EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Heterogeneous Memristive Neuromorphic Computing with Physics-Aware Cross-Modal Attention for Energy-Efficient Multimodal Emotion Recognition

Cong Xu, Le-Jian Xu, Wei Feng, Fei Yu, Jie Jin

Electronics · 12.11.2026 · 15 (22) · 5257

DOI: 10.3390/electronics15225257 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 12.11.2026. Verfasst von Cong Xu und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Emotion Recognition (MER) is a crucial technology for achieving emotional intelligence in human–computer interaction. However, the substantial computational demands of MER models currently overwhelm the capacities of resource-constrained edge devices. To address this problem, this paper proposes a Heterogeneous Memristive Neural Network for Multimodal Emotion Recognition. The proposed network adopts a hardware–algorithm co-design approach, in which different memristive devices are assigned to different modal processing tasks according to their physical characteristics. Specifically, highly stable filamentary memristors are utilized to construct deterministic feature extraction pathways for text and video, whereas highly plastic Phase-Change Memory (PCM) crossbar arrays are designed for audio encoding. Based on the high plasticity of PCM, a memristive cross-modal attention network is further designed to achieve dynamic and adaptive multimodal feature fusion. Furthermore, a collaborative weight update method is proposed. During weight updates, the inherent writing noise and conductance drift of PCM are utilized as hardware-level regularization and automatic sparse learning strategies. Simulation experiments conducted on the IEMOCAP and MELD datasets using the MemTorch framework demonstrate that the proposed neural network maintains recognition accuracy comparable to GPU baseline models, even when accounting for device non-idealities. Overall, the proposed neural network effectively resolves the “performance–power” conflict when deploying high-performance MER models on resource-constrained edge devices, providing an efficient and stable neuromorphic computing solution for edge applications.

Zitieren

APA

Xu, C., Xu, L., Feng, W., Yu, F., & Jin, J. (2026). Heterogeneous Memristive Neuromorphic Computing with Physics-Aware Cross-Modal Attention for Energy-Efficient Multimodal Emotion Recognition. Electronics, 15(22), 5257. https://doi.org/10.3390/electronics15225257

Vancouver

Xu C, Xu L, Feng W, Yu F, Jin J. Heterogeneous Memristive Neuromorphic Computing with Physics-Aware Cross-Modal Attention for Energy-Efficient Multimodal Emotion Recognition. Electronics. 2026;15(22):5257. doi:10.3390/electronics15225257

BibTeX
@article{Xu2026,
  author = {Xu, Cong and Xu, Le-Jian and Feng, Wei and Yu, Fei and Jin, Jie},
  title = {Heterogeneous Memristive Neuromorphic Computing with Physics-Aware Cross-Modal Attention for Energy-Efficient Multimodal Emotion Recognition},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {22},
  pages = {5257},
  doi = {10.3390/electronics15225257}
}
Kostenlos starten →