Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2026 · 22 (11) · 1-23
DOI: 10.1145/3829080 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.11.2026. Verfasst von Wenjuan Gong und 5 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Natural Science Foundation of Shandong Province, Shandong Data Open Innovative Application Laboratory.
Abstract (Originalsprache)
Sentiment analysis, crucial in fields like human–computer interaction and mental health, has advanced through multimodal fusion and large-scale language models. People exhibit significant variability in emotional expressions due to behavioral differences, yet existing methods often apply generic models across populations. By focusing primarily on objective features, these models overlook “personal” cues that may lead to inconsistencies between the extracted features and the corresponding emotion categories. Additionally, at the common level, this issue is aggravated by the accumulation of personal cues mapped to a common emotion category. Existing approaches tend to fuse these cues indiscriminately, whereas features that align consistently with the target emotion category should be emphasized. The attention mechanism, as the dominant fusion method, cannot adjust the misallocated weights it assigns. To address these limitations, we propose a “personal behavior-robust” sentiment analysis approach that disentangles personal cues from common ones through decoupling and learns the relative importance of features across modalities via a corrective feedback-based “interpretable” fusion strategy to adjust misallocated weights assigned by the attention mechanism. The proposed model effectively differentiates and prioritizes relevant information from multiple cues, guiding the model to focus on the cue that aligns with the overall emotion. Experimental results demonstrate that the proposed methods achieve good performance on two public datasets: CMU-MOSI and CMU-MOSEI.
Förderung
- Natural Science Foundation of Shandong Province
- Shandong Data Open Innovative Application Laboratory
Zitieren
APA
Gong, W., Li, J., Shi, T., Li, C., Chen, C., & Álvarez, J. M. (2026). Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(11), 1-23. https://doi.org/10.1145/3829080
Vancouver
Gong W, Li J, Shi T, Li C, Chen C, Álvarez JM. Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(11):1-23. doi:10.1145/3829080
BibTeX
@article{Gong2026,
author = {Gong, Wenjuan and Li, Jiarui and Shi, Tingbo and Li, Chunhan and Chen, Chenglizhao and Álvarez, José M.},
title = {Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {11},
pages = {1--23},
doi = {10.1145/3829080}
}
EUVIMED