Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2026 · 22 (11) · 1-23
DOI: 10.1145/3845997 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.11.2026. Verfasst von Cheng Jin und 5 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Shanghai, National Key Research and Development Program of China.
Abstract (Originalsprache)
3D hand reconstruction from monocular RGB images has attracted increasing attention due to its low cost and ease of deployment. However, accurately reconstructing interacting hands remains challenging, primarily because mutual occlusion leads to missing visual evidence, while image cropping weakens global spatial consistency. To address these issues, we propose the Context-Aware Interacting Hand Reconstruction Network (CANet), a monocular interacting-hand reconstruction framework that integrates multimodal context fusion with structured spatial attention. Specifically, CANet leverages estimated depth maps, edge contours, and center heatmaps to retain global contextual cues and guide reconstruction in occluded regions. A hierarchical spatial attention module further enhances spatial consistency by separately modeling intra-hand structural dependencies and inter-hand interactions, enabling more coherent reasoning about complex hand poses. Experiments across multiple public benchmarks demonstrate that CANet consistently improves the accuracy of both hand pose estimation and mesh reconstruction.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Shanghai
- National Key Research and Development Program of China
- Shanghai Key Laboratory of Forensic Medicine, Academy of Forensic Science, Ministry of Justice
- Shanghai Technical Service Center of Science and Engineering Computing, Shanghai University
Zitieren
APA
Jin, C., Cao, Y., Zhu, J., Li, T., Wang, H., & Fang, Y. (2026). Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(11), 1-23. https://doi.org/10.1145/3845997
Vancouver
Jin C, Cao Y, Zhu J, Li T, Wang H, Fang Y. Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(11):1-23. doi:10.1145/3845997
BibTeX
@article{Jin2026,
author = {Jin, Cheng and Cao, Yiting and Zhu, Junfeng and Li, Te and Wang, Hanzhang and Fang, Yuchun},
title = {Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {11},
pages = {1--23},
doi = {10.1145/3845997}
}
EUVIMED