EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention

Cheng Jin, Yiting Cao, Junfeng Zhu, Te Li, Hanzhang Wang, Yuchun Fang

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2026 · 22 (11) · 1-23

DOI: 10.1145/3845997 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.11.2026. Verfasst von Cheng Jin und 5 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Shanghai, National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

3D hand reconstruction from monocular RGB images has attracted increasing attention due to its low cost and ease of deployment. However, accurately reconstructing interacting hands remains challenging, primarily because mutual occlusion leads to missing visual evidence, while image cropping weakens global spatial consistency. To address these issues, we propose the Context-Aware Interacting Hand Reconstruction Network (CANet), a monocular interacting-hand reconstruction framework that integrates multimodal context fusion with structured spatial attention. Specifically, CANet leverages estimated depth maps, edge contours, and center heatmaps to retain global contextual cues and guide reconstruction in occluded regions. A hierarchical spatial attention module further enhances spatial consistency by separately modeling intra-hand structural dependencies and inter-hand interactions, enabling more coherent reasoning about complex hand poses. Experiments across multiple public benchmarks demonstrate that CANet consistently improves the accuracy of both hand pose estimation and mesh reconstruction.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Shanghai
  • National Key Research and Development Program of China
  • Shanghai Key Laboratory of Forensic Medicine, Academy of Forensic Science, Ministry of Justice
  • Shanghai Technical Service Center of Science and Engineering Computing, Shanghai University

Zitieren

APA

Jin, C., Cao, Y., Zhu, J., Li, T., Wang, H., & Fang, Y. (2026). Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(11), 1-23. https://doi.org/10.1145/3845997

Vancouver

Jin C, Cao Y, Zhu J, Li T, Wang H, Fang Y. Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(11):1-23. doi:10.1145/3845997

BibTeX
@article{Jin2026,
  author = {Jin, Cheng and Cao, Yiting and Zhu, Junfeng and Li, Te and Wang, Hanzhang and Fang, Yuchun},
  title = {Monocular Interacting-Hand Reconstruction with Multimodal Context Fusion and Spatial Attention},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {11},
  pages = {1--23},
  doi = {10.1145/3845997}
}
Kostenlos starten →