COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 15.06.2026 · 10 (2) · 1-29
DOI: 10.1145/3810218 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 15.06.2026. Verfasst von Baiyu Chen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: ARC Centre of Excellence for Automated Decision-Making and Society.
Abstract (Originalsprache)
The goal of creating intelligent, human-centered wearable systems for continuous activity understanding faces a fundamental trade-off: Egocentric video-based models capture rich semantic information and have demonstrated strong performance in human activity recognition (HAR), but their high power consumption, privacy concerns, and dependence on lighting limit their feasibility for continuous on-device recognition. In contrast, inertial measurement unit (IMU) sensors offer an energy-efficient, privacy-preserving alternative, yet lack large-scale annotated datasets, leading to weaker generalization. To bridge this gap, we propose COMODO, a cross-modal self-supervised distillation framework that transfers semantic knowledge from video to IMU without requiring labels. COMODO leverages a pretrained and frozen video encoder to construct a dynamic instance queue to align the feature distributions of video and IMU embeddings. This enables the IMU encoder to inherit rich semantic structure from video while maintaining its efficiency for real-world applications. Experiments on multiple egocentric HAR datasets show that COMODO consistently improves downstream performance, matching or surpassing fully supervised models, and demonstrating strong cross-dataset generalization. Benefiting from its simplicity and flexibility, COMODO is compatible with diverse pretrained video and time-series models, offering the potential to leverage more powerful teacher and student foundation models in future ubiquitous computing research. The code is available at this repository: https://github.com/cruiseresearchgroup/COMODO.
Förderung
- ARC Centre of Excellence for Automated Decision-Making and Society
Zitieren
APA
Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D. (2026). COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10(2), 1-29. https://doi.org/10.1145/3810218
Vancouver
Chen B, Wongso W, Li Z, Khaokaew Y, Xue H, Salim FD. COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2026;10(2):1-29. doi:10.1145/3810218
BibTeX
@article{Chen2026,
author = {Chen, Baiyu and Wongso, Wilson and Li, Zechen and Khaokaew, Yonchanok and Xue, Hao and Salim, Flora D.},
title = {COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition},
journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
year = {2026},
volume = {10},
number = {2},
pages = {1--29},
doi = {10.1145/3810218}
}
EUVIMED