Reliability-aware multimodal sensor fusion for human motion monitoring with safety-constrained neurorobotic feedback in sports training

Tiivistelmä

Introduction Embodied intelligent systems require reliable perception and safety-constrained feedback mechanisms for human motion understanding under heterogeneous sensing conditions. In this study, volleyball training is considered as a representative dynamic human-motion scenario rather than the sole application objective. The proposed system aims to provide a general neurorobotic perception and feedback framework for complex embodied environments. The framework must infer rapid technical actions while simultaneously tracking fatigue, jump loading, context information, and sensor reliability. Methods This study presents an embodied neurorobotic framework that combines a reliability-aware sensor-fusion transformer with a bidirectional long short-term memory (BiLSTM) estimator. Inertial, pose, plantar-pressure, ultra-wideband, and physiological measurements are synchronized, quality-gated, and exchanged through cross-modal attention before bidirectional temporal reasoning produces action, fatigue, and load estimates. The framework further couples these estimates to a barrier-based safety projection for haptic or robotic feedback. Theoretical analysis establishes a cross-modal perturbation bound and a conditional one-step barrier-satisfaction result under bounded estimation error, bounded disturbance, and maintained projection feasibility. Experiments use an athlete-disjoint, fully reproducible synthetic benchmark with controlled sensor noise and missing-modality conditions. Results The complete model achieves 87.0% accuracy and 0.872 macro-F1, exceeding transformer-only, BiLSTM-only, and no-quality-gate variants. Across the 10 held-out virtual-athlete profiles, accuracy is 0.870 ± 0.086 with a cluster-bootstrap 95% confidence interval of 0.820–0.920. Robustness tests show graceful degradation under moderate corruption, while the illustrative safety-filter simulation prevents all boundary violations observed under the unconstrained policy. Discussion Because no human-subject volleyball data are used, these findings establish reproducible computational behavior rather than field safety, injury-prevention efficacy, or clinical validity.