EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Experience Filter: Translating Heterogeneous Signals into Motion

Ekrem Yüksel iD, Alihan Bakir iD, Jamie Paik iD

Advanced Science · 10.11.2026

DOI: 10.1002/advs.78602 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Science, online erschienen am 10.11.2026. Verfasst von Ekrem Yüksel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human experience is inherently rich and multisensory, composed of intertwined visual, auditory, and kinesthetic feedback. Digital media capture only its audiovisual dimensions, omitting its mechanosensory dimensions. While full‐body (kinesthetic) haptic feedback can enrich media, no general method exists for deriving structured motion from heterogeneous signals and combining it with the audiovisual content for rendering. The Experience Filter maps multimedia signals into structured motion trajectories through a common pipeline of modality‐specific feature extraction followed by a modality‐agnostic shared encoding, recombining them into a synchronized multisensory experience: each signal is reduced to a shared descriptor space – signal intensity, dominant spectral feature, and distance from neutral axis – then mapped to a trajectory via a fixed, deterministic, training‐free encoding. Because this encoding space is hardware‐independent, the motion can be retargeted to robotic platforms of arbitrary degrees of freedom and morphology. A feature‐sensitivity analysis shows that the descriptor axes are independent and the encoding‐space topology follows from the chosen descriptors rather than being fixed. The framework is demonstrated on real‐world music and images, producing continuous, synchronized image‐audio‐motion experiences without manual authoring. The Experience Filter thus provides an analytical mapping from multimedia signals to combined image‐audio‐motion rendering, enabling expressive, user‐adjustable experience across robotic platforms.

Schlagwörter

Stichwörter der Autor:innen

haptic feedbackhuman‐robot interactionmultisensorial feedbacksignal‐to‐motion encodingtactile force feedback

Autor:innen und Institutionen

  • Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland.

ORCID-Profile: Ekrem Yüksel, Alihan Bakir, Jamie Paik

Förderung

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Zitieren

APA

Yüksel, E., Bakir, A., & Paik, J. (2026). Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. https://doi.org/10.1002/advs.78602

Vancouver

Yüksel E, Bakir A, Paik J. Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. 2026. doi:10.1002/advs.78602

BibTeX
@article{Yksel2026,
  author = {Yüksel, Ekrem and Bakir, Alihan and Paik, Jamie},
  title = {Experience Filter: Translating Heterogeneous Signals into Motion},
  journal = {Advanced Science},
  year = {2026},
  doi = {10.1002/advs.78602}
}
Kostenlos starten →