EUVIMEDEUVIMED entdecken
Crossref

Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity

Qi Ye iD, Qingtao Liu iD, Siyun Wang, Jiaying Chen, Yu Cui, Ke Jin iD, Huajin Chen iD, Xuan Cai, Gaofeng Li iD, Jiming Chen iD

Science Robotics · 28.01.2026 · 11 (110)

DOI: 10.1126/scirobotics.ady2869 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Robotics, erschienen am 28.01.2026. Verfasst von Qi Ye und 9 weiteren Autor:innen. Beteiligte Einrichtung: College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. Hauptthemen laut MEDLINE: Robotics. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 99 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Achieving humanlike dexterity with anthropomorphic multifingered robotic hands requires precise finger coordination. However, dexterous manipulation remains highly challenging because of high-dimensional action-observation spaces, complex hand-object contact dynamics, and frequent occlusions. To address this, we drew inspiration from the human learning paradigm of observation and practice and propose a two-stage learning framework by learning visual-tactile integration representations via self-supervised learning from human demonstrations. We trained a unified multitask policy through reinforcement learning and online imitation learning. This decoupled learning enabled the robot to acquire generalizable manipulation skills using only monocular images and simple binary tactile signals. With the unified policy, we built a multifingered hand manipulation system that performs multiple complicated tasks with low-cost sensing. It achieved an 85% success rate across five complex tasks and 25 objects and further generalized to three unseen tasks that share similar hand-object coordination patterns with the training tasks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RoboticsHumansHandTouchReinforcement Machine LearningLearningEquipment Design

Autor:innen und Institutionen

  • College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
  • School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China.

ORCID-Profile: Qi Ye, Qingtao Liu, Ke Jin, Huajin Chen, Gaofeng Li, Jiming Chen

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Ye, Q., Liu, Q., Wang, S., Chen, J., Cui, Y., Jin, K., Chen, H., Cai, X., Li, G., & Chen, J. (2026). Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity. Science Robotics, 11(110). https://doi.org/10.1126/scirobotics.ady2869

Vancouver

Ye Q, Liu Q, Wang S, Chen J, Cui Y, Jin K, et al. Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity. Science Robotics. 2026;11(110). doi:10.1126/scirobotics.ady2869

BibTeX
@article{Ye2026,
  author = {Ye, Qi and Liu, Qingtao and Wang, Siyun and Chen, Jiaying and Cui, Yu and Jin, Ke and Chen, Huajin and Cai, Xuan and Li, Gaofeng and Chen, Jiming},
  title = {Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity},
  journal = {Science Robotics},
  year = {2026},
  volume = {11},
  number = {110},
  doi = {10.1126/scirobotics.ady2869}
}
Kostenlos starten →