HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining
초록
Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck. Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment, yet their scalability is limited by high collection cost, acquisition difficulty, and low behavioral and environmental diversity. These limitations have sparked interest in egocentric human video as a scalable, substantially lower-cost, and more diverse alternative for embodied model pretraining. However, its effectiveness compared to teleoperated real-robot data remains underexplored. To address this question, we conduct a systematic study comparing egocentric human video and teleoperated real-robot trajectories as pretraining data sources for embodied foundation models, under fixed post-training and validation protocols. Surprisingly, we find that egocentric data, when processed through a carefully designed filtering and labeling pipeline, is not merely a viable substitute for model pretraining but can lead to superior performance. With the same amount of pretraining data, models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction, as well as 52.5% and 90% higher success rates on in-distribution and out-of-distribution real-robot task execution, respectively. This finding verifies a scalable paradigm for embodied foundation models: pretrain on egocentric human video to learn diverse world representations, then adapt with a small amount of labeled real-robot data for action-space alignment. We hope this study encourages broader exploration of egocentric data and offers guidance for data quality assessment before costly robot data collection.
저자 (22명)
- Juncheng Ma — LinkedIn 검색
- Jianxin Bi — LinkedIn 검색
- Yufan Deng — LinkedIn 검색
- Xuanran Zhai — LinkedIn 검색
- Kewei Zhang — LinkedIn 검색
- Ye Huang — LinkedIn 검색
- Bo Liang — LinkedIn 검색
- Shukai Gong — LinkedIn 검색
- Jiankai Tu — LinkedIn 검색
- Xiaotian Tang — LinkedIn 검색
- Jiaxin Li — LinkedIn 검색
- Kaiqi Chen — LinkedIn 검색
- Duomin Wang — LinkedIn 검색
- Yuqi Wang — LinkedIn 검색
- Bingyi Kang — LinkedIn 검색
- Eric Huang — LinkedIn 검색
- Zhiyang Dou — LinkedIn 검색
- Zhen Dong — LinkedIn 검색
- Enze Xie — LinkedIn 검색
- Wojciech Matusik — LinkedIn 검색
- Tat-Seng Chua — LinkedIn 검색
- Daquan Zhou — LinkedIn 검색
저자 LinkedIn 변경 추적은 추후 자동화 예정입니다. 현재는 검색 링크를 제공합니다.