📝 Publications

The UniT line

A shared physical language, extended through data scaling and world modeling.

UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modeling

Boyu Chen*,Yi Chen*,Lu Qiu,Jerry Bai,Yuying Ge†,Yixiao Ge

*Equal contribution †Corresponding author

TL;DR

A shared physical language that connects human and humanoid actions, enabling human-to-humanoid policy learning and world modeling.

Project · Paper · Code · BibTeX · Citation details

Abstract

Scaling humanoid foundation models is bottlenecked by the scarcity of robotic data. While massive egocentric human data offers a scalable alternative, bridging the cross-embodiment chasm remains a fundamental challenge due to kinematic mismatches. We introduce UniT (Unified Latent Action Tokenizer via Visual Anchoring), a framework that establishes a unified physical language for human-to-humanoid transfer. Grounded in the philosophy that heterogeneous kinematics share universal visual consequences, UniT employs a tri-branch cross-reconstruction mechanism: actions predict vision to anchor kinematics to physical outcomes, while vision reconstructs actions to filter out irrelevant visual confounders. Concurrently, a fusion branch synergies these purified modalities into a shared discrete latent space of embodiment-agnostic physical intents. We validate UniT across two paradigms: 1) Policy Learning (VLA-UniT): By predicting these unified tokens, it effectively leverages diverse human data to achieve state-of-the-art data efficiency and robust out-of-distribution (OOD) generalization on both humanoid simulation benchmark and real-world deployments, notably demonstrating zero-shot task transfer. 2) World Modeling (WM-UniT): By aligning cross-embodiment dynamics via unified tokens as conditions, it realizes direct human-to-humanoid action transfer. This alignment ensures that human data seamlessly translates into enhanced action controllability for humanoid video generation. Ultimately, by inducing a highly aligned cross-embodiment representation (empirically verified by t-SNE visualizations revealing the convergence of human and humanoid features into a shared manifold), UniT offers a scalable path to distill vast human knowledge into general-purpose humanoid capabilities.

Building on UniT

Data scaling

Inside Fe₀: What Cross-Embodiment Data Teaches an Embodied Foundation Model

Yizhuo Li*,Xiaoyu Zhang*,Yuying Ge*†,Teng Wang,Boyu Chen‡,Yi Chen,Bo Liu,Feng Qiu,Jiacheng Wei,Yuguo Gan,Hui Zhou,Yixiao Ge

*Co-first authors †Project lead ‡Core contributor

XPENG Robotics Blog · 2026

TL;DR

Building on UniT’s cross-embodiment premise, Fe₀ studies what heterogeneous data teaches robot policies—and where transfer reaches its limits.

Project · BibTeX · Citation details

Abstract

In embodied AI, it is a widely held belief that cross-embodiment data—a diverse mixture encompassing both heterogeneous robot data and egocentric human videos—enhances robot models. Yet, this consensus remains an opaque black box: we know it works, but we rarely understand exactly what this broad spectrum of data is teaching the robot. To demystify this, we introduce Fe0, an embodied foundation model. Rather than scaling data to chase benchmarks, we designed a rigorous controlled experiment. We deliberately constrained our in-domain teleoperation data to the minimum: just 3 hours of trajectories featuring only a single, atomic pick-and-place primitive, where 95% of the data involves moving a single object to a single receptacle from a highly restricted vocabulary. By establishing this extreme baseline, we guarantee that any zero-shot generalization Fe0 exhibits beyond in-domain grasping is unequivocally powered by cross-embodiment learning. But how do we measure this transfer scientifically? Instead of ad-hoc testing, we constructed a systematic L1–L5 capability framework—spanning from basic visual robustness and semantic grounding, up to complex relational reasoning, compositional planning, and novel action execution. We evaluate the model through both rigorous open-loop trajectory scoring and extensive closed-loop real-world trials on the IRON-R01-1.11 robot to unpack the true anatomy of cross-embodiment data. We reveal a clear capability boundary: while heterogeneous data effectively teaches the model visual semantics and basic task logic, it falters at mid-level relational and sequential reasoning, and hits a hard ceiling when transferring precise physical execution. By systematically evaluating exactly where cross-embodiment data succeeds and where it fails, we illuminate what this heterogeneous experience actually teaches an embodied foundation model, and what it takes to push beyond current limits.

World modeling

AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization

Cheng Chen,Jerry Bai,Jiacheng Wei,Boyu Chen,Xiaoji Zheng,Fan Wu,Minghao Yang,Tianrun Chen,Ruibo Li,Xiaoyu Yue,Xiaoyang Guo,Yixiao Ge,Guosheng Lin,Fayao Liu

TL;DR

Extending UniT through world modeling, AnyWorld turns human interactions into controllable, robot-native experience for policy adaptation.

Project · Paper · Code · BibTeX · Citation details

Abstract

Collecting contact-rich robot experiences at scale remains a major bottleneck for generalizable manipulation. Beyond data quantity, robot learning also requires diverse experiences across embodiments, viewpoints, and scenes. Human egocentric videos provide abundant physical interactions, but each video captures only a narrow slice of experience under a single body, camera trajectory, and environment. We propose AnyWorld, a cross-embodiment world modeling framework that expands a single human interaction into diverse robot-native rollouts without paired human-robot demonstrations. Our model factorizes an interaction into action, camera, and embodiment: action controls capture the motion structure, camera controls specify viewpoint evolution, and the target embodiment context defines the acting body and its interaction geometry. This formulation enables independent recomposition of embodiment, viewpoint, and scene factors, allowing a single model to generate many robot-domain experiences while preserving the underlying dynamics and object interactions. We train the model with large-scale human interaction pretraining followed by mixed-embodiment fine-tuning. Experiments show that our model supports controllable recomposition across embodiments, viewpoints, and scenes, and we further demonstrate that the generated data can improve manipulation performance on the RoboCasa GR1 tabletop benchmark and a real IRON humanoid robot. Beyond aggregate gains, we test whether unpaired human experience can be recomposed into robot-native video-action pairs that target a policy gap. Controlled IRON interventions correct a spurious completion prior and establish language-grounded spatial target selection; an action-only counterfactual intervention fails to learn the latter reliably, showing that both action calibration and visual recomposition are necessary.

Grounding for physical intelligence

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

Qize Yu*‡,Lianrui Fan*,Boyu Chen*,Jiaqi Liang*,Xini Ding*,Yue Chen,Zetian Song,Yuran Wang,Yi Zou,Kaixuan Wang,Tianxing Chen,Wenxuan Song,Bohan Zhou,Mingleyang Li,Siqiao Huang,Yuqi Ye,Caigao Jiang,Wei Wei,Ruihai Wu,Hang Zhang,Yixiao Ge,Shuchang Zhou,Shilong Liu,Xianming Liu,Ping Luo‡,Shiyu Huang‡

*Equal contribution ‡Corresponding authors

arXiv preprint · 2026

TL;DR

Points and boxes become reusable visual primitives, improving perception for robotic manipulation and autonomous driving.

Project · Paper · Code · BibTeX · Citation details

Abstract

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI’s pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding’s substantial benefits for both, and OCR’s potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.