EMBODIED AI · 2026-10-30
Embodied AI — Paper Daily, 2026-10-30
Today's embodied track is about making robot training cheaper and seeing clearly inside the models. SuperMimic uses a differentiable world model to cut the compute for teaching a humanoid whole-body motions from over a hundred GPUs to a few hours on one. Another paper takes apart three VLA models and finds that actions depend on only a few layers, so removing the other connections saves a third of the compute. The rest focus on real-world trouble: people moving nearby, long tasks that require remembering how the environment changed, learning new skills after deployment, and handling soft objects such as folding clothes.
4.74
Teaching a humanoid robot whole-body control by tracking large amounts of human motion data is currently very compute-hungry: the strongest motion tracker, SONIC (a previously released tracker trained with PPO on a cluster), trains for about 7 days on 128 GPUs. SuperMimic starts from a simple observation: the motions differ wildly, but the robot's body obeys one set of physics. It first learns a differentiable world model from simulator logs, then lets the policy imagine short rollouts inside that model and take gradients directly with respect to its actions. After 3 hours on a single GPU its return already exceeds SONIC's, using more than 1000× fewer GPU-hours. The bottleneck shifts from compute to the motion data itself, and adding more motions becomes cheap.
4.50
The action module of a VLA model usually reads the language-model backbone's information at every layer, yet only a narrow band of mid-to-late layers actually affects the action. The authors test this in three open-source VLA models. With a banana and an apple on the table, the robot is told once to pick up the banana, and once, with the two swapped in place, to pick up the apple. One layer's internal activations from the first run are copied into the second, and they watch, layer by layer, whether the robot switches to the banana. The target object's word in the instruction decides which object gets picked in early-to-mid layers, while the spatial information about where to move sits in mid-to-late layers. Acting on this, letting MolmoBot's action module read only 3 of those layers cuts inference compute by up to 33%, with success almost unchanged.
4.14
When a robot fails during manipulation, existing methods either give each frame a single failed-or-not score or must wait until the whole video is recorded. This work turns failure detection into judging while watching: the model reads live video chunk by chunk, keeps a memory bank of frames it has seen, and when something goes wrong outputs both when the failure starts and ends and a sentence describing what happened. Since general video models do not understand robot manipulation, the authors first run an extra round of pre-training on existing robot datasets. The final model locates failure time spans with an mIoU of 0.232, against 0.194 for the strongest online baseline. For anyone doing live monitoring, saying in real time where it went wrong is more useful than scoring afterwards.
4.07
Training robot policies on huge amounts of human manipulation video is hard for more than one reason. Human hands and robot arms move differently, and the visual encoder also places the same manipulation by a human and by a robot into two unrelated regions of feature space, so mixing them in training loses much of the benefit. HARP uses a small number of paired demonstrations, where a human and a robot do the same thing, as a bridge. It pulls robot visual features toward the human side while keeping different pairs distinguishable from one another. Plenty of unpaired video then trains a model that infers, from the change between adjacent frames, what action was taken, giving an action code shared by humans and robots. The policy learns to predict this code, and a small module turns it into robot control commands. Once aligned, human video actually helps: on CALVIN the policy completes 4.481 tasks in a row on average, and real-robot success is 7.1% higher than the strongest baseline.
4.38
When a robot follows an instruction, what to do and whether the motion is physically doable are two different kinds of requirements, and they get recombined into tasks never seen before. Existing policies learn them mixed in one representation, so they generalize poorly to new combinations and may produce motions that are jerky or out of the arm's reach. This work splits trajectory generation into a sum of energy terms. Each semantic piece of the instruction, such as the object or the location, gets its own learned term that scores a candidate trajectory, better the more it satisfies that requirement. Physical constraints such as smooth motion and how well the arm can exert force are written as explicit differentiable terms, and together they guide a diffusion model. Once split, a new combination is just a new sum of existing terms, and both compositional generalization and physical feasibility improve on real robots and public benchmarks.
4.12
Most robot manipulation benchmarks assume the robot is the only one moving at the workbench, yet in real settings a person is often right beside it. They may be pointing to hint at what they want, briefly taking away something needed, or finishing half the task themselves, or simply standing where the arm needs to pass. HARP-Bench treats these situations as part of the task itself, grouping them into four kinds, cueing, interruption, contribution and co-presence, and pairs every everyday task with a version without a person present for direct comparison. The authors test VLA, imitation-learning and rule-based policies, and all of them get clearly worse when a person is present; training on demonstrations collected with the benchmark improves performance in these situations, which real-robot experiments also confirm.
4.10
A deployed robot needs to keep learning new skills, but learning something new makes it forget the old, and holding on to the old makes it slow to learn the new. LifelongVLA attaches two sets of LoRA to a VLA model. One is trained only on the new task's data and learns fast; the other is trained only on review samples of old skills plus a distillation signal, with a lower learning rate, so it changes slowly and protects what was learned. A gate decides from the current input how much each set contributes. For review it keeps only a few randomly chosen samples per old skill instead of whole image-rich demonstrations. After learning several tasks in sequence, average success reaches 83.2%, 13 points above the strongest baseline, and the forgetting rate is 8.2 points lower.
4.05
For a long task like tidying a room, a mobile robot must keep track of where it is, what has changed around it and how far along the task is, which is hard to answer from the current image alone. SERF maintains a feature map that updates over time, representing both the objects in the environment and the robot's own body as feature points in one shared space. When an object moves its points follow rigid tracking, and the robot's points are placed from its joint angles. Besides images, the VLA policy takes features from this map centred on the gripper and on the robot base, both nearby and over a wide area. On three household tasks in BEHAVIOR-1K, average task progress rises from 44.0% to 58.7%, with larger gains in regions the robot has not visited.
3.99
Folding clothes is hard because most of the fabric is hidden by itself, and the same pull changes thick cloth and thin cloth in completely different ways. A person can guess a garment's shape and material after a few glances, and this work lets a robot do the same. A transformer world model, trained on large amounts of simulated cloth data, takes the partial point clouds from the last few frames, completes the shape of the whole garment, and predicts what the next action will turn it into. Material stiffness needs no retraining, since the model infers it from those few frames of history, so a new garment, material or robot needs no weight updates. Combined with a planner that picks actions toward a target shape, bimanual folding improves in reconstruction accuracy, prediction accuracy and success rate, both in simulation and on a real robot.