EMBODIED AI · 2026-11-09
Embodied AI — Paper Daily, 2026-11-09
Today's embodied track is about measuring general-purpose robot models accurately and putting them to use. RoboColosseum has models face each other blind on real robots at many sites and finds that rankings change a lot with the robot and the task, while RoboScope Kit speeds up simulation evaluation several times over. On the control and data side, HITCH lets a vision-language model operate a humanoid through an interface, and OopsieRecover studies how a few corrections generalize to new scenes. MoMaSplat fixes policies that fail when the base position shifts, and Social Humanoid teaches a humanoid to coordinate with people.
4.54
Ranking general-purpose robot policies is hard because real-robot experiments are difficult to reproduce: change the site or the object layout and the result changes. RoboColosseum tests five robot foundation models and three frontier LLM agents head to head on four robot types at many real sites, with both models facing the same local conditions.
4.37
Letting a VLM drive a humanoid robot has two hurdles: the model must tell where objects are and how far away, and the robot must turn high-level commands such as reach there or turn the torso into stable whole-body motion. HITCH gives the VLM an interface for both. The VLM picks a point in its image and gets back the 3D position of that point; it issues commands for the pelvis, torso, hands, walking and gaze, and a learned controller turns them into whole-body motion. Measuring the scene accurately and executing commands accurately each raise task success. Teacher policies then solve five base tasks through this interface, and a VLM fine-tuned on those trajectories goes from 3.3% to 51.1% success on SIMPLE without any SIMPLE trajectories. On a real G1 it carried a box in 13 of 20 trials with no real training data.
4.32
In homes and offices, paths are often blocked by chairs, toys and carts, and a robot has to decide which objects it can push aside and which it must go around: an empty cart can be pushed, but one carrying fragile glassware cannot. MessyNav needs no training in simulation and no per-scene instructions. It first has a vision-language model judge, from the image and its context, whether each object can be swiped aside, picked up or should be avoided, and then plans actions from those labels. Across 15 real-world scenes it reached the goal without unsafe interactions 67% of the time, against 40% for planning with the same model when judgment and planning share one prompt. Separating the two helps because whether an object can move depends on its state, such as whether the stack on top is stable, and that should be settled before planning.
4.29
Most humanoid robots are trained alone, so they cannot coordinate with a person next to them: when someone reaches out for a hug or a high-five, the robot neither anticipates it nor adjusts. Social Humanoid has a Unitree G1 generate matching motion from a person's motion, such as a hug, or a greeting followed by posing for a photo together. It has two layers. A high-level model turns human and robot motion into the same set of discrete codes and generates the robot's response from them, and it can keep predicting the person's motion when they are occluded. A low-level controller then executes the motion on the robot. The training data come from transferring five human-human motion-capture datasets onto the robot, giving 260 hours of human-humanoid interaction. In simulation 90% to 100% of generated motions were executed fully by the low-level controller, and on the real G1 the success rate matched simulation in every scenario.
4.27
Robot manipulation foundation models fail after deployment, for example by colliding or dropping objects. The usual fix is to have a human take over, demonstrate a correction and fine-tune on the data, but that only teaches do this action in this scene, so it breaks when the scene changes and often needs tens to hundreds of interventions. OopsieRecover tries to learn from a handful of corrections what went wrong and how to avoid it. It has a vision-language model propose several possible reasons from the scene, the actions before and after takeover, and a spoken description, then checks them against all corrections and keeps the one that explains them best. That reason becomes a rule saying when it applies and which actions are penalized then, for example tilting a mug while placing it. At deployment the original model proposes several candidate actions and the one with the lowest penalty is chosen. On new scenes of the CoffeeMug task, 8 corrections were enough to reach 84% success, while HG-DAgger fine-tuned on three times as many corrections reached only 28%.
4.24
Mobile manipulation robots rarely get to stand exactly where the demonstrations were recorded, and imitation-learned policies are sensitive to camera viewpoint, so a slightly different base position can break them. Earlier methods augment demonstrations with novel-view synthesis, but they handle only static scenes and cannot re-render an object once it has been picked up and moved, which is the most critical phase. MoMaSplat uses the multi-view footage that demonstration collection already provides to reconstruct the static scene and the manipulated object separately, renders the whole manipulation from base positions never seen before, and converts the actions to the new position to produce synthetic demonstrations. In real-world tests with the base placed at unseen positions, the policy trained on its demonstrations averaged 37% success. The strongest baseline reached only 17%, and training without augmentation only 6%.
4.12
A vision-based tactile sensor images the deformation of an elastomer, so one image stream mixes contact appearance, geometry and tangential deformation, yet existing policies usually feed tactile images in as ordinary images, and nobody can say which cue is doing the work. This study runs a controlled real-robot comparison. From the same tactile stream it derives an appearance image, normal and depth maps, and a flow field of marker displacements on the elastomer surface, and uses them alone or combined with identical demonstrations and the same policy backbone. In the real-robot trials, adding touch raised overall success from 57.8% to 78.9%. Among single cues, marker flow was the most useful at 74.4% success, especially for keeping an established contact. For tactile imitation learning, encoding each physical cue separately beats feeding tactile images in as RGB.
4.03
One way to give a VLA model, which maps images and instructions straight to actions, an understanding of physical dynamics is to attach a video world model that predicts future frames, but generating video online adds high latency to the control loop. PDRVLA borrows the world model only during training. A lightweight module uses learnable queries to pull a fixed number of physical tokens out of intermediate model features. In training these tokens predict future RGB, depth and optical-flow features, so they hold a summary of how the scene will change. The tokens are also given as conditioning to a frozen pretrained video world model that must reconstruct the future frames, with its reconstruction error also training the module. At deployment no video is generated, and the action module uses the tokens together with the last-layer features. Its task success rate on LIBERO is 99.3%. The ablation is modest: world-model supervision lowers future-feature prediction error by only 1.32% relative.
4.03
Simulation evaluation of VLAs is slow mainly because resources sit idle: five hundred RoboTwin Easy episodes on one GPU take nearly six hours, because simulation occupies the CPU cores while the GPU averages only 14.3% utilization. RoboScope Kit wraps each model in one uniform Policy class and each benchmark in an adapter that unifies the observation and action formats exchanged between them, leaving the benchmark's own rules and scoring unchanged. It then splits simulator workers and model replicas into two pools, picks their sizes by briefly trial-running different combinations and keeping the fastest one that works, and shards the work deterministically across GPUs. On LIBERO-Long this is 4.45 times faster than the official StarVLA evaluation, with equivalent success rates. On RoboTwin Easy with four GPUs, evaluation is 25.04 times faster than the single-GPU baseline, with identical success outcomes.
3.20
GoogleTeams building embodied systems are splitting them into two tiers: an embodied reasoning model in the cloud handles spatial reasoning, task planning and progress tracking, while a lightweight VLA or small model on the robot handles real-time motor control. Google Cloud describes this architecture in a post about robotics startups, and says cloud compute is used mainly for simulation, training and fleet orchestration. Examples include the Apollo humanoid integrating Gemini Robotics models, and Dyna Robotics training on Google Cloud GPU clusters and shipping updated models to production sites every day. It is a promotional article, so for people building embodied systems its main value is the split between cloud planning and local execution.