EMBODIED AI · 2026-11-02
Embodied AI — Paper Daily, 2026-11-02
Today's embodied track is about turning each stage of robot learning into a reusable foundation. UMI-Live lets people collect demonstrations on site with handheld grippers while a background pipeline turns them into training data. ProWorld uses generated video and LocoManiLab uses a shared simulation testbed, each replacing part of real-robot evaluation. Metatron and FORGE make humanoid whole-body control a general layer that higher-level code only has to give goals to. Other papers tackle planning over many steps, transferring data across controllers, and catching thrown objects, a task that demands fast reactions.
4.52
The most expensive part of teaching a robot a task is often collecting demonstrations: teleoperation means hauling the whole robot to each site. UMI lets a person demonstrate with a pair of handheld grippers instead, but you still have to build the hardware, calibrate it and process every recording, so the barrier stays high. UMI-Live packs the camera, LiDAR, compute and battery into the gripper. The operator presses one button to record the next take while a background pipeline turns finished takes into training-ready data. Most of the training data comes from the handheld grippers, collected at a different place and under different lighting than the test scene, with only a small share of real-robot teleoperation. Fine-tuned on this mix, pi0.5 succeeded in all 10 trials of a bimanual task that puts an orange into a bowl. Its value is not a new algorithm but a data-collection pipeline you can actually reproduce, and the authors also release about 40 hours of data.
4.47
Getting a humanoid to sprint, walk backwards, sidestep, spin in place and hop on one leg usually means preparing reference trajectories or training a separate policy for each behaviour. FORGE does it in two stages. First it trains a reference generator on a simplified body model: forces are computed as if the whole body were one rigid body, while joint poses follow the full robot, and it produces whole-body reference motions in real time from velocity commands. Then it freezes the generator and trains a second policy to track those references on the full robot dynamics. On a Unitree G1, one generator plus one tracking policy covers forward speed of 2.8 metres per second, backward speed of 2 metres per second, one-legged hopping, and smooth switching between these commands. Because the simplified model is cheap to compute, references can be generated online over a wide range of commands.
4.44
For a task like the Tower of Hanoi, where a robot must chain many moves, planning with a visual world model tends to drift further off the longer it plans. VQ-WM compresses images into a finite set of discrete states, searches for the shortest path to the goal over those states, then has a controller with visual feedback walk the path segment by segment. The key is how the discrete states are chosen. Instead of clustering by embedding distance, it runs the controller inside the learned world model, and records how many calls it takes to get from each state to a candidate state. It then picks as few candidates as possible so that every state in the data can reach one of them within a few calls. That way every step on the path is within the controller's reach. Trained only on short play data, it reaches 100% task progress on the physical Tower of Hanoi, while existing foundation models such as VLAs succeed only when the full solution happens to be in their training data.
4.41
Catching thrown objects is hard for a humanoid because objects differ so much: plush toys barely bounce, chip bags deform, hard balls rebound on contact, and a humanoid's wrist torque is small enough that overly aggressive motions can damage the hardware. The authors train a reinforcement learning policy only in simulation on simple shapes. Its grasp reward is based on the distance from the fingertips to the object surface, so the closer the fingers wrap the object the higher the reward. A penalty on exceeding rated torque grows over training, so the policy learns to stay within limits. They then deploy it on a Unitree G1 with no real-robot data, using a single camera from the robot's own viewpoint. On objects never seen in training, underhand catches, palm up and low, succeed 38.9% of the time, and overhand catches, hand raised high, succeed 36.1%, while an expert teleoperator manages only 6.8% on the same task. Motions this fast cannot be captured well by teleoperation, so reinforcement learning in simulation is the more realistic route.
4.37
Walking, recovering from a shove, picking things up and sitting down usually each need their own trained humanoid controller, even though all of them rely on the same balance and whole-body coordination. Metatron separates the two. It first trains one general whole-body policy and freezes it. The policy takes only partial goals, such as where a hand should be at a given moment, how fast to walk, or which style to move in. Each task is written as ordinary code that sends such goals to the policy at every control step, and the policy fills them in as coordinated whole-body motion. Coding agents can tune this task code from motion data without retraining the policy. On a real Unitree G1, one frozen model handles walking, recovery, teleoperated object pickup and sitting on furniture. When shoved during walking, two reference-tracking pipelines fail every time once the push exceeds 600 newtons, while Metatron stays upright in 75% of trials at even larger pushes.
4.27
Evaluating robot policies by running trial after trial on a real robot is slow and expensive, so people want to use frames generated by a world model as the test environment. There are two difficulties. The policy needs more than one image for its next step: it needs several camera views and the robot's own joint state. Generative models also tend to draw a missed grasp as a successful lift, making the policy look stronger than it is. ProWorld predicts multi-view video and robot state together and feeds both back to the policy under test. Its training also adds failed manipulations collected in a high-fidelity simulator to suppress that inflation. Across 3 policies and 4 manipulation tasks, the Pearson correlation between its success rates and real-robot success rates reaches 0.95, which suggests its ranking of policies can largely stand in for real-robot trials.
4.18
The same action command makes a robot move differently under different controllers: whether it controls joints or the end-effector, whether it sends positions or velocities, and whether stiffness is set high or low all change what the same numbers mean. Pool such data to train one policy and these meanings clash. The authors first show the policy a short interaction record of commands sent and how the robot actually moved afterwards. A Transformer summarises this record into features of the current controller, and the policy adjusts its actions accordingly. In real-robot experiments this raises average success by 33 percentage points over simply pooling the data. On contact-heavy tasks such as USB insertion and surface wiping, the policy changes its whole strategy with the controller, not just the size of its motions.
4.16
Research on humanoids that walk and manipulate at the same time is hard to compare, because every group uses different policies, controllers and tasks. LocoManiLab puts data collection, training, evaluation and real-robot deployment into one testbed: 234 tasks across five kinds of household scenes, with demonstrations collected by teleoperation in simulation, and twelve policies evaluated under the same controller. On unseen tasks and scenes, success rates generally fall, dropping further as visual perturbations grow, and policies differ clearly in how well they generalise. The authors also recreate 3 real-robot tasks in simulation, and simulation and the real robot rank the policies similarly, so the authors argue policies can be compared in simulation first and then taken to the real robot.
4.07
Getting a robot to do a new task from a few demonstrations without updating weights is limited above all by context length. One demonstration spans hundreds of steps of multi-view frames and still takes thousands of tokens after subsampling, so current methods usually fit only one or two. JEDI compresses each demonstration into a few vectors. Rather than training them to reconstruct the demonstrated actions, it trains them to predict the next-stage keyframe of the current episode. During training it also uses demonstrations of a different task or a different step order as negatives, forcing the vectors to keep execution details. In a test where two demonstrations differ only in the order objects are placed, JEDI follows the demonstrated order 84.1% of the time, while a version that compresses only actions manages 52.2%, roughly a coin flip. What a compressed demonstration should keep is the information that predicts what to do next.
4.05
AAgiBot
At this year's Innovation Day, AgiBot proposed a new unit, Worken (effective intelligent work hours), meant to answer how much useful work robots actually do. AgiBot's analogy: steam engines were measured in horsepower, electricity in kilowatt-hours, large models in tokens, and robots should be measured by the hours of valuable work they complete on their own and without interruption, not just by how many are built. At the same event AgiBot released its next-generation embodied foundation model GO-3 Preview, which it says ranks first on three embodied benchmarks and plans to open-source by year end. It also showed an open-source embodied operating system, Lingqu OS, a subcutaneous liquid-cooling system for joints, and a new family of dexterous hands. The industry's focus is shifting from units shipped to how much work robots can do by themselves.