EMBODIED AI · 2026-11-03
Embodied AI — Paper Daily, 2026-11-03
Today's embodied AI track is about where robot data comes from and how to get more out of it. CIMI and TacClaw-Infinity both collect data without a robot. CIMI records where the fingers touch the object during human demonstrations, and TacClaw-Infinity gathers demonstrations in many real settings with a handheld gripper that senses touch. After deployment, two papers let robots keep learning from their own runs. One has a whole fleet share experience using only success or failure labels, and I3L lets an operator step in only when the policy goes wrong. Another group looks at how actions are represented. Vela writes a stretch of motion as a curve, and DiffWBC turns a humanoid controller into a diffusion model so it can be steered online. A large real-robot study also warns that offline metrics are a poor substitute for hardware tests.
4.59
Teaching a robot hand from human demonstrations is hard because human and robot hands have different shapes. If the conversion only tries to make the robot hand look like the human hand, the robot's fingertips often touch the object in different places than the human did, and grasps and pushes fail. CIMI puts contact first. During demonstrations it records the contact surfaces between fingers and object plus fingertip camera views; when converting to robot motion it aligns the contact points first and the finger pose second. The learned policy likewise predicts how the contact surface should move next, and a controller coordinates wrist and fingers to match it. Across six hand-task pairs on the LEAP and Wuji hands, mean full-task success beats baseline policies by at least 50 percentage points. When transferring human skills, matching contact matters more than matching hand shape.
4.57
The robot learning toolchain is fragmented: motor communication, data collection, dataset formats, training algorithms and deployment each tend to use their own stack, so switching robots or datasets means re-assembling everything, and other people's results are hard to reproduce. LeRobot puts the whole chain into one open-source library: low-level motor control and support for low-cost hardware, a unified dataset format with streaming, implementations of several mainstream policy algorithms, and asynchronous inference. Its value is not a new algorithm but letting data and results from different labs be compared and reused within one set of tools.
4.52
A low validation loss does not mean an imitation-learning policy works on the real robot. The authors had a five-fingered robot hand grasp and insert an RJ45 network connector and tested 281 policies on hardware to find which design choices actually decide success. The answer: how observations and actions are represented matters far more than the optimizer or sampling scheme. For point-cloud policies, keeping only the region around the target and expressing the points in the robot-base frame each raised success by 70 to 100 percentage points. Among dozens of offline metrics, only a few measuring whether predicted motion is smooth and direct correlated clearly with insertion success, and only within a single policy family. Offline metrics cannot yet replace real-robot testing.
4.41
Deployed robots work every day, but improving their policy usually still requires experts to demonstrate or correct actions, which scales poorly as fleets grow. This paper lets a fleet learn from its own execution records: operators only label each episode as success or failure and reset the scene. During training, the context of each failure is used to retrieve similar segments from successful trajectories, whose recorded actions are prioritized for imitation, and the updated shared policy is sent back to every robot. With collection time per robot held fixed, more robots generally means better results; on multi-stage phone packaging, learning across 96 robots raised success from 79.2% to 98.3%. Without extra expert demonstrations, success-or-failure labels alone can keep a whole fleet improving.
4.41
Mobile manipulation policies trained on offline demonstrations tend to fail as soon as they meet situations not seen in training, and for long tasks where arm, torso and base must move together, recording more demonstrations cannot cover everything. I3L lets an operator take over when the robot goes wrong. The operator has a pair of leader arms that move in sync with the robot's arms, plus a pair of handheld controllers. Gripping a leader arm or pushing a controller hands full control to the person, who corrects the arms with the leader arms and steers torso and base with the controllers. The corrections are used for retraining, the new policy goes back on the robot, and the next round targets its new mistakes. After four rounds, autonomous success is on average 68.3 percentage points above the initial policy. With the same number of training trajectories it beats demonstration-only training by 35.6 points. Spending the demonstration budget where the policy actually fails beats recording more evenly.
4.39
Whole-body motion trackers for humanoids usually map a reference motion to a single set of joint commands, so making them dodge an obstacle means hand-editing the reference or retraining. DiffWBC turns the controller into a one-step diffusion model: it outputs a distribution over future trajectories while still needing only one forward pass per control step. The motion can then be steered online without changing any weights. Write a cost, such as how close the body gets to an obstacle, take its gradient with respect to the generator's input noise, and adjust the noise directly so the generated motion avoids the obstacle. When the robot must duck under a bar 1.25 m high, the share of runs that pass without touching the bar is 0.84, versus 0.10 for editing the reference motion. Sampling the same motion several times, the more the results spread, the more likely tracking will fail, which flags problem clips without physics simulation.
4.35
When a robot runs inference while it moves, the slower the model, the later a drifting action gets corrected, and current visuomotor policies mostly depend on heavy visual backbones that are hard to run fast on edge chips. RTPolicy lets the camera views attend to each other through one cross-view attention layer. It then pools each view down to 4 tokens for a flow-matching action generator, cutting the compute for both visual encoding and action generation. On the RoboTwin simulation benchmark it reaches a mean success rate of 52.26%, the highest among the compared methods. On a real bimanual platform, one inference takes only 30.35 ms on the RDK S600 edge chip. Real-time on the edge need not cost accuracy if the multi-view images are compressed first
4.34
Many dexterous actions depend on touch, such as telling whether a bottle is held firmly while sorting, yet tactile robot data is hard to collect at scale. TacClaw-Infinity collects data with a handheld gripper whose fingertips carry tactile sensors: people use it directly at home, in labs, factories, shops and outdoors, while it records camera views, fingertip touch and gripper trajectories, with no robot present. The result is over 10,000 hours of demonstrations covering more than a thousand task types, with uniform quality control, filtering and segment annotation. On three robots with different designs, adding tactile input raised the mean success rate of pi0.5 from 50.0% to 74.0%, with most of the gain on contact-dependent tasks such as bottle sorting.
4.15
Most of today's large robot models output a chunk of actions at a fixed rate, and neighbouring actions are nearly identical, so many output slots are wasted. Looking further ahead means giving up the local precision that contact requires. Vela instead outputs a fixed number of control points, each a position in action space, which together define a smooth spline that can be queried at any time. The model also outputs how far into the future this curve reaches. In training, that label is the longest span the fixed control points can represent within an error tolerance. So the curve reaches far when motion is smooth and stays short when contact needs precision, so the same control points are packed more densely. On a real long-horizon egg-cake cooking task, Vela's mean success across four subtasks is 67.5%, versus 37.5% for pi0.5. Switching from pointwise actions to curves lets the output budget go where it is needed
ICLR 2026
Hugging Face
NVIDIA