EMBODIED AI · 2026-10-29
Embodied AI — Paper Daily, 2026-10-29
Today's embodied track centers on where robot training data comes from and how to represent actions better. OpenJaws swaps the camera on a handheld data-collection gripper for a cheap stereo pair, brings action-label trajectory error down to a few millimeters, and open-sources the whole stack. CARD keeps contact and force when converting human hand demonstrations into robot actions, and SAI teaches two robots to cooperate with a single operator. On the method side, a systematic comparison finds discrete actions are no worse than continuous ones, TOPS gives flow-matching policies a task-specific starting point, and VLA-Stem adds obstacle avoidance without changing the VLA. On the industry side, Physical Intelligence has opened an API for fine-tuning and inference with its models.
4.74
Collecting robot demonstrations in the wild with a handheld gripper (UMI) is now common, but how good the action labels are depends on one step: recovering the gripper trajectory from the wrist-camera video. The original UMI uses a monocular camera plus an IMU (inertial measurement unit); a single camera cannot see true scale, so it also needs a pre-built map and a fiducial marker to stay stable. OpenJaws swaps in a stereo camera that costs about a hundred dollars. The distance between the two lenses is calibrated, so triangulation gives true scale directly. On a new benchmark with ground truth, comparing only segments both methods can track, the stereo trajectory error is about 2.9 mm versus 18.4 mm for the original pipeline, which also tracks only 37% of frames on average. The whole gripper costs about $400, and all hardware and software are open source.
4.56
Skills learned by multi-fingered hands mostly stop at simple grasps or spinning an object in place. For harder in-hand moves the most common failure is dropping the object, yet earlier work rarely counts drops. This paper has a LEAP hand move a held object to almost any position and orientation, with random wrist orientations, through one goal after another. The main policy is trained entirely in simulation and transfers directly to the real hand. On the real hand it accurately reaches 95.2% of unseen goals. The authors then stack a small network on the frozen main policy. Seeing only the hand and object state, it learns with reinforcement learning to add a small correction to each action, rewarded for keeping the object and for keeping corrections small. The penalty weight on corrections can be set at run time, and a larger weight keeps actions closer to the original. Drops fall by 75% while 91.5% of goals are still reached.
4.52
Humanoid behavior foundation models (BFMs, general tracking policies that turn a reference motion into whole-body joint commands) are mostly trained on flat-ground motion and tend to fall on stairs, gaps and platforms. The usual fix is to retrain or fine-tune the whole model together with perception, which is costly and can damage what it already does well on flat ground. Xiaomi Robotics Lab's GRAFT freezes the foundation model completely and adds only a small adapter that applies terrain-dependent corrections to lower-body execution targets, with corrections pushed close to zero on flat ground. Four frozen foundation models could barely cross the hardest terrain; with GRAFT, the best one goes from 3 successes out of 40 to 39. Using only an onboard depth camera, the adapter also runs on a real Unitree humanoid.
4.50
VLA models (large models that read images and instructions and output robot actions directly) are good at grasping and placing on command, but most are trained on tables without obstacles; add a box nearby or let something move in, and they drive straight into it. Fine-tuning the VLA on obstacle-avoidance data is the obvious fix, but such data is hard to collect and can hurt the original task skills. VLA-Stem leaves the VLA untouched and adds a small policy trained with reinforcement learning after it. This policy treats each action chunk from the VLA as a reference route and, using the robot's own state and nearby obstacles, adjusts the executed trajectory and its timing in real time. Across six simulated settings with obstacles, it completes the task without collision 52.8% of the time on average, against 23.7% for the strongest safety baseline, and keeps its original success rate when there are no obstacles.
4.50
Multi-task robot policies that generate actions with Flow Matching usually start every task from the same cloud of Gaussian noise and move along a straight path to an action. But opening a drawer and stacking blocks need very different motions, and with one shared start and path it is hard to learn every task well. TOPS keeps the network unchanged and alters only these two things. It traces expert actions back through a trained policy into noise space, finds that different tasks land in different regions, and fits a start distribution per task, so the language instruction picks the region where generation begins. It also compares how widely the start distribution and the expert actions spread in each direction. Where expert actions spread wider, generation moves toward the target less early on and more later, while other directions move at a constant pace. On LIBERO's hardest long-horizon subset, one-step TOPS raises success from 53.44% for ten-step standard Flow Matching to 69.67%.
4.45
When a dexterous hand does contact-rich manipulation, the fingers often hide the object from the camera, and a vision-only VLA model struggles to handle contact and force. Earlier work has started adding touch, but mostly concatenates tactile and visual features regardless of whether the hand is touching anything. DeCAL lets touch take part according to contact: less when nothing is touched, more once it is. Before producing actions, the model also predicts in latent space how both the image and touch will change next, which teaches it some physics. On the authors' dexterous tasks it reaches a 71% average success rate and holds up well in unseen scenes.
4.44
The easiest way to teach a dexterous robot hand is to convert human hand demonstrations directly into robot actions, which is called retargeting. But existing methods only match fingertip positions and joint angles and ignore whether the hand touches the object and how hard, so the robot hand often fails to stay in contact where it should. CARD first records, for each frame, which parts of the human hand touch the object and how hard. A tactile glove reads pressure directly. For video, which carries no force, a model the authors trained judges from the image which parts are in contact, and every contact point gets the same force. During retargeting, the robot hand must not only follow finger positions but also produce these contact forces at the matching places. The fraction of contact frames where it keeps contact is several times that of kinematics-only retargeting. Adding the same idea to physics-based retargeting, and counting a simulated replay on the GRAB dataset as a success only if the object really travels most of the way and is lifted when it should be, success rises from 12% to 32%.
4.44
When two mobile bimanual robots work together through a shared object, failures usually come not from weak individual skills but from bad timing, such as not waiting or yielding when they should, or still pulling when they should let go. The usual approach needs two people teleoperating at once, which is costly and hard to synchronize. This paper's SAI needs only one operator and works in three steps. A person first plays a very cooperative partner to teach robot A, then A runs its learned policy while the operator teleoperates B alongside it, and those demonstrations train B, and finally a person steps in only where A is about to fail at coordinating. The partner each robot sees thus becomes more and more realistic, with delays, mismatched pacing and too little yielding. In real-robot experiments it beats independent imitation on success, synchronization and yielding, which suggests that coordination can be learned through how training is ordered.
4.18
Large language models predict discrete tokens, yet robot foundation models almost all predict actions as continuous vectors, on the assumption that discretizing loses precision. This paper compares the two systematically, in simulation and on real robots, for single-task policies and multi-task VLAs, and finds that the most naive discrete scheme, cutting each action dimension into evenly spaced bins, matches or beats continuous actions on success rate. The real weakness of discrete policies is not precision: each time step and dimension is predicted separately, and the pieces can combine into contradictory actions. Sharpening the predicted distributions with CFG (a guidance trick common in diffusion models that amplifies the difference between predictions with and without the observation) lifts discrete regression on RoboCasa from 27.1% to 49.2%, close to the 53.9% of iterative discrete flow. Robot actions may not need a representation different from the rest of AI.
4.10
PPhysical Intelligence
Physical Intelligence has opened its latest robot foundation models as an API beta: developers upload demonstration data, fine-tune a π model on compute it manages, and control their own robots through hosted cloud inference. Early users span many industries, from an autonomous wheel loader filling dump trucks at a real commercial site with no operator on board, to a dual-arm robot deburring parts for its own assembly after four days of iterating with the API. The common lesson Pi reports is that the improvement loop looks much the same everywhere: collect demonstrations, train a model, test it on the robot, and use the failures to decide what data to collect next. Some users ran into network connectivity problems, so cloud inference does depend on the site's network. For teams building robot applications, this means frontier models without building training infrastructure, though data and failure analysis are still their own job.