EMBODIED AI · 2026-11-13

Embodied AI — Paper Daily, 2026-11-13

Today's thread in this track is how to adapt a robot policy to new conditions by changing only a small piece instead of retraining the large model. RAGu and GUARD both freeze the original policy and correct actions during generation with an outside signal, one using retrieved trajectories and the other an anomaly score of how far a state is from the training data. SFF refits only the last layer of a failure monitor online, and after deployment Rho likewise updates only a small module. REMAC and Thermo-VLA add a small change through LoRA or residuals, handling inference delay and the new thermal modality respectively.

4.67

1. Real-Time Robot Execution with Masked Action Chunking

Running the next model call while the robot executes the current action segment is meant to avoid pauses, yet done naively it lowers success. Earlier work blamed poor joins between consecutive segments. The authors point to an overlooked cause: by the time the model finishes, the robot has already taken a few steps from the old plan, so the first steps of the new segment never run, yet training still supervises them. REMAC randomly masks the first several steps of each segment in training, computes loss only on the part that will run, and learns a LoRA correction that is merged back before inference, adding no latency. Under the largest delay in simulation, success rises from 0.451 to 0.779. Inference delay is a known condition, so building it into training beats patching at inference time.

4.65

2. Runway’s Praxis-1 world action model for robotics is now available

Runway

Runway, a video generation company, has turned its video foundation model into a robot policy called Praxis-1, released with open weights under Apache 2.0 along with inference and fine-tuning code. It is a 27B-parameter MoE diffusion Transformer that predicts upcoming frames and robot actions together. Video prediction stays in training throughout, so the model predicts the scene's next frame and then outputs the motion that gets there. Runway says web video matches teleoperated robot video as pretraining data. On RoboLab-120 it reaches 46.2% success, against 28.0% for π0.5. If that holds, the data bottleneck for robot policies becomes how much general video a model can learn from.

4.51

3. Zero-Shot VLA Adaptation with Retrieved Guidance

Concatenating retrieved demonstrations into the model context is the standard RAG recipe for language models, but it does not carry over to robots. The authors retrieve demonstrations from real data such as DROID, whose viewpoints and objects do not match the target scene, and the context-concatenation baseline never succeeds on the real robot, doing worse than no retrieval. RAGu first warps the retrieved trajectories geometrically to the object positions in the target scene, then picks the one requiring the least change to the model's own actions. While the model generates actions, they are corrected toward that trajectory, with weights and context untouched. Real-robot success rises from 20% to 80%. Retrieved results need not enter the context; they can also guide generation.

4.50

4. Spatial Representation Distillation and Knowledge Routing for Vision-Language-Action Models

To give a VLA 3D understanding, one either feeds extra depth maps or point clouds at inference or trains 3D knowledge into the original visual encoder, which can damage the general visual ability from pretraining. SPARK-VLA keeps the original visual pathway and opens a second LoRA pathway beside it, which learns from a frozen 3D model, VGGT, during training; VGGT is not needed at inference. Each pathway yields one token per image position, and a small router decides from the instruction which ones go to the language model. The router's labels come from the gradient of the action loss with respect to each token, and tokens that lower the action error count as useful. In an ablation, concatenating all tokens gives 94.0% success, while routed selection gives 98.0%. With several feature sources, selecting beats passing everything.

4.50

5. Thermo-VLA: A Thermal-Aware Vision-Language-Action Model with Gated Residual Modulation for Robotic Manipulation

Some tasks require picking objects by temperature, such as removing the cage holding a dead mouse that is no longer warm, which an ordinary camera cannot tell apart, so thermal imaging is needed. Existing approaches feed the thermal image to the VLA as just another picture and succeed only about half the time on this task, often grabbing a salient heat source. Thermo-VLA uses the background temperature as a reference to split the thermal image into views colder and hotter than the background, weights the two by the instruction text, and adds them as residuals onto the RGB tokens, leaving the RGB representation intact. Across four temperature-dependent tasks it averages 0.86 success, 18 percentage points above π0.5 with thermal input added directly. Injecting a new modality as instruction-weighted residuals interferes least with existing visual ability.

4.48

6. Streaming Failure Forecasting: VLA Failure Monitors That Keep Learning After Deployment

After deployment, lighting, objects and cameras change, and the policy gets updated or moved to new tasks, yet the failure monitor is usually trained offline and then frozen. On real-robot runs of four VLAs, the authors find that once the task or policy version switches, the frozen monitor ranks failures barely better than chance. SFF's fix is cheap: the monitor network stays fixed, and after every episode the last linear layer is refit in closed form from one label, whether the episode succeeded. The alarm threshold is tuned separately and moves only after successes, to hold down false alarms. On 434 real-robot runs, the AUROC for ranking failures rises from 0.536 when frozen to 0.656. Ranking comes from refitting the last layer online, false alarms from a separate threshold, the same idea as freezing a recommender backbone and updating only its last layer online.

4.47

7. UniLatent: Cross-Embodiment Transfer via Latent Observation Alignment

Robot demonstration data is expensive and tied to one specific arm. Switch arms and the arm's shape and occlusions in the image change, so the policy stops working. The usual fix repaints the source robot frame by frame into the target robot, but repainting leaves artifacts, and the policy learns them too. UniLatent aligns in representation space instead: in simulation it poses both arms at the same end-effector pose, renders paired images, and trains the source robot's visual encoder so its embedding matches the frozen target encoder. The policy trains on embeddings of the source data and at deployment reads the target robot's images directly. In simulation, mean success rises from 44.5% with the strongest repainting baseline to 76.0%. Aligning in representation space sidesteps the artifacts of repainting pixels.

4.47

8. ReSkill: Selective Self-Improvement for Robot Manipulation

When a robot meets a new multi-step task, it usually fails at only a few steps, so retraining everything is wasteful. ReSkill has a coding agent break the task, during development, into an object-centric task graph in which each node calls a skill. The agent uses each node's execution records to find the failing step and revises only the flow or skill there. Every change must run on the full development layouts and win a paired comparison with the current version before it is kept. At deployment the robot runs only the revised program and never calls the agent. Across ten simulated tasks, average success rises from 30.0% for the initial versions to 86.2%, and the median revision changes only three functions. The large model only edits programs during development; production runs fixed code.

4.46

9. GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation

Robot policies trained with diffusion or flow matching tend to fail once execution drifts into states the training data did not cover, and the task reward is usually just final success, too late to correct course. GUARD learns an anomaly score from the policy's training data that measures how far the current state is from that data, treats it as a dense per-step reward, and trains a Q-function on it together with the sparse success reward. During action generation, the Q-function's gradient corrects the denoising, pushing actions toward completing the task without leaving the training distribution, while the policy weights stay unchanged. On four dexterous-hand tasks average success is 96.0%, and using the success reward alone scores 30 percentage points lower. Distance from the training distribution is a ready-made dense reward.

4.40

10. Rho: A Foundation for Efficiently Adaptable VLA Models

Microsoft has released Rho, an open-weights family of VLAs for dual-arm manipulation. Its core finding is that inserting an intermediate stage between general pretraining and task fine-tuning, trained on many tasks for one specific robot and called midtraining by the authors, halves the data needed for later task fine-tuning. The paper's explanation is that pretraining sees many robots but learns each shallowly, so learning a new task directly also means learning to control that robot. On a real FR3 Duo dual-arm setup, with the same plug-insertion training budget, success rises from 60.0% without midtraining to 86.7% after two midtraining epochs. Splitting robot adaptation from task adaptation makes the task step cheap, the same logic as domain continual pretraining before business fine-tuning in recommendation.