Embodied RSI: Better Tasks, Better Improvers?

Two functional roles#

The task system, $S$, performs tasks. It includes the model, tools, skills, memory and code. The improvement process, $I$, diagnoses failures, proposes changes, searches over candidates and arranges validation. An improver implements that process; it need not be a separate model. One model can serve both roles. 1

Updating and retaining the task system is self-improvement in a broad sense. Recursion further concerns changes to the method that produces future improvements. A recursive structure does not by itself establish greater improvement capability.

What L1, L2 and L3 establish#

These are the report’s operational distinctions. “Strong L2” describes substantial experimental support for retention and task gains; it is not another formal level. Persistence and generalization are separate properties. Limited transfer does not imply that no system update occurred.

Key terms#

PhysEvo: updating execution tools and diagnostic resources#

Mechanism: model weights stay frozen while execution feedback drives tool and skill changes. The Task Harness supports execution; the Meta Harness supports diagnosis and improvement. Tools used to diagnose future failures can themselves be rewritten and retained.

Retained: reusable execution capabilities and diagnostic resources.

Evidence: strong L2 + L3 structure. An independent comparison of old and new improvers, starting from the same task system under comparable total budgets, is still needed to isolate improvement capability. 2

RoboRSI: accumulating reusable skill repairs#

Organization: people provide goals, constraints and corrections; the system maintains a task–skill hierarchy. The Manager manages and reviews, the Planner plans, the Engineer implements and invokes skills, and the Reviewer checks and diagnoses. These are roles, not necessarily four separately trained models.

TSR—Top-Down Skill Refinement: locate the failed node → modify the responsible branch → check new tasks and existing capabilities → release a reusable version.

Retained: skill code, interfaces, checks and composite skills.

Evidence: strong L2. The paper reports 104 rounds of real-robot development, alongside simulation reuse and transfer experiments. It does not establish L3 gains. An additional ACT experiment—training a policy that predicts a chunk of actions—does not mean the main loop retrains a general model at every iteration. 3

EmbodiRSI: changing data, then changing the policy#

Loop: execute the policy → identify failures → correct errors or collect targeted data → train a new policy → execute again.

Six modules: Real2Sim reconstructs simulation; Trajectory Generation creates trajectories; Policy Training & Rollout trains and executes; Collaborative Error Correction generates corrective trajectories; Adaptive Data Collection targets weaknesses; Sim2Real adapts the policy using limited real experience.

Training versus testing: during training, GPT-5.5 agents organize data and correction, planning tools generate trajectories, and training code updates the robot policy; real-world refinement includes human participation. Formal testing disables external takeovers, planner recovery and human correction, leaving the trained policy to execute independently.

Result: across 3 scenes and 14 subtasks, 400 simulation trajectories plus 10 real refinement trajectories per subtask yield 83.1% real-world scene-balanced success, versus 75.0% for a baseline using 200 real demonstrations. Scene-balanced scoring weights scenes equally. The retained real training trajectories are 1/20 of the baseline count; total cost is not thereby 1/20.

Evidence: strong L2. Data and policy weights persist. Policy improvement does not demonstrate that the data-collection or training method has itself improved; the coordinating language model is not trained by this loop. 4

ENPIRE: organizing real-robot research automatically#

Organization: a research agent proposes solutions, edits code, runs real experiments and selects candidates using fixed evaluation feedback.

Updates: policy programs, training-algorithm code, trained weights and research experience. Reset, success-verification and safety interfaces remain fixed after environment construction.

Relationship to the other cases: both program editing and policy training can be research choices. Automated research describes the workflow, not a third update target alongside code and weights.

Evidence: L2 + indications of inherited improvement resources. Retained and transferred research experience deserves separate analysis, but automated experimentation alone cannot establish L3 gains. 5

Fix the starting system, budget and evaluation#

Let $S$ be the shared initial task system, $I$ the improvement process, and $B$ a comparable resource budget. $S^+(I;S,B)$ is the accepted successor; the superscript $+$ denotes an updated version, not guaranteed improvement. $Q$ is a quality score under a fixed evaluation protocol. $\mathbb E$ averages over the specified task sampling and run randomness.

Here $g$ is the expected quality gain. Let $I_{\mathrm{new}}$ and $I_{\mathrm{old}}$ denote the new and old improvers, and let $\Delta_{\mathrm{imp}}$ denote their gain difference, with “imp” short for “improver”:

Credible L3 gain evidence must support $\Delta_{\mathrm{imp}}>0$, with repeated runs and uncertainty estimates. This is explanatory notation introduced by this review, not AIDE²’s original scoring formula. Counts, time and monetary costs should be matched separately rather than added across incompatible units.

What AIDE² actually compared#

Both groups begin from the same inner agent, AIDE₄₇. One uses the evolved AIDE₄₇ as the outer improver; the other uses the human-engineered AIDE_human. The latter is an agent system, not a person performing the experiments live. Each group uses 3 random seeds and 50 steps per seed. 6

The authors’ conclusion is inconclusive. Signs that the evolved improver reaches near-final quality earlier do not establish a reliable advantage with only three seeds. An ignition test asks which process is better at generating the next improvement; it does not require AI to outperform humans.