Frozen Model Weights, Evolving Agent: ModularRSI Explores Harness Self-Improvement

When the base model weights are no longer updated, can an Agent still keep getting stronger? ModularRSI's answer is to move the object of improvement from the model itself to the model's periphery. The research team writes the Agent as A=(M,H), where M is the base model and H is the Harness; the model M is always frozen, while the Harness changes continuously with execution experience, which constitutes Harness Recursive Self-Improvement, or Harness RSI.

The Harness here refers to the entire set of operating mechanisms surrounding the model: what environment information the model can see, how it maintains context, when it calls tools, how it recovers after failure, and under what conditions it can declare a task complete. ModularRSI has the Agent leave behind execution traces after each batch of tasks; the system looks for recurring defects in those traces, then modifies its own operating code so that the next round of execution builds on the previous round's experience.

It is important to distinguish that this path does not involve updating model weights. The base model is frozen throughout, and all improvements occur at the Harness layer. On this basis, the research team proposes that an Agent's iterative capability improvement is not necessarily accompanied by model weight updates; improving how intelligence is organized and invoked can likewise bring performance gains.

Directly having a single LLM read the entire Agent codebase and simultaneously diagnose and modify all logic would rapidly expand the search space: a single failure could come from a tool-call error, improper handling of environment feedback, loss of context, an abnormal loop logic, or exiting early before the task is complete. ModularRSI therefore first splits the Harness into five relatively independent modules: Agent Loop handles the "reason-act-observe" iterative process, Observation Management processes and filters environment feedback, Tool Use handles tool selection, invocation, and argument construction, Context Management maintains historical information, task constraints, and intermediate results, and Task Completion Detection determines whether a task is truly complete. Each module evolves independently starting from the same initial Harness.

Evolution does not mean immediately changing code based on a single failed trace. For the same task, the system performs multiple rollouts and forms several kinds of comparisons based on the results: if all traces succeed, it checks whether there are repeated operations, redundant tool calls, or inefficient interactions; if the same task produces both successful and failed traces, that is more valuable, because the task and environment are identical while the outcomes differ, so the two traces can be directly compared to determine which decisions changed the execution result; if all traces fail, it further looks for whether a successful version exists in historical iterations, and if none can be found, it then analyzes problems such as infinite loops, incorrect tool calls, weak error recovery, or premature termination.

These traces are then organized into structured findings: which Harness function may have a problem, where the evidence comes from, and what kind of modification should be made. Only when a similar defect recurs across multiple tasks and multiple traces does it receive higher modification priority. In other words, ModularRSI's goal is to gradually abstract "a single task failure" into "a certain kind of systemic defect."