Google researchers have developed a new method called RRSI to prevent self-improving AI agents from memorizing their test tasks, which often causes performance gains to shrink or disappear on new tasks. RRSI improves scores on unseen benchmarks by up to 4.7 points while using about 30 percent fewer tokens than an unregularized version.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
Google Cloud AI Research and several universities propose RRSI, which suppresses memorization of test tasks by self-improving agents by limiting edit budgets and removing benchmark-specific tricks. The paper self-reports gains of up to 14.1 points on training tasks and up to 4.7 points on five unseen benchmarks, while using about 30% fewer tokens at runtime than the unregularized version.
Modern AI agents wrap a fixed language model in what is called a harness, a framework of prompts, workflows, tools, memory and logic that determines what the model sees at each step. The paper says much of the recent progress in agents has come from improvements to the harness rather than from new models.
The researchers call the practice of having a language model repeatedly rewrite the harness based on test-task feedback a practical form of recursive self-improvement. But the paper points out that because the agent repeatedly processes the same limited set of test tasks, it memorizes those tasks: training-task scores rise while gains on new tasks shrink or even disappear.
The researchers say memorization happens in several ways: the search remembers patterns that fit only one benchmark, prefers candidates that score high purely by luck, and stacks pointless complexity to raise test scores.
When the system proposes new changes, RRSI uses a budget to limit the number of independent edits a single candidate can bundle, and that budget shrinks over time: early on it allows larger rewrites, and later only small changes clearly traceable to results.
RRSI records previous attempts to avoid repeatedly chasing the same failed ideas, and when progress stalls it deliberately tries parts of the harness that have not yet been touched.
When selecting changes, a critic reviews each proposal and removes any that hardcode task names, answers or other benchmark-specific tricks. Another rule is that higher compute cost is accepted only when it brings measurable performance gains, and components that are no longer useful are removed.
The researchers tested RRSI on eight benchmarks covering coding, agentic office work and engineering design, with the underlying model Claude Opus 4.8 kept frozen throughout, and compared it against an unmodified baseline harness and four recent optimization methods.
According to the paper, RRSI improves up to 14.1 points on training tasks and up to 4.7 points on five never-before-seen benchmarks, while using about 30% fewer tokens at runtime than the unregularized version.
On all unseen benchmarks, RRSI's overall performance was never below the baseline, which is what typically happens with a harness that has memorized its tasks.
All methods performed well on training tasks, but the results reversed on new tasks, with two methods even falling below the baseline harness; RRSI's training gain was the smallest of all variants, and it was the only method clearly above baseline on unseen tasks.
This trade-off is exactly what RRSI's constraint mechanism pursues: trading part of the gain on training tasks for stable generalization on new tasks.
A coding harness optimized with Gemini 3.5 Flash, without any modification, raised the much weaker Gemini 3.1 Flash Lite's accuracy from 11.2 points to 14.6 points.
The paper's authors say the code has been made public on GitHub. The authors also note that the study covers only harnesses built around a frozen model, and does not address cases where model weights change.
The facts in this article come from THE DECODER's report on a paper by Google Cloud AI Research and several universities. The proposal of RRSI, the definition of a harness, the framing of recursive self-improvement, the memorization problem, the edit budget, history recording and exploration, the critic, the compute-cost rule and the test setup are facts confirmed in the report. Figures such as 14.1 points, 4.7 points, about 30% token savings, no unseen-benchmark result below baseline, the smallest training gain, and cross-model transfer from 11.2 to 14.6 points are all self-reported by the paper's authors and lack independent third-party verification; this article labels them as source claims. The study does not address self-improvement when model weights change, whether RRSI's results generalize to other task domains remains unclear, and the source provides no independent evaluation of how effective the critic actually is at removing benchmark-specific tricks.
The core of RRSI is not to make agents score higher on test tasks, but to use a shrinking edit budget and strict review to constrain self-improvement toward changes that are traceable and generalizable. The gap between the paper's self-reported 14.1-point training gain and 4.7-point unseen-benchmark gain, along with the roughly 30% token savings, points to a more modest judgment: self-improvement can make an agent genuinely more capable only when it turns repeated feedback into durable changes that do not depend on a specific benchmark.
Primary report
Primary source