Huawei AI beats 25 games via RSI, using 56% fewer actions than humans

Memento 3's report card is clean: all 25 games and 183 levels in the ARC-AGI-3 public set cleared, a perfect RHAE of 100, 7518 total operations; humans need 17135 operations to complete the same set of levels, so Memento 3 uses about 56% fewer. These numbers come from research by Huawei Noah's Ark Lab together with UCL and other teams, as reported by QbitAI.

But focusing only on the "perfect score" misses the more critical comparison: the same base model, Claude Opus 5, scores an average RHAE of only about 40.7 under the ARC Prize official standard test framework. The base model is unchanged, the weights are untouched, and there is no game-specific fine-tuning, yet the score rises from about 40.7 to 100, an improvement of 59.3 points. These 59.3 points are not growth in model capability but the increment brought by the agent system—this is the core question this article pursues.

Memento 3's core method is called Code as Model. It has the AI maintain two things simultaneously during gameplay: a continuously updated Rulebook, using Markdown to record discovered patterns and rules not yet figured out; and an Executable World Model, writing the summarized rules directly as runnable Python code, essentially building its own simplified game simulator. With these two, the AI can first reason through different routes in its self-built simulator, then decide how to take the next step.

The key lies in the error-correction mechanism. The report gives an example: the AI initially thought that after stepping on a special floor tile, the pattern carried by the character would mirror-flip according to the entry direction; only after actually trying it did it discover it was a 90-degree clockwise rotation unrelated to the entry direction, so it modified the rulebook, and the corresponding Python code updated in sync. The system then re-simulated previous operations, comparing the 64×64 game screen cell by cell, confirming the code could accurately reproduce the real results; if it didn't match, it kept revising.

This iteration leaves version traces. In the symbol-decryption game tr87, the same rulebook went through 11 versions over the course of a single game, and the research team also introduced Git version management for traceability—if a change was wrong, one could review historical versions or even restore them. This is exactly what the research team calls recursive self-improvement (RSI): the large model's parameters stay fixed, and the AI becomes more skilled at playing by accumulating experience and modifying rules and code.

The method's effectiveness cannot rest on the clear result alone; the research team ran controlled experiments. In three games, when the rulebook was retained, the AI used 617 operations in total; removing the rulebook and keeping only the code world model required 677. That is, this "walkthrough" brought about 9% fewer operations, and the agent's execution rounds also decreased by about 18%.

Further still is the dual world model: having the AI maintain two world models simultaneously, sharing the game operation record but each reasoning through different rule hypotheses. In the game wa30, the single world model needed 899 operations to clear, while switching to the dual world model reduced it to 597, about 34% fewer operations.

The boundary must be made clear: the controlled experiment on the rulebook and code world model was conducted in only three games, and the dual world model improvement has so far been verified only on wa30, so it cannot be directly generalized to other games. These two numbers are evidence of the method's effectiveness, but not a universal conclusion.

The research team took Memento 3 to challenge the Atari classic Pong. Unlike puzzle games, the ball keeps moving and bouncing, so the AI must not only figure out the rules but also judge trajectories in real time and control the paddle. The AI initially thought the paddle could be divided into four zones and the ball would bounce in the corresponding direction depending on where it hit; later, a horizontal bounce failed to match the summarized rule, so it re-analyzed the collision position, modified the bounce formula, and gradually derived motion patterns closer to reality.

Once the strategy matured, the research team let the program play independently: no longer calling Claude Opus 5, and no longer updating rules. The result was 21:0 in all three different starting configurations. By contrast, if the world model is not used and the large model is simply asked to decide the next operation based on the current screen, the final score is only -19, not much different from the -20.7 of random button-mashing.

In terms of efficiency, Memento 3 learned this control strategy using only about 9500 frames of gameplay, whereas some traditional reinforcement learning methods often require hundreds of thousands or even hundreds of millions of frames of training data. This comparison shows: rather than having the large model rethink every step, it is better to first let it learn how to play, then hand the playstyle to the program for execution.

Before Memento 3, there was already a set of scores on ARC-AGI-3. Princeton University's Continual Harness cleared 3 games for 20.5 points; DreamTeam, developed with participation from NVIDIA researchers, cleared 6 games for 38.1 points; OPINE-World scored 78.4, and NOOA scored 85.1.

Closest to a perfect score was independent developer Sergey Rodionov's baseline1: all 25 games cleared, scoring 99, with 8347 operations. Memento 3 filled in the last 1 point and pushed total operations down to 7518, 829 fewer than baseline1.

The significance of this coordinate line is that the perfect score did not appear out of nowhere; after a series of methods gradually approached it, it was closed out along the dimension of "fewer operations." Memento 3's differentiation is not just clearing, but clearing in fewer steps.

Cost is a clear constraint. Memento 3 uses Claude Opus 5 with Extra-High reasoning effort enabled, and the exploration, code generation, and verification processes all consume considerable tokens and compute. The source does not give specific scale, so it is impossible to judge whether this method's per-clear cost is acceptable.

The generalization boundary is equally clear. The current experiments mainly target game environments with relatively clear rules; whether this method still works in a more complex, changeable real world remains to be validated. Whether the operation reduction brought by the dual world model applies to games other than wa30 is also unclear.

There are also several pieces of information the source does not explain that need to be flagged: whether Memento 3's ARC-AGI-3 results have undergone third-party independent reproduction; the collection conditions and sample size of the human data in the human operation comparison; and the specific type or strength of the opponent in the Pong experiment. These gaps do not affect the validity of the confirmed facts, but they determine whether this method can be extrapolated.