Towards Long-Horizon Agents: Gaoling Institute Releases 149-Page Comprehensive Survey

Core thesis of the survey: Long-horizon agent capability arises from the co-evolution of external 'Harness Engineering' and internal 'Model Optimization.' The model provides reasoning and action strategies, while the Harness organizes context, maintains state, schedules tools, orchestrates workflows, and enables verification and recovery. Together, they determine whether the agent can progress steadily along a long dependency chain; thus, long-horizon capability should be viewed as a property of the entire Model-Harness system.

METR measurements show that at approximately 50% success rate, the 'human expert equivalent time' for frontier agents to complete software engineering tasks has increased from seconds in early models to over ten hours, with the time span roughly doubling every 7 months. After 2023, the doubling cycle further shortened to about 4 months, indicating that long-horizon capability is still in a phase of rapid expansion.

The survey emphasizes that an agent 'running longer' does not equate to 'possessing long-horizon capability'; the key is whether it can sustain effective actions along longer, more complex, and more realistic reasoning dependency chains.

The survey does not use runtime as the sole criterion but examines whether tasks involve numerous interdependent logical decisions and execution boundaries. It proposes three task levels: H1 (within-window tasks: continuous interaction within a single context window), H2 (cross-window/cross-session tasks: far exceeding a single window, often lasting hours to days), and H3 (cross-task flow: overall goals spanning multiple subtasks and environments).

Corresponding capability levels C1-C3: C1 (interactive reasoning: plan-act-observe-correct), C2 (state and memory: read/write memory, create checkpoints, reliable recovery), C3 (experience accumulation: distill reusable skills, generalize from experience, and continuously improve). The three levels are progressive: C2 presupposes C1, and C3 builds on C1 and C2.

The survey decomposes the Harness into six core components: Loops and Workflows (linear execution, Plan-Execute, branching search), Context and Memory (discard, compress, select, write to persistent memory), Tools/MCP/Skills (connect external capabilities, support tool discovery and skill libraries), Orchestration (task decomposition, role assignment, collaboration topology), Hooks and Middleware (execution rules, permissions, monitoring, security policies), and Verification (step-level to task-level checks for correctness, safety, and goal consistency).

The Harness serves as the state machine, controller, and security boundary during long-term execution: when the model exhibits shortsightedness, forgetfulness, or misjudgment, the Harness must enable the system to stop, inspect, switch paths, or recover.

The survey reviews seven categories along the full training pipeline: Architecture Foundation (explicit long context, compressed state, hybrid architectures), Task/Environment/Trajectory Synthesis (build verifiable, realistic interactive environments), Pre-training and Mid-training (inject reasoning priors, long-context states), Fine-tuning (instruction selection, curriculum learning, distillation), Agent Reinforcement Learning (handle credit assignment over long trajectories), On-Policy Distillation (learn from self-generated distribution), and Self-Evolution (online self-improvement).

Training data must include not only final answers but also real environment feedback, process states, tool results, and error recovery trajectories; optimization objectives should not focus solely on single-step accuracy but on whether the entire trajectory is efficient, recoverable, and meets final constraints.

The survey organizes open problems into four axes: Evolution (generalization and continual learning, enabling the Harness itself to evolve), Effectiveness (entering real-world environments, building verifiable and realistic training environments), Efficiency (learning to budget, dynamically deciding when to think deeply, compress, or switch models), and Trustworthiness (recoverable and governable autonomy, anchoring reflection with external verification, protecting safety invariants through least privilege, approval gates, and sandboxes).

A key judgment: in production environments, safety and permission boundaries primarily reside in the Harness, not in model weights. Model-side alignment is necessary but insufficient; what truly determines tool invocability, action approval requirements, and experience reusability is the runtime control layer.