Back to feed
News Story
meng shao (X)T3
1 sources

为 Claude Fable 5 设计「成本高效 Harness」 这种成本高效设计同样适用于 GPT-5.6 Sol，核心思路是：大多数任务的智能需求在 to…

Original

The article proposes a cost-efficient design for Claude Fable 5, also applicable to GPT-5.6 Sol, based on the insight that intelligence needs are asymmetric across tokens. It introduces three allocation modes (orchestrator, advisor, verifier) and two experiments showing significant cost savings with minimal performance loss. Key design guidelines include task shape analysis, delegation heuristics, coordination cost evaluation, and prompt cache optimization.

SynthePulse Insight · AI deep reading

Cost-Efficient Harness: Intelligent Asymmetric Allocation for Frontier Models

Version 1 · 1 source

By recognizing the asymmetric demand for intelligence at the token level, injecting frontier model judgment only into critical tokens can achieve performance close to full usage at significantly reduced cost.

  • Intelligence demand in most tasks is asymmetric across tokens: a few tokens require frontier judgment, while the majority only need cheap execution.
  • Three allocation modes: Orchestrator, Advisor, and Verifier, each suited to different task shapes.
  • Parameter Golf experiment: Fable 5 as Advisor + Sonnet 5 as Executor achieved ~90% of Fable-solo gain at 34% token cost.
  • BrowseComp experiment: Coordination arbitrage emerged in the full set (~31M tokens/question): 96% score at 46% cost; no advantage in the easy subset.
  • Two main sources of coordination cost: boundary duplication (tokens counted at least twice across models) and fan-out overlap (workers don't communicate).
  • Key design guidelines: examine task shape, use delegation heuristics, evaluate coordination costs, and ensure prompt cache hits.

Core Idea: Asymmetry of Intelligence Demand

Frontier models like Claude Fable 5 and GPT-5.6 Sol do not require equal intelligence for all tokens in most tasks. A few tokens depend on frontier judgment, while the majority only need cheap execution. The core of a cost-efficient harness lies in recognizing this asymmetry and injecting frontier intelligence only in the right places, thereby substantially reducing total cost.

Based on this idea, three allocation modes are designed: Orchestrator (Fable 5 orchestrates, delegating cheap workers), Advisor (Fable 5 acts as advisor, consulted by a lower-tier executor), and Verifier (Fable 5 verifies outputs). Different modes suit different task shapes: Advisor fits when judgment is spread throughout the process, while Orchestrator or Verifier fits when judgment is upfront or at the end.

Experimental Validation: Parameter Golf and BrowseComp

In the Parameter Golf experiment, Fable 5 as Advisor intervened at the start and at two checkpoints, while Sonnet 5 as Executor ran 20 rounds of ML experiment design. The result achieved ~90% of the performance gain of Fable running solo at 34% of token cost. Counterintuitively, upfront planning was ineffective—Fable's initial ranking was anti-correlated with effective direction; real value came from mid-stage checkpoints that corrected Sonnet 5's tendency to hill-climb on marginal gains.

The BrowseComp experiment tested multi-constraint web retrieval. In the easy subset (~0.37M tokens/question), Fable solo was actually cheaper; coordination added 60% cost with no benefit. But in the full set (~31M tokens/question), Fable orchestrating + Sonnet workers achieved 96% score at 46% cost—arbitrage emerged. Coordination costs mainly stem from boundary duplication and fan-out overlap, and savings only outweigh fixed handover costs when the worker absorbs enough tokens.

Design Guidelines and Ultimate Direction

Four harness design guidelines: 1) Examine the task shape and choose the corresponding allocation mode; 2) Use delegation heuristics to provide the model with the worker's taste and an intelligence prior ranking; 3) Evaluate coordination costs to ensure the delegated token volume is large enough; 4) Ensure prompt cache hits to avoid paying context write fees each time a new worker is spawned. Low cache hit rates directly offset the cost advantage of lower $/token.

The ultimate direction is for Fable 5 to write its own harness on the fly based on the task. The knowledge of task shape analysis, delegation cost trade-offs, and cache management provides the foundation for Claude to autonomously generate cost-effective harnesses that 'selectively apply frontier intelligence'.

Credibility boundary

This article is based on a technical post by AI analyst meng shao on X platform. The experiments were conducted by the author; data is source claim, not official benchmarks. Core ideas and design guidelines are analytical inferences. Overall credibility is moderate; it can serve as a reference for cost optimization strategies for frontier models.

Insight takeaway

The key to cost-effective usage of frontier models is selective application rather than substitution. Through asymmetric intelligence allocation and carefully designed coordination mechanisms, substantial cost reduction can be achieved while retaining most of the performance.

Sources for this version

  1. 为 Claude Fable 5 设计「成本高效 Harness」 这种成本高效设计同样适用于 GPT-5.6 Sol，核心思路是：大多数任务的智能需求在 to…

    meng shao (X)

Primary report

meng shao (X)T3

Primary source