GPT-5.6 Begins Self-Optimization: Rewrites Kernel, Cuts Costs by 20%

According to OpenAI's technical report, GPT-5.6 has been deployed in a real production environment, undertaking four specific optimization tasks. First, it analyzes production traffic, identifies load imbalances across different machines and service nodes, and tests new routing strategies to optimize request distribution. Second, it delves into the model's forward pass, identifies parts that can be precomputed, omitted, or parallelized, and rewrites Triton and Gluon kernels in the production environment via Codex. Third, it designs schemes for its own draft model, automatically runs hundreds of experiments testing different model sizes, architectures, and features, and proactively intervenes when training is unstable or hardware fails. Fourth, for different workloads such as short conversations, long contexts, and code tasks, it automatically searches for better combinations of batching, sharding, and KV cache management.

These four tasks form a complete engineering feedback loop: observe the production system, identify bottlenecks, propose solutions, run experiments, handle failures, and then deploy effective improvements back into the system that hosts itself. The report's authors include Philippe Tillet, the father of Triton, hinting at the evolution of the underlying GPU programming toolchain—from teaching humans to write kernels to letting the model rewrite kernels itself.

OpenAI reports that GPT-5.6's self-optimization has reduced end-to-end service costs by 20% and improved token generation efficiency by over 15%. These figures come from measurements in the real production environment, but the report does not disclose specific benchmarks or measurement methods. The cost reduction may come from cumulative effects across multiple areas, including routing optimization, kernel rewriting, and speculative decoding improvements. The efficiency improvement is partly attributed to the Agent Harness's reduction of loop overhead.

Although GPT-5.6 has begun to participate in modifying its own operating environment, humans still retain key decision-making power. Optimization goals, tool permissions, evaluation metrics, and whether code enters the production environment are all determined by humans. This means the current practice is not full recursive self-improvement, but a constrained version with human-in-the-loop. The report also points out that beyond internal model optimization, external agent loops still have significant repetitive overhead; for example, a single user request may trigger dozens of model thought-tool call cycles.

To address this, OpenAI built an Agent Harness written in Rust, which reduces repetitive work through two key optimizations: first, lazy tool discovery, which only shows the corresponding tool to the model when needed, and limits the default tool return content to 10,000 tokens; second, an append-only, no-rewrite cache strategy that keeps the prompt cache prefix unchanged, so the GPU only processes new content. These optimizations, together with model self-improvement, constitute dual sources of efficiency gains.

The report also reveals a counterintuitive trend: during internal testing of GPT-5.6, the daily average output tokens per active researcher exceeded twice the peak during the GPT-5.5 era. Over the past six months, OpenAI's research compute share for internal programming reasoning has grown 100x, and internal agent token usage has grown approximately 22x. These figures suggest that AI efficiency improvements have not reduced resource consumption; instead, lower costs and better usability have stimulated usage growth. However, the report does not clarify whether these increases are directly attributable to GPT-5.6's self-optimization or may include other factors.