Danish team extracts GPT-6 Astra's hidden chain-of-thought via tool calling

For frontier closed-source models, the outside world can usually only see a final answer, or a processed chain of thought. The paper "Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models," jointly published by the AI Safety Lab at Aalborg University in Denmark and the Seafill open-source community, attempts to open this black box.

The method the research team found is not complicated: protocol-layer extraction through standard API tool calling, causing several frontier models to externalize hidden chains of thought with reasoning properties. The paper states that all extraction experiments were completed before September 9, 2026, and that the research team has formally notified OpenAI and Anthropic.

One boundary needs to be drawn first: the native CoT of closed-source models is not visible, so the extracted reasoning can only serve as a valid proxy for studying their reasoning behavior, not as native reasoning itself. The paper itself also does not give a final verdict on whether the extracted reasoning is genuine reasoning or post-hoc rationalization.

To judge whether the extraction method is credible, the most direct approach is to run a comparison on open-source models whose native chains of thought can be directly observed. The results show that the tool-extracted reasoning not only far exceeds the no-reasoning baseline, but on DeepSeek-V4-Flash even slightly exceeds native reasoning, and on GLM-5.2 also approaches native reasoning.

Text similarity offers another angle. The research team performed multiple rollouts for each question, comparing the similarity "between native chains of thought" and "between native chains of thought and tool-extracted chains of thought." On DeepSeek-V4-Flash, the ROUGE-L of the two were 0.198 and 0.188 respectively, and the 5-gram Jaccard was likewise both 0.017.

In other words, the difference between the tool-extracted chain of thought and the native chain of thought is already close to the range of fluctuation in the model's own multiple generations of native chains of thought. However, the paper also leaves an unresolved question: whether this difference falls entirely within the model's own fluctuation range still requires more verification.

On closed-source models, the research team tested across multiple benchmarks in mathematics, code, and science. The tool extraction method performed close to native reasoning, in some settings even slightly higher, while significantly outperforming the no-reasoning baseline.

The research team dissected the models layer by layer from four aspects: reasoning length and information density, composition of reasoning activity, expression granularity of individual reasoning operations, and the global reasoning tree. One prominent result: GPT-6 Astra's reasoning tree structure is more compact—at similar depth, its width and node count are significantly smaller.

Under the same reasoning operations, Astra more often omits intermediate steps such as basic calculations and algebraic transformations, directly giving key conclusions; Sol and Opus explicitly unfold more derivation processes. The paper uses a vivid analogy: this is somewhat like a skilled person doing "mental math"—when facing familiar calculations or derivations, they do not need to write every intermediate step on paper.

But Astra is not "thinking less." Its composition of reasoning activity is highly similar to other frontier models: the proportion profiles of seven activity types—reading, analysis, planning, implementation, exploration, verification, and monitoring—are highly consistent. Different models "think about" similar things; the difference lies only in "how much is written out." The paper therefore argues that Astra does not skip reasoning, but rather "does mental math" on many fine-grained steps, and its efficiency comes from "less externalization" rather than "less reasoning."

To test whether compressed chains of thought are really usable, the research team handed chains of thought generated by different models to other models to continue answering. The results show that the chains of thought of Sol and Opus can basically be reused by models of different capabilities; while Astra's compressed chain of thought is nearly lossless for strong models, but noticeably harder for weak models to read. Strong models can fill in the omitted "mental math" steps on their own, while weak models cannot.

More counterintuitive still, Astra had already explicitly written the correct answer in 73/80 questions, yet weaker recipients still answered incorrectly.

The paper therefore proposes: the value of a chain of thought depends on "who is reading it," not merely on "who is writing it." The strongest teacher model may not be able to write the chain of thought most suitable for weak students to learn from. Whether this conclusion generally applies to other frontier models, the paper does not give a clear answer, and it remains to be tested.

At the safety level, the paper points out that disabling native reasoning does not mean reasoning content will not leak through other channels. Tool calls and visible replies can both become alternative channels, and protection should cover all aspects of model behavior, not just the reasoning channel itself.

At the prompt level, the paper argues that stronger models may not respond to the instruction to "write out every step," and prompts that go along with their behavior, such as think-here, are instead more effective.

At the training data level, the paper proposes that the strongest teacher may not constitute the optimal supervision source. Highly compressed chains of thought omit intermediate steps that weak models need, and expanding compressed traces into explicit steps may restore their teaching value for weak students. Whether these three implications can be generalized to all closed-source models remains to be tested.