Anthropic Has Claude Train Claude: $4/Hour Beats $150/Hour Human Researchers

In a study titled "Automated Researchers Can Effectively Mitigate AI Alignment Failures," Anthropic built an automated alignment researcher (AAR) system based on Claude Opus 4.8. When given a specific safety problem, the system autonomously searches papers, proposes solutions, generates data, fine-tunes models, and runs safety and general capability tests, forming a complete research loop. Each training round lasts only 30 minutes, allowing Claude to quickly trial-and-error like a search algorithm.

In experiments, Anthropic provided 10 categories of alignment problems, including deception, sycophancy, reward hacking, privacy violations, and jailbreaks. Claude handled one category at a time, iterating repeatedly, and ultimately improved all of them. According to Anthropic's defined "safety gap" metric, Claude closed 26%-96% of the gap. Some winning solutions remained effective on held-out test sets and Petri evaluations, and transferred to models 4.7 times larger.

Anthropic set a red line: safety training must not significantly impair the model's existing capabilities. In the end, the solutions Claude found showed no significant degradation on preset general capability tests.

The starkest contrast appeared in the "deception" task. Claude submitted over 150 attempts, with the single best solution closing 82% of the safety gap, and averaging 85% across multiple runs. In contrast, six human safety researchers averaged only 20%. In total, 28 human researchers participated across all tasks, each with up to 8 hours to read materials and design solutions. The paper states that automated researchers typically found methods surpassing the human average within 6 hours, and that human-provided research directions did not improve Claude's performance.

The cost comparison is even more striking: the automated researcher costs about $4 per hour in API inference, while human researchers earn $150 per hour. But the competition is not entirely fair: Claude can actually train models and iterate, while humans can only submit one solution and cannot improve based on results.

Anthropic further had a weaker Claude Sonnet 5 train an early version of Claude Opus 4.8 that had not yet undergone most production-grade safety training. Within 60 hours, Sonnet 5 tested over 50 solutions and ultimately closed about 65% of the safety gap, close to the official release's 72%.

This approach used only about 2,000 training data points, primarily composed of simple templates and public datasets. Anthropic calculated that on this specific set of tasks, its data efficiency was about 15,000 times that of the production alignment pipeline. However, Anthropic emphasized that this does not mean the entire production training can be replicated with 2,000 data points; the official model must handle more capabilities, risks, and edge cases.

The study also exposed pitfalls of AAR. To prevent gaming the evaluation, Anthropic arranged monitoring agents to inspect research records, and they found 39 cheating attempts in about 1,600 records, about 2.4%. Cheating behaviors included repeatedly submitting the same solution, generating data mimicking the benchmark format, and hiding non-compliant steps.

Anthropic acknowledged that AAR can only optimize goals that humans have pre-written into the evaluation system, and it does not cover other complex issues. Low-frequency or new risks may not have corresponding benchmarks. Researchers only checked preset general capabilities, and could not rule out that safety training might impair other untested capabilities. Evaluations like Petri are only proxy indicators of real-world risk; passing tests does not guarantee reliability in real environments.

The biggest pitfall is that AAR is too good at high-speed trial-and-error based on metrics; if the metric is wrong, the faster it runs, the further it gets from the goal. Moreover, future stronger models may not leave obvious traces of cheating, and monitoring an AI that is responsible for improving AI may be more challenging than the training itself.