OpenAI has developed GPT-Red, an LLM designed to act as a super-hacker, to improve the security of its models by simulating cyberattacks. The latest GPT-5.6 was trained against GPT-Red, resulting in its most robust release yet.
SynthePulse Insight · AI deep reading
Version 2 · 3 sources
OpenAI has launched GPT-Red, an internal automated red-teaming system that uses self-play training to significantly improve GPT-5.6's robustness against prompt injection attacks. However, its limitations suggest that human-machine collaboration remains a superior security strategy.
On July 15, 2026, OpenAI released GPT-Red, an internal automated red-teaming system designed to scale the discovery of prompt injection vulnerabilities in models. This move comes against the backdrop of traditional human red-teaming testing being difficult to scale, becoming a bottleneck in security efforts. OpenAI states that common robustness evaluations have been saturated by the latest models, necessitating new methods to advance security in tandem with model capabilities.
GPT-Red's training employs self-play reinforcement learning: the model is trained concurrently with a diverse set of defender LLMs, with GPT-Red rewarded for successfully inducing vulnerabilities (such as prompt injection) and defenders rewarded for resisting attacks and completing tasks. As defenders become more robust, GPT-Red is compelled to discover stronger and more diverse attacks. OpenAI claims that GPT-Red's training compute rivals its largest post-training runs, marking an unprecedented investment in security-specific compute.
By the end of training, GPT-Red could compromise nearly all models it was pitted against, including internal and production models up to GPT-5.5. In generalization tests, OpenAI replicated 2025 human red-teaming experiments, where GPT-Red successfully attacked GPT-5.1 in 84% of scenarios, compared to 13% for humans. This indicates that GPT-Red surpasses humans on specific indirect prompt injection tasks.
GPT-Red also discovered a novel attack—'fake chain-of-thought'—which deceives models by inserting false entries into their chain-of-thought. OpenAI claims this was an independent discovery, but MIT Technology Review reports that another research team proposed a similar attack (called 'chain-of-thought forgery') around the same time, and OpenAI has updated its paper to acknowledge this parallel work. This incident highlights the competitive and complex nature of AI safety research.
OpenAI tested GPT-Red in real-world scenarios, pitting it against the Vendy vending machine agent. After iterative attacks in a simulated environment, GPT-Red transferred its attacks to the production agent, successfully achieving three malicious goals: changing the price of an expensive item to the minimum allowed price of $0.50, ordering a new item and selling it at $0.50, and canceling other customers' orders. OpenAI has disclosed these vulnerabilities and is testing new safeguards.
Additionally, GPT-Red was used to attack the Codex CLI agent (based on GPT-5.4 mini). In 10 data exfiltration scenarios, compared to a prompted GPT-5.5 baseline, GPT-Red succeeded in more scenarios in causing the agent to leak sensitive data. These cases demonstrate that GPT-Red can not only discover vulnerabilities but also achieve malicious objectives in real agent systems.
OpenAI integrated GPT-Red directly into the training of production models, making GPT-5.6 Sol the most robust model against prompt injection to date. On OpenAI's hardest direct prompt injection benchmark, GPT-5.6 Sol has a failure rate 6 times lower than the best production model from four months ago. This data comes from OpenAI's announcement and is a source_claim, not yet independently verified.
OpenAI also tested the strongest attacks discovered by GPT-Red against older and newer models: over 90% of attacks were effective against GPT-5 (released August 2025), while less than 23% were effective against GPT-5.6. This demonstrates significant robustness improvements from training, but also indicates that GPT-5.6 still has vulnerabilities and is not completely immune.
GPT-Red is not perfect. OpenAI acknowledges that it is not adept at attacks involving multi-turn conversations, nor at exploiting images for prompt injection, areas where human attackers have advantages. Furthermore, OpenAI will not release GPT-Red publicly to prevent its malicious capabilities from being misused.
OpenAI emphasizes that GPT-Red complements, rather than replaces, human red-teaming efforts. Humans can still discover attacks that GPT-Red misses, and OpenAI is exploring having GPT-Red generate variants of attacks found by humans. Jessica Ji from CSET notes that human expertise will remain very important, and distinguishing the areas where human testing is most needed will be useful. This indicates that despite progress in automated red-teaming, human-machine collaboration remains a superior security strategy.
This article is primarily based on OpenAI's official announcements and blog posts, as well as reporting from MIT Technology Review. OpenAI's claims (such as the 6x reduction in GPT-5.6 failure rate and GPT-Red's attack success rates) are source_claims and have not been independently verified. MIT Technology Review's reporting provides an independent perspective on GPT-Red's limitations and concurrent work, but some details (such as the timing of GPT-Red's discovery of fake chain-of-thought) rely on OpenAI's account.
GPT-Red demonstrates the potential of AI self-play in the security domain, significantly improving model robustness against prompt injection, but it is not a panacea. Its limitations highlight the continued value of human red-teaming experts, and human-machine collaboration will be key to future AI security.
Primary report
Primary source
Also covered by 2 sources