Archestra Releases OpenAPPA, Achieving 0% Attack Success Rate on Two Major Security Benchmarks

Archestra has released an open-source security engine called OpenAPPA, aimed at stopping data exfiltration caused by prompt injection or model hallucination. The team reports that running OpenAPPA on two security benchmarks, Bench-Corp and AgentThreatBench, achieved zero successful attacks; in the same comparison, Claude Code's auto mode had a 10% attack success rate and Microsoft FIDES had 31%. The Bench-Corp benchmark contains 20 multi-step enterprise workflows.

The appeal of these numbers is that they rewrite "agent security" from a probabilistic problem into a deterministic one. But the boundary must be drawn first: the 0% attack success rate and the 10% and 31% comparisons all come from the Archestra team's own reporting, with no third-party independent verification found in the sources. So for now they can only be read as vendor self-reported benchmark results, not as reproduced conclusions.

The target of OpenAPPA's documentation criticism is very specific: the industry's answer to approval fatigue is to add another model to judge every tool call, such as Claude Code's auto mode and Codex's auto-review and other automatic modes. The documentation argues that this kind of approach has two design-level flaws.

The first is data flow tracking. The documentation points out that a second-model judge is architecturally unable to track data flow across tool calls; and because the classifier itself can be prompt-injected, the execution framework hides tool outputs from it, so the judge cannot see the data at all. The second is the probabilistic ceiling. The documentation states that because of the probabilistic design, even the best model judge has an accuracy ceiling of only 99.3%, and at millions of calls, 0.7% means a large number of leaks.

Both points are OpenAPPA's own arguments, and count as source-party claims. They explain why OpenAPPA chose a different path, but they do not themselves constitute an independent evaluation of competitors.

OpenAPPA's core idea is to place policy enforcement outside the agent's prompt and execution loop. According to the source description, this positional choice means the underlying language model cannot inspect, negotiate with, or bypass policy rules. In other words, the security decision is no longer made with the participation of the constrained model itself, but by a pluggable engine outside the loop.

This trade-off also explains why a simple tool allowlist or denylist is not enough. The article points out that agents have proven able to bypass such simple mechanisms—for example, a denied rm -rf can be replaced by an equivalent Python script. An allowlist constrains tool names, while an agent can migrate between semantically equivalent tools.

OpenAPPA implements the Agentic Permissions Policy Algebra (APPA) proposed in its paper, whose authors include Arseny Kravchenko, Vadim Liventsev, Innokentii Konstantinov, Ildar Iskhakov, and Matvey Kukuy. Policies are expressed as a single appa.toml configuration file, which defines data sources, audiences, trust levels, and permissions.

The mechanism simultaneously labels and monitors audience (the set of authorized consumers) and trust (the degree of data verification). Labels combine monotonically via lattice algebra and can only become stricter: reading restricted records narrows the audience, and reading unreviewed external web pages lowers trust. This monotonicity is the source of determinism—labels will not be relaxed because of what the model says.

Each tool contract defines three main operational attributes: requires (the audience membership and trust level needed to run the tool), delta (the restrictions imposed when the tool returns data), and effects (the audit trail of successful operations). Tool calls are therefore constrained by requires conditions—for example, process_internal_data requires the audience to be within internal, and apply_db_migration requires the data to be trusted.

OpenAPPA has explicit recovery semantics. When an agent attempts an illegal operation, the engine stops dispatch and provides structured paths, including a Sanitizer that can edit the payload, an Authority that routes the request to a human operator or an internal verification API, and Disposable Child Branches that are isolated on demand.

This design responds to the tension between security and usability. OpenAPPA's GitHub repository reminds developers that agent security has two axes: an agent that allows unauthorized flows is insecure, while an agent that rejects valid work is useless. Over-tightening policy reduces practicality—the agent does not leak data, but cannot successfully complete the task because of the restrictions.

It should be noted that the description of Disposable Child Branches at the end of the source summary is truncated, and its full mechanism and applicable conditions are unclear, so here we can only confirm that it is listed as one of the recovery paths and cannot elaborate further.

Even accepting the architectural logic of "enforcement outside the loop + monotonic labels," several questions remain unanswered in the sources. The 0% attack success rate is only self-reported by the Archestra team, lacking third-party independent reproduction or verification; the 10% and 31% comparison figures likewise come from vendor self-reporting, and their test conditions and comparability are unclear.

The benchmarks themselves are also not transparent. The specific composition, difficulty, and evaluation methodology of the two benchmarks, Bench-Corp and AgentThreatBench, are not fully disclosed in the sources. The specific origin and evaluation scope of the 99.3% probabilistic judge ceiling are likewise unexplained.

More critically, there is the production environment. OpenAPPA's actual protective effect in real production environments, rather than in benchmark tests, is still unclear. Zero successful attacks on a benchmark and maintaining zero leaks in a long-running, multi-tool production system with real data flows are two things that need to be proven separately.

Stringing these clues together, what OpenAPPA is really betting on is not some benchmark score, but auditability. The documentation notes that a rule set that is too tight will break the agent, while one that is too complex means no one can audit its permitted scope. A single appa.toml plus the three attributes requires, delta, and effects attempts to compress "what this agent is allowed to do" into a configuration a person can read through and review.

This is also the fundamental divergence from the probabilistic judge route: the latter hands the decision to a model that can likewise be injected and cannot see the full data; the former hands the decision to deterministic rules outside the loop, at the cost that the rules must be written clearly by humans and carry the risk of being written wrong.

So the reasonable attitude toward OpenAPPA is this: the architectural direction deserves serious consideration, while the 0% figure should be treated as a vendor claim awaiting verification. Until third-party reproduction and production-environment data appear, it proves that "this path may be viable," not that "this path is already proven."