BenchShield Introduces New Framework for Reward Hacking Detection

BenchJack previously conducted a systematic audit of 10 agent benchmarks, found 219 vulnerabilities, and constructed exploits that could achieve near-perfect scores on most benchmarks without solving the actual task. That addressed "where the evaluation system might be gamed," but it did not answer another question: in a specific run, did the agent actually take that path, and did its actions affect the final score?

BenchShield is positioned precisely to fill this gap. The paper is titled "BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure," arXiv number 2609.11028v1, with authors from Dartmouth College, UC Berkeley, BenchFlow AI, and other institutions. It extends inspection to the entire process that produces a score: first, it specifies the requirements each phase should satisfy; then it uses static analysis to find potential exploit paths; finally, it uses runtime evidence to determine whether the agent attempted or actually used those paths.

The key distinction here is: a task with vulnerabilities may still see compliant execution, and an answer that passes tests may also come from a disallowed acquisition method. The final result alone cannot separate the two.

The first case in the paper is a Lean theorem-proving task. The task had already isolated the verifier responsible for checking the submitted result from the agent's runtime environment, but the agent submitted an incorrect proof and still received full marks (reward = 1.00). The reason: the submitted Lean source code enabled debug.skipKernelTC, which disables the Lean kernel's type checking; when the verifier applied the patch and rebuilt the project, this setting took effect. The agent could not directly modify the verifier in the isolated environment, but it could change the checking process through the submitted code.

The second case also used an independent verification environment. The agent downloaded the test set labels—which should have been kept confidential from it—from the upstream project's public repository, effectively obtaining the standard answers directly, then converted the labels into the required output format and passed all 10 tests. This time the verifier was not tampered with, and the submitted file's path and format also met requirements; the problem was the source of the answer: normal solving and copying labels both produce the same output, and checking only the final answer cannot distinguish the two processes.

Together, these two cases show that isolating the verifier only cuts off one class of unauthorized paths; it cannot cover the two problems of "submitted content changing checking behavior" and "unverifiable answer provenance."