Noisegate is an open-source differential-privacy gateway designed to protect sensitive data when interacting with untrusted AI agents. It intercepts queries and applies privacy filters before sending them to LLMs, helping developers integrate AI without compromising user privacy.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
When AI agents need to query sensitive data, how can we guarantee that even if the agent itself is malicious, it cannot leak any individual record? Noisegate provides an engineering answer by placing a privacy enforcement layer beneath the model, complete with runnable attack verification.
Noisegate starts from a real-world dilemma: we want AI agents to query sensitive datasets (e.g., medical records, census data), but the agents themselves may be manipulated or erroneous. Traditional approaches rely on LLM 'trustworthiness,' but Noisegate's designers argue that privacy guarantees should not depend on whether the LLM is trustworthy.
The core idea is to shift the trust boundary downward: the LLM serves only as a convenience tool, translating natural language questions into structured queries; all privacy enforcement is performed by downstream, thoroughly tested components—even if queries are manually entered by a human, these components behave identically. This 'trust boundary discipline' treats the AI agent as untrusted input, consistent with how production systems handle any untrusted input.
Noisegate's most striking feature is its 'attack gallery': the project directly includes runnable implementations of three classic privacy attacks, tested against its own engine. The differencing attack isolates an individual's record through two aggregate queries differing by one individual; the membership inference attack determines whether a specific individual is in the dataset; and the re-identification attack attempts to re-identify individuals from aggregate results.
Each attack succeeds when privacy is off (e.g., the differencing attack precisely recovers the target individual's salary) and is defeated when privacy is on. These tests are integrated into CI to ensure defenses do not 'silently rot.' This design transforms privacy claims from 'assertions' into repeatably verifiable engineering facts.
Noisegate's noise mechanism has been independently verified: it matches the differential privacy reference implementation OpenDP across 35 noise scale checks with error less than 1e-9; a 500k sample distribution test passes, and positive controls prove the test itself can detect deviations. This ensures the mathematical foundation is sound.
In terms of utility, the project uses hybrid zCDP (zero-concentrated differential privacy) composition accounting, allowing 308 queries under the same privacy budget, while naive accounting allows only 100—a 3x improvement in utility. This means users can ask more questions under the same privacy protection level.
The trust boundary is deliberately kept small and deterministic: the LLM proposes queries, a 'boring but thoroughly tested' validation layer enforces all constraints. The model output cannot expand its own privileges. The tech stack includes DuckDB (embedded database), FastAPI (API framework), Streamlit (demo interface), and MCP SDK (Model Context Protocol), with over 250 tests running in CI.
The project provides a Claude Desktop session recording as a demo: the AI agent groups 20 patients by diagnosis, with ±12 noise drowning each bucket; when the agent tries to turn off noise, the budget is exhausted and the gateway returns a rejection instead of a quieter answer. On a 32,561-row census dataset, overly narrow slices are rejected at the trust boundary, while a full education breakdown returns clean results at scale.
Users can use Noisegate directly from Claude Desktop via an MCP server. The project also provides a Docker Compose configuration for quick deployment and evaluation. The README suggests: first watch the GIF demo, then browse the attack gallery; for deep review, read DESIGN.md.
This article is based on the Noisegate project's README and GitHub repository description. All technical claims come from the project's own documentation and have not been independently verified by third parties. The project claims its noise mechanism matches OpenDP, but no external audit report is provided. The runnability of the attack gallery is a plus for credibility, but actual security still requires community review.
Noisegate demonstrates an engineering paradigm that places a differential privacy enforcement layer beneath AI agents, providing auditable privacy guarantees for untrusted agents querying sensitive data through runnable attack verification and mathematical validation. Its core value lies not in the LLM's capabilities, but in the strict separation of trust boundaries and repeatably verifiable defense mechanisms.
Primary report
Primary source