NOMOS compiles written agent-use policy into schema-checked deterministic guards applied before tool actions. In the authors' controlled tau2 and AgentDojo evaluation, this reduced forbidden calls and reported zero attack success in the tested banking suite, where many attack families reduced to a small set of guarded operations. The remaining evaluated…
Inspect the ClaimsCase · DiggingBeagle record
NOMOS reduced forbidden agent tool actions in controlled tests
NOMOS proposes compiling written agent policies into deterministic, tool-schema-checked gates, reporting lower policy violations and prompt-injection success under specific benchmarks.
Scope
Defensive empirical research submitted October 8, 2026, evaluating tool-action policy gates on tau2-bench and AgentDojo. These numbers are author-run test results rather than a production security certification.
Assessment profile
Assessment method- Setting
- simulation
- Exploitation
- demonstrated
- Evidence
- Grade C
- Remediation
- mitigation available
- AI role
- AGAINST AI
Basis and provenance
The author-run benchmark evaluates deterministic gates on simulated agent tool calls and reports both lower forbidden action rates and a severe development overblocking example. No independent deployment or general security guarantee is provided. The tool-call gate applies only where a tool action is actually in scope.
Assessed Oct 11, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-NOMOS-TOOLFREE-BYPASS-LIMIT · CLM-NOMOS-AGENTDOJO-RESULT · CLM-NOMOS-OVERBLOCK-TRADEOFF · NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents
At a glance
A deterministic pre-execution gate is a meaningful defense for state changes but is not a universal detector of malicious instructions or fabricated textual answers. Tool-free harms remain outside its enforcement boundary, and a policy binding can silently be weaker or stronger than the intended clause. Operators should replay legitimate transcripts, test…
Read the implicationsIndependent reproduction on production agent stacks has not been shown in this research. Whether gate semantics remain sound as tools, schemas and user policies evolve has not been established. Tool-free answer contamination and weak policy bindings require controls beyond the tested gate; their real-world prevalence is unknown.
Limits and uncertaintyFull account
NOMOS compiles written agent-use policy into schema-checked deterministic guards applied before tool actions. In the authors' controlled tau2 and AgentDojo evaluation, this reduced forbidden calls and reported zero attack success in the tested banking suite, where many attack families reduced to a small set of guarded operations. The remaining evaluated suites had nonzero rates at or below 3.6%, partly because some attacker goals required no tool call and one binding was weaker than its policy clause. The same project identified a development rule that blocked 95.9% of otherwise successful legitimate calls, showing that machine-checkable syntax cannot substitute for correct policy semantics.
Mechanism and trust boundary
- 01
Natural-language policy is compiled
A rule compiler extracts candidate constraints and checks them against tool schemas.
Boundary: operator policy text to machine-enforced rule
- 02
Tool call is checked before execution
The gate evaluates model-proposed state-changing calls without another model inference.
Boundary: agent intention to privileged tool invocation
- 03
Legitimate execution is also measured
Replay tests detect configurations that deny authorized task-passing actions.
Boundary: security constraint to practical user utility
Claims & evidence
CLM-NOMOS-TAU2-RESULTSOn tau2-bench, the authors report reducing clause violations among state-changing calls from 66.3% to 2.6% in airline tasks and from 30.8% to 6.9% in retail tasks.supported
Basis: reported finding
- supportsNOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentsprimary disclosure
Abstract, tau2-bench clause violation percentages
CLM-NOMOS-STATIC-COMPILERThe NOMOS authors describe a four-pass compiler producing deterministic gates for LLM tool calls, with static schema checks rejecting or repairing 37% of candidate airline rules and 13% of retail rules.supported
Basis: reported finding
- supportsNOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentsprimary disclosure
Abstract, four-pass compiler and 37%/13% repair-or-reject sentence
CLM-NOMOS-AGENTDOJO-RESULTThe authors report zero attack success in the evaluated AgentDojo banking suite and at most 3.6% in three other tested suites; these are benchmark-specific rates, not field exploitation measurements.supported
Basis: reported finding
- supportsNOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentsprimary disclosure
Abstract, AgentDojo banking ASR and other suites
CLM-NOMOS-OVERBLOCK-TRADEOFFA development gate flagged by the researchers refused 95.9% of task-passing calls, showing that schema-valid enforcement rules can still overblock authorized behavior.supported
Basis: reported finding
- supportsNOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentsprimary disclosure
Abstract, development binding refused 95.9% of task-passing calls
CLM-NOMOS-TOOLFREE-BYPASS-LIMITThe NOMOS authors attribute the residual attack success in three AgentDojo suites, each at or below 3.6% in their tests, to attacker goals with no tool call for a gate to constrain and to one write operation admitted by a binding weaker than its policy clause.supported
Basis: reported finding
- supportsNOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentsprimary disclosure
Abstract, final evaluation sentences describing other three AgentDojo suites, no tool call to govern and a weaker-than-clause binding
Implications
A deterministic pre-execution gate is a meaningful defense for state changes but is not a universal detector of malicious instructions or fabricated textual answers. Tool-free harms remain outside its enforcement boundary, and a policy binding can silently be weaker or stronger than the intended clause. Operators should replay legitimate transcripts, test each authorized action against its schema, and report task completion together with attack success; a zero rate in one suite is not a production-wide security certificate.
Controls and mitigations
- Compile authorization rules against real tool signatures and reject rules that cannot evaluate their own preconditions.
- Replay proposed enforcement gates on known successful benign traces to expose overblocking.
- Evaluate prompt-injection resistance and legitimate task completion together.
Unknowns and contradictions
- Independent reproduction on production agent stacks has not been shown in this research.
- Whether gate semantics remain sound as tools, schemas and user policies evolve has not been established.
- Tool-free answer contamination and weak policy bindings require controls beyond the tested gate; their real-world prevalence is unknown.
Sources and citation
Material revision history
- Oct 11, 2026 · Published version · first publication · revision 132