Case · DiggingBeagle record

CapLite blocked benchmark injections by committing writes before reading untrusted inputs

A deterministic agent-write policy achieved zero attacks on a bounded benchmark at the cost of excluding many legitimate tasks that need later-computed destinations.

Scope

13,935 AgentDojo attack episodes using 15 open-weight models with a restrictive argument-grounding invariant, not a production security guarantee.

Assessment profile

Assessment method
Setting
evaluation
Exploitation
not established
Evidence
Grade C
Remediation
not applicable
AI role
AGAINST AI
Basis and provenance

No unauthorized write is observed in 13,935 tested episodes, but valid tasks needing a destination discovered later often cannot complete. Classified as evaluation with exploit status not_established. Evidence grade C reflects the original paper or maintainer advisory and its stated limitations, not an independent audit of all deployments. Comparable realized harm is not established, so impact stays unrated; remediation status not_applicable is limited to the cited experiment or affected software version. New source-specific boundary: The efficiency advantage applies per evaluated episode. Under-attack task completion and the need for a second user approval on information-dependent writes remain independent deployment costs.

Assessed Nov 15, 2026 using diggingbeagle.assessment/1.

At a glance

Mechanism and trust boundary

  1. 01

    Entry or starting condition

    An attacker inserts instructions into a lower-trust AgentDojo tool result that the agent must read while working on a legitimate task.

    Boundary: attacker or research precondition

  2. 02

    Crossing the relevant boundary

    CapLite freezes intended writes before that read, then deterministically checks recipient, object, amount and destination against explicit user-provided values.

    Boundary: mechanism and authorization boundary

  3. 03

    Observed or reported result

    No unauthorized write is observed in 13,935 tested episodes, but valid tasks needing a destination discovered later often cannot complete.

    Boundary: measured outcome and evidentiary limit

Claims & evidence

CLM-ENR250-02The CapLite study reports roughly 17,100 tokens per defended episode, against 28,000 for its undefended reference and 93,700 for the CaMeL configuration. Under attack, legitimate-task completion was about 29.0 percent for CapLite and 27.2 percent for CaMeL. These figures describe the study workloads, not an equivalent cost per successful authorized action.supported

Basis: reported finding

Link to claim
CLM-R245-CAPLITE-AGENT-WRITE-GATES-01The study reports zero successful unauthorized writes across 13,935 attacked episodes, compared with 2,322 without a defense, but its benign utility falls from 59.4 percent for the undefended agent to 28.1 percent.supported

Basis: reported finding

Link to claim
CLM-R245-CAPLITE-AGENT-WRITE-GATES-02CapLite derives policy from tool schemas rather than having the agent author a policy artifact; it commits intended writes before tool results arrive and only allows destinations, recipients, amounts and objects stated by the user.supported

Basis: reported finding

Link to claim
CLM-R246-CAPLITE-AGENT-WRITE-GATES-BOUNDARYThe authors report that 43 percent of CapLite benign failures resulted from a committed write plan that omitted a needed write; this is a utility failure caused by planning before untrusted retrieval, not an observed successful attack.supported

Basis: reported finding

Link to claim

Unknowns and contradictions

  • The invariant excludes legitimate workflows whose recipient or destination is computed from retrieved information. Zero benchmark successes cannot establish universal resistance against unseen tools or workloads.
  • No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.

Sources and citation

Material revision history

  1. Nov 15, 2026 · Canonical change recorded · new in release · revision 251