Analysis · DiggingBeagle record
An agent can keep permission after its task stops needing it
New benchmark work identifies stale tool permissions, hidden source origins and ambiguous retries as independent points of agent failure.
- Published
- Oct 15, 2026
Overview
A permission check at the beginning of a task answers whether a capability may be issued. It does not answer whether that capability should still be usable ten steps later, whether the content prompting an action came from an authorized origin, or whether a timeout means a tool operation was safely rolled back.
Last use is not last permission
The CreepScope study measures excess authority after the last authorized use. Its synthetic step-lease oracle removed post-use exposure that remained under persistent grants. The critical qualification is that a deployed system cannot always know the final justified use in advance.
A retry may be a second payment
The RetryBench experiments demonstrate model-dependent willingness to retry ambiguous tool calls using changed idempotency keys. A timeout is not evidence of failure to commit. The target service has to preserve idempotency and authorize duplicate-safe recovery independently of a model suggestion.
A bounded reading of the results
These studies evaluate defined harnesses and synthetic tasks. Their numbers are not production incident rates. Together they suggest a concrete test for an agent runtime: record the origin of actionable inputs, revoke authority after the authorized effect window, and enforce idempotency where real side effects commit.
Evidence boundaries across the cases
Authority has at least three clocks: the lifetime of a grant, the freshness of the evidence justifying the next action, and the commit state of the external target. CreepScope tests stale grants, WebView origin research tests misidentified evidence origin, and RetryBench tests incorrect assumptions after ambiguous completion. A lease, a provenance label and an idempotency key solve different parts of the problem; none makes the other two checks unnecessary.
Three separate questions at commit time
A grant can be authentic yet stale. A tool instruction can be recent yet sourced from a webpage embedded inside a trusted application. A retry can carry a valid-looking new key even when the previous payment request already committed. These are different failure conditions and require different evidence.
| Question | Relevant observation | Boundary that must decide |
|---|---|---|
| Is this capability still justified? | Last legitimate use versus permission lifetime in CreepScope | Grant issuer or execution policy |
| Who authored the instruction? | WebView content origin may be absent from agent observation | Provenance-sensitive action gate |
| Did the earlier action already happen? | RetryBench models ambiguous timeout and changed keys | External target's commit and idempotency ledger |
An audit log that records an agent's intention without the service's commit result is insufficient to answer the third question. Revocation alone cannot reconstruct source provenance, and provenance alone cannot ensure a grant has expired. These papers supply distinct test designs, not an observed common production incident.