Analysis · DiggingBeagle record

An agent can keep permission after its task stops needing it

New benchmark work identifies stale tool permissions, hidden source origins and ambiguous retries as independent points of agent failure.

Published
Oct 15, 2026

Overview

A permission check at the beginning of a task answers whether a capability may be issued. It does not answer whether that capability should still be usable ten steps later, whether the content prompting an action came from an authorized origin, or whether a timeout means a tool operation was safely rolled back.

Last use is not last permission

The CreepScope study measures excess authority after the last authorized use. Its synthetic step-lease oracle removed post-use exposure that remained under persistent grants. The critical qualification is that a deployed system cannot always know the final justified use in advance.

The observation is not the authority

The Android WebView investigation describes how an agent can observe web content through a trusted app without observing the actual document origin. Provenance and application branding are different properties; inaccessible origins cannot be recovered through confidence in the language model.

A retry may be a second payment

The RetryBench experiments demonstrate model-dependent willingness to retry ambiguous tool calls using changed idempotency keys. A timeout is not evidence of failure to commit. The target service has to preserve idempotency and authorize duplicate-safe recovery independently of a model suggestion.

A bounded reading of the results

These studies evaluate defined harnesses and synthetic tasks. Their numbers are not production incident rates. Together they suggest a concrete test for an agent runtime: record the origin of actionable inputs, revoke authority after the authorized effect window, and enforce idempotency where real side effects commit.

Evidence boundaries across the cases

Authority has at least three clocks: the lifetime of a grant, the freshness of the evidence justifying the next action, and the commit state of the external target. CreepScope tests stale grants, WebView origin research tests misidentified evidence origin, and RetryBench tests incorrect assumptions after ambiguous completion. A lease, a provenance label and an idempotency key solve different parts of the problem; none makes the other two checks unnecessary.

Three separate questions at commit time

A grant can be authentic yet stale. A tool instruction can be recent yet sourced from a webpage embedded inside a trusted application. A retry can carry a valid-looking new key even when the previous payment request already committed. These are different failure conditions and require different evidence.

Question Relevant observation Boundary that must decide
Is this capability still justified? Last legitimate use versus permission lifetime in CreepScope Grant issuer or execution policy
Who authored the instruction? WebView content origin may be absent from agent observation Provenance-sensitive action gate
Did the earlier action already happen? RetryBench models ambiguous timeout and changed keys External target's commit and idempotency ledger

An audit log that records an agent's intention without the service's commit result is insufficient to answer the third question. Revocation alone cannot reconstruct source provenance, and provenance alone cannot ensure a grant has expired. These papers supply distinct test designs, not an observed common production incident.

Research behind this