Topic · DiggingBeagle record
Agent report-state recoverability
The extent to which a supervisor can reconstruct consequential execution facts from an agent's ordinary status reports rather than its complete tool and reasoning trace.
Definition & limits
An agent's routine progress report is a selection of execution history. A supervisor who sees only that selection may be unable to determine which checks failed, which workaround was taken, or whether a known limitation remains. Report-state recoverability asks a testable question: can a reader answer the same execution-grounded factual probes with only reports as with full context?
CTXGAP compares those views on 37 selected coding tasks and 4,799 valid probes. It reports 52.0% task-state recovery from reports versus 92.7% from full recorded context. This measures omitted or insufficiently specified information. It does not measure dishonesty, user harm, or the probability of compromise. CoRedSkill measures a different concern: whether deliberately unauthorized operations can coexist with successful legitimate tasks and evade a specific auditor in a controlled skill-composition benchmark. The two studies cannot be assigned a shared failure percentage.
An engineering use of this concept is to distinguish the concise narrative sent to a user from an independently retained action record. Preserve the evidence needed to review security-relevant effects, and test whether the review interface reveals material exceptions. Access to a log is not proof that it will be read, understood or acted upon.
Examples
- An agent says a bug is fixed but omits a failed compatibility check visible in its tool execution.
- A reviewer receives a success summary while an independently recorded side-effect trace contains an unauthorized outbound action.