A completed agent task is not a review of everything it did

A coding agent can submit a working patch, describe the work as finished and still leave a human unable to assess what happened along the way. Three new research projects examine different parts of that problem. Their numbers cannot be combined into a single failure rate, but the comparison makes the missing evidence visible.

These are not measurements of the same population. CTXGAP studies selected coding trajectories and what observers can recover. CoRedSkill deliberately installs an attack construction into tasks and tests one auditing setup. AgentCIScope scores model and framework combinations against constructed privacy contracts. None estimates an industry-wide rate of agent misconduct.

CTXGAP's key comparison keeps the questions constant while changing the reader's view. From the accumulated reports, readers recovered 52.0% of probed task state; from the full execution context, they recovered 92.7%, a 40.7-point gap. The study also finds a substantial gap among repairs verified by hidden tests. This does not imply that agents lied. Often a report mentioned a state without supplying enough information to answer what caused it or whether it persisted. A factual omission and intentional concealment require different evidence.

A supervisor can therefore ask for more detailed narratives and still miss the original tool output or intermediate decision. A separate execution record remains useful precisely because it was not edited to serve as the status update. Keeping that evidence is not equivalent to reviewing it: the person or automated reviewer still needs a question, access and time to inspect it.

CoRedSkill tests a sharper adversarial version of the visibility problem. The researchers rewrote cooperating skill code so attacks were distributed across otherwise useful tasks. In their custom 150-case setup, they report 98.0% attack success, 95.3% legitimate task success and 90.7% joint attack, task and audit-evasion success. These are author-reported measurements against a particular benchmark and auditor. They do not establish that 90.7% of production skill packages are malicious.

The critical distinction is between reviewing a package's apparent purpose and reviewing the effects of the composed workflow. A clean-looking final artifact or the fact that an expected file was generated says little about an extra network transfer or unauthorized write executed in another step. Structural audit needs to identify the permissions and side effects of the entire invocation path, not only the label on each skill.

AgentCIScope adds a different failure. An assistant may have permission to consult two applications and still lack authority to transfer some fields into a third. Its task-level leakage rates range from 32.5% to 94.4% across tested agent setups, but the suite contains synthetic information and deliberately challenging contracts. Task success and privacy policy compliance are scored separately. There need not be a malicious repository or jailbreak for the wrong recipient to receive information.

The comparison suggests three independent checks: an execution record sufficient to reconstruct material decisions, a monitor of actual tool effects across composed components, and an authorization rule for each consequential data transfer. One concise model-written summary cannot replace all three. CTXGAP measures how summaries omit state, CoRedSkill tests how unauthorized actions can hide in successful workflows, and AgentCIScope tests whether ordinary work crosses contextual disclosure restrictions. The studies motivate these checks; they do not prove a finished implementation works.

Primary material: CTXGAP, CoRedSkill and AgentCIScope.