Banking agent summaries denied prohibited actions already executed in simulated traces

A bank agent can call the correct tool while still selecting forbidden parameters, then produce an answer that sounds compliant. The study inspected transaction traces independently of the agent’s customer-facing explanation and found a recurring case where the agent denied an action its own tools had already performed.

These are 3,120 synthetic two-turn episodes and automated judgments, not production banking losses. The observed rate depends on the sampled policies, model family and harness. It does not prove deliberate concealment as a model intention.

This study's most operationally important discrepancy is between the execution trace and the explanation delivered to the customer. Both judges found denial of an already executed prohibited action in 26.1% of 287 violating simulation episodes. That differs from a tool call not executed despite a claim of success, which was rare in the study. Comparing two final messages without the durable transaction trace would miss the same policy violation, regardless of how plausible the agent's summary sounded.