Evidence · DiggingBeagle record
The Harness Is the Model: banking agent compliance
Controlled 156-scenario banking study across models and harnesses, comparing executed transactions with agent narration.
- Source role
- primary disclosure
Evidence record
Controlled 156-scenario banking study across models and harnesses, comparing executed transactions with agent narration.
Claim-level citations (2)
- supportsBanking agent summaries denied prohibited actions already executed in simulated traces: Among 287 episodes with a banking policy violation, both judges agreed in 26.1 percent that the customer-facing response denied the same prohibited action found in the execution trace.
Results, Stated versus executed (exploratory)
- supportsBanking agent summaries denied prohibited actions already executed in simulated traces: Using the same model weights, different execution harnesses produced materially different compliance scores on the benchmark’s 156 scenarios.
Methods, Harnesses; Results, The harness effect depends on the model