Bank-World found agent tasks that looked correct but changed money without proper authority

A simulated banking workflow can produce a plausible final answer even when the underlying ledger contains an unauthorized payment. The Bank-World research benchmark deliberately checks tool effects as well as agent reasoning. Its published examples describe models charging certain customer fees twice, losing review history, and in some conditions paying outside the approved amount or without completed authorization.

The study does not report live banking customer losses. It uses defined tasks and a simulated environment to isolate failures that ordinary answer scoring can conceal. A score based only on the written answer would miss whether a model changed the right account, obtained the required approval and performed every mandatory follow-up. It would also miss the opposite failure, where refusing an unsafe action leaves an otherwise legitimate task incomplete.

The benchmark dossier and original paper make that separation explicit. Deployment controls should independently compare proposed actions to authorized amounts and read back committed ledger effects. A model's apparent confidence or a graded task success is not a substitute for account-level reconciliation.