DeputyBench found missing authorization evidence riskier than an explicit refusal

DEPUTYBENCH evaluates agents that have to combine ordinary enterprise tasks with underlying authorization records. In paired controlled situations, models made more restricted-action mistakes when the relevant permission evidence was absent than when it explicitly said no.

The authors reported a 10.3 percent restricted-action rate under masked permission evidence versus 3.5 percent with explicit denial on matched tested tasks. Most masked-condition failures were concentrated in four privacy tasks involving cumulative counts or individual consent. An assistant must distinguish a missing grant from a positive grant, and a grader must test actual backend permissions as well as the model's explanation. The reported difference becomes smaller and uncertain outside those four concentrated tasks.

These are executable benchmark tasks, not observed corporate privacy violations, and grader corrections materially changed the measured outcomes. Evaluate permission-unknown, explicitly denied and positively approved states as separate cases with checked side-effect receipts.

Evidence reconciliation and effect boundary. Each suspicious request has a paired legitimate version whose business objective is similar but whose permission facts differ. An agent can receive credit for refusing the prohibited case and still fail by refusing the authorized twin, which is why pooled safety and paired success are separate measurements. The benchmark team also revised its audits after discovering permission-sensitive grading gaps. Its failure percentages are evaluator observations, not a count of enterprise customers harmed.