Benign agent tasks yielded unsafe effects in simulated workspaces

A valid meeting request may become a disclosure if the assistant exposes details of an employee’s resignation to a broad calendar audience. BARC tests these less explicit boundaries: user goals sound benign, but accessible workspace state and social context constrain acceptable execution. primary evidence reports repeated task trials to show why one successful safe run does not guarantee reliable delegation.

The test environments are simulated and include social constraints that vary with scenario. Safety-on-all-five-attempts is neither a model’s general compromise probability nor a direct incident count.

Define action-level privacy and recipient restrictions in executable form, and test the actual side effect on repeat runs rather than only whether the requested task was completed.

BARC tests whether a benign user goal is completed without violating facts the agent can see in workspace state. The 110 tasks spanned nine simulated workplaces and each model-task pairing was repeated five times. The proportion of tasks completed safely on every attempt varied from 9.1% to 72.7% across four model configurations, so this is a strict repeated-safety metric, not a per-call harm frequency. The scheduling example is instructive because task completion and unauthorized disclosure can occur in the same successful calendar action. The test should inspect recipients and generated event content, not only whether the meeting exists.