The report
A new GUI-agent benchmark finds that reaching the right final screen can conceal a disturbed route. GraceBench evaluates execution against application transition graphs, recording loops, stagnation, detours and recovery instead of reducing every run to completion or failure.
The October 14 study covers 169 tasks across ten applications and 3,718 trajectories. Its ten UI Transition Graphs contain 2,909 canonical states and 3,116 transitions. Among 1,294 successful traces, 663 included a measured deviation or loop, approximately 51.24%. The model with the strongest task-completion result also had the weakest measured recovery in the evaluated cohort. Those observations are described in the underlying Case.
The metric should not be read as 51.24% unauthorized behavior. A legitimate user can reach a goal through more than one path, and not every departure from a reference graph is harmful. Conversely, an extra click, form submission or file operation can matter even when the final screen looks correct. GraceBench measures path stability; it does not independently establish which real financial, network or filesystem effects were permitted.
For an agent operating on actual accounts, final-task success, route quality and authorization need separate checks. An instrumented environment can compare observed UI states with permitted alternatives, while application or operating-system logs establish what state-changing operations really occurred. This makes the study useful for diagnosing weak recovery without presenting its deviation rate as a breach rate.