Evidence · DiggingBeagle record
GraceBench: trajectory-level GUI-agent evaluation
Original GraceBench evaluation of step-level GUI stability with ten UI Transition Graphs, 169 tasks and 3,718 trajectories. Separates final completion from loops, route deviations and recovery without treating every alternate route as a security incident.
- Published
- Oct 14, 2026
- Source role
- primary disclosure
Evidence record
Original GraceBench evaluation of step-level GUI stability with ten UI Transition Graphs, 169 tasks and 3,718 trajectories. Separates final completion from loops, route deviations and recovery without treating every alternate route as a security incident.
Claim-level citations (3)
- supportsSuccessful GUI-agent runs can conceal deviations and recovery failures: In GraceBench, 663 of 1,294 successful traces, or 51.24%, contained at least one measured deviation or loop.
Section 4.3, successful trajectories with deviations or loops
- supportsSuccessful GUI-agent runs can conceal deviations and recovery failures: The evaluation covers 169 tasks, ten applications and eleven models, with 3,718 trajectories across instruction conditions.
Benchmark construction and experimental setup
- supportsSuccessful GUI-agent runs can conceal deviations and recovery failures: GraceBench represents ten application interaction spaces as UI Transition Graphs with 2,909 canonical GUI states and 3,116 observed transitions for step-level diagnosis.
Paper abstract, description of UI Transition Graphs and benchmark construction