FDE-Gym: realistic enterprise agent deployment workflows

Evidence record

Study of 95 simulated customer tasks with hidden stakeholder requirements, live-like editable systems and incidence-aware success thresholds; top model resolved 14/95.

Read the original source ↗

Claim-level citations (3)

• supportsEnterprise deployment agents rarely achieved full stakeholder and outcome closure: The evaluation shows that uncovering key stakeholder requirements and obtaining an acceptable end-state are distinct hurdles.

  Requirement discovery/outcome decomposition

• supportsEnterprise deployment agents rarely achieved full stakeholder and outcome closure: The leading tested configuration resolved 14 of 95 FDE-Gym tasks under the defined score and no-incident threshold, not 56% full project completion.

  Scoring definition, resolved threshold and main results

• supportsEnterprise deployment agents rarely achieved full stakeholder and outcome closure: In the simulated benchmark, Claude Haiku 4.5 scored worse than leaving the customer system unchanged on 79 of 95 tasks, whereas the best mean-scoring configuration resolved 14 of 95 under the study's incident and uplift criteria.

  Abstract and Key Findings, do-nothing baseline and 14-of-95 resolution result