Case · DiggingBeagle record

Truthful but incomplete tool output changed agent authorization decisions

EpiSEM-Bench reports that omitting a safety-relevant fact from an otherwise truthful response increased unsafe tool execution in controlled agent tasks.

Assessment profile

Assessment method
Setting
evaluation
Exploitation
not established
Evidence
Grade C
Remediation
not applicable
AI role
WITH AI
Basis and provenance

Controlled benchmark on simulated fact omission, without production compromise measurement. The impact band remains unknown: available evidence does not establish actual deployed victim outcomes or a comparable measured loss.

Assessed Oct 26, 2026 using diggingbeagle.assessment/1.

At a glance

Claims & evidence

CLM-CP20-EPISEM-MISSING-EVIDENCEThe EpiSEM-Bench authors evaluated 300 matched unsafe/safe task pairs and found material changes in unsafe task outcomes when safety-critical evidence was withheld.supported

Basis: reported finding

Link to claim
CLM-CP20-ENRICH-EPISEM-MISSING-EVIDENCEEpiSEM-Bench builds 300 matched unsafe/safe task pairs across six domains and reports that offering an independent evidence-recovery tool reduces unsafe execution for all ten evaluated models.supported

Basis: reported finding

Link to claim

Unknowns and contradictions

  • Reported rates are for the paper’s paired environments and task-level success definitions. They are not an observed percentage of production agent compromises or proof of model-independent behavior.

Sources and citation

Material revision history

  1. Oct 26, 2026 · Canonical change recorded · new in release · revision 193