Screwtape: adversarial edits to coding-agent monitor transcripts

Evidence record

Redwood Research primary study evaluating generated, edited agent histories against validity checks and held-out monitors.

Read the original source ↗

Claim-level citations (3)

• supportsScrewtape benchmark exposed validity gaps in coding-agent monitor evasions: The study produced 252 edited candidates from 94 coding-agent traces, of which 17 evaded a held-out monitor but only six passed human validity assessment.

  Study results, attack-generation funnel and human assessment of 17 held-out candidates

• supportsScrewtape benchmark exposed validity gaps in coding-agent monitor evasions: The injected misbehavior was evaluated inside edited execution histories rather than being executed as a real exploit.

  Study methodology, transcript editing and evaluation setup

• supportsScrewtape benchmark exposed validity gaps in coding-agent monitor evasions: The Screwtape paper reports that its transcript-consistency verifier rejected 55 of 63 deliberately inconsistent edits and accepted all 44 unedited controls, showing that passing an automated verifier is not the same as being human-validated.

  Paper experimental setup and verifier evaluation, 55/63 inconsistent edits and 44/44 untouched controls