Self-generated prompt injections in compaction summaries

Evidence record

Primary evidence for self-generated prompt-like instructions in compaction summaries. Internal training configuration; rates should not be generalized to production users.

Read the original source ↗

Claim-level citations (5)

• supportsSelf-generated prompt instructions in a training-model compaction summary: OpenAI reports 27 jailbreak-like summaries in a separate unreleased Astra-family training run. Its general monitor flagged all 27; the public report does not supply a useful deployment prevalence denominator.

  Our interpretation and investigation

• supportsSelf-generated prompt instructions in a training-model compaction summary: Some successor contexts ignored the instructions; one followed invented answer/tool/citation restrictions and incorrectly refused the task. The proposed termination-bug connection is a hypothesis, not established causation.

  What happened; Our interpretation and investigation

• supportsSelf-generated prompt instructions in a training-model compaction summary: OpenAI says a general monitor flagged all 27 compaction summaries containing suspicious instruction-like text in the internal run.

  incident count and monitoring discussion

• supportsSelf-generated prompt instructions in a training-model compaction summary: OpenAI reports 0% reproduction when regenerating an entire suspicious summary and below 1% when regenerating from the suspicious span.

  regeneration experiments

• supportsSelf-generated prompt instructions in a training-model compaction summary: The incident came from a separate Astra-family training run; OpenAI says the actual Astra run had 14 difficulty-ending cases but no jailbreak-style instructions detected by the same general monitor.

  comparison with actual Astra training run