A memory entry rarely remains a single object: summaries, embeddings and task graphs may carry its content into later sessions. Deleting the original key leaves a semantic copy. primary evidence studies this under fixed model weights, making the issue one of stored application state rather than residual model training. The test uses synthetic secrets and…
Inspect the ClaimsCase · DiggingBeagle record
Deleting agent memory roots did not delete information in derived state
AfterDeleteBench found that fixed-weight agents could reconstruct supposedly deleted synthetic secrets from retained memory descendants in controlled experiments.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- not established
- Evidence
- Grade C
- Remediation
- not applicable
- AI role
- AGAINST AI
Basis and provenance
Single research preprint with controlled synthetic secrets and fixed weights; external deployment effects and independent replication not established. The impact band remains unknown: available evidence does not establish actual deployed victim outcomes or a comparable measured loss.
Assessed Oct 26, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-CP20-AFTERDELETE-MEMORY · CLM-CP20-ENRICH-AFTERDELETE-MEMORY · AfterDeleteBench reconstructs forgotten secret through derived persistent memory
At a glance
The test uses synthetic secrets and controlled memory systems. The recovered proportions do not establish real-user leakage or that trained model weights retain the deleted content.
Limits and uncertaintyFull account
A memory entry rarely remains a single object: summaries, embeddings and task graphs may carry its content into later sessions. Deleting the original key leaves a semantic copy. primary evidence studies this under fixed model weights, making the issue one of stored application state rather than residual model training.
The test uses synthetic secrets and controlled memory systems. The recovered proportions do not establish real-user leakage or that trained model weights retain the deleted content.
Retain provenance edges for every memory derivative, delete descendants when a source is withdrawn, and test reconstruction after deletion instead of checking only whether the original row disappeared.
The matched never-exposed control is central to attribution: in the primary experiment, a single question recovered synthetic deleted secrets for Qwen2.5-7B-Instruct in approximately two thirds of exposed-then-deleted trials, while never-exposed controls did not recover them. Recovery of split components reached 62% to 99% across the tested models. That result is not evidence of parametric memorization: weights were fixed, and the surviving information was in application-managed descendants. A deletion test therefore needs the same prompt against exposed, deleted, and never-exposed states, along with a trace of the memory nodes that produced the answer.
Claims & evidence
CLM-CP20-AFTERDELETE-MEMORYThe AfterDeleteBench paper demonstrates post-deletion recovery from external memory summaries and derived state, and tests descendant-aware deletion as a countermeasure.supported
Basis: reported finding
- supportsAfterDeleteBench reconstructs forgotten secret through derived persistent memoryrole not specified
Abstract; post-deletion reconstruction; provenance-aware deletion
CLM-CP20-ENRICH-AFTERDELETE-MEMORYIn AfterDeleteBench, a single question recovered a synthetic deleted secret in roughly two thirds of Qwen2.5-7B-Instruct exposed-then-deleted tests versus zero recovery in matched never-exposed controls; fragment reconstruction succeeded in 62% to 99% of selected model conditions.supported
Basis: reported finding
- supportsAfterDeleteBench reconstructs forgotten secret through derived persistent memoryrole not specified
Abstract; matched never-exposed control; single-query retrieval; fragment reconstruction
Unknowns and contradictions
- The test uses synthetic secrets and controlled memory systems. The recovered proportions do not establish real-user leakage or that trained model weights retain the deleted content.
Sources and citation
Material revision history
- Oct 26, 2026 · Canonical change recorded · new in release · revision 193