Persistent assistant memory can increase user disclosures through repeated contextual prompting

A study of assistant memory asks a different privacy question from whether an attacker has stolen a conversation log. It tests whether the assistant's own references to remembered attributes influence what the user reveals in later dialogue. Researchers used 300 synthetic profiles with 24 attribute types and 80 sessions per profile, plus a small eight-person human pilot. A separate inference model tried to recover attributes from the resulting record. The measured outcome is inferability from accumulated conversation content, not confirmed transfer to an unauthorized recipient. Full research paper.

The mechanism is a feedback loop: the assistant mentions or presupposes a remembered fact, the user affirms, corrects or declines it, and the new response itself becomes evidence about the attribute. An incorrect remembered value can also become entrenched because correcting it requires disclosing what the value should be. Even a refusal to discuss a topic may narrow what a model later infers. The absence of a malicious external URL therefore does not mean the privacy effect is absent; it arises through the ordinary conversational turn sequence.

When the study suppressed attribute references over selected sessions, the authors reported first disclosures falling from 58 to 37 per 1,000 and newly inferred attributes from 96 to 58 in their chosen evaluation scale. They describe response-side suppression as retaining more of the benefit of personalized answers than heavy filtering or anonymization in their studied setup. Those numbers are intervention-specific, not a prevalence estimate for actual consumers or proof that one response filter solves memory governance.

The operational test is to separate collection, memory retrieval, reuse in generated responses and external sharing. An agent may be allowed to remember a detail without being authorized to restate or presuppose it during unrelated tasks. Further work needs consenting real-user experiments, disclosure-quality tradeoffs and checks of what survives source disconnection and memory reset. The memory-disclosure loop names the mechanism without implying confirmed exfiltration.