First-hand information for everyone
Main finding Users sometimes want AI assistants to forget specific disclosures. The authors call this the “negative space” of AI memory and show it includes contextually sensitive disclosures beyond standard privacy lists (F1, F2).
Survey and taxonomy The team used a Critical Incident Technique survey to collect recalled incidents. Participants reported platform, context, what they disclosed, and why they later wanted it forgotten (F5). After screening, the dataset contained 179 incidents from 146 participants (F6). Iterative qualitative coding produced a taxonomy of 15 categories of information users want excluded from persistent AI memory. Two researchers independently coded the incidents and achieved κ = 0.83 (F7).
Fence: a policy-level gate prototype The authors implemented the taxonomy in a prototype called FENCE (Filtered Entry of seNsitive Conversation to mEmory). Users choose categories to protect as standing rules. When a later conversation matches selected categories, FENCE prevents that conversation from entering persistent memory by default. The system treats the conversation as the unit of enforcement and allows case-by-case overrides (F3).
Prototype implementation and defaults The prototype classifies disclosures locally (Qwen3.5-9B via Ollama) and uses a separate API for reply generation (Gemini in the implementation). The default protective state is “vanishing,” which excludes the conversation from memory and expires by default after 24 hours; users can extend this to three days or one week (F10).
Benchmarking automated category detection The authors created a 1,300-message benchmark with synthetic, real, and non-sensitive examples. Three researchers labeled each message on 15 binary categories, producing 19,500 message–category decisions; 93.2% of those decisions were unanimous (F11). They evaluated several models. Gemini 3.8 Flash achieved balanced accuracy .944 and recall .944. GPT-5.6 Luna was close (BA .935, recall .931). Qwen3.5-9B reached BA .886 and the highest reported precision (.682). Models performed better on cases where annotators agreed than on split decisions (F12).
User study In a within-subjects study (N = 15), participants tried a baseline memory-management interface (C0) and the Fence prototype (C1) in counterbalanced order. Both conditions were seeded to simulate an established account (12 saved memory items, 8 prior conversations) (F8). Participants rated Fence higher on usability, interpretability, and trust. Median usability rose from 5.75 to 6.25. Interpretability rose from 3.50 to 4.50. Trust rose from 3.00 to 4.00. Wilcoxon signed-rank tests with Bonferroni correction reported these improvements as statistically significant (Usability p_adj = .038; Interpretability p_adj = .028; Trust p_adj = .036). Mental effort did not show a clear reduction (F4).
Limitations and caveats The survey drew on participants’ recalled incidents and may miss less salient concerns or issues that only appear through later reuse (F13). Participants were U.S.-based, English-speaking adults; results may not generalize across cultures (F13). The authors did not report category frequencies as prevalence estimates because recruitment targeted people with prior forget experiences and participants could report multiple incidents (F14). The benchmark’s reference labels reflect majority judgments from three researchers and are an operationalization rather than a population-level ground truth; ambiguity at category boundaries reduced classifier performance on split cases (F15).