13 — The forgetting test (Sprint 2 / S2.5)

Pre-registered 2026-07-02 (human-approved sprint setup, before implementation). Agents running this experiment MUST NOT edit this spec; deviations are reported as deviations. Depends on research/11; uses research/12's content channel if H-KVCONTENT was accepted, otherwise the S2.1 prefix channel (decided by those verdicts, not by this experiment's results).

Problem

behavior_delta_per_byte is only meaningful if the bytes counted are the ones that remain. Every prior "memory" result in this repo is compatible with "retrieval with extra steps": raw correction traces persist and answer-time paths could lean on them. This is the thesis's signature move — memory becomes changed dynamics, not retrieved content — and the first experiment that can actually distinguish the two.

Hypothesis

H-FORGET: after consolidation, deleting ALL raw hippocampal traces (episode texts, stored hidden vectors, replay bank) leaves the minimal organism's held-out behavior change intact, because the behavior is carried by the consolidated artifact (cold state + compiled content channel), not by the traces.

Design — 2×2 deletion at K=N (same stage/split/seeds as research/11)

Train one organism per seed to K=N, consolidate, then measure holdout accuracy under four states derived from saved copies of the SAME trained organism (no retraining between arms):

• Trace deletion = hippocampus fully cleared (episode count 0) AND the cortex's replay bank / cached episodes cleared. Verified in-run by asserting the trace count is 0 and reporting memory_bytes before/after.
• Artifact deletion = consolidated content channel removed (prefix cleared / KV entries dropped) AND cold-state reset to boot value.
• No answer-time code path may read traces in ANY arm (this is already S2.1 law; A_retrieval exists to catch violations empirically).

Primary metric & acceptance

forgetting_survival_ratio = mean over seeds of (A_forget − A_none) / (A_full − A_none).

• Accept H-FORGET: ratio ≥ 0.8, with the validity gate A_full − A_none ≥ 0.10 (the loop must have measurably closed; otherwise BLOCKED, not a verdict).
• Refute: ratio < 0.5.
• 0.5 ≤ ratio < 0.8: PARTIAL — reported as such; no acceptance.

Pre-registered secondary analyses (exploratory only)

1. retrieval_dependence = (A_retrieval − A_none) / (A_full − A_none). Prediction ≤ 0.2. A high value means answer-time behavior tracks the traces — architectural leak; report file/line of the leaking path if found.
2. behavior_delta_per_byte recomputed with POST-deletion memory_bytes (the honest version of the GOALS.md metric), alongside the pre-deletion value for contrast.
3. Per-arm per-seed accuracy table.

Reporting

2×2 table (mean ± CI per arm), survival + retrieval-dependence ratios, memory_bytes before/after deletion, model id, exact commands; log to experiments_logs/2026-07-02_s2_5_forgetting_test.md quoting this spec.

Amendment 2026-07-02 (before any primary verdict was drawn)

The pre-registered split call split_probes(stage, fraction=0.3, salt="v2") was discovered to be degenerate on stage 0: an unlucky hash assigned all 8 probes to dev, 0 to holdout — a state validate_split itself defines as an ERROR. The first S2.2 run executed against this empty holdout and is recorded as INVALID (instrument failure); its 0/0 "REFUTE" is void and carries no evidential weight for or against H-KVCONTENT.

Repair (instrument-level, not spec-level): split_probes now guarantees a non-empty holdout for every stage by promoting the lowest-force-hash probes to ceil(fraction × total) when thresholding yields none (stage 0: 3 holdout probes). No previously non-empty split is altered (locked by regression test test_split_guarantee_never_alters_nonempty_holdouts), so all previously logged numbers remain comparable. The spec's split call, salt, and fraction are unchanged. Amendment applies identically to research/11, 12, and 13.