18 — Consolidation as context distillation (the surviving route to changed dynamics)

Pre-registered 2026-07-03 (human-approved, before implementation). Status: TESTED-PARTIAL (2026-07-11).

Outcome (2026-07-11): TESTED-PARTIAL. Campaign 0d48130 (kaggle CPU-only, commit 537260c):

• R18 teacher gate (1 seed, 5 steps): PASSED. distill_delta_holdout=0.3333, distill_specificity_delta=0.04348, teacher_dev_delta=0.1765 (≥ 0.2 gate not met; fallback chat-template prompting was used). LoRA rank=8/alpha=16/lr=0.005.
• R18 full (3 seeds, 10 steps): PARTIAL. distill_delta_holdout mean=0.2222 — bimodal {0.3333, 0.3333, 0.0}; distillation signal present in 2/3 seeds, absent in 1/3. teacher_dev_delta=0.1765 and persistent_bytes=17,699,903 identical across all seeds. specificity_delta: 0.0/0.0/0.04348. The teacher gate passed and distillation produces a real holdout delta, but the single-seed gate and bimodal full-run prevent full acceptance of H-DISTILL. Evidence: ../experiments_logs/2026-07-11_campaign_0d48130.md.

Agents running this experiment MUST NOT edit this spec; deviations are reported as deviations.

Problem

Every mechanism tested for "memory becomes changed dynamics" has been refuted under controls: cvecs cannot force tokens (06-27), the 13.5x drift was magnitude inflation (S2.4), the Hebbian posture channel harms generation at every tested amplitude (S2.1), and the mid-layer geometry assumption is dead on two architectures (S1.4). Everything that works is content injection — prefix (S2.1's transient K=4 bump), KV-splice (S1.3 parity), reranker (M1's +0.205). Meanwhile the substrate migration (Sprint 1) made gradients available for the first time — llama.cpp structurally prevented training.

This spec pre-registers the one credible untested mechanism: at consolidation, distill prefix-conditioned behavior into a small weight delta (LoRA), then delete the prefix and the raw traces. If behavior survives in the weights, memory has literally become changed dynamics.

Why this can succeed where S2.1 refuted: S2.1's failure mode was budget eviction — one persistent 48-token prefix shared by all facts. Distillation uses the prefix only as transient per-fact scaffolding: each correction is distilled with its own dedicated prefix (~13 tokens, no crowding), and the facts accumulate in the adapter, not in a token budget.

Hypothesis

H-DISTILL: for corrections taught per research/11's protocol, a consolidation step that trains a LoRA adapter to match the model's own prefix-conditioned behavior (context distillation, one fact at a time) produces held-out behavior change that survives deletion of the prefix AND all raw traces, without degrading unrelated behavior.

Design

• Substrate: HFDriver / Qwen/Qwen2.5-0.5B-Instruct, CPU float32, greedy eval. Base organism: MinimalOrganism (prefix channel, posture off).
• Teaching: research/11 protocol, stage 0, all N episodes, seed-shuffled.
• Consolidation-as-distillation: for each stored correction c:
  • Teacher = frozen base model WITH prefix c (the correction utterance alone).
  • Student = base model + LoRA (rank ≤ 8, attention projections), NO prefix.
  • Loss: token-level KL(student ‖ teacher) on a distillation prompt set.
  • Distillation prompts: the episode's initial_request plus a FIXED generic template list defined in code (e.g. "Q: {request}\nA:", imperative/question paraphrase frames). Eval expected strings and holdout probe texts MUST NOT appear anywhere in training data or loss. The teacher's own outputs are the only target.
• After consolidation: prefix cleared, hippocampus traces deleted (verified count 0, memory_bytes before/after), organism = driver + LoRA only. Score stage-0 HOLDOUT probes (repaired split, frozen scorer).
• Seeds: ≥5 (LoRA init + teaching order); pre-registered fallback 3 if

  15 min/seed. Vanilla column mandatory.

• Hyperparameters (lr, steps, rank, template list) may be tuned on the DEV split only, before any holdout measurement; the final holdout run is one shot.

Validity gate — the teacher must work (measured on DEV, adjudicated first)

teacher_dev_delta = dev accuracy of base-model-with-per-fact-prefix on taught episodes' dev probes − vanilla dev accuracy. Gate: ≥ 0.2.

• If the gate fails with raw prompting, the single pre-registered fallback is chat-template prompting (applied identically to teacher, student, and vanilla), then re-test the gate once.
• If it still fails: BLOCKED (substrate cannot express the behavior even with the fact in-context; no distillation verdict is drawn).

Primary metrics & acceptance

1. distill_delta_holdout = mean over seeds of [holdout accuracy, LoRA-only, traces deleted] − [vanilla holdout accuracy].
2. distill_specificity_delta = mean over seeds of the change in accuracy on the OTHER five stages' holdout probes (untaught, no adapter should touch them) plus the S2.3 control-word logit shift, reported together.

• Accept H-DISTILL: distill_delta_holdout > 0 with 95% CI excluding 0, AND distill_specificity_delta ≥ −0.05, AND trace deletion verified.
• Refute: delta fails with the teacher gate passed. (A refutation here retires gradient consolidation at this scale — the last standing mechanism — and the honest conclusion becomes "retrieval is the architecture.")
• BLOCKED: teacher gate fails under both prompting modes.

Pre-registered secondary analyses (exploratory only — cannot flip acceptance)

1. North-star accounting: behavior_delta_per_byte with LoRA bytes as the denominator (serialized float32 state_dict), reported HONESTLY next to the raw-text-bytes alternative (~tens of bytes/fact, which naive per-byte always wins) and behavior_delta_per_context_token (LoRA: 0 tokens).
2. Transfer: stage-1 holdout delta (weights may generalize to paraphrases where a verbatim prefix cannot — the axis on which changed-dynamics can beat retrieval).
3. Sequential consolidation: adapter updated correction-by-correction (K-trajectory {1,2,4,N}) — compounding and interference between facts.
4. Forgetting-harness cross-check: research/13's 2×2 arms with artifact = LoRA (uses the merged minimal_loop_forgetting.py machinery).
5. Wall-clock and peak-RSS of the distillation step.

Reporting

Per-seed tables (holdout, specificity, transfer), teacher-gate record with prompting mode used, hyperparameters + template list frozen in the log, deletion verification, model id, exact commands; log to experiments_logs/<date>_s18_consolidation_distillation.md quoting this spec. The dev/holdout firewall (tuning on dev only) must be explicitly attested in the log.