17 — Second-model generalization (Sprint 4 / S4.3)

Pre-registered 2026-07-02 (human-approved sprint setup). Agents MUST NOT edit this spec. Depends on research/11 (and research/12 if H-KVCONTENT was accepted — the content channel under test is whichever the Sprint-2 verdicts selected).

Problem

Every architecture result so far comes from one substrate at a time (LFM2.5 on the legacy path, Qwen2.5-0.5B on HF). Single-model results say nothing about the architecture generalizing versus exploiting one model's quirks — S1.4 already showed the two models disagree about layer geometry.

Hypothesis

H-GEN: the minimal metabolism loop's held-out effect (research/11's loop_delta_holdout > 0, compounding ρ ≥ 0.6) reproduces on a second, architecturally distinct small model with NO per-model tuning beyond the substrate adapter.

Second model (selection rule fixed now)

The already-benchmarked fallback Qwen/Qwen2.5-1.5B-Instruct is EXCLUDED as the second model (same family, same tokenizer — weak generalization evidence). Choose the fastest cached-or-downloadable model satisfying ALL of: plain decoder-only transformer with per-layer (k,v) cache (verified by check_kv_cache.py), different model family from Qwen, instruct-tuned, ≤ 2B params, ≤ 300 ms/tok CPU float32 on this host (bench_hf_cpu.py). Candidates to try in order: TinyLlama-1.1B-Chat (known EOS fragility — re-test with the chat template; if unusable, record why), HuggingFaceTB/SmolLM2-1.7B-Instruct, google/gemma-2-2b-it. The choice and benchmark numbers are recorded before any loop run.

Protocol

Exactly research/11's protocol (stage 0, split_probes(fraction=0.3, salt="v2"), K ∈ {0,1,2,4,N}, ≥5 seeds — fallback 3, reported), with the Sprint-2-selected content channel, on the second model. NO hyperparameter changes from the Qwen run: same prefix budget, same alpha, same gate. If the second model needs a chat template for coherent output, apply it identically in the vanilla column.

Primary metrics & acceptance

• gen_delta_holdout (= research/11's loop_delta_holdout on model 2)

• gen_compounding_rho (= loop_compounding_rho on model 2)

• Accept H-GEN: gen_delta_holdout > 0 with 95% CI excluding 0 AND gen_compounding_rho ≥ 0.6.

• Refute: either fails — recorded as "architecture does not yet generalize"; per-model tuning that rescues it belongs in a NEW pre-registered spec, not this one.

• Validity gates: model-2 vanilla holdout < 0.5 (else INVALID); research/11 must have ACCEPTED H-LOOP on Qwen (else this experiment is BLOCKED — there is nothing to generalize).

Pre-registered secondaries (exploratory only)

1. Effect-size comparison model 1 vs model 2 (ratio of deltas).
2. Drift triple on model 2.
3. ms/tok and memory cost table for the adapter on model 2.

Reporting

Model-2 selection record (candidates tried, benchmark numbers, exclusions), full research/11-style tables for model 2, side-by-side with the Qwen numbers; log to experiments_logs/<date>_s4_3_second_model.md quoting this spec.