Experiments Logs Ledger — Authoritative Index

Date: 2026-07-19 (updated through campaign 0d48130 adjudication, Exp03 ad77e93 real-driver closure, R18 5-seed diagnostic clarification, R18 5-seed diagnostic adjudication, R18 mechanism diagnostics adjudication, R19 DEV calibration adjudication, R20 DEV implementation/smoke adjudication, and R20 INT8 transport recovery/bounded calibration fanout) Purpose: This ledger classifies every experiment log against three invalidation events:

1. Scope-slot reranker bugs (fixed 2026-06-30): Three compounding bugs silently prevented the reranker from functioning correctly between its introduction on 2026-06-29 and the fix on 2026-06-30. See 2026-06-30_scope_slot_reranker_fix.md.
2. Test-set leakage removal (2026-07-01): Two leakage paths (_SCOPE_TEACHING per-episode-ID entries, prefix_targets=[probe.expected] for scope probes) were removed, superseding all prior Stage-2/Stage-5 scope claims. See 2026-07-01_honest_post_leakage_baseline.md.
3. Gameable-metrics retirement (2026-07-01, Sprint 0.4): lane_07's "0-by-construction" lexical baseline was replaced with a competitive token-overlap baseline (headline gap collapsed 1.0 → 0.0); lane_05's coverage-as-score was split from the honest result (= 0.0); lane_01's sub-metric set was frozen. See the retirement addenda in 2026-06-28_lane_07_world_model_critic.md and 2026-06-28_lane_05_metabolism_status.md, and the audit in 2026-07-01_remediation_audit.md.

Legacy v2 reference point: 2026-07-01_honest_post_leakage_baseline.md. Eval v2.2 repairs the runner protocol and split policy, so a new real-driver multi-seed baseline is pending. The legacy numbers must not be presented as a v2.2 difficulty curve. The 13.5x drift claim (044cb51) remains retracted (2026-07-01_s2_4_breakthrough_ablation.md).

Classification Key

Ledger

Summary

All files with INVALIDATED or SUPERSEDED content are classified PARTIAL because they also contain VALID content (either pre-reranker measurements, post-fix updates, mechanism analysis, or non-curriculum architecture/implementation documentation).

Reference Point

The honest post-leakage v2 baseline (2026-07-01_honest_post_leakage_baseline.md) is retained for historical comparison only. It is not the current v2.2 reference. Key legacy numbers:

These supersede earlier leakage-era Stage-2 and Stage-5 claims, but a new v2.2 baseline is required before current stage-to-stage comparisons are made.

Retracted headline claims (session f645e4af)

Three of the four headline claims the repo carried into July were re-adjudicated under pre-registered, leak-free conditions and fell:

What survives: the KV-splice mechanism (≡ text prefix at zero token cost, see S1.3), the scope-slot reranker's legitimate 0.92, and a frozen eval that can no longer be quietly bent. See SPRINT.md for the remediation plan and current sprint status.

Campaign 0d48130 Adjudication (2026-07-11)

Full curated evidence log: 2026-07-11_campaign_0d48130.md — per-run metrics, seed distributions, non-runnable inventory, artifact provenance paths, infrastructure fixes, and next steps. The summary below is a quick reference; the curated log is the durable record.

Five execution summaries adjudicated from three source commits: 0d48130 (Exp06 batch, kaggle), 537260c (colab-importfix + R18 gate + R18 full), and 2a22049 (R14 M2B fixed re-run, kaggle). All completed jobs ran under CPU-only contract (cuda_available=false, torch 2.10.0+cpu).

Scientific outcomes (complete):

Non-scientific outcomes:

Seed distributions: Exp06 — 5 seeds (0–4), m1_ratio zero variance, bytes_per_delta spread ≤20 B across all agents. R18 full — 3 seeds: distill_delta_holdout bimodal {0.3333, 0.3333, 0.0}; teacher_dev_delta and persistent_bytes identical across seeds. Colab experiments (01/02/04/05/07) are single-run with no cross-seed variance data.

R18 5-seed diagnostic adjudication: The 5-seed stage_0 rerun completed (exit 0, Kaggle CPU, kernel abdellahkadem/oczy-r18-5seed-5b5e93c63d76). teacher_dev_delta=0.17647058823529413 is identical across all 5 seeds and remains below the ≥ 0.2 validity gate. Scientific classification: BLOCKED at teacher validity gate / diagnostic only. 4/5 seeds show positive distill_delta_holdout=0.3333; seed 2 is null (0.0). Mean distill_delta_holdout=0.2667, mean specificity_delta=0.0261. No H-DISTILL verdict is permitted because the teacher gate failed after registered fallback. No threshold changes. Durable record: 2026-07-11_r18_five_seed_diagnostic.json.

Source: 2026-07-11_campaign_0d48130.md (adjudicated from /tmp/oczy-campaign-0d48130/ execution summaries), 2026-07-11_exp03_real_driver_closure.json (ad77e93 real-driver closure, from /tmp/oczy-exp03-real-run-v2/), and 2026-07-11_r18_five_seed_diagnostic.json (5-seed diagnostic adjudication, from /tmp/oczy-live-queue/ live state). No threshold changes or causal claims beyond measured metrics.

R19 DEV Calibration Adjudication (2026-07-12)

Full curated evidence log: 2026-07-11_campaign_0d48130.md § R19 DEV calibration. Durable execution/adjudication JSON: 2026-07-12_r19_dev_calibration.json.

Research/19 calibrate-dev phase ran from source commit bd1ead9a8358b675af5e929c53a01eb505839639 on Kaggle CPU. Infrastructure: COMPLETE (exit 0, all metrics collected, manifest hash verified). Scientific verdict: BLOCKED at the pre-registered DEV articulation gate.

Attempt history

Attempts v1–v3 were infrastructure failures with no valid scientific evidence. The v4 run was infrastructure-successful but scientifically BLOCKED.

v4 calibration metrics

Gate analysis

The oracle ceiling (0.357143 > 0) passes: the frozen LM can express the taught behavior with a direct text prefix. The blocker is the DEV articulation gate: the learned coupler (Arm B latent control) does not produce a measurable improvement over the no-update baseline (C1 random cortex) on DEV. No H-LATENT or H-LABEL verdict is permitted. No signoff was requested; no holdout was accessed.

C7 adapter discrepancy

The manifest carries c7_available=true (hardcoded in calibrate-dev), but _try_s3m2a_retrieval_adapter() returns None — no real S3.M2a adapter exists. The evaluate phase would block on C7 independently of the articulation gate. This must be resolved before any new claim run.

R19 vs R20 signoff separation

R19 signed evaluation is BLOCKED at the DEV articulation gate. No signoff was requested and no holdout was accessed. R20 (meta-trained cortex) remains separately blocked for lack of explicit human signoff. R19 signoff and R20 signoff are distinct: neither has been requested or granted. The R19 articulation gate failure does not change R20's blocked status.

Direction reassessment

Do not spend signed-eval or R20 budget. Before any new claim run, diagnose at DEV level: (1) why the learned coupler does not improve over the no-update baseline — coupler learning signal, latent interface, or articulation path; (2) resolve the C7 adapter discrepancy. No threshold, metric, baseline, episode, scoring, eval manifest, or research spec was changed.

Source: 2026-07-11_campaign_0d48130.md § R19 DEV calibration and 2026-07-12_r19_dev_calibration.json. No threshold changes or causal claims beyond measured metrics.

R20 DEV Implementation/Smoke Adjudication (2026-07-12)

Full curated evidence log: 2026-07-11_campaign_0d48130.md § R20 DEV implementation/smoke. Durable execution/adjudication JSON: 2026-07-12_r20_dev_smoke.json.

Research/20 (research/20-meta-trained-cortex-frozen-language-organ.md) DEV-only implementation/smoke ran from source commit e26d8291879d078b701f19802f72041e08cfd6a6 on Kaggle CPU (Qwen/Qwen2.5-0.5B-Instruct, frozen). Infrastructure: COMPLETE (exit 0, audit_status ok, all invariants verified). Scientific verdict: none — meta-test remains BLOCKED. This is infrastructure/mechanism smoke only.

Attempt history

Attempts v1 and v2 were infrastructure failures with no valid evidence collected. They are not scientific nulls or refutations. The v3 run was infrastructure-successful; the meta-test remains BLOCKED.

v3 smoke results

Audit invariants

Causal DEV deltas (observed mechanism smoke, not scientific results)

These DEV-level causal intervention deltas are from the validate-dev phase. They are recorded as observed mechanism smoke confirming that the causal intervention pipeline runs and produces output. They are not scientific results and cannot be used for an ACCEPT or REFUTE verdict.

Test suite results (engineering quality checks, not scientific evidence)

Meta-test block status

The R20 meta-test remains BLOCKED. The pre-registered protocol (§ Instrument freeze and threshold distribution check) requires all of the following before any meta-test run:

1. a frozen meta_cortex/v1 instrument (generators, seeds, family split, scorers, probe counts);
2. distribution checks (no-update and repeated-run distributions on meta-validation);
3. a power analysis freezing sample size from meta-validation effect sizes;
4. a manifest with SHA-256 hashes; and
5. human sign-off on the manifest, margin, and sample size.

None of these exist. The DEV-only smoke (train-dev, validate-dev, audit-dev) does not constitute a meta-test run and cannot produce a scientific verdict. No holdout or meta-test data was accessed. No signoff was requested or granted.

R19 vs R20 signoff separation

R19 signed evaluation is BLOCKED at the DEV articulation gate. R20 (meta-trained cortex) remains separately blocked for lack of a frozen instrument, manifest, and human signoff. R19 signoff and R20 signoff are distinct: neither has been requested or granted. The R20 DEV smoke does not change R20's blocked status.

Explicit non-claim

No ACCEPT or REFUTE verdict is claimed for H-META-CORTEX. The meta-test remains BLOCKED. The DEV smoke is infrastructure/mechanism verification only. The best DEV validation score (0.0), causal DEV deltas, frozen organ hash, trace count, and test suite results are recorded as observed infrastructure/mechanism smoke, not as scientific results. No threshold, metric, baseline, episode, scoring, eval manifest, or research spec was changed.

Source: 2026-07-11_campaign_0d48130.md § R20 DEV implementation/smoke and 2026-07-12_r20_dev_smoke.json. No threshold changes or causal claims beyond measured metrics.

Notes (conceptual, non-log)

Analysis documents live in notes/ (created 2026-07-03) — they interpret logged evidence but are not themselves experiment logs:

• notes/2026-07-03_steering_vs_posture_postmortem.md — why the steering/posture intuition failed (three broken assumptions: common-mode accumulation has magnitude not direction; constant vectors cannot condition; mention-space ≠ use-space), synthesizing S1.3, S1.4, S2.1, S2.4. Successor mechanism pre-registered in research/18-consolidation-as-distillation.md.