# Experiment Log: Organism Curriculum + LM Perception Layer **Date:** 2026-06-22 **Experiment:** Run the new 6-stage organism curriculum end-to-end through the multi-organ agent, baseline the existing eval suite against all six agents, and exercise the LM perception layer (LFM2.5-1.2B-Instruct Q4_K_M) on the original 12-lesson benchmark. **Evaluator:** NCode **Commands:** - `uv run python experiments/organism_curriculum/run_curriculum.py --report-name raw_full.json` - `uv run python experiments/run_experiment.py` - `uv run python experiments/lm_perception/run_perception_demo.py --lessons 12` --- ## Hypothesis / Goal Two questions: 1. Does the new organism curriculum (`experiments/organism_curriculum/`) actually exercise the organ metabolism (fast change -> replay -> compression -> slow change -> forgetting raw trace), and where does it surface real organ limitations versus curriculum flaws? 2. Does the restored LM perception layer (LFM2.5-1.2B-Instruct Q4_K_M via `oczy_lm/adapter.py`) successfully mediate the original 12-lesson correction benchmark end-to-end, and what is its parse fidelity on this host? The prior session had crippled the perception layer by stripping the few-shot examples from the parse prompt (parse rate dropped from 4/12 to 1/12). This experiment re-runs after restoring the examples and adding the LM stack (`llama-cpp-python`, `huggingface-hub`, `numba`, `numpy`) to `pyproject.toml` dependencies. ## Method 1. **Environment.** `uv sync` now installs the full LM perception stack into the repo's `.venv`. All commands run via `uv run python ...`. 2. **Raw organism curriculum.** Run all 6 stages (44 episodes total) against `OrganismAgent` in raw (`agent.learn(request, correction)`) mode. Report per-stage uptake latency, pre/post probe accuracy, and memory delta. 3. **Eval suite baseline.** Run `experiments/run_experiment.py` across `ZeroMemoryAgent`, `ContextOnlyAgent`, `FastOnlyAgent`, `HippocampusOnlyAgent`, `IdentityOnlyAgent`, and `OrganismAgent` on the existing 12-episode correction benchmark. 4. **LM perception demo.** Run `experiments/lm_perception/run_perception_demo.py --lessons 12` against `LiquidAI/LFM2.5-1.2B-Instruct-Q4_K_M`. Report raw vs LM absorption, parse rate, LM wallclock premium, and parse-miss list. 5. **No model retraining.** The small plastic-cortex LM and the GGUF are used as-is; only the parse/render prompts and ingestion glue have changed. ## Results ### Raw organism curriculum ``` Stage Episodes Uptake Pre Post Mem d -------------------------------------------------------------------------- Stage 0: Sense grounding 8/8 0.00 0.00 0.88 +15678B Stage 1: Transfer within domain 8/8 0.00 1.00 1.00 +3273B Stage 2: Scope control 0/8 1.00 0.50 0.50 +9431B Stage 3: Dialog 4/4 0.00 0.12 0.25 +2751B Stage 4: Consolidation stress 10/10 0.00 0.80 1.00 +6140B Stage 5: Cross-domain 5/6 0.17 0.42 0.33 +10000B ``` Stages 0, 1, 4 absorb cleanly (0.00 uptake latency, post-test 0.88-1.00). Stage 2 (scope control) fails 100%: the default `PlasticCortex` word-association backend learns a single corrected sense per word; teaching a second sense overwrites the first, so retention probes for the *new* (always-computing) sense pass but the *old* (corrected) sense from Stage 0 is lost. Stage 3 (dialog) shows uptake but low probe scores (0.12 -> 0.25) because the multi-turn follow-ups depend on context the toy backend does not maintain. Stage 5 (cross-domain) is partially tractable (5/6 retention, 0.00 scope) because the same two-senses-per-token limitation still applies. ### Existing eval suite baseline comparison ``` Agent Uptake Transfer Scope Forget Consol Identity Mem/Δ ---------------------------------------------------------------------------------------- ZeroMemoryAgent 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 61.0 ContextOnlyAgent 0.6667 0.0000 0.0000 0.0000 0.0000 0.0000 612.75 FastOnlyAgent 0.6667 0.1667 0.1667 1.0000 1.0000 1.0000 12.0 HippocampusOnlyAgent 0.6667 0.1667 0.0000 0.0000 0.0000 0.0000 1.25 IdentityOnlyAgent 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 493.0 OrganismAgent 0.6667 0.2500 0.1667 1.0000 1.0000 1.0000 68772.0 ``` OrganismAgent matches FastOnlyAgent on forgetting/consolidation/identity (both inherit PlasticCortex's behaviour) and adds a small transfer edge (0.25 vs 0.17) via the hippocampus replay path. Memory cost is dominated by the hippocampus + immune + autoencoder pickled state. ### LM perception demo (12 lessons) ``` Raw absorbed : 12/12 (100%) avg 0.00s/lesson LM parse OK : 5/12 (42%) LM absorbed : 11/12 (92%) avg 5.36s/lesson (LM+organism end-to-end) Wallclock premium per lesson : +5.36s (LM path - raw) LM parse misses (7): [ 1] Deploy the new model. | extracted='' [ 5] Start the run. | extracted='' [ 6] Edit the cell. | extracted='' [ 7] Play the record. | extracted='' [ 8] Add a module. | extracted='' [ 9] Press the key. | extracted='' [10] Restart the service. | extracted='service should be restarted' ``` Parse-miss pattern: lessons 1, 5, 6, 7, 8, 9, 10 fail. The parser succeeds on the first few lessons and the last one but fails consistently in the middle of the run, suggesting either KV-cache drift across calls or a systematic spurious-answer-clearing path triggered when the LM hallucinates a `corrected_answer` on what should be `accepted`. Absorption still reaches 11/12 because the raw fallback picks up the LM's misses; the LM adds latency (~5 s/lesson) but not correctness in its current state. ## Conclusion / Next steps - The organism curriculum works and surfaces a real organ limitation: the toy `PlasticCortex` cannot hold two senses per token, which is why Stage 2 and Stage 5 fail. This is an organ-level fix, not a curriculum-level fix. - The LM perception layer is restored to a usable state (1/12 -> 5/12). The raw fallback keeps overall absorption at 92%, so the system runs end-to-end, but the 42% parse rate is still the bottleneck. The positional miss pattern (lessons 5-10 fail) points at either KV-cache state leaking between calls or the spurious-answer-clearing sanity check firing too aggressively on short ambiguous tokens ("model", "run", "cell", "record", "module", "key", "service"). - Memory costs are stable: OrganismAgent at ~68 KB,~15788 B delta on Stage 0, decreasing on later stages as the hippocampus consolidates. - **Next steps.** 1. Investigate the positional parse-miss pattern: reset the LM KV-cache between calls (`Llama.reset()` or fresh context) and re-measure. 2. Sharpen the spurious-answer-clearing heuristic so it does not clear real corrections on short tokens; consider a length-aware gate. 3. Replace the toy `PlasticCortex` with a multi-sense backend so Stage 2 and Stage 5 become tractable. 4. Re-run the eval suite extended comparison (`eval_extended.py`) to confirm Oracle still ranks above PlasticCortex after the LM perception layer is in the loop. ## Artifacts - `experiments/organism_curriculum/reports/raw_full.json` - `experiments/organism_curriculum/reports/stage_0_lm.json` - `experiments/lm_perception/reports/demo_run.json` - `experiments/logs/` (run logs emitted by `ExperimentLogger`)