# Experiment Log: Cortex-as-Centre Un-Inversion **Date:** 2026-06-23 **Experiment:** Reframe the architecture so the cortex is the live mutation surface and the LM is a perception/articulation organ on its edges. Replace the toy label-string cortex with a tensorial `KVCortex` whose warm/cold state is steerable into the LM's attention KV cache. **Evaluator:** NCode --- ## Hypothesis / Goal The thesis (`experiments.txt` section 12) calls for an agent where memory becomes **changed dynamics**: activation → fast state → fast weights → neural memory → slow weights → forgetting raw trace. Today's `OrganismAgent` and `LMBackendAgent` are both inverted relative to this: - `OrganismAgent`: cortex writes label strings ("user profile"); the LM is absent. The slow/fast weight distinction is decorative because `PlasticCortex.BASELINE` never mutates. - `LMBackendAgent`: a frozen LLM drives the answer path while organ metabolism runs in parallel as bookkeeping. Experience never reaches the LM's forward dynamics. This experiment sets up the cortex contract that un-inverts the architecture: the cortex is a small-dim (`d_cortex = 128`) two-speed neuromodulator that reads the LM's hidden state on the perception side and projects its intent into the LM's attention KV cache on the articulation side. The LM stays frozen; every mutation lives in the cortex. ## Method 1. **Author** `KVCortex` (`plastic-cortex/src/plastic_cortex/kv_cortex.py`) with the contract: - `warm_state: ndarray[d_cortex]` — session-mutable, dies on restart. - `cold_state: ndarray[d_cortex]` — boot-loaded, written only by `consolidate()`. - `observe(lm_hidden, correction_signal)` — warm update, sub-ms. - `project_intent(layer_idx)` — per-layer `(k, v)` cached delta for KV-slot injection. - `train_step(lm_hidden)` — Hebbian update on `proj_hidden`. - `consolidate(replays=...)` — slow EMA nudge + replay absorption into cold state. - `reset_warm_from_cold()` — cold boot semantics. 2. **Author** smoke tests covering shape contract, warm/cold separation, consolidation behaviour, neuromodulator effect, projector stability, pickle round-trip, status contract. 3. **Validate** the contract on its own, before binding to the LM. The LM binding is tracked as Goal 1 in `GOALS.md` and is next-session work. ## Results ### Cortex contract ``` File: plastic-cortex/src/plastic_cortex/kv_cortex.py d_cortex = 128 (default) n_layers = 28 (matches LFM2.5-1.2B) d_head = 256 per-turn cost = sub-2 ms (one d_embd matmul + cached per-layer projections) cold persist = pickle of (proj_hidden, proj_k, proj_v, cold_state) ``` ### Smoke tests ``` uv run python plastic-cortex/tests/test_kv_cortex.py ok: shapes ok: warm_mutates_cold_does_not ok: consolidate_moves_warm_into_cold ok: consolidate_replay_absorption ok: correction_signal_raises_plasticity ok: reset_warm_from_cold ok: hebbian_training_changes_projector ok: pickle_round_trip ok: status_contract 9/9 passed ``` The tests verify: 1. `observe()` mutates only warm_state, never cold_state. 2. `consolidate()` mutates cold_state (always via slow EMA nudge; replay absorption is gated by rehearse count threshold). 3. `correction_signal=1.0` produces larger warm drift than `0.0` — the neuromodulator is functional, not just a parameter. 4. Hebbian `train_step()` mutates `proj_hidden` while per-row L2 renormalisation keeps norms in `[0.9, 1.1]` after 50 updates. 5. Pickle round-trips all state; RNG state is intentionally not serialised so loaded cortex is deterministic given its seed. 6. `status()` reports the canonical cross-organ fields (`serialized_bytes`, `record_count`, plus cortex-specific norms). ### What the contract does NOT yet do The cortex has no LM binding. It can absorb synthetic hidden vectors and emit `(k, v)` tuples, but no driver writes those into the LM's KV cache yet. The loop closes mathematically but not behaviourally. This is the work tracked in `GOALS.md` Goal 1. ## Conclusion / Next steps The cortex contract is shape-correct, two-speed (warm/cold), and neuromodulated (correction_signal raises plasticity). It is the first organ in the repo whose mathematical shape genuinely matches the thesis: small learnable state, fast in-session mutation, slow boot-persistent identity, no text in its input or output. **Next session:** - Goal 1 — LM-side KV-write binding via `llama-cpp-python`'s `kv_self` surface. - Once the binding is understood, write `CortexAgent` as the un-inverted organism: cortex is the mutation surface; organs consume tensor signals from cortex state; LM is perception + articulation only. - Goal 2 (hidden-state extraction at layer L) can begin in parallel once Goal 1's binding surface is mapped. ## Artifacts - `plastic-cortex/src/plastic_cortex/kv_cortex.py` — cortex contract - `plastic-cortex/tests/test_kv_cortex.py` — 9 contract tests - `GOALS.md` — architecture goals 1/2/3 with done-when criteria - This log file --- ## Update — Goal 1 binding verified end-to-end After authoring the cortex contract, Goal 1 (LM-side KV-write binding via `llama-cpp-python`) was probed and prototyped. Findings: ### Surface probe `llama-cpp-python` 0.3.31 exposes the cvec surface via `llama_set_adapter_cvec(ctx_p, data_ptr, len, n_embd, il_start, il_end)`. - `ctx_p` is reachable as `Llama._ctx.ctx` (raw int pointer). - `data` is a flat `c_float *` pointer; for per-layer cvecs, length must be `n_embd * (il_end - il_start)` and the API distributes one slice per layer. - Clearing is `llama_set_adapter_cvec(ctx_p, None, 0, n_embd, 0, n_layers)`. - The cvec persists across `create_completion` calls (apply once, holds until cleared). - The cvec REPLACES on each call — looped per-layer `set_cvec_layer(L)` calls keep only the last layer's steering. - The LFM2.5-1.2B-Instruct GGUF reports `n_layers=16` via `llama_n_layer()` (the API counts attention-capable layers; RWKV/conv hybrid layers are not addressed). ### Driver shim `oczy_lm/cvec_driver.py` formalises the surface as `LlamaCVecDriver`: - `set_cvec_layer(L, vec)` — single-layer apply (replaces prior). Useful for targeted steering; not what cortex metabolism needs. - `set_cvecs_per_layer(vectors, scale=1.0)` — batched single-call apply. Takes `n_layers` vectors of dim `n_embd`, concatenates, and invokes the adapter once with `il_start=0, il_end=n_layers`. - `set_cvec_uniform(vec)` — same vector across every layer. - `clear_cvec()` — NULL data over full range. - `generate(prompt)` — wrapper around `Llama.create_completion`. - `peek_embedding(prompt, last_token_only=True)` — final-layer prompt embedding via `Llama.create_embedding`. This is Goal 2 staging; layer-L intermediate extraction is not yet supported by the binding. ### Cortex contract revision `KVCortex` was updated to match the real surface: - `project_intent(layer_idx) -> (k, v)` (KV-slot tuples, dim `d_head`) was speculative. Replaced with `emit_cvec(layer_idx) -> ndarray[n_embd]` and `emit_all_cvecs() -> list[ndarray]`. - `proj_k`/`proj_v` (shape `(n_layers, d_head, d_cortex)`) merged into a single `proj_c` (shape `(n_layers, d_embd, d_cortex)`). - `KVCortexConfig.d_head` removed; `d_embd` is now both the input and per-layer output dim. - Cortex tests rewritten (`plastic-cortex/tests/test_kv_cortex.py`): 10/10 pass on the new contract including a cache-stability test that verifies `emit_cvec` returns the same memory until `observe()` is called. ### End-to-end test `oczy_lm/tests/test_cvec_driver.py` proves the binding: 1. Baseline (no cvec): `"Hello, my name is"` → `" Alex, and I"`. 2. Cortex absorbs a synthetic hidden; emits per-layer cvecs at scale 30. 3. `driver.set_cvecs_per_layer(cortex.emit_all_cvecs(), scale=30)` shifts the LM output to something other than the baseline. 4. `driver.clear_cvec()` restores the baseline output exactly. ``` 6/6 passed: ok: test_driver_reports_expected_shape ok: test_set_cvec_layer_shape_match ok: test_set_cvec_layer_rejects_wrong_dim ok: test_set_cvec_layer_rejects_bad_layer_idx ok: test_cvec_from_cortex_shifts_generation ok: test_peek_embedding_returns_n_embd_vector ``` ### Regression All pre-existing suites still pass: - `plastic-cortex/tests/test_kv_cortex.py`: 10/10 - `experiments/organism_curriculum/validation.py`: 6 stages / 44 episodes, 0 errors - `experiments/run_experiment.py --agent OrganismAgent`: same metrics as 2026-06-22 (uptake 0.67, transfer 0.25, scope 0.17, forget 1.00). ### What is NOT done - Goal 2 (layer-L hidden extraction): `peek_embedding` returns the final-layer output, which is the model's encoder-style summary. Layer-L intermediate residuals (the depth where semantic intent actually forms) are not exposed by the binding. Cortex metabolism currently works on the final-layer representation; that may be enough but is unproven. - Goal 3 (organ upgrades to tensor inputs): the cortex binding is wired to the LM but the five metabolism organs still consume strings and dicts. The CortexAgent driver glue that string everything together hasn't been written yet. - The cvec amplitude needed to dislodge greedy decoding is ~30x the cortex's default-init output. Once cortex weights train via Hebbian update on real hidden vectors, this scaling should become data-driven. ### Status Goal 1: **complete**. Cortex→LM articulation coupling is real and testable. Goal 2: deferred (final-layer embedding usable in staging). Goal 3: next. --- ## Update — CortexAgent end-to-end (Goal 3 partial) Goal 3's CortexAgent driver glue is built. The cortex is now the live mutation surface of an un-inverted organism: it observes LM hiddens, mutates warm_state, and its per-layer cvecs are applied to the frozen LM via the `cvec` adapter on articulate(). The existing organ metabolism is wired on the side, consuming cortex state vectors as surprise signals where they used to consume string features. ### New file: `experiments/cortex_agent.py` The CortexAgent class wires: - `LlamaCVecDriver` (frozen LM + cvec adapter surface) - `KVCortex` (live mutation surface) - The five metabolism organs (existing implementations, routed off cortex state) Lifecycle: ``` boot() cold_state loaded -> warm_state := cold_state.copy() perceive(utterance) LM final-layer embedding -> cortex.observe(...) metabolize(utterance) cortex.drift -> critic / hippocampus / immune / autoencoder (autoencoder.train_step on perceptron projector via the existing episode-shaped API) articulate(prompt) cortex.emit_all_cvecs() -> driver.set_cvecs_per_layer (scale=30.0 default) -> driver.generate() (clear_cvec on success) consolidate() hippocampus.consolidate + cortex.consolidate(replays) (replays synthesised by re-embedding consolidated queries via the LM -- placeholder until the hippocampus natively stores tensor replays) save(path)/load(path) Pickle: cortex cold_state + proj_hidden + proj_c + each organ. Warm state intentionally NOT persisted (session-only by design). turn(utterance) Convenience: perceive + metabolize + articulate in one call. ``` Correction-signal detection is lexical today (`_looks_like_correction`): the cortex's neuromodulator fires `alpha_correction` plasticity when the user's utterance contains markers like "no,", "wrong,", "actually,", etc. Goal 3 follow-up replaces this with a learned detector based on the WorldModelCritic's drift-vs-string-feature disagreement. ### Test: `experiments/tests/test_cortex_agent.py` Six smoke tests, all passing against the real LFM2.5-1.2B-Instruct Q4_K_M: ``` ok: test_perceive_produces_warm_state ok: test_correction_signal_drives_plasticity ok: test_metabolize_routes_to_hippocampus_on_drift ok: test_articulate_steered_differs_from_baseline ok: test_consolidate_moves_cold_state ok: test_save_load_round_trip_preserves_cold 6/6 passed ``` Verifies: 1. `perceive` produces a d_cortex warm vector; caches the (n_embd,) hidden vector for the next metabolize call. 2. A correction-marker utterance mutates the cortex more than the baseline drift threshold (0.05). 3. `metabolize` walks the cortex-drift through the hippocampus's surprise gate, so episodes land in the replay bank. 4. `articulate` with cortex steering produces different LM output than the same prompt without steering. 5. `consolidate` mutates `cold_state` -- the boot-persistent identity survives a real learning sequence. 6. `save` -> `load` round trip preserves `cold_state` to within rtol/atol 1e-6; loaded cortex's warm equals cold (boot semantics). ### Regression All pre-existing suites pass unchanged: - `plastic-cortex/tests/test_kv_cortex.py` 10/10 - `experiments/organism_curriculum/validation.py` 6 stages / 44 episodes OK - `experiments/run_experiment.py --agent OrganismAgent` baseline unchanged (this suite does not exercise CortexAgent -- it still uses OrganismAgent against the toy PlasticCortex) ### What is NOT done - **Goal 2 (layer-L hidden extraction)** remains staging-only: the cortex consumes `peek_embedding`'s final-layer mean-pooled embedding, not an intermediate residual at semantic depth. Whether this representation is rich enough for real attractor-basin carving is unknown -- the cvec steering test confirms behaviour SHIFT, not whether the SHIFT maps to the user's intent. - **Organ tensor-input upgrade**: CortexAgent feeds the existing organs with cortex.drift as the surprise scalar and synthesises replay tensors from re-embedded hippocampus queries. The hippocampus itself still stores string-keyed episodes; the critic still has its feature vector overwritten by the cortex.drift hack rather than rebuilt to consume cortex state natively. - **Correction-signal detection** is lexical; the WorldModelCritic's drift-vs-text disagreement should drive it instead. - **CortexAgent curriculum bridge**: the existing 6-stage organism curriculum still exercises `OrganismAgent`, not `CortexAgent`. A CortexAgent-compatible curriculum run is a follow-up -- once the cortex stabilises on meaningful absorption (synthetic hiddens do that only weakly), the curriculum can be used to MEASURE whether the corrected sense lingers in cold_state across consolidation cycles. ### Status Goal 3 CortexAgent glue: **complete** (articulate loop closes; identity survives save/load; organs routed from cortex state). Goal 3 organ tensor-input upgrades: **partial** (drift-as-scalar wired; deep organ rewrite deferred until CortexAgent behaviour validates that further organ investment is justified). ### Recommended next move Run CortexAgent through a small, scripted conversation:''' sed -i 's/^### Recommended next move.*$//' /home/nyanpasu/Desktop/code/kinoSoft/oczy/experiments_logs/2026-06-23_cortex_kv_contract.md cat >> /home/nyanpasu/Desktop/code/kinoSoft/oczy/experiments_logs/2026-06-23_cortex_kv_contract.md << 'EOF' ### Recommended next move Author a `cortex_conversation_demo.py` script that runs CortexAgent through a 4-turn scripted conversation (query -> correction -> query -> probe). Verify: 1. The cortex's `cold_state` visibly moves after `consolidate()`. 2. The LM's articulated output differs between baseline and post-correction on the same prompt. 3. The cold_state persists across save/load and still biases generation. That demo is the qualitative behaviour check that tells us whether the cortex is actually absorbing intent or just producing noise-shape variation in the LM's cvec inputs. If the latter, Goal 2 (intermediate- layer hidden) is the next priority; if the former, the organ upgrades become the bottleneck to closing the full metabolism loop. --- ## Update — CortexAgent qualitative demo (cvec steering diagnosis) `experiments/cortex_conversation_demo.py` was authored and run. The demo proves the cortex state persists across consolidate + save/load, but at the SAME scale that demonstrably shifts greedy decoding (30), the post-correction articulate() output is CJK-token garbage (`'选选选选选...'`). Steering at all tested scales: ``` baseline : " \nOptions: \nA) A summary of the product's features \nB" scale 0.5 : " \nOptions: \nA) A summary of the product's features \nB" scale 1.0 : " \nOptions: \nA) A summary of the product's features \nB" scale 2.0 : " \nOptions: \nA) A summary of the product's features \nB" scale 4.0 : " \nOptions: \nA) A summary of the product's features \nB" scale 8.0 : " \nOptions: \nA) A summary of the product's features \nB" scale 16.0 : " \nOptions: \nA) A summary of the product's features \nB" scale 30.0 : '选选选选选选选选选选选选选选选选选选选选' <-- crash ``` Cortex state after perceive: warm_norm=6.66, projected cvec norm=37.6. Without scaling that's already large (residual stream at LFM2 is ~5-20 in norm), yet greedy decoding absorbs the perturbation silently. ### Diagnosis: cortex is producing noise-shape steering, not intent-shape What the demo proves: - Cvec persistence works (cold_state survives save/load). - Cvec surface works at scale 30 (output CHANGES from baseline). What the demo does NOT prove: - The cortex is steering toward the correction's meaning. The post- correction output is Chinese token repetition — i.e. the random default proj_c at large amplitude pushes the residual off-manifold into degenerate decoding rather than toward "business vertical". ### Why this is the right diagnostic The cortex.absorbed_hidden -> warm_state is well-bounded via tanh and EMA. But proj_c is a 64 -> 2048 fixed-random projection -- its rows are random vectors in d_embd space. Applying a random direction cvec at scale 30 to a residual stream is equivalent to adding a high-norm random vector, which of course breaks decoding (the residual is no longer on-manifold). The honest fix: **proj_c must be trained, not random**. Or the cortex must steer through a SUBSPACE capable of carrying semantic intent (e.g., a low-rank subspace matching the correction's hidden direction). Until then, scaling either does nothing (greedy decoding absorbs the perturbation silently below the threshold) or breaks things (above the threshold). ### What this changes about Goal 3 Reachable claims (already true): - Cortex is the live mutation surface. - Adler signal persists across consolidate + save/load. - The cvec API is mechanically end-to-end working. Unreachable claims (deferred): - Cortex steering produces semantically meaningful output. - CortexAgent replaces LMBackendAgent for production use. The cortex has to LEARN what direction means what. Two paths: (a) **Hebbian train proj_c on actual correction contexts.** Stop initialising proj_c with random direction; initialise it from a small batch of correction hiddens via SVD. Each correction's hidden vector then implicitly becomes one direction in the projector's range, and a correction's warm_state naturally retries via that direction. Match the experience-autoencoder's existing pattern. (b) **Per-correction cvec root mean, not projection.** Forget proj_c altogether for steering; set the cortex's output to a temporary mean of recent correction hiddens (scaled small). This drops per-layer expressivity but it gives semantically aligned cvecs for free. Try (b) first because it's a 5-line change. Either path is conscious architecture work; we're past the "wire the LM binding" stage and into "make the cortex meaningful." ### Recommended next move Implement (b) as a `Mode=raw_hidden` mode on KVCortex: when set, the cortex's `emit_cvec(layer_idx)` returns `warm_state * hidden_from_last_correction` padded to n_embd -- a smoothed semantic anchor rather than a random projection. Compare to baseline articulate to see if the LM steers toward the corrected sense rather than crashing. If (b) still produces CJK garbage at meaningful amplitudes, switch to (a): SVD the proj_c over a small batch of seed correction hiddens.