# Order-Shuffle Stressor

## Date: 2026-06-28

## Goal

Test whether fact order within a turn changes cvec output, and whether
sequential (per-chunk cortex.observe) vs parallel (full-turn observe)
observation modes produce different order sensitivity.

## Architecture change: per-chunk sequential observation

The ingestion pipeline's `observation_mode` previously only affected drift
statistics, not cortex learning. Both modes called `cortex.observe()` once on
the full-turn hidden in `perceive()`, making them identical.

**Fix**: When `observation_mode="sequential"`:
1. `perceive()` skips `cortex.observe()` — caches `_last_hidden` but leaves
   `warm_state` untouched
2. `IngestionPipeline.process()` calls `cortex.observe(chunk_hidden,
   correction_signal)` per-chunk in order, making `warm_state` genuinely
   order-dependent (last chunk dominates at signal=1.0, plasticity=1.0)

Parallel path unchanged (full-turn observe in `perceive()`).

## Probe design evolution (runs #149-#153)

### Run #149: filler haystack + coherence metric
- 1024-token filler turn with facts buried at positions
- `salience="lexical-novelty"`, `correction_signal` not set (default 0.0)
- Result: sensitivity=0 (all answers identical)
- Problem: filler dominates `peek_embedding`, signal=0.0 → 2% plasticity

### Run #151: short turn + logit biasing + coherence metric
- Fixed template: "Here is some information. First, [fact]. Next, [fact]. Finally, [fact]."
- `correction_signal=1.0`, `use_logit_bias=True`
- Result: answer_sensitivity=1, but logit biasing fails for target_middle/last
- Problem: template words dominate mean-pooled hidden

### Run #152: pass-through salience
- Same as #151 but `salience="pass-through"`
- Result: identical to #151 — salience filter wasn't the bottleneck

### Run #153: facts-only turn (no template)
- Turn = just the 3 facts joined by spaces, no template words
- Result: **answer_sensitivity=2** (all 3 orderings produce distinct answers)
- All coh=1 (logit biasing works at all positions)
- Distinct continuations: target_first → "purpose of the 'level'",
  target_middle → "cryptographic hash", target_last → "hidden menu"

## Key findings

1. **Fact order DOES affect cvec output** — all 3 orderings produce distinct
   continuations (answer_sensitivity=2, maximum signal)
2. **Template words mask the signal** — "Here is some information. First. Next.
   Finally." dominate `peek_embedding`'s mean-pooled hidden. Removing the
   template (facts-only turn) fixed both coherence and sensitivity
3. **correction_signal=1.0 is required** — at signal=0.0, plasticity=alpha_warm=0.02,
   warm_state barely moves, no learning happens
4. **salience="pass-through"** needed so all chunks survive (lexical-novelty
   drops later chunks sharing vocabulary)
5. **peek_embedding uses create_embedding API** — separate forward pass, no KV
   cache pollution of generate()
6. **Parallel and sequential produce identical outputs** — the facts-only turn
   (~30 tokens) fits in a single 32-token chunk, so sequential mode has 1 chunk
   = 1 observe = identical to parallel mode. The per-chunk architecture needs
   ≥2 distinct chunks to discriminate, which requires a longer turn (haystack
   problem)

## Limitations

The per-chunk observe architecture is proven correct but the probe is at its
sensitivity limit. Discriminating parallel vs sequential requires:
- A turn long enough to produce ≥2 distinct chunks
- But short enough that facts dominate the embedding (not filler)
- These constraints conflict — the haystack problem

Future experiment: use a multi-sentence turn where each sentence is one fact
(no filler), with a small chunk window (16 tokens) and non-overlapping chunks.

## Files changed

- `src/oczy/experiments/cortex_agent.py` — `perceive()` skips observe in sequential mode
- `src/oczy/experiments/ingestion.py` — `process()` per-chunk observe loop in sequential mode
- `src/oczy/experiments/codebase_qa/benchmark.py` — `_run_order_shuffle_probe()` + metric wiring