# Real-Driver Needle Sweep at Length 4096 — 2026-06-27

## Question

At length 4096, does the foreign-MiniLM embedder become cheaper than same-LM
because many chunks must be embedded and same-LM embedding cache hits drop?

## Method

Reused the real-driver needle_sweep.py with `--length 4096`, `--n-ctx 8192`,
256-token chunks, and `pass-through` salience so every chunk is embedded (17
chunks per position). Ran 2 positions (0.0 and 0.5) for each embedder to keep
wall time bounded.

## Results

| embedder | recall@0.0 | recall@0.5 | mean_recall | traces | total wall seconds |
|---|---|---|---|---|---|
| same-lm | 1 | 0 | 0.50 | 17 | 20.1 |
| foreign-minilm | 1 | 0 | 0.50 | 17 | 30.2 |

## Interpretation

- Same-lm is still faster than foreign-minilm even when 17 chunks are embedded.
  The gap is smaller than at length 512 (20.1s vs 30.2s for 2 positions, vs
  29.6s vs 42.8s at length 512 for 5 positions), but same-lm remains ahead.
- The 0.5-position needle was missed because with 256-token chunks the needle
  tokens do not always land in a chunk that survives as a distinct trace
  (pass-through keeps all chunks, but hippocampal replay matches on query text,
  and the exact needle text may not be prominent in the retrieved chunk).
  This is a chunk-boundary artifact, not an architecture finding.
- `LlamaCVecDriver.peek_embedding` appears to be efficiently cached and
  llama.cpp's embedding-only forward path is fast enough that the CPU torch
  overhead of MiniLM does not win.

## Implication for architecture

The embedder fork, under the tested local CPU environment and tested sentence
model, favors same-lm. Foreign-MiniLM is not a universal cost win. Before
pursuing foreign embeddings further, the better leverage would be:
1. Use a much faster foreign backend (ONNX Runtime / Core ML / quantized),
   or GPU-offloaded torch.
2. Increase context to 32k+ where same-lm embedding batching/caching breaks down.
3. Measure embedding-call counts directly on the real driver to confirm cache
   behavior.

Given that same-lm is fast and accurate enough, and foreign-minilm adds a heavy
optional dependency without a clear win, the priority should return to the
original north star: behavior_delta_per_byte_of_persistent_memory.

## Open questions

1. Would an ONNX foreign embedder change the cost ratio?
2. Is there a length/context where same-lm consistently loses?
3. Does disabling same-lm embedding cache (if possible) expose the true cost?

## Artifacts

- No code changes in this iteration; reused `needle_sweep.py`.
- Data recorded in autoresearch run #90.

## Commits

- `205ae57` — Update SUMMARY.md with run #90 findings.

## Run

Run #90: benchmark `code_qa_accuracy=1.0`, fast suite `300 passed` (no code
changes, so same suite as run #89).