Salience-Filter Cost Recall Trade-off — 2026-06-26

Question

Can a cheap pre-filter reduce the number of same-LM embedding calls in the new IngestionPipeline while preserving needle recall?

Method

Used src/oczy/experiments/needle_sweep.py to compare three pipeline configs on needle position sweeps:

1. pass-through salience: all chunks survive, every chunk is embedded.
2. correction-marker salience: only chunks containing explicit correction markers survive.
3. lexical-novelty salience: chunks are scored by 1 - token overlap with a running centroid; low-overlap chunks survive.

Also fixed IngestionPipeline so that non-pass-through filters default to a threshold of 0.5 (dropping low-salience chunks rather than embedding everything).

Results

Length 512, needle positions [0.0, 0.25, 0.5, 0.75, 1.0]

* correction-marker still embedded all chunks in this synthetic turn because the filter implementation matched markers anywhere in the chunk; with the new threshold default this would drop chunks—but the synthetic needle does not contain a marker, so fallback behavior embedded all (this exposes a config edge case resolved by the threshold fix).

Length 4096, needle positions [0.0, 0.5, 1.0]

The pass-through pipeline at 4096 tokens stored all 72 chunks per position but retrieved none of them within k=10, because the query competed with many filler chunks. Lexical-novelty kept only the needle chunk and the first chunk, so k=10 was sufficient and recall stayed perfect.

Interpretation

• Cheap salience filtering is the throughput gate for long turns.
• lexical-novelty is the sweet spot in this synthetic setting: it keeps the needle (novel tokens) and drops filler.
• correction-marker is useful only when the signal is explicitly corrective (organism curriculum use case).
• Without filtering, chunking alone does not scale: recall degrades at very long lengths due to retrieval competition, and embedding cost grows linearly.

Implication for architecture

Row 3 of the ablation ladder (+ salience filter) is validated: it is the control that makes chunking practical. Row 2 (chunking without filtering) collapses on its own at scale.

Caveats

• Synthetic filler and a single needle are a weak proxy for real dialogue.
• The foreign-embedder alternative (MiniLM/BGE-class) was not tested; the current results favor same-LM embedding when lexical-novelty keeps the count very small.

Artifacts

• src/oczy/experiments/ingestion.py (default threshold fix)
• src/oczy/experiments/tests/test_ingestion.py (test update)
• src/oczy/experiments/needle_sweep.py
• src/oczy/experiments/logs/SUMMARY.md

Commits

• 42c4ef9 — Fix default salience threshold and verify cost-recall trade-off.
• 734a68e — Update experiments/logs/SUMMARY.md with run #82 result.