R5 — Longitudinal Collapse/Churn Detector for akm improve

Status (0.9.17): removed. The detector, its collapse_detector_alert event, the improve_cycle_metrics / canary_queries tables and scripts/refresh-canary-set.ts were deleted with the improve ledger change (migration 028 drops the tables); recorded telemetry shows no alert ever fired. The generation and lexical-diversity merge checks, which never refused a merge, went with it; only the random-cluster injection in src/commands/improve/anti-collapse.ts remains. The history below is kept for reference.

Earlier status: Partially implemented. The COLLAPSE alert class (§1, three sub-conditions) and the MERGE-FLOOR advisory are live. The CHURN alert class described throughout this document (§1, §5.2 accepted_actions column, §6.1 churnMinAcceptedActions config key, §8 item 2) was removed in the 0.9.0 backlog burn-down (2026-07) and NOT implemented. Its input signal — acceptedActions — was wired to a hardcoded 0 at every call site from the 0.9.0 confidence-gate deletion onward (the design's prescribed source, computeImproveRunMetrics(result).acceptedCount, is computed after envelope assembly, strictly AFTER the point in the pipeline where the detector fires, so it was never reachable there), which meant acceptedSum >= churnMinAcceptedActions (default 25) could never be satisfied and the alert had never once fired. Rather than wire in a differently-scaled substitute signal (e.g. consolidate's own merged/mergedSecondaries counts) and recalibrate a threshold that was tuned against the design's original (unreachable) quantity, the alert class was deleted: CollapseAlertKind no longer has a "churn" member, evaluateCollapseAlerts no longer evaluates it, and the config schema no longer accepts churnMinAcceptedActions. The improve_cycle_metrics. accepted_actions DB column remains (migrations are append-only; dropping it is a separate, unowned migration change) and is now always written as 0. See src/commands/improve/collapse-detector.ts's module doc comment for the live-code version of this note.

Implementation note (paths below are the ORIGINAL PLAN and no longer match the landed tree in a few places — the plan's intent held, the file layout didn't): the merge guards landed in checkGenerationGuard / checkLexicalDiversity / checkMergeInformationFloor, all in src/commands/improve/anti-collapse.ts (planned below as homeostatic.ts, which does not exist); the rank-metrics module landed at scripts/akm-eval/src/rank-metrics.ts (planned below as src/core/eval/rank-metrics.ts — the move stopped one directory short once it turned out src/ had zero importers of it, so there was no src/-side reason to relocate it there); the canary/cycle-metrics DB helpers landed in src/storage/repositories/canaries-repository.ts (planned below as state-db.ts, part of the broader repository-extraction split of src/core/state-db.ts / src/indexer/db/db.ts); the config schema landed in src/core/config/schema/improve.ts (planned below as the monolithic config-schema.ts); the health advisory landed in src/commands/health/advisories.ts (planned below as health.ts); the akm improve canary subcommand landed as documented. Section §10 carries the same corrections in its file-by-file table. One naming note beyond file layout: the frontmatter field this document calls source_refs landed as xrefs (see injectGenerationFrontmatter, which lives in consolidate.ts alongside its merge-branch call site) — read source_refs below as xrefs.

Scope discipline. The field has measured the two failure modes this detects (repeated LLM merge passes collapsing a store to one entry in ~10 passes; unbounded accumulation dropping task accuracy from 39%→13%) [research-08 §10]. AKM has strong preventive scaffolding (generation guard, lexical-diversity check, dedup, cooldowns, multi-run recombine confirmation) but no longitudinal signal to detect collapse or wasted churn if prevention fails. This is the one place the analysis judged adding a small mechanism justified. Everything below reuses existing machinery where it exists (curate-golden rank metrics, computeBigramDiversity, the anti-collapse generation guard, improve_runs metrics, the events log, the health advisory channel) and states the rejected alternative wherever a judgment call was made.

Hard constraints honored throughout: deterministic only — the detector NEVER calls an LLM and never loads an embedding model; bounded storage (state.db already suffered a 3.95 GB result_json bloat — every row this design writes is < 2 KB and retention is explicit); cheap enough to run on every consolidation/recombine cycle (~a few runs/day; improve itself fires every ~20–30 min but the detector only runs on cycles where consolidate/recombine actually did work); fail-open (a detector error warns and skips, never breaks an improve run).

1. Problem statement & detection targets

Two distinct pathologies, distinguishable only longitudinally:

Operational definitions (exact, threshold-bearing)

Let a cycle be one improve run in which the consolidate pass processed ≥ 1 op or the recombine pass evaluated ≥ 1 cluster. The detector snapshots one metrics row per cycle (§5). Let W = windowCycles (default 5) and hist = the last W rows.

COLLAPSE alert fires when ANY of:

1. meanCanaryRecall(current) ≤ median(meanCanaryRecall over hist) − recallDropThreshold (default recallDropThreshold = 0.15 absolute). Median-of-window, not previous-cycle, so a single noisy cycle can't fire or mask the alert.
2. distinct_content_ratio is monotonically non-increasing across all W rows AND hist[0].distinct_content_ratio − current.distinct_content_ratio ≥ entropyDropThreshold (default 0.05).
3. store_memory_count(current) < 0.8 × max(store_memory_count over hist) AND over_generation_count(current) > over_generation_count(hist[0]) — the store is shrinking because of re-merging, not deletion hygiene (over_generation_count = assets with frontmatter generation > maxGeneration).

CHURN alert fires when ALL of:

1. Σ acceptedCount over hist ≥ churnMinAcceptedActions (default 25 — from live data: ~471 accepts/3 days on the default profile means a genuinely idle store won't trip this, but a busy one accumulates it in ~1–2 days of cycles),
2. |meanCanaryScore(current) − meanCanaryScore(hist[0])| < 0.02, and
3. |distinct_content_ratio(current) − distinct_content_ratio(hist[0])| < 0.02.

i.e. real write volume, zero retrieval-visible or shape-visible movement. Churn is a budget-waste advisory, never a candidate for blocking.

MERGE-FLOOR violation (per-merge, not windowed): a consolidate merge whose output fails the information floor (§4). Counted per cycle; an advisory fires when merge_floor_violations > 0 in the current cycle.

Both alert evaluations are pure functions of the stored history rows — trivially unit testable with hand-built rows (§8).

2. Canary query mechanism

2.1 Where the canary set lives

Derived from the live bundle, frozen in state.db (table canary_queries, §5) — NOT a checked-in file. Rejected alternative: a checked-in fixture like tests/fixtures/stashes/curate-golden — a fixed corpus cannot measure the live bundle, and the live bundle content differs per install, so the canary set must be minted from what the store actually contains and then held fixed.

• How many: canaryCount = 40 (config; owner-approved 30–50 range so per-type trend lines are computable). Stable mean at K=10; 40 FTS queries cost single-digit milliseconds total.
• How chosen (deterministic, at first detector run): rank all memory, lesson, and knowledge entries by asset_salience.rank_score (fallback: utility_scores EMA, fallback: entry_key order), take a type-stratified top slice (⅓ memory, ⅓ lesson, ⅓ knowledge, backfilling from the global list when a type is short), and for each anchor asset build a query string from its name tokens + top 3 tags + first heading line of the body (all read from entries.entry_json / search_text). The expected relevant ref is the anchor asset itself.
• How refreshed: never automatically. akm improve canary --refresh (CLI, §6.3) deactivates the current set (active = 0, rows retained for history interpretation) and mints a new one; the detector records canary_set_id per cycle row so trend queries never compare across sets. Rejected alternative: auto-refresh when a canary's anchor is deleted — silent re-baselining is exactly how a slow collapse hides.

2.2 Merge-following (the false-positive killer)

A canary anchor being legitimately merged must not read as collapse. When scoring, a canary counts as a hit if the top-K contains (a) the anchor ref itself, OR (b) any entry whose frontmatter source_refs array contains the anchor ref (consolidate's injectGenerationFrontmatter in src/commands/improve/consolidate.ts already writes source_refs = all merge participants; recombine writes source_refs at recombine.ts:878). One level of indirection only — if a merged asset is merged again and provenance is dropped, the canary misses, and that is precisely the information loss we want to detect. This makes the recall metric a provenance-aware survival probe: "can the store still answer what this asset answered, and can it say where the answer came from."

2.3 Retrieval path: FTS-only BM25 (decision + justification)

The canary runs runFtsQuery-path BM25 search against the live index.db (searchFts in src/storage/repositories/index-fts-repository.ts; bm25 weights name 10 / description 5 / tags 3 / hints 2 / content 1), via a thin exported wrapper (§6.1). No embeddings, no LLM, no model download, byte-deterministic given the same index.

Rejected alternatives, in one sentence each:

• Deterministic embedder (AKM_EMBED_DETERMINISTIC=1) against the live index — wrong vector space: the live index's stored embeddings were produced by the real model, so a hash-embedded query would score garbage; the deterministic embedder is only valid when it embedded the corpus too (which is exactly how curate-golden uses it, and why the detector's CI test can still use it, §8.2).
• Real embedder / full akmCurate hybrid path — the local embedding model is deterministic in principle but heavyweight to load per cron cycle, version-drifts, and its failure modes (model missing, download blocked) would make the detector flaky; collapse is about whether canonical content survives and ranks, which BM25 over name/tags/body measures directly.

What we give up: the canary won't see a vector-only ranking regression. That is the curate-golden CI gate's job (frozen corpus, deterministic embedder, per-commit); the detector's job is longitudinal store health, not ranker health. The two are complementary and share their metric code (§2.4).

2.4 Metrics — reuse curate-golden's rank metrics (with a small move)

scripts/akm-eval/src/curate-metrics.ts already implemented (at design time) pure, IO-free ndcgAtK, recallAtK, mrr, scoreCurateCase, summarizeCurateMetrics. The detector must not duplicate them, and src/ should not import from scripts/ (bundling + layering). Move the module to src/core/eval/rank-metrics.ts and make scripts/akm-eval/src/curate-metrics.ts a one-line re-export so the bench, the CI test, and the detector share one implementation. The module has no imports, so the move is mechanical. Rejected alternative: copy the three functions into the detector — that's the parallel-mechanism accretion this repo bans. (Landed as scripts/akm-eval/src/rank-metrics.ts instead of src/core/eval/rank-metrics.ts — see the implementation note at the top of this document.)

Per-canary the detector computes recall@K, ndcg@K, rank of first hit (K = collapseDetector.k, default 10, binary relevance with the single anchor + its merge-followers as the relevant set); per-cycle it stores the means and a compact per-canary array ([canaryId, rankOfHit|-1] pairs — ints only, ≤ ~400 bytes for 40 canaries).

2.5 When it runs

Once per qualifying improve cycle, in the post-loop stage (runImprovePostLoopStage, src/commands/improve/loop-stages.ts), gated on: detector enabled AND consolidationRan === true AND not options.dryRun. The pass column is always written as "consolidate" today — there is no recombine pass wired into the loop (no recombination value reaches this gate), so the "recombine"/"both" pass values and the recombine-triggered snapshots this section originally described are not implemented. On non-qualifying runs (the ~93% of default-profile runs that touch no merges) the detector does nothing — zero cost on the 20–30-min hot path.

3. Entropy / store-shape metrics

All computed from a single pass over index.db entries for entry_type IN ('memory','lesson','knowledge') (columns: entry_key, entry_type, search_text, entry_json — see the entries table DDL in src/storage/repositories/index-schema.ts). No filesystem reads.

Cost analysis at 5–10k assets (the stated ceiling): one indexed SELECT of ~10k rows ≈ 10–30 ms in bun:sqlite; FNV-64 over ~1–2 KB × 10k ≈ 20–40 ms; bigram diversity over the 2,000-row sample ≈ 30–60 ms; 40 FTS queries ≈ 10–40 ms. Total < 250 ms and zero LLM/model cost, on a pass (consolidate/recombine) that already spends minutes in LLM calls. Memory: search_text strings are streamed row-by-row (prepare/iterate), never all materialized — peak overhead is the 2,000-row sample.

4. Merge-information floor

The analysis asks for two things: (a) a hard floor on re-generalization count, (b) a requirement that merges strictly increase information, not just shorten.

4.1 (a) The hard re-merge floor already exists — turn it on

checkGenerationGuard (src/commands/improve/anti-collapse.ts) already refuses a merge when ≥ 2 participants exceed maxGeneration (default 2), and computeMergedGeneration/readAssetGeneration maintain the frontmatter generation counter through injectGenerationFrontmatter, called from the merge branch of consolidate.ts. It is gated on processes.consolidate.antiCollapse.enabled, default OFF.

Change: flip the anti-collapse guard suite to default-ON (opt out via antiCollapse: { enabled: false }). Concretely, the three call-site gates change from config.enabled truthy-checks to config.enabled !== false: checkGenerationGuard and checkLexicalDiversity (both in anti-collapse.ts, landed name — planned below as homeostatic.ts), and the random-cluster injection gate in consolidate.ts. This matches this branch's precedent (distill quality judge and extract schema-similarity gate both flipped to default-ON, fail-open) and is narrow: the generation guard only refuses the pathological case of merging two already-twice-merged assets. Rejected alternative: a new separate re-merge counter — generation IS that counter; adding another would be the parallel-mechanism malfunction.

4.2 (b) Information floor — new deterministic check beside the generation guard

New pure function in homeostatic.ts (colocated with the other anti-collapse guards; landed in anti-collapse.ts as checkMergeInformationFloor):

export interface MergeInformationFloorResult {
  passed: boolean;
  /** Provenance: |union(source_refs ∪ participants)| after vs. before. */
  provenanceBefore: number;
  provenanceAfter: number;
  /** Specificity proxy: distinct-token retention of merged body vs. union of sources. */
  specificityRetention: number; // 0..1
  reason?: string;
}

/**
 * A merge must strictly increase information:
 *  1. Provenance: the merged asset's source_refs must be a superset of the union of
 *     all participants' source_refs plus the participant refs themselves
 *     (provenance never shrinks through a merge).
 *  2. Specificity: distinctTokens(mergedBody) >= minSpecificityRetention *
 *     distinctTokens(concat(participant bodies)) — a merge that only shortens/
 *     genericizes fails. Tokenization = the same lowercase whitespace split
 *     computeBigramDiversity uses. Default minSpecificityRetention = 0.6.
 */
export function checkMergeInformationFloor(
  mergedBody: string,
  mergedSourceRefs: string[],
  participants: Array<{ ref: string; body: string; sourceRefs: string[] }>,
  config: AntiCollapseConfig,
): MergeInformationFloorResult;

Enforcement point: consolidate.ts merge branch, immediately after the generation guard block (~line 1998, after injectGenerationFrontmatter) where mergedContent and all participant bodies are in hand. Provenance is made to pass mechanically: injectGenerationFrontmatter is extended to write source_refs as the union of participants + their existing source_refs (today it only sets it when absent — consolidate.ts:779-781 — which drops second-generation provenance; that is a bug this fixes). Specificity is measured, and in v1 a failure does not block the merge: it increments merge_floor_violations, pushes a warning, and emits the per-cycle advisory (§7). The documented promotion path (§9) turns it into a refusal (skipReason: "merge_information_floor", mirroring merge_generation_guard at consolidate.ts:1966) once live data confirms the 0.6 threshold's false-positive rate. Rejected alternative: blocking from day one — a wrong threshold silently freezing consolidation is worse than a wrong threshold generating advisories.

Config: two keys added to the existing antiCollapse object (§6.2). No new config surface.

5. Storage schema

5.1 Why not events alone

The events log is capped at 90-day retention (purgeOldEvents, purgeOldEvents in src/storage/repositories/events-repository.ts) and stores metadata as JSON blobs. Collapse is a slow failure — the field's measured case took ~10 merge passes, and AKM's consolidate/recombine cycles are days-to-weeks apart in practice (consolidate 4 runs and recombine-only 1 run in a recent 7-day window) — so a meaningful trend window exceeds 90 days, and trend queries need indexed numeric columns, not json_extract over a 100k-row event table. Decision: cycle history in its own table (365-day retention); alerts (rare, actionable) additionally go to the events log so they surface through the existing readEvents-based health plumbing exactly like outcome_proxy_inverted does. Rejected alternative: rows in improve_runs.metrics_json — that JSON is per-run not per-cycle, is the exact blob that already bloated to 3.95 GB, and is slated for retention-nulling (R9).

5.2 Migration 016 (append-only; 015 verified current tail of MIGRATIONS in src/core/state/migrations.ts)

-- ── Migration 016 — collapse/churn detector (R5) ─────────────────────────────
CREATE TABLE IF NOT EXISTS canary_queries (
  id            INTEGER PRIMARY KEY AUTOINCREMENT,
  canary_set_id TEXT    NOT NULL,          -- mint token; all rows of one set share it
  anchor_ref    TEXT    NOT NULL,          -- e.g. 'memory:alpha'
  query         TEXT    NOT NULL,          -- deterministic FTS query string
  source        TEXT    NOT NULL DEFAULT 'auto',  -- 'auto' | 'manual'
  active        INTEGER NOT NULL DEFAULT 1,
  created_at    TEXT    NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_canary_queries_active
  ON canary_queries(active, canary_set_id);

CREATE TABLE IF NOT EXISTS improve_cycle_metrics (
  id                      INTEGER PRIMARY KEY AUTOINCREMENT,
  run_id                  TEXT    NOT NULL,  -- improve_runs.id of the cycle
  ts                      TEXT    NOT NULL,  -- ISO-8601 UTC
  pass                    TEXT    NOT NULL,  -- 'consolidate' | 'recombine' | 'both'
  canary_set_id           TEXT    NOT NULL,
  mean_recall             REAL    NOT NULL,
  mean_ndcg               REAL    NOT NULL,
  mean_mrr                REAL    NOT NULL,
  canary_ranks_json       TEXT    NOT NULL,  -- [[canaryId, rankOfHit|-1], ...] ≤ ~200B
  store_total             INTEGER NOT NULL,
  store_by_type_json      TEXT    NOT NULL,  -- {"memory":n,"lesson":n,...} ≤ ~200B
  distinct_content_ratio  REAL    NOT NULL,
  mean_bigram_diversity   REAL    NOT NULL,
  over_generation_count   INTEGER NOT NULL,
  accepted_actions        INTEGER NOT NULL,
  merge_floor_violations  INTEGER NOT NULL DEFAULT 0,
  alerts_json             TEXT    NOT NULL DEFAULT '[]'  -- fired alert kinds, ≤ ~100B
);
CREATE INDEX IF NOT EXISTS idx_improve_cycle_metrics_ts
  ON improve_cycle_metrics(ts);

5.3 Footprint & retention

Every column is a scalar or a size-capped JSON array; a row is < 2 KB by construction (the only variable-size fields are the three small JSON columns, each bounded by canaryCount/type-count). At the realistic maximum of ~10 qualifying cycles/day, one year of history is < 7 MB. Retention: purgeOldCycleMetrics(db, retentionDays = 365) — same shape as purgeOldEvents — called from the same maintenance point that already runs purgeOldEvents in runImproveMaintenancePasses, emitting the purge count into the existing events_purged-style observability (new metadata key, no new event type needed). canary_queries rows are never purged (tens of rows, needed to interpret history).

6. Wiring points

6.1 New module: src/commands/improve/collapse-detector.ts (~250 LOC)

export interface CollapseDetectorConfig {
  enabled?: boolean;            // default true
  canaryCount?: number;         // default 40 (owner-approved 30–50 range)
  k?: number;                   // default 10
  windowCycles?: number;        // default 5
  recallDropThreshold?: number; // default 0.15
  entropyDropThreshold?: number;// default 0.05
  churnMinAcceptedActions?: number; // default 25
  retentionDays?: number;       // default 365
}

export interface CycleMetrics { /* mirrors improve_cycle_metrics columns */ }
export type CollapseAlertKind = "collapse-recall" | "collapse-entropy"
  | "collapse-shrink" | "churn" | "merge-floor";
export interface CollapseAlert { kind: CollapseAlertKind; detail: string;
  metrics: Record<string, number>; }

/** Mint (or return) the active canary set. Deterministic given the index +
 *  salience tables. Called lazily by runCollapseDetector on first use. */
export function ensureCanarySet(stateDb: Database, indexDb: IndexDatabase,
  cfg: CollapseDetectorConfig): { canarySetId: string;
  canaries: Array<{ id: number; anchorRef: string; query: string }> };

/** Compute one cycle's metrics. FTS + entries scan only; no LLM, no model. */
export function computeCycleMetrics(stateDb: Database, indexDb: IndexDatabase,
  args: { runId: string; pass: "consolidate" | "recombine" | "both";
    acceptedActions: number; mergeFloorViolations: number;
    cfg: CollapseDetectorConfig }): CycleMetrics;

/** PURE. Evaluate §1's alert definitions over stored history. */
export function evaluateCollapseAlerts(history: CycleMetrics[],
  current: CycleMetrics, cfg: CollapseDetectorConfig): CollapseAlert[];

/** Orchestrator: ensure canaries → compute → evaluate → persist row →
 *  appendEvent per alert. Fail-open: catches, warns, returns undefined. */
export function runCollapseDetector(args: { stateDbPath?: string;
  indexDbPath?: string; runId: string; pass: "consolidate"|"recombine"|"both";
  acceptedActions: number; mergeFloorViolations: number;
  config: AkmConfig; eventsCtx?: EventsContext }): CycleMetrics | undefined;

Canary retrieval goes through searchFts (src/storage/repositories/index-fts-repository.ts); rank metrics import from the relocated rank-metrics module (§2.4, landed at scripts/akm-eval/src/rank-metrics.ts); bigram diversity imports computeBigramDiversity from ./anti-collapse (landed name).

6.2 Hook + config + existing-file edits

6.3 Default-on vs opt-in: default-ON, justified

The branch precedent is explicit: quality gates that are deterministic, cheap, and fail-open ship default-ON with a config opt-out (distill qualityGate, extract schemaSimilarity, outcome weights). The detector is strictly observational in v1 (never blocks anything), costs < 250 ms only on merge-active cycles, uses no LLM/model, and fails open. An opt-in longitudinal detector is a contradiction — the operator who remembered to enable it is the one who didn't need it. Opt-out: improve.collapseDetector.enabled: false.

7. Alert semantics

• Per qualifying cycle, always: one improve_cycle_metrics row. No event (avoid event-volume creep; history lives in the table).
• On any alert (rare): one collapse_detector_alert event per fired kind (appendEvent, best-effort, same pattern as outcome_proxy_inverted at preparation.ts:1575-1594), and the kinds recorded in the row's alerts_json.
• Health surface: the akm health advisory (§6.2) — warn status, kind: "deterministic", confidence: "high" for collapse kinds (they're measured, not inferred), confidence: "medium" for churn (volume thresholds are tunable). The message includes the concrete numbers (current vs. window-median recall, entropy delta, accepted-action count) so the operator can act without opening the DB.
• Blocking: NO in v1. The detector observes; it never vetoes a consolidation pass. Recommendation is observe-first with this documented promotion path: (1) v1 ships observe-only; (2) after ≥ 30 days of live rows and threshold tuning (§9), an opt-in improve.collapseDetector.blockOnCollapse: true makes runConsolidationPass skip the merge phase (not the whole pass — dedup/archival still run) for a run whose previous cycle fired a collapse-kind alert, fail-open on any detector error, and emits a consolidation_blocked note into warnings; (3) the merge-information floor independently promotes from advisory to per-merge refusal (§4.2). Blocking on the previous cycle (not live re-evaluation) keeps the merge path free of detector latency and makes the block deterministic from persisted state.

8. Test plan

All tests use the _helpers/sandbox withIsolatedAkmStorage helper (per withSeededGolden in tests/integration/curate-golden-eval.test.ts); no raw process.env mutation (scripts/lint-tests-isolation.ts enforces this). No test touches a live DB. All new tests are CI-fast unit tests under tests/ (no Bun.spawn/Bun.serve/60s timeouts — the unit-vs-integration boundary rule).

(Landed layout differs from the plan below: the simulation/pure-function/orchestrator tests (items 1-3, 7) and the curate-golden reuse guard (item 6) all landed together in tests/integration/commands/improve/collapse-detector.test.ts; item 4 landed in tests/commands/improve/anti-collapse.test.ts; item 5 landed in tests/integration/state-db/improve-cycle-metrics.test.ts. See the implementation note at the top of this document.)

1. collapse-detector.test.ts — deterministic collapse simulation (the headline test). Seed an isolated bundle with 30 distinct memories (distinct topics, distinct vocabularies), akmIndex it (FTS only — the detector path needs no embeddings; set semanticSearchMode: "off" in the sandbox config), mint the canary set, snapshot cycle 0. Then simulate merge passes directly on the bundle files: each pass replaces groups of 3 memories with one generic merged body (progressively blander shared phrasing, xrefs written for pass 1 then deliberately dropped for pass 2+ to model provenance loss), reindex, run computeCycleMetrics + evaluateCollapseAlerts. Assert: distinct_content_ratio and mean_bigram_diversity decrease monotonically; mean recall survives pass 1 (merge-following via xrefs) and drops afterward; a collapse-recall or collapse-entropy alert fires by pass ≤ 4; the collapse_detector_alert event is written.
2. Churn simulation (same file): paraphrase bodies without merging (store shape and canary hits stable), feed acceptedActions ≥ threshold per cycle for W cycles, assert exactly a churn alert and no collapse alert.
3. evaluateCollapseAlerts pure-function table tests: hand-built history rows for each §1 clause boundary (just-below/just-above each threshold; median-window robustness against a single-cycle recall spike; window shorter than W never alerts).
4. Merge-information floor unit tests (anti-collapse.test.ts): provenance-union enforcement, specificity retention at the 0.6 boundary, shortening merge fails, genuinely-additive merge passes; plus the default-on flip (enabled absent ⇒ guards active; enabled: false ⇒ inert — byte-identical to the old default for opted-out configs).
5. improve-cycle-metrics.test.ts: migration 016 applies on a fresh and an existing DB; insert/query round-trip; purgeOldCycleMetrics deletes only rows past retention and returns the count; canary set mint / deactivate / re-mint preserves history rows with the old canary_set_id.
6. Curate-golden reuse guard: the metrics move in §2.4 keeps curate-golden-eval.test.ts green unchanged (it imports through the re-export) — that existing test IS the regression guard for the shared module; add one assertion in the detector test importing ndcgAtK from the landed rank-metrics module (scripts/akm-eval/src/rank-metrics.ts) to pin the canonical path. Additionally, one detector test seeds the curate-golden fixture corpus with AKM_EMBED_DETERMINISTIC: "1" (exactly as withSeededGolden does), mints canaries against it, and asserts cycle-0 mean recall ≥ 0.9 — proving the canary mechanism itself finds well-formed content on a known-good corpus.
7. Wiring test (same collapse-detector.test.ts file, "post-loop hook gating" group): an improve run with a stubbed consolidate that reports work writes exactly one cycle row; a run with no consolidate/recombine work writes none; dryRun writes none; a detector that throws (inject a bad indexDbPath) leaves the improve run green with a warning (fail-open).

9. Rollout plan

• Phase 0 — land observe-only (this design, one PR). Default-on detector + default-on generation guard + advisory-only information floor. Gate: full bun run check green (custom lints included), zero new warnings.
• Phase 1 — accumulate & tune (2–4 weeks of live cron). Watch, read-only:

  SELECT ts, pass, mean_recall, distinct_content_ratio, mean_bigram_diversity,
         over_generation_count, accepted_actions, merge_floor_violations, alerts_json
  FROM improve_cycle_metrics ORDER BY ts DESC LIMIT 30;
  SELECT ts, json_extract(metadata,'$.kind') FROM events
  WHERE event_type='collapse_detector_alert' ORDER BY ts DESC LIMIT 20;
  

  Tune recallDropThreshold / entropyDropThreshold / churnMinAcceptedActions against observed variance (the live synthesis lanes are currently near-idle, so early rows will be sparse — that sparsity is itself the §6.3 "synthesis lanes idle" signal made durable). Success criterion for the thresholds: zero false alerts on a healthy store over the window, and the simulated-collapse test values sitting well inside the alert region.
• Phase 2 — optional enforcement (separate PR, owner sign-off). blockOnCollapse opt-in (§7) and information-floor refusal (§4.2), each promoted independently and only if Phase 1 shows clean precision.

Non-goals (explicit): no LLM-judged merge quality (deterministic-only is a design invariant here); no embedding-drift tracking of the live vector space (curate-golden owns ranker quality); no automatic canary refresh; no changes to user-facing search/curate ranking; no retention change to improve_runs.result_json (that is R9, separately owned and sign-off-gated); no resurrection of the deleted homeostatic demotion pass.

10. Estimated diff size & file-by-file change list

Estimates below are as planned; paths marked "landed at" are where §6.2's implementation note says the code actually ended up.

Total: ~+1,290 / −195 (net ~+1,100 incl. ~520 test LOC; production code ~+660). For calibration: the moved metrics module and the simulation tests are the two biggest blocks; the runtime footprint added to the improve hot path is one gated function call.

Owner decisions (resolved 2026-07-02)

All three open questions were put to the owner and approved:

1. Anti-collapse guard suite flips to default-ON (§4.1) — approved. The generation guard may newly refuse the narrow two-participants-both-gen>2 merge case; acceptable as a live-pipeline behavior change.
2. 365-day retention for improve_cycle_metrics (§5.3) — confirmed (< 7 MB/yr worst case; the slow-failure detection window outweighs the bloat concern).
3. Canary count: 30–50 with per-type trend lines — approved. Size the canary set toward the top of the range (type-stratified across memory/lesson/knowledge so per-type collapse divergence is visible); cost stays negligible.