Mindbridge Β· track status

AMA-TRUST

The track exists because AMA's failure mode is confident wrongness β€” an answer that looks complete, reads fluently, and is quietly missing the thing that was asked for. Scope is answer correctness only; permissions moved to AMA-DISCLOSE on 2026-08-01.

Opened 2026-07-30 Β· source-file census and re-parse 2026-08-07 Β· production figures measured 2026-08-15 (bump progress, cell census, aggregation timings) Β· re-measured against origin/main and live issues 2026-08-17 Β· Fleet 1 wave 1 landed 2026-08-19 β€” four PRs merged, root cause of the aggregation gap found and fixed, three rulings closed Β· Fleet waves 2-3 landed 2026-08-19β†’08-24 β€” nine more PRs merged, the ratio operation and number formatting closed with them (the choke-point AST rules closed at the 08-25 reconciliation), and the undetected tail measured at 12.3% Β· 2026-08-29 and 09-01 track reconciliations closed the remaining spreadsheet-cluster verification items (schema typing, tail detection, cross-snapshot stitching, the under-reporting census) once their landed code was confirmed wired to a live caller Β· page re-derived 2026-09-02 against live issues and origin/main β€” 28 issues previously cited as open work are now closed; see History for what shipped.

Next β†’ #2900 coverage as a structured ledger β€” the last p1 on the board. ADR-148 exists on origin/main but stays status: proposed, and nothing is built (a repo-wide grep for CoverageLedger/coverage_ledger returns zero hits) β€” the ruling this issue asks for has not been made. Every item the spreadsheet cluster previously queued ahead of it β€” the ratio operation, the tail detector, cross-snapshot stitching, the typed-schema verification, the under-reporting census β€” has since closed, so this is now the head of the whole track. 28 previously-cited issues closed across 2026-08-19β†’09-01 (see History) Β· remaining open work: #2900 (next), #3185 (SQL-lane ruling), #3180, #3186 item 1, #3214, #2908, #2907/#2909 (parked), #2902 (upstream residue), #3314/#3327 (attachment backfill, cross-tracked)
33 open β€” 6 spreadsheet-pipeline items. Every item this page previously tracked as in-progress spreadsheet or answer-correctness work has shipped and closed β€” the render/RTL fix, the aggregation-tool routing fix, the detection/typing/aggregation pipeline, the tail detector, the ratio operation, cross-snapshot stitching, number formatting, the truncation notice, the under-reporting census, and the two choke-point AST rules. Grouped detail is in History. What remains open: the coverage ledger (next), a SQL-lane conversion ruling, a forked media-kind vocabulary, export-time disclosure, two parked directions, and the attachment-recovery backfill tracked jointly with ATTACHMENT-TRUTH.
33open in the track
6spreadsheet sub-track
0.677 / 0.875table detection precision / recall (08-17)
12.3%undetected tail, measured (08-22)
2,612 / 2,672xlsx nodes, empty body_text (08-15)
Open work Why this order Rulings History

Open work

The spreadsheet-pipeline cluster, the answer-correctness batch and the choke-point/AST items that used to fill this section (28 issues) have all shipped and closed β€” grouped detail is in History. What remains open:

Answer correctness
p1 #2900 AMA: coverage as a structured ledger, not a prose caveat next β€” ADR-148 exists, still proposed All
Calibrated hedging is the cheapest fix for confident wrongness: a coverage claim becomes a structured ledger rather than a prose caveat the model may or may not state. ADR-148 exists on origin/main but stays status: proposed; nothing is built yet. The last p1 on the board, and now the head of the whole track since every item that previously queued ahead of it has closed.
Attachment recovery β€” cross-tracked with ATTACHMENT-TRUTH
#3314 A backfilled message carries no provenance that it was repaired fixed forward, 3-row residue owner-gated Dev only
Fixed forward in PR #3545 β€” the backfill now threads provenance into update_raw_metadata before the body rewrite, so an interrupt leaves the row retryable. The 3 already-affected production rows are NOT retroactively marked and cannot be reached even post-merge; that residue needs a targeted one-off write that stays owner-gated. Carries state:not-in-effect until cleared.
p2 #3327 Recover the dropped attachments from retained maildir .eml (ADR-145 D6) recovery script merged β€” never run Enterprise
Every maildir .eml is retained, so the dropped documents are recoverable with no vendor refetch; scope is maildir only. The recovery script exists (PR #3559, prod-measured at 95,844 orphaned messages against 260,922 raw objects) but merging it recovers nothing β€” it has never been run. A dry run, a bounded pass and a re-derived cost model are still owed, and the connector_staged_events retention window stays paused on prod until it runs. Full detail on the ATTACHMENT-TRUTH page.
Coverage & choke-points β€” structural
p2 #2908 AMA: export an answer as a report β€” and re-run disclosure at export time export escapes disclosure gate All
Owner asked directly. Export escapes the disclosure gate; shares the fetch-by-id disclosure gap with the continuation tool and aggregation.
p3 #3180 AMA-TRUST: the media-kind vocabulary is forked across 85 files and three deploy targets vocabulary forked across files Dev only
85 files name two or more media-kind literals β€” 25 backend, 41 client, 19 platform β€” against the 19 estimated, across three deploy targets and toolchains. Do the backend lane first; the client/platform lane needs a generated artifact plus two checks.
p2 #3185 AMA-TRUST: document-body expressions β€” ruled, now the Wave-2 conversion tracker ruled: convert the SQL lane All
Ruled 2026-08-19: convert the document-body SQL lane anyway, via a producer-written column β€” one-way, not the safe-way the census would otherwise justify. Issue stays open as the Wave-2 conversion tracker, not as a decision blocker.
p3 #3186 AMA-TRUST: two salvage-usability gaps tool identity cannot close, and a dead prefix→media_type table item 2 shipped; item 1 open Dev only
Item 2 shipped (PR #3541 — the prefix→media_type build now raises at import time on a repeated prefix). Item 1 stays open: the two TestKnownGaps in test_salvage_usability.py need semantic judgement of answer text, not a mechanical fix.
p3 #3214 "Which text names this node in one line?" is the same expression written nine times across two lanes πŸ…ΏοΈ rehomed to VIEWER-TRUTH β€” not attempted here Dev only
One expression β€” "which text names this node in one line?" β€” written nine times across two lanes. The 2026-08-23 fleet wave picked it up and put it back down: the display-name expression is viewer-truth territory, and touching it from this track risked cross-fleet collision on the same files. It stays open, un-started, and is cheapest for whoever owns the viewer-name choke-point.
Decisions & directions
p2 #2902 AMA: recognise recurring reports as a series, and offer the complete snapshot shipped β€” upstream retrieval-breadth gap remains All
Series recognition shipped (PR #3564) β€” list_series_candidates + series_completeness_notice() wired into describe_spreadsheet. Known scope limitation kept on the issue rather than cut: the notice fires only once the model has already opened a truncated sheet β€” it does not cover the model never opening the fuller sibling at all. That upstream retrieval-breadth gap was ruled out of scope here and is not yet filed elsewhere; the issue stays open to carry it.
p3 #2907 AMA: a continuation tool so the model can fetch past a truncation marker πŸ…ΏοΈ PARKED β€” on-track, no fleet capacity Latent
Ruled PARKED 2026-08-19 (state:parked) β€” stays on the track, no reordering lever, un-park conditions on the issue. Superseded for spreadsheets β€” it pages past a message cliff, and that cap has fired zero times.
p3 #2909 AMA: standing answers with diffs (direction, needs product scoping) πŸ…ΏοΈ PARKED β€” on-track, no fleet capacity Latent
Ruled PARKED 2026-08-19 (state:parked) β€” stays on the track, no reordering lever, un-park conditions on the issue. Direction, not a ticket: pin an answer as a standing result set with a weekly diff. Depends on structured answers; needs product scoping before implementation.
Why is it in this order?

Ranked by what a user would notice first. The spreadsheet cluster sorted first because Regina asks a question about her own spreadsheets and gets nothing back β€” the track's own impact statement. Re-read 2026-09-02: that entire cluster has since closed β€” the render defect, the aggregation root cause, the ratio operation, the undetected-tail detector, the visible-degradation set (truncation notice, number formatting), and the cross-snapshot stitching defect the tail work uncovered. What a user would notice today is just the coverage LEDGER (#2900, the last p1 β€” hedging that is structured rather than prose), which is why it is now the whole track's head. Answer-correctness degradations come second, then structural/choke-point work, then decisions and directions last.

What it ignores: priority labels (the live render defect is p2 and sits first), filing order and age, cost or ease, and the needs:ruling flag as a reordering lever β€” a ruling that blocks work sorts with its work, not above it.

What the audit found β€” generated

Evidence below is generated from the read-only audit of 2026-08-29 and reconciled against the open issues at build time. Every claim cites the issue it came from. Nothing here is a decision β€” rulings stay in the prose above, which is written by hand.

#1279 fix(ama): AMA streaming timeout (60s) handling for long-form queriesprodhigh confidence

The 60s wall is unchanged and still cuts turns in production TODAY. Code: `AGENT_STREAM_TIMEOUT_SECONDS = 60` at backend/app/core/constants.py:1394, consumed at backend/app/services/ask/ask_agent_service.py:711 (`deadline = time.monotonic() + AGENT_STREAM_TIMEOUT_SECONDS`) and stream_turn.py:354. Prod (2026-08-29): `select outcome, count(*), round(avg(total_ms)), max(total_ms), count(*) filter (where driver_escalated) from public.ama_turns group by 1` β†’ ok 199 (avg 42.7s), **truncated 37 (avg 61.2s, max 64.2s)**, abandoned 4, error 1. 37/241 = 15.4% of every retained turn ends truncated, clustered exactly at the 60s deadline. `truncated` is NOT a graceful outcome: ask_agent_service.py:750-756 records it only when forced synthesis ran and did NOT recover β€” the user reads a cut-off answer. Partial mitigation HAS shipped (forced-synthesis floor `AGENT_SYNTHESIS_MIN_BUDGET_SECONDS = 20`, backend/app/services/ask/_driver_escalation.py, and the truncated outcome itself), which is why this is not 'silently drops' any more β€” but the drop rate is 15.4%, not zero. Premise staleness that does NOT rescue the issue: the blocker #1271 (grok-4.3 swap) is CLOSED and the driver is now anthropic/claude-sonnet-5 (llm_routing.py:310), so the issue's 'may become moot if the new model is faster' branch is resolved β€” measured, it did not become moot.

#2900 AMA: coverage as a structured ledger, not a prose caveatcodehigh confidence

ADR-148 EXISTS on origin/main (`docs/explanation/decisions/ADR-148-coverage-ledger-structured.md`) and its front-matter reads `status: proposed` β€” the ruling the issue asks for has not been made. Nothing is built: `git grep -rn "CoverageLedger\|coverage_ledger" origin/main -- backend client` returns ZERO hits (population = the whole backend and client trees). `git grep -rniE "coverage|ledger" origin/main -- backend/app/services/ask` (population = the entire AMA ask service) returns only prose: a `coverage_window()` tool referenced from _system_prompt.py:358/476/525 and a COVERAGE section at _system_prompt.py:515 that instructs the model to STATE the boundary in prose β€” precisely the prose-caveat mechanism the issue says is insufficient. The one structural signal the ADR builds on does exist: `class ServedCount` at backend/app/services/ask/tools/_result_header.py:54. Client-side: `git grep -rniE coverage -- client` hits only test comments and package-lock istanbul entries β€” no chip component.

#2902 AMA: recognise recurring reports as a series, and offer the complete snapshotabsencehigh confidence

Search: `git grep -rln "recurring_series\|report_series\|periodic_series" origin/main` over the ENTIRE repo at origin/main β€” zero files. Complementary: `git grep -rniE "coverage|ledger" origin/main -- backend/app/services/ask` (population = every file in the AMA ask service, the only place series-aware retrieval could be wired) shows retrieval helpers are `coverage_window`, `hybrid_search` and the spreadsheet tools; no series/periodicity concept and no 'a later snapshot is more complete' offer path. No ADR exists for it (`git ls-tree origin/main docs/explanation/decisions/` has no series entry alongside ADR-148). Nothing has shipped.

#2907 AMA: a continuation tool so the model can fetch past a truncation markerprodhigh confidence

BOTH unknowns closed. (a) UN-PARK CONDITION 1 IS STRUCTURALLY UNRECORDED, not merely unmeasured. `information_schema.columns` for `public.ama_turns` (29 columns read in full): there is NO `exec_metadata`; migration 20260823192813-era exec-accounting added `tool_calls`, `tool_exec_ms`, `provider_ms` (jsonb timings/ids, not result text). `select count(*) from public.ama_turns where tools::text ilike '%truncat%' or tool_calls::text ilike '%truncat%'` β†’ 0, against 130 turns created since the 2026-08-19 park. `public.chat_messages` = 12 rows, 0 matching '%TRUNCATED%'. The marker exists only in tool-result text emitted by the single choke-point `truncate_with_marker` (backend/app/services/ask/tools/_helpers.py:397, taught to the model at backend/app/services/ask/_system_prompt.py:574) and that text is never persisted to prod. No query can tell whether the cap has fired. (b) THE PERMISSIONS BLOCKER LANDED, but not where the issue says: `git grep -n "as_user|disclose" origin/main -- backend/app/repositories/message_repository.py` = ZERO hits, and `find_by_ids` exists only as the ungated generic at backend/app/repositories/base_repository.py:354. The gating landed in the ask service β€” `backend/app/services/ask/document_fetch_resolver.py` plus 9 sibling resolvers carry `as_user`. `gh issue view`: #3167 CLOSED (audience-predicate read choke-point), #3380 MERGED (tier/tenant-scoped document fetch-by-id resolver), #3408 MERGED (shared carrier-scope resolution for every document-serving surface).

#2908 AMA: export an answer as a report β€” and re-run disclosure at export timecodehigh confidence

No export/report surface exists in the ask stack on origin/main. `backend/app/services/ask/tools/__init__.py` enumerates the full tool set (aggregate_spreadsheet, count_entities, coverage_window, derive, find_attachment, get_my_profile, get_person_card(s), matches_and_gaps, now_feed, search_conversations, search_messages, semantic_search, surface_decisions, surface_open_questions, traverse_graph, web_search) β€” no export/download/csv/xlsx-emit tool. `git grep -inE "export|csv|xlsx" origin/main -- backend/app/services/ask/` returns only re-export comments, budget constants and xlsx *ingest* handling. Positive confirmation the work is unbuilt and known-unbuilt: origin/main:backend/app/services/ask/document_fetch_resolver.py:28-33 states verbatim that "Export (#2908) is a RENDER surface β€” it degrades a below-FULL document to a de-identified variant rather than dropping it β€” and that exit (``disclosed_for_render``) is not built here… it needs its own declared render path when it is built."

#2909 AMA: standing answers with diffs (direction, needs product scoping)codehigh confidence

The issue's own comment (aeldaly, 2026-08-18) parks it with an explicit un-park condition: #2902 (recurring reports as a series) lands AND the owner scopes the product question. Verified `gh issue view 2902` β†’ state OPEN, closedAt null, labels priority:p2 / track:ama-trust. Substrate prerequisite (structured row-set answers + series identity) is therefore absent; no scheduling/diff code exists in `backend/app/services/ask/` (tool registry enumerated above contains nothing schedule- or diff-shaped, and no pg_cron job in the 33-row prod census drives an ask re-run).

#3180 AMA-TRUST: the media-kind vocabulary is forked across 85 files and three deploy targetsabsencehigh confidence

No cross-language vocabulary artifact and no re-fork gate exist on origin/main. Searches run: `git grep -rln "media-kinds|mediaKinds|media_kinds" origin/main -- client platform backend/app scripts` β†’ one unrelated hit (backend/app/connectors/whatsapp/live_parser.py); `git grep -n "media" origin/main -- backend/scripts/check.py` β†’ zero hits (no AST rule). Population for the fork itself: `git grep -l -E "'(image|video|audio|pdf|docx|xlsx|pptx|vcf)'|\"…\"" origin/main -- <dir>` β†’ backend/app 39, client 53, platform/src 23 files carrying >=1 media-kind literal (a superset of the issue's >=2-literal count, so it neither confirms nor shrinks the 85). The only choke-point that DID ship is per-extractor and backend-only: backend/app/orchestration/phases/document_extraction/media_taxonomy.py:57 `_EXTRACTORS` with derived tables β€” it covers extractor version/media_type/governor bucket, not the kind vocabulary the client and platform declare.

#3185 AMA-TRUST βš–οΈ RULING NEEDED: the four remaining document-body expressions β€” convert or…prodhigh confidence

Rows 1, 2 and entity_projection_service are settled per the owner ruling; the one live row (refresh_library_documents_mat) has its CODE conversion shipped but its DATA not in effect. Code: supabase/migrations/20260819082749_library_documents_mat_producer_dedup_text.sql introduces the producer-written column pattern, carried forward through 20260821082456_channel_facet_source_type.sql:66. Prod (installed body, not migration text): `select pg_get_functiondef(oid) like '%dedup_text_hash%' from pg_proc … proname='refresh_library_documents_mat'` β†’ true. Prod data: `select properties->>'kind', count(*) filter (where (properties->>'dedup_text_hash') is null), count(*) from nodes_v2 where node_type='document' …` β†’ xlsx 2,817 null of 5,298; pdf 13,709/13,709 null; image 8,668/8,668; docx 701/701. The key-presence backfill defect the 2026-08-19 comment measured IS fixed in code β€” backend/scripts/tools/backfill_document_dedup_text.py:99 and :126 test `(properties->>'dedup_text_hash') IS NULL` (VALUE), not key presence β€” but the script has no scheduled caller (it is not among the two `dev_tools.py` GH-Actions hits, no pg_cron job shells out, and neither Fly scheduled machine runs it), and prod still shows 2,817 unstamped xlsx docs, so it has not been run to completion.

#3186 AMA-TRUST: two salvage-usability gaps tool identity cannot close, and a dead…codehigh confidence

Half the issue is closed and the issue text no longer describes state. Item 2 (the prefix table) is FIXED: backend/app/orchestration/phases/document_extraction/media_taxonomy.py:181-195 β€” `_version_prefix_to_media_type()` raises `ValueError("extractor_version prefix … is claimed by more than one extractor family")` on a repeated prefix, and line 195 binds `_VERSION_PREFIX_TO_MEDIA_TYPE` at module scope so the collision fails at import. The issue's other half (delete it as dead) was rejected in favour of the guard. Item 1 is UNCHANGED: backend/tests/unit/services/ask/test_salvage_usability.py:332 `class TestKnownGaps` still holds both tests, each asserting the wrong-ish current behaviour with an inline `# ← should be … once the gap closes` marker (lines ~349 and ~369).

#3214 [L2-g4-w4] "Which text names this node in one line?" is the same expression written nine…codehigh confidence

No shared expression exists and the site count is LARGER than filed. Absence of a helper: `git grep -nE "def (node_display_text|display_text_for_node|one_line_text|node_one_line)" origin/main -- backend/app` β†’ zero hits. Live Python sites of the exact three-field fallback (`git grep -n "searchable_text or " origin/main -- backend/app`, population = all of backend/app): services/analysis/conversation.py:808, services/analysis/conversation_matching.py:150, services/ask/connection_resolver.py:452 and :499, services/ask/content_resolver.py:370 and :684, services/ask/decision_resolver.py:325, services/ask/question_resolver.py:299, services/ask/tools/_unmet_needs.py:333/:404/:459, services/connections/pattern_matchers/duplicate_work.py:162/:163, services/connections/pattern_matchers/problem_solution.py:266/:268 β€” 16 occurrences, versus the three Python sites the issue names. Python-embedded SQL lane still forked: backend/app/repositories/feed_intelligence_repository.py:261, :320, :436 carry `COALESCE(NULLIF(tgt.searchable_text, ''), tgt.display_name, tgt.canonical_name, '')` verbatim. SQL lane: the same expression appears three times per redefinition of the needs-you view, latest at supabase/migrations/20260819104315_fix_viewer_needs_you_labeled_passthrough_3389.sql:99, :159, :211 β€” the issue's cited migration (20260807105944) has been superseded five times since, which is exactly the drift the issue predicts.

#3314 [L1-small-w1,L1-small-w2] A backfilled message carries no provenance that it was repairedprodhigh confidence

Code half IS shipped and reachable: backend/scripts/tools/backfill_inlined_rfc822_bodies.py:401 writes update_raw_metadata({'recovered_inlined_rfc822': records, 'recovered_inlined_rfc822_via': 'backfill'}) on origin/main, pinned by backend/tests/integration/test_backfill_inlined_rfc822_bodies.py:544-545. Data half is NOT in effect: prod query 2026-08-29 `select count(*) from public.messages where raw_metadata ? 'recovered_inlined_rfc822_via'` -> 0; and the three ids named in the issue body (bae749ce-39ed-4d60-aa1d-59ed2f925821, 228c4515-97d1-4c32-812b-5b5c23061846, 79ccf801-bbd3-438c-ae84-f176f546534f) each return m=false, v=false with content lengths 867/939/1163 (i.e. already rewritten, still unmarked). Only 2 rows carry the ingest-path key at all.

#3585 [L56-g4-w6] Regina: 73% of documents have stale embeddings after 2026-08-19 batch…prodhigh confidence

supabase execute_sql on public.nodes_v2, 2026-08-29: Regina (e9d96b09-…) documents total 26,014, stale (content_changed_at > embedding_updated_at) 8,959 = 34% (not 73% β€” the denominator grew from 12,188 to 26,014 while the stale count barely moved). Staleness is fleet-wide, not Regina-only: aeldaly 1,521/1,558, be0f3db0 1,198/1,567, 2394e9b2 527/527, c404c3ca 27/27, cb40cdb9 19/127. The issue's central causal premise β€” a single 2026-08-19 12:44–21:28 batch window β€” is contradicted: Regina's stale rows span content_changed_at 2026-08-27 02:07 through 2026-08-29 13:06 (i.e. generated continuously, up to the hour of this census). Lag distribution across all tenants: 3,676 rows stale by >=7 days, 6,900 by <7d, 1,680 by <1d β€” so this is not embedding merely trailing the writer by minutes.

Open questions β€” what the audit could not settle
  • #1279 Whether 15.4% is the true long-run rate or an artifact of the retention window: `ama_turns` is pruned nightly by pg_cron job 37 (`run_retention_policy('ama_turns')`, 03:44 UTC), so 241 rows is a recent slice, not all history. Closes with: `select date_trunc('day',created_at) d, count(*) filter (where outcome='truncated')::float/count(*) from public.ama_turns group by 1 order by 1` for the in-window trend, plus `select value from retention_policy_registry where table_name='ama_turns'` for the horizon. Also unknown: how many truncated turns had forced synthesis attempted-and-failed vs never reached β€” `select count(*) filter (where driver_escalated) from ama_turns where outcome='truncated'` gave 6 of 37, which bounds escalation but not synthesis attempts.
  • #2900 Whether the owner has ruled off-repo. Closing command: `gh issue view 2900 --json comments` β€” it has ZERO comments, and the ADR front-matter still says `status: proposed`, so no ruling is recorded anywhere I can read.
  • #2902 Whether Regina's daily collection reports still arrive as a detectable series in prod (the premise). Closing query: `select sender_person_id, date_trunc('day',sent_at), count(*) from public.messages m join public.message_email_headers h on h.message_id=m.id where m.tenant_id='e9d96b09…' and h.subject ilike '%collection%' group by 1,2 order by 2 desc limit 30` β€” I did not run it, so the series premise is unverified.
  • #2907 none unclosable remains. Correction to carry forward: a continuation tool must NOT be built on `MessageRepository`/`base_repository.find_by_ids` β€” those carry no tenant/tier gate; the gated fetch-by-id path is `backend/app/services/ask/document_fetch_resolver.py`.
  • #2908 Whether a client-side export exists outside the ask service β€” I searched backend only for tool surface; a client-only export is ruled out by the docstring at document_fetch_resolver.py:28 but not by an enumerated client census. Closing command: `git grep -inE "Sharing.shareAsync|FileSystem.writeAsStringAsync|toCSV|downloadAnswer" origin/main -- client/`.
  • #2909 Nothing engineering can close; the blocker is a product scoping decision the owner owns. Mechanical re-check when #2902 closes: `gh issue view 2902 --json state,closedAt`.
  • #3180 The issue's exact 85-file figure (files naming TWO OR MORE literals) was not re-derived; my grep counts >=1 literal. Closes with: `for d in backend/app client platform/src; do git grep -l -E "(image|video|audio|pdf|docx|xlsx|pptx|vcf)" origin/main -- $d | while read f; do [ $(git show origin/main:$f | grep -oE "'(image|video|audio|pdf|docx|xlsx|pptx|vcf)'" | sort -u | wc -l) -ge 2 ] && echo $f; done | wc -l; done`
  • #3185 Whether the remaining 2,817 nulls are backfill-eligible or are documents the extractor legitimately cannot fingerprint. Closes with: `poetry run python backend/scripts/tools/backfill_document_dedup_text.py --tenant e9d96b09… ` dry-run (it now prints the VALUE-null count) run on the oneoff machine, compared against the 2,817 figure above.
  • #3214 Whether the client/platform lanes carry a fourth expression of the same fallback. Closes with: `git grep -nE "searchable_text\s*\|\|\s*.*display_name" origin/main -- client platform/src`.
  • #3314 Whether the owner wants the 3 rows stamped or the residue scoped out β€” that is a production write and owner-gated. Closing command once decided: a targeted `update_raw_metadata` by message id, verifiable with `select id, raw_metadata ? 'recovered_inlined_rfc822_via' from public.messages where id in ('228c4515-97d1-4c32-812b-5b5c23061846','79ccf801-bbd3-438c-ae84-f176f546534f','bae749ce-39ed-4d60-aa1d-59ed2f925821')`.
  • #3585 WHICH writer advances content_changed_at without re-embedding is not identified β€” the issue's named suspect (the #3323 document-identity reconciler) cannot explain rows created 2026-08-27..29. Close with: `codegraph_explore "content_changed_at write sites nodes_v2 document embedding_updated_at"` and `git grep -n "content_changed_at" origin/main -- backend/app supabase/migrations`. Also unclosed: whether any scheduled job re-embeds at all (the cron census shows no embedding-refresh job among the 33 pg_cron rows).
Not yet assessed β€” filed after the audit
  • #3681 [ama-trust] nodes_v2 has three writer clocks and no content clock β€” add…
  • #3930 [ama-trust:spreadsheets] ADR-154 stages β€” spreadsheet structure: capture, clean, emit,…
  • #3967 [ama-trust:spreadsheets] Inverted day/month dates in daily report sheets β€” label the…
  • #3968 [ama-trust] Adopt extractor_version_prefix() in the nine extractors still stamping a…
  • #3982 [ama-trust] AMA deflection root cause: a cancelled round rewinds the checkpoint but not…
  • #3983 [ama-trust] viewer display_name is interpolated verbatim into the AMA system prompt and…
  • #3986 [ama-perf] Fix the node pool's row under-estimate so the planner picks HNSW on its own,…
  • #3996 [ama-trust:spreadsheets] AMA refuses to aggregate a column that IS clean numerics —…
  • #4009 [ama-trust] ANN retrieval silently under-returns when the filter is not in the index…
  • #4020 [ama-trust] the MESSAGE lane has the same ANN truncation shape and is unprotected (#4009…
  • #4032 [ama-perf] find_attachment is half the AMA turn budget β€” 29s of 60s, ~14s/call (Regina)
  • #4050 [ama-trust] now_feed regressed ~4.5x on 2026-09-03 β€” 0 of 35 calls hit 50s before, 9 of…
  • #4051 [ama-trust] deadline_cut flags 5 of 120 turns that blew the 60s deadline β€” 65 overran,…
  • #4055 [ama-trust] traverse_graph:format is 89% of the tool β€” 12.7s to render 8,695 chars from…
  • #4057 [ama-trust] find_by_ids chunks Supabase REST at 50 ids β€” a latent N+1 under every person…
  • #4059 aggregate_spreadsheet: a filter cell that fails to normalise drops its row silently β€” no…
  • #4153 [ama-trust] dated_report_9_12 oracle miss: the Sept-12 ask resolves the Sept-10 report…
  • #4476 [ask] How much of a source's text a surface shows is decided in three layers with five…
  • #5167 [L56-g1-w3,L56-g3-w2] Ask showed raw tool-call markup as the answer, and the eval harness…
  • #5170 Ask answered 'August production' from spreadsheets and never reached the email with the…
  • #6205 Stored spreadsheet grids: pad ragged rows or stay ragged (ADR-154 contract) β€” split from…

19 assessed issues closed since the audit and are not shown β€” a closed issue's evidence is history, and printing it as current is how a page starts lying.

Standing rulings

History & forensics

The 2026-08-29 and 09-01 reconciliations β€” the last three spreadsheet-cluster items verified and closed. Three issues that a landed PR had only partly addressed were re-censused against a live caller and closed once the wiring was confirmed, not just the code: #3320 (typed column metadata) β€” Deliverable A (the unconditional schema half, PR #3369) was already fully live: TableBlock.column_types carries the declared per-column type derived from the extractor's stored cell kinds, not text guessing; Deliverable B (materialized columnar projection) stays out of scope, deliberately never built. #3322 (LLM tail detection) β€” raw_sheet_fingerprint/is_cache_eligible confirmed imported by a live caller (spreadsheet_schema.py:44), closing the state:not-in-effect label from PR #3536's traffic gate. #3450 (cross-snapshot stitching) β€” snapshot_collision_notice() confirmed imported and called inside find_attachment.py's _render_result, a wired production caller. Separately, #3182 (shown-vs-matched under-reporting) closed via commit d6cc95b21 on the #3171 line β€” all four flagged sites now compare a pre-filter count against the substrate's real read bound. #2881 (rows shown vs rows matched) was briefly reopened when an adversarial review caught an over-claimed closure β€” 12 of 13 flagged tools route through the _result_header.py choke-point, but get_person_card.py does not and still shows six per-category caps with no matched-vs-shown signal β€” then re-closed once that was the acknowledged remaining scope rather than new work.

The 2026-08-19β†’24 batch β€” fleet waves 2-3, nine PRs merged, two issues closed in-window. In merge order: the per-group ratio operation (PR #3467, closing #3460 β€” the issue's close landed 2026-08-22, days after the merge) Β· the LLM tail-detection lane (PR #3536, #3322 β€” fingerprint + bounded fallback + cache; two panel rounds, 2 majors fixed in round 1, clean round 2) Β· the truncation notice (PR #3539, #3090 β€” all four lenses NO FINDING, single round) Β· the choke-point lesson fold-in (PR #3540, #3183 β€” verified both rules already widened; docs-only) Β· the prefix-collision guard (PR #3541, #3186 item 2) Β· the turn-time instrumentation (PR #3552, no issue) Β· the backfill provenance stamping (PR #3545, #3314 β€” its CI re-run survived three unrelated infra-red runs before merging) Β· number formatting (PR #3563, closing #2903 β€” 15 call sites collapsed, one real 4-lens panel after the lane self-disclosed it had skipped RULE ONE) Β· series recognition (PR #3564, #2902 β€” union-resolved a rebase against #3563; the retrieval-breadth residue stays open on the issue). Two process notes worth keeping: #3547's statement-timeout lift landed under this track's prefix but belongs to the REGINA-DEDUP repair (see that page), and two dedup-judge quality issues (#3551, #3553) are cross-listed onto REGINA-DEDUP where the judge is being consumed. Detail on two of that batch's closures: #2903 (number formatting) collapsed 15 bracketed-title call sites plus the ratio-computation site into one number_format.py module (signed_variance/format_variance_clause, so the sign and the word can never disagree, plus wrap_bidi_isolate) and reproduces the owner's screenshot figures exactly in unit tests. #3183 (evadable choke-point AST rules) closed on investigation rather than a fix β€” both named rules were already fully widened on main from earlier waves, so the only remaining item was folding the lesson into .ai/choke-points.md Part 5.

(the earlier) 2026-08-19 batch β€” Fleet 1 wave 1, four PRs merged, three rulings closed. In merge order: PR #3454 (human-safe, bounded spreadsheet grids for the entity surfaces, closing #3381 + #3382) Β· PR #3455 (deterministic spreadsheet aggregation β€” resolves the dual conflicting-prompt-instruction root cause of #3448) Β· PR #3458 (disclose the caps tools actually hit, closing #2881's get_person_card and #2878's now_feed sites) Β· PR #3457 (refuse INVISIBLE unconditionally on the renderable media path, closing #3096). Rulings the same day: #3309 (ADR-144 stays proposed, closed-lane-only authorized), #3185 (convert the SQL lane via a producer-written column, one-way), #2899 (cut, superseded by #3321/#3385), #2907 + #2909 (parked). Two new rows split out of #3448's repro: #3450 (cross-snapshot stitching) and #3460 (missing ratio operation). Separately measured: 7 days of prod tool-execution timing validated the owner's tool-latency Γ— round-count suspicion, folded into the tabled tooling-PoC ruling above. Detail on three of that wave's closures: #3448 (aggregation tools deployed but never invoked) found the same conflicting routing instruction duplicated in two prompt locations β€” the system-prompt bullets and attachment_block.py's routing block β€” so the model never took the second hop into describe_spreadsheet/aggregate_spreadsheet after find_attachment returned rendered rows; prod-validated post-fix at 50.6s, correct. #3096 (renderable-attachment leak on disclosure rollback) now refuses INVISIBLE unconditionally on the renderable media path. #2878 (now_feed four-times-silent cut) now cuts once at the render point with a single disclosed marker. #3313 (message keeps only its first attachment) is fixed going forward β€” message_attachments now stores every attachment per connector β€” but Regina's own already-ingested invoice PDF stays unstored until the maildir recovery (issue #3327, still open, see Open Work) actually runs.

The 2026-08-16/17 batch β€” five of the six spreadsheet items moved. In order of merge: the entity-projection render fix (PR #3363, server half of #3193, whole-grid not bounded) Β· the ground-truth corpus and scoring harness (PR #3364, #3318) Β· heuristic detection plus a structural fingerprint (PR #3365, #3319) Β· per-cell types served and declared per-column types (PR #3369, Deliverable A of #3320) Β· the type-free detection fallback and six scored fixes (PR #3370, taking header recall 0.292 β†’ 0.875) Β· the tier/tenant-scoped fetch-by-id resolver (PR #3380, the disclosure prerequisite for #3321). Reconciled 2026-08-17: #3193, #3318 and #3319 were closed against their own acceptance criteria; #3320 and #3321 stay open because each PR shipped a named half and said which (Deliverable A without B; the resolver prerequisite without the builder). The two things the batch created rather than closed β€” the client-side raw-markdown render and the list_by_type payload cliff β€” sat recorded in a panel writeup and unfiled until 2026-08-17, and are now #3381 and #3382. πŸ”‘ Worth naming as a pattern: a panel finding recorded in a merged PR body is invisible to every queue in this repo. Filing is what makes it work.

Merged in the cycle before β€” all verified as ancestors of main: the plan (PR #3145, 2026-08-07) Β· the choke-point pass (PR #3171, 2026-08-08, six built / four registered open) Β· the search-surface split (PR #3287, 2026-08-14, closing #3255) Β· the tsvector byte budget (PR #3303, 2026-08-14, closing #3288). Closed since the last pass: #3127, #3255 (state:not-in-effect), #3288, #3184, #3181 (ruled), and #3317 (the float binary-repr defect, completed 2026-08-15 β€” this page last listed it "Live"). Three earlier fixes also landed this cycle: reading the attachment for data questions, sizing the attachment budget, and the declined-salvage citability fix β€” whose metered-refund half was built, reviewed and deleted, because Regina's enterprise deployment never reaches the meter.

The census that founded the track (2026-08-07). 74 spreadsheets truncated, 51 of 74 lose real content, and 50,244,172 characters dropped β€” a lower bound, not exact (it subtracts two counts that are not the same unit). Three clippers: the per-sheet row cap (1,000 rows, 36 docs), the workbook character budget (250,000, 21 docs), and the budget dying inside the last sheet (20 docs, silent β€” the worst class, cut mid-row with no marker and reporting full coverage). The markers cannot be trusted as a census in either direction: the row cap genuinely binds 60 docs but only 36 say so, and the character budget binds 52 but only 21 do; the silent twenty sit between 250,448 and 250,822 characters, the budget expiring on the last sheet so the loop never reaches the marker-writing check the code comment at xlsx_extractor.py:324-330 predicts. Uncapped these render 127.4M characters against 12.4M stored, of which 64.9M (50.9%) is padding from sheets padded to Excel's 1,048,576-row ceiling. The denominator was also wrong β€” 2,198 spreadsheets, not 6,039 document kinds.

Three fixes, all measured, all dead. Raise the budget: 132,660 substantive rows carrying 33.1M characters sit beyond row 1,000, so no budget reaches them β€” it plateaus at 45 of 74. Filter the filler rows: 0.7% of the budget actually spent, zero documents completed (an earlier "3 docs" was refuted). Paginate past the cliff: reading is O(rows) and the answer is O(1) β€” you cannot read your way to an aggregate. One document alone holds 2,337,469 substantive characters against a 250,000 budget.

The design. Lossless capture: one row per worksheet in a side table, no embedding, no tsvector, not new nodes β€” both caps, the truncated suffix and every marker deleted. The search profile (sheet names, headers, labels, row counts) carries further than intended: searchable_text has four downstream consumers β€” the full-text generated column, the node embedding, the sensitivity tier classifier, and the fallback arm of two content chains. The split (PR #3287): searchable_text = profile + summary + roster, the embedded surface = profile + summary. The "length dilutes the embedding" theory is refuted β€” the variable is TOPICALITY. The profile is written at extraction β€” sheet names, column headers, label values, row counts β€” and ranking should improve, because 250k of numeric cells today dilute term weighting and the code already calls the current value "an unparseable digit run". No migration and no materializer change β€” both already land in node properties. A summary stage ships with it, because a spreadsheet's mechanical title was "Spreadsheet: Sheet2" β€” which is why images, whose vision model writes them one, outranked spreadsheets on every topical question.

Re-extraction is lazy, not a sweep. The bump ships with the code (PR #3287, xlsx-extractor v2 β†’ v3); a workbook re-extracts on the next enrichment pass of its own source, so a dormant tenant's sheets stay at v2. Re-measured 2026-08-15: 1,940 of 2,672 nodes at v3, 572 at v2, 60 at v1, one hundred unversioned; 2,507 documents hold lossless grids (6,373 sheets, 28 MB). The bump also turned a latent defect live: 2,612 of 2,672 nodes now carry an empty body_text β€” the inline-render row above. Median indexed characters recovered from the v2 trough of 313 to 1,707 (v1: 2,278). The embedding half needs no re-extraction: v2 nodes already carry a profile, so the read-time composer took the profile branch the moment this deployed.

Table detection. Header found on 94.7% of sheets, but only 39.6% on row 0; label column first 73.7%; both 53.5% β€” detect, don't assume. Prior art is not shippable: TableSense publishes annotations only (no model, no weights), ExcelTableCNN self-reports 40.3% recall. The "~71%" coverage figure is plausibility, not accuracy. Typing is nearly free once boundaries are known β€” 0.998 median column-type purity, 76% median of columns numeric β€” so labels really are the small minority worth indexing.

Storage reality. Node table 2,317 MB: embedding 469 MB, HNSW indexes 1,085 MB, trigram 57 MB, full-text 39 MB, all searchable text 26 MB. A vector is 6 KB whether the node holds 500 characters or 250,000, so node count is the expensive axis β€” why sheets get a side table, not nodes. Regina's rows sit in two indexes, ~21.9 KB per node marginal.

The tsvector budget (PR #3303). Ceiling 1,048,575 bytes; to_tsvector raises rather than truncating. A character budget never overflowed β€” it peaked at 89.0% (CJK 2-char tokens) β€” so the case for UTF-8 bytes is the margin (89.0% β†’ 45.8%), not a demonstrated overflow. Cost re-measured in bytes: 26 rows exceed 200,000 bytes (2 pdf, 24 xlsx), one more than by characters. The largest production PDF carries 959,521 characters and builds only 81,068 bytes (0.0845 per character) because prose repeats, while the worst real ratio is 2.1193. Truncation is not free even for surviving terms: ts_rank_cd cover density and an absolute-score filter both move.

Choke-points and the second census. Eleven questions named; six have a choke-point. The gates: two rulings — the document-body expressions (#3185, open) and the turn ceiling (#3181, ruled unbounded); the citation regex made unbounded (#3184); the prefix→media_type table deleted rather than guarded; the budget rule made name-blind and given its first test file. The pass itself refuted six claims (citability was decided from per-turn tool identity but gated on thread-scoped text; the per-turn character figure was 120,000, not 84,000). A second census found the fallback-order expression across five sites — document_scope_classifier, stage_inputs._cached_node_text, node_document_repository, entity_projection_service and sensitivity_classification_phase — each choosing among a node's content fields; two return nothing for a spreadsheet, and one of those gates conversation analysis — latent until the bump ran, live the moment it did. The one-looking-defect that is not: two chains are exact reverses, and the property whitelists are disjoint so the order cannot change the answer.

What this page does not claim. Post-merge production telemetry has not been measured. Corpus-wide re-extraction cost is estimated from 74 documents, not the 2,306 on production. The structural census covers Regina only. The template-signature count keys on sheet names (collisions unchecked). The search index is configured for English on Arabic content β€” pre-existing, out of scope. Corrected 2026-08-17: this section used to say the one-attachment-per-message limit had never been sized. It has β€” 57% of Regina's document-carrying maildir messages lost at least one file, β‰₯71,440 documents recoverable β€” and the model fix has since shipped forward-only. What is still unclaimed is any measurement of the 2026-08-16/17 batch on production: the detection scores come from a 22-sheet labelled sample, and no post-merge telemetry has been read for the render fix or the typed read path.