The track exists because AMA's failure mode is confident wrongness β an answer that looks complete, reads fluently, and is quietly missing the thing that was asked for. Scope is answer correctness only; permissions moved to AMA-DISCLOSE on 2026-08-01.
π’ The server half shipped in PR #3363 (merged 2026-08-16). The entity projection now routes through load_document_grids + document_text_for_serving β the same choke-point the three AMA serving tools use β so a spreadsheet's display body is its rendered grid, not its findability profile. The owner-facing decision was whole grid, not bounded: the entity screen is a document viewer, not a model prompt, and a silent character budget there would reintroduce the truncation class this track deleted.
π Two consequences its own panel recorded and never filed. They were filed on 2026-08-17 and this issue closed behind them β filing first, so closing lost nothing:
The original defect, kept for the reader: entity_projection_service.py:263 fell back body_text or searchable_text, so the screen rendered the profile as the body. On a real workbook: all 8 column headers and 6 row labels present, every figure absent β including a 2,544,000,000 five-year total. Worse than an empty body: what survived was the scaffolding and what was stripped was the content. Measured population: 2,612 of 2,672 xlsx nodes (97.75%) carry an empty body_text, of which 2,512 (94.0%) rendered the defect.
π’ Shipped in PR #3364 (merged 2026-08-16) β a labelled table-boundary corpus and the score_table_detection.py harness, which immediately became the instrument every detection claim below is now stated in. It did cover a second tenant, as the issue required.
π The sample is 22 sheets / 24 labelled blocks (Regina 20/22, aeldaly 2/2) against the 6,373 stored grids, so it is a scoring instrument rather than a corpus-representative measurement β and 10 of those 22 sheets carry no type signal at all. Every precision/recall figure on this page inherits that denominator. The issue closed on the harness, which is what it asked for β growing the corpus is worth a fresh issue if a figure ever needs to carry corpus-wide weight, and is not an unmet acceptance item here.
Why it was filed: every coverage figure sizing this track (the ~71% header heuristic) measured whether a plausible header row exists, not whether the right one was found.
π’ Built in PR #3365 and repaired in PR #3370, both merged 2026-08-17. Heuristic boundary detection over a stored sheet grid plus a structural fingerprint. Scored against the #3318 labels with the canonical harness:
Baseline β after the fixes (precision / recall): header row 0.583 / 0.292 β 0.677 / 0.875 (F1 0.764) Β· data start and data end 0.583 / 0.292 β 0.710 / 0.917 Β· column extent 0.500 / 0.250 β 0.710 / 0.917.
The first score exposed that the dominant cause was data availability, not the heuristic β 10 of 22 sheets carried no type signal (pre-v4 rows, style_ids/cell_styles empty) and the detector returned no blocks at all when typed=False. PR #3370 added a type-free fallback plus six algorithmic fixes, and the fallback only became reachable from served data because the typing work (#3320) wired the read path.
What remains is judgement, not code: whether that recall is enough to build the aggregation tool on, measured on a 22-sheet sample.
π’ Deliverable A β the unconditional schema half β shipped in PR #3369 (merged 2026-08-17). The serving read returns the type signal (grids_for_nodes selects cell_styles/style_ids through a second lateral join cut in parallel with the budgeted grid), and the per-column type is now declared, not guessed: unanimous β that type, no populated cell β EMPTY, mixed population β a new MIXED, never a majority vote. Derived from the stored per-cell type, never re-derived from rendered text. No migration β the per-cell type was already persisted and merely write-only on the read.
π Deliverable B β the materialized typed columnar projection β is explicitly NOT built and stays gated on its measured trigger: up to 6.74M narrow rows against a 28 MB source, roughly 30Γ the storage it projects, plus a staleness surface. That gate is why this row stays open on a shipped PR.
Two boundaries the PR stated rather than buried: a leading non-numeric record ("N/A") is indistinguishable from a units row to the detector and is skipped the same way, so such a column may still declare NUMBER β a limitation of the heuristic detector (#3319), not of this reduction. And pre-v4 rows carry an empty style list until the version bump re-extracts them, so their columns read untyped.
The head of the queue as of 2026-08-17. The model emits a constrained tenant- and tier-scoped query; Postgres executes it, which makes the dropped addend structurally impossible. Both things it was blocked on have landed: detection is built and scored (#3319), and the declared schema half shipped (#3320). It was never blocked on materialization.
π’ Its disclosure prerequisite also shipped β PR #3380 (merged 2026-08-17), the tier/tenant-scoped fetch-by-id resolver. The inherited find_by_id is service-role with no tier ceiling and no disclosure evaluation, so any model-facing surface built on it would read straight past the sensitivity tier on confidential content. resolve_document_by_id forces as_user=True, derives the ceiling from the viewer rather than the caller, and evaluates disclosure FULL-only so the model never reads a de-identified stand-in. A check.py AST hook rejects the service-role spelling inside the serving surface, with both polarities fixture-proven.
π That resolver is the PROMPT choke-point, FULL-only β stated, not assumed. Export (#2908) is a render surface (degrade, don't drop) and needs a separate render exit, deliberately unbuilt.
Sizing, re-measured 2026-08-15: only 12 sheets exceed 10,000 rows (0.19%), largest 826 ms cold.
Filed 2026-08-17 from the #3193 panel. load_document_grids renders model-oriented markdown pipe tables; the entity detail screen and the AMA node-detail modal render description as plain text with no markdown renderer (only chat-answer.tsx has one). So the figures now arrive β as literal | Year | Revenue |.
For Arabic sheets the bidi-neutral | reverses the columns, which is the exact defect three existing pieces of code were written to avoid: bodyExcerptOf, the 20260801204154 migration, and _xlsx_aggregation_text, whose comment reads "never |". π This is the track's own failure mode one layer out: the answer is no longer missing, it is unreadable β and on this corpus, reversed.
Filed 2026-08-17 from the #3193 panel, where three lenses raised it independently. The whole-grid decision was right for the entity screen β a viewer, not a prompt β but the same budget applies to the shared list_by_type projection serving up to 500 nodes, so a browse TOAST-decompresses and ships every spreadsheet's full grid behind a numberOfLines={3} card.
The fix is a per-surface budget: the list is a preview, the detail screen is a viewer. It does not reopen the whole-vs-bounded decision for the viewer. Not yet measured: the real payload size of a browse against Regina's corpus, which should set the preview budget.
π’ The cause is fixed going forward β message_attachments shipped 2026-08-17 (PR #3378, ADR-145 D1/D2, on ATTACHMENT-TRUTH as #3324), and all three connectors now emit every document attachment instead of returning on the first success.
π Regina's invoice is still unstored. The fix wrote no rows for messages already ingested, so message 228c4515 still holds one attachment and the PDF a human would actually look for is still missing. Repairing it is the maildir recovery (#3327), which is blocked. This row is the named acceptance case for that backfill rather than independent work β its own acceptance is that the invoice is retrievable by its content, not merely structurally present.
The limit is no longer unsized: 57% of Regina's document-carrying maildir messages lost at least one file, and β₯71,440 dropped documents are recoverable.
Ranked by what a user would notice first. The spreadsheet cluster sorts first because Regina asks a question about her own spreadsheets and gets nothing back β the track's own impact statement. Re-read 2026-08-17: that ordering held and the cluster is most of the way down. The render defect (#3193) is fixed and closed, and what a user would notice first is now the client half of that same fix β raw | pipes on the entity screen, reversed columns in Arabic (#3381). The aggregation tool (#3321) is the one rung with nothing left under it, which is why it takes the Next call. Answer-correctness degradations come second (the things that make any answer wrong today), then structural/choke-point work, then decisions and directions last.
What it ignores: priority labels (the live render defect is p2 and sits first), filing order and age, cost or ease, and the needs:ruling flag as a reordering lever β a ruling that blocks work sorts with its work, not above it.
The 2026-08-16/17 batch β five of the six spreadsheet items moved. In order of merge: the entity-projection render fix (PR #3363, server half of #3193, whole-grid not bounded) Β· the ground-truth corpus and scoring harness (PR #3364, #3318) Β· heuristic detection plus a structural fingerprint (PR #3365, #3319) Β· per-cell types served and declared per-column types (PR #3369, Deliverable A of #3320) Β· the type-free detection fallback and six scored fixes (PR #3370, taking header recall 0.292 β 0.875) Β· the tier/tenant-scoped fetch-by-id resolver (PR #3380, the disclosure prerequisite for #3321). Reconciled 2026-08-17: #3193, #3318 and #3319 were closed against their own acceptance criteria; #3320 and #3321 stay open because each PR shipped a named half and said which (Deliverable A without B; the resolver prerequisite without the builder). The two things the batch created rather than closed β the client-side raw-markdown render and the list_by_type payload cliff β sat recorded in a panel writeup and unfiled until 2026-08-17, and are now #3381 and #3382. π Worth naming as a pattern: a panel finding recorded in a merged PR body is invisible to every queue in this repo. Filing is what makes it work.
Merged in the cycle before β all verified as ancestors of main: the plan (PR #3145, 2026-08-07) Β· the choke-point pass (PR #3171, 2026-08-08, six built / four registered open) Β· the search-surface split (PR #3287, 2026-08-14, closing #3255) Β· the tsvector byte budget (PR #3303, 2026-08-14, closing #3288). Closed since the last pass: #3127, #3255 (state:not-in-effect), #3288, #3184, #3181 (ruled), and #3317 (the float binary-repr defect, completed 2026-08-15 β this page last listed it "Live"). Three earlier fixes also landed this cycle: reading the attachment for data questions, sizing the attachment budget, and the declined-salvage citability fix β whose metered-refund half was built, reviewed and deleted, because Regina's enterprise deployment never reaches the meter.
The census that founded the track (2026-08-07). 74 spreadsheets truncated, 51 of 74 lose real content, and 50,244,172 characters dropped β a lower bound, not exact (it subtracts two counts that are not the same unit). Three clippers: the per-sheet row cap (1,000 rows, 36 docs), the workbook character budget (250,000, 21 docs), and the budget dying inside the last sheet (20 docs, silent β the worst class, cut mid-row with no marker and reporting full coverage). The markers cannot be trusted as a census in either direction: the row cap genuinely binds 60 docs but only 36 say so, and the character budget binds 52 but only 21 do; the silent twenty sit between 250,448 and 250,822 characters, the budget expiring on the last sheet so the loop never reaches the marker-writing check the code comment at xlsx_extractor.py:324-330 predicts. Uncapped these render 127.4M characters against 12.4M stored, of which 64.9M (50.9%) is padding from sheets padded to Excel's 1,048,576-row ceiling. The denominator was also wrong β 2,198 spreadsheets, not 6,039 document kinds.
Three fixes, all measured, all dead. Raise the budget: 132,660 substantive rows carrying 33.1M characters sit beyond row 1,000, so no budget reaches them β it plateaus at 45 of 74. Filter the filler rows: 0.7% of the budget actually spent, zero documents completed (an earlier "3 docs" was refuted). Paginate past the cliff: reading is O(rows) and the answer is O(1) β you cannot read your way to an aggregate. One document alone holds 2,337,469 substantive characters against a 250,000 budget.
The design. Lossless capture: one row per worksheet in a side table, no embedding, no tsvector, not new nodes β both caps, the truncated suffix and every marker deleted. The search profile (sheet names, headers, labels, row counts) carries further than intended: searchable_text has four downstream consumers β the full-text generated column, the node embedding, the sensitivity tier classifier, and the fallback arm of two content chains. The split (PR #3287): searchable_text = profile + summary + roster, the embedded surface = profile + summary. The "length dilutes the embedding" theory is refuted β the variable is TOPICALITY. The profile is written at extraction β sheet names, column headers, label values, row counts β and ranking should improve, because 250k of numeric cells today dilute term weighting and the code already calls the current value "an unparseable digit run". No migration and no materializer change β both already land in node properties. A summary stage ships with it, because a spreadsheet's mechanical title was "Spreadsheet: Sheet2" β which is why images, whose vision model writes them one, outranked spreadsheets on every topical question.
Re-extraction is lazy, not a sweep. The bump ships with the code (PR #3287, xlsx-extractor v2 β v3); a workbook re-extracts on the next enrichment pass of its own source, so a dormant tenant's sheets stay at v2. Re-measured 2026-08-15: 1,940 of 2,672 nodes at v3, 572 at v2, 60 at v1, one hundred unversioned; 2,507 documents hold lossless grids (6,373 sheets, 28 MB). The bump also turned a latent defect live: 2,612 of 2,672 nodes now carry an empty body_text β the inline-render row above. Median indexed characters recovered from the v2 trough of 313 to 1,707 (v1: 2,278). The embedding half needs no re-extraction: v2 nodes already carry a profile, so the read-time composer took the profile branch the moment this deployed.
Table detection. Header found on 94.7% of sheets, but only 39.6% on row 0; label column first 73.7%; both 53.5% β detect, don't assume. Prior art is not shippable: TableSense publishes annotations only (no model, no weights), ExcelTableCNN self-reports 40.3% recall. The "~71%" coverage figure is plausibility, not accuracy. Typing is nearly free once boundaries are known β 0.998 median column-type purity, 76% median of columns numeric β so labels really are the small minority worth indexing.
Storage reality. Node table 2,317 MB: embedding 469 MB, HNSW indexes 1,085 MB, trigram 57 MB, full-text 39 MB, all searchable text 26 MB. A vector is 6 KB whether the node holds 500 characters or 250,000, so node count is the expensive axis β why sheets get a side table, not nodes. Regina's rows sit in two indexes, ~21.9 KB per node marginal.
The tsvector budget (PR #3303). Ceiling 1,048,575 bytes; to_tsvector raises rather than truncating. A character budget never overflowed β it peaked at 89.0% (CJK 2-char tokens) β so the case for UTF-8 bytes is the margin (89.0% β 45.8%), not a demonstrated overflow. Cost re-measured in bytes: 26 rows exceed 200,000 bytes (2 pdf, 24 xlsx), one more than by characters. The largest production PDF carries 959,521 characters and builds only 81,068 bytes (0.0845 per character) because prose repeats, while the worst real ratio is 2.1193. Truncation is not free even for surviving terms: ts_rank_cd cover density and an absolute-score filter both move.
Choke-points and the second census. Eleven questions named; six have a choke-point. The gates: two rulings β the document-body expressions (#3185, open) and the turn ceiling (#3181, ruled unbounded); the citation regex made unbounded (#3184); the prefixβmedia_type table deleted rather than guarded; the budget rule made name-blind and given its first test file. The pass itself refuted six claims (citability was decided from per-turn tool identity but gated on thread-scoped text; the per-turn character figure was 120,000, not 84,000). A second census found the fallback-order expression across five sites β document_scope_classifier, stage_inputs._cached_node_text, node_document_repository, entity_projection_service and sensitivity_classification_phase β each choosing among a node's content fields; two return nothing for a spreadsheet, and one of those gates conversation analysis β latent until the bump ran, live the moment it did. The one-looking-defect that is not: two chains are exact reverses, and the property whitelists are disjoint so the order cannot change the answer.
What this page does not claim. Post-merge production telemetry has not been measured. Corpus-wide re-extraction cost is estimated from 74 documents, not the 2,306 on production. The structural census covers Regina only. The template-signature count keys on sheet names (collisions unchecked). The search index is configured for English on Arabic content β pre-existing, out of scope. Corrected 2026-08-17: this section used to say the one-attachment-per-message limit had never been sized. It has β 57% of Regina's document-carrying maildir messages lost at least one file, β₯71,440 documents recoverable β and the model fix has since shipped forward-only. What is still unclaimed is any measurement of the 2026-08-16/17 batch on production: the detection scores come from a 22-sheet labelled sample, and no post-merge telemetry has been read for the render fix or the typed read path.