Give the benchmark SUT panel an extractor, and fit theta on non-nested spans (#700, #699)

[WesIngwersen]

Stacked on #712 (step3_underlying_ii_va). Two items off
intermediate_estimation_plan.md §What to build: S0a (#700) and S2 (#699).

S0a — the benchmark detail SUT panel gets an extractor (#700)

BEA publishes 2007, 2012 and 2017 detail Supply and Use SUT as one zip of
per-table workbooks with a sheet per year, all three already on the 2017 code
basis in one frame. It was a local drop that only the Step 3 drift diagnostic
read, through an ad-hoc zip reader.

io_2017 now carries _load_benchmark_detail_supply_use_usa(matrix, year),
GCS-backed like every other table, with load_benchmark_detail_U_intermediate_usa
and load_benchmark_detail_supply_usa as the typed 402 × 402 accessors.

• ✅ The panel's 2017 sheets are the single-year workbooks cell for cell, on
  both matrices — 0 differing cells on 413 × 424 and 405 × 415.
  assert_benchmark_panel_matches_2017() is the check.
• ⚠️ The two loaders are kept separate anyway: _load_2017_detail_supply_use_usa
  is what bea_parse emits as the BEA_Detail_Use_SUT / BEA_Detail_Supply
  FBAs, and those stay pinned to their published workbook. The FBAs are not
  extended to 2007 and 2012 — the year is hardcoded on bea_parse's detail
  branches and the yaml declares years: [2017].
• ⚠️ BEA's subsidy sign convention holds on all three years, so
  _assert_bea_subsidy_signs runs on every year the panel loader returns.
• SUPPLY-USE_2026-08-24.zip has been uploaded to
  extract/input-data/USA_AllTablesSUP/ and the download verified
  byte-identical (sha256 ea81c120…).
• --holdout reproduces unchanged, 2012→2017 best θ still 1.00.

S2 — fit θ, add the margin-rate factor (#699)

The experiment the plan asked for, and the answer is not the one it expected.

Every θ on the plan's table starts at 2017, so elapsed years, cumulative
inflation, price dispersion and accumulated structural drift all move together
with the calendar and none can be told from the others. The summary Use SUT
publishes 1997-2024 and the price index reaches 2012, so 78 non-nested spans
are free — different bases, different lengths, different inflation. New
--regime flag:

predictor of the fitted θ	R²
crosses the 2021-22 surge	0.613
cumulative price level	0.525
elapsed years	0.142
relative-price dispersion	0.014

• ❌ Dispersion is dead — the candidate §Inflation named. 1.4% of the
  variance, coefficient the wrong sign.
• ❌ So is elapsed time. Adding it to the regime binary moves that binary's
  coefficient to 0.002 and its R² not at all.
• ✅ The regime reading stands. Holding span length fixed, spans that cross
  the surge fit θ 0.0-0.5 and spans that do not fit 0.7-0.9, at every length
  from one to nine years.

What ships (default_theta): 0.75 off the surge, 0.0 across it — fitted
0.755 and 0.141. Rounding up to zero rather than to the target spans' own
−0.25/−0.50 is deliberate: a negative θ says nominal shares move against their
own price, which is a curve-fit rather than a mechanism, and it buys 0.6%.

⚠️ The decision matters much less than the diagnosis. Median gain of the
best θ over a frozen A is 5.44% of the score off the surge and 0.59%
across it — and every year this build targets from 2022 on crosses it. What
#497's θ = 1 cost was not a missed 0.6%, it was the 12.6% it gave away by
pointing the wrong way. DEFAULT_THETA is gone; THETA_497 keeps the name for
what #497 specified so the two stay scorable against each other.

The margin leg, built at detail

The deflator is now the purchaser one, price × margin rate. The non-obvious part
is reaching 402 rows annually: BEA publishes detail Supply only for benchmark
years, so the rate's level is detail-observed at 2017 and only its
movement is borrowed from the summary parent.

✅ S0a made that rule testable. Against the observed 2012 detail factor,
weighted by 2017 intermediate dollars:

rule	weighted MAE
detail level × parent movement (shipped)	0.756pp
the parent's factor, taken down unchanged	1.010pp
no factor at all	1.818pp

⚠️ A year with no published Supply table is refused, not carried.
MARGIN_YEARS (1997-2024, BEA's vintage) is a separate constraint from
INTERMEDIATE_YEARS (gross output). They agree at 2024, so nothing is blocked
today; a 2025 build (#707) would reach a year with one and not the other, and a
silent factor of 1.0 would read as "margins did not move".

⚠️ The margin leg is inert in exactly the years the build targets — at θ = 0
every factor is raised to the zero power. It moves 0.21% of the 2019 block,
0.54% of the 2021 block and 0.000% of the 2024 one. Kept because it is the
correct deflator for a purchaser-valued cell, not because it changes the current
answer.

⚠️ One vintage, not the pinned one. _load_usa_summary_sut picks the
workbook by year, which would take a ratio's numerator and denominator off
different vintages — a median 0.50pp on 2020's rates and 1.20pp on 2022's. They
agree exactly on 2017, so reading the current vintage throughout costs
nothing at the base and removes the seam at the target.

Also: plan-doc sections #712 left stale

PR #712 replaced vapro_seed with an observed control but did not restate the
sections describing it. Fixed here, since they now contradict the code:

• the §S1 level table (2018 was 15,754.5, is 15,848.0) — and a note that the
  block total is θ-independent to 12 decimal places, because every column is
  renormalised before the control is applied, so that table is a check on the
  control and never evidence about the carry;
• the column control table — 2.3% / 2.5→8.0% / GSLG −18.3% were the
  superseded seed; it is now 0.00007% economy-wide;
• the GSLG and T31005 paragraphs — the level problem is fixed, Step 3: can government spending by function explain the G* commodity mix? #578 (the
  commodity mix) is untouched, and T31005 supplies only a column total, which
  is now had for free.

Checks

black, ruff, mypy clean repo-wide (bar the 4 known Windows-only
settings.py resource errors CI does not see). pytest: 795 passed, 0
failed. 2017 reproduction unchanged — $6.02M max absolute, 1.05% max relative,
seven negatives — and use_intermediate_detail_sut still scores 100% on 44,281
populated cells.

Reproduce with intermediate_structure_drift --regime --holdout --theta --seed --control.

🤖 Generated with Claude Code

[WesIngwersen]

Consolidated into #742, which collapsed the #713 → #714 → #715 → #717 → #718 stack into one PR and merged into nowcast on 2026-08-27.

Verified superseded, not just replaced: every module this stack owns is byte-identical on nowcast — services_transport_expense_seed.py, services_transport_expense_resource.py, utilities_expense_seed.py, gov_function_bridge.py, benchmark_holdout.py, agriculture_expense_seed.py, trade_expense_supplement.py, inputs_structure.py, intermediate_structure_drift.py, underlying_industry_coverage.py, About_the_price_carry.md, USDA_ERS_FIWS.py, Census_AIES_Service_Expenses.yaml, test_benchmark_detail_sut.py. Nothing on this branch is unmerged.

Not re-targeting to nowcast: the branch now sits a net -8k to -40k lines behind it and would revert Step 2 (#740), Step 4a and the later Step 3 work.

#743 is the live Step 3 PR. Branch kept rather than deleted.