Paper: add the DC2018 extinction-convention systematic (C29) and its theta_star caveat

[jdeast]

For the paper agent on radish. The full spec is paper_extinction_convention.txt in the notes repo (jdeast/exozippy-notes, commit d146eac) -- read that first; it carries the quote-ready tables. This issue is the task, not the content.

Why

A band extinction is not a well-defined number until the convention is stated. The 2018 data challenge reddened monochromatically at each filter's effective wavelength; EXOZIPPy integrates a reddened spectrum through the passband, which is what a measurement is. For a filter as wide as W149 (0.896-2.073 um) those differ, so no single A_V reproduces both of the challenge's simulated band extinctions in our model. This is not a law error -- our law matches the challenge's own band ratio to 0.1%.

Per JDE, we do not engineer around it: we anchor the colour, budget the residual into the zeropoint prior, and report the leftover disagreement as a caveat beside the truth-recovery discussion.

What the paper needs

1. C29 in the Conventions section, matching src/exozippy/latex/convention.tex (already written there -- lift it). C29 currently exists in only two of the three copies, and convention.tex's header records it as OWED to the paper because the paper repo is not reachable from Hydra. Closing that is the main deliverable.
2. A methods sentence or two on the colour anchor, the correlated clump error, and the widened zeropoint prior (spec section 3).
3. A caveat beside the truth-recovery table (spec section 4). It must not overclaim: widening a prior does not remove a systematic, it stops it masquerading as a measurement. Recovered theta_star (hence theta_E, hence lens mass) carries a convention-induced low bias of up to ~23% on the most extincted events -- a property of comparing an integrated model to a monochromatic simulation, not a statement about real Roman data.
4. Optionally one sentence on the grid ceiling this uncovered (spec section 5): a posterior piling against a grid bound reports "the grid ran out", not "the data prefer the edge", and nothing said so. That is a general limitation of grid-backed priors.

The numbers, if you only read this far

event	A_V prior	grey (mag)	theta_star bias
008	2.26 +/- 0.14	-0.07	0.970x
152	3.70 +/- 0.23	-0.15	0.935x
223	5.46 +/- 0.61	-0.26	0.886x
128	6.44 +/- 0.94	-0.34	0.856x
062	7.26 +/- 0.79	-0.41	0.827x
194	9.01 +/- 1.20	-0.58	0.766x

On event 194 the convention accounts for 56% of the observed theta_star deficit in log space (-0.116 dex of -0.208); the fitted value is 0.62x truth. The rest is ours and is not explained by this.

Do not wait on

The six-event sweep is being re-run with these corrected priors, so the recovered values in any results table will change. The convention discussion above does not depend on that re-run -- write it now; refresh the numbers when the sweep lands.

Where the claims live in code

• src/exozippy/components/mulensing/conventions.md C29 (normative)
• src/exozippy/latex/convention.tex C29 (paper-facing twin)
• examples/DC2018/dc18_sweep_config.py -- AV_COLS, av_from_clump_colour
• notes/code_review_20260824.txt 7.7.3 -- the full measurement trail

[jdeast]

Correction to the table above — read this before writing section 3.

The issue body says the residual is budgeted into the zeropoint prior. That was the first implementation and it is wrong; it was corrected before the sweep launched (EXOZIPPy fe075b9f, notes spec updated in the same pass).

Why the zeropoint is the wrong knob. These configs run filters: [], so the SED carries no photometry of its own and the two mulensing zeropoints are the only colour information in the entire fit. Two independent priors of width 0.58 mag admit √2 × 0.58 = 0.82 mag of colour slack — an order of magnitude more than the colour signal separating plausible source temperatures. It buys an honest error bar on theta_star by throwing teffsed away, slackening precisely the constraint the colour anchor exists to exploit. The prior that would be right — one grey term shared between the two bands — is not expressible in the config format, and building it is the engineering JDE ruled out for a DC2018 problem.

What replaced it. av is the correctly correlated nuisance: moving it reddens both bands together in the ratio the law dictates — mostly grey, with exactly the colour term the physics implies — so it buys freedom along the reddening direction without opening the differential. Half the anchor spread now sits in the av prior width:

event	A_V prior	grey (mag)	theta_star bias	zeropoint
008	2.26 +/- 0.19	-0.07	0.970x	22.0 +/- 0.02
152	3.70 +/- 0.38	-0.15	0.935x	22.0 +/- 0.02
223	5.46 +/- 0.83	-0.26	0.886x	22.0 +/- 0.02
128	6.44 +/- 1.20	-0.34	0.856x	22.0 +/- 0.02
062	7.26 +/- 1.22	-0.41	0.827x	22.0 +/- 0.02
194	9.01 +/- 1.83	-0.58	0.766x	22.0 +/- 0.02

The consequence for the paper is sharper, not softer: the bias is not absorbed, it is reported. Nothing in the fit removes it. The caveat wording should say the av prior's width covers the spread so the bias does not masquerade as a measurement — not that widening priors fixes anything.

dc18_truth_table.py now prints this caveat beside the recovery table, so the numbers above can be read straight off any scoring run rather than transcribed from here.

[jdeast]

Second correction — this one retracts the theta_star bias table in the issue body. Do not write that version.

ab194/av_true, the first run with the colour-anchored prior (av = 9.01 ± 0.3, ptde, Rhat 1.00, ESS 23–25k, 78/78 chains in the good region), measured what the issue body predicted — and it came out differently.

Claimed: theta_star biased low by 10**(-0.2*grey), i.e. 0.766x on event 194.
Measured: 0.983x.

The CSB arithmetic is right about the CSB relation and wrong about where the residual lands. theta_star is protected, because the light curve pins rho * theta_E independently of the SED.

quantity	truth	fit	fit/truth	pull
theta_star (= R/D)	3.829e-4 mas	3.763e-4	0.983	—
R_source	0.822 R☉	0.331	0.403	−5.25
D_source	9980 pc	4090	0.410	−5.35
theta_E	0.3486 mas	0.27	0.774	−0.66

R and D are each ~2.45x low but slide in lockstep, so their ratio survives while each is individually −5 sigma. The over-predicted flux is absorbed by a source made too cool (teffsed 3040 K), with the distance following the luminosity inward.

What to write instead

1. The anchor fix is the headline, and it is attributable: theta_star moved from 0.62x truth (old arms, av ≈ 4.25 written from A_W149) to 0.983x, and R_source's pull from −13.10 to −5.25.
2. The surviving −5 sigma on (R_source, D_source) is ours, and must not be attributed to the convention. That is not established.
3. Drop the per-event theta_star-bias column from section 4, or relabel it as what it is: the bias the CSB relation would carry if theta_star came from dereddened photometry rather than from rho * theta_E.
4. Event 194's source is a dwarf (0.822 R☉ at 9980 pc), not a clump giant — I had that backwards in an earlier note.

The spec in the notes repo now carries this as section 4b and supersedes section 4's target claim; section 4's evidence for which convention the simulation used (the WavelengthEff table) is unaffected and still stands. EXOZIPPy PR #304 fixes the same wrong text where dc18_truth_table.py prints it.

[jdeast]

Third update, and this one is a confirmation rather than a retraction — it is the number the paper should lead with.

Three arms on event 194 differing only in the av prior (ptde, Rhat 1.00, 78/78 chains):

arm	av prior	av posterior	theta*/truth	R/truth
av_clump	8.61 ± 2.41	7.18	0.738	0.392
av_true	9.01 ± 0.30	8.99	0.983	0.403
av_band	11.65 ± 1.40	9.42	1.086	0.425

A log-linear fit crosses theta_star/truth = 1 at av = 9.00. The colour anchor, computed independently from the clump colour excess through the integrated BC grid with no reference to any fit, is 9.01. Agreement to 0.1%.

That is the C29 argument validated by two independent routes — one photometric-reddening, one light-curve-plus-source-physics. It is a much stronger claim than anything in the original spec, and it belongs in the paper ahead of the convention mechanics.

Also settled, and it changes what the caveat should say: R_source is ~0.4× truth at every av (0.392 → 0.425 across 2.24 mag). D_source and theta_star move with av; R_source does not. So the source-radius deficit is not an extinction problem, and the paper must not attribute it to the convention. That residual is ours and unexplained.

Supersedes the theta_star-bias table in the issue body (v1) and the "theta_star is protected" correction in my second comment (v2). Both were wrong; the notes spec's section 4b and EXOZIPPy PR #306 carry the measured version. The evidence in section 2 for which convention the simulation used (the WavelengthEff table) was never affected and still stands.

[jdeast]

Fourth update, and it retracts the "0.1%" in my third comment. The claim survives; the precision does not.

The sweep arm that was still sampling when I wrote the third comment finished overnight: sweep/194 (av = 9.01 +/- 1.83 -- the honest width the sweep runs on purpose -- fitu0te, ptde, Rhat 1.00, 78 chains) landed av = 7.29 (-0.94 sigma) and theta/truth = 0.698*, below the 0.83-0.94x I forecast.

The crossing, refit as runs were added

fit	slope (dex/mag)	crossing
the three arms of comment 3	0.0745	9.01 +/- 0.80
six runs with av > 5	0.0500	9.31 +/- 2.83
all eleven runs with a results.csv	0.0377	9.71 +/- 1.53

The relation is monotone but not log-linear: five runs near av = 4.25 sit at 0.63, where the three-arm line predicts 0.44. A global log-linear fit is therefore the wrong estimator and its crossing drifts with the fit range. The local estimate is the defensible one -- the two runs that straddle theta*/truth = 1 (8.99 -> 9.42) interpolate to 9.06, and three runs at av = 6.8-7.3 scatter 0.037 dex at fixed av, i.e. +/- 0.5 mag.

What to write

1. The anchor is confirmed to ~0.5 mag: crossing 9.1 +/- 0.5 against an independently computed 9.01. That is still a strong claim -- a reddening measurement and a light-curve-plus-source-physics fit agreeing within half a magnitude on a quantity whose band anchors disagree by 2.7 mag (9.01 vs 11.75). Do not write "0.1%".
2. The recovery table must quote the width the sweep actually used, and its cost. Every wide prior is pulled down, never up (-0.94 sigma from 9.01 +/- 1.83, -0.59 from 8.61 +/- 2.41, -1.59 from 11.65 +/- 1.40; only the tight 9.01 +/- 0.30 held). The data's own av preference is low, so on 194 the sweep gives 0.698x, not the tight prior's 0.983x.
3. R_source is 0.35-0.43x truth at every av, over 5.2 mag and eleven runs -- unchanged and stronger. Still not attributable to the convention.
4. fitu0te is not a confound between the sweep and the arms (0.634/0.630 vs 0.632/0.651 at av = 4.26).

One thing the paper should not do yet

The sweep is 3 of 6 events in, and only 194 is a clean test of the convention. 152 collapses its source to 1490 pc against 8292 (-17 sigma) and got worse when av rose from 1.10 to 3.06 -- opposite in sign to 194 -- and 223 found the wrong topology outright (s = 0.132 against 2.011, q = 0.046 against 7.9e-4). Neither is an extinction failure, and they must not be averaged into a recovery statement. 008, 062 and 128 are still sampling.

Notes spec paper_extinction_convention.txt section 4b carries this version, new 4c carries the sweep status, and the retracted theta_star-bias columns in sections 3c and 4 are now marked as such. EXOZIPPy PR #306 carries the same correction where the code prints it.