Block 8 (exp6, ⑤): self-repair repro driver + Figure-8 plot

[xiaohan2012]

Ties exp6 together end-to-end: a driver that runs the self-repair sweep across the 15 TabArena binary tasks, plus the Figure-8 trajectory plot. Everything downstream (sweep / metrics / eval) already landed in #16 / #17; this is the thin orchestration + visualization layer on top.

What's here

• scripts/run_self_repair.py — per-dataset: load → (optional) subsample → LimiX preprocess → ablation_sweep + native_final_auc [+ ablation_diffs]. Dumps JSON. --subsample-train/-test (0 = full), --skip-diffs, --out. Per-dataset try/except so one bad table can't sink the run.
• scripts/plot_self_repair.py — paper Figure 8: black baseline + per-ablation trajectories from the skip point + red-x immediate-drop connectors, normalized per-dataset by native final AUC (floored 0.5) then averaged.
• evaluation/layerwise.py::load_decoders — one fine-tuned decoder per capture depth from a weights dir (dedupes inline loading; used by the driver).
• pyproject.toml — viz dependency-group (matplotlib), kept out of core/CI.
• .gitignore — weights/, out/, run logs (trained decoders stay out of version control).

Verification

Full run (15/15 tasks, no subsample) reproduces the self-repair shape: black baseline climbs 0.59→1.0; early-layer ablations show a large immediate drop then recovery; late-layer ablations barely dent it.

Figure artifacts live under gitignored out/ (not committed).

[xiaohan2012]

Results — the figures this PR produced

The driver's output landed under a gitignored out/, so this PR carried no plot. Committing them now for the record (analysis/exp6-figures/, pinned).

Setup: LimiX-2M · 15 TabArena binary tasks · no subsample (natural OpenML split) · skip-ablate each layer, decode every depth, normalize per task by native final AUC (floored 0.5), then average.

LimiX-2M — the driver's own output

Black = no ablation. Colored = trajectory from each skip point, dark→light by ablated layer. Red ✗ = immediate post-skip reading, dashed line to where the baseline was.

• Baseline climbs 0.59 → 1.0 — the layerwise picture the tabular logit lens is supposed to show.
• Early layers do not recover. Skipping L0–L2 drops AUC to 0.59–0.66 and the trajectory never rejoins the baseline; it ends 0.92–0.95 at the output.
• Late layers barely dent it. From ~L4 on, the immediate drop is ≤0.03 and the curve is back on the baseline within a layer or two.
• The drop-then-recover shape is reproduced, and so is the early-layer exception.

All four models

Same figure for the other three (added after this PR, same driver and settings):

• Mitra (12 layers) — closest to LimiX: L0/L1 unrecoverable, everything later folds back.
• TabICLv2 (12 layers) — flattest baseline of the four (starts at 0.95); even the early-layer skips lose little. The one visible dip is a mid-stack layer that recovers.
• TabFM (24 layers) — only L1 is unrecoverable; every other skip is on the baseline by the output.

Caveat worth recording

The recovery here is measured on AUC, which is rank-based and ceiling-blind — once the classes are separated, later layers can keep doing signed work without moving the curve. That is why the late-layer skips look so flat above. Switching the y-axis to a magnitude metric (margin / GT-logit) makes dips visible that AUC hides entirely — see #39.

More importantly, all of this is a total effect: the ablation propagates freely downstream at every point of measurement. A trajectory that dips and returns to baseline is equally consistent with active repair and with plain redundancy. Separating the two needs a direct effect, which is #37/#45.