TabFM (②): self-repair preprocess + sweep/plot wiring

[xiaohan2012]

Adds TabFM to the exp6 self-repair sweep as the 4th backbone. Follows #30 (now merged); reopened against main after #30 squash-merged (original #31 auto-closed when its base branch was deleted).

What

• tabfm_preprocess — one clean (no-ensemble) member of TabFM's shared preprocessing: ordinal-encode categoricals + mean-impute numerics, then TabFM's own PreprocessingPipeline with the power member (CustomStandardScaler → yeo-johnson → log-based OutlierRemover). Ensemble views + the cat_mask embedding path are deferred, matching the single-clean-forward reproduction used for LimiX / TabICL / Mitra.
• Vendored (Apache-2.0, byte-for-byte) the numeric-normalization subset of classifier_and_regressor.py → vendor/tabfm/preprocess.py (the pipeline lives in a 3942-line un-vendored estimator file; same rationale as the LimiX preprocess vendoring).
• Wire tabfm into run_self_repair_sweep MODELS + plot_self_repair labels.
• configs/tabfm.yaml tuned for the 24GB box: micro_batch 8, n_jobs 1 (loky+CUDA crashes), max_seq_len 512 (row-attn is O(seq²)), prior_batch_size 128 (per-step readout buffer would OOM the 64GB cgroup at 512), max_steps 100.

Verified end-to-end (GPU)

• Finetune: 100 steps, loss 1.68 → 0.83, 25 per-depth decoders.
• Sweep: 15/15 TabArena binary tasks, final-layer AUC ≈ native everywhere.
• Figure 8: strong early-layer self-repair (ablate layer 1 → 0.50 chance, fully repaired by layers 2–4). Four-model comparison done.

Test

tabfm_preprocess shapes/dtype/finite on a mixed-type table (incl. a big-scale column). Passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_011HRBX8LFAsrTzsyLs9M4ad

[xiaohan2012]

The four-model comparison this PR completes

TabFM was the fourth and last backbone, so this is where the cross-architecture picture closes. The figure lived under a gitignored out/; committing it here for the record (analysis/exp6-figures/, pinned).

Setup: 15 TabArena binary tasks per model · natural OpenML split · identity-skip each layer, decode every depth, normalize per task by native final AUC (floored 0.5), then average. One measurement path, four adapters.

Black = no ablation. Colored = trajectory from each skip point, dark→light by ablated layer. Red ✗ = immediate post-skip reading.

model	layers	unrecoverable	late layers
LimiX-2M	12	L0–L2, still 0.05–0.08 short at the output	back on baseline within a layer or two
Mitra	12	L0/L1	fold back
TabICLv2	12	essentially none (baseline already starts at 0.95)	one mid-stack dip that recovers
TabFM	24	L1 only (drops to 0.50 chance, repaired by L2–L4)	fold back

The drop-then-recover shape and the early-layer exception hold in all four. The adapter contract is what makes this comparison meaningful: identical capture, intervention, and decode path across a 3D single-stream model (TabICL, TabFM), a 4D model (LimiX), and a double-stream one (Mitra) — the only per-model code is the adapter and the preprocessing.

Caveat worth recording

The y-axis is AUC, which is rank-based and ceiling-blind: once the classes are separated, later layers can keep doing signed work without moving the curve. That is why the late-layer skips look so flat. A magnitude coordinate makes dips visible that AUC hides entirely — see #39.

And all of it is a total effect: the ablation propagates freely downstream at every point of measurement, so a curve that dips and returns is equally consistent with active repair and with plain redundancy. Separating the two needs a direct effect — #45.