[CUDA] Add persistent fpA_intB MatMulNBits tactic autotune cache

[tianleiwu]

Stacked on #29585 (fpA_intB SM90/bs32/bias). Review/merge that first; this PR's base is its branch, so the diff here is cache-only. Rebase to main after #29585 merges.

Description

Adds an opt-in, hardware-keyed on-disk tactic cache for the fpA_intB weight-only MatMulNBits path, plus a reduced first-time profiling sweep with lazy fallback. Persistence is off by default — with no cache location configured, behavior is unchanged (in-process only).

Cache subsystem (gemm_tactic_cache.{h,cc})

• HardwareSignature (GPU name, SM, CUDA runtime, ORT version/commit/build config) names and strictly guards cache files, so tactics are never reused across incompatible GPUs/toolkits/builds.
• TSV format: percent-encoded free-text fields, a column-name header (appended columns are backward/forward compatible), file locking + atomic temp+rename write, and in-memory merge on flush so concurrent sessions tuning different shapes don't lose updates.
• MatMulNBitsTacticCache keyed by (N16b, K, act dtype, weight type, bits, block_size, zero-points, gemv_enabled, packing_sm). packing_sm keeps the SM80-compat and native-SM90 tactics for the same shape distinct on one device (e.g. H200).

Profiler (gemm_profiler.h, fpA_intB_gemm_profiler.{h,cc})

• Reduced initial sweep to a small M bucket set ({1,2,4,…,2048} default; override with ORT_FPA_INTB_PROFILE_M). getBestConfigOrProfile lazily profiles any missing bucket on first use.
• Persistent-cache load/store hooks are no-ops unless a cache is attached.

Wiring (matmul_nbits.{h,cc}): cache location resolved from session config (ep.cuda.gemm_tactic_cache_dir / _prefix) or env (ORT_CUDA_GEMM_TACTIC_CACHE_DIR / _PREFIX).

Offline tuning tool: onnxruntime/python/tools/fpa_intb_tune.py.

Production hardening (5 gaps addressed)

1. CUDA-graph-capture safety — ComputeInternal uses lookup-only getBestConfig while the compute stream is capturing (lazy profiling launches kernels/events/allocs, illegal during capture); run a warmup inference before capture to tune the needed buckets.
2. No hot-path disk I/O — lazily profiled buckets stay in the in-process map and are not flushed to disk on the inference path; disk writes happen only off the hot path (construction-time sweep + offline tool).
3. Per-location cache — the process-global cache is keyed by resolved location, so sessions with different dirs/prefixes each get their own cache instead of the first kernel's location winning.
4. Reduced-sweep ceiling — large M above the initial ceiling is covered by lazy profiling off-capture.
5. Stale-tactic guard — CUTLASS tactics loaded from disk are validated against the runner's current getConfigs() and dropped (re-profiled) if incompatible.

Testing

• gemm_tactic_cache_test.cc (8 cases): TSV round-trip, SM80/SM90 config serialize/parse, null tactic, store/load, signature-mismatch rejection, appended-column tolerance, concurrent flush-merge. Built with onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON, run via --gtest_filter=CUDA_EP_Unittest.All. 8/8 pass on H200.
• C++ *MatMulNBits* integration: 65 passed / 1 skipped (no regression with cache present).
• Python test_op_matmulnbits_prepacked_cuda.py: 7 passed.
• End-to-end persistence smoke test on H200: session with ep.cuda.gemm_tactic_cache_dir writes NVIDIA_H200_sm90.matmulnbits_fpa_intb.tsv and a second session reuses it.

[Copilot]

Pull request overview

This PR adds an opt-in, hardware-keyed persistent on-disk tactic autotune cache for the CUDA fpA_intB MatMulNBits path, and updates the profiler to reduce the initial M-bucket sweep with a lazy single-bucket fallback (with CUDA graph capture safety). It also includes a Python offline tuning tool, unit tests, and documentation for the cache format and configuration.

Changes:

• Introduce gemm_tactic_cache utilities (hardware signature, TSV encoding/decoding, config serialization, atomic/locked merge flush).
• Update GemmPluginProfiler / fpA_intB profiler to support reduced initial profiling + lazy per-bucket profiling and persistent-cache hooks.
• Wire cache resolution into MatMulNBits and add an offline tuner + docs/tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
onnxruntime/test/contrib_ops/cuda_kernels/gemm_tactic_cache_test.cc	New pure-logic unit tests for TSV/config round-trip and cache merge/signature behavior.
onnxruntime/python/tools/fpa_intb_tune.py	New offline tuning CLI tool that generates a persistent cache by constructing a CUDA session and optionally running dummy inferences.
onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.h	Use new profiler max-M logic for the initial sweep.
onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.cc	Add process-global persistent cache wiring + CUDA graph capture guard for lazy profiling.
onnxruntime/contrib_ops/cuda/llm/gemm_tactic_cache.h	New cache API: hardware signature, TSV helpers, MatMulNBits cache key and disk-backed cache wrapper.
onnxruntime/contrib_ops/cuda/llm/gemm_tactic_cache.cc	New cache implementation: TSV IO, signature checks, file locking, merge + atomic replace.
onnxruntime/contrib_ops/cuda/llm/gemm_profiler.h	Add reduced-bucket sweep support, persistent-cache hooks, and lazy getBestConfigOrProfile.
onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemm_profiler.h	Add env-controlled M-bucket override, persistent-cache attachment, and overrides for new hook points.
onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemm_profiler.cc	Implement reduced bucket set, persistent cache load/store, and stale-tactic validation.
docs/contrib_ops/cuda/matmul_nbits.md	Document cache enablement, reduced sweep, lazy profiling, CUDA graph capture constraints, and tooling/tests.
docs/contrib_ops/cuda/gemm_profiler_cache.md	New design/spec doc describing the cache format, signature strictness, and phased plan.