Kimi k3 sglang blackwell

[ezhong0211]

Purpose

Enables Kimi-K3 on the SGLang AL2023 images and fixes two Blackwell defects found while
verifying it on 8x B300.

Commit 1, Blackwell correctness. Three changes, each turning a load-time failure on
B200/B300 into either a correct build or a failed one. No effect on pre-Blackwell GPUs.

• torch_cuda_arch_list 10.3 to 10.3f. MXFP4 kernels emit cvt.e2m1x2, which ptxas
  accepts only under an a- or f-suffixed target. A bare 10.3 compiles and links without
  error but emits no usable FP4 device code for sm_103, so the gap appears only when a
  model with MXFP4 weights loads on the hardware. This diverges from upstream's own
  Dockerfile, which still uses a bare 10.3. Deliberate, not a catch-up.
• Assert cutlass._mlir imports. nvidia-cutlass-dsl ships pure Python and its MLIR
  extensions come from a separate -libs-cu13 wheel. Both are declared, yet a prior image
  shipped without _mlir, so import cutlass failed. That is a hard startup failure on
  Blackwell only, since bf16_gemm_backend='auto' selects the CuTe DSL path on SM100/SM103
  and cuBLAS elsewhere. Root cause is unresolved; this assertion does not fix it, it stops
  the same silent gap from reaching a released image again.
• Record reachable MXFP4 MoE backends to /etc/dlc-mxfp4-moe-backends.txt. With
  moe_runner_backend='auto' SGLang picks trtllm-gen on SM100/SM103 when its cubins are
  present and silently drops to marlin when they are not, so an image can ship on a slower
  expert path unnoticed. Warns rather than fails, because trtllm-gen's MoE cubins are
  unmerged upstream and a version bump does not help.

Also folded in, all found while building this image and all inert on a green build: a
single-tree pip bootstrap via get-pip.py, libz3 taken from the z3-solver wheel, and a
conditional tilelang cudart.cc patch.

Commit 2, the ref bump. sglang_ref moves to abddb1c7, with sglang_kernel_version
0.4.5 and flashinfer_version 0.6.15.post1. Kimi-K3 needs srt/models/kimi_k3.py, which
is in no released tag, so this is a bare commit hash. The two version pins are ABI-coupled
to the ref and are what upstream's own Dockerfile pins at that exact commit, read from the
tree rather than assumed. Both amzn2023 variants move together. The ubuntu variants pin no
sglang_ref (they build the 0.5.16 tag) and are untouched.

The two commits must land together and must not be split. This ref without the 10.3f
change produces an image that advertises K3 support, builds green, then fails at load time
on B300 because MXFP4 has no working device code.

Open items for the reviewer:

• framework_version stays 0.5.14+dlc1 and dlc_minor_version stays 2. Both feed
  image tags and are arguably stale. The only required_image_pattern in
  sglang-model-tests.yml keys on os_version, so neither breaks test matching. Flagged as
  a release-versioning decision rather than silently picked.
• Whether the trtllm-gen fallback costs accuracy or only throughput is unmeasured.

Test Plan

CI cannot validate Kimi-K3 at any point. It needs roughly 1.51 TiB of GPU memory against
640 GB on the largest runner, about 2.4x short, and B300 is not in the fleet. No model-test
entry is added, deliberately. So the plan splits into what CI can prove and what only
hardware can.

CI, this PR:

1. Image builds at abddb1c7 with sglang-kernel 0.4.5 and flashinfer 0.6.15.post1.
2. The cutlass._mlir assertion passes, proving the CuTe DSL native libs landed.
3. The MXFP4 probe runs and records a backend list.
4. Existing sanity, security, telemetry and model tests pass unchanged, confirming no
   regression on non-Blackwell hardware.

Local, already done:

5. resolve_build_args.py emits all four keys as upper-case build args on both variants,
   confirming the config overrides actually bind rather than silently falling back to the
   Dockerfile defaults.
6. Both configs parse as YAML and hold identical values.

Hardware, out of band on 8x B300, not gated on this PR:

7. Serve Kimi-K3 end to end on the built image.
8. cuobjdump --list-elf on the sgl_kernel shared objects, bare 10.3 against 10.3f,
   to replace the ptxas argument with direct evidence of emitted FP4 code.
9. Compare deep_gemm against marlin for throughput and output equivalence.

Test Result

Pending. Will update with the CI run once the PR is open.

Local checks pass:

• resolve_build_args.py emits SGLANG_REF=abddb1c7e9d61ddddeaf016d885c2f20aab426e8,
  SGLANG_KERNEL_VERSION=0.4.5, FLASHINFER_VERSION=0.6.15.post1,
  TORCH_CUDA_ARCH_LIST=9.0;10.0;10.3f on both ec2 and sagemaker.
• Both config files parse and agree.

Prior hardware evidence, for context rather than as validation of this exact tree: Kimi-K3
served on 8x B300 (p6-b300.48xlarge, compute capability 10.3) on 2026-08-04, roughly
229 GiB per rank under SGLang. That run used 40feea27, a dev-branch commit of the same
work, so the hardware evidence is one commit removed from the abddb1c7 shipped here. A
main commit was chosen anyway because it is immutable, where a dev branch can be rebased or
deleted. Item 7 above closes that gap.

---

Toggle if you are merging into main Branch

PR Checklist

• I ran pre-commit run --all-files locally before creating this PR. (Read DEVELOPMENT.md for details).