[Dev] [DO NOT MERGE] GLM-5.2 integration stack, recipes, and performance tracker by buptzyb · Pull Request #6944 · NVIDIA/Megatron-LM

buptzyb · 2026-08-28T08:48:18Z

Caution

This is a tracking-only integration PR. Do not merge it.
Each feature must be reviewed and merged through its source PR.

Purpose

This draft tracks the runnable GLM-5.2 development stack, representative benchmark recipes, current validation state, and the latest trustworthy performance results. The source PRs below remain the units of review and merge; the final compatibility commit only resolves cross-PR interactions in the combined stack.

Integration base: NVIDIA/Megatron-LM:dev@710925795
Integration head: 81997cd67
Tracking branch: buptzyb:codex/tracking/glm5-2-integration
Last refreshed: 2026-08-28

Integrated PR stack

CUDA Graph and dynamic execution

Order	Source PR	Capability	Aggregate commit	Source head on 2026-08-28
1	`PR 6022`	Paged stash with TE whole-MoE CUDA Graphs	`c7e2742d3`	`0a89681cc`
2	`PR 5807`	THD full-iteration graphs with fixed gradient accumulation	`b97261f49`	`26d0e6794`
3	`PR 5258`	Chunk-wise CUDA Graphs with a shared PP-rank pool	`df750123a`	`82761c05e`
4	`PR 6783`	Activation-offload handler lifecycle fix	`1fd15f1a6`	`42ee19e8f`
5	`PR 6801`	Parent communicator reuse for dynamic CP	`866ab006f`	`adf826b2a`
6	`PR 6479`	CUDA Graph slot sharing across dynamic-CP variants	`fa67bdfbd`	`853e321f0`

DSA

Order	Source PR	Capability	Aggregate commit	Source head on 2026-08-28
7	`PR 6206`	Plan-driven DSA scoring and CP metadata	`2bda9f548`	`5c3f4bd57`
8	`PR 6910`	Static-shape sparse-attention backward	`1a2d86435`	`c625acaca` — refresh pending; aggregate currently contains the preceding `b0acf2d93` revision
9	`PR 6753`	DSA sparsity/indexer FLOPs accounting	`04bd22a3d`	`06b971a8e`

Optimizer and repeated MTP

Order	Source PR	Capability	Aggregate commit	Source head on 2026-08-28
10	`PR 6326`	Per-head Muon for attention projections	`a4052318e`	`febf2f185`
11	`PR 6472`	Repeated-MTP DSA top-k/latent-KV sharing	`d4397793a`	`64b1daf44`
12	`PR 6473`	End-to-end TV loss for MTP	`ac1080aa4`	`38be10fa2`
13	`PR 6741`	Absolute future-row alignment for MTP	`14046685d`	`62fcc44cc`

81997cd67 is integration-only glue for dynamic-CP graph banks, paged-stash capture order, DSA metric reduction, and repeated-MTP compatibility. It is not a substitute for any source PR and must not be merged independently.

Representative GLM-5.2 recipe

The following is the 16-GPU long-context MXFP8 proxy. It preserves the full GLM-5.2 hidden, attention, DSA, expert-MLP, shared-expert, and MTP dimensions while reducing depth and global expert count. Two 32K samples are packed into THD64K; dynamic CP is configured with a maximum CP16, yielding 4K local tokens and four local experts per GPU.

ARGS:
  # Architecture: 21 main layers (2 dense + 19 MoE), plus repeated MTP7.
  num_layers: 21
  moe_layer_freq: "([0]*2+[1]*19)"
  num_experts: 64
  hidden_size: 6144
  num_attention_heads: 64
  kv_channels: 256
  q_lora_rank: 2048
  kv_lora_rank: 512
  qk_head_dim: 192
  qk_pos_emb_head_dim: 64
  v_head_dim: 256
  moe_ffn_hidden_size: 2048
  moe_shared_expert_intermediate_size: 2048

  # DSA / IndexShare.
  experimental_attention_variant: dsa
  dsa_kernel_backend: cudnn
  dsa_indexer_n_heads: 32
  dsa_indexer_head_dim: 128
  dsa_indexer_topk: 2048
  dsa_indexer_topk_freq: 4
  dsa_indexer_skip_topk_offset: 2
  dsa_indexer_loss_coeff: 1e-2
  dsa_indexer_use_sparse_loss: true

  # Repeated MTP and end-to-end TV objective.
  mtp_num_layers: 7
  mtp_use_repeated_layer: true
  dsa_mtp_index_kv_share: true
  mtp_loss_type: e2e_tv
  mtp_detach_heads: true
  mtp_loss_scaling_factor: 0.1

  # 16-GPU execution and THD64K packing.
  tensor_model_parallel_size: 1
  pipeline_model_parallel_size: 1
  expert_model_parallel_size: 16
  context_parallel_size: 16
  dynamic_context_parallel: true
  sequence_packing_scheduler: default_dynamic_cp
  seq_length: 32768
  use_varlen_dataset: true
  max_seqlen_per_dp_cp_rank: 4096
  micro_batch_size: 1
  global_batch_size: 8

  # Production precision and optimizer semantics.
  bf16: true
  fp8_recipe: mxfp8
  fp8_format: e4m3
  fp8_param_gather: true
  reuse_grad_buf_for_mxfp8_param_ag: true
  optimizer: muon
  muon_split_qkv_per_head: true

  # Benchmark window: 8 warmup + 20 measured iterations.
  train_iters: 28

The same configuration family also contains:

Model	Shape	Purpose
16-GPU proxy	21L, 64E, `TP1/PP1/EP16/max-CP16`, THD64K	Single-segment feature/performance iteration
64-GPU scale proxy	21L, 256E, `TP1/PP1/EP64/max-CP64`, THD256K	EP64/CP64 and one-NVL-domain scale validation
256-GPU full model	78L, 256E, `TP1/PP4/EP64/max-CP64`, layout `18/20/20/20`, THD256K	Four-domain full-model validation

BF16 and MXFP8 overlays exist for GB200 and GB300. These are benchmark/smoke recipes, not convergence recipes. MLA Muon split semantics remain an explicit area of ongoing parity work.

Latest formal performance

The latest low-variance formal result was measured on source 03017e849 on 2026-08-27. The current tracker head has subsequent integration/head refreshes and has not yet been rerun as a formal performance gate.

Workload: 16×GB200, GLM-5.2 21L (2 dense + 19 MoE) + repeated MTP7, 64 experts, MXFP8, THD4K, TP1/PP1/EP16/CP1/DP16, MBS1/GBS64, four fixed microbatches, cuDNN DSA, force-balanced routing, TE op fuser, paged stash, no recompute/offload. Each variant used 8 warmup + 20 measured iterations in a same-allocation ABBA run.

Mode	Median step	p95 range	CV range	Audited model TFLOPS/GPU	Tokens/s/GPU	Peak device-used memory
Eager / no CG	3493.75 ms	3499.70–3522.14 ms	0.309–0.327%	652.94	4689.5	162143.56 MiB
Full-iteration CG	3259.15 ms	3264.05–3266.23 ms	0.188–0.238%	699.94	5027.1	173059.56 MiB
Delta	7.198% faster	—	—	+7.198%	+7.198%	+10916 MiB

The TFLOPS numerator is 36.499350490 PFLOP/step after correcting detached-MTP output-weight wgrad, repeated-MTP IndexShare/KVShare, sparse-indexer-loss, and DSA pair overcounts. It is an audited model-GEMM estimate, not hardware-executed FLOPs.

There is no qualifying current-stack 64-GPU or 256-GPU formal result yet. Older full-model runs were short smokes, high-variance exploratory runs, or did not reliably prove CUDA Graph capture, so they are intentionally excluded from the formal table.