Add exact Qwen3.5 LoRA projection topology

[kiddyboots216]

Summary

• Keep Qwen3.5/3.6 GDN q, k, v, and z projections as independent rank-16 adapters.
• Add independent shared-expert gate, up, and down factors while preserving the fused gate/up base GEMM.
• Use the same folded weights in regular forward, ordered EP combine, and synchronized serving weights.
• Align the shipped Qwen3.5 LoRA example with exact admission: FlashAttention 4, Triton MoE, and all-to-all EP.
• Validate the parsed shipped configuration through the exact numerical-program and MoE admission resolvers.
• Add topology, checkpoint, one-GPU lifecycle, and two-rank FSDP2 coverage.

Stack

• Base: docs: refresh public documentation for current code #42 at 45a087aac147c94158aa953cb9658bc058cfb311
• Head: 89e4d2e4eb5313232ba8606a6676e7d905d08dd9

Validation

• Final focused model, argument, runner, and weight-sync suite: 103 passed, 31 expected skips.
• Changed Python files: Ruff passed.
• The one-GPU update/sync/checkpoint/reload and two-rank NCCL/FSDP2 replay payloads remain patch-equivalent across the restack.

[broly-code-security-scanner]

Broly Security Scan

Note

✅ Clean scan
No vulnerabilities detected in this PR.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.2) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Exploit Chains · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.

 

[kiddyboots216]

Superseded by #57, which consolidates the complete exact trainer-serving stack directly onto main.