Admit Ulysses head-sharding into the exact byte contract (attention side)

[kiddyboots216]

Admit Ulysses head sharding into the exact byte contract

Stacked on #52. This PR contains production code and conventional tests only.

What changes

• Make the context-parallel strategy explicit at call sites, so the ordering of RoPE and head exchange cannot silently change.
• Fail closed on uneven Q/KV head partitions and on exact-lane kernel fallback.
• Establish the attention-side byte contract for Ulysses degrees 2, 4, and 8 while keeping unsupported hybrid/GDN compositions closed.

Gates

• FA4 head-bucket invariance at the production head geometry: 5 passed.
• End-to-end Ulysses-1 versus Ulysses-N byte fixtures at degrees 2, 4, and 8, including padding and GQA replication branches: all verdicts passed.
• CPU admission and pipeline regressions: 45 passed.

Scope

This PR does not broaden the admitted production topology by itself. Hybrid models remain fail-closed until the GDN context-parallel program is present and jointly admitted. The byte claims are toolchain-scoped and were verified on H100 with the pinned Torch, CUDA, FA4, CUTLASS DSL, and Quack versions recorded by the test environment.

[broly-code-security-scanner]

Broly Security Scan

Note

✅ Clean scan
No vulnerabilities detected in this PR.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.2) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Exploit Chains · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.

 

[kiddyboots216]

Superseded by #57, which consolidates the complete exact trainer-serving stack directly onto main.