Dev (Qwen 3.5 recipe)#22
Merged
Merged
Conversation
New recipes examples/math/qwen3.5-4b-m2po-{full,delta} for training Qwen3.5
(dense, hybrid Gated-DeltaNet text backbone; model_type qwen3_5) with M2PO on
AstraFlow, mirroring the existing qwen3-8b-m2po recipe structure. Trained
text-only for math (the checkpoint ships as Qwen3_5ForConditionalGeneration).
Verified end-to-end on NVIDIA L40 (Ada, sm_89): a full run trained 86+ steps
with no crash and steadily rising eval — overall avg@k 47.8% -> 57.4% (+9.6) and
pass@k 56.5% -> 67.4% over the first 80 steps (AIME24/AIME25/AMC/Minerva/MATH500,
eval every 10 steps).
Minimal framework changes for Qwen3.5 / transformers>=5 compatibility:
- model.py: register qwen3_5 + is_qwen3_5_model()
- fsdp_engine.py: pass attention_mask=None for qwen3_5 (transformers>=5
create_causal_mask calls .ndim; the old dict form raised AttributeError)
- fsdp/__init__.py: normalize _no_split_modules set->list (qwen3_5 exposes a set)
- rlvr.py: unwrap BatchEncoding from apply_chat_template (transformers>=5)
Recipes use the standard packed training forward. attention_backend=flashinfer
(fa3 dispatches a Hopper-only kernel that fails on Ada/L40 for the GDN path;
flashinfer + triton both verified); max_running_requests=32, mem_fraction_static=0.7
on inference and FSDP dp=4 + max_tokens_per_mb=8192 on the trainer to fit 44GB L40.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The version bump (transformers 5.8.1, torch 2.11+cu130, sglang dev) broke the
existing plain-Qwen3 math recipes (qwen3-1.7b, qwen3-8b). Two fixes:
1. fsdp_engine.py: always pass attention_mask=None for the packed/varlen forward.
The old dict(full_attention=None, sliding_attention=None) form is a
transformers-4.x relic; on transformers>=5 a dense model (qwen3/qwen2) treats
that dict as a *precomputed* mask, skips creation, and crashes ('dict' object
has no attribute 'ndim'). None is correct for all archs (dense, moe, vl,
qwen3.5/GDN) — masking is driven by cu_seqlens + position_ids. Subsumes the
prior qwen3_5/moe/vl special-case (drop now-unused imports).
2. recipes + cli_args: set attn_impl: sdpa in actor+ref for qwen3-1.7b and
qwen3-8b recipes (flash_attn is ABI-broken vs torch 2.11+cu130 -> import crash;
default was flash_attention_2). Expand attn_impl choices to sdpa/eager.
Verified end-to-end: qwen3-1.7b full + delta train cleanly to step 100 on the
bumped stack; math500 avg@k rises 73.0->77.0 (full) / 72.2->78.8 (delta);
importance_weight=1.0000 (packed forward correct); zero crashes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Remove the unused is_qwen3_5_model() helper (its only caller, the mask-branch special-case, went away when the packed-forward mask was made unconditional). - Note in model.py why qwen3_5 is in VALID_VISION_MODELS (the checkpoint ships as Qwen3_5ForConditionalGeneration -> loaded via the ImageTextToText path). - Simplify the packed-forward attention_mask comment in fsdp_engine.py. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the validated runtime stack (transformers 5.8.1, kernels 0.14.1, SGLang dev with qwen3_5 + TritonGDNKernel, fla 0.5.0, flashinfer 0.6.11.post1, torch 2.11.0+cu130, attn_impl sdpa), GPU layout, run commands, and the validated eval (overall avg@k 47.8 -> 57.4, pass@k 56.5 -> 67.4 over 80 steps). The default pyproject pins load qwen3_5, but the validated stack is installed out of band; a pin bump is deferred to a separate, tested PR (the validated SGLang is a dev build, not a clean release). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump the pyproject transformers pin (core dep + uv override) from 5.6.1 to the 5.8.1 that the Qwen3.5 and Qwen3-dense math recipes were validated on, and move the coupled kernels constraint from <0.13 to >=0.14,<0.15 (what transformers 5.8.1 was validated against). torch is already 2.11.0; flashinfer comes in transitively via sglang. sglang stays pinned at the published 0.5.12.post1 — the Qwen3.5 inference path was validated on an sglang dev build that ships qwen3_5, as noted in the recipe README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….8.1) The published sglang 0.5.12.post1 predates qwen3_5 and pins transformers 5.6.x, so it is incompatible with the transformers==5.8.1 the recipes require. Pin sglang to the validated main-branch build (sgl-project/sglang @ 373cadc9): it ships qwen3_5 + TritonGDNKernel and itself requires transformers==5.8.1, flashinfer 0.6.11.post1, torch 2.11.0, kernels<0.15 -- all matching the validated env. Update the [tool.uv] comments to match. Verified: every pin/override matches the working astraflow35 env (transformers 5.8.1, kernels 0.14.1, outlines-core 0.1.26, torch 2.11.0, sglang @ g373cadc92). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment-only alignment of the Qwen3.5 recipes + touched code to the repo's conventions (qwen3-8b recipes / surrounding code as the baseline): - recipe experiment.yaml/raas.yaml: restore the "# -- ... --" section banners and the auto-derives / adaptive-availability rationale comments the dense recipes use; restyle the attn_impl comment to sentence-case prose. - model.py: capitalize the qwen3_5 registry comment to match the file. - fsdp_engine.py: reflow the attention_mask comment so dict(...) is not split mid-token, matching the file's other multi-line comments. - pyproject.toml: reflow the sglang-extra comment to the column band. - README: tag the GPU-layout code fence (text). No functional content changed (verified: all yaml/sh/py/toml parse; only comment lines were added/removed). Also refresh the recipe README install note, which still described SGLang as pinned at 0.5.12.post1 before the git-build pin landed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… 3.6.1 Two pyproject fixes so a clean install reproduces the validated astraflow35 env: - Add flash-linear-attention==0.5.0 to the sglang extra: the fast Gated-DeltaNet kernels Qwen3.5 GDN training used in the validated runs (optional -- transformers falls back to a slower pure-torch path when absent). Pulls fla-core==0.5.0. - networkx==3.3 -> 3.6.1 to match the validated env (the only pinned version that differed from astraflow35). Verified: uv pip install --dry-run -e ".[sglang]" resolves cleanly (299 packages, exit 0) to the validated versions (flash-linear-attention 0.5.0, fla-core 0.5.0, networkx 3.6.1, transformers 5.8.1, sglang @ 373cadc9, ...). All other pinned ML versions already matched astraflow35; the ~60 loose utility deps are left flexible per the repo's convention. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch the dense Qwen3 math recipes (qwen3-1.7b-m2po-2gpus-{full,delta},
qwen3-8b-m2po-{full,delta}) and the cli_args attn_impl default to
kernels-community/flash-attn2 -- a prebuilt, ABI-matched FlashAttention-2
kernel from the HF kernels hub (cached on first use, no source build).
The literal flash_attention_2 loads the local flash-attn wheel, which is
ABI-broken on torch 2.11+cu130 (undefined symbol); is_flash_attn_2_available()
only checks package metadata so it reports available and then crashes on import.
The kernels-hub variant is ABI-matched and is the upstream-faithful FA2 the
recipes were tuned with. A paired step-25 A/B/C on qwen3-1.7b-m2po-2gpus-full
gave overall avg@k FA2 >= sdpa >= sdpa+recompute_logprob, all within eval noise;
FA2 varlen also derives the packed block-diagonal mask from cu_seqlens, avoiding
sdpa's reliance on position_ids resets. Qwen3.5 recipes stay on sdpa.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ild) Switch the [sglang] extra from the git build (373cadc9) to the published sglang==0.5.13.post1 -- the release the recipes were recently validated against, which ships qwen3_5 support (sglang/srt/models/qwen3_5.py) so it covers both the Qwen3.5 and Qwen3-dense recipes. Update the Qwen3.5 READMEs' validated-stack tables to match (sglang 0.5.13.post1, flashinfer 0.6.12). The flash-attn-4 pre-release / transformers 5.8.1 / kernels 0.14.x [tool.uv] overrides are unchanged (0.5.13.post1 still pulls flash-attn-4 4.0.0b15). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both full and delta variants re-run end-to-end on the published-release pin (sglang==0.5.13.post1): training completes, full + delta weight transfer both work, eval holds at baseline (~49-51% overall avg@k) -- no regression vs the predecessor git build the step0->step80 table was produced on. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SGLang's default chunked_prefill_size=-1 (off) makes a batch's prefill upcast the LM-head logits to fp32 in one shot, stalling/OOMing the worker -> rollout starvation -> 'detokenizer unresponsive' health-check failures -> scheduler crash in forward_extend. Observed on 8xH100 during the 1000-step validation run. The sibling qwen3-8b recipe already sets this; the GDN hybrid needs it too. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two build fixes required for the Megatron backend on the current transformers 5.8 / torch 2.11 / CUDA 13 stack, plus a docs sync. Both were found by running an actual 2-step training smoke inside freshly built images (FSDP Qwen3.5-4B and Megatron Qwen3-1.7B). - Dockerfile.sglang: flash-attn 2.8.3 was silently not installed. The pyproject [tool.uv] exclude-dependencies entry also suppresses an explicit `uv pip install flash-attn` run from the project dir, so the image shipped without FA2. Install it from /tmp so uv ignores the project [tool.uv] and actually builds it from source (ABI-matched to torch; sm_80/90/100/120). Required by the Megatron TransformerEngine `flash` backend for THD/packed sequences (and dense-Qwen3 flash_attention_2); the FSDP image needs it too. - Dockerfile.sglang.megatron: transformers 5.8 moved rope_theta into a nested config.rope_parameters dict; mbridge 0.13.0 reads the old top-level attr and crashes converting Qwen3 configs to Megatron. Patch the installed mbridge to read either location. - docs: sync installation.md transformers references 5.6.1 -> 5.8.1.
Qwen3.5 is a Gated-DeltaNet hybrid; mbridge has no qwen3_5 bridge and Megatron-Core has no GDN layer spec, so the Megatron backend cannot build or weight-load it. Document this in both Qwen3.5 recipe READMEs (train on FSDP instead) and list the mbridge-supported architectures in the Megatron recipe README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tart isn't skipped The eval_at_start (version=0) eval fired at trainer loop entry, ~40s before the SGLang inference servers finished starting and registered with the dataflow pool. With no registered RaaS instance, EvalManager raised 'no healthy RaaS instance available for eval', the service caught it and skipped the eval, and the v=0 baseline was silently dropped (only 'Initial eval complete' printed, no eval-avg logged). Poll /status until >=1 RaaS instance is registered (300s cap) then let init settle before firing the pre-train eval. Verified in-container: the pre-train eval now waits for RaaSPool registration and runs (eval_start) instead of skipping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…training On sm_90 the Gated-DeltaNet backward must use fla's tilelang kernel (fla blocks its triton path on Hopper as numerically wrong, fla#640), and the tilelang wgmma JIT needs a full CUDA toolkit (nvcc + CCCL headers; the pip-shipped nvcc has none). Without these env vars the trainer crashes in the first backward on H100. Detect Hopper via nvidia-smi compute_cap in both Qwen3.5 trainer scripts and set FLA_TILELANG=1 and CUDA_HOME=/usr/local/cuda (only when unset, so user overrides win; no-op on Ada/L40 which stays on its validated default backend). Validated on 8xH100 in the v0.1.2 docker image: baseline overall avg@k 47.9 (matches L40 47.8), step 50 -> 57.8. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous commit (425221b) put the FLA_TILELANG/CUDA_HOME auto-setup only in 3_trainer_model0.sh, but the all-in-one run_*.sh scripts inline the trainer launch and never call that file — so the primary entry point still crashed in the first GDN backward on H100. Move the logic into a shared helper astraflow_setup_qwen35_hopper_env() in examples/_common/utils.sh and call it from all four Qwen3.5 entry points (both run_*.sh before the trainer launch, both 3_trainer_model0.sh replacing the duplicated block). Behavior unchanged: detect sm_90 via nvidia-smi compute_cap, default FLA_TILELANG=1 and CUDA_HOME=/usr/local/cuda only when unset; no-op on Ada/L40. Verified end-to-end in the astraflowai/astraflow:v0.1.2 image on 8xH100 with zero injected env: the launcher prints the Hopper banner, the GDN backward runs, and a full 400-step math-RL run completes cleanly (overall avg@k 48.4 -> 58.6, matching the reference curve).
- docker/README.md, installation.md: bump image tags v0.1.1 -> v0.1.2 and make the recommended Docker workflow explicit — the image provides the environment, a local checkout mounted over /workspace/astraflow provides the code (editable install picks it up; no rebuild for code changes). Quick Start now shows the full command with the repo mount, HF-cache mount, and WANDB key, plus the baked-code interactive mode as the alternative. Note the v0.1.2 images' 8xH100 end-to-end validation. - docs/en/recipes/math.md: add the missing Qwen3.5-4B section (run command, settings table, FSDP-only note, Hopper auto-env note). - README.md: add the 2026/07 v0.1.2 news entry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the Qwen3.5-4B (hybrid Gated-DeltaNet) math-RL recipes and moves the
validated stack to transformers 5.8.1 / sglang 0.5.13.post1 / torch 2.11
(18 commits, previously reviewed as PR #21 plus follow-up fixes).
New
qwen3.5-4b-m2po-full/-delta): M2PO, ctx 8k,full & delta TCP weight transfer. FSDP-only (mbridge has no
qwen3_5bridgeand Megatron-Core has no GDN layer spec — documented in the recipe READMEs).
attn_impl: kernels-community/flash-attn2).(mount over
/workspace/astraflow, no rebuild for code changes); recipes docgains the Qwen3.5 section; image tags synced to v0.1.2.
Fixes
chunked_prefill_size=8192for Qwen3.5 (SGLang rollout starvation → detokenizerhealth-check crash on H100 without it).
astraflow_setup_qwen35_hopper_env()setsFLA_TILELANG=1+CUDA_HOME(fla blocks its triton path on Hopper; the tilelang JIT needs nvcc + CCCL).
No-op on Ada/L40 and when the user pre-sets the vars.
eval_at_startno longer silently skipped: the pre-train eval now waits forRaaS-pool readiness (it used to fire ~40 s before the SGLang servers registered
and drop the step-0 baseline).
(flash-attn install, mbridge
rope_parameterspatch).Validation (8×H100, docker images
astraflowai/astraflow:v0.1.2*)400-step math-RL runs with sync eval every 50 steps (overall avg@k across
AIME24/AIME25/AMC/Minerva/MATH500; Megatron scope: AIME24/25 + MATH500):
i.e. the out-of-the-box workflow documented in
docker/README.md.