Repository navigation
Conversation
…it reads The page said the 27B has no CUDA backend; 1.1.0's native engine serves the MLX affine checkpoints on CUDA. Its CUDA section, which promises NVFP4 and EXL3 checkpoints, describes the Python engine (0.6.6); it now says so, and that neither reader is ported to the native engine yet.
Owner
|
Thank you. This is on main as d9107c1, with your authorship: the 27B recipe now says the native CUDA engine serves the MLX affine checkpoints (2 to 8 bits in groups of 64) with DFlash2 drafts and - Synthema, Ash's AI Assistant. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
docs/recipes/qwen3.8-27b.mdonly, two places:zig/src/families/qwen3_5/cuda_*.zig, with DFlash2 drafts and--kv-cache fp8. The intro now says that, and that it reads no NVFP4 or EXL3 checkpoint yet.## CUDAsection says the 27B "serves NVFP4 and EXL3 checkpoints" on CUDA. That's the Python engine (0.6.6). The page already says further up that the rest of it describes the Python engine, but a reader who jumps to## CUDAdoesn't see that. A two-line note at the top of the section now says it's the Python engine's, and that neither reader is ported to the native engine yet (1.1.0 carries EXL3 checkpoints on the native engine (tracker) #832's shared EXL3 module, but the 27B doesn't read EXL3 through it yet).Documentation only: no source, tests, fixtures or generated files change.
Receipt
d1018b31(main) plus this branch.d1018b31: a non-MLX 27B checkpoint getsUnsupportedQwenQuantizationfromzig/src/families/qwen3_5/cuda_geometry.zig(the native CUDA reader admits MLXaffinetables in groups of 64 only).nvidia/Qwen3.8-27B-NVFP4fails with it on a GB10.Gotchas