Skip to content

docs: the 27B recipe names its native CUDA backend and which formats it reads - #1146

Closed
jschmied wants to merge 1 commit into
ashhart:mainfrom
jschmied:docs-q27-cuda
Closed

jschmied wants to merge 1 commit into
ashhart:mainfrom
jschmied:docs-q27-cuda

Conversation

@jschmied

Copy link
Copy Markdown
Contributor

What this changes

docs/recipes/qwen3.8-27b.md only, two places:

Documentation only: no source, tests, fixtures or generated files change.

Receipt

  • Environment: TensorFold d1018b31 (main) plus this branch.
  • Checked against the source at d1018b31: a non-MLX 27B checkpoint gets UnsupportedQwenQuantization from zig/src/families/qwen3_5/cuda_geometry.zig (the native CUDA reader admits MLX affine tables in groups of 64 only). nvidia/Qwen3.8-27B-NVFP4 fails with it on a GB10.
  • Not run: nothing to run, documentation only.

Gotchas

…it reads

The page said the 27B has no CUDA backend; 1.1.0's native engine serves the MLX affine checkpoints
on CUDA. Its CUDA section, which promises NVFP4 and EXL3 checkpoints, describes the Python engine
(0.6.6); it now says so, and that neither reader is ported to the native engine yet.
@ashhart

ashhart commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Thank you. This is on main as d9107c1, with your authorship: the 27B recipe now says the native CUDA engine serves the MLX affine checkpoints (2 to 8 bits in groups of 64) with DFlash2 drafts and --kv-cache fp8, reads no NVFP4 or EXL3 checkpoint yet, and that its CUDA section describes the Python engine. It ships with the next release. Closing as landed.

- Synthema, Ash's AI Assistant.

@ashhart ashhart closed this Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants