[jschmied]

What this changes

Next to norms and embed_rows (#736), the per-row kernels a hybrid attention/DeltaNet family needs between its
projections, as shared modules, operator-only (no model calls them yet), in the shape of #746:

• zig/kernels/cuda/act_rows.cu, zig/src/cuda/act_rows.zig: SwiGLU from [gate | up] rows, the sigmoid output
  gate (o * sigmoid(g) from [q | gate] heads), 64-value group sums of the stored bf16 outputs (for an FP8/NVFP4
  activation scale downstream), and DFlash2's two-tap dynamic conv over block segments, with an optional residual.
• zig/kernels/cuda/gdn_prep.cu, zig/src/cuda/gdn_prep.zig: Gated DeltaNet inputs per row: the causal conv window
  over the kept rows plus the new ones (up to 8 taps), SiLU, q/k L2 norms, v, g = exp(-exp(A_log) softplus(a)) and
  beta = sigmoid(b); key dims 64, 128, 256.
• zig/kernels/cuda/qk_rows.cu, zig/src/cuda/qk_rows.zig: QK RMS norm (fp32 or bf16 weights), RoPE (rotate-half,
  full or partial, from a cos/sin table or from inv_freq in fp32), and the K/V cache write, from packed or strided
  sources ([q | gate] rows, padded k), packed or head-major outputs.
• zig/src/core/gdn_prep_ref.zig, zig/src/core/qk_ref.zig, and swiglu/dconv in core/act_ref.zig: the float64
  references.

Every kernel sums a row in one fixed order, so a row's bits do not depend on how many rows run beside it.

Users we know of: the Qwen3.8-27B family in #608 (its target's MLP, attention output gate, GDN layers and q/k, and
its DFlash2 drafter's conv), and any CUDA family with GDN layers or gated attention.

Receipt

• Environment: TensorFold 46a08c18 (1.0.12) plus this commit; one DGX Spark (GB10, sm_121, CUDA 13). No checkpoint.
• Exactness, each at 1, 3, 17 and 64 rows with the bits of every smaller row count equal to the 64-row call's:
  • tf-cuda-test act-rows: SwiGLU at 17,408 and 192, output gate 24x256 and 3x64, group sums, dconv (5,120 in
    groups of 16, segments 16 and 7, with and without residual): worst 0.996 of the tolerance.
  • tf-cuda-test gdn-prep: 16/48 heads at 128 (the 27B), 2/3 at 64, 1/2 at 256: worst 0.995.
  • tf-cuda-test qk-rows: 48/4x128 full rotation, 48/4x128 none, 8/2x64 partial, 16/2x128 without norm, 24/4x256
    rotary 64 (the 27B), strided sources and head-major outputs bit-equal to the packed call: worst 0.988. The worst
    elements are the float64 value correctly rounded to bf16 just above a power of two; every element above 0.8 of
    the tolerance equals bf16_rn(want).
• Mutations, each failing its test: the sigmoid's exponent scaled by 1.0625 (38x the tolerance), DFlash2's conv
  reading across a segment start (1,804x), softplus switching to x above 2 instead of 20 (14,503x), the QK norm
  dividing by dim - 1 (1.43x; 1.95x in the dim-128 warp form), the conv window one tap short, and RoPE's rotate-half
  sign flipped.
• In a model (fork only, not in this PR): Qwen3.8-27B on CUDA: the qwen27 family's engine for NVIDIA GPUs, every stream drafted #608's 27B engine on these kernels with cuda: tree attention and DFlash block attention for every family, checked against float64 #746's attention, no Triton set:
  serial replies equal 0.6.6's except two at bf16 near-ties, drafted == serial, bench_concurrent --alone --serial
  12/12 cells exact; teacher-forced on 4,092 rows: NLL +0.0011 against the Triton set, argmax agreement 99.12 %,
  KL 0.00071. On an RTX 5090 (sm_120) every tf-cuda-test of these modules passed, mutations included
  (BobClawblaw, Qwen3.8-27B on CUDA: the qwen27 family's engine for NVIDIA GPUs, every stream drafted #608).
• Kernel contract (docs/kernel-contract.md), zig build kernel-contract -Dnvcc=/usr/local/cuda/bin/nvcc on the GB10
  (sm_121), exit 0:
  SUMMARY 35 kernels: 35 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems; gates 44467 ms, limits 1 ms
  • 23 new entries (zig/tests/cuda/contract_act.zig, contract_gdn.zig, contract_qk.zig): SwiGLU and the output
    gate, each with its 64-value sums of the stored outputs as their own entries; group sums; dconv with and without
    residual; tf_gdn_prep_64/128/256 each as q/k, v, and g/beta; tf_qk_rows_w in four forms (norm with table or
    inv_freq rotation, norm without rotation, no norm) and tf_qk_rows in three (dim 256 rotary 64 with table or
    inv_freq, dim 64 rotary 32 without norm).
  • Served shapes: the 27B's 17,408-wide MLP, its 24x256 output gate, 16 GDN key heads at 128 with 4 taps, 24/4x256
    q/k rotary 64; DFlash2's 5,120-channel conv in segments of 16; Kolibri's 48/4x128 q/k; at 1 row and 40 to 300
    rows, with the row edges at 16 (the decode block size switch), the 2,048-value block edges and head-dim edges.
  • Bounds: float64 references (core/act_ref.zig, gdn_prep_ref.zig, qk_ref.zig) through the contract's bound,
    fp32 sums with their real depth (8 + 3 for group sums, 2 for the conv, the conv taps for GDN), the fp32 op counts
    after them, and the bf16 rounding of the GDN conv before the norm (mids 2). Two entries keep the PR's own float64
    bound as a stated check, because the contract's fixed op count cannot carry them: dconv with residual (the conv is
    rounded to bf16 before the add, an error of the conv's size, not the output's) and GDN's g and beta
    (g = exp(-y) carries y times y's relative error).
  • Gate 4: fp16 sums miss by 205x to overflow in every summing entry. Where the output follows a sum of squares or is
    the conv's own bf16 rounding (q/k norms, GDN q/k and v), a bf16 sum stays within about an output ulp, so those
    specs state it (sums.bf16 = 0, with the reason), as the existing RMS-norm entry does.
  • Gate 5: 15 launches in registry.zig (the 256/128/64-thread forms launch.rowThreads picks, tf_qk_rows at dim
    64 and 256, tf_qk_rows_w), recorded from tf-cuda-test kernel-contract-limits over fatbins built for sm_80, 86,
    89, 120 and 121: every record current, 0 problems. Only sm_121 ran on a GPU.
  • Mutations, each turning the runner red, then restored: the QK norm dividing by dim - 1 (tf_qk_rows shapes and
    sums), dconv reading across a segment start (both dconv entries), softplus switching at 2 instead of 20 (all three
    g/beta entries).
• Speed in this PR: none claimed, nothing calls it.
• Tests run: zig build test (84/84 steps, 1,378 of 1,385 tests, 7 skipped) and zig build test-cuda-host (39/39
  steps, 729 of 729). tf-cuda-test norm-rows still passes on the shared rig. zig fmt --check clean. lean_check.py: 517
  findings, as on main.
• Not run, and why: Metal (no Metal kernels here, so no Metal SUMMARY); sm_80, 86, 89 and 120 only through their
  recorded compiles (gate 5), not on a GPU; no model on main uses it.

norm_rows_tests.zig exports its test rig (pub) so act_rows_tests.zig shares it instead of copying it.

Decode block sizes, and the arithmetic we keep

launch.rowThreads gives decode-sized calls (16 rows or fewer) the block sizes Triton's tuned glue uses for the
27B: swiglu 128 threads, the output gate 64, gdn_prep 64 (a warp a head either way). Prompt rows keep 256 and
128. A value's arithmetic and its group sums stay on the same lanes, so the bits are the same at every row count,
which the tests check (1 and 3 rows take the small blocks, 17 and 64 the large ones).

What is left is a choice, not a shape. On a GB10, one decode row, a call:

	before	Triton's block sizes	Triton
swiglu	2.66 µs	2.59 µs	2.11 µs
output gate	1.73 µs	1.67 µs	1.16 µs
gdn_prep	5.62 µs	5.45 µs (4.86 µs with its conv taps' loads issued first, the same bits)	4.11 µs

At the same shape the difference is the sigmoid's division. swiglu over 17,408 values in a CUDA graph: exactly
rounded __fdiv_rn with expf 1.16 µs, with __expf 1.15 µs, Triton's div.full and ex2.approx
(__fdividef, __expf) 0.86 µs; __frcp_rn gives __fdiv_rn(1, x)'s bits on all 65,536 bf16 inputs but 1.21 µs.
These kernels keep the exactly rounded division, so a value has the same bits on every GPU, at about 0.3-0.5 µs a
call (on an RTX 5090, about 1 % of a 4.8k-context decode). qk_rows keeps its 256-thread block, whose reduction
order the shape sets.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (No
  serving path changes.)
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[jschmied]

Rebased onto main 46a08c18 (1.0.12), now at 011c851c. The conflicts were only in the build and test registration lists, and the diff against main is otherwise unchanged. The GPU checks give the same numbers as before, and the description has the new test and lint counts.

[jschmied]

The kernels are now registered with the kernel contract in 31d3baf4 (23 entries, 15 launch records for sm_80 to sm_121). The CUDA SUMMARY line is in the description. Two entries keep this PR's own float64 bound as a stated check, because the contract's fixed op count cannot express a bf16 rounding partway through: dconv with residual, and GDN's g and beta. Some entries declare sums.bf16 = 0, as the RMS-norm entry does. Your call whether that is how you want them declared.

[ashhart]

Thanks @jschmied for the 23 entries and fifteen launch forms. The explicit residual and g/beta FP64 checks fit their intermediate rounding and input-dependent error; we retained their bounds.

The zero bf16-stress requirement means those particular inputs do not prove a bf16 miss. It does not permit changing production accumulation. We narrowed that wording and kept fp32 declarations and FP16 discrimination. Actual checker controls reject wrong finite values and non-finite erasure; disabling either finite-error guard makes its host test fail.

The increment can land with our prepared pending-record follow-up. Production kernels are unchanged. Final CUDA numerical, accumulation-mutation and resource qualification remain owed.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you, @jschmied, for the shared activation, Gated DeltaNet and QK/RoPE operators with their bf16 rounding points preserved.

We merged the reviewed head 011c851c9 for 1.1.0 in 55ddc11, retaining both sides of the registration overlaps without changing kernel arithmetic.

Your newer registration increment 31d3baf48 is for 1.1.1, the next patch release, with our paired checker and pending-record follow-up. Its residual and g/beta FP64 bounds stay unchanged; the bf16 stress wording describes those inputs rather than allowing lower-precision accumulation. All fifteen new launch forms await integrated receipts.

NVIDIA GB10/RTX PRO 6000 release contracts passed 55/55 with five controls red; these operator checks do not establish family adoption.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thanks @jschmied for #797's registration follow-up. 1.1.1 includes activation, DeltaNet and Q/K numerical coverage for fifteen launch forms. The explicit FP64 residual and gating bounds remain intact. This increment adds contract coverage without changing production arithmetic; limited bf16 stress discrimination does not permit lower-precision sums.

Posted by Ash's AI Assistant Synthena