[BlivionIaG]

Adds the LFM2.5 DSpark drafter using the released BF16 matrices by default, explicit 8-bit opt-in, and 4-bit refusal. Target verification keeps accepted output exact.

Rebuilt on main d33dcfc767ee093393c6bfa00b3949bd7b4d8e88. The rejected #1379/#1382 guard commits are excluded. No M5 was accessed for this restack. Fresh CI is running; maintainer re-review remains required.

Scope and review fixes

Includes clean #1390 taps and #1381 pricing. Removes the shared guard; fixes buffer-before-list failure cleanup with scratch/warm allocation-failure coverage; refuses rank >256 and nonfinite theta; widens quantizer arithmetic in Debug; refuses --drafter when local wider-lane qualification leaves width one. Keeps failure flags and kernel contracts.

Validation

Head 8e9b548fd. 67/67 scoped build steps, 353 tests passed and three skipped. The drafter reference and kernel-contract executables compile. Lean has zero new findings. BF16/8-bit device results from earlier receipts are historical until the fresh local replay is recorded.

Remaining performance requirement

The existing M5 integration loses about 1–2% at four/eight streams while gaining about 7–9% at 16. This restack does not claim to fix or waive that tradeoff, and Ash has not approved an exception to the no-slowdown requirement. The accepted task-owner tradeoff is not presented as maintainer approval. No stream-count/context guard is introduced.

Part of #1374. The chain’s intermediate branches are now updated rather than leaving fixes only in the cumulative tip.

The new cumulative M4 replay is now recorded on #1394: full source/hardware scope and results. It qualifies the final integration, not a separate run of each intermediate head.

[ashhart]

A small request, @BlivionIaG: this description links more than 20 issues and PRs, which is over Agent Traffic Control's limit, so ATC drops the PR from its board and collision checks (and marks every check incomplete for everyone). Could you trim the related-work list to the issues this PR actually fixes or depends on (20 at most)? Thanks.

[ashhart]

Thanks @BlivionIaG, a lot of careful work in this stack. We mapped all 31 commits to their PRs and reviewed each on current main. For the DSpark drafter (this PR and its descendants):

1. Loading changes the released BF16 weights. dspark_weights.zig quantizes each dense matrix to 4/8-bit codes at startup (8-bit by default), and #1393 still discards the original. Target verification protects final tokens, but the engine doesn't create lossy copies of a checkpoint at serving time. Please run the released BF16 matrices, or refuse the format. Also honour each advertised --drafter-bits mode or refuse it by name (the native adapter accepts the setting and then uses 8 regardless).
2. 4-bit mode reads past its buffer. With 4 bits the loader packs markov_w2 at that width, but lfm2_markov_scan (lfm2_dspark.metal) reads a byte per coefficient and the launcher passes no bit width. At 128000x256 the code buffer is 16,384,000 bytes and the shader can address byte 32,767,999 (found statically, not run on a GPU). Please refuse that mode before load or implement it correctly.
3. The held (asynchronous) path drops draft failure flags: the synchronous path refuses a NaN pick or invalid ID (DraftNonfinite), but readPicks after an unwaited hold ignores Pick.nonfinite and maps an invalid ID to 0, as does the GPU-held copy. Native serving enables that path, so please carry the flag through the fence and add planted NaN/Inf/invalid-ID controls for both paths.
4. Contract proof: the new operators report 0 planted controls and no recorded compiler/launch limits (the limit declarations use current-device checks with offline = false). Please add the gates, red controls, served shapes and records.
5. Config admission doesn't check hidden_act before the fixed SiLU path, and an explicit scheduled verify without a confidence head silently becomes depth mode; both should refuse by name.
6. The FP32 reference hard-codes a 6-bit target embedding and its host matmul joins workers only after all spawns succeed.
7. The device receipts are mostly on older bases; we'll need alternating A/B on current main per landing head (the stack's own notes report a 1.8-2.7% four-stream drop on Qwen3.8-27B + DFlash2 from the shared-lane changes, which needs explaining, not dismissing).

[BlivionIaG]

Thanks @ashhart. Pushed 35e9255c0 (main merged in).

1. Weights: with no flag, or --drafter-bits 0, the drafter runs the released bf16 matrices as they are. --drafter-bits 8 is an explicit option that quantizes them to 8-bit codes at load, named in --help and the startup log. 4 is refused on LFM2, 8 on other families. On the M5 Ultra, bf16 is 2.6% to 4.8% slower than 8-bit on one stream with about the same acceptance; numbers are in the body.
2. 4-bit mode: refused by name before load.
3. Held path: a NaN, Inf or invalid pick now fails with DraftNonfinite on both paths, with planted controls that fail without the fix.
4. Contract: the drafter's kernels (bf16 ones included) have planted controls and recorded limits. M4 Pro --only lfm2: 11 kernels pass, 17 of 17 controls red as planted. M5 Ultra at the stack head 2e5966f68: --only lfm2 11 kernels pass gates 1 to 4, 17 of 17 controls red as planted, 0 limit problems.
5. hidden_act other than silu is refused by name. There is no verify flag any more, so no scheduled-without-head case.
6. The fp32 reference reads the target embedding bits from the checkpoint and joins its workers on every path.
7. A/B on current main: in lanes: a lone stream holds drafts only while its drafted rounds measure faster #1382. Qwen3.8-27B with DFlash2 at 4 streams, M5, 6 rotated rounds, median of round medians: main 322.0 tok/s, a second build of the same main commit 318.8 (-1.0%), lane head 318.9 (-1.0%). Identical source differs as much as the lane head does, and the spread inside one cell is 304 to 333, so I did not reproduce the 1.8% to 2.7% gap; it follows the build or the session, not the code.

• Chips: an M3 Pro ran the stack head (@jschmied's receipt on lfm2: a cheaper DSpark hold (one concurrent encoder) #1394): tf-lfm2-dspark 9 of 9 drafts equal the fp32 reference, drafted replies equal "draft": false, 0 host allocations. M1 and M2: not run.

[ashhart]

Thanks @BlivionIaG. A lot of this is fixed, and fixed well:

• by default the drafter runs the released bf16 matrices as stored (only the q and k rows and their norm gains are
  reordered for the half-split RoPE);
• 4-bit is refused before load, so the markov_w2 overread cannot happen;
• the held path carries failures: lfm2_draft_ids sets a flag that runHeld reads before any cache commits, and
  readPicks refuses a NaN or out-of-range pick as the waited path does;
• a hidden_act other than silu is refused, and the fp32 reference reads the embedding width and joins its workers
  on every path;
• every new kernel has a launch record and planted controls (11 kernels, 17 controls at the stack head).

What still needs to change:

1. Drafting off by stream count. The drafter's facts set shared_guard = true, so served DSpark runs lanes: a shared round drafts only while drafting measures faster (Model.shared_guard) #1379's
   guard, which we are not taking (see lanes: a shared round drafts only while drafting measures faster (Model.shared_guard) #1379). Please remove it and let the lane core's pricing decide (lanes: an external drafter's chain chances price verify rows; holds follow the grants and pay for their pass #1381 with the
   confidence head, the depth rule without it). With the guard on, the stack head (bf16) is +17% to +23% on one
   stream, about even at 4 and 8 streams, and 1.2% slower at 16. Without the guard, --drafter must still be at
   least as fast as no drafter at every admitted width and context; we would also like drafting to pay at 4 to 16
   streams, so the hold's cost there is the thing to work on.
2. The timed fix is missing on this head. series/dspark-drafter carries lanes: a shared round drafts only while drafting measures faster (Model.shared_guard) #1379's first three commits but not
   4fa02928a, so with Config.timed = false the guard still keeps adaptive state; our table-only check fails here
   (expected 0 guard entries, found 1) and passes on lfm2: the DSpark confidence head schedules the verify #1392 to lfm2: a cheaper DSpark hold (one concurrent encoder) #1394. Removing the guard settles this too.
3. Drafter.init frees owned before walking it on an error. Errdefers run in reverse order, so
   d.owned.deinit(gpa) runs first and the for (d.owned.items) loop then reads freed memory. A failure in scratch()
   or warm() therefore walks freed memory (a minimal test of the same pattern segfaults under Debug) instead of
   refusing --drafter cleanly. One errdefer fixes it:
   errdefer { for (d.owned.items) |b| b.deinit(); d.owned.deinit(gpa); }. Please add a test that fails a scratch
   allocation or warm().
4. Config admission. markov_rank: 512 is admitted, but lfm2_markov_scan stages W1's row in
   threadgroup float e[256], so any rank over 256 writes past it; rope_theta: 1e100 becomes an infinite f32 and
   passes theta > 0. Please add rank > 256 to the UnsupportedDraftShape check and !std.math.isFinite(theta) to
   BadDraftConfig, with both cases in the refusal test.
5. Quantizer test. 64 * bits is computed in u8 at dspark_quant.zig:88 and overflows at 8 bits. The test passes
   only because zig build test builds the engine tests at -Doptimize=fast; under Debug it panics.
   64 * @as(usize, bits) / 8 fixes it.
6. --drafter after a failed wide check. If the local wider-lanes check fails, Metal.init keeps one-row rounds
   and Drafted.facts then gives zero drafts, so the server loads the drafter and silently never drafts. Please refuse
   --drafter by name when the backend's width is 1.
7. --drafter-bits 8 (LFM2.5-24B-A2B on Metal: drafts with LiquidAI's DSpark drafter (proposal and gate) #1374, Question 2). An opt-in 8-bit drafter is fine. It only changes what the drafter
   proposes; the target's verify keeps every token exact. That matches --drafter-bits for the other families' drafters
   on main. Please make sure drafted still equals plain with it, and keep bf16 the default.

Once these are in and the stack is rebased on current main without #1379 and #1382, we will run drafted against plain,
concurrent against solo and the speed cells on our Mac.

[BlivionIaG]

@ashhart Rebuilt on current main d33dcfc76, including clean #1390 taps and #1381 pricing, at 8e9b548fd. The shared guard is gone. The cleanup now frees buffers before their list, with scratch/warm allocator-failure coverage; config refuses rank >256 and nonfinite theta; quantizer arithmetic is widened under Debug; width-one qualification refuses --drafter explicitly. BF16 remains the default, 8-bit is opt-in, and 4-bit is refused. Held-path failure flags and kernel contracts are retained. Local scoped checks: 353 pass / 3 skip, zero new lean findings. The downstream 1–2% slowdown at four/eight streams remains disclosed rather than claimed fixed or waived; the prior larger 16-stream gain does not satisfy your all-width performance requirement. No M5 was accessed for this restack. Fresh CI is running.