SME2 streaming-mode BLAKE2s - #20
Open
TomWambsgans wants to merge 2 commits into
Open
Conversation
CREDIT: https://github.com/zooko/blake3-sme2 M4 and later expose 512-bit streaming SVE2 behind SMSTART, so sixteen BLAKE2s lanes fit one register and `xar` fuses each xor with its rotation: a G function is ten instructions instead of sixteen. ZA carries both transposes, the message in as rows and out as columns, the digests out the same way. NEON is illegal inside streaming mode, so everything from the loads to the digest stores is hand-written. The width itself buys nothing. A streaming vector is four times a NEON one and issues at a quarter of the rate, and the two cancel: 251 GB/s of operand width per thread against 276 for one NEON core. What is left is the instruction saving, about 1.7x per thread. The block is shared by a whole core cluster and one thread saturates it, so extra streaming workers only take cores away from NEON: fmopa measures 3.94 G/s on one thread and 7.98 G/s on twelve, exactly the two performance clusters. Hence one slot per block, workers 0 and 1 plus the first efficiency worker, and NEON everywhere else. LEANVM_SME_WORKERS tunes it, 0 disabling the backend. A batch under sixteen inputs, and every remainder, still goes to NEON. Measured on an M4 Max, 2026-08-27, medians of interleaved runs: hash_bench multithreaded_throughput 441 -> 492 Mhash/s (+11.6%) aggregate --xmss 900 --log-inv-rate 1 1062 -> 1083 sig/s (+2.0%) The end-to-end gain is small because Merkle hashing is 14.5% of proving time and that stage is memory-bandwidth bound: it takes 0.198 s on four threads against 0.128 s on eleven, so it was never short of arithmetic. Same runs confirmed the pool defaults are already at their optimum, 11 performance workers against 1036 sig/s for 12 and 985 for 13, and all four efficiency workers against 995 with none. Also corrects the AGENTS.md claim that the M4 has no SVE. It has, in streaming mode, but with no 64-bit polynomial multiply: `pmullb z.q` faults there, which is what rules the field arithmetic out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The batch tail was the defect. Finishing a partial group on NEON meant leaving streaming mode mid-call, and that transition costs several hundred nanoseconds, far more than the handful of inputs it was there to hash, so any batch whose size was not a multiple of sixteen could be slower than plain NEON: at 31 inputs 1263 ns against NEON's 946, at 63 inputs 2149 against 1707. The remainder now rides a padded group instead, repeating the last input to fill the spare lanes, which is free because the kernel drives sixteen lanes either way. 31 inputs go to 589 ns and 63 to 1072, both now 1.6x ahead of NEON. A batch under one full vector still goes to NEON and never enters streaming mode at all. Between 17 and 24 inputs the two are within 0.8x to 1.0x of each other; that band is not worth a threshold fitted to noise, and the Merkle tiler does not land in it. `LEANVM_SME_WORKERS=0` did not do what its documentation said. `enabled` consulted `available` first, which probes by running the kernel, so the documented kill switch could not rescue a host that reports the feature but faults on SMSTART. The count is now checked before anything touches the hardware. Slot 0 needed a guard. `worker_id` returns 0 for the dispatcher and for every thread that is not a pool worker alike, so an off-pool thread hashing concurrently would have split the dispatcher's block, which is the one thing the slot list exists to prevent. `parallel::in_task` separates them, being set only inside a dispatch. A strictly sequential pool dispatches nothing, so that case is admitted explicitly, and it is where the kernel's own gain is visible: 40 -> 67 Mhash/s on one thread. The probe hashed sixteen copies of the same zero block, so it could not have caught a lane-routing error despite claiming to agree with the scalar backend. It now hashes sixteen distinct blocks. Two hazards are now recorded in the module documentation, both latent rather than live: a signal handler using Advanced SIMD takes SIGILL if it fires inside the kernel, because the handler runs with PSTATE.SM still set, and bare `smstart` enables ZA without committing a pending lazy save, which is safe only while nothing else in the process uses ZA. The assembly's own contract comment omitted that `len` must be nonzero, which the do-while loop requires and the Rust side already asserts. Measured again on the M4 Max, 2026-08-27, medians of interleaved runs: hash_bench multithreaded_throughput 452 -> 512 Mhash/s (+13%) aggregate --xmss 900 --log-inv-rate 1 1055 -> 1070 sig/s (+1.4%) That end-to-end figure corrects the +2.0% claimed in the previous commit, which sat at the optimistic end of the spread; six interleaved rounds put it at +1.4%, with five of six pairs favouring the backend. One thing is left alone deliberately. `pcs::merkle::BATCH_LEAVES` is `hash::LANES * 2`, which is 8 on aarch64 while both backends there consume 16 per whole batch (NEON interleaves four 4-lane groups), so some leaf shapes shed 8 leaves per tile to the slower path. It predates this work, it does not fire in any benchmarked configuration, and changing the tiler deserves its own measured commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CREDIT: https://github.com/zooko/blake3-sme2
a bit more than 15% performance gains on plaintext BLAKE2s hashing (multithreaded):
cargo test --release -p primitives --test hash_bench multithreaded_throughput -- --ignored --nocapture