Repository navigation
x86-64 Ed25519: the comb's AVX2 selection masks from one vpcmpeqd - #1360
Merged
Merged
Conversation
Each of the sixteen entries the AVX2 selection scans took six instructions to build its mask: combEqMask's four in the general registers, then vmovq and vpbroadcastq. The static now ends with the magnitudes 1 to 16, each in every doubleword of 32 bytes; the magnitude is broadcast to ymm14 once, and each entry's mask is one vpcmpeqd of it with the static's copy of the entry's magnitude. The proof is in Proof/Ed25519/X86_64/Ifma/CombSelect.lean (qword_pcmpeqd, vMagMask_ok, vselMag_ok); combWords is combTblWords ++ combMagWords, and the literal sizes of the static grow to 40448 bytes. From the analysis of the rustls provider's handshake signing in #991. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLVbycsRXKPfaW1swVZejG
alex
force-pushed
the
ed25519-comb-mask
branch
from
October 8, 2026 19:22
ac85696 to
4490b8b
Compare
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The comb's AVX2 selection (
Ifma.vselect, whichcombSelectY, the_adxvariants' selection, shares with the IFMA comb) scans all sixteen entries of a table and keeps each under a mask that is all ones exactly for the digit's magnitude. Each entry's mask took six instructions:combEqMask's four in the general registers (mov,xor,cmp,sbb), thenvmovqandvpbroadcastqto move it intoymm15. That is 96 of the 240 instructions of each scan, 52 selections per fixed-base multiplication.This builds each mask with one instruction:
VG_ED25519_COMBnow ends with the magnitudes1 … 16, each in all eight doublewords of 32 bytes (combMagWords, 512 bytes after the 26 tables).vselMagputs the static's address inraxand the digit's magnituder8into every doubleword ofymm14(vmovq,vpbroadcastd).vpcmpeqd ymm15, ymm14, [rax + 39936 + 32 (m - 1)], all ones in every doubleword exactly when the magnitude ism. Its address depends only onm.I also tried a counter in a register (
vpcmpeqdwith a register incremented byvpaddd, as the Weierstrass comb'sselEntryYdoes). It saved nothing on this core: the scan is bound by the vector ports, and the counter adds as many vector instructions as it removes in the general registers. With the selection's other registers taken (the IFMA comb's lanes live inymm0–ymm9), the memory operand also needs no free register.Proof
Ifma.vselect_okkeeps its conclusion, butymm14joins the registers it writes (selRegs; one caller's frame proof gains a case). The changes are inProof/Ed25519/X86_64/Ifma/CombSelect.lean:qword_pcmpeqd:pcmpeqdof quadwords holdingAandBin both doublewords is the mask ofA = B.vMagMask_ok(the compare against the static) andvselMag_ok(the setup).vselEntry_okandvselEntries_okcarryymm14and the magnitudes in the static (MagsAt) where they carriedrcx.vselect_okreads the magnitudes fromCombTbl, as the entries are read, throughcombWords_mag(Proof/Ed25519/X86_64/CombSelect.lean).combWordsis nowcombTblWords ++ combMagWords, socombWords_getDreads the tables through the append.The static grows from 4992 to 5056 words. The five witness states that write its size as a literal (per CLAUDE.md's "Tables of constants") now say 40448 bytes. The SSE selection (
combSelect, the baseline variants) and the AVX-512 one (zselect) are unchanged; they ignore the words after the tables. No change toSpec/orTCB/.Measurements
On a Sapphire Rapids Xeon restricted to
VG_CPU_FEATURES=avx,avx2,bmi1,bmi2,adx(the_adxvariants, as on a CPU without AVX512_IFMA), against #1358 (now merged), fastest of seven interleaved runs of each side (ci/bench_compare.py):ed25519_keygen/32ed25519_sign/64ed25519_sign/1024x25519_generate/32x25519_public_key/32Verification and the X25519 ladder, which don't select from the comb, moved by about 1% either way. With all of this CPU's features, the IFMA variants (whose fixed-base multiplication uses the
zmmselection) were within 2% either way.Why
From the analysis of the rustls provider's handshake signing in #991: on a CPU without AVX512_IFMA, Ed25519 signing was the gap against aws-lc-rs, and the fixed-base multiplication is nearly all of a signature's time outside SHA-512. The table scan was about a seventh of
vg_ed25519_scalar_base_adx; this removes a third of its instructions (each entry's fifteen become ten). X25519 key generation shares the comb and gains the same.🤖 Generated with Claude Code
https://claude.ai/code/session_01RLVbycsRXKPfaW1swVZejG