Repository navigation
AArch64 SHA-256: state in registers across blocks, vector digest output, initial hash value from a table - #2732
Merged
Merged
Conversation
…ters The feed-forward adds the copy in v30/v31 instead of storing the state and reloading it every block; the state is stored once, after the last block. Groups 14 and 15 load their constants into v3 each block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw
…able out256 writes the digest through v0/v1 (ldr q, rev32, str q) in place of three instructions a word, and initQ loads the initial hash value from a table (ldrqTbl) in place of building it with movz/movk, for vg_sha256_init, vg_sha224_init and the one-shot digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw
alex
marked this pull request as ready for review
October 18, 2026 05:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #991.
What
Three changes to AArch64 SHA-256, each doing the same work in fewer instructions.
1. Compression keeps the block's original state in registers.
vg_sha256_compress_sha2, which SHA-224, HMAC, HKDF and PBKDF2 call, no longer stores the state and reloads it after every block for the feed-forward:v30andv31(twomovs). The feed-forward adds them, and the state is stored once, after the last block.v30andv31held the constants of the last two groups of rounds. Those groups now load their constants straight intov3, the register the constants and schedule words are summed into, from the table already addressed byx4.ldr q+ 2str qof the state become 2 constantldr qs (off the SHA256H/H2 chain) and 2movs.This is aws-lc's
sha256_block_armv8arrangement of the state (v18/v19there), with our constants still kept in registers between blocks.2. The digest is written 16 bytes at a time (
out256):ldr q,rev32,str qofv0/v1. That's 6 instructions in place ofout32 8 true's 24 (ldr w,rev,str wper word). It applies to SHA-256'sfinalizeand one-shot digest, and so to HMAC-SHA-256's, PBKDF2-HMAC-SHA-256's and RSA-PSS-SHA-256's copies of them. SHA-224's 28-byte digest keepsout32 7 true.3. The initial hash value comes from a table (
initQ):adrp/add, twoldrqTbland twostr q, 6 instructions. It replaces 24 forvg_sha256_init/vg_sha224_init(movz,movk,str wper word) and 18 for the one-shot digests (movz/movkinto pairs,stp). The tables areVG_SHA256_H0andVG_SHA224_H0inconsts.rs.Generated files (all under
src/asm/aarch64/):sha256.rsconsts.rs: the two tableshmac_sha224.rs,hmac_sha256.rspbkdf2_sha224.rs,pbkdf2_sha256.rsrsa_pss_sha224_mgf1_sha224.rs,rsa_pss_sha256_mgf1_sha256.rsNo
TCB/orSpec/changes.Proof
Proof/Sha256/AArch64/Sha2/Compress.lean:VInvis the new loop invariant: the state is carried inv0/v1, and the memory is unchanged until the final store (final_ok).save_okandfold_okcover the copy and the feed-forward.rounds4_okhandles both kinds of group: constants fromkreg(sum4_ok), or loaded intov3(exec_kldTo).x4at the table in every block, the constant registers of the first 14 groups (kKept), andv30/v31.Proof/Sha256/AArch64/Stream/Out.lean:out256_ok, with the same postcondition asout32_ok (n := 8) true. The two 16-byte stores of the byte-reversed loads are the eight words' big-endian bytes (write_digest).Stream/Md.lean'sshapeuses it, and so does everything built on that shape: finalize, HMAC, PBKDF2 and the one-shot digests.Proof/Sha256/AArch64/Stream/Init.lean:initQ_correct.ldrqTblloads the table's pairs of words, which are the hash value's vectors (append_pairs), and two 16-byte stores write the state (write_iv).Stream/InitPairs.leangives the one-shot digests their facts aboutinitQ.taint_decide) of init, finalize, the one-shot digests, HMAC and PBKDF2 are unchanged and accept the new code.Performance
Benchmarks run 39098479382, against the merge base
50b726fe. Its scope is aarch64 (Neoverse N2),cpu_features='-',modules=sha256, which ran every benchmark that uses SHA-256. The hash, HMAC and PBKDF2 groups take the fastest of 8 binary copies each.sha256/64sha256/1024sha256/16384sha224/64/1024/16384hmac-sha256/64/1024/16384hmac-sha256-verify/64pbkdf2-hmac-sha256/*pbkdf2-hmac-sha224/*ecdsa_p256_sha256_sign/*Other rows: every other row (RSA-OAEP/PSS/PKCS#1, ECDSA over other curves and hashes, scrypt, EC public keys) is between 0.6% faster and 0.5% slower. The slowest is
ecdsa_p192_sha256_verifyat +0.5%; P-192's own code is unchanged, and it hashes a message with the faster SHA-256. No row is slower by more than 0.7%.Where the gain is: it is per call (digest output, initial value and the first block's state stores), not in the block loop. 16 KiB moves by 0.1%, inside the copies' spread.
Tests
lake build,Emit.lean --checkand theci/check_*.pyscripts pass. These tests pass under qemu (-cpu max), both by default and withVG_CPU_FEATURES=none(cargo test --release --target aarch64-unknown-linux-gnu):🤖 Generated with Claude Code
https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw