Skip to content

x86-64 AES-GCM: encrypt slices after an unaligned one straight to the output - #1467

Merged
reaperhulk merged 2 commits into
mainfrom
gcm-stream-head
Oct 9, 2026
Merged

reaperhulk merged 2 commits into
mainfrom
gcm-stream-head

Conversation

@alex

@alex alex commented Oct 9, 2026

Copy link
Copy Markdown
Member

What

vg_aes_gcm_stream_encrypt_to, which vg_aes_gcm_seal_gather (AesGcm::encrypt) calls once per slice on its streaming path, encrypted a slice straight from src to dst only when the text before it ended a block. A slice after one that ends mid-block was copied to dst byte by byte and encrypted in place by vg_aes_gcm_stream_encrypt, slow path and all. For example, encrypt(&[&p[..1], &p[1..]]) copied almost the whole message one byte at a time.

It now starts with a head step. If the text so far ends inside a block, the (-text_len) mod 16 bytes that finish it (or all of len, if fewer) are copied and encrypted by vg_aes_gcm_stream_encrypt. Then the whole blocks after them go to vg_aes_gcm_encrypt_blocks_to straight from src, and at most 15 bytes are left for the copy. The head is skipped when the text so far plus the head would pass 2⁶⁴ bytes, so the text length stays exact. blocks now works past the bytes done instead of from offset 0. Every instance of stream_encrypt_to gets this, for every variant: it is generic code.

This is part of #1228. The workload is the one from the rustls analysis in #991 (encrypt with a few slices, reused key).

Measurements

AesGcm::encrypt timed with a reused key, 13-byte AAD and 12-byte nonce, on an AVX-512 Xeon (VAES, VPCLMULQDQ). Each cell is the best of 9 runs in ns, base → this PR. Texts under 768 bytes still take the copy-and-seal path, so this change doesn't affect them.

bytes [n] [1, n-1] [n-1, 1] [5, n-5] [100, n-100] three thirds
768 165 → 166 504 → 236 171 → 173 509 → 236 516 → 275 191 → 193
1024 182 → 182 615 → 250 187 → 188 618 → 247 619 → 290 603 → 359
1360 224 → 226 755 → 269 185 → 187 759 → 272 762 → 317 748 → 407
2048 259 → 258 1047 → 323 263 → 263 1051 → 323 1054 → 356 958 → 445
4096 376 → 375 1925 → 456 403 → 403 1937 → 461 1923 → 500 1547 → 563
16384 1201 → 1203 7176 → 1193 1211 → 1206 7170 → 1236 7079 → 1268 5257 → 1346

With a slice after one that ends mid-block, encrypt is now 2.1–6.0× faster: within 1.0–1.7× of a single slice instead of 2.5–6×. Inputs where every slice boundary is block-aligned ([n], [n-1, 1]) are unchanged.

Proofs

Assembly comes only from the Lean code. Spec, TCB and contracts are unchanged; the proofs are against the existing streamEncryptTo contract.

  • Steps.lean: per-block lemmas for the new code (headLen, headOver, headPtrs, headArgs, headDone, blocksLoad, blocksCount). blocksArgs now works from any offset.
  • BlkCall.lean, EncCall.lean: the calls of encrypt_blocks_to and stream_encrypt now take an offset and a count, not just "from 0" or "the rest".
  • Blocks.lean: call_mid and blocks_ok now start from Mid s o instead of Mid s 0.
  • Head.lean (new): head_ok, split into per-piece lemmas that the constant-time proof reuses. The head's byte count stays in the free slot work + 72 across its call.
  • CT.lean: head_rel, plus blocks_rel from any offset. Every branch in the head depends only on text_len and len, which are public.

Checks

  • lake build (all 13,546 jobs), then the emitter. Only src/asm/x86_64/gcm.rs changes.
  • ci/check_lean_speed.py, check_lean_imports.py, check_vectors.py, check_arch_gates.py, check_variants.py, check_mcdt.py, cpu_tests.py lint, algorithms_table.py.
  • cargo fmt --check, cargo clippy --all-targets -- -D warnings.
  • cargo test with Wycheproof, with all CPU features, and the AES-GCM tests again with VG_CPU_FEATURES=none.
  • backends_agree now also covers slices after one that ends mid-block: a slice shorter than the bytes that end the open block, one exactly that long, and one longer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FoYWDDEDFq1DFev5X7n2aq


Generated by Claude Code

claude added 2 commits October 9, 2026 11:46
… (WIP: asm not yet regenerated)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoYWDDEDFq1DFev5X7n2aq
@reaperhulk
reaperhulk added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 01a0740 Oct 9, 2026
40 checks passed
@reaperhulk
reaperhulk deleted the gcm-stream-head branch October 9, 2026 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants