Add parallel AES-NI CBC-decrypt kernel (single fused pass)#144
Merged
Conversation
CBC decrypt is P[i] = DEC(C[i]) xor C[i-1]: the inverse transforms are
independent, so decrypt the run 8-wide (x86_64) / 4-wide (i386) and fold
the chain XOR into that same pass. This replaces the mode's two passes
over memory (bulk aesdec write, then a separate XOR re-read) with one,
which lifts the DRAM-bandwidth ceiling that bounded large buffers.
TAesNiCbcKernel now serves both directions (branch on FDirection, as the
CCM kernel does); the factory probes TryGetDecKeysPtr for the inverse-
MixColumns schedule. New AesNiCbcDecryptWide_{x86_64,i386}.inc are kept
separate from the serial encrypt include -- the two share no structure
(serial 1-wide leaf vs parallel wide with an xmm save frame), only the
shared round chains. In-place safe with no stack spill: every input read
completes before the first output store. A 1-wide tail drains the 1..7
(1..3 on i386) residual blocks, so the kernel takes any block count.
TCbcBlockCipher resolves a direction-matched kernel at Init and routes
CbcDecryptBulk through it, falling back to the existing path when nil.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CBC decrypt is P[i] = DEC(C[i]) xor C[i-1]: the inverse transforms are independent, so decrypt the run 8-wide (x86_64) / 4-wide (i386) and fold the chain XOR into that same pass. This replaces the mode's two passes over memory (bulk aesdec write, then a separate XOR re-read) with one, which lifts the DRAM-bandwidth ceiling that bounded large buffers.
TAesNiCbcKernel now serves both directions (branch on FDirection, as the CCM kernel does); the factory probes TryGetDecKeysPtr for the inverse- MixColumns schedule. New AesNiCbcDecryptWide_{x86_64,i386}.inc are kept separate from the serial encrypt include -- the two share no structure (serial 1-wide leaf vs parallel wide with an xmm save frame), only the shared round chains. In-place safe with no stack spill: every input read completes before the first output store. A 1-wide tail drains the 1..7 (1..3 on i386) residual blocks, so the kernel takes any block count.
TCbcBlockCipher resolves a direction-matched kernel at Init and routes CbcDecryptBulk through it, falling back to the existing path when nil.