Repository navigation
Conversation
cef0cfc to
76e88c0
Compare
|
refresh these benchmarks please |
|
Refreshed the benchmarks against current main, faef711, using this PR's provider/harness unchanged at 7aea648. Ran All ratios below are verified-garbage's speed relative to aws-lc-rs; higher is better. Handshake ranges span TLS 1.2 and 1.3 and are geometric means across cipher suites, after taking per-scenario medians.
Current resumed-handshake category ratios are 1.16–1.37× for TLS 1.3 and 1.13–1.56× for TLS 1.2. Selected provider-primitives medians (through rustls traits):
Improvements with relevant PRs landed since the recorded run
These are source-supported explanations, not isolated per-PR speedup measurements: I did not rerun 36ff93f on this same machine, so the exact software contribution is not separated from hardware. Differences likely driven substantially by CPUThe recorded run used a Cascade Lake-class Intel Xeon without IFMA, VAES, VPCLMULQDQ, or SHA-NI. This run used an AMD EPYC 9V74 with all four, pinned to CPU 2. Both providers dispatch differently.
Typical across-round handshake variation was about 3–4% (90th percentile roughly 5–7%, with larger outliers). Ratios close to 1×, including modest ChaCha bulk advantages, should be read cautiously. The full benchmark command completed successfully. These are native wall-clock measurements; no instruction-count measurements or separate correctness suite were run. |
7aea648 to
a0f24b5
Compare
|
Refreshed against main at The largest gains versus that comment are P-521 signing/verification (5.87× / 5.36×), RSA-3072 signing (3.41×), and Ed25519 verification (2.29×). P-256 and Ed25519 full handshakes are now faster than aws-lc-rs. Bulk record throughput is broadly unchanged. Ran Environment: AMD EPYC 9V74, pinned to CPU 2, with SHA-NI, IFMA, VAES and VPCLMULQDQ; Rust 1.99.0 ( TLS throughput relative to aws-lc-rs (higher is better). Medians per scenario, then geometric means across cipher suites; handshake ranges span TLS 1.2 and 1.3.
Provider primitives through rustls traits, median microseconds per operation. Speedup is the last comment’s VG time divided by current VG time; it is distinct from the final column’s comparison with aws-lc-rs.
Same-machine check against the old revision: I also reran
The same-machine primitive comparison confirms 5.89× / 5.37× for P-521 sign/verify, 2.30× for Ed25519 verification, and 3.36× for RSA-3072 signing. The direct-library benchmark also shows 1.35–1.45× faster AES-GCM sealing at 16 bytes and 1 KiB, despite broadly unchanged 16 KiB TLS bulk throughput. One adverse result: RSA-2048 private-key loading increased from 269.81 to 315.94 µs (+17% time), but the three-pass ranges overlap (old 268–296 µs; new 274–344 µs), so this is inconclusive rather than an established regression. Remaining gaps include P-384 verification/ECDH, P-256 ECDH, RSA verification, and bulk AES-GCM. The large-key RSA gain is consistent with #1096 adding the IFMA private-key path for 3072/4096-bit keys; recent P-256/P-384/P-521 and Ed25519 changes are also present. This measures their combined result, not isolated per-PR contributions. Measurement limits: these are native wall-clock measurements, with current and old revisions run in sequential blocks. Typical TLS scenario variation across samples was about 4–5%; differences below roughly 10% deserve caution. Current RSA-4096 signing varied from 1.69–2.51 ms across its three passes (median 2.26 ms, versus 6.24 ms in the last comment). The benchmark runs completed with their built-in handshake/verification/decryption checks; no separate correctness suite or instruction-count measurement was run. |
a0f24b5 to
f219344
Compare
|
Reran the complete benchmark suite against main Command: Environment: AMD EPYC 9V74, x86-64 Linux VM, pinned to CPU 2, with IFMA, SHA-NI, VAES and VPCLMULQDQ; Rust 1.99.0 ( Ratios are VG speed / aws-lc-rs speed; higher is better. Timing ratios are aws-lc-rs time / VG time. TLS values use the harness's median per scenario, then geometric means across cipher suites. Handshake ranges span TLS 1.2 and 1.3; P-384 is TLS 1.2 only. On this machine, VG full-handshake throughput is 15–35% higher for P-256 and 39–66% higher for Ed25519 than aws-lc-rs. RSA-2048 is near parity. TLS 1.3 bulk AES-GCM reaches 0.90–0.96× at 16 KiB, and ChaCha20-Poly1305 1.14×. RSA signing and private-key loading remain slower in the primitive measurements. TLS handshakes
Absolute full-handshake throughput (handshakes/s; geometric means across cipher suites):
TLS 1.3 bulkThroughput in MiB/s (the harness labels this
Provider primitives through rustls traits: all updated valuesMedian nanoseconds per operation across three passes. Time Δ = (VG − aws-lc-rs) / aws-lc-rs; negative means VG takes less time. The harness warms up and takes an internal median of seven batches for provider primitives or nine batches for direct-library measurements.
Direct-library primitives: all updated valuesMedian nanoseconds per operation across three passes. Time Δ = (VG − aws-lc-rs) / aws-lc-rs; negative means VG takes less time. The harness warms up and takes an internal median of seven batches for provider primitives or nine batches for direct-library measurements. Direct-library ECDH includes key generation on the aws-lc-rs side, as labeled. VG-only RSA breakdown rows isolate parts of verification and are not equivalent to aws-lc-rs whole verification; their comparative columns are omitted.
Direct-library AEAD size sweep: all updated valuesMedian nanoseconds per operation across three passes. Time Δ = (VG − aws-lc-rs) / aws-lc-rs; negative means VG takes less time. The harness warms up and takes an internal median of seven batches for provider primitives or nine batches for direct-library measurements.
Measurement limits: the PR's current Across the TLS samples, the median coefficient of variation was 6.3% for aws-lc-rs and 5.0% for VG; the 90th percentiles were 11.0% and 8.4%. RSA primitive timing also varies: VG provider RSA-2048 signing spans 257–354 µs (median 343 µs), RSA-4096 signing 2.26–2.50 ms (median 2.50 ms), and direct-library RSA-3072 signing 0.82–1.17 ms (median 1.14 ms). Differences around or below 10%, especially near parity, deserve caution. Bulk scenarios repeated across certificate types are pooled as in the existing summarizer. These are native wall-clock measurements; allocation/memory and instruction counts were not measured. The benchmark runs completed with their built-in handshake/verification/decryption checks; no separate correctness suite was run. |
f219344 to
42af3cd
Compare
Each of the sixteen entries the AVX2 selection scans took six instructions to build its mask: combEqMask's four in the general registers, then vmovq and vpbroadcastq. The static now ends with the magnitudes 1 to 16, each in every doubleword of 32 bytes; the magnitude is broadcast to ymm14 once, and each entry's mask is one vpcmpeqd of it with the static's copy of the entry's magnitude. The proof is in Proof/Ed25519/X86_64/Ifma/CombSelect.lean (qword_pcmpeqd, vMagMask_ok, vselMag_ok); combWords is combTblWords ++ combMagWords, and the literal sizes of the static grow to 40448 bytes. From the analysis of the rustls provider's handshake signing in #991. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLVbycsRXKPfaW1swVZejG
The AVX2 instance of vg_chacha20_poly1305_seal (and seal_gather, which calls it) absorbs each 512-byte chunk's ciphertext with the scalar Poly1305 while the AVX2 ChaCha20 kernel computes the next chunk's keystream, so out-of-order cores overlap the two (`sealStitched`, chosen by `sealFor`). Proven correct and constant time against the existing contracts; no TCB or spec change. From the rustls provider analysis in #991. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ma2fA9rUTPXEEQjYyCMCRw
All 224 tests against aws-lc-rs 1.18.1 on an AVX-512 IFMA Xeon, with the IFMA RSA public-key operation: verified-garbage is faster in 214, and the other 10 are within noise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAf76RQfGxGymBaZASPY7q
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAf76RQfGxGymBaZASPY7q
3 participants
What
A new package,
rustls-provider/(rustls-verified-garbage): a rustlsCryptoProviderbuilt on verified-garbage, with tooling to benchmark it against rustls's aws-lc-rs provider and to rerun those benchmarks.It is a separate package, like
bench/, with its own[workspace]. The library's tests, lints and coverage never build it. Nothing undersrc/,tests/,lean/,Spec/orTCB/changes. It builds against rustls from git at a pinned revision (91aebe5d).The provider
It covers what aws-lc-rs's provider does:
aes_ecb::AesEcb) or ChaCha20 (RFC 9001 §5.4).AES-GCM records of 512 bytes or more are sealed out of place with
AesGcm::encrypt(#1228), reading the payload where it is. Shorter ones, and ChaCha20-Poly1305 records, are copied and encrypted in place, which measured faster (seebench/RESULTS.md).Not supported:
Testing
bench/setup-rustls.shchecks out rustls and appliesbench/rustls.patch. The patch adds atests_with_verified_garbagemodule to rustls's API test suite, and averified-garbageprovider and aprovider-primitivesbinary torustls-bench.tests/records.rs: every suite seals contiguous and fragmented plaintext identically and opens what it sealed.cargo test --releasewithWYCHEPROOF_ROOTchecks every verification algorithm. Wycheproof has no vectors for P-521 with SHA-256 or SHA-384, so those two combinations are untested.Benchmarks
bench/run.sh [rounds]reruns everything, each benchmark pinned to one CPU:provider-primitives, which times each provider's primitives through the rustls traits;bench/aws-lc-compare, which times each library's primitives through its own API.bench/summarize.pyreduces the runs.bench/RESULTS.mdhas the runs, the analysis and the history.Every ratio is verified-garbage's speed relative to aws-lc-rs's (below 1 is slower). The detailed run is main at 93f5444, on a Xeon with SHA-NI, IFMA and VAES:
The latest run is on main at b5b6785, on a Cascade Lake-class CPU (no IFMA or VAES) and a noisy host:
Separate sessions are working on those in the library.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DAf76RQfGxGymBaZASPY7q