deps: switch reqwest from native-tls to rustls with aws-lc-rs - #39415
Conversation
Build reqwest 0.13 with its `rustls` feature (aws-lc-rs provider, platform verifier) instead of `native-tls-vendored`, and switch sentry from `transport` to `reqwest` + `rustls`. Sentry's `transport` feature enables `reqwest/native-tls-no-alpn`, and reqwest defaults to native-tls whenever it is compiled in, so leaving it would keep every reqwest 0.13 client on OpenSSL. The schema registry client (mz-ccsr) used native-tls-only APIs: * `Identity` now holds a PEM key and certificate chain instead of a PKCS #12 archive. `Identity::from_pem` checks that the key matches the leaf certificate up front, and the buffer is zeroized on drop. `Debug` no longer prints the key. * `Identity::from_pkcs12_der` is removed. testdrive, its only caller, converts its PKCS #12 keystore to PEM with OpenSSL. * `Certificate` parsing uses rustls instead of native-tls. mz-ccsr no longer depends on native-tls, openssl or mz-tls-util, and mz-storage-types drops its native-tls and openssl dependencies. The kafka-auth schema registry tests expect the rustls error strings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5895178 to
bdc9b59
Compare
QA LLM Review1. MEDIUM -- Existing schema registry connections break on upgrade when the registry's certificate is one OpenSSL accepts but webpki rejects
Schema registry TLS moves from OpenSSL to rustls/webpki, which is stricter about server certificates. A registry whose certificate has no subjectAltName (CN only), or which uses a self-signed DetailsI checked this against reqwest 0.13.5, rustls 0.23.45 and rustls-webpki 0.103.15, the versions this PR locks. I used
The kafka-auth fixtures ( This change is not listed under "Behavior changes". At minimum it needs a release note. A pre-upgrade check or a per-connection fallback would avoid breaking these connections silently. |
|
Addressed: self-signed |
…CA (#39459) ### Motivation This addresses the QA review finding on #39415. Under rustls, a self-signed `CA:TRUE` schema registry certificate supplied as its own `SSL CERTIFICATE AUTHORITY` fails with `CaUsedAsEndEntity`, though OpenSSL accepted it. Follows #39415. ### Description - When a ccsr client has configured root certificates, mz-ccsr builds the rustls config (aws-lc-rs) and hands it to reqwest with `tls_backend_preconfigured`. - A server certificate byte-for-byte identical to a configured root skips the chain, basic-constraints and key-usage checks, but its validity period and server name are still checked. The name is checked against SANs, falling back to any subject CN only when the pinned cert has no DNS or IP SAN. This is stricter than OpenSSL, which also falls back when only IP SANs are present, matches wildcard CNs, and decodes other CN string types. Validity is read with `x509-cert` because rustls-webpki exposes no validity accessor. - Every other certificate goes to `rustls-platform-verifier`, with the same native and extra roots reqwest uses. Handshake signatures are always verified. - The client identity moves into the rustls config. - Without configured roots, reqwest's default path is unchanged. Non-pinned CN-only leaves without a SAN are still rejected. ### Verification New `test_tls_server_verification` runs against a local TLS server. It accepts a SAN leaf and exact-match self-signed CAs matched by SAN or by any of several CNs, and rejects an exact match with the wrong SAN or CN, or with only an IP SAN, an expired or not-yet-valid exact match, a different self-signed cert, and a CN-only leaf. New `test_tls_client_identity_with_roots` covers mTLS with configured roots. Each check was confirmed to make its test fail when removed. A local differential run against `openssl verify -verify_hostname` (3.6.4 and 4.0.2) over 18 certificate shapes found no case we accept that OpenSSL rejects. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Jason Hernandez <7144515+jasonhernandez@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Motivation
Move reqwest 0.13 off OpenSSL (native-tls) onto rustls with the aws-lc-rs provider, as part of removing OpenSSL and
ringfrom the tree. Prior art: #35947.Description
rustlsfeature: aws-lc-rs plusrustls-platform-verifier, which trusts the system store on Linux and macOS.transporttoreqwest+rustls.transportenables reqwest's native-tls, and reqwest defaults to native-tls whenever it is compiled in, so every client would otherwise stay on OpenSSL.Identityholds the PEM key and certificates and passes them to rustls. Schema registry connections already take PEM (SSL KEY,SSL CERTIFICATE). The PKCS#12 archiveIdentityused to build from them was only an internal step for native-tls, so the SQL surface is unchanged.Identitychecks that the key matches the certificate, is zeroized on drop, and redactsDebug. It is built from secrets at connect time and never persisted.--certstill takes a PKCS#12 keystore and converts it to PEM itself. No CI job passes--cert.mz_ccsr::tls::TlsErrorreplaces the native-tls and OpenSSL variants ofCsrConnectError. mz-ccsr and mz-storage-types drop native-tls and openssl.mysql_async.Behavior changes
invalid peer certificate: UnknownIssuer. The kafka-auth.tdfiles are updated.keys may not be consistent: KeyMismatchinstead of an OpenSSL PKCS#12 error.CA:TRUEcertificate given as its ownSSL CERTIFICATE AUTHORITYfails withCaUsedAsEndEntity. Follow-up ccsr: trust a server certificate that exactly matches the configured CA #39459 accepts it by exact match. A certificate with only a CN and no subjectAltName now fails with a name error and must be reissued with a SAN. The same stricter validation applies toCOPY FROMURLs and the OIDC issuer fetch, with no exact-match fallback.Release notes
Schema registry connections and
COPY FROMURLs now reject server certificates that identify the host only by common name, without a subjectAltName. If a server uses such a certificate, reissue it with a subjectAltName that matches its hostname before upgrading.Verification
Locally:
bin/fmt, workspacecargo check, clippy on the touched crates, cargo-deny, lint-cargo, and ccsr and testdrive unit tests, including a newtest_pem_identity.CI: PR CI covers
kafka-auth(schema registry ssl, ssl-basic, mssl, mssl-basic),testdrive, and the environmentdauth/serverand balancerdservertests. Nightly (ci-nightly) coverstestdrive-old-kafka-src-syntax,cloudtestandorchestratord.🤖 Generated with Claude Code