Repository navigation
send PROXY protocol v1/v2 on configured inbound ports - #4625
Draft
daniel-garcia wants to merge 2 commits into
Draft
daniel-garcia wants to merge 2 commits into
daniel-garcia wants to merge 2 commits into
Conversation
Linkerd exposes the verified client identity to HTTP applications via the l5d-client-id header, but has no equivalent for opaque TCP protocols: the application sees a connection from the loopback with no caller identity. This adds an opt-in mechanism to relay that connection metadata on the TCP forwarding path. When LINKERD2_PROXY_INBOUND_PORTS_PROXY_PROTOCOL_V2 (a comma-separated port/range list, empty by default) includes the target port, the inbound proxy prepends a HAProxy PROXY protocol v2 header to the connection it opens to the local application, before splicing bytes. The header carries the real client address in the standard PP v2 address fields and, when the connection was mutually authenticated, the verified client identity in a custom TLV of type 0xE0. Mixed address families are encoded by promoting the IPv4 address to its IPv6-mapped form. The header is written by a new SendProxyProtocol connector middleware (modeled on the outbound TaggedTransport) in the shared TCP forward stack, covering both the origin-destination path and the direct transport-header path. TcpEndpoint now carries the client address and server-side TLS status across the forwarding boundary to make that metadata available at connect time. HTTP proxying is unaffected. The control-plane configuration surface (annotation config.linkerd.io/proxy-protocol-v2-inbound-ports rendering this environment variable) lands separately in linkerd/linkerd2. See linkerd/linkerd2#15637 Signed-off-by: Daniel Garcia <dgarcia@infoblox.com>
Some applications only understand the text-based PROXY protocol v1. Add
LINKERD2_PROXY_INBOUND_PORTS_PROXY_PROTOCOL_V1, a comma-separated
port/range list (empty by default), alongside the existing v2 setting.
For listed ports the inbound proxy prepends a v1 header, e.g.
PROXY TCP4 10.1.2.3 10.9.8.7 33000 5432\r\n
to the connection it opens to the local application.
Version 1 has no extension mechanism, so the header carries the client
and server addresses only; the verified client identity is only
available through v2. As with v2, mixed address families are encoded as
TCP6 by promoting the IPv4 address to its IPv6-mapped form.
The v1 and v2 port sets must be disjoint: configuration parsing fails if
a port appears in both.
Signed-off-by: Daniel Garcia <dgarcia@infoblox.com>
daniel-garcia
changed the title
daniel-garcia
changed the title
daniel-garcia
changed the title
toidiu
reviewed
Oct 12, 2026
| /// address only; it cannot carry the client identity. | ||
| /// | ||
| /// Must not overlap with `proxy_protocol_v2_ports`. | ||
| pub proxy_protocol_v1_ports: RangeInclusiveSet<u16>, |
There was a problem hiding this comment.
Lets drop V1 for now since V2 supersedes it. We can evaluate adding it if a usecase comes up.
| /// established, iff the connection's target port is in one of the configured | ||
| /// port sets. | ||
| #[derive(Clone, Debug)] | ||
| pub(crate) struct SendProxyProtocol<S> { |
There was a problem hiding this comment.
nit: maybe ProxyProtocol or ProxyProtocolHeader
Comment on lines
+44
to
+45
| /// A Linkerd-specific TLV carrying the verified mTLS client identity. | ||
| const TLV_TYPE_CLIENT_ID: u8 = 0xE0; |
There was a problem hiding this comment.
-
TLV 0xE0 collides with Google Cloud PSC so we will need another one.
-
AWS seems to use subtype so it might be worth doing that here also.
[Type][Len][SubType + Data]
Type 1 byte vendor type AWS 0xEA Azure 0xEE
Length 2 bytes value length (big-endian, per PP2 spec)
Value 1 byte subtype AWS 0x01 = VPCE_ID Azure 0x01 = PRIVATEENDPOINT_LINKID
n bytes payload AWS: ASCII endpoint ID Azure: u32 little-endian LinkID
- We should document the known types we know about in a comment. So we should document AWS, Azure, GCP. I did a quick audit of other known usages and didnt find anything.
There was a problem hiding this comment.
Some of this should change once we remove V1 so i'll try and avoid comments around those sections for now.
| server: SocketAddr, | ||
| client_id: Option<&str>, | ||
| ) -> Vec<u8> { | ||
| let (family, src_bytes, dst_bytes): (u8, Vec<u8>, Vec<u8>) = match (client.ip(), server.ip()) { |
There was a problem hiding this comment.
This function is pretty critical but hard to understand. e.g. .octets().to_vec() is repeated many times.
Can we use some abstraction (struct or builder) to make this function cleaner?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Backend applications behind the inbound proxy see every opaque TCP connection as originating from the loopback, with no way to learn the real client address or the mTLS-verified client identity — the TCP counterpart of what
l5d-client-idprovides for HTTP. linkerd/linkerd2#15474 asks for opt-in PROXY protocol support on inbound TCP.Solution
Two opt-in port lists (comma-separated ports/ranges, empty by default) select which inbound ports get a HAProxy PROXY protocol header, prepended on the connection the inbound proxy opens to the local application before splicing bytes:
LINKERD2_PROXY_INBOUND_PORTS_PROXY_PROTOCOL_V2— binary v2 header:0xE0)LINKERD2_PROXY_INBOUND_PORTS_PROXY_PROTOCOL_V1— text v1 header (PROXY TCP4 10.1.2.3 10.9.8.7 33000 5432\r\n) for applications that only support v1. v1 has no extension mechanism, so it carries addresses only, never the identity. Mixed families are sent asTCP6with the IPv4 address in IPv6-mapped form.Implementation: a
SendProxyProtocolconnector middleware (modeled on outbound'sTaggedTransport) in the shared TCP forward stack, covering both the origin-destination path and the direct transport-header path, selecting the header version per target port;TcpEndpointnow carries the client address and server-side TLS status across the forwarding boundary. Opaque/TCP forwarding only — HTTP proxying is unaffected. Config parsing mirrorsENV_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION. No new dependencies. The control-plane surface (annotations → env vars) is linkerd/linkerd2#15676.Validation
13 new unit tests:
tokio_test::io)cargo test -p linkerd-app-inbound -p linkerd-appgreen;cargo clippy -p linkerd-app-inbound -p linkerd-app --all-targetsclean with-D warnings. Also exercised end to end on kind with a companion linkerd2 build: an application received the expected v1 line on a v1 port and a v2 header on a v2 port, with the identity TLV only for meshed clients.Note: the
fuzzersjob'slinkerd/app/inbound/fuzzbuild fails identically on cleanmain(cargo-fuzz 0.13.2 needs rustc 1.91 in the rust:1.90 container, andhttp/fuzz.rspredates the hyper 1.x migration) — unrelated to this change.WIP: opened for early feedback on the approach; the TLV type assignment and per-version behavior are up for discussion.
Part of linkerd/linkerd2#15474