Skip to content

fix(mcp): pin outbound connections to IPv4 to avoid unreachable-IPv6 hangs#5798

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/mcp-pin-ipv4-egress
Jul 21, 2026
Merged

fix(mcp): pin outbound connections to IPv4 to avoid unreachable-IPv6 hangs#5798
waleedlatif1 merged 2 commits into
stagingfrom
fix/mcp-pin-ipv4-egress

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

The real root cause (empirically confirmed)

The MCP "connecting forever" / tool-discovery 30s timeouts in production — affecting Gauge, Exa, and PlanetScale alike — were caused by IP-family pinning, not OAuth or HTTP/2 (which the earlier PRs fixed but were different layers).

The chain, every link verified against production:

  1. SSRF protection pins each outbound connection to a single resolved IP (createPinnedLookup), which strips Happy Eyeballs' IPv4 fallback.
  2. dns.lookup(host, { verbatim: true }) returns the IPv6 address first for Cloudflare-fronted dual-stack hosts — verified: app.withgauge.com2606:4700:…, api.exa.ai2606:4700:….
  3. Production's app subnets have zero IPv6 egress — no IPv6 CIDR, IPv4-only via NAT Gateway (AWS NAT Gateways are IPv4-only). Verified on both subnets + route tables.
  4. So the connection pinned to IPv6 connects into a void and hangs → the SDK's 30s timeout fires (AbortError, durationMs=30002). Intermittent because the resolver rotates A/AAAA order (works on retry when it picks IPv4).

This explains everything that didn't add up: why #5797 (h1.1) didn't fix it (protocol-independent), and why it never reproduced locally (dev machines have IPv6 egress).

Empirical proof (undici, pinned lookup):

pin IPv6-blackhole ONLY (current behavior)   → FAIL after 8001ms (TimeoutError)   ← the prod hang
prefer IPv4 / include IPv4 in the set        → status 200 in ~600ms               ← reachable

Fix

At both pinned-resolution sites — validateMcpServerSsrf (MCP) and validateUrlWithDNS (providers / A2A / SMTP, same latent bug) — resolve all addresses ({ all: true, verbatim: true }) and prefer an IPv4 address; IPv6-only hosts still pin their sole address. This keeps the single-IP pinning API and threading untouched (near-zero blast radius) and SSRF validation of the pinned IP is unchanged.

Verified end-to-end: app.withgauge.com and api.exa.ai now pin their IPv4 address instead of the unreachable IPv6.

Type of Change

  • Bug fix

Testing

  • Full lib/mcp suite green (374); lib/core/security green; tsc + biome clean.
  • Added tests: dual-stack host prefers IPv4; IPv6-only host pins its sole address.
  • The definitive confirmation is re-testing on staging/prod once deployed.

Note

A fuller fix (pin the validated set of all resolved IPs and let undici do Happy Eyeballs, so it's egress-agnostic) is a good follow-up, but it threads string → string[] through ~10 security-critical sites; preferring IPv4 fixes the confirmed bug now with minimal risk.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…hangs

Root cause of the MCP 'connecting forever' / tool-discovery timeouts in
production: SSRF pinning forces each outbound connection to a single resolved
IP, which strips Happy Eyeballs' IPv4 fallback. dns.lookup(verbatim) returns the
IPv6 address first for Cloudflare-fronted dual-stack hosts (Gauge, api.exa.ai),
so the connection was pinned to IPv6 — but the production app subnets have no
IPv6 egress (AWS NAT gateways are IPv4-only), so the pinned IPv6 connection
connects into a void and hangs until the 30s timeout. Intermittent because the
resolver rotates A/AAAA order; works on retry when it happens to pick IPv4.

This is why h1.1 (#5797) did not fix it (protocol-independent) and why it never
reproduced locally (dev machines have IPv6 egress). Empirically verified: pinning
only the IPv6 address times out; preferring the IPv4 address connects in ~600ms.

Fix: at both pinned-resolution sites (validateMcpServerSsrf and validateUrlWithDNS)
resolve all addresses and prefer an IPv4 one; IPv6-only hosts still pin their sole
address. No signature/threading changes; SSRF validation of the pinned IP is
unchanged.
@vercel

vercel Bot commented Jul 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 21, 2026 5:13am

Request Review

@cursor

cursor Bot commented Jul 21, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches security-critical DNS pinning for MCP and general outbound URL validation; behavior change is narrow (address family preference) but affects all pinned connections to dual-stack hosts.

Overview
Fixes production MCP connection hangs (~30s timeouts) on dual-stack hosts when SSRF pinning chose an IPv6 address first (dns.lookup with verbatim: true) while egress is IPv4-only (e.g. AWS NAT), so pinned outbound traffic never completed.

validateMcpServerSsrf and validateUrlWithDNS now resolve all addresses ({ all: true, verbatim: true }) and prefer an IPv4 entry for the pinned IP; IPv6-only hosts still use their sole address. SSRF checks still run on the chosen address; only selection order changed.

Tests mock the multi-record lookup shape and add cases for dual-stack (IPv4 preferred) and IPv6-only pinning.

Reviewed by Cursor Bugbot for commit a866409. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR changes pinned outbound connections to prefer IPv4 while preserving IPv6-only resolution. The main changes are:

  • Resolve every address before selecting the pinned IP.
  • Prefer IPv4 for dual-stack MCP and general outbound URLs.
  • Add MCP tests for dual-stack and IPv6-only hosts.

Confidence Score: 5/5

This looks safe to merge.

  • Both validation paths keep the existing SSRF checks and pinning behavior.
  • IPv6 remains available when DNS returns no IPv4 address.
  • No additional blocking issue was found in the updated code.

Important Files Changed

Filename Overview
apps/sim/lib/core/security/input-validation.server.ts Resolves all DNS addresses and prefers IPv4 for pinned outbound requests.
apps/sim/lib/mcp/domain-check.ts Applies IPv4-preferred address selection to MCP SSRF validation.
apps/sim/lib/mcp/domain-check.test.ts Updates DNS mocks and tests dual-stack preference and IPv6-only fallback.

Reviews (2): Last reviewed commit: "chore(mcp): trim inline comments on the ..." | Re-trigger Greptile

Comment thread apps/sim/lib/core/security/input-validation.server.ts
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a866409. Configure here.

@waleedlatif1
waleedlatif1 merged commit d381eef into staging Jul 21, 2026
20 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/mcp-pin-ipv4-egress branch July 21, 2026 05:29
waleedlatif1 added a commit that referenced this pull request Jul 21, 2026
…5799)

Same class of bug fixed for MCP in #5798: SSRF pinning forces a single resolved
IP, which strips Happy Eyeballs' IPv4 fallback. dns.lookup(verbatim) returns the
IPv6 address first for dual-stack hosts, so a database host or 1Password Connect
server that is dual-stack gets pinned to IPv6 — unreachable on IPv4-only egress
(AWS NAT gateways) and hangs until timeout.

validateDatabaseHost and validateConnectServerUrl now resolve all addresses and
prefer IPv4; IPv6-only hosts still pin their sole address. SSRF validation of the
pinned IP is unchanged (the selected address is the one validated and pinned).
waleedlatif1 added a commit that referenced this pull request Jul 21, 2026
…lower

- Restore validateUrlWithDNS's prefer-IPv4 resolution (a stale working-tree hunk
  accidentally reverted #5798 for the still-pinned non-MCP consumers).
- Validate the INITIAL url's IP-literal in followRedirectsGuarded, not just
  redirect hops, so the exported guard is self-contained.
- Drop entity headers (content-length/type/encoding) when a 301/302/303 switches
  a POST to a bodyless GET, which undici would otherwise reject.
- Lift method/headers/body/signal from a Request input instead of silently
  downgrading a guarded POST Request to a bare GET.
waleedlatif1 added a commit that referenced this pull request Jul 22, 2026
…rd (#5823)

* fix(mcp): replace single-IP pinning with validate-at-connect SSRF guard

Swaps the MCP transport + OAuth guard from pin-one-resolved-IP to the standard
pattern (LibreChat getSSRFConnect): DNS resolves normally and EVERY socket
connect filters the resolved addresses against the private/reserved blocklist,
closing the validate-then-trust window a rebind could race and removing the
non-standard mechanism implicated in the headers-then-no-body stalls (no
reference MCP client pins IPs; a pinned attempt welded to a bad flow cannot
escape, while retries rotate via resolver round-robin and succeed).

Adversarially reviewed before shipping; both findings closed here:
- Redirects are now followed manually with per-hop validation: an IP-literal
  redirect target (which bypasses ANY connect-time lookup - Node skips the
  custom lookup for numeric hosts) is checked explicitly, closing a metadata-
  endpoint redirect hole the old pin also had.
- Custom request headers are dropped on cross-origin hops, so a redirect to a
  second attacker host cannot harvest configured auth headers.

Policy gating unchanged: the guard activates exactly where the pin did
(validateMcpServerSsrf non-null; allowlist mode / localhost-on-self-hosted stay
unguarded). Non-MCP consumers of the pinned fetch are untouched. 397 tests
incl. new rebinding, mixed-answer, IP-literal-redirect, and hop-cap coverage.

* fix(mcp): restore IPv4 preference and harden the guarded redirect follower

- Restore validateUrlWithDNS's prefer-IPv4 resolution (a stale working-tree hunk
  accidentally reverted #5798 for the still-pinned non-MCP consumers).
- Validate the INITIAL url's IP-literal in followRedirectsGuarded, not just
  redirect hops, so the exported guard is self-contained.
- Drop entity headers (content-length/type/encoding) when a 301/302/303 switches
  a POST to a bodyless GET, which undici would otherwise reject.
- Lift method/headers/body/signal from a Request input instead of silently
  downgrading a guarded POST Request to a bare GET.

* fix(mcp): annotate headers double-cast and cancel redirect body before throw paths

- Add the missing double-cast-allowed annotation on the sanitized-headers cast
  (strict boundary audit).
- Cancel the redirect response body before the hop-cap / blocked-target throws
  so those paths can't leave a socket checked out on the long-lived Agent.

* fix(mcp): keep self-hosted private-resolving hosts unguarded (old-pin parity)

A DNS alias resolving to loopback/private is only reachable on self-hosted,
where the policy explicitly permits it; the guarded lookup would filter the
address and strand the connect where the old pin connected. Both MCP gates
(transport + OAuth guard) now route private/loopback resolutions over the
unguarded path, same as the localhost carve-out. Test IPs moved off RFC-5737
TEST-NET (which is correctly classified reserved).

* fix(mcp): pin (not skip) self-hosted private resolutions; refuse cross-origin body forwards

- The private/loopback carve-out now keeps the LEGACY PIN to the validated
  address instead of falling back to unguarded fetch — preserving both the old
  behavior and its anti-rebinding property for self-hosted DNS aliases.
- Cross-origin redirect hops now also refuse to forward a request body (307/308
  preserve method+body; post-pin those redirects really dial the new origin, so
  an open redirect could exfiltrate OAuth client secrets). Bodyless cross-origin
  redirects still follow. Tests for both.

* chore(mcp): true up stale pinned-era doc comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant