Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions deploy/native/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,11 @@ driven by the dashboard's Statuspage pusher, which keeps running on every host.

Both hosts set `MCP_SERVE_METRICS=1` (hardcoded in the unit), so the app serves
the Prometheus exposition at `/metrics` on its own port for a scraper to reach on
the host's private address. Caddy answers `/metrics` on the public origin with a
404, which is what keeps it off the internet.
the host's private address. Caddy answers `/metrics` with a 404, which is what keeps
it off the internet. `deploy.sh` asserts that 404 against Caddy on the host itself,
since a public name behind a fronting edge (production's is) answers with the edge's
response rather than Caddy's; it separately fails the deploy if the exposition itself
ever shows up through the public name, redirects followed.

The mechanics live in [`deploy-native.yml`](../../.github/workflows/deploy-native.yml),
a reusable workflow both call. It first runs the status dashboard's unit tests (a
Expand Down
35 changes: 22 additions & 13 deletions deploy/native/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -274,27 +274,36 @@ echo ">> external check:"
curl -sS --max-time 20 -o /dev/null -w "https://$DOMAIN/ -> HTTP %{http_code} (TLS verify %{ssl_verify_result})\n" "https://$DOMAIN/" || true
curl -sS --max-time 20 -o /dev/null -w "https://$DOMAIN/status/ -> HTTP %{http_code}\n" "https://$DOMAIN/status/" || true

# ...and /metrics must NOT be reachable on the public origin. Caddy answers it
# with a 404; losing that one block would publish the exposition, so assert it on
# every deploy. The assertion is "exactly the configured 404": any other code —
# a proxied 500, a `000` from an unreachable or stalled origin — has not disproved
# exposure, so it retries (Caddy may be reloading) and then fails hard.
# /metrics must NOT be reachable on the public origin: the Caddyfile answers it
# with a 404, and losing that one block would publish the exposition. Ask Caddy
# itself, from the host, with $DOMAIN pinned to loopback so its site block
# answers: production's public name now resolves to a fronting edge that
# redirects every non-MCP path, so the public answer says nothing about this
# host. -k because the question is the route, not the certificate.
hidden=""
for attempt in 1 2 3 4 5; do
code="$(curl -sS --max-time 20 -o /dev/null -w '%{http_code}' "https://$DOMAIN/metrics" || echo 000)"
if [ "$code" = 200 ]; then
echo "FATAL: https://$DOMAIN/metrics answered 200 — the Prometheus exposition is public" >&2
exit 1
fi
code="$($SSH "curl -ksS --max-time 10 -o /dev/null -w '%{http_code}' --resolve '$DOMAIN:443:127.0.0.1' 'https://$DOMAIN/metrics'" || echo 000)"
if [ "$code" = 404 ]; then
hidden=1
echo "https://$DOMAIN/metrics -> HTTP 404 (not published, as intended)"
echo "caddy on the host: https://$DOMAIN/metrics -> HTTP 404 (not published, as intended)"
break
fi
echo "https://$DOMAIN/metrics -> HTTP $code, expected 404 (attempt $attempt)" >&2
echo "caddy on the host: https://$DOMAIN/metrics -> HTTP $code, expected 404 (attempt $attempt)" >&2
sleep 3
done
if [ -z "$hidden" ]; then
echo "FATAL: https://$DOMAIN/metrics never answered the configured 404, so its exposure is unproven" >&2
echo "FATAL: Caddy never answered the configured 404 for /metrics, so its exposure is unproven" >&2
exit 1
fi

# Through the public name too, redirects followed: the exposition's own marker
# in the body is what proves it public, whatever sits in front. A status alone
# proves nothing either way (an edge's 200 page, a redirect elsewhere).
public="$(mktemp)"
summary="$(curl -sSL --max-redirs 5 --max-time 20 -o "$public" -w '%{http_code} at %{url_effective}' "https://$DOMAIN/metrics")" || summary="${summary:-000} (request failed)"
if grep -q imcp2_build_info "$public"; then
echo "FATAL: https://$DOMAIN/metrics serves the Prometheus exposition publicly ($summary)" >&2
exit 1
fi
rm -f "$public"
echo "https://$DOMAIN/metrics -> $summary; no exposition in the body"
Loading