Skip to content

Fix Docs MCP deployment reliability - #446

Merged
MuncleUscles merged 1 commit into
mainfrom
agent/fix-docs-mcp-reliability
Jul 29, 2026
Merged

Fix Docs MCP deployment reliability#446
MuncleUscles merged 1 commit into
mainfrom
agent/fix-docs-mcp-reliability

Conversation

@MuncleUscles

@MuncleUscles MuncleUscles commented Jul 29, 2026

Copy link
Copy Markdown
Member

What changed

  • add a protocol-level MCP smoke test that completes initialize and tools/list
  • run the canary tests before building the image and monitor production every 15 minutes
  • remove the obsolete, silently failing /web/jobs/scrape deployment step
  • recover automatically from an unusable derived SQLite index and emit useful diagnostics
  • document the build/deploy ownership boundary and required production guardrails

Root cause

The public nginx sidecar could report /healthz as healthy while the MCP container was unavailable, so infrastructure health was green even as MCP initialization returned HTTP 502. The image workflow also called a route that the current server does not expose and swallowed both failures, producing a false-success deployment signal. Production additionally uses a mutable latest image and keeps derived index state on a persistent volume, which makes rollouts non-reproducible and can preserve a bad index across restarts.

Impact

This makes broken MCP handshakes fail visibly in CI and on a schedule, ensures unusable derived index state is rebuilt, and gives the production rollout an immutable, testable artifact.

Validation

  • node --test docs-mcp/smoke-test.test.mjs
  • shellcheck docs-mcp/entrypoint.sh
  • actionlint .github/workflows/docs-mcp.yml .github/workflows/docs-mcp-health.yml
  • node --check docs-mcp/smoke-test.mjs
  • node --check docs-mcp/smoke-test.test.mjs
  • end-to-end smoke test against the production image running locally
  • verified the public endpoint currently fails the new canary with HTTP 502

Summary by CodeRabbit

  • New Features

    • Added automated health checks for the documentation service, including MCP initialization and tool discovery.
    • Added diagnostics and automatic index rebuilding when the documentation index is missing or unusable.
    • Added operational guidance for deployment, monitoring, troubleshooting, and recovery.
  • Bug Fixes

    • Improved error reporting when index verification fails or the service endpoint is unavailable.
  • Tests

    • Added automated coverage for successful MCP handshakes and upstream endpoint failures.
    • Added pre-release validation before publishing service updates.

@netlify

netlify Bot commented Jul 29, 2026

Copy link
Copy Markdown

Deploy Preview for genlayer-docs ready!

Name Link
🔨 Latest commit 72b6bbd
🔍 Latest deploy log https://app.netlify.com/projects/genlayer-docs/deploys/6a69d797bf05040007c52b16
😎 Deploy Preview https://deploy-preview-446--genlayer-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds an MCP SSE handshake smoke test with local fixtures, integrates it into build and scheduled health workflows, improves SQLite index failure diagnostics and rebuilding, and documents deployment, health-check, and incident-response procedures.

Changes

Docs MCP health and deployment

Layer / File(s) Summary
MCP handshake smoke test
docs-mcp/smoke-test.mjs, docs-mcp/smoke-test.test.mjs
Parses SSE events, performs MCP initialize and tools/list requests, validates advertised tools, handles timeouts and HTTP errors, and tests success and failure paths against local HTTP fixtures.
CI and production canaries
.github/workflows/docs-mcp.yml, .github/workflows/docs-mcp-health.yml
Runs the handshake canary for pull requests and builds, publishes images only on pushes, and schedules production health checks every 15 minutes with manual dispatch support.
Index failure diagnostics and rebuild
docs-mcp/entrypoint.sh
Reports package versions, removes unusable SQLite index artifacts, rebuilds the derived index, and emits diagnostics when verification fails.
Deployment and incident runbook
docs-mcp/OPERATIONS.md
Documents MCP deployment criteria, protocol health checks, Kubernetes triage, index recovery, immutable image guardrails, rolling deployments, and dedicated indexing jobs.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant SmokeTest
  participant DocsMCP
  GitHubActions->>SmokeTest: Run MCP canary
  SmokeTest->>DocsMCP: Open SSE endpoint
  DocsMCP-->>SmokeTest: Return messages URL
  SmokeTest->>DocsMCP: Send initialize and tools/list
  DocsMCP-->>SmokeTest: Return server info and tools
  SmokeTest-->>GitHubActions: Report pass or failure
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and clearly matches the PR's main focus on fixing Docs MCP deployment reliability.
Description check ✅ Passed The description covers what changed, root cause, impact, and validation, matching the template's required detail.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/fix-docs-mcp-reliability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@MuncleUscles
MuncleUscles marked this pull request as ready for review July 29, 2026 10:36

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
.github/workflows/docs-mcp.yml (1)

32-32: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Disable persisted checkout credentials.

Both workflows retain the GITHUB_TOKEN in the workspace Git configuration after checkout, although neither needs Git authentication afterward.

  • .github/workflows/docs-mcp.yml#L32-L32: set persist-credentials: false on checkout before running tests and the image build.
  • .github/workflows/docs-mcp-health.yml#L20-L20: set persist-credentials: false on checkout before running the canary.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/docs-mcp.yml at line 32, Disable persisted checkout
credentials in the checkout steps of .github/workflows/docs-mcp.yml at lines
32-32 and .github/workflows/docs-mcp-health.yml at lines 20-20 by setting
persist-credentials to false before the subsequent tests, image build, and
canary steps.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/docs-mcp-health.yml:
- Around line 22-23: Update the workflow step named “Initialize MCP and list
tools” to invoke the repository’s established failure-alert integration when the
scheduled canary fails, following the configuration and usage required by
docs-mcp/OPERATIONS.md. Preserve the existing smoke-test command and ensure
alerts are emitted only for workflow failure.

In `@docs-mcp/smoke-test.mjs`:
- Around line 84-90: Update the JSON-RPC response validation around the payload
handling in the smoke test to reject malformed responses: require
payload.jsonrpc to equal "2.0" and require exactly one of payload.result or
payload.error to be present. Preserve existing error propagation for valid error
responses, and add a regression fixture covering an incomplete response such as
{"id":1} so the canary cannot pass without a valid initialization exchange.
- Line 147: Validate the URL created in the smoke test’s messagesUrl flow before
any JSON-RPC POST, ensuring endpointEvent.data remains same-origin with sseUrl
or matches an explicit approved-origin allowlist. Reject disallowed origins and
do not send requests to them; preserve the existing flow for validated message
URLs.

---

Nitpick comments:
In @.github/workflows/docs-mcp.yml:
- Line 32: Disable persisted checkout credentials in the checkout steps of
.github/workflows/docs-mcp.yml at lines 32-32 and
.github/workflows/docs-mcp-health.yml at lines 20-20 by setting
persist-credentials to false before the subsequent tests, image build, and
canary steps.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6ccc4fee-e86c-48d9-8a5e-cef13a351bb7

📥 Commits

Reviewing files that changed from the base of the PR and between 5c1c29d and 72b6bbd.

📒 Files selected for processing (6)
  • .github/workflows/docs-mcp-health.yml
  • .github/workflows/docs-mcp.yml
  • docs-mcp/OPERATIONS.md
  • docs-mcp/entrypoint.sh
  • docs-mcp/smoke-test.mjs
  • docs-mcp/smoke-test.test.mjs

Comment on lines +22 to +23
- name: Initialize MCP and list tools
run: node docs-mcp/smoke-test.mjs https://docs-mcp.genlayer.com/sse

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Alert when the scheduled canary fails.

This only creates a failed Actions run; it does not notify an owning channel. Add the repository’s failure-alert integration so outages are actionable, as required by docs-mcp/OPERATIONS.md lines 47-55.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/docs-mcp-health.yml around lines 22 - 23, Update the
workflow step named “Initialize MCP and list tools” to invoke the repository’s
established failure-alert integration when the scheduled canary fails, following
the configuration and usage required by docs-mcp/OPERATIONS.md. Preserve the
existing smoke-test command and ensure alerts are emitted only for workflow
failure.

Comment thread docs-mcp/smoke-test.mjs
Comment on lines +84 to +90
const payload = JSON.parse(event.data);
if (payload.error) {
throw new Error(
`JSON-RPC response ${id} failed: ${JSON.stringify(payload.error)}`,
);
}
return payload;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Reject malformed JSON-RPC responses.

{"id":1} is accepted as a successful initialize response. If tools/list then succeeds, this canary passes without a valid initialization exchange. Require jsonrpc: "2.0" plus exactly one of result/error, and add a regression fixture.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs-mcp/smoke-test.mjs` around lines 84 - 90, Update the JSON-RPC response
validation around the payload handling in the smoke test to reject malformed
responses: require payload.jsonrpc to equal "2.0" and require exactly one of
payload.result or payload.error to be present. Preserve existing error
propagation for valid error responses, and add a regression fixture covering an
incomplete response such as {"id":1} so the canary cannot pass without a valid
initialization exchange.

Comment thread docs-mcp/smoke-test.mjs
(candidate) => candidate.event === "endpoint",
"the MCP message endpoint",
);
const messagesUrl = new URL(endpointEvent.data, sseUrl);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Do not POST to arbitrary SSE-advertised origins.

endpointEvent.data is remote-controlled. A compromised public SSE service can make the scheduled GitHub runner POST JSON-RPC requests to an arbitrary URL. Require the message URL to remain same-origin, or validate it against an explicit allowlist.

Suggested guard
     const messagesUrl = new URL(endpointEvent.data, sseUrl);
+    if (messagesUrl.origin !== sseUrl.origin) {
+      throw new Error("MCP message endpoint must use the SSE endpoint origin");
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const messagesUrl = new URL(endpointEvent.data, sseUrl);
const messagesUrl = new URL(endpointEvent.data, sseUrl);
if (messagesUrl.origin !== sseUrl.origin) {
throw new Error("MCP message endpoint must use the SSE endpoint origin");
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs-mcp/smoke-test.mjs` at line 147, Validate the URL created in the smoke
test’s messagesUrl flow before any JSON-RPC POST, ensuring endpointEvent.data
remains same-origin with sseUrl or matches an explicit approved-origin
allowlist. Reject disallowed origins and do not send requests to them; preserve
the existing flow for validated message URLs.

@MuncleUscles
MuncleUscles merged commit b7d0884 into main Jul 29, 2026
8 checks passed
@MuncleUscles
MuncleUscles deleted the agent/fix-docs-mcp-reliability branch July 29, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant