feat: MCP client support — expose external MCP server tools to agents

[kokhlo]

Motivation

Closes #296.

PentAGI agents currently reach external security tooling only through the built-in terminal/browser/file tools. Many tools teams already use (Burp Suite Pro, Nuclei, Shodan, custom internal scanners) increasingly ship Model Context Protocol servers. This PR makes PentAGI an MCP client: tools advertised by external MCP servers are discovered at startup and offered to agents as first-class tools, next to the built-in ones.

What is added

• backend/pkg/mcp — a small MCP client on top of the official modelcontextprotocol/go-sdk:
  • transports: stdio (local command managed as a PentAGI child process, with optional extra env), http (streamable HTTP) and sse (legacy servers);
  • tools are namespaced as mcp_<server>_<tool> (sanitized to [a-z0-9_], capped at 64 chars — the strictest provider function-name limit), so they can never collide with or shadow built-in tool names;
  • an unreachable server or a failed tool discovery is logged as a warning and skipped — the rest of the fleet keeps working and PentAGI starts normally;
  • tool-level errors (IsError results) are surfaced as Go errors so the existing tool-call fixing flow can react.
• backend/pkg/config — new settings following the existing env conventions (see backend/docs/config.md for the full reference):
  • MCP_SERVERS — JSON array of server definitions (name, transport, url/command+args+env, optional contexts);
  • MCP_ALLOWED_TOOLS / MCP_DENIED_TOOLS — comma-separated selectors (*, <server>, <server>/*, <server>/<tool>), deny always wins;
  • MCP_TOOL_TIMEOUT — per-call deadline in seconds (default 120, 0 = no deadline).
• backend/pkg/tools — integration with the agent tool registry:
  • discovered MCP tools are appended to the assistant, primary, coder, installer, searcher and pentester executors (same pattern as the optional browser/graphiti/web-search tools);
  • per-server contexts restricts visibility to specific agents (primary_agent, assistant, coder, installer, searcher, pentester);
  • the server-announced JSON Schema is passed through as the function parameters;
  • mcp_* tool names map to EnvironmentToolType for observability; the shared client is closed on graceful shutdown.
• Docs — new section in README.md (Advanced Setup → External Tools (MCP)), full reference in backend/docs/config.md, .env.example block, and the docker-compose.yml env mapping.

Design notes / open questions

1. Config format — the issue sketch proposed per-server env vars (MCP_BURP_TRANSPORT=http, MCP_BURP_URL=...). I went with one JSON variable (MCP_SERVERS) because the number of servers is open-ended and a JSON array scales without inventing a new env naming scheme; happy to switch if maintainers prefer the per-server form.
2. SDK choice — the official modelcontextprotocol/go-sdk (pinned to v1.3.1, the newest version that does not force a golang-jwt/jwt/v5 upgrade on the rest of the tree) instead of mark3labs/mcp-go. It supports all three transports plus in-memory transports used by the tests.
3. Agent scope — tools are offered to the six tool-capable executors listed above. Generator/memorist/enricher/reporter executors were left out for now; trivial to add if wanted.
4. Lifecycle — servers are contacted once, lazily, on first executor creation (process-level shared client, closed on shutdown). No reconnect/re-discovery loop yet; that can be a follow-up if needed.

Testing

• unit tests for config parsing/validation (valid/invalid JSON, per-transport checks, duplicate names, unknown contexts), tool naming (sanitization, truncation, prefix inversion), allow/deny selector semantics, and the tools-layer mapping/registration (fake client);
• client tests run against real in-memory MCP servers from the SDK (dispatch, context filtering, unreachable-server degradation, tool-error surfacing);
• verified end-to-end against a standalone stdio MCP server: env parsing → subprocess spawn with extra env → namespaced discovery → tool call round-trip;
• go build ./..., go vet ./... clean; full go test ./... shows no new failures (the three pre-existing cmd/installer failures reproduce on main untouched).