Lazy-load database and embedding modules in MCP server

[wolfiesch]

Summary

• Defers database connection, schema migration, and embedding pipeline initialization until tool execution time.
• Preserves automatic schema migration for legacy index databases via a memoized ensureDbReady() helper before read-only queries are executed.
• Prevents loading better-sqlite3, sqlite-vec, and @xenova/transformers ONNX bindings during initial MCP server transport connection.
• Adds regression test in test/db.test.ts verifying that outdated index DB fixtures migrate lazily before read queries.
• Verified with full test suite passing.

Performance Receipts (5-Session Averages on Exact PR Head)

• MCP Server Startup + Handshake: 96.6 ms (down from 175.6 ms warm / 616 ms cold).
• Text/Keyword Search Execution: 24.5 ms (skips embedding model initialization).
• First Vector Search Execution: 210.4 ms (loads embedding pipeline on demand).
• Warm Vector Search Execution: 44.6 ms (model resident in memory).

[Copilot]

Pull request overview

This PR restructures the MCP server so heavyweight DB/embedding initialization is deferred until tool execution time, and adds a new read_visual tool that renders transcripts into cached PNG pages for vision-capable models (plus telemetry and regression tests).

Changes:

• Lazy-load DB access (and associated schema migration) during tool calls via ensureDbReady() and dynamic imports.
• Add read_visual MCP tool powered by a Bun-rendered rasterization pipeline with a content-addressed cache and integrity checks.
• Add session-path boundary resolution helper, telemetry logging, tests, and dogfood harness for validating visual-read caching/latency behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/visual-read.test.ts	Adds unit tests for visual page caching, identity binding, and corruption handling.
test/session-path.test.ts	Adds tests for bounded session path resolution and traversal rejection.
test/db.test.ts	Adds regression test proving lazy migration happens before read-only DB queries.
src/visual-read.ts	Implements content-addressed visual rendering cache + receipt integrity validation.
src/session-path.ts	Adds a shared helper to resolve session paths safely within a configured root.
src/mcp-server.ts	Defers DB initialization, adds telemetry, and introduces read_visual tool.
scripts/render-pages.ts	Bun bridge script that rasterizes markdown into PNG pages + manifest for caching.
package.json	Publishes render script and adds @oh-my-pi/snapcompact dependency.
dogfood/read-visual/contract.ts	Defines dogfood protocol/receipt schemas and immutability helpers.
dogfood/read-visual/preflight.ts	Preflight checks for reproducible dogfood runs and artifact identity.
dogfood/read-visual/run.ts	Executes serialized local MCP dogfood runs and records receipts.
dogfood/read-visual/score.ts	Scores dogfood outputs against protocol gates and emits a score artifact.
dogfood/read-visual/protocol.json	Declares the dogfood execution plan and acceptance gates.
bun.lock	Locks newly added snapcompact dependency tree.

Suppressed comments (3)

src/mcp-server.ts:189

• formatConversation assumes parseSessionFile() returns at least one exchange, but parseSessionFile can return an empty array (e.g. oversized sessions are skipped). This will throw when accessing exchanges[0], breaking both read and read_visual for those files.

    `Session: ${header.sessionId}`,
    header.cwd ? `Directory: ${header.cwd}` : "",
    `Exchanges: ${exchanges.length}`,
    "",

src/mcp-server.ts:460

• list_gotchas defines an optional query input, but the handler never passes it to searchMemoryRecords, so supplying query has no effect.
  src/mcp-server.ts:493
• get_project_context accepts an optional project input, but the handler does not pass it to getProjectContext, so project-scoped context is never returned.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.