[DO NOT MERGE] Experiment: model-free tool CLI and MCP server on one adapter

docs/docs/configure/mcp-servers.md

@@ -6,6 +6,43 @@ description: "Configure Model Context Protocol (MCP) servers in Altimate Code to
 
 altimate supports the Model Context Protocol (MCP) for connecting to external tool servers.
 
+## Use Altimate Code tools from another MCP client
+
+Run `altimate-code mcp serve` to expose data tools over stdio. Your MCP client's model handles reasoning; this server runs tools directly without calling a model.
+
+Add this entry to your MCP client's configuration (for example, Claude Code's `.mcp.json`):
+
+```json
+{
+  "mcpServers": {
+    "altimate-code": {
+      "command": "altimate-code",
+      "args": ["mcp", "serve", "--connections", "/absolute/path/to/connections.json"]
+    }
+  }
+}
+```
+
+By default the server exposes `warehouse_list`, `warehouse_add`, `sql_execute`, `sql_materialize` and `answer_commit`, plus `impact_analysis`, `dbt_lineage`, `altimate_core_column_lineage`, `schema_detect_pii`, `data_diff`, `altimate_core_equivalence` and `sql_translate`. Pass `tables` to `sql_execute` to get those tables' columns and value profiles in the same call as its queries. Start with `--all-tools` to expose every data tool, the same set `altimate-code tool list` shows. Without `--connections`, the server reads the usual connection sources and re-reads them while none are configured, so a connections file written mid-session, or a connection added with `warehouse_add`, is picked up without a restart. A connections file is a JSON object keyed by connection name:
+
+```json
+{
+  "local": { "type": "sqlite", "path": "./analytics.sqlite" }
+}
+```
+
+Relative database paths resolve beside this file. With `--connections`, this file is the only source of connections: global and project `connections.json` files and `ALTIMATE_CODE_CONN_*` overrides are ignored. Omit `--connections` to use normal connection discovery: global and project `.altimate-code/connections.json` files, followed by `ALTIMATE_CODE_CONN_*` environment overrides. Set your client's working directory to the project when using project discovery.
+
+At startup the server lists each connection's name and type in its instructions, read from configuration only: it does not connect to remote or server databases or run queries on them. For local SQLite and DuckDB files (not behind an SSH tunnel) it also lists table names from one catalog query (no row counts); a file with more than 40 tables is noted as such instead. Call `warehouse_list` for columns and sample rows. Listing tools use catalog row estimates where available (including DuckDB); SQLite listings omit row counts. They never count rows by scanning each table.
+
+<!-- altimate_change start — MCP read-only limits and private output lifecycle -->
+
+Source writes are refused by default. Without `--allow-write`, source queries request database-enforced read-only execution on PostgreSQL, SQLite and DuckDB. For other connection types, read-only is enforced by SQL classification only; stored functions with side effects are not blocked — use a read-only database user. The server prints this caveat on stderr at startup and includes it in `warehouse_list` for each affected connection. Add `--allow-write` to enable SQL write permissions; existing hard safety blocks still apply. `sql_materialize` always requires a read-only source query and writes only into the server session's temporary `scratch` warehouse. Scratch tables disappear when the server exits. Large results may include a local spill file path that your client can open with any file reader. Spill files use a private directory and are deleted when the server exits. Local `tool_output.max_lines` and `tool_output.max_bytes` settings control output limits; remote agent configuration is not loaded.
+
+<!-- altimate_change end -->
+
+The server writes only MCP protocol messages to stdout and sends logs to stderr. It does not initialize telemetry and honors `ALTIMATE_TELEMETRY_DISABLED`.
+
 ## Local MCP Servers
 
 Run an MCP server as a local subprocess:

docs/docs/usage/cli.md

@@ -28,6 +28,7 @@ altimate --agent analyst
 | ----------- | ------------------------------ |
 | `run`       | Run a prompt non-interactively |
 | `check`     | Run deterministic SQL checks (no LLM required) -- see [SQL Check](check.md) |
+| `tool`      | Run the data tools without a model, optionally via a warm daemon -- see [Tool CLI](tool.md) |
 | `serve`     | Start the HTTP API server      |
 | `web`       | Start the web UI               |
 | `agent`     | Agent management               |

docs/docs/usage/tool.md

@@ -0,0 +1,72 @@
+---
+title: "Tool CLI (Headless) — Altimate Code"
+description: "Call Altimate Code data tools from any shell, agent or CI job without an LLM. Optional warm daemon keeps connections and the schema cache ready."
+---
+
+# Tool CLI (Headless)
+
+Expose the data tools -- warehouse, SQL, schema, lineage, impact, PII, data-diff, FinOps and dbt -- as plain commands. No model or API key is involved, so Claude Code, Codex and CI jobs can call them directly.
+
+## List tools
+
+```bash
+altimate-code tool list          # id and description
+altimate-code tool list --json   # id, description and JSON input schema
+```
+
+`tool` and `mcp serve` share one implementation, so compact schemas, `sql_execute` with `tables`, `query_from` chaining, spilled result files, read-only enforcement and `answer_commit` (with its `request` argument) behave the same in both. `tool` exposes the full tool list; pass `--workspace-tools` to add `workspace_context`, `workspace_skill` and `workspace_execute`.
+
+## Startup context
+
+```bash
+altimate-code tool context [--connections <file>] [--json]
+```
+
+Prints the analyst method and the connections known at startup (the same text `mcp serve` sends as its instructions), so a shell agent gets it in one call.
+
+## Run a tool
+
+```bash
+altimate-code tool run sql_format --args '{"sql":"select a,b from t","dialect":"postgres"}'
+altimate-code tool run warehouse_list
+altimate-code tool run schema_inspect --args-file inspect.json --json
+echo '{"query":"select 1"}' | altimate-code tool run sql_execute --args -
+```
+
+| Option | Description |
+|---|---|
+| `--args '<json>'` | Tool arguments as a JSON object. `-` reads them from stdin. |
+| `--args-file <path>` | Read the arguments from a file. |
+| `--allow-write` | Allow writes. Without it, only reads run. |
+| `--no-daemon` | Run in this process instead of using the daemon. |
+| `--json` | Print `{"status","title","output","metadata"}`. Otherwise only `output` is printed. |
+
+Arguments are validated against the tool's schema. An unknown tool, invalid arguments or a failing tool exits non-zero.
+
+Only the data tools are exposed. Coding tools (`bash`, `read`, `edit`, ...), `dbt_pr_review` (use `altimate-code review`), memory and training tools are not available here.
+
+## Permissions
+
+Runs are read-only by default. Reads such as `sql_execute` on a SELECT and `schema_inspect` are allowed; anything else is refused with `Permission denied: <kind>. Re-run with --allow-write to enable writes.` Tools that change local configuration (`warehouse_add`, `warehouse_remove`, `warehouse_install_driver`) also need `--allow-write`.
+
+`DROP DATABASE`, `DROP SCHEMA` and `TRUNCATE` stay blocked even with `--allow-write`.
+
+## Daemon
+
+`tool run` starts a background process per project directory and CLI version on first use, and reuses it afterwards so warehouse connections and the schema cache stay warm. If it cannot be reached within about 3 seconds, the call runs in-process. `scratch` tables created by `sql_materialize` live as long as the daemon does.
+
+```bash
+altimate-code tool daemon status
+altimate-code tool daemon stop
+```
+
+| Setting | Description |
+|---|---|
+| `ALTIMATE_TOOL_DAEMON=0` | Always run in-process. |
+| `ALTIMATE_TOOL_DAEMON_IDLE_MS` | Idle time before the daemon exits (default `600000`). |
+
+The daemon listens on a local socket (a named pipe on Windows) under the Altimate Code state directory and removes it on exit.
+
+## Claude Code routing
+
+For clients that cannot use the `altimate-code` MCP server, the Claude Code plugin adds CLI routing guidance at session start: `altimate-code tool context` followed by `altimate-code tool run` for data questions, and `altimate-code run` for long multi-step jobs. To persist the same guidance in the repository, run `altimate-code link --claude-md`; it writes a managed block to the project's `CLAUDE.md` and never changes content outside the `<!-- altimate-code:routing start -->` / `<!-- altimate-code:routing end -->` markers.

docs/mcp-workspaces.md

@@ -0,0 +1,31 @@
+# Workspaces for external coding clients (experimental)
+
+`altimate-code mcp serve --workspace-tools` adds workspace context, named skills and known read operations from the linked workspace's existing `datamate` engine. The default MCP server remains unchanged. This draft builds on the experimental MCP server; no rollout or merge is implied.
+
+The client must run in the intended project directory. Sign-in and linking may complete after connecting: `workspace_context` returns structured setup status and the catalog remains stable. `workspace_context` returns the current workspace, bounded project memory, a summary of skills/knowledge/integrations, and skill names. `workspace_skill` reads a named skill from the validated snapshot; refresh missing skills with `altimate-code workspace refresh`. It rejects traversal, symlinks, oversized skills and a changed snapshot/account.
+
+The engine is pinned with its existing `--datamate` option. Only knowledge, supported metadata reads and SQL reads are exposed; arbitrary tools, writes and engine memory mutations are excluded. SQL uses the existing classifier. Unsupported SQL argument schemas are withheld. The engine authenticates through its normal local configuration. Use read-only warehouse credentials: classification is not database isolation. Local SQL remains available if workspace authentication, linking or engine startup fails. An account/binding change invalidates stale receipts. Read fresh context and use the new receipt; reconnecting MCP is not required.
+
+For non-model clients:
+
+```
+altimate-code workspace-client list
+altimate-code workspace-client context --directory /absolute/project
+printf '%s' 'Use booked ARR for recurring revenue.' | altimate-code workspace-client remember --directory /absolute/project --token=<context-workspace-token> --id=metrics/arr
+```
+
+These commands output JSON. A context receipt binds account, workspace and project. `remember` requires that receipt, a valid memory id and 1–2,048 characters on stdin; it saves exactly one project block. It suppresses the ordinary fire-and-forget mirror, then uses the existing sync path with the explicitly selected binding. Requests pin credentials across async API/index operations. A backend failure after local persistence reports `savedLocally: true, cloudConfirmed: false`; retrying is a user decision. Confirmed cloud publication is reported only when the sync reports one successful write. No model-callable memory-write tool or bulk-sync operation is added.
+
+Workspace context adds bounded data to a client's prompt; skills load on demand. Explicit knowledge calls can use backend models, memory publication can invoke the existing backend extractor, and warehouse calls retain warehouse costs. There is no new reasoning agent or per-row model loop. A CLI link can normally backfill existing memory; clients that expose linking without a backfill confirmation must disable memory for that subprocess, as the Claude plugin does. Do not pass `--yes` unless replacing a link was explicitly requested.
+
+Verification covers unit contracts, a real stdio protocol fixture and existing SQLite MCP behavior. Production read-only testing listed visible workspaces and loaded matching context through a temporary local project. Production bindings and memory were not written. A non-empty cloud memory round trip and real workspace-engine SQL/knowledge calls remain rollout checks.
+
+## Automatic-client setup and stable gateway
+
+The opt-in workspace catalog is always `workspace_context`, `workspace_skill` and `workspace_execute`, including when initially unlinked. `workspace_context` reports `needs_auth`, `needs_workspace`, `disabled` or `unavailable` without returning raw credential errors. A ready context includes the workspace receipt. Context loads coalesce and cache for 30 seconds; identity is verified on every use. `refresh: true` bypasses that cache.
+
+Pass `include_tools: true` to request lazy engine discovery. The response includes `engineStatus` (`ready`, `needs_engine` or `unavailable`) and an operation catalog with the engine's actual input schemas. After explicit engine installation, request both `refresh: true` and `include_tools: true`. `workspace_execute` takes `workspace_token`, `operation` and an `arguments` object matching the returned schema. Unknown operations, invalid schemas, stale receipts and source-write SQL are refused. `workspace_skill` also requires `workspace_token`.
+
+This gateway intentionally changes the host permission surface: allowing `workspace_execute` permits its entire closed read-operation allowlist, including knowledge requests that can invoke backend models. It does not inherit per-operation approvals from the old dynamically discovered tool names. No workspace memory write is exposed as an MCP tool.
+
+`workspace-client context` returns the same structured setup status. `workspace-client engine-install` invokes the existing native installer; a UI must require an explicit user action before invoking it. The Claude pilot runs native sign-in and link actions from its first-use panel. Linking disables memory backfill for that subprocess. Authentication output stays outside model context. Default MCP startup without `--workspace-tools` is unchanged.

docs/mkdocs.yml

@@ -104,6 +104,7 @@ nav:
     - TUI: usage/tui.md
     - CLI: usage/cli.md
     - SQL Check: usage/check.md
+    - Tool CLI: usage/tool.md
     - Web UI: usage/web.md
     - CI: usage/ci-headless.md
     - IDE: usage/ide.md
@@ -186,6 +187,7 @@ nav:
     - TUI: usage/tui.md
     - CLI: usage/cli.md
     - SQL Check: usage/check.md
+    - Tool CLI: usage/tool.md
     - Web UI: usage/web.md
     - CI: usage/ci-headless.md
     - IDE: usage/ide.md

packages/drivers/src/mongodb.ts

@@ -42,6 +42,9 @@ type MqlCommand =
   | "dropCollection"
   | "createIndex"
   | "listIndexes"
+  // altimate_change start — scalar database catalog counts
+  | "dbStats"
+  // altimate_change end
   | "listCollections"
   | "ping"
 
@@ -514,7 +517,7 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
       maxResultBytes?: number,
       signal?: AbortSignal,
     ): Promise<ConnectorResult> {
-      async function readCursor(cursor: any): Promise<Record<string, unknown>[]> {
+      async function readCursor(cursor: any, rowLimit?: number): Promise<Record<string, unknown>[]> {
         const consume = createResultByteBudget(maxResultBytes)
         let stopped = false
         let failure: unknown
@@ -539,7 +542,7 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
           })
           const read = async () => {
             if (stopped) throw failure
-            if (maxResultBytes === undefined && maxTimeMS === undefined && !signal) return cursor.toArray()
+            if (rowLimit === undefined && maxResultBytes === undefined && maxTimeMS === undefined && !signal) return cursor.toArray()
             // One document per round trip made a 127k-document read take ~10 minutes. A reply
             // batch is still bounded (a server reply never exceeds 16 MiB) and the byte budget
             // stops the read at the first document past the cap.
@@ -548,6 +551,7 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
               if (stopped) throw failure
               if (maxResultBytes === undefined) {
                 docs.push(doc)
+                if (rowLimit !== undefined && docs.length >= rowLimit) break
                 continue
               }
               const serialized = Object.fromEntries(
@@ -605,17 +609,20 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
         return { columns: ["ok"], rows: [[result.ok]], row_count: 1, truncated: false }
       }
 
+      // altimate_change start — catalog discovery consumes only a bounded cursor, never the metadata cache
+      if (cmd === "dbStats") {
+        const stats = await db.command({ dbStats: 1, ...withMaxTime(maxTimeMS) })
+        return { columns: ["collections"], rows: [[stats.collections]], row_count: 1, truncated: false }
+      }
       if (cmd === "listCollections") {
-        const collections = await readCollections(db, maxTimeMS)
-        const matches = parsed.filter
-          ? await db.listCollections(parsed.filter, {
-            nameOnly: true, authorizedCollections: true, ...withMaxTime(maxTimeMS),
-          }).toArray()
-          : collections
+        const matches = await readCursor(db.listCollections(parsed.filter ?? {}, {
+          nameOnly: true, authorizedCollections: true, batchSize: Math.min(effectiveLimit + 1, 501), ...withMaxTime(maxTimeMS),
+        }), effectiveLimit + 1)
         const rows = matches.slice(0, effectiveLimit)
-          .map((collection: { name: string; type: string }) => [collection.name, collection.type])
+          .map((collection) => [collection.name, collection.type])
         return { columns: ["name", "type"], rows, row_count: rows.length, truncated: matches.length > effectiveLimit }
       }
+      // altimate_change end
 
       if (!MONGO_COLLECTION_READS.has(cmd)) clearMetadata()
 

packages/drivers/test/mongodb-catalog-bounds.test.ts

@@ -0,0 +1,74 @@
+// altimate_change start — schema discovery never materializes all collection metadata
+import { expect, mock, test } from "bun:test"
+import { connect } from "../src/mongodb"
+import { restoreModuleMocksAfterTests } from "./module-mock"
+
+await restoreModuleMocksAfterTests("mongodb")
+
+test("two million MongoDB collections stop at 501 and close the cursor", async () => {
+  let fetched = 0
+  let closed = 0
+  const options: unknown[] = []
+  const commands: unknown[] = []
+  mock.module("mongodb", () => ({
+    default: {
+      MongoClient: class {
+        async connect() {}
+        async close() {}
+        db() {
+          return {
+            databaseName: "large",
+            command: async (command: unknown) => {
+              commands.push(command)
+              return { collections: 2_000_000 }
+            },
+            listCollections: (_filter: unknown, opts: unknown) => {
+              options.push(opts)
+              return {
+                async toArray() {
+                  throw new Error("unbounded collection fetch")
+                },
+                async close() {
+                  closed++
+                },
+                async *[Symbol.asyncIterator]() {
+                  for (let i = 0; i < 2_000_000; i++) {
+                    fetched++
+                    yield { name: `c${i}`, type: "collection" }
+                  }
+                },
+              }
+            },
+          }
+        }
+      },
+    },
+  }))
+  const connector = await connect({ type: "mongodb" })
+  await connector.connect()
+  try {
+    const start = performance.now()
+    const result = await connector.execute(
+      JSON.stringify({ command: "listCollections", database: "large" }),
+      500,
+      undefined,
+      { readOnly: true, queryTimeoutMs: 1000 },
+    )
+    expect(result.rows).toHaveLength(500)
+    expect(result.truncated).toBe(true)
+    expect(fetched).toBe(501)
+    expect(closed).toBe(1)
+    expect(performance.now() - start).toBeLessThan(1000)
+    expect(options[0]).toMatchObject({ nameOnly: true, authorizedCollections: true, batchSize: 501, maxTimeMS: 1000 })
+    const counts = await connector.execute(JSON.stringify({ command: "dbStats", database: "large" }), 1, undefined, {
+      readOnly: true,
+      queryTimeoutMs: 1000,
+    })
+    expect(counts.rows).toEqual([[2_000_000]])
+    expect(fetched).toBe(501)
+    expect(commands).toEqual([{ dbStats: 1, maxTimeMS: 1000 }])
+  } finally {
+    await connector.close()
+  }
+})
+// altimate_change end