DEV-1810: MCP query row cap — default 20 rows, LIMIT push-down, truncation notice

.claude/skills/slayer-query.md

@@ -20,6 +20,8 @@ A `SlayerQuery` is a JSON/dict object. The same shape works across the REST API,
 }
 ```
 
+Over MCP, a query without `limit` returns at most 20 rows plus a truncation notice — set an explicit `limit` to get more (`query_nested`: the root stage's `limit`).
+
 `order[].column` uses the short alias (`count`, `revenue_sum`) to order by a measure declared in the same query; undeclared order targets use formula (colon) syntax — see below.
 
 **Ordering by something you don't project.** `order` may name an undeclared column/aggregate/expression ("top-N by X, show only Y, Z"). Computed hidden, sorted on, and stripped from the result: an **aggregate** (`amount:sum`, `customers.revenue:sum`), an inline **transform** (`rank(amount:sum)`, `change(...)`, `cumsum`/`lag`/`lead`/`ntile`), an inline **composite** (`revenue:sum / cnt:sum`, `abs(amount:sum)`), and a **windowed** aggregate (`amount:sum(window='90d')`, alone or inside a composite). A **raw row column** sorts directly in a raw-rows query (`distinct_dimension_values: false`); in a grouped/dedup query there is no single value per group, so it sorts **per group** by the extreme the direction puts first — `asc` by each group's `min`, `desc` by each group's `max`. Write `{"column": "created_at:max", "direction": "asc"}` explicitly for the other one. A **joined** row column (`customers.regions.name`), and a derived column whose `sql` reaches through a join, behave the same way — the join is pulled in for the sort, and in a grouped query the wrap is computed per host row-group rather than globally. NULLs sort **last** in both directions on every database (SQL Server excepted: its native ordering is used, because the portable emulation makes the statement fail there). An order target SLayer cannot resolve is an error, never a silently unsorted result. Order expressions must use formula syntax for their operands, not the `name`s of measures declared in the same query: `{"column": "revenue:sum / cnt:sum"}` works, `{"column": "rev / cnt"}` is rejected.

docs/reference/mcp.md

@@ -91,8 +91,8 @@ claude mcp list
 
 | Tool | Description |
 |------|-------------|
-| `query` | Execute a semantic query. See [Queries](../concepts/queries.md) for format. |
-| `query_nested` | Execute a multi-stage DAG of named sub-queries that can reference one another via `source_model` or `joins.target_model`. Companion to `query`; the engine auto-sorts the list (Kahn's algorithm), so order doesn't matter. Params: `queries: List[Dict[str, Any]]`, plus `variables` / `show_sql` / `dry_run` / `explain` / `format` mirroring `query`. See [Multistage Queries](../examples/06_multistage_queries/multistage_queries.md). |
+| `query` | Execute a semantic query. See [Queries](../concepts/queries.md) for format. Without an explicit `limit` the response is capped at 20 rows (the generated SQL carries `LIMIT 21` so truncation is detectable) and a truncation notice is appended via the warnings channel. |
+| `query_nested` | Execute a multi-stage DAG of named sub-queries that can reference one another via `source_model` or `joins.target_model`. Companion to `query`; the engine auto-sorts the list (Kahn's algorithm), so order doesn't matter. Params: `queries: List[Dict[str, Any]]`, plus `variables` / `show_sql` / `dry_run` / `explain` / `format` mirroring `query`. The 20-row cap keys on the ROOT (last) stage's `limit` only — non-root limits neither lift nor lower it. See [Multistage Queries](../examples/06_multistage_queries/multistage_queries.md). |
 
 **`query` parameters:**
 
@@ -104,15 +104,15 @@ claude mcp list
 | `filters` | list[str] | Filter formula strings, e.g. `["status = 'active'", "amount > 100"]`. Supports operators (`=`, `<>`, `>`, `>=`, `<`, `<=`, `IN`, `IS NULL`, `IS NOT NULL`, `LIKE`, `NOT LIKE`), boolean logic (`AND`, `OR`, `NOT`), and inline transform expressions (`"change(revenue) > 0"`). Filters on measures are automatically routed to HAVING. |
 | `time_dimensions` | list[dict] | Time grouping. Each entry supports an optional `label` for display. |
 | `order` | list[dict] | Sorting, e.g. `[{"column": "count", "direction": "desc"}]` |
-| `limit` | int | Max rows |
+| `limit` | int | Max rows, trusted verbatim; without it the response is capped at 20 rows with a truncation notice |
 | `offset` | int | Skip rows |
 | `whole_periods_only` | bool | Snap date filters to time bucket boundaries, exclude the current incomplete time bucket |
 | `distinct_dimension_values` | bool | Default `true` — auto-dedup dim-only queries (`GROUP BY <dim/td aliases>`). Set `false` to emit raw rows (no top-level `GROUP BY`); rejects any measure reference in `measures` / `filters` / `order`. |
 | `strict` | bool | Default `false` — error instead of warn when a [cross-model measure broadcasts](../concepts/queries.md#cross-model-measures) or a filter is excluded from its producer. Rejected with the run-by-name shortcut — declare it on the stored query instead |
 | `show_sql` | bool | Include the generated SQL in the response for debugging |
 | `dry_run` | bool | Generate and return the SQL without executing it |
 | `explain` | bool | Run EXPLAIN ANALYZE and return the query plan |
-| `format` | string | Output format: `"markdown"` (default, compact), `"json"` (structured), or `"csv"` (most compact). Case-insensitive |
+| `format` | string | Output format: `"markdown"` (default, compact), `"json"` (structured), or `"csv"` (most compact). Case-insensitive. Warnings (including the truncation notice) render as a trailing `Warnings:` block in markdown, leading `#` comment lines in csv, and turn the json payload into `{"data", "warnings"}` instead of a bare array |
 
 ### Memories + semantic search
 

openspec/changes/archive/2026-09-04-dev-1810-mcp-query-tool-add-a-response-row-cap-with-a-truncation/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-04

openspec/changes/archive/2026-09-04-dev-1810-mcp-query-tool-add-a-response-row-cap-with-a-truncation/design.md

@@ -0,0 +1,28 @@
+# Design
+
+## Context
+
+See proposal.md — Why. Relevant current state: `_format_output` (slayer/mcp/server.py) dispatches markdown/json/csv rendering, and warnings already render in all three formats (`_format_warnings`, `_csv_warning_comments`, json `{"data","warnings"}` shape). The `query` tool takes a run-by-name shortcut for a bare stored-query-model name with no overrides; passing `limit` disables that shortcut (pre-existing behavior). `SlayerQuery.limit` is emitted as the outermost SQL LIMIT.
+
+## Goals / Non-Goals
+
+- Goal: cap lives entirely in the MCP layer; the engine, REST API, Flight, and PG facade are untouched.
+- Non-goals: no config/env knob for the default cap; no exact-total row counting; no ceiling on explicit limits.
+
+## Decisions
+
+1. **`limit` is the knob; no new tool argument** (over a separate `max_rows` arg). One knob matches Storyline's agent-facing contract and avoids two interacting parameters on an already 16-parameter tool. Explicit `limit` is fully trusted — the maintainer explicitly rejected a hard ceiling (Storyline's 10,000): a caller overriding the default knows what they're doing.
+2. **Push-down + universal slice** (over post-hoc slice alone). When no explicit limit, the structured path sets the query limit to cap+1 (21) so the database never ships the full runaway result; the post-execution slice is the universal guarantee for paths push-down can't reach (run-by-name, DAG output, explain plans). Consequence: notices say "more rows exist" uniformly; exact totals are unknowable in the pushed-down case.
+3. **Notice rides the warnings union** (over dedicated per-format footers). A `ResponseTruncationWarning` (kind `"truncated"`, fields `returned_rows`, `hint`) joins `AnySlayerWarning` in slayer/core/warnings.py; the MCP layer appends it before formatting. Zero new rendering branches; the three formats stay in sync by construction. The engine never emits this kind — it is additive schema only from the REST API's perspective.
+4. **Cap decision uses the caller's arguments, not the executed result.** `len(result.data) > limit` must never be the trigger — with an explicit limit the MCP layer does no slicing at all (even for explain plans whose row count is unrelated to the SQL limit). Only the no-limit paths slice, at 20.
+5. **query_nested: root stage only.** The root (last) entry of `queries` controls the cap; other stages' limits are irrelevant. Push-down copies the root dict rather than mutating the caller's input.
+
+## Risks / Trade-offs
+
+- [Pushed-down `LIMIT 21` visible in show_sql/dry_run output] → honest: it is the SQL that runs; docstrings note the default cap.
+- [Run-by-name notice says "pass a higher 'limit'", but passing `limit` switches to structured execution with different variable precedence] → pre-existing sharp edge, kept out of the one-line notice deliberately; uniform hint text wins.
+- [Truncating an explain plan (no-limit case) can mangle plan readability] → accepted for uniformity; a giant EXPLAIN ANALYZE floods context the same way rows do, and `limit` lifts the cap.
+
+## Migration Plan
+
+Additive behavior change in one release; no storage or schema migration. Rollback = revert.

openspec/changes/archive/2026-09-04-dev-1810-mcp-query-tool-add-a-response-row-cap-with-a-truncation/proposal.md

@@ -0,0 +1,29 @@
+# Proposal: MCP query response row cap with truncation notice
+
+## Why
+
+The MCP `query` and `query_nested` tools have no response-side row cap: a query without a `limit` returns the full result set, which can flood the calling agent's context window. Storyline's MCP query tool already caps responses; SLayer should behave the same way.
+
+## What Changes
+
+- The MCP `query` tool caps returned rows at 20 when the caller passes no `limit`. An explicit `limit` is trusted verbatim — no ceiling, no truncation.
+- `query_nested` applies the same rule keyed on the ROOT stage's `limit` (last entry of `queries`).
+- When no explicit limit exists, the structured path pushes `LIMIT cap+1` (21) into the generated query so truncation is detectable without fetching the full result; a universal post-execution slice guards paths push-down cannot reach (run-by-name stored queries, DAG stages, explain plans).
+- A truncated response carries a `ResponseTruncationWarning` (new kind `"truncated"` in the `AnySlayerWarning` union) stating the returned row count and how to get more rows; it renders through the existing warnings machinery in all three output formats (markdown `Warnings:` block, csv leading `#` comment, json `{"data", "warnings"}`).
+
+## Capabilities
+
+### New Capabilities
+
+- `mcp/response-row-cap`: response-side row capping and truncation notices for the MCP query tools.
+
+### Modified Capabilities
+
+(none)
+
+## Impact
+
+- `slayer/core/warnings.py` — new `ResponseTruncationWarning` + union member (additive; the engine never emits it, REST unaffected).
+- `slayer/mcp/server.py` — default-cap constant, cap/push-down and slice+notice helpers, wiring in `query` and `query_nested`, docstring updates.
+- `docs/reference/mcp.md`, `.claude/skills/slayer-query.md` — document the cap, the notice, the root-stage rule, and the json shape change on truncation.
+- REST API, Flight, PG facade, stored-query semantics: unchanged.

openspec/changes/archive/2026-09-04-dev-1810-mcp-query-tool-add-a-response-row-cap-with-a-truncation/specs/mcp/response-row-cap/spec.md

@@ -0,0 +1,145 @@
+# mcp/response-row-cap
+
+## Purpose
+
+Protects the calling agent's context window: MCP query responses are capped at a small default row count unless the caller sets an explicit limit, and a truncated response says so and tells the caller how to get more rows.
+
+## ADDED Requirements
+
+### Requirement: Default row cap on MCP query responses
+
+When the MCP `query` tool is called without a `limit`, the response SHALL contain at most 20 data rows. When the underlying result has more rows than the cap, the response SHALL be truncated to exactly 20 rows and carry a truncation notice.
+
+#### Scenario: Uncapped query over a large result
+
+- WHEN `query` runs without `limit` against a model whose result has more than 20 rows
+- THEN the response contains exactly 20 data rows and a truncation notice
+
+#### Scenario: Result exactly at the cap
+
+- WHEN `query` runs without `limit` and the result has exactly 20 rows
+- THEN all 20 rows are returned and no truncation notice appears
+
+#### Scenario: Result one past the cap
+
+- WHEN `query` runs without `limit` and the result has exactly 21 rows
+- THEN the response contains exactly 20 rows and a truncation notice
+
+### Requirement: Explicit limit is trusted verbatim
+
+When the caller passes an explicit `limit`, the MCP layer SHALL return the rows as executed, with no response-side truncation and no truncation notice — regardless of how many rows come back, including `explain` plan rows.
+
+#### Scenario: Explicit limit honored
+
+- WHEN `query` runs with `limit=25` against a result with 30 available rows
+- THEN 25 rows are returned and no truncation notice appears
+
+#### Scenario: Explicit small limit
+
+- WHEN `query` runs with `limit=5`
+- THEN 5 rows are returned and no truncation notice appears
+
+#### Scenario: Rows exceeding an explicit limit are not sliced by the MCP layer
+
+- WHEN the engine returns more rows than an explicit `limit` (e.g. a mocked execution)
+- THEN the MCP layer neither slices the rows nor adds a truncation notice
+
+#### Scenario: Explain with explicit limit untouched
+
+- WHEN `query` runs with `explain=True` and an explicit `limit`, and the plan has more rows than the limit
+- THEN all plan rows are returned and no truncation notice appears
+
+### Requirement: Cap push-down into the generated query
+
+When no explicit `limit` is given on the structured query path, the generated SQL SHALL carry `LIMIT 21` (cap + 1) so truncation is detectable without fetching the full result. Run-by-name execution of a stored query SHALL leave the stored query's SQL untouched.
+
+#### Scenario: Pushed-down limit visible in SQL
+
+- WHEN `query` runs without `limit` on the structured path with `show_sql=True` or `dry_run=True`
+- THEN the generated SQL contains `LIMIT 21`, not `LIMIT 20`
+
+#### Scenario: Stored query SQL untouched
+
+- WHEN a stored query runs by bare name (run-by-name path) without `limit`
+- THEN the SQL executed is the stored query's own, with no injected LIMIT
+
+### Requirement: Run-by-name responses are capped response-side
+
+A stored query executed by bare name without a `limit` SHALL have its response sliced to 20 rows with a truncation notice when it returns more, and returned whole with no notice when it returns 20 or fewer.
+
+#### Scenario: Stored query above the cap
+
+- WHEN a run-by-name stored query returns more than 20 rows
+- THEN the response contains exactly 20 rows and a truncation notice
+
+#### Scenario: Stored query at the cap
+
+- WHEN a run-by-name stored query returns exactly 20 rows
+- THEN all 20 rows are returned and no truncation notice appears
+
+### Requirement: query_nested capped by the root stage's limit only
+
+The `query_nested` tool SHALL apply the same rule keyed on the ROOT stage (last entry of `queries`): an explicit root `limit` is trusted verbatim; without one, the final response is capped at 20 with a truncation notice whose hint points at the root query's `limit`. Non-root stages' limits SHALL NOT affect the cap. The tool SHALL NOT mutate the caller's `queries` dicts.
+
+#### Scenario: Root without limit is capped
+
+- WHEN `query_nested` runs with a root stage that has no `limit` and the final result has more than 20 rows
+- THEN the response contains exactly 20 rows and a truncation notice telling the caller to set a higher `limit` on the root query
+
+#### Scenario: Non-root limit does not lift the cap
+
+- WHEN a non-root stage has an explicit `limit` but the root stage has none
+- THEN the default cap of 20 still applies to the final response
+
+#### Scenario: Root limit trusted
+
+- WHEN the root stage has an explicit `limit`
+- THEN no response-side truncation occurs and no notice appears
+
+#### Scenario: Caller dicts unchanged
+
+- WHEN `query_nested` pushes the cap into the root stage
+- THEN the caller's submitted `queries` dicts are structurally unchanged afterwards (no `limit` key added)
+
+### Requirement: Truncation notice content and rendering
+
+The truncation notice SHALL state the returned row count, say that more rows exist, and tell the caller how to get more rows. It SHALL appear in every output format through the warnings channel: the markdown `Warnings:` block, a leading `#` comment line in csv, and a warning entry with kind `"truncated"` in the json `{"data", "warnings"}` payload. It SHALL coexist with other warnings, appended last.
+
+#### Scenario: Markdown notice
+
+- WHEN a truncated result is formatted as markdown
+- THEN the output ends with a `Warnings:` block containing "showing first 20 rows — more rows exist" and a hint to pass a higher `limit`
+
+#### Scenario: CSV notice
+
+- WHEN a truncated result is formatted as csv
+- THEN a leading `#` comment line carries the notice and the data rows below keep a uniform column count
+
+#### Scenario: JSON notice
+
+- WHEN a truncated result is formatted as json
+- THEN the payload has the `{"data", "warnings"}` shape and `warnings` contains an entry with `kind == "truncated"` and the returned row count
+
+#### Scenario: Coexists with other warnings
+
+- WHEN a truncated result already carries an engine warning
+- THEN both warnings render in every format and the truncation notice comes last
+
+#### Scenario: Warning round-trips through the union
+
+- WHEN a response carrying the truncation warning is serialized and re-validated
+- THEN the warning deserializes back to the truncation kind with its fields intact
+
+### Requirement: Explain plan rows capped without a limit
+
+When `query` runs with `explain=True` and no `limit`, the returned plan rows SHALL be subject to the same 20-row cap and notice. `dry_run` output (SQL only) SHALL never carry a truncation notice.
+
+#### Scenario: Large explain plan capped
+
+- WHEN `explain=True` without `limit` yields a plan of more than 20 rows
+- THEN 20 plan rows are returned with a truncation notice
+
+#### Scenario: Dry run unaffected
+
+- WHEN `dry_run=True`
+- THEN the response contains only SQL and never a truncation notice