diff --git a/.changeset/17551-dataset-selection-schema.md b/.changeset/17551-dataset-selection-schema.md new file mode 100644 index 00000000000..71bba028484 --- /dev/null +++ b/.changeset/17551-dataset-selection-schema.md @@ -0,0 +1,15 @@ +--- +"@objectstack/spec": minor +"@objectstack/rest": patch +"@objectstack/service-analytics": patch +--- + +`DatasetSelectionSchema` — the ADR-0021 dataset selection is a Zod declaration now, and `POST /api/v1/analytics/dataset/query` parses the whole selection against it (#17551). + +`DatasetSelection` was a TypeScript **interface** with no Zod schema anywhere in the repo. PR #17548 doored that route, but only over the **seven** members the selection shares with `AnalyticsQuery`; the other **four** — `runtimeFilter`, `dateGranularity`, `compareTo`, `totals` — were declared in TypeScript, published in the api-surface, and enforced by nothing on the wire. The measured consequence is #17550: `compareTo: { kind: 'nonsense' }` came back as a previous-period comparison under an ordinary **200**, a number a dashboard renders and a person reads as fact. + +- **One declaration, in `packages/spec`.** `DatasetSelectionSchema`, `DatasetCompareToSchema` and `DatasetTotalsSchema` are authored in `api/analytics.zod.ts`, beside the `AnalyticsQueryRequestSchema` the sibling routes parse. `@objectstack/spec/contracts` now **re-exports** the `DatasetSelection` and `DatasetCompareTo` types from that schema instead of declaring interfaces of its own — the same move `AnalyticsQuery` made in #4538, taken here before a mirror could drift. +- **A transcription, not a new contract.** The seven shared members are read straight off `AnalyticsQuerySchema.shape`, so the claim that the two agree is structural rather than a hand-written list; the four dataset-only members are the already-published TypeScript members made executable. No member is added and nothing the interface permitted is refused. +- **Refusals carry a prescription.** An unrecognised `compareTo.kind` answers the sentence `datasetCompareKindRefusalMessage` builds — what arrived, the two windows the executor implements, what to do — and `@objectstack/service-analytics`' `shiftRange` now raises that same sentence with its own origin clause, so one condition keeps one wording. An unknown key is named, echoed and pointed at the canonical spelling (`where` → `runtimeFilter`, `granularity` → `dateGranularity`), and the retired `{ offset }` arm and the pre-#5011 bare-string form each carry their rewrite. +- ⚠️ **What narrows on the wire**, so an upgrading caller can look for it: a selection member whose value the published interface never permitted now answers `400 VALIDATION_FAILED` with `details.fields[]` instead of travelling into the executor. Measured against the sibling route spelling for spelling, `runtimeFilter` now behaves exactly as `/analytics/query`'s `where` does — three structurally-malformed filter spellings (`{ $or: 'x' }`, an `$or` branch that is not a filter object, `{ $not: 5 }`) are refused at the schema on both routes, and the four semantic ones (`{ stage: {} }`, `{ amount: { $between: [10] } }`, `{ $nor: […] }`, `{ $or: [] }`) still pass both and are answered deeper. The dataset route was the looser of the two; it is not any more. +- **No valid selection changes.** Every in-repo specimen and all five `@object-ui` call sites that build a selection today still pass, pinned in both packages; the route still forwards the caller's object to the service by identity, never a parse output, and the schema carries no default or transform that could override the engine's own timezone resolution chain. diff --git a/content/docs/references/api/analytics.mdx b/content/docs/references/api/analytics.mdx index 717020c4f84..a9fff472ad9 100644 --- a/content/docs/references/api/analytics.mdx +++ b/content/docs/references/api/analytics.mdx @@ -17,8 +17,8 @@ Provides endpoints for executing analytical queries and discovering metadata. ## TypeScript Usage ```typescript -import { AnalyticsEndpoint, AnalyticsMetadataResponseSchema, AnalyticsQueryRequestSchema, AnalyticsResultResponseSchema, AnalyticsSqlResponseSchema, GetAnalyticsMetaRequestSchema } from '@objectstack/spec/api'; -import type { AnalyticsEndpoint, AnalyticsMetadataResponse, AnalyticsQueryRequest, AnalyticsResultResponse, AnalyticsSqlResponse, GetAnalyticsMetaRequest } from '@objectstack/spec/api'; +import { AnalyticsEndpoint, AnalyticsMetadataResponseSchema, AnalyticsQueryRequestSchema, AnalyticsResultResponseSchema, AnalyticsSqlResponseSchema, DatasetCompareToSchema, DatasetSelectionSchema, DatasetTotalsSchema, GetAnalyticsMetaRequestSchema } from '@objectstack/spec/api'; +import type { AnalyticsEndpoint, AnalyticsMetadataResponse, AnalyticsQueryRequest, AnalyticsResultResponse, AnalyticsSqlResponse, DatasetCompareTo, DatasetSelection, DatasetTotals, GetAnalyticsMetaRequest } from '@objectstack/spec/api'; // Validate data const result = AnalyticsEndpoint.parse(data); @@ -193,6 +193,71 @@ const result = AnalyticsEndpoint.parse(data); | **traceId** | `string` | optional | | +--- + +## DatasetCompareTo + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **kind** | `Enum<'previousPeriod' \| 'previousYear'>` | ✅ | Comparison window: previousPeriod (equal-length, immediately before) or previousYear (the same window one calendar year back) | +| **dimension** | `string` | optional | Time dimension to shift; omit when the selection has exactly one dated time dimension | + + +--- + +## DatasetSelection + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **dimensions** | `string[]` | optional | List of dimensions to group by | +| **measures** | `string[]` | ✅ | List of metrics to calculate | +| **runtimeFilter** | `any` | optional | Presentation-scope filter (canonical Query DSL FilterCondition), ANDed with the dataset's intrinsic filter at render | +| **timeDimensions** | `{ dimension: string; granularity?: Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; dateRange?: Enum<'today' \| 'yesterday' \| 'this_week' \| 'last_week' \| 'this_month' \| 'last_month' \| …> \| [string, string] }[]` | optional | Time-bucketed dimensions. Each entry names a dimension, an optional bucket `granularity`, and an optional `dateRange` — a preset name from the closed date-range vocabulary (e.g. `'last_7_days'`) or an explicit `[start, end]` window; an unrecognised string answers `400 ANALYTICS_DATE_RANGE_UNRECOGNIZED` instead of silently widening. | +| **dateGranularity** | `Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>` | optional | Presentation-scope date bucketing applied to every selected `date` dimension; an explicit `timeDimensions` entry wins over it, and the dataset dimension's own default is used when neither is set | +| **order** | `Record>` | optional | | +| **limit** | `number` | optional | | +| **offset** | `number` | optional | | +| **compareTo** | `{ kind: Enum<'previousPeriod' \| 'previousYear'>; dimension?: string }` | optional | Period-over-period comparison window (`{ kind, dimension? }`); attaches `__compare` columns | +| **totals** | `{ groupings: string[][] }` | optional | Server-side marginal aggregates; each grouping is a dimension subset to additionally aggregate by, `[]` being the grand total | +| **timezone** | `string` | optional | | + +### Nested Shape: `DatasetSelection.timeDimensions[number]` + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **dimension** | `string` | ✅ | | +| **granularity** | `Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>` | optional | | +| **dateRange** | `Enum<'today' \| 'yesterday' \| 'this_week' \| 'last_week' \| 'this_month' \| 'last_month' \| …> \| [string, string]` | optional | Time window for this dimension: a date-range PRESET name from the closed vocabulary in `data/date-range-presets.ts` (today, yesterday, this_week, last_week, this_month, last_month, this_quarter, last_quarter, this_year, last_year, last_7_days, last_30_days, last_90_days — e.g. `'last_7_days'`), or an explicit `[start, end]` array of ISO dates / `{date-macro}` tokens (e.g. `["2023-01-01", "2023-01-31"]`). Any other string is refused at the schema with `400 ANALYTICS_DATE_RANGE_UNRECOGNIZED`. | + +### Nested Shape: `DatasetSelection.compareTo` + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **kind** | `Enum<'previousPeriod' \| 'previousYear'>` | ✅ | Comparison window: previousPeriod (equal-length, immediately before) or previousYear (the same window one calendar year back) | +| **dimension** | `string` | optional | Time dimension to shift; omit when the selection has exactly one dated time dimension | + +### Nested Shape: `DatasetSelection.totals` + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **groupings** | `string[][]` | ✅ | Dimension subsets to additionally aggregate by, in request order; the empty subset is the grand total | + + +--- + +## DatasetTotals + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **groupings** | `string[][]` | ✅ | Dimension subsets to additionally aggregate by, in request order; the empty subset is the grand total | + + --- ## GetAnalyticsMetaRequest diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 47d11b4cc71..7e1f9b6f13e 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,6 +1,6 @@ --- title: Protocol Reference -description: Every schema published by @objectstack/spec — 1531 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1534 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -20,7 +20,7 @@ counts are sums of the rows they head. Regenerate with | Module | Pages | Schemas | Description | | :--- | ---: | ---: | :--- | | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. | -| [API Protocol](/docs/references/api) | 31 | 441 | REST contracts, endpoints, routing, realtime, batch, discovery. | +| [API Protocol](/docs/references/api) | 31 | 444 | REST contracts, endpoints, routing, realtime, batch, discovery. | | [Automation Protocol](/docs/references/automation) | 14 | 74 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. | | [Data Protocol](/docs/references/data) | 29 | 175 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. | @@ -33,7 +33,7 @@ counts are sums of the rows they head. Regenerate with | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | | [System Protocol](/docs/references/system) | 34 | 273 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 158 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **195** | **1531** | 14 protocol modules | +| **Total** | **195** | **1534** | 14 protocol modules | --- @@ -62,13 +62,13 @@ Agents, tools, skills, RAG and knowledge sources, model registry, conversations. ## API Protocol -**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **31 pages, 441 schemas** +**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **31 pages, 444 schemas** REST contracts, endpoints, routing, realtime, batch, discovery. | File | Schemas | | :--- | :--- | -| [`analytics.zod.ts`](/docs/references/api/analytics) | `AnalyticsEndpoint`, `AnalyticsMetadataResponse`, `AnalyticsQueryRequest`, `AnalyticsResultResponse`, `AnalyticsSqlResponse`, `GetAnalyticsMetaRequest` | +| [`analytics.zod.ts`](/docs/references/api/analytics) | `AnalyticsEndpoint`, `AnalyticsMetadataResponse`, `AnalyticsQueryRequest`, `AnalyticsResultResponse`, `AnalyticsSqlResponse`, `DatasetCompareTo`, `DatasetSelection`, `DatasetTotals`, `GetAnalyticsMetaRequest` | | [`auth.zod.ts`](/docs/references/api/auth) | `AuthProvider`, `LoginRequest`, `LoginType`, `RefreshTokenRequest`, `RegisterRequest`, `Session`, `SessionResponse`, `SessionUser`, `UserProfileResponse` | | [`auth-endpoints.zod.ts`](/docs/references/api/auth-endpoints) | `AuthEndpoint`, `AuthFeaturesConfig`, `AuthProviderInfo`, `DeviceRequestResponse`, `DeviceTokenResponse`, `EmailPasswordConfigPublic`, `GetAuthConfigResponse` | | [`automation-api.zod.ts`](/docs/references/api/automation-api) | `AutomationApiErrorCode`, `AutomationFlowPathParams`, `AutomationRunPathParams`, `CreateFlowRequest`, `CreateFlowResponse`, `DeleteFlowRequest`, `DeleteFlowResponse`, `FlowSummary`, `GetFlowRequest`, `GetFlowResponse`, `GetRunRequest`, `GetRunResponse`, `ListFlowsRequest`, `ListFlowsResponse`, `ListRunsRequest`, `ListRunsResponse`, `ResumeFailureDetails`, `ToggleFlowRequest`, `ToggleFlowResponse`, `TriggerFlowRequest`, `TriggerFlowResponse`, `UpdateFlowRequest`, `UpdateFlowResponse` | diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md index f1e5d637489..195fe7375f9 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md @@ -257,7 +257,7 @@ directory rather than per file. | Dir | Sites | |---|---| | `ai/` | 78 | -| `api/` | 451 | +| `api/` | 454 | | `identity/` | 32 | | `integration/` | 8 | | `kernel/` | 247 | diff --git a/packages/core/src/utils/analytics-date-range.ts b/packages/core/src/utils/analytics-date-range.ts index 9bcc53dec3d..045249934f1 100644 --- a/packages/core/src/utils/analytics-date-range.ts +++ b/packages/core/src/utils/analytics-date-range.ts @@ -235,10 +235,10 @@ export function resolveAnalyticsDateRangePreset( * Reachability: on EVERY REST analytics route a schema door parses * `timeDimensions` ahead of the reader. `POST /analytics/query` and * `/analytics/sql` parse the whole body; `POST /analytics/dataset/query` has - * parsed its selection's shared members — `timeDimensions` included — against - * `AnalyticsQuerySchema.pick(…)` since PR #17548, the PR that landed that door - * for card #17058 (`rest/src/analytics-selection-door.ts`, wired ahead of the - * executor). So every `dateRange` the union CAN refuse is refused there, with + * parsed its selection's shared members — `timeDimensions` included — since + * PR #17548, and parses the WHOLE selection against + * `DatasetSelectionSchema` since #17551 + * (`rest/src/analytics-selection-door.ts`, wired ahead of the executor). So every `dateRange` the union CAN refuse is refused there, with * the schema's own sentence, and this constructor contributes only its * `.code`/`.status` to that answer. * diff --git a/packages/rest/src/analytics-dataset-selection-door.test.ts b/packages/rest/src/analytics-dataset-selection-door.test.ts index 7b5534cb435..42bb5ca1c11 100644 --- a/packages/rest/src/analytics-dataset-selection-door.test.ts +++ b/packages/rest/src/analytics-dataset-selection-door.test.ts @@ -12,11 +12,28 @@ * * The card asked whether the dataset route's `selection` is genuinely the same * shape as the siblings' before reusing their schema. It is **not** — §1 below - * drives that against the real schema — so the door parses a PROJECTION of the - * members whose declarations coincide, and the four dataset-only members are - * projected away rather than refused. §5 is the other half of that answer and - * the one that matters most: a fully-loaded VALID selection still passes. - * A door that refuses too much is a worse defect than the one being fixed. + * drives that against the real schema — which is why the door has never parsed + * `AnalyticsQueryRequestSchema`. §5 is the other half of that answer and the + * one that matters most: a fully-loaded VALID selection still passes. A door + * that refuses too much is a worse defect than the one being fixed. + * + * ## [#17551, ruled] The half #17058 could not door + * + * #17058 parsed a PROJECTION — the seven members whose declarations coincide + * with `AnalyticsQuery`'s — and projected `runtimeFilter`, `dateGranularity`, + * `compareTo` and `totals` AWAY, because `DatasetSelection` had no Zod schema + * anywhere in the repo and authoring one in this consumer is the second + * declaration of a spec-owned wire shape PD #12 forbids. Decision batch #204 + * item 3 ruled letter A: the schema is authored in `packages/spec` and this + * door parses the WHOLE selection against it. So §4 flips from 「these four are + * not judged here」 to 「these four are judged here, both directions」, and §6 + * drives #17550's own specimen — `compareTo: { kind: 'nonsense' }`, which used + * to return a previous-period comparison under a 200 — through the real route. + * + * ⚠️ The SCHEMA's own two-directional pins live beside the schema + * (`spec/src/api/dataset-selection.test.ts`). What is pinned HERE is the + * ENVELOPE: which refusal shape a failure lands in, how a field path is spelled + * against the request body, and that the executor is never reached. */ // The dynamic `import()`s below are paid HERE, at module scope, so the @@ -29,10 +46,7 @@ import '@objectstack/spec/data'; import { describe, it, expect, vi } from 'vitest'; import { RestServer } from './rest-server'; -import { - SELECTION_MEMBERS_SHARED_WITH_ANALYTICS_QUERY, - datasetSelectionRefusal, -} from './analytics-selection-door'; +import { datasetSelectionRefusal } from './analytics-selection-door'; // ── harness (the shape `analytics-routes.test.ts` uses) ────────────────────── @@ -141,20 +155,27 @@ describe('#17058 §1 — the dataset route\'s `selection` is not the sibling rou }); /** - * The projection list is a claim about two declarations agreeing. Pin both - * directions so a later edit cannot quietly move a member into or out of it. + * [#17551] The door no longer carries a member list of its own — the shape + * it parses IS the spec's declaration. Pin that this module reaches the + * schema rather than a local copy, in the one way a consumer can: the four + * dataset-only members are judged here and are still not `AnalyticsQuery` + * members. (The schema's own structural pins — the seven shared members + * taken off `AnalyticsQuerySchema.shape` BY IDENTITY — live beside it.) */ - it('every projected member is an `AnalyticsQuery` member; no dataset-only member is', async () => { + it('the four dataset-only members are judged, and are still not `AnalyticsQuery` members', async () => { const { AnalyticsQuerySchema } = await import('@objectstack/spec/data'); + const { DatasetSelectionSchema } = await import('@objectstack/spec/api'); const analyticsMembers = Object.keys((AnalyticsQuerySchema as any).shape); - for (const member of SELECTION_MEMBERS_SHARED_WITH_ANALYTICS_QUERY) { - expect(analyticsMembers, `${member} must be declared on AnalyticsQuery`).toContain(member); - } + const selectionMembers = Object.keys((DatasetSelectionSchema as any).shape); for (const datasetOnly of ['runtimeFilter', 'dateGranularity', 'compareTo', 'totals']) { + expect(selectionMembers, `${datasetOnly} must be declared on the selection`) + .toContain(datasetOnly); expect(analyticsMembers).not.toContain(datasetOnly); - expect(SELECTION_MEMBERS_SHARED_WITH_ANALYTICS_QUERY as readonly string[]) - .not.toContain(datasetOnly); } + // …and the door really parses THAT schema: a value only it can refuse + // must be refused here. + expect(await datasetSelectionRefusal({ measures: ['revenue'], compareTo: { kind: 'nope' } })) + .toBeDefined(); }); }); @@ -275,26 +296,104 @@ describe('#17058 §3 — the generic refusal is 400 VALIDATION_FAILED + details. }); // ───────────────────────────────────────────────────────────────────────────── -// §4 — the four dataset-only members keep passing (they are projected away) +// §4 — [#17551] the four dataset-only members, both directions // ───────────────────────────────────────────────────────────────────────────── -describe('#17058 §4 — the dataset-only members are NOT judged by the sibling schema', () => { - const datasetOnly: Array<[string, unknown]> = [ +/** + * ⚠️ This section is the one #17551 turned over. It used to assert that these + * four 「reach the service untouched」 BECAUSE the door projected them away — + * true of the projection, and the gap the ruling closed. What survives + * unchanged is the half that still has to hold: a LEGAL value of each one still + * reaches the service, by identity. What is added is the other half: a + * malformed value of each is now refused at the door, with a remedy, and the + * executor never sees it. + */ +describe('#17551 §4 — the dataset-only members are judged at the door now', () => { + const legal: Array<[string, unknown]> = [ ['runtimeFilter', { region: { $ne: 'EU' } }], ['dateGranularity', 'quarter'], ['compareTo', { kind: 'previousYear' }], ['totals', { groupings: [[]] }], ]; - for (const [member, value] of datasetOnly) { - it(`\`${member}\` reaches the service untouched`, async () => { + for (const [member, value] of legal) { + it(`CONTROL — a legal \`${member}\` still reaches the service untouched`, async () => { const selection = { dimensions: ['region'], measures: ['revenue'], [member]: value }; const { res, queryDataset } = await post({ dataset: inlineDataset, selection }); expect(res.statusCode).toBe(200); expect(queryDataset).toHaveBeenCalledTimes(1); + // Validation-only: the CALLER's object, by identity — never a parse + // output that could carry a schema default. expect(queryDataset.mock.calls[0][1]).toBe(selection); }); } + + const malformed: Array<{ member: string; value: unknown; field: string; says: string }> = [ + { + // ⚠️ No dataset-only sentence is invented for this one, deliberately. + // `runtimeFilter` carries the canonical `FilterCondition`, so its + // refusals are that vocabulary’s own — byte-identical to what the + // sibling body's `where` answers for the same input. A second + // wording here would be exactly the #5240 defect this card's own + // `compareTo.kind` builder exists to avoid. + member: 'runtimeFilter', + value: 'region = NA', + field: 'selection.runtimeFilter', + says: 'expected record', + }, + { + member: 'dateGranularity', + value: 'fortnight', + field: 'selection.dateGranularity', + says: 'month', + }, + { + member: 'compareTo', + value: { kind: 'previousPeriod', offset: '7d' }, + field: 'selection.compareTo', + says: 'offset', + }, + { + member: 'totals', + value: { groupings: ['region'] }, + field: 'selection.totals.groupings.0', + says: 'array', + }, + ]; + + for (const c of malformed) { + it(`a malformed \`${c.member}\` answers 400 and never reaches the service`, async () => { + const selection = { dimensions: ['region'], measures: ['revenue'], [c.member]: c.value }; + const { res, queryDataset } = await post({ dataset: inlineDataset, selection }); + expect(res.statusCode).toBe(400); + expect(res.body.code).toBe('VALIDATION_FAILED'); + const fields: Array<{ field: string }> = res.body.details.fields; + expect(fields.map((f) => f.field)).toContain(c.field); + expect(String(res.body.message).toLowerCase()).toContain(c.says.toLowerCase()); + expect(queryDataset).not.toHaveBeenCalled(); + }); + } + + it('an UNDECLARED key is named against the request body, not dropped', async () => { + // ⚠️ The root rename is live here and was inert before #17551: a + // `.strict()` parse of the whole selection puts an unrecognized-keys + // issue at the ROOT, which the shared mapper spells `(body)` — true for + // the sibling routes, false here, where the object sits under + // `selection`. + const { res, queryDataset } = await post({ + dataset: inlineDataset, + selection: { measures: ['revenue'], runtimeFillter: { region: 'NA' } }, + }); + expect(res.statusCode).toBe(400); + expect(res.body.code).toBe('VALIDATION_FAILED'); + const fields: Array<{ field: string }> = res.body.details.fields; + expect(fields.map((f) => f.field)).toContain('selection'); + expect(fields.map((f) => f.field)).not.toContain('selection.(body)'); + // The refusal carries the fix, which is the whole point of doing this + // in the schema rather than with a key list here. + expect(res.body.message).toContain('\`runtimeFillter\` → \`runtimeFilter\`'); + expect(queryDataset).not.toHaveBeenCalled(); + }); }); // ───────────────────────────────────────────────────────────────────────────── @@ -452,3 +551,96 @@ describe('#17598 §5 — the arity refusal carries one wording on the wire', () expect(res.body.message).toContain('selection.timeDimensions.0.dateRange.1'); }); }); + +// ───────────────────────────────────────────────────────────────────────────── +// §6 — ⭐ [#17550] the card this door closes, driven through the real route +// ───────────────────────────────────────────────────────────────────────────── + +/** + * #17550: `shiftRange` branched only on `previousYear` and fell through to the + * `previousPeriod` arm, so `compareTo: { kind: 'nonsense' }` returned a + * previous-period comparison under an ordinary **200**. Nothing in the response + * distinguished it from a real answer, and the wrong answer is a comparison + * WINDOW — a number a dashboard renders and a person reads as fact. + * + * That card's own fix put an exhaustive `switch` in `shiftRange`, which closes + * it for an IN-PROCESS caller. ⚠️ It could not close it at the door: the door + * projected `compareTo` away, so a body still travelled into the executor and + * was answered there, one layer past the boundary that owns request shape — + * and only for a `compareTo` that survived long enough to be shifted at all. + * This section is that half: the refusal now happens AT THE DOOR, in the + * route's own envelope, before the analytics service is called. + * + * ⛔ Both halves stay. The executor's refusal is not redundant — `shiftRange` + * is a published export of `service-analytics` and `queryDataset` is reachable + * in-process by a caller that never posted a body. What is shared between them + * is the SENTENCE (`datasetCompareKindRefusalMessage`), so one condition keeps + * one wording (#5240). + */ +describe('#17550 §6 — an unrecognised `compareTo.kind` is refused at the door', () => { + it('the card\'s specimen answers 400 and the executor is never called', async () => { + const { res, queryDataset } = await post({ + dataset: inlineDataset, + selection: { + dimensions: ['region'], + measures: ['revenue'], + timeDimensions: [{ dimension: 'close_date', dateRange: 'last_30_days' }], + compareTo: { kind: 'nonsense' }, + }, + }); + + // ⛔ Not a 200 with a comparison in it — that IS the defect. + expect(res.statusCode).toBe(400); + expect(res.body.code).toBe('VALIDATION_FAILED'); + const fields: Array<{ field: string }> = res.body.details.fields; + expect(fields.map((f) => f.field)).toContain('selection.compareTo.kind'); + // ⭐ The whole point of the ruling's North Star clause: loud, AND with a + // prescription. What arrived, the closed vocabulary, and what to do. + expect(res.body.message).toContain('"nonsense"'); + expect(res.body.message).toContain("'previousPeriod'"); + expect(res.body.message).toContain("'previousYear'"); + expect(res.body.message).toContain('drop compareTo'); + // The refusal is the whole reason a door exists. + expect(queryDataset).not.toHaveBeenCalled(); + }); + + it('the envelope\'s code is a registered vocabulary member, not a dialect', async () => { + const { ApiErrorSchema } = await import('@objectstack/spec/api'); + const { res } = await post({ + dataset: inlineDataset, + selection: { measures: ['revenue'], compareTo: { kind: 'nonsense' } }, + }); + const parsed = (ApiErrorSchema as any).safeParse({ + code: res.body.code, + message: res.body.message, + httpStatus: res.statusCode, + }); + expect(parsed.success, JSON.stringify(parsed.success ? null : parsed.error.issues)).toBe(true); + }); + + it('CONTROL — the same selection with a declared kind still answers 200', async () => { + const selection = { + dimensions: ['region'], + measures: ['revenue'], + timeDimensions: [{ dimension: 'close_date', dateRange: 'last_30_days' }], + compareTo: { kind: 'previousPeriod' }, + }; + const { res, queryDataset } = await post({ dataset: inlineDataset, selection }); + expect(res.statusCode).toBe(200); + expect(queryDataset).toHaveBeenCalledTimes(1); + expect(queryDataset.mock.calls[0][1]).toBe(selection); + }); + + it('the door and the executor answer the SAME sentence, differing only in where', async () => { + const { datasetCompareKindRefusalMessage } = await import('@objectstack/spec/api'); + const { res } = await post({ + dataset: inlineDataset, + selection: { measures: ['revenue'], compareTo: { kind: 'nonsense' } }, + }); + // The verdict clause — everything before the origin clause — is what + // both raise. Pinning it here is what keeps a second wording from + // arriving at this door later. + const verdict = datasetCompareKindRefusalMessage('nonsense', 'schema').split(' Refused at')[0]; + expect(res.body.message).toContain(verdict); + }); +}); diff --git a/packages/rest/src/analytics-filter-refusal-envelope.test.ts b/packages/rest/src/analytics-filter-refusal-envelope.test.ts index 52808e27f43..b75092c4128 100644 --- a/packages/rest/src/analytics-filter-refusal-envelope.test.ts +++ b/packages/rest/src/analytics-filter-refusal-envelope.test.ts @@ -45,6 +45,13 @@ * be re-labelled with a code of its own choosing. */ +// [#17551] The dynamic `import()`s below are paid HERE, at module scope, so the +// transform lands during COLLECTION rather than inside a clocked window +// (`pnpm check:test-source-alias`; this package resolves the specifier through +// `dist/`). The dynamic calls stay where they are — this only decides where the +// first load is paid. +import '@objectstack/spec/api'; + import { describe, it, expect, vi, beforeAll, afterAll } from 'vitest'; import type { Logger } from '@objectstack/spec/contracts'; import { AnalyticsService } from '@objectstack/service-analytics'; @@ -191,26 +198,6 @@ describe('[#5352] POST /analytics/dataset/query — a filter refusal reaches the runtimeFilter: { amount: { $between: [10] } }, message: /needs a two-element \[min, max\] array/, }, - { - // FLIPPED with the #5322 ruling (2026-08-04): this entry was `{$or: []}` - // pinning the "requires a non-empty array" refusal. The empty array is - // now the OR identity — FALSE, zero rows, asserted in the #5322 block - // below — so the refusal that survives at the same guard site is the - // non-array spelling, same envelope. - name: 'an $or that is not an array', - runtimeFilter: { $or: 'won' }, - message: /"\$or" requires an array of filter objects/, - }, - { - name: 'an $or branch that is not a filter object', - runtimeFilter: { $or: [{ stage: 'won' }, 'nope'] }, - message: /branches must be filter objects/, - }, - { - name: 'a $not of a non-object', - runtimeFilter: { $not: 5 }, - message: /"\$not" requires a filter object/, - }, { name: 'an unsupported top-level operator', runtimeFilter: { $nor: [{ stage: 'won' }] }, @@ -229,6 +216,85 @@ describe('[#5352] POST /analytics/dataset/query — a filter refusal reaches the } }); +/** + * [#17551] Three spellings that used to reach the normalizer now stop one layer + * earlier — at the route's schema door — and the block above no longer claims + * them. + * + * ⚠️ This is a CODE change on a live wire surface, so it is recorded with the + * measurement that justifies it rather than as a test edit. Since #17551 the + * dataset route parses its whole `selection` against `DatasetSelectionSchema`, + * whose `runtimeFilter` IS the canonical `FilterConditionSchema` — the same + * declaration the SIBLING route's `where` carries. Measured on both schemas, + * spelling for spelling: + * + * | `runtimeFilter` / `where` | `/analytics/query` | `/analytics/dataset/query` | + * |:---|:---|:---| + * | `{ $or: 'won' }` | refused at the schema | refused at the schema | + * | `{ $or: [{…}, 'nope'] }` | refused at the schema | refused at the schema | + * | `{ $not: 5 }` | refused at the schema | refused at the schema | + * | `{ stage: {} }` | passes the schema | passes the schema | + * | `{ amount: { $between: [10] } }` | passes the schema | passes the schema | + * | `{ $nor: [{…}] }` | passes the schema | passes the schema | + * | `{ $or: [] }` | passes the schema | passes the schema | + * + * ⇒ the two routes now answer this field IDENTICALLY, which is the whole reason + * the door exists ("one family, two postures" was the defect). The three rows + * that changed changed because the dataset route used to be the LOOSER of the + * two, not because anything narrowed past `FilterCondition`. + * + * ⛔ Nothing here weakens #5352's subject: the seam it exists for — a real + * `AnalyticsService`, a real `normalizeAnalyticsFilterTree` refusal, and this + * route's catch reading the envelope rather than a message list — is still + * driven by every case left in the block above, `$sortOf` included. + */ +describe('[#17551] the structurally-malformed filter spellings are refused at the door', () => { + const AT_THE_DOOR: Array<{ name: string; runtimeFilter: unknown }> = [ + { name: 'an $or that is not an array', runtimeFilter: { $or: 'won' } }, + { name: 'an $or branch that is not a filter object', runtimeFilter: { $or: [{ stage: 'won' }, 'nope'] } }, + { name: 'a $not of a non-object', runtimeFilter: { $not: 5 } }, + ]; + + for (const c of AT_THE_DOOR) { + it(`${c.name} → 400 VALIDATION_FAILED, located on the member`, async () => { + const route = buildRoute(async () => realAnalytics()); + const res = await post(route, { dataset, selection: { ...selection, runtimeFilter: c.runtimeFilter } }); + + expect(res.statusCode).toBe(400); + // Still the caller's mistake, still a 400 — #5352's own invariant. + expect(res.statusCode).not.toBe(500); + expect(res.body.code).not.toBe('ANALYTICS_QUERY_FAILED'); + expect(res.body.code).toBe('VALIDATION_FAILED'); + // …and it says WHICH member, which the deeper refusal never did. + const fields: Array<{ field: string }> = res.body.details.fields; + expect(fields.map((f) => f.field).some((f) => f.startsWith('selection.runtimeFilter'))).toBe(true); + }); + } + + it('CONTROL — the sibling route\'s own schema refuses the same three, so this is one posture', async () => { + const { AnalyticsQueryRequestSchema } = await import('@objectstack/spec/api'); + for (const c of AT_THE_DOOR) { + const parsed = (AnalyticsQueryRequestSchema as any).safeParse({ + cube: 'opportunity', + measures: ['revenue'], + where: c.runtimeFilter, + }); + expect(parsed.success, `${c.name} must be refused by the sibling schema too`).toBe(false); + } + }); + + it('CONTROL — the four spellings the schema PASSES still cross the seam', async () => { + const { AnalyticsQueryRequestSchema } = await import('@objectstack/spec/api'); + const passes = [{ stage: {} }, { amount: { $between: [10] } }, { $nor: [{ stage: 'won' }] }, { $or: [] }]; + for (const where of passes) { + const parsed = (AnalyticsQueryRequestSchema as any).safeParse({ + cube: 'opportunity', measures: ['revenue'], where, + }); + expect(parsed.success, `${JSON.stringify(where)} must still pass the schema`).toBe(true); + } + }); +}); + describe('[#5322] empty combinators are boolean identities at the REST face — evaluated, not refused', () => { // Until the 2026-08-04 #5322 ruling, `{$or: []}` sat in REFUSALS above and // this route answered it 400 ("requires a non-empty array"). The ruling took diff --git a/packages/rest/src/analytics-selection-door.ts b/packages/rest/src/analytics-selection-door.ts index b9dd1734ba8..c773dbebf96 100644 --- a/packages/rest/src/analytics-selection-door.ts +++ b/packages/rest/src/analytics-selection-door.ts @@ -1,59 +1,50 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [#17058] The door parse for `POST {basePath}/analytics/dataset/query`'s - * `selection` — the half of the analytics family this route never had. + * The door parse for `POST {basePath}/analytics/dataset/query`'s `selection`. * - * ## The gap + * ## The gap, and the two rounds that closed it * - * `/analytics/query` and `/analytics/sql` Zod-parse their body at the entry - * (`runtime/src/domains/analytics.ts` → `assertAnalyticsQueryBody`) and lift a - * malformed member to a 400 before the service is reached. The dataset route - * checked only that `selection.measures` was a non-empty array, so every other - * member travelled into `dataset-executor` unrefused and was answered by + * ⚠️ The first round is cited by its PULL REQUEST throughout this file. The card + * it closed is no longer on this board — `check:issue-citations` classes that + * number `allocated-but-absent`, and deleted-vs-transferred is NOT MEASURED — + * so PR #17548 is the live record, and its own body names the card it closed. + * + * [PR #17548] `/analytics/query` and `/analytics/sql` Zod-parse their body at the + * entry (`runtime/src/domains/analytics.ts` → `assertAnalyticsQueryBody`) and + * lift a malformed member to a 400 before the service is reached. The dataset + * route checked only that `selection.measures` was a non-empty array, so every + * other member travelled into `dataset-executor` unrefused and was answered by * whatever the face behind it happened to do with it. That is the same door, * one family, two postures — the inconsistency a client cannot predict. * - * ## Why this is a PROJECTION and not a reuse of the siblings' schema - * - * ⚠️ Measured before writing a line, because the card left it open: **the - * dataset route's `selection` is NOT the sibling routes' shape.** It is - * `DatasetSelection` (`spec/contracts/analytics-service.ts`), and against - * `AnalyticsQueryRequestSchema` a perfectly legal selection fails twice over — - * the sibling schema requires `cube` (a dataset selection never carries one: - * the dataset is addressed by `body.dataset` / `body.datasetName`) and it is - * `.strict()`, so `runtimeFilter`, `dateGranularity`, `compareTo` and `totals` - * are all rejected as unrecognized keys. ⛔ Reusing it would refuse every real - * dashboard widget — a far worse defect than the one being fixed. - * - * What IS shared is member-by-member, and it is most of the shape. Seven of - * `DatasetSelection`'s eleven members declare exactly the type the - * `AnalyticsQuery` member of the same name declares: - * - * | member | `DatasetSelection` | `AnalyticsQuery` | - * |:---|:---|:---| - * | `dimensions` | `string[]?` | `string[]?` | - * | `measures` | `string[]` | `string[]` | - * | `timeDimensions` | `AnalyticsQuery['timeDimensions']` — declared BY REFERENCE | itself | - * | `order` | `Record?` | same | - * | `limit` / `offset` | `number?` | same | - * | `timezone` | `string?` | same | - * - * So parsing those seven against `AnalyticsQuerySchema.pick(…)` enforces the - * contract `DatasetSelection` already declares — a pull-back onto published - * text, never a narrowing past it. The four dataset-only members - * (`runtimeFilter`, `dateGranularity`, `compareTo`, `totals`) are PROJECTED - * AWAY before the parse, deliberately: `.pick()` carries `.strict()` through, - * so handing the raw selection to the picked schema would reject them. - * - * ⚠️ Those four therefore still have no door. `DatasetSelection` is a - * TypeScript interface with no Zod schema anywhere in the repo, and authoring - * one belongs in `packages/spec` beside the interface (Prime Directive #1), - * not here in a consumer — a second declaration of a spec-owned wire shape is - * the dialect Prime Directive #12 exists to prevent. Filed separately; this - * module is deliberately the derivable half. - * - * ## The refusal shapes, and why the date-range code is not spelled here + * That PR's own answer was PARTIAL and said so: `DatasetSelection` had no Zod + * schema anywhere in the repo, so this module parsed a PROJECTION — the seven + * members whose declarations coincide with `AnalyticsQuery`'s — and + * deliberately projected the four dataset-only members (`runtimeFilter`, + * `dateGranularity`, `compareTo`, `totals`) AWAY. Reusing the siblings' schema + * for the whole selection was ⛔ not available and that was measured rather + * than assumed: `AnalyticsQueryRequestSchema` requires `cube` (a dataset + * selection carries none — the dataset is addressed by `body.dataset` / + * `body.datasetName`) and is `.strict()`, so a legal selection failed it on + * `cube` **plus** all four members above, which would have 400'd every real + * dashboard widget. + * + * [#17551, ruled — decision batch #204 item 3, letter A] The missing half is + * now declared where it belongs: `DatasetSelectionSchema` + * (`@objectstack/spec/api`, beside the `AnalyticsQueryRequestSchema` the + * sibling routes parse) is the ONE declaration of this wire shape, and + * `@objectstack/spec/contracts` re-exports its type rather than carrying a + * second interface. So this module parses the **whole** selection against it, + * and the projection is gone. + * + * ⛔ Assembling the missing members out of spec-exported parts HERE was + * refused by name in that ruling: it is exactly the second declaration of a + * spec-owned wire shape Prime Directive #12 exists to prevent. This module + * owns the ENVELOPE — which refusal shape a failure lands in, and how a field + * path is spelled against the request body — and owns no part of the contract. + * + * ## The refusal shapes * * Two answers, matching the family: * @@ -75,6 +66,13 @@ * keep one wording, which a second spelling quietly ends. That constructor's * own TSDoc names this route as the caller it was waiting for. * + * The same convention governs the newly-doored members, and it is why no + * sentence about them is written here either: an unrecognised `compareTo.kind` + * answers {@link datasetCompareKindRefusalMessage}, the builder + * `service-analytics`'s `shiftRange` also raises for the in-process caller that + * never posted a body, and every unknown-key refusal is the schema's own + * `strictObject` prescription. + * * The `message` is built the way the sibling builds it — `: ` * joined — over `fieldsFromZodIssues` (`@objectstack/types`), which is * `zodIssuesToFields`, the one ADR-0114 D3 mapper, plus the two things every @@ -82,12 +80,16 @@ * of the date-range union's own arm RESTATEMENT, so an arity refusal reaches * the wire with ONE wording rather than the prescription followed by zod's * `Too small: expected array to have >=2 items`. That collapse lives in the one - * mapper both analytics doors share, ⛔ never as a second copy here — which is - * why this door reads the wrapper rather than the raw D3 function. (The rename - * is inert here: the projection is always an object built from declared members - * only, so no issue of this parse lands at the root.) Field paths are prefixed - * `selection.` because they are reported against the REQUEST body, where the - * parsed object sits one level down. + * mapper both analytics doors share, ⛔ never as a second copy here. + * + * ⚠️ The root rename is LIVE here since #17551 and was inert before it. The + * projection was an object this module built out of declared members only, so + * no issue of that parse could land at the root; the full selection is + * `.strict()`, and an unrecognized-keys issue lands at exactly the root. The + * mapper spells that position `(body)`, which is true for the sibling routes — + * their body IS the query — and false here, where the parsed object sits under + * `selection`. So the root is re-spelled `selection` and every deeper path is + * prefixed `selection.`, because both are reported against the REQUEST body. * * Validation-only: the caller's `selection` is forwarded to the service * untouched, never the parse output — the rule `assertAnalyticsQueryBody` @@ -98,29 +100,6 @@ import { fieldsFromZodIssues } from '@objectstack/types'; import { analyticsDateRangeUnrecognizedError } from '@objectstack/core'; -/** - * The `DatasetSelection` members whose declared type IS the `AnalyticsQuery` - * member of the same name — the projection this door parses. - * - * ⛔ Adding a member here is a claim about the two declarations agreeing: - * check `DatasetSelection` in `spec/contracts/analytics-service.ts` against - * `AnalyticsQuerySchema` in `spec/data/analytics.zod.ts` first. A member that - * only LOOKS alike (`runtimeFilter` vs `where` — same `FilterCondition`, a - * different key on each side) does not belong: this list is what makes the - * parse a pull-back rather than a new contract. `.pick()` is type-checked - * against the schema, so a member that leaves `AnalyticsQuery` fails the - * build here rather than silently dropping out of coverage. - */ -export const SELECTION_MEMBERS_SHARED_WITH_ANALYTICS_QUERY = [ - 'dimensions', - 'measures', - 'timeDimensions', - 'order', - 'limit', - 'offset', - 'timezone', -] as const; - /** A door refusal, ready for `res.status(...).json(...)`. */ export interface DatasetSelectionRefusal { status: number; @@ -128,51 +107,36 @@ export interface DatasetSelectionRefusal { } /** - * Built on first use and memoised — `@objectstack/spec/data` stays off this + * Built on first use and memoised — `@objectstack/spec/api` stays off this * module's init path, the same lazy `await import` the analytics route already * performs for `DatasetSchema`. */ -let sharedSelectionSchema: { safeParse(input: unknown): any } | undefined; +let selectionSchema: { safeParse(input: unknown): any } | undefined; -async function getSharedSelectionSchema(): Promise<{ safeParse(input: unknown): any }> { - if (!sharedSelectionSchema) { - const { AnalyticsQuerySchema } = await import('@objectstack/spec/data'); - sharedSelectionSchema = (AnalyticsQuerySchema as any).pick({ - dimensions: true, - measures: true, - timeDimensions: true, - order: true, - limit: true, - offset: true, - timezone: true, - }); +async function getSelectionSchema(): Promise<{ safeParse(input: unknown): any }> { + if (!selectionSchema) { + const { DatasetSelectionSchema } = await import('@objectstack/spec/api'); + selectionSchema = DatasetSelectionSchema as unknown as { safeParse(input: unknown): any }; } - return sharedSelectionSchema!; + return selectionSchema!; } /** - * Parse the shared members of a dataset `selection` and describe the refusal, - * or `undefined` when the selection passes. + * Parse a dataset `selection` and describe the refusal, or `undefined` when the + * selection passes. * * A non-object `selection` answers `undefined`: the route's own check ahead of * this one (`selection.measures` must be a non-empty array) already owns that * case and answers it with a message naming the member, which is the better - * sentence for by far the most common mistake. This function is about the - * members that had no door at all. + * sentence for by far the most common mistake. */ export async function datasetSelectionRefusal( selection: unknown, ): Promise { if (!selection || typeof selection !== 'object' || Array.isArray(selection)) return undefined; - const source = selection as Record; - const projection: Record = {}; - for (const member of SELECTION_MEMBERS_SHARED_WITH_ANALYTICS_QUERY) { - if (member in source) projection[member] = source[member]; - } - - const schema = await getSharedSelectionSchema(); - const parsed = schema.safeParse(projection); + const schema = await getSelectionSchema(); + const parsed = schema.safeParse(selection); if (parsed.success) return undefined; const issues: Array<{ @@ -181,9 +145,12 @@ export async function datasetSelectionRefusal( message: string; input?: unknown; }> = parsed.error.issues; - const fields = fieldsFromZodIssues(issues, projection).map((entry) => ({ + const fields = fieldsFromZodIssues(issues, selection).map((entry) => ({ ...entry, - field: `selection.${entry.field}`, + // `(body)` is the mapper's name for the ROOT, correct on the sibling + // routes whose body IS the parsed object and wrong here, where it sits + // one level down under `selection`. + field: entry.field === '(body)' ? 'selection' : `selection.${entry.field}`, })); const message = `Invalid dataset selection: ${fields .map((f) => `${f.field}: ${f.message}`) diff --git a/packages/rest/src/rest-server.ts b/packages/rest/src/rest-server.ts index a5a7ed3dbc1..5004bb98b64 100644 --- a/packages/rest/src/rest-server.ts +++ b/packages/rest/src/rest-server.ts @@ -305,7 +305,7 @@ import { runImport } from './import-runner.js'; // [#16581] The public picker's authoring-dialect → parser-grammar lowering. import { lowerViewFilterRules } from './view-filter-rule-lowering.js'; import { prepareImportRequest } from './import-prepare.js'; -// [#17058] The `POST …/analytics/dataset/query` door parse — the half of the +// [#17551] The `POST …/analytics/dataset/query` door parse — the half of the // analytics family this route never had. See the module header for the // measurement that decides its shape. import { datasetSelectionRefusal } from './analytics-selection-door.js'; @@ -11131,7 +11131,7 @@ export class RestServer { }); } - // [#17058] …and every OTHER member of `selection` had no + // [PR #17548] …and every OTHER member of `selection` had no // door at all, so a malformed one travelled into // `dataset-executor` and was answered by whatever the face // behind it happened to do with it — while the sibling @@ -11139,13 +11139,18 @@ export class RestServer { // identical failure to a 400 at the entry. One family, two // postures, decided by which door the client knocked on. // - // The parse is a PROJECTION, never the siblings' schema: - // `selection` is a `DatasetSelection`, which is NOT the - // `AnalyticsQuery` the siblings parse — it carries no - // `cube` and has four members of its own, so the sibling - // schema would 400 every real dashboard widget. - // {@link datasetSelectionRefusal} carries that measurement - // and the reason those four are deliberately left out. + // [#17551, ruled] The parse is the WHOLE selection, against + // `DatasetSelectionSchema` — the one declaration of this + // wire shape, authored in `packages/spec` beside the + // sibling routes' own request body. ⛔ Never the siblings' + // schema: `selection` is a `DatasetSelection`, which + // carries no `cube` and has four members of its own, so + // `AnalyticsQueryRequestSchema` would 400 every real + // dashboard widget. PR #17548 could only door the seven + // members whose declarations coincided; the four that were + // left — `runtimeFilter`, `dateGranularity`, `compareTo`, + // `totals` — are what this closes. {@link datasetSelectionRefusal} + // carries both measurements. // // Validation-only: the caller's `selection` is what reaches // `queryDataset` below, never a parse output. diff --git a/packages/services/service-analytics/src/dataset-executor.ts b/packages/services/service-analytics/src/dataset-executor.ts index 9bd01563857..d07fa7f5c7f 100644 --- a/packages/services/service-analytics/src/dataset-executor.ts +++ b/packages/services/service-analytics/src/dataset-executor.ts @@ -7,6 +7,7 @@ import type { DatasetSelection, DatasetCompareTo, } from '@objectstack/spec/contracts'; +import { datasetCompareKindRefusalMessage } from '@objectstack/spec/api'; import { emptyGroupValueFor, type FilterCondition } from '@objectstack/spec/data'; import type { ExecutionContext } from '@objectstack/spec/kernel'; import { @@ -717,12 +718,21 @@ function resolveCompareDimension(selection: DatasetSelection): string { * ## Why an unrecognised `kind` is REFUSED here, not fallen through * * `kind` is declared as the closed pair `'previousPeriod' | 'previousYear'` - * (`DatasetCompareTo`, `spec/contracts/analytics-service.ts`) and enforced on - * the wire by NOTHING: `DatasetSelection` has no Zod schema anywhere, and - * `/analytics/dataset/query`'s own door projects `compareTo` away before its - * parse and forwards the caller's selection untouched (`analytics-selection-door` - * says so in its header). So a body carrying `compareTo: { kind: 'previousQuarter' }` - * reaches this function with its declared type unenforced. + * (`DatasetCompareTo`, re-exported by `spec/contracts/analytics-service.ts` + * from `DatasetCompareToSchema`). ⚠️ When this refusal was written the wire + * enforced NOTHING — `DatasetSelection` had no Zod schema anywhere, and + * `/analytics/dataset/query`'s door projected `compareTo` away before its + * parse — so a body carrying `compareTo: { kind: 'previousQuarter' }` reached + * this function with its declared type unenforced. #17551 closed that: the + * route now parses the whole selection against `DatasetSelectionSchema`. + * + * ⛔ That does NOT make this refusal redundant, and the reason is the reason it + * was written here: `shiftRange` is a PUBLISHED export of this package + * (`src/index.ts:35`, advertised by `README.md:160`) and this module is + * reachable in-process through `IAnalyticsService.queryDataset` by a caller + * that never posted a body at all. The HTTP door is one of its callers, not its + * only one — which is exactly why the SENTENCE is shared rather than copied + * (see the `default` arm below). * * The shape this function used to have — one `if` for `previousYear`, then the * previousPeriod arm as a FALL-THROUGH — answered that body with a @@ -761,11 +771,15 @@ export function shiftRange(range: [string, string], kind: CompareTo['kind']): [s } default: { const exhaustive: never = kind; + // ⛔ The sentence is NOT spelled here. It is the one builder the schema + // door also raises (`datasetCompareKindRefusalMessage`, @objectstack/spec/api), + // called with `'runtime'` because this site is reached by a caller that + // never posted a body — one condition, one wording, and the clause that + // says WHERE it was refused is the only part that differs (#5240; the + // date-range family's `analyticsDateRangeRefusalMessage` is the same + // split, for the same reason). throw datasetInvalidError( - `[dataset-executor] compareTo.kind ${JSON.stringify(exhaustive)} is not a comparison window ` - + 'this executor implements. The two it runs are \'previousPeriod\' (the equal-length window ' - + 'ending the day before this one starts) and \'previousYear\' (the same window one calendar ' - + 'year back). Name one of those, or drop compareTo.', + `[dataset-executor] ${datasetCompareKindRefusalMessage(exhaustive, 'runtime')}`, ); } } diff --git a/packages/services/service-analytics/src/date-range-array-arm.ts b/packages/services/service-analytics/src/date-range-array-arm.ts index bc042a70ed4..cb534637d9a 100644 --- a/packages/services/service-analytics/src/date-range-array-arm.ts +++ b/packages/services/service-analytics/src/date-range-array-arm.ts @@ -59,9 +59,9 @@ * - so tightening `AnalyticsDateRangeSchema` is ⛔ not "deliberately NOT done * here" any more: it is done, upstream, where the contract lives; * - and `POST /analytics/dataset/query` is ⛔ no longer a route that never - * Zod-parses its selection — since PR #17548 it parses the selection's - * shared members, `timeDimensions` among them, against - * `AnalyticsQuerySchema.pick(…)` ahead of the executor + * Zod-parses its selection — PR #17548 doored the selection's shared + * members, `timeDimensions` among them, and #17551 widened that parse to + * the whole selection against `DatasetSelectionSchema` * (`rest/src/analytics-selection-door.ts`, wired in `rest-server.ts`), so on * THAT route the schema door is AHEAD of these faces, not behind them. * diff --git a/packages/spec/api-surface/api.json b/packages/spec/api-surface/api.json index 831b944abfe..789b68f032f 100644 --- a/packages/spec/api-surface/api.json +++ b/packages/spec/api-surface/api.json @@ -261,6 +261,12 @@ "DataLoaderConfigParsed (type)", "DataLoaderConfigSchema (const)", "DataProtocol (interface)", + "DatasetCompareTo (type)", + "DatasetCompareToSchema (const)", + "DatasetSelection (type)", + "DatasetSelectionSchema (const)", + "DatasetTotals (type)", + "DatasetTotalsSchema (const)", "DeduplicationStrategy (const)", "DeduplicationStrategy (type)", "DeleteDataRequest (type)", @@ -1118,6 +1124,7 @@ "WebSocketServerConfigSchema (const)", "WellKnownCapabilities (type)", "WellKnownCapabilitiesSchema (const)", + "datasetCompareKindRefusalMessage (function)", "envelopeViolations (function)", "getAuthEndpointUrl (function)", "getDefaultRouteRegistrations (function)", diff --git a/packages/spec/api-surface/contracts.json b/packages/spec/api-surface/contracts.json index 51ee881a05a..66834273eb0 100644 --- a/packages/spec/api-surface/contracts.json +++ b/packages/spec/api-surface/contracts.json @@ -70,8 +70,8 @@ "CryptoHandle (interface)", "CubeMeta (interface)", "DEFAULT_STORAGE_LIST_LIMIT (const)", - "DatasetCompareTo (interface)", - "DatasetSelection (interface)", + "DatasetCompareTo (type)", + "DatasetSelection (type)", "DefineSharingRuleInput (interface)", "DelegableAdminScope (interface)", "DelegableScope (interface)", diff --git a/packages/spec/authorable-surface/api.json b/packages/spec/authorable-surface/api.json index 3770b6729d1..dcfe6ac3aef 100644 --- a/packages/spec/authorable-surface/api.json +++ b/packages/spec/authorable-surface/api.json @@ -467,6 +467,20 @@ "api/DataLoaderConfig:coalesceRequests", "api/DataLoaderConfig:maxBatchSize", "api/DataLoaderConfig:maxConcurrency", + "api/DatasetCompareTo:dimension", + "api/DatasetCompareTo:kind", + "api/DatasetSelection:compareTo", + "api/DatasetSelection:dateGranularity", + "api/DatasetSelection:dimensions", + "api/DatasetSelection:limit", + "api/DatasetSelection:measures", + "api/DatasetSelection:offset", + "api/DatasetSelection:order", + "api/DatasetSelection:runtimeFilter", + "api/DatasetSelection:timeDimensions", + "api/DatasetSelection:timezone", + "api/DatasetSelection:totals", + "api/DatasetTotals:groupings", "api/DeleteDataRequest:expectedVersion", "api/DeleteDataRequest:id", "api/DeleteDataRequest:object", diff --git a/packages/spec/declaration-map/api.json b/packages/spec/declaration-map/api.json index 8548a1eee5b..1ec93321851 100644 --- a/packages/spec/declaration-map/api.json +++ b/packages/spec/declaration-map/api.json @@ -189,6 +189,12 @@ "DataEventType": "api/DataEventType", "DataLoaderConfig": "api/DataLoaderConfig", "DataLoaderConfigSchema": "api/DataLoaderConfig", + "DatasetCompareTo": "api/DatasetCompareTo", + "DatasetCompareToSchema": "api/DatasetCompareTo", + "DatasetSelection": "api/DatasetSelection", + "DatasetSelectionSchema": "api/DatasetSelection", + "DatasetTotals": "api/DatasetTotals", + "DatasetTotalsSchema": "api/DatasetTotals", "DeduplicationStrategy": "api/DeduplicationStrategy", "DeleteDataRequest": "api/DeleteDataRequest", "DeleteDataRequestSchema": "api/DeleteDataRequest", diff --git a/packages/spec/dropped-refinements.baseline.json b/packages/spec/dropped-refinements.baseline.json index d8981a2bc98..38c06eb037d 100644 --- a/packages/spec/dropped-refinements.baseline.json +++ b/packages/spec/dropped-refinements.baseline.json @@ -2,8 +2,8 @@ "description": "Shrink-only ledger of every PUBLISHED JSON Schema that is STILL WIDER than the Zod type it was generated from, because a rule written as `.refine()` reaches the runtime and not the file (#18670). `z.toJSONSchema()` has no arm for a `custom` check: a plain record, the same record with a `.refine()`, and the same record with an ABORTING `.refine()` all project byte-identically (measured on zod 4.4.3, the version packages/spec resolves). So a document one of these files ACCEPTS can still be refused at parse time, and an author -- or an AI -- validating against packages/spec/json-schema/** finds out a release later. Each `sites` path is a position under that schema at which a refinement is dropped; the same paths are written onto the artifact itself as `x-dropped-refinements`. Item 2 closed the first patterns: a refinement DECLARED through the closed list in src/shared/refinement-projection.ts is emitted into the published file, reads `projected` rather than `dropped`, and its row LEAVES this ledger in the same PR -- which is why the ledger shrinks and never grows on a repair. Every refinement outside that closed list stays here, and adding an arm to the list is a public-contract decision, not a refactor. Hand-edited on purpose and with no `gen:` script: a generator would let a new gap be admitted by running a command instead of by a decision, which is the silence this ledger exists to end. Adding, removing or moving a site fails packages/spec/scripts/build-schemas.ts until the line moves with it, and the failure prints the corrected entry in full. ⛔ Do not delete or weaken a refinement to shorten this file -- the runtime rule is correct; it is the projection that is silent, and the remedy is to teach the closed list a NAMED pattern, never to drop the rule.", "measured": { "zod": "4.4.3", - "publishedSchemasWithDroppedRefinements": 204, - "droppedRefinementSites": 560, + "publishedSchemasWithDroppedRefinements": 205, + "droppedRefinementSites": 561, "refinementSitesThatDidProject": 366, "refinementSitesWithNoJsonFormToCompare": 9 }, @@ -109,6 +109,11 @@ "" ] }, + "api/DatasetSelection": { + "sites": [ + "runtimeFilter.lazy" + ] + }, "api/DisablePackageResponse": { "sites": [ "package.manifest.navigationContributions.element.items.element.lazy.options[0]" diff --git a/packages/spec/export-origins/api.json b/packages/spec/export-origins/api.json index e0c7d725f3f..05b68eced5d 100644 --- a/packages/spec/export-origins/api.json +++ b/packages/spec/export-origins/api.json @@ -247,6 +247,12 @@ "DataLoaderConfigParsed": "src/api/contract.zod.ts#DataLoaderConfigParsed (type)", "DataLoaderConfigSchema": "src/api/contract.zod.ts#DataLoaderConfigSchema (const)", "DataProtocol": "src/api/protocol.zod.ts#DataProtocol (interface)", + "DatasetCompareTo": "src/api/analytics.zod.ts#DatasetCompareTo (type)", + "DatasetCompareToSchema": "src/api/analytics.zod.ts#DatasetCompareToSchema (const)", + "DatasetSelection": "src/api/analytics.zod.ts#DatasetSelection (type)", + "DatasetSelectionSchema": "src/api/analytics.zod.ts#DatasetSelectionSchema (const)", + "DatasetTotals": "src/api/analytics.zod.ts#DatasetTotals (type)", + "DatasetTotalsSchema": "src/api/analytics.zod.ts#DatasetTotalsSchema (const)", "DeduplicationStrategy": "src/api/export.zod.ts#DeduplicationStrategy (type)", "DeleteDataRequest": "src/api/protocol.zod.ts#DeleteDataRequest (type)", "DeleteDataRequestSchema": "src/api/protocol.zod.ts#DeleteDataRequestSchema (const)", @@ -1070,6 +1076,7 @@ "WebSocketServerConfigSchema": "src/api/websocket.zod.ts#WebSocketServerConfigSchema (const)", "WellKnownCapabilities": "src/api/discovery.zod.ts#WellKnownCapabilities (type)", "WellKnownCapabilitiesSchema": "src/api/discovery.zod.ts#WellKnownCapabilitiesSchema (const)", + "datasetCompareKindRefusalMessage": "src/api/analytics.zod.ts#datasetCompareKindRefusalMessage (function)", "envelopeViolations": "src/api/contract.zod.ts#envelopeViolations (function)", "getAuthEndpointUrl": "src/api/auth-endpoints.zod.ts#getAuthEndpointUrl (function)", "getDefaultRouteRegistrations": "src/api/plugin-rest-api.zod.ts#getDefaultRouteRegistrations (function)", diff --git a/packages/spec/export-origins/contracts.json b/packages/spec/export-origins/contracts.json index 5f2927f21a1..846c1823023 100644 --- a/packages/spec/export-origins/contracts.json +++ b/packages/spec/export-origins/contracts.json @@ -70,8 +70,8 @@ "CryptoHandle": "src/contracts/crypto-provider.ts#CryptoHandle (interface)", "CubeMeta": "src/contracts/analytics-service.ts#CubeMeta (interface)", "DEFAULT_STORAGE_LIST_LIMIT": "src/contracts/storage-service.ts#DEFAULT_STORAGE_LIST_LIMIT (const)", - "DatasetCompareTo": "src/contracts/analytics-service.ts#DatasetCompareTo (interface)", - "DatasetSelection": "src/contracts/analytics-service.ts#DatasetSelection (interface)", + "DatasetCompareTo": "src/api/analytics.zod.ts#DatasetCompareTo (type)", + "DatasetSelection": "src/api/analytics.zod.ts#DatasetSelection (type)", "DefineSharingRuleInput": "src/contracts/sharing-service.ts#DefineSharingRuleInput (interface)", "DelegableAdminScope": "src/contracts/security-service.ts#DelegableAdminScope (interface)", "DelegableScope": "src/contracts/security-service.ts#DelegableScope (interface)", diff --git a/packages/spec/json-schema.manifest/api.json b/packages/spec/json-schema.manifest/api.json index 4a1c8946a86..237945c8387 100644 --- a/packages/spec/json-schema.manifest/api.json +++ b/packages/spec/json-schema.manifest/api.json @@ -101,6 +101,9 @@ "api/DataEvent", "api/DataEventType", "api/DataLoaderConfig", + "api/DatasetCompareTo", + "api/DatasetSelection", + "api/DatasetTotals", "api/DeduplicationStrategy", "api/DeleteDataRequest", "api/DeleteDataResponse", diff --git a/packages/spec/src/api/analytics.zod.ts b/packages/spec/src/api/analytics.zod.ts index 6d18da78182..d48e2ed0bae 100644 --- a/packages/spec/src/api/analytics.zod.ts +++ b/packages/spec/src/api/analytics.zod.ts @@ -2,7 +2,9 @@ import { z } from 'zod'; import { AnalyticsQuerySchema } from '../data/analytics.zod'; -import { AggregationFunction } from '../data/query.zod'; +import { FilterConditionSchema } from '../data/filter.zod'; +import { AggregationFunction, DateGranularity } from '../data/query.zod'; +import { strictObject } from '../shared/strict-object'; import { BaseResponseSchema } from './contract.zod'; import { retiredKey } from '../shared/retired-key'; @@ -234,6 +236,384 @@ export const AnalyticsSqlResponseSchema = lazySchema(() => BaseResponseSchema.ex }), })); +// ========================================== +// 5. Dataset Selection (ADR-0021) +// ========================================== + +/** + * [#17551] The refusal sentence for a `compareTo.kind` outside the closed pair + * — ONE wording for ONE condition, shared by the two doors that can raise it. + * + * The condition is reachable from two places and they must not disagree + * (the #5240 convention; `analyticsDateRangeRefusalMessage` in + * `data/analytics.zod.ts` is the same builder for the date-range vocabulary, + * and this one is written to its shape deliberately): + * + * - **`'schema'`** — {@link DatasetCompareToSchema} refused the value at parse + * time. Every `POST /analytics/dataset/query` body passes through it. + * - **`'runtime'`** — a reader PAST that door refused it: `shiftRange` + * (`service-analytics/src/dataset-executor.ts`), reached in-process by a + * caller that never posted a body at all. + * + * ⛔ There is no default `origin`. A defaulted one makes the same false + * assertion, silently, for every caller who does not think about it: an author + * refused past the door would be sent to inspect a parse that never ran. + * + * @param input - the refused value, exactly as it arrived. + * @param origin - `'schema'` when {@link DatasetCompareToSchema} refused it at + * parse time, `'runtime'` when a reader past that door did. + */ +export function datasetCompareKindRefusalMessage( + input: unknown, + origin: 'schema' | 'runtime', +): string { + const refusedAt = origin === 'schema' + ? 'Refused at the schema (VALIDATION_FAILED / 400)' + : 'Refused past the schema door, by the analytics executor that received it (DATASET_INVALID / 400)'; + return ( + `compareTo.kind ${JSON.stringify(input)} is not a comparison window this platform implements. ` + + "The two it runs are 'previousPeriod' (the equal-length window ending the day before this one " + + "starts) and 'previousYear' (the same window one calendar year back). Name one of those, or " + + `drop compareTo. ${refusedAt}: an unrecognised spelling used to reach the executor and answer ` + + 'with a previous-period comparison under an ordinary 200, which no status, header or field in ' + + 'the response distinguished from the comparison the caller asked for.' + ); +} + +// The `{ offset }` arm authors carry from before #5011. It was never a member +// of this contract: the dashboard widget declared it, `DatasetWidget` forwarded +// it verbatim into a shape with no `offset` in it, and the executor threw +// `compareTo requires a timeDimension "undefined"` — which took the whole +// widget down. `//` rather than a doc comment deliberately (the `COMPARE_TO_*` +// convention in `ui/dashboard.zod.ts`): build-docs lifts JSDoc onto the +// reference page, and an upgrade note is not a doc for a shape that exists. +const COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED = + '`compareTo.offset` is not a member of this contract and never was — the analytics executor has ' + + 'no `offset` concept, so a body carrying one reached it and threw. Write the kind instead: ' + + "`compareTo: { kind: 'previousPeriod' }` for the equal-length window immediately before, " + + "`compareTo: { kind: 'previousYear' }` for the same window a calendar year back — " + + "`{ offset: '1y' }` is exactly `previousYear`. For any other duration (`'7d'`, `'1M'`, …) there " + + 'is no faithful one-key rewrite: state the window you want on the `timeDimensions[]` entry this ' + + 'comparison anchors on, and compare it with `previousPeriod`, which shifts by whatever length ' + + 'that window resolves to.'; + +// The bare-string form the pre-#5011 dashboard documented. It is not a key, so +// `guidance` cannot reach it: it arrives as `invalid_type` (a string where an +// object is declared), which is what `retiredForms` answers. +const COMPARE_TO_STRING_ON_THE_WIRE_RETIRED = (kind: 'previousPeriod' | 'previousYear') => + `\`compareTo: '${kind}'\` (the bare string form) is not a member of this contract — the ` + + 'comparison directive is an object. Write `compareTo: { kind: ' + + `'${kind}' }\` instead: same comparison, spelled the way the analytics executor reads it. Add ` + + '`dimension` only when the selection has more than one dated time dimension; with one, the ' + + 'executor resolves it.'; + +/** + * `DatasetSelection.compareTo` — the period-over-period directive the ADR-0021 + * dataset executor implements, as a closed wire shape. + * + * ## Why this is a TRANSCRIPTION and not a new contract (#17551) + * + * `DatasetCompareTo` has been a published TypeScript interface since #5011 + * (`contracts/analytics-service.ts`) and `dashboard.widgets[].compareTo` + * already declares the same two members as a `strictObject` — the widget's + * copy being, in its own words, 「a thin projection of + * `DatasetSelection.compareTo`」. What had no declaration was the WIRE: the + * dataset route forwarded `compareTo` to the executor unparsed, so the + * authoring path was doored and the HTTP path was not. This schema is the + * already-published text made executable; it adds no member and no accepted + * value. + * + * ⚠️ It is NOT the widget's schema re-used. The two carry different + * prescriptions because they are read by authors at different moments — the + * widget's point at neighbouring WIDGET keys (`options.dateGranularity`, the + * widget's own `filter`), which do not exist on a wire selection. What they + * share is the vocabulary, and that is shared by construction: `kind`'s refusal + * sentence is {@link datasetCompareKindRefusalMessage}, the one builder the + * executor also raises. + */ +export const DatasetCompareToSchema = lazySchema(() => strictObject( + { + surface: 'this compareTo directive', + history: 'Until this shape was closed, an undeclared key here rode the wire into the dataset ' + + 'executor, which read the two members it knows and ignored the rest.', + // The same near-misses `dashboard.widgets[].compareTo` curated in #5042 and + // #5011, minus the two that name widget-only slots: an author reaching for + // a word on the widget reaches for it here too, and the wire is where an + // AI-written body arrives. + aliases: { + type: 'kind', + mode: 'kind', + field: 'dimension', + dateField: 'dimension', + timeDimension: 'dimension', + }, + guidance: { + // The retired `{ offset }` arm (#5011) and the words measured beside it. + // The executor never had an `offset` concept, so this is not a rename. + offset: COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED, + period: COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED, + duration: COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED, + interval: COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED, + shift: COMPARE_TO_OFFSET_ON_THE_WIRE_RETIRED, + granularity: 'a comparison window carries no granularity — it shifts a window, it does not ' + + 'bucket one. Bucketing is `dateGranularity` on the selection itself (or a ' + + "`timeDimensions[]` entry's own `granularity`), and the comparison pass reuses whatever " + + 'the primary pass resolved.', + dateRange: '`compareTo` shifts a window it does not declare — put the window on the ' + + '`timeDimensions[]` entry this comparison anchors on (`{ dimension, dateRange }`) and ' + + 'name that dimension here, or omit `dimension` and let the executor resolve it.', + }, + retiredForms: { + previousPeriod: COMPARE_TO_STRING_ON_THE_WIRE_RETIRED('previousPeriod'), + previousYear: COMPARE_TO_STRING_ON_THE_WIRE_RETIRED('previousYear'), + }, + }, + { + /** + * Which comparison window to run. + * + * `previousPeriod` = the equal-length window immediately before the + * resolved one; `previousYear` = the same window one calendar year back. + * Those are the two the executor implements, and the refusal for anything + * else is the shared sentence, not zod's bare option list — an + * unrecognised spelling used to come back as a previous-period comparison + * under a 200. + */ + kind: z.enum(['previousPeriod', 'previousYear'], { + error: (issue) => datasetCompareKindRefusalMessage(issue.input, 'schema'), + }).describe( + 'Comparison window: previousPeriod (equal-length, immediately before) or previousYear ' + + '(the same window one calendar year back)', + ), + /** + * The time dimension (by name) whose `dateRange` is shifted. + * + * OPTIONAL, and resolved BY THE EXECUTOR when omitted — exactly one dated + * candidate is shifted; zero or several is a loud error naming what it + * found. That rule lives at the producer of the comparison (PD #12), so + * every caller gets the same answer or the same error; ⛔ a consumer must + * never paper over it by guessing a dimension. + */ + dimension: z.string().optional().describe( + 'Time dimension to shift; omit when the selection has exactly one dated time dimension', + ), + }, +)); + +/** + * `DatasetSelection.totals` — the ADR-0021 marginal-aggregate request. + * + * Each grouping is a subset of `dimensions` to additionally aggregate by, and + * `[]` requests the grand total. The selection is re-run grouped only by those + * dimensions, so every total is the measure's TRUE aggregate over the + * underlying rows — the ADR-0021 governance line that forbids client-side + * re-aggregation. + */ +export const DatasetTotalsSchema = lazySchema(() => strictObject( + { + surface: 'this totals request', + history: 'Until this shape was closed, an undeclared key here rode the wire into the dataset ' + + 'executor, which reads `groupings` and nothing else.', + aliases: { + grouping: 'groupings', + groupBy: 'groupings', + subtotals: 'groupings', + }, + guidance: { + // The RESPONSE spells the same idea `dimensions` (`AnalyticsResult.totals[].dimensions`), + // so reaching for it on the REQUEST is a cross-direction near-miss, not a typo. + dimensions: '`dimensions` is how a total is reported back ' + + '(`AnalyticsResult.totals[].dimensions`), not how it is requested. Ask for it as ' + + '`totals: { groupings: [[...dimension names], []] }` — one entry per marginal, `[]` for ' + + 'the grand total.', + grandTotal: 'the grand total is the EMPTY grouping, not a flag: ' + + '`totals: { groupings: [[]] }`.', + }, + }, + { + /** + * One entry per marginal, in request order; the results arrive on + * `AnalyticsResult.totals` in that same order. `[]` is the grand total, so + * a matrix report asks for `{ groupings: [rowDims, columnDims, []] }`. + * + * `order` / `limit` / `offset` do not apply to totals queries — a total + * always covers the full selection. + */ + groupings: z.array(z.array(z.string())).describe( + 'Dimension subsets to additionally aggregate by, in request order; the empty subset is the ' + + 'grand total', + ), + }, +)); + +/** + * `DatasetSelection` — a presentation's selection against a dataset (ADR-0021), + * and the body `POST /api/v1/analytics/dataset/query` posts under `selection`. + * + * Report/dashboard widgets bind to a dataset and pick dimensions/measures BY + * NAME; this is the wire shape that request carries. + * + * ## Why this schema exists, and why it is a NARROWING onto published text + * + * [#17551, ruled] `DatasetSelection` was a TypeScript **interface** with no Zod + * schema anywhere in the repo. PR #17548 put a door on the route, but only over + * the SEVEN members the selection shares with `AnalyticsQuery`, parsed as a + * projection; the four dataset-only members — `runtimeFilter`, + * `dateGranularity`, `compareTo`, `totals` — were 「declared in TypeScript, + * published in the api-surface, and enforced by nothing on the wire」. The + * measured consequence was #17550: `compareTo: { kind: 'nonsense' }` came back + * as a previous-period comparison under a 200, a number a dashboard renders and + * a person reads as fact. + * + * Every member below is a transcription of a member this file's sibling + * (`contracts/analytics-service.ts`) has published since ADR-0021. Nothing is + * added, and nothing that the interface permits is refused. + * + * ## The seven shared members are taken BY REFERENCE, not retyped + * + * ⭐ `dimensions` / `measures` / `timeDimensions` / `order` / `limit` / + * `offset` / `timezone` are read straight off `AnalyticsQuerySchema.shape`. + * The interface already declared `timeDimensions` by reference + * (`AnalyticsQuery['timeDimensions']`), and the door's own projection list is + * a standing claim that the other six agree. Taking the declarations + * themselves makes that claim STRUCTURAL: there is no second copy to drift, + * and a member that leaves `AnalyticsQuery` fails the build here rather than + * silently becoming a private dialect. + * + * ⛔ `runtimeFilter` is deliberately NOT read off `where`. They carry the same + * `FilterCondition`, but they are different keys on each side, and the alias + * table below is what tells an author so. + * + * ## Strict, like every other analytics door + * + * `AnalyticsQuerySchema` has been `.strict()` since #4001 and the sibling + * request body is `.strict()` too. An undeclared key on a selection was never + * declared; it was silently dropped, and the widget behind it answered a + * narrower question than its author asked. It is now named, echoed back, and + * pointed at the canonical key where one exists. + */ +export const DatasetSelectionSchema = lazySchema(() => { + const shared = AnalyticsQuerySchema.shape; + return strictObject( + { + surface: 'this dataset selection', + history: 'Until this shape was closed, `runtimeFilter`, `dateGranularity`, `compareTo` and ' + + '`totals` were declared in TypeScript and enforced by nothing on the wire, and any other ' + + 'key was dropped without a word.', + aliases: { + // ⭐ MEASURED, not invented: `analytics-selection-door.ts` names this + // exact pair as the near-miss its projection list must keep out — + // 「`runtimeFilter` vs `where` — same `FilterCondition`, a different key + // on each side」. The sibling body spells it `where`; a dataset + // selection spells it `runtimeFilter`. + where: 'runtimeFilter', + filter: 'runtimeFilter', + filters: 'runtimeFilter', + // The sibling record dialect spells sorting `orderBy`; the analytics + // dialect spells it `order` (`AnalyticsQuerySchema` carries the same + // entry for the same reason). + orderBy: 'order', + // `ui/dataset.zod.ts` curates exactly these for the same target. + granularity: 'dateGranularity', + granularities: 'dateGranularity', + dateBucket: 'dateGranularity', + bucket: 'dateGranularity', + // The dashboard widget spells its measure list `values`; a selection + // spells it `measures`. + values: 'measures', + }, + guidance: { + cube: '`cube` is an `AnalyticsQuery` member, not a selection member — a dataset selection ' + + 'names no cube. The dataset is addressed one level up, beside `selection`: ' + + '`dataset` (an inline definition) or `datasetName` (a saved one).', + dataset: '`dataset` belongs one level up, beside `selection` — the request body is ' + + '`{ dataset | datasetName, selection }`, and the selection itself carries only the ' + + "dataset's dimension and measure NAMES.", + datasetName: '`datasetName` belongs one level up, beside `selection` — the request body is ' + + '`{ dataset | datasetName, selection }`.', + previewDrafts: '`previewDrafts` is a request-body flag, one level up beside `selection`, ' + + 'not a selection member.', + }, + }, + { + /** Dimension names from the dataset. */ + dimensions: shared.dimensions, + /** Measure names from the dataset (may include derived measures). */ + measures: shared.measures, + /** + * Presentation-scope filter, ANDed with the dataset's intrinsic filter + * at render. Same canonical `FilterCondition` the sibling body spells + * `where` — a different key, deliberately, because it composes with the + * dataset's own filter rather than replacing it. + */ + runtimeFilter: FilterConditionSchema.optional().describe( + "Presentation-scope filter (canonical Query DSL FilterCondition), ANDed with the dataset's " + + 'intrinsic filter at render', + ), + /** Optional time-dimension windows passed through to the runtime. */ + timeDimensions: shared.timeDimensions, + /** + * Presentation-scope date bucketing (framework#3588). Applies to every + * selected dimension the dataset declares as a `date` dimension, so a + * widget can bucket a trend by month without the dataset having to + * declare that granularity for every consumer. + * + * Precedence, per dimension: an explicit `timeDimensions` entry for that + * dimension wins, then this selection-level granularity, then the + * dataset dimension's own `dateGranularity` default. Unset leaves each + * dimension on its dataset default (which may be no bucketing at all — + * grouping by the raw column). + */ + dateGranularity: DateGranularity.optional().describe( + 'Presentation-scope date bucketing applied to every selected `date` dimension; an explicit ' + + "`timeDimensions` entry wins over it, and the dataset dimension's own default is used " + + 'when neither is set', + ), + /** + * Result ordering, applied by key in insertion order + * (`{ revenue: 'desc' }`). + * + * Every key must be a selected dimension, a selected measure, or a + * `__compare` column; anything else is rejected by the executor + * rather than silently ignored. Ordering is applied AFTER measure-scoped + * filters are merged, `compareTo` columns are attached, and derived + * measures are evaluated — so a derived measure (e.g. a win-rate ratio) + * is a valid sort key even though no single SQL statement computes it. + * + * ⛔ The KEYS are judged by the executor, against the dataset this + * selection runs on; this schema judges only the DIRECTION, which is all + * a dataset-free parse can know. + */ + order: shared.order, + /** + * Max rows to return, applied after `order`. When `limit` is set without + * `order`, rows are ordered by the selected dimensions ascending first, + * so the truncated window is deterministic rather than an arbitrary + * subset. + */ + limit: shared.limit, + offset: shared.offset, + /** + * Compare-to directive — runs a shifted query and attaches + * `__compare` columns. + */ + compareTo: DatasetCompareToSchema.optional().describe( + 'Period-over-period comparison window ({ kind, dimension? }); attaches `__compare` ' + + 'columns', + ), + /** + * Server-side totals (matrix subtotals + grand total). Results arrive on + * `AnalyticsResult.totals` in request order. + */ + totals: DatasetTotalsSchema.optional().describe( + 'Server-side marginal aggregates; each grouping is a dimension subset to additionally ' + + 'aggregate by, `[]` being the grand total', + ), + timezone: shared.timezone, + }, + ); +}); + export type AnalyticsEndpoint = z.input; export type AnalyticsQueryRequest = z.input; /** @@ -252,3 +632,19 @@ export type AnalyticsSqlResponse = z.input; /** Post-parse shape of {@link AnalyticsSqlResponse} — defaults applied, transforms run (ADR-0122). */ export type AnalyticsSqlResponseParsed = z.infer; export type GetAnalyticsMetaRequest = z.input; + +/** + * [#17551] The ADR-0021 comparison directive. THE declaration: the + * `DatasetCompareTo` name `@objectstack/spec/contracts` publishes is this type + * re-exported, never a second interface beside it. + */ +export type DatasetCompareTo = z.input; +/** The ADR-0021 marginal-aggregate request (`DatasetSelection.totals`). */ +export type DatasetTotals = z.input; +/** + * [#17551] A presentation’s selection against a dataset (ADR-0021) — the body + * `POST /api/v1/analytics/dataset/query` posts under `selection`. THE + * declaration: the `DatasetSelection` name `@objectstack/spec/contracts` + * publishes is this type re-exported. + */ +export type DatasetSelection = z.input; diff --git a/packages/spec/src/api/dataset-selection.test.ts b/packages/spec/src/api/dataset-selection.test.ts new file mode 100644 index 00000000000..0ae442c1266 --- /dev/null +++ b/packages/spec/src/api/dataset-selection.test.ts @@ -0,0 +1,318 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#17551, ruled] `DatasetSelectionSchema` — the ONE declaration of the wire + * shape `POST /api/v1/analytics/dataset/query` posts under `selection`. + * + * ## What was wrong + * + * `DatasetSelection` was a TypeScript **interface** (`contracts/analytics-service.ts`) + * with no Zod schema anywhere in the repo. PR #17548 doored the route, but only + * over the SEVEN members the selection shares with `AnalyticsQuery`; the four + * dataset-only ones — `runtimeFilter`, `dateGranularity`, `compareTo`, + * `totals` — were 「declared in TypeScript, published in the api-surface, and + * enforced by nothing on the wire」. #17550 is the measured consequence: + * `compareTo: { kind: 'nonsense' }` came back as a previous-period comparison + * under an ordinary **200**, a number a dashboard renders and a person reads as + * fact. + * + * ## What is pinned, and why BOTH directions are here + * + * Every refusal owes two cases. ⛔ A schema that refuses everything passes the + * first kind and is still wrong — so each member below has a malformed value + * that must now be refused WITH ITS REMEDY, and a legal value that must still + * pass. §5 drives the whole eleven-member shape as one specimen. + * + * §1 pins the 「transcription, not a new contract」 claim the ruling turns on, + * and pins it STRUCTURALLY: the seven shared members are asserted to be + * `AnalyticsQuerySchema`'s own declarations BY IDENTITY, so there is no second + * copy that could drift. + */ + +import { describe, it, expect } from 'vitest'; + +import { AnalyticsQuerySchema } from '../data/analytics.zod'; +import { + DatasetCompareToSchema, + DatasetSelectionSchema, + DatasetTotalsSchema, + datasetCompareKindRefusalMessage, +} from './analytics.zod'; + +/** A legal, ordinary dashboard-widget selection — all eleven members. */ +const FULLY_LOADED = { + dimensions: ['region'], + measures: ['revenue'], + runtimeFilter: { region: 'NA' }, + timeDimensions: [{ dimension: 'close_date', granularity: 'month', dateRange: 'last_30_days' }], + dateGranularity: 'month', + order: { revenue: 'desc' }, + limit: 10, + offset: 0, + compareTo: { kind: 'previousPeriod', dimension: 'close_date' }, + totals: { groupings: [['region'], []] }, + timezone: 'Asia/Shanghai', +} as const; + +/** Every issue message a failed parse raised, joined — what an author reads. */ +function refusalText(input: unknown): string { + const parsed = DatasetSelectionSchema.safeParse(input); + expect(parsed.success, `expected a refusal, got a pass: ${JSON.stringify(input)}`).toBe(false); + return parsed.success ? '' : parsed.error.issues.map((i) => i.message).join('\n'); +} + +// ───────────────────────────────────────────────────────────────────────────── +// §1 — the transcription claim, pinned structurally +// ───────────────────────────────────────────────────────────────────────────── + +/** + * The seven members `DatasetSelection` shares with `AnalyticsQuery` — the list + * the REST door used to carry as a hand-written array, which was a standing + * CLAIM that two declarations agreed. Taking the declarations themselves makes + * the claim structural; this block is what keeps it that way. + */ +const SHARED_WITH_ANALYTICS_QUERY = [ + 'dimensions', + 'measures', + 'timeDimensions', + 'order', + 'limit', + 'offset', + 'timezone', +] as const; + +const DATASET_ONLY = ['runtimeFilter', 'dateGranularity', 'compareTo', 'totals'] as const; + +describe('#17551 §1 — a transcription of published text, not a new contract', () => { + it('the seven shared members ARE `AnalyticsQuery`’s declarations, by identity', () => { + const selection = DatasetSelectionSchema.shape as Record; + const query = AnalyticsQuerySchema.shape as Record; + for (const member of SHARED_WITH_ANALYTICS_QUERY) { + expect(query[member], `${member} must be declared on AnalyticsQuery`).toBeDefined(); + // ⭐ Identity, not equality: a retyped copy would pass a structural + // comparison and drift the day either side moved. + expect(selection[member], `${member} must BE the AnalyticsQuery declaration`) + .toBe(query[member]); + } + }); + + it('the four dataset-only members are declared here and on no sibling', () => { + const selection = DatasetSelectionSchema.shape as Record; + const query = Object.keys(AnalyticsQuerySchema.shape as Record); + for (const member of DATASET_ONLY) { + expect(selection[member], `${member} must be declared on the selection`).toBeDefined(); + expect(query, `${member} must NOT be an AnalyticsQuery member`).not.toContain(member); + } + }); + + it('eleven members, no more — the interface published exactly these', () => { + expect(Object.keys(DatasetSelectionSchema.shape as Record).sort()).toEqual( + [...SHARED_WITH_ANALYTICS_QUERY, ...DATASET_ONLY].sort(), + ); + }); + + it('`runtimeFilter` is NOT taken off `where` — same shape, different key', () => { + const selection = DatasetSelectionSchema.shape as Record; + const query = AnalyticsQuerySchema.shape as Record; + expect(selection.runtimeFilter).not.toBe(query.where); + // …and the alias table is what tells an author so, rather than a 400 they + // have to guess at. + expect(refusalText({ measures: ['revenue'], where: { region: 'NA' } })) + .toContain('`where` → `runtimeFilter`'); + }); +}); + +// ───────────────────────────────────────────────────────────────────────────── +// §2 — ⭐ #17550's case: an unrecognised `compareTo.kind` +// ───────────────────────────────────────────────────────────────────────────── + +describe('#17550 §2 — `compareTo: { kind: … }` outside the closed pair is refused, with a remedy', () => { + it('the card’s own specimen is refused', () => { + const text = refusalText({ measures: ['revenue'], compareTo: { kind: 'nonsense' } }); + // ① what arrived — so the caller can find it in the body they sent. + expect(text).toContain('"nonsense"'); + // ② the whole legal set, both members, spelled as an author would write them. + expect(text).toContain("'previousPeriod'"); + expect(text).toContain("'previousYear'"); + // ③ the fix, and that it is THIS key's value that is wrong. + expect(text).toContain('compareTo.kind'); + expect(text).toContain('drop compareTo'); + // ④ ⭐ the prescription is the North Star clause this card cites: the + // refusal is loud AND it says what used to happen instead. + expect(text).toContain('200'); + }); + + it('every spelling an unparsed body can carry is refused, not just a plausible one', () => { + for (const kind of ['previousQuarter', 'previous_period', 7, null]) { + const text = refusalText({ measures: ['revenue'], compareTo: { kind } }); + expect(text, `kind=${JSON.stringify(kind)}`).toContain('compareTo.kind'); + } + }); + + it('CONTROL — both declared kinds still pass, with and without `dimension`', () => { + for (const kind of ['previousPeriod', 'previousYear'] as const) { + expect(DatasetSelectionSchema.safeParse({ measures: ['revenue'], compareTo: { kind } }).success) + .toBe(true); + expect(DatasetSelectionSchema.safeParse({ + measures: ['revenue'], + compareTo: { kind, dimension: 'close_date' }, + }).success).toBe(true); + } + }); + + it('the retired `{ offset }` arm and the bare-string form each carry their rewrite', () => { + const offset = refusalText({ measures: ['revenue'], compareTo: { kind: 'previousPeriod', offset: '7d' } }); + expect(offset).toContain('`offset`'); + expect(offset).toContain("kind: 'previousYear'"); + + const bare = refusalText({ measures: ['revenue'], compareTo: 'previousPeriod' }); + expect(bare).toContain("compareTo: { kind: 'previousPeriod' }"); + }); + + it('ONE condition, ONE wording — the two origins differ only in where it was refused', () => { + const schema = datasetCompareKindRefusalMessage('previousQuarter', 'schema'); + const runtime = datasetCompareKindRefusalMessage('previousQuarter', 'runtime'); + // The verdict sentence is byte-identical on both sides of the door… + const verdict = 'compareTo.kind "previousQuarter" is not a comparison window this platform implements.'; + expect(schema.startsWith(verdict)).toBe(true); + expect(runtime.startsWith(verdict)).toBe(true); + // …and only the clause that says WHERE differs — the one clause no input + // can supply, which is why it is a required parameter with no default. + expect(schema).toContain('Refused at the schema'); + expect(runtime).toContain('Refused past the schema door'); + expect(schema).not.toEqual(runtime); + // The schema really raises it — not a sentence only the builder knows. + expect(refusalText({ measures: ['revenue'], compareTo: { kind: 'previousQuarter' } })) + .toContain(verdict); + }); +}); + +// ───────────────────────────────────────────────────────────────────────────── +// §3 — the other three undoored members, both directions each +// ───────────────────────────────────────────────────────────────────────────── + +describe('#17551 §3 — `runtimeFilter` / `dateGranularity` / `totals`', () => { + it('`dateGranularity` outside the closed vocabulary is refused; every member of it passes', () => { + expect(refusalText({ measures: ['revenue'], dateGranularity: 'fortnight' })) + .toMatch(/fortnight|Invalid option/); + for (const g of ['day', 'week', 'month', 'quarter', 'year']) { + expect( + DatasetSelectionSchema.safeParse({ measures: ['revenue'], dateGranularity: g }).success, + `dateGranularity: ${g} must still pass`, + ).toBe(true); + } + }); + + it('`runtimeFilter` must be a FilterCondition; a real one passes', () => { + expect(DatasetSelectionSchema.safeParse({ measures: ['revenue'], runtimeFilter: 'region = NA' }).success) + .toBe(false); + for (const f of [{ region: 'NA' }, { region: { $ne: 'EU' } }, { $and: [{ region: 'NA' }] }]) { + expect( + DatasetSelectionSchema.safeParse({ measures: ['revenue'], runtimeFilter: f }).success, + `runtimeFilter ${JSON.stringify(f)} must still pass`, + ).toBe(true); + } + }); + + it('`totals` is `{ groupings: string[][] }`; the grand total and a matrix both pass', () => { + // The response spells the same idea `dimensions` — the cross-direction + // near-miss the guidance entry exists for. + expect(refusalText({ measures: ['revenue'], totals: { dimensions: ['region'] } })) + .toContain('totals: { groupings:'); + // A flat list where a list OF LISTS is declared. + expect(DatasetSelectionSchema.safeParse({ measures: ['revenue'], totals: { groupings: ['region'] } }).success) + .toBe(false); + for (const t of [{ groupings: [[]] }, { groupings: [['region'], ['stage'], []] }]) { + expect( + DatasetSelectionSchema.safeParse({ measures: ['revenue'], totals: t }).success, + `totals ${JSON.stringify(t)} must still pass`, + ).toBe(true); + } + }); + + it('the two nested directives are closed too — an unknown key is named, not dropped', () => { + expect(DatasetCompareToSchema.safeParse({ kind: 'previousPeriod', dimensoin: 'x' }).success).toBe(false); + expect(DatasetTotalsSchema.safeParse({ groupings: [[]], grandTotal: true }).success).toBe(false); + }); +}); + +// ───────────────────────────────────────────────────────────────────────────── +// §4 — an unknown key is named, echoed and pointed somewhere +// ───────────────────────────────────────────────────────────────────────────── + +describe('#17551 §4 — the selection is `.strict()`, like every other analytics door', () => { + it('names the surface, echoes the key, and carries the history', () => { + const text = refusalText({ measures: ['revenue'], totaIs: { groupings: [[]] } }); + expect(text).toContain('Unrecognized key(s) on this dataset selection'); + expect(text).toContain('`totaIs`'); + expect(text).toContain('enforced by nothing on the wire'); + }); + + it('a member of the REQUEST BODY written one level too deep gets its wrong-layer pointer', () => { + const text = refusalText({ measures: ['revenue'], datasetName: 'sales' }); + expect(text).toContain('one level up'); + expect(text).toContain('`{ dataset | datasetName, selection }`'); + }); + + it('`cube` — the sibling body’s required member — is answered, not merely rejected', () => { + const text = refusalText({ measures: ['revenue'], cube: 'opportunity' }); + expect(text).toContain('a dataset selection names no cube'); + }); +}); + +// ───────────────────────────────────────────────────────────────────────────── +// §5 — ⭐ the negative side: the whole shape still passes, unchanged +// ───────────────────────────────────────────────────────────────────────────── + +describe('#17551 §5 — a valid selection still passes, and the parse adds nothing', () => { + it('the fully-loaded eleven-member selection parses', () => { + const parsed = DatasetSelectionSchema.safeParse(FULLY_LOADED); + expect(parsed.success, parsed.success ? '' : JSON.stringify(parsed.error.issues)).toBe(true); + }); + + it('⭐ the parse output equals the input — no default, no transform', () => { + // The route forwards the CALLER's object, never the parse output, and that + // is only safe while this holds. A `.default()` added to any member later + // turns this red instead of silently overriding the engine's own + // resolution chain (the `timezone` rule #1982/#2018 records). + expect(DatasetSelectionSchema.parse(FULLY_LOADED)).toEqual(FULLY_LOADED); + }); + + it('the minimal selection — `measures` alone — passes', () => { + expect(DatasetSelectionSchema.safeParse({ measures: ['revenue'] }).success).toBe(true); + }); + + it('every in-repo selection specimen still passes', () => { + // The same leniency sweep #17058 ran at the door, re-run against the WHOLE + // schema: if any in-repo caller had been relying on the four undoored + // members going unparsed, this is where it shows. + const specimens: Array> = [ + { measures: ['account_count'], dimensions: ['bogus_dim'] }, + { measures: ['account_count'], runtimeFilter: { bogus_col: 'x' } }, + { dimensions: ['stage'], measures: ['revenue'], order: { profit: 'desc' } }, + { dimensions: ['stage'], measures: ['revenue'], totals: { groupings: [['region']] } }, + { + dimensions: ['stage'], + measures: ['revenue'], + timeDimensions: [{ dimension: 'close_date', granularity: 'month' }], + compareTo: { kind: 'previousPeriod' }, + }, + { measures: ['amount_sum'] }, + { measures: ['cnt'], timeDimensions: [{ dimension: 'issued', granularity: 'month' }] }, + // The five objectui call sites, as they build their selection today + // (DatasetWidget, DatasetReportRenderer, DashboardFilterBar, + // ObjectChart, DatasetPreview — measured at objectui @98178b2). + { dimensions: ['region'], measures: ['revenue'], runtimeFilter: { region: 'NA' }, dateGranularity: 'month', order: { revenue: 'desc' }, limit: 20 }, + { dimensions: ['region'], measures: ['revenue'], totals: { groupings: [['region'], []] }, order: { revenue: 'asc' } }, + { dimensions: ['industry'], measures: ['option_count'], runtimeFilter: { is_active: true }, order: { industry: 'asc' }, limit: 1000 }, + { dimensions: [], measures: [] }, + ]; + for (const selection of specimens) { + const parsed = DatasetSelectionSchema.safeParse(selection); + expect( + parsed.success, + `specimen must still pass: ${JSON.stringify(selection)} — ${parsed.success ? '' : JSON.stringify(parsed.error.issues)}`, + ).toBe(true); + } + }); +}); diff --git a/packages/spec/src/contracts/analytics-service.ts b/packages/spec/src/contracts/analytics-service.ts index ab1b7ed9a7f..ee4a87e0865 100644 --- a/packages/spec/src/contracts/analytics-service.ts +++ b/packages/spec/src/contracts/analytics-service.ts @@ -1,6 +1,7 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. import type { AnalyticsQuery, Cube } from '../data/analytics.zod.js'; +import type { DatasetSelection } from '../api/analytics.zod.js'; import type { FilterCondition } from '../data/filter.zod.js'; import type { AggregationFunction } from '../data/query.zod.js'; import type { PercentScale } from '../data/percent-scale.js'; @@ -144,93 +145,33 @@ export interface CubeMeta { /** * Compare-to directive (ADR-0021): runs a time-shifted second query and * attaches `__compare` columns to each row. + * + * [#17551] Re-exported from the zod source (`DatasetCompareToSchema`, + * api/analytics.zod.ts) instead of a hand-written interface — the same move + * `AnalyticsQuery` above made for the same reason, taken here BEFORE a mirror + * could drift rather than after. The members are unchanged: `kind` is the + * closed pair the executor implements and `dimension` is optional and + * resolved by that executor. */ -export interface DatasetCompareTo { - /** previousPeriod = equal-length window immediately before; previousYear = same window −1y. */ - kind: 'previousPeriod' | 'previousYear'; - /** - * The time dimension (by name) whose `dateRange` is shifted. - * - * **Optional since #5011, resolved by the EXECUTOR — not by any consumer.** - * When omitted the executor takes the selection's shiftable time dimensions - * (its own long-standing criterion: a `timeDimensions` entry carrying a - * `dateRange`) and: - * - * - exactly one candidate → that one is shifted; - * - zero candidates → throws, saying a comparison needs a dated window; - * - two or more → throws, listing the candidates by name so the author can - * pick one. - * - * The ambiguous and empty cases are LOUD by design. A consumer must never - * paper over them by guessing a dimension (PD #12): the resolution rule - * lives at the producer of the comparison — the executor — precisely so - * every caller gets the same answer or the same error. - */ - dimension?: string; -} +export type { DatasetCompareTo } from '../api/analytics.zod.js'; /** * A presentation's selection against a dataset (ADR-0021). Report/dashboard * widgets bind to a dataset and pick dimensions/measures BY NAME; this is the * wire shape a preview/query endpoint posts. + * + * [#17551, ruled] Re-exported from the zod source (`DatasetSelectionSchema`, + * api/analytics.zod.ts). It was a hand-written interface here, which is why + * four of its eleven members — `runtimeFilter`, `dateGranularity`, + * `compareTo`, `totals` — were published and enforced by nothing on the + * wire: `POST /analytics/dataset/query` could only door the seven whose + * declarations coincided with `AnalyticsQuery`'s, and the rest travelled into + * the executor unrefused (#17550 is the measured consequence). The schema is a + * transcription of the text this interface already published; the members, their + * types and their documentation live there now, in ONE place, and this name is + * that declaration re-exported — ⛔ never a second declaration beside it. */ -export interface DatasetSelection { - /** Dimension names from the dataset. */ - dimensions?: string[]; - /** Measure names from the dataset (may include derived measures). */ - measures: string[]; - /** Presentation-scope filter, ANDed with the dataset's intrinsic filter at render. */ - runtimeFilter?: FilterCondition; - /** Optional time-dimension windows passed through to the runtime. */ - timeDimensions?: AnalyticsQuery['timeDimensions']; - /** - * Presentation-scope date bucketing (framework#3588). Applies to every - * selected dimension the dataset declares as a `date` dimension, so a - * widget can bucket a trend by month without the dataset having to declare - * that granularity for every consumer. - * - * Precedence, per dimension: an explicit `timeDimensions` entry for that - * dimension wins, then this selection-level granularity, then the dataset - * dimension's own `dateGranularity` default. Unset leaves each dimension on - * its dataset default (which may be no bucketing at all — grouping by the - * raw column). - */ - dateGranularity?: 'day' | 'week' | 'month' | 'quarter' | 'year'; - /** - * Result ordering, applied by key in insertion order (`{ revenue: 'desc' }`). - * - * Every key must be a selected dimension, a selected measure, or a - * `__compare` column; anything else is rejected rather than - * silently ignored. Ordering is applied AFTER measure-scoped filters are - * merged, `compareTo` columns are attached, and derived measures are - * evaluated — so a derived measure (e.g. a win-rate ratio) is a valid sort - * key even though no single SQL statement computes it. - */ - order?: Record; - /** - * Max rows to return, applied after `order`. When `limit` is set without - * `order`, rows are ordered by the selected dimensions ascending first, so - * the truncated window is deterministic rather than an arbitrary subset. - */ - limit?: number; - offset?: number; - /** Compare-to directive — runs a shifted query and attaches `__compare`. */ - compareTo?: DatasetCompareTo; - /** - * Server-side totals (matrix subtotals + grand total). Each grouping is a - * subset of `dimensions` to additionally aggregate by; the selection is - * re-run grouped only by those dimensions, so every total is the measure's - * TRUE aggregate over the underlying rows — an `avg` total is the average - * over all rows, not an average of bucket averages (the ADR-0021 - * governance line that forbids client-side re-aggregation). `[]` requests - * the grand total. A matrix report asks for - * `{ groupings: [rowDims, columnDims, []] }`. Results arrive on - * `AnalyticsResult.totals` in request order. `order`/`limit`/`offset` do - * not apply to totals queries — totals always cover the full selection. - */ - totals?: { groupings: string[][] }; - timezone?: string; -} +export type { DatasetSelection } from '../api/analytics.zod.js'; export interface IAnalyticsService { /** diff --git a/packages/spec/src/data/analytics.zod.ts b/packages/spec/src/data/analytics.zod.ts index b92bd69179a..4bab38c7e78 100644 --- a/packages/spec/src/data/analytics.zod.ts +++ b/packages/spec/src/data/analytics.zod.ts @@ -498,11 +498,12 @@ function describeRefusedDateRange(input: unknown): string { * (`@objectstack/core`) — `AnalyticsService.query`, `queryDataset` and the * dataset executor behind it reached IN PROCESS, and a driver's cube face * called directly. ⚠️ Every REST analytics route is a SCHEMA-origin door, - * `POST /analytics/dataset/query` included: since PR #17548, the PR that - * landed that door for card #17058, the route parses its selection's shared - * members — `timeDimensions` among them — against - * `AnalyticsQuerySchema.pick(…)` ahead of the executor, so that route's - * refusal is THIS schema's and says so. Until this parameter existed the + * `POST /analytics/dataset/query` included: PR #17548 landed a + * door there that parsed the selection's shared members — `timeDimensions` + * among them — and #17551 widened it to the WHOLE selection, against + * `DatasetSelectionSchema` (`api/analytics.zod.ts`, which takes those members + * off this schema's own shape). Either way the parse is ahead of the executor, + * so that route's refusal is THIS schema's and says so. Until this parameter existed the * shared sentence asserted the SCHEMA origin for both, so an author refused past * the door was sent to inspect a parse call that never ran; the one package that * noticed (`service-analytics`, #17593) had to OVERWRITE the message instead of diff --git a/packages/spec/src/type-alias-convention.pin.test.ts b/packages/spec/src/type-alias-convention.pin.test.ts index 276d3fc3460..41c0adefeb1 100644 --- a/packages/spec/src/type-alias-convention.pin.test.ts +++ b/packages/spec/src/type-alias-convention.pin.test.ts @@ -275,7 +275,7 @@ import type * as M187 from './shared/duration.zod.js'; import type * as M188 from './ai/build-progress.zod.js'; // --------------------------------------------------------------------------- -// 784 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. +// 787 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. // // That number is machine-checked, not hand-kept. The runtime companion at the // bottom of this file recomputes the pin count from the source and asserts that @@ -345,6 +345,14 @@ export type Iso36 = Assert, z.infe // api/analytics.zod.ts export type Iso37 = Assert, z.infer< typeof M11.AnalyticsEndpoint > >>; export type Iso38 = Assert, z.infer< typeof M11.AnalyticsQueryRequestSchema > >>; +// [#17551] The ADR-0021 dataset selection and its two nested directives. Their +// seven shared members ARE `AnalyticsQuerySchema`'s own declarations (Iso300 +// above pins that schema isomorphic), and the four dataset-only members carry +// no default, transform, catch or pipe — so the author state and the parsed +// state coincide and no `XParsed` name would be anything but a synonym. +export type Iso877 = Assert, z.infer< typeof M11.DatasetCompareToSchema > >>; +export type Iso878 = Assert, z.infer< typeof M11.DatasetTotalsSchema > >>; +export type Iso879 = Assert, z.infer< typeof M11.DatasetSelectionSchema > >>; // api/auth-endpoints.zod.ts export type Iso39 = Assert, z.infer< typeof M12.AuthEndpointSchema > >>; @@ -1683,7 +1691,7 @@ describe('ADR-0122 type-alias convention', () => { // this title and the section header above the pin list — are now asserted // against the recomputed count below, so neither can go stale without a red // test naming it. - it('still declares all 784 isomorphic pins', () => { + it('still declares all 787 isomorphic pins', () => { // The truth of each pin is proved by tsc, not here — an `Assert>` // that stops holding is a compile error with the alias named. What tsc // cannot notice is a pin that was DELETED: removing the assertion removes @@ -2268,7 +2276,14 @@ describe('ADR-0122 type-alias convention', () => { // carried both halves of the pair — which is also why only ONE of the three // was ever on this list. -1 converted to an `XParsed` pair; the Iso number // stays vacant (ids are claims about pins, not positions). - expect(pins).toHaveLength(784); + // 784 -> 787 is #17551's ADR-0021 dataset selection (api/analytics.zod.ts, + // module slot M11): `DatasetSelectionSchema` and the two nested directives + // `DatasetCompareToSchema` / `DatasetTotalsSchema`, the (RISE) case three + // times. The selection takes its seven shared members straight off + // `AnalyticsQuerySchema.shape` — already pinned isomorphic as Iso300 — and + // the four it adds carry no default, transform, catch or pipe, so no + // `XParsed` is declared and all three come here instead. +3 added. + expect(pins).toHaveLength(787); // The count is stated in PROSE twice as well — this case's title and the // section header above the pin list — and until #6605 nothing read either