diff --git a/.claude/rules/cross-language-porting.md b/.claude/rules/cross-language-porting.md index ae51970c5..ccd8b7eef 100644 --- a/.claude/rules/cross-language-porting.md +++ b/.claude/rules/cross-language-porting.md @@ -17,7 +17,7 @@ Preserve the following contracts exactly across all language ports: **Metamodel subtype vocabularies (must be identical across languages):** the `registry-conformance` gate (`fixtures/registry-conformance/`) is the structural enforcer of this rule — each port emits its registry as a canonical manifest byte-matched to `expected-registry.json`. **All five ports (TS / C# / Java / Kotlin / Python) are live + green** (SP-G Java/Kotlin reconciliation complete; the JVM runners compose from the defined metamodel provider set so codegen-base/om classpath SPI does not pollute the measured vocabulary). See `fixtures/registry-conformance/README.md`. - Filter operators: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `like`, `isNull` -- Object subtypes: `entity` (owns data: own identity, writable sources, lifecycle), `value` (pure shape: NO identity, NO source, ever; constructed — by caller/embedding — never populated; may `extends` entity fields for shape; a value-hosted field may carry `origin.passthrough` but never an assembly origin), `projection` (derived read-only representation: fields `extends`-bound / origin-derived / self-declared-under-external-assembly, all read-only at subtype level; identity optional and MUST extend an entity identity; sources restricted to read-only `@kind`s; the declared field set IS the exposure — inclusive list, fail-closed). A field carrying `origin.*` is derived ⇒ read-only wherever it lives (incl. on entities). An entity's primary source must be a writable `@kind` (read-only kinds only in read role). See [ADR-0028](spec/decisions/ADR-0028-object-taxonomy-projection-value-purity.md). (FR-024 Phase E — `object.projection`/`value` are registered in `expected-registry.json` and the projection/value validation passes [identity pass-through, value-purity, projection-licensing, `@via` inference/cardinality, extends/origin agreement, derived-field providability] are enforced cross-port in all 5 ports. The **B4b** entity-primary-source-readonly cutover [the "writable `@kind`" clause above — `ERR_ENTITY_PRIMARY_SOURCE_READONLY`] + the projection codegen fan-out (read-only DTOs for view-kind projections; FR-015 proc-callables for proc-kind projections in TypeScript, C# and Kotlin ONLY — Java and Python ship no callable generator at all, so the cross-port claim does NOT cover that clause; api-docs label `object.projection` units as `projection` and document their generated `Dto`) are now shipped cross-port; the remaining FR-024 work is the declared-API surface — tracked in #10.) **`report` (FR-044 Plan 1)** is a root object subtype registered in all five ports, with the `dimension.attribute` / `dimension.time` / `measure.aggregate` / `measure.ratio` / `segment.filter` children on `object.entity` and the relative-date filter value — loader-validated (`ERR_INVALID_DIMENSION` / `ERR_INVALID_MEASURE` / `ERR_INVALID_REPORT` / `ERR_REPORT_FOREIGN_MEASURE`, plus `ERR_BAD_ATTR_FILTER` for relative dates off a reporting host) and **inert in every generator and in `meta migrate`**: an `object.report` emits nothing until its lowering plans land, gated by the 27 `reporting` conformance fixtures and the `codegen-noop` corpus. See [docs/features/reporting.md](docs/features/reporting.md). +- Object subtypes: `entity` (owns data: own identity, writable sources, lifecycle), `value` (pure shape: NO identity, NO source, ever; constructed — by caller/embedding — never populated; may `extends` entity fields for shape; a value-hosted field may carry `origin.passthrough` but never an assembly origin), `projection` (derived read-only representation: fields `extends`-bound / origin-derived / self-declared-under-external-assembly, all read-only at subtype level; identity optional and MUST extend an entity identity; sources restricted to read-only `@kind`s; the declared field set IS the exposure — inclusive list, fail-closed). A field carrying `origin.*` is derived ⇒ read-only wherever it lives (incl. on entities). An entity's primary source must be a writable `@kind` (read-only kinds only in read role). See [ADR-0028](spec/decisions/ADR-0028-object-taxonomy-projection-value-purity.md). (FR-024 Phase E — `object.projection`/`value` are registered in `expected-registry.json` and the projection/value validation passes [identity pass-through, value-purity, projection-licensing, `@via` inference/cardinality, extends/origin agreement, derived-field providability] are enforced cross-port in all 5 ports. The **B4b** entity-primary-source-readonly cutover [the "writable `@kind`" clause above — `ERR_ENTITY_PRIMARY_SOURCE_READONLY`] + the projection codegen fan-out (read-only DTOs for view-kind projections; FR-015 proc-callables for proc-kind projections in TypeScript, C# and Kotlin ONLY — Java and Python ship no callable generator at all, so the cross-port claim does NOT cover that clause; api-docs label `object.projection` units as `projection` and document their generated `Dto`) are now shipped cross-port; the remaining FR-024 work is the declared-API surface — tracked in #10.) **`report` (FR-044 Plan 1)** is a root object subtype registered in all five ports, with the `dimension.attribute` / `dimension.time` / `measure.aggregate` / `measure.ratio` / `segment.filter` children on `object.entity` and the relative-date filter value — loader-validated (`ERR_INVALID_DIMENSION` / `ERR_INVALID_MEASURE` / `ERR_INVALID_REPORT` / `ERR_REPORT_FOREIGN_MEASURE`, plus `ERR_BAD_ATTR_FILTER` for relative dates off a reporting host) and, since FR-044 Plan 2, **lowered only when it declares a read-only `source.rdb` of `@kind: view`**: `meta migrate` creates that view (TypeScript only, ADR-0015), every port reads it (persistence corpus, `report-shapes.json`), C# generates its typed row and Kotlin its Exposed table object, and everything else (routes, typed clients, filter allowlists, api-docs, and a report with no view source at all) stays inert, gated by the 27 `reporting` conformance fixtures and the `codegen-noop` corpus. See [docs/features/reporting.md](docs/features/reporting.md). - Source subtypes: `rdb` (paradigm; ADR-0007). The pre-v2 `dbTable`/`dbView` subtypes are RETIRED — `source.rdb` + `@kind: table|view|materializedView|storedProc|tableFunction` is the form, with read-only-ness derived from `@kind`. Multi-source via `@role` (exactly one `primary` per object). Source physical name = `@table` (NOT `@name`); field physical name = `@column` (renamed from `@dbColumn`). Referential actions on relationships: `@onDelete` / `@onUpdate`. - Origin subtypes: `passthrough`, `aggregate`, `collection`, `computed`, `first` (concrete; `base` is the abstract root). `passthrough` is legal on an `object.value`-hosted field (FR-015 parameter lineage); the four assembly origins (`aggregate`/`computed`/`collection`/`first`) live on `object.projection` only — a value-hosted assembly origin is `ERR_SUBTYPE_RULE_VIOLATION` (#210). - Relationship subtypes: `association`, `aggregation`, `composition`. Cardinality via `@cardinality: one|many`; target via `@objectRef`. **M:N (FR-018) slim vocabulary:** `@cardinality: "many"` + `@objectRef` (target) + `@through` (the junction/through entity — a third entity that MUST declare two `identity.reference` children, one per FK side). The relationship's FK fields are **derived** from those references (the `identity.reference` SSOT for FK direction), never restated. `@sourceRefField` (optional) disambiguates a *directed* self-join by naming the source-side FK field on the junction (the other reference is the target side); on a `@cardinality: one` relationship it instead names which of several `identity.reference` nodes onto the same target this relationship navigates, short-circuiting the unique-candidate/`@sourceRefField`/name-pairing ladder (#368, [ADR-0029](spec/decisions/ADR-0029-entity-child-extends-and-via-inference.md) Amendment 1) — an unresolvable 1:N reference set is `ERR_INVALID_RELATIONSHIP` at load. `@symmetric` (optional boolean) marks an *undirected* self-join (union-on-read) — valid only when `@objectRef` == the declaring entity, and mutually exclusive with `@sourceRefField`. The pre-FR-018 `@joinEntity`/`@joinFields` attrs are REMOVED. Validation errors: symmetric-on-hetero / symmetric+sourceRefField → `ERR_BAD_ATTR_VALUE`; junction-missing-two-references / sourceRefField-not-matching / M:N-attr-on-1:N / **junction-unpairable** → `ERR_INVALID_RELATIONSHIP`. **Unpairable means the junction declares its two references but neither resolves to the navigating entity, or neither to the `@objectRef` target** — declaring two references is NOT enough, and the loader now checks WHAT they point at (owner ruling 2026-09-20). It does so by running the real FK derivation and converting its failure, never a parallel re-implementation, so the loader and the derivation cannot drift; scope mirrors codegen's own iteration exactly — every CONCRETE, non-projection object crossed with its EFFECTIVE relationships, NOT deduped by declaration, because pairing is a property of the navigating entity and an inherited M:N can pair from one subtype and not another. Before the ruling this loaded clean and then diverged: TS and C# warned and emitted no traversal route (a silent 404), while Java, Kotlin and Python failed the build. Gated by `fixtures/conformance/error-relationship-m2m-junction-unpairable/`. diff --git a/AGENTS.md b/AGENTS.md index 88d233c83..4ead16eb7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -82,7 +82,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - YAML / verify corpora green across the ports that ship those layers. - **Codegen-compile gate** (all five ports; a GATE, not a corpus — it has no fixtures of its own and no row in the matrix). Every corpus above gates BEHAVIOUR; none asks whether the emitted code BUILDS, which is how four "generated code does not compile" defects shipped in 1.0.4 with the whole matrix green — `gen` exits 0 in all four cases and the adopter's build is the first thing that disagrees. Each port generates from `fixtures/persistence-conformance/canonical/meta.fitness.json` (reused deliberately: a second kitchen sink would drift from the one the other corpora already maintain) and compiles the emitted tree with its real compiler — `ts.createProgram` / Roslyn / `javac` / `KotlinCompilation` / (Python, having no static compiler) importing the generated package plus `ruff` F821. **Every port excludes its framework-bound route tier** (TS `routesFile`, C# `RoutesGenerator`, Java `SpringControllerGenerator`, Kotlin `KotlinSpringControllerGenerator`): those imports are not on an in-memory compile's classpath and stubbing them drowns the signal, so that tier is proven by the api-contract integration lane instead. One cross-port rule, not four local concessions. Found 5 further real defects on first run. Boundary detail: `docs/CONFORMANCE.md` → "Split coverage". -**Key cross-language features shipped:** FR5 family (a/b/c/d/e + WARN envelope-shape — actionable loader errors per ADR-0009); FR-003 (Java RDB runtime persistence + projections; schema migrations are TS-only — the Java migration engine was removed); FR-006 (template.output parser-on-receipt codegen per ADR-0010 in all 5 ports); FR-008 + FR-009 (cross-port REST API contract + the nine filter operators); FR-018 (M:N relationship codegen in all 5 ports — entity navigation + idiomatic ORM wiring [Drizzle m2m / EF Core `UsingEntity` / Spring repo+JPA / Exposed / Pydantic+route as the SQLAlchemy-secondary equivalent] + REST traversal `GET //{id}/` + Tier-2 docs, gated by the shared api-contract m2m corpus in both lanes + persistence-conformance; the TanStack M:N client hook is a deferred client-ergonomics follow-up); SP-H (field-subtype end-to-end hardening: every concrete `field.*` subtype write+read round-trips cross-port via the persistence `op: roundtrip` gate; cut `field.byte`/`field.short`/`field.class` non-functional stubs; cross-port filter-op reconciliation for uuid/currency); source v2 paradigm (ADR-0007); metadata-ktx Kotlin facade; per-target output directories (TS codegen); FR-044 Plan 1 (the reporting vocabulary — `dimension.attribute`/`dimension.time`, `measure.aggregate`/`measure.ratio`, `segment.filter`, `object.report`, plus the relative-date filter value — registered and loader-validated in all five ports behind 27 conformance fixtures + the `codegen-noop` corpus; **reports generate nothing yet**, see [docs/features/reporting.md](docs/features/reporting.md)). +**Key cross-language features shipped:** FR5 family (a/b/c/d/e + WARN envelope-shape — actionable loader errors per ADR-0009); FR-003 (Java RDB runtime persistence + projections; schema migrations are TS-only — the Java migration engine was removed); FR-006 (template.output parser-on-receipt codegen per ADR-0010 in all 5 ports); FR-008 + FR-009 (cross-port REST API contract + the nine filter operators); FR-018 (M:N relationship codegen in all 5 ports — entity navigation + idiomatic ORM wiring [Drizzle m2m / EF Core `UsingEntity` / Spring repo+JPA / Exposed / Pydantic+route as the SQLAlchemy-secondary equivalent] + REST traversal `GET //{id}/` + Tier-2 docs, gated by the shared api-contract m2m corpus in both lanes + persistence-conformance; the TanStack M:N client hook is a deferred client-ergonomics follow-up); SP-H (field-subtype end-to-end hardening: every concrete `field.*` subtype write+read round-trips cross-port via the persistence `op: roundtrip` gate; cut `field.byte`/`field.short`/`field.class` non-functional stubs; cross-port filter-op reconciliation for uuid/currency); source v2 paradigm (ADR-0007); metadata-ktx Kotlin facade; per-target output directories (TS codegen); FR-044 Plan 1 (the reporting vocabulary — `dimension.attribute`/`dimension.time`, `measure.aggregate`/`measure.ratio`, `segment.filter`, `object.report`, plus the relative-date filter value — registered and loader-validated in all five ports behind 27 conformance fixtures + the `codegen-noop` corpus; a report that declares a read-only `source.rdb @kind: view` is lowered to a SQL view by `meta migrate` and read by every port (Plan 2), and no route or typed client is generated for a report yet, see [docs/features/reporting.md](docs/features/reporting.md)). **Latest release: 1.0.13** (2026-10-03) — npm `1.0.13`, PyPI `1.0.13`, NuGet `1.0.13`, Maven Central `8.0.13`. A PATCH: an already-plural entity name (`Stats`, `Settings`) no longer doubles in API-surface names (REST paths, hooks, finders, DbSets) in any port, while default physical table names stay frozen on the old rule; two entities that would share one API name are now a generation error; two Kotlin controller compile fixes (`field.inet` filter ops, `@dbColumnType: uuid` on a string field). Gated by a private `1.0.13-rc.1` build on the adopter estate (`rc-gate.sh` 7/7) and a full `--strict-toolchains` local CI run. The previous release, 1.0.12 (2026-10-02), added `fmt` in every CLI, a deprecated-reference `verify` advisory, the `onLocate` extract hook, and an Exposed 1.x Kotlin output mode. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ea4555b8..76af5ef35 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,13 +27,53 @@ it until 1.1 ships._ `measure.aggregate` or an `object.report`. Four new error codes (`ERR_INVALID_DIMENSION`, `ERR_INVALID_MEASURE`, `ERR_INVALID_REPORT`, `ERR_REPORT_FOREIGN_MEASURE`) and extended `ERR_BAD_ATTR_FILTER` carry the load-time rules, gated by 27 new shared conformance fixtures. - `measure.derived` is not registered (it waits for FR-037 R5). **No generated output yet:** an - `object.report` emits no view DDL, route, client code or docs page, `meta migrate` proposes - nothing for it, and a model using the new names generates exactly what it did without them. - See [docs/features/reporting.md](docs/features/reporting.md). + `measure.derived` is not registered (it waits for FR-037 R5). A model that does not use the + new names generates exactly what it did without them; the next entry says what a report + becomes. See [docs/features/reporting.md](docs/features/reporting.md). +- **A report with a view source becomes a SQL view, and every port reads it (FR-044).** An + `object.report` that declares a read-only `source.rdb` of `@kind: view` is now lowered by + TypeScript: `meta migrate` creates the view on Postgres, SQLite and D1 (a changed report view + is dropped and re-created; a derived report view whose `@from` entity has no table fails + migrate naming the report and the entity; a report with an `@sql` source skips that check). A + derived report view is also refused, by name, when its `@from` is a TPH subtype (the subtype + shares its base's table, so the view would count every subtype's rows: declare it from the base + with an `@filter` on the discriminator field), when a `@via` hop has no `identity.reference` + behind it, and when a filter's `in` list is empty. A bare `@of` or `@via` on a dimension or + measure inherited from a base in another package resolves in that base's package, as the loader + resolves it. MySQL SQL comes from `buildReportViews(root, + { dialect: "mysql" })` and the "Reports" section of `docs/recipes/mysql.md`, since `meta + migrate` does not target MySQL. A report with no `source.*` still generates nothing. Time + grains and relative dates are UTC, weeks start on Monday, a `sum` of nothing and a ratio over + zero are null, and a count is zero. The TypeScript `ObjectManager`, Java OMDB and the Python + `ObjectManager` read a view-backed report (list and count, with filter, sort and limit on the + derived fields; by-id and writes are refused); C# generates a keyless EF Core row type and + `DbContext` mapping for it and Kotlin an Exposed table object. `meta docs` lists the view on + the agent schema page. **Still absent:** no route, typed client, filter allowlist or api-docs + entry for a report in any port, no `measure.derived`, and no query-time grouping. Six shared + persistence scenarios (`report-*.yaml`) and `report-shapes.json` hold the ports to the same + columns; the `metaobjects-authoring` skill now teaches reports (`references/reporting.md`). + Anyone who declared a view-sourced report under the unreleased 1.1 vocabulary will now see a + `CREATE VIEW` from `meta migrate`. Kotlin `gen` fails, naming the report and the dimension or + measure, for a view-backed report with a derived field named after a Kotlin hard keyword or + with two derived fields that land on one column property. A view-backed report with a + dimension or measure over a `field.object` is refused by name by Java OMDB (on read), Kotlin + `gen` and C# `gen`; group by a scalar field. A `@measures` item may be written dotted + (`Sale.total`) and reads the same as the bare name in every port. Java OQL + (`executeQuery`) with a report as its result class builds rows from the report's derived + fields. ### Fixed +- **Java OMDB reads a projection whose view is named by `@view`.** The read mapping took the + view name from `@table` only, so a projection declared with the kind-matching `@view` alias + had no read mapping. It now resolves the source's physical name (`@view`, then the legacy + `@table`), the same rule the TypeScript toolchain creates the view under. +- **Kotlin: a field named after an Exposed `Table` property that was not reserved now gets the + `Column` suffix.** An entity or report field named `schemaName` now emits the column property + `schemaNameColumn`; it collided with Exposed's `Table.schemaName` and did not compile before. + With `exposedApi=1` the same now holds for `options` and `storageParameters`, which are + `Table` properties only in Exposed 1.x; `exposedApi=0` output for those two names is unchanged. + The physical column names do not change. - **Java: a bare string authored for an `isArray` attribute is now ONE item.** The Java parser used to split it on commas, which left stray quotes in the items; TypeScript and Python already kept it whole. All three now agree, so a Java model that relied on the split (a diff --git a/README.md b/README.md index 37ae799df..ede493bf2 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ first-week wedge plan — and `meta init` picks up from there. | Template-drift verify | Yes | Yes (`Verify.check`) | Yes (via Java) | Yes (`dotnet meta verify`) | Yes (`metaobjects.render.verify`) | | YAML authoring (sigil-free → JSON) | Yes | Yes | Yes (via Java) | Yes | Yes | | Capability requirements (`requirement.*`) | Registered + `meta verify` gate | Registered (loads + validates) | Registered (via Java) | Registered (loads + validates) | Registered (loads + validates) | -| Reporting vocabulary (`dimension` / `measure` / `segment` / `object.report`, FR-044) | Registered (loads + validates); no generated output yet | Registered (loads + validates); no generated output yet | Registered (via Java); no generated output yet | Registered (loads + validates); no generated output yet | Registered (loads + validates); no generated output yet | +| Reporting vocabulary (`dimension` / `measure` / `segment` / `object.report`, FR-044) | Registered; a view-backed report becomes a SQL view in `meta migrate` and is read by `ObjectManager`; no routes yet | Registered; OMDB reads a view-backed report; no generated output | Registered (via Java); generates an Exposed table object per view-backed report | Registered; generates a keyless EF Core row type per view-backed report | Registered; `ObjectManager` reads a view-backed report; no generated output | | Libraries (`libraries: [...]`) | Yes | Yes | Yes (via Java) | Yes | Yes | | Metadata dependencies (`dependencies`) | Yes (`meta deps sync`, `path` transport) | Phase 2 | Phase 2 | Phase 2 | Yes (loads the synced snapshot) | | Runtime metadata (ObjectManager-style) | Yes (`runtime-ts`) | Yes (OMDB) | Yes (via Java OMDB + Exposed) | Roadmap | Yes (ObjectManager) | diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/agent-context/skills/metaobjects-authoring/references/reporting.md b/agent-context/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/agent-context/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md b/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md index 945e40148..eb54c6be2 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md @@ -59,6 +59,51 @@ the reference: its column builders name the MySQL types. A column named after a MySQL reserved word (`rank`, `order`) needs backticks in your DDL. The generated code and both runtimes quote identifiers themselves. +### Reports + +An `object.report` (the reporting vocabulary; see `references/reporting.md` in the +`metaobjects-authoring` skill) is a compiled view, and on MySQL you create that view +yourself, because `meta migrate` does not. Declare the report with a read-only +`source.rdb` of `@kind: view` and no `@unmanaged`, since `meta migrate` never targets MySQL +and so nothing manages the view either way: + +```json +{ "source.rdb": { "@kind": "view", "@view": "v_program_minutes" } } +``` + +`buildReportViews` skips a report whose source is `@unmanaged: true`, so generate the SQL +before marking a source unmanaged if a shared model needs that flag for another database. + +`buildReportViews` returns the body of each view-backed report for the `mysql` dialect. Put +each one in your own migration as `CREATE VIEW AS `: + +```ts +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { loadDirectory } from "@metaobjectsdev/metadata"; + +const { root } = await loadDirectory("metaobjects"); // wherever your metadata lives +for (const view of buildReportViews(root, { dialect: "mysql" })) { + console.log(`CREATE VIEW \`${view.name}\` AS\n${view.sql};`); +} +``` + +The loop above ignores `view.schema`, which is the report source's `@schema` when it declares +one. If yours does, create the view in that schema yourself (qualify the name in your +migration); the loop will not. + +Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). +The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a +change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your +migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: + +- **Ratios and averages have four fractional digits by default.** MySQL divides to + `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns + `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. +- **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, + so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with + `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches + what the generated tier and the ObjectManager store when the pool uses `timezone: "Z"`. + ## Behaviour that differs from Postgres and SQLite - **Writes read the row back.** MySQL has no `RETURNING`: diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 97954023d..9ed4560fc 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -33,7 +33,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/render-conformance/`](../fixtures/render-conformance/) | 15 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | -| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 33 (27 query + 6 migration) | all 33 | 27 query (migrations TS-only, ADR-0015) | 27 query (via Exposed) | 27 query | 27 query | +| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 39 (33 query + 6 migration) | all 39 | 33 query (migrations TS-only, ADR-0015) | 33 query (via Exposed) | 33 query | 33 query | | [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 61 (31 core + 10 tph + 9 m2m + 2 jsonb + 2 write-through + 7 projection) | ✓ (Fastify reference + generated lane) | ✓ (embedded HTTP + JDBC) | ✓ (embedded HTTP + Exposed) | ✓ (HttpListener + Npgsql) | ✓ (FastAPI + pg8000) | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 16 cases | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | @@ -250,10 +250,10 @@ trailing-newline preservation, and unicode multibyte handling. All 31 fixtures → [features/migrations-and-drift.md](features/migrations-and-drift.md) (template drift section — `Renderer.verify`). -### `fixtures/persistence-conformance/` (33 — 27 query + 6 migration) +### `fixtures/persistence-conformance/` (39 — 33 query + 6 migration) - `migrations/*` (6) → [features/migrations-and-drift.md](features/migrations-and-drift.md) (schema migration section) -- `queries/*` (27) → [features/source-kinds.md](features/source-kinds.md) (query semantics against `source.rdb`) +- `queries/*` (33) → [features/source-kinds.md](features/source-kinds.md) (query semantics against `source.rdb`) ### `fixtures/api-contract-conformance/` (61) @@ -398,7 +398,7 @@ own those two functions), and ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 361 + yaml 16 + verify 31 -+ render 15 + persistence 33 + api-contract 61 + source-resolution 25 + scope 10 + ++ render 15 + persistence 39 + api-contract 61 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider composition, agent context, docs emit) rather than user-facing metamodel behaviour, diff --git a/docs/README.md b/docs/README.md index 0268e0502..661720df0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -66,7 +66,7 @@ this tree is documentation, not the source of truth. | Build on a metadata model another repository publishes (`dependencies`, `meta deps sync`, overlay/extend across the boundary) | [`features/metadata-dependencies.md`](features/metadata-dependencies.md) | | Adopt a design MetaObjects already ships — users/groups/roles, an LLM trace envelope — instead of authoring it (`libraries`, `meta eject `) | [`features/libraries.md`](features/libraries.md) | | Record what the system is supposed to do, and stop agents reviving retired features | [`features/requirements.md`](features/requirements.md) | -| Declare what a dashboard groups by and counts (`dimension`, `measure`, `segment`, `object.report`; load-time checked, no generated output yet) | [`features/reporting.md`](features/reporting.md) | +| Declare what a dashboard groups by and counts (`dimension`, `measure`, `segment`, `object.report`; load-time checked; a view-backed report becomes a SQL view, no routes yet) | [`features/reporting.md`](features/reporting.md) | | Wire prompt construction (FR-004) | [`features/templates-and-payloads.md`](features/templates-and-payloads.md) | | Share a metadata shape across multiple instances (abstracts, `extends:`) | [`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md) | | Add a custom metamodel subtype or attribute to a downstream project | [`features/extending-with-providers.md`](features/extending-with-providers.md) + [`recipes/extending-metaobjects-with-providers.md`](recipes/extending-metaobjects-with-providers.md) | diff --git a/docs/features/reporting.md b/docs/features/reporting.md index c350784ff..c0930a3bf 100644 --- a/docs/features/reporting.md +++ b/docs/features/reporting.md @@ -6,13 +6,18 @@ and counts, as metadata, validated when the model loads._ **Status:** registered and loader-validated in all five ports (TypeScript, C#, Java, Python, Kotlin through Java). Arrived with **metamodel 1.1** (FR-044). -**Reports generate nothing yet.** This release ships the vocabulary and its load-time -rules, and no more. There is no view DDL, no `meta migrate` proposal, no REST route, no -generated client hook and no docs page for an `object.report`, and `meta docs` and the API -docs skip it. A model that declares dimensions, measures, segments and reports generates -byte-for-byte what the same model without them generates, in every port. Generated output -for reports lands in later plans of FR-044; until then the declarations are a checked -statement of intent that an agent or a person can read. +**What a report becomes.** A report that declares a read-only `source.rdb` of `@kind: view` +is **lowered to a SQL view**: `meta migrate` creates it (Postgres, SQLite and D1; MySQL SQL +comes from `buildReportViews`, see [MySQL](#mysql)), and every port reads it through its own +runtime. [What a report lowers to](#what-a-report-lowers-to) is the contract. A report with +no `source.*` stays inert: it is a checked statement of intent that generates nothing. + +**What does not exist yet.** There is no REST route, no typed client or hook, no filter +allowlist and no api-docs entry for a report in any port (the later plans of FR-044). There +is no `measure.derived`, no query-time choice of dimensions or measures (a report is a fixed +combination, compiled once), and no time-zone vocabulary: time grains and relative dates are +UTC. A model that declares none of this generates byte-for-byte what it did before, in every +port. **Entirely opt-in.** A model that declares none of this sees no change at all. @@ -81,6 +86,8 @@ A report is a top-level object that names an entity as its `@from`: { "field.string": { "name": "status" } }, { "field.timestamp": { "name": "purchasedAt" } }, { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], + "@references": "Program" } }, { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, @@ -109,7 +116,10 @@ A report is a top-level object that names an entity as its `@from`: { "object.report": { "name": "StoreTotals", "@from": "Purchase", - "@measures": ["purchases", "buyers", "revenue"] + "@measures": ["purchases", "buyers", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_store_totals" } } + ] }} ] }} @@ -117,11 +127,14 @@ A report is a top-level object that names an entity as its `@from`: What each piece means: -- **`dimension.attribute`** groups by a column's value as-is. `@of` is `Entity.field`. +- **`dimension.attribute`** groups by a column's value as-is. `@of` is `Entity.field`. `@via` + reaches a to-one related entity's column, and each hop needs a **foreign key the model + declares**: an `identity.reference` between the two entities (`fkProgram` above). A + `relationship.*` alone names the hop but says nothing about which column joins it. - **`dimension.time`** groups by a date or timestamp truncated to a grain. It declares which grains it supports in `@grains`. - **`measure.aggregate`** is one aggregate over the entity's own rows. `@agg: count` without - `@distinct` counts rows. With `@distinct: true` it counts distinct values of `@of`, and a + `@distinct` counts the rows whose `@of` is not null. With `@distinct: true` it counts distinct values of `@of`, and a list in `@of` is a distinct count of the tuple. This deliberately differs from `origin.aggregate`, whose `count` is always distinct as a join-inflation guard: a measure aggregates its own entity's rows and a dimension reaches only to-one paths, so no join @@ -148,7 +161,193 @@ order: one per dimension, then one per measure. | measure `revenue` | `revenue` | A `@dimensions` item is a dimension name, or `name:grain` for a time dimension (a single -colon, so it cannot collide with the `::` package separator). +colon, so it cannot collide with the `::` package separator). A `@measures` item is a measure +name, or `Entity.name` where `Entity` is the `@from` entity or one it extends; both forms name +the same measure and derive the same field. + +`StoreTotals` above declares the source that makes it **served**; `DailyRevenue` declares +none, so it is checked at load and nothing more. A report is served only when it declares a +`source.rdb` with `@kind: view` (the next section says what that does). + +## What a report lowers to + +### Which reports lower + +The report's **own** read-only source decides. Dimensions, measures and segments are never +lowered alone. When a report declares several read-only sources, the one with `@role: primary` +decides (else the first): it is the source the view is named by, the one `meta migrate` creates +and the one every runtime reads. + +| The report declares | Result | +|---|---| +| no `source.*` | Inert: no view, no migrate statement, no runtime read. Reading it through an `ObjectManager` fails as "not served". | +| `source.rdb` with `@kind: view` | A derived view. `meta migrate` creates `CREATE VIEW `, where the name is the source's `@view` (or the legacy `@table`). | +| the same, plus `@sql` | Your SQL is the body, exactly as for a projection. The column shape below still defines what the runtime reads. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it (you or a migration tool own the DDL), but the runtime still reads it through the shape below. | +| `@kind: materializedView`, `storedProc` or `tableFunction` | `meta migrate` skips it, as for a projection. | + +A **derived** report view (no `@sql`) whose `@from` entity has no table (it is abstract, or +declares no writable `source.rdb`) fails `meta migrate` with an error naming the report and the +entity, rather than emitting a view over a table that does not exist. A report with an `@sql` +source is not derived, so that check does not apply to it: your SQL is used as written. + +A derived report view is refused in three more cases, each with an error naming the report: + +- **`@from` is a TPH subtype** (an entity with `@discriminatorValue` under a base with + `@discriminator`). The subtype shares its base's table with every other subtype, so a view + derived from it would count all of their rows. Declare the report `@from` the base, with an + `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). A report `@from` the + base is unaffected, and an `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **a `@via` hop has no foreign key in the model.** The error names the hop and the + `identity.reference` it needs. +- **a filter's `in` list is empty**, which no database accepts as SQL. + +### The columns you get + +A report has no primary key and declares no fields; its read shape is derived. One column +per `@dimensions` item in listed order, then one per `@measures` item in listed order. The +physical column name is your naming strategy applied to the **derived field name**; an +`@column` on the `@of` field is never inherited. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only when the dimension has no `@via` and the `@of` field declares `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date`, the first day of the bucket | same rule | +| `count` (with or without `@distinct`) | the measure's name | `long` | yes: a count is never null | +| `sum` of `int` or `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (integer minor units, with the field's `@currency`) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` or `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency` or `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` or `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A derived column carries the type-shaping attributes of its `@of` field where they apply +(`@currency`, `@values`, `@intValueMap`, `@maxLength`, `@precision`, `@scale`, `@localTime`, +`@objectRef`, `@storage`, `@dbColumnType`, `isArray`) and nothing else: no `@column`, no +`@required` beyond the rule above, no `@default`, no validators. + +### Measures + +A measure's rows are the report's rows after its own `@segment` and `@filter` (ANDed) are +applied. The aggregates: + +- **`count`** counts the rows whose `@of` column is not null. On a non-null column that is + every row. With `@distinct` it counts distinct non-null values. A tuple (`@of` with several + items) counts distinct tuples, and a tuple with any null component is not counted, on every + engine. +- **`sum`** of nothing is **null**, not zero: a report with no matching rows, or a filtered + measure that matched none of a group's rows, shows null. A `sum` of an integer type is a + 64-bit integer on every engine (Postgres casts it to `BIGINT`, MySQL to `SIGNED`, and SQLite's + integer `SUM` already is one). +- **`avg`, `min`, `max`** are the engine's own. +- **`measure.ratio`** is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**, + never an error. Each operand is repeated inline with its own conditions, so an operand need + not be listed in `@measures`. + +A report with no dimensions is one row over the whole table. Over an **empty** table that row +still exists: counts are `0`, sums and ratios are null. + +### Dimensions, time grains and joins + +- **`@via`** reaches a column of a to-one related entity, through a join. Every hop is joined + through an `identity.reference` the model declares between the two entities; a hop without one + loads, and then fails `meta migrate` naming the hop. The join type is the + projection rule, unchanged: a required belongs-to foreign key joins `INNER`, anything else + `LEFT OUTER`, and an `INNER` survives only when every join above it is `INNER`. The + consequence to know: **a dimension reached through a required reference drops a fact row + whose reference matches no row, from that report.** A dimension that is not listed in + `@dimensions` adds no join. +- **Grains** are `hour, day, week, month, quarter, year`. A bucket is the first instant (for + `hour`) or first day (for the rest) of the period. **Weeks start on Monday (ISO-8601)** on + every engine: the week of Sunday 2026-05-17 starts 2026-05-11, and Monday 2026-06-01 starts + its own week. +- **UTC only.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session + time zone is, so two readers get the same buckets. A `field.timestamp` with `@localTime` and + a `field.date` are bucketed as stored. There is no vocabulary for another zone. +- `GROUP BY` is every listed dimension, in `@dimensions` order. The report's `@segment` and + `@filter` are the `WHERE`: rows are scoped before grouping, and there is no `HAVING`. +- **Relative dates** in a view are evaluated when the view is **queried**, against the UTC + clock. A naive (`@localTime`) timestamp is compared with the UTC wall clock. + +### What the runtime does with it + +| Port | Read side | +|---|---| +| TypeScript | `ObjectManager` reads a view-backed report through a detached read model: `list` and `count`, with filter, sort and limit on the derived fields. | +| Java | OMDB, the same read. | +| Python | `ObjectManager`, the same read. | +| C# | codegen writes a keyless EF Core row class per view-backed report and maps it with `HasNoKey().ToView(...)` plus a `DbSet`. | +| Kotlin | codegen writes an Exposed table object per view-backed report. | + +By-id and every write are refused (a report has no identity and is read-only); a report with no +view source is refused as not served; an `@unmanaged` view-backed report is still read. C# also +refuses a report whose derived field name, in Pascal case, equals the report's own class name, +since the row class could not have a member named like itself. Kotlin refuses a view-backed +report in two cases, because the generated table would not compile: a derived field named after +a Kotlin hard keyword (`in`, `is`, `object`, `when`, …), and two derived fields that land on one +column property (a name that collides with a member of Exposed's `Table`, such as `source`, gets +a `Column` suffix, which can meet a second field already called `sourceColumn`). Both fail `gen` +with an error naming the report and the dimension or measure. No port generates a route, typed +client, filter allowlist or api-docs entry for a report. + +`meta docs` lists a report's view on the agent schema page (`agent/schema.md`) and on no other +page. + +### What differs by engine + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| Created by | `meta migrate` | `meta migrate` | you (see below) | +| `avg` and ratio of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: SQLite has no decimal, so `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants and dates | `TIMESTAMPTZ`, `DATE` | ISO-8601 text (an hour bucket is `...:00:00.000Z`) | `DATETIME(3)` read as the UTC wall clock | + +A changed report is dropped and re-created by `meta migrate` (it does not `CREATE OR +REPLACE`, since the diff does not know the old column list). + +#### MySQL + +`meta migrate` never targets MySQL (ADR-0015), so on MySQL you create the view yourself. +`buildReportViews(root, { dialect: "mysql" })` from `@metaobjectsdev/codegen-ts` returns the +body of each view-backed report; the recipe in [`docs/recipes/mysql.md`](../recipes/mysql.md) +("Reports") shows the loop and its caveats. It skips a report whose source is `@unmanaged`, and +the bodies are valid under MySQL's default `ONLY_FULL_GROUP_BY`. + +### Known limits + +- **A derived report from a TPH subtype is refused.** Declare it from the base with an `@filter` + on the discriminator field (see "Which reports lower"). +- **An abstract view-backed report gets no C# row class and no Kotlin table object.** The + TypeScript, Java and Python runtimes still read it. The same holds for a report whose source + `@kind` is `materializedView`, `storedProc` or `tableFunction`: C# and Kotlin generate nothing + for it, `meta migrate` skips it, and the three runtimes issue a `SELECT` against whatever + relation the source names. That works for a materialized view you created and is a database + error for a stored procedure or a table function. +- **A report over a `field.object` is not supported across ports.** A dimension whose `@of` is a + `field.object` (or a field carrying `@objectRef`) loads in every port. Java OMDB then refuses + the read, and Kotlin `gen` and C# `gen` refuse to generate the table or row, each with an + error naming the report and the dimension (the same refusal covers a measure whose column is + typed by such a field). The TypeScript and Python runtimes read it and return the parsed JSON. + Group by a scalar field. +- **With `@via`, `@of` must name an entity that has the field** (declared on it or inherited by + it); naming a base of the reached entity for a field only the subtype declares loads and then + fails `meta migrate`. The quiet form of the same rule: a `@via` dimension reads its field from + the entity `@of` names, so when the reached subtype **redeclares** that field and `@of` names + the base, the view selects the base's column and type with no error. Qualify `@of` with the + subtype that declares the field you mean. Without `@via` the field is read from the `@from` + entity itself. + +### What the corpus gates + +Six shared scenarios under `fixtures/persistence-conformance/queries/report-*.yaml` read the +canonical reports through every port's runtime (list and count, filter, sort, an empty table, +the Monday boundary, an hour bucket, a relative window). The derived columns are pinned by +`fixtures/persistence-conformance/report-shapes.json`, produced by TypeScript and byte-matched +by every port. The SQL is produced by TypeScript only, so the other ports read the view the +TypeScript migrate engine produced and never lower a report themselves. ## The rules the loader enforces @@ -238,8 +437,11 @@ of the four operators a relative date may sit under, so it is refused. The loader checks that the declarations are consistent with each other and with the model. It does not check that a measure means what its name says, that a segment's filter selects -the rows you intend, or that the data exists. And since a report generates nothing yet, a -passing load says nothing about any query: there is no query. +the rows you intend, or that the data exists. A green `meta migrate` proves the view was +created, not that its numbers are the ones you mean: a dimension reached through a required +reference leaves out the fact rows whose reference matches nothing, and a report's +`@filter` may select no rows at all. A report with no view source is still only checked at +load, and a passing load says nothing about a query against it: there is none. ## Compatibility diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index 30e8cc235..8ad59f127 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -107,6 +107,10 @@ Schema migrations are owned by the Node `meta` CLI (ADR-0015) — the C# CLI is The codegen emits: - `Author.g.cs` — class per entity (a mutable attributed POCO, not a record). +- `.g.cs` — keyless row class per **view-backed report** (`object.report` with a + read-only `@kind: view` source); mapped in `AppDbContext` with `DbSet` + + `HasNoKey().ToView(...)`. No routes or filter allowlist for a report. See + [reporting](../features/reporting.md). - `AppDbContext.g.cs` — `DbSet`, projection `.ToView()`, `@storage` owned types via `OwnsOne` (single) / `OwnsMany(...).ToJson(...)` (`@isArray` array-of-VO), enum-as-string via `HasConversion()`. diff --git a/docs/ports/kotlin.md b/docs/ports/kotlin.md index 20b8f5912..9f8683e86 100644 --- a/docs/ports/kotlin.md +++ b/docs/ports/kotlin.md @@ -354,6 +354,51 @@ class AuthorService(private val db: Database) { } ``` +### Reports + +For an `object.report` that declares a read-only `source.rdb` of `@kind: view`, +`KotlinExposedTableGenerator` writes one read-only Exposed table object, `Table`, bound +to that view, with one column per derived field (dimensions, then measures). That is all Kotlin +generates for a report: no row class, no `Names`, and nothing from any other generator. +A report with no view source generates nothing. See [reporting](../features/reporting.md) for +the vocabulary and the columns a report gets. An excerpt, under the default snake_case column +naming: + +```kotlin +object ProgramMinutesTable : Table("v_program_minutes") { + val program = long("program") + val weeks = long("weeks") + val totalMinutes = long("total_minutes").nullable() + val avgMinutes = decimal("avg_minutes", 38, 18).nullable() + // … one column per derived field +} +``` + +- A report has no identity, so the object has no `primaryKey`. List it and count it; there is + no by-id read and no write. +- A column is nullable exactly when the derived field can be null: a `sum`, `avg`, `min`, `max` + or ratio, and a dimension reached through `@via`. +- A derived decimal with no declared precision (an `avg`, a ratio, a `sum` of a decimal) is + read as `decimal(name, 38, 18)`. Exposed rounds a decimal to the column's scale when it reads + it, so the value is exact to 18 places. The object maps a view, so those numbers never reach + DDL. +- An enum dimension is typed by the enum class of the entity it reads (`ProgramStatus` for a + dimension over `Program.status`), so it compares against the same constants as the entity's + own column. No per-report enum is generated. +- The view and its columns are bound by string literal even when `useNames` is on. +- `gen` fails, naming the report and the dimension or measure, when a derived field is named + after a Kotlin hard keyword or when two derived fields land on one column property (see + below for the `Column` suffix). Rename the item. +- `gen` also fails, naming the report and the dimension or measure, when a derived field reads a + `field.object` (a dimension over an embedded value object, say). A report over a + `field.object` is not supported; group by a scalar field. +- An abstract report generates nothing, view or not. + +A column property whose name is a member of Exposed's `Table` gets a `Column` suffix +(`source` becomes `sourceColumn`); the physical column name does not change. The reserved set +follows the output mode: `options` and `storageParameters` are `Table` members only in Exposed +1.x, so they are suffixed only with `exposedApi=1`. + ### `Names` — the physical names, as constants `names` (`KotlinNamesGenerator`) is **not** wired above — it is opt-in, like diff --git a/docs/recipes/mysql.md b/docs/recipes/mysql.md index 59a3db74e..d67dae1be 100644 --- a/docs/recipes/mysql.md +++ b/docs/recipes/mysql.md @@ -77,6 +77,51 @@ the reference: its column builders name the MySQL types. A column named after a MySQL reserved word (`rank`, `order`) needs backticks in your DDL. The generated code and both runtimes quote identifiers themselves. +### Reports + +An `object.report` (the reporting vocabulary; see `references/reporting.md` in the +`metaobjects-authoring` skill) is a compiled view, and on MySQL you create that view +yourself, because `meta migrate` does not. Declare the report with a read-only +`source.rdb` of `@kind: view` and no `@unmanaged`, since `meta migrate` never targets MySQL +and so nothing manages the view either way: + +```json +{ "source.rdb": { "@kind": "view", "@view": "v_program_minutes" } } +``` + +`buildReportViews` skips a report whose source is `@unmanaged: true`, so generate the SQL +before marking a source unmanaged if a shared model needs that flag for another database. + +`buildReportViews` returns the body of each view-backed report for the `mysql` dialect. Put +each one in your own migration as `CREATE VIEW AS `: + +```ts +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { loadDirectory } from "@metaobjectsdev/metadata"; + +const { root } = await loadDirectory("metaobjects"); // wherever your metadata lives +for (const view of buildReportViews(root, { dialect: "mysql" })) { + console.log(`CREATE VIEW \`${view.name}\` AS\n${view.sql};`); +} +``` + +The loop above ignores `view.schema`, which is the report source's `@schema` when it declares +one. If yours does, create the view in that schema yourself (qualify the name in your +migration); the loop will not. + +Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). +The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a +change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your +migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: + +- **Ratios and averages have four fractional digits by default.** MySQL divides to + `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns + `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. +- **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, + so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with + `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches + what the generated tier and the ObjectManager store when the pool uses `timezone: "Z"`. + ## Behaviour that differs from Postgres and SQLite - **Writes read the row back.** MySQL has no `RETURNING`: diff --git a/docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md b/docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md index 183bfc30b..2ee6168a5 100644 --- a/docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md +++ b/docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md @@ -311,7 +311,7 @@ These shapes were run on the three engines with conditionally quoted identifiers | Gate | Path | Ports | |---|---|---| | Canonical reports (model) | `fixtures/persistence-conformance/canonical/meta.fitness.json` | all (shared input) | -| Report shapes artifact | `fixtures/persistence-conformance/canonical/report-shapes.json` (new, TS-produced, committed) | TS produces and drift-checks; C#, Java, Kotlin (through Java), Python byte-match their own derivation in a container-free unit test | +| Report shapes artifact | `fixtures/persistence-conformance/report-shapes.json` (new, TS-produced, committed) | TS produces and drift-checks; C#, Java, Kotlin (through Java), Python byte-match their own derivation in a container-free unit test | | Canonical schema | `fixtures/persistence-conformance/canonical/schema.postgres.sql` (regenerated: six views added) | all execute it | | `queries/report-grouped-measures.yaml` | every Table C row; an attribute dimension reached by `@via`; `filter`, `sort`, `count` on derived fields | all five | | `queries/report-totals.yaml` | no dimensions → one row; ratio | all five | @@ -480,7 +480,7 @@ The fixture's fields carry no `@required`, so every dimension is `required: fals ```ts // Table B of docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md: // a report's derived fields. The single definition; every port has a rule-for-rule copy, -// gated by fixtures/persistence-conformance/canonical/report-shapes.json. +// gated by fixtures/persistence-conformance/report-shapes.json. const SUM_LONG: ReadonlySet = new Set([FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG]); const FLOATING: ReadonlySet = new Set([FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT]); @@ -953,7 +953,7 @@ function literal(v: unknown, d: ReportDialect): string { - Modify: `server/typescript/packages/codegen-ts/src/index.ts` (export `buildReportViews` beside `buildProjectionViews` at line 270) - Modify: `fixtures/persistence-conformance/canonical/meta.fitness.json` - Regenerate: `fixtures/persistence-conformance/canonical/schema.postgres.sql` -- Create: `server/typescript/packages/integration-tests/src/gen-report-shapes.ts`, `fixtures/persistence-conformance/canonical/report-shapes.json` +- Create: `server/typescript/packages/integration-tests/src/gen-report-shapes.ts`, `fixtures/persistence-conformance/report-shapes.json` - Modify: `server/typescript/packages/integration-tests/package.json` (script `"gen:report-shapes": "bun run src/gen-report-shapes.ts"`) - Test: `server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts` (new), `test/schema-artifact.test.ts` (existing), `server/typescript/packages/codegen-ts/test/projection/build-projection-views.test.ts` (add cases) - Modify: `server/typescript/packages/cli/test/unit/reporting-inert.test.ts`, `fixtures/codegen-noop/reporting/README.md` diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md index 945e40148..eb54c6be2 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md @@ -59,6 +59,51 @@ the reference: its column builders name the MySQL types. A column named after a MySQL reserved word (`rank`, `order`) needs backticks in your DDL. The generated code and both runtimes quote identifiers themselves. +### Reports + +An `object.report` (the reporting vocabulary; see `references/reporting.md` in the +`metaobjects-authoring` skill) is a compiled view, and on MySQL you create that view +yourself, because `meta migrate` does not. Declare the report with a read-only +`source.rdb` of `@kind: view` and no `@unmanaged`, since `meta migrate` never targets MySQL +and so nothing manages the view either way: + +```json +{ "source.rdb": { "@kind": "view", "@view": "v_program_minutes" } } +``` + +`buildReportViews` skips a report whose source is `@unmanaged: true`, so generate the SQL +before marking a source unmanaged if a shared model needs that flag for another database. + +`buildReportViews` returns the body of each view-backed report for the `mysql` dialect. Put +each one in your own migration as `CREATE VIEW AS `: + +```ts +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { loadDirectory } from "@metaobjectsdev/metadata"; + +const { root } = await loadDirectory("metaobjects"); // wherever your metadata lives +for (const view of buildReportViews(root, { dialect: "mysql" })) { + console.log(`CREATE VIEW \`${view.name}\` AS\n${view.sql};`); +} +``` + +The loop above ignores `view.schema`, which is the report source's `@schema` when it declares +one. If yours does, create the view in that schema yourself (qualify the name in your +migration); the loop will not. + +Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). +The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a +change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your +migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: + +- **Ratios and averages have four fractional digits by default.** MySQL divides to + `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns + `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. +- **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, + so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with + `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches + what the generated tier and the ObjectManager store when the pool uses `timezone: "Z"`. + ## Behaviour that differs from Postgres and SQLite - **Writes read the row back.** MySQL has no `RETURNING`: diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index 6dbd5e4ed..0e34c3503 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,6 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -677,6 +678,81 @@ Several variants of one thing sharing **one table**: the base `object.entity` de `@discriminatorValue`. Codegen emits per-subtype routes with the discriminator injected and immutable. Supported in all five ports; the worked example is in `references/inheritance-tph.md`. +## Reporting — dimensions, measures and reports + +Reach for it when a dashboard number would otherwise be a hand-written `GROUP BY`: revenue per +day, buyers per program, a total. You name the pieces once, on the entity that owns the rows, +and an `object.report` combines them by name. Four node kinds: + +- `dimension.attribute` / `dimension.time` — what to group by (`@of: Entity.field`; a time + dimension lists the `@grains` it supports: `hour`, `day`, `week`, `month`, `quarter`, `year`); +- `measure.aggregate` (`@agg`: `count`, `sum`, `avg`, `min`, `max`) and `measure.ratio` + (`@numerator` / `@denominator`, both measures of the entity); +- `segment.filter` — a named, reusable `@filter` ("active purchase"); +- `object.report` — a top-level object: `@from` an entity, `@dimensions` (`name` or + `name:grain`), `@measures`. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Purchase", + "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "status" } }, + { "field.currency": { "name": "amountCents" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, + { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", + "@grains": ["day", "month"] } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id", + "@segment": "active" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents" } } + ] + }}, + { "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_daily_revenue" } } + ] + }} + ] +}} +``` + +Three rules an author trips on: + +1. **Every measure belongs to `@from`.** A report cannot mix measures of two entities (joining + two fact tables multiplies each side's rows); two fact tables are two reports. +2. **`@via` is to-one only, and each hop needs a declared foreign key.** A dimension reaches a + related entity's column through a `relationship.*` with `@cardinality: one` (or an + `identity.reference`), never through a to-many, which would repeat fact rows and double-count + a `sum`. The view joins the hop through an `identity.reference` between the two entities + (`{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": + "Program" } }`); a relationship with none behind it loads and then fails `meta migrate`. +3. **A report declares no fields.** Its columns are derived: one per dimension, then one per + measure (a time dimension at a grain is ``, so `purchasedAt:day` is + `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. + +**A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration +is what makes `meta migrate` create the view (Postgres, SQLite, D1) and what every port's +runtime reads; a report with no `source.*` is checked at load and generates nothing. + +A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its +base's table): declare it `@from` the base with an `@filter` on the discriminator field. + +What does not exist: no REST route and no typed client for a report yet, no `measure.derived` +(arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or +measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and +relative dates are UTC). Column types, the null rules, Monday weeks and per-engine differences +are in `references/reporting.md`. + ## Requirements — capability ledger (opt-in) **This capability exists whether or not the project uses it yet.** `requirement.functional` and `requirement.architectural` are registered metadata types, declared in `metaobjects/` beside the entities they describe and loaded by the same loader — no side file, no bespoke parser. They record *why* each part of the model exists, so a field with no reason to exist becomes visible as one. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md new file mode 100644 index 000000000..1116503c6 --- /dev/null +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -0,0 +1,83 @@ +# Reporting: the columns a report gets, and how it behaves + +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. + +## A report is served only with a view source + +A report is a compiled view. The report's **own** read-only source decides what happens: + +| The report declares | Result | +|---|---| +| no `source.*` | Checked at load, nothing else. No view, no migrate statement, no runtime read (an `ObjectManager` refuses it as "not served"). | +| `source.rdb` with `@kind: view` | `meta migrate` creates the view (Postgres, SQLite, D1) under the source's `@view` name, and every port reads it. | +| the same, plus `@sql` | Your SQL is the view body. The columns below still define what is read. | +| the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it; the runtime still reads it. | +| `@kind: materializedView`, `storedProc`, `tableFunction` | `meta migrate` skips it. | + +A derived report view (no `@sql`) whose `@from` entity has no table (abstract, or no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity; a report with an `@sql` source skips that check, since your SQL is used as written. A changed report is dropped and re-created by `meta migrate`. When a report declares several read-only sources, the one with `@role: primary` decides (else the first). + +## The columns you get + +A report declares no fields. Its columns are one per `@dimensions` item in listed order, then one per `@measures` item in listed order, named by the derived field name (your naming strategy applies to that name; an `@column` on the `@of` field is not inherited). A report has no primary key: read it with list and count (filter, sort and limit work on the derived columns); get-by-id and every write are refused. + +| Item | Column | Type | Never null? | +|---|---|---|---| +| `dimension.attribute` | the dimension's name | the `@of` field's type | only with no `@via` and an `@of` field with `@required: true` | +| `dimension.time` at `hour` | `Hour` | `timestamp` | same rule | +| `dimension.time` at `day`, `week`, `month`, `quarter`, `year` | `` | `date` (first day of the bucket) | same rule | +| `count`, with or without `@distinct` | the measure's name | `long` | yes | +| `sum` of `int` / `long` | the measure's name | `long` | no | +| `sum` of `currency` | the measure's name | `currency` (minor units) | no | +| `sum` of `decimal` | the measure's name | `decimal` | no | +| `sum` of `double` / `float` | the measure's name | `double` | no | +| `avg` of `int`, `long`, `currency`, `decimal` | the measure's name | `decimal` | no | +| `avg` of `double` / `float` | the measure's name | `double` | no | +| `min` / `max` | the measure's name | the `@of` field's type | no | +| `measure.ratio` | the measure's name | `decimal` | no | + +A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. + +## Time grains + +`hour, day, week, month, quarter, year`; `hour` is illegal on a `field.date`. **Weeks start on Monday (ISO-8601)** on every engine: Sunday 2026-05-17 falls in the week of 2026-05-11, and Monday 2026-06-01 opens its own week. + +**Bucketing is UTC.** A `field.timestamp` instant is bucketed in UTC whatever the reader's session time zone is, so every reader gets the same buckets. A `@localTime` timestamp and a `field.date` are bucketed as stored. There is no vocabulary for another time zone; do not look for one. + +## Relative dates + +A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 duration) is legal only on a `field.date` or `field.timestamp`, under `gt`, `gte`, `lt` or `lte`, and only in the `@filter` of a `segment`, a `measure.aggregate` or an `object.report`. It is evaluated when the view is **queried**, against the UTC clock. + +## Nulls and zeros + +- A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. +- A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). + +## Joins: a dimension through a required reference drops rows + +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. + +**Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. + +## Engine differences + +| | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| View created by | `meta migrate` | `meta migrate` | you: see below | +| A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | +| `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | + +**MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. + +## Known limits + +- **A report `@from` a TPH subtype is refused** when its view is derived. The subtype shares its base's table with every other subtype, so the view would count all of their rows. Declare the report `@from` the base, with an `@filter` on the discriminator field (`"@filter": { "kind": "ADMIN" }`). An `@sql` or `@unmanaged` report over a subtype is yours to scope. +- **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. +- **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). +- **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Kotlin `gen` and C# `gen`; only the TypeScript and Python runtimes read it (as parsed JSON). Group by a scalar field. + +## What a report does not have + +No REST route, typed client, filter allowlist or api-docs entry is generated for a report in any port. There is no `measure.derived`, no query-time choice of dimensions or measures, and no time-zone vocabulary. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md index 945e40148..eb54c6be2 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md @@ -59,6 +59,51 @@ the reference: its column builders name the MySQL types. A column named after a MySQL reserved word (`rank`, `order`) needs backticks in your DDL. The generated code and both runtimes quote identifiers themselves. +### Reports + +An `object.report` (the reporting vocabulary; see `references/reporting.md` in the +`metaobjects-authoring` skill) is a compiled view, and on MySQL you create that view +yourself, because `meta migrate` does not. Declare the report with a read-only +`source.rdb` of `@kind: view` and no `@unmanaged`, since `meta migrate` never targets MySQL +and so nothing manages the view either way: + +```json +{ "source.rdb": { "@kind": "view", "@view": "v_program_minutes" } } +``` + +`buildReportViews` skips a report whose source is `@unmanaged: true`, so generate the SQL +before marking a source unmanaged if a shared model needs that flag for another database. + +`buildReportViews` returns the body of each view-backed report for the `mysql` dialect. Put +each one in your own migration as `CREATE VIEW AS `: + +```ts +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { loadDirectory } from "@metaobjectsdev/metadata"; + +const { root } = await loadDirectory("metaobjects"); // wherever your metadata lives +for (const view of buildReportViews(root, { dialect: "mysql" })) { + console.log(`CREATE VIEW \`${view.name}\` AS\n${view.sql};`); +} +``` + +The loop above ignores `view.schema`, which is the report source's `@schema` when it declares +one. If yours does, create the view in that schema yourself (qualify the name in your +migration); the loop will not. + +Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). +The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a +change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your +migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: + +- **Ratios and averages have four fractional digits by default.** MySQL divides to + `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns + `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. +- **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, + so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with + `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches + what the generated tier and the ObjectManager store when the pool uses `timezone: "Z"`. + ## Behaviour that differs from Postgres and SQLite - **Writes read the row back.** MySQL has no `RETURNING`: diff --git a/fixtures/codegen-noop/reporting/README.md b/fixtures/codegen-noop/reporting/README.md index 1f6ec4ffe..bc3139e8f 100644 --- a/fixtures/codegen-noop/reporting/README.md +++ b/fixtures/codegen-noop/reporting/README.md @@ -1,4 +1,4 @@ -# Reporting vocabulary is inert (FR-044 Plan 1) +# Reporting vocabulary: what is lowered, what stays inert (FR-044) Two models that differ ONLY by the FR-044 reporting vocabulary: @@ -10,9 +10,21 @@ Two models that differ ONLY by the FR-044 reporting vocabulary: a view-backed report passes every source-keyed codegen gate and is the shape most likely to leak output. -Until a report's lowering lands (Plan 2/3), every generator in every port must emit -byte-identical files for the two models, and TypeScript migrate must propose nothing for -the difference. The per-port tests that hold this: +What is lowered (FR-044 Plan 2): a report that declares a read-only `source.rdb @kind: view` +becomes that view. `StoreTotals` is that report, so `with/` differs from `without/` in exactly +these places and no others: + +- TypeScript `meta migrate` proposes one extra statement, `CREATE VIEW v_store_totals`, and the + `meta docs` agent schema page lists that view (the `## Views` section). +- C# codegen (`dotnet meta gen`) writes one extra file, the keyless row class `StoreTotals.g.cs`, and two extra + lines in `AppDbContext.g.cs` (a `DbSet` and `HasNoKey().ToView("v_store_totals")`). +- Kotlin codegen (`metaobjects:generate`) writes an Exposed table object for `StoreTotals`. + +What stays inert: a report with no read-only source (`ProgramEngagement`, `DailyRevenue`), +everywhere; every generator in TypeScript, Java and Python, for every report; routes, typed +clients, filter allowlists and api-docs in every port; and every other C# and Kotlin generator. +No port but TypeScript emits SQL for a report (ADR-0015), so for Java and Python `with/` and +`without/` still generate byte-identical files. The per-port tests that hold this: | Port | Test | |---|---| @@ -22,10 +34,11 @@ the difference. The per-port tests that hold this: | Kotlin | `server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt` | | Python | `server/python/tests/test_reporting_inert.py` | -The documentation tier is held to the same rule (FR-044 Plan 1 ruling): `meta docs` (model, -agent, requirements and site pages) and every port's api-docs builder emit nothing for a -report, because its fields are derived by the lowering and a page today would show none of -them. The TypeScript, C#, Java, Kotlin and Python tests above compare that output too. +The documentation tier is held to the same rule (FR-044 Plan 1 ruling), with the one entry +above: `meta docs` model, requirements and site pages and every port's api-docs builder emit +nothing for a report, because its fields are derived by the lowering and a page would show +none of them. Only the agent schema page lists a view-backed report's view. The five tests +above compare that output too. Regenerate `without/` from `with/` by deleting every `dimension.*`, `measure.*` and `segment.*` child and every `object.report` node — nothing else may differ. diff --git a/fixtures/generator-registry-conformance/registry.json b/fixtures/generator-registry-conformance/registry.json index 3d292e001..5c69ac6ca 100644 --- a/fixtures/generator-registry-conformance/registry.json +++ b/fixtures/generator-registry-conformance/registry.json @@ -124,7 +124,7 @@ "ports": ["java"] }, "exposed-table": { - "concept": "Per-entity Kotlin Exposed table object.", + "concept": "Kotlin Exposed table object per table-backed or view-backed entity or projection, and per view-backed report.", "tier": "native", "layer": "persistence", "ports": ["kotlin"] diff --git a/fixtures/persistence-conformance/README.md b/fixtures/persistence-conformance/README.md index 3a2ceacd3..52555d156 100644 --- a/fixtures/persistence-conformance/README.md +++ b/fixtures/persistence-conformance/README.md @@ -26,6 +26,7 @@ commands), and are required before any release publish — see fixtures/persistence-conformance/ ├── README.md # this file — spec + DSL ├── normalization.md # how each port serializes result rows +├── report-shapes.json # TS-produced derived fields of each report; every port byte-matches it ├── canonical/ # SHARED "kitchen-sink" metadata + committed schema DDL │ ├── meta.*.json # the kitchen-sink metadata every query scenario reads │ └── schema.postgres.sql # TS-produced canonical DDL every port executes to set up its DB @@ -38,8 +39,9 @@ fixtures/persistence-conformance/ ### The canonical schema artifact `canonical/schema.postgres.sql` is **generated by TypeScript** from -`canonical/meta.fitness.json` (base tables + the `origin.aggregate` / -`origin.passthrough` projection views) and **committed**. It carries a +`canonical/meta.fitness.json` (base tables, the `origin.aggregate` / +`origin.passthrough` projection views and the view of each view-backed `object.report`) +and **committed**. It carries a `@generated … DO NOT EDIT` header; every port's query runner executes it verbatim to provision its test DB, so the *runtime* layer is exercised against a schema no port synthesized. @@ -312,6 +314,32 @@ Because M:N membership is a **set**, the runner compares `relate` results > (both columns equal X) — the historical Kotlin failure mode this scenario > pins; all five ports now retain the `(a,a)` row. +### Report scenarios (FR-044) + +Six `queries/report-*.yaml` scenarios read a view-backed `object.report` through the +port's runtime. They use only `op: list` and `op: count`, single-key `sort`, `filter` and +`limit`: a report has no primary key, so there is no `op: get` and no write. The schema is +still the committed `canonical/schema.postgres.sql`, which creates each report's view, so a +port reads the view the TypeScript migrate engine produced and never lowers a report itself. +Rows are keyed by the report's **derived field names** (dimensions in `@dimensions` order, +then measures in `@measures` order; a time dimension is named ``, for example +`createdAtMonth`). + +| Scenario | What it pins | +|---|---| +| `report-grouped-measures` | every measure kind; an attribute dimension reached through a to-one reference; `filter`, `sort` and `limit` on a measure; `count` of groups | +| `report-totals` | no dimensions: one row for the whole table; a ratio | +| `report-totals-empty` | one row over an empty table: `count` is `0`, `sum` is `null`, a zero-denominator ratio is `null` | +| `report-time-grains` | `month` grain beside an enum dimension; a segment-filtered `sum` that is `null`; the ISO Monday week boundary; a report-level `@segment` | +| `report-time-hour-and-date` | `hour` on an instant, `week` on a `field.date` | +| `report-relative-date` | a `{ now: "-P30D" }` filter, with a seed relative to the database clock | + +Wire shapes follow the view's column types and [`normalization.md`](./normalization.md): +`count` and the integral `sum` are `BIGINT` (string), `min`/`max` of an int are `INTEGER` +(number), `avg` and a ratio are `NUMERIC` (canonical decimal string, so `"60"`, `"0.75"` +and `"0"`), a `date`-grain bucket is a `DATE` string, and an instant `hour` bucket is a +`TIMESTAMPTZ` string in UTC (`"2026-05-04T03:00:00Z"`). + ### Filter operators Same vocabulary as the cross-language filter spec (Project D): diff --git a/fixtures/persistence-conformance/canonical/meta.fitness.json b/fixtures/persistence-conformance/canonical/meta.fitness.json index 2e52c10aa..d743dbd67 100644 --- a/fixtures/persistence-conformance/canonical/meta.fitness.json +++ b/fixtures/persistence-conformance/canonical/meta.fitness.json @@ -15,9 +15,11 @@ { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, { "identity.secondary": { "name": "byTitle", "@fields": "title" } }, { "index.lookup": { "name": "idx_programs_title_status", "@fields": ["title", "status"], "@orders": ["asc", "asc"] } }, - { "dimension.time": { "name": "createdAt", "@of": "Program.createdAt", "@grains": ["day", "month"] } }, + { "dimension.time": { "name": "createdAt", "@of": "Program.createdAt", "@grains": ["day", "week", "month", "quarter", "year"] } }, { "measure.aggregate": { "name": "listValue", "@agg": "sum", "@of": "Program.priceCents", "@segment": "published" } }, - { "segment.filter": { "name": "published", "@filter": { "status": "PUBLISHED" } } } + { "segment.filter": { "name": "published", "@filter": { "status": "PUBLISHED" } } }, + { "dimension.attribute": { "name": "status", "@of": "Program.status" } }, + { "measure.aggregate": { "name": "programs", "@agg": "count", "@of": "Program.id" } } ] }}, { "object.entity": { @@ -29,7 +31,20 @@ { "field.string": { "name": "label", "@maxLength": 80 } }, { "field.int": { "name": "durationMinutes", "@required": true } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, - { "identity.reference": { "name": "fkProgram", "@fields": "programId", "@references": "Program" } } + { "identity.reference": { "name": "fkProgram", "@fields": "programId", "@references": "Program" } }, + { "segment.filter": { "name": "long", "@filter": { "durationMinutes": { "gte": 60 } } } }, + { "dimension.attribute": { "name": "program", "@of": "Week.programId" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { "name": "weeks", "@agg": "count", "@of": "Week.id" } }, + { "measure.aggregate": { "name": "longWeeks", "@agg": "count", "@of": "Week.id", "@segment": "long" } }, + { "measure.aggregate": { "name": "labels", "@agg": "count", "@distinct": true, "@of": "Week.label" } }, + { "measure.aggregate": { "name": "slots", "@agg": "count", "@distinct": true, + "@of": ["Week.programId", "Week.durationMinutes"] } }, + { "measure.aggregate": { "name": "totalMinutes", "@agg": "sum", "@of": "Week.durationMinutes" } }, + { "measure.aggregate": { "name": "avgMinutes", "@agg": "avg", "@of": "Week.durationMinutes" } }, + { "measure.aggregate": { "name": "minMinutes", "@agg": "min", "@of": "Week.durationMinutes" } }, + { "measure.aggregate": { "name": "maxMinutes", "@agg": "max", "@of": "Week.durationMinutes" } }, + { "measure.ratio": { "name": "longShare", "@numerator": "longWeeks", "@denominator": "weeks" } } ] }}, { "object.entity": { @@ -66,7 +81,10 @@ { "field.timestamp": { "name": "observedAt", "@required": true, "@localTime": true } }, { "field.date": { "name": "asOfDate", "@required": true } }, { "field.time": { "name": "atTime", "@required": true } }, - { "identity.primary": { "name": "id", "@fields": "id", "@generation": "uuid" } } + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "uuid" } }, + { "dimension.time": { "name": "recordedAt", "@of": "Asset.recordedAt", "@grains": ["hour", "day"] } }, + { "dimension.time": { "name": "asOfDate", "@of": "Asset.asOfDate", "@grains": ["week", "month"] } }, + { "measure.aggregate": { "name": "assets", "@agg": "count", "@of": "Asset.id" } } ] }}, { "object.projection": { @@ -303,6 +321,26 @@ }}, + { "object.report": { "name": "ProgramMinutes", "@from": "Week", + "@dimensions": ["program", "programTitle"], + "@measures": ["Week.weeks", "longWeeks", "labels", "slots", "totalMinutes", "avgMinutes", "minMinutes", "maxMinutes", "longShare"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_minutes" } } ] } }, + { "object.report": { "name": "FitnessTotals", "@from": "Week", + "@measures": ["weeks", "totalMinutes", "longShare"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_fitness_totals" } } ] } }, + { "object.report": { "name": "ProgramsByMonth", "@from": "Program", + "@dimensions": ["createdAt:month", "status"], "@measures": ["programs", "listValue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_programs_by_month" } } ] } }, + { "object.report": { "name": "ProgramsByWeek", "@from": "Program", + "@dimensions": ["createdAt:week"], "@measures": ["programs"], "@segment": "published", + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_programs_by_week" } } ] } }, + { "object.report": { "name": "RecentPrograms", "@from": "Program", + "@measures": ["programs"], "@filter": { "createdAt": { "gte": { "now": "-P30D" } } }, + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_recent_programs" } } ] } }, + { "object.report": { "name": "AssetActivity", "@from": "Asset", + "@dimensions": ["recordedAt:hour", "asOfDate:week"], "@measures": ["assets"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_asset_activity" } } ] } }, + { "template.prompt": { "name": "coachNote", "@payloadRef": "ProgramBrief", diff --git a/fixtures/persistence-conformance/canonical/schema.postgres.sql b/fixtures/persistence-conformance/canonical/schema.postgres.sql index 212fceb74..27f33a8a9 100644 --- a/fixtures/persistence-conformance/canonical/schema.postgres.sql +++ b/fixtures/persistence-conformance/canonical/schema.postgres.sql @@ -175,3 +175,64 @@ CREATE VIEW "v_program_stat" AS LEFT OUTER JOIN weeks w ON w."programId" = p.id GROUP BY p.id; COMMENT ON VIEW "v_program_stat" IS 'metaobjects:v1:sha256:9120c9f8899e257b52a087c4841b8e55f679d7f9c7a6e58d8823912855f2ab57'; + +CREATE VIEW "v_program_minutes" AS + SELECT + w."programId" AS "program", + p."title" AS "programTitle", + COUNT(w."id") AS "weeks", + COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS "longWeeks", + COUNT(DISTINCT w."label") AS "labels", + COUNT(DISTINCT (w."programId", w."durationMinutes")) FILTER (WHERE w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL) AS "slots", + CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes", + AVG(w."durationMinutes") AS "avgMinutes", + MIN(w."durationMinutes") AS "minMinutes", + MAX(w."durationMinutes") AS "maxMinutes", + CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare" + FROM "weeks" w + INNER JOIN "programs" p ON p."id" = w."programId" + GROUP BY w."programId", p."title"; +COMMENT ON VIEW "v_program_minutes" IS 'metaobjects:v1:sha256:06b54e731e0221acab4b6e0a7cb7f5914662eff39af6ca57031a7df69cbbec93'; + +CREATE VIEW "v_fitness_totals" AS + SELECT + COUNT(w."id") AS "weeks", + CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes", + CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare" + FROM "weeks" w; +COMMENT ON VIEW "v_fitness_totals" IS 'metaobjects:v1:sha256:92ac554bcb44ee0bc7f3ddab40cf64c52fd964865dfe155104621babdd766e80'; + +CREATE VIEW "v_programs_by_month" AS + SELECT + CAST(date_trunc('month', p."created_ts") AS DATE) AS "createdAtMonth", + p."status" AS "status", + COUNT(p."id") AS "programs", + CAST(SUM(p."priceCents") FILTER (WHERE p."status" = 'PUBLISHED') AS BIGINT) AS "listValue" + FROM "programs" p + GROUP BY CAST(date_trunc('month', p."created_ts") AS DATE), p."status"; +COMMENT ON VIEW "v_programs_by_month" IS 'metaobjects:v1:sha256:8310fbae2f38c07296f2c22297c5e83ef2fbd12faaed5f90bd4211dbc8b45935'; + +CREATE VIEW "v_programs_by_week" AS + SELECT + CAST(date_trunc('week', p."created_ts") AS DATE) AS "createdAtWeek", + COUNT(p."id") AS "programs" + FROM "programs" p + WHERE p."status" = 'PUBLISHED' + GROUP BY CAST(date_trunc('week', p."created_ts") AS DATE); +COMMENT ON VIEW "v_programs_by_week" IS 'metaobjects:v1:sha256:656478aaf3e58443a6123f4f016bd61fbebfe2b0122175260155fb92d754e5ae'; + +CREATE VIEW "v_recent_programs" AS + SELECT + COUNT(p."id") AS "programs" + FROM "programs" p + WHERE p."created_ts" >= ((now() AT TIME ZONE 'UTC') - INTERVAL 'P30D'); +COMMENT ON VIEW "v_recent_programs" IS 'metaobjects:v1:sha256:9e675bb71fe4a3c74f06f5db65d7908e246a59c2a8fc062fc2c899948841eccc'; + +CREATE VIEW "v_asset_activity" AS + SELECT + date_trunc('hour', a."recordedAt", 'UTC') AS "recordedAtHour", + CAST(date_trunc('week', CAST(a."asOfDate" AS TIMESTAMP)) AS DATE) AS "asOfDateWeek", + COUNT(a."id") AS "assets" + FROM "assets" a + GROUP BY date_trunc('hour', a."recordedAt", 'UTC'), CAST(date_trunc('week', CAST(a."asOfDate" AS TIMESTAMP)) AS DATE); +COMMENT ON VIEW "v_asset_activity" IS 'metaobjects:v1:sha256:02b68a9ec0c67e11c6e505a47591c24e8dece737a46161549167c2cafece0a26'; diff --git a/fixtures/persistence-conformance/queries/report-grouped-measures.yaml b/fixtures/persistence-conformance/queries/report-grouped-measures.yaml new file mode 100644 index 000000000..bf2afad4a --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-grouped-measures.yaml @@ -0,0 +1,53 @@ +name: report-grouped-measures +description: | + ProgramMinutes is an object.report over Week: one row per (program, programTitle), + with every measure kind. programTitle is reached through the to-one reference + Week.fkProgram, so the view joins programs. Wire shapes follow the view's column + types: counts and the integral sum are BIGINT (string), min/max of an int are + INTEGER (number), avg and the ratio are NUMERIC (canonical decimal string). + + Program 1's slots are (1,30), (1,60), (1,90): three distinct tuples from four rows. + Its labels are 'Week 1', 'Week 2' and one NULL: two distinct, the null uncounted. + + A report has no primary key, so every query here is a list or a count; get-by-id + and every write are refused. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45); +queries: + - name: all-groups + op: list + entity: ProgramMinutes + sort: [{ field: program, dir: asc }] + expect: + - { program: "1", programTitle: "Foundations", weeks: "4", longWeeks: "3", labels: "2", slots: "3", + totalMinutes: "240", avgMinutes: "60", minMinutes: 30, maxMinutes: 90, longShare: "0.75" } + - { program: "2", programTitle: "Strength", weeks: "1", longWeeks: "0", labels: "1", slots: "1", + totalMinutes: "45", avgMinutes: "45", minMinutes: 45, maxMinutes: 45, longShare: "0" } + - name: filter-on-a-measure + op: list + entity: ProgramMinutes + filter: { weeks: { gte: 2 } } + sort: [{ field: program, dir: asc }] + expect: + - { program: "1", programTitle: "Foundations", weeks: "4", longWeeks: "3", labels: "2", slots: "3", + totalMinutes: "240", avgMinutes: "60", minMinutes: 30, maxMinutes: 90, longShare: "0.75" } + - name: sort-desc-on-a-measure + op: list + entity: ProgramMinutes + sort: [{ field: totalMinutes, dir: desc }] + limit: 1 + expect: + - { program: "1", programTitle: "Foundations", weeks: "4", longWeeks: "3", labels: "2", slots: "3", + totalMinutes: "240", avgMinutes: "60", minMinutes: 30, maxMinutes: 90, longShare: "0.75" } + - name: count-groups + op: count + entity: ProgramMinutes + expect: 2 diff --git a/fixtures/persistence-conformance/queries/report-relative-date.yaml b/fixtures/persistence-conformance/queries/report-relative-date.yaml new file mode 100644 index 000000000..6669beac2 --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-relative-date.yaml @@ -0,0 +1,18 @@ +name: report-relative-date +description: | + RecentPrograms carries the report-level filter createdAt >= now - P30D. A relative-date + value is evaluated by the database when the view is QUERIED (the view calls now()), not + when it is created, so a fixed seed date would rot as the clock advances. The seed is + therefore clock-relative: seed SQL is executed raw, so an expression is legal. One + program is three days old (inside the window), the other sixty (outside). + created_ts is a naive timestamp holding the UTC wall clock. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Fresh', 1000, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '3 days'), + (2, 'Stale', 2000, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '60 days'); +queries: + - name: only-the-recent-program-counts + op: list + entity: RecentPrograms + expect: + - { programs: "1" } diff --git a/fixtures/persistence-conformance/queries/report-time-grains.yaml b/fixtures/persistence-conformance/queries/report-time-grains.yaml new file mode 100644 index 000000000..bf13bcc4f --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-time-grains.yaml @@ -0,0 +1,38 @@ +name: report-time-grains +description: | + Time-grain dimensions over Program.createdAt (a naive timestamp, @column created_ts). + A grain bucket is a DATE: the first day of the bucket, wire form "YYYY-MM-DD". + + ProgramsByMonth groups by (createdAt month, status). The PUBLISHED group's listValue is + the sum of priceCents over the published programs: 4999 + 2500 + 300 = 7799. The DRAFT + and ARCHIVED groups have no published row, so the measure's `published` segment matches + nothing there and the sum is NULL (not 0). + + ProgramsByWeek buckets by ISO week (Monday start) and carries a report-level segment, + `published`, so the DRAFT and ARCHIVED programs are absent. Program 2 (Sunday 23:30) and + program 5 (Monday 00:00) are thirty minutes apart and land in different weeks: that is + the ISO Monday boundary. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00'), + (4, 'Legacy', 700, 'ARCHIVED', '2026-05-31T23:59:59'), + (5, 'Monday', 300, 'PUBLISHED', '2026-05-18T00:00:00'); +queries: + - name: by-month-and-status + op: list + entity: ProgramsByMonth + sort: [{ field: status, dir: asc }] + expect: + - { createdAtMonth: "2026-05-01", status: "ARCHIVED", programs: "1", listValue: null } + - { createdAtMonth: "2026-06-01", status: "DRAFT", programs: "1", listValue: null } + - { createdAtMonth: "2026-05-01", status: "PUBLISHED", programs: "3", listValue: "7799" } + - name: by-week-published-only + op: list + entity: ProgramsByWeek + sort: [{ field: createdAtWeek, dir: asc }] + expect: + - { createdAtWeek: "2026-04-27", programs: "1" } + - { createdAtWeek: "2026-05-11", programs: "1" } + - { createdAtWeek: "2026-05-18", programs: "1" } diff --git a/fixtures/persistence-conformance/queries/report-time-hour-and-date.yaml b/fixtures/persistence-conformance/queries/report-time-hour-and-date.yaml new file mode 100644 index 000000000..a9c7df9de --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-time-hour-and-date.yaml @@ -0,0 +1,35 @@ +name: report-time-hour-and-date +description: | + AssetActivity buckets Asset.recordedAt by hour and Asset.asOfDate by week. + + recordedAt is an instant (TIMESTAMPTZ); its hour bucket is itself an instant, so it is + bucketed in UTC and read back in the instant wire form "2026-05-04T03:00:00Z". + asOfDate is a field.date; its week bucket is a DATE, the Monday that starts the ISO week. + 2026-05-03 is a Sunday, so its week starts 2026-04-27; 2026-05-04 is a Monday. +seed-data: | + INSERT INTO "assets" + ("id","ownerId","externalId","payload","recordedAt","observedAt","asOfDate","atTime") + VALUES + ('11111111-1111-4111-8111-111111111111', + 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', + '22222222-2222-4222-8222-222222222222', + '{"k": 1}', + '2026-05-04T03:30:00Z', '2026-05-04T03:30:00', '2026-05-03', '03:30:00'), + ('33333333-3333-4333-8333-333333333333', + '44444444-4444-4444-8444-444444444444', + '55555555-5555-4555-8555-555555555555', + '{"k": 2}', + '2026-05-04T03:45:00Z', '2026-05-04T03:45:00', '2026-05-03', '03:45:00'), + ('66666666-6666-4666-8666-666666666666', + '77777777-7777-4777-8777-777777777777', + '88888888-8888-4888-8888-888888888888', + '{"k": 3}', + '2026-05-04T04:10:00Z', '2026-05-04T04:10:00', '2026-05-04', '04:10:00'); +queries: + - name: by-hour-and-week + op: list + entity: AssetActivity + sort: [{ field: recordedAtHour, dir: asc }] + expect: + - { recordedAtHour: "2026-05-04T03:00:00Z", asOfDateWeek: "2026-04-27", assets: "2" } + - { recordedAtHour: "2026-05-04T04:00:00Z", asOfDateWeek: "2026-05-04", assets: "1" } diff --git a/fixtures/persistence-conformance/queries/report-totals-empty.yaml b/fixtures/persistence-conformance/queries/report-totals-empty.yaml new file mode 100644 index 000000000..ec12d2b4f --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-totals-empty.yaml @@ -0,0 +1,14 @@ +name: report-totals-empty +description: | + FitnessTotals over an EMPTY weeks table. Three pins, each a place where an engine + could plausibly differ: + - a report with no dimensions returns exactly ONE row even over an empty table + (an aggregate without GROUP BY always yields one row); + - a count of nothing is 0, but a sum of nothing is NULL (not 0); + - a ratio whose denominator is zero is NULL (the view divides by NULLIF(den, 0)). +queries: + - name: one-row-over-an-empty-table + op: list + entity: FitnessTotals + expect: + - { weeks: "0", totalMinutes: null, longShare: null } diff --git a/fixtures/persistence-conformance/queries/report-totals.yaml b/fixtures/persistence-conformance/queries/report-totals.yaml new file mode 100644 index 000000000..12d79f81f --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-totals.yaml @@ -0,0 +1,21 @@ +name: report-totals +description: | + FitnessTotals is an object.report over Week with no dimensions: the whole table is + one group, so a list returns exactly one row. longShare is a ratio (NUMERIC): three + of the five weeks are long, 3/5 = 0.6. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45); +queries: + - name: one-row-for-the-whole-table + op: list + entity: FitnessTotals + expect: + - { weeks: "5", totalMinutes: "285", longShare: "0.6" } diff --git a/fixtures/persistence-conformance/report-shapes.json b/fixtures/persistence-conformance/report-shapes.json new file mode 100644 index 000000000..16dfe26aa --- /dev/null +++ b/fixtures/persistence-conformance/report-shapes.json @@ -0,0 +1,214 @@ +{ + "reports": [ + { + "report": "fitness::ProgramMinutes", + "from": "fitness::Week", + "view": "v_program_minutes", + "fields": [ + { + "name": "program", + "role": "dimension", + "subType": "long", + "required": true, + "typeSource": "fitness::Week.programId" + }, + { + "name": "programTitle", + "role": "dimension", + "subType": "string", + "required": false, + "typeSource": "fitness::Program.title" + }, + { + "name": "weeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "longWeeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "labels", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "slots", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "totalMinutes", + "role": "measure", + "subType": "long", + "required": false, + "typeSource": null + }, + { + "name": "avgMinutes", + "role": "measure", + "subType": "decimal", + "required": false, + "typeSource": null + }, + { + "name": "minMinutes", + "role": "measure", + "subType": "int", + "required": false, + "typeSource": "fitness::Week.durationMinutes" + }, + { + "name": "maxMinutes", + "role": "measure", + "subType": "int", + "required": false, + "typeSource": "fitness::Week.durationMinutes" + }, + { + "name": "longShare", + "role": "measure", + "subType": "decimal", + "required": false, + "typeSource": null + } + ] + }, + { + "report": "fitness::FitnessTotals", + "from": "fitness::Week", + "view": "v_fitness_totals", + "fields": [ + { + "name": "weeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "totalMinutes", + "role": "measure", + "subType": "long", + "required": false, + "typeSource": null + }, + { + "name": "longShare", + "role": "measure", + "subType": "decimal", + "required": false, + "typeSource": null + } + ] + }, + { + "report": "fitness::ProgramsByMonth", + "from": "fitness::Program", + "view": "v_programs_by_month", + "fields": [ + { + "name": "createdAtMonth", + "role": "dimension", + "subType": "date", + "required": true, + "typeSource": null + }, + { + "name": "status", + "role": "dimension", + "subType": "enum", + "required": true, + "typeSource": "fitness::Program.status" + }, + { + "name": "programs", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "listValue", + "role": "measure", + "subType": "currency", + "required": false, + "typeSource": "fitness::Program.priceCents" + } + ] + }, + { + "report": "fitness::ProgramsByWeek", + "from": "fitness::Program", + "view": "v_programs_by_week", + "fields": [ + { + "name": "createdAtWeek", + "role": "dimension", + "subType": "date", + "required": true, + "typeSource": null + }, + { + "name": "programs", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + } + ] + }, + { + "report": "fitness::RecentPrograms", + "from": "fitness::Program", + "view": "v_recent_programs", + "fields": [ + { + "name": "programs", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + } + ] + }, + { + "report": "fitness::AssetActivity", + "from": "fitness::Asset", + "view": "v_asset_activity", + "fields": [ + { + "name": "recordedAtHour", + "role": "dimension", + "subType": "timestamp", + "required": true, + "typeSource": "fitness::Asset.recordedAt" + }, + { + "name": "asOfDateWeek", + "role": "dimension", + "subType": "date", + "required": true, + "typeSource": null + }, + { + "name": "assets", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + } + ] + } + ] +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs index d756da04d..3b6927de7 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs @@ -146,6 +146,20 @@ public void Every_generated_file_compiles_with_zero_errors(string selection, boo var files = generators.SelectMany(g => g.Generate(ctx)).ToList(); Assert.True(files.Count > 0, $"selection '{selection}' generated no files at all"); + // FR-044 — the corpus's six view-backed reports each generate a keyless row class. + // Named, because a compile gate passes trivially over a file that was never emitted. + foreach (var report in new[] + { + "ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", + "RecentPrograms", "AssetActivity", + }) + { + Assert.True(files.Any(f => f.Path == report + ".g.cs"), $"no row class was generated for report {report}"); + Assert.False(files.Any(f => f.Path.StartsWith(report + "Names", StringComparison.Ordinal) + || f.Path.StartsWith(report + "FilterAllowlist", StringComparison.Ordinal)), + $"report {report} leaked into the names or filter-allowlist tier"); + } + if (isTemplateTier) { // A compile gate passes trivially on an empty emit, so name what this selection diff --git a/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs new file mode 100644 index 000000000..9b4902a04 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs @@ -0,0 +1,463 @@ +// ReportRowCodegenTests — the keyless EF Core row a view-backed `object.report` generates +// (FR-044 Plan 2), and the Table A cases that decide whether one is generated at all. +// +// ReportingInertTests holds the shared with/without corpus to "exactly the row and its +// mapping"; this file covers what that corpus does not reach: the `@unmanaged` and `@sql` +// arms, the kinds that stay inert, every Table B row's C# type and nullability, the +// naming strategy on the derived name, and a real compile. + +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using MetaObjects.Codegen; +using MetaObjects.Codegen.Generators; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +public class ReportRowCodegenTests +{ + private const string Namespace = "MetaObjects.ReportRows.Generated"; + + // One entity carrying a dimension or measure for every Table B row, and a to-one + // relationship for the `@via` case. `<>` is replaced per test. + private const string ModelTemplate = + """ + { "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Store", "children": [ + { "source.rdb": { "@table": "stores" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.string": { "name": "region", "@required": true, "@maxLength": 40 } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.long": { "name": "storeId", "@required": true, "@column": "store_fk" } }, + { "field.string": { "name": "channel" } }, + { "field.enum": { "name": "status", "@required": true, "@values": ["OPEN", "PAID"] } }, + { "field.int": { "name": "units", "@required": true } }, + { "field.currency": { "name": "amountCents", "@required": true, "@currency": "USD" } }, + { "field.decimal": { "name": "weight", "@precision": 12, "@scale": 3 } }, + { "field.double": { "name": "score" } }, + { "field.timestamp": { "name": "soldAt", "@required": true } }, + { "field.timestamp": { "name": "bookedAt", "@localTime": true } }, + { "field.date": { "name": "soldOn" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "storeRef", "@references": "Store", "@fields": ["storeId"] } }, + { "relationship.association": { "name": "store", "@objectRef": "Store", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "store", "@of": "Sale.storeId" } }, + { "dimension.attribute": { "name": "channel", "@of": "Sale.channel" } }, + { "dimension.attribute": { "name": "status", "@of": "Sale.status" } }, + { "dimension.attribute": { "name": "storeRegion", "@of": "Store.region", "@via": "Sale.store" } }, + { "dimension.time": { "name": "soldAt", "@of": "Sale.soldAt", "@grains": ["hour", "day", "month"] } }, + { "dimension.time": { "name": "bookedAt", "@of": "Sale.bookedAt", "@grains": ["hour"] } }, + { "dimension.time": { "name": "soldOn", "@of": "Sale.soldOn", "@grains": ["week"] } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } }, + { "measure.aggregate": { "name": "channels", "@agg": "count", "@distinct": true, "@of": "Sale.channel" } }, + { "measure.aggregate": { "name": "unitsSold", "@agg": "sum", "@of": "Sale.units" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "totalWeight", "@agg": "sum", "@of": "Sale.weight" } }, + { "measure.aggregate": { "name": "totalScore", "@agg": "sum", "@of": "Sale.score" } }, + { "measure.aggregate": { "name": "avgUnits", "@agg": "avg", "@of": "Sale.units" } }, + { "measure.aggregate": { "name": "avgScore", "@agg": "avg", "@of": "Sale.score" } }, + { "measure.aggregate": { "name": "minUnits", "@agg": "min", "@of": "Sale.units" } }, + { "measure.aggregate": { "name": "lastSoldAt", "@agg": "max", "@of": "Sale.soldAt" } }, + { "measure.aggregate": { "name": "maxWeight", "@agg": "max", "@of": "Sale.weight" } }, + { "measure.ratio": { "name": "unitsPerSale", "@numerator": "unitsSold", "@denominator": "sales" } } + ] } } + <> + ] } } + """; + + private const string AllDimensions = + "\"store\", \"channel\", \"status\", \"storeRegion\", \"soldAt:hour\", \"soldAt:day\", \"soldAt:month\", \"bookedAt:hour\", \"soldOn:week\""; + + private const string AllMeasures = + "\"sales\", \"channels\", \"unitsSold\", \"revenue\", \"totalWeight\", \"totalScore\", \"avgUnits\", \"avgScore\", \"minUnits\", \"lastSoldAt\", \"maxWeight\", \"unitsPerSale\""; + + private static string Report(string name, string sourceBody, string dims = "", string measures = "\"sales\"") + { + string children = sourceBody.Length == 0 + ? "" + : ", \"children\": [ { \"source.rdb\": { " + sourceBody + " } } ]"; + return ", { \"object.report\": { \"name\": \"" + name + "\", \"@from\": \"Sale\", " + + "\"@dimensions\": [" + dims + "], \"@measures\": [" + measures + "]" + children + " } }"; + } + + private static MetaRoot Load(params string[] reports) + { + var json = ModelTemplate.Replace("<>", string.Concat(reports)); + var result = new MetaDataLoader().Load([new InMemoryStringSource(json, id: "meta.shop.json")]); + Assert.True(result.Errors.Count == 0, + "model did not load:\n" + string.Join("\n", result.Errors.Select(e => $" {e.Code}: {e.Message}"))); + return result.Root; + } + + private static GenConfig Config(ColumnNamingStrategy strategy = ColumnNamingStrategy.Literal, bool includeNames = true) => new() + { + OutDir = "/unused", + Namespace = Namespace, + ColumnNamingStrategy = strategy, + IncludeNames = includeNames, + }; + + /// + /// The context builds: no report in the entity set. + /// + private static GenContext RunnerContext(MetaRoot root, GenConfig? config = null) => new() + { + Entities = root.Objects().Where(o => !o.IsReport()).ToList(), + Root = root, + Config = config ?? Config(), + }; + + private static Dictionary Emit(GenContext ctx, params IGenerator[] generators) => + generators.SelectMany(g => g.Generate(ctx)).ToDictionary(f => f.Path, f => f.Content, StringComparer.Ordinal); + + private static Dictionary EmitAll(GenContext ctx) => + Emit(ctx, new EntityGenerator(), new DbContextGenerator(), new NamesGenerator(), + new FilterAllowlistGenerator(), new RoutesGenerator()); + + // --------------------------------------------------------------------- + // Table A — which reports generate a row + // --------------------------------------------------------------------- + + public static TheoryData ViewBackedSources => new() + { + { "a managed derived view", "\"@kind\": \"view\", \"@view\": \"v_sales\"" }, + // Migrate never creates or drops it; the view exists all the same (the MySQL case). + { "an unmanaged view", "\"@kind\": \"view\", \"@view\": \"v_sales\", \"@unmanaged\": true" }, + // The author's body replaces the derived one; Table B still defines the columns. + { "a hand-written @sql view", "\"@kind\": \"view\", \"@view\": \"v_sales\", \"@sql\": \"SELECT COUNT(id) AS sales FROM sales\"" }, + // The legacy physical-name slot. + { "a view named by @table", "\"@kind\": \"view\", \"@table\": \"v_sales\"" }, + }; + + [Theory] + [MemberData(nameof(ViewBackedSources))] + public void A_view_backed_report_generates_its_row_and_mapping(string what, string source) + { + var files = EmitAll(RunnerContext(Load(Report("SalesTotal", source)))); + + Assert.True(files.ContainsKey("SalesTotal.g.cs"), $"{what}: no row class was generated"); + Assert.Contains("public class SalesTotal", files["SalesTotal.g.cs"]); + Assert.Contains(" public long Sales { get; set; }", files["SalesTotal.g.cs"]); + Assert.DoesNotContain("[Key]", files["SalesTotal.g.cs"]); + Assert.DoesNotContain("[Table(", files["SalesTotal.g.cs"]); + + var ctx = files["AppDbContext.g.cs"]; + Assert.Contains(" public DbSet SalesTotals { get; set; } = default!;", ctx); + Assert.Contains(" modelBuilder.Entity().HasNoKey().ToView(\"v_sales\");", ctx); + + // Nothing else: no names artifact, filter allowlist or routes for a report. + Assert.Equal( + ["SalesTotal.g.cs"], + files.Keys.Where(k => k.Contains("SalesTotal", StringComparison.Ordinal)).ToList()); + } + + public static TheoryData InertSources => new() + { + { "no source", "" }, + // The lowering skips these kinds, so no relation with the Table B columns is promised. + { "a materialized view", "\"@kind\": \"materializedView\", \"@materializedView\": \"mv_sales\"" }, + { "a stored procedure", "\"@kind\": \"storedProc\", \"@proc\": \"sp_sales\"" }, + { "a table function", "\"@kind\": \"tableFunction\", \"@function\": \"fn_sales\"" }, + }; + + [Theory] + [MemberData(nameof(InertSources))] + public void A_report_that_is_not_a_view_generates_nothing(string what, string source) + { + var with = EmitAll(RunnerContext(Load(Report("SalesTotal", source)))); + var without = EmitAll(RunnerContext(Load())); + + Assert.True(without.Keys.OrderBy(k => k).SequenceEqual(with.Keys.OrderBy(k => k)), $"{what}: the file set changed"); + foreach (var (path, content) in without) + Assert.True(content == with[path], $"{what}: {path} changed"); + } + + [Fact] + public void A_context_built_from_the_unfiltered_root_generates_the_same_files() + { + // Many callers (and most tests) pass `root.Objects()` as the entity set, reports + // included. The row must come out the same, and the raw report node must not leak + // into the names, allowlist or routes tiers as an empty projection. + var root = Load( + Report("SalesTotal", "\"@kind\": \"view\", \"@view\": \"v_sales\""), + Report("Sourceless", "")); + var unfiltered = new GenContext { Entities = root.Objects(), Root = root, Config = Config() }; + + var expected = EmitAll(RunnerContext(root)); + var actual = EmitAll(unfiltered); + Assert.Equal(expected.Keys.OrderBy(k => k).ToList(), actual.Keys.OrderBy(k => k).ToList()); + foreach (var (path, content) in expected) + Assert.True(content == actual[path], $"{path} differs for an unfiltered entity set"); + Assert.DoesNotContain(actual.Keys, k => k.Contains("Sourceless", StringComparison.Ordinal)); + } + + // --------------------------------------------------------------------- + // Table B — the C# type and nullability of every derived field + // --------------------------------------------------------------------- + + private static string RowOf(MetaRoot root, GenConfig? config = null) => + Emit(RunnerContext(root, config), new EntityGenerator())["SalesCube.g.cs"]; + + private static MetaRoot Cube() => + Load(Report("SalesCube", "\"@kind\": \"view\", \"@view\": \"v_sales_cube\"", AllDimensions, AllMeasures)); + + [Theory] + // Dimensions: the @of field's type; nullable unless it has no @via and @of is @required. + [InlineData("public long Store { get; set; }")] + [InlineData("public string? Channel { get; set; }")] + [InlineData("public SalesCubeStatus Status { get; set; }")] + [InlineData("public string? StoreRegion { get; set; }")] // @required at the source, reached by @via + [InlineData("public DateTimeOffset SoldAtHour { get; set; }")] // hour of an instant + [InlineData("public DateOnly SoldAtDay { get; set; }")] // day or coarser is a date + [InlineData("public DateOnly SoldAtMonth { get; set; }")] + [InlineData("public DateTime? BookedAtHour { get; set; }")] // hour of a @localTime timestamp + [InlineData("public DateOnly? SoldOnWeek { get; set; }")] + // Measures. + [InlineData("public long Sales { get; set; }")] // count is never null + [InlineData("public long Channels { get; set; }")] // count distinct + [InlineData("public long? UnitsSold { get; set; }")] // sum of int + [InlineData("public long? Revenue { get; set; }")] // sum of currency: minor units + [InlineData("public decimal? TotalWeight { get; set; }")] // sum of decimal + [InlineData("public double? TotalScore { get; set; }")] // sum of double + [InlineData("public decimal? AvgUnits { get; set; }")] // avg of int + [InlineData("public double? AvgScore { get; set; }")] // avg of double + [InlineData("public int? MinUnits { get; set; }")] // min keeps the type, loses @required + [InlineData("public DateTimeOffset? LastSoldAt { get; set; }")] // max of a required instant + [InlineData("public decimal? MaxWeight { get; set; }")] + [InlineData("public decimal? UnitsPerSale { get; set; }")] // ratio + public void A_derived_field_has_the_type_and_nullability_of_its_Table_B_row(string property) + { + Assert.Contains(" " + property + "\n", RowOf(Cube()).ReplaceLineEndings("\n")); + } + + [Fact] + public void Fields_are_emitted_in_Table_B_order() + { + var row = RowOf(Cube()); + var order = new[] + { + "Store", "Channel", "Status", "StoreRegion", "SoldAtHour", "SoldAtDay", "SoldAtMonth", + "BookedAtHour", "SoldOnWeek", "Sales", "Channels", "UnitsSold", "Revenue", "TotalWeight", + "TotalScore", "AvgUnits", "AvgScore", "MinUnits", "LastSoldAt", "MaxWeight", "UnitsPerSale", + }.Select(p => row.IndexOf($" {p} {{ get; set; }}", StringComparison.Ordinal)).ToList(); + Assert.DoesNotContain(-1, order); + Assert.Equal(order.OrderBy(i => i).ToList(), order); + } + + [Fact] + public void A_column_is_the_naming_strategy_on_the_derived_name_and_never_the_of_fields_column() + { + // `Sale.storeId` declares @column: store_fk. The report column is `store`. + var literal = RowOf(Cube()); + Assert.Contains("[Column(\"store\")]", literal); + Assert.Contains("[Column(\"soldAtHour\")]", literal); + Assert.DoesNotContain("store_fk", literal); + + var snake = RowOf(Cube(), Config(ColumnNamingStrategy.SnakeCase)); + Assert.Contains("[Column(\"sold_at_hour\")]", snake); + Assert.Contains("[Column(\"units_per_sale\")]", snake); + Assert.DoesNotContain("store_fk", snake); + } + + [Fact] + public void A_report_row_binds_by_literal_even_when_the_names_generator_runs() + { + // A report has no names artifact, so a reference to one would not compile. + var files = EmitAll(RunnerContext(Cube(), Config(includeNames: true))); + Assert.DoesNotContain(files.Keys, k => k.StartsWith("SalesCubeNames", StringComparison.Ordinal)); + Assert.DoesNotContain("SalesCubeNames", files["SalesCube.g.cs"]); + Assert.DoesNotContain("SalesCubeNames", files["AppDbContext.g.cs"]); + // An entity in the same run still binds through its artifact. + Assert.Contains("[Table(SaleNames.SourcePrimaryTable)]", files["Sale.g.cs"]); + } + + [Fact] + public void An_enum_dimension_declares_its_enum_and_reads_it_as_text() + { + var files = EmitAll(RunnerContext(Cube())); + Assert.Contains(" public enum SalesCubeStatus { OPEN, PAID }", files["SalesCube.g.cs"]); + Assert.Contains( + " modelBuilder.Entity().Property(x => x.Status).HasConversion();", + files["AppDbContext.g.cs"]); + } + + [Theory] + [InlineData(true)] + [InlineData(false)] + public void The_row_and_its_mapping_compile(bool includeNames) + { + var ctx = RunnerContext(Cube(), Config(includeNames: includeNames)); + var generators = new List { new EntityGenerator(), new DbContextGenerator() }; + if (includeNames) generators.Add(new NamesGenerator()); + var files = generators.SelectMany(g => g.Generate(ctx)).ToList(); + + var trees = files + .Select(f => CSharpSyntaxTree.ParseText(f.Content, new CSharpParseOptions(LanguageVersion.CSharp12), path: f.Path)) + .ToList(); + var comp = CSharpCompilation.Create( + "report_rows_" + Guid.NewGuid().ToString("N"), trees, + DbContextCompileTests.BuildReferences(), + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + var errors = comp.GetDiagnostics() + .Where(d => d.Severity == DiagnosticSeverity.Error) + .Select(d => $"{d.Location.GetLineSpan().Path}: {d.Id}: {d.GetMessage()}") + .ToList(); + Assert.True(errors.Count == 0, string.Join("\n", errors)); + } + + // --------------------------------------------------------------------- + // The runner + // --------------------------------------------------------------------- + + [Fact] + public void A_report_whose_DbSet_name_collides_with_an_entitys_is_refused() + { + // Report `Sales` and entity `Sale` both claim the DbSet property `Sales`. + var root = Load(Report("Sales", "\"@kind\": \"view\", \"@view\": \"v_sales\"")); + var outDir = Path.Combine(Path.GetTempPath(), "report-rows-" + Guid.NewGuid().ToString("N")); + try + { + var ex = Assert.Throws(() => + CodegenRunner.Run(Config() with { OutDir = outDir }, root, [new EntityGenerator(), new DbContextGenerator()])); + Assert.Contains("\"Sale\" and \"Sales\" both pluralize", ex.Message); + } + finally + { + if (Directory.Exists(outDir)) Directory.Delete(outDir, recursive: true); + } + } + + public static TheoryData RowNameCollisions => new() + { + // report name, @dimensions, @measures, the phrase the message must carry + { "Sales", "", "\"sales\"", "its measure \"sales\"" }, + { "Channel", "\"channel\"", "\"sales\"", "its dimension \"channel\"" }, + // A time dimension collides through its DERIVED name; the message names the item. + { "SoldAtDay", "\"soldAt:day\"", "\"sales\"", "its dimension \"soldAt\"" }, + }; + + [Theory] + [MemberData(nameof(RowNameCollisions))] + public void A_report_whose_derived_field_is_named_after_its_row_class_is_refused( + string report, string dims, string measures, string names) + { + // CS0542: a member cannot be named after its enclosing type. `gen` must say so + // rather than exit 0 and leave it to the adopter's build. + var root = Load(Report(report, "\"@kind\": \"view\", \"@view\": \"v_x\"", dims, measures)); + foreach (var generator in new IGenerator[] { new EntityGenerator(), new DbContextGenerator() }) + { + var ex = Assert.Throws(() => generator.Generate(RunnerContext(root)).ToList()); + Assert.Contains($"report \"{report}\"", ex.Message); + Assert.Contains(names, ex.Message); + Assert.Contains("rename the report or the", ex.Message); + } + } + + [Fact] + public void A_sourceless_report_whose_item_is_named_after_it_is_not_refused() + { + // It generates no row, so there is no class for the name to collide with. + var with = EmitAll(RunnerContext(Load(Report("Channel", "", "\"channel\"", "\"sales\"")))); + var without = EmitAll(RunnerContext(Load())); + Assert.Equal(without.Keys.OrderBy(k => k).ToList(), with.Keys.OrderBy(k => k).ToList()); + foreach (var (path, content) in without) + Assert.True(content == with[path], $"{path} changed"); + } + + [Fact] + public void A_sourceless_report_with_a_colliding_name_is_not_refused() + { + // It generates nothing, so it claims no name. + var root = Load(Report("Sales", "")); + var outDir = Path.Combine(Path.GetTempPath(), "report-rows-" + Guid.NewGuid().ToString("N")); + try + { + var result = CodegenRunner.Run(Config() with { OutDir = outDir }, root, [new EntityGenerator(), new DbContextGenerator()]); + Assert.DoesNotContain(result.Files, f => f.Path == "Sales.g.cs"); + } + finally + { + if (Directory.Exists(outDir)) Directory.Delete(outDir, recursive: true); + } + } + + // --------------------------------------------------------------------- + // What generates nothing, and what is refused + // --------------------------------------------------------------------- + + private const string ObjectDimensionModel = + """ + { "metadata.root": { "package": "acme::shop", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "city" } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "destination", "@of": "Sale.shipTo" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "SalesByDestination", "@from": "Sale", + "@dimensions": ["destination"], "@measures": ["sales"]<> } } + ] } } + """; + + private static MetaRoot LoadObjectDimension(bool viewBacked) + { + string source = viewBacked + ? ", \"children\": [ { \"source.rdb\": { \"@kind\": \"view\", \"@view\": \"v_by_destination\" } } ]" + : ""; + var result = new MetaDataLoader().Load( + [new InMemoryStringSource(ObjectDimensionModel.Replace("<>", source), id: "meta.shop.json")]); + Assert.True(result.Errors.Count == 0, + "model did not load:\n" + string.Join("\n", result.Errors.Select(e => $" {e.Code}: {e.Message}"))); + return result.Root; + } + + [Fact] + public void A_dimension_over_a_field_object_is_refused_naming_the_report_and_the_dimension() + { + // The loader accepts it. Left alone, the row class silently has no property for + // the dimension, so the report would read without the column it groups by. + var root = LoadObjectDimension(viewBacked: true); + foreach (var generator in new IGenerator[] { new EntityGenerator(), new DbContextGenerator() }) + { + var ex = Assert.Throws(() => generator.Generate(RunnerContext(root)).ToList()); + Assert.Equal( + "report \"SalesByDestination\": its dimension \"destination\" reads \"acme::shop::Sale.shipTo\", " + + "a field.object. A report over a field.object is not supported; group by a scalar field.", + ex.Message); + } + } + + [Fact] + public void A_sourceless_report_over_a_field_object_generates_nothing_and_is_not_refused() + { + var files = EmitAll(RunnerContext(LoadObjectDimension(viewBacked: false))); + Assert.DoesNotContain(files.Keys, k => k.Contains("SalesByDestination", StringComparison.Ordinal)); + } + + [Fact] + public void An_abstract_view_backed_report_generates_nothing() + { + // An abstract object gets no class in this port, report or not. The TypeScript, + // Java and Python runtimes still read the view (docs/features/reporting.md, Known limits). + string report = Report("SalesTotal", "\"@kind\": \"view\", \"@view\": \"v_sales\"") + .Replace("\"name\": \"SalesTotal\",", "\"name\": \"SalesTotal\", \"abstract\": true,"); + Assert.Contains("\"abstract\": true", report); + var with = EmitAll(RunnerContext(Load(report))); + var without = EmitAll(RunnerContext(Load())); + + Assert.Equal(without.Keys.OrderBy(k => k).ToList(), with.Keys.OrderBy(k => k).ToList()); + foreach (var (path, content) in without) + Assert.True(content == with[path], $"{path} changed"); + } +} diff --git a/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs index 2eee34142..b1872250a 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs @@ -1,17 +1,21 @@ -// FR-044 Plan 1 — the reporting vocabulary is INERT in every C# generator. +// FR-044 — what the reporting vocabulary generates in C#, and what it does not. // -// Plan 1 registers `dimension.*`, `measure.*`, `segment.*` and `object.report` and -// validates them at load, but gives none of them output: a report's lowering lands in -// Plan 2/3. Until then a model that USES the vocabulary must generate exactly what the -// same model without it generates, byte for byte, through every registered generator. +// `dimension.*`, `measure.*` and `segment.*` generate nothing. A report generates nothing +// either, with ONE exception that landed in Plan 2: a report that declares a read-only +// `source.rdb @kind: view` is a database view, and C# reads it through a generated keyless +// row class and its `HasNoKey().ToView(...)` mapping plus a DbSet. So: +// +// - a SOURCELESS report (`ProgramEngagement`, `DailyRevenue`) stays inert in every +// registered generator and in the api docs; +// - the VIEW-BACKED report (`StoreTotals`) adds exactly one file (its row class) and +// exactly its lines in AppDbContext.g.cs, and nothing in the routes, filter-allowlist, +// names or api-docs tiers (Plan 3). // // The model pair is fixtures/codegen-noop/reporting/{with,without}, shared with the other -// four ports' copies of this test. `with/` carries a report that declares a read-only -// `source.rdb @kind: view` (R5 allows one) — the case that used to leak here as an empty -// entity class, a filter allowlist, a GET-only route and a keyless DbSet with ToView. +// four ports' copies of this test. // // Runs through CodegenRunner.Run — the path `dotnet meta gen` takes — not a hand-built -// GenContext, because the skip lives at the runner's entity-set choke point. +// GenContext, because the report skip lives at the runner's entity-set choke point. using MetaObjects.Codegen; using MetaObjects.Codegen.ApiDocs; @@ -95,6 +99,89 @@ public void The_with_model_really_carries_the_vocabulary() Assert.DoesNotContain(Load("without").Objects(), o => o.IsReport()); } + // The two generators that lower a view-backed report (ReportRows). Every other + // registered generator must not notice a report at all. + private const string EntityGeneratorName = "entity"; + private const string DbContextGeneratorName = "db-context"; + private const string RowFile = "StoreTotals.g.cs"; + private const string DbContextFile = "AppDbContext.g.cs"; + + // Snake-case is the naming strategy of this suite's GenConfig, so the columns prove the + // strategy is applied to the DERIVED field name. `revenue` is a sum of a currency: + // integer minor units, nullable because a sum of nothing is null. A count is never null. + private const string ExpectedRow = + """ + // + // Generated by MetaObjects entity-generator. Do not edit by hand. + #nullable enable + using System; + using System.Collections.Generic; + using System.ComponentModel.DataAnnotations; + using System.ComponentModel.DataAnnotations.Schema; + + namespace MetaObjects.ReportingInert.Generated; + + public class StoreTotals + { + [Column("purchases")] + public long Purchases { get; set; } + [Column("buyers")] + public long Buyers { get; set; } + [Column("revenue")] + public long? Revenue { get; set; } + } + + """; + + private static readonly string[] ExpectedDbContextLines = + [ + " public DbSet StoreTotals { get; set; } = default!;", + " modelBuilder.Entity().HasNoKey().ToView(\"v_store_totals\");", + ]; + + /// The lines of that lacks, + /// asserting that is otherwise an in-order subsequence of it + /// (so nothing was removed, changed or reordered). + private static List AddedLines(string expected, string actual) + { + var want = expected.Split('\n'); + var added = new List(); + int i = 0; + foreach (var line in actual.Split('\n')) + { + if (i < want.Length && want[i] == line) i++; + else added.Add(line); + } + Assert.True(i == want.Length, "AppDbContext.g.cs lost or changed a line once a report was declared"); + return added; + } + + /// + /// is plus exactly what the + /// view-backed report adds through the generators in . + /// + private static void AssertSameExceptTheReportRow( + SortedDictionary expected, SortedDictionary actual, + IReadOnlyCollection selection) + { + var allowedNew = selection.Contains(EntityGeneratorName) ? new[] { RowFile } : []; + Assert.Equal( + expected.Keys.Concat(allowedNew).OrderBy(k => k, StringComparer.Ordinal).ToList(), + actual.Keys.ToList()); + if (allowedNew.Length > 0) + Assert.Equal(ExpectedRow.ReplaceLineEndings("\n"), actual[RowFile].ReplaceLineEndings("\n")); + + foreach (var (path, content) in expected) + { + if (path == DbContextFile && selection.Contains(DbContextGeneratorName)) + { + Assert.Equal(ExpectedDbContextLines, AddedLines(content, actual[path])); + continue; + } + Assert.True(content == actual[path], $"{path} differs once reporting nodes are declared"); + } + } + [Theory] [MemberData(nameof(GeneratorNames))] public void Generator_emits_the_same_files_with_and_without_reporting_nodes(string name) @@ -102,7 +189,8 @@ public void Generator_emits_the_same_files_with_and_without_reporting_nodes(stri var entry = GeneratorRegistry.Entries[name]; var expected = Emit(Load("without"), [Build(entry)]); var actual = Emit(Load("with"), [Build(entry)]); - AssertSame(expected, actual); + // For every generator but `entity` and `db-context` this is plain equality. + AssertSameExceptTheReportRow(expected, actual, [name]); } [Fact] @@ -122,20 +210,43 @@ public void Exactly_these_generators_cannot_run_from_a_bare_model() [Fact] public void Every_runnable_generator_in_one_run_emits_the_same_files() { - var runnable = GeneratorRegistry.Entries.Values - .Where(e => !Emit(Load("without"), [Build(e)]).ContainsKey("")) + var runnable = GeneratorRegistry.Entries + .Where(e => !Emit(Load("without"), [Build(e.Value)]).ContainsKey("")) .ToList(); - var expected = Emit(Load("without"), runnable.Select(Build).ToList()); - var actual = Emit(Load("with"), runnable.Select(Build).ToList()); + var generators = runnable.Select(e => Build(e.Value)).ToList(); + var expected = Emit(Load("without"), generators); + var actual = Emit(Load("with"), runnable.Select(e => Build(e.Value)).ToList()); Assert.False(expected.ContainsKey(""), expected.GetValueOrDefault("")); + Assert.False(actual.ContainsKey(""), actual.GetValueOrDefault("")); Assert.True(expected.Count > 10, $"only {expected.Count} files — the suite barely ran"); - AssertSame(expected, actual); + AssertSameExceptTheReportRow(expected, actual, runnable.Select(e => e.Key).ToList()); + } + + [Fact] + public void A_report_reaches_no_tier_but_its_row_and_its_DbContext_mapping() + { + var files = Emit(Load("with"), GeneratorRegistry.Entries.Values.Select(Build).ToList()); + Assert.False(files.ContainsKey(""), files.GetValueOrDefault("")); + + // The view-backed report: one file, named for it; no routes, allowlist or names. + Assert.Equal([RowFile], files.Keys.Where(k => k.Contains("StoreTotals", StringComparison.Ordinal)).ToList()); + // A sourceless report: no file at all, and no mention in any file. + foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue" }) + { + Assert.DoesNotContain(files.Keys, k => k.Contains(sourceless, StringComparison.Ordinal)); + Assert.DoesNotContain(files.Values, c => c.Contains(sourceless, StringComparison.Ordinal)); + } + // Outside its row and the DbContext, no generated file mentions the report. + var mentions = files.Where(f => f.Value.Contains("StoreTotals", StringComparison.Ordinal)) + .Select(f => f.Key).OrderBy(k => k, StringComparer.Ordinal).ToList(); + Assert.Equal([DbContextFile, RowFile], mentions); } /// /// The api docs surface (`dotnet meta docs`): every unit page, the index and the agent /// page, rendered exactly as DocsCommand renders them. A report has no generated API to - /// document, and its derived fields do not exist until its lowering lands. + /// document (its routes are Plan 3), so the docs carry nothing for any report, + /// view-backed or not. /// private static SortedDictionary ApiDocs(MetaRoot root) { diff --git a/server/csharp/MetaObjects.Codegen/CSharpNaming.cs b/server/csharp/MetaObjects.Codegen/CSharpNaming.cs index c4b41293b..49ac6494d 100644 --- a/server/csharp/MetaObjects.Codegen/CSharpNaming.cs +++ b/server/csharp/MetaObjects.Codegen/CSharpNaming.cs @@ -509,6 +509,12 @@ public static bool HasPrimarySource(MetaObject obj) => /// public static ObjectNames? ResolveObjectNames(MetaObject obj, ColumnNamingStrategy strategy = ColumnNamingStrategy.Literal) { + // FR-044 — a report resolves no names artifact, so NamesGenerator emits none and + // the report's generated row (ReportRows) spells its view and columns as literals + // through the same fallback a sourceless object takes. One gate, here, so the + // artifact and every reference to it cannot disagree about whether it exists. + if (obj.IsReport()) return null; + // SourceResolution.PrimaryRdbSource, not a scan of our own: ADR-0039's RESOLVING // source accessor (an inherited primary must be seen, or an entity extending an // abstract base with its own primary source would wrongly read as unpersisted), diff --git a/server/csharp/MetaObjects.Codegen/CodegenRunner.cs b/server/csharp/MetaObjects.Codegen/CodegenRunner.cs index 794a35b4d..fe87ec940 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenRunner.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenRunner.cs @@ -29,10 +29,15 @@ public static RunResult Run(GenConfig config, MetaRoot root, IReadOnlyList(); var ctx = new GenContext { - // FR-044 Plan 1: object.report has no output until its lowering lands (Plan 2/3). - // Dropped here, at the entity set every generator reads, and not per generator: - // a report may declare a read-only `source.rdb @kind: view` (R5), which would - // otherwise pass every source-keyed gate and emit an empty projection tier. + // FR-044: an object.report is never in the entity set. Dropped here, at the set + // every generator reads, and not per generator: a report may declare a read-only + // `source.rdb @kind: view` (R5), which would otherwise pass every source-keyed + // gate and emit an empty projection tier (routes, allowlist, names). + // + // A view-backed report DOES generate its keyless row and DbContext mapping + // (Plan 2). The two generators that emit those ask for the report's row model + // themselves, through ReportRows, so every other generator stays report-free + // without having to know what a report is. Entities = root.Objects().Where(o => !o.IsReport()).ToList(), Root = root, Config = config, @@ -41,7 +46,9 @@ public static RunResult Run(GenConfig config, MetaRoot root, IReadOnlyList(); diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 72a9d4f07..45add1e90 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -72,7 +72,10 @@ public virtual IEnumerable Generate(GenContext ctx) // FR-017 TPH: a concrete subtype shares the base's single table — it gets NO // DbSet and no per-subtype model config; the hierarchy is reached via the base // DbSet (`.OfType()`). Filter subtypes out of the emitted set entirely. - var objects = ctx.Entities + // FR-044 — a view-backed report joins the set as its ROW MODEL, which the + // read-only-projection arm below maps as `HasNoKey().ToView(...)`; every other + // report node is dropped. See ReportRows. + var objects = ReportRows.WithReportRows(ctx) .Where(o => AppliesTo(o, ctx.Root)) .OrderBy(o => o.Name, StringComparer.Ordinal) .ToList(); @@ -291,7 +294,7 @@ protected virtual void EmitUsings(StringBuilder sb, bool needsMetadataUsing, Gen // references EVERY entity by short name (DbSet, modelBuilder.Entity()), // so it needs a `using` for each distinct namespace the entities resolve to. var dbCtxNs = ctx.Config.Namespace; - var refNamespaces = ctx.Entities + var refNamespaces = ReportRows.WithReportRows(ctx) .Where(o => o.IsEntity() || o.DbView is not null) .Select(o => PackageBindingResolver.Resolve(ctx.Config, PackageBindingResolver.EffectivePackage(o), o.Name)) .Where(ns => !string.IsNullOrEmpty(ns) && ns != dbCtxNs) diff --git a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs index f11fdfe6d..e67be7609 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs @@ -44,7 +44,10 @@ public class EntityGenerator : IGenerator public virtual IEnumerable Generate(GenContext ctx) { - var candidates = ctx.Entities + // FR-044 — a view-backed report joins the set as its ROW MODEL (a keyless, + // projection-shaped object carrying its derived fields); every other report node + // is dropped. See ReportRows. + var candidates = ReportRows.WithReportRows(ctx) .Where(o => o.IsEntity() || o.DbView is not null) .OrderBy(o => o.Name, StringComparer.Ordinal) .ToList(); diff --git a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs index 74aa2f41e..5893e4b88 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs @@ -53,9 +53,13 @@ public class FilterAllowlistGenerator : PerEntityGenerator /// /// public static bool AppliesTo(MetaObject entity) => - ((entity.IsEntity() || entity.DbView is not null) - && InstanceArtifacts.EmitsInstanceArtifacts(entity)) - || InstanceArtifacts.IsSourcelessEntity(entity); + // FR-044 — a report has no filter allowlist (it has no routes to name one). Stated + // here as well as at CodegenRunner's entity set, for the same reason as + // RoutesGenerator.AppliesTo. + !entity.IsReport() + && (((entity.IsEntity() || entity.DbView is not null) + && InstanceArtifacts.EmitsInstanceArtifacts(entity)) + || InstanceArtifacts.IsSourcelessEntity(entity)); protected override EmittedFile GenerateOne(MetaObject entity, GenContext ctx) { diff --git a/server/csharp/MetaObjects.Codegen/Generators/NamesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/NamesGenerator.cs index 4cd0579be..2cd250ff3 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/NamesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/NamesGenerator.cs @@ -81,7 +81,10 @@ public override IEnumerable Generate(GenContext ctx) // ctx.Config.ColumnNamingStrategy. The divergence refusal is not this generator's to // own and never was — it lives in MetaObjects.Meta.SourceResolution, which every // caller that resolves a physical name goes through, codegen and runtime alike. - public override bool Filter(MetaObject entity) => CSharpNaming.HasPrimarySource(entity); + // FR-044 — a report has no names artifact (ResolveObjectNames answers null for one): + // its generated row binds its view and columns by literal. + public override bool Filter(MetaObject entity) => + !entity.IsReport() && CSharpNaming.HasPrimarySource(entity); protected override EmittedFile GenerateOne(MetaObject entity, GenContext ctx) => Render(entity, ctx, fragment: false) diff --git a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs index 502731ec7..1632e467a 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/RoutesGenerator.cs @@ -47,7 +47,8 @@ public class RoutesGenerator : PerEntityGenerator private const string HelperRuntimeNamespace = "MetaObjects.Codegen.Runtime"; public override bool Filter(MetaObject entity) => - (entity.IsEntity() || entity.DbView is not null) && InstanceArtifacts.EmitsInstanceArtifacts(entity); + !entity.IsReport() // FR-044: a report has no routes (see AppliesTo) + && (entity.IsEntity() || entity.DbView is not null) && InstanceArtifacts.EmitsInstanceArtifacts(entity); /// /// True iff this entity gets a generated routes file: it passes @@ -56,7 +57,11 @@ public override bool Filter(MetaObject entity) => /// loop AND the api-docs builder (so docs never claim REST a routes-off entity lacks). /// public static bool AppliesTo(MetaObject entity, MetaRoot root) => - (entity.IsEntity() || entity.DbView is not null) + // FR-044 — a report has no routes. CodegenRunner already keeps reports out of the + // entity set; this holds for a caller that builds its context from the unfiltered + // root, where a view-backed report would otherwise pass as a projection. + !entity.IsReport() + && (entity.IsEntity() || entity.DbView is not null) && InstanceArtifacts.EmitsInstanceArtifacts(entity) && !TphPlanBuilder.IsTphSubtype(entity, root); diff --git a/server/csharp/MetaObjects.Codegen/ReportRows.cs b/server/csharp/MetaObjects.Codegen/ReportRows.cs new file mode 100644 index 000000000..2de5a582c --- /dev/null +++ b/server/csharp/MetaObjects.Codegen/ReportRows.cs @@ -0,0 +1,202 @@ +// report-rows — how a view-backed `object.report` reaches the EF Core generators (FR-044). +// +// WHAT IS GENERATED +// +// A report that declares a read-only `source.rdb @kind: view` is a database view +// (contract Table A). C# has no metadata-driven runtime, so reading that view means +// generating its row: a keyless entity class (EntityGenerator) and its +// `HasNoKey().ToView(...)` mapping plus a DbSet (DbContextGenerator). Nothing else is +// generated for a report: no routes, filter allowlist, names artifact or api docs. +// +// The view's existence is what matters, not who creates it. `@unmanaged: true` (migrate +// never creates it) and `@sql` (the author wrote the body) both still name a view with +// the Table B columns, so both still get a row. A report with no source stays inert, and +// so does one whose read source is a materialized view, a stored procedure or a table +// function: the lowering skips those kinds, so no relation with the Table B columns is +// promised to exist. An ABSTRACT report generates nothing either, view or not: an abstract +// object gets no class in this port. And a view-backed report with a derived field over a +// `field.object` is refused by name (RefuseObjectField): a keyless row cannot own it. +// +// WHY A SYNTHESIZED OBJECT, NOT A REPORT BRANCH IN EACH GENERATOR +// +// A report declares no fields; its read shape is derived (ReportShapes, Table B). The EF +// generators read an object's fields through `Fields()` in a dozen places (members, enum +// declarations, usings, the enum and jsonb conversions in the DbContext). A view-backed +// report already satisfies their projection predicates (`IsReadOnlyProjection()`, +// `DbView`), so it is handed to them as a ROW MODEL: a detached object with one real +// `field.*` child per derived field and a copy of the report's read source. They then +// emit it exactly as they emit a keyless read-only projection. +// +// The row model is never added to the root and nothing in the loaded tree is mutated to +// build it (the source is copied, not re-parented). It keeps the report's name, package +// and `object.report` subtype, so `IsReport()` still identifies it. +// +// Mirrors server/typescript/packages/metadata/src/core/reporting/report-read-model.ts. + +using MetaObjects.Core.Reporting; +using MetaObjects.Meta; +using static MetaObjects.Core.Field.FieldConstants; +using static MetaObjects.Persistence.Db.DbConstants; +using static MetaObjects.Persistence.Source.SourceConstants; +using static MetaObjects.Shared.BaseTypes; + +namespace MetaObjects.Codegen; + +/// Row models for view-backed reports, and the object set the EF generators iterate. +public static class ReportRows +{ + /// + /// Table B: the type-shaping attrs a derived field carries from its type source. + /// @dbColumnType and isArray are handled separately. Nothing else is + /// carried: no @column, @required, @default, validators or views. + /// + private static readonly string[] CarriedAttrs = + [ + FIELD_ATTR_CURRENCY, + FIELD_ATTR_VALUES, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_PRECISION, + FIELD_ATTR_SCALE, + FIELD_ATTR_LOCAL_TIME, + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_STORAGE, + ]; + + /// + /// True iff is a report whose read source is a view, the one + /// shape that generates a row (see the file header for the other kinds). + /// + public static bool IsViewBacked(MetaObject obj) => + obj.IsReport() && !obj.IsAbstract + && ReportShapes.ReadSource(obj)?.EffectiveKind == SOURCE_KIND_VIEW; + + /// + /// The row model of a view-backed report: one field per Table B row, in Table B + /// order, and a copy of the report's read source. Frozen. Throws what + /// throws when a reference does not resolve. + /// + public static MetaObject RowModel(MetaObject report, MetaRoot root) + { + var source = ReportShapes.ReadSource(report) + ?? throw new InvalidOperationException($"report '{report.Name}' declares no read-only source."); + var shape = ReportShapes.Of(report, root); + RefuseFieldNamedAfterTheRow(report, shape); + RefuseObjectField(report, shape); + + var model = new MetaObject(new TypeId(report.Type, report.SubType), report.Name); + if (report.Package is { } pkg) model.SetPackage(pkg); + // The file-default package has no getter; the effective package is what it resolves to. + string effectivePkg = NamingRefs.EffectivePackage(report); + if (effectivePkg.Length > 0) model.SetFileDefaultPackage(effectivePkg); + model.SetSource(report.Source); + + foreach (var f in shape.Fields) model.AddChild(DerivedField(f)); + model.AddChild(CopySource(source)); + model.Freeze(); + return model; + } + + /// + /// Refuse a report one of whose derived fields would become a property with the row + /// class's own name. C# forbids a member named after its enclosing type (CS0542), and + /// the loader relates a report's name to none of its item names, so a report named for + /// what it measures (Revenue with a measure revenue) loads clean and would + /// otherwise generate a file that does not compile. Reached only for a report that + /// generates a row; a report that generates nothing claims no name. + /// + private static void RefuseFieldNamedAfterTheRow(MetaObject report, ReportShape shape) + { + string className = CSharpNaming.Pascal(report.Name); + foreach (var f in shape.Fields) + { + if (CSharpNaming.Pascal(f.Name) != className) continue; + string role = ReportShapes.RoleName(f.Role); + string item = f.Role == ReportFieldRole.Dimension ? f.Dimension!.Name : f.Measure!.Name; + throw new InvalidOperationException( + $"report \"{report.Name}\" and its {role} \"{item}\" both generate the C# name " + + $"\"{className}\" (the row class, and the property for derived field \"{f.Name}\"), " + + $"and a C# member cannot be named after its enclosing type — rename the report " + + $"or the {role}."); + } + } + + /// + /// Refuse a report with a derived field typed by a field.object (a dimension over + /// an embedded value object, say). The loader accepts it, but a keyless row cannot own + /// the value object: the entity generator would emit the row with no property for the + /// field at all, and the report would read without the column it groups by. Reached + /// only for a report that generates a row. The Java read model and the Kotlin table + /// generator refuse the same report with the same sentence. + /// + private static void RefuseObjectField(MetaObject report, ReportShape shape) + { + foreach (var f in shape.Fields) + { + if (f.TypeSource is not { } src) continue; + // ADR-0039: resolving — an @objectRef the @of field inherits counts. + if (src.SubType != FIELD_SUBTYPE_OBJECT && src.Attr(FIELD_ATTR_OBJECT_REF) is null) continue; + string role = ReportShapes.RoleName(f.Role); + string item = f.Role == ReportFieldRole.Dimension ? f.Dimension!.Name : f.Measure!.Name; + string owner = src.Parent?.ResolutionKey() ?? ""; + throw new InvalidOperationException( + $"report \"{report.Name}\": its {role} \"{item}\" reads \"{owner}.{src.Name}\", " + + $"a field.{src.SubType}. A report over a field.object is not supported; group by a scalar field."); + } + } + + private static MetaField DerivedField(ReportField f) + { + var field = new MetaField(new TypeId(TYPE_FIELD, f.SubType), f.Name); + // From the derived shape, never from the type source: a `min` of a required column + // is still nullable, and a dimension reached by `@via` is nullable. + field.SetAttr(FIELD_ATTR_REQUIRED, f.Required); + if (f.TypeSource is { } src) + { + // ADR-0039: resolving, so a value the `@of` field inherits through extends is carried. + foreach (string name in CarriedAttrs) + if (src.Attr(name) is { } value) field.SetAttr(name, value); + // ADR-0039: own — `@dbColumnType` is the one deliberately own-only attr (a + // physical column-type override is never inherited), so the derived field + // carries exactly what the `@of` field itself declares. + if (src.OwnAttr(FIELD_ATTR_DB_COLUMN_TYPE) is { } dbColumnType) + field.SetAttr(FIELD_ATTR_DB_COLUMN_TYPE, dbColumnType); + // `isArray` is a native flag, not an attr; ResolvedIsArray() is its resolving read. + if (src.ResolvedIsArray()) field.SetIsArray(true); + } + return field; + } + + /// + /// A detached copy of a source node: same type, name and effective attrs, pinned to + /// @role: primary so (which considers primary + /// sources only) names it. + /// + private static MetaSource CopySource(MetaSource source) + { + var copy = new MetaSource(new TypeId(source.Type, source.SubType), source.Name); + // ADR-0039: resolving — the copy carries the source's effective configuration. + foreach (var (name, value) in source.Attrs()) copy.SetAttr(name, value); + copy.SetAttr(SOURCE_ATTR_ROLE, SOURCE_ROLE_PRIMARY); + return copy; + } + + /// The row model of every view-backed report in the model, in declaration order. + public static IReadOnlyList For(MetaRoot root) => + root.Objects().Where(IsViewBacked).Select(r => RowModel(r, root)).ToList(); + + /// + /// The objects a row-emitting generator iterates: the run's entity set with every + /// report node removed, then the row model of each view-backed report. + /// + /// The row models come from the root, not from : + /// keeps every report out of that set, which is what + /// keeps a report inert in every generator that does not ask for it here. A caller + /// that builds a context from the unfiltered root gets the same answer, because a + /// raw report node in the entity set is dropped rather than emitted as a class with + /// no members. + /// + /// + public static IReadOnlyList WithReportRows(GenContext ctx) => + [.. ctx.Entities.Where(o => !o.IsReport()), .. For(ctx.Root)]; +} diff --git a/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs b/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs new file mode 100644 index 000000000..f2184620d --- /dev/null +++ b/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs @@ -0,0 +1,239 @@ +// ReportShapeTests — the C# derivation of a report's read shape (FR-044, Table B) +// byte-matches the committed, TypeScript-produced artifact +// fixtures/persistence-conformance/report-shapes.json. +// +// Container-free: pure metadata in, JSON out. The same artifact gates the Java, Kotlin +// and Python derivations, so the five ports cannot drift on a derived field's name, +// subtype, nullability or type source. + +using System.IO; +using System.Linq; +using MetaObjects.Core.Reporting; +using MetaObjects.Loader; +using MetaObjects.Meta; +using Xunit; +using static MetaObjects.Core.Field.FieldConstants; + +namespace MetaObjects.Conformance.Tests; + +public class ReportShapeTests +{ + // fixtures/persistence-conformance, the sibling of the conformance corpus root. + private static readonly string PersistenceCorpus = + Path.Combine(Path.GetDirectoryName(CorpusRoot.Path)!, "persistence-conformance"); + + private static MetaRoot LoadCanonical() + { + var result = new MetaDataLoader().Load( + [new FileSource(Path.Combine(PersistenceCorpus, "canonical", "meta.fitness.json"))]); + Assert.True(result.Errors.Count == 0, + "canonical model failed to load: " + string.Join("; ", result.Errors.Select(e => e.ToString()))); + return result.Root; + } + + private static ReportShape Shape(MetaRoot root, string report) => + ReportShapes.Of(root.Objects().Single(o => o.Name == report), root); + + [Fact] + public void Derived_shapes_byte_match_the_committed_artifact() + { + string expected = File.ReadAllText(Path.Combine(PersistenceCorpus, "report-shapes.json")); + string actual = ReportShapes.ToArtifactJson(LoadCanonical()); + Assert.True(expected == actual, + "The C# report shapes differ from fixtures/persistence-conformance/report-shapes.json " + + "(TypeScript produces that file; a difference is a defect in ReportShapes).\n--- C# ---\n" + actual); + } + + [Fact] + public void The_artifact_covers_the_six_canonical_reports() + { + // Else the byte comparison could pass over an empty list on both sides. + var reports = LoadCanonical().Objects().Where(o => o.IsReport()).Select(o => o.Name).ToList(); + Assert.Equal( + ["ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", "RecentPrograms", "AssetActivity"], + reports); + } + + [Fact] + public void Dimensions_come_first_in_listed_order_then_measures() + { + var shape = Shape(LoadCanonical(), "ProgramMinutes"); + Assert.Equal("fitness::Week", shape.From.ResolutionKey()); + Assert.Equal( + ["program", "programTitle", "weeks", "longWeeks", "labels", "slots", "totalMinutes", + "avgMinutes", "minMinutes", "maxMinutes", "longShare"], + shape.Fields.Select(f => f.Name).ToList()); + Assert.All(shape.Fields.Take(2), f => Assert.Equal(ReportFieldRole.Dimension, f.Role)); + Assert.All(shape.Fields.Skip(2), f => Assert.Equal(ReportFieldRole.Measure, f.Role)); + } + + [Fact] + public void A_dimension_reached_by_via_is_nullable_and_a_count_never_is() + { + var fields = Shape(LoadCanonical(), "ProgramMinutes").Fields.ToDictionary(f => f.Name); + Assert.True(fields["program"].Required); // no @via, the @of field is @required + Assert.False(fields["programTitle"].Required); // reached by @via + Assert.True(fields["weeks"].Required); // count + Assert.False(fields["totalMinutes"].Required); // a sum of nothing is null + Assert.Equal(FIELD_SUBTYPE_LONG, fields["totalMinutes"].SubType); + Assert.Equal(FIELD_SUBTYPE_DECIMAL, fields["avgMinutes"].SubType); + Assert.Equal(FIELD_SUBTYPE_INT, fields["minMinutes"].SubType); + Assert.Equal(FIELD_SUBTYPE_DECIMAL, fields["longShare"].SubType); + } + + [Fact] + public void An_hour_grain_is_a_timestamp_and_a_coarser_grain_is_a_date() + { + var fields = Shape(LoadCanonical(), "AssetActivity").Fields.ToDictionary(f => f.Name); + Assert.Equal(FIELD_SUBTYPE_TIMESTAMP, fields["recordedAtHour"].SubType); + Assert.Equal("recordedAt", fields["recordedAtHour"].TypeSource?.Name); + Assert.Equal(FIELD_SUBTYPE_DATE, fields["asOfDateWeek"].SubType); + Assert.Null(fields["asOfDateWeek"].TypeSource); + } + + [Fact] + public void A_sourceless_report_has_a_shape_and_no_view() + { + var result = new MetaDataLoader().Load([new InMemoryStringSource( + """ + { "metadata.root": { "package": "shop", "children": [ + { "object.entity": { "name": "Sale", "children": [ + { "field.long": { "name": "id", "@required": true } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "Totals", "@from": "Sale", "@measures": ["sales"] } } + ] } } + """, id: "meta.shop.json")]); + Assert.Empty(result.Errors); + var report = result.Root.Objects().Single(o => o.IsReport()); + Assert.Equal(["sales"], ReportShapes.Of(report, result.Root).Fields.Select(f => f.Name).ToList()); + // A sourceless report has a shape and no view (Table A). + Assert.Null(ReportShapes.ReadSource(report)); + Assert.Contains("\"view\": null", ReportShapes.ToArtifactJson(result.Root)); + } + + // ----------------------------------------------------------------------- + // Reference resolution: the shape must agree with the loader's ValidateReporting + // about what a reference names, or a model that loads clean fails (or is silently + // mistyped) when it is generated. The same cases as the TypeScript report-shape.test.ts. + // ----------------------------------------------------------------------- + + // `a::Base` (abstract): members whose bare `@of` names `Base`. + private const string SharedBase = + """ + { "metadata.root": { "package": "a", "children": [ + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "kind" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "kind", "@of": "Base.kind" } }, + { "measure.aggregate": { "name": "events", "@agg": "count", "@of": "Base.id" } }, + { "measure.aggregate": { "name": "lastKind", "@agg": "max", "@of": "Base.kind" } } + ] } } + ] } } + """; + + private const string Decoy = + """ + { "object.entity": { "name": "Base", "children": [ + { "field.int": { "name": "id" } }, { "field.int": { "name": "kind" } } ] } }, + """; + + // Package `b`: `Ev extends a::Base` and report `R` over it. + private static string EvFile(string before = "", string evExtra = "", string measures = "[\"events\", \"lastKind\"]") => + "{ \"metadata.root\": { \"package\": \"b\", \"children\": [" + before + + " { \"object.entity\": { \"name\": \"Ev\", \"extends\": \"a::Base\", \"children\": [" + + " { \"source.rdb\": { \"@table\": \"evs\" } }" + evExtra + " ] } }," + + " { \"object.report\": { \"name\": \"R\", \"@from\": \"Ev\", \"@dimensions\": [\"kind\"]," + + " \"@measures\": " + measures + ", \"children\": [" + + " { \"source.rdb\": { \"@kind\": \"view\", \"@view\": \"v_r\" } } ] } } ] } }"; + + private static MetaRoot LoadInline(params string[] files) + { + var result = new MetaDataLoader().Load( + files.Select((json, i) => (IMetaDataSource)new InMemoryStringSource(json, id: $"meta.inline{i}.json")).ToList()); + Assert.True(result.Errors.Count == 0, + "model failed to load: " + string.Join("; ", result.Errors.Select(e => e.ToString()))); + return result.Root; + } + + // `name subType ` per derived field of `R`. + private static List Typed(MetaRoot root) => + Shape(root, "R").Fields + .Select(f => $"{f.Name} {f.SubType} {f.TypeSource?.Parent?.ResolutionKey() ?? "-"}") + .ToList(); + + [Fact] + public void A_bare_of_on_a_member_inherited_from_another_package_resolves_in_the_declaring_entitys_package() + { + var root = LoadInline(SharedBase, EvFile()); + Assert.Equal(["kind string a::Base", "events long -", "lastKind string a::Base"], Typed(root)); + } + + [Fact] + public void A_same_named_decoy_in_the_reports_package_does_not_capture_the_reference() + { + var root = LoadInline(SharedBase, EvFile(before: Decoy)); + Assert.Equal(["kind string a::Base", "events long -", "lastKind string a::Base"], Typed(root)); + } + + [Fact] + public void Without_via_the_field_is_read_from_from_so_a_field_from_redeclares_wins() + { + var root = LoadInline(SharedBase, EvFile(evExtra: ", { \"field.int\": { \"name\": \"kind\" } }")); + Assert.Equal(["kind int b::Ev", "events long -", "lastKind int b::Ev"], Typed(root)); + } + + [Fact] + public void A_dotted_measures_item_names_the_measure_by_its_last_segment() + { + var root = LoadInline(SharedBase, EvFile(measures: "[\"Ev.events\", \"a::Base.lastKind\"]")); + Assert.Equal(["kind", "events", "lastKind"], Shape(root, "R").Fields.Select(f => f.Name).ToList()); + } + + [Fact] + public void ReportMeasureItemName_is_the_last_segment() + { + Assert.Equal("total", ReportAccessors.ReportMeasureItemName("total")); + Assert.Equal("total", ReportAccessors.ReportMeasureItemName("Sale.total")); + Assert.Equal("total", ReportAccessors.ReportMeasureItemName("acme::shop::Sale.total")); + Assert.Null(ReportAccessors.ReportMeasureItemOwner("total")); + Assert.Equal("acme::shop::Sale", ReportAccessors.ReportMeasureItemOwner("acme::shop::Sale.total")); + } + + // A report built in code (never added to the root): what the loader would refuse. + private static MetaObject Stray(string pkg, string from, string attr, string item) + { + var report = new MetaObject(new TypeId(TYPE_OBJECT, OBJECT_SUBTYPE_REPORT), "Stray"); + report.SetPackage(pkg); + report.SetAttr(OBJECT_REPORT_ATTR_FROM, from); + report.SetAttr(attr, new List { item }); + return report; + } + + [Fact] + public void A_dotted_measures_item_whose_qualifier_is_not_from_or_an_ancestor_does_not_resolve() + { + var root = LoadInline(SharedBase, EvFile(before: Decoy)); + // Past the loader, which refuses these as ERR_INVALID_REPORT / ERR_REPORT_FOREIGN_MEASURE. + var ex = Assert.Throws(() => + ReportShapes.Of(Stray("b", "Ev", OBJECT_REPORT_ATTR_MEASURES, "Nope.events"), root)); + Assert.Equal("report 'Stray': measure 'Nope.events' on 'Ev' does not resolve.", ex.Message); + // The qualifier resolves in the REPORT's package: b::Base is the decoy, not an ancestor of Ev. + ex = Assert.Throws(() => + ReportShapes.Of(Stray("b", "Ev", OBJECT_REPORT_ATTR_MEASURES, "Base.events"), root)); + Assert.Equal("report 'Stray': measure 'Base.events' on 'Ev' does not resolve.", ex.Message); + } + + [Fact] + public void A_time_dimension_item_with_no_grain_or_a_grain_outside_the_closed_set_does_not_resolve() + { + var root = LoadCanonical(); + var ex = Assert.Throws(() => + ReportShapes.Of(Stray("fitness", "Program", OBJECT_REPORT_ATTR_DIMENSIONS, "createdAt"), root)); + Assert.Equal("report 'Stray': time dimension 'createdAt' grain '' does not resolve.", ex.Message); + ex = Assert.Throws(() => + ReportShapes.Of(Stray("fitness", "Program", OBJECT_REPORT_ATTR_DIMENSIONS, "createdAt:fortnight"), root)); + Assert.Equal("report 'Stray': time dimension 'createdAt' grain 'fortnight' does not resolve.", ex.Message); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs index bf069be10..414ad1e5a 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs @@ -11,7 +11,9 @@ public AppDbContext(DbContextOptions options) : base(options) { } public DbSet AllTypes { get; set; } = default!; public DbSet Assets { get; set; } = default!; + public DbSet AssetActivities { get; set; } = default!; public DbSet Auths { get; set; } = default!; + public DbSet FitnessTotals { get; set; } = default!; public DbSet Follows { get; set; } = default!; public DbSet Friendships { get; set; } = default!; public DbSet Measurements { get; set; } = default!; @@ -21,16 +23,27 @@ public AppDbContext(DbContextOptions options) : base(options) { } public DbSet PostReferrals { get; set; } = default!; public DbSet PostTags { get; set; } = default!; public DbSet Programs { get; set; } = default!; + public DbSet ProgramMinutes { get; set; } = default!; public DbSet ProgramStats { get; set; } = default!; public DbSet ProgramViews { get; set; } = default!; + public DbSet ProgramsByMonths { get; set; } = default!; + public DbSet ProgramsByWeeks { get; set; } = default!; + public DbSet RecentPrograms { get; set; } = default!; public DbSet Tags { get; set; } = default!; public DbSet Weeks { get; set; } = default!; protected override void OnModelCreating(ModelBuilder modelBuilder) { + modelBuilder.Entity().HasNoKey().ToView("v_asset_activity"); + modelBuilder.Entity().HasNoKey().ToView("v_fitness_totals"); + modelBuilder.Entity().HasNoKey().ToView("v_program_minutes"); modelBuilder.Entity().ToView(ProgramStatNames.SourcePrimaryView); modelBuilder.Entity().ToView(ProgramViewNames.SourcePrimaryView); modelBuilder.Entity().Property(x => x.Status).HasConversion(); + modelBuilder.Entity().HasNoKey().ToView("v_programs_by_month"); + modelBuilder.Entity().Property(x => x.Status).HasConversion(); + modelBuilder.Entity().HasNoKey().ToView("v_programs_by_week"); + modelBuilder.Entity().HasNoKey().ToView("v_recent_programs"); modelBuilder.Entity().OwnsOne(x => x.Settings, b => b.ToJson(AllTypesNames.SettingsColumn)); modelBuilder.Entity().OwnsMany(x => x.Labels, b => b.ToJson(AllTypesNames.LabelsColumn)); modelBuilder.Entity().Property(x => x.EnumVal).HasConversion(); diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AssetActivity.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AssetActivity.g.cs new file mode 100644 index 000000000..45048514c --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AssetActivity.g.cs @@ -0,0 +1,19 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class AssetActivity +{ + [Column("recordedAtHour")] + public DateTimeOffset RecordedAtHour { get; set; } + [Column("asOfDateWeek")] + public DateOnly AsOfDateWeek { get; set; } + [Column("assets")] + public long Assets { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotals.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotals.g.cs new file mode 100644 index 000000000..63c89ca40 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotals.g.cs @@ -0,0 +1,19 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class FitnessTotals +{ + [Column("weeks")] + public long Weeks { get; set; } + [Column("totalMinutes")] + public long? TotalMinutes { get; set; } + [Column("longShare")] + public decimal? LongShare { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramMinutes.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramMinutes.g.cs new file mode 100644 index 000000000..86d9ac20c --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramMinutes.g.cs @@ -0,0 +1,36 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class ProgramMinutes +{ + [Column("program")] + public long Program { get; set; } + [Column("programTitle")] + [MaxLength(200)] + public string? ProgramTitle { get; set; } + [Column("weeks")] + public long Weeks { get; set; } + [Column("longWeeks")] + public long LongWeeks { get; set; } + [Column("labels")] + public long Labels { get; set; } + [Column("slots")] + public long Slots { get; set; } + [Column("totalMinutes")] + public long? TotalMinutes { get; set; } + [Column("avgMinutes")] + public decimal? AvgMinutes { get; set; } + [Column("minMinutes")] + public int? MinMinutes { get; set; } + [Column("maxMinutes")] + public int? MaxMinutes { get; set; } + [Column("longShare")] + public decimal? LongShare { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByMonth.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByMonth.g.cs new file mode 100644 index 000000000..c7973ae97 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByMonth.g.cs @@ -0,0 +1,22 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class ProgramsByMonth +{ + public enum ProgramsByMonthStatus { DRAFT, PUBLISHED, ARCHIVED } + [Column("createdAtMonth")] + public DateOnly CreatedAtMonth { get; set; } + [Column("status")] + public ProgramsByMonthStatus Status { get; set; } + [Column("programs")] + public long Programs { get; set; } + [Column("listValue")] + public long? ListValue { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByWeek.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByWeek.g.cs new file mode 100644 index 000000000..1c39b13de --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramsByWeek.g.cs @@ -0,0 +1,17 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class ProgramsByWeek +{ + [Column("createdAtWeek")] + public DateOnly CreatedAtWeek { get; set; } + [Column("programs")] + public long Programs { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/RecentPrograms.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/RecentPrograms.g.cs new file mode 100644 index 000000000..5051760cc --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/RecentPrograms.g.cs @@ -0,0 +1,15 @@ +// +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class RecentPrograms +{ + [Column("programs")] + public long Programs { get; set; } +} diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs b/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs index 51ea8e1ec..8226e81f5 100644 --- a/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs +++ b/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs @@ -48,10 +48,36 @@ public static IReadOnlyList ReportDimensionItems(MetaData o .ToList() .AsReadOnly(); - /// The @measures names. + /// + /// The @measures items AS WRITTEN: each a bare measure name, or a dotted + /// Entity.name (loader rule R3). Use for the measure name. + /// public static IReadOnlyList ReportMeasureNames(MetaData obj) => ReportingValues.StringList(obj.Attr(OBJECT_REPORT_ATTR_MEASURES)); + /// + /// The measure a @measures item names: the segment after its LAST . + /// (total, Sale.total and acme::shop::Sale.total all name + /// total). It is also the derived report field's name. The part before that + /// ., when present, is an entity qualifier (). + /// + public static string ReportMeasureItemName(string item) + { + int dot = item.LastIndexOf(CHILD_REF_SEPARATOR, StringComparison.Ordinal); + return dot == -1 ? item : item[(dot + CHILD_REF_SEPARATOR.Length)..]; + } + + /// + /// The entity qualifier of a dotted @measures item (Sale in + /// Sale.total), or null for a bare item. Loader rule R3: it names @from + /// or an entity @from extends. + /// + public static string? ReportMeasureItemOwner(string item) + { + int dot = item.LastIndexOf(CHILD_REF_SEPARATOR, StringComparison.Ordinal); + return dot == -1 ? null : item[..dot]; + } + /// /// The derived report field for a dimension item: name (attribute) or /// name + Capitalized(grain) (time), e.g. purchasedAt:day → purchasedAtDay. diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs b/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs new file mode 100644 index 000000000..a5ebf7300 --- /dev/null +++ b/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs @@ -0,0 +1,305 @@ +// A report's derived fields (FR-044, contract Table B): one field per `@dimensions` item +// in listed order, then one per `@measures` item in listed order. +// +// Ported rule for rule from +// server/typescript/packages/metadata/src/core/reporting/report-shape.ts, and gated +// against it by fixtures/persistence-conformance/report-shapes.json (the TS-produced +// artifact every port byte-matches; see ReportShapeTests). + +using System.Text; +using MetaObjects.Loader; +using MetaObjects.Meta; + +namespace MetaObjects.Core.Reporting; + +/// Whether a derived report field comes from a dimension or a measure. +public enum ReportFieldRole +{ + Dimension, + Measure, +} + +/// One derived field of a report (one Table B row). +/// The derived field name; the physical column is the naming strategy applied to it. +/// Dimension or measure. +/// A field subtype name (FIELD_SUBTYPE_*). +/// False when the column can be null. +/// The @of field whose type-shaping attrs this field carries, or null. +/// The dimension node, for a dimension field. +/// The time grain, for a dimension.time field. +/// The measure node, for a measure field. +public sealed record ReportField( + string Name, + ReportFieldRole Role, + string SubType, + bool Required, + MetaField? TypeSource = null, + MetaDimension? Dimension = null, + string? Grain = null, + MetaMeasure? Measure = null); + +/// The read shape of an object.report. +/// The report node. +/// The resolved @from entity. +/// The derived fields, dimensions first, in declared order. +public sealed record ReportShape(MetaObject Report, MetaObject From, IReadOnlyList Fields); + +/// Derives a report's read shape (Table B) and serialises the conformance artifact. +public static class ReportShapes +{ + private static readonly HashSet SumLong = + new(StringComparer.Ordinal) { FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG }; + + private static readonly HashSet Floating = + new(StringComparer.Ordinal) { FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT }; + + /// + /// The entity that DECLARES a dimension or measure reached through : + /// the member's parent, which is itself or an entity it extends. A + /// bare entity name inside the member (@of, @via) resolves in THIS entity's + /// package, exactly as the loader's ValidateReporting resolves it + /// (EffectivePackage(ctx.Declaring)), never in 's package or + /// the report's. + /// + public static MetaData ReportingMemberOwner(MetaData member, MetaObject from) => member.Parent ?? from; + + /// + /// Resolve a dimension's or measure's Entity.field reference to the field node. + /// The ONE rule, the same as the loader's (ValidateReporting D1 / M1) and as the + /// TypeScript resolveReportingFieldRef: + /// + /// The entity half resolves relative to the package of , + /// the entity that declares the member (). + /// With (a measure, or a dimension without @via: the + /// reference is about the @from entity's own rows) the named entity must be + /// or an entity it extends, and the field is read from + /// , so a field it redeclares wins. + /// Without (a dimension with @via) the field is read + /// from the named entity. + /// + /// Null when any step fails. + /// + public static MetaField? ResolveReportingFieldRef( + string reference, MetaData declaring, MetaRoot root, MetaObject? host = null) + { + // `Entity.field`; a package qualifier uses `::`, so the member separator is the LAST dot. + int dot = reference.LastIndexOf(CHILD_REF_SEPARATOR, StringComparison.Ordinal); + if (dot <= 0) return null; + if (NamingRefs.ResolveObjectRef(root, reference[..dot], NamingRefs.EffectivePackage(declaring)) + is not MetaObject named) return null; + if (host is not null && !ValidationPasses.IsSelfOrAncestor(named, host)) return null; + // ADR-0039: resolving, so a field inherited through extends is found. + return (host ?? named).FindField(reference[(dot + CHILD_REF_SEPARATOR.Length)..]); + } + + private static InvalidOperationException Unresolved(string reportName, string what) => + new($"report '{reportName}': {what} does not resolve."); + + private static T? DeclaredMember(MetaObject from, string type, string name) where T : MetaData => + // ADR-0039: resolving Children(), so a member declared on an abstract base is found. + from.Children().OfType().FirstOrDefault(c => c.Type == type && c.Name == name); + + private static ReportField DimensionField(ReportDimensionItem item, MetaObject from, MetaRoot root, string reportName) + { + var dim = DeclaredMember(from, TYPE_DIMENSION, item.Name) + ?? throw Unresolved(reportName, $"dimension '{item.Name}' on '{from.Name}'"); + bool vialess = dim.Via() is null; + var of = ResolveReportingFieldRef(dim.Of() ?? "", ReportingMemberOwner(dim, from), root, vialess ? from : null) + ?? throw Unresolved(reportName, $"dimension '{item.Name}' @of"); + string name = ReportAccessors.ReportDerivedFieldName(item); + // The @required ATTR only, read resolving (ADR-0039); a validator.required child does not count. + bool required = vialess && of.Attr(FIELD_ATTR_REQUIRED) is true; + if (dim.IsTime()) + { + // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. + string? grain = item.Grain; + if (grain is null || !TIME_GRAINS.Contains(grain, StringComparer.Ordinal)) + throw Unresolved(reportName, $"time dimension '{item.Name}' grain '{grain ?? ""}'"); + return grain == GRAIN_HOUR + ? new ReportField(name, ReportFieldRole.Dimension, FIELD_SUBTYPE_TIMESTAMP, required, of, dim, grain) + : new ReportField(name, ReportFieldRole.Dimension, FIELD_SUBTYPE_DATE, required, null, dim, grain); + } + return new ReportField(name, ReportFieldRole.Dimension, of.SubType, required, of, dim); + } + + /// + /// One @measures item, bare (total) or dotted (Sale.total, loader + /// rule R3). The measure is named by the item's last segment and looked up on + /// ; a qualifier resolves in the REPORT's package and must be + /// or an entity it extends. + /// + private static ReportField MeasureField(string item, MetaObject report, MetaObject from, MetaRoot root) + { + string reportName = report.Name; + string name = ReportAccessors.ReportMeasureItemName(item); + if (ReportAccessors.ReportMeasureItemOwner(item) is { } qualifier) + { + var owner = NamingRefs.ResolveObjectRef(root, qualifier, NamingRefs.EffectivePackage(report)); + if (owner is null || !ValidationPasses.IsSelfOrAncestor(owner, from)) + throw Unresolved(reportName, $"measure '{item}' on '{from.Name}'"); + } + var m = DeclaredMember(from, TYPE_MEASURE, name) + ?? throw Unresolved(reportName, $"measure '{item}' on '{from.Name}'"); + if (m.IsRatio()) + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_DECIMAL, false, Measure: m); + string? agg = m.Agg(); + if (agg == AGG_COUNT) + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_LONG, true, Measure: m); + var of = ResolveReportingFieldRef(m.OfColumns().FirstOrDefault() ?? "", ReportingMemberOwner(m, from), root, from) + ?? throw Unresolved(reportName, $"measure '{name}' @of"); + string src = of.SubType; + if (agg == AGG_SUM) + { + if (src == FIELD_SUBTYPE_CURRENCY) + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_CURRENCY, false, of, Measure: m); + string sumType = SumLong.Contains(src) ? FIELD_SUBTYPE_LONG + : Floating.Contains(src) ? FIELD_SUBTYPE_DOUBLE + : FIELD_SUBTYPE_DECIMAL; + return new ReportField(name, ReportFieldRole.Measure, sumType, false, Measure: m); + } + if (agg == AGG_AVG) + { + string avgType = Floating.Contains(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; + return new ReportField(name, ReportFieldRole.Measure, avgType, false, Measure: m); + } + // min / max keep the source field's type. + return new ReportField(name, ReportFieldRole.Measure, src, false, of, Measure: m); + } + + /// + /// Table B. Throws naming the report when a + /// reference does not resolve (a report that passed the loader's report validation + /// always resolves). + /// + public static ReportShape Of(MetaObject report, MetaRoot root) + { + string fromName = ReportAccessors.ReportFrom(report) ?? throw Unresolved(report.Name, "@from"); + var from = NamingRefs.ResolveObjectRef(root, fromName, NamingRefs.EffectivePackage(report)) as MetaObject + ?? throw Unresolved(report.Name, $"@from '{fromName}'"); + var fields = new List(); + foreach (var item in ReportAccessors.ReportDimensionItems(report)) + fields.Add(DimensionField(item, from, root, report.Name)); + foreach (string item in ReportAccessors.ReportMeasureNames(report)) + fields.Add(MeasureField(item, report, from, root)); + return new ReportShape(report, from, fields.AsReadOnly()); + } + + /// + /// The source a report is READ from: its own read-only source with @role: primary, + /// else its first own read-only source; null when it declares none (Table A: not lowered). + /// The same rule that names the lowered view in the TypeScript lowering, so a reader + /// lands on the relation the lowering created. + /// + public static MetaSource? ReadSource(MetaObject report) + { + // ADR-0039: own — source classification reads the sources the report declares + // ITSELF, exactly as the lowering's view-name rule does. + var readOnly = report.OwnSources().Where(s => s.IsReadOnly()).ToList(); + return readOnly.FirstOrDefault(s => s.Role == SOURCE_ROLE_PRIMARY) ?? readOnly.FirstOrDefault(); + } + + // ----------------------------------------------------------------------- + // The conformance artifact (fixtures/persistence-conformance/report-shapes.json) + // ----------------------------------------------------------------------- + + /// + /// The report-shapes artifact for a loaded model: every object.report in + /// declaration order, serialised byte for byte as the TypeScript generator + /// (integration-tests/src/gen-report-shapes.ts) writes it — keys in a fixed + /// order, two-space indent, every object expanded, one trailing newline. + /// + public static string ToArtifactJson(MetaRoot root) + { + var reports = root.Objects().Where(o => o.IsReport()).ToList(); + var sb = new StringBuilder(); + sb.Append("{\n \"reports\": "); + if (reports.Count == 0) + { + sb.Append("[]"); + } + else + { + sb.Append("[\n"); + for (int i = 0; i < reports.Count; i++) + { + AppendReport(sb, Of(reports[i], root)); + sb.Append(i < reports.Count - 1 ? ",\n" : "\n"); + } + sb.Append(" ]"); + } + sb.Append("\n}\n"); + return sb.ToString(); + } + + private static void AppendReport(StringBuilder sb, ReportShape shape) + { + sb.Append(" {\n"); + sb.Append(" \"report\": ").Append(JsonString(shape.Report.ResolutionKey())).Append(",\n"); + sb.Append(" \"from\": ").Append(JsonString(shape.From.ResolutionKey())).Append(",\n"); + sb.Append(" \"view\": ").Append(JsonStringOrNull(ReadSource(shape.Report)?.PhysicalName)).Append(",\n"); + sb.Append(" \"fields\": "); + if (shape.Fields.Count == 0) + { + sb.Append("[]\n"); + } + else + { + sb.Append("[\n"); + for (int i = 0; i < shape.Fields.Count; i++) + { + var f = shape.Fields[i]; + sb.Append(" {\n"); + sb.Append(" \"name\": ").Append(JsonString(f.Name)).Append(",\n"); + sb.Append(" \"role\": ").Append(JsonString(RoleName(f.Role))).Append(",\n"); + sb.Append(" \"subType\": ").Append(JsonString(f.SubType)).Append(",\n"); + sb.Append(" \"required\": ").Append(f.Required ? "true" : "false").Append(",\n"); + sb.Append(" \"typeSource\": ").Append(JsonStringOrNull(TypeSourceKey(f.TypeSource))).Append('\n'); + sb.Append(i < shape.Fields.Count - 1 ? " },\n" : " }\n"); + } + sb.Append(" ]\n"); + } + sb.Append(" }"); + } + + /// The wire name of a role, as the artifact spells it. + public static string RoleName(ReportFieldRole role) => + role == ReportFieldRole.Dimension ? "dimension" : "measure"; + + // `.`, or null. + private static string? TypeSourceKey(MetaField? field) + { + if (field is null) return null; + var owner = field.Parent + ?? throw new InvalidOperationException($"field '{field.Name}' has no owning entity."); + return $"{owner.ResolutionKey()}{CHILD_REF_SEPARATOR}{field.Name}"; + } + + private static string JsonStringOrNull(string? s) => s is null ? "null" : JsonString(s); + + // JSON.stringify's string escaping: `"`, `\`, the short escapes, and \u00XX for the + // remaining control characters. Everything else is written as is. + private static string JsonString(string s) + { + var sb = new StringBuilder(s.Length + 2); + sb.Append('"'); + foreach (char c in s) + { + switch (c) + { + case '"': sb.Append("\\\""); break; + case '\\': sb.Append("\\\\"); break; + case '\b': sb.Append("\\b"); break; + case '\f': sb.Append("\\f"); break; + case '\n': sb.Append("\\n"); break; + case '\r': sb.Append("\\r"); break; + case '\t': sb.Append("\\t"); break; + default: + if (c < 0x20) sb.Append("\\u").Append(((int)c).ToString("x4")); + else sb.Append(c); + break; + } + } + sb.Append('"'); + return sb.ToString(); + } +} diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs index f1670a03d..281e00e45 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs @@ -143,7 +143,8 @@ private static (string Owner, string[] Path)? ReportingSplitDotted(string refere } /// True when is or an entity it extends. - private static bool IsSelfOrAncestor(MetaData? candidate, MetaData entity) + // Internal: the report shape (ReportShapes) applies the same test, so it uses this one. + internal static bool IsSelfOrAncestor(MetaData? candidate, MetaData entity) { var visited = new HashSet(ReferenceEqualityComparer.Instance); for (MetaData? n = entity; n is not null && !visited.Contains(n); n = n.SuperData) diff --git a/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/GeneratorUtil.java b/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/GeneratorUtil.java index 449ee6a56..580effd26 100644 --- a/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/GeneratorUtil.java +++ b/server/java/codegen-base/src/main/java/com/metaobjects/generator/util/GeneratorUtil.java @@ -19,9 +19,10 @@ public static Collection getFilteredMetaData(MetaDataLoader loader, Me } public static Collection getFilteredMetaData(MetaDataLoader loader, Class clazz, MetaDataFilters filters ) { - // FR-044 Plan 1: object.report has no output until its lowering lands (Plan 2/3). - // Dropped here because every direct per-object generator (the Java model tier, the - // Mustache and PlantUML generators) selects its objects through this overload. + // FR-044: no generator that selects its objects here emits for an object.report, with + // or without a view source. Dropped here because every direct per-object generator + // (the Java model tier, the Mustache and PlantUML generators) selects its objects + // through this overload. List generatable = new ArrayList<>(); for (T md : loader.getMetaData( clazz )) { if (!isReport(md)) generatable.add(md); @@ -30,10 +31,11 @@ public static Collection getFilteredMetaData(MetaDataLoa } /** - * True for an {@code object.report} (FR-044). Plan 1 registers and validates the - * reporting vocabulary but gives a report no lowering yet, so no generator emits for - * one — including a report that declares a read-only {@code source.rdb @kind: view} - * (R5 allows one), which would otherwise pass every source-keyed gate. + * True for an {@code object.report} (FR-044). The generators that ask this skip a + * report: every Java generator, and every Kotlin generator but one. A report that + * declares a read-only {@code source.rdb @kind: view} would otherwise pass every + * source-keyed gate. The one exception is the Kotlin Exposed table generator, which + * emits the read-only table object of a view-backed report. */ public static boolean isReport(MetaData md) { return md instanceof MetaObject && MetaObject.SUBTYPE_REPORT.equals(md.getSubType()); diff --git a/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xCodegenCompileTest.kt b/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xCodegenCompileTest.kt index 84be44e5e..18d12dca5 100644 --- a/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xCodegenCompileTest.kt +++ b/server/java/codegen-kotlin-exposed1x-check/src/test/kotlin/com/metaobjects/generator/kotlin/exposed1x/Exposed1xCodegenCompileTest.kt @@ -9,6 +9,7 @@ import com.metaobjects.generator.kotlin.KotlinRelationsGenerator import com.metaobjects.generator.kotlin.KotlinValidatorGenerator import com.metaobjects.generator.util.GeneratedFileWriter import com.metaobjects.metadata.ktx.loadDirectory +import com.metaobjects.metadata.ktx.loadString import com.tschuchort.compiletesting.KotlinCompilation import com.tschuchort.compiletesting.SourceFile import java.nio.file.Files @@ -127,4 +128,47 @@ class Exposed1xCodegenCompileTest { outDir.toFile().deleteRecursively() } } + + /** + * Exposed 1.x adds the open `Table` properties `options` and `storageParameters`, which + * 0.x lacks. A column property of either name hides that member and does not compile, so + * under `exposedApi=1` the generator suffixes it (`optionsColumn`). Compiled here because + * this is the only module with Exposed 1.x on its classpath. + */ + @Test + fun `exposedApi=1 table with fields named after 1x-only Table members compiles`() { + val model = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Plan", "children": [ + { "source.rdb": { "@table": "plans" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "options" } }, + { "field.string": { "name": "storageParameters" } }, + { "identity.primary": { "name": "id", "@fields": ["id"], "@generation": "increment" } }, + { "identity.secondary": { "name": "by_options", "@fields": ["options"] } } + ] } } + ] } + }""" + val outDir = Files.createTempDirectory("exposed1x-reserved-") + try { + val gen = KotlinExposedTableGenerator() + gen.setArgs(mapOf("outputDir" to outDir.toString(), "exposedApi" to "1")) + gen.execute(loadString("exposed1x-reserved", model)) + + val table = outDir.resolve("acme/shop/PlanTable.kt").readText() + assertTrue("val optionsColumn = text(\"options\")" in table, table) + assertTrue("val storageParametersColumn = text(\"storage_parameters\")" in table, table) + + val result = KotlinCompilation().apply { + sources = listOf(SourceFile.kotlin("PlanTable.kt", table)) + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, + "a table with fields named options / storageParameters does not compile against " + + "Exposed 1.3.x:\n${result.messages}") + } finally { + outDir.toFile().deleteRecursively() + } + } } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt index 0c5e89e74..fdf3a5dcf 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt @@ -186,7 +186,7 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( ), "exposed-table" to GeneratorInfo( name = "exposed-table", - description = "Per-entity Kotlin Exposed table object.", + description = "Kotlin Exposed table object per table-backed or view-backed entity or projection, and per view-backed report.", tier = GeneratorTier.NATIVE, layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinExposedTableGenerator, diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt index ad6206879..a6293244c 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt @@ -15,10 +15,13 @@ import com.metaobjects.generator.kotlin.PackageMapping import com.metaobjects.MetaData import com.metaobjects.database.CoreDBMetaDataProvider import com.metaobjects.database.IndexNaming +import com.metaobjects.MetaRoot import com.metaobjects.field.EnumField import com.metaobjects.field.MapField import com.metaobjects.field.MetaField +import com.metaobjects.field.DecimalField import com.metaobjects.field.ObjectField +import com.metaobjects.generator.GeneratorException import com.metaobjects.generator.GeneratorIOWriter import com.metaobjects.generator.direct.MultiFileDirectGeneratorBase import com.metaobjects.identity.MetaIdentity @@ -29,6 +32,8 @@ import com.metaobjects.`object`.MetaObject import com.metaobjects.relationship.CompositionRelationship import com.metaobjects.relationship.MetaRelationship import com.metaobjects.relationship.RelationshipReferences +import com.metaobjects.reporting.ReportReadModel +import com.metaobjects.reporting.ReportShape import com.metaobjects.source.MetaSource import com.metaobjects.source.RdbSource import com.squareup.kotlinpoet.ClassName @@ -44,6 +49,10 @@ import com.metaobjects.generator.util.GeneratedFileWriter * Generator: one Exposed Table `object` per `object.entity` that has a `source.rdb` child. * Entities without source.rdb are skipped (no persistence layer). * + *

A view-backed `object.report` (FR-044) also gets one — the read-only mapping of the view + * `meta migrate` creates, with one column per derived field. See [emitReport] for which + * reports emit and which do not. + * *

Exposed's `Column` types are inferred by the Kotlin compiler from the initialiser * expressions (e.g., `val name = varchar("name", 100)`). KotlinPoet's [com.squareup.kotlinpoet.PropertySpec] * requires an explicit type, which would force `val name: Column = ...` — verbose and @@ -154,6 +163,16 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBaseTable` of a view-backed `object.report`. + * + * Which reports emit (contract Table A): + * + * - no read-only source: NOTHING. The report has no view, so there is nothing to map + * (a sourceless object generates nothing, #248). + * - `source.rdb @kind: view`: the table object, bound to the source's physical name. + * The same for a view marked `@unmanaged: true` or carrying an authored `@sql` body: + * migrate does not derive (or does not create) that view, but the view exists and + * Table B still defines the columns a reader gets, so the mapping is needed either way. + * - `@kind: materializedView` / `storedProc` / `tableFunction`: NOTHING. The lowering + * skips those kinds, so no relation with the Table B columns is promised. + * - an abstract report: NOTHING. + * + * The columns are the report's DERIVED fields — one per dimension, then one per measure — + * taken from the JVM's single definition of that shape ([ReportShape], via + * [ReportReadModel], which presents them as ordinary field nodes). So the report goes + * through the same [emit] as a view-kind projection, with three differences, which reach + * it through the report's [ReportTablePlan] and its being a [ReportReadModel]: + * + * - it binds its view and columns by LITERAL even when the names generator is in the + * run ([bindsThroughNames]): [KotlinNamesGenerator] emits nothing for a report; + * - an enum column references the enum class of the entity the dimension reads + * ([reportEnumClass]): no generator emits a per-report enum; + * - a derived decimal with no declared precision reads at [REPORT_DECIMAL_PRECISION] / + * [REPORT_DECIMAL_SCALE] ([scalarColumnSpec]). + * + * A report has no identity, so the table has no `primaryKey`, and no index or reference. + * Every other Kotlin generator skips reports. + */ + private fun emitReport( + report: MetaObject, + outRoot: Path, + loader: MetaDataLoader, + packagesNeedingInstantTzHelper: MutableSet, + packagesNeedingInetUriHelper: MutableSet, + packagesNeedingJacksonMapper: MutableSet, + packagesNeedingUuidStringHelper: MutableSet, + ) { + if (KotlinGenUtil.isAbstractEntity(report)) return + // The source the report is READ from, by the rule that names the lowered view. + val source = ReportShape.readSource(report) as? RdbSource ?: return + if (source.effectiveKind != MetaSource.KIND_VIEW) return + + // Table B, derived ONCE for the report; everything below reads this one shape. + val shape = ReportShape.of(report, loader.root) + refuseUncompilableReportColumns(shape) + refuseObjectFieldReportColumns(shape) + val model = ReportReadModel.of(report) + val pkg = PackageMapping.splitFqn(report.name).first + reportPlans[model] = ReportTablePlan( + enumClasses = shape.fields().filter { it.typeSource is EnumField } + .associate { it.name to reportEnumClass(shape, it) }, + unsizedDecimals = shape.fields() + .filter { it.typeSource == null && it.subType == DecimalField.SUBTYPE_DECIMAL } + .mapTo(HashSet()) { it.name }, + ) + try { + if (emit(model, source, outRoot, loader, emptyList(), emptyMap())) packagesNeedingInstantTzHelper += pkg + } finally { + reportPlans.remove(model) + } + if (entityNeedsInetUriHelper(model, loader)) packagesNeedingInetUriHelper += pkg + if (entityNeedsJacksonMapper(model, loader)) packagesNeedingJacksonMapper += pkg + if (entityNeedsUuidStringHelper(model, loader)) packagesNeedingUuidStringHelper += pkg + } + + /** + * What [emit] needs to know about the report it is emitting that the read model's + * field nodes do not say. Built once per report by [emitReport]. + * + * @property enumClasses derived enum field name → the generated enum class its column is typed by + * @property unsizedDecimals derived decimal fields the shape gives no type source, so no precision + */ + private class ReportTablePlan(val enumClasses: Map, val unsizedDecimals: Set) + + /** + * The plan of the report being emitted, keyed by its read model for the duration of + * that one [emit] call. Identity-keyed because node equality is structural. It is how + * the three report differences reach [emit] without a parameter on a `protected open` + * function an adopter's subclass may override. + */ + private val reportPlans = java.util.IdentityHashMap() + + /** + * Refuse a report whose derived field names cannot become the column properties of one + * Kotlin `object`. `gen` would otherwise exit 0 and the adopter's build would be the + * first thing to disagree. + * + * Two cases, both loadable. A derived field named after a Kotlin hard keyword (`in`, + * `is`, `object`, …) is not a legal property name. And [KotlinNaming.safeColumnProperty] + * renames a field that collides with an Exposed `Table` member (`source` becomes + * `sourceColumn`), which can land on a second derived field already called that. + * + * Scoped to reports: an entity field has the same two hazards and they are left as they + * were. A report that generates no table (see [emitReport]) is never checked. + */ + private fun refuseUncompilableReportColumns(shape: ReportShape) { + val report = shape.report() + val seen = HashMap() + for (f in shape.fields()) { + if (f.name in KOTLIN_HARD_KEYWORDS) { + throw GeneratorException( + "report \"${report.shortName}\": its ${describeItem(f)} generates the Exposed column " + + "property \"${f.name}\", and `${f.name}` is a Kotlin keyword, so the generated " + + "table would not compile. Rename the ${f.role.wireName()}." + ) + } + val property = KotlinNaming.safeColumnProperty(f.name, exposedApi()) + val prior = seen.put(property, f) ?: continue + throw GeneratorException( + "report \"${report.shortName}\": its ${describeItem(prior)} and its ${describeItem(f)} both " + + "generate the Exposed column property \"$property\" (a name that collides with a member " + + "of Exposed's Table gets a \"Column\" suffix), so the generated table would not " + + "compile. Rename one of them." + ) + } + } + + /** + * Refuse a report with a derived field typed by a `field.object` (a dimension over an + * embedded value object, say). The loader accepts it, but no port reads one through a + * report's row: the read model the table is built from ([ReportReadModel]) refuses it + * too, with the same sentence. Refused here first so `gen` fails as a generator error + * naming the report and the dimension or measure. + */ + private fun refuseObjectFieldReportColumns(shape: ReportShape) { + for (f in shape.fields()) { + val src = f.typeSource ?: continue + // ADR-0039: resolving — an @objectRef the @of field inherits counts. + if (src.subType != ObjectField.SUBTYPE_OBJECT && !src.hasMetaAttr(MetaField.ATTR_OBJECT_REF)) continue + throw GeneratorException( + "report \"${shape.report().shortName}\": its ${describeItem(f)} reads " + + "\"${f.typeSourceKey()}\", a field.${src.subType}. A report over a field.object is " + + "not supported; group by a scalar field." + ) + } + } + + /** + * `measure "x"` or `dimension "x"`. The derived name IS the item name here: only a time + * dimension derives a different one (``), and that is never a keyword, a + * `Table` member or a `…Column` name, so a time dimension is never refused. + */ + private fun describeItem(f: ReportShape.Field): String = "${f.role.wireName()} \"${f.name}\"" + + /** + * Whether the table of [entity] references `Names` constants: only when the + * names generator is in the run ([useNames]) AND emits an artifact for [entity]. It + * emits none for a report (FR-044), so a report binds its view and columns by literal. + */ + private fun bindsThroughNames(entity: MetaObject): Boolean = useNames() && entity !is ReportReadModel + + /** + * The generated enum class a report's derived enum field [f] is typed by. A report gets + * no entity class and so no enum of its own; its enum column carries the values of the + * field the dimension (or min/max measure) reads, and is typed by THAT field's class: + * the one [KotlinEntityGenerator] emits for the entity the item reads from. Without + * `@via` that is the report's `@from` entity (which is how a field `@from` inherits from + * an abstract base still names a class that exists); with `@via` it is the entity the + * `@of` reference names. [ReportShape.ofEntity] answers both, by the rule that derived + * the field (a bare name resolves in the package of the entity DECLARING the dimension). + * + * @throws GeneratorException naming the report and the dimension when that entity does + * not resolve. The shape resolved the same reference to derive [f], so a loaded model + * cannot reach this; it guards a tree built in code, where typing the column by a + * guessed class would compile against the wrong enum. + */ + private fun reportEnumClass(shape: ReportShape, f: ReportShape.Field): ClassName { + val report = shape.report() + // The entity the field is read from, by the rule that derived the field: nothing + // about packages or @via is restated here. + val owner = shape.ofEntity(f) + ?: throw GeneratorException( + "report \"${report.shortName}\": its dimension \"${f.dimension?.shortName}\" reads the enum " + + "\"${f.dimension?.of}\" through @via, and the entity that reference names does not " + + "resolve, so the generated column has no enum class to be typed by." + ) + return KotlinTypeMapper.enumTypeName(f.typeSource, owner) + ?: throw GeneratorException( + "report \"${report.shortName}\": its ${describeItem(f)} is an enum with no generated enum class." + ) + } + + /** + * The generated enum class a `field.enum` column of [entity]'s table is typed by: + * [KotlinTypeMapper.enumTypeName] for an entity or projection — the class + * [KotlinEntityGenerator] emits for it — and [reportEnumClass] for a report. + */ + private fun enumClassFor(field: MetaField<*>, entity: MetaObject): ClassName? = + reportPlans[entity]?.enumClasses?.get(field.name) ?: KotlinTypeMapper.enumTypeName(field, entity) + + /** + * The Exposed column spec of a non-enum scalar [field] of [entity]: the type mapper's, + * except for a report's derived decimal that carries no declared precision (an `avg`, a + * ratio, or a `sum` of a decimal — Table B gives those no type source). + * + * Exposed's decimal column rounds every value it reads to the column's declared scale, + * so the mapper's default of four places would silently turn a ratio of 2/3 into 0.6667. + * These columns are an unconstrained NUMERIC in the view, so they are read at the widest + * precision and scale a Postgres NUMERIC is commonly declared with. The object maps a + * view, so the two numbers never reach DDL. + */ + private fun scalarColumnSpec(entity: MetaObject, field: MetaField<*>, colExpr: String, api: ExposedApi): String { + if (reportPlans[entity]?.unsizedDecimals?.contains(field.name) == true) { + return "decimal($colExpr, $REPORT_DECIMAL_PRECISION, $REPORT_DECIMAL_SCALE)" + } + return KotlinTypeMapper.exposedColumnSpec(field, colExpr, api) + } + /** * True iff [entity] carries at least one `field.uri`/`field.inet` column — on a direct * field OR a flattened `object.value` sub-field. Mirrors the instant-tz detection inside @@ -516,7 +746,7 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase "$baseSpec.autoIncrement()" @@ -840,7 +1070,7 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase", col1, ...) }` for identity.secondary @@ -889,11 +1119,11 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase): String { - if (!useNames()) return "\"${KotlinGenUtil.resolveColumnName(f, columnNaming())}\"" + if (!bindsThroughNames(entity)) return "\"${KotlinGenUtil.resolveColumnName(f, columnNaming())}\"" // ADR-0039: metaFields is the RESOLVING accessor — an inherited field is a HIT here, // and the artifact of `entity` is where its constant is read from. if (entity.metaFields.any { it.name == f.name }) return ownColumnExpr(entity, f) @@ -996,7 +1226,7 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase): String { val (_, shortName) = PackageMapping.splitFqn(entity.name) - return if (useNames()) + return if (bindsThroughNames(entity)) "${KotlinNaming.namesObjectName(shortName)}.${KotlinNaming.namesMember(f.name)}_COLUMN" else "\"${KotlinGenUtil.resolveColumnName(f, columnNaming())}\"" } @@ -1175,6 +1405,24 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase = setOf( + "as", "break", "class", "continue", "do", "else", "false", "for", "fun", "if", "in", + "interface", "is", "null", "object", "package", "return", "super", "this", "throw", + "true", "try", "typealias", "typeof", "val", "var", "when", "while", + ) + /** * Exposed column suffix that renders a Postgres `DEFAULT gen_random_uuid()` * server-side mint on a native uuid column (R6 Plan 2a, `@generation: uuid`). @@ -1977,7 +2225,7 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase() for (entity in loader.metaObjects) { - // FR-044 Plan 1: object.report has no output until its lowering lands (Plan 2/3). + // FR-044: a report gets no names artifact. Its Exposed table (the one thing Kotlin + // generates for a view-backed report) binds its view and columns by literal. if (GeneratorUtil.isReport(entity)) continue if (emit(entity, outRoot, strategy)) emitted += entity.name } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt index 34d46978b..bdbbf4569 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinNaming.kt @@ -54,20 +54,35 @@ object KotlinNaming { */ val RESERVED_TABLE_MEMBERS: Set = setOf( "source", "fields", "columns", "index", "indices", "primaryKey", - "tableName", "ddl", "foreignKeys", "checkConstraints", "sequences", + "tableName", "schemaName", "ddl", "foreignKeys", "checkConstraints", "sequences", "autoIncColumn", "realFields", "defaultExpression", "generatedSignature", "tableNameWithoutScheme", "tableNameWithoutSchemeSanitized", ) + /** + * The `Table` properties Exposed 1.x ADDS to [RESERVED_TABLE_MEMBERS] (checked with + * `javap` against exposed-core 1.3.1: `options`, `storageParameters`). Reserved only in + * the Exposed 1.x output mode: under 0.x neither is a `Table` member, a column property + * of that name compiles, and renaming it would change working generated code. + */ + val RESERVED_TABLE_MEMBERS_EXPOSED_1X: Set = setOf("options", "storageParameters") + /** * [KotlinExposedTableGenerator]: the Kotlin property name for a column. Identity for a * normal field; a field whose camelCase name collides with an Exposed `Table`/`ColumnSet` - * member ([RESERVED_TABLE_MEMBERS]) gets a `Column` suffix (e.g. `source` → `sourceColumn`). + * member ([RESERVED_TABLE_MEMBERS], plus [RESERVED_TABLE_MEMBERS_EXPOSED_1X] when + * [exposedApi] is 1.x) gets a `Column` suffix (e.g. `source` → `sourceColumn`). * The PHYSICAL column name is unaffected — only the Kotlin val identifier changes — so the * persisted schema is unchanged. + * + * Every site that names the property passes the run's [exposedApi], so the table that + * declares it and the code that references it agree. */ - fun safeColumnProperty(name: String): String = - if (name in RESERVED_TABLE_MEMBERS) name + "Column" else name + fun safeColumnProperty(name: String, exposedApi: ExposedApi = ExposedApi.V0): String { + val reserved = name in RESERVED_TABLE_MEMBERS || + (exposedApi == ExposedApi.V1 && name in RESERVED_TABLE_MEMBERS_EXPOSED_1X) + return if (reserved) name + "Column" else name + } /** [KotlinSpringControllerGenerator]: `shortName + "Controller"`. */ fun controllerName(shortName: String): String = shortName + "Controller" diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinRelationsGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinRelationsGenerator.kt index 02ff8e8db..b4b9f1e39 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinRelationsGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinRelationsGenerator.kt @@ -281,7 +281,7 @@ open class KotlinRelationsGenerator : MultiFileDirectGeneratorBase() for (fk in reverseFks) { if (!first) append("\n") first = false - val col = KotlinNaming.safeColumnProperty(fk.fkField) + val col = KotlinNaming.safeColumnProperty(fk.fkField, exposedApi()) val single = KotlinNaming.reverseFinderName(fk.fkField) val batched = KotlinNaming.reverseFinderInName(fk.fkField) append("/** Reverse nav: the $ownerShort rows whose `${fk.fkField}` FK points at the given ${fk.targetShortName} id (single indexed query). */\n") diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 63889b078..28e4e7034 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -884,7 +884,7 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase): String { +private fun sortColumnExpr(tableVar: String, shortName: String, sortFields: List, api: ExposedApi): String { if (sortFields.isEmpty()) return "error(\"$shortName has no sortable fields\")" val sb = StringBuilder("when (field) {\n") for (name in sortFields) { - val prop = KotlinNaming.safeColumnProperty(name) + val prop = KotlinNaming.safeColumnProperty(name, api) sb.append(" \"").append(name).append("\" -> ") .append(tableVar).append(".").append(prop).append("\n") } diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt new file mode 100644 index 000000000..95a79ae51 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt @@ -0,0 +1,533 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.generator.Generator +import com.metaobjects.generator.GeneratorException +import com.metaobjects.loader.MetaDataLoader +import com.metaobjects.metadata.ktx.loadDirectory +import com.metaobjects.metadata.ktx.loadString +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import java.nio.file.Files +import java.nio.file.Path +import java.nio.file.Paths +import java.util.TreeMap +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * FR-044 — [KotlinExposedTableGenerator] emits the read-only Exposed table of a view-backed + * `object.report`, with one column per derived field (contract Table B), and nothing for a + * report that has no view. + * + * The six canonical reports are generated from the shared persistence corpus, the same + * model the hand-written reference tables in `integration-tests-kotlin` map and the + * persistence lane reads. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class KotlinReportTableGeneratorTest { + + private fun canonicalDir(): Path { + var cur: Path? = Paths.get("").toAbsolutePath() + while (cur != null) { + val candidate = cur.resolve("fixtures/persistence-conformance/canonical") + if (Files.isDirectory(candidate)) return candidate + cur = cur.parent + } + throw IllegalStateException("Could not locate fixtures/persistence-conformance/canonical") + } + + /** Run [generators] over [loader] into one directory; relative path to contents. */ + private fun emit( + loader: MetaDataLoader, + args: Map = emptyMap(), + generators: List = listOf(KotlinExposedTableGenerator()), + ): Map { + val outDir = Files.createTempDirectory("report-table-") + try { + for (gen in generators) { + gen.setArgs(mapOf("outputDir" to outDir.toString(), "packageName" to "acme.shop") + args) + gen.execute(loader) + } + val files = TreeMap() + Files.walk(outDir).use { s -> + s.filter { it.isRegularFile() }.forEach { files[outDir.relativize(it).toString()] = it.readText() } + } + return files + } finally { + outDir.toFile().deleteRecursively() + } + } + + private fun canonical(args: Map = mapOf("columnNaming" to "literal")) = + emit(loadDirectory("report-table-canonical", canonicalDir()), args) + + private fun assertCompiles(files: Map) { + val sources = files.filterKeys { it.endsWith(".kt") } + .map { (path, text) -> SourceFile.kotlin(path.substringAfterLast('/'), text) } + val result = KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true // Exposed, off the test classpath + messageOutputStream = System.out + }.compile() + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + } + + // --- The canonical reports --------------------------------------------------------- + + @Test + fun `a grouped report emits one column per derived field, typed and nullable by Table B`() { + assertEquals( + """ + |package fitness + | + |import org.jetbrains.exposed.sql.Table + | + |/** READ-ONLY VIEW — generated from view metadata; do not insert/update/delete directly. */ + |/** GENERATED — do not hand-edit. Regenerated from metadata. */ + |object ProgramMinutesTable : Table("v_program_minutes") { + | val program = long("program") + | val programTitle = varchar("programTitle", 200).nullable() + | val weeks = long("weeks") + | val longWeeks = long("longWeeks") + | val labels = long("labels") + | val slots = long("slots") + | val totalMinutes = long("totalMinutes").nullable() + | val avgMinutes = decimal("avgMinutes", 38, 18).nullable() + | val minMinutes = integer("minMinutes").nullable() + | val maxMinutes = integer("maxMinutes").nullable() + | val longShare = decimal("longShare", 38, 18).nullable() + |} + |""".trimMargin(), + canonical().getValue("fitness/ProgramMinutesTable.kt"), + ) + } + + @Test + fun `every canonical report emits its table and none has a primary key`() { + val files = canonical() + val reports = listOf( + "ProgramMinutes" to "v_program_minutes", "FitnessTotals" to "v_fitness_totals", + "ProgramsByMonth" to "v_programs_by_month", "ProgramsByWeek" to "v_programs_by_week", + "RecentPrograms" to "v_recent_programs", "AssetActivity" to "v_asset_activity", + ) + for ((report, view) in reports) { + val src = files.getValue("fitness/${report}Table.kt") + assertTrue("object ${report}Table : Table(\"$view\") {" in src, src) + assertFalse("primaryKey" in src, src) + assertFalse("init {" in src, src) + assertFalse(".references(" in src, src) + assertFalse("autoIncrement" in src, src) + } + } + + @Test + fun `a day-or-coarser bucket is a date and an enum dimension is typed by the source entity's enum`() { + val src = canonical().getValue("fitness/ProgramsByMonthTable.kt") + assertTrue("import org.jetbrains.exposed.sql.javatime.date\n" in src, src) + assertTrue(" val createdAtMonth = date(\"createdAtMonth\")\n" in src, src) + // Program.status's own generated class — no generator emits a ProgramsByMonthStatus. + assertTrue( + " val status = enumerationByName(\"status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, ProgramStatus::class)\n" in src, + src, + ) + assertFalse("ProgramsByMonthStatus" in src, src) + assertTrue(" val programs = long(\"programs\")\n" in src, src) + // A sum of a currency is integer minor units, and null over no rows. + assertTrue(" val listValue = long(\"listValue\").nullable()\n" in src, src) + } + + @Test + fun `an hour bucket of an instant is the instant column and brings the package helper`() { + val files = canonical() + val src = files.getValue("fitness/AssetActivityTable.kt") + assertTrue(" val recordedAtHour = instantWithTimeZone(\"recordedAtHour\")\n" in src, src) + assertTrue(" val asOfDateWeek = date(\"asOfDateWeek\")\n" in src, src) + assertTrue(" val assets = long(\"assets\")\n" in src, src) + assertTrue("fitness/MetaInstantWithTimeZoneColumnType.kt" in files.keys, files.keys.toString()) + } + + @Test + fun `the naming strategy applies to the derived field name`() { + val src = canonical(emptyMap()).getValue("fitness/ProgramMinutesTable.kt") // snake_case default + assertTrue(" val avgMinutes = decimal(\"avg_minutes\", 38, 18).nullable()\n" in src, src) + assertTrue(" val programTitle = varchar(\"program_title\", 200).nullable()\n" in src, src) + } + + @Test + fun `the Exposed 1x mode emits the same report table against the v1 packages`() { + val files = canonical(mapOf("columnNaming" to "literal", "exposedApi" to "1")) + val byMonth = files.getValue("fitness/ProgramsByMonthTable.kt") + assertTrue("import org.jetbrains.exposed.v1.core.Table\n" in byMonth, byMonth) + assertTrue("import org.jetbrains.exposed.v1.javatime.date\n" in byMonth, byMonth) + assertFalse("org.jetbrains.exposed.sql" in byMonth, byMonth) + assertTrue(" val createdAtMonth = date(\"createdAtMonth\")\n" in byMonth, byMonth) + val minutes = files.getValue("fitness/ProgramMinutesTable.kt") + assertTrue(" val avgMinutes = decimal(\"avgMinutes\", 38, 18).nullable()\n" in minutes, minutes) + } + + @Test + fun `with the names generator in the run a report still binds by literal and gets no names artifact`() { + val files = emit( + loadDirectory("report-table-names", canonicalDir()), + mapOf("columnNaming" to "literal", "useNames" to "true"), + listOf(KotlinNamesGenerator(), KotlinExposedTableGenerator()), + ) + // The entity beside it does reference its artifact, so the arg really was on. + assertTrue("ProgramNames." in files.getValue("fitness/ProgramTable.kt")) + val src = files.getValue("fitness/ProgramMinutesTable.kt") + assertTrue("object ProgramMinutesTable : Table(\"v_program_minutes\") {" in src, src) + assertTrue(" val weeks = long(\"weeks\")\n" in src, src) + assertFalse("Names" in src, src) + assertFalse(files.keys.any { it.endsWith("ProgramMinutesNames.kt") }, files.keys.toString()) + } + + // --- Which reports emit (Table A) -------------------------------------------------- + + /** A `Sale` entity with [measures], and a `SaleTotals` report over them with [reportSource]. */ + private fun model( + measures: List = listOf("sales"), + reportSource: String? = """{ "source.rdb": { "@kind": "view", "@view": "v_sale_totals" } }""", + extraMembers: String = "", + dimensions: List = emptyList(), + ): String { + val measureNodes = measures.joinToString(",\n") { + """{ "measure.aggregate": { "name": "$it", "@agg": "count", "@of": "Sale.id" } }""" + } + val children = reportSource?.let { """, "children": [ $it ]""" } ?: "" + val dims = if (dimensions.isEmpty()) "" else + """ "@dimensions": [${dimensions.joinToString(",") { "\"$it\"" }}],""" + return """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.timestamp": { "name": "soldAt" } }, + { "field.string": { "name": "channel" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + $extraMembers + $measureNodes + ] } }, + { "object.report": { "name": "SaleTotals", "@from": "Sale",$dims + "@measures": [${measures.joinToString(",") { "\"$it\"" }}]$children } } + ] } + }""" + } + + private fun reportFiles(json: String, args: Map = emptyMap()): Map = + emit(loadString("report-table-model", json), args).filterKeys { "SaleTotals" in it } + + @Test + fun `a sourceless report generates nothing`() { + assertEquals(emptyMap(), reportFiles(model(reportSource = null))) + } + + @Test + fun `a managed view-backed report generates its table`() { + val files = reportFiles(model()) + assertEquals(setOf("acme/shop/SaleTotalsTable.kt"), files.keys) + val src = files.values.single() + assertTrue("object SaleTotalsTable : Table(\"v_sale_totals\") {" in src, src) + assertTrue(" val sales = long(\"sales\")\n" in src, src) + } + + @Test + fun `an unmanaged view and an authored-sql view exist, so each still generates the table`() { + for (attrs in listOf( + """"@unmanaged": true""", + """"@sql": "SELECT COUNT(id) AS sales FROM sales"""", + )) { + val files = reportFiles(model( + reportSource = """{ "source.rdb": { "@kind": "view", "@view": "v_sale_totals", $attrs } }""")) + assertEquals(setOf("acme/shop/SaleTotalsTable.kt"), files.keys, attrs) + assertTrue(" val sales = long(\"sales\")\n" in files.values.single(), attrs) + } + } + + @Test + fun `a view named by the legacy table attr and a schema-qualified view bind that name`() { + val legacy = reportFiles(model( + reportSource = """{ "source.rdb": { "@kind": "view", "@table": "v_legacy" } }""")) + assertTrue("Table(\"v_legacy\")" in legacy.values.single(), legacy.values.single()) + val qualified = reportFiles(model( + reportSource = """{ "source.rdb": { "@kind": "view", "@view": "v_sale_totals", "@schema": "rpt" } }""")) + assertTrue("Table(\"rpt.v_sale_totals\")" in qualified.values.single(), qualified.values.single()) + } + + @Test + fun `a report over a kind the lowering skips generates nothing`() { + for (kind in listOf( + """"@kind": "materializedView", "@materializedView": "mv_sale_totals"""", + """"@kind": "storedProc", "@procedure": "sale_totals"""", + """"@kind": "tableFunction", "@function": "sale_totals"""", + )) { + assertEquals(emptyMap(), reportFiles(model(reportSource = """{ "source.rdb": { $kind } }""")), kind) + } + } + + // --- Names that would not compile --------------------------------------------------- + + @Test + fun `names that are SQL keywords or Exposed Table members compile`() { + // `order`, `user`, `group`, `rank` are ordinary dashboard names; the rest are (or + // look like) members of Exposed's Table, which safeColumnProperty renames. + val names = listOf( + "order", "user", "group", "rank", "count", "name", "columns", "tableName", "source", + "index", "fields", "primaryKey", "schemaName", + ) + val files = reportFiles(model(measures = names)) + val src = files.getValue("acme/shop/SaleTotalsTable.kt") + assertTrue(" val order = long(\"order\")\n" in src, src) + // The physical column keeps the derived name; only the Kotlin property is renamed. + assertTrue(" val columnsColumn = long(\"columns\")\n" in src, src) + assertTrue(" val tableNameColumn = long(\"table_name\")\n" in src, src) + assertTrue(" val schemaNameColumn = long(\"schema_name\")\n" in src, src) + assertCompiles(files) + } + + @Test + fun `a measure named after a Kotlin keyword is refused, naming the report and the measure`() { + for (keyword in listOf("in", "is", "object", "when", "fun")) { + val e = assertFailsWith(keyword) { reportFiles(model(measures = listOf("sales", keyword))) } + val message = e.message.orEmpty() + assertTrue("report \"SaleTotals\"" in message, message) + assertTrue("measure \"$keyword\"" in message, message) + assertTrue("Kotlin keyword" in message && "Rename the measure" in message, message) + } + } + + @Test + fun `a dimension named after a Kotlin keyword is refused, naming the dimension`() { + val e = assertFailsWith { + reportFiles(model( + extraMembers = """{ "dimension.attribute": { "name": "class", "@of": "Sale.channel" } },""", + dimensions = listOf("class"), + )) + } + val message = e.message.orEmpty() + assertTrue("report \"SaleTotals\"" in message && "dimension \"class\"" in message, message) + } + + @Test + fun `two derived fields that land on one column property are refused, naming both`() { + val e = assertFailsWith { + reportFiles(model(measures = listOf("source", "sourceColumn"))) + } + val message = e.message.orEmpty() + assertTrue("report \"SaleTotals\"" in message, message) + assertTrue("measure \"source\"" in message && "measure \"sourceColumn\"" in message, message) + assertTrue("\"sourceColumn\"" in message && "Rename one of them" in message, message) + } + + @Test + fun `a dimension and a measure that land on one column property are refused, naming both`() { + val e = assertFailsWith { + reportFiles(model( + measures = listOf("fieldsColumn"), + extraMembers = """{ "dimension.attribute": { "name": "fields", "@of": "Sale.channel" } },""", + dimensions = listOf("fields"), + )) + } + val message = e.message.orEmpty() + assertTrue("dimension \"fields\"" in message && "measure \"fieldsColumn\"" in message, message) + } + + @Test + fun `a time dimension's derived name is never refused`() { + val files = reportFiles(model( + extraMembers = """{ "dimension.time": { "name": "source", "@of": "Sale.soldAt", "@grains": ["day"] } },""", + dimensions = listOf("source:day"), + )) + assertTrue(" val sourceDay = date(\"source_day\")" in files.values.single(), files.values.single()) + } + + @Test + fun `a report that generates no table is not refused for its names`() { + assertEquals(emptyMap(), reportFiles(model(measures = listOf("in", "source", "sourceColumn"), reportSource = null))) + } + + @Test + fun `an enum dimension reached by via is typed by the enum of the entity it reads, and compiles`() { + val json = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Store", "children": [ + { "source.rdb": { "@table": "stores" } }, + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "tier", "@values": ["GOLD", "SILVER"], "@required": true } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "storeId", "@required": true } }, + { "field.enum": { "name": "channel", "@values": ["WEB", "SHOP"], "@intValueMap": { "WEB": 1, "SHOP": 2 }, "@required": true } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "storeRef", "@references": "Store", "@fields": ["storeId"] } }, + { "relationship.association": { "name": "store", "@objectRef": "Store", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "storeTier", "@of": "Store.tier", "@via": "Sale.store" } }, + { "dimension.attribute": { "name": "channel", "@of": "Sale.channel" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "SalesByTier", "@from": "Sale", + "@dimensions": ["storeTier", "channel"], "@measures": ["sales"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_sales_by_tier" } } ] } } + ] } + }""" + val files = emit( + loadString("report-table-via-enum", json), + generators = listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator()), + ) + val src = files.getValue("acme/shop/SalesByTierTable.kt") + // Store's class, and nullable: a dimension reached by @via can be null. + assertTrue( + " val storeTier = enumerationByName(\"store_tier\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, StoreTier::class).nullable()\n" in src, + src, + ) + // An int-backed enum keeps its mapping, typed by Sale's class. + assertTrue(" val channel = customEnumeration(\"channel\", \"INTEGER\", " in src, src) + assertTrue("1 -> SaleChannel.WEB" in src && "SaleChannel.SHOP -> 2" in src, src) + assertFalse("SalesByTier" in src.replace("SalesByTierTable", ""), src) + assertCompiles(files) + } + + // --- References resolve as the loader resolves them ----------------------------------- + + /** `a::Base` (abstract) declares members whose bare `@of` names `Base`. */ + private val sharedBase = """{ + "metadata.root": { "package": "a", "children": [ + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "kind", "@maxLength": 12 } }, + { "field.enum": { "name": "tier", "@values": ["GOLD", "SILVER"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "kind", "@of": "Base.kind" } }, + { "dimension.attribute": { "name": "tier", "@of": "Base.tier" } }, + { "measure.aggregate": { "name": "events", "@agg": "count", "@of": "Base.id" } } + ] } } + ] } + }""" + + /** Package `b`: `Ev extends a::Base` and report `R` over it. [decoy] adds a same-named, + * differently-typed `b::Base` that a package-of-`@from` resolution would capture. */ + private fun evFile(decoy: Boolean, measures: String = """["events"]"""): String { + val decoyNode = if (!decoy) "" else """ + { "object.entity": { "name": "Base", "children": [ + { "field.int": { "name": "id" } }, + { "field.int": { "name": "kind" } }, + { "field.int": { "name": "tier" } } + ] } },""" + return """{ + "metadata.root": { "package": "b", "children": [$decoyNode + { "object.entity": { "name": "Ev", "extends": "a::Base", "children": [ + { "source.rdb": { "@table": "evs" } } + ] } }, + { "object.report": { "name": "R", "@from": "Ev", "@dimensions": ["kind", "tier"], + "@measures": $measures, + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_r" } } ] } } + ] } + }""" + } + + private fun loadFiles(vararg json: String): MetaDataLoader = + MetaDataLoader.createManual(false, "report-table-cross-package").apply { + init() + load(json.mapIndexed { i, text -> + com.metaobjects.loader.InMemoryStringSource( + text, "meta.inline$i.json", com.metaobjects.loader.MetaDataSource.MetaDataFormat.JSON) + }) + assertEquals(emptyList(), errors.map { it.message }) + register() + } + + private fun crossPackageTable(decoy: Boolean, measures: String = """["events"]"""): Map = + emit( + loadFiles(sharedBase, evFile(decoy, measures)), + mapOf("columnNaming" to "literal"), + listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator()), + ) + + private fun assertTypedFromTheDeclaringBase(files: Map) { + val src = files.getValue("b/RTable.kt") + assertTrue(" val kind = varchar(\"kind\", 12).nullable()\n" in src, src) + // The enum class of the @from entity, which is the class the entity generator emits. + assertTrue("EvTier::class).nullable()\n" in src, src) + assertTrue(" val events = long(\"events\")\n" in src, src) + assertCompiles(files.filterKeys { !it.startsWith("b/Base") }) + } + + @Test + fun `a bare of on a member inherited from another package resolves in the declaring entity's package`() { + assertTypedFromTheDeclaringBase(crossPackageTable(decoy = false)) + } + + @Test + fun `a same-named decoy in the report's package does not capture the reference`() { + // b::Base.kind and b::Base.tier are ints: captured, the columns would be integer(...). + assertTypedFromTheDeclaringBase(crossPackageTable(decoy = true)) + } + + @Test + fun `a dotted measures item names the measure by its last segment`() { + val src = crossPackageTable(decoy = false, measures = """["a::Base.events"]""").getValue("b/RTable.kt") + assertTrue(" val events = long(\"events\")\n" in src, src) + val viaFrom = crossPackageTable(decoy = false, measures = """["Ev.events"]""").getValue("b/RTable.kt") + assertEquals(src, viaFrom) + } + + // --- What generates nothing, and what is refused --------------------------------------- + + @Test + fun `an abstract view-backed report generates nothing`() { + // An abstract object gets no table object in this port, report or not. The + // TypeScript, Java and Python runtimes still read the view (docs: Known limits). + val json = model().replace(""""name": "SaleTotals",""", """"name": "SaleTotals", "abstract": true,""") + assertTrue("\"abstract\": true" in json) + assertEquals(emptyMap(), reportFiles(json)) + } + + @Test + fun `a dimension over a field object is refused, naming the report and the dimension`() { + val json = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "city" } } ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "destination", "@of": "Sale.shipTo" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "SaleTotals", "@from": "Sale", "@dimensions": ["destination"], + "@measures": ["sales"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_sale_totals" } } ] } } + ] } + }""" + val e = assertFailsWith { reportFiles(json) } + assertEquals( + "report \"SaleTotals\": its dimension \"destination\" reads \"acme::shop::Sale.shipTo\", a " + + "field.object. A report over a field.object is not supported; group by a scalar field.", + e.message, + ) + // The same report with no view generates nothing and is not refused. + assertEquals(emptyMap(), reportFiles(json.replace( + """"children": [ { "source.rdb": { "@kind": "view", "@view": "v_sale_totals" } } ]""", """"children": []"""))) + } + + // --- The emitted reports build ------------------------------------------------------ + + @Test + fun `the canonical report tables compile beside the entities they reference`() { + val files = emit( + loadDirectory("report-table-compile", canonicalDir()), + mapOf("columnNaming" to "literal", "packageName" to "fitness"), + listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator()), + ) + assertTrue(files.keys.count { it.endsWith("Table.kt") } >= 6, files.keys.toString()) + assertCompiles(files) + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReservedTableMembersTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReservedTableMembersTest.kt new file mode 100644 index 000000000..b3221bea8 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReservedTableMembersTest.kt @@ -0,0 +1,108 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.generator.Generator +import com.metaobjects.metadata.ktx.loadString +import java.nio.file.Files +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * A column property named after a member of Exposed's `Table` gets a `Column` suffix + * ([KotlinNaming.safeColumnProperty]), and WHICH names are members depends on the Exposed + * version the output targets. `options` and `storageParameters` are `Table` properties only + * in Exposed 1.x (checked with `javap` against exposed-core 0.55.0 and 1.3.1), so they are + * reserved only under `exposedApi=1`: under 0.x a property of that name compiles, and + * renaming it would change working generated code. + */ +class KotlinReservedTableMembersTest { + + private val model = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Plan", "children": [ + { "source.rdb": { "@table": "plans" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "options" } }, + { "field.string": { "name": "storageParameters" } }, + { "field.string": { "name": "schemaName" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"], "@generation": "increment" } }, + { "identity.secondary": { "name": "by_options", "@fields": ["options"] } }, + { "measure.aggregate": { "name": "options", "@agg": "count", "@of": "Plan.id" } } + ] } }, + { "object.report": { "name": "PlanTotals", "@from": "Plan", "@measures": ["options"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_plan_totals" } } ] } } + ] } + }""" + + private fun emit(exposedApi: String?, gen: Generator = KotlinExposedTableGenerator()): Map { + val outDir = Files.createTempDirectory("reserved-members-") + try { + val args = mapOf("outputDir" to outDir.toString(), "packageName" to "acme.shop") + + (exposedApi?.let { mapOf("exposedApi" to it) } ?: emptyMap()) + gen.setArgs(args) + gen.execute(loadString("reserved-members", model)) + return Files.walk(outDir).use { s -> + s.filter { it.isRegularFile() }.toList().associate { outDir.relativize(it).toString() to it.readText() } + } + } finally { + outDir.toFile().deleteRecursively() + } + } + + @Test + fun `the helper reserves the 1x-only members in 1x mode alone`() { + for (name in listOf("options", "storageParameters")) { + assertEquals(name, KotlinNaming.safeColumnProperty(name)) + assertEquals(name, KotlinNaming.safeColumnProperty(name, ExposedApi.V0)) + assertEquals(name + "Column", KotlinNaming.safeColumnProperty(name, ExposedApi.V1)) + } + // A member of Table in both versions is reserved in both. + for (api in ExposedApi.values()) { + assertEquals("schemaNameColumn", KotlinNaming.safeColumnProperty("schemaName", api)) + assertEquals("sourceColumn", KotlinNaming.safeColumnProperty("source", api)) + assertEquals("title", KotlinNaming.safeColumnProperty("title", api)) + } + } + + @Test + fun `under Exposed 0x a field named options keeps its property name`() { + for (api in listOf(null, "0")) { + val table = emit(api).getValue("acme/shop/PlanTable.kt") + assertTrue(" val options = text(\"options\").nullable()\n" in table, table) + assertTrue(" val storageParameters = text(\"storage_parameters\").nullable()\n" in table, table) + assertTrue("uniqueIndex(\"by_options\", options)" in table, table) + assertFalse("optionsColumn" in table, table) + assertFalse("storageParametersColumn" in table, table) + // schemaName is a Table member in 0.x too. + assertTrue(" val schemaNameColumn = text(\"schema_name\").nullable()\n" in table, table) + val report = emit(api).getValue("acme/shop/PlanTotalsTable.kt") + assertTrue(" val options = long(\"options\")\n" in report, report) + } + } + + @Test + fun `under Exposed 1x a field named options gets the suffixed property, declared and referenced`() { + val files = emit("1") + val table = files.getValue("acme/shop/PlanTable.kt") + // The physical column keeps its name; only the Kotlin property is renamed. + assertTrue(" val optionsColumn = text(\"options\").nullable()\n" in table, table) + assertTrue(" val storageParametersColumn = text(\"storage_parameters\").nullable()\n" in table, table) + assertTrue("uniqueIndex(\"by_options\", optionsColumn)" in table, table) + assertFalse("val options =" in table, table) + assertTrue(" val schemaNameColumn = text(\"schema_name\").nullable()\n" in table, table) + val report = files.getValue("acme/shop/PlanTotalsTable.kt") + assertTrue(" val optionsColumn = long(\"options\")\n" in report, report) + } + + @Test + fun `the controller's sort dispatch references the property the table declares, per mode`() { + val v0 = emit(null, KotlinSpringControllerGenerator()).values.joinToString("\n") + assertTrue("\"options\" -> PlanTable.options\n" in v0, v0) + val v1 = emit("1", KotlinSpringControllerGenerator()).values.joinToString("\n") + assertTrue("\"options\" -> PlanTable.optionsColumn\n" in v1, v1) + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt index 0eeabc7a9..315c8c569 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt @@ -18,16 +18,20 @@ import kotlin.test.assertFalse import kotlin.test.assertTrue /** - * FR-044 Plan 1 — the reporting vocabulary is INERT in every Kotlin generator. + * FR-044 — what the reporting vocabulary generates in Kotlin, and what stays INERT. * - * Plan 1 registers `dimension.*`, `measure.*`, `segment.*` and `object.report` and validates - * them at load, but gives none of them output: a report's lowering lands in Plan 2/3. Until - * then a model that USES the vocabulary must generate exactly what the same model without it - * generates, byte for byte, through every generator in [GENERATOR_REGISTRY]. + * `dimension.*`, `measure.*` and `segment.*` generate nothing anywhere. An `object.report` + * with no read-only source generates nothing anywhere. A report that declares a read-only + * `source.rdb @kind: view` is lowered to that view (by TypeScript migrate), and Kotlin + * generates exactly one thing for it: its read-only Exposed table, from + * [KotlinExposedTableGenerator]. Every other generator in [GENERATOR_REGISTRY] — entity, + * names, relations, repository, controller, filter allowlist, the docs tier — emits for a + * model that USES the vocabulary exactly what it emits for the same model without it, byte + * for byte. * * The model pair is `fixtures/codegen-noop/reporting/{with,without}`, shared with the other - * four ports' copies of this test. `with/` carries a report that declares a read-only - * `source.rdb @kind: view` (R5 allows one) — the shape that leaked in C#. + * four ports' copies of this test. `with/` carries two sourceless reports + * (`ProgramEngagement`, `DailyRevenue`) and one view-backed one (`StoreTotals`). */ class ReportingInertTest { @@ -82,14 +86,28 @@ class ReportingInertTest { } } - private fun sameOrLeak(label: String, expected: Map, actual: Map): String? { - if (expected.keys.toList() != actual.keys.toList()) { - return "$label: emitted file set ${expected.keys} became ${actual.keys}" + /** + * Null when [actual] is [expected] plus exactly the files in [added] (path to contents, + * empty for a generator that must stay inert); else what leaked. + */ + private fun sameOrLeak( + label: String, + expected: Map, + actual: Map, + added: Map = emptyMap(), + ): String? { + val wanted = TreeMap(expected).apply { putAll(added) } + if (wanted.keys.toList() != actual.keys.toList()) { + return "$label: emitted file set ${wanted.keys} became ${actual.keys}" } - val differing = expected.keys.filter { expected[it] != actual[it] } + val differing = wanted.keys.filter { wanted[it] != actual[it] } return if (differing.isEmpty()) null else "$label: $differing differ once reporting nodes are declared" } + /** What a generator may add for the with-model: the view-backed report's table, and only from `exposed-table`. */ + private fun allowedFor(info: GeneratorInfo): Map = + if (info.name == EXPOSED_TABLE) mapOf(STORE_TOTALS_TABLE_PATH to STORE_TOTALS_TABLE) else emptyMap() + @Test fun `the with-model really carries the vocabulary`() { // Else every comparison below is vacuously green. @@ -102,15 +120,36 @@ class ReportingInertTest { } @Test - fun `every generator emits the same files with and without reporting nodes`() { + fun `only the Exposed table generator emits for a report, and only the view-backed one's table`() { // Every generator is compared before anything is asserted, so one red run names // every leak rather than the first. val leaks = GENERATOR_REGISTRY.values.mapNotNull { info -> - sameOrLeak(info.name, emit("without", listOf(info)), emit("with", listOf(info))) + sameOrLeak(info.name, emit("without", listOf(info)), emit("with", listOf(info)), allowedFor(info)) } assertTrue(leaks.isEmpty(), leaks.joinToString("\n")) } + @Test + fun `the view-backed report emits exactly its Exposed table`() { + // Else the allowance above is vacuous: the table really is emitted, with this content. + val info = GENERATOR_REGISTRY.getValue(EXPOSED_TABLE) + val added = emit("with", listOf(info)) - emit("without", listOf(info)).keys + assertEquals(mapOf(STORE_TOTALS_TABLE_PATH to STORE_TOTALS_TABLE), added) + } + + @Test + fun `a sourceless report appears in no generated file`() { + val files = emit("with", GENERATOR_REGISTRY.values.toList()) + assertFalse(THREW in files, "the combined suite threw: ${files[THREW]}") + for (report in listOf("ProgramEngagement", "DailyRevenue")) { + val hits = files.filter { (path, text) -> report in path || report in text }.keys + assertTrue(hits.isEmpty(), "$report leaked into $hits") + } + // The view-backed report is named by its table and by nothing else. + val hits = files.filter { (path, text) -> "StoreTotals" in path || "StoreTotals" in text }.keys + assertEquals(setOf(STORE_TOTALS_TABLE_PATH), hits) + } + @Test fun `exactly these generators cannot run from a bare model`() { // Each is compared above on its error message alone, which proves nothing about its @@ -126,13 +165,14 @@ class ReportingInertTest { val expected = emit("without", runnable) assertFalse(THREW in expected, "the combined suite threw: ${expected[THREW]}") assertTrue(expected.size > 10, "only ${expected.size} files — the suite barely ran") - val leak = sameOrLeak("combined", expected, emit("with", runnable)) + val leak = sameOrLeak( + "combined", expected, emit("with", runnable), mapOf(STORE_TOTALS_TABLE_PATH to STORE_TOTALS_TABLE)) assertTrue(leak == null, leak) } /** * The api docs surface: every unit page, the index and the agent page. A report has no - * generated API to document, and its derived fields do not exist until its lowering lands. + * generated API to document — no route, repository or DTO — whether or not it has a view. */ private fun apiDocs(variant: String): Map { val model = KotlinApiModelBuilder().build(load(variant), "shop") @@ -157,5 +197,25 @@ class ReportingInertTest { private companion object { const val THREW = "" + + /** The registry id of [KotlinExposedTableGenerator]. */ + const val EXPOSED_TABLE = "exposed-table" + + const val STORE_TOTALS_TABLE_PATH = "acme/shop/StoreTotalsTable.kt" + + /** `StoreTotals`: three measures over `Purchase` — two counts and a sum of a currency. */ + val STORE_TOTALS_TABLE = """ + |package acme.shop + | + |import org.jetbrains.exposed.sql.Table + | + |/** READ-ONLY VIEW — generated from view metadata; do not insert/update/delete directly. */ + |/** GENERATED — do not hand-edit. Regenerated from metadata. */ + |object StoreTotalsTable : Table("v_store_totals") { + | val purchases = long("purchases") + | val buyers = long("buyers") + | val revenue = long("revenue").nullable() + |} + |""".trimMargin() } } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt index 27b35ef5e..73bf6a2a2 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt @@ -104,6 +104,62 @@ internal class KotlinCodegenMatchesReferenceTest { ExpectedColumn("recordedAt", families = setOf("instantWithTimeZone")), ), ), + // FR-044: the six view-backed reports. Each mirrors its hand-written reference + // (`tables/View.kt`) column for column: the family is the view's real column + // type, and a column is nullable exactly when the reference's is. A report has no + // identity, so none carries a primary key. + "ProgramMinutes" to EntityExpectation( + columns = listOf( + ExpectedColumn("program", families = setOf("long"), nullable = false), + ExpectedColumn("programTitle", families = setOf("varchar"), nullable = true), + ExpectedColumn("weeks", families = setOf("long"), nullable = false), + ExpectedColumn("longWeeks", families = setOf("long"), nullable = false), + ExpectedColumn("labels", families = setOf("long"), nullable = false), + ExpectedColumn("slots", families = setOf("long"), nullable = false), + ExpectedColumn("totalMinutes", families = setOf("long"), nullable = true), + ExpectedColumn("avgMinutes", families = setOf("decimal"), nullable = true), + ExpectedColumn("minMinutes", families = setOf("integer"), nullable = true), + ExpectedColumn("maxMinutes", families = setOf("integer"), nullable = true), + ExpectedColumn("longShare", families = setOf("decimal"), nullable = true), + ), + report = "v_program_minutes", + ), + "FitnessTotals" to EntityExpectation( + columns = listOf( + ExpectedColumn("weeks", families = setOf("long"), nullable = false), + ExpectedColumn("totalMinutes", families = setOf("long"), nullable = true), + ExpectedColumn("longShare", families = setOf("decimal"), nullable = true), + ), + report = "v_fitness_totals", + ), + "ProgramsByMonth" to EntityExpectation( + columns = listOf( + ExpectedColumn("createdAtMonth", families = setOf("date"), nullable = false), + ExpectedColumn("status", families = setOf("varchar", "enumerationByName"), nullable = false), + ExpectedColumn("programs", families = setOf("long"), nullable = false), + ExpectedColumn("listValue", families = setOf("long"), nullable = true), + ), + report = "v_programs_by_month", + ), + "ProgramsByWeek" to EntityExpectation( + columns = listOf( + ExpectedColumn("createdAtWeek", families = setOf("date"), nullable = false), + ExpectedColumn("programs", families = setOf("long"), nullable = false), + ), + report = "v_programs_by_week", + ), + "RecentPrograms" to EntityExpectation( + columns = listOf(ExpectedColumn("programs", families = setOf("long"), nullable = false)), + report = "v_recent_programs", + ), + "AssetActivity" to EntityExpectation( + columns = listOf( + ExpectedColumn("recordedAtHour", families = setOf("instantWithTimeZone"), nullable = false), + ExpectedColumn("asOfDateWeek", families = setOf("date"), nullable = false), + ExpectedColumn("assets", families = setOf("long"), nullable = false), + ), + report = "v_asset_activity", + ), ) @Test @@ -124,6 +180,7 @@ internal class KotlinCodegenMatchesReferenceTest { val source = tableFile.readText() assertSourceContainsColumns(entity, source, expected.columns) assertSourceContainsForeignKeys(entity, source, expected.foreignKeys) + expected.report?.let { view -> assertSourceIsExactlyTheReportTable(entity, source, view, expected.columns) } } } finally { outDir.deleteRecursively() @@ -147,9 +204,41 @@ internal class KotlinCodegenMatchesReferenceTest { "${entity}Table.kt: expected `val ${col.name} = <${col.families.joinToString("|")}>(...)`. " + "Source was:\n$source" ) + // Nullability is asserted only where the expectation states it (the reports). + col.nullable?.let { nullable -> + val line = source.lineSequence().first { Regex("""\bval\s+${Regex.escape(col.name)}\s*=""").containsMatchIn(it) } + assertTrue( + line.trimEnd().endsWith(".nullable()") == nullable, + "${entity}Table.kt: expected column '${col.name}' to be ${if (nullable) "nullable" else "non-null"}; saw `${line.trim()}`" + ) + } } } + /** + * A report's table (FR-044) is held tighter than an entity's: it binds [view], declares + * EXACTLY the expected columns in the expected order (the derived fields — dimensions, + * then measures — and nothing else), and has no primary key, because a report has no + * identity. + */ + private fun assertSourceIsExactlyTheReportTable( + entity: String, + source: String, + view: String, + expected: List, + ) { + assertTrue( + "object ${entity}Table : Table(\"$view\")" in source, + "${entity}Table.kt: expected the table to bind the view '$view'; saw:\n$source", + ) + val declared = Regex("""^\s*val\s+(\w+)\s*=""", RegexOption.MULTILINE).findAll(source).map { it.groupValues[1] }.toList() + assertTrue( + declared == expected.map { it.name }, + "${entity}Table.kt: expected exactly the columns ${expected.map { it.name }} in that order; saw $declared", + ) + assertTrue("primaryKey" !in source, "${entity}Table.kt: a report has no identity, so no primaryKey; saw:\n$source") + } + /** * Assert each expected FK is present as a `.references(.id...)` * decoration on the named column. Catches the regression where the generator @@ -173,10 +262,12 @@ internal class KotlinCodegenMatchesReferenceTest { } } - private data class ExpectedColumn(val name: String, val families: Set) + private data class ExpectedColumn(val name: String, val families: Set, val nullable: Boolean? = null) private data class ExpectedFk(val columnName: String, val targetTable: String) private data class EntityExpectation( val columns: List, val foreignKeys: List = emptyList(), + /** For an `object.report` (FR-044): the view its table binds. Null for every other object. */ + val report: String? = null, ) } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index dde93bf71..149c5f8c7 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -16,6 +16,12 @@ import com.metaobjects.integration.kotlin.tables.ProgramStatView import com.metaobjects.integration.kotlin.tables.ProgramTable import com.metaobjects.integration.kotlin.tables.ProgramView import com.metaobjects.integration.kotlin.tables.WeekTable +import com.metaobjects.integration.kotlin.tables.AssetActivityView +import com.metaobjects.integration.kotlin.tables.FitnessTotalsView +import com.metaobjects.integration.kotlin.tables.ProgramMinutesView +import com.metaobjects.integration.kotlin.tables.ProgramsByMonthView +import com.metaobjects.integration.kotlin.tables.ProgramsByWeekView +import com.metaobjects.integration.kotlin.tables.RecentProgramsView import org.jetbrains.exposed.sql.AndOp import org.jetbrains.exposed.sql.Column import org.jetbrains.exposed.sql.Database @@ -484,6 +490,14 @@ object QueryScenarioRunner { "Measurement" -> MeasurementTable "ProgramStat" -> ProgramStatView "ProgramView" -> ProgramView + // FR-044: a view-backed report is read through the view its lowering created. It has + // no primary key, so only `list` and `count` reach these. + "ProgramMinutes" -> ProgramMinutesView + "FitnessTotals" -> FitnessTotalsView + "ProgramsByMonth" -> ProgramsByMonthView + "ProgramsByWeek" -> ProgramsByWeekView + "RecentPrograms" -> RecentProgramsView + "AssetActivity" -> AssetActivityView "Asset" -> AssetTable "AllTypes" -> AllTypesTable // FR-017 TPH: the discriminator base + all its subtypes share the single `auths` table. diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AssetActivityView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AssetActivityView.kt new file mode 100644 index 000000000..599ed3f85 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AssetActivityView.kt @@ -0,0 +1,27 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.javatime.date + +/** + * Hand-written reference Exposed Table mapping the `AssetActivity` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_asset_activity`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * - `recordedAtHour` = the hour bucket of Asset.recordedAt, an instant → TIMESTAMPTZ → + * [instantWithTimeZone], the same `Column` [AssetTable] reads the base + * column with (so it normalizes to the `…Z` wire form) + * - `asOfDateWeek` = the ISO week bucket of Asset.asOfDate, a date → DATE → `date` + * - `assets` = a count → BIGINT → `long` + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object AssetActivityView : Table("v_asset_activity") { + val recordedAtHour = instantWithTimeZone("recordedAtHour") + val asOfDateWeek = date("asOfDateWeek") + val assets = long("assets") +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsView.kt new file mode 100644 index 000000000..4fb81819f --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsView.kt @@ -0,0 +1,24 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `FitnessTotals` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_fitness_totals`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * No dimensions, so the view returns exactly one row, over an empty table too: `weeks` + * (a count, BIGINT) is then `0`, and `totalMinutes` (a sum, BIGINT) and `longShare` (a ratio, + * NUMERIC) are NULL — hence nullable. + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object FitnessTotalsView : Table("v_fitness_totals") { + val weeks = long("weeks") + val totalMinutes = long("totalMinutes").nullable() + val longShare = decimal("longShare", 38, 18).nullable() +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramMinutesView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramMinutesView.kt new file mode 100644 index 000000000..2fd7108eb --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramMinutesView.kt @@ -0,0 +1,40 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `ProgramMinutes` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_program_minutes`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * One column per derived field (dimensions, then measures), each mirroring the view's REAL + * column type and nullable exactly where `fixtures/persistence-conformance/report-shapes.json` + * says `required: false`: + * - `program` = Week.programId (required FK) → BIGINT → `long` + * - `programTitle` = Program.title, reached by @via → VARCHAR → `varchar`, nullable + * - a `count`, with or without @distinct → BIGINT → `long` (never null) + * - `sum` of an int, cast by the view → BIGINT → `long`, nullable + * - `avg`, and a ratio → NUMERIC → `decimal`, nullable + * - `min` / `max` of an int → INTEGER → `integer`, nullable + * The decimal precision and scale never reach DDL (this maps a view); they are the scale + * Exposed reads the unconstrained NUMERIC back at. + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object ProgramMinutesView : Table("v_program_minutes") { + val program = long("program") + val programTitle = varchar("programTitle", 200).nullable() + val weeks = long("weeks") + val longWeeks = long("longWeeks") + val labels = long("labels") + val slots = long("slots") + val totalMinutes = long("totalMinutes").nullable() + val avgMinutes = decimal("avgMinutes", 38, 18).nullable() + val minMinutes = integer("minMinutes").nullable() + val maxMinutes = integer("maxMinutes").nullable() + val longShare = decimal("longShare", 38, 18).nullable() +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByMonthView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByMonthView.kt new file mode 100644 index 000000000..3ed7d789e --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByMonthView.kt @@ -0,0 +1,30 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.javatime.date + +/** + * Hand-written reference Exposed Table mapping the `ProgramsByMonth` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_programs_by_month`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * - `createdAtMonth` = the month bucket of Program.createdAt → DATE (the first day of the + * month) → `date` + * - `status` = Program.status, an enum → VARCHAR; read as its member symbol, the + * way [ProgramTable] reads the base column + * - `programs` = a count → BIGINT → `long` + * - `listValue` = a filtered sum of a currency → BIGINT minor units → `long`, nullable + * (NULL when no row in the group matches the measure's filter) + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object ProgramsByMonthView : Table("v_programs_by_month") { + val createdAtMonth = date("createdAtMonth") + val status = varchar("status", 64) + val programs = long("programs") + val listValue = long("listValue").nullable() +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByWeekView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByWeekView.kt new file mode 100644 index 000000000..441b67d15 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramsByWeekView.kt @@ -0,0 +1,24 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table +import org.jetbrains.exposed.sql.javatime.date + +/** + * Hand-written reference Exposed Table mapping the `ProgramsByWeek` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_programs_by_week`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * - `createdAtWeek` = the ISO week bucket of Program.createdAt → DATE (the Monday that + * starts the week) → `date` + * - `programs` = a count → BIGINT → `long` + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object ProgramsByWeekView : Table("v_programs_by_week") { + val createdAtWeek = date("createdAtWeek") + val programs = long("programs") +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/RecentProgramsView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/RecentProgramsView.kt new file mode 100644 index 000000000..6b02e2419 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/RecentProgramsView.kt @@ -0,0 +1,21 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `RecentPrograms` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_recent_programs`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * One measure and no dimensions: a single row holding a count (BIGINT → `long`) of the + * programs created in the last 30 days, evaluated when the view is queried. + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object RecentProgramsView : Table("v_recent_programs") { + val programs = long("programs") +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/ObjectManagerDbAdapter.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ObjectManagerDbAdapter.java index 9af4b38d5..f3d77b8f6 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/ObjectManagerDbAdapter.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ObjectManagerDbAdapter.java @@ -81,8 +81,12 @@ static Object execute(ObjectManagerDB omdb, ObjectConnection conn, MetaObject mc if (spec.limit() != null) opts.setRange(buildRange(spec.offset(), spec.limit())); Collection raw = omdb.getObjects(conn, mc, opts); + // FR-044: a report's rows are instances of its read model (one field per derived + // field); the declared report node has no fields to walk. Any other object is its + // own read object. + MetaObject rowMeta = omdb.readObjectFor(mc); List> rows = new ArrayList<>(raw.size()); - for (Object o : raw) rows.add(toRowMap(mc, o, columnSqlTypes)); + for (Object o : raw) rows.add(toRowMap(rowMeta, o, columnSqlTypes)); if ("get".equals(spec.op())) return rows.isEmpty() ? null : rows.get(0); return rows; // op:list diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java index 1f1bc4db2..2078e59d5 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java @@ -7,6 +7,7 @@ import com.metaobjects.manager.db.ObjectManagerDB; import com.metaobjects.manager.db.driver.PostgresDriver; import com.metaobjects.object.MetaObject; +import com.metaobjects.reporting.ReportReadModel; import javax.sql.DataSource; import java.io.PrintWriter; @@ -129,7 +130,11 @@ private static Object dispatch(ObjectManagerDB omdb, ObjectConnection oc, MetaOb * {@link ResultSetMetaData}. Returns an empty map for a non-persistent object. */ private static Map probeColumnSqlTypes(PostgresContainer pg, MetaObject mc) { - String relation = mc.getPrimaryRdbViewName(); + // FR-044: a report's relation is its read model's view (named by the source's + // kind-matching @view alias); the declared node carries no fields to key the probe by. + String relation = ReportReadModel.isReport(mc) + ? ReportReadModel.of(mc).viewName() + : mc.getPrimaryRdbViewName(); if (relation == null) relation = mc.getPrimaryRdbTableName(); if (relation == null) return Map.of(); Map types = new LinkedHashMap<>(); diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index d9db53618..a782a4e4a 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -4187,7 +4187,7 @@ private static WalkedViaPath validateViaPath(String viaAttr, MetaRoot root, * * @param referrerPkg the effective package of the node carrying the ref ("" for root-level) */ - static MetaObject resolveRootObject(MetaRoot root, String ref, String referrerPkg) { + public static MetaObject resolveRootObject(MetaRoot root, String ref, String referrerPkg) { if (ref == null) return null; String pkg = (referrerPkg == null) ? "" : referrerPkg; if (ref.indexOf(MetaData.PKG_SEPARATOR) >= 0) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/object/ReportMetaObject.java b/server/java/metadata/src/main/java/com/metaobjects/object/ReportMetaObject.java index fc869c2be..a5d3f3562 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/object/ReportMetaObject.java +++ b/server/java/metadata/src/main/java/com/metaobjects/object/ReportMetaObject.java @@ -23,8 +23,10 @@ * derived from {@code @dimensions} and {@code @measures}, never declared; its rules * (R1-R7) are enforced by the loader's reporting validation pass. * - *

Registration + canonical serialization only in this plan; the view lowering and - * the generated read surface arrive with FR-044 Plan 2.

+ *

The declared node carries no field children. Its read shape is derived by + * {@link com.metaobjects.reporting.ReportShape} (contract Table B), and a runtime reads it + * through {@link com.metaobjects.reporting.ReportReadModel}. The view itself is lowered by + * the TypeScript toolchain only (ADR-0015); the Java generators emit nothing for a report.

*/ @SuppressWarnings("serial") public class ReportMetaObject extends AbstractObjectRepresentation { diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java index e7a3fe089..b86065302 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java @@ -16,6 +16,7 @@ package com.metaobjects.reporting; import com.metaobjects.MetaData; +import com.metaobjects.util.MetaDataUtil; import com.metaobjects.object.MetaObject; import java.util.ArrayList; @@ -56,11 +57,31 @@ public static List reportDimensionItems(MetaData report) { return out; } - /** The {@code @measures} names. */ + /** The {@code @measures} items AS WRITTEN: each a bare measure {@code name}, or a dotted + * {@code Entity.name} (loader rule R3). Use {@link #reportMeasureItemName} for the measure name. */ public static List reportMeasureNames(MetaData report) { return ReportingAttrs.stringList(report, MetaObject.ATTR_REPORT_MEASURES); } + /** + * The measure a {@code @measures} item names: the segment after its LAST {@code .} + * ({@code total}, {@code Sale.total} and {@code acme::shop::Sale.total} all name + * {@code total}). It is also the derived report field's name. The part before that + * {@code .}, when present, is an entity qualifier ({@link #reportMeasureItemOwner}). + */ + public static String reportMeasureItemName(String item) { + int dot = item.lastIndexOf(MetaDataUtil.CHILD_REF_SEPARATOR); + return dot == -1 ? item : item.substring(dot + MetaDataUtil.CHILD_REF_SEPARATOR.length()); + } + + /** The entity qualifier of a dotted {@code @measures} item ({@code Sale} in + * {@code Sale.total}), or {@code null} for a bare item. Loader rule R3: it names + * {@code @from} or an entity {@code @from} extends. */ + public static String reportMeasureItemOwner(String item) { + int dot = item.lastIndexOf(MetaDataUtil.CHILD_REF_SEPARATOR); + return dot == -1 ? null : item.substring(0, dot); + } + /** The derived report field for a dimension item: {@code name} (attribute) or * {@code name + Capitalized(grain)} (time), e.g. {@code purchasedAtDay}. */ public static String reportDerivedFieldName(ReportDimensionItem item) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportReadModel.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportReadModel.java new file mode 100644 index 000000000..e5ae0ec81 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportReadModel.java @@ -0,0 +1,264 @@ +/* + * Copyright 2026 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.reporting; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaDataException; +import com.metaobjects.MetaRoot; +import com.metaobjects.attr.BooleanAttribute; +import com.metaobjects.attr.MetaAttribute; +import com.metaobjects.attr.StringAttribute; +import com.metaobjects.database.CoreDBMetaDataProvider; +import com.metaobjects.field.CurrencyField; +import com.metaobjects.field.DateField; +import com.metaobjects.field.DecimalField; +import com.metaobjects.field.DoubleField; +import com.metaobjects.field.EnumField; +import com.metaobjects.field.LongField; +import com.metaobjects.field.MetaField; +import com.metaobjects.field.ObjectField; +import com.metaobjects.object.MetaObject; +import com.metaobjects.object.ReportMetaObject; +import com.metaobjects.source.MetaSource; + +import java.util.List; + +/** + * A report's READ MODEL (FR-044): a detached {@code object.report} node carrying one real + * {@code field.*} child per derived field ({@link ReportShape}, contract Table B) and a + * copy of the source the report is read from. + * + *

Why it exists

+ * An {@code object.report} declares no fields: its read shape is derived from its + * dimensions and measures. A metadata-driven runtime walks an object's field children + * everywhere (column mapping, filter and sort resolution, instance construction, every + * read codec). Rather than teach each of those what a report is, a runtime reads a report + * through this model and sees ordinary fields. + * + *

Why it is detached

+ * The model is never added to the root: it has no parent, the loader does not list it, + * and the canonical serializer, {@code fmt}, codegen and every other tree walker never see + * it. Nothing in the loaded tree is mutated to build it — the type-shaping attrs and the + * source are COPIED (attrs only), never re-parented ({@code addChild} rewrites a child's + * parent). Nodes are constructed directly, not through the loader, and no vocabulary is + * added: every node is an already-registered {@code type.subType}. + * + *

It keeps the report's name, package and {@code object.report} subtype, so a consumer + * holding it can still tell it is a report (no identity, read-only). It is its own class so + * it can be told apart from the declared node, which has the same name and subtype.

+ * + *

Mirrors the TypeScript {@code report-read-model.ts}.

+ */ +@SuppressWarnings("serial") +public final class ReportReadModel extends ReportMetaObject { + + private static final String CACHE_KEY = "ReportReadModel.of()"; + + /** + * Table B: the type-shaping attrs a derived field carries from its type source, read + * with the RESOLVING accessor (ADR-0039) so a value the {@code @of} field inherits + * through {@code extends} is carried too. {@code @dbColumnType} and array-ness are + * handled separately. Nothing else is carried: no {@code @column}, {@code @required}, + * {@code @default}, validators or views. + */ + private static final List CARRIED_ATTRS = List.of( + CurrencyField.ATTR_CURRENCY, + EnumField.ATTR_VALUES, + EnumField.ATTR_INT_VALUE_MAP, + MetaField.ATTR_MAX_LENGTH, + MetaField.ATTR_PRECISION, + MetaField.ATTR_SCALE, + CoreDBMetaDataProvider.LOCAL_TIME, + MetaField.ATTR_OBJECT_REF, + MetaField.ATTR_STORAGE); + + /** The declared report this model was built from; {@code null} only on a bare instance. */ + private transient MetaObject report; + + /** + * Constructs an EMPTY model node. Public only because the node contract requires a + * {@code (String name)} constructor ({@link MetaData#clone()}); obtain a model with + * {@link #of(MetaObject)}. + */ + public ReportReadModel(String name) { + super(name); + } + + /** True for an {@code object.report}: the declared node or its read model. */ + public static boolean isReport(MetaObject object) { + return object != null && MetaObject.SUBTYPE_REPORT.equals(object.getSubType()); + } + + /** + * The read model of a report attached to a loaded model; the root is found from the + * report. Passing a read model returns it unchanged. + * + * @throws MetaDataException what {@link ReportShape#of(MetaObject)} throws + */ + public static ReportReadModel of(MetaObject report) { + if (report instanceof ReportReadModel) return (ReportReadModel) report; + return report.useFrozenCache(CACHE_KEY, () -> build(ReportShape.of(report))); + } + + /** + * The read model of an {@code object.report}: one field per Table B row, in Table B + * order, plus a copy of the source the report is read from + * ({@link ReportShape#readSource(MetaObject)}) when it declares one. A sourceless report + * yields a model with no source: it has a shape and no view, and the caller decides + * what that means ({@link #isServed()}). + * + *

Cached on the report node once the loaded tree is frozen (the loader freezes it + * when the load completes), so a report has one model for its lifetime; before that + * each call builds a fresh, equal model, so nothing derived from a still-mutable tree is + * served stale.

+ * + * @throws MetaDataException what {@link ReportShape#of(MetaObject, MetaRoot)} throws + */ + public static ReportReadModel of(MetaObject report, MetaRoot root) { + if (report instanceof ReportReadModel) return (ReportReadModel) report; + return report.useFrozenCache(CACHE_KEY, () -> build(ReportShape.of(report, root))); + } + + /** The declared {@code object.report} node this model reads. */ + public MetaObject report() { + return report; + } + + /** True when the report has a view to read (Table A); a sourceless report is not served. */ + public boolean isServed() { + // ADR-0039: own — findPrimaryReadOnlySource() reads getSources(false). Sanctioned: + // the model's sources are exactly the one copy build() added (pinned to primary); + // the model extends nothing, so there is no inherited layer to drop. + return findPrimaryReadOnlySource().isPresent(); + } + + /** + * The physical name of the view the model is read from, or {@code null} when the + * report is not served. Resolved through the source's kind-matching alias + * ({@code @view} for a view), so a report is read under the name the lowering created. + */ + public String viewName() { + // ADR-0039: own — as isServed(): the model's one source is its own copy. + return findPrimaryReadOnlySource().map(MetaSource::getPhysicalName).orElse(null); + } + + private static ReportReadModel build(ReportShape shape) { + MetaObject report = shape.report(); + // The resolution key carries the package, so the model resolves as the report does. + ReportReadModel model = new ReportReadModel(report.getName()); + model.report = report; + for (ReportShape.Field f : shape.fields()) { + refuseObjectField(report, f); + model.addChild(derivedField(f)); + } + + MetaSource source = ReportShape.readSource(report); + if (source != null) model.addChild(copySource(source)); + + model.freeze(); + return model; + } + + /** + * Refuse a derived field typed by a {@code field.object} (or by any field carrying + * {@code @objectRef}). The loader puts no subtype restriction on a dimension's + * {@code @of}, so such a report loads; but the derived field is a detached node, and an + * {@code @objectRef} on it cannot be resolved (there is no loader to resolve it in), so + * a read would fail deep in the codec with no report named. Refused here, by name. + */ + private static void refuseObjectField(MetaObject report, ReportShape.Field f) { + MetaField src = f.typeSource(); + if (src == null) return; + // ADR-0039: resolving — an @objectRef the @of field inherits counts. + if (!ObjectField.SUBTYPE_OBJECT.equals(src.getSubType()) && !src.hasMetaAttr(MetaField.ATTR_OBJECT_REF)) return; + throw new MetaDataException("report '" + report.getShortName() + "': " + f.role().wireName() + " '" + + (f.dimension() != null ? f.dimension().getShortName() : f.name()) + "' reads '" + f.typeSourceKey() + + "', a field." + src.getSubType() + ". A report over a field.object is not supported;" + + " group by a scalar field."); + } + + private static MetaField derivedField(ReportShape.Field f) { + MetaField field = newField(f); + // From the derived shape, never from the type source: a `min` of a required column + // is still nullable, and a dimension reached by @via is nullable. + field.addMetaAttr(BooleanAttribute.create(MetaField.ATTR_REQUIRED, f.required())); + MetaField src = f.typeSource(); + if (src == null) return field; + + for (String name : CARRIED_ATTRS) { + if (src.hasMetaAttr(name)) field.addMetaAttr(copyAttr(src.getMetaAttr(name))); + } + // ADR-0039: own — @dbColumnType is the one deliberately own-only attr (a physical + // column-type override is never inherited). So it is read own from the type source: + // the derived field carries exactly what the @of field itself declares, and nothing + // its supers declare. + if (src.hasMetaAttr(CoreDBMetaDataProvider.DB_COLUMN_TYPE, false)) { + field.addMetaAttr(copyAttr(src.getMetaAttr(CoreDBMetaDataProvider.DB_COLUMN_TYPE, false))); + } + // Array-ness is a native flag, not an attr; isArrayType() is its resolving read. + if (src.isArrayType()) field.setArray(true); + return field; + } + + /** + * A new, parentless field node of the derived subtype. A derived field that has a type + * source always has that source's subtype (Table B), so it is built as the same node + * class; the rows with no type source produce one of four fixed subtypes. + */ + @SuppressWarnings({"unchecked", "rawtypes"}) + private static MetaField newField(ReportShape.Field f) { + MetaField src = f.typeSource(); + if (src != null && f.subType().equals(src.getSubType())) { + return (MetaField) src.newInstanceFromClass((Class) src.getClass(), MetaField.TYPE_FIELD, f.subType(), f.name()); + } + switch (f.subType()) { + case LongField.SUBTYPE_LONG: return new LongField(f.name()); + case DecimalField.SUBTYPE_DECIMAL: return new DecimalField(f.name()); + case DoubleField.SUBTYPE_DOUBLE: return new DoubleField(f.name()); + case DateField.SUBTYPE_DATE: return new DateField(f.name()); + default: + throw new MetaDataException("report read model: derived field '" + f.name() + + "' has subtype field." + f.subType() + ", which Table B does not derive without a type source."); + } + } + + /** + * A detached copy of a source node: same node class, name and effective attrs, and + * nothing else (attrs only — the loaded node is never re-parented). + * + *

The copy is the model's ONLY source, and it is pinned to {@code @role: primary}: + * a runtime resolves an object's relation through its primary source, so this is what + * makes the read land on the selected source's physical name rather than on a default + * table name nobody declared.

+ */ + @SuppressWarnings({"unchecked", "rawtypes"}) + private static MetaSource copySource(MetaSource source) { + MetaSource copy = (MetaSource) source.newInstanceFromClass( + (Class) source.getClass(), source.getType(), source.getSubType(), source.getName()); + // ADR-0039: resolving — the copy carries the source's effective configuration + // (@kind, the physical-name alias, @schema, @unmanaged, @sql). + for (MetaAttribute attr : (List) source.getMetaAttrs()) { + if (!MetaSource.ATTR_ROLE.equals(attr.getShortName())) copy.addMetaAttr(copyAttr(attr)); + } + copy.addMetaAttr(StringAttribute.create(MetaSource.ATTR_ROLE, MetaSource.ROLE_PRIMARY)); + return copy; + } + + /** A parentless copy of an attr node (same class, name and value). */ + private static MetaAttribute copyAttr(MetaAttribute attr) { + return (MetaAttribute) attr.clone(); + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java new file mode 100644 index 000000000..a0c6f66bc --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java @@ -0,0 +1,389 @@ +/* + * Copyright 2026 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.reporting; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaDataException; +import com.metaobjects.MetaRoot; +import com.metaobjects.field.CurrencyField; +import com.metaobjects.field.DateField; +import com.metaobjects.field.DecimalField; +import com.metaobjects.field.DoubleField; +import com.metaobjects.field.FloatField; +import com.metaobjects.field.IntegerField; +import com.metaobjects.field.LongField; +import com.metaobjects.field.MetaField; +import com.metaobjects.field.TimestampField; +import com.metaobjects.loader.ValidationPhase; +import com.metaobjects.object.MetaObject; +import com.metaobjects.reporting.ReportAccessors.ReportDimensionItem; +import com.metaobjects.source.MetaSource; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.IdentityHashMap; +import java.util.List; +import java.util.Set; + +/** + * A report's derived fields (FR-044, contract Table B): the read shape of an + * {@code object.report}, which declares no fields of its own. One field per + * {@code @dimensions} item in listed order, then one per {@code @measures} item in + * listed order. + * + *

The single definition in the JVM ports — the Kotlin generators consume this class + * rather than restating the table. Rule-for-rule the TypeScript {@code report-shape.ts}; + * gated by {@code fixtures/persistence-conformance/report-shapes.json}, which every port + * byte-matches. Every read is RESOLVING (ADR-0039) unless a comment says otherwise.

+ * + *

Pure metadata: nothing here touches a database, emits SQL (ADR-0015) or mutates the + * loaded tree.

+ */ +public final class ReportShape { + + /** Whether a derived field comes from a dimension or from a measure. */ + public enum Role { + DIMENSION("dimension"), + MEASURE("measure"); + + private final String wireName; + + Role(String wireName) { + this.wireName = wireName; + } + + /** The role as the shapes artifact spells it. */ + public String wireName() { + return wireName; + } + } + + /** + * One derived field. + * + * @param name the derived field name: the dimension name, {@code } + * for a time dimension, or the measure name. The physical column is the + * naming strategy applied to THIS name; an {@code @column} on the + * {@code @of} field is never inherited + * @param role dimension or measure + * @param subType a field subtype name ({@code long}, {@code decimal}, {@code date}, …) + * @param required whether the column can never be null + * @param typeSource the {@code @of} field whose type-shaping attrs the derived field + * carries ({@code @currency}, {@code @values}, {@code @intValueMap}, + * {@code @maxLength}, {@code @precision}, {@code @scale}, + * {@code @localTime}, {@code @objectRef}, {@code @storage}, + * {@code @dbColumnType} and array-ness), or {@code null} when it carries none + * @param dimension the dimension node, for a dimension field; else {@code null} + * @param grain the time grain, for a time dimension; else {@code null} + * @param measure the measure node, for a measure field; else {@code null} + */ + public record Field(String name, Role role, String subType, boolean required, MetaField typeSource, + MetaDimension dimension, String grain, MetaMeasure measure) { + + /** + * {@code .}, + * or {@code null} without a type source. The form the shapes artifact records. + */ + public String typeSourceKey() { + if (typeSource == null) return null; + // The parent of a field is the object that declares it — for a field the + // @of entity inherits through extends, that is the base, not the @of entity. + MetaData owner = typeSource.getParent(); + if (owner == null) { + throw new MetaDataException("field '" + typeSource.getName() + "' has no owning entity."); + } + return owner.getName() + SEP + typeSource.getName(); + } + } + + /** The member separator of an {@code Entity.field} reference. */ + private static final String SEP = "."; + + private static final Set SUM_LONG = Set.of(IntegerField.SUBTYPE_INT, LongField.SUBTYPE_LONG); + private static final Set FLOATING = Set.of(DoubleField.SUBTYPE_DOUBLE, FloatField.SUBTYPE_FLOAT); + + private final MetaObject report; + private final MetaObject from; + private final MetaRoot root; + private final List fields; + + private ReportShape(MetaObject report, MetaObject from, MetaRoot root, List fields) { + this.report = report; + this.from = from; + this.root = root; + this.fields = Collections.unmodifiableList(fields); + } + + /** The {@code object.report} node this shape was derived from. */ + public MetaObject report() { + return report; + } + + /** The {@code @from} entity. */ + public MetaObject from() { + return from; + } + + /** The derived fields, dimensions then measures, each in listed order. */ + public List fields() { + return fields; + } + + /** + * The entity a derived field's {@code @of} field is READ from, by the rule that derived + * the field ({@link #resolveFieldRef}): the {@code @from} entity for a measure or a + * dimension without {@code @via}, and the entity the {@code @of} reference names for a + * dimension with {@code @via}. {@code null} when that entity does not resolve, which a + * shape derived from a loaded model cannot reach. + * + *

It is the entity whose generated artifacts describe the field (a Kotlin enum class, + * say), which for an inherited field is not the object that declares it.

+ */ + public MetaObject ofEntity(Field field) { + MetaDimension dim = field.dimension(); + if (dim == null || dim.getVia() == null) return from; + return resolveFieldRefEntity(dim.getOf(), memberOwner(dim, from), root); + } + + /** + * The physical name of the view the report is read from, or {@code null} when the + * report declares no read-only source (Table A: not lowered, not served). + */ + public String viewName() { + MetaSource source = readSource(report); + return source == null ? null : source.getPhysicalName(); + } + + /** + * The source a report is READ from: its own read-only source with {@code @role: primary}, + * else its first own read-only source; {@code null} when it declares none. + * + *

This is the rule that NAMES the lowered view in the TypeScript toolchain + * ({@code viewName} / {@code projectionViewSource}). It is restated here because no port + * but TypeScript lowers a report; the two must stay the same rule, or a runtime reads a + * relation the lowering did not create. For every model that loads, the primary branch + * fires (a report whose sources include no primary is refused at load); the fallback + * covers a tree built in code.

+ */ + public static MetaSource readSource(MetaObject report) { + MetaSource first = null; + // ADR-0039: own — source classification reads the sources the report declares + // ITSELF (getSources(false)), exactly as the lowering does. A report inherits no source. + for (MetaSource source : report.getSources(false)) { + if (!source.isReadOnly()) continue; + if (MetaSource.ROLE_PRIMARY.equals(source.getRole())) return source; + if (first == null) first = source; + } + return first; + } + + /** + * Table B for a report attached to a loaded model; the root is found from the report. + * + * @throws MetaDataException naming the report, when it is not under a root or a + * reference does not resolve + */ + public static ReportShape of(MetaObject report) { + return of(report, rootOf(report)); + } + + /** + * Table B. + * + * @param report an {@code object.report} + * @param root the model its references resolve in + * @throws MetaDataException naming the report, when a reference does not resolve (a + * report that passed the loader's reporting validation always resolves) + */ + public static ReportShape of(MetaObject report, MetaRoot root) { + String fromName = ReportAccessors.reportFrom(report); + if (fromName == null) throw unresolved(report, "@from"); + MetaObject from = ValidationPhase.resolveRootObject(root, fromName, packageOf(report)); + if (from == null) throw unresolved(report, "@from '" + fromName + "'"); + + List fields = new ArrayList<>(); + for (ReportDimensionItem item : ReportAccessors.reportDimensionItems(report)) { + fields.add(dimensionField(item, from, root, report)); + } + for (String item : ReportAccessors.reportMeasureNames(report)) { + fields.add(measureField(item, from, root, report)); + } + return new ReportShape(report, from, root, fields); + } + + /** + * The entity that DECLARES a dimension or measure reached through {@code from}: the + * member's parent, which is {@code from} itself or an entity {@code from} extends. A bare + * entity name inside the member ({@code @of}, {@code @via}) resolves in THIS entity's + * package, exactly as the loader's reporting validation resolves it + * ({@code pkgOf(ctx.declaring())}), never in {@code from}'s package or the report's. + */ + public static MetaData memberOwner(MetaData member, MetaObject from) { + MetaData parent = member.getParent(); + return parent != null ? parent : from; + } + + /** + * Resolve a dimension's or measure's {@code Entity.field} reference to the field node, + * or {@code null}. The ONE rule, the same as the loader's (reporting validation D1 / M1) + * and as the TypeScript {@code resolveReportingFieldRef}: + * + *
    + *
  1. A package qualifier uses {@code ::}, so the member separator is the LAST dot. + * The entity half resolves relative to the package of {@code declaring}, the + * entity that declares the member ({@link #memberOwner}; ADR-0042).
    2. + *
  2. With {@code host} (a measure, or a dimension without {@code @via}: the reference + * is about the {@code @from} entity's own rows) the named entity must be + * {@code host} or an entity it extends, and the field is read from {@code host}, + * so a field {@code host} redeclares wins.
    3. + *
  3. Without {@code host} ({@code null}: a dimension with {@code @via}) the field is + * read from the named entity.
    4. + *
+ */ + public static MetaField resolveFieldRef(String ref, MetaData declaring, MetaRoot root, MetaObject host) { + MetaObject named = resolveFieldRefEntity(ref, declaring, root); + if (named == null) return null; + if (host != null && !isSelfOrAncestor(named, host)) return null; + String fieldName = ref.substring(ref.lastIndexOf(SEP) + SEP.length()); + // ADR-0039: resolving, so a field inherited through extends is found. + for (MetaField f : (host != null ? host : named).getMetaFields()) { + if (fieldName.equals(f.getName())) return f; + } + return null; + } + + /** + * The entity an {@code Entity.field} reference NAMES, or {@code null}: the entity half + * of {@link #resolveFieldRef}, resolved relative to the package of {@code declaring} + * (the entity that declares the dimension or measure carrying the reference). It is the + * entity the reference is written against, which for an inherited field is not the + * object that declares it. + */ + public static MetaObject resolveFieldRefEntity(String ref, MetaData declaring, MetaRoot root) { + if (ref == null) return null; + int dot = ref.lastIndexOf(SEP); + if (dot <= 0) return null; + return ValidationPhase.resolveRootObject(root, ref.substring(0, dot), packageOf(declaring)); + } + + /** True when {@code candidate} is {@code entity} or an entity it extends (the super chain). */ + private static boolean isSelfOrAncestor(MetaData candidate, MetaData entity) { + Set visited = Collections.newSetFromMap(new IdentityHashMap<>()); + for (MetaData n = entity; n != null && !visited.contains(n); n = n.getSuperData()) { + if (n == candidate) return true; + visited.add(n); + } + return false; + } + + private static Field dimensionField(ReportDimensionItem item, MetaObject from, MetaRoot root, MetaObject report) { + MetaDimension dim = declaredMember(from, MetaDimension.class, item.name()); + if (dim == null) throw unresolved(report, "dimension '" + item.name() + "' on '" + from.getShortName() + "'"); + boolean vialess = dim.getVia() == null; + MetaField of = dim.getOf() == null ? null + : resolveFieldRef(dim.getOf(), memberOwner(dim, from), root, vialess ? from : null); + if (of == null) throw unresolved(report, "dimension '" + item.name() + "' @of"); + + String name = ReportAccessors.reportDerivedFieldName(item); + // Attr only: a validator.required child does not make the column non-null. + boolean required = vialess && ReportingAttrs.isTrue(of, MetaField.ATTR_REQUIRED); + if (dim.isTime()) { + // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. + String grain = item.grain(); + if (grain == null || !ReportingConstants.TIME_GRAINS.contains(grain)) { + throw unresolved(report, "time dimension '" + item.name() + "' grain '" + (grain == null ? "" : grain) + "'"); + } + if (ReportingConstants.GRAIN_HOUR.equals(grain)) { + return new Field(name, Role.DIMENSION, TimestampField.SUBTYPE_TIMESTAMP, required, of, dim, grain, null); + } + // day / week / month / quarter / year: the first day of the bucket. + return new Field(name, Role.DIMENSION, DateField.SUBTYPE_DATE, required, null, dim, grain, null); + } + return new Field(name, Role.DIMENSION, of.getSubType(), required, of, dim, null, null); + } + + /** + * One {@code @measures} item, bare ({@code total}) or dotted ({@code Sale.total}, loader + * rule R3). The measure is named by the item's last segment and looked up on + * {@code from}; a qualifier resolves in the REPORT's package and must be {@code from} or + * an entity {@code from} extends. + */ + private static Field measureField(String item, MetaObject from, MetaRoot root, MetaObject report) { + String name = ReportAccessors.reportMeasureItemName(item); + String qualifier = ReportAccessors.reportMeasureItemOwner(item); + if (qualifier != null) { + MetaObject owner = ValidationPhase.resolveRootObject(root, qualifier, packageOf(report)); + if (owner == null || !isSelfOrAncestor(owner, from)) { + throw unresolved(report, "measure '" + item + "' on '" + from.getShortName() + "'"); + } + } + MetaMeasure m = declaredMember(from, MetaMeasure.class, name); + if (m == null) throw unresolved(report, "measure '" + item + "' on '" + from.getShortName() + "'"); + if (m.isRatio()) { + return new Field(name, Role.MEASURE, DecimalField.SUBTYPE_DECIMAL, false, null, null, null, m); + } + String agg = m.getAgg(); + if (ReportingConstants.AGG_COUNT.equals(agg)) { + // A count is never null, with or without @distinct. + return new Field(name, Role.MEASURE, LongField.SUBTYPE_LONG, true, null, null, null, m); + } + List columns = m.getOfColumns(); + MetaField of = columns.isEmpty() ? null + : resolveFieldRef(columns.get(0), memberOwner(m, from), root, from); + if (of == null) throw unresolved(report, "measure '" + name + "' @of"); + String src = of.getSubType(); + if (ReportingConstants.AGG_SUM.equals(agg)) { + if (CurrencyField.SUBTYPE_CURRENCY.equals(src)) { + return new Field(name, Role.MEASURE, CurrencyField.SUBTYPE_CURRENCY, false, of, null, null, m); + } + String subType = SUM_LONG.contains(src) ? LongField.SUBTYPE_LONG + : FLOATING.contains(src) ? DoubleField.SUBTYPE_DOUBLE + : DecimalField.SUBTYPE_DECIMAL; + return new Field(name, Role.MEASURE, subType, false, null, null, null, m); + } + if (ReportingConstants.AGG_AVG.equals(agg)) { + String subType = FLOATING.contains(src) ? DoubleField.SUBTYPE_DOUBLE : DecimalField.SUBTYPE_DECIMAL; + return new Field(name, Role.MEASURE, subType, false, null, null, null, m); + } + // min / max keep the source field's type. + return new Field(name, Role.MEASURE, src, false, of, null, null, m); + } + + private static T declaredMember(MetaObject from, Class type, String name) { + // ADR-0039: resolving children, so a member declared on an abstract base is found. + for (T member : from.getChildren(type, true)) { + if (name.equals(member.getShortName())) return member; + } + return null; + } + + /** The node's package for ADR-0042 bare-reference resolution. */ + private static String packageOf(MetaData node) { + return node.getPackage() == null ? "" : node.getPackage(); + } + + private static MetaRoot rootOf(MetaObject report) { + for (MetaData node = report.getParent(); node != null; node = node.getParent()) { + if (node instanceof MetaRoot) return (MetaRoot) node; + } + throw new MetaDataException("report '" + report.getShortName() + + "': is not attached to a model root; pass the root its references resolve in."); + } + + private static MetaDataException unresolved(MetaObject report, String what) { + return new MetaDataException("report '" + report.getShortName() + "': " + what + " does not resolve."); + } +} diff --git a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java new file mode 100644 index 000000000..d9bc51e28 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java @@ -0,0 +1,352 @@ +/* + * Copyright 2026 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.reporting; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaDataException; +import com.metaobjects.MetaRoot; +import com.metaobjects.database.CoreDBMetaDataProvider; +import com.metaobjects.field.CurrencyField; +import com.metaobjects.field.EnumField; +import com.metaobjects.field.MetaField; +import com.metaobjects.io.json.CanonicalJsonSerializer; +import com.metaobjects.loader.LoaderOptions; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.InMemoryStringSource; +import com.metaobjects.object.MetaObject; +import com.metaobjects.registry.SharedRegistryTestBase; +import com.metaobjects.source.MetaSource; +import org.junit.BeforeClass; +import org.junit.Test; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotSame; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * FR-044 — {@link ReportReadModel}: the detached object a runtime reads a report through. + * Container-free; the database read is gated by the persistence-conformance lane. + */ +public class ReportReadModelTest extends SharedRegistryTestBase { + + private static MetaDataLoader canonicalLoader; + private static MetaRoot canonical; + + private static Path corpusDir() { + Path dir = Paths.get("").toAbsolutePath(); + while (dir != null) { + Path candidate = dir.resolve("fixtures/persistence-conformance"); + if (Files.isDirectory(candidate)) return candidate; + dir = dir.getParent(); + } + throw new AssertionError("fixtures/persistence-conformance not found"); + } + + @BeforeClass + public static void loadCanonical() { + canonicalLoader = MetaDataLoader.fromDirectory("report-read-model-test", corpusDir().resolve("canonical")); + canonical = canonicalLoader.getRoot(); + } + + /** Loads and REGISTERS (so the tree is frozen, as every production load is). */ + private static MetaRoot loadJson(String json) { + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "report-read-model-inline"); + loader.setSourceURIs(java.util.Collections.emptyList()); + loader.init(); + loader.load(List.of(new InMemoryStringSource(json, "meta.inline.json"))); + assertTrue("no load errors: " + loader.getErrors(), loader.getErrors().isEmpty()); + loader.register(); + return loader.getRoot(); + } + + private static MetaObject object(MetaRoot root, String name) { + // ADR-0039: own — the root's own children in declaration order (a root has no super). + for (MetaObject o : root.getChildren(MetaObject.class, false)) { + if (name.equals(o.getShortName())) return o; + } + throw new AssertionError("no object " + name); + } + + private static List fieldNames(MetaObject o) { + List names = new ArrayList<>(); + for (MetaField f : o.getMetaFields()) names.add(f.getName()); + return names; + } + + private static boolean required(MetaField f) { + return Boolean.TRUE.equals(f.getMetaAttr(MetaField.ATTR_REQUIRED).getValue()); + } + + // --------------------------------------------------------------------------- + // Shape + // --------------------------------------------------------------------------- + + @Test + public void carriesOneRealFieldPerTableBRowInOrder() { + ReportReadModel model = ReportReadModel.of(object(canonical, "ProgramMinutes"), canonical); + assertEquals( + List.of("program", "programTitle", "weeks", "longWeeks", "labels", "slots", "totalMinutes", + "avgMinutes", "minMinutes", "maxMinutes", "longShare"), + fieldNames(model)); + assertEquals("long", model.getMetaField("program").getSubType()); + assertEquals("string", model.getMetaField("programTitle").getSubType()); + assertEquals("long", model.getMetaField("weeks").getSubType()); + assertEquals("decimal", model.getMetaField("avgMinutes").getSubType()); + assertEquals("min keeps the @of field's subtype", "int", model.getMetaField("minMinutes").getSubType()); + assertEquals("decimal", model.getMetaField("longShare").getSubType()); + } + + @Test + public void requiredComesFromTheShapeNotFromTheTypeSource() { + ReportReadModel model = ReportReadModel.of(object(canonical, "ProgramMinutes"), canonical); + assertTrue(required(model.getMetaField("program"))); + assertFalse("reached by @via", required(model.getMetaField("programTitle"))); + assertTrue("a count is never null", required(model.getMetaField("weeks"))); + // Week.durationMinutes is @required, yet a min over it is nullable (an empty group). + assertFalse(required(model.getMetaField("minMinutes"))); + } + + @Test + public void keepsTheReportsNamePackageAndSubtypeAndHasNoIdentity() { + MetaObject report = object(canonical, "ProgramMinutes"); + ReportReadModel model = ReportReadModel.of(report, canonical); + assertEquals(report.getName(), model.getName()); + assertEquals("ProgramMinutes", model.getShortName()); + assertEquals("fitness", model.getPackage()); + assertEquals(MetaObject.SUBTYPE_REPORT, model.getSubType()); + assertTrue(ReportReadModel.isReport(model)); + assertTrue(ReportReadModel.isReport(report)); + assertFalse(ReportReadModel.isReport(object(canonical, "Week"))); + assertNull("a report has no primary key", model.getPrimaryIdentity()); + assertSame(report, model.report()); + assertTrue(model.isFrozen()); + } + + // --------------------------------------------------------------------------- + // Type-shaping attrs (Table B) + // --------------------------------------------------------------------------- + + @Test + public void anEnumDimensionCarriesItsValues() { + MetaField status = ReportReadModel.of(object(canonical, "ProgramsByMonth"), canonical).getMetaField("status"); + assertEquals("enum", status.getSubType()); + assertEquals( + object(canonical, "Program").getMetaField("status").getMetaAttr(EnumField.ATTR_VALUES).getValue(), + status.getMetaAttr(EnumField.ATTR_VALUES).getValue()); + } + + @Test + public void theHourBucketCarriesLocalTimeAndADateBucketCarriesNothing() { + ReportReadModel model = ReportReadModel.of(object(canonical, "AssetActivity"), canonical); + MetaField source = object(canonical, "Asset").getMetaField("recordedAt"); + MetaField hour = model.getMetaField("recordedAtHour"); + assertEquals("timestamp", hour.getSubType()); + assertEquals(source.hasMetaAttr(CoreDBMetaDataProvider.LOCAL_TIME), hour.hasMetaAttr(CoreDBMetaDataProvider.LOCAL_TIME)); + MetaField week = model.getMetaField("asOfDateWeek"); + assertEquals("date", week.getSubType()); + assertEquals("only @required", 1, week.getMetaAttrs().size()); + } + + private static final String SALES_MODEL = """ + { "metadata.root": { "package": "shop", "children": [ + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.currency": { "name": "amountCents", "@required": true, "@currency": "EUR" } } + ] } }, + { "object.entity": { "name": "Sale", "extends": "Base", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } }, + { "field.string": { "name": "region", "@required": true, "@maxLength": 8, "@column": "region_code" } }, + { "field.string": { "name": "payload", "@dbColumnType": "jsonb" } }, + { "dimension.attribute": { "name": "region", "@of": "Sale.region" } }, + { "dimension.attribute": { "name": "payload", "@of": "Sale.payload" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "SalesByRegion", "@from": "Sale", "@dimensions": ["region", "payload"], + "@measures": ["revenue", "sales"], "children": [ + { "source.rdb": { "name": "replica", "@kind": "view", "@view": "v_sales_replica", "@role": "replica" } }, + { "source.rdb": { "name": "main", "@kind": "view", "@view": "v_sales_by_region", "@schema": "rpt" } } + ] } }, + { "object.report": { "name": "UnmanagedSales", "@from": "Sale", "@measures": ["sales"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_hand_made", "@unmanaged": true } } + ] } }, + { "object.report": { "name": "InertSales", "@from": "Sale", "@measures": ["sales"] } } + ] } } + """; + + @Test + public void carriesOnlyTheTypeShapingAttrsAndNeverTheColumn() { + MetaRoot root = loadJson(SALES_MODEL); + ReportReadModel model = ReportReadModel.of(object(root, "SalesByRegion"), root); + + MetaField region = model.getMetaField("region"); + assertEquals("8", region.getMetaAttr(MetaField.ATTR_MAX_LENGTH).getValueAsString()); + assertFalse("@column on the @of field is never inherited: the column is the derived name", + region.hasMetaAttr(CoreDBMetaDataProvider.COLUMN)); + + MetaField revenue = model.getMetaField("revenue"); + assertEquals("currency", revenue.getSubType()); + assertEquals("a sum of currency carries @currency, here inherited through extends", + "EUR", revenue.getMetaAttr(CurrencyField.ATTR_CURRENCY).getValueAsString()); + assertFalse(required(revenue)); + + assertEquals("jsonb", model.getMetaField("payload").getMetaAttr(CoreDBMetaDataProvider.DB_COLUMN_TYPE).getValueAsString()); + } + + // --------------------------------------------------------------------------- + // The source + // --------------------------------------------------------------------------- + + @Test + public void theCanonicalReportsResolveTheirRelationToTheDeclaredView() { + assertEquals("v_program_minutes", ReportReadModel.of(object(canonical, "ProgramMinutes"), canonical).viewName()); + assertEquals("v_fitness_totals", ReportReadModel.of(object(canonical, "FitnessTotals"), canonical).viewName()); + assertEquals("v_asset_activity", ReportReadModel.of(object(canonical, "AssetActivity"), canonical).viewName()); + } + + @Test + public void aReplicaDeclaredBeforeThePrimaryViewIsNotTheOneRead() { + MetaRoot root = loadJson(SALES_MODEL); + MetaObject report = object(root, "SalesByRegion"); + ReportReadModel model = ReportReadModel.of(report, root); + + assertTrue(model.isServed()); + assertEquals("v_sales_by_region", model.viewName()); + List sources = new ArrayList<>(model.getSources()); + assertEquals("the copy is the model's only source", 1, sources.size()); + MetaSource copy = sources.get(0); + assertEquals(MetaSource.ROLE_PRIMARY, copy.getRole()); + assertEquals(MetaSource.KIND_VIEW, copy.getEffectiveKind()); + assertEquals("the source's other attrs are carried", "rpt", copy.getSchema()); + + // The loaded source is copied, never re-parented. + MetaSource declared = ReportShape.readSource(report); + assertNotSame(declared, copy); + assertSame(report, (MetaData) declared.getParent()); + assertSame(model, (MetaData) copy.getParent()); + // ADR-0039: own — counting the sources the report itself declares. + assertEquals(2, report.getSources(false).size()); + } + + @Test + public void anUnmanagedSourceIsCopiedLikeAnyOther() { + MetaRoot root = loadJson(SALES_MODEL); + ReportReadModel model = ReportReadModel.of(object(root, "UnmanagedSales"), root); + assertTrue(model.isServed()); + assertEquals("v_hand_made", model.viewName()); + assertTrue(model.getSources().iterator().next().isUnmanaged()); + } + + @Test + public void aSourcelessReportHasAShapeAndNoView() { + MetaRoot root = loadJson(SALES_MODEL); + ReportReadModel model = ReportReadModel.of(object(root, "InertSales"), root); + assertEquals(List.of("sales"), fieldNames(model)); + assertFalse(model.isServed()); + assertNull(model.viewName()); + assertTrue(model.getSources().isEmpty()); + } + + // --------------------------------------------------------------------------- + // Detached, and the loaded tree untouched + // --------------------------------------------------------------------------- + + @Test + public void isDetachedAndLeavesTheLoadedModelUntouched() { + String before = CanonicalJsonSerializer.canonicalSerialize(canonical); + int objectsBefore = canonicalLoader.getMetaObjects().size(); + int childrenBefore = canonical.getChildren().size(); + + List models = new ArrayList<>(); + // ADR-0039: own — the root's own children in declaration order (a root has no super). + for (MetaObject o : canonical.getChildren(MetaObject.class, false)) { + if (ReportReadModel.isReport(o)) models.add(ReportReadModel.of(o, canonical)); + } + assertEquals(6, models.size()); + + for (ReportReadModel model : models) { + assertNull("never attached", model.getParent()); + assertFalse(canonical.getChildren().contains(model)); + assertFalse(canonicalLoader.getMetaObjects().contains(model)); + assertTrue("the declared node still declares no fields", model.report().getMetaFields().isEmpty()); + } + assertEquals(objectsBefore, canonicalLoader.getMetaObjects().size()); + assertEquals(childrenBefore, canonical.getChildren().size()); + assertEquals(before, CanonicalJsonSerializer.canonicalSerialize(canonical)); + } + + @Test + public void isCachedPerReportNodeAndIdempotent() { + MetaObject report = object(canonical, "FitnessTotals"); + ReportReadModel model = ReportReadModel.of(report, canonical); + assertSame(model, ReportReadModel.of(report, canonical)); + assertSame(model, ReportReadModel.of(report)); + assertSame("a read model is its own read model", model, ReportReadModel.of(model)); + assertNotSame(model, ReportReadModel.of(object(canonical, "ProgramMinutes"), canonical)); + } + + // --------------------------------------------------------------------------- + // A derived field over a field.object is refused by name + // --------------------------------------------------------------------------- + + private static final String OBJECT_DIMENSION_MODEL = """ + { "metadata.root": { "package": "shop", "children": [ + { "object.value": { "name": "Address", "children": [ + { "field.string": { "name": "city" } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "field.object": { "name": "shipTo", "@objectRef": "Address", "@storage": "jsonb" } }, + { "dimension.attribute": { "name": "destination", "@of": "Sale.shipTo" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } } + ] } }, + { "object.report": { "name": "SalesByDestination", "@from": "Sale", + "@dimensions": ["destination"], "@measures": ["sales"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_sales_by_destination" } } + ] } } + ] } } + """; + + @Test + public void aDimensionOverAFieldObjectIsRefusedByName() { + MetaRoot root = loadJson(OBJECT_DIMENSION_MODEL); + MetaObject report = object(root, "SalesByDestination"); + // The shape still derives (the loader accepts the model); the read model refuses. + assertEquals("object", ReportShape.of(report, root).fields().get(0).subType()); + try { + ReportReadModel.of(report, root); + fail("a report over a field.object must be refused"); + } catch (MetaDataException e) { + assertEquals("report 'SalesByDestination': dimension 'destination' reads 'shop::Sale.shipTo'," + + " a field.object. A report over a field.object is not supported; group by a scalar field.", + e.getMessage()); + } + } +} diff --git a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java new file mode 100644 index 000000000..ddd8bf1b5 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java @@ -0,0 +1,447 @@ +/* + * Copyright 2026 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.metaobjects.reporting; + +import com.metaobjects.MetaData; +import com.metaobjects.MetaDataException; +import com.metaobjects.MetaRoot; +import com.metaobjects.field.MetaField; +import com.metaobjects.loader.LoaderOptions; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.InMemoryStringSource; +import com.metaobjects.object.MetaObject; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.BeforeClass; +import org.junit.Test; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.List; +import java.util.stream.Collectors; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** + * FR-044 — {@link ReportShape} (contract Table B) in the Java port. The gate is the + * byte-comparison with {@code fixtures/persistence-conformance/report-shapes.json}, the + * committed TypeScript-produced artifact every port derives from the same canonical model. + * Container-free: metadata in, JSON out. + */ +public class ReportShapeTest extends SharedRegistryTestBase { + + private static MetaRoot canonical; + + private static Path corpusDir() { + Path dir = Paths.get("").toAbsolutePath(); + while (dir != null) { + Path candidate = dir.resolve("fixtures/persistence-conformance"); + if (Files.isDirectory(candidate)) return candidate; + dir = dir.getParent(); + } + throw new AssertionError("fixtures/persistence-conformance not found"); + } + + @BeforeClass + public static void loadCanonical() { + canonical = MetaDataLoader.fromDirectory("report-shape-test", corpusDir().resolve("canonical")).getRoot(); + } + + private static MetaRoot loadJson(String... files) { + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "report-shape-inline"); + loader.setSourceURIs(java.util.Collections.emptyList()); + loader.init(); + List sources = new java.util.ArrayList<>(); + for (int i = 0; i < files.length; i++) { + sources.add(new InMemoryStringSource(files[i], "meta.inline" + i + ".json")); + } + loader.load(sources); + assertTrue("no load errors: " + loader.getErrors(), loader.getErrors().isEmpty()); + return loader.getRoot(); + } + + private static MetaObject object(MetaRoot root, String name) { + // ADR-0039: own — the root's own children in declaration order (a root has no super). + for (MetaObject o : root.getChildren(MetaObject.class, false)) { + if (name.equals(o.getShortName())) return o; + } + throw new AssertionError("no object " + name); + } + + private static ReportShape.Field field(ReportShape shape, String name) { + for (ReportShape.Field f : shape.fields()) { + if (name.equals(f.name())) return f; + } + throw new AssertionError("no derived field " + name); + } + + // --------------------------------------------------------------------------- + // The artifact — every port serialises the same bytes + // --------------------------------------------------------------------------- + + private static String quote(String s) { + return s == null ? "null" : "\"" + s.replace("\\", "\\\\").replace("\"", "\\\"") + "\""; + } + + /** The artifact's bytes for a loaded model: the format documented in the TypeScript + * generator ({@code integration-tests/src/gen-report-shapes.ts}). */ + private static String shapesJson(MetaRoot root) { + StringBuilder b = new StringBuilder("{\n \"reports\": ["); + boolean firstReport = true; + // ADR-0039: own — the root's own children in declaration order (a root has no super). + for (MetaObject report : root.getChildren(MetaObject.class, false)) { + if (!MetaObject.SUBTYPE_REPORT.equals(report.getSubType())) continue; + ReportShape shape = ReportShape.of(report, root); + b.append(firstReport ? "\n" : ",\n"); + firstReport = false; + b.append(" {\n"); + b.append(" \"report\": ").append(quote(report.getName())).append(",\n"); + b.append(" \"from\": ").append(quote(shape.from().getName())).append(",\n"); + b.append(" \"view\": ").append(quote(shape.viewName())).append(",\n"); + b.append(" \"fields\": ["); + boolean firstField = true; + for (ReportShape.Field f : shape.fields()) { + b.append(firstField ? "\n" : ",\n"); + firstField = false; + b.append(" {\n"); + b.append(" \"name\": ").append(quote(f.name())).append(",\n"); + b.append(" \"role\": ").append(quote(f.role().wireName())).append(",\n"); + b.append(" \"subType\": ").append(quote(f.subType())).append(",\n"); + b.append(" \"required\": ").append(f.required()).append(",\n"); + b.append(" \"typeSource\": ").append(quote(f.typeSourceKey())).append("\n"); + b.append(" }"); + } + b.append(firstField ? "]\n" : "\n ]\n"); + b.append(" }"); + } + b.append(firstReport ? "]\n" : "\n ]\n"); + return b.append("}\n").toString(); + } + + @Test + public void canonicalShapesByteMatchTheCommittedArtifact() throws IOException { + String expected = Files.readString(corpusDir().resolve("report-shapes.json"), StandardCharsets.UTF_8); + assertEquals(expected, shapesJson(canonical)); + } + + // --------------------------------------------------------------------------- + // Table B, row by row, on the canonical reports + // --------------------------------------------------------------------------- + + @Test + public void fieldsAreDimensionsThenMeasuresInListedOrder() { + ReportShape shape = ReportShape.of(object(canonical, "ProgramMinutes"), canonical); + assertEquals( + List.of("program", "programTitle", "weeks", "longWeeks", "labels", "slots", "totalMinutes", + "avgMinutes", "minMinutes", "maxMinutes", "longShare"), + shape.fields().stream().map(ReportShape.Field::name).collect(Collectors.toList())); + assertSame(object(canonical, "ProgramMinutes"), shape.report()); + assertSame(object(canonical, "Week"), shape.from()); + assertEquals("v_program_minutes", shape.viewName()); + } + + @Test + public void attributeDimensionKeepsTheOfFieldsSubtypeAndIsRequiredOnlyWithoutVia() { + ReportShape shape = ReportShape.of(object(canonical, "ProgramMinutes"), canonical); + + ReportShape.Field program = field(shape, "program"); + assertEquals(ReportShape.Role.DIMENSION, program.role()); + assertEquals("long", program.subType()); + assertTrue("no @via and a required @of field", program.required()); + assertEquals("fitness::Week.programId", program.typeSourceKey()); + assertNotNull(program.dimension()); + assertNull(program.grain()); + assertNull(program.measure()); + + ReportShape.Field title = field(shape, "programTitle"); + assertEquals("string", title.subType()); + assertFalse("reached by @via, so nullable", title.required()); + assertEquals("fitness::Program.title", title.typeSourceKey()); + } + + @Test + public void measureRowsOfTableB() { + ReportShape shape = ReportShape.of(object(canonical, "ProgramMinutes"), canonical); + + ReportShape.Field count = field(shape, "weeks"); + assertEquals(ReportShape.Role.MEASURE, count.role()); + assertEquals("long", count.subType()); + assertTrue("a count is never null", count.required()); + assertNull(count.typeSource()); + assertNotNull(count.measure()); + + assertEquals("long", field(shape, "labels").subType()); // count + @distinct + assertEquals("long", field(shape, "slots").subType()); // tuple count + assertEquals("long", field(shape, "totalMinutes").subType()); // sum of int + assertFalse(field(shape, "totalMinutes").required()); + assertNull(field(shape, "totalMinutes").typeSource()); + assertEquals("decimal", field(shape, "avgMinutes").subType()); + + ReportShape.Field min = field(shape, "minMinutes"); + assertEquals("min keeps the @of field's subtype", "int", min.subType()); + assertFalse(min.required()); + assertEquals("fitness::Week.durationMinutes", min.typeSourceKey()); + + ReportShape.Field ratio = field(shape, "longShare"); + assertEquals("decimal", ratio.subType()); + assertFalse(ratio.required()); + assertNull(ratio.typeSource()); + } + + @Test + public void timeDimensionIsNamedByGrainAndTypedByGrain() { + ReportShape byMonth = ReportShape.of(object(canonical, "ProgramsByMonth"), canonical); + ReportShape.Field month = byMonth.fields().get(0); + assertEquals("createdAtMonth", month.name()); + assertEquals("date", month.subType()); + assertEquals("month", month.grain()); + assertNull("a date bucket carries no type source", month.typeSource()); + + ReportShape activity = ReportShape.of(object(canonical, "AssetActivity"), canonical); + ReportShape.Field hour = field(activity, "recordedAtHour"); + assertEquals("timestamp", hour.subType()); + assertEquals("hour", hour.grain()); + assertEquals("the hour bucket carries @localTime from the @of field", + "fitness::Asset.recordedAt", hour.typeSourceKey()); + assertEquals("date", field(activity, "asOfDateWeek").subType()); + } + + @Test + public void aReportWithNoDimensionsHasOnlyMeasures() { + ReportShape shape = ReportShape.of(object(canonical, "FitnessTotals"), canonical); + assertEquals(List.of("weeks", "totalMinutes", "longShare"), + shape.fields().stream().map(ReportShape.Field::name).collect(Collectors.toList())); + for (ReportShape.Field f : shape.fields()) assertEquals(ReportShape.Role.MEASURE, f.role()); + } + + @Test + public void theRootIsFoundFromTheReportWhenNotPassed() { + MetaObject report = object(canonical, "ProgramsByWeek"); + assertEquals( + ReportShape.of(report, canonical).fields().stream().map(ReportShape.Field::name).collect(Collectors.toList()), + ReportShape.of(report).fields().stream().map(ReportShape.Field::name).collect(Collectors.toList())); + } + + // --------------------------------------------------------------------------- + // Rows the canonical model does not exercise + // --------------------------------------------------------------------------- + + private static final String SALES_MODEL = """ + { "metadata.root": { "package": "shop", "children": [ + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.currency": { "name": "amountCents", "@required": true, "@currency": "USD" } } + ] } }, + { "object.entity": { "name": "Sale", "extends": "Base", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } }, + { "field.decimal": { "name": "weight", "@precision": 10, "@scale": 2 } }, + { "field.double": { "name": "score" } }, + { "field.float": { "name": "ratio" } }, + { "field.string": { "name": "region", "@required": true, "@maxLength": 8 } }, + { "dimension.attribute": { "name": "region", "@of": "Sale.region" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "avgRevenue", "@agg": "avg", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "totalWeight", "@agg": "sum", "@of": "Sale.weight" } }, + { "measure.aggregate": { "name": "avgWeight", "@agg": "avg", "@of": "Sale.weight" } }, + { "measure.aggregate": { "name": "totalScore", "@agg": "sum", "@of": "Sale.score" } }, + { "measure.aggregate": { "name": "avgScore", "@agg": "avg", "@of": "Sale.score" } }, + { "measure.aggregate": { "name": "totalRatio", "@agg": "sum", "@of": "Sale.ratio" } }, + { "measure.aggregate": { "name": "avgRatio", "@agg": "avg", "@of": "Sale.ratio" } }, + { "measure.aggregate": { "name": "maxRevenue", "@agg": "max", "@of": "Sale.amountCents" } } + ] } }, + { "object.report": { "name": "SalesByRegion", "@from": "Sale", "@dimensions": ["region"], + "@measures": ["revenue", "avgRevenue", "totalWeight", "avgWeight", "totalScore", "avgScore", + "totalRatio", "avgRatio", "maxRevenue"] } } + ] } } + """; + + @Test + public void sumAndAvgRowsByOfSubtype() { + MetaRoot root = loadJson(SALES_MODEL); + ReportShape shape = ReportShape.of(object(root, "SalesByRegion"), root); + + ReportShape.Field revenue = field(shape, "revenue"); + assertEquals("currency", revenue.subType()); + assertNotNull("sum of currency carries @currency from the @of field", revenue.typeSource()); + assertEquals("decimal", field(shape, "avgRevenue").subType()); + assertEquals("decimal", field(shape, "totalWeight").subType()); + assertEquals("decimal", field(shape, "avgWeight").subType()); + assertEquals("double", field(shape, "totalScore").subType()); + assertEquals("double", field(shape, "avgScore").subType()); + assertEquals("double", field(shape, "totalRatio").subType()); + assertEquals("double", field(shape, "avgRatio").subType()); + assertEquals("currency", field(shape, "maxRevenue").subType()); + assertTrue("no @via and a required @of field", field(shape, "region").required()); + assertNull("a sourceless report has no view", shape.viewName()); + } + + @Test + public void anInheritedOfFieldIsFoundAndItsTypeSourceNamesTheDeclaringEntity() { + MetaRoot root = loadJson(SALES_MODEL); + ReportShape.Field revenue = field(ReportShape.of(object(root, "SalesByRegion"), root), "revenue"); + MetaField typeSource = revenue.typeSource(); + assertEquals("amountCents", typeSource.getName()); + assertSame("the field lives on the base that declares it", object(root, "Base"), (MetaData) typeSource.getParent()); + assertEquals("shop::Base.amountCents", revenue.typeSourceKey()); + } + + @Test + public void anUnresolvedReferenceNamesTheReport() { + MetaRoot root = loadJson(SALES_MODEL); + // A report built in code (never added to the root), naming a measure that does not exist. + com.metaobjects.object.ReportMetaObject stray = new com.metaobjects.object.ReportMetaObject("shop::Stray"); + stray.addMetaAttr(com.metaobjects.attr.StringAttribute.create(MetaObject.ATTR_REPORT_FROM, "Sale")); + com.metaobjects.attr.StringArrayAttribute measures = + new com.metaobjects.attr.StringArrayAttribute(MetaObject.ATTR_REPORT_MEASURES); + measures.setValue(List.of("nope")); + stray.addMetaAttr(measures); + try { + ReportShape.of(stray, root); + fail("an unresolved measure must throw"); + } catch (MetaDataException e) { + assertTrue(e.getMessage(), e.getMessage().contains("report 'Stray'")); + assertTrue(e.getMessage(), e.getMessage().contains("measure 'nope'")); + } + } + + // --------------------------------------------------------------------------- + // Reference resolution: the shape must agree with the loader's reporting validation + // about what a reference names, or a model that loads clean fails (or is silently + // mistyped) when it is read. The same cases as the TypeScript report-shape.test.ts. + // --------------------------------------------------------------------------- + + /** {@code a::Base} (abstract): members whose bare {@code @of} names {@code Base}. */ + private static final String SHARED_BASE = """ + { "metadata.root": { "package": "a", "children": [ + { "object.entity": { "name": "Base", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "kind" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "kind", "@of": "Base.kind" } }, + { "measure.aggregate": { "name": "events", "@agg": "count", "@of": "Base.id" } }, + { "measure.aggregate": { "name": "lastKind", "@agg": "max", "@of": "Base.kind" } } + ] } } + ] } } + """; + + private static final String DECOY = + "{ \"object.entity\": { \"name\": \"Base\", \"children\": [" + + " { \"field.int\": { \"name\": \"id\" } }, { \"field.int\": { \"name\": \"kind\" } } ] } },"; + + /** Package {@code b}: {@code Ev extends a::Base} and report {@code R} over it. */ + private static String evFile(String before, String evExtra, String measures) { + return "{ \"metadata.root\": { \"package\": \"b\", \"children\": [" + before + + " { \"object.entity\": { \"name\": \"Ev\", \"extends\": \"a::Base\", \"children\": [" + + " { \"source.rdb\": { \"@table\": \"evs\" } }" + evExtra + " ] } }," + + " { \"object.report\": { \"name\": \"R\", \"@from\": \"Ev\", \"@dimensions\": [\"kind\"]," + + " \"@measures\": " + measures + ", \"children\": [" + + " { \"source.rdb\": { \"@kind\": \"view\", \"@view\": \"v_r\" } } ] } } ] } }"; + } + + private static final String BARE_MEASURES = "[\"events\", \"lastKind\"]"; + + /** {@code name subType typeSourceKey} per derived field of report {@code R}. */ + private static List typed(MetaRoot root) { + return ReportShape.of(object(root, "R"), root).fields().stream() + .map(f -> f.name() + " " + f.subType() + " " + f.typeSourceKey()) + .collect(Collectors.toList()); + } + + @Test + public void aBareOfOnAMemberInheritedFromAnotherPackageResolvesInTheDeclaringEntitysPackage() { + MetaRoot root = loadJson(SHARED_BASE, evFile("", "", BARE_MEASURES)); + assertEquals(List.of("kind string a::Base.kind", "events long null", "lastKind string a::Base.kind"), + typed(root)); + } + + @Test + public void aSameNamedDecoyInTheReportsPackageDoesNotCaptureTheReference() { + MetaRoot root = loadJson(SHARED_BASE, evFile(DECOY, "", BARE_MEASURES)); + assertEquals(List.of("kind string a::Base.kind", "events long null", "lastKind string a::Base.kind"), + typed(root)); + } + + @Test + public void withoutViaTheFieldIsReadFromFromSoAFieldFromRedeclaresWins() { + MetaRoot root = loadJson(SHARED_BASE, + evFile("", ", { \"field.int\": { \"name\": \"kind\" } }", BARE_MEASURES)); + assertEquals(List.of("kind int b::Ev.kind", "events long null", "lastKind int b::Ev.kind"), typed(root)); + } + + @Test + public void aDottedMeasuresItemNamesTheMeasureByItsLastSegment() { + MetaRoot root = loadJson(SHARED_BASE, evFile("", "", "[\"Ev.events\", \"a::Base.lastKind\"]")); + assertEquals(List.of("kind string a::Base.kind", "events long null", "lastKind string a::Base.kind"), + typed(root)); + } + + @Test + public void measureItemNameIsTheLastSegment() { + assertEquals("total", ReportAccessors.reportMeasureItemName("total")); + assertEquals("total", ReportAccessors.reportMeasureItemName("Sale.total")); + assertEquals("total", ReportAccessors.reportMeasureItemName("acme::shop::Sale.total")); + assertNull(ReportAccessors.reportMeasureItemOwner("total")); + assertEquals("acme::shop::Sale", ReportAccessors.reportMeasureItemOwner("acme::shop::Sale.total")); + } + + /** A report built in code (never added to the root): what the loader would refuse. */ + private static com.metaobjects.object.ReportMetaObject stray(String name, String from, String attr, String item) { + com.metaobjects.object.ReportMetaObject stray = new com.metaobjects.object.ReportMetaObject(name); + stray.addMetaAttr(com.metaobjects.attr.StringAttribute.create(MetaObject.ATTR_REPORT_FROM, from)); + com.metaobjects.attr.StringArrayAttribute items = new com.metaobjects.attr.StringArrayAttribute(attr); + items.setValue(List.of(item)); + stray.addMetaAttr(items); + return stray; + } + + private static void assertUnresolved(String expected, MetaObject report, MetaRoot root) { + try { + ReportShape.of(report, root); + fail("expected: " + expected); + } catch (MetaDataException e) { + assertEquals(expected, e.getMessage()); + } + } + + @Test + public void aDottedMeasuresItemWhoseQualifierIsNotFromOrAnAncestorDoesNotResolve() { + MetaRoot root = loadJson(SHARED_BASE, evFile(DECOY, "", BARE_MEASURES)); + // Past the loader, which refuses these as ERR_INVALID_REPORT / ERR_REPORT_FOREIGN_MEASURE. + assertUnresolved("report 'R': measure 'Nope.events' on 'Ev' does not resolve.", + stray("b::R", "Ev", MetaObject.ATTR_REPORT_MEASURES, "Nope.events"), root); + // The qualifier resolves in the REPORT's package: b::Base is the decoy, not an ancestor of Ev. + assertUnresolved("report 'R': measure 'Base.events' on 'Ev' does not resolve.", + stray("b::R", "Ev", MetaObject.ATTR_REPORT_MEASURES, "Base.events"), root); + } + + @Test + public void aTimeDimensionItemWithNoGrainOrAGrainOutsideTheClosedSetDoesNotResolve() { + assertUnresolved("report 'Stray': time dimension 'createdAt' grain '' does not resolve.", + stray("fitness::Stray", "Program", MetaObject.ATTR_REPORT_DIMENSIONS, "createdAt"), canonical); + assertUnresolved("report 'Stray': time dimension 'createdAt' grain 'fortnight' does not resolve.", + stray("fitness::Stray", "Program", MetaObject.ATTR_REPORT_DIMENSIONS, "createdAt:fortnight"), canonical); + } +} diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/ObjectManagerDB.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/ObjectManagerDB.java index 39dbd8d75..474881692 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/ObjectManagerDB.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/ObjectManagerDB.java @@ -18,6 +18,7 @@ import com.metaobjects.field.MetaField; import com.metaobjects.manager.StateAwareMetaObject; import com.metaobjects.object.MetaObject; +import com.metaobjects.reporting.ReportReadModel; import com.metaobjects.*; import com.metaobjects.manager.*; import com.metaobjects.manager.db.driver.*; @@ -255,10 +256,80 @@ protected ObjectMapping getCreateMapping(MetaObject mc) { * Gets the read mapping */ protected ObjectMapping getReadMapping(MetaObject mc) { + // FR-044: a declared report has no fields to map. It is mapped through its read + // model, and has no read mapping at all when it declares no view (not served). + if (isDeclaredReport(mc)) { + ReportReadModel model = ReportReadModel.of(mc); + return model.isServed() ? getReadMapping(model) : null; + } return readMappings.computeIfAbsent(mc, k -> Optional.ofNullable(getMappingHandler().getReadMapping(k))).orElse(null); } + /////////////////////////////////////////////////////// + // REPORTS (FR-044) + // + + /** True for an {@code object.report} node as loaded (not its read model). */ + private static boolean isDeclaredReport(MetaObject mc) { + return ReportReadModel.isReport(mc) && !(mc instanceof ReportReadModel); + } + + /** + * The object a READ is planned against. Every object but a report is returned + * unchanged. An {@code object.report} declares no fields — its read shape is derived + * from its dimensions and measures — so it is read through its detached + * {@link ReportReadModel}: ordinary fields (one per derived field) over the report's + * view. The column mapping, filter and sort resolution, instance construction and the + * read codecs then see nothing unusual. The model is never attached to the loaded tree. + * + *

Rows of a report are instances of the returned model: read their values through + * it ({@code readObjectFor(report).getMetaFields()}), not through the declared node, + * which has no fields.

+ * + * @throws PersistenceException when the report declares no read-only source: it has a + * shape and no view, so it is not served + */ + public MetaObject readObjectFor(MetaObject mc) { + if (!ReportReadModel.isReport(mc)) return mc; + ReportReadModel model; + try { + model = ReportReadModel.of(mc); + } catch (MetaDataException e) { + throw new PersistenceException("Report [" + mc.getName() + "] cannot be read: " + e.getMessage(), e); + } + if (!model.isServed()) { + throw new PersistenceException("Report [" + mc.getName() + "] is not served: it declares no" + + " read-only source, so it has no view to read"); + } + return model; + } + + /** + * Refuse an operation that needs an identity or writes. A report is a compiled view + * with no primary key: it is listed and counted, nothing else. Checked on the subtype, + * before any source or mapping check, so a write on a sourceless report is refused as + * read-only rather than as unserved. + */ + private static void requireNotReport(MetaObject mc, String operation) { + if (ReportReadModel.isReport(mc)) { + throw new PersistenceException(operation + " is not supported on [" + mc.getName() + + "]: a report is read-only and has no identity (read it with getObjects / getObjectsCount)"); + } + } + + /** + * Gets an object's reference. A report row has no identity, so it has no reference. + */ + @Override + public ObjectRef getObjectRef(Object obj) { + // The same lookup the base method does. An object with no metadata is left to the + // base, so everything but a report row behaves exactly as it did. + MetaObject mc = MetaDataUtil.findMetaObject(obj, this); + if (mc != null) requireNotReport(mc, "getObjectRef"); + return super.getObjectRef(obj); + } + /** * Gets the update mapping */ @@ -373,6 +444,8 @@ public Object getObjectByRef(ObjectConnection c, String refStr) { ObjectRef ref = getObjectRef(refStr); MetaObject mc = ref.getMetaClass(); + requireNotReport(mc, "getObjectByRef"); + if (!isReadableClass(mc)) { throw new PersistenceException("MetaClass [" + mc + "] is not readable"); } @@ -489,6 +562,8 @@ private void requireInSubtypeScope(ObjectConnection c, MetaObject mc, Object obj @Override public int deleteObjects(ObjectConnection c, MetaObject mc, Expression exp) { + requireNotReport(mc, "deleteObjects"); + if (!isDeleteableClass(mc)) { throw new PersistenceException("MetaClass [" + mc + "] is not deletable"); } @@ -528,6 +603,8 @@ public int deleteObjects(ObjectConnection c, MetaObject mc, Expression exp) { */ @Override public long getObjectsCount(ObjectConnection c, MetaObject mc, Expression exp) throws MetaDataException { + mc = readObjectFor(mc); // FR-044: a report is counted through its read model + if (!isReadableClass(mc)) { throw new PersistenceException("MetaClass [" + mc + "] is not persistable"); } @@ -554,6 +631,8 @@ public long getObjectsCount(ObjectConnection c, MetaObject mc, Expression exp) t */ @Override public Collection getObjects(ObjectConnection c, MetaObject mc, QueryOptions options) throws MetaDataException { + mc = readObjectFor(mc); // FR-044: a report is read through its read model + if (!isReadableClass(mc)) { throw new PersistenceException("MetaClass [" + mc + "] is not persistable"); } @@ -605,6 +684,8 @@ public void loadObject(ObjectConnection c, Object o) throws MetaDataException { // Get the MetaClass for the object MetaObject mc = getMetaObjectFor(o); + requireNotReport(mc, "loadObject"); + // If it's not a readable class throw an exception if (!isReadableClass(mc)) { throw new PersistenceException("MetaClass [" + mc + "] is not persistable"); @@ -657,6 +738,8 @@ public void createObject(ObjectConnection c, Object obj) throws PersistenceExcep MetaObject mc = getMetaObjectFor(obj); + requireNotReport(mc, "createObject"); + if (!isCreateableClass(mc)) { throw new PersistenceException("Object of class [" + mc + "] is not createable"); } @@ -699,6 +782,7 @@ public void updateObject(ObjectConnection c, Object obj) throws PersistenceExcep // Get the metaclass and make sure it is updateable MetaObject mc = getMetaObjectFor(obj); + requireNotReport(mc, "updateObject"); if (!isUpdateableClass(mc)) { throw new PersistenceException("Object of class [" + mc + "] is not writeable"); } @@ -785,6 +869,8 @@ public void deleteObject(ObjectConnection c, Object obj) throws PersistenceExcep MetaObject mc = getMetaObjectFor(obj); + requireNotReport(mc, "deleteObject"); + if (!isDeleteableClass(mc)) { throw new PersistenceException("Object [" + obj + "] of class [" + mc + "] is not deleteable"); } @@ -989,6 +1075,15 @@ protected MetaField getFieldForColumn(MetaObject resultClass, ObjectMapping mapp return rc; } + /** + * The object an OQL query names as its result class ({@code [Name] SELECT ...}). + * Resolved through the loader registry, as it always was; a seam so a manager wired to + * a specific loader can resolve the name against it. + */ + protected MetaObject findResultClass(String className) throws MetaDataNotFoundException { + return MetaDataUtil.findMetaObjectByName(className, this); + } + /** * Executes the specified query and maps it to the given object. * @@ -1022,7 +1117,7 @@ public Collection executeQuery(ObjectConnection c, String query, Collection executeQuery(ObjectConnection c, String query, Collection data = new LinkedList(); try { + // FR-044: a declared report has no fields, so its rows are built from its + // read model (one field per derived field). In OQL the author supplies the + // SQL, so a sourceless report is a legitimate result shape too: it has a + // model and simply no mapping, and columns bind by derived field name. + if (ReportReadModel.isReport(resultClass)) resultClass = ReportReadModel.of(resultClass); ObjectMappingDB mapping = (ObjectMappingDB) getReadMapping(resultClass); while (rs.next()) { @@ -1078,6 +1178,7 @@ public Collection executeQuery(ObjectConnection c, String query, Collection objects) throws MetaDataException { + requireNotReport(mc, "createObjectsBulk"); if (!isCreateableClass(mc)) { throw new PersistenceException("Object of class [" + mc + "] is not createable"); } @@ -1108,6 +1209,7 @@ public void createObjectsBulk(ObjectConnection c, MetaObject mc, Collection objects) throws MetaDataException { + requireNotReport(mc, "updateObjectsBulk"); if (!isUpdateableClass(mc)) { throw new PersistenceException("Object of class [" + mc + "] is not updateable"); } diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java index 3e352ed8a..b088aee8d 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java @@ -9,6 +9,8 @@ import com.metaobjects.MetaDataNotFoundException; import com.metaobjects.database.CoreDBMetaDataProvider; import com.metaobjects.object.MetaObject; +import com.metaobjects.reporting.ReportReadModel; +import com.metaobjects.source.MetaSource; import com.metaobjects.MetaData; import com.metaobjects.MetaDataException; @@ -68,7 +70,14 @@ public ObjectMapping getCreateMapping( MetaObject mc ) { @Override public ObjectMapping getReadMapping(MetaObject mc) { - + + // FR-044: a declared report has no field children, so mapping it would yield a + // column-less SELECT. Refuse by name instead: a report is mapped through its read model. + if ( ReportReadModel.isReport( mc ) && !( mc instanceof ReportReadModel )) { + throw new MetaDataException( "Report [" + mc.getName() + "] has no fields to map: a report is read" + + " through its read model (ReportReadModel.of), not through the declared node" ); + } + // Try to get a view first String name = getViewRef( mc ); if ( name != null ) { @@ -117,8 +126,8 @@ protected ObjectMappingDB getTableMapping( MetaObject mc ) { /** Get the table mapping */ protected ObjectMapping getViewMapping( MetaObject mc ) { - // Create the view definition. The view name comes from source.rdb @table - // (@kind=view); OMDB reads from a view that already exists in the database + // Create the view definition. The view name is the read-only source.rdb's physical + // name (@view, or the legacy @table); OMDB reads from a view that already exists in the database // (created by the migrate toolchain) — it does not synthesize view DDL. ViewDef v = new ViewDef( NameDef.parseName( getViewRef( mc ))); @@ -468,14 +477,22 @@ private String getPersistenceAttribute( MetaData md, String ref ) { } /** - * Retrieves the view name from the MetaObject — the {@code @table} of its - * primary read-only {@code source.rdb} child (source-v2 ADR-0007). + * Retrieves the view name from the MetaObject: the physical name of its primary + * read-only {@code source.rdb} child (source-v2 ADR-0007), resolved by the source's own + * rule ({@link MetaSource#getPhysicalName()}, ADR-0018): the kind-matching alias + * ({@code @view} for a view) first, then the legacy {@code @table}. One rule for a + * projection and for a report's read model, and the rule the TypeScript toolchain + * creates the view under. * * @return the view name, or {@code null} if no primary read-only source */ protected String getViewRef( MetaObject mc ) { - return mc.getPrimaryRdbViewName(); + // ADR-0039: own — findPrimaryReadOnlySource() reads getSources(false). Sanctioned: + // an object is read through the read-only source it declares ITSELF (a projection's + // own view; a report read model's one source copy); an inherited writable source is + // reached through getTableRef below. + return mc.findPrimaryReadOnlySource().map( MetaSource::getPhysicalName ).orElse( null ); } /** diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java new file mode 100644 index 000000000..d08bf909a --- /dev/null +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java @@ -0,0 +1,597 @@ +/* + * Copyright 2026 Doug Mealing LLC dba Meta Objects + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/* + * FR-044 — OMDB reads a view-backed object.report: getObjects / getObjectsCount with + * filter, sort and range on derived fields; by-id and every write refused; a sourceless + * report not served; nothing changed for a non-report object. + * + * The views are created here by literal DDL. That is this test's fixture, not a port + * emitting SQL: in a real project the view comes from the TypeScript toolchain (ADR-0015). + */ +package com.metaobjects.manager.db; + +import com.metaobjects.MetaDataException; +import com.metaobjects.field.MetaField; +import com.metaobjects.io.json.CanonicalJsonSerializer; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.manager.ObjectConnection; +import com.metaobjects.manager.ObjectRef; +import com.metaobjects.manager.PersistenceException; +import com.metaobjects.manager.QueryOptions; +import com.metaobjects.manager.db.defs.BaseDef; +import com.metaobjects.manager.db.driver.DerbyDriver; +import com.metaobjects.manager.exp.Expression; +import com.metaobjects.manager.exp.Range; +import com.metaobjects.manager.exp.SortOrder; +import com.metaobjects.object.MetaObject; +import com.metaobjects.object.value.ValueObject; +import com.metaobjects.registry.MetaDataLoaderRegistry; +import com.metaobjects.registry.ServiceRegistryFactory; +import com.metaobjects.reporting.ReportReadModel; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +import javax.sql.DataSource; +import java.io.PrintWriter; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.sql.SQLNonTransientConnectionException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.logging.Logger; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class ReportReadTest { + + private static ObjectManagerDB omdb; + private static String dbFile; + private static MetaDataLoader loader; + private static MetaDataLoaderRegistry registry; + + @BeforeClass + public static void setupDB() throws Exception { + registry = new MetaDataLoaderRegistry(ServiceRegistryFactory.getDefault()); + loader = MetaDataLoader.fromResources("test-report", List.of("meta.report.json")); + registry.registerLoader(loader); + + dbFile = "omb-report-" + System.currentTimeMillis(); + Class.forName("org.apache.derby.jdbc.EmbeddedDriver"); + getConnection().close(); + + DataSource ds = new DataSource() { + @Override public Connection getConnection() throws SQLException { return ReportReadTest.getConnection(); } + @Override public Connection getConnection(String u, String p) throws SQLException { return getConnection(); } + @Override public PrintWriter getLogWriter() { return new PrintWriter(System.out); } + @Override public void setLogWriter(PrintWriter out) {} + @Override public void setLoginTimeout(int s) {} + @Override public int getLoginTimeout() { return 100; } + @Override public Logger getParentLogger() { throw new UnsupportedOperationException(); } + @Override public T unwrap(Class iface) { throw new UnsupportedOperationException(); } + @Override public boolean isWrapperFor(Class iface) { return false; } + }; + + omdb = new ObjectManagerDB() { + // A string reference names its object through service-discovered loaders, which a + // plain unit test does not have; resolve the name against this test's loader instead. + @Override + public ObjectRef getObjectRef(String refStr) { + String rest = refStr.substring("objectref://".length()); + int slash = rest.indexOf('/'); + return new ObjectRef(registry.findMetaObjectByName(rest.substring(0, slash)), + new String[] { rest.substring(slash + 1) }); + } + + // An OQL result class is named the same way, and resolved the same way here. + @Override + protected MetaObject findResultClass(String className) { + return registry.findMetaObjectByName(className); + } + }; + omdb.setDatabaseDriver(new DerbyDriver()); + omdb.setDataSource(ds); + omdb.init(); + + try (Connection c = getConnection(); Statement s = c.createStatement()) { + s.execute("CREATE TABLE RPT_SALES (id BIGINT PRIMARY KEY, region VARCHAR(8) NOT NULL," + + " status INTEGER NOT NULL, amountCents BIGINT NOT NULL)"); + s.execute("INSERT INTO RPT_SALES VALUES (1, 'east', 1, 100), (2, 'east', 2, 300), (3, 'west', 1, 50)"); + s.execute("CREATE VIEW RPT_V_BY_REGION (region, sales, revenue, minAmount) AS" + + " SELECT region, COUNT(id), SUM(amountCents), MIN(amountCents) FROM RPT_SALES GROUP BY region"); + s.execute("CREATE VIEW RPT_V_BY_STATUS (status, sales) AS" + + " SELECT status, COUNT(id) FROM RPT_SALES GROUP BY status"); + s.execute("CREATE VIEW RPT_V_HAND_MADE (sales) AS SELECT COUNT(id) FROM RPT_SALES"); + s.execute("CREATE VIEW RPT_V_PRIMARY (sales) AS SELECT COUNT(id) FROM RPT_SALES"); + // Decoys: a read that lands on the replica, or on a default table name nobody + // declared, returns a value no real view produces. + s.execute("CREATE VIEW RPT_V_REPLICA (sales) AS SELECT COUNT(id) + 100 FROM RPT_SALES"); + s.execute("CREATE TABLE REPLICATED_SALES (sales BIGINT)"); + s.execute("INSERT INTO REPLICATED_SALES VALUES (999)"); + // A projection whose view is named by @view (not the legacy @table). + s.execute("CREATE VIEW RPT_V_SALE_REGIONS (id, region) AS SELECT id, region FROM RPT_SALES"); + s.execute("CREATE TABLE INERT_SALES (sales BIGINT)"); + s.execute("INSERT INTO INERT_SALES VALUES (999)"); + } + } + + private static Connection getConnection() throws SQLException { + return DriverManager.getConnection("jdbc:derby:memory:" + dbFile + ";create=true"); + } + + @AfterClass + public static void teardown() throws Exception { + if (dbFile != null) { + try { DriverManager.getConnection("jdbc:derby:memory:" + dbFile + ";drop=true"); } + catch (SQLNonTransientConnectionException ignored) {} + } + if (loader != null) loader.destroy(); + } + + private static MetaObject object(String shortName) { + for (MetaObject mo : loader.getMetaObjects()) { + if (shortName.equals(mo.getShortName())) return mo; + } + throw new AssertionError("no object " + shortName); + } + + /** Read a report and flatten each row by the read model's fields, in field order. */ + private static List> read(String report, QueryOptions options) { + MetaObject declared = object(report); + ObjectConnection oc = omdb.getConnection(); + try { + List> rows = new ArrayList<>(); + for (Object o : omdb.getObjects(oc, declared, options)) { + Map row = new LinkedHashMap<>(); + for (MetaField f : omdb.readObjectFor(declared).getMetaFields()) { + row.put(f.getName(), f.getObject(o)); + } + rows.add(row); + } + return rows; + } finally { + omdb.releaseConnection(oc); + } + } + + private static long count(String report, Expression filter) { + ObjectConnection oc = omdb.getConnection(); + try { + return omdb.getObjectsCount(oc, object(report), filter); + } finally { + omdb.releaseConnection(oc); + } + } + + private static Map row(Object... keyValues) { + Map row = new LinkedHashMap<>(); + for (int i = 0; i < keyValues.length; i += 2) row.put((String) keyValues[i], keyValues[i + 1]); + return row; + } + + private static QueryOptions sortedBy(String field, int direction) { + QueryOptions options = new QueryOptions(); + options.setSortOrder(new SortOrder(field, direction)); + return options; + } + + private static Set columnsOf(ObjectMappingDB mapping) { + Set columns = new HashSet<>(); + mapping.getArguments().forEach(a -> columns.add(a.getName())); + return columns; + } + + private interface Op { + void run(ObjectConnection oc) throws Exception; + } + + /** Runs {@code op} and returns the PersistenceException it must throw. */ + private static PersistenceException refused(String what, Op op) throws Exception { + ObjectConnection oc = omdb.getConnection(); + try { + op.run(oc); + } catch (PersistenceException e) { + return e; + } finally { + omdb.releaseConnection(oc); + } + fail(what + " must be refused"); + return null; + } + + private static void assertReadOnlyNoIdentity(String operation, String report, PersistenceException e) { + String message = e.getMessage(); + assertTrue(message, message.startsWith(operation + " is not supported on [reporttest::" + report + "]")); + assertTrue(message, message.contains("a report is read-only and has no identity")); + } + + // --------------------------------------------------------------------------- + // getObjects / getObjectsCount on a view-backed report + // --------------------------------------------------------------------------- + + @Test + public void getObjectsReturnsTheViewsRowsKeyedByDerivedFieldName() { + List> rows = read("SalesByRegion", sortedBy("region", SortOrder.ASC)); + assertEquals(List.of( + row("region", "east", "sales", 2L, "revenue", 400L, "minAmount", 100L), + row("region", "west", "sales", 1L, "revenue", 50L, "minAmount", 50L)), rows); + } + + @Test + public void rowsAreInstancesOfTheReadModelNotOfTheDeclaredNode() { + MetaObject declared = object("SalesByRegion"); + ObjectConnection oc = omdb.getConnection(); + try { + Object first = omdb.getObjects(oc, declared, new QueryOptions()).iterator().next(); + MetaObject rowMeta = omdb.getMetaObjectFor(first); + assertTrue(rowMeta instanceof ReportReadModel); + assertSame(omdb.readObjectFor(declared), rowMeta); + assertSame("a read model is its own read object", rowMeta, omdb.readObjectFor(rowMeta)); + // The model can be handed back in directly. + assertEquals(2, omdb.getObjects(oc, rowMeta, new QueryOptions()).size()); + assertEquals(2L, omdb.getObjectsCount(oc, rowMeta, null)); + } finally { + omdb.releaseConnection(oc); + } + } + + @Test + public void filtersOnADimensionAndOnAMeasure() { + assertEquals(List.of(row("region", "west", "sales", 1L, "revenue", 50L, "minAmount", 50L)), + read("SalesByRegion", new QueryOptions(new Expression("region", "west")))); + assertEquals(List.of(row("region", "east", "sales", 2L, "revenue", 400L, "minAmount", 100L)), + read("SalesByRegion", new QueryOptions(new Expression("sales", 2L, Expression.EQUAL_GREATER)))); + assertTrue(read("SalesByRegion", new QueryOptions(new Expression("revenue", 1000L, Expression.GREATER))).isEmpty()); + } + + @Test + public void sortsOnAMeasureWithALimit() { + QueryOptions options = new QueryOptions(); + options.setSortOrder(new SortOrder("revenue", SortOrder.DESC)); + options.setRange(new Range(1, 1)); + List> rows = read("SalesByRegion", options); + assertEquals(1, rows.size()); + assertEquals("east", rows.get(0).get("region")); + + options.setSortOrder(new SortOrder("revenue", SortOrder.ASC)); + assertEquals("west", read("SalesByRegion", options).get(0).get("region")); + } + + @Test + public void countWorksWithAndWithoutAFilter() { + assertEquals(2L, count("SalesByRegion", null)); + assertEquals(1L, count("SalesByRegion", new Expression("sales", 2L, Expression.EQUAL_GREATER))); + assertEquals(0L, count("SalesByRegion", new Expression("region", "north"))); + } + + @Test + public void rowsAreDecodedByDerivedSubtype_anIntBackedEnumDimensionReadsAndFiltersAsItsSymbol() { + // The column holds 1 / 2; the derived field carries @values + @intValueMap from Sale.status. + assertEquals(List.of(row("status", "CLOSED", "sales", 1L), row("status", "OPEN", "sales", 2L)), + read("SalesByStatus", sortedBy("status", SortOrder.DESC))); + assertEquals(List.of(row("status", "OPEN", "sales", 2L)), + read("SalesByStatus", new QueryOptions(new Expression("status", "OPEN")))); + } + + @Test + public void anUnknownFieldInAReportFilterOrSortIsRefusedByName() { + try { + read("SalesByRegion", new QueryOptions(new Expression("amountCents", 1L))); + fail("a field of @from is not a field of the report"); + } catch (MetaDataException e) { + assertTrue(e.getMessage(), e.getMessage().contains("amountCents")); + } + try { + read("SalesByRegion", sortedBy("nope", SortOrder.ASC)); + fail("an unknown sort field must be refused"); + } catch (MetaDataException e) { + assertTrue(e.getMessage(), e.getMessage().contains("nope")); + } + } + + @Test + public void anUnmanagedReportIsStillRead() { + assertEquals(List.of(row("sales", 3L)), read("UnmanagedSales", new QueryOptions())); + assertEquals(1L, count("UnmanagedSales", null)); + } + + @Test + public void aReplicaDeclaredBeforeThePrimaryView_readsComeFromThePrimaryView() { + // RPT_V_REPLICA answers 103 and the REPLICATED_SALES fallback table answers 999. + assertEquals(List.of(row("sales", 3L)), read("ReplicatedSales", new QueryOptions())); + } + + // --------------------------------------------------------------------------- + // The mapping + // --------------------------------------------------------------------------- + + @Test + public void theMappingIsTheViewWithExactlyTheDerivedColumnsAndNoKeyColumn() { + ObjectMappingDB mapping = (ObjectMappingDB) omdb.getReadMapping(object("SalesByRegion")); + assertEquals("RPT_V_BY_REGION", ((BaseDef) mapping.getDBDef()).getNameDef().getName()); + // A mapping's arguments are unordered; the set is what is asserted. + assertEquals(Set.of("region", "sales", "revenue", "minAmount"), columnsOf(mapping)); + assertSame("the declared node and its model share one mapping", + mapping, omdb.getReadMapping(omdb.readObjectFor(object("SalesByRegion")))); + } + + @Test + public void theNamingStrategyAppliesToTheDerivedFieldName() { + SimpleMappingHandlerDB handler = new SimpleMappingHandlerDB(); + handler.setColumnNaming("snake_case"); + ObjectMappingDB mapping = (ObjectMappingDB) handler.getReadMapping(omdb.readObjectFor(object("SalesByRegion"))); + assertEquals(Set.of("region", "sales", "revenue", "min_amount"), columnsOf(mapping)); + } + + @Test + public void theDeclaredReportNodePassedStraightToTheMappingHandlerIsRefusedByName() { + try { + new SimpleMappingHandlerDB().getReadMapping(object("SalesByRegion")); + fail("a declared report has no fields to map"); + } catch (MetaDataException e) { + assertTrue(e.getMessage(), e.getMessage().contains("Report [reporttest::SalesByRegion] has no fields to map")); + } + } + + @Test + public void aReportIsReadableAndNothingElse() { + MetaObject report = object("SalesByRegion"); + assertTrue(omdb.isReadableClass(report)); + assertFalse(omdb.isCreateableClass(report)); + assertFalse(omdb.isUpdateableClass(report)); + assertFalse(omdb.isDeleteableClass(report)); + assertFalse("a sourceless report has no read mapping", omdb.isReadableClass(object("InertSales"))); + } + + // --------------------------------------------------------------------------- + // By-id and every write: read-only, no identity + // --------------------------------------------------------------------------- + + @Test + public void byIdAndEveryWriteOnAReportAreRefused_readOnlyNoIdentity() throws Exception { + MetaObject report = object("SalesByRegion"); + ObjectConnection reader = omdb.getConnection(); + Object row; + try { + row = omdb.getObjects(reader, report, new QueryOptions()).iterator().next(); + } finally { + omdb.releaseConnection(reader); + } + // A row read from the view (an instance of the read model) and one built from the declared node. + Object built = report.newInstance(); + + for (Object instance : List.of(row, built)) { + assertReadOnlyNoIdentity("createObject", "SalesByRegion", + refused("createObject", oc -> omdb.createObject(oc, instance))); + assertReadOnlyNoIdentity("updateObject", "SalesByRegion", + refused("updateObject", oc -> omdb.updateObject(oc, instance))); + assertReadOnlyNoIdentity("deleteObject", "SalesByRegion", + refused("deleteObject", oc -> omdb.deleteObject(oc, instance))); + assertReadOnlyNoIdentity("loadObject", "SalesByRegion", + refused("loadObject", oc -> omdb.loadObject(oc, instance))); + assertReadOnlyNoIdentity("getObjectRef", "SalesByRegion", + refused("getObjectRef", oc -> omdb.getObjectRef(instance))); + } + + assertReadOnlyNoIdentity("deleteObjects", "SalesByRegion", + refused("deleteObjects", oc -> omdb.deleteObjects(oc, report, new Expression("region", "east")))); + assertReadOnlyNoIdentity("createObjectsBulk", "SalesByRegion", + refused("createObjectsBulk", oc -> omdb.createObjectsBulk(oc, report, new ArrayList<>(List.of(built))))); + assertReadOnlyNoIdentity("updateObjectsBulk", "SalesByRegion", + refused("updateObjectsBulk", oc -> omdb.updateObjectsBulk(oc, report, new ArrayList<>(List.of(built))))); + assertReadOnlyNoIdentity("getObjectByRef", "SalesByRegion", + refused("getObjectByRef", oc -> omdb.getObjectByRef(oc, "objectref://reporttest::SalesByRegion/east"))); + + // Nothing was written: the view still answers what the seed rows produce. + assertEquals(2L, count("SalesByRegion", null)); + } + + // --------------------------------------------------------------------------- + // A sourceless report is not served + // --------------------------------------------------------------------------- + + @Test + public void aSourcelessReportIsNotServed() throws Exception { + MetaObject inert = object("InertSales"); + // INERT_SALES holds a decoy row: a fallback to a default table name would return it. + PersistenceException read = refused("getObjects", oc -> omdb.getObjects(oc, inert, new QueryOptions())); + assertTrue(read.getMessage(), read.getMessage().contains("Report [reporttest::InertSales] is not served")); + assertTrue(read.getMessage(), read.getMessage().contains("no view to read")); + PersistenceException counted = refused("getObjectsCount", oc -> omdb.getObjectsCount(oc, inert, null)); + assertTrue(counted.getMessage(), counted.getMessage().contains("is not served")); + } + + @Test + public void aWriteOnASourcelessReportIsRefusedAsReadOnlyNotAsUnserved() throws Exception { + MetaObject inert = object("InertSales"); + assertReadOnlyNoIdentity("createObject", "InertSales", + refused("createObject", oc -> omdb.createObject(oc, inert.newInstance()))); + assertReadOnlyNoIdentity("deleteObjects", "InertSales", + refused("deleteObjects", oc -> omdb.deleteObjects(oc, inert, null))); + } + + // --------------------------------------------------------------------------- + // Nothing else changes + // --------------------------------------------------------------------------- + + @Test + public void readingReportsLeavesTheLoadedModelUntouched() { + String before = CanonicalJsonSerializer.canonicalSerialize(loader.getRoot()); + int objects = loader.getMetaObjects().size(); + + read("SalesByRegion", new QueryOptions()); + read("SalesByStatus", new QueryOptions()); + read("UnmanagedSales", new QueryOptions()); + read("ReplicatedSales", new QueryOptions()); + count("SalesByRegion", null); + + assertEquals(objects, loader.getMetaObjects().size()); + assertEquals(before, CanonicalJsonSerializer.canonicalSerialize(loader.getRoot())); + assertTrue("the declared node still declares no fields", object("SalesByRegion").getMetaFields().isEmpty()); + assertFalse(loader.getMetaObjects().contains(omdb.readObjectFor(object("SalesByRegion")))); + } + + @Test + public void anEntityInTheSameModelIsReadAndWrittenAsBefore() throws Exception { + MetaObject sale = object("Sale"); + assertSame("a non-report object is its own read object", sale, omdb.readObjectFor(sale)); + assertTrue(omdb.isCreateableClass(sale)); + + ObjectConnection oc = omdb.getConnection(); + try { + ValueObject vo = (ValueObject) sale.newInstance(); + vo.setLong("id", 10L); + vo.setString("region", "north"); + vo.setString("status", "CLOSED"); + vo.setLong("amountCents", 700L); + omdb.createObject(oc, vo); + assertEquals(4L, omdb.getObjectsCount(oc, sale, null)); + + ObjectRef ref = omdb.getObjectRef(vo); + assertNotNull(ref); + ValueObject byRef = (ValueObject) omdb.getObjectByRef(oc, "objectref://reporttest::Sale/10"); + assertEquals("north", byRef.getString("region")); + + Collection found = omdb.getObjects(oc, sale, new QueryOptions(new Expression("id", 10L))); + assertEquals(1, found.size()); + ValueObject loaded = (ValueObject) found.iterator().next(); + assertSame(sale, omdb.getMetaObjectFor(loaded)); + assertEquals("CLOSED", loaded.getString("status")); + + loaded.setLong("amountCents", 800L); + omdb.updateObject(oc, loaded); + ValueObject reloaded = (ValueObject) sale.newInstance(); + reloaded.setLong("id", 10L); + omdb.loadObject(oc, reloaded); + assertEquals(Long.valueOf(800L), reloaded.getLong("amountCents")); + + // The report sees the entity's write through its view, and the entity's delete. + assertEquals(3L, omdb.getObjectsCount(oc, object("SalesByRegion"), null)); + omdb.deleteObject(oc, reloaded); + assertEquals(1, omdb.deleteObjects(oc, sale, new Expression("id", 3L))); + assertEquals(2L, omdb.getObjectsCount(oc, sale, null)); + vo = (ValueObject) sale.newInstance(); + vo.setLong("id", 3L); + vo.setString("region", "west"); + vo.setString("status", "OPEN"); + vo.setLong("amountCents", 50L); + omdb.createObject(oc, vo); // restore the seed row the other tests read + assertEquals(3L, omdb.getObjectsCount(oc, sale, null)); + } finally { + omdb.releaseConnection(oc); + } + } + + // --------------------------------------------------------------------------- + // OQL with a report as the result class + // --------------------------------------------------------------------------- + + /** Run an OQL query and flatten each row by the fields of {@code rowShape}. */ + private static List> query(String oql, MetaObject rowShape) { + ObjectConnection oc = omdb.getConnection(); + try { + List> rows = new ArrayList<>(); + for (Object o : omdb.executeQuery(oc, oql, new ArrayList<>())) { + assertSame("an OQL row of a report is an instance of its read model", + rowShape, omdb.getMetaObjectFor(o)); + Map row = new LinkedHashMap<>(); + for (MetaField f : rowShape.getMetaFields()) row.put(f.getName(), f.getObject(o)); + rows.add(row); + } + return rows; + } finally { + omdb.releaseConnection(oc); + } + } + + @Test + public void oqlWithAReportResultClassBuildsRowsFromTheReadModel() { + MetaObject declared = object("SalesByRegion"); + // The author supplies the SQL; the report supplies the row shape. + assertEquals(List.of(row("region", "west", "sales", 1L, "revenue", 50L, "minAmount", 50L)), + // Aliases are quoted because OQL binds a result column by its exact name and + // Derby upper-cases an unquoted one (true of any OQL result class). + query("[reporttest::SalesByRegion] SELECT region AS \"region\", sales AS \"sales\"," + + " revenue AS \"revenue\", minAmount AS \"minAmount\"" + + " FROM RPT_V_BY_REGION WHERE sales < 2", ReportReadModel.of(declared))); + } + + @Test + public void oqlWithASourcelessReportResultClassStillBuildsRows() { + // InertSales has no view, so it is not served by getObjects; as an OQL result shape + // it only names the columns, and they bind by derived field name. + assertEquals(List.of(row("sales", 3L)), + query("[reporttest::InertSales] SELECT COUNT(id) AS \"sales\" FROM RPT_SALES", + ReportReadModel.of(object("InertSales")))); + } + + // --------------------------------------------------------------------------- + // A projection declared with @view (the same physical-name rule as a report) + // --------------------------------------------------------------------------- + + @Test + public void aProjectionDeclaredWithViewIsReadFromThatView() { + MetaObject projection = object("SaleRegionView"); + ObjectMappingDB mapping = (ObjectMappingDB) omdb.getReadMapping(projection); + assertNotNull("a projection whose view is named by @view has a read mapping", mapping); + assertEquals("RPT_V_SALE_REGIONS", ((BaseDef) mapping.getDBDef()).getNameDef().getName()); + + ObjectConnection oc = omdb.getConnection(); + try { + QueryOptions options = new QueryOptions(new Expression("region", "west")); + Collection found = omdb.getObjects(oc, projection, options); + assertEquals(1, found.size()); + assertEquals(Long.valueOf(3L), ((ValueObject) found.iterator().next()).getLong("id")); + } finally { + omdb.releaseConnection(oc); + } + } + + // --------------------------------------------------------------------------- + // getObjectRef on an object that is not a report + // --------------------------------------------------------------------------- + + @Test + public void getObjectRefOnAnObjectWithNoMetadataFailsExactlyAsTheBaseManagerDoes() { + Object stranger = new Object(); + Throwable base = null; + try { + com.metaobjects.util.MetaDataUtil.findMetaObject(stranger, omdb); + } catch (RuntimeException e) { + base = e; + } + assertNotNull("the base lookup refuses an object with no metadata", base); + try { + omdb.getObjectRef(stranger); + fail("an object with no metadata has no reference"); + } catch (RuntimeException e) { + assertSame(base.getClass(), e.getClass()); + assertEquals(base.getMessage(), e.getMessage()); + } + } +} diff --git a/server/java/omdb/src/test/resources/meta.report.json b/server/java/omdb/src/test/resources/meta.report.json new file mode 100644 index 000000000..8af329aaf --- /dev/null +++ b/server/java/omdb/src/test/resources/meta.report.json @@ -0,0 +1,43 @@ +{ + "metadata.root": { + "package": "reporttest", + "children": [ + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "RPT_SALES" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "region", "@required": true, "@maxLength": 8 } }, + { "field.enum": { "name": "status", "@required": true, "@values": ["OPEN", "CLOSED"], + "@intValueMap": { "OPEN": 1, "CLOSED": 2 } } }, + { "field.long": { "name": "amountCents", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "assigned" } }, + { "dimension.attribute": { "name": "region", "@of": "Sale.region" } }, + { "dimension.attribute": { "name": "status", "@of": "Sale.status" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "minAmount", "@agg": "min", "@of": "Sale.amountCents" } } + ] } }, + { "object.report": { "name": "SalesByRegion", "@from": "Sale", "@dimensions": ["region"], + "@measures": ["sales", "revenue", "minAmount"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "RPT_V_BY_REGION" } } + ] } }, + { "object.report": { "name": "SalesByStatus", "@from": "Sale", "@dimensions": ["status"], + "@measures": ["sales"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "RPT_V_BY_STATUS" } } + ] } }, + { "object.report": { "name": "UnmanagedSales", "@from": "Sale", "@measures": ["sales"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "RPT_V_HAND_MADE", "@unmanaged": true } } + ] } }, + { "object.report": { "name": "ReplicatedSales", "@from": "Sale", "@measures": ["sales"], "children": [ + { "source.rdb": { "name": "replica", "@kind": "view", "@view": "RPT_V_REPLICA", "@role": "replica" } }, + { "source.rdb": { "name": "main", "@kind": "view", "@view": "RPT_V_PRIMARY" } } + ] } }, + { "object.report": { "name": "InertSales", "@from": "Sale", "@measures": ["sales"] } }, + { "object.projection": { "name": "SaleRegionView", "children": [ + { "source.rdb": { "@kind": "view", "@view": "RPT_V_SALE_REGIONS" } }, + { "field.long": { "name": "id", "extends": "reporttest::Sale.id" } }, + { "field.string": { "name": "region", "extends": "reporttest::Sale.region" } }, + { "identity.primary": { "name": "pk", "extends": "reporttest::Sale.pk" } } + ] } } + ] + } +} diff --git a/server/python/src/metaobjects/meta/core/reporting/report_accessors.py b/server/python/src/metaobjects/meta/core/reporting/report_accessors.py index 482b171f7..1cc4335c8 100644 --- a/server/python/src/metaobjects/meta/core/reporting/report_accessors.py +++ b/server/python/src/metaobjects/meta/core/reporting/report_accessors.py @@ -9,6 +9,7 @@ from dataclasses import dataclass +from ....naming_refs import CHILD_REF_SEP from ...meta_data import MetaData from ..object.object_constants import ( OBJECT_REPORT_ATTR_DIMENSIONS, @@ -52,10 +53,29 @@ def report_dimension_items(obj: MetaData) -> list[ReportDimensionItem]: def report_measure_names(obj: MetaData) -> list[str]: - """ADR-0039: resolving. The ``@measures`` names.""" + """ADR-0039: resolving. The ``@measures`` items AS WRITTEN: each a bare measure + ``name``, or a dotted ``Entity.name`` (loader rule R3). Use + :func:`report_measure_item_name` for the measure name.""" return _string_list(obj.get_meta_attr(OBJECT_REPORT_ATTR_MEASURES)) +def report_measure_item_name(item: str) -> str: + """The measure a ``@measures`` item names: the segment after its LAST ``.`` + (``total``, ``Sale.total`` and ``acme::shop::Sale.total`` all name ``total``). It is + also the derived report field's name. The part before that ``.``, when present, is an + entity qualifier (:func:`report_measure_item_owner`).""" + dot = item.rfind(CHILD_REF_SEP) + return item if dot == -1 else item[dot + len(CHILD_REF_SEP):] + + +def report_measure_item_owner(item: str) -> str | None: + """The entity qualifier of a dotted ``@measures`` item (``Sale`` in ``Sale.total``), + or ``None`` for a bare item. Loader rule R3: it names ``@from`` or an entity ``@from`` + extends.""" + dot = item.rfind(CHILD_REF_SEP) + return None if dot == -1 else item[:dot] + + def report_derived_field_name(item: ReportDimensionItem) -> str: """The derived report field for a dimension item: ``name`` (attribute) or ``name`` + Capitalized(grain) (time), e.g. ``purchasedAt:day`` -> ``purchasedAtDay``.""" diff --git a/server/python/src/metaobjects/meta/core/reporting/report_read_model.py b/server/python/src/metaobjects/meta/core/reporting/report_read_model.py new file mode 100644 index 000000000..854305ac1 --- /dev/null +++ b/server/python/src/metaobjects/meta/core/reporting/report_read_model.py @@ -0,0 +1,169 @@ +"""A report's READ MODEL (FR-044): a detached object carrying one real ``field.*`` +child per derived field (Table B) and a copy of the report's own read-only source. + +WHY IT EXISTS + +An ``object.report`` declares no fields: its read shape is derived from its +dimensions and measures. A metadata-driven runtime walks an object's field +children in a dozen places (column list, filter and sort resolution, the name +map, every read coercion). Rather than teach each of them what a report is, the +runtime reads a report through this model and sees ordinary fields. + +WHY IT IS DETACHED + +The model is never added to the root: it has no parent, ``root.children()`` does +not list it, and the canonical serializer, ``fmt``, codegen and every other tree +walker never see it. Nothing in the loaded tree is mutated to build it; in +particular the report's own source node is COPIED, not re-parented +(``add_child`` rewrites the child's ``parent``). The nodes are constructed +directly, not through the loader or the registry, so no vocabulary is added: +every node is an already-registered ``type.subType``. + +It keeps the report's name, package and ``object.report`` subtype, so a consumer +holding it can still tell it is a report (no identity, read-only). + +Mirrors TS ``core/reporting/report-read-model.ts``. ADR-0039 / Python naming +inversion: ``attr()`` is OWN-ONLY here, so the TS ``attr()`` reads are +``get_meta_attr()`` below. +""" +from __future__ import annotations + +from weakref import WeakKeyDictionary + +from ....shared.base_types import TYPE_FIELD +from ...meta_root import MetaRoot +from ...persistence.db.db_constants import FIELD_ATTR_DB_COLUMN_TYPE, FIELD_ATTR_LOCAL_TIME +from ...persistence.source.meta_source import MetaSource +from ...persistence.source.source_constants import SOURCE_ATTR_ROLE, SOURCE_ROLE_PRIMARY +from ..field.field_constants import ( + FIELD_ATTR_CURRENCY, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_PRECISION, + FIELD_ATTR_REQUIRED, + FIELD_ATTR_SCALE, + FIELD_ATTR_STORAGE, + FIELD_ATTR_VALUES, +) +from ..field.meta_field import MetaField +from ..object.meta_object import MetaObject +from .report_shape import ReportField, report_shape + +#: Table B: the type-shaping attrs a derived field carries from its ``type_source``, +#: read with the RESOLVING accessor (ADR-0039) so a value the ``@of`` field inherits +#: through ``extends`` is carried too. ``@dbColumnType`` and ``isArray`` are handled +#: separately below. Nothing else is carried: no ``@column``, ``@required``, +#: ``@default``, validators or views. +_CARRIED_ATTRS: tuple[str, ...] = ( + FIELD_ATTR_CURRENCY, + FIELD_ATTR_VALUES, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_PRECISION, + FIELD_ATTR_SCALE, + FIELD_ATTR_LOCAL_TIME, + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_STORAGE, +) + +#: Read models cached per report node (identity-keyed; nodes define no ``__eq__``). +_READ_MODELS: "WeakKeyDictionary[MetaObject, MetaObject]" = WeakKeyDictionary() + + +def _derived_field(f: ReportField) -> MetaField: + field = MetaField(TYPE_FIELD, f.sub_type, f.name) + # From the derived shape, never from the type source: a ``min`` of a required column + # is still nullable, and a dimension reached by ``@via`` is nullable. + field.set_attr(FIELD_ATTR_REQUIRED, f.required) + src = f.type_source + if src is not None: + for name in _CARRIED_ATTRS: + value = src.get_meta_attr(name) + if value is not None: + field.set_attr(name, value) + # ADR-0039: own — ``@dbColumnType`` is the one deliberately own-only attr (a + # physical column-type override is never inherited), and every consumer reads + # it with ``attr()`` (own). So it is read own from the type source and set OWN + # here: the derived field carries exactly what the ``@of`` field itself + # declares, and nothing its supers declare. + db_column_type = src.attr(FIELD_ATTR_DB_COLUMN_TYPE) + if db_column_type is not None: + field.set_attr(FIELD_ATTR_DB_COLUMN_TYPE, db_column_type) + # ``is_array`` is a native flag, not an attr; ``resolved_is_array()`` is its resolving read. + if src.resolved_is_array(): + field.is_array = True + return field + + +def report_read_source(report: MetaObject) -> MetaSource | None: + """The source a report is READ from: its own read-only source with ``@role: primary``, + else its first own read-only source. ``None`` when it declares none (Table A: not + lowered, not served). + + This is the rule that NAMES the lowered view (``viewName`` / ``projectionViewSource`` + in codegen-ts's ``projection/extract-view-spec.ts``). It is restated here because + this package cannot depend on the TS codegen; the two must stay the same rule, or + the runtime reads a relation the lowering did not create. + + What the loader permits: a report may declare several read-only sources (a + ``@role: replica`` view beside its primary view loads clean, in either order); + ``@role`` defaults to ``primary``; a report whose sources include no primary is + ``ERR_SOURCE_NO_PRIMARY``; a writable source on a report is refused. So for every + model that loads, the primary branch fires. The first-read-only fallback covers a + tree built in code and keeps this rule identical to the lowering's. + """ + # ADR-0039: own — source classification reads the sources the report declares + # ITSELF, exactly as the lowering's ``viewName`` does. + read_only = [c for c in report.own_children() if isinstance(c, MetaSource) and c.is_read_only()] + return next((s for s in read_only if s.role() == SOURCE_ROLE_PRIMARY), read_only[0] if read_only else None) + + +def _copy_source(source: MetaSource) -> MetaSource: + """A detached copy of a source node: same ``type.subType``, name and effective attrs, + and nothing else (attrs only — the loaded node is never re-parented). + + The copy is the model's ONLY source, and it is pinned to ``@role: primary``: the + runtime resolves an object's table through ``primary_rdb_source``, which considers + primary sources only, so this is what makes the read land on the selected source's + physical name rather than on a default table name nobody declared.""" + copy = MetaSource(source.type, source.sub_type, source.name) + # ADR-0039: resolving — the copy carries the source's effective configuration + # (@kind, the physical-name alias, @schema, @unmanaged, @sql). + for name, value in source.attrs().items(): + copy.set_attr(name, value) + copy.set_attr(SOURCE_ATTR_ROLE, SOURCE_ROLE_PRIMARY) + return copy + + +def report_read_model(report: MetaObject, root: MetaRoot) -> MetaObject: + """The read model of an ``object.report``: one field per Table B row, in Table B + order, plus a copy of the source the report is read from (see + :func:`report_read_source`) when it declares one (Table A). A sourceless report + yields a model with no source: it has a shape and no view, and the caller decides + what that means (the runtime refuses to serve it). + + Cached per report node (identity-keyed); the model is frozen. Raises what + :func:`report_shape` raises when a reference does not resolve. + """ + cached = _READ_MODELS.get(report) + if cached is not None: + return cached + + shape = report_shape(report, root) + model = MetaObject(report.type, report.sub_type, report.name) + model.package = report.package + model.file_default_package = report.file_default_package + for f in shape.fields: + model.add_child(_derived_field(f)) + + source = report_read_source(report) + if source is not None: + model.add_child(_copy_source(source)) + + model.freeze() + _READ_MODELS[report] = model + return model + + +__all__ = ["report_read_model", "report_read_source"] diff --git a/server/python/src/metaobjects/meta/core/reporting/report_shape.py b/server/python/src/metaobjects/meta/core/reporting/report_shape.py new file mode 100644 index 000000000..0cbfd268b --- /dev/null +++ b/server/python/src/metaobjects/meta/core/reporting/report_shape.py @@ -0,0 +1,235 @@ +"""Table B of the FR-044 Plan 2 contract: a report's derived fields. The single +Python definition; every port has a rule-for-rule copy, gated by +``fixtures/persistence-conformance/report-shapes.json``. + +ADR-0039: every read is RESOLVING. Python naming inversion: ``attr()`` is +OWN-ONLY here, so the TypeScript reference's ``attr()`` is ``get_meta_attr()`` +below, and ``fields()`` / ``children()`` are the resolving member accessors. +Mirrors TS ``core/reporting/report-shape.ts``. +""" +from __future__ import annotations + +from dataclasses import dataclass + +from ....naming_refs import CHILD_REF_SEP, resolve_object_ref +from ....shared.separators import PACKAGE_SEP +from ...meta_data import MetaData +from ...meta_root import MetaRoot +from ..field.field_constants import ( + FIELD_ATTR_REQUIRED, + FIELD_SUBTYPE_CURRENCY, + FIELD_SUBTYPE_DATE, + FIELD_SUBTYPE_DECIMAL, + FIELD_SUBTYPE_DOUBLE, + FIELD_SUBTYPE_FLOAT, + FIELD_SUBTYPE_INT, + FIELD_SUBTYPE_LONG, + FIELD_SUBTYPE_TIMESTAMP, +) +from ..field.meta_field import MetaField +from ..object.meta_object import MetaObject +from .meta_dimension import MetaDimension +from .meta_measure import MetaMeasure +from .report_accessors import ( + ReportDimensionItem, + report_derived_field_name, + report_dimension_items, + report_from, + report_measure_item_name, + report_measure_item_owner, + report_measure_names, +) +from .reporting_constants import ( + AGG_AVG, + AGG_COUNT, + AGG_SUM, + GRAIN_HOUR, + TIME_GRAINS, + TYPE_DIMENSION, + TYPE_MEASURE, +) + +ROLE_DIMENSION = "dimension" +ROLE_MEASURE = "measure" + +_SUM_LONG = frozenset({FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG}) +_FLOATING = frozenset({FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT}) + + +@dataclass(frozen=True) +class ReportField: + name: str + role: str # ROLE_DIMENSION | ROLE_MEASURE + #: A field subtype name (``FIELD_SUBTYPE_*``). + sub_type: str + required: bool + #: The ``@of`` field whose type-shaping attrs this field carries (Table B). + type_source: MetaField | None = None + dimension: MetaDimension | None = None + grain: str | None = None + measure: MetaMeasure | None = None + + +@dataclass(frozen=True) +class ReportShape: + report: MetaObject + from_: MetaObject + fields: tuple[ReportField, ...] + + +def _package_of_key(key: str) -> str: + """Effective package of a node, taken from its resolution key (``::``).""" + i = key.rfind(PACKAGE_SEP) + return key[:i] if i >= 0 else "" + + +def _is_self_or_ancestor(candidate: MetaData | None, entity: MetaData) -> bool: + """True when ``candidate`` is ``entity`` or an entity it extends (the super chain). + The loader's own test (``validate_reporting._is_self_or_ancestor``), restated because + this package cannot import the loader (the loader imports it).""" + visited: set[int] = set() + n: MetaData | None = entity + while n is not None and id(n) not in visited: + if n is candidate: + return True + visited.add(id(n)) + n = n.super_data + return False + + +def reporting_member_owner(member: MetaData, from_: MetaObject) -> MetaData: + """The entity that DECLARES a dimension or measure reached through ``from_``: the + member's parent, which is ``from_`` itself or an entity ``from_`` extends. A bare + entity name inside the member (``@of``, ``@via``) resolves in THIS entity's package, + exactly as the loader's ``validate_reporting`` resolves it (``_pkg_of(ctx.declaring)``), + never in ``from_``'s package or the report's.""" + return member.parent if member.parent is not None else from_ + + +def resolve_reporting_field_ref( + ref: str, declaring: MetaData, root: MetaRoot, host: MetaObject | None = None +) -> MetaField | None: + """Resolve a dimension's or measure's ``Entity.field`` reference to the field node. + The ONE rule, the same as the loader's (``validate_reporting`` D1 / M1) and as the + TypeScript ``resolveReportingFieldRef``: + + 1. The entity half resolves relative to the package of ``declaring``, the entity that + declares the member (:func:`reporting_member_owner`). + 2. With ``host`` (a measure, or a dimension without ``@via``: the reference is about + the ``@from`` entity's own rows) the named entity must be ``host`` or an entity it + extends, and the field is read from ``host``, so a field ``host`` redeclares wins. + 3. Without ``host`` (a dimension with ``@via``) the field is read from the named entity. + + ``None`` when any step fails. + """ + # ``Entity.field``; a package qualifier uses ``::``, so the member separator is the LAST dot. + dot = ref.rfind(CHILD_REF_SEP) + if dot <= 0: + return None + named = resolve_object_ref(root, ref[:dot], _package_of_key(declaring.resolution_key())) + if not isinstance(named, MetaObject): + return None + if host is not None and not _is_self_or_ancestor(named, host): + return None + # ADR-0039: resolving fields(), so a field inherited through extends is found. + member = ref[dot + len(CHILD_REF_SEP):] + return next((f for f in (host if host is not None else named).fields() if f.name == member), None) + + +def _unresolved(report_name: str, what: str) -> ValueError: + return ValueError(f"report '{report_name}': {what} does not resolve.") + + +def _declared_member(from_: MetaObject, type_: str, name: str, cls: type[MetaData]) -> MetaData | None: + # ADR-0039: resolving children(), so a member declared on an abstract base is found. + return next( + (c for c in from_.children() if c.type == type_ and c.name == name and isinstance(c, cls)), + None, + ) + + +def _dimension_field( + item: ReportDimensionItem, from_: MetaObject, root: MetaRoot, report_name: str +) -> ReportField: + dim = _declared_member(from_, TYPE_DIMENSION, item.name, MetaDimension) + if not isinstance(dim, MetaDimension): + raise _unresolved(report_name, f"dimension '{item.name}' on '{from_.name}'") + vialess = dim.via() is None + of = resolve_reporting_field_ref( + dim.of() or "", reporting_member_owner(dim, from_), root, from_ if vialess else None + ) + if of is None: + raise _unresolved(report_name, f"dimension '{item.name}' @of") + name = report_derived_field_name(item) + # ADR-0039 resolving: the @of field's effective @required (the attr only; a + # validator.required child does not count). + required = vialess and of.get_meta_attr(FIELD_ATTR_REQUIRED) is True + if dim.is_time(): + # Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. + grain = item.grain + if grain is None or grain not in TIME_GRAINS: + raise _unresolved(report_name, f"time dimension '{item.name}' grain '{grain or ''}'") + if grain == GRAIN_HOUR: + return ReportField( + name, ROLE_DIMENSION, FIELD_SUBTYPE_TIMESTAMP, required, of, dimension=dim, grain=grain + ) + return ReportField(name, ROLE_DIMENSION, FIELD_SUBTYPE_DATE, required, None, dimension=dim, grain=grain) + return ReportField(name, ROLE_DIMENSION, of.sub_type, required, of, dimension=dim) + + +def _measure_field(item: str, report: MetaObject, from_: MetaObject, root: MetaRoot) -> ReportField: + """One ``@measures`` item, bare (``total``) or dotted (``Sale.total``, loader rule R3). + The measure is named by the item's last segment and looked up on ``from_``; a qualifier + resolves in the REPORT's package and must be ``from_`` or an entity ``from_`` extends.""" + report_name = report.name + name = report_measure_item_name(item) + qualifier = report_measure_item_owner(item) + if qualifier is not None: + owner = resolve_object_ref(root, qualifier, _package_of_key(report.resolution_key())) + if owner is None or not _is_self_or_ancestor(owner, from_): + raise _unresolved(report_name, f"measure '{item}' on '{from_.name}'") + m = _declared_member(from_, TYPE_MEASURE, name, MetaMeasure) + if not isinstance(m, MetaMeasure): + raise _unresolved(report_name, f"measure '{item}' on '{from_.name}'") + if m.is_ratio(): + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_DECIMAL, False, measure=m) + agg = m.agg() + if agg == AGG_COUNT: + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_LONG, True, measure=m) + cols = m.of_columns() + of = resolve_reporting_field_ref(cols[0] if cols else "", reporting_member_owner(m, from_), root, from_) + if of is None: + raise _unresolved(report_name, f"measure '{name}' @of") + src = of.sub_type + if agg == AGG_SUM: + if src == FIELD_SUBTYPE_CURRENCY: + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_CURRENCY, False, of, measure=m) + sub_type = ( + FIELD_SUBTYPE_LONG + if src in _SUM_LONG + else FIELD_SUBTYPE_DOUBLE + if src in _FLOATING + else FIELD_SUBTYPE_DECIMAL + ) + return ReportField(name, ROLE_MEASURE, sub_type, False, measure=m) + if agg == AGG_AVG: + sub_type = FIELD_SUBTYPE_DOUBLE if src in _FLOATING else FIELD_SUBTYPE_DECIMAL + return ReportField(name, ROLE_MEASURE, sub_type, False, measure=m) + # min / max keep the source field's type. + return ReportField(name, ROLE_MEASURE, src, False, of, measure=m) + + +def report_shape(report: MetaObject, root: MetaRoot) -> ReportShape: + """Table B. Raises a ``ValueError`` naming the report when a reference does not + resolve (a report that passed ``validate_reporting`` always resolves).""" + from_name = report_from(report) + if from_name is None: + raise _unresolved(report.name, "@from") + from_ = resolve_object_ref(root, from_name, _package_of_key(report.resolution_key())) + if not isinstance(from_, MetaObject): + raise _unresolved(report.name, f"@from '{from_name}'") + fields = ( + *(_dimension_field(item, from_, root, report.name) for item in report_dimension_items(report)), + *(_measure_field(item, report, from_, root) for item in report_measure_names(report)), + ) + return ReportShape(report, from_, tuple(fields)) diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index 009ecc125..c111e024d 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -34,6 +34,8 @@ from ..meta.meta_root import MetaRoot from ..meta.core.object.meta_object import MetaObject +from ..meta.core.object.object_constants import OBJECT_SUBTYPE_REPORT +from ..meta.core.reporting.report_read_model import report_read_model from ..meta.core.field.meta_field import MetaField from ..meta.core.field import field_constants as fc from ..naming import DEFAULT_COLUMN_NAMING, resolve_column_name @@ -213,6 +215,7 @@ def __init__( # --- Public API ---------------------------------------------------------- def find_by_id(self, entity_name: str, id_value: Any) -> dict[str, Any] | None: + self._refuse_report("find_by_id", entity_name) entity = self._require_entity(entity_name) pk_field = self._primary_pk_field(entity) rows = self.find_many(entity_name, {pk_field: id_value}, sort=None, limit=1, offset=None) @@ -238,6 +241,7 @@ def create(self, entity_name: str, data: dict[str, Any]) -> dict[str, Any]: timestamp equals its created one. Use :meth:`insert_preserving` for the import/restore path that must keep original timestamps. """ + self._refuse_report("create", entity_name) entity = self._require_entity(entity_name) # #203: stamp every onCreate AND onUpdate column with one shared now() (the # caller's value is ignored). No-op for entities that declare no @autoSet field. @@ -259,6 +263,7 @@ def insert_preserving(self, entity_name: str, data: dict[str, Any]) -> dict[str, same TPH discriminator injection, same ``RETURNING`` row). For an entity that declares no ``@autoSet`` field this is identical to :meth:`create`. """ + self._refuse_report("insert_preserving", entity_name) entity = self._require_entity(entity_name) return self._insert_row(entity, entity_name, data) @@ -365,6 +370,7 @@ def update( subtype's patch strips it and the by-id write is scoped to the subtype (a cross-subtype id matches no row → the same not-found path). """ + self._refuse_report("update", entity_name) entity = self._require_entity(entity_name) table = self._table_name(entity) pk_field = self._primary_pk_field(entity) @@ -466,6 +472,7 @@ def delete(self, entity_name: str, id_value: Any) -> bool: Returns ``True`` when a row was deleted, ``False`` when the PK matched nothing. Mirrors the TS ``om.delete`` boolean outcome contract. """ + self._refuse_report("delete", entity_name) entity = self._require_entity(entity_name) table = self._table_name(entity) pk_field = self._primary_pk_field(entity) @@ -560,6 +567,7 @@ def relate( mirror the TS reference resolver. ``record`` is a source-key dict (e.g. ``{"id": 1}``); only the source PK is read from it. """ + self._refuse_report("relate", entity_name) entity = self._require_entity(entity_name) desc = resolve_n2m_descriptor(entity, relation_name, self._entity_by_name) if desc is None: @@ -623,12 +631,50 @@ def _relate_n2m( # --- Helpers ------------------------------------------------------------- - def _require_entity(self, name: str) -> MetaObject: + def _declared_entity(self, name: str) -> MetaObject: + """The object node exactly as loaded (a report is NOT swapped for its read model).""" e = self._entity_by_name.get(name) if e is None: raise KeyError(f"No entity named '{name}' in loaded metadata") return e + def _require_entity(self, name: str) -> MetaObject: + """The object the runtime reads and writes through. FR-044: a report declares no + fields (its read shape is derived), so it is read through its detached read model + (:func:`report_read_model`) — ordinary derived ``field.*`` children plus a copy of + its own read-only source — and everything downstream (column list, filter and sort + resolution, read coercion, table resolution) sees an ordinary view-backed object. + The model is built once per report node and never joins the loaded tree. + + A report with no read-only source of its own has no view (Table A), so there is + nothing to read: refused as not served rather than read from a table nobody made.""" + e = self._declared_entity(name) + if e.sub_type != OBJECT_SUBTYPE_REPORT: + return e + try: + model = report_read_model(e, self._root) + except ValueError as exc: + raise ValueError(f"Report '{name}' cannot be read: {exc}") from exc + # ADR-0039: own — the read model is a detached object that extends nothing; its + # sources are exactly the one copy report_read_model() added, so there is no + # inherited layer for an own read to drop. + if not any(isinstance(c, MetaSource) for c in model.own_children()): + raise ValueError( + f"Report '{name}' is not served: it declares no read-only source, " + f"so it has no view to read" + ) + return model + + def _refuse_report(self, op: str, name: str) -> None: + """Refuse an operation a report cannot support — get-by-id, relationship + traversal and every write — on the DECLARED subtype, before anything else is + looked at (a sourceless report included: it is read-only, not merely unserved).""" + if self._declared_entity(name).sub_type == OBJECT_SUBTYPE_REPORT: + raise ValueError( + f"{op} is not supported on '{name}': a report is read-only and has no identity " + f"(it is a view over aggregates; only find_many and count read it)" + ) + def _table_name(self, entity: MetaObject) -> str: """The physical relation *entity* lives in. @@ -699,6 +745,7 @@ def primary_key_field(self, entity_name: str) -> str: """The single-field primary-key NAME for an entity, from its ``identity.primary`` ``@fields``. ``op: roundtrip`` reads the inserted row back by this key (composite PKs are not supported by roundtrip).""" + self._refuse_report("primary_key_field", entity_name) return self._primary_pk_field(self._require_entity(entity_name)) def _primary_pk_field(self, entity: MetaObject) -> str: diff --git a/server/python/tests/runtime/test_object_manager_report.py b/server/python/tests/runtime/test_object_manager_report.py new file mode 100644 index 000000000..26d9886f4 --- /dev/null +++ b/server/python/tests/runtime/test_object_manager_report.py @@ -0,0 +1,270 @@ +"""ObjectManager reads a view-backed ``object.report`` (FR-044 Plan 2, Task 14). + +A report declares no fields: its read shape is derived (Table B). The runtime reads +it through a detached READ MODEL (``report_read_model``) swapped in at +``_require_entity``, so the column list, filter/sort resolution, read coercion and +table resolution all see an ordinary view-backed object. These tests use a recording +driver (no database): they pin the SQL, the refusals, and that the loaded tree is +never touched. The rows themselves are proven by the six shared ``report-*`` persistence +scenarios against Postgres (``tests/integration/test_query_scenarios.py``). +""" +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import pytest + +from metaobjects import load_directory +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.meta.core.object.meta_object import MetaObject +from metaobjects.meta.core.object.object_constants import OBJECT_SUBTYPE_REPORT +from metaobjects.meta.core.reporting.report_read_model import report_read_model +from metaobjects.runtime.object_manager import ObjectManager, SelectResult +from metaobjects.serializer_json import canonical_serialize +from metaobjects.shared.base_types import TYPE_OBJECT + +CORPUS = Path(__file__).parents[4] / "fixtures" / "persistence-conformance" + + +class RecordingDriver: + """Records every statement; answers with no rows (and a zero scalar).""" + + def __init__(self) -> None: + self.sql: list[str] = [] + self.params: list[tuple[Any, ...]] = [] + + def select(self, sql: str, params: tuple[Any, ...] = ()) -> SelectResult: + self.sql.append(sql) + self.params.append(params) + return SelectResult([], {}) + + def scalar(self, sql: str, params: tuple[Any, ...] = ()) -> Any: + self.sql.append(sql) + self.params.append(params) + return 0 + + def insert_returning(self, *a: Any, **k: Any) -> SelectResult: # pragma: no cover - must not be reached + raise AssertionError("a report write reached the driver") + + update_returning = insert_returning + + def execute_rowcount(self, *a: Any, **k: Any) -> int: # pragma: no cover - must not be reached + raise AssertionError("a report write reached the driver") + + +def _canonical_root(): + result = load_directory(CORPUS / "canonical") + assert not result.errors, "\n".join(e.message for e in result.errors) + return result.root + + +def _om(root: Any, driver: RecordingDriver, **kw: Any) -> ObjectManager: + return ObjectManager(root, driver, **kw) # type: ignore[arg-type] + + +def _load(text: str): + result = MetaDataLoader().load([InMemoryStringSource(text, "t.json")]) + assert result.errors == [], [e.message for e in result.errors] + return result.root + + +# A report beside an entity; `source` is spliced into the report's children. +def _model(report_sources: str, *, extra_report: str = "") -> str: + return """{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "name": "primary", "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "region", "@required": true } }, + { "field.int": { "name": "units" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "dimension.attribute": { "name": "region", "@of": "Sale.region" } }, + { "measure.aggregate": { "name": "total", "@agg": "sum", "@of": "Sale.units" } } + ]} }, + { "object.report": { "name": "SalesByRegion", "@from": "Sale", + "@dimensions": ["region"], "@measures": ["total"], "children": [ + %s + ]%s } } +]} }""" % (report_sources, extra_report) + + +# --- the canonical reports through the runtime ------------------------------------------------ + + +def test_find_many_selects_the_derived_columns_from_the_view() -> None: + root = _canonical_root() + drv = RecordingDriver() + _om(root, drv).find_many("ProgramMinutes") + assert drv.sql[0] == ( + 'SELECT "program", "programTitle", "weeks", "longWeeks", "labels", "slots", ' + '"totalMinutes", "avgMinutes", "minMinutes", "maxMinutes", "longShare" ' + 'FROM "v_program_minutes"' + ) + + +def test_filter_sort_limit_on_derived_fields() -> None: + root = _canonical_root() + drv = RecordingDriver() + _om(root, drv).find_many( + "ProgramMinutes", {"weeks": {"gte": 2}}, sort=[("totalMinutes", "desc")], limit=1 + ) + sql = drv.sql[0] + assert sql.endswith('FROM "v_program_minutes" WHERE "weeks" >= %s ORDER BY "totalMinutes" DESC LIMIT 1') + assert drv.params[0] == (2,) + + +def test_count_reads_the_view() -> None: + root = _canonical_root() + drv = RecordingDriver() + assert _om(root, drv).count("ProgramMinutes", {"weeks": {"gte": 1}}) == 0 + assert drv.sql[0] == 'SELECT COUNT(*) FROM "v_program_minutes" WHERE "weeks" >= %s' + + +def test_the_naming_strategy_applies_to_the_derived_name() -> None: + """The derived field carries no @column, so snake_case turns ``programTitle`` into + ``program_title`` (a column the lowering emits under the same strategy).""" + root = _canonical_root() + drv = RecordingDriver() + _om(root, drv, column_naming="snake_case").find_many("ProgramMinutes", {"programTitle": "x"}) + assert '"program_title"' in drv.sql[0] + assert '"programTitle"' not in drv.sql[0] + + +def test_an_unknown_field_in_a_report_filter_is_not_silently_mapped() -> None: + """Same as any object: an unknown field name is used verbatim as the column, so the + database (not the runtime) refuses it. Pins that a report adds no special case.""" + root = _canonical_root() + drv = RecordingDriver() + _om(root, drv).find_many("ProgramMinutes", {"nope": 1}) + assert '"nope" = %s' in drv.sql[0] + + +@pytest.mark.parametrize("op", ["find_by_id", "create", "insert_preserving", "update", "delete"]) +def test_by_id_and_every_write_are_refused_read_only_no_identity(op: str) -> None: + root = _canonical_root() + drv = RecordingDriver() + om = _om(root, drv) + args: dict[str, tuple[Any, ...]] = { + "find_by_id": ("ProgramMinutes", 1), + "create": ("ProgramMinutes", {"weeks": 1}), + "insert_preserving": ("ProgramMinutes", {"weeks": 1}), + "update": ("ProgramMinutes", 1, {"weeks": 1}), + "delete": ("ProgramMinutes", 1), + } + with pytest.raises(ValueError, match="read-only and has no identity"): + getattr(om, op)(*args[op]) + assert drv.sql == [] + + +def test_relate_and_primary_key_field_are_refused() -> None: + om = _om(_canonical_root(), RecordingDriver()) + with pytest.raises(ValueError, match="read-only and has no identity"): + om.relate("ProgramMinutes", {"id": 1}, "weeks") + with pytest.raises(ValueError, match="read-only and has no identity"): + om.primary_key_field("ProgramMinutes") + + +def test_a_non_report_object_is_unchanged() -> None: + root = _canonical_root() + drv = RecordingDriver() + om = _om(root, drv) + declared = next(c for c in root.children() if c.type == TYPE_OBJECT and c.name == "Program") + assert om._require_entity("Program") is declared + om.find_many("Program", {"id": 1}, limit=1) + assert drv.sql[0].startswith('SELECT ') and 'FROM "programs"' in drv.sql[0] + assert om.primary_key_field("Program") == "id" + + +def test_reading_reports_leaves_the_loaded_tree_untouched() -> None: + root = _canonical_root() + before = canonical_serialize(root) + n_objects = len([c for c in root.children() if c.type == TYPE_OBJECT]) + om = _om(root, RecordingDriver()) + for name in ("ProgramMinutes", "FitnessTotals", "ProgramsByMonth"): + om.find_many(name) + om.count(name) + assert len([c for c in root.children() if c.type == TYPE_OBJECT]) == n_objects + assert canonical_serialize(root) == before + # The report's source node still belongs to the report (never re-parented). + report = next(c for c in root.children() if c.name == "ProgramMinutes") + # ADR-0039: own — the assertion is about the nodes the report DECLARES (its source), + # whose parent must still be the report; an inherited child belongs to its base. + assert all(c.parent is report for c in report.own_children()) + + +def test_the_runtime_reads_through_the_cached_read_model() -> None: + root = _canonical_root() + om = _om(root, RecordingDriver()) + assert om._require_entity("ProgramMinutes") is om._require_entity("ProgramMinutes") + report = next(c for c in root.children() if c.name == "ProgramMinutes") + assert report_read_model(report, root) is om._require_entity("ProgramMinutes") + # Detached: the model is not the declared node and has no parent. + assert om._require_entity("ProgramMinutes") is not report + assert om._require_entity("ProgramMinutes").parent is None + assert om._require_entity("ProgramMinutes").sub_type == OBJECT_SUBTYPE_REPORT + + +# --- shapes the loader permits, built inline -------------------------------------------------- + + +def test_a_sourceless_report_is_not_served() -> None: + root = _load(_model("")) + drv = RecordingDriver() + om = _om(root, drv) + for read in (lambda: om.find_many("SalesByRegion"), lambda: om.count("SalesByRegion")): + with pytest.raises(ValueError, match="is not served"): + read() + assert drv.sql == [] + + +def test_a_write_on_a_sourceless_report_is_refused_as_read_only_not_unserved() -> None: + om = _om(_load(_model("")), RecordingDriver()) + with pytest.raises(ValueError, match="read-only and has no identity"): + om.create("SalesByRegion", {"total": 1}) + with pytest.raises(ValueError, match="read-only and has no identity"): + om.find_by_id("SalesByRegion", 1) + + +def test_an_unmanaged_view_backed_report_is_still_read() -> None: + root = _load(_model('{ "source.rdb": { "@kind": "view", "@view": "v_sales_by_region", "@unmanaged": true } }')) + drv = RecordingDriver() + om = _om(root, drv) + om.find_many("SalesByRegion") + om.count("SalesByRegion") + assert drv.sql[0] == 'SELECT "region", "total" FROM "v_sales_by_region"' + assert drv.sql[1] == 'SELECT COUNT(*) FROM "v_sales_by_region"' + + +def test_a_replica_declared_before_the_primary_view_reads_the_primary() -> None: + """The read model names the view by the same rule the lowering uses: the own read-only + source with @role primary, else the first. Decoy names would show up in the SQL.""" + root = _load(_model( + '{ "source.rdb": { "name": "r", "@kind": "view", "@view": "v_replica", "@role": "replica" } },' + '{ "source.rdb": { "name": "p", "@kind": "view", "@view": "v_primary", "@role": "primary" } }' + )) + drv = RecordingDriver() + _om(root, drv).find_many("SalesByRegion") + assert 'FROM "v_primary"' in drv.sql[0] + assert "v_replica" not in drv.sql[0] + + +def test_a_view_with_no_explicit_role_is_read() -> None: + root = _load(_model('{ "source.rdb": { "@kind": "view", "@view": "v_plain" } }')) + drv = RecordingDriver() + _om(root, drv).find_many("SalesByRegion") + assert 'FROM "v_plain"' in drv.sql[0] + + +def test_required_and_carried_attrs_come_from_the_derived_shape() -> None: + root = _load(_model('{ "source.rdb": { "@kind": "view", "@view": "v" } }')) + report = next(c for c in root.children() if c.name == "SalesByRegion") + model = report_read_model(report, root) + assert isinstance(model, MetaObject) + by_name = {f.name: f for f in model.fields()} + assert list(by_name) == ["region", "total"] + assert by_name["region"].sub_type == "string" + assert by_name["region"].get_meta_attr("required") is True # dimension: the @of field's @required + assert by_name["total"].sub_type == "long" + assert by_name["total"].get_meta_attr("required") is False # a sum is nullable + assert all(f.get_meta_attr("column") is None for f in model.fields()) # @column is never carried diff --git a/server/python/tests/test_report_read_model.py b/server/python/tests/test_report_read_model.py new file mode 100644 index 000000000..4dfcc3fda --- /dev/null +++ b/server/python/tests/test_report_read_model.py @@ -0,0 +1,164 @@ +"""FR-044 Plan 2 — the report READ MODEL's carry rules (Table B's type-source column). + +A derived field is a new, detached ``field.*`` node. It takes its ``@required`` from the +derived shape and, when Table B gives it a type source, exactly these from that field: +``@currency``, ``@values``, ``@intValueMap``, ``@maxLength``, ``@precision``, ``@scale``, +``@localTime``, ``@objectRef``, ``@storage`` (read RESOLVING), its OWN ``@dbColumnType`` +(never an inherited one), and array-ness. Nothing else: no ``@column``, no ``@default``. + +The canonical corpus exercises few of these, so they are pinned here with inline models. +""" +from __future__ import annotations + +import json + +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.meta.core.field.field_constants import ( + FIELD_ATTR_COLUMN, + FIELD_ATTR_CURRENCY, + FIELD_ATTR_DEFAULT, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_PRECISION, + FIELD_ATTR_REQUIRED, + FIELD_ATTR_SCALE, + FIELD_ATTR_VALUES, +) +from metaobjects.meta.core.field.meta_field import MetaField +from metaobjects.meta.core.reporting.report_read_model import report_read_model +from metaobjects.meta.persistence.db.db_constants import ( + FIELD_ATTR_DB_COLUMN_TYPE, + FIELD_ATTR_LOCAL_TIME, +) +from metaobjects.shared.base_types import TYPE_OBJECT + +_MODEL = { + "metadata.root": { + "package": "shop", + "children": [ + { + "object.entity": { + "name": "Base", + "abstract": True, + "children": [ + # Inherited by Sale.code below: @maxLength must be carried, the + # physical @dbColumnType must not. + {"field.string": {"name": "code", "@maxLength": 12, "@dbColumnType": "uuid"}}, + ], + } + }, + { + "object.entity": { + "name": "Sale", + "children": [ + {"source.rdb": {"@table": "sales"}}, + {"field.long": {"name": "id"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"field.string": {"name": "code", "extends": "shop::Base.code"}}, + {"field.string": {"name": "ref", "@dbColumnType": "uuid", "@column": "ref_col", + "@required": True, "@default": "x"}}, + {"field.string": {"name": "tags", "isArray": True}}, + {"field.enum": {"name": "status", "@values": ["OPEN", "PAID"]}}, + {"field.currency": {"name": "amountCents", "@currency": "EUR", "@required": True}}, + {"field.decimal": {"name": "weight", "@precision": 10, "@scale": 2}}, + {"field.timestamp": {"name": "bookedAt", "@localTime": True}}, + {"dimension.attribute": {"name": "code", "@of": "Sale.code"}}, + {"dimension.attribute": {"name": "ref", "@of": "Sale.ref"}}, + {"dimension.attribute": {"name": "tags", "@of": "Sale.tags"}}, + {"dimension.attribute": {"name": "status", "@of": "Sale.status"}}, + {"dimension.time": {"name": "bookedAt", "@of": "Sale.bookedAt", "@grains": ["hour", "day"]}}, + {"measure.aggregate": {"name": "sales", "@agg": "count", "@of": "Sale.id"}}, + {"measure.aggregate": {"name": "revenue", "@agg": "sum", "@of": "Sale.amountCents"}}, + {"measure.aggregate": {"name": "minAmount", "@agg": "min", "@of": "Sale.amountCents"}}, + {"measure.aggregate": {"name": "totalWeight", "@agg": "sum", "@of": "Sale.weight"}}, + {"measure.aggregate": {"name": "maxWeight", "@agg": "max", "@of": "Sale.weight"}}, + ], + } + }, + { + "object.report": { + "name": "R", + "@from": "Sale", + "@dimensions": ["code", "ref", "tags", "status", "bookedAt:hour", "bookedAt:day"], + "@measures": ["sales", "revenue", "minAmount", "totalWeight", "maxWeight"], + "children": [{"source.rdb": {"@kind": "view", "@view": "v_r"}}], + } + }, + ], + } +} + + +def _fields() -> dict[str, MetaField]: + result = MetaDataLoader().load([InMemoryStringSource(json.dumps(_MODEL), "meta.shop.json")]) + assert result.errors == [], [e.message for e in result.errors] + report = next(c for c in result.root.children() if c.type == TYPE_OBJECT and c.name == "R") + return {f.name: f for f in report_read_model(report, result.root).fields()} + + +def test_fields_are_one_per_table_b_row_in_order_with_the_derived_subtype() -> None: + assert [(f.name, f.sub_type) for f in _fields().values()] == [ + ("code", "string"), ("ref", "string"), ("tags", "string"), ("status", "enum"), + ("bookedAtHour", "timestamp"), ("bookedAtDay", "date"), + ("sales", "long"), ("revenue", "currency"), ("minAmount", "currency"), + ("totalWeight", "decimal"), ("maxWeight", "decimal"), + ] + + +def test_currency_is_carried_by_a_sum_and_by_a_min() -> None: + fields = _fields() + assert fields["revenue"].get_meta_attr(FIELD_ATTR_CURRENCY) == "EUR" + assert fields["minAmount"].get_meta_attr(FIELD_ATTR_CURRENCY) == "EUR" + + +def test_required_comes_from_the_derived_shape_never_from_the_type_source() -> None: + fields = _fields() + # A min of a required column is still nullable (no rows -> null); a count never is. + assert fields["minAmount"].get_meta_attr(FIELD_ATTR_REQUIRED) is False + assert fields["revenue"].get_meta_attr(FIELD_ATTR_REQUIRED) is False + assert fields["sales"].get_meta_attr(FIELD_ATTR_REQUIRED) is True + assert fields["ref"].get_meta_attr(FIELD_ATTR_REQUIRED) is True + assert fields["code"].get_meta_attr(FIELD_ATTR_REQUIRED) is False + + +def test_values_and_max_length_are_carried_resolving() -> None: + fields = _fields() + assert fields["status"].get_meta_attr(FIELD_ATTR_VALUES) == ["OPEN", "PAID"] + # Sale.code declares no @maxLength of its own: it inherits 12 from Base.code. + assert fields["code"].get_meta_attr(FIELD_ATTR_MAX_LENGTH) == 12 + + +def test_db_column_type_is_carried_only_when_the_of_field_declares_it_itself() -> None: + fields = _fields() + # ADR-0039: own — @dbColumnType is the one deliberately own-only attr; these + # assertions are about exactly that, so they read it with the OWN accessor attr(). + assert fields["ref"].attr(FIELD_ATTR_DB_COLUMN_TYPE) == "uuid" + assert fields["code"].attr(FIELD_ATTR_DB_COLUMN_TYPE) is None + assert fields["code"].get_meta_attr(FIELD_ATTR_DB_COLUMN_TYPE) is None + + +def test_array_ness_is_carried() -> None: + fields = _fields() + assert fields["tags"].resolved_is_array() is True + assert fields["code"].resolved_is_array() is False + + +def test_precision_scale_and_local_time_follow_the_type_source() -> None: + fields = _fields() + # max keeps the @of field as its type source; a sum of a decimal has none. + assert fields["maxWeight"].get_meta_attr(FIELD_ATTR_PRECISION) == 10 + assert fields["maxWeight"].get_meta_attr(FIELD_ATTR_SCALE) == 2 + assert fields["totalWeight"].get_meta_attr(FIELD_ATTR_PRECISION) is None + assert fields["totalWeight"].get_meta_attr(FIELD_ATTR_SCALE) is None + # An hour bucket is the instant itself and keeps @localTime; a day bucket is a bare date. + assert fields["bookedAtHour"].get_meta_attr(FIELD_ATTR_LOCAL_TIME) is True + assert fields["bookedAtDay"].get_meta_attr(FIELD_ATTR_LOCAL_TIME) is None + + +def test_nothing_else_is_carried() -> None: + ref = _fields()["ref"] + # The physical column is the naming strategy applied to the DERIVED name, so the + # @of field's @column is never inherited; nor is its @default. + assert ref.get_meta_attr(FIELD_ATTR_COLUMN) is None + assert ref.get_meta_attr(FIELD_ATTR_DEFAULT) is None + assert sorted(ref.attrs()) == sorted([FIELD_ATTR_REQUIRED, FIELD_ATTR_DB_COLUMN_TYPE]) diff --git a/server/python/tests/test_report_shape.py b/server/python/tests/test_report_shape.py new file mode 100644 index 000000000..97f3fe349 --- /dev/null +++ b/server/python/tests/test_report_shape.py @@ -0,0 +1,358 @@ +"""FR-044 Plan 2 — Table B (a report's derived fields), byte-matched across ports. + +TypeScript produces ``fixtures/persistence-conformance/report-shapes.json`` from the +canonical model; every other port derives the same shapes from the same model and +compares BYTES, in a container-free test, so the derivation cannot drift between ports. +The format is a contract (reports in declaration order; keys ``report, from, view, +fields``, then per field ``name, role, subType, required, typeSource``; two-space +indent; one trailing newline), and ``type_source`` is the resolution key of the entity +that DECLARES the ``@of`` field, a dot, and the field name. +""" +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from metaobjects import load_directory +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.meta.core.field.meta_field import MetaField +from metaobjects.meta.core.object.meta_object import MetaObject +from metaobjects.meta.core.object.object_constants import ( + OBJECT_REPORT_ATTR_DIMENSIONS, + OBJECT_REPORT_ATTR_FROM, + OBJECT_REPORT_ATTR_MEASURES, + OBJECT_SUBTYPE_REPORT, +) +from metaobjects.meta.core.reporting.report_accessors import ( + report_measure_item_name, + report_measure_item_owner, +) +from metaobjects.meta.core.reporting.report_read_model import report_read_source +from metaobjects.meta.core.reporting.report_shape import report_shape +from metaobjects.shared.base_types import TYPE_OBJECT + +CORPUS = Path(__file__).parents[3] / "fixtures" / "persistence-conformance" + + +def _file(package: str, children: list) -> InMemoryStringSource: + return InMemoryStringSource( + json.dumps({"metadata.root": {"package": package, "children": children}}), f"meta.{package}.json" + ) + + +def _load(*files: InMemoryStringSource): + result = MetaDataLoader().load(list(files)) + assert result.errors == [], [e.message for e in result.errors] + return result.root + + +def _object(root, name: str) -> MetaObject: + return next(c for c in root.children() if c.type == TYPE_OBJECT and c.name == name) + + +#: A minimal entity with one count measure, for the tests that are about the report. +_SALE = { + "object.entity": { + "name": "Sale", + "children": [ + {"source.rdb": {"@table": "sales"}}, + {"field.long": {"name": "id"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"measure.aggregate": {"name": "sales", "@agg": "count", "@of": "Sale.id"}}, + ], + } +} + + +def _root(): + result = load_directory(CORPUS / "canonical") + assert not result.errors, "\n".join(e.message for e in result.errors) + return result.root + + +def _type_source(field: MetaField | None) -> str | None: + if field is None: + return None + owner = field.parent + assert owner is not None, f"field '{field.name}' has no owning entity" + return f"{owner.resolution_key()}.{field.name}" + + +def generate_report_shapes_json(root) -> str: + reports = [] + # ADR-0039: resolving — the root is never extended, so children() == own_children(). + for report in (c for c in root.children() if c.type == TYPE_OBJECT): + if report.sub_type != OBJECT_SUBTYPE_REPORT: + continue + shape = report_shape(report, root) + # The source the lowering names and the runtime reads: the report's own read-only + # source with @role primary, else its first own read-only source. + source = report_read_source(report) + reports.append( + { + "report": report.resolution_key(), + "from": shape.from_.resolution_key(), + "view": None if source is None else source.physical_name(), + "fields": [ + { + "name": f.name, + "role": f.role, + "subType": f.sub_type, + "required": f.required, + "typeSource": _type_source(f.type_source), + } + for f in shape.fields + ], + } + ) + # json.dumps(indent=2) is JSON.stringify(_, null, 2) for this data: same layout, + # empty containers aside (none occur), and no non-ASCII to escape. + return json.dumps({"reports": reports}, indent=2, ensure_ascii=False) + "\n" + + +def test_derived_shapes_byte_match_the_committed_artifact() -> None: + expected = (CORPUS / "report-shapes.json").read_text(encoding="utf-8") + assert generate_report_shapes_json(_root()) == expected + + +def test_the_canonical_model_has_six_reports() -> None: + reports = [c for c in _root().children() if c.type == TYPE_OBJECT and c.sub_type == OBJECT_SUBTYPE_REPORT] + assert len(reports) == 6 + assert all(isinstance(r, MetaObject) for r in reports) + + +def test_the_view_is_the_primary_read_only_source_else_the_first() -> None: + """The artifact's ``view`` is the source the lowering names and the runtime reads: a + replica declared BEFORE the primary does not name the view.""" + root = _load( + _file( + "shop", + [ + _SALE, + { + "object.report": { + "name": "R", + "@from": "Sale", + "@measures": ["sales"], + "children": [ + {"source.rdb": {"name": "replica", "@kind": "view", "@view": "v_replica", "@role": "replica"}}, + {"source.rdb": {"name": "main", "@kind": "view", "@view": "v_primary"}}, + ], + } + }, + ], + ) + ) + assert json.loads(generate_report_shapes_json(root))["reports"][0]["view"] == "v_primary" + + +# --------------------------------------------------------------------------- +# Reference resolution: the shape must agree with the loader's validate_reporting +# about what a reference names, or a model that loads clean fails (or is silently +# mistyped) when it is read. The same cases as the TypeScript report-shape.test.ts. +# --------------------------------------------------------------------------- + +#: ``a::Base`` (abstract): members whose bare ``@of`` names ``Base``. +_SHARED_BASE = { + "object.entity": { + "name": "Base", + "abstract": True, + "children": [ + {"field.long": {"name": "id"}}, + {"field.string": {"name": "kind"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"dimension.attribute": {"name": "kind", "@of": "Base.kind"}}, + {"measure.aggregate": {"name": "events", "@agg": "count", "@of": "Base.id"}}, + {"measure.aggregate": {"name": "lastKind", "@agg": "max", "@of": "Base.kind"}}, + ], + } +} + +_DECOY = { + "object.entity": {"name": "Base", "children": [{"field.int": {"name": "id"}}, {"field.int": {"name": "kind"}}]} +} + + +def _ev(extra: list | None = None) -> dict: + return { + "object.entity": { + "name": "Ev", + "extends": "a::Base", + "children": [{"source.rdb": {"@table": "evs"}}, *(extra or [])], + } + } + + +def _ev_report(**attrs) -> dict: + return { + "object.report": { + "name": "R", + "@from": "Ev", + "@dimensions": ["kind"], + "@measures": ["events", "lastKind"], + **attrs, + "children": [{"source.rdb": {"@kind": "view", "@view": "v_r"}}], + } + } + + +def _typed(root) -> list[tuple[str, str, str | None]]: + shape = report_shape(_object(root, "R"), root) + return [ + (f.name, f.sub_type, None if f.type_source is None else f.type_source.parent.resolution_key()) + for f in shape.fields + ] + + +def test_a_bare_of_on_a_member_inherited_from_another_package_resolves_in_the_declaring_package() -> None: + root = _load(_file("a", [_SHARED_BASE]), _file("b", [_ev(), _ev_report()])) + assert _typed(root) == [("kind", "string", "a::Base"), ("events", "long", None), ("lastKind", "string", "a::Base")] + + +def test_a_same_named_decoy_in_the_reports_package_does_not_capture_the_reference() -> None: + root = _load(_file("a", [_SHARED_BASE]), _file("b", [_DECOY, _ev(), _ev_report()])) + assert _typed(root) == [("kind", "string", "a::Base"), ("events", "long", None), ("lastKind", "string", "a::Base")] + + +def test_without_via_the_field_is_read_from_from_so_a_field_from_redeclares_wins() -> None: + root = _load(_file("a", [_SHARED_BASE]), _file("b", [_ev([{"field.int": {"name": "kind"}}]), _ev_report()])) + assert _typed(root) == [("kind", "int", "b::Ev"), ("events", "long", None), ("lastKind", "int", "b::Ev")] + + +def test_a_dotted_measures_item_names_the_measure_by_its_last_segment() -> None: + root = _load( + _file("a", [_SHARED_BASE]), + _file("b", [_ev(), _ev_report(**{"@measures": ["Ev.events", "a::Base.lastKind"]})]), + ) + assert [f[0] for f in _typed(root)] == ["kind", "events", "lastKind"] + + +def test_report_measure_item_name_is_the_last_segment() -> None: + assert report_measure_item_name("total") == "total" + assert report_measure_item_name("Sale.total") == "total" + assert report_measure_item_name("acme::shop::Sale.total") == "total" + assert report_measure_item_owner("total") is None + assert report_measure_item_owner("acme::shop::Sale.total") == "acme::shop::Sale" + + +def _stray(package: str, from_: str, attr: str, item: str) -> MetaObject: + """A report built in code (never added to the root): what the loader would refuse.""" + report = MetaObject(TYPE_OBJECT, OBJECT_SUBTYPE_REPORT, "Stray") + report.package = package + report.set_attr(OBJECT_REPORT_ATTR_FROM, from_) + report.set_attr(attr, [item]) + return report + + +def test_a_dotted_measures_item_whose_qualifier_is_not_from_or_an_ancestor_does_not_resolve() -> None: + root = _load(_file("a", [_SHARED_BASE]), _file("b", [_DECOY, _ev(), _ev_report()])) + # Past the loader, which refuses these as ERR_INVALID_REPORT / ERR_REPORT_FOREIGN_MEASURE. + with pytest.raises(ValueError) as e: + report_shape(_stray("b", "Ev", OBJECT_REPORT_ATTR_MEASURES, "Nope.events"), root) + assert str(e.value) == "report 'Stray': measure 'Nope.events' on 'Ev' does not resolve." + # The qualifier resolves in the REPORT's package: b::Base is the decoy, not an ancestor of Ev. + with pytest.raises(ValueError) as e: + report_shape(_stray("b", "Ev", OBJECT_REPORT_ATTR_MEASURES, "Base.events"), root) + assert str(e.value) == "report 'Stray': measure 'Base.events' on 'Ev' does not resolve." + + +def test_a_time_dimension_item_with_no_grain_or_a_grain_outside_the_closed_set_does_not_resolve() -> None: + root = _root() + with pytest.raises(ValueError) as e: + report_shape(_stray("fitness", "Program", OBJECT_REPORT_ATTR_DIMENSIONS, "createdAt"), root) + assert str(e.value) == "report 'Stray': time dimension 'createdAt' grain '' does not resolve." + with pytest.raises(ValueError) as e: + report_shape(_stray("fitness", "Program", OBJECT_REPORT_ATTR_DIMENSIONS, "createdAt:fortnight"), root) + assert str(e.value) == "report 'Stray': time dimension 'createdAt' grain 'fortnight' does not resolve." + + +# --------------------------------------------------------------------------- +# Table B rows the canonical model does not contain +# --------------------------------------------------------------------------- + +_CUBE = { + "object.entity": { + "name": "Sale", + "extends": "Base", + "children": [ + {"source.rdb": {"@table": "sales"}}, + {"field.long": {"name": "id"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"field.decimal": {"name": "weight", "@precision": 10, "@scale": 2}}, + {"field.double": {"name": "score"}}, + {"field.float": {"name": "ratio"}}, + {"field.string": {"name": "region", "@required": True, "@maxLength": 8}}, + {"field.timestamp": {"name": "bookedAt", "@localTime": True}}, + {"dimension.attribute": {"name": "region", "@of": "Sale.region"}}, + {"dimension.time": {"name": "bookedAt", "@of": "Sale.bookedAt", "@grains": ["hour", "day"]}}, + {"measure.aggregate": {"name": "revenue", "@agg": "sum", "@of": "Sale.amountCents"}}, + {"measure.aggregate": {"name": "avgRevenue", "@agg": "avg", "@of": "Sale.amountCents"}}, + {"measure.aggregate": {"name": "totalWeight", "@agg": "sum", "@of": "Sale.weight"}}, + {"measure.aggregate": {"name": "avgWeight", "@agg": "avg", "@of": "Sale.weight"}}, + {"measure.aggregate": {"name": "totalScore", "@agg": "sum", "@of": "Sale.score"}}, + {"measure.aggregate": {"name": "avgScore", "@agg": "avg", "@of": "Sale.score"}}, + {"measure.aggregate": {"name": "totalRatio", "@agg": "sum", "@of": "Sale.ratio"}}, + {"measure.aggregate": {"name": "avgRatio", "@agg": "avg", "@of": "Sale.ratio"}}, + {"measure.aggregate": {"name": "maxRevenue", "@agg": "max", "@of": "Sale.amountCents"}}, + {"measure.aggregate": {"name": "minWeight", "@agg": "min", "@of": "Sale.weight"}}, + ], + } +} + +_CUBE_BASE = { + "object.entity": { + "name": "Base", + "abstract": True, + "children": [{"field.currency": {"name": "amountCents", "@required": True, "@currency": "USD"}}], + } +} + +_CUBE_REPORT = { + "object.report": { + "name": "R", + "@from": "Sale", + "@dimensions": ["region", "bookedAt:hour", "bookedAt:day"], + "@measures": [ + "revenue", "avgRevenue", "totalWeight", "avgWeight", "totalScore", "avgScore", + "totalRatio", "avgRatio", "maxRevenue", "minWeight", + ], + } +} + + +def test_sum_avg_min_and_max_rows_by_of_subtype() -> None: + root = _load(_file("shop", [_CUBE_BASE, _CUBE, _CUBE_REPORT])) + fields = {f.name: f for f in report_shape(_object(root, "R"), root).fields} + + def row(name: str) -> tuple[str, bool, str | None]: + f = fields[name] + return (f.sub_type, f.required, _type_source(f.type_source)) + + # sum: currency stays currency (and carries its type source); int/long -> long; + # double/float -> double; anything else -> decimal. No sum is required. + assert row("revenue") == ("currency", False, "shop::Base.amountCents") + assert row("totalWeight") == ("decimal", False, None) + assert row("totalScore") == ("double", False, None) + assert row("totalRatio") == ("double", False, None) + # avg: double/float -> double, anything else -> decimal; never a type source. + assert row("avgRevenue") == ("decimal", False, None) + assert row("avgWeight") == ("decimal", False, None) + assert row("avgScore") == ("double", False, None) + assert row("avgRatio") == ("double", False, None) + # min / max keep the @of field's subtype and name it as the type source: the entity + # that DECLARES the field, which for an inherited field is the base. + assert row("maxRevenue") == ("currency", False, "shop::Base.amountCents") + assert row("minWeight") == ("decimal", False, "shop::Sale.weight") + # Dimensions: required only from the @of field's own @required; an hour bucket keeps + # the field as its type source (for @localTime), a coarser one is a bare date. + assert row("region") == ("string", True, "shop::Sale.region") + assert row("bookedAtHour") == ("timestamp", False, "shop::Sale.bookedAt") + assert row("bookedAtDay") == ("date", False, None) + + +def test_a_sourceless_report_has_a_shape_and_no_view() -> None: + root = _load(_file("shop", [_CUBE_BASE, _CUBE, _CUBE_REPORT])) + assert json.loads(generate_report_shapes_json(root))["reports"][0]["view"] is None diff --git a/server/typescript/packages/cli/test/unit/reporting-inert.test.ts b/server/typescript/packages/cli/test/unit/reporting-inert.test.ts index c002edd20..d7b767e66 100644 --- a/server/typescript/packages/cli/test/unit/reporting-inert.test.ts +++ b/server/typescript/packages/cli/test/unit/reporting-inert.test.ts @@ -1,11 +1,13 @@ -// FR-044 Plan 1 — the reporting vocabulary is INERT in every generator and in migrate. +// FR-044 — what a report generates, and what it does not. // -// Plan 1 registers `dimension.*`, `measure.*`, `segment.*` and `object.report` and -// validates them at load, but gives none of them output: a report's lowering (a view, a -// typed row, a route) lands in Plan 2/3. Until then a model that USES the vocabulary must -// generate exactly what the same model without it generates — byte for byte, in every -// catalog generator — and `meta migrate` must propose nothing for it. Anything else is -// churn an adopter sees the day they declare a measure. +// Plan 1 registered `dimension.*`, `measure.*`, `segment.*` and `object.report` and gave +// them no output. Plan 2 lowers exactly ONE thing: a report that declares a read-only +// `source.rdb @kind: view` becomes that view in TypeScript migrate (and so on the +// `meta docs` agent schema page, which lists the views migrate would create). Everything +// else stays inert, and this file holds it there: a sourceless report is inert everywhere, +// every catalog generator emits the same files with and without the reporting nodes (no +// TypeScript generator emits for a report; routes and the typed row are Plan 3), and every +// docs surface other than that one schema entry is byte-identical. // // The model pair lives in fixtures/codegen-noop/reporting/ and is shared with the other // four ports' copies of this test. `with/` carries a report that declares a read-only @@ -15,7 +17,7 @@ // `meta docs` is held to the same rule (controller ruling, 2026-10-03): a report's fields // are derived by its lowering, so a page for one today would show none of them. Every docs // surface — model pages, agent pages, requirements, the HTML site, and the api surface — -// must come out identical with and without the reporting nodes. +// must come out identical with and without the reporting nodes, bar the one view entry. import { describe, test, expect, beforeAll } from "bun:test"; import { mkdtempSync, mkdirSync, copyFileSync, rmSync, readFileSync, readdirSync, statSync } from "node:fs"; @@ -162,24 +164,35 @@ describe("FR-044 a selection of only reports", () => { }); }); -describe("FR-044 reporting nodes are inert in migrate", () => { - test("the expected postgres schema is identical, and diff() proposes no statement", async () => { - const withSchema: SchemaSnapshot = buildExpectedSchema(withReporting, { dialect: "postgres" }); - const withoutSchema: SchemaSnapshot = buildExpectedSchema(withoutReporting, { dialect: "postgres" }); - expect(withSchema).toEqual(withoutSchema); +describe("FR-044 a sourceless report is inert in migrate; a view-backed report proposes exactly its view", () => { + test("the expected postgres schemas differ by exactly v_store_totals, and diff() proposes exactly that view", async () => { + const views = (m: MetaRoot) => buildProjectionViews(m, { dialect: "postgres" }); + const withSchema: SchemaSnapshot = buildExpectedSchema(withReporting, { dialect: "postgres", views: views(withReporting) }); + const withoutSchema: SchemaSnapshot = buildExpectedSchema(withoutReporting, { dialect: "postgres", views: views(withoutReporting) }); + + // Only StoreTotals declares a view; ProgramEngagement and DailyRevenue are sourceless. + expect(withoutSchema.views).toEqual([]); + expect(withSchema.views.map((v) => v.name)).toEqual(["v_store_totals"]); + // Everything else is the same: the tables do not move. + expect(withSchema.tables).toEqual(withoutSchema.tables); // Live DB = the model without reporting nodes; metadata = the model with them. - const forward = await diff(withSchema, withoutSchema, { dialect: "postgres" }); - expect(forward.changes).toEqual([]); - // And from an empty database, the report adds nothing to what the entities need. + const forward = await diff({ expected: withSchema, actual: withoutSchema }); + expect(forward.changes.map((c) => [c.kind, c.kind === "create-view" ? c.view.name : undefined])).toEqual([ + ["create-view", "v_store_totals"], + ]); + // And from an empty database, the report adds exactly that one view to what the entities need. const empty: SchemaSnapshot = { tables: [], views: [] }; - const fromEmptyWith = await diff(withSchema, empty, { dialect: "postgres" }); - const fromEmptyWithout = await diff(withoutSchema, empty, { dialect: "postgres" }); - expect(fromEmptyWith.changes).toEqual(fromEmptyWithout.changes); + const fromEmptyWith = await diff({ expected: withSchema, actual: empty }); + const fromEmptyWithout = await diff({ expected: withoutSchema, actual: empty }); + expect(fromEmptyWith.changes.filter((c) => c.kind !== "create-view")).toEqual( + fromEmptyWithout.changes.filter((c) => c.kind !== "create-view"), + ); + expect(fromEmptyWith.changes.filter((c) => c.kind === "create-view")).toHaveLength(1); }); }); -describe("FR-044 reporting nodes are inert in meta docs", () => { +describe("FR-044 reporting nodes are inert in meta docs, bar the one view entry", () => { /** Run `meta docs` over a project holding one variant, once per surface flag set, and * read back everything written. The project directory has the SAME basename for both * variants: the site stamps it into every page title. */ @@ -251,7 +264,7 @@ describe("FR-044 reporting nodes are inert in meta docs", () => { compare(expected, await api(withReporting)); }); - test("the agent surface (schema, ui, requirements) is identical with the UI tier wired", async () => { + test("the agent surface differs only by the schema page's v_store_totals view, with the UI tier wired", async () => { // Same reason as above: `meta docs --agent` needs a loadable gen config. The schema // input is built exactly as docs.ts's buildAgentSchemaInput builds it for postgres. const agent = async (metadata: MetaRoot): Promise> => { @@ -279,6 +292,20 @@ describe("FR-044 reporting nodes are inert in meta docs", () => { // ui.md is the page that leaked a view-backed report; it must actually be rendered. expect(Object.keys(expected).some((p) => p.endsWith("ui.md"))).toBe(true); expect(Object.keys(expected).some((p) => p.endsWith("schema.md"))).toBe(true); - compare(expected, await agent(withReporting)); + const actual = await agent(withReporting); + + // The schema page lists the views migrate would create (docs.ts feeds it + // buildProjectionViews), so the one view-backed report appears there and nowhere else. + const schemaPage = Object.keys(expected).find((p) => p.endsWith("schema.md"))!; + const entry = "## Views\n\n" + + "A view is generated from its projection's `origin.*` children or its report's dimensions and measures — it is derived, never hand-written. " + + "Editing the view SQL directly is drift the tool cannot see.\n\n" + + "### `v_store_totals`\n\nDeclared by `acme::shop::StoreTotals`.\n\n"; + expect(actual[schemaPage]).toContain(entry); + expect(actual[schemaPage]!.replace(entry, "")).toBe(expected[schemaPage]!); + delete actual[schemaPage]; + const rest = { ...expected }; + delete rest[schemaPage]; + compare(rest, actual); }); }); diff --git a/server/typescript/packages/codegen-ts/src/generators/agent-docs-file.ts b/server/typescript/packages/codegen-ts/src/generators/agent-docs-file.ts index 8b5ab1cf5..c2e97b064 100644 --- a/server/typescript/packages/codegen-ts/src/generators/agent-docs-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/agent-docs-file.ts @@ -63,6 +63,7 @@ import type { ColumnNamingStrategy, MetaField, MetaObject } from "@metaobjectsde import type { EmittedFile, Generator, GeneratorFactory } from "../generator.js"; import { resolveObjectNames } from "../names.js"; import { isAbstract } from "../instance-artifacts.js"; +import { isReport } from "../source-detect.js"; import { enumValues, intValueMapOf } from "../enum-meta.js"; import { renderAgentSchemaPage } from "./agent-schema-page.js"; import { renderAgentUiPage } from "./agent-ui-page.js"; @@ -231,6 +232,9 @@ export const agentDocsFile = function agentDocsFile(opts?: AgentDocsFileOpts): G // so this mapping cannot disagree with the column it labels. const declaredBy = new Map>(); const viewLineage = new Map(); + // Qualified names of the views a view-backed object.report owns (FR-044). Empty for a + // model with no report, so the page is byte-identical to what it was before reports. + const reportViews = new Set(); for (const obj of objects) { const names = resolveObjectNames(obj, opts.columnNamingStrategy); // The PRIMARY source's physical name and schema — `names.name` is the object's @@ -249,6 +253,7 @@ export const agentDocsFile = function agentDocsFile(opts?: AgentDocsFileOpts): G // page printed the base's enum a second time under a different owner. // `buildExpectedSchema`'s Pass 1 skips abstracts; this reads the same rule. if (isTable && !isAbstract(obj)) tableBacked.push(obj); + if (!isTable && isReport(obj)) reportViews.add(key); let map = declaredBy.get(key); if (map === undefined) { map = new Map(); @@ -271,6 +276,7 @@ export const agentDocsFile = function agentDocsFile(opts?: AgentDocsFileOpts): G const content = renderAgentSchemaPage(schema, { declaredBy, viewLineage, + reportViews, relationships: relationshipLines(objects), enums: enumLines(tableBacked), }); diff --git a/server/typescript/packages/codegen-ts/src/generators/agent-schema-page.ts b/server/typescript/packages/codegen-ts/src/generators/agent-schema-page.ts index 111d8298d..6b1c40737 100644 --- a/server/typescript/packages/codegen-ts/src/generators/agent-schema-page.ts +++ b/server/typescript/packages/codegen-ts/src/generators/agent-schema-page.ts @@ -181,6 +181,9 @@ export interface AgentSchemaPageOptions { readonly declaredBy: ReadonlyMap>; /** Per-projection lineage lines, keyed by QUALIFIED view name. */ readonly viewLineage: ReadonlyMap; + /** QUALIFIED names of the views owned by a view-backed `object.report`. Omitted or empty + * means no report view is on the page, and the Views intro keeps its pre-report wording. */ + readonly reportViews?: ReadonlySet; /** Relationship lines, already rendered from the model. */ readonly relationships: readonly string[]; /** Enum lines, already rendered from the model. */ @@ -263,8 +266,13 @@ export function renderAgentSchemaPage( if (input.views.length > 0) { out.push("## Views"); out.push(""); + // A model with no report view must render exactly what it rendered before FR-044 + // (no-churn: `meta verify --docs` would flag drift after an upgrade otherwise). + const hasReportView = input.views.some((v) => opts.reportViews?.has(input.qualify(v)) === true); out.push( - "A view is generated from its projection's `origin.*` children — it is derived, " + + (hasReportView + ? "A view is generated from its projection's `origin.*` children or its report's dimensions and measures — it is derived, " + : "A view is generated from its projection's `origin.*` children — it is derived, ") + "never hand-written. Editing the view SQL directly is drift the tool cannot see.", ); out.push(""); diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index e86a34b21..631033b6e 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -267,8 +267,12 @@ export { extractViewSpec } from "./projection/extract-view-spec.js"; export type { ExtractContext } from "./projection/extract-view-spec.js"; export { emitViewDdl } from "./projection/view-ddl-emit.js"; export type { EmitOptions as ViewDdlEmitOptions } from "./projection/view-ddl-emit.js"; -export { buildProjectionViews } from "./projection/build-projection-views.js"; -export type { ExpectedView, BuildProjectionViewsOptions } from "./projection/build-projection-views.js"; +export { emitReportViewDdl } from "./projection/report-ddl-emit.js"; +export type { ReportEmitOptions } from "./projection/report-ddl-emit.js"; +export { extractReportSpec } from "./projection/extract-report-spec.js"; +export type { ReportViewSpec } from "./projection/report-spec.js"; +export { buildProjectionViews, buildReportViews } from "./projection/build-projection-views.js"; +export type { ExpectedView, BuildProjectionViewsOptions, BuildReportViewsOptions } from "./projection/build-projection-views.js"; export type { JoinNode, JoinTree, SelectColumn, SelectSpec, ViewSpec } from "./projection/view-spec.js"; // Prompt construction (FR-004): ADR-0056 — a template's payload is its value object's own // interface (entityFile()), so there is no template-tier payload emitter to export. The one diff --git a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts index cd8a850d9..4d8a046b9 100644 --- a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts +++ b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts @@ -17,6 +17,8 @@ import { isMetaRoot, isReadOnlySource, isWritableSource, + reportFrom, + resolveObjectRef, SOURCE_KIND_VIEW, TYPE_FIELD, TYPE_IDENTITY, @@ -25,7 +27,12 @@ import { resolveTableSchema, } from "@metaobjectsdev/metadata"; import { isProjection, isWriteThrough } from "./projection-detector.js"; -import { extractViewSpec, refNamedOwner } from "./extract-view-spec.js"; +import { extractViewSpec, packageOf, projectionViewSource, refNamedOwner } from "./extract-view-spec.js"; +import { extractReportSpec } from "./extract-report-spec.js"; +import { emitReportViewDdl } from "./report-ddl-emit.js"; +import type { ReportViewSpec } from "./report-spec.js"; +import type { ReportDialect } from "./time-sql.js"; +import { isReport } from "../source-detect.js"; import { emitViewDdl } from "./view-ddl-emit.js"; import type { JoinNode, ViewSpec } from "./view-spec.js"; import type { ColumnNamingStrategy } from "../metaobjects-config.js"; @@ -108,7 +115,10 @@ export function buildProjectionViews( for (const obj of root.objects()) joinTables[obj.resolutionKey()] = resolveTableName(obj); const out: ExpectedView[] = []; - for (const projection of root.objects().filter(isProjection)) { + // A view-backed object.report satisfies isProjection (a read-only source, no writable one) + // but is lowered by buildReportViews below, not here: name it out rather than rely on + // viewIsDerived happening to drop it. + for (const projection of root.objects().filter((o) => isProjection(o) && !isReport(o))) { // #208 §6 — classify DDL ownership BEFORE viewIsDerived (see classifyReadOnlySource), // so an escape-valve view carrying extends-bound identity/fields (pure shape / row // identity) is never mis-synthesized into a wrong base-table passthrough SELECT. @@ -149,9 +159,91 @@ export function buildProjectionViews( } emitViewFor(entity, root, joinTables, dialect, columnNamingStrategy, out); } + + // FR-044 — the view of every view-backed object.report, appended AFTER the two loops + // above so the views a report-free model returns keep their order. + out.push(...buildReportViews(root, { dialect: opts.dialect, columnNamingStrategy })); return out; } +export interface BuildReportViewsOptions { + dialect: "postgres" | "sqlite" | "d1" | "mysql"; + columnNamingStrategy?: ColumnNamingStrategy; +} + +/** + * The view of every view-backed `object.report` (contract Table A). Called by + * buildProjectionViews; exported separately because MySQL is accepted here and nowhere + * else (migrate does not target MySQL; the SQL ships through this function and a recipe). + * + * The Table A gate (classifySource, over the source `projectionViewSource` selects) runs + * BEFORE extractReportSpec: a sourceless + * report must never reach it, because projectionViewName falls back to `v_` and + * would invent a view nobody declared. A report whose `@from` has no table, or whose + * `@via` chain does not resolve, throws out of extractReportSpec naming the report; that + * propagates so `meta migrate` fails loudly instead of emitting a view over nothing. + */ +export function buildReportViews(root: MetaData, opts: BuildReportViewsOptions): ExpectedView[] { + if (!isMetaRoot(root)) { + throw new Error("buildReportViews: root must be a loaded MetaRoot."); + } + // D1 is SQLite at the SQL level. + const dialect: ReportDialect = opts.dialect === "d1" ? "sqlite" : opts.dialect; + const columnNamingStrategy = opts.columnNamingStrategy ?? "snake_case"; + const joinTables: Record = {}; + for (const obj of root.objects()) joinTables[obj.resolutionKey()] = resolveTableName(obj); + + const out: ExpectedView[] = []; + for (const report of root.objects().filter(isReport)) { + // Table A, decided by the SAME source the view is named by (`projectionViewName`) and + // the runtime reads (`reportReadModel`): the own read-only source with role primary, + // else the first own read-only source. Reports only: the projection and write-through + // loops above keep classifying their FIRST own read-only source. + const cls = classifySource(projectionViewSource(report)); + if (cls.kind === "skip") continue; + if (cls.kind === "sql") { + emitSqlView(report, cls.source, root, joinTables, out); + continue; + } + const spec = extractReportSpec(report, root, { columnNamingStrategy }); + const baseTableName = joinTables[spec.joinTree.baseEntity]; + if (!baseTableName) { + // extractReportSpec refuses a table-less @from first, so this is a defect, not an + // authoring error; skipping would silently drop a view the report declares. + throw new Error( + `report '${report.name}': no table name is known for its @from entity '${spec.joinTree.baseEntity}', ` + + `so its view '${spec.viewName}' cannot be emitted.`, + ); + } + const schema = resolveTableSchema(report); + out.push({ + name: spec.viewName, + sql: emitReportViewDdl(spec, { dialect, baseTableName, joinTables, bodyOnly: true }), + dependsOn: reportDependsOn(spec, baseTableName, joinTables), + fqn: report.resolutionKey(), + ...(schema !== undefined ? { schema } : {}), + // `columns` omitted on purpose: unknown, so migrate takes the fail-safe drop+create (Table F). + }); + } + return out; +} + +/** The base table plus every joined table, deduped — the physical tables a report view reads. */ +function reportDependsOn( + spec: ReportViewSpec, + baseTableName: string, + joinTables: Readonly>, +): string[] { + const tables = new Set([baseTableName]); + const walk = (node: JoinNode): void => { + const t = joinTables[node.targetEntity]; + if (t) tables.add(t); + for (const child of node.children) walk(child); + }; + for (const j of spec.joinTree.joins) walk(j); + return [...tables]; +} + /** * #208 §6 — classify a host's read-only source by DDL OWNERSHIP, BEFORE any derivation * decision. Shared by the projection and write-through loops so the ownership rules can @@ -177,7 +269,11 @@ type ReadOnlySourceClass = | { kind: "derive"; source: MetaSource }; function classifyReadOnlySource(host: MetaObject): ReadOnlySourceClass { - const source = host.ownChildren().find(isReadOnlySource); + return classifySource(host.ownChildren().find(isReadOnlySource)); +} + +/** The classification itself, for a source the caller has already selected. */ +function classifySource(source: MetaSource | undefined): ReadOnlySourceClass { if (source === undefined) return { kind: "skip" }; if (source.isUnmanaged) return { kind: "skip" }; // external — Flyway/hand-migration owns it if (source.sqlBody !== undefined) return { kind: "sql", source }; // author-supplied body @@ -258,7 +354,7 @@ function emitSqlView( /** * The physical tables an `@sql` view depends on. migrate-ts uses this to drop+recreate * the view around a column-altering change on a source table (Postgres blocks ALTER on a - * column a view depends on). Two sources, no `@dependsOn` attr: + * column a view depends on). Three sources, no `@dependsOn` attr: * * - A **write-through host** (a writable table source + an `@sql` read-view source) * reads from its OWN table — its one certain dependency. It has NO extends anchors @@ -267,6 +363,9 @@ function emitSqlView( * - A **projection** `@sql` view's dependencies are its extends-bound anchor tables * (D7 — the `extends` bindings that anchor the read model's shape ARE the dependency * declaration). + * - A **report** `@sql` view reads its `@from` entity's table (FR-044). A report has + * neither a writable table nor extends anchors, so without this its dependsOn would + * be empty and a column ALTER on the `@from` table would fail at apply. * * Deduped. (A table the opaque body JOINs but neither hosts nor anchors is NOT tracked — * the deferred `@dependsOn` escape, ADR-0043.) @@ -291,6 +390,13 @@ function collectSqlDependsOn( const t = joinTables[owner.resolutionKey()]; if (t !== undefined) tables.add(t); } + // A report's `@from` table (resolved package-locally, as reportShape does). + if (isReport(host)) { + const fromName = reportFrom(host); + const from = fromName === undefined ? undefined : resolveObjectRef(root, fromName, packageOf(host)).node; + const t = from === undefined ? undefined : joinTables[from.resolutionKey()]; + if (t !== undefined) tables.add(t); + } return [...tables]; } diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts new file mode 100644 index 000000000..250c25731 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts @@ -0,0 +1,379 @@ +// FR-044 Plan 2 — lower an `object.report` plus its shape to a dialect-neutral +// ReportViewSpec (contract Table F). Dimension joins reuse the projection walk +// (`walkViaPath` / `pathsToJoins`), so hop resolution, ambiguity errors and the #209 +// join type are one implementation. The renderer (report-ddl-emit) turns the spec to SQL. + +import { + AGG_SUM, + FIELD_ATTR_LOCAL_TIME, + FIELD_SUBTYPE_CURRENCY, + FIELD_SUBTYPE_DATE, + FIELD_SUBTYPE_DOUBLE, + FIELD_SUBTYPE_ENUM, + FIELD_SUBTYPE_FLOAT, + FIELD_SUBTYPE_INT, + FIELD_SUBTYPE_LONG, + FIELD_SUBTYPE_TIMESTAMP, + FILTER_COMPOSE_AND, + FILTER_COMPOSE_OR, + FILTER_OP_IN, + FILTER_RELATIVE_NOW, + IDENTITY_REFERENCE_ATTR_REFERENCES, + IDENTITY_SUBTYPE_REFERENCE, + OBJECT_REPORT_ATTR_FILTER, + OBJECT_REPORT_ATTR_SEGMENT, + RELATIONSHIP_ATTR_OBJECT_REF, + TYPE_IDENTITY, + TYPE_MEASURE, + TYPE_RELATIONSHIP, + TYPE_SEGMENT, + reportShape, + reportingMemberOwner, + reportingViaHops, + resolveObjectRef, + resolveReportingFieldRef, + type MetaData, + type MetaField, + type MetaMeasure, + type MetaObject, + type MetaRoot, + type MetaSegment, + type ReportField, +} from "@metaobjectsdev/metadata"; +import { intValueMapOf } from "../enum-meta.js"; +import { columnNameFromField } from "../naming.js"; +import { hasWritableRdbSource } from "../source-detect.js"; +import { isTphSubtype, tphDiscriminatorBase, tphDiscriminatorPin } from "../templates/zod-validators.js"; +import { + desugarClause, + encodeIntEnumFilterValue, + packageOf, + pathsToJoins, + projectionViewName, + shortAliasFor, + sourceColumnNameFor, + walkViaPath, + type ExtractContext, + type Path, +} from "./extract-view-spec.js"; +import type { ReportAggregate, ReportColumn, ReportViewSpec } from "./report-spec.js"; +import type { ReportTemporal } from "./time-sql.js"; +import type { JoinNode, ViewFilterClause } from "./view-spec.js"; + +/** Table D's column kind for a `field.date` / `field.timestamp`. */ +export function temporalOf(field: MetaField): ReportTemporal { + if (field.subType === FIELD_SUBTYPE_DATE) return "date"; + return field.attr(FIELD_ATTR_LOCAL_TIME) === true ? "naive" : "instant"; +} + +const INTEGRAL_SUM: ReadonlySet = new Set([FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_CURRENCY]); +const FLOATING_SUM: ReadonlySet = new Set([FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT]); + +function isPlainObject(v: unknown): v is Record { + return typeof v === "object" && v !== null && !Array.isArray(v); +} + +function isRelativeValue(v: unknown): v is Record { + return isPlainObject(v) && FILTER_RELATIVE_NOW in v; +} + +/** AND of the present clauses, a lone clause as itself, none as undefined. */ +function andOf(clauses: readonly (ViewFilterClause | undefined)[]): ViewFilterClause | undefined { + const present = clauses.filter((c): c is ViewFilterClause => c !== undefined); + if (present.length === 0) return undefined; + return present.length === 1 ? present[0]! : { kind: "and", clauses: present }; +} + +/** + * A reporting filter (`{ field: value | { op: value }, and?, or? }`) over the `@from` + * entity's own fields on the base alias. Differs from `resolveAggregateFilter` in that every + * operator on a field survives (a range keeps both ends) and a relative-date operand + * (`{ now: "-P90D" }`, legal only on reporting hosts, rule F1) lowers to a `RelativeNow`. + * Projection and `origin.aggregate` filters keep refusing relative dates. + */ +function resolveReportFilter( + filter: unknown, + entity: MetaObject, + alias: string, + ctx: ExtractContext, + where: string, +): ViewFilterClause | undefined { + if (!isPlainObject(filter)) return undefined; + const clauses: ViewFilterClause[] = []; + for (const [key, val] of Object.entries(filter)) { + if (key === FILTER_COMPOSE_AND || key === FILTER_COMPOSE_OR) { + const subs = (Array.isArray(val) ? val : []) + .map((s) => resolveReportFilter(s, entity, alias, ctx, where)) + .filter((c): c is ViewFilterClause => c !== undefined); + if (subs.length > 0) clauses.push({ kind: key === FILTER_COMPOSE_AND ? "and" : "or", clauses: subs }); + continue; + } + // ADR-0039: resolving fields(), so a field inherited through extends is found. + const field = entity.fields().find((f) => f.name === key); + if (field === undefined) { + throw new Error(`${where}: filter field "${key}" is not a field of '${entity.name}'.`); + } + const ref = `${alias}.${sourceColumnNameFor(field, ctx)}`; + for (const [op, raw] of Object.entries(desugarClause(val))) { + // `IN ()` is a syntax error on Postgres and MySQL, so it would fail when the migration is + // applied, far from the report. The loader accepts the empty list; refuse it here by name. + if (op === FILTER_OP_IN && Array.isArray(raw) && raw.length === 0) { + throw new Error( + `${where}: the 'in' list on "${key}" is empty, which no row can match and no database accepts ` + + `as SQL (IN ()). List at least one value, or remove the clause.`, + ); + } + clauses.push({ kind: "cmp", ref, op, value: lowerFilterValue(raw, op, field, key, where) }); + } + } + return andOf(clauses); +} + +function lowerFilterValue(raw: unknown, op: string, field: MetaField, key: string, where: string): unknown { + const relative = (v: Record) => { + if (field.subType !== FIELD_SUBTYPE_DATE && field.subType !== FIELD_SUBTYPE_TIMESTAMP) { + throw new Error(`${where}: a relative-date value on "${key}" needs a field.date or field.timestamp.`); + } + return { + kind: "relativeNow" as const, + duration: String(v[FILTER_RELATIVE_NOW]), + temporal: temporalOf(field), + }; + }; + if (isRelativeValue(raw)) return relative(raw); + if (Array.isArray(raw) && raw.some(isRelativeValue)) { + return raw.map((v) => (isRelativeValue(v) ? relative(v) : v)); + } + return encodeIntEnumFilterValue( + raw, + op, + field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined, + key, + where, + ); +} + +/** A named member (segment or measure) declared on the `@from` entity. */ +function declared(from: MetaObject, type: string, name: string): MetaSegment | MetaMeasure | undefined { + // ADR-0039: resolving children(), so a member declared on an abstract base is found. The + // type string identifies the node (no `instanceof` across packages); the cast is type-only. + return from.children().find((c) => c.type === type && c.name === name) as MetaSegment | MetaMeasure | undefined; +} + +/** The filter of a named segment on `from`, resolved over `from`'s fields. */ +function segmentClause( + segmentName: string | undefined, + from: MetaObject, + alias: string, + ctx: ExtractContext, + where: string, +): ViewFilterClause | undefined { + if (segmentName === undefined) return undefined; + const segment = declared(from, TYPE_SEGMENT, segmentName) as MetaSegment | undefined; + if (segment === undefined) throw new Error(`${where}: segment '${segmentName}' is not declared on '${from.name}'.`); + return resolveReportFilter(segment.filter(), from, alias, ctx, `${where} segment '${segmentName}'`); +} + +function castFor(agg: string, of: MetaField | undefined): ReportAggregate["cast"] { + if (agg !== AGG_SUM || of === undefined) return undefined; + if (INTEGRAL_SUM.has(of.subType)) return "bigint"; + if (FLOATING_SUM.has(of.subType)) return "double"; + return undefined; +} + +/** One aggregate (Table C) for a `measure.aggregate` on `from`. */ +function aggregateOf( + measure: MetaMeasure, + report: MetaObject, + from: MetaObject, + baseAlias: string, + root: MetaRoot, + ctx: ExtractContext, +): ReportAggregate { + const where = `report '${report.name}' measure '${measure.name}'`; + const agg = measure.agg(); + if (agg === undefined) throw new Error(`${where}: has no @agg.`); + // The same rule as reportShape: the entity half resolves in the DECLARING entity's package, + // and the column is read from `from` (a measure aggregates `from`'s own rows). + const declaring = reportingMemberOwner(measure, from); + const fields = measure.ofColumns().map((ref) => { + const f = resolveReportingFieldRef(ref, declaring, root, from); + if (f === undefined) throw new Error(`${where}: @of '${ref}' does not resolve.`); + return f; + }); + const filter = andOf([ + segmentClause(measure.segmentName(), from, baseAlias, ctx, where), + resolveReportFilter(measure.filter(), from, baseAlias, ctx, `${where} @filter`), + ]); + const cast = castFor(agg, fields[0]); + return { + agg, + distinct: measure.distinct(), + refs: fields.map((f) => `${baseAlias}.${sourceColumnNameFor(f, ctx)}`), + ...(filter !== undefined ? { filter } : {}), + ...(cast !== undefined ? { cast } : {}), + }; +} + +/** The join alias a dimension's path ends on: walk the deduplicated tree by relationship name. */ +function aliasAtEndOf(path: Path, joins: readonly JoinNode[]): string { + let level = joins; + let alias = ""; + for (const step of path) { + const node = level.find((j) => j.relationship === step.relationship); + if (node === undefined) throw new Error(`report join tree lost the hop '${step.relationship}'.`); + alias = node.alias; + level = node.children; + } + return alias; +} + +/** + * Why a dimension's `@via` walk stopped at `hop`: the error names the hop, the entity it was + * looked up on, and what the model is missing. The loader (rule D2) accepts a to-one + * `relationship.*` with no `identity.reference` behind it, so the missing-foreign-key case is + * reachable from a model that loads clean. + */ +function viaHopError(where: string, via: string, hop: string, at: MetaData, root: MetaRoot): Error { + const head = `${where} @via '${via}' cannot be joined at hop '${hop}' on '${at.resolutionKey()}'`; + // ADR-0039: resolving children(), so an inherited relationship or reference is found. + const node = at + .children() + .find( + (c) => + c.name === hop && + (c.type === TYPE_RELATIONSHIP || (c.type === TYPE_IDENTITY && c.subType === IDENTITY_SUBTYPE_REFERENCE)), + ); + if (node === undefined) { + return new Error(`${head}: it names no relationship or identity.reference of that entity.`); + } + const targetRef = node.attr( + node.type === TYPE_IDENTITY ? IDENTITY_REFERENCE_ATTR_REFERENCES : RELATIONSHIP_ATTR_OBJECT_REF, + ); + const target = typeof targetRef === "string" ? resolveObjectRef(root, targetRef, packageOf(at)).node : undefined; + if (target === undefined) { + return new Error(`${head}: its target '${String(targetRef ?? "")}' does not resolve to an object.`); + } + return new Error( + `${head}: the model declares no foreign key for it. A view joins a hop through an identity.reference; ` + + `declare one on '${at.name}' whose @references is '${target.name}' (with the foreign-key field in @fields).`, + ); +} + +export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: ExtractContext): ReportViewSpec { + const shape = reportShape(report, root); + const from = shape.from; + // A view over a table that does not exist: refuse, naming both, rather than emit SQL that fails at apply. + if (from.isAbstract || !hasWritableRdbSource(from)) { + throw new Error( + `report '${report.name}': @from '${from.name}' has no table (it is abstract or declares no writable ` + + `source.rdb), so no view can be derived. Give '${from.name}' a source, or remove the report's source.`, + ); + } + // A TPH subtype has no table of its own: its rows sit in the discriminator base's table beside + // every other subtype's. A derived view has no discriminator predicate, so it would aggregate + // all of them and report wrong numbers with nothing failing. Refuse, and say how to scope it. + if (isTphSubtype(from)) { + const base = tphDiscriminatorBase(from); + const pin = tphDiscriminatorPin(from); + throw new Error( + `report '${report.name}': @from '${from.name}' is a TPH subtype: it shares the table of ` + + `'${base?.name ?? ""}' with every other subtype, so a view derived from it would aggregate all of ` + + `their rows. Declare the report @from '${base?.name ?? ""}' with an @filter on the discriminator ` + + `field '${pin?.fieldName ?? ""}' (for example { ${JSON.stringify(pin?.fieldName ?? "")}: ` + + `${JSON.stringify(pin?.value ?? "")} }).`, + ); + } + const used = new Set(); + const baseAlias = shortAliasFor(from.name, used); + + // One path per LISTED dimension that has @via (Table F); an unlisted dimension adds no join. + const pathOf = new Map(); + for (const f of shape.fields) { + const dim = f.dimension; + const via = dim?.via(); + if (dim === undefined || via === undefined) continue; + const where = `report '${report.name}': dimension '${f.name}'`; + // The loader's rule D2: the owner half resolves in the DECLARING entity's package and must be + // `from` or an entity it extends; the walk then starts AT `from`. + const hops = reportingViaHops(via, reportingMemberOwner(dim, from), from, root); + if (hops === undefined) { + throw new Error( + `${where} @via '${via}' must be Owner.hop[.hop...], starting at @from '${from.name}' or an entity it extends.`, + ); + } + // The head is `from`'s SHORT name, resolved in `from`'s own package: a bare name binds the + // referrer's package first, so it is `from` itself even when another package has an entity + // of that name. Never its resolution key: walkViaPath splits on every `.`, and a package + // name may contain one (`com.acme::F`). + const path = walkViaPath([from.name, ...hops].join("."), root, packageOf(from), ctx); + // walkViaPath stops at the first hop it cannot resolve; a partial path would pin the + // dimension to the wrong alias, so the whole chain must be walked. + if (path.length !== hops.length) { + const last = path[path.length - 1]; + const at = last === undefined ? from : root.objects().find((o) => o.resolutionKey() === last.targetEntity); + throw viaHopError(where, via, hops[path.length]!, at ?? from, root); + } + pathOf.set(f, path); + } + const joins = pathsToJoins([...pathOf.values()], used); + + const columns = shape.fields.map((f): ReportColumn => { + const dbColAlias = columnNameFromField(f.name, ctx.columnNamingStrategy); + if (f.role === "dimension") { + const dim = f.dimension; + // A time dimension below the hour grain carries no typeSource; resolve by reportShape's rule. + const of = + f.typeSource ?? + (dim === undefined + ? undefined + : resolveReportingFieldRef( + dim.of() ?? "", + reportingMemberOwner(dim, from), + root, + dim.via() === undefined ? from : undefined, + )); + if (of === undefined) throw new Error(`report '${report.name}': dimension '${f.name}' @of does not resolve.`); + const path = pathOf.get(f); + const alias = path === undefined ? baseAlias : aliasAtEndOf(path, joins); + const ref = `${alias}.${sourceColumnNameFor(of, ctx)}`; + if (f.grain !== undefined) { + return { kind: "timeDimension", fieldName: f.name, dbColAlias, ref, grain: f.grain, temporal: temporalOf(of) }; + } + return { kind: "dimension", fieldName: f.name, dbColAlias, ref }; + } + const measure = f.measure!; + if (measure.isRatio()) { + const operand = (name: string | undefined): ReportAggregate => { + const m = name === undefined ? undefined : (declared(from, TYPE_MEASURE, name) as MetaMeasure | undefined); + if (m === undefined || m.isRatio()) { + throw new Error( + `report '${report.name}': ratio '${measure.name}' operand '${name ?? ""}' is not a measure.aggregate on '${from.name}'.`, + ); + } + return aggregateOf(m, report, from, baseAlias, root, ctx); + }; + return { + kind: "ratio", + fieldName: f.name, + dbColAlias, + numerator: operand(measure.numerator()), + denominator: operand(measure.denominator()), + }; + } + return { kind: "aggregate", fieldName: f.name, dbColAlias, aggregate: aggregateOf(measure, report, from, baseAlias, root, ctx) }; + }); + + const reportWhere = `report '${report.name}'`; + const segment = report.attr(OBJECT_REPORT_ATTR_SEGMENT); + const where = andOf([ + segmentClause(typeof segment === "string" ? segment : undefined, from, baseAlias, ctx, reportWhere), + resolveReportFilter(report.attr(OBJECT_REPORT_ATTR_FILTER), from, baseAlias, ctx, `${reportWhere} @filter`), + ]); + return { + viewName: projectionViewName(report, ctx.columnNamingStrategy), + joinTree: { baseEntity: from.resolutionKey(), baseAlias, joins }, + columns, + ...(where !== undefined ? { where } : {}), + }; +} diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts index 70868f2da..eb47af8f6 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts @@ -105,7 +105,7 @@ const EXPR_COMPARISON_OPS: ReadonlySet = new Set([ * would lower as op `now` with value `"x"`, and `assertNoRelativeDate` — which inspects the * VALUE — would never see it. */ -function desugarClause(raw: unknown): Record { +export function desugarClause(raw: unknown): Record { if (raw === null) return { [FILTER_OP_IS_NULL]: true }; if (Array.isArray(raw)) return { [FILTER_OP_IN]: raw }; if (typeof raw === "object") { @@ -119,7 +119,8 @@ function desugarClause(raw: unknown): Record { * `@filter` of a segment, measure.aggregate or object.report (the loader's F1 rule), and * this lowering has no rendering for it: it would otherwise land as a SQL literal of * `[object Object]`. A programmatic caller skips the loader, so refuse it here, loudly. - * The report lowering (FR-044 Plan 2) replaces this throw. + * The throw stays for projection and `origin.aggregate` filters. A report lowers its relative + * values through `extract-report-spec.ts` and renders them in `report-ddl-emit.ts`. */ function assertNoRelativeDate(value: unknown, where: string): void { const isRelative = (v: unknown): boolean => @@ -211,7 +212,7 @@ function intEnumMapsOf(projection: MetaObject): ReadonlyMap | undefined, @@ -539,7 +540,7 @@ export function refNamedOwner(node: MetaData, root: MetaRoot): MetaObject | unde } /** Effective package of an object, taken from its resolution key ("::"). */ -function packageOf(obj: MetaData): string { +export function packageOf(obj: MetaData): string { const key = obj.resolutionKey(); const i = key.lastIndexOf("::"); return i >= 0 ? key.slice(0, i) : ""; @@ -597,7 +598,7 @@ function baseEntityFor( ); } -function sourceColumnNameFor( +export function sourceColumnNameFor( entityField: MetaData, ctx: ExtractContext, ): string { @@ -777,7 +778,7 @@ function resolveExprNode( return undefined; } -function shortAliasFor(entityName: string, used: Set): string { +export function shortAliasFor(entityName: string, used: Set): string { // Derive from the SHORT name — an entity ref may now be a resolutionKey ("pkg::Name", // #244); the alias must stay the first letter of the entity, so existing single-package // view SQL is byte-identical (a changed alias would churn `verify --db` fingerprints). @@ -796,7 +797,7 @@ function shortAliasFor(entityName: string, used: Set): string { // prefix into a trie, then converts to JoinNode tree. // --------------------------------------------------------------------------- -interface PathStep { +export interface PathStep { entity: MetaData; relationship: string; cardinality: "one" | "many"; @@ -809,13 +810,178 @@ interface PathStep { targetEntity: string; } -type Path = PathStep[]; +export type Path = PathStep[]; interface TrieNode { children: Map; step?: PathStep; } +/** Walk one dotted `@via` (`Owner.hop[.hop…]`) into join steps. Returns [] when the + * head or any hop does not resolve. Throws on an ambiguous hop (#368). A report dimension + * joins through this same walk as a projection origin, so hop resolution, the ambiguity + * errors and the #209 join type are one implementation. */ +export function walkViaPath(via: string, root: MetaRoot, referrerPkg: string, ctx: ExtractContext): Path { + const segments = via.split("."); + const rawEntity = segments[0]; + const relSegments = segments.slice(1); + if (!rawEntity) return []; + // @via may be package-qualified ("pkg::Entity.rel"). Resolve package-aware and key + // the joinTree on resolutionKey() (FQN) so a same-bare-named entity in another + // package can't win — the passthrough @from lookups key on the same FQN (#244). + let currentObj = resolveEntityRef(root, rawEntity, referrerPkg); + if (!currentObj) return []; + + const path: Path = []; + for (const relName of relSegments) { + // FR-024: a hop may name a relationship OR a reference-only FK + // (identity.reference — a to-one forward-FK edge). ADR-0039: resolving — + // a traversed relationship/reference may inherit its target via extends. + const resolved = resolveHop(currentObj, relName); + if (!resolved) break; + const { hop, targetName, cardinality } = resolved; + // @objectRef/@references may be package-qualified ("pkg::Entity"); resolve it + // package-aware relative to the hop's source entity (the loader qualifies a + // same-package ref even when authored bare), so the join binds the exact target. + const target = resolveEntityRef(root, targetName, packageOf(currentObj)); + if (!target) break; + + // #368: two identity.reference declarations onto the same target are legal + // (e.g. Match.homeTeamRef/awayTeamRef -> Team) — resolveHopReference prefers + // the SPECIFIC reference/relationship the hop already named over re-deriving + // one from the target alone, so an explicit `@via: "Match.homeTeamRef"` (or a + // relationship disambiguated by @sourceRefField/name-pairing) resolves cleanly. + // Only a relationship hop that even the ladder cannot choose reaches the throw. + const resolvedRef = resolveHopReference(currentObj as MetaObject, hop, relName, target); + let ref: ReferenceLookup | undefined; + if (Array.isArray(resolvedRef)) { + if (resolvedRef.length > 1) { + // #368 round 2: @sourceRefField cannot fix this, but WHY differs by shape, and + // asserting the wrong reason for a given shape is itself a bug (fix round 1 of + // this cleanup caught exactly that). resolveRelationshipReference's ladder reads + // ONLY the hop's own entity's candidates (referenceCandidatesFor(currentObj, ...)); + // it never even looks at `target`'s references. So: + // - If `currentObj` itself holds one of the ambiguous candidates, resolution + // already tried @sourceRefField/name-pairing against it and failed — and that + // is only reachable at all when @cardinality isn't "one": a @cardinality "one" + // relationship with 2+ own-side candidates is rejected at LOAD by rule (e) + // (validateOneSideReferenceResolution) using this exact same ladder, so if we + // got this far with an own-side candidate, @cardinality is provably not "one", + // and @sourceRefField is provably illegal here (rule (d)). + // - If NONE of the candidates are `currentObj`'s own, @sourceRefField could not + // have mattered regardless of @cardinality — it only ever consults the hop's + // OWN identity.reference children, and it has none targeting `target`. This is + // rule (e)'s zero-candidate gap (validation-passes.ts:2226, `<= 1` skips 0 too): + // a @cardinality "one" relationship can reach here with the FK entirely on the + // far side, so @cardinality itself must NOT be asserted in this branch. + const holderName = (currentObj as MetaObject).name; + const holderOwnsACandidate = resolvedRef.some((r) => r.holder.name === holderName); + const whySourceRefFieldCannotHelp = holderOwnsACandidate + ? `it only disambiguates a @cardinality "${CARDINALITY_ONE}" relationship, and this ` + + `relationship's @cardinality is not "${CARDINALITY_ONE}" (declaring @sourceRefField on it ` + + `is itself a load error)` + : `it only consults "${holderName}"'s own identity.reference children, and "${holderName}" ` + + `declares none targeting "${target.name}" -- every candidate above belongs to the other side ` + + `of this join`; + throw new Error( + `projection join hop "${relName}" from "${holderName}" to "${target.name}" is ambiguous: ` + + `${resolvedRef.map((r) => r.referenceIdentity.name).join(", ")}. ` + + `@sourceRefField cannot resolve this: ${whySourceRefFieldCannotHelp}. There is no attribute ` + + `that disambiguates a hop like this -- remove the extra identity.reference between these two ` + + `entities, or restructure the model so only one remains.`, + ); + } + ref = resolvedRef[0]; + } else { + ref = resolvedRef; + } + if (!ref) break; + + const fkField = ref.referenceIdentity.fields[0]; + if (!fkField) break; + + const resolvedPkField = ref.referenceIdentity.resolvedTargetPkField(root) ?? "id"; + + const referenceHolder: "source" | "target" = + ref.holder.name === currentObj.name ? "source" : "target"; + + // FK lives on the holder; PK on the entity it references. Resolve both to + // physical columns now so the ON clause is naming-strategy correct. + const fkHolder = referenceHolder === "source" ? currentObj : target; + const pkHolder = referenceHolder === "source" ? target : currentObj; + + // #209 — a belongs-to hop (FK on the parent) whose FK is NOT NULL is + // semantically INNER: every base row has a match, so INNER and LEFT OUTER + // return the same set, and INNER matches the hand-written view it stands in + // for (and keeps `verify --db` fingerprints aligned). A nullable belongs-to + // FK, or ANY has-many hop (FK on the child — a base row may have zero + // children), stays LEFT OUTER so no base row is dropped. + // `@enforce: false` does NOT change this. An unenforced NOT NULL reference can name a + // row that does not exist, so INNER filters that base row out — and that filter is + // what the hand-written view did: a legacy account view joins `ref_id` INNER to the + // user table precisely to exclude the accounts whose `ref_id` holds a group id. Making + // the hop LEFT OUTER (tried, then reverted before release) silently changed which rows + // such views return. To keep unmatched rows, make the FK field nullable. + const fkFieldObj = (fkHolder as MetaObject).findField(fkField); + const selfInner = + referenceHolder === "source" && fkFieldObj !== undefined && isRequired(fkFieldObj); + // Nested-chain safety: joins render flat + left-associative, so an INNER hop + // BELOW any LEFT ancestor drops the base row (its ON references a column the + // LEFT ancestor NULLed). An INNER only survives when the ENTIRE ancestor chain + // is INNER; otherwise demote to LEFT (lossless — under a LEFT ancestor, LEFT is + // the correct type). `path` holds this chain's ancestor hops accumulated so far. + const joinType: "inner" | "left" = + selfInner && path.every((prior) => prior.joinType === "inner") ? "inner" : "left"; + + path.push({ + entity: currentObj, + relationship: relName, + cardinality, + fkColumn: joinColumnFor(fkHolder, fkField, ctx), + pkColumn: joinColumnFor(pkHolder, resolvedPkField, ctx), + referenceHolder, + joinType, + targetEntity: target.resolutionKey(), + }); + currentObj = target; + } + return path; +} + +/** Prefix-dedupe paths into JoinNodes, assigning aliases. */ +export function pathsToJoins(paths: readonly Path[], usedAliases: Set): JoinNode[] { + // Dedupe by prefix: paths sharing a prefix collapse into one join branch. + const trieRoot: TrieNode = { children: new Map() }; + for (const path of paths) { + let node = trieRoot; + for (const step of path) { + let child = node.children.get(step.relationship); + if (!child) { + child = { children: new Map(), step }; + node.children.set(step.relationship, child); + } + node = child; + } + } + + function toJoinNode(node: TrieNode): JoinNode { + const step = node.step!; + return { + relationship: step.relationship, + targetEntity: step.targetEntity, + alias: shortAliasFor(step.targetEntity, usedAliases), + cardinality: step.cardinality, + fkColumn: step.fkColumn, + pkColumn: step.pkColumn, + referenceHolder: step.referenceHolder, + joinType: step.joinType, + children: Array.from(node.children.values()).map(toJoinNode), + }; + } + + return Array.from(trieRoot.children.values()).map(toJoinNode); +} + function buildJoinTree( projection: MetaObject, base: MetaObject, @@ -853,166 +1019,15 @@ function buildJoinTree( } if (!viaAttr) continue; - const segments = viaAttr.split("."); - const rawEntity = segments[0]; - const relSegments = segments.slice(1); - if (!rawEntity) continue; - // @via may be package-qualified ("pkg::Entity.rel"). Resolve package-aware and key - // the joinTree on resolutionKey() (FQN) so a same-bare-named entity in another - // package can't win — the passthrough @from lookups key on the same FQN (#244). - let currentObj = resolveEntityRef(root, rawEntity, projPkg); - if (!currentObj) continue; - - const path: Path = []; - for (const relName of relSegments) { - // FR-024: a hop may name a relationship OR a reference-only FK - // (identity.reference — a to-one forward-FK edge). ADR-0039: resolving — - // a traversed relationship/reference may inherit its target via extends. - const resolved = resolveHop(currentObj, relName); - if (!resolved) break; - const { hop, targetName, cardinality } = resolved; - // @objectRef/@references may be package-qualified ("pkg::Entity"); resolve it - // package-aware relative to the hop's source entity (the loader qualifies a - // same-package ref even when authored bare), so the join binds the exact target. - const target = resolveEntityRef(root, targetName, packageOf(currentObj)); - if (!target) break; - - // #368: two identity.reference declarations onto the same target are legal - // (e.g. Match.homeTeamRef/awayTeamRef -> Team) — resolveHopReference prefers - // the SPECIFIC reference/relationship the hop already named over re-deriving - // one from the target alone, so an explicit `@via: "Match.homeTeamRef"` (or a - // relationship disambiguated by @sourceRefField/name-pairing) resolves cleanly. - // Only a relationship hop that even the ladder cannot choose reaches the throw. - const resolvedRef = resolveHopReference(currentObj as MetaObject, hop, relName, target); - let ref: ReferenceLookup | undefined; - if (Array.isArray(resolvedRef)) { - if (resolvedRef.length > 1) { - // #368 round 2: @sourceRefField cannot fix this, but WHY differs by shape, and - // asserting the wrong reason for a given shape is itself a bug (fix round 1 of - // this cleanup caught exactly that). resolveRelationshipReference's ladder reads - // ONLY the hop's own entity's candidates (referenceCandidatesFor(currentObj, ...)); - // it never even looks at `target`'s references. So: - // - If `currentObj` itself holds one of the ambiguous candidates, resolution - // already tried @sourceRefField/name-pairing against it and failed — and that - // is only reachable at all when @cardinality isn't "one": a @cardinality "one" - // relationship with 2+ own-side candidates is rejected at LOAD by rule (e) - // (validateOneSideReferenceResolution) using this exact same ladder, so if we - // got this far with an own-side candidate, @cardinality is provably not "one", - // and @sourceRefField is provably illegal here (rule (d)). - // - If NONE of the candidates are `currentObj`'s own, @sourceRefField could not - // have mattered regardless of @cardinality — it only ever consults the hop's - // OWN identity.reference children, and it has none targeting `target`. This is - // rule (e)'s zero-candidate gap (validation-passes.ts:2226, `<= 1` skips 0 too): - // a @cardinality "one" relationship can reach here with the FK entirely on the - // far side, so @cardinality itself must NOT be asserted in this branch. - const holderName = (currentObj as MetaObject).name; - const holderOwnsACandidate = resolvedRef.some((r) => r.holder.name === holderName); - const whySourceRefFieldCannotHelp = holderOwnsACandidate - ? `it only disambiguates a @cardinality "${CARDINALITY_ONE}" relationship, and this ` + - `relationship's @cardinality is not "${CARDINALITY_ONE}" (declaring @sourceRefField on it ` + - `is itself a load error)` - : `it only consults "${holderName}"'s own identity.reference children, and "${holderName}" ` + - `declares none targeting "${target.name}" -- every candidate above belongs to the other side ` + - `of this join`; - throw new Error( - `projection join hop "${relName}" from "${holderName}" to "${target.name}" is ambiguous: ` + - `${resolvedRef.map((r) => r.referenceIdentity.name).join(", ")}. ` + - `@sourceRefField cannot resolve this: ${whySourceRefFieldCannotHelp}. There is no attribute ` + - `that disambiguates a hop like this -- remove the extra identity.reference between these two ` + - `entities, or restructure the model so only one remains.`, - ); - } - ref = resolvedRef[0]; - } else { - ref = resolvedRef; - } - if (!ref) break; - - const fkField = ref.referenceIdentity.fields[0]; - if (!fkField) break; - - const resolvedPkField = ref.referenceIdentity.resolvedTargetPkField(root) ?? "id"; - - const referenceHolder: "source" | "target" = - ref.holder.name === currentObj.name ? "source" : "target"; - - // FK lives on the holder; PK on the entity it references. Resolve both to - // physical columns now so the ON clause is naming-strategy correct. - const fkHolder = referenceHolder === "source" ? currentObj : target; - const pkHolder = referenceHolder === "source" ? target : currentObj; - - // #209 — a belongs-to hop (FK on the parent) whose FK is NOT NULL is - // semantically INNER: every base row has a match, so INNER and LEFT OUTER - // return the same set, and INNER matches the hand-written view it stands in - // for (and keeps `verify --db` fingerprints aligned). A nullable belongs-to - // FK, or ANY has-many hop (FK on the child — a base row may have zero - // children), stays LEFT OUTER so no base row is dropped. - // `@enforce: false` does NOT change this. An unenforced NOT NULL reference can name a - // row that does not exist, so INNER filters that base row out — and that filter is - // what the hand-written view did: a legacy account view joins `ref_id` INNER to the - // user table precisely to exclude the accounts whose `ref_id` holds a group id. Making - // the hop LEFT OUTER (tried, then reverted before release) silently changed which rows - // such views return. To keep unmatched rows, make the FK field nullable. - const fkFieldObj = (fkHolder as MetaObject).findField(fkField); - const selfInner = - referenceHolder === "source" && fkFieldObj !== undefined && isRequired(fkFieldObj); - // Nested-chain safety: joins render flat + left-associative, so an INNER hop - // BELOW any LEFT ancestor drops the base row (its ON references a column the - // LEFT ancestor NULLed). An INNER only survives when the ENTIRE ancestor chain - // is INNER; otherwise demote to LEFT (lossless — under a LEFT ancestor, LEFT is - // the correct type). `path` holds this chain's ancestor hops accumulated so far. - const joinType: "inner" | "left" = - selfInner && path.every((prior) => prior.joinType === "inner") ? "inner" : "left"; - - path.push({ - entity: currentObj, - relationship: relName, - cardinality, - fkColumn: joinColumnFor(fkHolder, fkField, ctx), - pkColumn: joinColumnFor(pkHolder, resolvedPkField, ctx), - referenceHolder, - joinType, - targetEntity: target.resolutionKey(), - }); - currentObj = target; - } + const path = walkViaPath(viaAttr, root, projPkg, ctx); if (path.length > 0) allPaths.push(path); } } - // Dedupe by prefix: paths sharing a prefix collapse into one join branch. - const trieRoot: TrieNode = { children: new Map() }; - for (const path of allPaths) { - let node = trieRoot; - for (const step of path) { - let child = node.children.get(step.relationship); - if (!child) { - child = { children: new Map(), step }; - node.children.set(step.relationship, child); - } - node = child; - } - } - - function toJoinNode(node: TrieNode): JoinNode { - const step = node.step!; - return { - relationship: step.relationship, - targetEntity: step.targetEntity, - alias: shortAliasFor(step.targetEntity, usedAliases), - cardinality: step.cardinality, - fkColumn: step.fkColumn, - pkColumn: step.pkColumn, - referenceHolder: step.referenceHolder, - joinType: step.joinType, - children: Array.from(node.children.values()).map(toJoinNode), - }; - } - return { baseEntity: base.resolutionKey(), baseAlias, - joins: Array.from(trieRoot.children.values()).map(toJoinNode), + joins: pathsToJoins(allPaths, usedAliases), }; } diff --git a/server/typescript/packages/codegen-ts/src/projection/index.ts b/server/typescript/packages/codegen-ts/src/projection/index.ts index a3c4c5bc1..c89f7f9a2 100644 --- a/server/typescript/packages/codegen-ts/src/projection/index.ts +++ b/server/typescript/packages/codegen-ts/src/projection/index.ts @@ -2,3 +2,6 @@ export * from "./view-spec.js"; export * from "./extract-view-spec.js"; export * from "./view-ddl-emit.js"; export * from "./projection-detector.js"; +export * from "./report-spec.js"; +export * from "./extract-report-spec.js"; +export * from "./report-ddl-emit.js"; diff --git a/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts b/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts new file mode 100644 index 000000000..e40a45654 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts @@ -0,0 +1,177 @@ +// FR-044 Plan 2 (contract Tables C, F) — renders a ReportViewSpec to view SQL for +// Postgres, SQLite and MySQL. Kept apart from view-ddl-emit.ts on purpose: the projection +// emitter quotes conditionally (`quoteIfNeeded`) and is Postgres/SQLite only; a report +// quotes every identifier unconditionally, so a measure named `order` is valid DDL. +import type { JoinNode, ViewFilterClause } from "./view-spec.js"; +import type { ReportAggregate, ReportColumn, ReportViewSpec } from "./report-spec.js"; +import { isRelativeNow } from "./report-spec.js"; +import { relativeNowSql, truncateToGrain, type ReportDialect } from "./time-sql.js"; + +export interface ReportEmitOptions { + readonly dialect: ReportDialect; + readonly baseTableName: string; + /** Map from entity name → table name for every entity referenced in joins. */ + readonly joinTables: Readonly>; + /** Body only (no CREATE VIEW wrapper, no trailing `;`), as migrate-ts consumes it. */ + readonly bodyOnly?: boolean; +} + +/** An identifier, quoted unconditionally. */ +function q(ident: string, d: ReportDialect): string { + return d === "mysql" ? "`" + ident.replace(/`/g, "``") + "`" : `"${ident.replace(/"/g, '""')}"`; +} + +/** `alias.column` → `alias."column"`. The alias is generated, never quoted. */ +function ref(r: string, d: ReportDialect): string { + const dot = r.indexOf("."); + return dot < 0 ? q(r, d) : `${r.slice(0, dot)}.${q(r.slice(dot + 1), d)}`; +} + +function literal(v: unknown, d: ReportDialect): string { + if (isRelativeNow(v)) return relativeNowSql(v.duration, v.temporal, d); + if (v === null || v === undefined) return "NULL"; + if (typeof v === "number") return String(v); + if (typeof v === "boolean") return d === "sqlite" ? (v ? "1" : "0") : v ? "TRUE" : "FALSE"; + const s = String(v).replace(/'/g, "''"); + return `'${d === "mysql" ? s.replace(/\\/g, "\\\\") : s}'`; +} + +const FILTER_OP_SQL: Readonly> = { + eq: "=", ne: "<>", gt: ">", gte: ">=", lt: "<", lte: "<=", like: "LIKE", +}; + +/** A resolved filter clause as a SQL boolean expression; `and` / `or` groups are parenthesised. */ +function cond(clause: ViewFilterClause, d: ReportDialect): string { + switch (clause.kind) { + case "and": + case "or": + return `(${clause.clauses.map((c) => cond(c, d)).join(clause.kind === "and" ? " AND " : " OR ")})`; + case "exprCmp": + throw new Error("report-ddl-emit: a report filter never lowers to an exprCmp clause."); + case "cmp": { + const lhs = ref(clause.ref, d); + if (clause.op === "isNull") return clause.value === false ? `${lhs} IS NOT NULL` : `${lhs} IS NULL`; + if (clause.op === "in") { + const vals = (Array.isArray(clause.value) ? clause.value : [clause.value]).map((v) => literal(v, d)); + return `${lhs} IN (${vals.join(", ")})`; + } + const op = FILTER_OP_SQL[clause.op]; + if (op === undefined) throw new Error(`report-ddl-emit: unsupported filter operator "${clause.op}".`); + return `${lhs} ${op} ${literal(clause.value, d)}`; + } + } +} + +function castType(cast: "bigint" | "double", d: ReportDialect): string | undefined { + switch (d) { + case "postgres": + return cast === "bigint" ? "BIGINT" : "DOUBLE PRECISION"; + case "mysql": + return cast === "bigint" ? "SIGNED" : undefined; // MySQL SUM(double) is already DOUBLE + case "sqlite": + return undefined; // SQLite has one integer and one real affinity; SUM already fits + } +} + +/** One aggregate (Table C): the bare aggregate, the condition by dialect, the cast last. */ +function aggregate(a: ReportAggregate, d: ReportDialect): string { + const refs = a.refs.map((r) => ref(r, d)); + const c = a.filter === undefined ? undefined : cond(a.filter, d); + const fn = a.agg.toUpperCase(); + let sql: string; + if (refs.length > 1) { + // A distinct tuple count: a tuple with any NULL component is not counted, on every dialect. + const notNull = refs.map((r) => `${r} IS NOT NULL`); + const both = [...notNull, ...(c === undefined ? [] : [c])].join(" AND "); + switch (d) { + case "postgres": + sql = `COUNT(DISTINCT (${refs.join(", ")})) FILTER (WHERE ${both})`; + break; + case "sqlite": + sql = `COUNT(DISTINCT CASE WHEN ${both} THEN json_array(${refs.join(", ")}) END)`; + break; + case "mysql": { + // MySQL's multi-argument COUNT(DISTINCT …) already skips a tuple with a NULL component. + const [first, ...rest] = refs; + const head = c === undefined ? first! : `CASE WHEN ${c} THEN ${first} END`; + sql = `COUNT(DISTINCT ${[head, ...rest].join(", ")})`; + break; + } + } + } else { + const x = refs[0]!; + const distinct = a.distinct ? "DISTINCT " : ""; + if (c === undefined) sql = `${fn}(${distinct}${x})`; + else if (d === "postgres") sql = `${fn}(${distinct}${x}) FILTER (WHERE ${c})`; + else sql = `${fn}(${distinct}CASE WHEN ${c} THEN ${x} END)`; + } + const type = a.cast === undefined ? undefined : castType(a.cast, d); + return type === undefined ? sql : `CAST(${sql} AS ${type})`; +} + +interface RenderedColumn { + readonly expr: string; + readonly alias: string; + /** Present for a dimension: its expression is the GROUP BY term. */ + readonly grouped: boolean; +} + +function column(c: ReportColumn, d: ReportDialect): RenderedColumn { + const alias = q(c.dbColAlias, d); + switch (c.kind) { + case "dimension": + return { expr: ref(c.ref, d), alias, grouped: true }; + case "timeDimension": + return { expr: truncateToGrain(ref(c.ref, d), c.grain, c.temporal, d), alias, grouped: true }; + case "aggregate": + return { expr: aggregate(c.aggregate, d), alias, grouped: false }; + case "ratio": { + // Each operand is its FULL Table C expression (condition and cast included). + const num = aggregate(c.numerator, d); + const den = aggregate(c.denominator, d); + const top = d === "postgres" ? "NUMERIC" : d === "sqlite" ? "REAL" : undefined; + return { + expr: `${top === undefined ? num : `CAST(${num} AS ${top})`} / NULLIF(${den}, 0)`, + alias, + grouped: false, + }; + } + } +} + +function renderJoin(node: JoinNode, parentAlias: string, options: ReportEmitOptions): string { + const table = options.joinTables[node.targetEntity]; + if (!table) { + throw new Error(`report-ddl-emit: no table name registered for joined entity "${node.targetEntity}".`); + } + const d = options.dialect; + const fk = q(node.fkColumn, d); + const pk = q(node.pkColumn, d); + // referenceHolder "source": FK on the parent (belongs-to); "target": FK on the child (has-many). + const on = node.referenceHolder === "source" + ? `${node.alias}.${pk} = ${parentAlias}.${fk}` + : `${node.alias}.${fk} = ${parentAlias}.${pk}`; + const kw = node.joinType === "inner" ? "INNER JOIN" : "LEFT OUTER JOIN"; + let sql = ` ${kw} ${q(table, d)} ${node.alias} ON ${on}`; + for (const child of node.children) sql += "\n" + renderJoin(child, node.alias, options); + return sql; +} + +export function emitReportViewDdl(spec: ReportViewSpec, options: ReportEmitOptions): string { + const d = options.dialect; + // Rendered once: a dimension's SELECT expression is its GROUP BY term, so they cannot differ. + const cols = spec.columns.map((c) => column(c, d)); + const select = cols.map((c) => ` ${c.expr} AS ${c.alias}`).join(",\n"); + const groupBy = cols.filter((c) => c.grouped).map((c) => c.expr); + + const base = spec.joinTree.baseAlias; + const joins = spec.joinTree.joins.map((j) => renderJoin(j, base, options)).join("\n"); + const body = + ` SELECT\n${select}\n FROM ${q(options.baseTableName, d)} ${base}` + + (joins === "" ? "" : `\n${joins}`) + + (spec.where === undefined ? "" : `\n WHERE ${cond(spec.where, d)}`) + + (groupBy.length === 0 ? "" : `\n GROUP BY ${groupBy.join(", ")}`); + + if (options.bodyOnly) return body; + return `CREATE VIEW ${q(spec.viewName, d)} AS\n${body};`; +} diff --git a/server/typescript/packages/codegen-ts/src/projection/report-spec.ts b/server/typescript/packages/codegen-ts/src/projection/report-spec.ts new file mode 100644 index 000000000..5fcc0f770 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/projection/report-spec.ts @@ -0,0 +1,63 @@ +// FR-044 Plan 2 (contract Table F) — the dialect-neutral shape of a report's view. +// `extractReportSpec` produces it; the report DDL emitter renders it per dialect. Column +// references are already resolved to unquoted `alias.column`, as in `ViewSpec`. + +import type { MeasureAgg, TimeGrain } from "@metaobjectsdev/metadata"; +import type { JoinTree, ViewFilterClause } from "./view-spec.js"; +import type { ReportTemporal } from "./time-sql.js"; + +/** A relative-date operand, carried as the `value` of a ViewFilterClause `cmp`. */ +export interface RelativeNow { + readonly kind: "relativeNow"; + /** The signed ISO-8601 duration as authored, e.g. "-P90D". */ + readonly duration: string; + readonly temporal: ReportTemporal; +} + +export function isRelativeNow(v: unknown): v is RelativeNow { + return typeof v === "object" && v !== null && (v as { kind?: unknown }).kind === "relativeNow"; +} + +/** One aggregate (Table C). `refs` are unquoted `alias.column`. */ +export interface ReportAggregate { + readonly agg: MeasureAgg; + readonly distinct: boolean; + readonly refs: readonly string[]; + /** The measure's condition: its `@segment` filter AND its `@filter`. */ + readonly filter?: ViewFilterClause; + /** The Table C cast: integral sum → "bigint", floating sum → "double". */ + readonly cast?: "bigint" | "double"; +} + +export type ReportColumn = + | { readonly kind: "dimension"; readonly fieldName: string; readonly dbColAlias: string; readonly ref: string } + | { + readonly kind: "timeDimension"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly ref: string; + readonly grain: TimeGrain; + readonly temporal: ReportTemporal; + } + | { + readonly kind: "aggregate"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly aggregate: ReportAggregate; + } + | { + readonly kind: "ratio"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly numerator: ReportAggregate; + readonly denominator: ReportAggregate; + }; + +export interface ReportViewSpec { + readonly viewName: string; + readonly joinTree: JoinTree; + /** One column per `@dimensions` item then one per `@measures` item, in listed order. */ + readonly columns: readonly ReportColumn[]; + /** The report's `@segment` filter, then its `@filter`, ANDed. Absent when neither is declared. */ + readonly where?: ViewFilterClause; +} diff --git a/server/typescript/packages/codegen-ts/src/projection/time-sql.ts b/server/typescript/packages/codegen-ts/src/projection/time-sql.ts new file mode 100644 index 000000000..02dd3633d --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/projection/time-sql.ts @@ -0,0 +1,174 @@ +// Time-grain truncation (contract Table D) and relative-date values (Table E), +// rendered as SQL for the three view dialects. Pure string functions: the report +// DDL emitter supplies an already-quoted column reference and picks the dialect. +import { + GRAIN_DAY, GRAIN_HOUR, GRAIN_MONTH, GRAIN_QUARTER, GRAIN_WEEK, GRAIN_YEAR, + ISO_DURATION_RE, + TIME_GRAINS, + type TimeGrain, +} from "@metaobjectsdev/metadata"; + +export type ReportDialect = "postgres" | "sqlite" | "mysql"; +/** Table D's three column kinds: `instant` is a `field.timestamp` (TIMESTAMPTZ), + * `naive` one with `@localTime: true`, `date` a `field.date`. */ +export type ReportTemporal = "date" | "instant" | "naive"; + +export interface IsoDurationParts { + readonly sign: "+" | "-"; + readonly years: number; + readonly months: number; + readonly weeks: number; + readonly days: number; + readonly hours: number; + readonly minutes: number; + readonly seconds: number; + /** The duration text without its sign, e.g. "P7D". */ + readonly magnitude: string; +} + +/** Leading integer of a capture such as `"12H"`; an absent group is 0. */ +function component(group: string | undefined): number { + return group === undefined ? 0 : Number.parseInt(group, 10); +} + +/** Parse a signed ISO-8601 duration. Capture groups of `ISO_DURATION_RE`, in order: + * 1 `nY`, 2 `nM` (months), 3 `nW`, 4 `nD`, 5 the whole `T…` block, 6 `nH`, 7 `nM` + * (minutes), 8 `nS`. */ +export function parseIsoDuration(duration: string): IsoDurationParts { + const m = ISO_DURATION_RE.exec(duration); + if (m === null) throw new Error(`time-sql: "${duration}" is not an ISO-8601 duration.`); + return { + sign: duration.startsWith("-") ? "-" : "+", + years: component(m[1]), + months: component(m[2]), + weeks: component(m[3]), + days: component(m[4]), + hours: component(m[6]), + minutes: component(m[7]), + seconds: component(m[8]), + magnitude: duration.replace(/^[+-]/, ""), + }; +} + +/** Table D. `ref` is an already-quoted `alias."column"` reference. */ +export function truncateToGrain( + ref: string, + grain: TimeGrain, + temporal: ReportTemporal, + dialect: ReportDialect, +): string { + // The Postgres arm writes the grain into `date_trunc('', ...)`. The loader validates + // it (rule R2); a programmatic caller skips the loader, so check the closed set here. + if (!(TIME_GRAINS as readonly string[]).includes(grain)) { + throw new Error(`time-sql: "${String(grain)}" is not a time grain (${TIME_GRAINS.join(", ")}).`); + } + if (grain === GRAIN_HOUR && temporal === "date") { + // Rule D4 forbids this at load; a programmatic caller skips the loader. + throw new Error(`time-sql: the "hour" grain cannot truncate a date column (${ref}).`); + } + switch (dialect) { + case "postgres": + return truncatePostgres(ref, grain, temporal); + case "sqlite": + return truncateSqlite(ref, grain, temporal); + case "mysql": + return truncateMysql(ref, grain); + } +} + +function truncatePostgres(x: string, grain: TimeGrain, temporal: ReportTemporal): string { + switch (temporal) { + case "instant": + return grain === GRAIN_HOUR + ? `date_trunc('hour', ${x}, 'UTC')` + : `CAST(date_trunc('${grain}', ${x} AT TIME ZONE 'UTC') AS DATE)`; + case "naive": + return grain === GRAIN_HOUR + ? `date_trunc('hour', ${x})` + : `CAST(date_trunc('${grain}', ${x}) AS DATE)`; + case "date": + // date_trunc(text, date) resolves to the timestamptz overload and truncates in + // the session zone, so the date is cast to TIMESTAMP first. + return grain === GRAIN_DAY + ? x + : `CAST(date_trunc('${grain}', CAST(${x} AS TIMESTAMP)) AS DATE)`; + } +} + +function truncateSqlite(x: string, grain: TimeGrain, temporal: ReportTemporal): string { + switch (grain) { + case GRAIN_HOUR: + return temporal === "instant" + ? `strftime('%Y-%m-%dT%H:00:00.000Z', ${x})` + : `strftime('%Y-%m-%dT%H:00:00', ${x})`; + case GRAIN_DAY: + return `date(${x})`; + case GRAIN_WEEK: + return `date(${x}, 'weekday 0', '-6 days')`; + case GRAIN_MONTH: + return `date(${x}, 'start of month')`; + case GRAIN_QUARTER: + return `date(${x}, 'start of month', '-' || ((CAST(strftime('%m', ${x}) AS INTEGER) - 1) % 3) || ' months')`; + case GRAIN_YEAR: + return `date(${x}, 'start of year')`; + } +} + +function truncateMysql(x: string, grain: TimeGrain): string { + switch (grain) { + case GRAIN_HOUR: + return `CAST(DATE_FORMAT(${x}, '%Y-%m-%d %H:00:00') AS DATETIME(3))`; + case GRAIN_DAY: + return `DATE(${x})`; + case GRAIN_WEEK: + return `DATE(DATE_SUB(${x}, INTERVAL WEEKDAY(${x}) DAY))`; + case GRAIN_MONTH: + return `DATE(DATE_FORMAT(${x}, '%Y-%m-01'))`; + case GRAIN_QUARTER: + return `MAKEDATE(YEAR(${x}), 1) + INTERVAL (QUARTER(${x}) - 1) QUARTER`; + case GRAIN_YEAR: + return `MAKEDATE(YEAR(${x}), 1)`; + } +} + +/** Table E. `{ now: "" }` as SQL, evaluated when the view is queried. */ +export function relativeNowSql( + duration: string, + temporal: ReportTemporal, + dialect: ReportDialect, +): string { + const d = parseIsoDuration(duration); + switch (dialect) { + case "postgres": { + // ISO-8601 interval input is accepted as written. + const interval = `INTERVAL '${d.magnitude}'`; + if (temporal === "instant") return `(now() ${d.sign} ${interval})`; + const utcWall = `((now() AT TIME ZONE 'UTC') ${d.sign} ${interval})`; + return temporal === "naive" ? utcWall : `CAST(${utcWall} AS DATE)`; + } + case "sqlite": { + // One modifier per non-zero component, Y M W D H M S order; a week is 7 days. + const mods = [ + [d.years, "years"], [d.months, "months"], [d.weeks * 7, "days"], [d.days, "days"], + [d.hours, "hours"], [d.minutes, "minutes"], [d.seconds, "seconds"], + ] + .filter(([n]) => (n as number) !== 0) + .map(([n, unit]) => `'${d.sign}${n} ${unit}'`); + const args = ["'now'", ...mods].join(", "); + if (temporal === "date") return `date(${args})`; + const fmt = temporal === "instant" ? "%Y-%m-%dT%H:%M:%fZ" : "%Y-%m-%dT%H:%M:%f"; + return `strftime('${fmt}', ${args})`; + } + case "mysql": { + const intervals = [ + [d.years, "YEAR"], [d.months, "MONTH"], [d.weeks, "WEEK"], [d.days, "DAY"], + [d.hours, "HOUR"], [d.minutes, "MINUTE"], [d.seconds, "SECOND"], + ] + .filter(([n]) => (n as number) !== 0) + .map(([n, unit]) => ` ${d.sign} INTERVAL ${n} ${unit}`) + .join(""); + const now = `UTC_TIMESTAMP(3)${intervals}`; + return temporal === "date" ? `DATE(${now})` : `(${now})`; + } + } +} diff --git a/server/typescript/packages/codegen-ts/test/agent-docs-surface.test.ts b/server/typescript/packages/codegen-ts/test/agent-docs-surface.test.ts index 9f72f7a13..343169f08 100644 --- a/server/typescript/packages/codegen-ts/test/agent-docs-surface.test.ts +++ b/server/typescript/packages/codegen-ts/test/agent-docs-surface.test.ts @@ -614,6 +614,44 @@ describe("agent/schema.md — the claims it makes about the model", () => { expect(page).not.toContain("one-to-one"); }); + // FR-044 no-churn: the Views intro names a report only when a report-backed view is on the + // page. A projection-only model must keep the wording it had before reports existed, or + // `meta verify --docs` flags drift after an upgrade. + test("the Views intro of a model with no report view is the pre-report sentence, byte for byte", async () => { + const page = (await emit(await load(SHAPES), { schema: fleetSchema() })).get("agent/schema.md") ?? ""; + expect(page).toContain( + "## Views\n\n" + + "A view is generated from its projection's `origin.*` children — it is derived, never hand-written. " + + "Editing the view SQL directly is drift the tool cannot see.\n\n", + ); + expect(page).not.toContain("report"); + }); + + test("the Views intro names a report's dimensions and measures once a report view is on the page", () => { + const fleet = fleetSchema(); + const base = { + ...fleet, + views: [{ name: "v_store_totals" }, { name: "v_owner_summary" }], + provenance: new Map([ + ...fleet.provenance, + ["public.v_store_totals", "acme::shop::StoreTotals"], + ]), + }; + const withReport = renderAgentSchemaPage(base, { + declaredBy: new Map(), viewLineage: new Map(), relationships: [], enums: [], + reportViews: new Set(["public.v_store_totals"]), + }); + expect(withReport).toContain( + "A view is generated from its projection's `origin.*` children or its report's dimensions and measures — " + + "it is derived, never hand-written. ", + ); + const without = renderAgentSchemaPage(base, { + declaredBy: new Map(), viewLineage: new Map(), relationships: [], enums: [], reportViews: new Set(), + }); + expect(without).toContain("A view is generated from its projection's `origin.*` children — it is derived, never hand-written. "); + expect(without).not.toContain("report's"); + }); + test("a view carries its `origin.*` lineage, which is what makes it a derived artifact", async () => { const page = (await emit(await load(SHAPES), { schema: fleetSchema() })).get("agent/schema.md") ?? ""; expect(page).toContain("## Views"); diff --git a/server/typescript/packages/codegen-ts/test/projection/build-projection-views.test.ts b/server/typescript/packages/codegen-ts/test/projection/build-projection-views.test.ts index 32eb524b5..bca095cfd 100644 --- a/server/typescript/packages/codegen-ts/test/projection/build-projection-views.test.ts +++ b/server/typescript/packages/codegen-ts/test/projection/build-projection-views.test.ts @@ -9,8 +9,10 @@ // `origin.passthrough` renames, and the bodyOnly emit shape consumed by migrate-ts. import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; -import { buildProjectionViews } from "../../src/projection/build-projection-views.js"; +import { buildProjectionViews, buildReportViews } from "../../src/projection/build-projection-views.js"; async function load(children: unknown[]) { const json = JSON.stringify({ "metadata.root": { package: "acme", children } }); @@ -299,3 +301,232 @@ describe("buildProjectionViews — #208 @sql / @unmanaged DDL-ownership escape v ); }); }); + +// FR-044 Plan 2 Task 6 — a view-backed object.report lowers through buildReportViews, which +// buildProjectionViews calls after its two existing loops (contract Table A). +describe("buildReportViews — view-backed reports (FR-044 Plan 2, Table A)", () => { + type Json = Record; + const REPORTING = resolve(import.meta.dir, "../../../../../../fixtures/codegen-noop/reporting"); + + function shop(variant: "with" | "without", mutate?: (children: Json[]) => void): Json { + const model = JSON.parse(readFileSync(resolve(REPORTING, variant, "meta.shop.json"), "utf8")) as { + "metadata.root": { children: Json[] }; + }; + mutate?.(model["metadata.root"].children); + return model; + } + async function loadModel(model: Json) { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(model))]); + expect(errors).toEqual([]); + return root; + } + const PG = { dialect: "postgres", columnNamingStrategy: "literal" } as const; + + test("a view-backed report yields one ExpectedView named by its source", async () => { + const root = await loadModel(shop("with")); + const views = buildReportViews(root, PG); + expect(views.map((v) => v.name)).toEqual(["v_store_totals"]); + const v = views[0]!; + expect(v.fqn).toBe("acme::shop::StoreTotals"); + expect(v.dependsOn).toEqual(["purchases"]); + expect(v.columns).toBeUndefined(); + expect("columns" in v).toBe(false); + expect(v.sql).toContain("FROM \"purchases\" p"); + expect(v.sql).toContain('COUNT(p."id") FILTER (WHERE p."status" = \'active\') AS "purchases"'); + expect(v.sql).not.toContain("CREATE VIEW"); + }); + + test("a sourceless report yields nothing", async () => { + const root = await loadModel(shop("with")); + const names = buildProjectionViews(root, PG).map((v) => v.name); + expect(names).toContain("v_store_totals"); + expect(names.some((n) => /engagement|daily/i.test(n))).toBe(false); + expect(names).toHaveLength(1); + }); + + test("an @unmanaged report source yields nothing", async () => { + const root = await loadModel( + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + (kids[0]!["source.rdb"] as Json)["@unmanaged"] = true; + }), + ); + expect(buildReportViews(root, PG)).toEqual([]); + }); + + test("a non-view report source kind yields nothing", async () => { + const root = await loadModel( + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + (kids[0]!["source.rdb"] as Json)["@kind"] = "materializedView"; + }), + ); + expect(buildReportViews(root, PG)).toEqual([]); + }); + + test("an @sql report source keeps the author's body and depends on the @from table", async () => { + const body = "SELECT COUNT(*) AS purchases FROM purchases"; + const root = await loadModel( + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + (kids[0]!["source.rdb"] as Json)["@sql"] = body; + }), + ); + const views = buildReportViews(root, PG); + expect(views).toHaveLength(1); + expect(views[0]!.sql).toBe(body); + expect(views[0]!.dependsOn).toEqual(["purchases"]); + expect(views[0]!.fqn).toBe("acme::shop::StoreTotals"); + expect("columns" in views[0]!).toBe(false); + }); + + test("the projection loop does not see an @sql report either: one view, not two", async () => { + // An @sql report is the shape most like a projection (a read-only source with a body). + // Were the `!isReport` filter on the projection loop dropped, it would be emitted twice. + const root = await loadModel( + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + (kids[0]!["source.rdb"] as Json)["@sql"] = "SELECT COUNT(*) AS purchases FROM purchases"; + }), + ); + expect(buildProjectionViews(root, PG)).toHaveLength(1); + }); + + // Source selection (final fix wave A4): Table A is decided by the SAME source the view is + // named by and the runtime reads: the own read-only source with role primary, else the + // first own read-only source. A replica declared first must not decide it. + const replicaFirst = (replica: Json) => + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + (kids[0]!["source.rdb"] as Json)["@role"] = "primary"; + kids.unshift({ "source.rdb": { "@kind": "view", "@table": "v_store_totals_replica", "@role": "replica", ...replica } }); + }); + + test("a replica declared before the primary, @unmanaged: the primary view is still created", async () => { + const root = await loadModel(replicaFirst({ "@unmanaged": true })); + const views = buildReportViews(root, PG); + expect(views.map((v) => v.name)).toEqual(["v_store_totals"]); + expect(views[0]!.sql).toContain('FROM "purchases" p'); + }); + + test("a replica declared before the primary, with @sql: the primary is derived and named, not the replica's body", async () => { + const root = await loadModel(replicaFirst({ "@sql": "SELECT 1 AS purchases" })); + const views = buildReportViews(root, PG); + expect(views.map((v) => v.name)).toEqual(["v_store_totals"]); + expect(views[0]!.sql).not.toContain("SELECT 1 AS purchases"); + expect(views[0]!.sql).toContain('FROM "purchases" p'); + }); + + test("a primary that is @unmanaged is skipped even when a managed replica is declared first", async () => { + const root = await loadModel( + shop("with", (children) => { + const r = children.find((c) => (c["object.report"] as Json | undefined)?.name === "StoreTotals"); + const kids = (r!["object.report"] as { children: Json[] }).children; + Object.assign(kids[0]!["source.rdb"] as Json, { "@role": "primary", "@unmanaged": true }); + kids.unshift({ "source.rdb": { "@kind": "view", "@table": "v_store_totals_replica", "@role": "replica" } }); + }), + ); + expect(buildReportViews(root, PG)).toEqual([]); + }); + + test("an @sql report @from a TPH subtype is not refused: the author owns the body", async () => { + const tph = (source: Json): Json => ({ + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "User", + "@discriminator": "kind", + children: [ + { "source.rdb": { "@table": "users" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "kind" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "measure.aggregate": { name: "users", "@agg": "count", "@of": "User.id" } }, + ], + }, + }, + { "object.entity": { name: "Admin", extends: "User", "@discriminatorValue": "ADMIN", children: [] } }, + { "object.report": { name: "Admins", "@from": "Admin", "@measures": ["users"], children: [{ "source.rdb": source }] } }, + ], + }, + }); + const body = "SELECT COUNT(id) AS users FROM users WHERE kind = 'ADMIN'"; + const sql = buildReportViews(await loadModel(tph({ "@kind": "view", "@view": "v_admins", "@sql": body })), PG); + expect(sql.map((v) => [v.name, v.sql, v.dependsOn])).toEqual([["v_admins", body, ["users"]]]); + // An @unmanaged view is likewise the author's: nothing is created and nothing is refused. + expect(buildReportViews(await loadModel(tph({ "@kind": "view", "@view": "v_admins", "@unmanaged": true })), PG)).toEqual([]); + // The derived path is the one that refuses. + await expect( + loadModel(tph({ "@kind": "view", "@view": "v_admins" })).then((root) => buildReportViews(root, PG)), + ).rejects.toThrow(/report 'Admins': @from 'Admin' is a TPH subtype/); + }); + + test("the projection loop does not see a report", async () => { + const root = await loadModel(shop("with")); + const all = buildProjectionViews(root, PG); + expect(all).toHaveLength(1); + expect(all).toEqual(buildReportViews(root, PG)); + }); + + test("a model with no report returns exactly what it returned before; reports come last", async () => { + const withReport = (children: Json[]) => { + children.push( + { "object.projection": { name: "ProgramLite", children: [ + { "source.rdb": { "@kind": "view", "@view": "v_program_lite" } }, + { "field.long": { name: "id", extends: "Program.id" } }, + { "identity.primary": { extends: "Program.id" } }, + ] } }, + ); + }; + // Re-use the with-model's reports but drop them again: a projection-only twin. + const twin = (keepReports: boolean) => + shop("with", (children) => { + withReport(children); + if (!keepReports) { + for (let i = children.length - 1; i >= 0; i--) if ("object.report" in children[i]!) children.splice(i, 1); + } + }); + const without = buildProjectionViews(await loadModel(twin(false)), PG); + const withR = buildProjectionViews(await loadModel(twin(true)), PG); + expect(without.map((v) => v.name)).toEqual(["v_program_lite"]); + expect(withR.map((v) => v.name)).toEqual(["v_program_lite", "v_store_totals"]); + expect(withR.slice(0, without.length)).toEqual(without); + // And the report-free codegen-noop model is still empty. + expect(buildProjectionViews(await loadModel(shop("without")), PG)).toEqual([]); + }); + + test("mysql is accepted by buildReportViews and emits backticks", async () => { + const root = await loadModel(shop("with")); + const views = buildReportViews(root, { dialect: "mysql", columnNamingStrategy: "literal" }); + expect(views).toHaveLength(1); + expect(views[0]!.sql).toContain("FROM `purchases` p"); + expect(views[0]!.sql).toContain("CASE WHEN p.`status` = 'active' THEN p.`id` END"); + }); + + test("d1 returns exactly the sqlite bodies", async () => { + const root = await loadModel(shop("with")); + const d1 = buildReportViews(root, { dialect: "d1", columnNamingStrategy: "literal" }); + const sqlite = buildReportViews(root, { dialect: "sqlite", columnNamingStrategy: "literal" }); + expect(d1).toEqual(sqlite); + expect(d1[0]!.sql).toContain('COUNT(CASE WHEN p."status" = \'active\' THEN p."id" END)'); + }); + + test("a view-backed report over a @from with no table throws, naming the report and the entity", async () => { + const root = await loadModel( + shop("with", (children) => { + const purchase = children.find((c) => (c["object.entity"] as Json | undefined)?.name === "Purchase"); + const kids = (purchase!["object.entity"] as { children: Json[] }).children; + kids.splice(kids.findIndex((k) => "source.rdb" in k), 1); + }), + ); + expect(() => buildReportViews(root, PG)).toThrow(/report 'StoreTotals'.*'Purchase'/); + expect(() => buildProjectionViews(root, PG)).toThrow(/StoreTotals/); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts b/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts new file mode 100644 index 000000000..15ec7690d --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts @@ -0,0 +1,618 @@ +// FR-044 Plan 2, Task 4 — an object.report plus its shape lowers to a dialect-neutral +// ReportViewSpec (contract Table F). These tests assert on the SPEC, never on SQL: the +// renderer is a separate task. + +import { describe, test, expect } from "bun:test"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { + MetaDataLoader, + InMemoryStringSource, + OBJECT_REPORT_ATTR_FILTER, + reportReadModel, + type MetaObject, + type MetaRoot, +} from "@metaobjectsdev/metadata"; +import { extractReportSpec, temporalOf } from "../../src/projection/extract-report-spec.js"; +import { isRelativeNow } from "../../src/projection/report-spec.js"; +import type { ReportViewSpec } from "../../src/projection/report-spec.js"; + +type Json = Record; + +const FIXTURE = resolve(import.meta.dir, "../../../../../../fixtures/codegen-noop/reporting/with/meta.shop.json"); + +/** The shared reporting fixture, with extra members and reports appended for these cases. */ +function shopModel(mutate?: (children: Json[]) => void): Json { + const model = JSON.parse(readFileSync(FIXTURE, "utf8")) as { "metadata.root": { children: Json[] } }; + const children = model["metadata.root"].children; + const entity = (name: string): Json[] => { + const hit = children.find((c) => (c["object.entity"] as Json | undefined)?.name === name); + return (hit!["object.entity"] as { children: Json[] }).children; + }; + const purchase = entity("Purchase"); + purchase.push( + { "field.timestamp": { name: "createdAt", "@column": "created_ts" } }, + { "field.double": { name: "score" } }, + { "dimension.time": { name: "createdAt", "@of": "Purchase.createdAt", "@grains": ["month"] } }, + { "measure.aggregate": { name: "scoreTotal", "@agg": "sum", "@of": "Purchase.score" } }, + ); + children.push( + { "object.report": { name: "ProgramTitles", "@from": "Purchase", "@dimensions": ["programTitle"], "@measures": ["purchases"] } }, + { "object.report": { name: "ProgramOnly", "@from": "Purchase", "@dimensions": ["program"], "@measures": ["purchases"] } }, + { + "object.report": { + name: "SegmentAndFilter", "@from": "Purchase", "@measures": ["purchases"], + "@segment": "active", "@filter": { status: "refunded" }, + }, + }, + { + "object.report": { + name: "Range", "@from": "Purchase", "@measures": ["purchases"], + "@filter": { amountCents: { gte: 100, lte: 500 } }, + }, + }, + { "object.report": { name: "Scores", "@from": "Purchase", "@measures": ["scoreTotal"] } }, + { + "object.report": { + name: "CreatedByMonth", "@from": "Purchase", "@dimensions": ["createdAt:month"], "@measures": ["purchases"], + }, + }, + ); + mutate?.(children); + return model; +} + +async function load(model: Json): Promise { + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(model))]); + expect(errors).toEqual([]); + return root; +} + +async function spec( + name: string, + opts: { model?: Json; strategy?: "snake_case" | "literal" } = {}, +): Promise { + const root = await load(opts.model ?? shopModel()); + const report = root.findObject(name); + if (report === undefined) throw new Error(`no report ${name}`); + return extractReportSpec(report, root, { columnNamingStrategy: opts.strategy ?? "snake_case" }); +} + +describe("extractReportSpec", () => { + test("a no-dimension report has no joins and only aggregate columns", async () => { + const s = await spec("StoreTotals"); + expect(s.joinTree.joins).toEqual([]); + expect(s.columns.map((c) => c.kind)).toEqual(["aggregate", "aggregate", "aggregate"]); + expect(s.viewName).toBe("v_store_totals"); + expect(s.joinTree.baseEntity).toBe("acme::shop::Purchase"); + expect(s.joinTree.baseAlias).toBe("p"); + expect(s.where).toBeUndefined(); + }); + + test("a @via dimension adds one join with the #209 join type", async () => { + const s = await spec("ProgramTitles"); + expect(s.joinTree.joins).toHaveLength(1); + const join = s.joinTree.joins[0]!; + expect(join.relationship).toBe("program"); + // programId carries no @required in this fixture, so the hop is LEFT. + expect(join.joinType).toBe("left"); + const dim = s.columns[0]!; + expect(dim.kind).toBe("dimension"); + if (dim.kind !== "dimension") throw new Error("unreachable"); + expect(dim.fieldName).toBe("programTitle"); + expect(dim.dbColAlias).toBe("program_title"); + // The ref lands on the JOIN alias, not the base alias. + expect(dim.ref).toBe(`${join.alias}.title`); + expect(join.alias).not.toBe(s.joinTree.baseAlias); + }); + + test("a required belongs-to FK joins INNER", async () => { + const model = shopModel((children) => { + const purchase = children.find((c) => (c["object.entity"] as Json | undefined)?.name === "Purchase")!; + const fields = (purchase["object.entity"] as { children: Json[] }).children; + const programId = fields.find((f) => (f["field.long"] as Json | undefined)?.name === "programId")!; + (programId["field.long"] as Json)["@required"] = true; + }); + const s = await spec("ProgramTitles", { model }); + expect(s.joinTree.joins[0]!.joinType).toBe("inner"); + }); + + test("an unlisted @via dimension adds no join", async () => { + const s = await spec("ProgramOnly"); + expect(s.joinTree.joins).toEqual([]); + const dim = s.columns[0]!; + if (dim.kind !== "dimension") throw new Error("expected a dimension"); + expect(dim.ref).toBe("p.program_id"); + }); + + test("report @segment and @filter combine with AND, segment first", async () => { + const s = await spec("SegmentAndFilter"); + expect(s.where).toEqual({ + kind: "and", + clauses: [ + { kind: "cmp", ref: "p.status", op: "eq", value: "active" }, + { kind: "cmp", ref: "p.status", op: "eq", value: "refunded" }, + ], + }); + }); + + test("a relative value becomes a RelativeNow with the field's temporal kind", async () => { + const s = await spec("DailyRevenue"); + expect(s.where).toEqual({ + kind: "cmp", + ref: "p.purchased_at", + op: "gte", + value: { kind: "relativeNow", duration: "-P90D", temporal: "instant" }, + }); + expect(isRelativeNow((s.where as { value: unknown }).value)).toBe(true); + expect(isRelativeNow({ now: "-P90D" })).toBe(false); + expect(isRelativeNow(null)).toBe(false); + }); + + test("two operators on one field both survive", async () => { + const s = await spec("Range"); + expect(s.where).toEqual({ + kind: "and", + clauses: [ + { kind: "cmp", ref: "p.amount_cents", op: "gte", value: 100 }, + { kind: "cmp", ref: "p.amount_cents", op: "lte", value: 500 }, + ], + }); + }); + + test("a measure's @segment and @filter become its aggregate filter", async () => { + const totals = await spec("StoreTotals"); + const purchases = totals.columns[0]!; + if (purchases.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(purchases.aggregate.agg).toBe("count"); + expect(purchases.aggregate.filter).toEqual({ kind: "cmp", ref: "p.status", op: "eq", value: "active" }); + // A measure with neither a segment nor a filter has no aggregate filter. + const engagement = await spec("ProgramEngagement"); + const lastActivity = engagement.columns.find((c) => c.fieldName === "lastActivityAt")!; + if (lastActivity.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(lastActivity.aggregate.filter).toBeUndefined(); + }); + + test("a measure's own @filter is read without a segment", async () => { + const model = shopModel((children) => { + const purchase = children.find((c) => (c["object.entity"] as Json | undefined)?.name === "Purchase")!; + (purchase["object.entity"] as { children: Json[] }).children.push( + { "measure.aggregate": { name: "bigOnes", "@agg": "count", "@of": "Purchase.id", "@segment": "active", "@filter": { refunded: false } } }, + ); + children.push({ "object.report": { name: "Big", "@from": "Purchase", "@measures": ["bigOnes"] } }); + }); + const s = await spec("Big", { model }); + const c = s.columns[0]!; + if (c.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(c.aggregate.filter).toEqual({ + kind: "and", + clauses: [ + { kind: "cmp", ref: "p.status", op: "eq", value: "active" }, + { kind: "cmp", ref: "p.refunded", op: "eq", value: false }, + ], + }); + }); + + test("a tuple @of yields several refs and distinct: true", async () => { + const s = await spec("ProgramEngagement"); + const days = s.columns.find((c) => c.fieldName === "daysEngaged")!; + if (days.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(days.aggregate.refs).toEqual(["w.program_id", "w.week_number", "w.day_number"]); + expect(days.aggregate.distinct).toBe(true); + expect(days.aggregate.cast).toBeUndefined(); + // The report's @segment scopes the whole report, not each measure. + expect(s.where).toEqual({ kind: "cmp", ref: "w.event_type", op: "eq", value: "exercise_complete" }); + }); + + test("a ratio carries both operand aggregates in full", async () => { + const s = await spec("ProgramEngagement"); + const ratio = s.columns.find((c) => c.fieldName === "avgDaysPerStarter")!; + if (ratio.kind !== "ratio") throw new Error("expected a ratio"); + expect(ratio.numerator.refs).toHaveLength(3); + expect(ratio.numerator.distinct).toBe(true); + expect(ratio.denominator.refs).toEqual(["w.customer_email"]); + expect(ratio.denominator.distinct).toBe(true); + expect(ratio.dbColAlias).toBe("avg_days_per_starter"); + }); + + test("an operand that is not listed in @measures is still resolved", async () => { + const model = shopModel((children) => { + children.push({ + "object.report": { name: "RatioOnly", "@from": "WorkoutEvent", "@measures": ["avgDaysPerStarter"] }, + }); + }); + const s = await spec("RatioOnly", { model }); + expect(s.columns.map((c) => c.kind)).toEqual(["ratio"]); + const ratio = s.columns[0]!; + if (ratio.kind !== "ratio") throw new Error("expected a ratio"); + // Neither operand is listed, and both arrive as full aggregates over the base alias. + const listed = await spec("ProgramEngagement"); + const same = listed.columns.find((c) => c.fieldName === "avgDaysPerStarter")!; + if (same.kind !== "ratio") throw new Error("expected a ratio"); + expect(ratio.numerator).toEqual(same.numerator); + expect(ratio.denominator).toEqual(same.denominator); + expect(ratio.numerator.refs).toHaveLength(3); + expect(ratio.denominator.refs).toEqual(["w.customer_email"]); + }); + + test("an integral sum is cast to bigint; a currency sum too; a floating sum to double", async () => { + const revenue = (await spec("DailyRevenue")).columns.find((c) => c.fieldName === "revenue")!; + if (revenue.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(revenue.aggregate.cast).toBe("bigint"); + const scores = (await spec("Scores")).columns[0]!; + if (scores.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(scores.aggregate.cast).toBe("double"); + // count and max never cast. + const lastActivity = (await spec("ProgramEngagement")).columns.find((c) => c.fieldName === "lastActivityAt")!; + if (lastActivity.kind !== "aggregate") throw new Error("expected an aggregate"); + expect(lastActivity.aggregate.cast).toBeUndefined(); + }); + + test("a time dimension carries its grain and the field's temporal kind", async () => { + const s = await spec("DailyRevenue"); + const dim = s.columns[0]!; + if (dim.kind !== "timeDimension") throw new Error("expected a timeDimension"); + expect(dim).toEqual({ + kind: "timeDimension", + fieldName: "purchasedAtDay", + dbColAlias: "purchased_at_day", + ref: "p.purchased_at", + grain: "day", + temporal: "instant", + }); + }); + + test("refuses a @from with no writable source", async () => { + const model = shopModel((children) => { + children.push( + { + "object.entity": { + name: "Ghost", + children: [ + { "field.long": { name: "id" } }, + { "identity.primary": { name: "id", "@fields": ["id"] } }, + { "measure.aggregate": { name: "ghosts", "@agg": "count", "@of": "Ghost.id" } }, + ], + }, + }, + { "object.report": { name: "GhostReport", "@from": "Ghost", "@measures": ["ghosts"] } }, + ); + }); + const root = await load(model); + expect(() => extractReportSpec(root.findObject("GhostReport")!, root, { columnNamingStrategy: "snake_case" })) + .toThrow(/report 'GhostReport'.*'Ghost'.*no table/); + }); + + test("the output alias is the naming strategy applied to the derived name, not an inherited @column", async () => { + const s = await spec("CreatedByMonth"); + const dim = s.columns[0]!; + if (dim.kind !== "timeDimension") throw new Error("expected a timeDimension"); + expect(dim.dbColAlias).toBe("created_at_month"); + expect(dim.ref).toBe("p.created_ts"); + const literal = await spec("CreatedByMonth", { strategy: "literal" }); + const lit = literal.columns[0]!; + expect(lit.dbColAlias).toBe("createdAtMonth"); + expect((lit as { ref: string }).ref).toBe("p.created_ts"); + }); +}); + +describe("temporalOf", () => { + test("date, instant and naive", async () => { + const root = await load(shopModel((children) => { + const purchase = children.find((c) => (c["object.entity"] as Json | undefined)?.name === "Purchase")!; + (purchase["object.entity"] as { children: Json[] }).children.push( + { "field.timestamp": { name: "localAt", "@localTime": true } }, + ); + })); + const purchase = root.findObject("Purchase")!; + const field = (n: string) => purchase.fields().find((f) => f.name === n)!; + expect(temporalOf(field("purchasedOn"))).toBe("date"); + expect(temporalOf(field("purchasedAt"))).toBe("instant"); + expect(temporalOf(field("localAt"))).toBe("naive"); + }); +}); + +// --------------------------------------------------------------------------- +// Final fix wave (FR-044): refusals and reference resolution. +// --------------------------------------------------------------------------- + +const CTX = { columnNamingStrategy: "snake_case" } as const; + +const file = (pkg: string, children: Json[]): InMemoryStringSource => + new InMemoryStringSource(JSON.stringify({ "metadata.root": { package: pkg, children } })); + +async function loadFiles(files: InMemoryStringSource[]): Promise { + const { root, errors } = await new MetaDataLoader().load(files); + expect(errors).toEqual([]); + return root; +} + +const view = (name: string): Json => ({ "source.rdb": { "@kind": "view", "@view": name } }); + +/** The node with one attr replaced, WITHOUT the loader (the loaded tree is frozen and the + * loader refuses these values): what a caller building a tree in code can hand in. */ +function withAttr(node: MetaObject, name: string, value: unknown): MetaObject { + const stub = Object.create(node) as MetaObject; + Object.defineProperty(stub, "attr", { value: (n: string) => (n === name ? value : node.attr(n)) }); + return stub; +} + +describe("extractReportSpec: a @from in a TPH hierarchy", () => { + /** `User` owns the table and the discriminator; `Admin` is a TPH subtype sharing it. */ + const tph = (reports: Json[]): InMemoryStringSource => + file("acme", [ + { + "object.entity": { + name: "User", + "@discriminator": "kind", + children: [ + { "source.rdb": { "@table": "users" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "kind" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "measure.aggregate": { name: "users", "@agg": "count", "@of": "User.id" } }, + ], + }, + }, + { "object.entity": { name: "Admin", extends: "User", "@discriminatorValue": "ADMIN", children: [] } }, + ...reports, + ]); + const report = (name: string, from: string, extra: Json = {}, source: Json = view("v_r")): Json => ({ + "object.report": { name, "@from": from, "@measures": ["users"], ...extra, children: [source] }, + }); + + test("a derived report @from a TPH subtype is refused, naming the report and the subtype", async () => { + const root = await loadFiles([tph([report("Admins", "Admin")])]); + expect(() => extractReportSpec(root.findObject("Admins")!, root, CTX)).toThrow( + "report 'Admins': @from 'Admin' is a TPH subtype: it shares the table of 'User' with every other " + + "subtype, so a view derived from it would aggregate all of their rows. Declare the report " + + "@from 'User' with an @filter on the discriminator field 'kind' (for example { \"kind\": \"ADMIN\" }).", + ); + }); + + test("a report @from the TPH base is accepted and reads the shared table unscoped", async () => { + const root = await loadFiles([tph([report("Users", "User")])]); + const s = extractReportSpec(root.findObject("Users")!, root, CTX); + expect(s.joinTree.baseEntity).toBe("acme::User"); + expect(s.where).toBeUndefined(); + }); + + test("a base report with an @filter on the discriminator lowers to a WHERE on that column", async () => { + const root = await loadFiles([tph([report("Admins", "User", { "@filter": { kind: "ADMIN" } })])]); + const s = extractReportSpec(root.findObject("Admins")!, root, CTX); + expect(s.where).toEqual({ kind: "cmp", ref: "u.kind", op: "eq", value: "ADMIN" }); + }); + + test("the runtime read model does not look at @from's TPH position: it serves whatever relation the source names", async () => { + // An @sql or @unmanaged view over a subtype is the author's body, so it is not refused + // (build-projection-views.test.ts); the read model's shape is the same Table B either way. + const root = await loadFiles([tph([report("Admins", "Admin")])]); + const model = reportReadModel(root.findObject("Admins")!, root); + expect(model?.fields().map((f) => f.name)).toEqual(["users"]); + }); +}); + +describe("extractReportSpec: references resolve as the loader resolves them", () => { + /** `a::Base` (abstract) declares members with BARE references; `b::Ev extends a::Base`. */ + const shared = (): InMemoryStringSource => + file("a", [ + { + "object.entity": { + name: "Owner", + children: [ + { "source.rdb": { "@table": "owners" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "label" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + name: "Base", + abstract: true, + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "kind" } }, + { "field.long": { name: "ownerId" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "identity.reference": { name: "ownerRef", "@fields": ["ownerId"], "@references": "a::Owner" } }, + { "relationship.association": { name: "owner", "@objectRef": "a::Owner", "@cardinality": "one" } }, + { "dimension.attribute": { name: "kind", "@of": "Base.kind" } }, + { "dimension.attribute": { name: "ownerLabel", "@of": "Owner.label", "@via": "Base.owner" } }, + { "measure.aggregate": { name: "events", "@agg": "count", "@of": "Base.id" } }, + ], + }, + }, + ]); + const consumer = (extra: Json[] = []): InMemoryStringSource => + file("b", [ + ...extra, + { "object.entity": { name: "Ev", extends: "a::Base", children: [{ "source.rdb": { "@table": "evs" } }] } }, + { + "object.report": { + name: "R", + "@from": "Ev", + "@dimensions": ["kind", "ownerLabel"], + "@measures": ["Ev.events"], + children: [view("v_r")], + }, + }, + ]); + + const refs = (s: ReportViewSpec): unknown[] => + s.columns.map((c) => (c.kind === "aggregate" ? c.aggregate.refs : (c as { ref: string }).ref)); + + test("a bare @of / @via inherited from another package resolves in the declaring entity's package", async () => { + const root = await loadFiles([shared(), consumer()]); + const s = extractReportSpec(root.findObject("R")!, root, CTX); + expect(s.joinTree.baseEntity).toBe("b::Ev"); + expect(s.joinTree.joins.map((j) => [j.relationship, j.targetEntity])).toEqual([["owner", "a::Owner"]]); + expect(refs(s)).toEqual(["e.kind", `${s.joinTree.joins[0]!.alias}.label`, ["e.id"]]); + }); + + test("same-named decoys in the report's package do not capture the references", async () => { + const decoys: Json[] = [ + { "object.entity": { name: "Base", children: [{ "field.int": { name: "id" } }, { "field.int": { name: "kind" } }] } }, + { + "object.entity": { + name: "Owner", + children: [{ "source.rdb": { "@table": "decoy_owners" } }, { "field.int": { name: "label", "@column": "decoy" } }], + }, + }, + ]; + const root = await loadFiles([shared(), consumer(decoys)]); + const s = extractReportSpec(root.findObject("R")!, root, CTX); + expect(s.joinTree.joins.map((j) => j.targetEntity)).toEqual(["a::Owner"]); + expect(refs(s)).toEqual(["e.kind", `${s.joinTree.joins[0]!.alias}.label`, ["e.id"]]); + }); +}); + +describe("extractReportSpec: the @via walk starts at @from whatever its package looks like", () => { + /** `F` with a to-one reference to `P`, and a report grouping by P's title through it. */ + const facts = (pkg: string, pTable = "ps"): InMemoryStringSource => + file(pkg, [ + { + "object.entity": { + name: "P", + children: [ + { "source.rdb": { "@table": pTable } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + name: "F", + children: [ + { "source.rdb": { "@table": `${pTable}_facts` } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "pId" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "identity.reference": { name: "pRef", "@fields": ["pId"], "@references": "P" } }, + { "dimension.attribute": { name: "pTitle", "@of": "P.title", "@via": "F.pRef" } }, + { "measure.aggregate": { name: "facts", "@agg": "count", "@of": "F.id" } }, + ], + }, + }, + { + "object.report": { + name: "ByP", + "@from": "F", + "@dimensions": ["pTitle"], + "@measures": ["facts"], + children: [view("v_by_p")], + }, + }, + ]); + const joinsOf = (root: MetaRoot, reportKey: string): unknown[] => { + const report = root.objects().find((o) => o.resolutionKey() === reportKey)!; + const s = extractReportSpec(report, root, CTX); + return [s.joinTree.baseEntity, ...s.joinTree.joins.map((j) => [j.relationship, j.targetEntity])]; + }; + + test.each(["acme", "com.acme", "com.acme::shop.v2"])( + "a @via dimension lowers to one join in package '%s' (a package name may contain a dot)", + async (pkg) => { + const root = await loadFiles([facts(pkg)]); + expect(joinsOf(root, `${pkg}::ByP`)).toEqual([`${pkg}::F`, ["pRef", `${pkg}::P`]]); + }, + ); + + test("the walk starts at THIS report's @from when another package has an entity of the same short name", async () => { + // Loaded in both orders: neither `F` may win by load order. + for (const files of [[facts("one", "ps1"), facts("two", "ps2")], [facts("two", "ps2"), facts("one", "ps1")]]) { + const root = await loadFiles(files); + expect(joinsOf(root, "one::ByP")).toEqual(["one::F", ["pRef", "one::P"]]); + expect(joinsOf(root, "two::ByP")).toEqual(["two::F", ["pRef", "two::P"]]); + } + }); +}); + +describe("extractReportSpec: refusals that name what is wrong", () => { + test("refuses an abstract @from, naming the report and the entity", async () => { + const root = await loadFiles([ + file("acme", [ + { + "object.entity": { + name: "Shape", + abstract: true, + children: [ + { "source.rdb": { "@table": "shapes" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "measure.aggregate": { name: "shapes", "@agg": "count", "@of": "Shape.id" } }, + ], + }, + }, + { "object.report": { name: "Shapes", "@from": "Shape", "@measures": ["shapes"], children: [view("v_shapes")] } }, + ]), + ]); + expect(() => extractReportSpec(root.findObject("Shapes")!, root, CTX)).toThrow( + /report 'Shapes'.*'Shape'.*no table \(it is abstract/, + ); + }); + + test("a @via hop with no foreign key in the model is refused, naming the hop and what it needs", async () => { + // The loader accepts a to-one relationship with no identity.reference behind it. + const model = shopModel((children) => { + const purchase = children.find((c) => (c["object.entity"] as Json | undefined)?.name === "Purchase")!; + const kids = (purchase["object.entity"] as { children: Json[] }).children; + kids.splice(kids.findIndex((k) => "identity.reference" in k), 1); + }); + const root = await load(model); + expect(() => extractReportSpec(root.findObject("ProgramTitles")!, root, CTX)).toThrow( + "report 'ProgramTitles': dimension 'programTitle' @via 'Purchase.program' cannot be joined at hop 'program' " + + "on 'acme::shop::Purchase': the model declares no foreign key for it. A view joins a hop through an " + + "identity.reference; declare one on 'Purchase' whose @references is 'Program' (with the foreign-key " + + "field in @fields).", + ); + }); + + test("a @via whose later hop does not resolve is refused, not joined part-way", async () => { + const root = await load(shopModel()); + const titles = root.findObject("ProgramTitles")!; + const from = root.findObject("Purchase")!; + const dim = from.children().find((c) => c.name === "programTitle")!; + // Past the loader (rule D2 refuses an unknown hop): a two-hop path whose second hop is nothing. + const via = Object.create(dim) as typeof dim & { via(): string }; + Object.defineProperty(via, "via", { value: () => "Purchase.program.nowhere" }); + const fromStub = Object.create(from) as MetaObject; + Object.defineProperty(fromStub, "children", { value: () => from.children().map((c) => (c === dim ? via : c)) }); + const rootStub = Object.create(root) as MetaRoot; + const fromKey = from.resolutionKey(); + Object.defineProperty(rootStub, "children", { + value: () => root.children().map((c) => (c.resolutionKey() === fromKey ? fromStub : c)), + }); + Object.defineProperty(rootStub, "objects", { + value: () => root.objects().map((c) => (c.resolutionKey() === fromKey ? fromStub : c)), + }); + expect(() => extractReportSpec(titles, rootStub, CTX)).toThrow( + /report 'ProgramTitles': dimension 'programTitle' @via 'Purchase.program.nowhere' cannot be joined at hop 'nowhere' on 'acme::shop::Program': it names no relationship or identity.reference/, + ); + }); + + test("a filter field that is not a field of @from is refused by name", async () => { + const root = await load(shopModel()); + // Past the loader (rule S1 refuses an unknown filter field). + const r = withAttr(root.findObject("StoreTotals")!, OBJECT_REPORT_ATTR_FILTER, { nope: 1 }); + expect(() => extractReportSpec(r, root, CTX)).toThrow( + `report 'StoreTotals' @filter: filter field "nope" is not a field of 'Purchase'.`, + ); + }); + + test("an empty `in` list is refused at lowering, naming the report and the field", async () => { + const model = shopModel((children) => { + children.push({ + "object.report": { name: "NoStatuses", "@from": "Purchase", "@measures": ["purchases"], "@filter": { status: { in: [] } } }, + }); + }); + const root = await load(model); + expect(() => extractReportSpec(root.findObject("NoStatuses")!, root, CTX)).toThrow( + `report 'NoStatuses' @filter: the 'in' list on "status" is empty, which no row can match and no ` + + `database accepts as SQL (IN ()). List at least one value, or remove the clause.`, + ); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts b/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts new file mode 100644 index 000000000..d5ace4927 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts @@ -0,0 +1,485 @@ +// FR-044 Plan 2 Task 5 — emitReportViewDdl. Specs are hand-built (not extracted): this +// file pins the TEXT of contract Tables C, E, F and the golden bodies of Table G. +import { describe, test, expect } from "bun:test"; +import type { TimeGrain } from "@metaobjectsdev/metadata"; +import { emitReportViewDdl, type ReportEmitOptions } from "../../src/projection/report-ddl-emit.js"; +import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; +import type { + ReportAggregate, + ReportColumn, + ReportViewSpec, +} from "../../src/projection/report-spec.js"; +import type { JoinNode, ViewFilterClause, ViewSpec } from "../../src/projection/view-spec.js"; + +const pg = (baseTableName: string, joinTables: Record = {}): ReportEmitOptions => + ({ dialect: "postgres", baseTableName, joinTables, bodyOnly: true }); +const sqlite = (baseTableName: string, joinTables: Record = {}): ReportEmitOptions => + ({ dialect: "sqlite", baseTableName, joinTables, bodyOnly: true }); +const mysql = (baseTableName: string, joinTables: Record = {}): ReportEmitOptions => + ({ dialect: "mysql", baseTableName, joinTables, bodyOnly: true }); + +const lines = (...l: string[]): string => l.join("\n"); + +const cmp = (ref: string, op: string, value: unknown): ViewFilterClause => ({ kind: "cmp", ref, op, value }); +const count = (ref: string, extra: Partial = {}): ReportAggregate => + ({ agg: "count", distinct: false, refs: [ref], ...extra }); +const col = (fieldName: string, aggregate: ReportAggregate): ReportColumn => + ({ kind: "aggregate", fieldName, dbColAlias: fieldName, aggregate }); + +function baseSpec( + alias: string, + entity: string, + columns: readonly ReportColumn[], + rest: { joins?: readonly JoinNode[]; where?: ViewFilterClause; viewName?: string } = {}, +): ReportViewSpec { + return { + viewName: rest.viewName ?? "v_test", + joinTree: { baseEntity: entity, baseAlias: alias, joins: rest.joins ?? [] }, + columns, + ...(rest.where !== undefined ? { where: rest.where } : {}), + }; +} + +// ── v_program_minutes (Table G, all three dialects) ──────────────────────────────── + +const longWeek = cmp("w.durationMinutes", "gte", 60); +const programJoin: JoinNode = { + relationship: "program", targetEntity: "Program", alias: "p", cardinality: "one", + fkColumn: "programId", pkColumn: "id", referenceHolder: "source", joinType: "inner", children: [], +}; +const programMinutes: ReportViewSpec = baseSpec( + "w", + "Week", + [ + { kind: "dimension", fieldName: "program", dbColAlias: "program", ref: "w.programId" }, + { kind: "dimension", fieldName: "programTitle", dbColAlias: "programTitle", ref: "p.title" }, + col("weeks", count("w.id")), + col("longWeeks", count("w.id", { filter: longWeek })), + col("labels", count("w.label", { distinct: true })), + col("slots", count("w.programId", { distinct: true, refs: ["w.programId", "w.durationMinutes"] })), + col("totalMinutes", { agg: "sum", distinct: false, refs: ["w.durationMinutes"], cast: "bigint" }), + col("avgMinutes", { agg: "avg", distinct: false, refs: ["w.durationMinutes"] }), + col("minMinutes", { agg: "min", distinct: false, refs: ["w.durationMinutes"] }), + col("maxMinutes", { agg: "max", distinct: false, refs: ["w.durationMinutes"] }), + { + kind: "ratio", fieldName: "longShare", dbColAlias: "longShare", + numerator: count("w.id", { filter: longWeek }), denominator: count("w.id"), + }, + ], + { joins: [programJoin], viewName: "v_program_minutes" }, +); + +describe("emitReportViewDdl — Table G v_program_minutes", () => { + const tables = { Program: "programs" }; + + test("postgres", () => { + expect(emitReportViewDdl(programMinutes, pg("weeks", tables))).toBe(lines( + ` SELECT`, + ` w."programId" AS "program",`, + ` p."title" AS "programTitle",`, + ` COUNT(w."id") AS "weeks",`, + ` COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS "longWeeks",`, + ` COUNT(DISTINCT w."label") AS "labels",`, + ` COUNT(DISTINCT (w."programId", w."durationMinutes")) FILTER (WHERE w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL) AS "slots",`, + ` CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes",`, + ` AVG(w."durationMinutes") AS "avgMinutes",`, + ` MIN(w."durationMinutes") AS "minMinutes",`, + ` MAX(w."durationMinutes") AS "maxMinutes",`, + ` CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare"`, + ` FROM "weeks" w`, + ` INNER JOIN "programs" p ON p."id" = w."programId"`, + ` GROUP BY w."programId", p."title"`, + )); + }); + + test("sqlite", () => { + expect(emitReportViewDdl(programMinutes, sqlite("weeks", tables))).toBe(lines( + ` SELECT`, + ` w."programId" AS "program",`, + ` p."title" AS "programTitle",`, + ` COUNT(w."id") AS "weeks",`, + ` COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS "longWeeks",`, + ` COUNT(DISTINCT w."label") AS "labels",`, + ` COUNT(DISTINCT CASE WHEN w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL THEN json_array(w."programId", w."durationMinutes") END) AS "slots",`, + ` SUM(w."durationMinutes") AS "totalMinutes",`, + ` AVG(w."durationMinutes") AS "avgMinutes",`, + ` MIN(w."durationMinutes") AS "minMinutes",`, + ` MAX(w."durationMinutes") AS "maxMinutes",`, + ` CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "longShare"`, + ` FROM "weeks" w`, + ` INNER JOIN "programs" p ON p."id" = w."programId"`, + ` GROUP BY w."programId", p."title"`, + )); + }); + + test("mysql", () => { + expect(emitReportViewDdl(programMinutes, mysql("weeks", tables))).toBe(lines( + " SELECT", + " w.`programId` AS `program`,", + " p.`title` AS `programTitle`,", + " COUNT(w.`id`) AS `weeks`,", + " COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) AS `longWeeks`,", + " COUNT(DISTINCT w.`label`) AS `labels`,", + " COUNT(DISTINCT w.`programId`, w.`durationMinutes`) AS `slots`,", + " CAST(SUM(w.`durationMinutes`) AS SIGNED) AS `totalMinutes`,", + " AVG(w.`durationMinutes`) AS `avgMinutes`,", + " MIN(w.`durationMinutes`) AS `minMinutes`,", + " MAX(w.`durationMinutes`) AS `maxMinutes`,", + " COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0) AS `longShare`", + " FROM `weeks` w", + " INNER JOIN `programs` p ON p.`id` = w.`programId`", + " GROUP BY w.`programId`, p.`title`", + )); + }); +}); + +// created_ts is a NAIVE timestamp (@localTime) in the canonical model; recordedAt is an instant. +// ── The other five canonical views, Postgres (Table G) ───────────────────────────── + +const totalsSpec: ReportViewSpec = baseSpec( + "w", + "Week", + [ + col("weeks", count("w.id")), + col("totalMinutes", { agg: "sum", distinct: false, refs: ["w.durationMinutes"], cast: "bigint" }), + { + kind: "ratio", fieldName: "longShare", dbColAlias: "longShare", + numerator: count("w.id", { filter: longWeek }), denominator: count("w.id"), + }, + ], + { viewName: "v_fitness_totals" }, +); + +const byMonthSpec: ReportViewSpec = baseSpec( + "p", + "Program", + [ + { + kind: "timeDimension", fieldName: "createdAtMonth", dbColAlias: "createdAtMonth", + ref: "p.created_ts", grain: "month" as TimeGrain, temporal: "naive", + }, + { kind: "dimension", fieldName: "status", dbColAlias: "status", ref: "p.status" }, + col("programs", count("p.id")), + col("listValue", { + agg: "sum", distinct: false, refs: ["p.priceCents"], cast: "bigint", + filter: cmp("p.status", "eq", "PUBLISHED"), + }), + ], + { viewName: "v_programs_by_month" }, +); + +const byWeekSpec: ReportViewSpec = baseSpec( + "p", + "Program", + [ + { + kind: "timeDimension", fieldName: "createdAtWeek", dbColAlias: "createdAtWeek", + ref: "p.created_ts", grain: "week" as TimeGrain, temporal: "naive", + }, + col("programs", count("p.id")), + ], + { where: cmp("p.status", "eq", "PUBLISHED"), viewName: "v_programs_by_week" }, +); + +const recentSpec: ReportViewSpec = baseSpec( + "p", + "Program", + [col("programs", count("p.id"))], + { + where: cmp("p.created_ts", "gte", { kind: "relativeNow", duration: "-P30D", temporal: "naive" }), + viewName: "v_recent_programs", + }, +); + +const assetActivitySpec: ReportViewSpec = baseSpec( + "a", + "Asset", + [ + { + kind: "timeDimension", fieldName: "recordedAtHour", dbColAlias: "recordedAtHour", + ref: "a.recordedAt", grain: "hour" as TimeGrain, temporal: "instant", + }, + { + kind: "timeDimension", fieldName: "asOfDateWeek", dbColAlias: "asOfDateWeek", + ref: "a.asOfDate", grain: "week" as TimeGrain, temporal: "date", + }, + col("assets", count("a.id")), + ], + { viewName: "v_asset_activity" }, +); + +describe("emitReportViewDdl — Table G, remaining Postgres bodies", () => { + test("v_fitness_totals", () => { + expect(emitReportViewDdl(totalsSpec, pg("weeks"))).toBe(lines( + ` SELECT`, + ` COUNT(w."id") AS "weeks",`, + ` CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes",`, + ` CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare"`, + ` FROM "weeks" w`, + )); + }); + + test("v_programs_by_month", () => { + expect(emitReportViewDdl(byMonthSpec, pg("programs"))).toBe(lines( + ` SELECT`, + ` CAST(date_trunc('month', p."created_ts") AS DATE) AS "createdAtMonth",`, + ` p."status" AS "status",`, + ` COUNT(p."id") AS "programs",`, + ` CAST(SUM(p."priceCents") FILTER (WHERE p."status" = 'PUBLISHED') AS BIGINT) AS "listValue"`, + ` FROM "programs" p`, + ` GROUP BY CAST(date_trunc('month', p."created_ts") AS DATE), p."status"`, + )); + }); + + test("v_programs_by_week", () => { + expect(emitReportViewDdl(byWeekSpec, pg("programs"))).toBe(lines( + ` SELECT`, + ` CAST(date_trunc('week', p."created_ts") AS DATE) AS "createdAtWeek",`, + ` COUNT(p."id") AS "programs"`, + ` FROM "programs" p`, + ` WHERE p."status" = 'PUBLISHED'`, + ` GROUP BY CAST(date_trunc('week', p."created_ts") AS DATE)`, + )); + }); + + test("v_recent_programs", () => { + expect(emitReportViewDdl(recentSpec, pg("programs"))).toBe(lines( + ` SELECT`, + ` COUNT(p."id") AS "programs"`, + ` FROM "programs" p`, + ` WHERE p."created_ts" >= ((now() AT TIME ZONE 'UTC') - INTERVAL 'P30D')`, + )); + }); + + test("v_asset_activity", () => { + expect(emitReportViewDdl(assetActivitySpec, pg("assets"))).toBe(lines( + ` SELECT`, + ` date_trunc('hour', a."recordedAt", 'UTC') AS "recordedAtHour",`, + ` CAST(date_trunc('week', CAST(a."asOfDate" AS TIMESTAMP)) AS DATE) AS "asOfDateWeek",`, + ` COUNT(a."id") AS "assets"`, + ` FROM "assets" a`, + ` GROUP BY date_trunc('hour', a."recordedAt", 'UTC'), CAST(date_trunc('week', CAST(a."asOfDate" AS TIMESTAMP)) AS DATE)`, + )); + }); +}); + +// ── One test per rule ────────────────────────────────────────────────────────────── + +function specWithMeasure(name: string): ReportViewSpec { + return baseSpec("p", "Program", [col(name, count("p.id"))]); +} + +describe("emitReportViewDdl — rules", () => { + test("quotes a keyword-named measure and dimension", () => { + expect(emitReportViewDdl(specWithMeasure("order"), pg("programs"))).toContain(`AS "order"`); + expect(emitReportViewDdl(specWithMeasure("order"), sqlite("programs"))).toContain(`AS "order"`); + expect(emitReportViewDdl(specWithMeasure("order"), mysql("programs"))).toContain("AS `order`"); + const dim = baseSpec("p", "Program", [ + { kind: "dimension", fieldName: "group", dbColAlias: "group", ref: "p.user" }, + col("rank", count("p.id")), + ]); + const sql = emitReportViewDdl(dim, pg("programs")); + expect(sql).toContain(`p."user" AS "group"`); + expect(sql).toContain(`AS "rank"`); + expect(sql).toContain(`GROUP BY p."user"`); + }); + + test("quotes an embedded quote character in an identifier", () => { + expect(emitReportViewDdl(specWithMeasure(`a"b`), pg("programs"))).toContain(`AS "a""b"`); + expect(emitReportViewDdl(specWithMeasure("a`b"), mysql("programs"))).toContain("AS `a``b`"); + }); + + test("no dimensions: no GROUP BY", () => { + expect(emitReportViewDdl(totalsSpec, pg("weeks"))).not.toContain("GROUP BY"); + }); + + test("a time dimension groups by the same expression it selects", () => { + const sql = emitReportViewDdl(byMonthSpec, pg("programs")); + const expr = `CAST(date_trunc('month', p."created_ts") AS DATE)`; + expect(sql).toContain(`${expr} AS "createdAtMonth"`); + expect(sql).toContain(`GROUP BY ${expr}, p."status"`); + }); + + test("a relative value renders Table E inside WHERE", () => { + expect(emitReportViewDdl(recentSpec, pg("programs"))) + .toContain(`WHERE p."created_ts" >= ((now() AT TIME ZONE 'UTC') - INTERVAL 'P30D')`); + expect(emitReportViewDdl(recentSpec, sqlite("programs"))) + .toContain(`WHERE p."created_ts" >= strftime('%Y-%m-%dT%H:%M:%f', 'now', '-30 days')`); + expect(emitReportViewDdl(recentSpec, mysql("programs"))) + .toContain("WHERE p.`created_ts` >= (UTC_TIMESTAMP(3) - INTERVAL 30 DAY)"); + }); + + test("a relative value inside an `in` list renders Table E per element", () => { + const spec = baseSpec("p", "Program", [col("programs", count("p.id"))], { + where: cmp("p.created_ts", "in", [{ kind: "relativeNow", duration: "-P1D", temporal: "naive" }, "x"]), + }); + expect(emitReportViewDdl(spec, pg("programs"))) + .toContain(`WHERE p."created_ts" IN (((now() AT TIME ZONE 'UTC') - INTERVAL 'P1D'), 'x')`); + }); + + const tupleCondSpec: ReportViewSpec = baseSpec("w", "Week", [ + col("slots", count("w.programId", { + distinct: true, + refs: ["w.programId", "w.durationMinutes"], + filter: longWeek, + })), + ]); + + test("a tuple distinct count with a condition, per dialect", () => { + expect(emitReportViewDdl(tupleCondSpec, pg("weeks"))).toContain( + `COUNT(DISTINCT (w."programId", w."durationMinutes")) FILTER (WHERE w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL AND w."durationMinutes" >= 60)`); + expect(emitReportViewDdl(tupleCondSpec, sqlite("weeks"))).toContain( + `COUNT(DISTINCT CASE WHEN w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL AND w."durationMinutes" >= 60 THEN json_array(w."programId", w."durationMinutes") END)`); + expect(emitReportViewDdl(tupleCondSpec, mysql("weeks"))).toContain( + "COUNT(DISTINCT CASE WHEN w.`durationMinutes` >= 60 THEN w.`programId` END, w.`durationMinutes`)"); + }); + + test("a distinct count with a condition", () => { + const spec = baseSpec("w", "Week", [col("labels", count("w.label", { distinct: true, filter: longWeek }))]); + expect(emitReportViewDdl(spec, pg("weeks"))) + .toContain(`COUNT(DISTINCT w."label") FILTER (WHERE w."durationMinutes" >= 60)`); + expect(emitReportViewDdl(spec, sqlite("weeks"))) + .toContain(`COUNT(DISTINCT CASE WHEN w."durationMinutes" >= 60 THEN w."label" END)`); + expect(emitReportViewDdl(spec, mysql("weeks"))) + .toContain("COUNT(DISTINCT CASE WHEN w.`durationMinutes` >= 60 THEN w.`label` END)"); + }); + + test("Table C casts: bigint and double sums, per dialect", () => { + const big = baseSpec("w", "Week", [ + col("t", { agg: "sum", distinct: false, refs: ["w.m"], cast: "bigint" }), + ]); + const dbl = baseSpec("w", "Week", [ + col("t", { agg: "sum", distinct: false, refs: ["w.m"], cast: "double" }), + ]); + const dec = baseSpec("w", "Week", [col("t", { agg: "sum", distinct: false, refs: ["w.m"] })]); + expect(emitReportViewDdl(big, pg("weeks"))).toContain(`CAST(SUM(w."m") AS BIGINT)`); + expect(emitReportViewDdl(big, sqlite("weeks"))).toContain(` SUM(w."m") AS "t"`); + expect(emitReportViewDdl(big, mysql("weeks"))).toContain("CAST(SUM(w.`m`) AS SIGNED)"); + expect(emitReportViewDdl(dbl, pg("weeks"))).toContain(`CAST(SUM(w."m") AS DOUBLE PRECISION)`); + expect(emitReportViewDdl(dbl, sqlite("weeks"))).toContain(` SUM(w."m") AS "t"`); + expect(emitReportViewDdl(dbl, mysql("weeks"))).toContain(" SUM(w.`m`) AS `t`"); + expect(emitReportViewDdl(dec, pg("weeks"))).toContain(` SUM(w."m") AS "t"`); + }); + + test("a conditional cast sum: the cast wraps the FILTER, per dialect", () => { + const spec = baseSpec("w", "Week", [ + col("t", { agg: "sum", distinct: false, refs: ["w.m"], cast: "bigint", filter: longWeek }), + ]); + expect(emitReportViewDdl(spec, pg("weeks"))) + .toContain(`CAST(SUM(w."m") FILTER (WHERE w."durationMinutes" >= 60) AS BIGINT)`); + expect(emitReportViewDdl(spec, sqlite("weeks"))) + .toContain(`SUM(CASE WHEN w."durationMinutes" >= 60 THEN w."m" END)`); + expect(emitReportViewDdl(spec, mysql("weeks"))) + .toContain("CAST(SUM(CASE WHEN w.`durationMinutes` >= 60 THEN w.`m` END) AS SIGNED)"); + }); + + test("a ratio repeats each operand's FULL expression, operand cast nested inside the ratio cast", () => { + const sumOp: ReportAggregate = { + agg: "sum", distinct: false, refs: ["w.m"], cast: "bigint", filter: longWeek, + }; + const sumAll: ReportAggregate = { agg: "sum", distinct: false, refs: ["w.m"], cast: "bigint" }; + const spec = baseSpec("w", "Week", [ + { kind: "ratio", fieldName: "r", dbColAlias: "r", numerator: sumOp, denominator: sumAll }, + ]); + expect(emitReportViewDdl(spec, pg("weeks"))).toContain( + `CAST(CAST(SUM(w."m") FILTER (WHERE w."durationMinutes" >= 60) AS BIGINT) AS NUMERIC) / NULLIF(CAST(SUM(w."m") AS BIGINT), 0) AS "r"`); + expect(emitReportViewDdl(spec, sqlite("weeks"))).toContain( + `CAST(SUM(CASE WHEN w."durationMinutes" >= 60 THEN w."m" END) AS REAL) / NULLIF(SUM(w."m"), 0) AS "r"`); + expect(emitReportViewDdl(spec, mysql("weeks"))).toContain( + "CAST(SUM(CASE WHEN w.`durationMinutes` >= 60 THEN w.`m` END) AS SIGNED) / NULLIF(CAST(SUM(w.`m`) AS SIGNED), 0) AS `r`"); + }); + + test("a MySQL string literal doubles backslashes and quotes; the others double only quotes", () => { + const spec = baseSpec("p", "Program", [col("programs", count("p.id"))], { + where: cmp("p.label", "eq", "a\\b'c"), + }); + expect(emitReportViewDdl(spec, mysql("programs"))).toContain("WHERE p.`label` = 'a\\\\b''c'"); + expect(emitReportViewDdl(spec, pg("programs"))).toContain(`WHERE p."label" = 'a\\b''c'`); + expect(emitReportViewDdl(spec, sqlite("programs"))).toContain(`WHERE p."label" = 'a\\b''c'`); + }); + + test("boolean literals: TRUE/FALSE on Postgres and MySQL, 1/0 on SQLite", () => { + const spec = baseSpec("p", "Program", [col("programs", count("p.id"))], { + where: cmp("p.active", "eq", true), + }); + expect(emitReportViewDdl(spec, pg("programs"))).toContain(`p."active" = TRUE`); + expect(emitReportViewDdl(spec, mysql("programs"))).toContain("p.`active` = TRUE"); + expect(emitReportViewDdl(spec, sqlite("programs"))).toContain(`p."active" = 1`); + }); + + test("operators: ne, like, in, isNull, and/or grouping", () => { + const where: ViewFilterClause = { + kind: "and", + clauses: [ + cmp("p.a", "ne", 1), + cmp("p.b", "like", "x%"), + cmp("p.c", "in", ["u", "v"]), + cmp("p.d", "isNull", true), + cmp("p.e", "isNull", false), + { kind: "or", clauses: [cmp("p.f", "lt", 2), cmp("p.g", "lte", 3)] }, + ], + }; + const spec = baseSpec("p", "Program", [col("programs", count("p.id"))], { where }); + expect(emitReportViewDdl(spec, pg("programs"))).toContain( + `WHERE (p."a" <> 1 AND p."b" LIKE 'x%' AND p."c" IN ('u', 'v') AND p."d" IS NULL AND p."e" IS NOT NULL AND (p."f" < 2 OR p."g" <= 3))`); + }); + + test("an exprCmp or unknown operator is refused", () => { + const bad = baseSpec("p", "Program", [col("programs", count("p.id"))], { + where: { kind: "exprCmp", expr: { kind: "lit", value: 1 }, op: "eq", value: 1 }, + }); + expect(() => emitReportViewDdl(bad, pg("programs"))).toThrow(/exprCmp/); + const badOp = baseSpec("p", "Program", [col("programs", count("p.id"))], { + where: cmp("p.a", "regex", "x"), + }); + expect(() => emitReportViewDdl(badOp, pg("programs"))).toThrow(/regex/); + }); + + test("a join to an entity with no registered table is refused", () => { + expect(() => emitReportViewDdl(programMinutes, pg("weeks"))).toThrow(/Program/); + }); + + test("a nested join and a LEFT OUTER join render with their ON clauses", () => { + const nested: JoinNode = { + relationship: "program", targetEntity: "Program", alias: "p", cardinality: "one", + fkColumn: "programId", pkColumn: "id", referenceHolder: "source", joinType: "left", + children: [{ + relationship: "weeks", targetEntity: "Week", alias: "w0", cardinality: "many", + fkColumn: "programId", pkColumn: "id", referenceHolder: "target", joinType: "left", children: [], + }], + }; + const spec = baseSpec("w", "Week", [col("n", count("w.id"))], { joins: [nested] }); + const sql = emitReportViewDdl(spec, mysql("weeks", { Program: "programs", Week: "weeks" })); + expect(sql).toContain(lines( + " LEFT OUTER JOIN `programs` p ON p.`id` = w.`programId`", + " LEFT OUTER JOIN `weeks` w0 ON w0.`programId` = p.`id`", + )); + }); + + test("bodyOnly false wraps in CREATE VIEW with a quoted name and a trailing semicolon", () => { + const spec = baseSpec("p", "Program", [col("programs", count("p.id"))], { viewName: "v_x" }); + const full = emitReportViewDdl(spec, { dialect: "postgres", baseTableName: "programs", joinTables: {} }); + expect(full).toBe(`CREATE VIEW "v_x" AS\n SELECT\n COUNT(p."id") AS "programs"\n FROM "programs" p;`); + const my = emitReportViewDdl(spec, { dialect: "mysql", baseTableName: "programs", joinTables: {}, bodyOnly: false }); + expect(my.startsWith("CREATE VIEW `v_x` AS\n")).toBe(true); + expect(my.endsWith(";")).toBe(true); + }); + + test("the projection emitter's output is untouched", () => { + const spec: ViewSpec = { + viewName: "v_program_summary", + joinTree: { baseEntity: "Program", baseAlias: "p", joins: [] }, + selectSpec: { + columns: [ + { kind: "passthrough", fieldName: "id", dbColAlias: "id", sourceAlias: "p", sourceColumn: "id" }, + { kind: "passthrough", fieldName: "title", dbColAlias: "title", sourceAlias: "p", sourceColumn: "title" }, + ], + }, + groupBy: [], + }; + // quoteIfNeeded leaves these lower-case identifiers bare; the report emitter would quote them. + expect(emitViewDdl(spec, { dialect: "postgres", baseTableName: "programs", joinTables: {} })).toBe( + "CREATE VIEW v_program_summary AS\n SELECT\n p.id AS id,\n p.title AS title\n FROM programs p;", + ); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/time-sql.test.ts b/server/typescript/packages/codegen-ts/test/projection/time-sql.test.ts new file mode 100644 index 000000000..37301f76a --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/time-sql.test.ts @@ -0,0 +1,145 @@ +import { describe, expect, test } from "bun:test"; +import { parseIsoDuration, relativeNowSql, truncateToGrain } from "../../src/projection/time-sql.js"; + +const X = `p."created_ts"`; + +describe("truncateToGrain (Table D)", () => { + test.each([ + ["hour", "instant", `date_trunc('hour', ${X}, 'UTC')`], + ["day", "instant", `CAST(date_trunc('day', ${X} AT TIME ZONE 'UTC') AS DATE)`], + ["week", "instant", `CAST(date_trunc('week', ${X} AT TIME ZONE 'UTC') AS DATE)`], + ["month", "instant", `CAST(date_trunc('month', ${X} AT TIME ZONE 'UTC') AS DATE)`], + ["quarter", "instant", `CAST(date_trunc('quarter', ${X} AT TIME ZONE 'UTC') AS DATE)`], + ["year", "instant", `CAST(date_trunc('year', ${X} AT TIME ZONE 'UTC') AS DATE)`], + ["hour", "naive", `date_trunc('hour', ${X})`], + ["day", "naive", `CAST(date_trunc('day', ${X}) AS DATE)`], + ["week", "naive", `CAST(date_trunc('week', ${X}) AS DATE)`], + ["month", "naive", `CAST(date_trunc('month', ${X}) AS DATE)`], + ["quarter", "naive", `CAST(date_trunc('quarter', ${X}) AS DATE)`], + ["year", "naive", `CAST(date_trunc('year', ${X}) AS DATE)`], + ["day", "date", X], + ["week", "date", `CAST(date_trunc('week', CAST(${X} AS TIMESTAMP)) AS DATE)`], + ["month", "date", `CAST(date_trunc('month', CAST(${X} AS TIMESTAMP)) AS DATE)`], + ["quarter", "date", `CAST(date_trunc('quarter', CAST(${X} AS TIMESTAMP)) AS DATE)`], + ["year", "date", `CAST(date_trunc('year', CAST(${X} AS TIMESTAMP)) AS DATE)`], + ] as const)("postgres %s on %s", (grain, temporal, sql) => { + expect(truncateToGrain(X, grain, temporal, "postgres")).toBe(sql); + }); + + const SQLITE_QUARTER = `date(${X}, 'start of month', '-' || ((CAST(strftime('%m', ${X}) AS INTEGER) - 1) % 3) || ' months')`; + test.each([ + ["hour", "instant", `strftime('%Y-%m-%dT%H:00:00.000Z', ${X})`], + ["hour", "naive", `strftime('%Y-%m-%dT%H:00:00', ${X})`], + ...(["instant", "naive", "date"] as const).flatMap((t) => [ + ["day", t, `date(${X})`], + ["week", t, `date(${X}, 'weekday 0', '-6 days')`], + ["month", t, `date(${X}, 'start of month')`], + ["quarter", t, SQLITE_QUARTER], + ["year", t, `date(${X}, 'start of year')`], + ] as const), + ] as const)("sqlite %s on %s", (grain, temporal, sql) => { + expect(truncateToGrain(X, grain, temporal, "sqlite")).toBe(sql); + }); + + const MYSQL = [ + ["hour", `CAST(DATE_FORMAT(${X}, '%Y-%m-%d %H:00:00') AS DATETIME(3))`], + ["day", `DATE(${X})`], + ["week", `DATE(DATE_SUB(${X}, INTERVAL WEEKDAY(${X}) DAY))`], + ["month", `DATE(DATE_FORMAT(${X}, '%Y-%m-01'))`], + ["quarter", `MAKEDATE(YEAR(${X}), 1) + INTERVAL (QUARTER(${X}) - 1) QUARTER`], + ["year", `MAKEDATE(YEAR(${X}), 1)`], + ] as const; + test.each(MYSQL)("mysql %s", (grain, sql) => { + expect(truncateToGrain(X, grain, "naive", "mysql")).toBe(sql); + }); + test.each(MYSQL.filter(([g]) => g !== "hour"))("mysql %s is the same for every column kind", (grain, sql) => { + for (const temporal of ["instant", "date"] as const) { + expect(truncateToGrain(X, grain, temporal, "mysql")).toBe(sql); + } + }); + test("mysql hour is the same on an instant", () => { + expect(truncateToGrain(X, "hour", "instant", "mysql")).toBe(MYSQL[0][1]); + }); + + test.each(["postgres", "sqlite", "mysql"] as const)("hour on a date column throws (%s)", (dialect) => { + expect(() => truncateToGrain(X, "hour", "date", dialect)).toThrow(/hour/); + }); + + test.each(["postgres", "sqlite", "mysql"] as const)( + "a grain outside the closed set is refused, never interpolated into SQL (%s)", + (dialect) => { + // A programmatic caller skips the loader; the Postgres arm writes the grain into + // date_trunc('', ...), so an unchecked string would reach the DDL. + const hostile = "day', now()); DROP TABLE t; --" as unknown as "day"; + expect(() => truncateToGrain(X, hostile, "instant", dialect)).toThrow( + `time-sql: "day', now()); DROP TABLE t; --" is not a time grain (hour, day, week, month, quarter, year).`, + ); + }, + ); +}); + +describe("relativeNowSql (Table E)", () => { + test.each([ + ["-P7D", "instant", `(now() - INTERVAL 'P7D')`], + ["+P7D", "instant", `(now() + INTERVAL 'P7D')`], + ["P7D", "instant", `(now() + INTERVAL 'P7D')`], + ["-P1Y2M3WT4H5M6S", "instant", `(now() - INTERVAL 'P1Y2M3WT4H5M6S')`], + ["PT12H", "naive", `((now() AT TIME ZONE 'UTC') + INTERVAL 'PT12H')`], + ["-P2W", "naive", `((now() AT TIME ZONE 'UTC') - INTERVAL 'P2W')`], + ["-P1Y", "date", `CAST(((now() AT TIME ZONE 'UTC') - INTERVAL 'P1Y') AS DATE)`], + ["+P1D", "date", `CAST(((now() AT TIME ZONE 'UTC') + INTERVAL 'P1D') AS DATE)`], + ] as const)("postgres %s on %s", (duration, temporal, sql) => { + expect(relativeNowSql(duration, temporal, "postgres")).toBe(sql); + }); + + test.each([ + ["-P7D", "instant", `strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-7 days')`], + ["P1D", "instant", `strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '+1 days')`], + ["-P1Y2M3WT4H", "naive", `strftime('%Y-%m-%dT%H:%M:%f', 'now', '-1 years', '-2 months', '-21 days', '-4 hours')`], + ["PT5M6S", "naive", `strftime('%Y-%m-%dT%H:%M:%f', 'now', '+5 minutes', '+6 seconds')`], + ["-P1W2D", "instant", `strftime('%Y-%m-%dT%H:%M:%fZ', 'now', '-7 days', '-2 days')`], + ["+P1D", "date", `date('now', '+1 days')`], + ["-P1Y2M3DT4H5M6S", "date", `date('now', '-1 years', '-2 months', '-3 days', '-4 hours', '-5 minutes', '-6 seconds')`], + ["P0D", "date", `date('now')`], + ] as const)("sqlite %s on %s", (duration, temporal, sql) => { + expect(relativeNowSql(duration, temporal, "sqlite")).toBe(sql); + }); + + test.each([ + ["-P30D", "instant", `(UTC_TIMESTAMP(3) - INTERVAL 30 DAY)`], + ["-P1Y2W", "naive", `(UTC_TIMESTAMP(3) - INTERVAL 1 YEAR - INTERVAL 2 WEEK)`], + ["-P1Y2M3W4DT5H6M7S", "instant", + `(UTC_TIMESTAMP(3) - INTERVAL 1 YEAR - INTERVAL 2 MONTH - INTERVAL 3 WEEK - INTERVAL 4 DAY - INTERVAL 5 HOUR - INTERVAL 6 MINUTE - INTERVAL 7 SECOND)`], + ["PT12H", "naive", `(UTC_TIMESTAMP(3) + INTERVAL 12 HOUR)`], + ["-P7D", "date", `DATE(UTC_TIMESTAMP(3) - INTERVAL 7 DAY)`], + ["P0D", "instant", `(UTC_TIMESTAMP(3))`], + ] as const)("mysql %s on %s", (duration, temporal, sql) => { + expect(relativeNowSql(duration, temporal, "mysql")).toBe(sql); + }); + + test("a malformed duration throws, naming it", () => { + expect(() => parseIsoDuration("P")).toThrow(/P/); + expect(() => relativeNowSql("7 days", "instant", "postgres")).toThrow(/7 days/); + }); +}); + +describe("parseIsoDuration", () => { + test("splits every component and the sign", () => { + expect(parseIsoDuration("-P1Y2M3W4DT5H6M7S")).toEqual({ + sign: "-", years: 1, months: 2, weeks: 3, days: 4, hours: 5, minutes: 6, seconds: 7, + magnitude: "P1Y2M3W4DT5H6M7S", + }); + }); + test("an unsigned duration is positive and absent components are zero", () => { + expect(parseIsoDuration("PT90M")).toEqual({ + sign: "+", years: 0, months: 0, weeks: 0, days: 0, hours: 0, minutes: 90, seconds: 0, + magnitude: "PT90M", + }); + expect(parseIsoDuration("+P7D").sign).toBe("+"); + expect(parseIsoDuration("+P7D").magnitude).toBe("P7D"); + }); + test("a month is not a minute", () => { + const p = parseIsoDuration("P3MT4M"); + expect([p.months, p.minutes]).toEqual([3, 4]); + }); +}); diff --git a/server/typescript/packages/docs-site/src/coverage.ts b/server/typescript/packages/docs-site/src/coverage.ts index 84f5968d1..060b7f7c8 100644 --- a/server/typescript/packages/docs-site/src/coverage.ts +++ b/server/typescript/packages/docs-site/src/coverage.ts @@ -4,8 +4,8 @@ import { } from "@metaobjectsdev/metadata"; /** FR-044 reporting vocabulary — all four types: `dimension.*`, `measure.*`, `segment.*` - * and `object.report`. Inert in every generator until the report lowering lands (FR-044 - * Plan 2/3), so the site renders none of it BY DESIGN. + * and `object.report`. A view-backed report is lowered to a view (FR-044 Plan 2), but no + * site page renders any of this vocabulary until Plan 3, so the site shows none of it BY DESIGN. * * The audit reports these as DEFERRED, not as "not rendered by any page" and not by * silently dropping them: the gap stays visible on the returned report (`deferred`, one diff --git a/server/typescript/packages/integration-tests/package.json b/server/typescript/packages/integration-tests/package.json index fb465d8fa..8fb9f00d5 100644 --- a/server/typescript/packages/integration-tests/package.json +++ b/server/typescript/packages/integration-tests/package.json @@ -7,6 +7,7 @@ "scripts": { "test": "bun test", "gen:schema": "bun run src/gen-canonical-schema.ts", + "gen:report-shapes": "bun run src/gen-report-shapes.ts", "oracle": "bun run src/oracle-cli.ts" }, "dependencies": { diff --git a/server/typescript/packages/integration-tests/src/gen-report-shapes.ts b/server/typescript/packages/integration-tests/src/gen-report-shapes.ts new file mode 100644 index 000000000..53bd9fb10 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/gen-report-shapes.ts @@ -0,0 +1,103 @@ +// gen-report-shapes.ts — (re)generate the committed report-shapes artifact. +// +// Run: `bun run gen:report-shapes` (from this package). Pure metadata → JSON, no DB. +// Writes fixtures/persistence-conformance/report-shapes.json: the derived +// fields (contract Table B) of every object.report in the canonical model, in declaration +// order. TypeScript produces it; the C#, Java, Kotlin and Python ports each derive the +// same shapes from the same model and byte-match this file in a container-free unit test, +// so the derivation cannot drift between ports. +// +// Format (a contract, every port serialises the same bytes): reports in declaration order; +// keys in the order below; two-space indent; a trailing newline. `typeSource` is +// `.` or null; `view` is the report's +// OWN read-only source's physical name or null. +// +// The artifact sits BESIDE canonical/, not inside it: every port directory-loads +// canonical/ as metadata, and a non-metadata .json there fails the load. + +import { readFileSync, writeFileSync } from "node:fs"; +import { resolve } from "node:path"; + +import { + OBJECT_SUBTYPE_REPORT, + reportReadSource, + reportShape, + type MetaField, + type MetaRoot, +} from "@metaobjectsdev/metadata"; + +import { loadMetadataDir } from "./load-metadata.ts"; +import { CANONICAL_DIR, CORPUS_DIR } from "./paths.ts"; + +/** Absolute path to the committed report-shapes artifact. */ +export const REPORT_SHAPES_PATH = resolve(CORPUS_DIR, "report-shapes.json"); + +interface ShapeFieldJson { + name: string; + role: string; + subType: string; + required: boolean; + typeSource: string | null; +} + +interface ShapeReportJson { + report: string; + from: string; + view: string | null; + fields: ShapeFieldJson[]; +} + +function typeSourceOf(field: MetaField | undefined): string | null { + if (field === undefined) return null; + const owner = field.parent; + if (owner === undefined) throw new Error(`field '${field.name}' has no owning entity.`); + return `${owner.resolutionKey()}.${field.name}`; +} + +/** The artifact's bytes for a loaded model. Deterministic: declaration order, no clock. */ +export function generateReportShapesJson(root: MetaRoot): string { + const reports: ShapeReportJson[] = []; + for (const report of root.objects()) { + if (report.subType !== OBJECT_SUBTYPE_REPORT) continue; + const shape = reportShape(report, root); + // The ONE source-selection rule (own read-only source with role primary, else the first + // own read-only source): the source the lowering names the view by and the runtime reads. + // A sourceless report has no view. + const source = reportReadSource(report); + reports.push({ + report: report.resolutionKey(), + from: shape.from.resolutionKey(), + view: source === undefined ? null : source.physicalName, + fields: shape.fields.map((f) => ({ + name: f.name, + role: f.role, + subType: f.subType, + required: f.required, + typeSource: typeSourceOf(f.typeSource), + })), + }); + } + return `${JSON.stringify({ reports }, null, 2)}\n`; +} + +/** Read the committed report-shapes artifact. */ +export function readReportShapesJson(): string { + return readFileSync(REPORT_SHAPES_PATH, "utf8"); +} + +async function main(): Promise { + const root = await loadMetadataDir(CANONICAL_DIR); + const json = generateReportShapesJson(root); + writeFileSync(REPORT_SHAPES_PATH, json, "utf8"); + /* eslint-disable no-console */ + console.log(`wrote ${REPORT_SHAPES_PATH} (${json.length} bytes)`); + /* eslint-enable no-console */ +} + +if (import.meta.main) { + main().catch((err: unknown) => { + // eslint-disable-next-line no-console + console.error(err); + process.exit(1); + }); +} diff --git a/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts new file mode 100644 index 000000000..39c09f30b --- /dev/null +++ b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts @@ -0,0 +1,90 @@ +// report-shapes-artifact.test.ts — drift-check for the committed report-shapes artifact. +// +// No DB required. Regenerates the derived fields of every canonical object.report from +// canonical/meta.fitness.json and asserts they are byte-identical to the committed +// fixtures/persistence-conformance/report-shapes.json, which every other port +// byte-matches against its own derivation (contract Table B). +// +// Regenerate the artifact with: `bun run gen:report-shapes` (in this package). + +import { describe, expect, test } from "bun:test"; + +import { InMemoryStringSource, MetaDataLoader } from "@metaobjectsdev/metadata"; + +import { + generateReportShapesJson, + readReportShapesJson, + REPORT_SHAPES_PATH, +} from "../src/gen-report-shapes.ts"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; + +describe("canonical report-shapes artifact (report-shapes.json)", () => { + test("committed shapes match what TS derives from metadata (no drift)", async () => { + const root = await loadMetadataDir(CANONICAL_DIR); + const generated = generateReportShapesJson(root); + const committed = readReportShapesJson(); + + if (generated !== committed) { + throw new Error( + `Report-shapes artifact is stale.\n` + + ` ${REPORT_SHAPES_PATH}\n` + + `differs from what TS derives from canonical/meta.fitness.json.\n` + + `Run \`bun run gen:report-shapes\` to regenerate, then commit the result.`, + ); + } + expect(generated).toBe(committed); + }); + + test("the six canonical reports appear in declaration order, with the contract's byte format", () => { + const committed = readReportShapesJson(); + expect(committed.endsWith("}\n")).toBe(true); + expect(committed.startsWith('{\n "reports": [\n {\n "report": "fitness::ProgramMinutes"')).toBe(true); + const parsed = JSON.parse(committed) as { reports: { report: string; view: string | null }[] }; + expect(parsed.reports.map((r) => [r.report, r.view])).toEqual([ + ["fitness::ProgramMinutes", "v_program_minutes"], + ["fitness::FitnessTotals", "v_fitness_totals"], + ["fitness::ProgramsByMonth", "v_programs_by_month"], + ["fitness::ProgramsByWeek", "v_programs_by_week"], + ["fitness::RecentPrograms", "v_recent_programs"], + ["fitness::AssetActivity", "v_asset_activity"], + ]); + }); + + test("`view` is the source the lowering names and the runtime reads: primary, else first", async () => { + // A replica declared BEFORE the primary must not name the report's view. + const model = { + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Sale", + children: [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "measure.aggregate": { name: "sales", "@agg": "count", "@of": "Sale.id" } }, + ], + }, + }, + { + "object.report": { + name: "Totals", + "@from": "Sale", + "@measures": ["sales"], + children: [ + { "source.rdb": { "@kind": "view", "@view": "v_totals_replica", "@role": "replica" } }, + { "source.rdb": { "@kind": "view", "@view": "v_totals", "@role": "primary" } }, + ], + }, + }, + ], + }, + }; + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(model))]); + expect(errors).toEqual([]); + const parsed = JSON.parse(generateReportShapesJson(root)) as { reports: { view: string | null }[] }; + expect(parsed.reports.map((r) => r.view)).toEqual(["v_totals"]); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts b/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts new file mode 100644 index 000000000..caa584c28 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts @@ -0,0 +1,475 @@ +/** + * Report views — FR-044 Plan 2, against a REAL MySQL 8.4. + * + * `meta migrate` does not own a MySQL schema (ADR-0015; `--dialect mysql` is refused), so + * there is no convergence gate here. What ships for MySQL is the SQL: + * `buildReportViews(root, { dialect: "mysql" })` returns each view-backed report's body for + * the adopter to put in their own DDL (docs/recipes/mysql.md, "Reports"). This file proves + * that SQL is ACCEPTED and RIGHT: + * + * - every canonical view is created under the server's DEFAULT `sql_mode`, which includes + * ONLY_FULL_GROUP_BY (the mode the view bodies must satisfy; nothing here relaxes it); + * - the rows are the ones the shared Task 10 persistence scenarios assert, read with raw + * SQL straight off the views, so a wrong value is a LOWERING defect; + * - Review Focus 5: MySQL divides to four fractional digits, so 2/3 is `0.6667` (Postgres + * is `0.66666666666666666667`, SQLite `0.6666666666666666`). Pinned so the documented + * behaviour cannot drift unnoticed; + * - the view's column types follow contract Table B (`CAST(SUM(...) AS SIGNED)` is BIGINT); + * - the Table D grain expressions and the Table E relative-date expressions return the + * documented values. + * + * `DATETIME(3)` holds the UTC wall clock, so the relative-date seeds are written from + * `UTC_TIMESTAMP(3)`. Values are read with `dateStrings` and `bigNumberStrings` so nothing is + * reshaped by the driver: BIGINT, DECIMAL, DATE and DATETIME arrive as the text MySQL sent; + * only INT (the min/max of an int column) is a number. + * + * Requires Docker (or METAOBJECTS_TEST_MYSQL_URL). + */ + +import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import mysql from "mysql2/promise"; +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { MetaDataLoader, InMemoryStringSource, loadDirectory, type MetaRoot } from "@metaobjectsdev/metadata"; +import { startMysql, type MysqlContainerHandle } from "../src/mysql-container.ts"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; + +const REPO_ROOT = resolve(import.meta.dir, "../../../../.."); + +let container: MysqlContainerHandle; +let conn: mysql.Connection; +let canonical: MetaRoot; + +const CANONICAL_VIEWS = [ + "v_program_minutes", "v_fitness_totals", "v_programs_by_month", + "v_programs_by_week", "v_recent_programs", "v_asset_activity", +]; + +/** The adopter's hand-written DDL (MySQL schema is not MetaObjects'). */ +const DDL = [ + `CREATE TABLE programs ( + id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + title VARCHAR(200) NOT NULL, + priceCents BIGINT NOT NULL, + status VARCHAR(9) NOT NULL, + created_ts DATETIME(3) NOT NULL + )`, + `CREATE TABLE weeks ( + id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + programId BIGINT NOT NULL, + label VARCHAR(80), + durationMinutes INT NOT NULL, + FOREIGN KEY (programId) REFERENCES programs (id) + )`, + `CREATE TABLE assets ( + id VARCHAR(36) NOT NULL DEFAULT (UUID()) PRIMARY KEY, + ownerId VARCHAR(36) NOT NULL, + recordedAt DATETIME(3) NOT NULL, + observedAt DATETIME(3) NOT NULL, + asOfDate DATE NOT NULL, + atTime TIME(3) NOT NULL + )`, +]; + +async function loadInline(metaJson: string): Promise { + const r = await new MetaDataLoader().load([new InMemoryStringSource(metaJson)]); + // The loader collects errors instead of throwing; a refused inline model must not reach MySQL. + expect(r.errors).toEqual([]); + return r.root; +} + +function reportViews(root: MetaRoot) { + return buildReportViews(root, { dialect: "mysql", columnNamingStrategy: "literal" }); +} + +/** `CREATE VIEW` for every report view of `root`, exactly as the recipe shows. */ +async function createViews(root: MetaRoot): Promise { + const views = reportViews(root); + for (const v of views) { + await conn.query(`DROP VIEW IF EXISTS \`${v.name}\``); + await conn.query(`CREATE VIEW \`${v.name}\` AS\n${v.sql}`); + } + return views.map((v) => v.name); +} + +async function select(query: string): Promise[]> { + const [rows] = await conn.query(query); + return rows as Record[]; +} + +const SEED_PROGRAMS_AND_WEEKS = ` + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'); + INSERT INTO weeks (id, programId, label, durationMinutes) VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45);`; + +const SEED_PROGRAMS_BY_TIME = ` + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00'), + (4, 'Legacy', 700, 'ARCHIVED', '2026-05-31T23:59:59'), + (5, 'Monday', 300, 'PUBLISHED', '2026-05-18T00:00:00');`; + +const SEED_ASSETS = ` + INSERT INTO assets (ownerId, recordedAt, observedAt, asOfDate, atTime) VALUES + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T03:30:00', '2026-05-04T03:30:00', '2026-05-03', '03:30:00'), + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T03:45:00', '2026-05-04T03:45:00', '2026-05-03', '03:45:00'), + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T04:10:00', '2026-05-04T04:10:00', '2026-05-04', '04:10:00');`; + +/** Seed scripts hold several statements; the connection runs one at a time. */ +async function exec(script: string): Promise { + for (const stmt of script.split(/;\s*\n/).map((s) => s.trim()).filter(Boolean)) { + await conn.query(stmt.endsWith(";") ? stmt.slice(0, -1) : stmt); + } +} + +// --------------------------------------------------------------------------------- +// Inline models: the Table D grains and the Table E durations the canonical corpus does +// not exercise (it has no quarter or year grain and only `-P30D`). +// --------------------------------------------------------------------------------- + +/** One report with EVERY grain over a timestamp, grouped by all of them (one row per instant). */ +const GRAIN_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Event", children: [ + { "source.rdb": { "@table": "events" } }, + { "field.long": { name: "id" } }, + { "field.timestamp": { name: "recordedAt", "@required": true } }, + { "field.date": { name: "happenedOn", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "dimension.time": { name: "recordedAt", "@of": "Event.recordedAt", + "@grains": ["hour", "day", "week", "month", "quarter", "year"] } }, + { "dimension.time": { name: "happenedOn", "@of": "Event.happenedOn", "@grains": ["week", "quarter"] } }, + { "measure.aggregate": { name: "events", "@agg": "count", "@of": "Event.id" } }, + ] } }, + { "object.report": { name: "EventsByGrain", "@from": "Event", + "@dimensions": ["recordedAt:hour", "recordedAt:day", "recordedAt:week", "recordedAt:month", + "recordedAt:quarter", "recordedAt:year", "happenedOn:week", "happenedOn:quarter"], + "@measures": ["events"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_events_by_grain" } } ] } }, +]}}); + +/** Three windows, one per Table E branch: an instant duration, a date duration, a forward one. */ +const RELATIVE_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Event", children: [ + { "source.rdb": { "@table": "events" } }, + { "field.long": { name: "id" } }, + { "field.timestamp": { name: "recordedAt", "@required": true } }, + { "field.date": { name: "happenedOn", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "events", "@agg": "count", "@of": "Event.id" } }, + ] } }, + { "object.report": { name: "LastTwelveHours", "@from": "Event", "@measures": ["events"], + "@filter": { recordedAt: { gte: { now: "-PT12H" } } }, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_last_twelve_hours" } } ] } }, + { "object.report": { name: "LastTwoWeeks", "@from": "Event", "@measures": ["events"], + "@filter": { happenedOn: { gte: { now: "-P2W" } } }, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_last_two_weeks" } } ] } }, + { "object.report": { name: "UpToTomorrow", "@from": "Event", "@measures": ["events"], + "@filter": { recordedAt: { lte: { now: "P1DT1H" } } }, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_up_to_tomorrow" } } ] } }, +]}}); + +/** Every view and table this file creates: the canonical six, the inline models' and the recipe's. */ +const OWN_VIEWS = [ + ...CANONICAL_VIEWS, + "v_events_by_grain", "v_last_twelve_hours", "v_last_two_weeks", "v_up_to_tomorrow", + "v_program_minutes_recipe", +] as const; +const OWN_TABLES = ["weeks", "programs", "assets", "events"] as const; + +beforeAll(async () => { + container = await startMysql(); + conn = await mysql.createConnection({ + uri: container.url, + timezone: "Z", + dateStrings: true, + supportBigNumbers: true, + bigNumberStrings: true, + }); + // Idempotent: a rerun against a persistent METAOBJECTS_TEST_MYSQL_URL starts clean, and + // every test below is independent of test order (or of `-t` selecting one of them). + // Only the views and tables THIS file creates, by name: METAOBJECTS_TEST_MYSQL_URL may point + // at a shared database, and sweeping information_schema would drop somebody else's views. + for (const v of OWN_VIEWS) await conn.query(`DROP VIEW IF EXISTS \`${v}\``); + for (const t of OWN_TABLES) await conn.query(`DROP TABLE IF EXISTS \`${t}\``); + for (const ddl of DDL) await conn.query(ddl); + canonical = await loadMetadataDir(CANONICAL_DIR); + await createViews(canonical); +}, 240_000); + +afterAll(async () => { + await conn?.end(); + container?.stop(); +}, 60_000); + +beforeEach(async () => { + await conn.query("DELETE FROM weeks"); + await conn.query("DELETE FROM programs"); + await conn.query("DELETE FROM assets"); +}); + +describe("report views — canonical model on real MySQL 8.4", () => { + test("every canonical view is accepted under the server's default sql_mode (ONLY_FULL_GROUP_BY)", async () => { + // Assert the mode rather than assume it: the whole claim is "valid under the default". + const [mode] = await select(`SELECT @@GLOBAL.sql_mode AS g, @@SESSION.sql_mode AS s`); + expect(String(mode?.g)).toContain("ONLY_FULL_GROUP_BY"); + expect(String(mode?.s)).toContain("ONLY_FULL_GROUP_BY"); + + // Re-create them here, under the mode just asserted, so this test does not lean on beforeAll. + const names = await createViews(canonical); + expect([...names].sort()).toEqual([...CANONICAL_VIEWS].sort()); + for (const name of names) { + // Selecting proves the stored body still resolves, not merely that it parsed. + await select(`SELECT * FROM \`${name}\``); + } + const created = await select( + `SELECT table_name AS n FROM information_schema.views WHERE table_schema = DATABASE() ORDER BY table_name`, + ); + // The inline-model tests below add views of their own, so assert inclusion, not equality. + expect(created.map((r) => r.n)).toEqual(expect.arrayContaining([...CANONICAL_VIEWS])); + }, 60_000); + + describe("values", () => { + test("v_program_minutes: every measure kind, one row per (program, programTitle); longShare 0.7500 and 0.0000", async () => { + await exec(SEED_PROGRAMS_AND_WEEKS); + const rows = await select("SELECT * FROM `v_program_minutes` ORDER BY `program`"); + expect(rows).toEqual([ + { program: "1", programTitle: "Foundations", weeks: "4", longWeeks: "3", labels: "2", slots: "3", + totalMinutes: "240", avgMinutes: "60.0000", minMinutes: 30, maxMinutes: 90, longShare: "0.7500" }, + { program: "2", programTitle: "Strength", weeks: "1", longWeeks: "0", labels: "1", slots: "1", + totalMinutes: "45", avgMinutes: "45.0000", minMinutes: 45, maxMinutes: 45, longShare: "0.0000" }, + ]); + + // The measure filter, sort and count the shared scenario runs through the view. + expect((await select("SELECT `program` FROM `v_program_minutes` WHERE `weeks` >= 2 ORDER BY `program`")).map((r) => r.program)) + .toEqual(["1"]); + expect((await select("SELECT `program` FROM `v_program_minutes` ORDER BY `totalMinutes` DESC LIMIT 1")).map((r) => r.program)) + .toEqual(["1"]); + expect((await select("SELECT count(*) AS n FROM `v_program_minutes`"))[0]?.n).toBe("2"); + }); + + test("REVIEW FOCUS 5: a ratio of 2/3 is 0.6667 on MySQL (four fractional digits, not Postgres' 20)", async () => { + await exec(` + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES + (1, 'Thirds', 100, 'PUBLISHED', '2026-05-01T10:00:00'); + INSERT INTO weeks (programId, label, durationMinutes) VALUES + (1, 'a', 30), (1, 'b', 60), (1, 'c', 90);`); + // Two of three weeks are >= 60 minutes. + expect((await select("SELECT `longShare` FROM `v_program_minutes`"))[0]?.longShare).toBe("0.6667"); + expect((await select("SELECT `longShare` FROM `v_fitness_totals`"))[0]?.longShare).toBe("0.6667"); + }); + + test("v_fitness_totals: no dimensions, one row over the whole table", async () => { + await exec(SEED_PROGRAMS_AND_WEEKS); + expect(await select("SELECT * FROM `v_fitness_totals`")) + .toEqual([{ weeks: "5", totalMinutes: "285", longShare: "0.6000" }]); + }); + + test("EMPTY GROUPS (Review Focus 4): v_fitness_totals over an empty weeks table is one row (0, NULL, NULL)", async () => { + expect((await select("SELECT count(*) AS n FROM weeks"))[0]?.n).toBe("0"); + expect(await select("SELECT * FROM `v_fitness_totals`")) + .toEqual([{ weeks: "0", totalMinutes: null, longShare: null }]); + }); + + test("COLUMN TYPES: totalMinutes is bigint (CAST ... AS SIGNED); counts are bigint, avg and ratio decimal, min/max keep the field's int", async () => { + const cols = await select( + `SELECT column_name AS c, data_type AS t FROM information_schema.columns + WHERE table_schema = DATABASE() AND table_name = 'v_program_minutes' ORDER BY ordinal_position`, + ); + expect(Object.fromEntries(cols.map((r) => [r.c, r.t]))).toEqual({ + program: "bigint", programTitle: "varchar", + weeks: "bigint", longWeeks: "bigint", labels: "bigint", slots: "bigint", + totalMinutes: "bigint", avgMinutes: "decimal", minMinutes: "int", maxMinutes: "int", longShare: "decimal", + }); + // A bare SUM(int) would be decimal(32,0); the cast is what makes it bigint. + const fitness = await select( + `SELECT data_type AS t FROM information_schema.columns + WHERE table_schema = DATABASE() AND table_name = 'v_fitness_totals' AND column_name = 'totalMinutes'`, + ); + expect(fitness[0]?.t).toBe("bigint"); + }); + + test("v_programs_by_month and v_programs_by_week: month grain, enum dimension, null filtered sum, ISO Monday boundary, report @segment", async () => { + await exec(SEED_PROGRAMS_BY_TIME); + + expect(await select("SELECT * FROM `v_programs_by_month` ORDER BY `status`")).toEqual([ + { createdAtMonth: "2026-05-01", status: "ARCHIVED", programs: "1", listValue: null }, + { createdAtMonth: "2026-06-01", status: "DRAFT", programs: "1", listValue: null }, + { createdAtMonth: "2026-05-01", status: "PUBLISHED", programs: "3", listValue: "7799" }, + ]); + + // Program 2 (Sunday 23:30) and program 5 (Monday 00:00) are thirty minutes apart and + // land in different weeks; DRAFT / ARCHIVED are scoped out by the segment. + expect(await select("SELECT * FROM `v_programs_by_week` ORDER BY `createdAtWeek`")).toEqual([ + { createdAtWeek: "2026-04-27", programs: "1" }, + { createdAtWeek: "2026-05-11", programs: "1" }, + { createdAtWeek: "2026-05-18", programs: "1" }, + ]); + }); + + test("v_asset_activity: hour on a timestamp, week on a field.date", async () => { + await exec(SEED_ASSETS); + expect(await select("SELECT * FROM `v_asset_activity` ORDER BY `recordedAtHour`")).toEqual([ + { recordedAtHour: "2026-05-04 03:00:00.000", asOfDateWeek: "2026-04-27", assets: "2" }, + { recordedAtHour: "2026-05-04 04:00:00.000", asOfDateWeek: "2026-05-04", assets: "1" }, + ]); + // The hour bucket is a DATETIME(3), so it compares with a stored instant. + const t = await select( + `SELECT data_type AS t, datetime_precision AS p FROM information_schema.columns + WHERE table_schema = DATABASE() AND table_name = 'v_asset_activity' AND column_name = 'recordedAtHour'`, + ); + expect(t[0]).toEqual({ t: "datetime", p: 3 }); + }); + + test("RELATIVE WINDOW: v_recent_programs counts the program 3 days old, not the one 60 days old", async () => { + await exec(` + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES + (1, 'Recent', 100, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 3 DAY), + (2, 'Stale', 100, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 60 DAY);`); + expect(await select("SELECT * FROM `v_recent_programs`")).toEqual([{ programs: "1" }]); + }); + }); +}); + +describe("report views — inline models on real MySQL 8.4", () => { + test("TABLE D: hour, day, week, month, quarter and year grains, on a timestamp and on a date", async () => { + await conn.query("DROP TABLE IF EXISTS events"); + await conn.query( + `CREATE TABLE events (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, recordedAt DATETIME(3) NOT NULL, happenedOn DATE NOT NULL)`, + ); + const root = await loadInline(GRAIN_MODEL); + const names = await createViews(root); + expect(names).toEqual(["v_events_by_grain"]); + + await exec(` + INSERT INTO events (recordedAt, happenedOn) VALUES + ('2026-05-01T10:15:00', '2026-05-01'), + ('2026-05-17T23:30:00', '2026-05-17'), + ('2026-06-01T00:00:00', '2026-06-01'), + ('2026-01-01T00:00:00', '2026-01-01'), + ('2026-07-01T05:05:05', '2026-07-01'), + ('2026-12-31T23:59:59', '2026-12-31');`); + + const grain = (hour: string, day: string, week: string, month: string, quarter: string, year: string, + dWeek: string, dQuarter: string) => ({ + recordedAtHour: hour, recordedAtDay: day, recordedAtWeek: week, recordedAtMonth: month, + recordedAtQuarter: quarter, recordedAtYear: year, + happenedOnWeek: dWeek, happenedOnQuarter: dQuarter, events: "1", + }); + expect(await select("SELECT * FROM `v_events_by_grain` ORDER BY `recordedAtHour`")).toEqual([ + // 2026-01-01 is a Thursday: ISO week starts Monday 2025-12-29, in the previous year. + grain("2026-01-01 00:00:00.000", "2026-01-01", "2025-12-29", "2026-01-01", "2026-01-01", "2026-01-01", "2025-12-29", "2026-01-01"), + // Contract Table D checked values: day, week, month, quarter, year of 2026-05-01T10:00. + grain("2026-05-01 10:00:00.000", "2026-05-01", "2026-04-27", "2026-05-01", "2026-04-01", "2026-01-01", "2026-04-27", "2026-04-01"), + // A Sunday at 23:30 is still the week of Monday 2026-05-11. + grain("2026-05-17 23:00:00.000", "2026-05-17", "2026-05-11", "2026-05-01", "2026-04-01", "2026-01-01", "2026-05-11", "2026-04-01"), + // A Monday midnight starts its own week. + grain("2026-06-01 00:00:00.000", "2026-06-01", "2026-06-01", "2026-06-01", "2026-04-01", "2026-01-01", "2026-06-01", "2026-04-01"), + grain("2026-07-01 05:00:00.000", "2026-07-01", "2026-06-29", "2026-07-01", "2026-07-01", "2026-01-01", "2026-06-29", "2026-07-01"), + grain("2026-12-31 23:00:00.000", "2026-12-31", "2026-12-28", "2026-12-01", "2026-10-01", "2026-01-01", "2026-12-28", "2026-10-01"), + ]); + + // The grain's column types (Table B): hour is a DATETIME(3), every other grain a DATE. + const cols = await select( + `SELECT column_name AS c, data_type AS t FROM information_schema.columns + WHERE table_schema = DATABASE() AND table_name = 'v_events_by_grain' AND column_name <> 'events'`, + ); + for (const r of cols) expect(r.t).toBe(r.c === "recordedAtHour" ? "datetime" : "date"); + }, 60_000); + + test("TABLE E: relative-date windows read the UTC wall clock (an instant, a date, a forward duration)", async () => { + await conn.query("DROP TABLE IF EXISTS events"); + await conn.query( + `CREATE TABLE events (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, recordedAt DATETIME(3) NOT NULL, happenedOn DATE NOT NULL)`, + ); + const root = await loadInline(RELATIVE_MODEL); + expect((await createViews(root)).sort()).toEqual(["v_last_twelve_hours", "v_last_two_weeks", "v_up_to_tomorrow"]); + + // Clock-relative on purpose: the view calls UTC_TIMESTAMP(3) when it is QUERIED. + await exec(` + INSERT INTO events (recordedAt, happenedOn) VALUES + (UTC_TIMESTAMP(3) - INTERVAL 1 HOUR, DATE(UTC_TIMESTAMP(3) - INTERVAL 1 DAY)), + (UTC_TIMESTAMP(3) - INTERVAL 11 HOUR, DATE(UTC_TIMESTAMP(3) - INTERVAL 13 DAY)), + (UTC_TIMESTAMP(3) - INTERVAL 13 HOUR, DATE(UTC_TIMESTAMP(3) - INTERVAL 15 DAY)), + (UTC_TIMESTAMP(3) + INTERVAL 1 HOUR, DATE(UTC_TIMESTAMP(3) - INTERVAL 40 DAY)), + (UTC_TIMESTAMP(3) + INTERVAL 3 DAY, DATE(UTC_TIMESTAMP(3) - INTERVAL 60 DAY));`); + + // gte -PT12H: only the 13h-old instant is out (1h and 11h ago, and both future rows, are in). + expect(await select("SELECT * FROM `v_last_twelve_hours`")).toEqual([{ events: "4" }]); + // -P2W is 14 days, applied to the date: 1 and 13 days ago are in, 15, 40 and 60 are out. + expect(await select("SELECT * FROM `v_last_two_weeks`")).toEqual([{ events: "2" }]); + // P1DT1H forward: everything up to a day and an hour ahead; the 3-days-ahead row is out. + expect(await select("SELECT * FROM `v_up_to_tomorrow`")).toEqual([{ events: "4" }]); + }, 60_000); +}); + +describe("the recipe's declaration and script", () => { + /** The first fenced `json` block under "### Reports" in docs/recipes/mysql.md. */ + function recipeDeclaration(): string { + const doc = readFileSync(join(REPO_ROOT, "docs/recipes/mysql.md"), "utf8"); + const section = doc.slice(doc.indexOf("### Reports")); + const m = /```json\n([\s\S]*?)\n```/.exec(section); + if (m === null) throw new Error("docs/recipes/mysql.md has no json block under '### Reports'"); + return m[1]!; + } + + /** A model whose one report carries `sourceJson` as its source, written where `loadDirectory` reads it. */ + function modelDir(sourceJson: string): string { + const dir = mkdtempSync(join(tmpdir(), "report-recipe-")); + const source = JSON.parse(sourceJson) as Record; + writeFileSync(join(dir, "meta.fitness.json"), JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Week", children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.long": { name: "id" } }, + { "field.int": { name: "durationMinutes", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "weeks", "@agg": "count", "@of": "Week.id" } }, + ] } }, + { "object.report": { name: "ProgramMinutes", "@from": "Week", "@measures": ["weeks"], children: [source] } }, + ]}})); + return dir; + } + + /** The recipe's script, minus the console.log. */ + async function recipeViews(sourceJson: string) { + const dir = modelDir(sourceJson); + try { + const { root } = await loadDirectory(dir); + return buildReportViews(root, { dialect: "mysql" }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + } + + test("the declaration the recipe shows yields exactly one view, and MySQL accepts its body", async () => { + const declaration = recipeDeclaration(); + expect(declaration).not.toContain("@unmanaged"); + const views = await recipeViews(declaration); + expect(views.map((v) => v.name)).toEqual(["v_program_minutes"]); + + // The body is built over the default snake_case column names; the recipe's own `weeks` + // table has `id`, so it resolves as it stands. + await conn.query("DROP VIEW IF EXISTS `v_program_minutes_recipe`"); + await conn.query(`CREATE VIEW \`v_program_minutes_recipe\` AS\n${views[0]!.sql}`); + await exec(` + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES (1, 'P', 1, 'DRAFT', '2026-05-01T10:00:00'); + INSERT INTO weeks (programId, label, durationMinutes) VALUES (1, 'a', 30), (1, 'b', 45);`); + expect(await select("SELECT * FROM `v_program_minutes_recipe`")).toEqual([{ weeks: "2" }]); + await conn.query("DROP VIEW `v_program_minutes_recipe`"); + }, 60_000); + + test("the same declaration with @unmanaged: true yields no view: buildReportViews skips an unmanaged source", async () => { + const unmanaged = JSON.parse(recipeDeclaration()) as { "source.rdb": Record }; + unmanaged["source.rdb"]["@unmanaged"] = true; + expect(await recipeViews(JSON.stringify(unmanaged))).toEqual([]); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/report-views-pg.test.ts b/server/typescript/packages/integration-tests/test/report-views-pg.test.ts new file mode 100644 index 000000000..7921e357a --- /dev/null +++ b/server/typescript/packages/integration-tests/test/report-views-pg.test.ts @@ -0,0 +1,504 @@ +/** + * Report views — FR-044 Plan 2, against a REAL Postgres. + * + * Earlier tasks proved the lowering as text (emitter goldens) and through the unit-level + * view builder. This is where a report view first meets an engine: it must (1) CONVERGE + * under `meta migrate` (emit -> apply -> re-diff empty, which is what Postgres deparsing + * the stored view body makes non-trivial), and (2) return the expected rows. + * + * The rows are the ones the shared Task 10 persistence scenarios assert. They are read + * here with raw SQL straight off the views, so a wrong value is a LOWERING defect, not + * an ObjectManager one. + * + * Raw values are compared as the engine sends them (dates and timestamps as text, int8 + * and numeric as strings). Postgres pads numeric precision (`AVG(int)` is + * `60.0000000000000000`), so decimals are compared through `canonicalDecimal`, which is + * what the conformance runner's wire normalisation does too. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from "bun:test"; +import { + buildExpectedSchema, diff, emit, introspectPostgres, collectUnmanagedNames, + type AllowOptions, type SchemaSnapshot, +} from "@metaobjectsdev/migrate-ts"; +import { buildProjectionViews } from "@metaobjectsdev/codegen-ts"; +import { MetaDataLoader, InMemoryStringSource, type MetaRoot } from "@metaobjectsdev/metadata"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import { Client, Pool } from "pg"; +import { startPostgres, type RunningPg } from "../src/postgres-container.ts"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; + +// --------------------------------------------------------------------------------- +// Container + pipeline helpers (the same shape as view-lifecycle-pg.test.ts). +// --------------------------------------------------------------------------------- + +let pg: RunningPg; +let k: Kysely>; +let canonical: MetaRoot; + +beforeAll(async () => { + pg = await startPostgres(); + k = new Kysely>({ + dialect: new PostgresDialect({ pool: new Pool({ connectionString: pg.connectionUri }) }), + }); + canonical = await loadMetadataDir(CANONICAL_DIR); +}, 120_000); + +afterAll(async () => { + await k.destroy(); + await pg.stop(); +}, 60_000); + +beforeEach(async () => { + await sql.raw("DROP SCHEMA public CASCADE").execute(k); + await sql.raw("CREATE SCHEMA public").execute(k); +}); + +async function applyRaw(text: string): Promise { + for (const stmt of text.split(/;\s*\n/).map((s) => s.trim()).filter(Boolean)) { + await sql.raw(stmt.endsWith(";") ? stmt : `${stmt};`).execute(k); + } +} + +async function loadInline(metaJson: string): Promise { + const r = await new MetaDataLoader().load([new InMemoryStringSource(metaJson)]); + // The loader collects errors instead of throwing; a refused inline model must not be migrated. + expect(r.errors).toEqual([]); + return r.root; +} + +function expectedFor(root: MetaRoot): SchemaSnapshot { + return buildExpectedSchema(root, { + columnNamingStrategy: "literal", + views: buildProjectionViews(root, { dialect: "postgres", columnNamingStrategy: "literal" }), + }); +} + +/** build -> introspect -> diff -> emit -> apply, exactly the production pipeline. */ +async function migrate(root: MetaRoot, allow: AllowOptions = {}) { + const expected = expectedFor(root); + const unmanagedNames = collectUnmanagedNames(root); + const result = await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres", allow, unmanagedNames }); + const emittable = result.changes.filter((c) => c.status.state !== "blocked"); + const { up } = emittable.length === 0 ? { up: "" } : emit(emittable, { dialect: "postgres" }); + if (result.blocked.length === 0 && up.trim().length > 0) await applyRaw(up); + return { expected, unmanagedNames, result, up }; +} + +/** THE gate: re-diffing the just-migrated database proposes nothing. */ +async function assertConverged(expected: SchemaSnapshot, unmanagedNames: string[] = []): Promise { + const followup = await diff({ expected, actual: await introspectPostgres(k), dialect: "postgres", unmanagedNames }); + if (followup.changes.length > 0) { + console.error("NOT CONVERGED — a further `meta migrate` would emit:"); + for (const c of followup.changes) console.error(" -", c.kind, JSON.stringify(c).slice(0, 200)); + } + expect(followup.changes).toEqual([]); +} + +/** + * Raw, text-faithful reads: only int4 becomes a number; int8, numeric, dates and + * timestamps stay the strings Postgres sent, so nothing is reshaped by the driver. + * `sessionZone` runs the connection in that time zone (Review Focus 3). + */ +async function select(query: string, sessionZone?: string): Promise[]> { + const c = new Client({ + connectionString: pg.connectionUri, + types: { getTypeParser: (oid: number) => (oid === 23 ? (v: string) => Number(v) : (v: string) => v) }, + }); + await c.connect(); + try { + if (sessionZone !== undefined) await c.query(`SET TIME ZONE '${sessionZone}'`); + return (await c.query(query)).rows as Record[]; + } finally { + await c.end(); + } +} + +/** `60.0000000000000000` -> `60`, `0.75000000000000000000` -> `0.75`; null stays null. */ +function canonicalDecimal(v: unknown): string | null { + if (v === null || v === undefined) return null; + const s = String(v); + return s.includes(".") ? s.replace(/\.?0+$/, "") : s; +} + +/** The `CREATE VIEW` body this model's builder produces for `name`. */ +function viewSql(root: MetaRoot, name: string): string { + const v = buildProjectionViews(root, { dialect: "postgres", columnNamingStrategy: "literal" }) + .find((x) => x.name === name); + if (v === undefined) throw new Error(`no view ${name}`); + return v.sql; +} + +// --------------------------------------------------------------------------------- +// Seeds — verbatim from the Task 10 scenarios. +// --------------------------------------------------------------------------------- + +const SEED_PROGRAMS_AND_WEEKS = ` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45);`; + +const SEED_PROGRAMS_BY_TIME = ` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00'), + (4, 'Legacy', 700, 'ARCHIVED', '2026-05-31T23:59:59'), + (5, 'Monday', 300, 'PUBLISHED', '2026-05-18T00:00:00');`; + +const SEED_ASSETS = ` + INSERT INTO "assets" ("ownerId","recordedAt","observedAt","asOfDate","atTime") VALUES + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T03:30:00Z', '2026-05-04T03:30:00', '2026-05-03', '03:30:00'), + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T03:45:00Z', '2026-05-04T03:45:00', '2026-05-03', '03:45:00'), + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-04T04:10:00Z', '2026-05-04T04:10:00', '2026-05-04', '04:10:00');`; + +const HOUR_BUCKET = `to_char("recordedAtHour" AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')`; + +describe("report views — canonical model on real Postgres", () => { + // ------------------------------------------------------------------------- + // Convergence (spec §7: emit -> apply -> re-diff empty). + // ------------------------------------------------------------------------- + + test("CONVERGENCE: the six canonical report views migrate from empty, then a second and third migrate propose nothing", async () => { + const first = await migrate(canonical); + + for (const view of [ + "v_program_minutes", "v_fitness_totals", "v_programs_by_month", + "v_programs_by_week", "v_recent_programs", "v_asset_activity", + ]) { + expect(first.up).toContain(`CREATE VIEW "${view}" AS`); + const r = await sql.raw(`SELECT to_regclass('public.${view}') IS NOT NULL AS ok`).execute(k); + expect((r.rows[0] as { ok: boolean }).ok).toBe(true); + } + + await assertConverged(first.expected, first.unmanagedNames); + + const second = await migrate(canonical); + expect(second.up.trim()).toBe(""); + expect(second.result.changes).toEqual([]); + await assertConverged(second.expected, second.unmanagedNames); + + const third = await migrate(canonical); + expect(third.up.trim()).toBe(""); + expect(third.result.changes).toEqual([]); + }, 60_000); + + // ------------------------------------------------------------------------- + // Join type: the existing #209 rule, unchanged. + // ------------------------------------------------------------------------- + + test("JOIN TYPE: Week.fkProgram is a required belongs-to, so v_program_minutes joins programs INNER", () => { + const body = viewSql(canonical, "v_program_minutes"); + expect(body).toContain(`INNER JOIN "programs" p ON p."id" = w."programId"`); + expect(body).not.toContain("LEFT OUTER JOIN"); + }); + + // ------------------------------------------------------------------------- + // Values — the Task 10 scenarios, read straight off the views. + // ------------------------------------------------------------------------- + + describe("values", () => { + beforeEach(async () => { + await migrate(canonical); + }); + + test("v_program_minutes: every measure kind, one row per (program, programTitle)", async () => { + await applyRaw(SEED_PROGRAMS_AND_WEEKS); + const rows = await select(`SELECT * FROM "v_program_minutes" ORDER BY "program"`); + expect(rows.map((r) => ({ ...r, avgMinutes: canonicalDecimal(r.avgMinutes), longShare: canonicalDecimal(r.longShare) }))) + .toEqual([ + { program: "1", programTitle: "Foundations", weeks: "4", longWeeks: "3", labels: "2", slots: "3", + totalMinutes: "240", avgMinutes: "60", minMinutes: 30, maxMinutes: 90, longShare: "0.75" }, + { program: "2", programTitle: "Strength", weeks: "1", longWeeks: "0", labels: "1", slots: "1", + totalMinutes: "45", avgMinutes: "45", minMinutes: 45, maxMinutes: 45, longShare: "0" }, + ]); + + // The measure-filter, sort and count the shared scenario runs through the view. + expect( + (await select(`SELECT "program" FROM "v_program_minutes" WHERE "weeks" >= 2 ORDER BY "program"`)).map((r) => r.program), + ).toEqual(["1"]); + expect( + (await select(`SELECT "program" FROM "v_program_minutes" ORDER BY "totalMinutes" DESC LIMIT 1`)).map((r) => r.program), + ).toEqual(["1"]); + expect((await select(`SELECT count(*) AS n FROM "v_program_minutes"`))[0]?.n).toBe("2"); + }); + + test("v_fitness_totals: no dimensions, one row over the whole table", async () => { + await applyRaw(SEED_PROGRAMS_AND_WEEKS); + const rows = await select(`SELECT * FROM "v_fitness_totals"`); + expect(rows.map((r) => ({ ...r, longShare: canonicalDecimal(r.longShare) }))) + .toEqual([{ weeks: "5", totalMinutes: "285", longShare: "0.6" }]); + }); + + test("EMPTY GROUPS (Review Focus 4): v_fitness_totals over an empty weeks table is one row (0, NULL, NULL)", async () => { + const count = await select(`SELECT count(*) AS n FROM "weeks"`); + expect(count[0]?.n).toBe("0"); + const rows = await select(`SELECT * FROM "v_fitness_totals"`); + // One row, count 0, a sum of nothing is NULL (not 0), a zero denominator is NULL. + expect(rows).toEqual([{ weeks: "0", totalMinutes: null, longShare: null }]); + }); + + test("v_programs_by_month and v_programs_by_week: month grain, enum dimension, null filtered sum, ISO Monday boundary, report @segment", async () => { + await applyRaw(SEED_PROGRAMS_BY_TIME); + + const month = await select(`SELECT * FROM "v_programs_by_month" ORDER BY "status"`); + expect(month).toEqual([ + { createdAtMonth: "2026-05-01", status: "ARCHIVED", programs: "1", listValue: null }, + { createdAtMonth: "2026-06-01", status: "DRAFT", programs: "1", listValue: null }, + { createdAtMonth: "2026-05-01", status: "PUBLISHED", programs: "3", listValue: "7799" }, + ]); + + // Program 2 (Sunday 23:30) and program 5 (Monday 00:00) are thirty minutes apart + // and land in different weeks; DRAFT / ARCHIVED are scoped out by the segment. + const week = await select(`SELECT * FROM "v_programs_by_week" ORDER BY "createdAtWeek"`); + expect(week).toEqual([ + { createdAtWeek: "2026-04-27", programs: "1" }, + { createdAtWeek: "2026-05-11", programs: "1" }, + { createdAtWeek: "2026-05-18", programs: "1" }, + ]); + }); + + test("v_asset_activity: hour on an instant, week on a field.date", async () => { + await applyRaw(SEED_ASSETS); + const rows = await select( + `SELECT ${HOUR_BUCKET} AS "recordedAtHour", "asOfDateWeek"::text AS "asOfDateWeek", "assets" + FROM "v_asset_activity" ORDER BY "recordedAtHour"`, + ); + expect(rows).toEqual([ + { recordedAtHour: "2026-05-04T03:00:00Z", asOfDateWeek: "2026-04-27", assets: "2" }, + { recordedAtHour: "2026-05-04T04:00:00Z", asOfDateWeek: "2026-05-04", assets: "1" }, + ]); + }); + + test("UTC BUCKETS (Review Focus 3): under a New York session zone an instant still lands in its UTC hour bucket", async () => { + // 23:30 on 2026-05-03 in New York (UTC-4 in May) is 03:30 on 2026-05-04 UTC. + await sql.raw( + `INSERT INTO "assets" ("ownerId","recordedAt","observedAt","asOfDate","atTime") VALUES + ('aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', '2026-05-03T23:30:00-04:00', '2026-05-03T23:30:00', '2026-05-03', '23:30:00')`, + ).execute(k); + const rows = await select( + `SELECT ${HOUR_BUCKET} AS "recordedAtHour", "assets" FROM "v_asset_activity"`, + "America/New_York", + ); + expect(rows).toEqual([{ recordedAtHour: "2026-05-04T03:00:00Z", assets: "1" }]); + // And the session really was not UTC while it ran. + expect((await select(`SHOW TIME ZONE`, "America/New_York"))[0]).toEqual({ TimeZone: "America/New_York" }); + }); + + test("RELATIVE WINDOW: v_recent_programs counts the program 3 days old, not the one 60 days old", async () => { + await applyRaw(` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Recent', 100, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '3 days'), + (2, 'Stale', 100, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '60 days');`); + expect(await select(`SELECT * FROM "v_recent_programs"`)).toEqual([{ programs: "1" }]); + }); + }); +}); + +// --------------------------------------------------------------------------------- +// Inline models: the behaviours the canonical corpus does not exercise. +// --------------------------------------------------------------------------------- + +/** An Event table, hour and day reports over its UTC instant (Review Focus 3, day grain). */ +const EVENT_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Event", children: [ + { "source.rdb": { "@table": "events" } }, + { "field.long": { name: "id" } }, + { "field.timestamp": { name: "recordedAt", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "dimension.time": { name: "recordedAt", "@of": "Event.recordedAt", "@grains": ["hour", "day"] } }, + { "measure.aggregate": { name: "events", "@agg": "count", "@of": "Event.id" } }, + ] } }, + { "object.report": { name: "EventsByDay", "@from": "Event", "@dimensions": ["recordedAt:day"], "@measures": ["events"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_events_by_day" } } ] } }, + { "object.report": { name: "EventsByHour", "@from": "Event", "@dimensions": ["recordedAt:hour"], "@measures": ["events"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_events_by_hour" } } ] } }, +]}}); + +/** A Fact whose program reference is NULLABLE, grouped by the referenced program's title. */ +const FACT_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Prog", children: [ + { "source.rdb": { "@table": "progs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Fact", children: [ + { "source.rdb": { "@table": "facts" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "progId" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "fkProg", "@fields": "progId", "@references": "Prog" } }, + { "dimension.attribute": { name: "progTitle", "@of": "Prog.title", "@via": "Fact.fkProg" } }, + { "measure.aggregate": { name: "facts", "@agg": "count", "@of": "Fact.id" } }, + ] } }, + { "object.report": { name: "FactsByProg", "@from": "Fact", "@dimensions": ["progTitle"], "@measures": ["facts"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_facts_by_prog" } } ] } }, +]}}); + +/** A Metric table with one measure, optionally a second; the report lists what it has. */ +function metricModel(measures: string[]): string { + const all = [ + { "measure.aggregate": { name: "samples", "@agg": "count", "@of": "Metric.id" } }, + { "measure.aggregate": { name: "total", "@agg": "sum", "@of": "Metric.amount" } }, + ]; + return JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Metric", children: [ + { "source.rdb": { "@table": "metrics" } }, + { "field.long": { name: "id" } }, + { "field.int": { name: "amount", "@required": true } }, + { "field.string": { name: "kind", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "dimension.attribute": { name: "kind", "@of": "Metric.kind" } }, + ...all, + ] } }, + { "object.report": { name: "MetricsByKind", "@from": "Metric", "@dimensions": ["kind"], "@measures": measures, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_metrics_by_kind" } } ] } }, + ]}}); +} + +describe("report views — inline models on real Postgres", () => { + test("UTC BUCKETS (Review Focus 3): a report at recordedAt:day puts 23:30 New York on the next UTC day", async () => { + const root = await loadInline(EVENT_MODEL); + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + await sql.raw( + `INSERT INTO "events" ("recordedAt") VALUES ('2026-05-03T23:30:00-04:00')`, + ).execute(k); + + const day = await select(`SELECT "recordedAtDay"::text AS "recordedAtDay", "events" FROM "v_events_by_day"`, "America/New_York"); + expect(day).toEqual([{ recordedAtDay: "2026-05-04", events: "1" }]); + const hour = await select( + `SELECT ${HOUR_BUCKET} AS "recordedAtHour", "events" FROM "v_events_by_hour"`, + "America/New_York", + ); + expect(hour).toEqual([{ recordedAtHour: "2026-05-04T03:00:00Z", events: "1" }]); + + // The same rows under a UTC session: the buckets are the reader's-zone-independent. + expect(await select(`SELECT "recordedAtDay"::text AS "recordedAtDay", "events" FROM "v_events_by_day"`, "UTC")) + .toEqual(day); + }, 60_000); + + test("JOIN TYPE: a NULLABLE @via FK lowers to LEFT OUTER JOIN, and a fact with a NULL FK lands in a NULL group", async () => { + const root = await loadInline(FACT_MODEL); + const body = viewSql(root, "v_facts_by_prog"); + expect(body).toContain(`LEFT OUTER JOIN "progs"`); + expect(body).not.toContain("INNER JOIN"); + + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + await applyRaw(` + INSERT INTO "progs" ("id","title") VALUES (1, 'Alpha'); + INSERT INTO "facts" ("id","progId") VALUES (1, 1), (2, 1), (3, NULL), (4, NULL), (5, NULL);`); + const rows = await select(`SELECT * FROM "v_facts_by_prog" ORDER BY "progTitle" NULLS LAST`); + expect(rows).toEqual([ + { progTitle: "Alpha", facts: "2" }, + { progTitle: null, facts: "3" }, + ]); + }, 60_000); + + test("CHANGE A REPORT: adding a measure is a drop and a create of that view, and then converges", async () => { + const before = await loadInline(metricModel(["samples"])); + const first = await migrate(before); + await assertConverged(first.expected, first.unmanagedNames); + + const after = await loadInline(metricModel(["samples", "total"])); + // A plain `meta migrate`, no --allow: the drop is PAIRED with the create of the same view, + // which is not a destructive change. Passing dropView here would hide a broken pairing. + const { result, up, expected, unmanagedNames } = await migrate(after); + expect(result.blocked).toEqual([]); + + const viewChanges = result.changes.filter((c) => c.kind.endsWith("-view")); + expect(viewChanges.map((c) => c.kind).sort()).toEqual(["create-view", "drop-view"]); + expect(up).toContain(`DROP VIEW`); + expect(up).toContain(`CREATE VIEW "v_metrics_by_kind" AS`); + // A fail-safe drop and create, not CREATE OR REPLACE (the report carries no column list). + expect(up).not.toContain("CREATE OR REPLACE"); + + const cols = await sql.raw( + `SELECT column_name FROM information_schema.columns WHERE table_name = 'v_metrics_by_kind' ORDER BY ordinal_position`, + ).execute(k); + expect((cols.rows as { column_name: string }[]).map((c) => c.column_name)).toEqual(["kind", "samples", "total"]); + + await assertConverged(expected, unmanagedNames); + const again = await migrate(after); + expect(again.up.trim()).toBe(""); + }, 60_000); + + test("a tuple with a NULL component is not counted, read through the lowered view", async () => { + // The canonical tuple's components are both required, so only an inline model can put a + // NULL through the FILTER guard on an engine. + const root = await loadInline(JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Pair", children: [ + { "source.rdb": { "@table": "pairs" } }, + { "field.long": { name: "id" } }, + { "field.int": { name: "a" } }, + { "field.int": { name: "b" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "combos", "@agg": "count", "@distinct": true, "@of": ["Pair.a", "Pair.b"] } }, + ] } }, + { "object.report": { name: "PairTotals", "@from": "Pair", "@measures": ["combos"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_pair_totals" } } ] } }, + ]}})); + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + await applyRaw(` + INSERT INTO "pairs" ("a","b") VALUES (1, 2), (1, 2), (2, 1), (1, NULL), (NULL, 3), (NULL, NULL);`); + expect(await select(`SELECT * FROM "v_pair_totals"`)).toEqual([{ combos: "2" }]); + }, 60_000); + + test("SUM TYPES (Table C): a decimal sum stays numeric and a double sum is double precision, on the engine", async () => { + const root = await loadInline(JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Reading", children: [ + { "source.rdb": { "@table": "readings" } }, + { "field.long": { name: "id" } }, + { "field.decimal": { name: "amount", "@precision": 12, "@scale": 2 } }, + { "field.double": { name: "score" } }, + { "field.float": { name: "ratio" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "amountTotal", "@agg": "sum", "@of": "Reading.amount" } }, + { "measure.aggregate": { name: "scoreTotal", "@agg": "sum", "@of": "Reading.score" } }, + { "measure.aggregate": { name: "ratioTotal", "@agg": "sum", "@of": "Reading.ratio" } }, + ] } }, + { "object.report": { name: "ReadingTotals", "@from": "Reading", + "@measures": ["amountTotal", "scoreTotal", "ratioTotal"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_reading_totals" } } ] } }, + ]}})); + const body = viewSql(root, "v_reading_totals"); + expect(body).toContain(`SUM(r."amount") AS "amountTotal"`); + expect(body).toContain(`CAST(SUM(r."score") AS DOUBLE PRECISION) AS "scoreTotal"`); + expect(body).toContain(`CAST(SUM(r."ratio") AS DOUBLE PRECISION) AS "ratioTotal"`); + + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + // The view's column types are what Table B promises the readers. + const cols = await sql.raw( + `SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'v_reading_totals' ORDER BY ordinal_position`, + ).execute(k); + expect(cols.rows).toEqual([ + { column_name: "amountTotal", data_type: "numeric" }, + { column_name: "scoreTotal", data_type: "double precision" }, + { column_name: "ratioTotal", data_type: "double precision" }, + ]); + + // Over zero rows every sum is NULL, never 0. + expect(await select(`SELECT * FROM "v_reading_totals"`)).toEqual([ + { amountTotal: null, scoreTotal: null, ratioTotal: null }, + ]); + await applyRaw(`INSERT INTO "readings" ("amount","score","ratio") VALUES (10.25, 1.5, 0.5), (0.50, 2.25, 0.25);`); + const [row] = await select(`SELECT * FROM "v_reading_totals"`); + expect(canonicalDecimal(row!.amountTotal)).toBe("10.75"); + expect(Number(row!.scoreTotal)).toBe(3.75); + expect(Number(row!.ratioTotal)).toBe(0.75); + }, 60_000); +}); diff --git a/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts b/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts new file mode 100644 index 000000000..af9744695 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts @@ -0,0 +1,342 @@ +/** + * Report views — FR-044 Plan 2, against a REAL SQLite (libsql). + * + * The Postgres counterpart is `report-views-pg.test.ts`. SQLite lowers a report + * differently on every axis the contract tables name: conditional aggregates are + * `CASE WHEN`, the tuple distinct count goes through `json_array`, time grains are + * `strftime` / `date` modifiers over ISO-8601 TEXT, and a ratio divides as `REAL`. + * + * CONVERGENCE here is a different claim than on Postgres. SQLite stores a view's SQL + * text verbatim and introspection reads it back, so a second diff is empty only when the + * emitter is DETERMINISTIC and its text survives the round trip byte for byte. + * + * THE INSTANT'S LITERAL (contract Table D, UNVERIFIED item, resolved below). Both + * TypeScript SQLite writers spell an instant `YYYY-MM-DDTHH:MM:SS.sssZ`, with a + * three-digit fraction even when it is zero: + * - the application stamp is `new Date().toISOString()` (`@autoSet`), and + * - the DDL default is `strftime('%Y-%m-%dT%H:%M:%fZ','now')` (migrate-ts + * `SQLITE_ISO_NOW`, which `drizzle-schema.ts` mirrors byte for byte). + * Table D's hour bucket for an instant is `strftime('%Y-%m-%dT%H:00:00.000Z', x)`, which + * has that same spelling, so a bucket is text-comparable with a stored instant. A value + * written without the fraction (`...:00Z`, what a client hands the wire in) is parsed by + * `strftime` identically and lands in the same bucket; only its own stored text differs. + */ + +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { Kysely, sql } from "kysely"; +import { LibsqlDialect } from "@libsql/kysely-libsql"; +import { + buildExpectedSchema, diff, emit, introspectSqlite, type Change, type SchemaSnapshot, +} from "@metaobjectsdev/migrate-ts"; +import { buildProjectionViews } from "@metaobjectsdev/codegen-ts"; +import { MetaDataLoader, InMemoryStringSource, type MetaRoot } from "@metaobjectsdev/metadata"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; + +let tmpDir: string; +let k: Kysely>; + +beforeEach(() => { + tmpDir = mkdtempSync(join(tmpdir(), "report-views-sqlite-")); + k = new Kysely({ dialect: new LibsqlDialect({ url: `file:${join(tmpDir, "test.db")}` }) }); +}); + +afterEach(async () => { + await k.destroy(); + rmSync(tmpDir, { recursive: true, force: true }); +}); + +// libsql execute() is single-statement: split on ";" (no view body or seed carries an inner ";"). +async function applyRaw(text: string): Promise { + for (const stmt of text.trim().split(";").map((s) => s.trim()).filter(Boolean)) { + await sql.raw(stmt).execute(k); + } +} + +async function loadInline(metaJson: string): Promise { + const r = await new MetaDataLoader().load([new InMemoryStringSource(metaJson)]); + // The loader collects errors instead of throwing; a refused inline model must not be migrated. + expect(r.errors).toEqual([]); + return r.root; +} + +function expectedFor(root: MetaRoot): SchemaSnapshot { + return buildExpectedSchema(root, { + dialect: "sqlite", + columnNamingStrategy: "literal", + views: buildProjectionViews(root, { dialect: "sqlite", columnNamingStrategy: "literal" }), + }); +} + +/** + * `field.inet` has no SQLite storage class of its own: the canonical model's `all_types` + * table declares two, SQLite introspects them back as TEXT, and the diff reports a blocked + * `text -> inet` change on every run. That is a property of the table, not of any report + * (nothing here reads `all_types`), and it is the only residue the canonical model leaves. + * It is named, not swallowed: a residual change on any OTHER table or on any view fails. + */ +const INET_COLUMNS: ReadonlySet = new Set(["inetVal", "inet6Val"]); +const isInetResidue = (c: Change): boolean => + c.kind === "change-column-type" && c.table === "all_types" && INET_COLUMNS.has(c.column) && c.to.kind === "inet"; + +/** build -> introspect -> diff -> emit -> apply. */ +async function migrate(root: MetaRoot) { + const expected = expectedFor(root); + const actual = await introspectSqlite(k); + const result = await diff({ expected, actual, dialect: "sqlite" }); + expect(result.blocked.filter((c) => !isInetResidue(c))).toEqual([]); + const { up } = emit(result.changes.filter((c) => !isInetResidue(c)), { + dialect: "sqlite", + expectedSchema: expected, + ...(actual.meta !== undefined && { actualMeta: actual.meta }), + }); + if (up.trim().length > 0) await applyRaw(up); + return { expected, result, up }; +} + +async function assertConverged(expected: SchemaSnapshot): Promise { + const followup = await diff({ expected, actual: await introspectSqlite(k), dialect: "sqlite" }); + const residual = followup.changes.filter((c) => !isInetResidue(c)); + if (residual.length > 0) { + console.error("NOT CONVERGED (sqlite) — a further migrate would emit:"); + for (const c of residual) console.error(" -", c.kind, JSON.stringify(c).slice(0, 200)); + } + expect(residual).toEqual([]); +} + +async function select(query: string): Promise[]> { + return (await sql.raw(query).execute(k)).rows as Record[]; +} + +function viewSql(root: MetaRoot, name: string): string { + const v = buildProjectionViews(root, { dialect: "sqlite", columnNamingStrategy: "literal" }) + .find((x) => x.name === name); + if (v === undefined) throw new Error(`no view ${name}`); + return v.sql; +} + +const SEED_PROGRAMS_AND_WEEKS = ` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45)`; + +const SEED_PROGRAMS_BY_TIME = ` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00'), + (4, 'Legacy', 700, 'ARCHIVED', '2026-05-31T23:59:59'), + (5, 'Monday', 300, 'PUBLISHED', '2026-05-18T00:00:00')`; + +const OWNER = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"; + +describe("report views — canonical model on real SQLite", () => { + let canonical: MetaRoot; + let expected: SchemaSnapshot; + + beforeEach(async () => { + canonical = await loadMetadataDir(CANONICAL_DIR); + ({ expected } = await migrate(canonical)); + }); + + test("CONVERGENCE: the six canonical views apply, then a second and third migrate propose nothing (the emitter is deterministic)", async () => { + const views = (await select(`SELECT name FROM sqlite_master WHERE type = 'view' ORDER BY name`)).map((r) => r.name); + for (const v of [ + "v_program_minutes", "v_fitness_totals", "v_programs_by_month", + "v_programs_by_week", "v_recent_programs", "v_asset_activity", + ]) { + expect(views).toContain(v); + } + + await assertConverged(expected); + const second = await migrate(canonical); + expect(second.up.trim()).toBe(""); + expect(second.result.changes.filter((c) => !isInetResidue(c))).toEqual([]); + const third = await migrate(canonical); + expect(third.up.trim()).toBe(""); + }); + + test("SQLite lowering shapes: CASE WHEN conditions, json_array tuple, REAL ratio, INNER JOIN", () => { + const body = viewSql(canonical, "v_program_minutes"); + expect(body).toContain(`COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS "longWeeks"`); + expect(body).toContain(`json_array(w."programId", w."durationMinutes")`); + expect(body).toContain(`CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0)`); + expect(body).toContain(`INNER JOIN "programs" p ON p."id" = w."programId"`); + }); + + test("v_program_minutes: every measure kind, incl. the tuple distinct count through json_array", async () => { + await applyRaw(SEED_PROGRAMS_AND_WEEKS); + const rows = await select(`SELECT * FROM "v_program_minutes" ORDER BY "program"`); + // SQLite has no decimal type: avg and the ratio are REAL (numbers), not decimal strings. + expect(rows).toEqual([ + { program: 1, programTitle: "Foundations", weeks: 4, longWeeks: 3, labels: 2, slots: 3, + totalMinutes: 240, avgMinutes: 60, minMinutes: 30, maxMinutes: 90, longShare: 0.75 }, + { program: 2, programTitle: "Strength", weeks: 1, longWeeks: 0, labels: 1, slots: 1, + totalMinutes: 45, avgMinutes: 45, minMinutes: 45, maxMinutes: 45, longShare: 0 }, + ]); + // Program 1 has four rows but three distinct (programId, durationMinutes) tuples, and + // its labels are 'Week 1', 'Week 2' and one NULL: two distinct, the null uncounted. + expect(await select(`SELECT "program" FROM "v_program_minutes" WHERE "weeks" >= 2`)).toEqual([{ program: 1 }]); + expect(await select(`SELECT "program" FROM "v_program_minutes" ORDER BY "totalMinutes" DESC LIMIT 1`)).toEqual([{ program: 1 }]); + }); + + test("v_fitness_totals: no dimensions, one row; ratio is REAL", async () => { + await applyRaw(SEED_PROGRAMS_AND_WEEKS); + expect(await select(`SELECT * FROM "v_fitness_totals"`)).toEqual([{ weeks: 5, totalMinutes: 285, longShare: 0.6 }]); + }); + + test("EMPTY GROUPS (Review Focus 4): v_fitness_totals over an empty weeks table is one row (0, NULL, NULL)", async () => { + expect(await select(`SELECT count(*) AS n FROM "weeks"`)).toEqual([{ n: 0 }]); + expect(await select(`SELECT * FROM "v_fitness_totals"`)).toEqual([{ weeks: 0, totalMinutes: null, longShare: null }]); + }); + + test("v_programs_by_month / v_programs_by_week: month grain, null filtered sum, ISO Monday boundary, report @segment", async () => { + await applyRaw(SEED_PROGRAMS_BY_TIME); + expect(await select(`SELECT * FROM "v_programs_by_month" ORDER BY "status"`)).toEqual([ + { createdAtMonth: "2026-05-01", status: "ARCHIVED", programs: 1, listValue: null }, + { createdAtMonth: "2026-06-01", status: "DRAFT", programs: 1, listValue: null }, + { createdAtMonth: "2026-05-01", status: "PUBLISHED", programs: 3, listValue: 7799 }, + ]); + expect(await select(`SELECT * FROM "v_programs_by_week" ORDER BY "createdAtWeek"`)).toEqual([ + { createdAtWeek: "2026-04-27", programs: 1 }, + { createdAtWeek: "2026-05-11", programs: 1 }, + { createdAtWeek: "2026-05-18", programs: 1 }, + ]); + }); + + test("WEEK BOUNDARY: 2026-05-17T23:30:00 (Sunday) is in the week of 2026-05-11; 2026-05-18T00:00:00 (Monday) opens 2026-05-18", async () => { + const r = await select( + `SELECT date(t, 'weekday 0', '-6 days') AS wk FROM + (SELECT '2026-05-17T23:30:00' AS t UNION ALL SELECT '2026-05-18T00:00:00' ORDER BY 1)`, + ); + expect(r).toEqual([{ wk: "2026-05-11" }, { wk: "2026-05-18" }]); + // And through the view's own expression, on the programs seeded exactly there. + await applyRaw(` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Sunday', 1, 'PUBLISHED', '2026-05-17T23:30:00'), + (2, 'Monday', 1, 'PUBLISHED', '2026-05-18T00:00:00')`); + expect(await select(`SELECT "createdAtWeek" FROM "v_programs_by_week" ORDER BY 1`)) + .toEqual([{ createdAtWeek: "2026-05-11" }, { createdAtWeek: "2026-05-18" }]); + }); + + test("v_asset_activity: hour on an instant, week on a field.date; the hour bucket's literal is `…:00:00.000Z`", async () => { + // Instants are stored the way the TS writers spell them: a three-digit fraction + Z. + await applyRaw(` + INSERT INTO "assets" ("id","ownerId","recordedAt","observedAt","asOfDate","atTime") VALUES + ('00000000-0000-4000-8000-000000000001', '${OWNER}', '2026-05-04T03:30:00.000Z', '2026-05-04T03:30:00.000', '2026-05-03', '03:30:00'), + ('00000000-0000-4000-8000-000000000002', '${OWNER}', '2026-05-04T03:45:00.000Z', '2026-05-04T03:45:00.000', '2026-05-03', '03:45:00'), + ('00000000-0000-4000-8000-000000000003', '${OWNER}', '2026-05-04T04:10:00.000Z', '2026-05-04T04:10:00.000', '2026-05-04', '04:10:00')`); + const rows = await select(`SELECT * FROM "v_asset_activity" ORDER BY "recordedAtHour"`); + expect(rows).toEqual([ + { recordedAtHour: "2026-05-04T03:00:00.000Z", asOfDateWeek: "2026-04-27", assets: 2 }, + { recordedAtHour: "2026-05-04T04:00:00.000Z", asOfDateWeek: "2026-05-04", assets: 1 }, + ]); + + // PIN: the bucket has the same spelling as every instant the TS adapters store, so it + // sorts and compares as text against them. Both writers' spellings, derived live: + const written = await select(`SELECT strftime('%Y-%m-%dT%H:%M:%fZ', '2026-05-04T03:00:00Z') AS ddlDefault`); + expect(written[0]?.ddlDefault).toBe("2026-05-04T03:00:00.000Z"); // migrate-ts SQLITE_ISO_NOW + expect(new Date("2026-05-04T03:00:00Z").toISOString()).toBe("2026-05-04T03:00:00.000Z"); // @autoSet + expect(rows[0]?.recordedAtHour).toBe(written[0]?.ddlDefault as string); + expect(rows[0]?.recordedAtHour).toBe(new Date("2026-05-04T03:00:00Z").toISOString()); + + // A value stored WITHOUT the fraction (an unpadded wire string) still buckets the same. + await applyRaw(` + INSERT INTO "assets" ("id","ownerId","recordedAt","observedAt","asOfDate","atTime") VALUES + ('00000000-0000-4000-8000-000000000004', '${OWNER}', '2026-05-04T03:59:59Z', '2026-05-04T03:59:59', '2026-05-04', '03:59:59')`); + expect(await select(`SELECT "assets" FROM "v_asset_activity" WHERE "recordedAtHour" = '2026-05-04T03:00:00.000Z' AND "asOfDateWeek" = '2026-05-04'`)) + .toEqual([{ assets: 1 }]); + }); + + test("RELATIVE WINDOW: v_recent_programs counts the program 3 days old, not the one 60 days old", async () => { + // created_ts is a naive timestamp, stored as a naive wall clock (no Z). + await applyRaw(` + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Recent', 100, 'PUBLISHED', strftime('%Y-%m-%dT%H:%M:%f','now','-3 days')), + (2, 'Stale', 100, 'PUBLISHED', strftime('%Y-%m-%dT%H:%M:%f','now','-60 days'))`); + expect(await select(`SELECT * FROM "v_recent_programs"`)).toEqual([{ programs: 1 }]); + }); +}); + +describe("report views — inline model on real SQLite", () => { + /** A Stamp table with date and naive-timestamp columns for the quarter / year grains. */ + const STAMP_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Stamp", children: [ + { "source.rdb": { "@table": "stamps" } }, + { "field.long": { name: "id" } }, + { "field.date": { name: "day", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "dimension.time": { name: "day", "@of": "Stamp.day", "@grains": ["quarter", "year"] } }, + { "measure.aggregate": { name: "stamps", "@agg": "count", "@of": "Stamp.id" } }, + ] } }, + { "object.report": { name: "StampsByQuarter", "@from": "Stamp", "@dimensions": ["day:quarter"], "@measures": ["stamps"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_stamps_by_quarter" } } ] } }, + { "object.report": { name: "StampsByYear", "@from": "Stamp", "@dimensions": ["day:year"], "@measures": ["stamps"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_stamps_by_year" } } ] } }, + ]}}); + + /** A tuple distinct count whose two components are both NULLABLE (the canonical model's are required). */ + const PAIR_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Pair", children: [ + { "source.rdb": { "@table": "pairs" } }, + { "field.long": { name: "id" } }, + { "field.int": { name: "a" } }, + { "field.int": { name: "b" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "combos", "@agg": "count", "@distinct": true, "@of": ["Pair.a", "Pair.b"] } }, + ] } }, + { "object.report": { name: "PairTotals", "@from": "Pair", "@measures": ["combos"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_pair_totals" } } ] } }, + ]}}); + + test("a tuple with a NULL component is not counted, read through the lowered view", async () => { + const root = await loadInline(PAIR_MODEL); + expect(viewSql(root, "v_pair_totals")).toContain( + `COUNT(DISTINCT CASE WHEN p."a" IS NOT NULL AND p."b" IS NOT NULL THEN json_array(p."a", p."b") END)`, + ); + const { expected } = await migrate(root); + await assertConverged(expected); + // (1,2) twice, (2,1) once, and three rows with a NULL component: two distinct tuples. + await applyRaw(` + INSERT INTO "pairs" ("a","b") VALUES + (1, 2), (1, 2), (2, 1), (1, NULL), (NULL, 3), (NULL, NULL)`); + expect(await select(`SELECT * FROM "v_pair_totals"`)).toEqual([{ combos: 2 }]); + }); + + test("QUARTER and YEAR grains: every month of a quarter lands on its first day, and the view converges", async () => { + const root = await loadInline(STAMP_MODEL); + expect(viewSql(root, "v_stamps_by_quarter")).toContain( + `date(s."day", 'start of month', '-' || ((CAST(strftime('%m', s."day") AS INTEGER) - 1) % 3) || ' months')`, + ); + const { expected } = await migrate(root); + await assertConverged(expected); + + await applyRaw(` + INSERT INTO "stamps" ("day") VALUES + ('2026-01-01'), ('2026-03-31'), + ('2026-04-01'), ('2026-05-01'), ('2026-06-30'), + ('2026-07-01'), + ('2026-12-31'), + ('2027-02-14')`); + expect(await select(`SELECT * FROM "v_stamps_by_quarter" ORDER BY "dayQuarter"`)).toEqual([ + { dayQuarter: "2026-01-01", stamps: 2 }, + { dayQuarter: "2026-04-01", stamps: 3 }, + { dayQuarter: "2026-07-01", stamps: 1 }, + { dayQuarter: "2026-10-01", stamps: 1 }, + { dayQuarter: "2027-01-01", stamps: 1 }, + ]); + expect(await select(`SELECT * FROM "v_stamps_by_year" ORDER BY "dayYear"`)).toEqual([ + { dayYear: "2026-01-01", stamps: 7 }, + { dayYear: "2027-01-01", stamps: 1 }, + ]); + }); +}); diff --git a/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts b/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts index af3a6e324..58e118f71 100644 --- a/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts +++ b/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts @@ -8,6 +8,7 @@ import { OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, } from "../object/object-constants.js"; +import { CHILD_REF_SEPARATOR } from "../../shared/structural.js"; import { REPORT_DIMENSION_GRAIN_SEPARATOR } from "./reporting-constants.js"; export interface ReportDimensionItem { @@ -34,11 +35,30 @@ export function reportDimensionItems(obj: MetaData): ReportDimensionItem[] { }); } -/** The `@measures` names. */ +/** The `@measures` items AS WRITTEN: each a bare measure `name`, or a dotted + * `Entity.name` (loader rule R3). Use {@link reportMeasureItemName} for the measure name. */ export function reportMeasureNames(obj: MetaData): string[] { return stringList(obj.attr(OBJECT_REPORT_ATTR_MEASURES)); } +/** + * The measure a `@measures` item names: the segment after its LAST `.` + * (`total`, `Sale.total` and `acme::shop::Sale.total` all name `total`). It is also + * the derived report field's name. The part before that `.`, when present, is an + * entity qualifier, which {@link reportMeasureItemOwner} returns. + */ +export function reportMeasureItemName(item: string): string { + const dot = item.lastIndexOf(CHILD_REF_SEPARATOR); + return dot === -1 ? item : item.slice(dot + CHILD_REF_SEPARATOR.length); +} + +/** The entity qualifier of a dotted `@measures` item (`Sale` in `Sale.total`), or + * undefined for a bare item. Loader rule R3: it names `@from` or an entity `@from` extends. */ +export function reportMeasureItemOwner(item: string): string | undefined { + const dot = item.lastIndexOf(CHILD_REF_SEPARATOR); + return dot === -1 ? undefined : item.slice(0, dot); +} + /** The derived report field for a dimension item: `name` (attribute) or `name` + Capitalized(grain) (time). */ export function reportDerivedFieldName(item: ReportDimensionItem): string { if (item.grain === undefined || item.grain === "") return item.name; diff --git a/server/typescript/packages/metadata/src/core/reporting/report-read-model.ts b/server/typescript/packages/metadata/src/core/reporting/report-read-model.ts new file mode 100644 index 000000000..d499bc23c --- /dev/null +++ b/server/typescript/packages/metadata/src/core/reporting/report-read-model.ts @@ -0,0 +1,163 @@ +// A report's READ MODEL (FR-044): a detached object carrying one real `field.*` +// child per derived field (Table B) and a copy of the report's own read-only source. +// +// WHY IT EXISTS +// +// An `object.report` declares no fields: its read shape is derived from its +// dimensions and measures. A metadata-driven runtime walks an object's field +// children in a dozen places (column list, filter and sort resolution, the name +// map, every read coercion). Rather than teach each of them what a report is, the +// runtime reads a report through this model and sees ordinary fields. +// +// WHY IT IS DETACHED +// +// The model is never added to the root: it has no parent, `root.objects()` does +// not list it, and the canonical serializer, `fmt`, codegen and every other tree +// walker never see it. Nothing in the loaded tree is mutated to build it; in +// particular the report's own source node is COPIED, not re-parented +// (`addChild` rewrites the child's parent). The nodes are constructed directly, +// not through the registry, so the sealed registry is not involved and no +// vocabulary is added: every node is an already-registered `type.subType`. +// +// It keeps the report's name, package and `object.report` subtype, so a consumer +// holding it can still tell it is a report (no identity, read-only). + +import { TypeId } from "../../registry.js"; +import { TYPE_FIELD } from "../../shared/base-types.js"; +import type { MetaRoot } from "../../shared/meta-root.js"; +import { isReadOnlySource } from "../../shared/node-guards.js"; +import { MetaSource } from "../../persistence/source/meta-source.js"; +import { SOURCE_ATTR_ROLE, SOURCE_ROLE_PRIMARY } from "../../persistence/source/source-constants.js"; +import { FIELD_ATTR_DB_COLUMN_TYPE, FIELD_ATTR_LOCAL_TIME } from "../../persistence/db/db-constants.js"; +import { MetaObject } from "../object/meta-object.js"; +import { MetaField } from "../field/meta-field.js"; +import { + FIELD_ATTR_CURRENCY, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_PRECISION, + FIELD_ATTR_REQUIRED, + FIELD_ATTR_SCALE, + FIELD_ATTR_STORAGE, + FIELD_ATTR_VALUES, +} from "../field/field-constants.js"; +import { reportShape, type ReportField } from "./report-shape.js"; + +/** + * Table B: the type-shaping attrs a derived field carries from its `typeSource`, + * read with the RESOLVING accessor (ADR-0039) so a value the `@of` field inherits + * through `extends` is carried too. `@dbColumnType` and `isArray` are handled + * separately below. Nothing else is carried: no `@column`, `@required`, + * `@default`, validators or views. + */ +const CARRIED_ATTRS = [ + FIELD_ATTR_CURRENCY, + FIELD_ATTR_VALUES, + FIELD_ATTR_INT_VALUE_MAP, + FIELD_ATTR_MAX_LENGTH, + FIELD_ATTR_PRECISION, + FIELD_ATTR_SCALE, + FIELD_ATTR_LOCAL_TIME, + FIELD_ATTR_OBJECT_REF, + FIELD_ATTR_STORAGE, +] as const; + +function derivedField(f: ReportField): MetaField { + const field = new MetaField(new TypeId(TYPE_FIELD, f.subType), f.name); + // From the derived shape, never from the type source: a `min` of a required column + // is still nullable, and a dimension reached by `@via` is nullable. + field.setAttr(FIELD_ATTR_REQUIRED, f.required); + const src = f.typeSource; + if (src !== undefined) { + for (const name of CARRIED_ATTRS) { + const value = src.attr(name); + if (value !== undefined) field.setAttr(name, value); + } + // ADR-0039: own — `@dbColumnType` is the one deliberately own-only attr (a + // physical column-type override is never inherited), and every consumer reads + // it with `ownAttr`. So it is read own from the type source and set OWN here: + // the derived field carries exactly what the `@of` field itself declares, and + // nothing its supers declare. A field that `extends` another does not get the + // parent's `@dbColumnType` either, so this matches how a projection field + // would see it. + const dbColumnType = src.ownAttr(FIELD_ATTR_DB_COLUMN_TYPE); + if (dbColumnType !== undefined) field.setAttr(FIELD_ATTR_DB_COLUMN_TYPE, dbColumnType); + // `isArray` is a native flag, not an attr; resolvedIsArray() is its resolving read. + if (src.resolvedIsArray()) field.setIsArray(true); + } + return field; +} + +/** + * The source a report is READ from: its own read-only source with `@role: primary`, + * else its first own read-only source. Undefined when it declares none (Table A: + * not lowered, not served). + * + * This is the rule that NAMES the lowered view — `viewName` / `projectionViewSource` + * in codegen-ts's `projection/extract-view-spec.ts`, reached for a report through + * `projectionViewName`. It is restated here because the metadata package cannot + * depend on a codegen package; the two must stay the same rule, or the runtime + * reads a relation the lowering did not create. + * + * What the loader permits, measured: a report may declare several read-only sources + * (a `@role: replica` view beside its primary view loads clean, in either order); + * `@role` defaults to `primary`; a report whose sources include no primary is + * `ERR_SOURCE_NO_PRIMARY` and a writable source on a report is refused. So for every + * model that loads, the primary branch fires. The first-read-only fallback covers a + * tree built in code, and keeps this rule identical to the lowering's. + */ +export function reportReadSource(report: MetaObject): MetaSource | undefined { + // ADR-0039: own — source classification reads the sources the report declares + // ITSELF, exactly as the lowering's `viewName` does. + const readOnly = report.ownChildren().filter(isReadOnlySource); + return readOnly.find((s) => s.role === SOURCE_ROLE_PRIMARY) ?? readOnly[0]; +} + +/** + * A detached copy of a source node: same `type.subType`, name and effective attrs, + * and nothing else (attrs only — the loaded node is never re-parented). + * + * The copy is the model's ONLY source, and it is pinned to `@role: primary`: the + * runtime resolves an object's table through `primaryRdbSource`, which considers + * primary sources only, so this is what makes the read land on the selected + * source's physical name rather than on a default table name nobody declared. + */ +function copySource(source: MetaSource): MetaSource { + const copy = new MetaSource(source.typeId, source.name); + // ADR-0039: resolving — the copy carries the source's effective configuration + // (@kind, the physical-name alias, @schema, @unmanaged, @sql). + for (const [name, value] of source.attrs()) copy.setAttr(name, value); + copy.setAttr(SOURCE_ATTR_ROLE, SOURCE_ROLE_PRIMARY); + return copy; +} + +const READ_MODELS = new WeakMap(); + +/** + * The read model of an `object.report`: one field per Table B row, in Table B + * order, plus a copy of the source the report is read from (see + * `reportReadSource`) when it declares one (Table A). A sourceless report yields a model with no source: it has a shape + * and no view, and the caller decides what that means (the runtime refuses to + * serve it). + * + * Cached per report node; the model is frozen. Throws what `reportShape` throws + * when a reference does not resolve. + */ +export function reportReadModel(report: MetaObject, root: MetaRoot): MetaObject { + const cached = READ_MODELS.get(report); + if (cached !== undefined) return cached; + + const shape = reportShape(report, root); + const model = new MetaObject(report.typeId, report.name); + if (report.package !== undefined) model.setPackage(report.package); + if (report.fileDefaultPackage !== undefined) model.setFileDefaultPackage(report.fileDefaultPackage); + for (const f of shape.fields) model.addChild(derivedField(f)); + + const source = reportReadSource(report); + if (source !== undefined) model.addChild(copySource(source)); + + model.freeze(); + READ_MODELS.set(report, model); + return model; +} diff --git a/server/typescript/packages/metadata/src/core/reporting/report-shape.ts b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts new file mode 100644 index 000000000..e7c04ac27 --- /dev/null +++ b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts @@ -0,0 +1,246 @@ +// Table B of docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md: +// a report's derived fields. The single definition; every port has a rule-for-rule copy, +// gated by fixtures/persistence-conformance/report-shapes.json. + +import type { MetaData } from "../../shared/meta-data.js"; +import type { MetaRoot } from "../../shared/meta-root.js"; +import { isMetaObject } from "../../shared/node-guards.js"; +import { resolveObjectRef } from "../../naming-refs.js"; +import { CHILD_REF_SEPARATOR, PACKAGE_SEPARATOR } from "../../shared/structural.js"; +import type { MetaObject } from "../object/meta-object.js"; +import type { MetaField } from "../field/meta-field.js"; +import { + FIELD_ATTR_REQUIRED, + FIELD_SUBTYPE_CURRENCY, + FIELD_SUBTYPE_DATE, + FIELD_SUBTYPE_DECIMAL, + FIELD_SUBTYPE_DOUBLE, + FIELD_SUBTYPE_FLOAT, + FIELD_SUBTYPE_INT, + FIELD_SUBTYPE_LONG, + FIELD_SUBTYPE_TIMESTAMP, +} from "../field/field-constants.js"; +import { MetaDimension } from "./meta-dimension.js"; +import { MetaMeasure } from "./meta-measure.js"; +import { + reportDerivedFieldName, + reportDimensionItems, + reportFrom, + reportMeasureItemName, + reportMeasureItemOwner, + reportMeasureNames, +} from "./report-accessors.js"; +import { + AGG_AVG, + AGG_COUNT, + AGG_SUM, + GRAIN_HOUR, + TIME_GRAINS, + TYPE_DIMENSION, + TYPE_MEASURE, + type TimeGrain, +} from "./reporting-constants.js"; + +export type ReportFieldRole = "dimension" | "measure"; + +export interface ReportField { + readonly name: string; + readonly role: ReportFieldRole; + /** A field subtype name (FIELD_SUBTYPE_*). */ + readonly subType: string; + readonly required: boolean; + /** The `@of` field whose type-shaping attrs this field carries (Table B). */ + readonly typeSource?: MetaField; + readonly dimension?: MetaDimension; + readonly grain?: TimeGrain; + readonly measure?: MetaMeasure; +} + +export interface ReportShape { + readonly report: MetaObject; + readonly from: MetaObject; + readonly fields: readonly ReportField[]; +} + +const SUM_LONG: ReadonlySet = new Set([FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG]); +const FLOATING: ReadonlySet = new Set([FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT]); + +/** Effective package of a node, taken from its resolution key ("::"). */ +function packageOfKey(key: string): string { + const i = key.lastIndexOf(PACKAGE_SEPARATOR); + return i >= 0 ? key.slice(0, i) : ""; +} + +/** True when `candidate` is `entity` or an entity it extends (the super chain). */ +function isSelfOrAncestor(candidate: MetaData, entity: MetaData): boolean { + const visited = new Set(); + for (let n: MetaData | undefined = entity; n !== undefined && !visited.has(n); n = n.superData) { + if (n === candidate) return true; + visited.add(n); + } + return false; +} + +/** + * The entity that DECLARES a dimension, measure or segment reached through `from`: the + * member's parent, which is `from` itself or an entity `from` extends. A bare entity name + * inside the member (`@of`, `@via`) resolves in THIS entity's package, exactly as the + * loader's `validateReporting` resolves it (`pkgOf(ctx.declaring)`), never in `from`'s + * package or the report's. + */ +export function reportingMemberOwner(member: MetaData, from: MetaObject): MetaData { + return member.parent ?? from; +} + +/** + * Resolve a dimension's or measure's `Entity.field` reference to the field node. The ONE + * rule, the same as the loader's (`validateReporting` D1 / M1): + * + * 1. The entity half resolves relative to the package of `declaring`, the entity that + * declares the member ({@link reportingMemberOwner}). + * 2. With `host` (a measure, or a dimension without `@via`: the reference is about the + * `@from` entity's own rows) the named entity must be `host` or an entity it extends, + * and the field is read from `host`, so a field `host` redeclares wins. + * 3. Without `host` (a dimension with `@via`) the field is read from the named entity. + * + * Undefined when any step fails. + */ +export function resolveReportingFieldRef( + ref: string, + declaring: MetaData, + root: MetaRoot, + host?: MetaObject, +): MetaField | undefined { + // `Entity.field`; a package qualifier uses `::`, so the member separator is the LAST dot. + const dot = ref.lastIndexOf(CHILD_REF_SEPARATOR); + if (dot <= 0) return undefined; + const named = resolveObjectRef(root, ref.slice(0, dot), packageOfKey(declaring.resolutionKey())).node; + if (!isMetaObject(named)) return undefined; + if (host !== undefined && !isSelfOrAncestor(named, host)) return undefined; + // ADR-0039: resolving, so a field inherited through extends is found. + return (host ?? named).fields().find((f) => f.name === ref.slice(dot + 1)); +} + +/** + * The hop names of a dimension's `@via` (`Owner.hop[.hop...]`), read as the loader reads it + * (`validateReporting` rule D2): `Owner` resolves in the package of `declaring` + * ({@link reportingMemberOwner}) and must be `from` or an entity `from` extends. The walk + * itself then starts AT `from`, whichever of the two `Owner` named. Undefined when the + * reference has no owner, no hop, or an owner that is not `from` or an ancestor of it. + */ +export function reportingViaHops( + via: string, + declaring: MetaData, + from: MetaObject, + root: MetaRoot, +): string[] | undefined { + // The owner ends at the first `.` after the last `::` (a package qualifier has no `.`). + const lastSep = via.lastIndexOf(PACKAGE_SEPARATOR); + const segStart = lastSep === -1 ? 0 : lastSep + PACKAGE_SEPARATOR.length; + const dot = via.indexOf(CHILD_REF_SEPARATOR, segStart); + if (dot <= segStart) return undefined; + const hops = via.slice(dot + CHILD_REF_SEPARATOR.length).split(CHILD_REF_SEPARATOR); + if (hops.some((h) => h === "")) return undefined; + const owner = resolveObjectRef(root, via.slice(0, dot), packageOfKey(declaring.resolutionKey())).node; + if (owner === undefined || !isSelfOrAncestor(owner, from)) return undefined; + return hops; +} + +function unresolved(reportName: string, what: string): Error { + return new Error(`report '${reportName}': ${what} does not resolve.`); +} + +function declaredMember( + from: MetaObject, + type: string, + name: string, + cls: new (...args: never[]) => T, +): T | undefined { + // ADR-0039: resolving children(), so a member declared on an abstract base is found. + return from.children().find((c): c is T => c.type === type && c.name === name && c instanceof cls); +} + +function isTimeGrain(grain: string | undefined): grain is TimeGrain { + return grain !== undefined && (TIME_GRAINS as readonly string[]).includes(grain); +} + +function dimensionField( + item: { name: string; grain?: string }, + from: MetaObject, + root: MetaRoot, + reportName: string, +): ReportField { + const dim = declaredMember(from, TYPE_DIMENSION, item.name, MetaDimension); + if (dim === undefined) throw unresolved(reportName, `dimension '${item.name}' on '${from.name}'`); + const vialess = dim.via() === undefined; + const of = resolveReportingFieldRef(dim.of() ?? "", reportingMemberOwner(dim, from), root, vialess ? from : undefined); + if (of === undefined) throw unresolved(reportName, `dimension '${item.name}' @of`); + const name = reportDerivedFieldName(item); + const required = vialess && of.attr(FIELD_ATTR_REQUIRED) === true; + if (dim.isTime()) { + // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. + const grain = item.grain; + if (!isTimeGrain(grain)) throw unresolved(reportName, `time dimension '${item.name}' grain '${grain ?? ""}'`); + if (grain === GRAIN_HOUR) { + return { name, role: "dimension", subType: FIELD_SUBTYPE_TIMESTAMP, required, typeSource: of, dimension: dim, grain }; + } + return { name, role: "dimension", subType: FIELD_SUBTYPE_DATE, required, dimension: dim, grain }; + } + return { name, role: "dimension", subType: of.subType, required, typeSource: of, dimension: dim }; +} + +/** + * One `@measures` item, bare (`total`) or dotted (`Sale.total`, loader rule R3). The measure + * is named by the item's last segment and looked up on `from`; a qualifier resolves in the + * REPORT's package and must be `from` or an entity `from` extends. + */ +function measureField(item: string, report: MetaObject, from: MetaObject, root: MetaRoot): ReportField { + const reportName = report.name; + const name = reportMeasureItemName(item); + const qualifier = reportMeasureItemOwner(item); + if (qualifier !== undefined) { + const owner = resolveObjectRef(root, qualifier, packageOfKey(report.resolutionKey())).node; + if (owner === undefined || !isSelfOrAncestor(owner, from)) { + throw unresolved(reportName, `measure '${item}' on '${from.name}'`); + } + } + const m = declaredMember(from, TYPE_MEASURE, name, MetaMeasure); + if (m === undefined) throw unresolved(reportName, `measure '${item}' on '${from.name}'`); + if (m.isRatio()) { + return { name, role: "measure", subType: FIELD_SUBTYPE_DECIMAL, required: false, measure: m }; + } + const agg = m.agg(); + if (agg === AGG_COUNT) { + return { name, role: "measure", subType: FIELD_SUBTYPE_LONG, required: true, measure: m }; + } + const of = resolveReportingFieldRef(m.ofColumns()[0] ?? "", reportingMemberOwner(m, from), root, from); + if (of === undefined) throw unresolved(reportName, `measure '${name}' @of`); + const src = of.subType; + if (agg === AGG_SUM) { + if (src === FIELD_SUBTYPE_CURRENCY) { + return { name, role: "measure", subType: FIELD_SUBTYPE_CURRENCY, required: false, typeSource: of, measure: m }; + } + const subType = SUM_LONG.has(src) ? FIELD_SUBTYPE_LONG : FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; + return { name, role: "measure", subType, required: false, measure: m }; + } + if (agg === AGG_AVG) { + const subType = FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; + return { name, role: "measure", subType, required: false, measure: m }; + } + // min / max keep the source field's type. + return { name, role: "measure", subType: src, required: false, typeSource: of, measure: m }; +} + +/** Table B. Throws a plain Error naming the report when a reference does not resolve + * (a report that passed `validateReporting` always resolves). */ +export function reportShape(report: MetaObject, root: MetaRoot): ReportShape { + const fromName = reportFrom(report); + if (fromName === undefined) throw unresolved(report.name, "@from"); + const from = resolveObjectRef(root, fromName, packageOfKey(report.resolutionKey())).node; + if (!isMetaObject(from)) throw unresolved(report.name, `@from '${fromName}'`); + const fields = [ + ...reportDimensionItems(report).map((item) => dimensionField(item, from, root, report.name)), + ...reportMeasureNames(report).map((item) => measureField(item, report, from, root)), + ]; + return { report, from, fields }; +} diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index bd01d7b6c..19364d602 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -50,9 +50,21 @@ export { reportFrom, reportDimensionItems, reportMeasureNames, + reportMeasureItemName, + reportMeasureItemOwner, reportDerivedFieldName, type ReportDimensionItem, } from "./core/reporting/report-accessors.js"; +export { + reportShape, + reportingMemberOwner, + reportingViaHops, + resolveReportingFieldRef, + type ReportField, + type ReportFieldRole, + type ReportShape, +} from "./core/reporting/report-shape.js"; +export { reportReadModel, reportReadSource } from "./core/reporting/report-read-model.js"; // Shared `@implementedBy` resolution — one resolver for the CLI's requirement // checks and codegen's requirement-test fan-out (FR-038). export { diff --git a/server/typescript/packages/metadata/test/report-read-model.test.ts b/server/typescript/packages/metadata/test/report-read-model.test.ts new file mode 100644 index 000000000..a552a03c5 --- /dev/null +++ b/server/typescript/packages/metadata/test/report-read-model.test.ts @@ -0,0 +1,226 @@ +import { describe, expect, test } from "bun:test"; +import { join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; +import { + FIELD_ATTR_COLUMN, + FIELD_ATTR_CURRENCY, + FIELD_ATTR_LOCAL_TIME, + FIELD_ATTR_REQUIRED, + FIELD_ATTR_VALUES, + InMemoryStringSource, + MetaDataLoader, + OBJECT_SUBTYPE_REPORT, + SOURCE_KIND_VIEW, + TYPE_FIELD, + TYPE_IDENTITY, + canonicalSerialize, + isMetaSource, + loadUris, + reportReadModel, + resolveTableName, + type MetaObject, + type MetaRoot, +} from "../src/index.js"; + +const REPO_ROOT = resolve(import.meta.dir, "..", "..", "..", "..", ".."); +const MODEL = join(REPO_ROOT, "fixtures", "persistence-conformance", "canonical", "meta.fitness.json"); + +async function load(): Promise { + const result = await loadUris([pathToFileURL(MODEL).href]); + expect(result.errors).toEqual([]); + return result.root; +} +const object = (root: MetaRoot, name: string): MetaObject => { + const found = root.objects().find((o) => o.name === name); + if (found === undefined) throw new Error(`no object ${name}`); + return found; +}; +const model = (root: MetaRoot, name: string): MetaObject => reportReadModel(object(root, name), root); +const fieldsOf = (m: MetaObject) => m.children().filter((c) => c.type === TYPE_FIELD); + +describe("reportReadModel (FR-044 Table B as a detached read model)", () => { + test("ProgramMinutes has eleven field children in Table B order with the Table B subtypes", async () => { + const root = await load(); + expect(fieldsOf(model(root, "ProgramMinutes")).map((f) => [f.name, f.subType])).toEqual([ + ["program", "long"], + ["programTitle", "string"], + ["weeks", "long"], + ["longWeeks", "long"], + ["labels", "long"], + ["slots", "long"], + ["totalMinutes", "long"], + ["avgMinutes", "decimal"], + ["minMinutes", "int"], + ["maxMinutes", "int"], + ["longShare", "decimal"], + ]); + }); + + test("min keeps the source field's subtype: minMinutes is field.int", async () => { + const root = await load(); + const min = model(root, "ProgramMinutes").fields().find((f) => f.name === "minMinutes"); + expect(min?.type).toBe(TYPE_FIELD); + expect(min?.subType).toBe("int"); + }); + + test("@required comes from the derived shape, not from the type source", async () => { + const root = await load(); + const m = model(root, "ProgramMinutes"); + const required = Object.fromEntries(m.fields().map((f) => [f.name, f.attr(FIELD_ATTR_REQUIRED)])); + expect(required["program"]).toBe(true); // no @via, @of required + expect(required["programTitle"]).toBe(false); // reached by @via + expect(required["weeks"]).toBe(true); // a count is never null + expect(required["minMinutes"]).toBe(false); // Week.durationMinutes is required; a min is not + }); + + test("a currency sum carries @currency from its type source", async () => { + const root = await load(); + const listValue = model(root, "ProgramsByMonth").fields().find((f) => f.name === "listValue"); + expect(listValue?.subType).toBe("currency"); + expect(listValue?.attr(FIELD_ATTR_CURRENCY)).toBe("USD"); + }); + + test("an enum dimension carries @values; an hour bucket carries its source's @localTime", async () => { + const root = await load(); + const status = model(root, "ProgramsByMonth").fields().find((f) => f.name === "status"); + expect(status?.subType).toBe("enum"); + expect(status?.attr(FIELD_ATTR_VALUES)).toEqual(["DRAFT", "PUBLISHED", "ARCHIVED"]); + // Asset.recordedAt is an instant: no @localTime to carry. + const hour = model(root, "AssetActivity").fields().find((f) => f.name === "recordedAtHour"); + expect(hour?.subType).toBe("timestamp"); + expect(hour?.hasAttr(FIELD_ATTR_LOCAL_TIME)).toBe(false); + }); + + test("@column is never carried: Program.createdAt's created_ts does not reach a derived field", async () => { + const root = await load(); + for (const name of ["ProgramMinutes", "ProgramsByMonth", "ProgramsByWeek", "AssetActivity"]) { + for (const f of model(root, name).fields()) expect(f.hasAttr(FIELD_ATTR_COLUMN)).toBe(false); + } + }); + + test("the model keeps the report's name and subtype, has no identity, and is frozen", async () => { + const root = await load(); + const m = model(root, "ProgramMinutes"); + expect(m.name).toBe("ProgramMinutes"); + expect(m.subType).toBe(OBJECT_SUBTYPE_REPORT); + expect(m.resolutionKey()).toBe(object(root, "ProgramMinutes").resolutionKey()); + expect(m.children().some((c) => c.type === TYPE_IDENTITY)).toBe(false); + expect(m.isFrozen()).toBe(true); + }); + + test("the model's read-only source has the report's physical name", async () => { + const root = await load(); + const m = model(root, "ProgramMinutes"); + const sources = m.children().filter(isMetaSource); + expect(sources).toHaveLength(1); + expect(sources[0]!.isReadOnly()).toBe(true); + expect(sources[0]!.effectiveKind).toBe(SOURCE_KIND_VIEW); + expect(sources[0]!.physicalName).toBe("v_program_minutes"); + // A copy: the report's own source node is not re-parented. + const own = object(root, "ProgramMinutes").children().find(isMetaSource)!; + expect(sources[0]).not.toBe(own); + expect(own.parent).toBe(object(root, "ProgramMinutes")); + }); + + test("the model is detached: it has no parent and the root does not list it", async () => { + const root = await load(); + const m = model(root, "ProgramMinutes"); + expect(m.parent).toBeUndefined(); + expect(root.objects()).not.toContain(m); + expect(root.children()).not.toContain(m); + }); + + test("root.objects() is unchanged in length and the root serialises byte-identically", async () => { + const root = await load(); + const before = canonicalSerialize(root); + const count = root.objects().length; + for (const o of root.objects()) { + if (o.subType === OBJECT_SUBTYPE_REPORT) reportReadModel(o, root); + } + expect(root.objects().length).toBe(count); + expect(canonicalSerialize(root)).toBe(before); + }); + + test("the model is cached per report node", async () => { + const root = await load(); + expect(model(root, "ProgramMinutes")).toBe(model(root, "ProgramMinutes")); + expect(model(root, "ProgramMinutes")).not.toBe(model(root, "FitnessTotals")); + }); + + // What the loader permits (asserted by `errors` below, not assumed): a report may + // declare two read-only sources, and @role defaults to primary. + const multiSource = async (sources: unknown[]): Promise => { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource( + JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Sale", children: [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "sales", "@agg": "count", "@of": "Sale.id" } }, + ] } }, + { "object.report": { name: "Totals", "@from": "Sale", "@measures": ["sales"], children: sources } }, + ], + }, + }), + ), + ]); + expect(result.errors.map((e) => e.message)).toEqual([]); + return result.root; + }; + + test("a replica read-only source declared before the primary view: the model holds the primary", async () => { + const root = await multiSource([ + { "source.rdb": { name: "rep", "@kind": "view", "@view": "v_totals_replica", "@role": "replica" } }, + { "source.rdb": { name: "pri", "@kind": "view", "@view": "v_totals", "@role": "primary" } }, + ]); + const m = model(root, "Totals"); + const sources = m.children().filter(isMetaSource); + expect(sources.map((s) => [s.physicalName, s.role])).toEqual([["v_totals", "primary"]]); + expect(resolveTableName(m)).toBe("v_totals"); + }); + + test("a read-only source with no explicit @role is the one read", async () => { + const root = await multiSource([{ "source.rdb": { "@kind": "view", "@view": "v_only" } }]); + const m = model(root, "Totals"); + expect(m.children().filter(isMetaSource).map((s) => [s.physicalName, s.role])).toEqual([["v_only", "primary"]]); + expect(resolveTableName(m)).toBe("v_only"); + }); + + test("the canonical reports resolve their table to the declared view", async () => { + const root = await load(); + expect(resolveTableName(model(root, "ProgramMinutes"))).toBe("v_program_minutes"); + expect(resolveTableName(model(root, "AssetActivity"))).toBe("v_asset_activity"); + }); + + test("a sourceless report yields a model with the fields and no source", async () => { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource( + JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Sale", children: [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { name: "id" } }, + { "field.currency": { name: "amountCents", "@currency": "EUR" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + ] } }, + { "object.report": { name: "Revenue", "@from": "Sale", "@measures": ["revenue"] } }, + ], + }, + }), + ), + ]); + expect(result.errors.map((e) => e.message)).toEqual([]); + const m = model(result.root, "Revenue"); + expect(m.fields().map((f) => [f.name, f.subType, f.attr(FIELD_ATTR_CURRENCY)])).toEqual([ + ["revenue", "currency", "EUR"], + ]); + expect(m.children().some(isMetaSource)).toBe(false); + }); +}); diff --git a/server/typescript/packages/metadata/test/report-shape.test.ts b/server/typescript/packages/metadata/test/report-shape.test.ts new file mode 100644 index 000000000..c9c3b1960 --- /dev/null +++ b/server/typescript/packages/metadata/test/report-shape.test.ts @@ -0,0 +1,188 @@ +import { describe, expect, test } from "bun:test"; +import { join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; +import { + InMemoryStringSource, + MetaDataLoader, + OBJECT_REPORT_ATTR_DIMENSIONS, + OBJECT_REPORT_ATTR_MEASURES, + loadUris, + reportMeasureItemName, + reportShape, + type MetaObject, + type MetaRoot, +} from "../src/index.js"; + +const REPO_ROOT = resolve(import.meta.dir, "..", "..", "..", "..", ".."); +const MODEL = join(REPO_ROOT, "fixtures", "conformance", "reporting-vocabulary", "input", "meta.shop.json"); + +async function load(): Promise { + const result = await loadUris([pathToFileURL(MODEL).href]); + expect(result.errors).toEqual([]); + return result.root; +} +const report = (root: MetaRoot, name: string): MetaObject => { + const found = root.objects().find((o) => o.name === name); + if (found === undefined) throw new Error(`no object ${name}`); + return found; +}; +const brief = (root: MetaRoot, name: string) => + reportShape(report(root, name), root).fields.map((f) => [f.name, f.role, f.subType, f.required]); + +describe("reportShape (FR-044 Table B)", () => { + test("dimensions come first, in listed order, then measures", async () => { + const root = await load(); + expect(brief(root, "ProgramEngagement")).toEqual([ + ["program", "dimension", "long", false], + ["starters", "measure", "long", true], + ["daysEngaged", "measure", "long", true], + ["avgDaysPerStarter", "measure", "decimal", false], + ["lastActivityAt", "measure", "timestamp", false], + ]); + }); + + test("a time dimension derives typed date", async () => { + const root = await load(); + expect(brief(root, "DailyRevenue")).toEqual([ + ["purchasedAtDay", "dimension", "date", false], + ["purchases", "measure", "long", true], + ["revenue", "measure", "currency", false], + ]); + }); + + test("sum of a currency keeps the currency field as its type source", async () => { + const root = await load(); + const revenue = reportShape(report(root, "DailyRevenue"), root).fields.find((f) => f.name === "revenue"); + expect(revenue?.typeSource?.name).toBe("amountCents"); + }); + + test("no dimensions yields measures only", async () => { + const root = await load(); + expect(brief(root, "StoreTotals").map((f) => f[1])).toEqual(["measure", "measure", "measure"]); + }); +}); + +// --------------------------------------------------------------------------- +// Reference resolution (final fix wave A2 / A3 / A8). The shape must agree with the +// loader's `validateReporting` about what a reference names, or a model that loads +// clean fails (or is silently mistyped) when it is lowered or read. +// --------------------------------------------------------------------------- + +const file = (pkg: string, children: unknown[]): InMemoryStringSource => + new InMemoryStringSource(JSON.stringify({ "metadata.root": { package: pkg, children } })); + +async function loadInline(files: InMemoryStringSource[]): Promise { + const { root, errors } = await new MetaDataLoader().load(files); + expect(errors).toEqual([]); + return root; +} + +/** `a::Base` (abstract): members whose bare `@of` names `Base`. */ +const sharedBase = { + "object.entity": { + name: "Base", + abstract: true, + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "kind" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "dimension.attribute": { name: "kind", "@of": "Base.kind" } }, + { "measure.aggregate": { name: "events", "@agg": "count", "@of": "Base.id" } }, + { "measure.aggregate": { name: "lastKind", "@agg": "max", "@of": "Base.kind" } }, + ], + }, +}; +const ev = (extra: unknown[] = []) => ({ + "object.entity": { + name: "Ev", + extends: "a::Base", + children: [{ "source.rdb": { "@table": "evs" } }, ...extra], + }, +}); +const evReport = (attrs: Record = {}) => ({ + "object.report": { + name: "R", + "@from": "Ev", + "@dimensions": ["kind"], + "@measures": ["events", "lastKind"], + ...attrs, + children: [{ "source.rdb": { "@kind": "view", "@view": "v_r" } }], + }, +}); +/** The report with one attr replaced, WITHOUT the loader (the loaded tree is frozen, and + * the loader refuses these values): what a caller building a tree in code can hand in. */ +function withAttr(node: MetaObject, name: string, value: unknown): MetaObject { + const stub = Object.create(node) as MetaObject; + Object.defineProperty(stub, "attr", { value: (n: string) => (n === name ? value : node.attr(n)) }); + return stub; +} + +const typed = (root: MetaRoot) => + reportShape(report(root, "R"), root).fields.map((f) => [f.name, f.subType, f.typeSource?.parent?.resolutionKey()]); + +describe("reportShape reference resolution", () => { + test("a bare @of on a member inherited from another package resolves in the DECLARING entity's package", async () => { + const root = await loadInline([file("a", [sharedBase]), file("b", [ev(), evReport()])]); + expect(typed(root)).toEqual([ + ["kind", "string", "a::Base"], + ["events", "long", undefined], + ["lastKind", "string", "a::Base"], + ]); + }); + + test("a same-named decoy in the report's package does not capture the reference", async () => { + const decoy = { + "object.entity": { name: "Base", children: [{ "field.int": { name: "id" } }, { "field.int": { name: "kind" } }] }, + }; + const root = await loadInline([file("a", [sharedBase]), file("b", [decoy, ev(), evReport()])]); + expect(typed(root)).toEqual([ + ["kind", "string", "a::Base"], + ["events", "long", undefined], + ["lastKind", "string", "a::Base"], + ]); + }); + + test("without @via the field is read from @from, so a field @from redeclares wins (as in the loader)", async () => { + const root = await loadInline([ + file("a", [sharedBase]), + file("b", [ev([{ "field.int": { name: "kind" } }]), evReport()]), + ]); + expect(typed(root)).toEqual([ + ["kind", "int", "b::Ev"], + ["events", "long", undefined], + ["lastKind", "int", "b::Ev"], + ]); + }); + + test("a dotted @measures item names the measure by its last segment (loader rule R3)", async () => { + const root = await loadInline([ + file("a", [sharedBase]), + file("b", [ev(), evReport({ "@measures": ["Ev.events", "a::Base.lastKind"] })]), + ]); + expect(typed(root).map((f) => f[0])).toEqual(["kind", "events", "lastKind"]); + }); + + test("reportMeasureItemName: bare, dotted and package-qualified", () => { + expect(reportMeasureItemName("total")).toBe("total"); + expect(reportMeasureItemName("Sale.total")).toBe("total"); + expect(reportMeasureItemName("acme::shop::Sale.total")).toBe("total"); + }); + + test("a dotted @measures item whose qualifier is not @from (or an ancestor of it) does not resolve", async () => { + const root = await loadInline([file("a", [sharedBase]), file("b", [ev(), evReport()])]); + // Past the loader, which refuses this as ERR_INVALID_REPORT / ERR_REPORT_FOREIGN_MEASURE. + const r = withAttr(report(root, "R"), OBJECT_REPORT_ATTR_MEASURES, ["Nope.events"]); + expect(() => reportShape(r, root)).toThrow("report 'R': measure 'Nope.events' on 'Ev' does not resolve."); + }); + + test("a time dimension item with no grain, or a grain outside the closed set, does not resolve", async () => { + const root = await load(); + const r = report(root, "DailyRevenue"); + expect(() => reportShape(withAttr(r, OBJECT_REPORT_ATTR_DIMENSIONS, ["purchasedAt"]), root)).toThrow( + "report 'DailyRevenue': time dimension 'purchasedAt' grain '' does not resolve.", + ); + expect(() => reportShape(withAttr(r, OBJECT_REPORT_ATTR_DIMENSIONS, ["purchasedAt:fortnight"]), root)).toThrow( + "report 'DailyRevenue': time dimension 'purchasedAt' grain 'fortnight' does not resolve.", + ); + }); +}); diff --git a/server/typescript/packages/runtime-ts/src/object-manager.ts b/server/typescript/packages/runtime-ts/src/object-manager.ts index bff125065..85ad535ed 100644 --- a/server/typescript/packages/runtime-ts/src/object-manager.ts +++ b/server/typescript/packages/runtime-ts/src/object-manager.ts @@ -1,6 +1,8 @@ import type { MetaData } from "@metaobjectsdev/metadata"; import { TYPE_OBJECT, TYPE_FIELD, + OBJECT_SUBTYPE_REPORT, + isMetaObject, isMetaRoot, isReadOnlySource, reportReadModel, FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_DECIMAL, } from "@metaobjectsdev/metadata"; @@ -111,7 +113,7 @@ export class ObjectManager { } async findById(entityName: string, id: unknown, opts: ReadOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "findById"); const pkField = resolvePkFields(entity)[0]!; return this.findFirst(entityName, { [pkField]: this.coerceIdArg(entity, id) as string | number }, opts); } @@ -151,7 +153,7 @@ export class ObjectManager { async load(refString: string): Promise { const { entity: entityName, pkValues } = decodeRef(refString); - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "load"); const pkFields = resolvePkFields(entity); if (pkValues.length !== pkFields.length) { throw new MetadataError( @@ -169,12 +171,12 @@ export class ObjectManager { } refOf(entityName: string, record: Row): string { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "refOf"); return encodeRef(entityName, record, resolvePkFields(entity)); } async create(entityName: string, data: Row, opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "create"); const driver = opts.tx ?? this.driver; const restricted0 = this.applyViewRestriction(entity, data, opts.view); @@ -197,7 +199,7 @@ export class ObjectManager { } async update(entityName: string, id: unknown, data: Row, opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "update"); const driver = opts.tx ?? this.driver; const restricted0 = this.applyViewRestriction(entity, data, opts.view); @@ -229,7 +231,7 @@ export class ObjectManager { } async delete(entityName: string, id: unknown, opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "delete"); const driver = opts.tx ?? this.driver; // FR-017 TPH: scope the by-id delete to the subtype (cross-subtype → not found). const spec = buildDeleteSpec(entity, this.coerceIdArg(entity, id), this.columnNamingStrategy, this.tphScope(entity)); @@ -243,7 +245,7 @@ export class ObjectManager { } async createMany(entityName: string, dataArray: Row[], opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "createMany"); const driver = opts.tx ?? this.driver; // Validate + identity-resolve every row before any insert so a late failure can't leave partial state. @@ -274,7 +276,7 @@ export class ObjectManager { } async updateMany(entityName: string, filter: Filter, partial: Row, opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "updateMany"); const driver = opts.tx ?? this.driver; const restricted = this.applyViewRestriction(entity, partial, opts.view); const v = runValidators(entity, restricted, { partial: true }); @@ -292,7 +294,7 @@ export class ObjectManager { } async deleteMany(entityName: string, filter: Filter, opts: WriteOpts = {}): Promise { - const entity = this.requireEntity(entityName); + const entity = this.requireIdentified(entityName, "deleteMany"); const driver = opts.tx ?? this.driver; const spec: DeleteManySpec = { table: resolveTableName(entity), @@ -464,6 +466,12 @@ export class ObjectManager { } private requireEntity(entityName: string): MetaData { + const entity = this.requireObject(entityName); + return entity.subType === OBJECT_SUBTYPE_REPORT ? this.reportReadModelOf(entity) : entity; + } + + /** The declared object node, exactly as loaded (a report is NOT swapped for its read model). */ + private requireObject(entityName: string): MetaData { if (!VALID_ENTITY_NAME.test(entityName)) { throw new UnsafeNameError( `Unsafe entity name '${entityName}'`, @@ -478,6 +486,57 @@ export class ObjectManager { return entity; } + /** + * FR-044: a report declares no fields — its read shape is derived — so it is read + * through a detached read model carrying one real field per derived field and the + * report's own read-only source. Everything downstream (column list, filter and + * sort resolution, the name map, read coercion) then treats it as it treats a + * projection. The model is built once per report node and never joins the tree. + * + * A report with no read-only source of its own has no view (Table A), so there is + * nothing to read: refused here rather than falling through to a default table name. + */ + private reportReadModelOf(report: MetaData): MetaData { + if (!isMetaObject(report) || !isMetaRoot(this.metadata)) { + throw new MetadataError( + `Report '${report.name}' cannot be read: the ObjectManager's metadata is not a loaded root`, + { entity: report.name }, + ); + } + let model: MetaData; + try { + model = reportReadModel(report, this.metadata); + } catch (cause) { + const detail = cause instanceof Error ? cause.message : String(cause); + throw new MetadataError(`Report '${report.name}' cannot be read: ${detail}`, { entity: report.name, cause }); + } + if (!model.children().some((c) => isReadOnlySource(c))) { + throw new MetadataError( + `Report '${report.name}' is not served: it declares no read-only source, so it has no view to read`, + { entity: report.name }, + ); + } + return model; + } + + /** + * Resolve an entity for an operation that needs an identity or writes: get-by-id, + * reference encode/decode, and every create/update/delete. A report (FR-044) is an + * aggregate over a view — it has no primary key and no write target — so these are + * refused by name before anything else is looked at (a sourceless report included). + */ + private requireIdentified(entityName: string, op: string): MetaData { + const entity = this.requireObject(entityName); + if (entity.subType === OBJECT_SUBTYPE_REPORT) { + throw new MetadataError( + `${op} is not supported on '${entityName}': a report is read-only and has no identity ` + + `(read it with findMany, findFirst or count)`, + { entity: entityName }, + ); + } + return entity; + } + private toJsRow(entity: MetaData, dbRow: Row): Row { const { dbToJs } = this.nameMap(entity); const out: Row = {}; diff --git a/server/typescript/packages/runtime-ts/src/query-builder.ts b/server/typescript/packages/runtime-ts/src/query-builder.ts index 8e0fbb78f..ef8ec0079 100644 --- a/server/typescript/packages/runtime-ts/src/query-builder.ts +++ b/server/typescript/packages/runtime-ts/src/query-builder.ts @@ -3,6 +3,7 @@ import { TYPE_FIELD, TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY, IDENTITY_ATTR_FIELDS, + OBJECT_SUBTYPE_REPORT, DEFAULT_COLUMN_NAMING_STRATEGY, resolveTableName, resolveColumnName, } from "@metaobjectsdev/metadata"; @@ -193,7 +194,19 @@ export function buildSelectSpec( strategy: ColumnNamingStrategy = DEFAULT_COLUMN_NAMING_STRATEGY, ): SelectSpec { const allFields = projectedFields ?? listFieldNames(entity); - const pkFields = resolvePkFields(entity); + // A report's read model (FR-044) has no identity, so there is no key to add to the + // column list. Scoped to the report subtype: every other object still requires one. + const isReport = entity.subType === OBJECT_SUBTYPE_REPORT; + // A report node as DECLARED has no field children (its shape is derived); only its + // read model does. Selecting from the bare node would be a query with no columns. + if (isReport && listFieldNames(entity).length === 0) { + throw new MetadataError( + `Report '${entity.name}' has no fields to select: a report is read through its read model ` + + `(reportReadModel), not through the declared node`, + { entity: entity.name }, + ); + } + const pkFields = isReport ? [] : resolvePkFields(entity); const fieldSet = new Set(allFields); for (const pk of pkFields) fieldSet.add(pk); diff --git a/server/typescript/packages/runtime-ts/test/object-manager-report.test.ts b/server/typescript/packages/runtime-ts/test/object-manager-report.test.ts new file mode 100644 index 000000000..9d47cace7 --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/object-manager-report.test.ts @@ -0,0 +1,348 @@ +// FR-044: ObjectManager reads a view-backed `object.report`. +// +// A report has no field children: its read shape is DERIVED (Table B). The runtime +// reads it through a detached read model built by `reportReadModel`, so the query +// builder and the type coercer see ordinary fields. These tests pin the contract: +// list + count work, filter and sort work on any derived field, by-id and every +// write are refused, a sourceless report is not served, and the loaded model is +// never touched. + +import { describe, test, expect } from "bun:test"; +import { join, resolve } from "node:path"; +import { + MetaDataLoader, + InMemoryStringSource, + canonicalSerialize, + isMetaRoot, + reportReadModel, +} from "@metaobjectsdev/metadata"; +import type { MetaRoot } from "@metaobjectsdev/metadata"; +import { FileSource } from "@metaobjectsdev/metadata/core"; +import { ObjectManager } from "../src/object-manager.js"; +import { inMemoryDriver } from "../src/drivers/in-memory-driver.js"; +import { MetadataError } from "../src/errors.js"; +import { buildSelectSpec } from "../src/query-builder.js"; +import type { Row } from "../src/persistence-driver.js"; + +const REPO_ROOT = resolve(import.meta.dir, "..", "..", "..", "..", ".."); +const CANONICAL = join(REPO_ROOT, "fixtures", "persistence-conformance", "canonical", "meta.fitness.json"); + +async function loadCanonical(): Promise { + const result = await new MetaDataLoader().load([new FileSource(CANONICAL)]); + expect(result.errors).toEqual([]); + if (!isMetaRoot(result.root)) throw new Error("not a root"); + return result.root; +} + +// The view's columns, under the default snake_case strategy applied to the DERIVED names. +const PROGRAM_MINUTES_ROWS: Row[] = [ + { program: 1, program_title: "Alpha", weeks: 3, long_weeks: 1, labels: 3, slots: 3, + total_minutes: 150, avg_minutes: "50.0000", min_minutes: 30, max_minutes: 75, long_share: "0.3333" }, + { program: 2, program_title: "Bravo", weeks: 2, long_weeks: 2, labels: 2, slots: 2, + total_minutes: 140, avg_minutes: "70.0000", min_minutes: 60, max_minutes: 80, long_share: "1.0000" }, + { program: 3, program_title: "Charlie", weeks: 1, long_weeks: 0, labels: 1, slots: 1, + total_minutes: 20, avg_minutes: "20.0000", min_minutes: 20, max_minutes: 20, long_share: "0.0000" }, +]; + +async function canonicalOm(): Promise<{ om: ObjectManager; root: MetaRoot }> { + const root = await loadCanonical(); + const driver = inMemoryDriver({ + seed: { v_program_minutes: PROGRAM_MINUTES_ROWS }, + pkFields: { v_program_minutes: ["program"] }, + }); + return { om: new ObjectManager({ metadata: root, driver }), root }; +} + +// An inline model for the cases the canonical corpus does not carry: an int-backed enum +// dimension (the derived field must carry @values + @intValueMap from its type source), +// an @unmanaged view, an @sql view, and a sourceless report. +const SALES = { + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Sale", children: [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", "@required": true, "@values": ["OPEN", "CLOSED"], + "@intValueMap": { OPEN: 1, CLOSED: 2 } } }, + { "field.currency": { name: "amountCents", "@required": true, "@currency": "EUR" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "dimension.attribute": { name: "status", "@of": "Sale.status" } }, + { "measure.aggregate": { name: "sales", "@agg": "count", "@of": "Sale.id" } }, + { "measure.aggregate": { name: "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + ] } }, + { "object.report": { name: "SalesByStatus", "@from": "Sale", "@dimensions": ["status"], + "@measures": ["sales", "revenue"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_sales_by_status" } }, + ] } }, + { "object.report": { name: "UnmanagedSales", "@from": "Sale", "@dimensions": ["status"], + "@measures": ["sales"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_unmanaged_sales", "@unmanaged": true } }, + ] } }, + { "object.report": { name: "AuthoredSales", "@from": "Sale", "@measures": ["sales"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_authored_sales", + "@sql": "SELECT COUNT(id) AS sales FROM sales" } }, + ] } }, + // Loads clean: a report may declare a replica read-only source beside its primary view. + { "object.report": { name: "ReplicatedSales", "@from": "Sale", "@measures": ["sales"], children: [ + { "source.rdb": { name: "rep", "@kind": "view", "@view": "v_replicated_sales_replica", "@role": "replica" } }, + { "source.rdb": { name: "pri", "@kind": "view", "@view": "v_replicated_sales", "@role": "primary" } }, + ] } }, + { "object.report": { name: "InertSales", "@from": "Sale", "@measures": ["sales"] } }, + ], + }, +}; + +async function salesOm(): Promise<{ om: ObjectManager; root: MetaRoot }> { + const result = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(SALES))]); + expect(result.errors.map((e) => e.message)).toEqual([]); + if (!isMetaRoot(result.root)) throw new Error("not a root"); + const driver = inMemoryDriver({ + seed: { + v_sales_by_status: [ + { status: 1, sales: 4, revenue: 4000 }, + { status: 2, sales: 6, revenue: 9000 }, + ], + v_unmanaged_sales: [ + { status: 1, sales: 4 }, + { status: 2, sales: 6 }, + ], + v_authored_sales: [{ sales: 10 }], + v_replicated_sales: [{ sales: 10 }], + // Decoys: the replica view, and the default table name a model with no primary + // source would fall back to. + v_replicated_sales_replica: [{ sales: 77 }], + replicated_sales: [{ sales: 88 }], + // A decoy: if a report read ever fell back to the entity-name default table + // ("inert_sales") or to the @from table, these rows would surface. + sales: [{ id: 1, status: 1, amount_cents: 1000 }], + inert_sales: [{ sales: 99 }], + }, + pkFields: { + v_sales_by_status: ["status"], v_unmanaged_sales: ["status"], + v_authored_sales: ["sales"], inert_sales: ["sales"], + v_replicated_sales: ["sales"], v_replicated_sales_replica: ["sales"], replicated_sales: ["sales"], + }, + }); + return { om: new ObjectManager({ metadata: result.root, driver }), root: result.root }; +} + +describe("ObjectManager reads a view-backed report (FR-044)", () => { + test("findMany returns the view's rows keyed by derived field name", async () => { + const { om } = await canonicalOm(); + const rows = await om.findMany("ProgramMinutes", undefined, { orderBy: ["program", "asc"] }); + expect(rows).toHaveLength(3); + expect(rows[0]).toEqual({ + program: 1, programTitle: "Alpha", weeks: 3, longWeeks: 1, labels: 3, slots: 3, + totalMinutes: 150, avgMinutes: "50.0000", minMinutes: 30, maxMinutes: 75, longShare: "0.3333", + }); + }); + + test("count works with and without a filter", async () => { + const { om } = await canonicalOm(); + expect(await om.count("ProgramMinutes")).toBe(3); + expect(await om.count("ProgramMinutes", { totalMinutes: { $gte: 100 } })).toBe(2); + expect(await om.count("ProgramMinutes", { programTitle: "Charlie" })).toBe(1); + }); + + test("filters on a dimension and on a measure", async () => { + const { om } = await canonicalOm(); + const byDimension = await om.findMany("ProgramMinutes", { programTitle: { $like: "B%" } }); + expect(byDimension.map((r) => r.program)).toEqual([2]); + const byMeasure = await om.findMany("ProgramMinutes", { minMinutes: { $lt: 60 } }, { orderBy: ["program", "asc"] }); + expect(byMeasure.map((r) => r.programTitle)).toEqual(["Alpha", "Charlie"]); + const both = await om.findMany("ProgramMinutes", { $and: [{ weeks: { $gte: 2 } }, { longWeeks: { $gte: 2 } }] }); + expect(both.map((r) => r.programTitle)).toEqual(["Bravo"]); + }); + + test("sorts on a measure, with limit and offset", async () => { + const { om } = await canonicalOm(); + const desc = await om.findMany("ProgramMinutes", undefined, { orderBy: ["totalMinutes", "desc"] }); + expect(desc.map((r) => r.programTitle)).toEqual(["Alpha", "Bravo", "Charlie"]); + const page = await om.findMany("ProgramMinutes", undefined, { + orderBy: ["maxMinutes", "asc"], limit: 1, offset: 1, + }); + expect(page.map((r) => r.programTitle)).toEqual(["Alpha"]); + }); + + test("findFirst reads one row", async () => { + const { om } = await canonicalOm(); + const row = await om.findFirst("ProgramMinutes", { program: 2 }); + expect(row?.programTitle).toBe("Bravo"); + }); + + test("the literal naming strategy addresses the view by the derived names as written", async () => { + const root = await loadCanonical(); + const driver = inMemoryDriver({ + seed: { v_fitness_totals: [{ weeks: 6, totalMinutes: 310, longShare: "0.5000" }] }, + pkFields: { v_fitness_totals: ["weeks"] }, + }); + const om = new ObjectManager({ metadata: root, driver, columnNamingStrategy: "literal" }); + expect(await om.findMany("FitnessTotals")).toEqual([{ weeks: 6, totalMinutes: 310, longShare: "0.5000" }]); + }); + + test("rows are coerced by derived subtype: an int-backed enum dimension reads and filters as its symbol", async () => { + const { om } = await salesOm(); + const rows = await om.findMany("SalesByStatus", undefined, { orderBy: ["revenue", "desc"] }); + expect(rows).toEqual([ + { status: "CLOSED", sales: 6, revenue: 9000 }, + { status: "OPEN", sales: 4, revenue: 4000 }, + ]); + expect(await om.findMany("SalesByStatus", { status: "OPEN" })).toEqual([{ status: "OPEN", sales: 4, revenue: 4000 }]); + expect(await om.count("SalesByStatus", { status: { $in: ["OPEN", "CLOSED"] } })).toBe(2); + }); + + test("an @unmanaged report is served: findMany and count read its view", async () => { + const { om } = await salesOm(); + const rows = await om.findMany("UnmanagedSales", undefined, { orderBy: ["sales", "asc"] }); + expect(rows).toEqual([{ status: "OPEN", sales: 4 }, { status: "CLOSED", sales: 6 }]); + expect(await om.count("UnmanagedSales")).toBe(2); + expect(await om.count("UnmanagedSales", { sales: { $gt: 4 } })).toBe(1); + }); + + test("an @sql report is served from its view", async () => { + const { om } = await salesOm(); + expect(await om.findMany("AuthoredSales")).toEqual([{ sales: 10 }]); + expect(await om.count("AuthoredSales")).toBe(1); + }); + + test("a replica read-only source declared before the primary view: reads come from the primary view", async () => { + const { om } = await salesOm(); + expect(await om.findMany("ReplicatedSales")).toEqual([{ sales: 10 }]); + expect(await om.count("ReplicatedSales")).toBe(1); + expect(await om.count("ReplicatedSales", { sales: 10 })).toBe(1); + }); + + test("a report whose only read-only source has no explicit @role is read from it", async () => { + // SalesByStatus, UnmanagedSales and AuthoredSales all declare no @role. + const { om, root } = await salesOm(); + const report = root.objects().find((o) => o.name === "SalesByStatus")!; + expect(buildSelectSpec(reportReadModel(report, root), undefined, {}).table).toBe("v_sales_by_status"); + expect(await om.count("SalesByStatus")).toBe(2); + }); + + test("the select spec: the view as the table, the derived columns in Table B order, no key column", async () => { + const root = await loadCanonical(); + const report = root.objects().find((o) => o.name === "ProgramMinutes")!; + const spec = buildSelectSpec(reportReadModel(report, root), undefined, {}); + expect(spec.table).toBe("v_program_minutes"); + expect(spec.columns).toEqual([ + "program", "program_title", "weeks", "long_weeks", "labels", "slots", + "total_minutes", "avg_minutes", "min_minutes", "max_minutes", "long_share", + ]); + expect(spec.where).toBeUndefined(); + const literal = buildSelectSpec(reportReadModel(report, root), undefined, {}, undefined, "literal"); + expect(literal.columns).toEqual([ + "program", "programTitle", "weeks", "longWeeks", "labels", "slots", + "totalMinutes", "avgMinutes", "minMinutes", "maxMinutes", "longShare", + ]); + }); + + test("the declared report node, passed straight to buildSelectSpec, is refused by name", async () => { + const root = await loadCanonical(); + const report = root.objects().find((o) => o.name === "ProgramMinutes")!; + let err: unknown; + try { + buildSelectSpec(report, undefined, {}); + } catch (e) { + err = e; + } + expect(err).toBeInstanceOf(MetadataError); + expect((err as MetadataError).message).toContain("ProgramMinutes"); + expect((err as MetadataError).message).toContain("no fields"); + }); + + test("a sourceless report is not served", async () => { + const { om } = await salesOm(); + for (const read of [ + () => om.findMany("InertSales"), + () => om.count("InertSales"), + () => om.findFirst("InertSales", {}), + ]) { + const err = await read().then(() => undefined, (e: unknown) => e); + expect(err).toBeInstanceOf(MetadataError); + expect((err as MetadataError).message).toContain("InertSales"); + expect((err as MetadataError).message).toContain("not served"); + expect((err as MetadataError).message).toContain("no view"); + } + }); + + test("by-id and every write on a report throw: read-only, no identity", async () => { + const { om } = await canonicalOm(); + const attempts: Array<[string, () => unknown]> = [ + ["findById", () => om.findById("ProgramMinutes", 1)], + ["create", () => om.create("ProgramMinutes", { program: 9 })], + ["update", () => om.update("ProgramMinutes", 1, { weeks: 9 })], + ["delete", () => om.delete("ProgramMinutes", 1)], + ["createMany", () => om.createMany("ProgramMinutes", [{ program: 9 }])], + ["updateMany", () => om.updateMany("ProgramMinutes", { program: 1 }, { weeks: 9 })], + ["deleteMany", () => om.deleteMany("ProgramMinutes", { program: 1 })], + ["load", () => om.load("ProgramMinutes:1")], + ["refOf", () => om.refOf("ProgramMinutes", { program: 1 })], + ]; + for (const [op, attempt] of attempts) { + let err: unknown; + try { + await attempt(); + } catch (e) { + err = e; + } + expect(err, op).toBeInstanceOf(MetadataError); + const message = (err as MetadataError).message; + expect(message, op).toContain("ProgramMinutes"); + expect(message, op).toContain("read-only"); + expect(message, op).toContain("no identity"); + expect(message, op).toContain(op); + expect((err as MetadataError).entity, op).toBe("ProgramMinutes"); + } + // Nothing was written. + expect(await om.count("ProgramMinutes")).toBe(3); + }); + + test("a write on a sourceless report is refused as read-only, not as unserved", async () => { + const { om } = await salesOm(); + const err = await om.create("InertSales", { sales: 1 }).then(() => undefined, (e: unknown) => e); + expect(err).toBeInstanceOf(MetadataError); + expect((err as MetadataError).message).toContain("read-only"); + }); + + test("an unknown field in a report filter or sort is refused by name", async () => { + const { om } = await canonicalOm(); + // `durationMinutes` is a Week field, not a derived field of the report. + await expect(om.findMany("ProgramMinutes", { durationMinutes: 60 })).rejects.toThrow( + "Unknown field 'durationMinutes' on entity 'ProgramMinutes'", + ); + await expect(om.findMany("ProgramMinutes", undefined, { orderBy: ["title", "asc"] })).rejects.toThrow( + "Unknown field 'title'", + ); + }); + + test("reading reports leaves the loaded model untouched", async () => { + const { om, root } = await canonicalOm(); + const before = canonicalSerialize(root); + const objects = root.objects().length; + await om.findMany("ProgramMinutes", { weeks: { $gte: 1 } }, { orderBy: ["weeks", "desc"] }); + await om.count("FitnessTotals"); + await om.findMany("ProgramsByMonth"); + await om.findMany("AssetActivity"); + expect(root.objects().length).toBe(objects); + expect(canonicalSerialize(root)).toBe(before); + }); + + test("the runtime reads through the cached read model", async () => { + const { om, root } = await canonicalOm(); + const report = root.objects().find((o) => o.name === "ProgramMinutes")!; + const model = reportReadModel(report, root); + await om.findMany("ProgramMinutes"); + expect(reportReadModel(report, root)).toBe(model); + }); + + test("entities in the same model are read and written as before", async () => { + const { om } = await salesOm(); + expect(await om.count("Sale")).toBe(1); + const created = await om.create("Sale", { status: "CLOSED", amountCents: 2500 }); + expect(created.status).toBe("CLOSED"); + expect((await om.findById("Sale", created.id))?.amountCents).toBe(2500); + expect(await om.count("Sale")).toBe(2); + }); +}); diff --git a/spec/roadmap.md b/spec/roadmap.md index 479cb2a50..a025a461a 100644 --- a/spec/roadmap.md +++ b/spec/roadmap.md @@ -161,7 +161,7 @@ under **Shipped**; planned FRs under **Planned** + the **Release plan**. ✅ shi | FR-041 | Public A/B drift benchmark — coding agents with vs without MetaObjects, pre-registered, friction-first | 📋 **design settled 2026-09-12, unbuilt** — the "proving the value" work; it is what licenses the claims FR-042 §4 withholds. Revised after an adversarial two-reviewer design review (spec §14): the scored task set is now held out from the friction pass (the draft tuned Arm B on the tasks it would later be scored on, with no symmetric loop for the control), Arm A is derived from Arm B's generated output so the seeds differ only in the model and the gate, `n` is calibrated from a pilot instead of asserted (one identical config measured 25/51/28 turns at temperature 0 — 41% CV), H2 is time-to-**correct** rather than time-to-done, the primary is analysed intention-to-treat so an arm cannot win by not finishing, and escaped defects are reported split by whether `meta verify` already covers the invariant class (it covers four of five, so H1 is partly definitional for those). First deliverable is the Phase 0b friction log. Design: `docs/superpowers/specs/2026-09-11-fr-041-drift-ab-benchmark-design.md` | 1.x | — | | FR-042 | First-touch positioning — one typed model, two verbs (README, llms, sites) | 🟢 shipped on the four first-touch surfaces — pitch **locked** 2026-09-12 (two verbs: Generate + Verify; requirements fold into Verify; H1 model-first), Verify clause amended 2026-09-14 to "fails or warns", six pillars carry maturity labels, do-not-say list gated in the `gates` lane. **Remaining:** the drift terminal recording for the .dev hero, and the drift-demo page its CTA should point at (spec §8). Design: `docs/superpowers/specs/2026-09-11-fr-042-first-touch-positioning-design.md` | — | — | | FR-043 | **Libraries** — reusable declared design: model metadata + the requirements that make it checkable (+ an implied generator selection), opted into by name | 🟢 **design approved 2026-09-13, un-deferred** — proposed as a SIXTH pillar. Not greenfield: `library/ai/llm-call.yaml` already ships one (opt-in via the loader's `libraries: ["ai"]`, embedded per port under an `embedded-library drift` gate, with `trace-helper` codegen beside it). Generalises it — requirements as a component, discovery through the codegen catalog's `kind: "library"`, declared package→generator coupling replacing `trace-helper`'s hard-coded `LlmCallBase`, `overlay: true` as the adaptation door and `meta eject` for repackaging. **No new metamodel vocabulary; `metamodelVersion` does not move.** Second library `iam` (users, typed nestable groups, roles as permission bundles, global and group-scoped grants) ships `stability: preview`; its model is verified to load clean under strict. Phase 2: third-party authoring over FR-023's deferred transports. Design: `docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md` | 1.1 | — | -| FR-044 | **Core reporting** — declared measures, dimensions, segments and reports (compiled to SQL views in every port), plus Cube and dbt MetricFlow exporters | 🟢 **Plan 1 of 5 shipped on `main` (2026-10-03)** — the vocabulary (`dimension.attribute`, `dimension.time`, `measure.aggregate`, `measure.ratio`, `segment.filter`, `object.report`, plus the relative-date filter value) is registered and loader-validated in all five ports, gated by 27 new conformance fixtures and the new `codegen-noop` corpus; decisions D1–D6 are settled. `metamodelVersion` reads **`1.1`** on `main` (1.0.x PATCH releases held until 1.1 ships). **Reports generate nothing yet** — view lowering and REST are Plans 2–3, then the Cube / dbt MetricFlow exporters; `measure.derived` stays unregistered until FR-037 R5. A query-time engine remains parked with its re-entry trigger. Feature doc: `docs/features/reporting.md`. Design: `docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md` | 1.1 | [#391](https://github.com/metaobjectsdev/metaobjects/issues/391) | +| FR-044 | **Core reporting** — declared measures, dimensions, segments and reports (compiled to SQL views in every port), plus Cube and dbt MetricFlow exporters | 🟢 **Plans 1–2 of 5 shipped on `main` (Plan 1 2026-10-03, Plan 2 2026-10-04)** — the vocabulary (`dimension.attribute`, `dimension.time`, `measure.aggregate`, `measure.ratio`, `segment.filter`, `object.report`, plus the relative-date filter value) is registered and loader-validated in all five ports, gated by 27 new conformance fixtures and the new `codegen-noop` corpus; decisions D1–D6 are settled. `metamodelVersion` reads **`1.1`** on `main` (1.0.x PATCH releases held until 1.1 ships). **Plan 2 lowered the view-backed report**: a report that declares a read-only `source.rdb @kind: view` becomes that view in `meta migrate` (Postgres / SQLite / D1; MySQL through `buildReportViews` + the recipe), every port's runtime reads it (six shared persistence scenarios, columns pinned by `report-shapes.json`), C# generates a keyless EF Core row and Kotlin an Exposed table object, and `meta docs` lists the view on the agent schema page. **Still to come:** REST routes and typed clients (Plan 3), then the Cube / dbt MetricFlow exporters; `measure.derived` stays unregistered until FR-037 R5. A query-time engine remains parked with its re-entry trigger. Feature doc: `docs/features/reporting.md`. Design: `docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md` | 1.1 | [#391](https://github.com/metaobjectsdev/metaobjects/issues/391) | _(FR-001 was the original metamodel foundation — pre-dates the FR-numbered tracking.)_ _(FR-032 was developed under the working number "FR-026" — see commit history; renumbered to avoid the FR-026=Forms collision. Design: `docs/superpowers/specs/2026-06-13-fr-032-canonical-fqn-refs-design.md`, ADR-0032.)_