Repository navigation
feat(reporting): lower view-backed object.report to SQL views and read it in every port (FR-044 Plan 2) - #399
Merged
Merged
Conversation
…ews; canonical reports and shape artifact (FR-044)
…anonical metadata directory (FR-044)
… on Postgres and SQLite (FR-044) Postgres: convergence x3, the Task 10 values read straight off the six canonical views, UTC buckets under a New York session zone, empty-group row, INNER vs LEFT OUTER join, relative window, and a changed report taking the drop-and-create path. SQLite: convergence (verbatim text), the same values, week boundary, quarter and year grains, relative window, tuple distinct count via json_array, and the hour bucket's literal pinned to the .000Z spelling the TS adapters store. No emitter change.
…ipe (FR-044) A MySQL 8.4 value test creates the six canonical report views under the default sql_mode (ONLY_FULL_GROUP_BY asserted, not assumed) and pins the Task 10 rows, the 2/3 ratio at 0.6667 (Review Focus 5), the bigint type of a SIGNED-cast sum, and the Table D grain and Table E relative-date values. No emitter change was needed. docs/recipes/mysql.md gains a Reports section (meta migrate still does not own a MySQL schema); the skill reference and the regenerated agent-context goldens follow.
…e recipe's declaration in the test (FR-044) buildReportViews skips a report whose source.rdb is @Unmanaged, so the recipe's @Unmanaged: true declaration returned no SQL. The recipe now declares @kind: view only (meta migrate never targets MySQL, so nothing manages the view either way) and says that an unmanaged source is skipped. The MySQL test now reads the recipe's own fenced declaration, runs the recipe's loadDirectory + buildReportViews shape, and asserts one view, and none once @Unmanaged is added. Tables and views are created in beforeAll after dropping stale ones, so tests run alone (-t) and rerun against a persistent server.
…ring names (FR-044)
Six shared read scenarios over the view-backed reports in the canonical model: grouped measures, totals, totals over an empty table, time grains, hour and week buckets, and a relative-date filter. List and count only; no get, no write. The TypeScript runner discovers them from queries/ with no list to update and passes all 33 query scenarios. Persistence corpus count 33 -> 39 (27 -> 33 query) in docs/CONFORMANCE.md; README gains a Report scenarios subsection. Other ports' persistence lanes are red on these until their own tasks.
…d report (FR-044)
…class name (FR-044)
… no report view; correct reporting doc details (FR-044)
KotlinExposedTableGenerator emits the read-only Exposed table of an object.report that declares a source.rdb @kind: view, with one column per derived field (contract Table B) taken from the JVM ReportShape. A report with no view, or over a kind the lowering skips, still generates nothing, and every other Kotlin generator still skips reports. A report table binds by literal (no names artifact is emitted for a report), types an enum column by the enum of the entity the dimension reads, and reads a derived decimal with no declared precision at 38,18 so Exposed does not round a ratio to four places. A derived field named after a Kotlin keyword, or two that land on one column property, is a generation error naming the report and the item. Also reserves schemaName as an Exposed Table member in safeColumnProperty: a column property of that name did not compile. Six hand-written reference tables put the Kotlin persistence lane on the six shared report scenarios.
…ve Exposed 1.x table members (FR-044)
…what cannot be lowered (FR-044) The lowering and the read shape disagreed with validateReporting about what a loadable model means. Each case loaded clean and then failed, or was silently wrong, at migrate or at read. - @Of and @via resolve in the package of the entity that DECLARES the dimension or measure, not the @from entity's. A member inherited from a base in another package now resolves, and a same-named entity in the report's package can no longer capture the reference and mistype the column. - Without @via the field is read from @from, and a @via walk starts at @from, as the loader's does. - A dotted @measures item (Sale.total, loader rule R3) names the measure by its last segment. One canonical report now uses the dotted form; schema.postgres.sql and report-shapes.json are byte-identical. - A report is classified (skip / @SQL / derive) by the same source its view is named by and the runtime reads: primary, else first. A replica declared first no longer decides it. The report-shapes generator uses the same selector. - A derived report @from a TPH subtype is refused: the subtype shares its base's table, so the view aggregated every subtype's rows. An @SQL or @Unmanaged report is the author's body and is not refused. - A @via hop with no identity.reference behind it says which hop and what it needs. An empty in list, a time dimension without a grain and a grain outside the closed set are refused by name instead of reaching the DDL.
…they do not own (FR-044) - MySQL: drop only the views and tables this file creates, by name, instead of every view in the database (a shared test database lost unrelated views). - Postgres: changing a report runs a plain migrate with no dropView allowance, so a broken drop-and-create pairing would fail the test. - The NULL-component tuple count is read through a lowered view over nullable columns on Postgres and SQLite, not through hand-written SQL. - A decimal sum and a double / float sum are created and read on Postgres, with the view's column types asserted. - Every inline model asserts the loader returned no errors. - The SQLite inet residue is narrowed to the two inet columns.
…nown limits (FR-044) - The documented @via example could not be lowered: its relationship had no identity.reference behind it. The example now declares one, and the feature doc, authoring skill rule 2 and the skill reference say a hop needs it. - Known limits: a derived report from a TPH subtype is refused (declare it from the base with a filter on the discriminator); an abstract view-backed report and non-view source kinds get no C# row or Kotlin table; a dimension over a field.object is not supported across ports. - Which source decides when a report declares several (primary, else first), the dotted @measures form, and that count counts non-null @Of. - CHANGELOG [Unreleased] and the agent-context goldens follow.
…ort over a field.object (FR-044) The report shape resolved a bare @Of in the @from entity's package. The loader resolves it in the package of the entity that DECLARES the dimension or measure, so a member inherited from a base in another package either failed to resolve or was typed from a same-named decoy. The shape now follows the loader: declaring package, the named entity must be @from or an ancestor, and without @via the field is read from @from. A dotted @measures item (Sale.total) names the measure by its last segment. A time dimension item with no grain, or one outside the closed set, does not resolve. OMDB: a derived field over a field.object is refused by name when the read model is built (it was a NullPointerException on read). OQL with a report result class builds rows from the read model. getObjectRef leaves an object with no metadata to the base method. A projection whose view is named by @view now has a read mapping: the view name is the source's physical name, one rule for projections and reports.
…eport over a field.object (FR-044) The enum class of a report column is taken from ReportShape.ofEntity, so the generator restates nothing about packages or @via. gen fails, naming the report and the dimension or measure, when a derived field reads a field.object. Tests cover a member inherited across packages (with and without a same-named decoy), a dotted @measures item, and an abstract view-backed report (generates nothing).
…eport over a field.object (FR-044) ReportShapes resolves @Of in the declaring entity's package, checks the named entity is @from or an ancestor, and reads the field from @from when there is no @via. A dotted @measures item names the measure by its last segment. A report row with a dimension over a field.object silently lost that property; gen now refuses it by name. An abstract view-backed report generates nothing (tested).
…able B rows and carry rules (FR-044) report_shape resolves @Of in the declaring entity's package, checks the named entity is @from or an ancestor, and reads the field from @from when there is no @via. A dotted @measures item names the measure by its last segment. The shapes test helper names the view by the read model's rule (primary, else first). New direct tests for sum/avg/min/max by @Of subtype and for what a derived field carries from its type source. ADR-0039 comments on the own-only reads.
…object refusal, OMDB @view projections (FR-044)
Java OMDB (on read), Kotlin gen and C# gen now refuse a report whose column is typed by a field.object, naming the report and the item; the TypeScript and Python runtimes read it as parsed JSON. The Known limits entry and the skill reference said the other ports were not gated for it.
…s a dot; document the redeclared-field limit (FR-044) The @via walk was handed the @from entity's resolution key as its head, and the walk splits on every dot, so a report in a package such as com.acme resolved no hop and was refused with a false 'no foreign key' message. The head is now the entity's short name, resolved in its own package, which is still that entity when another package has one of the same name. Known limits gains the quiet form of the @via rule: when the reached subtype redeclares the field and @Of names the base, the view reads the base's column.
…degen inventory for report lowering
This was referenced Oct 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Execute FR-044 Plan 2 (report view lowering), as written in
docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md(merged in PR #398), using subagent-driven development, then gate it and merge.Plan 2 makes reports real after Plan 1 (PR #397) made them loadable but inert: TypeScript view lowering for
object.reportand relative-date filter values on Postgres, SQLite/D1 and MySQL; shared persistence-conformance read scenarios green in every port (TypeScript, C#, Java, Kotlin, Python); the reporting section of themetaobjects-authoringskill, docs and changelog. Tasks 1-16 in the plan's order.The plan's seven open questions are answered, each as the plan already assumes:
source.rdbwith@kind: view; a sourceless report stays inert.buildReportViewsplus the documented recipe, tested against MySQL 8.4; no CLI surface.nowis the UTC clock; no time-zone vocabulary.@viajoin follows the settled projection rule unchanged (a required belongs-to FK is INNER).meta docsagent schema page; model and API pages for reports wait for Plan 3.countcounts rows whose@ofcolumn is not null; asumof nothing is null; ratio and average precision are engine-native.Standing decisions: no query-time engine (compiled SQL views only); all measures from the report's own
@fromentity and@viato-one only; relative-date values legal only insegment,measure.aggregateandobject.reportfilters;measure.derivedexcluded; nothing outside spec section 3.1; Plans 3-5 are later work.What Changed
object.reportwith a read-onlysource.rdb @kind: viewis now lowered bymeta migrateon Postgres, SQLite and D1 (extractReportSpec+emitReportViewDdl; a changed view is dropped and re-created; refusals by name for a TPH-subtype@from, a@viahop with noidentity.reference, an emptyinlist, and a tableless@from).buildReportViews(root, { dialect: "mysql" })emits MySQL SQL with a new recipe indocs/recipes/mysql.md. Time-grain and relative-date filter SQL (time-sql.ts) is UTC-only, weeks start Monday; a sourceless report still generates nothing.ObjectManagerand Java OMDB (list and count with filter/sort/limit on derived fields; by-id and writes refused; Java OQL builds rows from a report result class) read view-backed reports; C# generates a keyless EF Core row type plusDbContextmapping and Kotlin an Exposed table object, both refusing reports they cannot lower (e.g. derived fields overfield.object, Kotlin hard-keyword or colliding member names).meta docslists the view on the agent schema page. Also fixed: Java OMDB resolves a projection's physical name from@viewbefore legacy@table, and Kotlin reserves ExposedTablemember names.fixtures/persistence-conformance/queries/report-*.yaml) plus thereport-shapes.jsonartifact hold all five ports to the same columns and semantics (count of non-null@ofis zero over nothing, nullsum, null ratio over zero); themetaobjects-authoringskill gainsreferences/reporting.md, and the reporting feature doc, port docs, and CHANGELOG describe the lowering and its remaining limits (no routes, typed clients, filter allowlists, api-docs,measure.derived, or query-time grouping — Plan 3+).Risk Assessment
✅ Low: No substantiated source findings: every risky surface (dialect-specific SQL, cross-port shape parity, read/write refusals, fail-closed refusals naming the report) is either verified against a real engine here or gated by executable behavior tests (six shared scenarios on real Postgres/SQLite/MySQL, byte-matched shape artifact, convergence tests), and the near-miss wrong-SQL paths I constructed are each unreachable behind a loader rule.
Testing
Baseline
scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchainswas green before this run. I then drove the change live on all five ports: the six shared persistence-conformance report scenarios read through each port's real runtime against a containerized Postgres (TypeScript ObjectManager 6/6; C# xUnit theories 33/33; Java QueryScenarioTests 33/33; Kotlin Exposed QueryScenarioConformanceTest 33/33; Python runner 6/6 report-filtered), and the TS lowering was exercised against all three engines end-to-end (report-views-pg 14, report-views-sqlite 11, report-views-mysql 13, all converging to an empty second diff and returning the scenario rows). A hand-driven CLI pass lowered the canonical reports throughmeta docs --agent(all six report views on the schema page) and reproduced the adversarial refusal — a view-backed report over a sourceless@fromfails with 'report WidgetTotals: @from Widget has no table...', while the lowerable control report in the same model renders; transcripts saved as artifacts alongside a SQLite transcript showing emitted view SQL and actual rows. Port report unit/read suites (C# 74, Java ReportReadTest 22 + shape tests, Kotlin 30, Python 40) and the TS inert corpus (31) all passed. The complete regression suitescripts/ci-local.sh --strict-toolchainswas launched and was still running its final integration lane at phase close; its completion notification will land in this session. Worktree left clean; no source files modified.meta docsagent schema page, and a model with no report keeps its pre-report page byte for byteEvidence: meta docs --agent schema page with all six report views (live CLI drive)
Evidence: Views section excerpt: v_asset_activity, v_fitness_totals, v_program_minutes, v_programs_by_month, v_programs_by_week, v_recent_programs
Evidence: CLI refusal transcript: view-backed report over a sourceless @from (error names report and entity; page skipped, not silently wrong)
Evidence: SQLite live run: emitted CREATE VIEW SQL for four reports plus rows read back (weeks 5, totalMinutes 285, longShare 0.6; ProgramMinutes 240/0.75) and empty second diff
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
scripts/ci-local.sh --strict-toolchains) was launched and was still executing its final lane (integration-tests, 5-port + docker: fresh postgres:16 + mysql:8.4 containers active) when this phase closed. Baseline--only ts-fast --only ts-unitpassed, and every port's full query-scenario suite was driven directly and green (C# 33/33, Java 33/33, Kotlin 33/33, Python report 6/6, TS 6/6 + engine tests), so the missing outcome is the remaining breadth (other 27 TS scenarios, full Python suite, codegen-compile gates, hygiene lanes). The run's exit notification will arrive in this session; its log (tail -120) is in task output bjibylid0.meta docsagent schema page, and a model with no report keeps its pre-report page byte for bytescripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchainsscripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (baseline, green)cd server/typescript/packages/integration-tests && bun test test/report-views-pg.test.ts (14 pass, real Postgres via Testcontainers)cd server/typescript/packages/integration-tests && bun test test/report-views-sqlite.test.ts (11 pass)cd server/typescript/packages/integration-tests && bun test test/report-views-mysql.test.ts (13 pass, real MySQL 8.4 via Testcontainers)cd server/typescript/packages/integration-tests && bun test test/query.test.ts -t report (6/6 report scenarios through ObjectManager on real Postgres)bun test test/report-shapes-artifact.test.ts (3 pass, cross-port shape byte contract)bun test test/regen-reports-divergence.test.ts (6 pass)cd server/typescript/packages/cli && bun test test/unit/reporting-inert.test.ts (31 pass, sourceless reports generate nothing)dotnet test MetaObjects.IntegrationTests --filter FullyQualifiedName~QueryScenarioTests (33/33 on real Postgres, six report theories included)dotnet test server/csharp --filter ReportShape|ReportingInert|ReportRow (13 + 61 pass)uv run --extra integration pytest tests/integration/test_query_scenarios.py -k report (6/6 on real Postgres); pytest tests/test_report_shape.py tests/test_report_read_model.py tests/runtime/test_object_manager_report.py (40 pass)mvn -f integration-tests/pom.xml test -Dtest=QueryScenarioTests (33/33 on real Postgres); mvn -pl omdb -am test -Dtest=ReportReadTest (22 pass, embedded Derby executing the lowered view); mvn -pl metadata test -Dtest=ReportShapeTest,ReportReadModelTestmvn -f integration-tests-kotlin/pom.xml test -Dtest=QueryScenarioConformanceTest (33/33 on real Postgres); mvn -pl codegen-kotlin test -Dtest=KotlinReportTableGeneratorTest,KotlinReservedTableMembersTest (30 pass)meta docs --agent against a consumer project over the canonical fixture (report views on agent/schema.md) and against an adversarial sourceless-@from model (named refusal)ad-hoc SQLite demo lowering the canonical reports through buildProjectionViews + migrate-ts and reading rows back (evidence artifact)scripts/ci-local.sh --strict-toolchains (full regression — launched, still in its final integration lane at phase close)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.