Skip to content

core: analyze every dialect through the new analysis core - #4538

Open
kyleconroy wants to merge 2 commits into
mainfrom
claude/analyze-core-analyzer-migration-dv25w5
Open

core: analyze every dialect through the new analysis core#4538
kyleconroy wants to merge 2 commits into
mainfrom
claude/analyze-core-analyzer-migration-dv25w5

Conversation

@kyleconroy

Copy link
Copy Markdown
Collaborator

sqlc analyze ran ClickHouse and GoogleSQL on the core catalog and analyzer while PostgreSQL, MySQL and SQLite still went through the legacy compiler. This routes all five dialects through the core.

What changed

  • compiler: NewCompiler takes options; WithCoreAnalysis, which the analyze command passes, builds a core catalog seeded with the engine's dialect and skips the legacy catalog and the analyzer connection entirely. ClickHouse and GoogleSQL keep doing so unconditionally. generate, vet and compile are untouched.
  • internal/core/seed: a declarative description of a dialect — its types and their aliases, the operators and casts between them, and the functions it ships with — that Apply turns into catalog rows. The PostgreSQL, MySQL and SQLite seeds are built on it and reuse each engine's existing function catalog (pg_catalog, and the MySQL and SQLite stdlibs), so count, max, upper and friends resolve with real return types.
  • internal/core: literals take the type each dialect names rather than PostgreSQL's (the analyzer hardcoded int4/numeric/text/bool, which only PostgreSQL has); a type a schema declares — an enum, or one of SQLite's free-form type names — gains the dialect's comparison operators; array types are named after their element.
  • internal/core/schema: enums, functions, views, CREATE TABLE AS, ALTER TABLE and the rename statements now load into the catalog, and a column's array-ness is read from either the type name or the column. Five new catalog queries back the ALTER TABLE mutations, generated by sqlc itself.
  • internal/core/analyzer: CTEs, set operations, subqueries in FROM and as expressions (including correlated ones), functions in FROM, IN, BETWEEN, CASE, COALESCE, array expressions, casts, and output-name references in GROUP BY/HAVING. Overloads are chosen by argument type, polymorphic return types (max(anyelement)) resolve to the argument's type, and an unknown function or operator leaves the expression untyped instead of failing the whole query.

Behavior change

analyze reports type names as the catalog stores them, in lower case — so SQLite's INTEGER is now integer. The analyze_basic/sqlite golden reflects that, and docs/howto/analyze.md calls it out.

Testing

Nine new end-to-end cases (analyze_select, analyze_dml, analyze_params for each of the three migrated dialects) cover joins, aggregates, CTEs, RETURNING, multi-table DELETE, and sqlc.arg/narg/slice.

As a wider check I ran sqlc analyze over every PostgreSQL/MySQL/SQLite schema+query pair in examples/ and internal/endtoend/testdata — 262 pairs, of which 253 pass. Roughly twelve of the remaining eighteen are intentionally-invalid fixtures (invalid_table_alias, sqlc_arg_invalid, relation_does_not_exist, MySQL files written with $1). The genuine gaps left are recursive CTEs, MySQL schema-qualified column references, SQLite virtual tables and table-valued functions (json_each), and one MySQL self-join that references a table by a name it aliased away.

TestReplay, TestParse and TestFormat pass. The TestValidSchema failures in my environment are pre-existing — no databases run there, and the count is identical on main.


Generated by Claude Code

claude added 2 commits July 31, 2026 19:28
`sqlc analyze` ran ClickHouse and GoogleSQL on the core catalog and
analyzer while PostgreSQL, MySQL and SQLite still went through the legacy
compiler. This routes all five dialects through the core.

- compiler: NewCompiler takes options; WithCoreAnalysis, which the analyze
  command passes, builds a core catalog seeded with the engine's dialect
  and skips the legacy catalog and the analyzer connection entirely.
  ClickHouse and GoogleSQL keep doing so unconditionally. `generate`,
  `vet` and `compile` are untouched.
- core/seed: a declarative description of a dialect — its types and their
  aliases, the operators and casts between them, and the functions it
  ships with — that Apply turns into catalog rows. PostgreSQL, MySQL and
  SQLite seeds are built on it, reusing each engine's existing function
  catalog (pg_catalog and the MySQL and SQLite stdlibs).
- core: literals take the type each dialect names rather than PostgreSQL's,
  a type a schema declares gains the dialect's comparison operators, and
  array types are named after their element.
- core/schema: enums, functions, views, CREATE TABLE AS, ALTER TABLE and
  the rename statements now load into the catalog, and a column's array-ness
  is read from either the type name or the column.
- core/analyzer: CTEs, set operations, subqueries in FROM and as
  expressions, correlated references, functions in FROM, IN, BETWEEN, CASE,
  COALESCE, array expressions, casts and output-name references in GROUP BY
  and HAVING. Overloads are chosen by argument type, polymorphic return
  types resolve to the argument's, and an unknown function or operator
  leaves the expression untyped instead of failing the query.

The SQLite golden changes because the core reports the type name the
catalog stores, which is lower case.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011MnoUabwBWW9gaEn2Nj7eG
Each engine now embeds a dialect.json and hands it to seed.Dialect, which
parses it into a Spec and applies it. An unknown field is an error, so a
misspelled key is caught where it is written rather than silently ignored.

ClickHouse and GoogleSQL move onto the seed package at the same time,
replacing their hand-rolled type and operator loops. Both gain the two
things the shared spec gives every dialect and their loops left out: the
type each literal takes on, and a count() aggregate.

Generated function catalogs stay in Go — pg_catalog and the MySQL and
SQLite stdlibs are passed to seed.Dialect rather than restated as JSON —
and a dialect can declare a handful of functions inline where there is no
generated list.

A new test loads every engine's dialect.json into a catalog and checks
that its literals and a sample of its type spellings resolve and compare.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011MnoUabwBWW9gaEn2Nj7eG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants