core: analyze every dialect through the new analysis core - #4538
Open
kyleconroy wants to merge 2 commits into
Open
core: analyze every dialect through the new analysis core#4538kyleconroy wants to merge 2 commits into
kyleconroy wants to merge 2 commits into
Conversation
`sqlc analyze` ran ClickHouse and GoogleSQL on the core catalog and analyzer while PostgreSQL, MySQL and SQLite still went through the legacy compiler. This routes all five dialects through the core. - compiler: NewCompiler takes options; WithCoreAnalysis, which the analyze command passes, builds a core catalog seeded with the engine's dialect and skips the legacy catalog and the analyzer connection entirely. ClickHouse and GoogleSQL keep doing so unconditionally. `generate`, `vet` and `compile` are untouched. - core/seed: a declarative description of a dialect — its types and their aliases, the operators and casts between them, and the functions it ships with — that Apply turns into catalog rows. PostgreSQL, MySQL and SQLite seeds are built on it, reusing each engine's existing function catalog (pg_catalog and the MySQL and SQLite stdlibs). - core: literals take the type each dialect names rather than PostgreSQL's, a type a schema declares gains the dialect's comparison operators, and array types are named after their element. - core/schema: enums, functions, views, CREATE TABLE AS, ALTER TABLE and the rename statements now load into the catalog, and a column's array-ness is read from either the type name or the column. - core/analyzer: CTEs, set operations, subqueries in FROM and as expressions, correlated references, functions in FROM, IN, BETWEEN, CASE, COALESCE, array expressions, casts and output-name references in GROUP BY and HAVING. Overloads are chosen by argument type, polymorphic return types resolve to the argument's, and an unknown function or operator leaves the expression untyped instead of failing the query. The SQLite golden changes because the core reports the type name the catalog stores, which is lower case. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011MnoUabwBWW9gaEn2Nj7eG
Each engine now embeds a dialect.json and hands it to seed.Dialect, which parses it into a Spec and applies it. An unknown field is an error, so a misspelled key is caught where it is written rather than silently ignored. ClickHouse and GoogleSQL move onto the seed package at the same time, replacing their hand-rolled type and operator loops. Both gain the two things the shared spec gives every dialect and their loops left out: the type each literal takes on, and a count() aggregate. Generated function catalogs stay in Go — pg_catalog and the MySQL and SQLite stdlibs are passed to seed.Dialect rather than restated as JSON — and a dialect can declare a handful of functions inline where there is no generated list. A new test loads every engine's dialect.json into a catalog and checks that its literals and a sample of its type spellings resolve and compare. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011MnoUabwBWW9gaEn2Nj7eG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sqlc analyzeran ClickHouse and GoogleSQL on the core catalog and analyzer while PostgreSQL, MySQL and SQLite still went through the legacy compiler. This routes all five dialects through the core.What changed
NewCompilertakes options;WithCoreAnalysis, which the analyze command passes, builds a core catalog seeded with the engine's dialect and skips the legacy catalog and the analyzer connection entirely. ClickHouse and GoogleSQL keep doing so unconditionally.generate,vetandcompileare untouched.Applyturns into catalog rows. The PostgreSQL, MySQL and SQLite seeds are built on it and reuse each engine's existing function catalog (pg_catalog, and the MySQL and SQLite stdlibs), socount,max,upperand friends resolve with real return types.int4/numeric/text/bool, which only PostgreSQL has); a type a schema declares — an enum, or one of SQLite's free-form type names — gains the dialect's comparison operators; array types are named after their element.CREATE TABLE AS,ALTER TABLEand the rename statements now load into the catalog, and a column's array-ness is read from either the type name or the column. Five new catalog queries back theALTER TABLEmutations, generated by sqlc itself.FROMand as expressions (including correlated ones), functions inFROM,IN,BETWEEN,CASE,COALESCE, array expressions, casts, and output-name references inGROUP BY/HAVING. Overloads are chosen by argument type, polymorphic return types (max(anyelement)) resolve to the argument's type, and an unknown function or operator leaves the expression untyped instead of failing the whole query.Behavior change
analyzereports type names as the catalog stores them, in lower case — so SQLite'sINTEGERis nowinteger. Theanalyze_basic/sqlitegolden reflects that, anddocs/howto/analyze.mdcalls it out.Testing
Nine new end-to-end cases (
analyze_select,analyze_dml,analyze_paramsfor each of the three migrated dialects) cover joins, aggregates, CTEs,RETURNING, multi-tableDELETE, andsqlc.arg/narg/slice.As a wider check I ran
sqlc analyzeover every PostgreSQL/MySQL/SQLite schema+query pair inexamples/andinternal/endtoend/testdata— 262 pairs, of which 253 pass. Roughly twelve of the remaining eighteen are intentionally-invalid fixtures (invalid_table_alias,sqlc_arg_invalid,relation_does_not_exist, MySQL files written with$1). The genuine gaps left are recursive CTEs, MySQL schema-qualified column references, SQLite virtual tables and table-valued functions (json_each), and one MySQL self-join that references a table by a name it aliased away.TestReplay,TestParseandTestFormatpass. TheTestValidSchemafailures in my environment are pre-existing — no databases run there, and the count is identical onmain.Generated by Claude Code