A ClickHouse warehouse can be mounted read-only, with slow queries stopped by the server timeout or KILL QUERY

[asaf-shitrit]

Why

Closes #1092.

What changes

clickhouse://user[:password]@host[:port]/[database] mounts a warehouse over HTTP. ssl=true, secure=true, or port 443 or 8443 selects https.

Each request asks for readonly=1 and max_execution_time. If the account's profile refuses one of them, the engine sends the request again without it. It drops readonly only for an account that is read-only itself. Each request has a query_id. The engine kills the query after 12 s, when the caller cancels the request, or when the connection drops during the reply. A gateway error in front of ClickHouse, such as a 504, also starts a KILL.

Some accounts do not let the server hold the shared 10 s statement timeout: a readonly=1 or readonly=2 profile, or a CONST timeout. For these accounts the KILL needs GRANT SELECT ON system.processes. An account in readonly=1 mode whose own max_execution_time is 10 s or less also works. The engine checks this before it sends a query without the server timeout. If the check fails, the engine refuses every query and the Test connection button, with a message that names the grant.

is_primary_key stays false: a ClickHouse primary key allows duplicate rows. data_type keeps Nullable(...) and LowCardinality(...). The engine reads JSONCompactStrings and converts values by column type.

The quoting of 64-bit integers in JSON changes with the server version: output_format_json_quote_64bit_integers is off by default from 25.8. A read-only account cannot change this setting. Unquoted 128-bit and 256-bit integers also lose precision. So integers that fit in 64 bits come back as exact JSON numbers. Larger 128-bit and 256-bit values, and Decimal(P, S) values with P above 15, come back as strings.

Limits:

• When the URL names a database, fetch_schema reads only that database. Without one, it reads every non-system database.
• In a Nullable(String) column, JSONCompactStrings writes NULL and the text ᴺᵁᴸᴸ the same way, so both read as NULL. With the ASCII grid charset the marker is NULL, and both read as the text NULL. This is the cost of exact integers.
• ClickHouse has no Settings form and uses the connection string box, as Utopia cannot mount a ClickHouse warehouse #1092 asked.
• A password does not select https, unlike Trino. ClickHouse accepts passwords on its plain HTTP port, and many self-hosted servers have only that port.
• A normal account sends one request per query. The other accounts send three or four, because engines keep no state between calls. These are the refused attempt, a KILL check, a setting probe when necessary, and the query.
• Behind a load balancer with several replicas, a KILL can reach a different replica. The engine then reports that the query can still run. For such setups, give the account a server-side max_execution_time.

The roadmap line also names Feishu, so I moved ClickHouse to the feature table, as #315 did for MySQL. The Chinese feature table lists no engines.

How it was checked

The live test passed on 26.9.12.8 with four accounts: the server's default user, a readonly=1 profile, a readonly=2 profile and a CONST timeout. Each account refused a write, and no slow query stayed on the server. The engine refused an account without the grant, on a query and on the Test connection check.

I also checked these cases on a real server:

• A cancel by the caller, and a connection that drops during the query. Each time, the engine sent a KILL and the query stopped.
• A readonly=1 account with a 5 s max_execution_time and no grant. The engine accepted it, and the server stopped a slow query at 5 s.
• An error after part of the result, on 25.3 and 26.9, with and without compression. The error holds only the server's message.

Before review

• Every commit is signed off (git commit -s)
• cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass
• For changes under web/: pnpm build and pnpm test pass
• A new migration takes the next free number on dev, and CURRENT_SCHEMA_VERSION in crates/utopia-cli/src/main.rs equals the number of files in migrations/
• UI strings are in both web/src/i18n/en.ts and zh.ts

[WaylandYang]

Thank you. This is careful work, and every point from #1092 is answered with a measurement.

I ran it against a real server (26.9.12.8 in Docker, the fixture from the doc comment):

account	Test connection	slow query, as recorded in system.query_log
default	ok	stopped by the server at 10 s (159)
readonly=1 with the grant	ok	killed at 12 s (394)
readonly=2 with the grant	ok	stopped by the server at 10 s (159)
CONST timeout with the grant	ok	killed at 12 s (394)
readonly=1 without the grant	refused, message names the grant	none sent

• The live tests pass with each of the four accounts, and no utopia- query is left in system.processes afterwards.
• Through the API on a fresh database: migration 0106 applies, and create, grant, mount and schema sync work with the readonly=1 account.
• A wrong password and a closed port each return their own error.

One correction, and it is mine. I wrote on #1092 that Settings has a single connection-string box. That was wrong: each engine has its own form, and the box is the last option. You followed what I said, so I pushed 59b086d to add the ClickHouse form (host, port, optional database, user and password) with a test. The same commit names MySQL and ClickHouse in the query tool's dialect list, which had been missing MySQL since #303.

For the record, not a request: the gate parses with sqlparser 0.62, which rejects four ClickHouse forms I tried, namely WITH <expr> AS name, tuple access t.1, GLOBAL IN and ASOF JOIN … ON. Seventeen other idioms pass. The gate fails closed, so this limits what the model can write and does not open anything.

Merging when CI is green on the new commit. Thanks again.

[asaf-shitrit]

Thank you for the review, and for the ClickHouse form in Settings. The note about the four forms that sqlparser rejects is useful to know. 🙏