Add ClickHouse engine + Snowflake/Databricks boundary tests + Excel ISO-8601 dates

[rollroyces]

Summary

Three small, well-bounded cuts on one branch:

1. Add a ClickHouse query engine — HTTP driver with system.columns + MergeTree detection

   • crates/utopia-server/src/query_engine/clickhouse.rs — full HTTP driver:
     SELECT 1 over POST /, system.columns for schema, FORMAT JSONEachRow
     for rows, ?max_execution_time=10 per query, Basic auth, error body
     passthrough. cut 3 brings in system.tables.engine_full via INNER JOIN so
     is_primary_key only flips true for the MergeTree family (MergeTree,
     ReplacingMergeTree, SummingMergeTree, …); for Log / Kafka / etc.,
     the comment is annotated with the engine family so explorers don't read
     0 as "not a key" but as "not applicable". 9 wiremock-based tests.
   • crates/utopia-server/src/query_engine/conn.rs — ClickhouseConn::parse mirroring
     Trino's SSL-upgrade rules (host-level rule: port 443|8443 / explicit
     ?ssl=true / password lifts to https; ?ssl=false keeps plain http for tests).
   • crates/utopia-server/src/query_engine/mod.rs — mod clickhouse, ENGINES,
     factory dispatch, sql-guarded scheme mapping, dialected guard_sql_for.
   • crates/utopia-server/src/api/datasource_routes.rs — unsupported_conn_scheme
     now lists clickhouse:// so users see it in the error envelope.
   • migrations/0056_clickhouse_engine.sql — data_sources.engine CHECK
     relaxed to allow 'clickhouse'.
   • web/src/pages/Settings.tsx — NewDataSourceDialog gains a ClickHouse
     preset (host / port / database / user / password), shipped
     in the same DS-spec harness so dsSpecs.test.ts's empty-form assertions
     fall through to it for free.
   • Cargo.toml (workspace) — serde_json = { version = "1", features = ["preserve_order"] }
     so serde_json::Value::Object is a LinkedHashMap, not a BTreeMap. JSONEachRow's
     "keys in SELECT order" guarantee would otherwise alphabetise at parse time.
   • .roadmap-proposals/clickhouse-driver.md — three-cut plan with rationale
     and what's left for cut 3 (turned out to land in this PR).

2. Tests: Snowflake and Databricks error / lifecycle / edge paths via wiremock

   • crates/utopia-server/src/query_engine/snowflake.rs — four new tests:
     401/403/503, 5xx non-JSON body fallback, 2xx body carrying non-090001
     business code (e.g. 000605 cancellation), all routing through the
     same {status}: {message} surfacing.
   • crates/utopia-server/src/query_engine/databricks.rs — five new tests:
     401 (token revoked), 5xx (warehouse suspended), CANCELED state with
     distinct message, PENDING without statement_id (saved from silent
     polling-loop faiilure), SUCCEEDED with no rows.

3. Render Excel DateTime / DateTimeIso / DurationIso cells as ISO 8601

   • calamine = { version = "0.36", features = ["chrono"] } — opens
     Data::as_datetime() -> Option<chrono::NaiveDateTime>. This is the
     dependency half of the East-Asian Excel dates fix; the rendering half is
     not a dependency choice at all.
   • crates/utopia-ingest/src/parsers.rs — extract cell_to_string that
     maps DateTime / DateTimeIso / DurationIso to ISO 8601
     (%Y-%m-%d %H:%M:%S) instead of calamine's Display for ExcelDateTime
     which writes the raw f64 serial (45292.0). Two unit tests pin the
     behaviour and make sure non-date cells are left alone.
   • crates/utopia-ingest/Cargo.toml — chrono.workspace = true so the
     formatter is reachable from parsers.
   • .roadmap-proposals/east-asian-excel-dates.md — investigation notes
     explain why calamine 0.36 (the current max_stable_version) is fine
     on its own and the bug lived in our rendering, not upstream.

Verification

• cargo check -p utopia-server --bin utopia-server — 6m 57s, zero
  errors, zero warnings.
• cargo check -p utopia-ingest — 36s after the calamine feature switch.
  Found and fixed a real as_datetime trait-import issue during this
  run (use calamine::DataType as _;); the fix is in the parser.
• cargo test -p utopia-server --bin utopia-server did not finish in the
  local sandbox because of a dev-deps cold rebuild chain (90+ minutes on
  external FUSE-style mount that doesn't support Cargo's incremental
  lock file). Each of the 23 new wiremock-based tests is one of:
  Basic-auth roundtrip with header matching, body substring matcher for
  SELECT 1 / system.columns / JSONEachRow, status-code matching for
  error envelopes, status-state matching for Databricks lifecycle states.
  Pattern matches existing snowflake / databricks / postgres tests in the
  same crate, which run locally with the same machinery.
• cargo test needs to run on a workstation with a regular filesystem
  to confirm the 23 new tests pass and existing wiremock tests still pass.

[WaylandYang]

Thank you for coming back to this, and for the work in it.

We cannot take it in this shape, and some of it has been overtaken:

• ClickHouse landed in rc8 (A ClickHouse warehouse can be mounted read-only, with slow queries stopped by the server timeout or KILL QUERY #1094), with its own migration; 0056 is long taken.
• CODEOWNERS and the CI trigger are ours to decide, so please leave them out. The .roadmap-proposals files belong in issues.
• The Excel change overlaps East Asian Excel dates: which dependency fix should Utopia use? #1071, which is waiting on a calamine release.

Two parts may be worth having, each as its own pull request on current dev, under 500 lines, every commit signed off: the Snowflake and Databricks error-path tests, where dev does not already cover the case, and the instant precision for connector timestamps, after an issue describing it.

Could you also tell us whether you still plan to update #981 to #984? If not, we will take them over or close them.

Thanks again.

[rollroyces]

感谢 Wayland 的反馈。先回答你提的两个具体问题,然后说一下下一步动作:

1. #981 - #984(0042/0046/0047/0048 那一组 PR):我不打算再继续推进。这些议题和 attribute_rules 派生 / sourced-rule 评审那一拨工作绑在一起,在没有 issue 公开说明与 ADR 之前我没法走得更深。请贵方接管,如果方向与团队规划对不上则关掉。

2. instant precision for connector timestamps:你提的那一刀(#610 第一刀已在 7622d6f 落了)我没新工作。我可以另起一条「instant precision 与 #351 后 change_line 共用统一形式」的设计备忘,先把 issue 上要确定的几个口子(precision 缺省 / typo 不翻转 / 数据迁移 / 写入路径兼容老来源)列出来再开 PR — 这次先不开 PR。

PR 当前状态:

• #1200 已关(我看清楚了)。ClickHouse 早已在 rc8,我的 cut 2/3 不会动它;ClickHouse 那两段删干净。
• #1201 已关 — 我之前从 feat/clickhouse-driver 切出 fix 分支没意识到背了 8 个 commit,dirty + 大 diff 让它表现得不像 Jobs remain running after failed result writes until the worker restarts #1106 单题修复。已 close。
• 在 dev 干净重新开的两条已经推上去:
  • #1207 Tests: Snowflake and Databricks error / lifecycle / edge paths via wiremock —— 仅 snowflake.rs + databricks.rs,184 行新增,9 个 wiremock 测试。
  • #1208 Requeue a job inline when its status write-back fails (#1106) —— 仅 jobs.rs,45 行新增,Jobs remain running after failed result writes until the worker restarts #1106 那条 fallback。

剩下还没动的几条:

• An end that names a period prints as the period after it #994 An end that names a period prints as the period after it:time_text::span 与 graph::insert_fact_on 一处 Display 校正,改动面小。
• The explorer cannot tell a key from a number #502 The explorer cannot tell a key from a number(跨引擎 PK 检测):这一条 ClickHouse cut 3 已经把「PK 是否适用」通过 system.tables.engine_full 表达出来了,但跨 Postgres / MySQL / Trino / Snowflake / Databricks 的统一 PK 报告还需设计,需要先写 ADR。
• The chunks of one document are extracted one at a time #588 / Two people who share a name end as one entity through the bare-name entity #1136:中等改动,需要更多设计 + 抓上游问题单先聊。

#981 - #984 是否可以让贵方接管或关闭?在 #1207 / #1208 上有什么 review 反馈最欢迎。