Skip to content

feat(plugins): native delete and export for the engines with no SQL - #2899

Merged
datlechin merged 1 commit into
mainfrom
feat/non-sql-native-operations
Sep 15, 2026
Merged

datlechin merged 1 commit into
mainfrom
feat/non-sql-native-operations

Conversation

@datlechin

Copy link
Copy Markdown
Member

Follows #2896, which stopped the app inventing DROP TABLE for engines that have no SQL. That change was the root-cause half: once the app stopped guessing, four gaps it had been papering over became visible. This closes all four.

Elasticsearch and Weaviate export

ExportDataSourceAdapter asks the driver for defaultExportQuery and, on nil, builds SELECT * FROM "<object>". Elasticsearch and Weaviate implement neither that hook nor streamRows, so Export sent them a statement they have no parser for and the file came out empty.

Both now answer with an export tag their own streamRows decodes and pages:

  • Elasticsearch opens a point-in-time and walks it with search_after, so an export is not capped at the 10,000-document max_result_window. The point-in-time also pins the index for the whole run, so a document written while it streams cannot shift the sort and be exported twice or skipped.
  • Weaviate pages GET /v1/objects by offset.

A tag rather than a query with a large size, because the protocol's default streamRows runs execute once and buffers the whole result. That is fine for a grid page and not for an index.

Kafka topic delete

Kafka had no dropObjectStatement, so after #2896 a topic could not be deleted at all. It now answers DROP TOPIC <name>, which is also a KafkaQL verb, and the driver issues DeleteTopics.

That needed more than a statement. KafkaApiKey had no DeleteTopics case (API key 20) at all; KafkaCluster parsed controllerId only to print "yes" in the cluster view and had no way to reach the controller, which is the only broker that accepts the request; and KafkaErrorCode had neither NOT_CONTROLLER nor TOPIC_DELETION_DISABLED. All three are added. Capped at v5 deliberately: v6 addresses topics by UUID instead of by name, the same line KafkaApiKey already draws for Fetch and Metadata.

DROP TABLE stays a Kafka syntax error. A topic is not a table, and that text is exactly what the app used to invent for engines with no SQL, so accepting it would make the mistake look supported.

Truncate is not offered. Kafka removes records by retention or by an offset per partition, neither of which empties a topic the way Truncate means.

DynamoDB table delete

PartiQL has no DDL, so the driver returned nil and the table could not be deleted. It now answers DROP TABLE "<name>" and execute routes that to the DeleteTable API. The spelling is deliberate: the confirmation shows the statement verbatim, and the generic classifier tiers a leading DROP as destructive without needing a DynamoDB arm. The cached table description is dropped with it, or a table recreated under the same name would be read through the old key schema.

Truncate stays unoffered: emptying a DynamoDB table means scanning it and deleting every item, which is a long billed job rather than a statement.

iOS

TableProMobile/Views/TableListView.swift built TRUNCATE TABLE and DROP TABLE in the view for every engine, gated only on the object being a view and on Safe Mode. On Redis the rows are keys, so a long-press offered Drop Table and sent DROP TABLE "session:42", which the driver tokenised as an unknown Redis command.

The policy moved to TableProConnectionLibrary, keyed by the raw database type id, and both apps read it. That target is one of five the two apps already link, and it carries no plugin ABI, so this needs no currentPluginKitVersion bump. A string key rather than a DatabaseType because the two apps do not share that type: the Mac app has its own and iOS has TableProCoreTypes.DatabaseType, and their constant lists have already drifted (36 versus 33, with Elasticsearch, PGlite and Typesense missing from the shared one).

Tests

  • KafkaIntegrationTests gains two cases that drop a topic through the driver against a real Apache Kafka broker and wait for it to leave the cluster listing. Hand-written wire protocol is only worth trusting against a real broker, and scripts/kafka-test-broker.sh provides one.
  • KafkaQLTests covers DROP TOPIC and keeps DROP TABLE as a syntax error.
  • DynamoDBOperationsTests round-trips every generated drop back to its table name, refuses names DynamoDB could not have, and asserts the statement classifies destructive.
  • SQLDDLFallbackPolicyTests on iOS asserts Redis is the only engine iOS ships without SQL DDL, walking IOSDriverFactory.supportedTypes() so a new non-SQL driver fails it.

Verified

Build PASS. 278 cases across 17 suites PASS, including 28 Kafka integration cases against a live broker. Elasticsearch, Weaviate, DynamoDB and Kafka plugin schemes each build. swiftlint --strict introduces no new violation in any changed file, checked by comparing each file against its state at HEAD.

The iOS app does not build in this checkout, for a pre-existing reason: the vendored oracle-nio fails with a @TaskLocal macro error. No Oracle file is in this change, and no error names a file it touches.

Not in this change

  • ExportDataSourceAdapter also fabricates a scoped SELECT for a row-range export, on a path that never consults defaultExportQuery. Gating that one needs a scope-aware driver hook.
  • SQLExportPlugin writes DROP IF EXISTS for every engine, which needs an export-format flag rather than the DDL policy.
  • Kafka's SELECT * FROM topic is accepted as CONSUME sugar and exports only the newest 100 messages with no warning. Bounding it honestly is a product decision about what exporting a live topic should mean.
  • On iOS, SQLBuilder, the query editor's SELECT template and DataBrowserViewModel build literal SQL for every engine, so Redis row browsing there is broken beyond these two menu items.

@mintlify

mintlify Bot commented Sep 15, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
TablePro 🟢 Ready View Preview Sep 15, 2026, 12:33 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@datlechin
datlechin merged commit 3190110 into main Sep 15, 2026
7 of 8 checks passed
@datlechin
datlechin deleted the feat/non-sql-native-operations branch September 15, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant