Parent issue
Part of #4185 — Kafka Protocols Compatibility: Basic Produce.
Motivation
Define the DDL table mapping contract before Metadata and Produce so both APIs agree on which Fluss tables can be exposed as Kafka topics and how records map to physical columns. This is PR03 in the foundation sequence; Metadata is PR04 (#4266).
Scope
- Map a topic name to the same-named table in the configured Kafka database, table IDs to stable topic IDs, and Kafka partitions to Fluss buckets.
- Define the initial supported table shape: non-partitioned Arrow log tables without a primary key.
- Resolve
kafka.* custom properties for raw/string key and value formats, field projections, and optional timestamp/headers columns.
- Validate field names, types, projection conflicts, and unsupported mapping options through a shared resolver that Metadata and later Produce can reuse.
- Cover valid/invalid mappings and a native Fluss create-table → metadata-readback round trip.
Acceptance criteria
Dependencies
Depends on #4265, which builds on #4264. This issue is a prerequisite for Metadata (#4266) and subsequent Produce capabilities.
Out of scope
Kafka CreateTopics/DeleteTopics, automatic table creation, Metadata request handling, Produce, byte decoding, Arrow encoding, JSON conversion, and authentication.
Implementation and review
Implementation PR: #4277 (Draft), from gyang94:codex/kafka-ddl-table-mapping, based directly on PR02 (#4260). Local Java 11 validation passed 47 unit tests and 2 integration tests, including the native DDL round trip, plus Checkstyle, Spotless and RAT checks. These results do not imply remote CI or review completion.
Parent issue
Part of #4185 — Kafka Protocols Compatibility: Basic Produce.
Motivation
Define the DDL table mapping contract before Metadata and Produce so both APIs agree on which Fluss tables can be exposed as Kafka topics and how records map to physical columns. This is PR03 in the foundation sequence; Metadata is PR04 (#4266).
Scope
kafka.*custom properties for raw/string key and value formats, field projections, and optional timestamp/headers columns.Acceptance criteria
Dependencies
Depends on #4265, which builds on #4264. This issue is a prerequisite for Metadata (#4266) and subsequent Produce capabilities.
Out of scope
Kafka CreateTopics/DeleteTopics, automatic table creation, Metadata request handling, Produce, byte decoding, Arrow encoding, JSON conversion, and authentication.
Implementation and review
Implementation PR: #4277 (Draft), from
gyang94:codex/kafka-ddl-table-mapping, based directly on PR02 (#4260). Local Java 11 validation passed 47 unit tests and 2 integration tests, including the native DDL round trip, plus Checkstyle, Spotless and RAT checks. These results do not imply remote CI or review completion.