Skip to content

[kafka] Integrate Produce with native Fluss append #4283

Description

@gyang94

Purpose

Part of #4185. PR07 of the Kafka compatibility sequence.

Wire the Produce handler and raw/string transcoder to the TabletServer append gateway. Apply the same table admission contract as Metadata, isolate conversion and append errors per partition, map acknowledgements and offsets, and register Produce v3-v11.

Acceptance criteria

  • Only compatible pre-created tables and valid bucket IDs reach append.
  • One invalid table or partition does not suppress independent valid writes.
  • Gateway acks/timeout and partition offsets/replication errors map correctly.
  • Kafka calls tryCompleteActions immediately after synchronous produceLog invocation, including throws, without waiting for async acknowledgements.
  • A real Kafka producer discovers native DDL tables and writes raw/string records with acks=0/1/all; native Fluss readback verifies data and metadata.

Dependencies

PR06 raw/string conversion (#4282) and independent core PR #3412 / #3455. Core changes remain owned by #3412.

Scope boundary

No new core implementation, Kafka table creation, JSON, authentication, idempotence or transactions.

Implementation

Draft PR #4286: #4286

Java 11 validation: 82 unit tests and 4 integration tests passed at dc5fb6599b8c5e155cb17280df71a74ee34c9ffa. Review and merge remain pending.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions