Skip to content

[cdc] Support per-table configuration for MySQL database CDC - #9871

Merged
JingsongLi merged 2 commits into
apache:masterfrom
lilei1128:cdc-table-config
Sep 17, 2026
Merged

JingsongLi merged 2 commits into
apache:masterfrom
lilei1128:cdc-table-config

Conversation

@lilei1128

Copy link
Copy Markdown
Contributor

Purpose

  • Add --table_conf_by_table for mysql_sync_database.
  • Allow source tables to override global --table_conf table properties.
  • Support per-table configuration in divided and combined modes.
  • Reject per-table sink/job runtime options such as sink.parallelism.
  • Document the new configuration and inheritance rules.

Tests

  • Add parser unit tests for per-table configuration and duplicate keys.
  • Add validation test for rejecting per-table sink configuration.
  • Add MySQL CDC integration tests for divided and combined modes.
  • Add coverage for per-table overrides and global configuration inheritance.

@JingsongLi

Copy link
Copy Markdown
Contributor

Useful feature — per-table engine options remove the all-or-nothing --table-conf for database sync.

One issue: CdcActionCommonUtilsTest.testRejectSinkConfiguration() is missing its @Test annotation, so it never runs. It covers exactly the new validation (rejecting per-table sink.* keys), so it should be executed.

One behavior question: per-table keys that match no synchronized source table are silently accepted (parseTableConfigByTable just builds the map, and the override is looked up by exact table name). A typo such as oder:bucket=8, or a case mismatch, configures nothing with no signal at all. A warning listing unmatched keys (or validating against the discovered source tables) would save users a lot of debugging.

@JingsongLi
JingsongLi merged commit ce6884b into apache:master Sep 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants