Skip to content

[FLINK-40577][table] Support ChangelogModes in ProcessTableFunctionTestHarness - #29130

Draft
autophagy wants to merge 1 commit into
apache:masterfrom
autophagy:FLINK-40577-changelog-modes
Draft

[FLINK-40577][table] Support ChangelogModes in ProcessTableFunctionTestHarness#29130
autophagy wants to merge 1 commit into
apache:masterfrom
autophagy:FLINK-40577-changelog-modes

Conversation

@autophagy

Copy link
Copy Markdown
Contributor

What is the purpose of the change

This PR extends the ProcessTableFunctionTestHarness so it can model changelog semantics, rather than assuming insert-only streams.

The harness lets the user supply a table argument's changelog mode and upsert keys directly. To do this, the PR ads a changelogMode per table argument, and (for upsert changelogs) one or more upsertKey candidates (each call adds another candidate).

For the output side, the harness derives the produced changelog mode. If the PTF implements ChangelogFunction its mode is resolved from that (insert-only otherwise).
Collected rows whose RowKind the resolved output mode does not allow are rejected.
Input rows and configurations that the planner could never produce (e.g. upsert output with
row-semantic table args) are likewise rejected.

Brief change log

  • Model update traits (SUPPORT_UPDATES, REQUIRE_UPDATE_BEFORE, REQUIRE_FULL_DELETE) on table arguments in the PTF test harness.
  • Add per-argument changelog-mode / upsert-key configuration, output-mode inference for ChangelogFunction PTFs, and input/output/config validation.

Verifying this change

This change added tests and can be verified as follows:

  • Added tests in ProcessTableFunctionTestHarnessTest, as well as tests for the changelog validation and resolvers.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (docs / JavaDocs)

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Opus 5)

@flinkbot

flinkbot commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants