Skip to content

[spark] Close the commit after truncating a table or partitions - #9900

Open
cxzl25 wants to merge 1 commit into
apache:masterfrom
cxzl25:spark-truncate-close-commit
Open

cxzl25 wants to merge 1 commit into
apache:masterfrom
cxzl25:spark-truncate-close-commit

Conversation

@cxzl25

@cxzl25 cxzl25 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Purpose

Both Spark truncate paths create a BatchTableCommit and never close it:

  • TruncatePaimonTableWithFilterExec.run() handles TRUNCATE TABLE with or
    without a partition spec on the driver.
  • PaimonSparkTableBase.truncateTable implements Spark's TruncatableTable.

TableCommitImpl.close() shuts down the underlying commit and its
maintainExecutor thread pool, so every TRUNCATE TABLE leaked a thread pool
on the driver. Both call sites now wrap the commit in try/finally, matching
PaimonFormatTable.withCommit and the other commit call sites in this module.

Tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant