Skip to content

fix(dataset): Restrict FlatXmlProducer empty-table backfill to DTD me… #952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jeffjensen wants to merge 3 commits into from
Closed

fix(dataset): Restrict FlatXmlProducer empty-table backfill to DTD me… #952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4674c5d
fix(dataset): Restrict FlatXmlProducer empty-table backfill to DTD me…
jeffjensen Aug 20, 2026
2c76618
chore(release): Prep 3.5.1
jeffjensen Aug 20, 2026
a1f5944
docs(site): Update index.adoc with 3.5.1 release
jeffjensen Aug 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

<groupId>org.dbunit</groupId>
<artifactId>dbunit</artifactId>
<version>3.5.1-SNAPSHOT</version>
<version>3.5.1</version>
<packaging>jar</packaging>
<name>dbUnit Extension</name>
<url>https://github.com/dbunit/dbunit-extension</url>
Expand All @@ -23,7 +23,7 @@

<properties>
<!-- Maven configuration -->
<project.build.outputTimestamp>2026-08-12T02:19:02Z</project.build.outputTimestamp>
<project.build.outputTimestamp>2026-08-20T13:48:02Z</project.build.outputTimestamp>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<project.scm.id>github</project.scm.id>
Expand Down Expand Up @@ -154,7 +154,7 @@
<connection>scm:git:https://github.com/dbunit/dbunit-extension.git</connection>
<developerConnection>scm:git:https://github.com/dbunit/dbunit-extension.git</developerConnection>
<url>https://github.com/dbunit/dbunit-extension.git</url>
<tag>dbunit-3.5.0</tag>
<tag>dbunit-3.5.1</tag>
</scm>
<issueManagement>
<system>github</system>
Expand Down Expand Up @@ -1175,6 +1175,33 @@
</configuration>
</reportSet>
-->
<reportSet>
<id>3.5.0-to-3.5.1</id>
<reports>
<report>cmp-report</report>
</reports>
<configuration>
<parameter>
<reportLinkName>3.5.0-to-3.5.1</reportLinkName>
</parameter>
<oldVersion>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>3.5.0</version>
<type>jar</type>
</dependency>
</oldVersion>
<newVersion>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>3.5.1</version>
<type>jar</type>
</dependency>
</newVersion>
</configuration>
</reportSet>
<reportSet>
<id>3.4.0-to-3.5.0</id>
<reports>
Expand Down
5 changes: 5 additions & 0 deletions src/changes/changes.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
</properties>

<body>
<release version="3.5.1" date="Aug 20, 2026" description="A regression fix for FlatXmlProducer incorrectly treating any explicitly-supplied metadata IDataSet as a DTD-style enumeration of the fixture's tables">
<action dev="jeffjensen" type="fix" issue="951" system="github" due-to="tkrah">
Fix a 3.5.0 regression (introduced by issue #496's fix) where FlatXmlProducer added every table from an explicitly-supplied metadata IDataSet as an empty table, not just tables actually present in the flat XML body: FlatXmlDataSetBuilder#setMetaDataSet(IDataSet) is documented as supplying column metadata only, but a broad metadata source such as a live database's full IDataSet (e.g. via DatabaseConnection#createDataSet()) was being read as if it enumerated the fixture's own tables, so DELETE_ALL/CLEAN_INSERT ended up touching every table in that broader source instead of only the ones the XML body mentions. FlatXmlProducer's issue #496 empty-table backfill now only runs when the metadata source is DTD-derived (a FlatDtdDataSet, whether parsed inline from the flat XML's own DOCTYPE or supplied via FlatXmlDataSetBuilder#setMetaDataSetFromDtd), restoring pre-3.5.0 behavior for any other explicitly-supplied metadata dataset.
</action>
</release>
<release version="3.5.0" date="Aug 11, 2026" description="A documentation site overhaul (new tutorials, 10 database vendor guides, a class-by-class Core Components reference, and a new Developing DbUnit contributor section); several new capabilities including DbUnitExtension for native JUnit 5/6 lifecycle management, a JSON dataset format with a matching JSON ValueComparer, MariaDB and H2 2.x database support, expanded PostgreSQL json/jsonb and array type support, and an opt-in escape hatch for schemas with circular foreign keys; and a broad set of correctness and performance fixes spanning metadata handling, DTD/XML export, PostgreSQL null-safety and large-object handling, and FlatXmlProducer memory use">
<action dev="jeffjensen" type="add" issue="840" system="github" due-to="jeffjensen">
Add repo-root README.adoc, rendered natively by GitHub via Asciidoctor, so the repository landing page shows a pitch, build/reproducible-build badges, a pointer to the "dbUnit in 5 Minutes" tutorial, and links to the documentation site, Maven coordinates, GitHub Discussions, and CONTRIBUTING.md instead of nothing.
Expand Down
19 changes: 11 additions & 8 deletions src/main/java/org/dbunit/dataset/xml/FlatXmlProducer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,19 +333,22 @@ private boolean isNewTable(String tableName)
}

/**
* Notifies the consumer of every table declared in {@link #_metaDataSet} (DTD or
* explicit metadata dataset) that never appeared as a row element in the XML body,
* as an empty table using that source's column metadata. Without this, a table with
* zero rows in a given fixture is silently absent from the produced dataset, which
* can make operations like {@code CLEAN_INSERT}/{@code DELETE_ALL} skip it entirely
* even though the DTD declares it. No-op when no DTD/metadata dataset is available,
* so behavior is unchanged for plain flat XML.
* Notifies the consumer of every table declared in a DTD-derived {@link #_metaDataSet}
* that never appeared as a row element in the XML body, as an empty table using that
* source's column metadata. Without this, a table with zero rows in a given fixture is
* silently absent from the produced dataset, which can make operations like
* {@code CLEAN_INSERT}/{@code DELETE_ALL} skip it entirely even though the DTD declares
* it. No-op when {@link #_metaDataSet} is not a {@link FlatDtdDataSet}, i.e. when there
* is plain flat XML or an arbitrary metadata {@link IDataSet} supplied via
* {@link FlatXmlDataSetBuilder#setMetaDataSet}: such a dataset is consulted only for
* column metadata of tables actually present in the XML body, not as an enumeration of
Comment thread
coderabbitai[bot] marked this conversation as resolved.
* every table the produced dataset should contain.
*
* @throws DataSetException if the consumer cannot be notified.
*/
private void addMissingDtdTables() throws DataSetException
{
if (_metaDataSet == null)
if (!(_metaDataSet instanceof FlatDtdDataSet))
{
return;
}
Expand Down
18 changes: 13 additions & 5 deletions src/site/asciidoc/index.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,22 @@ We will gladly help you as needed with your ideas and contributions and look to
|Date |News

// |TBD
// |Please try the 3.5.0-SNAPSHOT snapshot build and let us know how it works!
// It adds `DbUnitExtension` for *native JUnit 5/6 lifecycle management*, a new *JSON dataset format* and JSON `ValueComparer`, *MariaDB and H2 2.x* database support, expanded PostgreSQL `json`/`jsonb` and array type support, and an opt-in escape hatch for *circular foreign keys*, plus a ground-up *documentation site overhaul* and numerous other bug fixes.
// |Please try the 3.5.1-SNAPSHOT snapshot build and let us know how it works!
// It fixes a 3.5.0 regression where `FlatXmlDataSetBuilder.setMetaDataSet()` pulled *every table in the supplied metadata source* into the produced dataset, not just the ones in the flat XML body.
// See link:https://dbunit.github.io/dbunit-extension/repos.html#snapshots[SNAPSHOTS] for how to use them.
// Refer to the link:changes.html#a3.5.0-SNAPSHOT[changes report],
// the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.0%20type%3AFeature[feature list], and
// the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.0%20type%3ABug[bug list]
// Refer to the link:changes.html#a3.5.1-SNAPSHOT[changes report],
// the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.1%20type%3AFeature[feature list], and
// the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.1%20type%3ABug[bug list]
// for the snapshot contents (and subsequent updates).

|2026-08-20
|Release 3.5.1 available.
It fixes a 3.5.1 regression where `FlatXmlDataSetBuilder.setMetaDataSet()` pulled *every table in the supplied metadata source* into the produced dataset, not just the ones in the flat XML body.
Refer to the link:changes.html#a3.5.1[changes report],
the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.1%20type%3AFeature[feature list], and
the link:https://github.com/dbunit/dbunit-extension/issues?q=is%3Aissue+milestone%3A3.5.1%20type%3ABug[bug list]
for the release contents.

|2026-08-11
|Release 3.5.0 available.
It adds `DbUnitExtension` for *native JUnit 5/6 lifecycle management*, a new *JSON dataset format* and JSON `ValueComparer`, *MariaDB and H2 2.x* database support, expanded PostgreSQL `json`/`jsonb` and array type support, and an opt-in escape hatch for *circular foreign keys*, plus a ground-up *documentation site overhaul* and numerous other bug fixes.
Expand Down
1 change: 1 addition & 0 deletions src/site/site.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@
<!--
<item name="3.5.0 to snapshot" href="/3.5.0-to-snapshot.html"/>
-->
<item name="3.5.0 to 3.5.1" href="/3.5.0-to-3.5.1.html"/>
<item name="3.4.0 to 3.5.0" href="/3.4.0-to-3.5.0.html"/>
<item name="3.3.0 to 3.4.0" href="/3.3.0-to-3.4.0.html"/>
<item name="3.2.0 to 3.3.0" href="/3.2.0-to-3.3.0.html"/>
Expand Down
60 changes: 54 additions & 6 deletions src/test/java/org/dbunit/dataset/xml/FlatXmlProducerTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ void testProduceMetaDataSet_withMetaDataSetProvided_usesMetaDataSetColumnsForEmp
}

@Test
void testProduceMetaDataSet_withTableAbsentFromXmlBody_addsEmptyTableFromMetaDataSet() throws Exception
void testProduceMetaDataSet_withNonDtdMetaDataSetAndTableAbsentFromXmlBody_doesNotAddMissingTable()
throws Exception
{
// Setup consumer
final String presentTable = "PRESENT_TABLE";
Comment on lines 161 to 166

@sourcery-ai sourcery-ai Bot Aug 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add a test ensuring DTD tables that do appear in the XML body are not duplicated by the backfill

To fully exercise addMissingDtdTables(), please add a complementary test where a DTD-declared table also appears in the XML body and verify that only one table is produced (no extra empty backfilled table). This will guard against regressions where backfill logic re-emits already-seen tables.

Suggested implementation:

    @Test
    void testProduceMetaDataSet_withNonDtdMetaDataSetAndTableAbsentFromXmlBody_doesNotAddMissingTable()
            throws Exception
    {
        // Setup consumer
        final String presentTable = "PRESENT_TABLE";
        final MockDataSetConsumer consumer = new MockDataSetConsumer();
        consumer.addExpectedStartDataSet();
        consumer.addExpectedEmptyTable(presentTable, presentColumns);
    }

    @Test
    void testProduceMetaDataSet_withDtdMetaDataSetAndTablePresentInXmlBody_doesNotDuplicateTable()
            throws Exception
    {
        // Setup consumer
        final String presentTable = "PRESENT_TABLE";
        final MockDataSetConsumer consumer = new MockDataSetConsumer();
        consumer.addExpectedStartDataSet();

        // Expect exactly one non-empty table event for the DTD-declared table that is also present in the XML body
        // (adapt the row values to match the XML body used by this test)
        consumer.addExpectedTable(presentTable, presentColumns, new Object[][] {
                { "row1col1", "row1col2" }
        });
        consumer.addExpectedEndDataSet();

        // Build a FlatXmlProducer whose MetaDataSet comes from a DTD and where the same table appears in the XML body.
        // The important part is that PRESENT_TABLE is declared in the DTD and also has at least one row in the XML body.
        final String xmlWithDtdAndPresentTable =
                "<?xml version=\"1.0\"?>\n" +
                "<!DOCTYPE dataset [\n" +
                "  <!ELEMENT dataset (PRESENT_TABLE*)>\n" +
                "  <!ELEMENT PRESENT_TABLE EMPTY>\n" +
                "]>\n" +
                "<dataset>\n" +
                "  <PRESENT_TABLE col1=\"row1col1\" col2=\"row1col2\"/>\n" +
                "</dataset>";

        final FlatXmlProducer producer = new FlatXmlProducer(
                new StringReader(xmlWithDtdAndPresentTable)
        );
        producer.setConsumer(consumer);

        // Exercise: this should not backfill an extra empty PRESENT_TABLE, only the one coming from the XML body.
        producer.produce();

        consumer.verify();

To fully integrate this test with the existing codebase, you will likely need to:

  1. Adjust the construction of FlatXmlProducer to match how other tests in FlatXmlProducerTest create producers (e.g., using FlatXmlDataSetBuilder, setting ColumnSensingDataSet, or passing flags that enable DTD metadata).
  2. Ensure the MockDataSetConsumer API matches the calls:
    • If the existing tests use a different method to assert non-empty tables (e.g., addExpectedTable vs. addExpectedTableWithRow), update the invocation accordingly.
    • Make sure the presentColumns array and row values ("row1col1", "row1col2") are consistent with the column metadata used elsewhere in the test file.
  3. If the project uses shared XML fixtures instead of inline XML strings, move xmlWithDtdAndPresentTable into the appropriate helper or resource file and reference it from the test.
  4. Confirm that the DTD snippet aligns with how addMissingDtdTables() discovers tables from the DTD in other tests; you may need to mirror the exact DTD structure used in the existing “backfill” test to ensure coverage of the same code path.

Expand All @@ -175,11 +176,12 @@ void testProduceMetaDataSet_withTableAbsentFromXmlBody_addsEmptyTableFromMetaDat
final MockDataSetConsumer consumer = new MockDataSetConsumer();
consumer.addExpectedStartDataSet();
consumer.addExpectedEmptyTable(presentTable, presentColumns);
// MISSING_TABLE is declared in the supplied metaDataSet but never appears as a
// row element in the XML body; it must still be reported, with zero rows and its
// own column metadata, or a CLEAN_INSERT/DELETE_ALL relying on the produced
// dataset's table list would silently skip it (issue #496).
consumer.addExpectedEmptyTable(missingTable, missingColumns);
// MISSING_TABLE is declared in the supplied metaDataSet but never appears as a row
// element in the XML body. Unlike DTD-derived metadata, an arbitrary metaDataSet
// (e.g. a live database's full IDataSet, supplied only to resolve column info) is
// not an enumeration of the fixture's tables, so it must NOT be added: doing so
// previously made DELETE_ALL/CLEAN_INSERT touch every table in that broader source,
// not just the ones the XML body actually mentions (issue #951 regression from #496).
consumer.addExpectedEndDataSet();

// Setup producer
Expand All @@ -199,6 +201,52 @@ void testProduceMetaDataSet_withTableAbsentFromXmlBody_addsEmptyTableFromMetaDat
consumer.verify();
}

@Test
void testProduceMetaDataSet_withFlatDtdDataSetAndTableAbsentFromXmlBody_addsEmptyTableFromMetaDataSet()
throws Exception
{
// Setup consumer
final String presentTable = "PRESENT_TABLE";
final String missingTable = "MISSING_TABLE";
// Deliberately different shapes (name and column count) per table, so a producer
// bug that mixed up which table's metadata to use would make this test fail
// instead of passing by coincidence.
final Column[] presentColumns = new Column[] {
new Column("PRESENT_COL", DataType.UNKNOWN, Column.NULLABLE)};
final Column[] missingColumns = new Column[] {
new Column("MISSING_COL0", DataType.UNKNOWN, Column.NULLABLE),
new Column("MISSING_COL1", DataType.UNKNOWN, Column.NULLABLE)};
final MockDataSetConsumer consumer = new MockDataSetConsumer();
consumer.addExpectedStartDataSet();
consumer.addExpectedEmptyTable(presentTable, presentColumns);
// MISSING_TABLE is declared in the DTD-derived metaDataSet but never appears as a
// row element in the XML body; it must still be reported, with zero rows and its
// own column metadata, or a CLEAN_INSERT/DELETE_ALL relying on the produced
// dataset's table list would silently skip it (issue #496). This mirrors what
// FlatXmlDataSetBuilder#setMetaDataSetFromDtd builds internally, so that path must
// keep the #496 backfill even though it is not the inline-parsed-DTD case.
consumer.addExpectedEmptyTable(missingTable, missingColumns);
consumer.addExpectedEndDataSet();

// Setup producer
final String content = "<?xml version=\"1.0\"?>"
+ "<!DOCTYPE dataset SYSTEM \"urn:/dummy.dtd\">" + "<dataset>"
+ "<PRESENT_TABLE/>" + "</dataset>";
final InputSource source = new InputSource(new StringReader(content));
final String dtdContent = "<!ELEMENT dataset (PRESENT_TABLE*,MISSING_TABLE*)>"
+ "<!ATTLIST PRESENT_TABLE PRESENT_COL CDATA #IMPLIED>"
+ "<!ATTLIST MISSING_TABLE MISSING_COL0 CDATA #IMPLIED MISSING_COL1 CDATA #IMPLIED>";
final FlatDtdDataSet metaDataSet =
new FlatDtdDataSet(new StringReader(dtdContent));
final IDataSetProducer producer =
new FlatXmlProducer(source, metaDataSet);
producer.setConsumer(consumer);

// Produce and verify consumer
producer.produce();
consumer.verify();
}

@Test
void testProduceCustomEntityResolver_withCustomEntityResolver_usesResolverToLoadDtd() throws Exception
{
Expand Down
Loading