Skip to content

fix(tsv): honor infer_table_structure when partitioning TSV files - #4513

Open
feiiiiii5 wants to merge 1 commit into
Unstructured-IO:mainfrom
feiiiiii5:fix/tsv-infer-table-structure
Open

feiiiiii5 wants to merge 1 commit into
Unstructured-IO:mainfrom
feiiiiii5:fix/tsv-infer-table-structure

Conversation

@feiiiiii5

@feiiiiii5 feiiiiii5 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

partition_tsv() silently ignores infer_table_structure=False, so callers still receive Table.metadata.text_as_html. The flag is passed through by partition() but absorbed by **kwargs in the TSV partitioner.

Add the explicit parameter with the same default (True) as the other table partitioners, and set HTML metadata only when enabled. Tests cover both flag values through partition_tsv() and partition(). Plain table text and the default HTML output are preserved.

The branch includes current main (2c0c7a6c), retaining its TSV resource-limit tests and implementation. The required changelog/version bump is now 0.27.18.

Test

  • Frozen TSV/test/lint dependencies installed from uv.lock.
  • TSV: 28 passed; adjacent CSV: 70 passed.
  • The four new flag cases against current main: 2 failed, 2 passed; against this branch: 4 passed.
  • Ruff check and format check pass for both changed Python files; git diff --check passes.

Full partitioner/extras suites and cross-platform tests were not run for this update. The broader 786-pass run in the earlier comment is historical.

Type of change: bug fix; no new dependency. Related #4373 changes numeric HTML generation, and #3870 fixed a duplicate auto-dispatch kwarg; neither addresses this ignored TSV flag.

@feiiiiii5
feiiiiii5 force-pushed the fix/tsv-infer-table-structure branch from 3a9ada9 to a27bf88 Compare October 1, 2026 04:41
@feiiiiii5

Copy link
Copy Markdown
Contributor Author

Merged current main (2c0c7a6c) and resolved the changelog/version and TSV-test conflicts, preserving the upstream resource-limit safeguards. The PR still changes only the original four files; the required version bump is now 0.27.18.

With frozen dependencies, TSV has 28 passing tests and adjacent CSV has 70. The new flag regressions still fail on current main (2 failed, 2 passed) and pass on this branch (4 passed). Ruff and formatting pass. Full partitioner/extras suites were not rerun.

partition_tsv() had no infer_table_structure parameter, so the kwarg landed
in **kwargs and was dropped: Table.metadata.text_as_html was always set, even
for partition(..., infer_table_structure=False) or a skip_infer_table_types
list containing tsv. csv, xlsx, docx and odt all gate text_as_html on that
flag; tsv did not.

Adds the parameter with the same docstring as the other partitioners and
gates the html on it, plus the two regression tests the csv and xlsx
partitioner tests already have.
@feiiiiii5
feiiiiii5 force-pushed the fix/tsv-infer-table-structure branch from bf6728d to 622458a Compare October 5, 2026 04:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant