Repository navigation
fix(html): extract definition lists instead of discarding them - #4502
Merged
Merged
Conversation
<dl>, <dt> and <dd> were mapped to RemovedBlock, so partition_html() silently dropped glossaries and every Sphinx-generated API reference, where each documented function, its parameters and its return value live in a <dl>. Map <dl> to ListBlock and <dd> to ListItemBlock, as the ListBlock docstring and the list-depth code already anticipate, and <dt> to a plain BlockItem.
cragwolfe
approved these changes
Sep 27, 2026
cragwolfe
left a comment
Contributor
There was a problem hiding this comment.
SAFE TO MERGE at reviewed head 89f382c4799a1c0d991123d1141bdab879f5f044 against main at 1bedf7be0bea9db5c5dc3a9a2dab83078b4db54d. No actionable Critical or High findings remain.
The changed tags reuse existing traversal classes; the parser already counts <dl> ancestors for <dd> depth. The added tests cover exact glossary element types and depth, plus nested Sphinx text retention in source order. All six required exact-head checks passed at review time.
(authored by codex)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
<dl>,<dt>and<dd>are mapped toRemovedBlock, sopartition_html()drops definition lists and everything inside them. Documentation gets hit hardest: Sphinx renders every documented function, class and attribute as a<dl>, so partitioning an API reference page returns the headings and nothing else.Glossaries and key/value lists disappear the same way, with no error.
Fix
Map
<dl>toListBlockand<dd>toListItemBlock, which the parser already anticipates: theListBlockdocstring says "maybe a<dl>element at some point", and the list-depth code already countsdlancestors fordditems.<dt>becomes a plainBlockItem, so a term gets its type from its text like any paragraph. The v2 (ontology) parser already keeps definition lists.Tests
Two new tests in
test_partition.py: a glossary (exact element types, text andcategory_depth) and a nested Sphinx-style API reference (all text kept, in order). Without the fix they return[('Title', 'Glossary', 1)]and['API reference']. The html, chunking, documents, md, text and email tests go from 1201 passed on main to 1203, with no failures.This touches the same mapping table as #4451 (
<details>/<summary>), so whichever lands second will need a small rebase.