Describe a type with its prose, carry a parse error's findings, and close the writer controls - #37
Merged
samuelduchesne merged 14 commits intoSep 4, 2026
Conversation
Closes the introspection entry on the parity record. Three families of difference, all of them in what the schema bundle carried rather than in the introspection code, and all three closed additively. The prose. `@idfkit/schemas` dropped `memo` and `note` by omission: slimType and slimField never read them. It now emits them as a deduplicated pool to data/docs.json.gz, 4,878 distinct strings standing in for roughly 119,000 occurrences, referenced by one integer per record and per property. describeObjectType takes the pool as an optional third argument and stays synchronous. Reading a file is not, and making the function async to fetch something most callers do not want would be a breaking change serving the minority. A caller who passes nothing gets exactly what they got before, which is what keeps this additive, and the corpus runner is a caller that opts in. The accepted values. The blank is legal for 21,962 enum-bearing fields across the versions and Python reports it. It stays filtered out of `e`, because `e` is what validate() checks against and admitting it there would change what validation accepts. `eb` carries it alongside and the description path puts it back. `se`, the collapsed anyOf string branch, was already in the bundle and already read by validation; the description path now reads it too, so Autosize and Autocalculate are visible on both sides. The field order. Three types in every one of the seventeen schemas carry a positional list holding only the name, so the description fell back to the key order of the property map, which the content-addressing serializer had sorted. `fo` records the declaration order that sort destroys, for those three and nothing else. Two of them actually diverged; the third agreed by alphabetical luck and is fixed anyway, because a schema edit renaming either field would otherwise break it in silence. Proven against Python, not against our own assumptions: a digest over every memo and note of all 858 types in 26.1.0 matches the one Python produces. MEASURED COST, and it is not what the plan predicted. The plan expected the prose to add 190,471 bytes and land the install at 86.2 percent of budget. It adds 310,253 and lands at 92.9 percent, 1,705,108 bytes against 1,835,008, with 126.9 KB of headroom. The pool itself is 175,035 bytes, near the prediction. The references cost 135,218 rather than 18,652, because a memo that differs between versions makes two previously byte-identical definitions distinct: unique defs go from 2,570 to 2,735. The estimate did not account for deduplication degrading. Measured by rebuilding without the references, `eb` and `fo` together cost nothing at all, 626 bytes less than before. The gate passes and the budget is not moved again to make it pass.
…elds IdfParseError held a single diagnostic's line and typeName, so a caller who wanted the rest had nowhere to look. Python's IDFParseError has always carried the collection; this is the half of the recorded divergence that was real. `.line` and `.typeName` still resolve to the first finding's values, so no existing caller breaks. They are a convenience over diagnostics[0] rather than a second source of truth. ParseDiagnostic gains the location it lacked, a file path, a column and the object name, so both languages carry the same kinds of location. typeName is NOT renamed to match Python's obj_type: that is idiomatic casing, the naming register records it as such, and spending a rename to make the register harder to read is the wrong trade. Both diagnostic types gain a code from the shared vocabulary, and every site that reports one populates it. The corpus compares findings on (code, line, typeName) and never on message text, so wording stays free to improve without turning every improvement into a conformance failure. typeName moves from ParseDiagnostic up to LexDiagnostic, because the lexer knows it too: an unterminated object has read its type name before it runs out of input, and the finding used to drop it. The corpus case for a truncated object was failing on exactly that.
The other language has had output_type=compressed since it was written and this writer had no compact path at all to extend: comments:false skips the padding and the comment and still puts every field on its own line, which is a different, coarser output. Branches at both places compressed has to reach: the per-field line break and the blank separator after each object. Type name and every value on one line, comma-separated, terminated once, which is the shape the other language produces. Asking for compressed and comments together is not an error. Compressed has no comments by definition, so the narrower request is unambiguous. The two are still not byte-identical under compressed, and are not meant to be: it removes comments, indentation and blank lines and does not touch float rendering, which is one of the six pinned defaults. What it guarantees, and what the corpus checks structurally, is that a document written under it re-reads to the document it came from. All six defaults are pinned by tests first, at the values THIS writer uses: four-space indent, column 30, insertion order, schema-consulted floats, minor words lowercase, no generator header. Five of the six differ from the other language and none of them moves. The docs snippet for the new explanation page lives here rather than in the site, because the site vendors TypeScript snippets from a docs release.
… set ParseDiagnostic gained column, filepath and objectName, and nothing filled any of them: they were type declarations only, while the changelog and the naming register both said both libraries now carry the same kinds of location. parseIdf now sets objectName at both diagnostic sites, by the same best-effort first-positional-value rule the Python parser uses. column and filepath are still not filled, and the changelog now says so rather than implying otherwise. A column needs the lexer to track one, and a filepath needs the file-reading edge to attach it; both are real changes rather than oversights, and claiming them was the defect.
…declared ordering closes the last one-sided control (SC-007, US4 scenario 2). It takes sorted or source rather than a boolean, matching the other language, and defaults to source, which is insertion order and what this writer always did. It composes with versionFirst: ordering decides the sequence, versionFirst lifts Version out of it. This also makes the corpus case honest. writer-option-ordering asked for source, which was already this writer's only behaviour, so it passed here even when the runner dropped the option entirely. column and filepath were declared on the diagnostic types and set by nothing, so the location parity FR-033 claims was a type declaration. The lexer now tracks the start of each line and reports the column of the object's first NON-BLANK character, which is what Python reports: its regex matches the type name, so an object indented three spaces is column 4 on both sides. Verified against the same input in both languages. filepath is attached at the node edge, the one place that knows where the text came from, on both the returning path and the raising one. parseIdf takes text and cannot know, so it is not given a new option to carry. RawObject gains an optional column, which is what the three lexer tests that changed were asserting the absence of.
The level covering assertion 4, the eight new cases, and the three closures. conformance.ts is regenerated from the pin rather than edited.
Fixes #36. The same defect and the same fix as idfkit#191; the two had to land together, or the corpus would report whichever moved first as a new divergence. A missing semicolon swallows the object below it and slides that object's type name into a numeric field. The field count still fits, so nothing overflows and no parser notices by counting. Positioned at the offending FIELD, not the object. The lexer records one offset per object rather than a line per field, and the reporting path rescans from it counting separators, so the common path pays nothing for a position almost no object needs. NOT ROUTED THROUGH `report`, which throws under `strict`. That is right for a finding leaving nothing to return, such as an unknown type, and wrong here: the object is built and the document is complete. Routing it through `report` made strict parses fail on files that used to load, which the conformance corpus caught immediately on two cases. Any numeric field accepts a sizing sentinel, whatever its own `se` branch says. The schema EnergyPlus ships is narrower about which sentinel belongs where than the engine it ships with, and reading each branch literally produced 3,775 findings across the 760 example files of one release, every one against a model EnergyPlus accepts. Whether the sentinel is the documented one is a schema question, and validateObject already answers it. Measured on the same 8,581 real files as the Python side, both languages report the same 41 findings in the same 27 files, and every one is a true positive.
…g EnergyPlus The conformance job installed EnergyPlus 26.1, roughly a gigabyte, to get at one release's example files, and read them with one schema. It needed neither. The library parses against its own bundled schemas, so the only thing wanted from a release is the files, and a blobless, shallow, sparse clone of testfiles/*.idf is about 210 MB. Cached on the release tag, which is immutable, so seventeen downloads happen once rather than per pull request. That is what makes sweeping all seventeen versions affordable, and sweeping all of them is what catches the two defects a single release cannot show: a parser change safe on the newest schema and wrong on an older one, and a file whose content belongs to one version while its Version object declares another. Seventeen parallel jobs, fail-fast off, each held to the count measured for that version so a regression of one is caught. FOUND BY RUNNING THIS AGAINST THE PYTHON SWEEP. The two agreed on the file and the field and disagreed on the line by one, on every finding that had a comment between the separator and the value. `fieldLine` stopped at the `!` and reported the line the previous value sits on. A field's comma is followed by `!- Field Name` in essentially every real file, so this was wrong nearly everywhere and right in the conformance case, whose input carries no comments. Both languages now report line 318 for the same value in the same file. `errors` is zero on every version, against three in the Python sweep. Those are three files whose Version object sits where its detector does not look, which this library reads without trouble: idfkit#185, now measured at exactly three files out of 12,644.
The sweep job ran 'npm run build', which this repository does not define, so all seventeen jobs failed identically before reading a single file. The build is 'typecheck', which is 'tsc --build'. The sweep reads dist/ rather than the sources, deliberately: that is what an npm consumer receives, and a sweep of 12,644 real files is worth pointing at the artifact rather than at something a bundler would transform first.
The sweep called bundle.load(version) directly, which demands an exact match, so every file declaring 9.0 was reported unreadable against a bundle carrying 9.0.1: 627 of 627 files on three versions. A defect in the sweep that looked exactly like one in the library. It now calls schemaFor, which is the path loadIdf takes and the one that resolves a declared version onto a bundled one. 9.2.0 is held at four findings where Python is held at three. The extra is real: _1a-Long0.0.idf carries an un-migrated 'report variable dictionary;', which this lexer reports as an unknown type and Python's pattern discards in silence because it has no comma. idfkit#193. The two numbers agree again once that is decided, and holding them apart until then is the point of measuring each side separately.
_1a-Long0.0.idf exists at 8.9.0, 9.0.1, 9.1.0 and 9.2.0 and not after, so the extra finding this lexer reports and Python's pattern discards appears on all four. Only 9.2.0 was held at four; the other three failed for the same reason. The schemaFor fix worked: unreadable is zero on every version now, against 627 of 627 before it.
_1a-Long0.0.idf carries an un-migrated 'report variable dictionary;' at the four tags it exists in. That is a defect in the file rather than in either library, and the two libraries disagree about it: one reports an unknown type, the other's pattern discards it for having no comma. The disagreement is real and is tracked as idfkit#193. Carrying it in the sweep meant holding four versions at a count that said nothing about this repository's own regressions, which is what the sweep is for. It is excluded in the sparse pattern, so it never lands, with the reason and the issue at the point of exclusion. Every version is back to its true count, and the two libraries' matrices are now identical version for version. That is the useful property: they sweep the same files, so a number moving on one side alone is a divergence.
The exclusion added in the last commit never took effect. The cache key was `ep-testfiles-v<version>`, the tag is immutable so the key never changed, and the restored directory was the pre-exclusion one. The clone step was skipped entirely and the sweep read the 665 files it was meant to stop reading. The tag is immutable but the SELECTION is not, and the key described only the first. That is the bug, and bumping the key by hand would have fixed this instance and left the next one. The selection now lives in .github/ep-testfiles.sparse and the key is a hash of that file, so changing what is swept necessarily changes the key. The clone step reads its patterns from the same file, so the cache and the checkout cannot disagree about what should be present. Verified from a fresh clone: 664 files, the excluded one absent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature 002, the TypeScript half. Stacked on #33: this targets
amend-install-size-budget, because the prose below needs the budget that PR raises.Depends on idfkit/idfkit-conformance#1. This bumps the governance pin to
governance-2026.10and the conformance pin toconformance-2026.8, neither of which exists until that merges and the tags are cut, so the gates are red until then.Describing a type now reads the same in both languages
Three families of difference, all in what the schema bundle carried rather than in the introspection code.
The prose.
@idfkit/schemasdroppedmemoandnoteby omission:slimTypeandslimFieldnever read them. It now emits them as a deduplicated pool todata/docs.json.gz, 4,878 distinct strings standing in for roughly 119,000 occurrences.describeObjectTypetakes the pool as an optional third argument and stays synchronous. Reading a file is not, and making the function async to fetch something most callers do not want would be a breaking change serving the minority. A caller who passes nothing gets exactly what they got before.The prose never reaches the model-reading path: it is a separate file under
data/, whichcheck-bundle-purity.mjsalready fences, and the check now covers it with no change.The accepted values. The blank is legal for 21,962 enum-bearing fields and Python reports it. It stays filtered out of
e, becauseeis what validation checks against and admitting it there would change whatvalidate()accepts;ebcarries it alongside and the description path puts it back.se, the collapsedanyOfstring branch, was already in the bundle and already read by validation; the description path now reads it too.The field order. Three types in every one of the seventeen schemas carry a positional list holding only the name, so the description fell back to the key order of the property map, which the content-addressing serializer had sorted.
forecords the declaration order. Two of the three actually diverged; the third agreed by alphabetical luck and is fixed anyway.Proven against Python rather than against our own assumptions: a digest over every memo and note of all 858 types in 26.1.0 matches the one Python produces.
The cost, and it is not what the plan predicted
The plan expected the prose to add 190,471 bytes and land the install at 86.2 percent of budget. It adds 310,253 and lands at 92.9 percent, 1,705,108 bytes against 1,835,008, with 126.9 KB of headroom.
The pool itself is 175,035 bytes, near the prediction. The references cost 135,218 rather than 18,652, because a memo that differs between versions makes two previously byte-identical definitions distinct: unique defs go from 2,570 to 2,735. The estimate did not account for deduplication degrading. Measured by rebuilding without the references,
ebandfotogether cost nothing at all.The gate passes and the budget is not moved again to make it pass.
A parse error carries its findings
IdfParseErrorheld one diagnostic flattened into.lineand.typeName, and a caller who wanted the rest had nowhere to look. It now carries the collection, and both accessors still resolve to the first finding's values so no existing caller breaks.ParseDiagnosticgains acodefrom the shared vocabulary, plus acolumn, afilepathand anobjectName. The first pass declared those three and filled none of them, which made the location parity a type declaration; the lexer now tracks line starts and reports the column of the object's first non-blank character, which is what Python reports, and the path is attached at the node edge on both the returning and raising paths.The writer controls
compressed, the fifth control, which had no compact path here to extend. Thenordering, added in a second pass because SC-007 asks for zero controls that exist on one writer and not the other and this one still did.orderingalso makes the corpus case honest: it asked forsource, already this writer's only behaviour, so it passed even when the runner dropped the option entirely.No default moves. All six are pinned by tests first, at the values this writer uses: four-space indent, column 30, insertion order, schema-consulted floats, minor words lowercase, no generator header. Five of the six differ from the other language and none of them moves.
Known, and tracked rather than fixed here
#36, a missing semicolon corrupts a field and the parse reports nothing. Recorded in
known-divergence.toml. The same defect exists inidfkitas #191, and the two must be fixed together or the corpus reports whichever moved first as a new divergence.Review
A maintainer of the other language, per FR-067 and CODEOWNERS.
Verification
475 tests,
tsc --build,format:check, and everycheck:*gate green, at the tags once they exist.