Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 44 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ happened on the first fixture written.

## Read this first: the coverage this corpus does NOT have

Partial coverage is easy to mistake for complete coverage. Five gaps are open.
Partial coverage is easy to mistake for complete coverage. Six gaps are open.

### EnergyPlus 26.1.0 only

Expand Down Expand Up @@ -75,7 +75,19 @@ for. It has no members yet.

A second consequence, worth stating separately: cases are curated from a sweep of the EnergyPlus
example files, so the corpus only sees hazards that EnergyPlus's own files exhibit. Byte-order marks,
CRLF line endings, and the other things real editors emit are not in that set.
CRLF line endings, and the other things real editors emit are not in that set. Nine `preserve-*`
cases were donated in `conformance-2026.9` to close part of that, and they close it for the parse
and the write; they do not close it for the read from disk, which is this section's subject.

**Assertion 9 lives on this side of the boundary too.** `preserved-text` proves that a library
reproduces the text its read was GIVEN, not that it reproduces a file's bytes. The runners decode
the input themselves, as above, so what the assertion holds each library to is a string in, the same
string out. Two defects live on the other side of that line and neither is closed by it: the
TypeScript reader refuses an IDF file carrying a byte-order mark, recorded above against
idfkit-js#28, and Python's `save_idf` opens its destination without `newline=""`, so the standard
library translates every line feed to the platform's own convention on the way out. On Windows a
byte-identical write therefore becomes a byte-different file, and the first language's byte-identity
claim is platform-dependent. Both need the reserved `checks/` directory to close.

### Weather retrieval is not covered, and `tier1` does not claim it

Expand All @@ -95,17 +107,21 @@ build-time warm-up and a run with the network switched off, which is where the s
it. The `tier1` tag therefore means "the Tier 1 capabilities this corpus can express", not "every
Tier 1 capability", and a green `--tag tier1` is not a statement about weather.

### Writer output is not compared as text, and cannot be
### The two writers' output is not compared against each other, and cannot be

The naming register once said this corpus proves that both libraries render the same string for
the same model. It does not, and it is not able to.

`runners/compare.md` forbids textual comparison outright, for a good reason set out there: a
formatting difference and a value difference are not the same finding, and a comparator that is
textual anywhere is textual everywhere. The assertion enum has no writer kind. What assertion 3
does instead is re-parse each library's own IDF output and compare the resulting *document* to the
original, which catches a field that moved or a value that was lost and says nothing about the
bytes in between.
`runners/compare.md` forbids comparing the two libraries textually, for a good reason set out
there: a formatting difference and a value difference are not the same finding, and a comparator
that is textual anywhere is textual everywhere. What assertion 3 does instead is re-parse each
library's own IDF output and compare the resulting *document* to the original, which catches a field
that moved or a value that was lost and says nothing about the bytes in between.

Assertion 9 is not an exception to this and must not be read as one. It compares a library's output
to **that same library's own input**, never to the other library's output, and its whole content is
that nothing was re-rendered. Rule 1 states the exception as a direction rather than as an assertion
number precisely so the next byte comparison cannot be added by precedent.

The bytes do differ, and the differences are real. Round-tripping one file through both writers
gives 2-space indentation against 4, insertion order against sorted order, a `!-Generator idfkit`
Expand All @@ -116,6 +132,25 @@ about output belongs.

So a green run is a statement about what each library understood, never about what it typed.

### Three byte-hazard cases declare an assertion neither library can evaluate

`preserve-empty`, `preserve-comments-only` and `preserve-duplicate-name` declare `preserved-text`
and it is skipped in both languages, because both readers refuse their inputs and a refused read
leaves nothing to write. `preserve-unterminated-final` is skipped on the TypeScript side alone, for
the same reason.

The first three fail for one cause between them: both libraries resolve their schema from a Version
statement in the file, and an empty file, a file of comments and a file whose duplicate name stops
the read have no readable model. The declared parse outcome is that refusal, which both libraries
agree on exactly, so those cases still earn their keep on assertion 1.

What they do not yet pin is the writer hazard they were donated for: that a file with nothing to
anchor comes back whole, and that the characters of a statement the read REJECTED survive a write.
The second needs a read that succeeds while rejecting a statement, which neither library offers
today. The assertion is declared rather than dropped so that a library which later reads one of
these files starts being held to the property with no corpus change, and so that the gap is visible
as a skip in every run rather than as an absence nobody sees.

## Why `known-divergence.toml` ships populated

The corpus lands with real, currently failing disagreements already recorded in
Expand Down
2 changes: 1 addition & 1 deletion cases/encoding-latin1-high-bytes/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ into one.
tags = ["encoding"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/encoding-utf8-object-name/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ reason.
tags = ["encoding"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/extensible-empty-group/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ else in the corpus catches it.
tags = ["extensible"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/extensible-partial-group/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ is a difference the oracle expectation catches.
tags = ["extensible", "positional"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ reported difference rather than being rounded away.
tags = ["extensible", "numeric", "types"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/extensible-single-vs-multiple/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ multi-field group can tell those apart.
tags = ["extensible"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/extensible-wrapper-key/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ numeric coercion.
tags = ["extensible"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/naming-blank-vs-absent/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ raised it, because no shipped example leaves an optional Name blank.
tags = ["naming", "positional"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/naming-synthetic-collision/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ present only to satisfy the oracle's pre-conversion validation, so this case pin
tags = ["naming"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/naming-synthetic-key/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ present only to satisfy the oracle's pre-conversion validation, so this case pin
tags = ["naming"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-autocalculate-casing/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ schema enum names, which is not always the word the file used.
tags = ["numeric"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-autosize-casing/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ fails schema validation everywhere at once rather than in one place.
tags = ["numeric"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-integer-field/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ would turn every integer field in the corpus red without a single library changi
tags = ["numeric", "types"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-scientific-notation/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ library and only show up against an external expectation.
tags = ["numeric"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-sentinel-schema-remap/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ and that pointer is the finding.
tags = ["numeric", "types"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/numeric-zero-against-blank/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ detail of the text, not of the model.
tags = ["numeric", "positional"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ in the corpus rather than one standing in for the rest.
tags = ["positional", "extensible"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ that second failure, since it is a field-order bug and field order is compared o
tags = ["positional", "extensible"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
2 changes: 1 addition & 1 deletion cases/positional-trailing-unset-no-extensible/case.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ positional-extensible-group-trailing-unset.
tags = ["positional"]
energyplus_version = "26.1.0"
truth = "oracle"
assertions = ["parse-outcome", "epjson", "round-trip"]
assertions = ["parse-outcome", "epjson", "round-trip", "preserved-text"]
16 changes: 16 additions & 0 deletions cases/preserve-comment-hazards/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
title = "A comma and a semicolon inside a trailing comment, neither of which is a delimiter"
why = """
A comment trailing a value carries both delimiter characters. Neither is a delimiter, because a
comment runs to the end of its line and there are no string literals in this grammar.

A reader that took either for a delimiter would shift every field after it by one position, so this
is a positional hazard reported as a comment one. For a preserving writer it is a second hazard on
the same characters: the comment lives in the gap between two statements, belongs to no object, and
is reproduced exactly or the file changes in a place no object was edited.

Donated from the syntax fixtures of the second language.
"""
tags = ["positional"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
6 changes: 6 additions & 0 deletions cases/preserve-comment-hazards/input.idf
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Version, 26.1;

Zone,
Zone One, !- Name, which is followed here by a comma; and a semicolon
0.0, !- Direction of Relative North
0.0; !- X Origin
20 changes: 20 additions & 0 deletions cases/preserve-comments-only/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
title = "A file of comments and nothing else, which neither reader will open"
why = """
Comments, no statements, and therefore no Version.

For a preserving writer the property is that the output is the input: there is nothing to anchor,
so every character is text that belongs to no object. Neither library can be asked, because both
resolve their schema from a Version statement in the file and both refuse a file that has none. The
declared outcome is that refusal, which the two agree on exactly.

The preserved-text assertion is declared and is skipped in both languages today. It is declared
rather than dropped so that a library which later reads a versionless file starts being held to the
property without a corpus change, and so that the skip is visible in every run rather than being an
absence nobody sees. The gap is recorded in the README.

Donated from the syntax fixtures of the second language.
"""
tags = ["malformed"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
6 changes: 6 additions & 0 deletions cases/preserve-comments-only/input.idf
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
! A model somebody started and never wrote a statement into.
!-Generator IDFEditor 1.51
!-Option SortedOrder
!
! There is no Version statement here either, so nothing below can be
! resolved against a schema. Classification must still cover every byte.
19 changes: 19 additions & 0 deletions cases/preserve-duplicate-name/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
title = "Two objects of one type declaring the same name, which both readers refuse"
why = """
Two Zone objects share a name, which the schema forbids and real files contain.

Both libraries refuse the file, and they agree, which is worth pinning on its own: a duplicate name
is the one malformed shape where the two have historically been asked to differ, and this case
records that they do not.

The writer hazard this fixture was donated for is the other half, and it is not reachable here: it
is what happens to the characters of a statement the read REJECTED while the read still succeeded,
which is a state neither reader offers, since both stop at the duplicate instead. The preserved-text
assertion is declared and skipped, and the gap is recorded in the README with the other two.

Donated from the syntax fixtures of the second language.
"""
tags = ["naming"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
9 changes: 9 additions & 0 deletions cases/preserve-duplicate-name/input.idf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Version, 26.1;

Zone,
Zone One, !- Name
0.0; !- Direction of Relative North

Zone,
Zone One, !- Name
90.0; !- Direction of Relative North
19 changes: 19 additions & 0 deletions cases/preserve-edit-no-op/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
title = "A field written the value it already holds leaves the object as it was"
why = """
The same input, and an operation that writes the Building's Terrain the value the file already
gives it. Nothing changed, so nothing may be reformatted, and the operations block excludes no
extent: the write is compared against the input in full.

This case exists because it caught a real defect. The first language cleared its retained source
text on every field write, without comparing, so this operation reformatted the Building and turned
its `0.0` into `0`. It was found by running the library rather than by reading it, and it is
written as a corpus case rather than as that library's unit test for exactly that reason: the
second language got this right for free through a comparing accessor, and a claim that only one
library's own tests hold is a claim the other can drift away from.

FR-004 and the fourth acceptance scenario of the one-object-diff story.
"""
tags = ["numeric"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
24 changes: 24 additions & 0 deletions cases/preserve-edit-no-op/input.idf
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Version,
26.1;

Building,
Scientific Notation,
0.0, !- North Axis {deg}
City, !- Terrain
0.04, !- Loads Convergence Tolerance Value {W}
0.4, !- Temperature Convergence Tolerance Value {deltaC}
FullExterior, !- Solar Distribution
25, !- Maximum Number of Warmup Days
6; !- Minimum Number of Warmup Days

GlobalGeometryRules,
UpperLeftCorner, !- Starting Vertex Position
Counterclockwise, !- Vertex Entry Direction
Relative; !- Coordinate System

Curve:Linear,
Exponent Forms, !- Name
1.0E-5, !- Coefficient1 Constant
-2.5e-3, !- Coefficient2 x
-1.5E+2, !- Minimum Value of x
1.5E2; !- Maximum Value of x
18 changes: 18 additions & 0 deletions cases/preserve-edit-one-field/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
title = "One field changes and exactly one object looks changed"
why = """
The input is the one from numeric-scientific-notation, unaltered. The operation writes the
Building's Terrain, and the assertion is about every byte that is not the Building's.

This is SC-002 as a case rather than as a hope. The Curve:Linear object below carries `1.0E-5`,
`-2.5e-3`, `-1.5E+2` and `1.5E2`, four notations no writer reproduces from a parsed number, so a
writer that reformats the whole document to satisfy one edit fails here loudly and immediately.
The comments, the blank lines and the trailing newline are outside every object and are compared
too.

An existing input rather than a new one, deliberately: the property is about the file the operation
did NOT touch, so the more ordinary the file the better the evidence.
"""
tags = ["numeric"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
24 changes: 24 additions & 0 deletions cases/preserve-edit-one-field/input.idf
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Version,
26.1;

Building,
Scientific Notation,
0.0, !- North Axis {deg}
City, !- Terrain
0.04, !- Loads Convergence Tolerance Value {W}
0.4, !- Temperature Convergence Tolerance Value {deltaC}
FullExterior, !- Solar Distribution
25, !- Maximum Number of Warmup Days
6; !- Minimum Number of Warmup Days

GlobalGeometryRules,
UpperLeftCorner, !- Starting Vertex Position
Counterclockwise, !- Vertex Entry Direction
Relative; !- Coordinate System

Curve:Linear,
Exponent Forms, !- Name
1.0E-5, !- Coefficient1 Constant
-2.5e-3, !- Coefficient2 x
-1.5E+2, !- Minimum Value of x
1.5E2; !- Maximum Value of x
21 changes: 21 additions & 0 deletions cases/preserve-edit-remove-and-add/case.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
title = "An object removed and another added leave the text between them alone"
why = """
The same input as preserve-edit-rename. One Construction is removed and one Material:NoMass is
added.

Two properties in one case, and they fail in opposite directions. A removal must take the object's
own extent and nothing else: the blank line and the comment around it belong to no object and must
survive, and a writer that removes a line too many or too few is reproducing a file the author did
not write. An addition must go to the end, formatted, and must not run onto the last line of a file
whose final statement is where the author left it.

The removal half is also where the object notation's terms were found to be wrong in the first
language: its preserving write asked the surviving objects whether they were unmodified, and a
removed object is not among them to ask, so the original text came back with the removed object
still in it. That is fixed at the library, and this case is the text format's half of the same
question.
"""
tags = ["references"]
energyplus_version = "26.1.0"
truth = "convention"
assertions = ["parse-outcome", "preserved-text"]
Loading
Loading