From 7c27a1724cac89fcccd3c03597e6b045c86284a2 Mon Sep 17 00:00:00 2001 From: Samuel Letellier-Duchesne Date: Mon, 7 Sep 2026 21:47:13 -0400 Subject: [PATCH] Document reading a weather file, once, for both languages One page, prose written once, code varying by language, filed as a how-to beside its siblings in `weather/`. Nothing about reading an EPW differs between the two libraries in a way that needs different explanation: the retrieval before it does, and the page says so and links to where that is explained. It declares `weather-file-reading` through the parity macro, so the build fails if the capability resolves to nothing. It records the decision against a public summary reader where a reader would look for one, rather than leaving them to conclude it was overlooked: the `.stat` member is a report written for a person, in around forty section shapes whose set varies with the station's climate, every figure of which is derivable from the EPW itself. It is read in one place, as the conformance corpus's oracle, where a brittle parse is an offline maintenance task rather than something in a caller's path. BLOCKED UNTIL THE PINS MOVE, on exactly three things and nothing else: the parity macro cannot resolve `weather-file-reading` at governance-2026.14, the vendored TypeScript snippets are not in docs-2026.3, and the Python snippets call functions idfkit==1.0.0-rc.3 does not have. All three clear with the pin bump that follows the release, which is where `weather/station-search.md` is already waiting. --- .../snippets/weather/reading/absent_values.py | 23 +++ docs/snippets/weather/reading/read_an_epw.py | 21 +++ docs/weather/reading.md | 148 ++++++++++++++++++ mkdocs.yml | 1 + 4 files changed, 193 insertions(+) create mode 100644 docs/snippets/weather/reading/absent_values.py create mode 100644 docs/snippets/weather/reading/read_an_epw.py create mode 100644 docs/weather/reading.md diff --git a/docs/snippets/weather/reading/absent_values.py b/docs/snippets/weather/reading/absent_values.py new file mode 100644 index 0000000..7e99901 --- /dev/null +++ b/docs/snippets/weather/reading/absent_values.py @@ -0,0 +1,23 @@ +from __future__ import annotations + +import math + +from idfkit.weather import WeatherFile, monthly_means + +epw: WeatherFile = ... # type: ignore[assignment] +# --8<-- [start:example] +# A measurement the file says was not taken is absent, not the number the format +# reserves for it. Absence is nan: no weather file can express one, so it is not +# a value the column could otherwise have held. +albedo = epw.hours.albedo +print(math.isnan(albedo[0])) # True where the file wrote its missing value + +# How many hours of a column are present is carried rather than recomputed. +print(epw.hours.present_count["albedo"]) + +# A monthly mean excludes the absent hours from the sum AND from the divisor, so +# it is the mean of the values that exist. The count says how many that was. +january = monthly_means(epw, "dry_bulb_temperature")[0] +print(january.mean) # nan when the month held no measurement at all +print(january.count) # and 0 beside it, which says why +# --8<-- [end:example] diff --git a/docs/snippets/weather/reading/read_an_epw.py b/docs/snippets/weather/reading/read_an_epw.py new file mode 100644 index 0000000..5747051 --- /dev/null +++ b/docs/snippets/weather/reading/read_an_epw.py @@ -0,0 +1,21 @@ +from __future__ import annotations + +from idfkit.weather import parse_epw + +epw_text: str = ... # type: ignore[assignment] +# --8<-- [start:example] +# Reading takes the text you already hold. No network, no filesystem, nothing +# asynchronous, so it costs a fraction of the retrieval that produced the text. +epw = parse_epw(epw_text) + +print(epw.location.city) # Chicago Ohare Intl Ap +print(epw.hours.row_count) # 8760, taken from the file's declared data period + +# Every numeric field is a named column, packed and the same length. +temperature = epw.hours.dry_bulb_temperature # array("d"), degrees C +print(temperature[0]) # the first hour of 1 January + +# The hour column runs 1 to 24, and hour 24 is the last hour of its OWN day +# rather than hour 0 of the next. +print(epw.hours.hour[23], epw.hours.day[23]) # 24.0 1.0 +# --8<-- [end:example] diff --git a/docs/weather/reading.md b/docs/weather/reading.md new file mode 100644 index 0000000..6feb2fd --- /dev/null +++ b/docs/weather/reading.md @@ -0,0 +1,148 @@ +# How to read a weather file + +You have the text of an EPW file, retrieved by [downloading a +station's files](downloads.md) or read off disk. This guide turns it into the +header records and the hourly table, and computes monthly figures from it +without treating an unmeasured hour as a number. + +Reading is the same operation in both languages and is spelled the same way. The +retrieval that precedes it is not: Python hands you a path and TypeScript hands +you text, for the reasons [How to download weather files](downloads.md) sets +out. None of that reaches a reader whose input comes from the caller, which is +why this page has one set of instructions rather than two. + +{{ parity("weather-file-reading") }} + +!!! info "In JavaScript, weather is a separate install" + `pip install idfkit` installs weather support and its station index + unconditionally. `npm install idfkit` installs neither. Add `@idfkit/weather` + by name. The reader lives on the portable surface of that package, not on + `@idfkit/weather/node`: it takes a string and touches no disk, so it runs in + a browser tab, a worker or an edge runtime as readily as in Node. + +## Read the file + +=== "Python" + + ```python + --8<-- "docs/snippets/weather/reading/read_an_epw.py:example" + ``` + +=== "TypeScript" + + ```ts + --8<-- "docs/snippets/js/weather/reading/read_an_epw.ts:example" + ``` + +`parse_epw` / `parseEpw` takes text. Each language also has the source-taking +form for a path, `load_epw` and `loadEpw`, which reads the bytes, decodes them +as the weather formats are written, and adds nothing else. In TypeScript that +one is in `@idfkit/weather/node`, because it is the half that touches a disk. + +## Reach a field by its name, never by its position + +An EPW row is thirty-five comma-separated fields with no header line, so a +caller reading one positionally counts to it. The reader does that once and +gives every field a name: `dry_bulb_temperature` / `dryBulbTemperature`, +`relative_humidity` / `relativeHumidity`, `wind_speed` / `windSpeed`, and so on +for every column of the format. + +Two fields are text and stay text. The source-and-uncertainty flags are +obviously so. The present weather codes are not: they hold a nine-digit value +such as `999999999`, which is nine single-digit observations written side by +side rather than a number near a billion. Coercing that yields a column of +meaningless integers, so it is never parsed. + +!!! warning "Hour 24 is the last hour of its own day" + The `hour` column runs 1 to 24, not 0 to 23. Hour 24 belongs to the day its + row names, not to midnight of the next one. A chart that assumes otherwise is + off by one for every day of the year and looks right until somebody checks a + single value against the file. + +## A measurement that was not taken is not a number + +EPW has no blanks. A station that never measured a quantity writes a reserved +value in the slot, so a file that never recorded its albedo carries `999.000` in +that column for all 8,760 hours. Averaged naively, that is a plausible-looking +number that is wrong by three orders of magnitude. + +Both libraries map those values to an absent value instead, per field and per +value, from the reserved-value table published in the EnergyPlus Weather File +Data Dictionary and committed once for both languages. + +=== "Python" + + ```python + --8<-- "docs/snippets/weather/reading/absent_values.py:example" + ``` + +=== "TypeScript" + + ```ts + --8<-- "docs/snippets/js/weather/reading/absent_values.ts:example" + ``` + +Absence is `nan` in Python and `NaN` in TypeScript. That is chosen rather than +convenient: no physical quantity in this format can be nan and no EPW can +express one, so it is not a value the column could otherwise have held. It is +therefore distinguishable from a real zero, which matters because zero solar +radiation at night is a measurement, and it propagates loudly through arithmetic +instead of quietly biasing a result. + +!!! warning "A reserved value is not the same as a large number" + Ceiling height reserves three values: `99999` for a measurement that was not + made, `77777` for an unlimited ceiling, and `88888` for a cirroform ceiling. + The last two are observations. Across the sampled corpus that column holds + `77777` in 55% of its rows and `99999` in none, so a reader applying the rule + "a large number means missing" blanks more than half of it and reports a real + sky condition as unmeasured. The table is per field and per value for exactly + this reason. + +## Compute a monthly figure + +`monthly_means` / `monthlyMeans` returns twelve entries for one column, January +first, each carrying the mean and the count of hours that went into it. + +The count is part of the answer rather than something to derive from it. Without +it you cannot tell a mean over 744 hours from a mean over three, and both look +like numbers. + +The mean excludes absent hours from the sum **and from the divisor**, so it is +the mean of the values that exist rather than their sum spread over hours that do +not. A month in which every hour is absent gives an absent mean with a count of +zero, which is distinguishable from a month whose mean is genuinely zero. + +## What the reader does not do + +- **Write an EPW.** Reading only. +- **Interpret the design conditions record.** It is kept as text. The DDY member + of the same archive is the supported path to design conditions; see [How to + apply design days](design-days.md). +- **Read the climate summary.** The `.stat` member of an archive is a report + written for a person, in around forty section shapes whose set varies with the + station's climate. Parsing it would be brittle in a way this reader is not, and + every figure it carries is derivable from the EPW itself. It is used as an + oracle in the shared conformance corpus, where a brittle parse is somebody's + offline maintenance task rather than something in a caller's path, and that is + the only place it is read. +- **Interpolate, gap-fill or repair.** An absent value is reported as absent. + What to do about it depends on what you are computing, which makes it your + judgement rather than the reader's. +- **Guess at what it cannot represent.** A file whose rows disagree with its own + declared data period fails, naming the disagreement. So does a row with the + wrong field count, naming the row. Nothing partial is ever returned. + +## How the two languages are held to the same answer + +Both readers are checked against the climate summary the EnergyPlus Weather +Converter produced from the same archive, in the shared conformance corpus rather +than in either library's own tests. That summary is not written by either library +and ships in the same archive as the file it describes, so the expectation and the +input cannot drift apart. + +The check compares monthly means of dry bulb, dew point, relative humidity and +wind speed over six stations spanning the climate range, and separately asserts +that the reserved values in a seventh file read as absent. Beyond the four fields +the summary aggregates, the two libraries are held only to agreeing with each +other, which is weaker; the corpus says so rather than implying the whole table +is checked against something external. diff --git a/mkdocs.yml b/mkdocs.yml index ab39059..2c0c93f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -78,6 +78,7 @@ nav: - weather/index.md - Search for weather stations: weather/station-search.md - Download weather files: weather/downloads.md + - Read a weather file: weather/reading.md - Apply design days: weather/design-days.md - Geocode addresses: weather/geocoding.md - Warm the weather cache for an offline run: how-to/warm-the-weather-cache.md