Skip to content

Template the per-reference-year timeslice window patterns#121

Merged
nick-gorman merged 2 commits into
mainfrom
template-timeslice-windows
Jun 24, 2026
Merged

Template the per-reference-year timeslice window patterns#121
nick-gorman merged 2 commits into
mainfrom
template-timeslice-windows

Conversation

@nick-gorman

@nick-gorman nick-gorman commented Jun 18, 2026

Copy link
Copy Markdown
Member

AEMO's PLEXOS data package ships a timeslice calendar (timeslice_RefYear5000.csv): one row per on/off event recording, on absolute dates across the whole model horizon, when each region's demand-condition timeslice — peak (hot day), typical-summer, or winter — switches on (-1) or off (0). The dates aren't arbitrary: AEMO assigns each planning financial year a reference (weather) year from a rolling 16-year sequence (Table 1 of the Draft 2026 ISP Market Model Instructions) and lays that year's conditions onto the calendar. So a given weather year recurs every 16 years, and the calendar is really one annual pattern per reference year, stamped repeatedly across the horizon.

This PR adds the templater step that inverts that into one month-day window pattern per reference year. It pairs each on-event with its following off to form a window, labels the window by the financial year it starts in (and so by that year's reference year), checks every occurrence of a reference year carries the same pattern, and keeps one. The translator (a later PR) re-sequences these patterns onto the user's configured reference_year_cycle.

timeslice_RefYear5000.csv  (FY2026 maps to reference year 2015):
  DATETIME     NAME         TIMESLICE
  18/11/2025   NSW Hot Day  -1          # peak on
  20/11/2025   NSW Hot Day   0          # peak off
  01/04/2026   NSW Winter   -1          # winter on
  01/11/2026   NSW Winter    0          # winter off

timeslices  (templated, abridged — a real reference year has several peak/summer windows):
  timeslice_id          reference_year  start_month_day  end_month_day
  nsw_peak_demand       2015            11-18            11-20          # end exclusive
  nsw_winter_reference  2015            04-01            11-01          # spans 1 July

The decode stays this simple — pair an on with its own off, label by start year — because of how AEMO's data is shaped. Within a reference year the windows tile the year exactly, with peak and summer carrying the weather variation; winter is a fixed seasonal block (April–November on the mainland), identical across reference years, and the only timeslice whose window crosses the 1 July financial-year boundary. Peak and summer therefore sit wholly inside one reference year, so labelling them by start date is unambiguous; winter does straddle the boundary, but being invariant, which reference year "owns" its end can't change the result. So the decode needs no cross-reference-year or leap-day reconciliation — and tests fail if a future calendar breaks either property (a weather-varying window crossing 1 July, or winter differing between reference years).

src/ispypsa/templater/
  timeslices.py                                  → decode the PLEXOS on/off calendar → one window pattern per reference year
  plexos/7.5/timeslice_RefYear5000.csv           → the shipped PLEXOS timeslice calendar (v7.5)
  manually_extracted_template_tables/7.5/
    reference_year_sequence.csv                  → AEMO's planning-year → reference-year sequence
src/ispypsa/validation/schemas/timeslices.yaml   → schema for the templated table
tests/test_templater/test_timeslices.py

Not wired into the templater orchestrator yet.

AEMO timeslice calendar (timeslice_RefYear5000.csv, shipped under
templater/plexos/ with the other PLEXOS data-package extracts) marks hot
day / typical summer / winter windows on absolute dates, with each
planning year's dates derived from the weather of the reference year
AEMO assigned to it. Templating the calendar as absolute windows would
bake AEMO's reference-year sequence into every run regardless of the
user's configured reference_year_cycle, so peak-day limits would fall on
days that aren't peaks in the modelled traces.

Instead the templater inverts the calendar: it assigns windows to the
planning year they start in, labels them via Table 1 of the Draft 2026
ISP Market Model Instructions (transcribed as the manually extracted
reference_year_sequence table, extended cyclically per AEMO's repeating
sequence), and emits one month-day window pattern per reference year for
the translator to re-sequence. Every occurrence of a reference year must
carry an identical pattern — the check that proves the Table 1 decode
against the data; it passes for all 33 usable planning years. Planning
years truncated at the calendar horizon (windows that never turn off)
are dropped whole: their reference years recur earlier with complete
windows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/ispypsa/templater/timeslices.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The templater attributes each calendar window to a reference year by its start
date and emits one month-day pattern per reference year. That is only sound
while a few structural invariants of AEMO's calendar hold, which were previously
assumed silently. Make them fail loud:

- partition: each reference year's windows must tile the year exactly (checked
  inline in _template_timeslices; boundary-based, so leap-safe)
- attribution: only winter may cross the 1 July financial-year boundary, and
  winter must be identical across reference years, so a boundary-crossing window
  cannot blur reference years (standalone guards, run on the shipped calendar)

Also declare the no-overlap and full-coverage rules in the timeslices schema for
the future validation layer, and add tests: a shipped-calendar smoke test,
focused guard tests, and a leap-day case pinning that the templater preserves
02-29 (clamping to 02-28 in non-leap model years is the translator's job).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
return windows[["timeslice_id", "start_date", "end_date"]].reset_index(drop=True)


def _drop_horizon_truncated_planning_years(windows: pd.DataFrame) -> pd.DataFrame:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is one of those moot questions (assuming we can know the templater input pretty confidently), but is there a chance that this drops a planning year that doesn't have a complete occurrence of the same ref year?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it's relying on the templater input having at least one complete year for each reference year.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observation - This test structure seems a little different than others, testing the sub-functions just by calling the orchestrator with different input configurations, rather than testing each function explicitly. No comment really just noting in case there's a specific reasoning or preference behind this style!

@nick-gorman nick-gorman merged commit dfaddc8 into main Jun 24, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants