Skip to content

Removed renaming of output (demand only)#58

Merged
dylanjmcconnell merged 11 commits into
mainfrom
drop-output-rename-demand
Jun 24, 2026
Merged

Removed renaming of output (demand only)#58
dylanjmcconnell merged 11 commits into
mainfrom
drop-output-rename-demand

Conversation

@dylanjmcconnell

Copy link
Copy Markdown
Member

This PR removes functions that renamed the output file generated by the restructuring functions (but only for the demand trace data).

Background note

Renaming of the output parquet files is kind of unneccessary / not used since the release of V2 (which is when we moved to using the hive partitioning format). That is, the metadata contained in the renamed outputs isn't actually used / is a historical artifact of the previous approach (it also didn't hurt / didn't cause any problems to leave as is - so wasn't initially changed). It does introduce a pain point in migrating to the 2026 trace data, (and unnecessary to maintain) hence being actually removed now.

Summary

  • Demand parser now writes <input_stem>.parquet instead of constructing a new filename from scenario/subregion/poe/demand_type metadata.
  • write_new_demand_filename and get_save_scenario_for_demand_trace removed
    • (scenario mapping is now a one-liner in the restructure function).
    • Removed associated tests.
  • Restructure function receives per-file metadata directly (no more all_input_file_metadata lookup inside function).
  • Test data name changed (only name - contents identical).
    • tests/test_data/output/Green_Energy_Exports_RefYear2011_CNSW_POE10_OPSO_MODELLING.parquettests/test_data/output/CNSW_RefYear_2011_HYDROGEN_EXPORT_POE10_OPSO_MODELLING.parquet

@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/isp_trace_parser/demand_traces.py 97.50% <100.00%> (-0.42%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nick-gorman nick-gorman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good Dylan!

Comment thread src/isp_trace_parser/demand_traces.py Outdated
Comment thread src/isp_trace_parser/demand_traces.py Outdated
@dylanjmcconnell dylanjmcconnell merged commit 34eb9c0 into main Jun 24, 2026
18 checks passed
@dylanjmcconnell dylanjmcconnell deleted the drop-output-rename-demand branch June 24, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants