Bc cvar updates by bcakire · Pull Request #107 · ReEDS-Model/ReEDS

bcakire · 2026-05-26T18:30:14Z

Summary

Add CVaR/NCVAR resource adequacy reporting and target checks to the PRAS and stress-period workflow.

This PR extends the existing PRAS risk-metric functionality so that CVaR is no longer only calculated internally during a PRAS run. The PRAS workflow can now also write CVaR/NCVAR reporting outputs and sample-level total shortfall files that can be reused by the Python stress-period workflow.

This PR adds:

CVAR/NCVAR reporting from run_pras.jl
A compact PRAS output file with per-sample total shortfall by region
Annual CVAR/NCVAR calculation in stress_periods.py
Threshold-based CVAR/NCVAR target checks
Check-only behavior for CVAR/NCVAR so risk targets can be evaluated without adding stress periods or changing PRM

Technical details

Implementation notes

run_pras.jl updates:

Added --cvar_alpha argument to control the CVaR confidence level
Added --write_shortfall_samples_totals argument
Added CVAR/NCVAR reporting when PRAS.ShortfallSamples() is available
Writes:
PRAS_{t}i{iteration}-risk_metrics.csv
PRAS_{t}i{iteration}-shortfall_totals_by_sample.h5
risk_metrics.csv reports:
CVAR in MWh
NCVAR in ppm
alpha
estimate
standard error
VaR
shortfall_totals_by_sample.h5 stores:
sample index
USA total shortfall by sample
regional total shortfall by sample

stress_periods.py updates:

Added CVAR_METRICS = {'CVAR', 'NCVAR'}
Added support for reading:
PRAS_{t}i{iteration}-shortfall_totals_by_sample.h5
Added:
get_cvar_alpha()
get_shortfall_totals_by_sample()
_sample_cvar()
get_annual_cvar_stress_metric()
evaluate_cvar_target_check()
CVAR/NCVAR are calculated annually by hierarchy level
NCVAR is normalized by annual load and reported in ppm
CVAR/NCVAR are intentionally excluded from stress-period selection

Additional changes

Replaced memory-intensive hourly shortfall sample aggregation with direct use of PRAS_*shortfall_totals_by_sample.h5 where sample-level total shortfall is sufficient
This avoids needing to read PRAS_*shortfall_samples.h5 only to aggregate it again by sample and region
Existing EUE/NEUE/LOLE stress-period selection behavior is unchanged

Switches added/removed/changed

Added:

GSw_PRM_CVARAlpha
GSw_PRM_StressThresholdNCVAR
GSw_PRM_StressThresholdCVAR

Example usage:

GSw_PRM_StressThresholdMetrics = EUE/NCVAR
GSw_PRM_StressThresholdEUE = transgrp_1000_EUE_sum
GSw_PRM_StressThresholdNCVAR = transgrp_10_NCVAR_cvar
GSw_PRM_CVARAlpha = 0.95

For PRAS reporting, run_pras.jl can be called with:

--write_shortfall_samples_totals 1
--cvar_alpha 0.95

Validation, testing, and comparison report(s)

Validation completed on the Pacific test case.

Tests performed:

Ran run_pras.jl with:
--write_shortfall_samples_totals 1
--cvar_alpha 0.95
Verified generation of:
PRAS_2032i0-risk_metrics.csv
PRAS_2032i0-shortfall_totals_by_sample.h5
Verified that shortfall_totals_by_sample.h5 contains sample-level total shortfall by USA and region
Verified that risk_metrics.csv reports CVAR and NCVAR
Verified:
get_shortfall_totals_by_sample()
get_annual_cvar_stress_metric()
evaluate_cvar_target_check()
Confirmed NCVAR threshold failures do NOT:
add stress periods
trigger PRM updates
Confirmed standard EUE stress-period workflow remains unchanged

Example validation output:

GSw_PRM_StressThreshold = country_-1_NCVAR_cvar failed for:
region
USA 0.0
Name: NCVAR, dtype: float64
NCVAR is check-only: no stress periods and no PRM increment will be added.

Checklist for author

Details to double-check

General information to guide review

[X ] Zero impact on results of default case
[X ] No large data file(s) added/modified
[X ] No substantive impact on runtime for full-US reference case
No substantive impact on folder size for full-US reference case
No change to process flow (runreeds.py, reeds/core/solve/solve.py)
No change to code organization
No change to package requirements (environment.yml or Project.toml)

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

Tag points of contact here if you would like additional review of the relevant parts of the model

bsergi

Here are some initial thoughts from a quick look. Happy to touch base when you get back to discuss any of these. Let me know when you've taken another pass here and I can review again.

One high-level comment: can you organize the PR summary text a bit more? The technical details section has a lot of information but is a bit difficult to follow. Also, I didn't quite understand your test output GSw_PRM_StressThreshold = country_-1_NCVAR_cvar failed. What does it mean to have a -1 threshold? Ultimately for the PR we'll also want a comparison of default case from cases.csv from both this branch and main to show there weren't any changes.

bsergi · 2026-06-18T14:54:30Z

revert the changes to this file before merging

bsergi · 2026-06-18T14:56:39Z

            required = false
        "--write_shortfall_samples"
-            help = "Write the sample-level shortfall"
+            help = "Write the sample-level shortfall (hourly, large)"


What are the large and small designations? I might suggest just "Write per-sample hourly shortfall by region" for this one and "Write per-sample total shortfall by region" for the one below.

bsergi · 2026-06-18T14:57:02Z

            default = 0
            required = false
+        "--write_shortfall_samples_totals"
+            help = "Write per-sample total shortfall by region (small)"


this is total over the entire PRAS time period and not annual, right?

bsergi · 2026-06-18T14:58:35Z

+        _,_,_,_,energyunit = PRAS.get_params(sys)
+        alpha = Float64(args["cvar_alpha"])
+        cvar_obj = PRAS.CVAR(energyunit, results["short_samples"], alpha)
+        @info(cvar_obj)


I think PRAS is setup so that the above info statements provide context (e.g., @info "$(PRAS.EUE(results["short"]))" results in EUE = 851000±5000 MWh/131400h MWh) being printed). Does this do something similar?

bsergi · 2026-06-18T15:47:19Z

-        end
-        ## Write it
+        sf = results["short_samples"]
+        region_names = sf.regions.names


the previous code filtered using sys.regions.names. I think this was because there are some pseudo regions for DC converter stations that don't have load and thus don't need to be included here, so we may want to continue to drop them (@patrickbrown4 might be able to weigh in on this one).

bsergi · 2026-06-18T16:27:28Z

+    return x.sort_values(ascending=False).iloc[:n_tail].mean()
+
+
+def get_annual_cvar_stress_metric(case, t, stress_metric='NCVAR', iteration=0, alpha=0.95):


this function and the next one get pretty long, so it would be good to add comments throughout to provide some documentation on what they are doing.

bsergi · 2026-06-18T16:31:40Z

+    x = pd.Series(samples).dropna().astype(float)
+    if x.empty:
+        return np.nan
+    if not (0 <= alpha < 1):


Feels like this check might fit better in get_cvar_alpha. An even better approach might be to add a check before the ReEDS run which would avoid having runs get this far and then running into an error (best place for that would probably be here:

ReEDS/runreeds.py

Line 222 in 1a8ab63

def check_compatibility(sw):

)

bsergi · 2026-06-18T16:39:35Z

+
+def get_annual_cvar_stress_metric(case, t, stress_metric='NCVAR', iteration=0, alpha=0.95):
+    stress_metric = stress_metric.upper()
+    if stress_metric not in CVAR_METRICS:


The multi-metrics PR (#99) now drops the metric tag in the switch value; for example, GSw_PRM_StressThresholdNEUE now just has a value of transgrp_1_sum instead of transgrp_1_EUE_sum (see here). I think we could do the same in this PR and then you could drop the CVAR_METRICS and the check and just iterate over whichever switches are activated.

bsergi · 2026-06-18T16:48:21Z

-                                                                   )
+        for criterion in sw[f'GSw_PRM_StressThreshold{metric}'].split('/'):
+            print(f"Evaluating GSw_PRM_StressThreshold {metric} with criterion: {criterion}")
+            stress_criteria = _evaluate_stress_threshold_criterion(stress_criteria, criterion, sw, t, iteration, dfenergy_r, stressperiods_this_iteration,


This line looks like just a formatting change--I'd suggest keeping as the original had it.

bsergi · 2026-06-18T16:53:02Z

+    return pd.concat(_metric, names=['level','metric','region']).rename(stress_metric)
+
+
+def evaluate_cvar_target_check(sw, t, iteration=0, stress_metrics=None):


Much of this function, including the core check of failed = this_test.loc[this_test > threshold], seems to replicate behavior in _evaluate_stress_threshold_criterion. Do you think we could merge the CVAR treatment into that existing function and just have some control statements to turn off adding new stress periods for now when using CVAR metrics? It's possible having a second function makes the most sense here, but I worry a little bit about maintaining two similar functions.

Burcin Cakir Erdener and others added 3 commits May 20, 2026 05:38

add CVaR reporting to run_pras.jl

29b6be0

fix fix

33d2a02

Add CVaR target checks for stress periods

e0ae25e

bcakire marked this pull request as draft May 28, 2026 06:19

merged with aa/multimetrics/06_02_26

9bc6424

patrickbrown4 self-requested a review June 2, 2026 23:50

bsergi reviewed Jun 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bc cvar updates#107

Bc cvar updates#107
bcakire wants to merge 4 commits into
ReEDS-Model:aa/multi_metricsfrom
bcakire:bc_cvar_updates

bcakire commented May 26, 2026 •

edited

Loading

Uh oh!

bsergi left a comment •

edited

Loading

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

bsergi Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		return x.sort_values(ascending=False).iloc[:n_tail].mean()


		def get_annual_cvar_stress_metric(case, t, stress_metric='NCVAR', iteration=0, alpha=0.95):

		return pd.concat(_metric, names=['level','metric','region']).rename(stress_metric)


		def evaluate_cvar_target_check(sw, t, iteration=0, stress_metrics=None):

Uh oh!

Conversation

bcakire commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Technical details

Implementation notes

Additional changes

Switches added/removed/changed

Validation, testing, and comparison report(s)

Checklist for author

Details to double-check

General information to guide review

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

Tag points of contact here if you would like additional review of the relevant parts of the model

Uh oh!

bsergi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bcakire commented May 26, 2026 •

edited

Loading

bsergi left a comment •

edited

Loading