Add water consumption from electricity mix - #1407
Draft
benoit-cty wants to merge 6 commits into
Draft
Conversation
Report in water_consumed the water used to generate the electricity consumed, on top of the direct cooling water given by the wue parameter. The water intensity (L/kWh) is computed as a weighted average of per-source water consumption factors over the energy mix of the country (or Canadian region) where the machine runs, with a world-average fallback when coverage of the mix is below 90%. The intensity is cached per location and a failure of the water estimation can never break the emissions tracking. Ported and reworked from feat/water_consumption_S, started by students: - fix the source keys of water_consumption_per_source.json so they match global_energy_mix.json (the original computation always fell back to the world average), and add oil/hydro/biofuel factors - fix a gal/kWh vs gal/MWh unit error in the cloud fallback path - make the world-average fallback consistent with the computed country values (weighted by the world energy mix) to avoid a 5x discontinuity at the coverage threshold - accumulate water incrementally in _update_emissions alongside emissions, instead of recomputing from the total at report time - drop code duplicated from the Emissions class and the dead USA regional branch (only Canada has regional energy mix data) Co-authored-by: Sifflet_blanc <flo.crahay@gmail.com> Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add the water_consumed column to the emissions table (alembic migration included), accept it on emission creation, and sum it in the run, experiment, project and organization reports. Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Show a water radial chart next to the energy one on the organization, project and public project dashboards, and add water_consumed to the CSV exports. Co-authored-by: Simon Decostanzi <gbsny8zh@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1407 +/- ##
==========================================
- Coverage 91.58% 91.58% -0.01%
==========================================
Files 49 50 +1
Lines 5100 5216 +116
==========================================
+ Hits 4671 4777 +106
- Misses 429 439 +10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Member
|
Thanks for this integration Benoît! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an estimation of the water consumed by your compute, reported in the existing
water_consumedfield (litres), across the whole stack:water_consumedis now the sum ofwueparameter (unchanged, defaults to 0), andcodecarbon/data/private_infra/water_consumption_per_source.json(Meldrum et al. 2013, Macknick et al. 2012). When sources with known water intensity cover less than 90% of a country's mix (e.g. geothermal-heavy Kenya/Iceland), a world average of ~3.75 L/kWh is used (computed from the same per-source table weighted by the world mix, so the fallback is consistent with computed country values). Cloud providers publish no per-region water data, so cloud runs fall back to the water intensity of the host country.water_consumedcolumn onemissions(alembic migration included), accepted on emission creation and summed in run/experiment/project/organization reports.water_consumedadded to CSV exports.examples/water_consumption.pyadded.This is a rework of the student branches
feat/water_consumptionandfeat/water_consumption_S(the latter is a superset; both can be deleted once this merges). The branches were 481 commits behind master, so the feature was ported onto master rather than rebased, keeping the students as co-authors. Beyond the rebase, the main fixes over the original branch:global_energy_mix.json(natural_gasvsgas, missingoil/hydroelectricity/biofuel), so the energy-mix computation always fell back to the world average — and one fallback path called a method that did not exist (WaterPerKWh.from_g_per_kWh). Keys now match, hydro/oil/biofuel factors were added, and coverage is ≥90% for 207 of 213 countries.gal/kWhvsgal/MWh) in the cloud fallback path._update_emissions()alongside emissions instead of being recomputed from the total at report time;wuewater is added rather than replacing the estimate (direct + indirect, as in Li et al. 2023, "Making AI Less Thirsty").Emissionsclass, the noisy per-measurement warnings, an unrelatedalembic/env.pychange, and a migration that carried unrelated auto-generated schema changes with a wrongdown_revision.An adversarial multi-angle review was then run on the branch and its accepted findings applied: the water intensity is cached per location (it is constant for a run, and its fallback warnings now log once instead of every measurement cycle), the water estimation is wrapped so a failure can never break the carbon tracking, the new data file was added to
[tool.setuptools.package-data](verified present in a built wheel), the dead USA regional branch was dropped (only Canada ships a regional energy mix), anullwater_consumedis coalesced to 0 before insert (the column is NOT NULL), and the duplicated webappRadialChartDatatype/skeleton markup were consolidated.Behavior change to be aware of (not an API break):
water_consumedwas previouslywue × energyand therefore 0 unlesswuewas set. It is now non-zero for every run, since the indirect water of electricity generation is always estimated. Orders of magnitude: France ≈ 3.8 L/kWh (nuclear+hydro), Poland ≈ 1.8 L/kWh (coal), Norway ≈ 15 L/kWh (hydro reservoir evaporation — high but consistent with the literature; flagged as uncertain in the docs). If maintainers prefer the estimate to stay opt-in, gating it behind a parameter is a small follow-up.Related Issue
Extends #894 (water usage estimation), which was partially addressed by the
wueparameter in #941. Supersedes the branchesfeat/water_consumptionandfeat/water_consumption_S.Motivation and Context
Computing does not only emit CO₂: generating the electricity it uses also consumes fresh water (thermal/nuclear plant cooling, reservoir evaporation). Reporting only the datacenter WUE misses this — usually dominant — indirect part, and required the user to know their WUE. With this change every run gets a water estimate out of the box, computed from the same energy-mix data already used for carbon intensity.
How Has This Been Tested?
tests/test_water_consumption.py(14 tests): unit conversions, per-country values (FRA/POL), scaling with energy, world-average fallbacks (unknown country, low-coverage country, cloud without geo), CAN regional mix, USA regional fallback, and a data-file consistency test that fails if the JSON keys ever drift fromglobal_energy_mix.json.uv run task test-package— passes excepttest_task_energy_with_live_update_interference, which fails identically on master (pre-existing, unrelated: energy sum mismatch).uv run task test-api-unit— 119 passed (fixtures extended withwater_consumed; the existing wue default/custom tests now also assert the new field).tsc --noEmitclean,vitest run— same 3 pre-existing interaction-test failures as master, everything else passes; prettier clean.water_consumedtoemissions.csvwith the expected 3.83 L/kWh ratio for France;examples/water_consumption.pyruns.Screenshots (if appropriate):
Types of changes
AI Usage Disclosure
Checklist: