Skip to content

Commit 7ff13f5

Browse files
jirhikerclaude
andcommitted
Run product automations monthly on staggered days
Move all product schedules from daily to monthly, spread across the month so the three source-fetch groups don't contend: - Well correlation: 1st (0 6 1 * *) - Water levels: 15th (0 6 15 * *) - Analytes / chemistry: 25th (0 6 25 * *) All at 06:00 America/Denver. Water-level products (parameter: waterlevels) and chemistry products each share a cohort, so every member now carries its category's cron and the cohort resolves to a single monthly schedule; well correlation is standalone. Update the stale "quarterly" comments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 7b95fb1 commit 7ff13f5

2 files changed

Lines changed: 24 additions & 24 deletions

File tree

orchestration/config/products.yaml

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ products:
66
output_type: ogc_summary
77
title: "NM Unified Water Levels Summary"
88
description: "Summary stats for water levels, all NM sources"
9-
schedule: "0 6 * * *"
9+
schedule: "0 6 15 * *"
1010
spatial_filter:
1111
state: NM
1212
sources:
@@ -17,7 +17,7 @@ products:
1717
output_type: ogc_timeseries
1818
title: "NM Water Levels Time Series"
1919
description: "Per-observation water level measurements, all NM sources"
20-
schedule: "0 7 * * *"
20+
schedule: "0 6 15 * *"
2121
spatial_filter:
2222
state: NM
2323
sources:
@@ -28,7 +28,7 @@ products:
2828
output_type: ogc_summary
2929
title: "NM Arsenic Summary"
3030
description: "Arsenic concentration summary stats, all NM sources"
31-
schedule: "0 9 * * *"
31+
schedule: "0 6 25 * *"
3232
spatial_filter:
3333
state: NM
3434
sources:
@@ -39,7 +39,7 @@ products:
3939
output_type: ogc_summary
4040
title: "NM TDS Summary"
4141
description: "TDS concentration summary stats, all NM sources"
42-
schedule: "0 10 * * *"
42+
schedule: "0 6 25 * *"
4343
spatial_filter:
4444
state: NM
4545
sources:
@@ -52,7 +52,7 @@ products:
5252
output_type: ogc_major_chemistry
5353
title: "NM Major Chemistry"
5454
description: "Wells with major-ion chemistry (latest value per analyte), well depth, all NM sources"
55-
schedule: "0 11 * * *"
55+
schedule: "0 6 25 * *"
5656
spatial_filter:
5757
state: NM
5858
sources:
@@ -67,7 +67,7 @@ products:
6767
output_type: ogc_waterlevel_trend
6868
title: "NM Water Level Trends"
6969
description: "Per-well depth-to-water trend (slope and category), all NM sources"
70-
schedule: "0 12 * * *"
70+
schedule: "0 6 15 * *"
7171
spatial_filter:
7272
state: NM
7373
sources:
@@ -80,7 +80,7 @@ products:
8080
output_type: ogc_analyte_trend
8181
title: "NM Arsenic Trends"
8282
description: "Per-well arsenic concentration trend (slope and category), all NM sources"
83-
schedule: "0 13 * * *"
83+
schedule: "0 6 25 * *"
8484
spatial_filter:
8585
state: NM
8686
sources:
@@ -91,7 +91,7 @@ products:
9191
output_type: ogc_analyte_trend
9292
title: "NM Nitrate Trends"
9393
description: "Per-well nitrate concentration trend (slope and category), all NM sources"
94-
schedule: "0 14 * * *"
94+
schedule: "0 6 25 * *"
9595
spatial_filter:
9696
state: NM
9797
sources:
@@ -104,7 +104,7 @@ products:
104104
output_type: ogc_mcl_exceedance
105105
title: "NM MCL Exceedances"
106106
description: "Wells flagged against drinking-water MCLs, all NM sources"
107-
schedule: "0 15 * * *"
107+
schedule: "0 6 25 * *"
108108
analytes: [arsenic, nitrate, fluoride, uranium, sulfate, chloride, tds]
109109
spatial_filter:
110110
state: NM
@@ -118,7 +118,7 @@ products:
118118
output_type: ogc_monitoring_recency
119119
title: "NM Monitoring Recency"
120120
description: "Per-well water-level monitoring recency and status, all NM sources"
121-
schedule: "0 16 * * *"
121+
schedule: "0 6 15 * *"
122122
stale_days: 365
123123
spatial_filter:
124124
state: NM
@@ -132,7 +132,7 @@ products:
132132
output_type: ogc_hardness
133133
title: "NM Water Hardness"
134134
description: "Per-well total hardness as CaCO3 (calcium + magnesium), all NM sources"
135-
schedule: "0 11 * * *"
135+
schedule: "0 6 25 * *"
136136
analytes: [calcium, magnesium]
137137
spatial_filter:
138138
state: NM
@@ -146,7 +146,7 @@ products:
146146
output_type: ogc_water_type
147147
title: "NM Water Type"
148148
description: "Per-well hydrochemical (Piper) water type from major-ion chemistry, all NM sources"
149-
schedule: "0 11 * * *"
149+
schedule: "0 6 25 * *"
150150
analytes: [calcium, magnesium, sodium, potassium, bicarbonate, carbonate, chloride, sulfate]
151151
spatial_filter:
152152
state: NM
@@ -161,7 +161,7 @@ products:
161161
output_type: ogc_data_density
162162
title: "NM Water Level Data Density"
163163
description: "Per-well water-level measurement coverage and frequency, all NM sources"
164-
schedule: "0 7 * * *"
164+
schedule: "0 6 15 * *"
165165
spatial_filter:
166166
state: NM
167167
sources:
@@ -175,7 +175,7 @@ products:
175175
output_type: ogc_waterlevel_change
176176
title: "NM Water Level Change"
177177
description: "Per-well depth-to-water change over a trailing window, all NM sources"
178-
schedule: "0 7 * * *"
178+
schedule: "0 6 15 * *"
179179
window_years: 5
180180
spatial_filter:
181181
state: NM
@@ -189,7 +189,7 @@ products:
189189
output_type: ogc_sar
190190
title: "NM Sodium Adsorption Ratio"
191191
description: "Per-well sodium adsorption ratio and irrigation hazard class, all NM sources"
192-
schedule: "0 11 * * *"
192+
schedule: "0 6 25 * *"
193193
analytes: [sodium, calcium, magnesium]
194194
spatial_filter:
195195
state: NM
@@ -203,7 +203,7 @@ products:
203203
output_type: ogc_ion_balance
204204
title: "NM Ion Balance"
205205
description: "Per-well major-ion charge balance error (QA screen), all NM sources"
206-
schedule: "0 11 * * *"
206+
schedule: "0 6 25 * *"
207207
analytes: [calcium, magnesium, sodium, potassium, bicarbonate, carbonate, chloride, sulfate]
208208
spatial_filter:
209209
state: NM
@@ -217,7 +217,7 @@ products:
217217
output_type: ogc_wqi
218218
title: "NM Water Quality Index"
219219
description: "Per-well CCME water quality index against drinking-water MCLs, all NM sources"
220-
schedule: "0 15 * * *"
220+
schedule: "0 6 25 * *"
221221
analytes: [arsenic, nitrate, fluoride, uranium, sulfate, chloride, tds]
222222
spatial_filter:
223223
state: NM
@@ -232,7 +232,7 @@ products:
232232
output_type: ogc_waterlevel_status
233233
title: "NM Water Level Status"
234234
description: "Per-well current water level vs the well's own record (percentile and status), all NM sources"
235-
schedule: "0 7 * * *"
235+
schedule: "0 6 15 * *"
236236
spatial_filter:
237237
state: NM
238238
sources:
@@ -246,7 +246,7 @@ products:
246246
output_type: ogc_seasonal_amplitude
247247
title: "NM Water Level Seasonal Amplitude"
248248
description: "Per-well within-year depth-to-water amplitude, all NM sources"
249-
schedule: "0 7 * * *"
249+
schedule: "0 6 15 * *"
250250
min_days_per_year: 4
251251
spatial_filter:
252252
state: NM
@@ -262,7 +262,7 @@ products:
262262
output_type: ogc_depletion_projection
263263
title: "NM Well Depletion Projection"
264264
description: "Per-well screening projection of years until depth-to-water reaches well depth, all NM sources"
265-
schedule: "0 12 * * *"
265+
schedule: "0 6 15 * *"
266266
spatial_filter:
267267
state: NM
268268
sources:
@@ -273,14 +273,14 @@ products:
273273
# combine gathers sites from EVERY source (sites-only, incl. the OSE POD
274274
# source) and correlates them via backend/well_correlation.py — explicit id
275275
# references (usgs_site_id / alternate_site_id) plus spatial+depth agreement,
276-
# since coordinates are only approximate. Generated quarterly (1st of Jan/Apr/
277-
# Jul/Oct, 06:00 America/Denver). Correlation thresholds are optional overrides
276+
# since coordinates are only approximate. Generated monthly (1st, 06:00
277+
# America/Denver). Correlation thresholds are optional overrides
278278
# (defaults live in backend/well_correlation.py).
279279
- id: nm_well_correlation
280280
output_type: ogc_well_correlation
281281
title: "NM Cross-Agency Well Correlation"
282282
description: "Per-well links to associated wells at other agencies and to OSE PODs, all NM sources"
283-
schedule: "0 6 1 */3 *"
283+
schedule: "0 6 1 * *"
284284
spatial_filter:
285285
state: NM
286286
sources:

orchestration/definitions.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -226,7 +226,7 @@ def _build_standalone_jobs_and_schedules(
226226
These products depend on no shared source asset, so they are not part of any
227227
cohort. Each gets its own job (selecting just its combine + geoserver assets)
228228
and its own schedule at the product's cron. The well correlation product runs
229-
quarterly (see products.yaml)."""
229+
monthly (see products.yaml)."""
230230
jobs: dict[str, "UnresolvedAssetJobDefinition"] = {}
231231
schedules: list[dg.ScheduleDefinition] = []
232232
for product in _products(products_config):

0 commit comments

Comments
 (0)