fix(edr): source water-chemistry EDR from the legacy NMA tables

[jirhiker]

Problem

water_chemistry is advertised in /ogcapi/collections and returns nothing on staging:

GET /ogcapi/collections/water_chemistry/locations  ->  {"type":"FeatureCollection","features":[]}

Not a missing view this time — the view exists and the query runs. It reads a model the data isn't in. ogc_water_chemistry sources the normalized chain (observation → sample → field_activity → field_event → thing), but per docs/chemistry-ingestion-runbook.md the live ingestion path writes only to the legacy tables:

Target tables: NMA_MajorChemistry, NMA_MinorTraceChemistry, NMA_Chemistry_SampleInfo

Which is why the sibling layers, on the same database, are full:

Collection	Source	Rows
major_chemistry_results	NMA_MajorChemistry	3333
minor_chemistry_wells	NMA_MinorTraceChemistry	2934
water_chemistry (EDR)	observation + sample	0

Approach

Repoint both EDR chemistry relations — public and internal mirror — at the legacy tables, at the per-result grain EDR needs. The existing pivot views are per-well summaries (one row per thing, analyte_count, latest date), so they aren't reusable; this goes to the base tables and unions the four families hanging off NMA_Chemistry_SampleInfo:

Table	Analyte	Value	Units	Date
NMA_MajorChemistry	"Analyte" / "Symbol"	"SampleValue"	"Units"	"AnalysisDate"
NMA_MinorTraceChemistry	analyte / symbol	sample_value	units	analysis_date
NMA_Radionuclides	"Analyte" / "Symbol"	"SampleValue"	"Units"	"AnalysisDate"
NMA_FieldParameters	"FieldParameter"	"SampleValue"	"Units"	(none — sample's CollectionDate)

Radionuclides and field parameters aren't in any current layer; they're the same shape, so including them brings pH, temperature, and conductivity into EDR at no extra cost. Rows that end up with no timestamp are dropped; EDR needs a time axis.

Interim by design. When chemistry reaches the normalized model, the relations move back and the EDR contract doesn't change — same collection, parameter-names, CoverageJSON. downgrade() restores the normalized definitions by importing them from the revisions that own them (z9a0b1c2d3e4, 2d3c3a268652) rather than copying, so they can't drift.

Materialized, not plain views

Both are MATERIALIZED, matching ogc_major_chemistry_results and ogc_minor_chemistry_wells. As plain views, every request re-planned a four-way UNION over the full legacy result tables — and get_fields() runs SELECT DISTINCT parameter_name, unit at provider construction, i.e. a full scan per request. Staging and production are already well past the size where that's affordable.

• Unique index on id → REFRESH MATERIALIZED VIEW CONCURRENTLY works.
• Indexes on thing_id, datetime, parameter_name → the provider's three filter columns.
• Freshness matches the other chemistry layers: the nightly pg_cron job discovers materialized views from the catalog, and both are registered in services/materialized_views.py for oco refresh-materialized-views after an ad-hoc ingestion.

thing_type

These relations deliberately carry no thing_type filter, so thing_type is exposed as a column and surfaces as a property on /locations features — otherwise a consumer couldn't tell a well from a spring. ogc_waterlevels has no such column, so the provider detects it instead of assuming; that collection is unchanged.

Detection reads pg_attribute, not information_schema.columns — the latter doesn't list materialized views at all, and silently reported "no thing_type" until this was caught end-to-end.

Three deliberate differences from the pivot views

1. No thing_type filter. ogc_major_chemistry_results restricts to 'water well' because it's a wells layer. This is a chemistry collection — chemistry collected at a spring belongs in it.
2. Publication gated on thing.release_status = 'public' (the convention f4a5b6c7d8e9 set for legacy-backed views) plus NMA_Chemistry_SampleInfo."PublicRelease" not being explicitly false. The pivot views ignore PublicRelease; honouring it errs toward withholding, and NULL counts as "not suppressed" so the layers agree on rows carrying no opinion.
3. parameter_name is raw trimmed analyte text, falling back to symbol. The pivot views canonicalize through long CASE blocks that only cover the analytes they expose as columns. Raw text keeps every analyte reachable, at the cost of aliases surfacing separately (Ca and Calcium both appear). That's ADR3's open chemistry-cardinality question — canonicalizing is follow-up and changes only the vocabulary, not this plumbing.

Verification

Seeded ocotilloapi_test across all four families, then removed the fixtures:

Case	Public	Internal
Public well — major, minor, radionuclide, field param	4 rows (all families)	4
Spring, public	included (no thing_type filter)	included
Draft thing	excluded	included
Sample with PublicRelease = false	excluded	included
NULL analyte	dropped	dropped
Total	5	7

Field-parameter row correctly picked up the sample's CollectionDate as its timestamp.

Also confirmed: both relations are relkind 'm' with the four expected indexes; REFRESH ... CONCURRENTLY succeeds and picks up newly inserted rows; the provider's own SQL runs against them (_read projection, get_fields DISTINCT, bbox / ST_Intersects / datetime predicates); thing_type appears on chemistry /locations features and is absent — without error — on ogc_waterlevels; a missing relation yields False rather than raising; and a downgrade → upgrade cycle restores each definition.

tests/test_edr_provider.py (new), test_cli_commands.py, test_migration_view_parity.py, test_pygeoapi_mount.py — 34 passed.

After deploy

Chemistry EDR is materialized, so it is populated by the migration and refreshed nightly:

curl -s "https://ocotillo-api-staging.newmexicowaterdata.org/ogcapi/collections/water_chemistry/locations?f=json"

🤖 Generated with Claude Code