feat(ogc): add the water well field operations internal layer - #913
Closed
jirhiker wants to merge 2 commits into
Closed
feat(ogc): add the water well field operations internal layer#913jirhiker wants to merge 2 commits into
jirhiker wants to merge 2 commits into
Conversation
Adds water_well_field_operations, an OGC API - Features collection on
/ogcapi-internal only: one feature per water well carrying what a field crew
needs to plan and execute a visit. Every other well layer answers a scientific
question -- water level, chemistry, trend. This one answers an operational one:
may we go to this well, what are we allowed to do when we get there, and is it
overdue?
86 columns: identity and location (including latitude and longitude in decimal
degrees and how the elevation was obtained), construction detail, the current
measuring point, all five status types, the three standing landowner
permissions, monitoring frequency and group membership, manual water-level and
chemistry-sampling statistics, field-visit history, the installed logger and
its continuous record, landowner contacts, and access and directions notes.
Two relations rather than one (migration e1f2a3b4c5d6):
ogc_internal_water_well_field_operations_stats materialized, nightly refresh
Aggregates over observation and transducer_observation.
ogc_internal_water_well_field_operations plain view, served by pygeoapi
Live join of thing, location, status_history, permission_history,
measuring_point_history, monitoring_frequency_history, deployment,
sensor, contact and notes, LEFT JOINed to the stats matview.
The split is not tidiness. Staleness is dangerous on exactly the columns that
are cheap to read: a revoked sampling permission that still reads true until
the next nightly refresh sends a crew onto land they are no longer welcome on.
The current-record rule is also written against CURRENT_DATE, which inside a
materialized view would freeze at refresh time.
Three decisions worth review:
* Internal-only with no public twin. The layer publishes landowner and operator
contact details -- name, organization, role, phone, email -- alongside
staff-written access notes. ogc_water_well_field_operations does not exist
and must never be created. Every credential the internal mount accepts now
reaches personal information, the static desktop-GIS keys included; notices
added to CLAUDE.md and docs/internal-ogc-desktop-gis.md.
* The three permission columns are three-valued. NULL means no permission is on
record, which is not a refusal -- it means nobody has asked the landowner
yet. Collapsing it to false would tell a crew a well is off limits when the
truth is unknown.
* History reads honour end_date, unlike ogc_actively_monitored_wells, which
takes the greatest start_date and lets a status closed in 2019 read as
current. Written up in docs/ogc_conventions.md.
Requested as amp_field_well_operations and renamed before merge per the
pre-merge check in docs/ogc_conventions.md: amp is an unexpanded abbreviation
whose expansion is recorded nowhere in this codebase, AMP labels the layer's
audience rather than filtering its rows (the row set is every water well), and
Group B analytic layers prefer a water_well_ prefix. The AMP crews stay named
in the collection description.
Multi-valued columns are comma-joined text rather than arrays: this layer is
pulled into ArcGIS Pro and QGIS and exported to File Geodatabase and GeoPackage
for offline field use, and neither format has a list type.
Depth to water uses the (value - COALESCE(measuring_point_height, 0))
convention shared with ogc_water_well_summary, ogc_latest_depth_to_water_wells
and ogc_well_water_column, and elevation_method reuses that summary's
provenance lookup verbatim, so the layers cannot disagree.
Twelve supporting indexes come with the migration; none of the foreign keys
these lookups traverse was indexed.
Design rationale in docs/water-well-field-operations-layer.md, column-by-column
sources in docs/water-well-field-operations-columns.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Coverage✅ 79.35% total — gate is 75%. Coverage for the Python files changed in this PR
|
Contributor
|
@jirhiker I was working on this today (see ticket Jira ticket 1202) and was about to push a PR. I'll take a look at your work as well and compare. |
jirhiker
marked this pull request as draft
September 1, 2026 03:30
Member
Author
|
@ksmuczynski oops. sorry i didn't see you were already working on this. ill close this PR, but you can still compare to feat/amp-field-operations-ogc-layer |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new OGC API - Features collection,
water_well_field_operations, on/ogcapi-internalonly. One feature per water well, carrying what a field crewneeds to plan and execute a visit.
Every other well layer answers a scientific question — water level, chemistry,
trend. This one answers an operational one: may we go to this well, what are
we allowed to do when we get there, and is it overdue?
86 columns: identity and location (latitude/longitude in decimal degrees, plus
how the elevation was obtained), construction detail, the current measuring
point, all five status types, the three standing landowner permissions,
monitoring frequency and group membership, manual water-level and
chemistry-sampling statistics, field-visit history, the installed logger and
its continuous record, landowner contacts, and access and directions notes.
Column-by-column sources:
docs/water-well-field-operations-columns.md.Design rationale:
docs/water-well-field-operations-layer.md.Two relations, not one
ogc_internal_water_well_field_operations_statsobservationandtransducer_observationogc_internal_water_well_field_operationsthing,location, the four history tables,deployment,sensor,contact,notes, LEFT JOINed to the stats matviewThe split is not tidiness. Staleness is dangerous on exactly the columns that
are cheap to read: a revoked sampling permission that still reads
trueuntil the next nightly refresh sends a crew onto land they are no longer
welcome on. The current-record rule is also written against
CURRENT_DATE,which inside a materialized view would freeze at refresh time — a permission
that expired this morning would still read current tonight.
Please review these three decisions
1. Internal-only, with no public twin — this layer publishes PII.
Landowner and operator contact details (name, organization, role, phone,
email) alongside staff-written access notes that routinely contain gate codes.
ogc_water_well_field_operationsdoes not exist and must never be created.Every credential the internal mount accepts now reaches personal information,
including the static desktop-GIS keys, which are shared secrets revocable
only by redeploy. Notices added to
CLAUDE.mdanddocs/internal-ogc-desktop-gis.md.docs/api-key-management.mdneeds the samenotice — that file lives on
feat/api-key-managementand does not exist onthis branch, so it could not be edited here.
2. The three permission columns are three-valued.
true= a current grantallows it,
false= a current grant refuses it,NULL= no permission is onrecord, which is not a refusal — it means nobody has asked the landowner yet.
Collapsing NULL to false would tell a crew a well is off limits when the truth
is unknown. Any consumer-facing rendering has to carry the third state; the
field descriptions say so, since that prose is the only place a consumer reads
it.
3. History reads honour
end_date.ogc_actively_monitored_wellstakesthe greatest
start_dateand ignoresend_date, so a status closed in 2019still reads as current there. Tolerable on a summary layer, not on one whose
job is to say what is true today. The divergence is written into
docs/ogc_conventions.mdso the two views don't read as one being a mistake.Naming
Requested as
amp_field_well_operationsand renamed before merge, per thepre-merge check in
docs/ogc_conventions.md. Three rules pushed the same way:ampis an unexpanded abbreviation whose expansion is recorded nowhere in thiscodebase; AMP labels the layer's audience rather than filtering its rows (the
row set is every water well, no group predicate), so the name would claim a
scope the data doesn't have; and Group B analytic layers prefer a
water_well_prefix. The AMP crews stay named in the collection description,where an audience belongs.
Consistency with the existing catalogue
(value - COALESCE(measuring_point_height, 0))— theconvention shared with
ogc_water_well_summary,ogc_latest_depth_to_water_wellsandogc_well_water_column.elevation_methodreuseswater_well_summary's provenance lookup verbatim.pulled into ArcGIS Pro and QGIS and exported to File Geodatabase and
GeoPackage for offline field use, and neither format has a list type.
water_well_summary, wells with no measurements still appear — awell nobody has measured is exactly the well a crew needs to find.
Also in this PR
was indexed; Postgres does not index foreign keys on its own.
discovers matviews from the catalog, so no schedule change.
blackreformat intests/test_cli_commands.py, applied bythe repo's own pre-commit hook.
Testing
Full suite green: 1224 passed, 84 skipped, 6 xpassed.
blackandflake8clean. Migration downgrade/upgrade round trip verified, and
REFRESH MATERIALIZED VIEW CONCURRENTLYconfirmed working against the statsview's unique index.
18 new tests in
tests/test_ogc_water_well_field_operations.pycover theload-bearing rules: unmeasured wells still appear, permission
NULL≠ false,an expired grant is not current, a barometer is not a datalogger, the
measuring-point convention, coordinates agree with the geometry, provenance on
a different field doesn't leak into
elevation_method, and the layer 404s onthe public mount.
Open before this ships
docs/api-key-management.mdneeds the PII notice (see decision 1).Access Statushas no values.status_typecarries it, but thestatus_valuelexicon has no access-related terms — all eleven map to Well,Monitoring, Open, or Datalogger Suitability. The column publishes and reads
null until the vocabulary gains terms scoped to it. Worth a look at
production data.
continuous_reading_countaggregates the largest table in the schema. If thenightly refresh proves too slow at production scale, the fallback is to
source the three continuous columns from
transducer_daily_data.🤖 Generated with Claude Code