Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,18 @@ ArcGIS Pro cannot send a bearer token at all and neither desktop client can
refresh an Authentik token. Read **`docs/internal-ogc-desktop-gis.md`** before
changing the credential paths.

**`/ogcapi-internal` carries landowner PII.** The `water_well_field_operations`
collection publishes contact name, organization, role, phone and email for well
owners and operators, plus staff-written access notes. Every credential the
mount accepts reaches it, the static keys included. It is internal-only with
**no public twin** — `ogc_water_well_field_operations` does not exist and must
never be created. The layer also honours `end_date` when reading history
tables, unlike `ogc_actively_monitored_wells`, so "may we sample here" cannot
outlive the permission that granted it. Read
**`docs/water-well-field-operations-layer.md`** before changing it, and
**`docs/water-well-field-operations-columns.md`** for where each column comes
from.

### OGC field descriptions

Per-column `title`/`description`/unit for every collection lives in
Expand Down
831 changes: 831 additions & 0 deletions alembic/versions/e1f2a3b4c5d6_add_water_well_field_operations_layer.py

Large diffs are not rendered by default.

458 changes: 458 additions & 0 deletions core/ogc-field-descriptions.yml

Large diffs are not rendered by default.

47 changes: 47 additions & 0 deletions core/pygeoapi-config-internal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,53 @@ resources:
table: ogc_internal_locations
geom_field: point

water_well_field_operations:
type: collection
title: Water Well Field Operations
description: >-
One record per water well, assembled for the crews who visit them. Where
every other well layer answers a scientific question, this one answers an
operational one: whether a crew may go to a well, what they are permitted
to do when they get there, and whether it is overdue. It pairs the well's
construction, completion formation and aquifer system with its
measuring-point detail, current well, monitoring, open and
datalogger-suitability status, the standing landowner permissions for
water-level measurement, chemistry sampling and logger installation, every
sensor currently deployed at it (not just data loggers), how long it has
been since it was last measured and last sampled, and the landowner
contact and staff-written notes -- access, directions, maintenance and
the rest -- needed to reach it. Published on the internal mount only,
because it carries personal contact details for landowners and operators
alongside staff-written access instructions.
keywords:
[
water-wells,
field-operations,
landowner-contacts,
access-permissions,
monitoring-status,
currently-installed-equipment,
aquifer-system,
internal,
]
extents:
spatial:
bbox: [-109.05, 31.33, -103.00, 37.00]
crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
providers:
- type: feature
name: core.feature_provider.DescribedPostgreSQLProvider
data:
host: {postgres_host}
port: {postgres_port}
dbname: {postgres_db}
user: {postgres_user}
password: {postgres_password_env}
search_path: [public]
id_field: id
table: ogc_internal_water_well_field_operations
geom_field: point

latest_depth_to_water_wells:
type: collection
title: Latest Depth to Water (Water Wells)
Expand Down
10 changes: 10 additions & 0 deletions docs/internal-ogc-desktop-gis.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ collections. It is gated by `core/internal_ogc_auth.py`, an ASGI middleware that
runs in front of the raw Starlette Mount — FastAPI's `Depends()` machinery never
sees these requests, so none of the `*_dependency` role parameters apply here.

> **An internal credential dispenses personal information.** The
> `water_well_field_operations` collection publishes landowner and operator
> contact details — name, organisation, role, phone number, email address —
> alongside staff-written access notes that routinely contain gate codes and
> names. Every credential accepted by this mount reaches it: an Authentik JWT
> carrying `OGCInternal`, and the static keys in `INTERNAL_OGC_API_KEYS`, which
> are shared secrets revocable only by redeploy. Any credential path added
> later inherits the same reach. Issue keys with that in mind, and read
> `docs/water-well-field-operations-layer.md` before widening who holds one.

## Why there are static API keys at all

The mount originally accepted only `Authorization: Bearer <Authentik JWT>`.
Expand Down
27 changes: 27 additions & 0 deletions docs/ogc_conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,33 @@ human-readable display name).
| `locations` | N/A | Locations | N/A | N/A | Conforms. Hidden from public catalog — scope decision, not a naming fix |
| `avg_tds_wells` | `water_well_average_total_dissolved_solids` | Average Total Dissolved Solids (Water Wells) | N/A | 2 | Title implemented. `avg` abbreviation remains in the ID; hidden from public catalog but the rule still applies for internal consumers |
| `latest_depth_to_water_wells` | N/A | Latest Depth to Water (Water Wells) | N/A | N/A | Conforms. Hidden from public catalog — audit flagged it as redundant with `water_well_summary`, not a naming defect |
| `water_well_field_operations` | N/A | Water Well Field Operations | N/A | N/A | Conforms. Internal-only with no public form at all — it publishes landowner contact details and staff-written access notes |

## Current-record semantics

Layers that read a history table (`status_history`, `permission_history`,
`measuring_point_history`, `monitoring_frequency_history`) disagree about what
"current" means, and the disagreement is deliberate.

`ogc_actively_monitored_wells` takes the row with the greatest `start_date` and
**ignores `end_date`**, so a monitoring status closed in 2019 still reads as
current there. That is tolerable on a summary layer, where the question is
roughly "is this well in the programme".

`ogc_internal_water_well_field_operations` honours the window:

```sql
WHERE h.start_date <= CURRENT_DATE
AND (h.end_date IS NULL OR h.end_date >= CURRENT_DATE)
ORDER BY h.start_date DESC, h.id DESC
```

Its columns answer "may a crew do this today", and a permission that ran out
last month must not read as a permission. A row whose window has not opened yet
is not current either, and reads null.

New layers should follow the second form. The first is kept only because
changing it would move rows in a published layer.

## References

Expand Down
105 changes: 105 additions & 0 deletions docs/water-well-field-operations-columns.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# water well field operations — column sources

Every column published by `ogc_internal_water_well_field_operations`, in view
order, and where its value comes from. Columns marked *(stats)* are read from
the `ogc_internal_water_well_field_operations_stats` materialized view and are as
fresh as its last refresh; everything else is joined live on each request.

"Current record" means the history row satisfying
`start_date <= CURRENT_DATE AND (end_date IS NULL OR end_date >= CURRENT_DATE)`,
latest `start_date` first. "Most recent association" means the
`location_thing_association` row with no `effective_end`, latest
`effective_start` first.

Generated from the view definition in
`alembic/versions/e1f2a3b4c5d6_add_water_well_field_operations_layer.py`. Field
prose lives in `core/ogc-field-descriptions.yml`; the design rationale lives in
`docs/water-well-field-operations-layer.md`.

| Column | Source |
| --- | --- |
| `id` | `thing.id` |
| `name` | `thing.name` |
| `station_type` | Literal `'water well'` — the view's row filter. Named `station_type` rather than `thing_type` to match the naming already established on the public thing-type views |
| `release_status` | `thing.release_status` |
| `alternate_ids` | `thing_id_link.alternate_organization` + `.alternate_id`, comma-joined |
| `latitude` | `ST_Y(location.point)` — decimal degrees, WGS 84 |
| `longitude` | `ST_X(location.point)` — decimal degrees, WGS 84 |
| `elevation` | `location.elevation`, most recent association |
| `well_depth` | `thing.well_depth` |
| `hole_depth` | `thing.hole_depth` |
| `well_casing_diameter` | `thing.well_casing_diameter` |
| `well_casing_depth` | `thing.well_casing_depth` |
| `well_completion_date` | `thing.well_completion_date` |
| `well_driller_name` | `thing.well_driller_name` |
| `well_construction_method` | `thing.well_construction_method` |
| `well_pump_type` | `thing.well_pump_type` |
| `well_pump_depth` | `thing.well_pump_depth` |
| `formation_completion_code` | `thing.formation_completion_code` |
| `formation_completion_description` | `lexicon_term.definition` where `term = thing.formation_completion_code`. Neither this nor `aquifer_system_name` below is date-windowed — neither source table carries a `start_date`/`end_date` |
| `well_purpose` | `well_purpose.purpose`, comma-joined |
| `well_casing_material` | `well_casing_material.material`, comma-joined |
| `aquifer_system_name` | `aquifer_system.name` via `thing_aquifer_association`, comma-joined |
| `screen_count` | `count(well_screen)` |
| `screen_depth_top` | `well_screen.screen_depth_top`, every interval, semicolon-joined, ordered shallowest first, `COALESCE(..., '')` for the same reason as the equipment columns |
| `screen_depth_bottom` | `well_screen.screen_depth_bottom`, same intervals, same order, same `COALESCE(..., '')` treatment |
| `screen_description` | `well_screen.screen_description`, same intervals, same order, same `COALESCE(..., '')` treatment |
| `mp_height` | `measuring_point_history.measuring_point_height`, current record |
| `mp_description` | `measuring_point_history.measuring_point_description`, current record |
| `well_status` | `status_history.status_value` where `status_type = 'Well Status'`, current record |
| `monitoring_status` | `status_history.status_value` where `status_type = 'Monitoring Status'`, current record |
| `open_status` | `status_history.status_value` where `status_type = 'Open Status'`, current record |
| `datalogger_suitability_status` | `status_history.status_value` where `status_type = 'Datalogger Suitability Status'`, current record |
| `may_measure_water_level` | `permission_history.permission_allowed` where `permission_type = 'Water Level Sample'`, current record |
| `may_sample_water_chemistry` | `permission_history.permission_allowed` where `permission_type = 'Water Chemistry Sample'`, current record |
| `may_install_datalogger` | `permission_history.permission_allowed` where `permission_type = 'Datalogger Installation'`, current record |
| `permission_granted_by` | `contact.name` via `permission_history.contact_id` on the current water-level grant |
| `monitoring_frequency` | `monitoring_frequency_history.monitoring_frequency`, current record |
| `group_names` | `group.name` via `group_thing_association`, comma-joined |
| `group_types` | `group.group_type`, same order as `group_names` |
| `manual_water_level_count` | `count(observation)` via `sample` → `field_activity` → `field_event`, `activity_type = 'groundwater level'` *(stats)* |
| `manual_water_level_first_date` | `min(observation.observation_datetime)`, UTC date, same chain *(stats)* |
| `manual_water_level_last_date` | `max(observation.observation_datetime)`, UTC date, same chain *(stats)* |
| `days_since_manual_water_level` | `CURRENT_DATE - manual_water_level_last_date` |
| `last_depth_to_water_ft` | `observation.value - COALESCE(observation.measuring_point_height, 0)` on the latest reading *(stats)* |
| `chemistry_sample_count` | `count(DISTINCT sample.id)`, `activity_type = 'water chemistry'` *(stats)* |
| `chemistry_sample_last_date` | `max(sample.sample_date)`, UTC date, same filter *(stats)* |
| `days_since_chemistry_sample` | `CURRENT_DATE - chemistry_sample_last_date` |
| `field_event_count` | `count(field_event)` for the well *(stats)* |
| `date_last_visited` | `max(field_event.event_date)`, UTC date *(stats, column named `field_event_last_date` there)* |
| `has_datalogger` | `true` when a currently-installed deployment exists whose `sensor.sensor_type` is Data Logger / Pressure Transducer / DiverLink / Diver Cable. Stays logger-scoped even though the columns below do not |
| `datalogger_deployment_count` | `count(deployment)`, same logger-only filter as `has_datalogger` |
| `sensor_type` | `sensor.sensor_type` for every currently-installed deployment (`installation_date IS NOT NULL AND removal_date IS NULL`), **any sensor type, not just loggers** — semicolon-joined, ordered by `sensor_type` |
| `model` | `sensor.model`, same deployments, same order as `sensor_type`. `COALESCE(..., '')` before aggregating, so a null value is an empty segment, not a dropped position |
| `serial_no` | `sensor.serial_no`, same deployments, same order, same `COALESCE(..., '')` treatment |
| `sensor_status` | `sensor.sensor_status`, same deployments, same order, same `COALESCE(..., '')` treatment |
| `installed_date` | `deployment.installation_date`, same deployments, same order. In practice never null -- `installed_deployments` filters on `installation_date IS NOT NULL` -- but `COALESCE`d anyway for consistency with its siblings |
| `recording_interval` | `deployment.recording_interval`, same deployments, same order, same `COALESCE(..., '')` treatment — `text`, not `integer`, because `string_agg` produces `text` regardless of how many deployments a given well has |
| `recording_interval_units` | `deployment.recording_interval_units`, same deployments, same order, same `COALESCE(..., '')` treatment |
| `hanging_point_desc` | `deployment.hanging_point_description`, same deployments, same order, same `COALESCE(..., '')` treatment |
| `continuous_reading_count` | `count(transducer_observation)` via `deployment` *(stats)* |
| `continuous_first_datetime` | `min(transducer_observation.observation_datetime)` *(stats)* |
| `continuous_last_datetime` | `max(transducer_observation.observation_datetime)` *(stats)* |
| `days_since_continuous_reading` | `CURRENT_DATE - continuous_last_datetime::date` |
| `contact_count` | `count(DISTINCT contact)` via `thing_contact_association` |
| `primary_contact_name` | `contact.name`, `contact_type = 'Primary'` preferred, else lowest `contact.id` |
| `primary_contact_organization` | `contact.organization`, same contact |
| `primary_contact_role` | `contact.role`, same contact |
| `primary_contact_type` | `contact.contact_type`, same contact — says whether the row above is a real primary or a fallback |
| `primary_contact_phone` | `phone.phone_number`, lowest `phone.id` for that contact |
| `primary_contact_email` | `email.email`, lowest `email.id` for that contact |
| `contact_names` | `contact.name` for every associated contact, comma-joined |
| `access_notes` | `notes.content` where `note_type = 'Access'`, newest first, joined with ` | ` |
| `directions_notes` | `notes.content` where `note_type = 'Directions'`, newest first, joined with ` | ` |
| `communication_notes` | `notes.content` where `note_type = 'Communication'`, newest first, joined with ` | ` |
| `construction_notes` | `notes.content` where `note_type = 'Construction'`, newest first, joined with ` | ` |
| `maintenance_notes` | `notes.content` where `note_type = 'Maintenance'`, newest first, joined with ` | ` |
| `historical_notes` | `notes.content` where `note_type = 'Historical'`, newest first, joined with ` | ` |
| `general_notes` | `notes.content` where `note_type = 'General'`, newest first, joined with ` | ` |
| `water_notes` | `notes.content` where `note_type = 'Water'`, newest first, joined with ` | ` |
| `water_quality_notes` | `notes.content` where `note_type = 'Water Quality'`, newest first, joined with ` | ` |
| `sampling_procedure_notes` | `notes.content` where `note_type = 'Sampling Procedure'`, newest first, joined with ` | ` |
| `coordinate_notes` | `notes.content` where `note_type = 'Coordinate'`, newest first, joined with ` | ` |
| `owner_comment_notes` | `notes.content` where `note_type = 'OwnerComment'`, newest first, joined with ` | ` |
| `site_notes_legacy` | `notes.content` where `note_type = 'Site Notes (legacy)'`, newest first, joined with ` | `. Not `site_notes_legacy_notes` — the lexicon term already says "notes" |
| `point` | `location.point`, most recent association (PostGIS Point, EPSG:4326) |
Loading
Loading