chore(lexicon): describe every lexicon category - #915
Open
jirhiker wants to merge 1 commit into
Open
Conversation
61 of the 62 seeded categories carried `description = NULL`; only `data_maturity` had one. The column has existed since the initial migration, so nothing needed a schema change -- the text was simply never written. Descriptions disambiguate the categories whose term lists overlap, which is where the vocabulary is easiest to misread. Grounded in the mapping comments on `db/data_provenance.py`: `collection_method` is the union of the legacy NM_Aquifer `LU_AltitudeMethod` and `LU_CoordinateMethod` lookups, while `elevation_method` and `coordinate_method` are those two lookups on their own; `origin_type` is likewise the union of `LU_DataSource` and `LU_Depth_CompletionSource`, which also exist separately. Also separates `casing_material` from `screen_type` (same terms, different well component) and `status` from the `status_type` / `status_value` pair used by `status_history`. `country`, `county` and `state` have no seeded terms, and say so rather than implying content. `release_status` deliberately does not mention `embargoed`; that term is not in this branch's lexicon and arrives with the access-control work. `init_lexicon` inserted only categories that were missing, so a database seeded before the descriptions existed would never receive them. It now upserts, wrapped in coalesce: a NULL description is filled from the seed and a description edited through /lexicon is left alone. That covers a fresh seed and a re-run of `oco initialize-lexicon`, but neither happens on deploy, so a data migration backfills the populated databases. It reads the same JSON, touches only NULL descriptions, and is gated on `66ac1af4ba69` -- the initial migration, which created the column. Pinning it to the current head would risk the runner silently skipping it on an environment whose alembic state lags. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Coverage✅ 79.39% total — gate is 75%. Coverage for the Python files changed in this PR
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
61 of the 62 seeded lexicon categories carried
description = NULL— onlydata_maturityhad one. This writes the missing text, and gets it into databases that are already seeded.lexicon_category.descriptionhas existed since the initial migration, so nothing here needs a schema change. The text was simply never written.The descriptions
Most categories are self-evident from their name. The ones worth care are those whose term lists overlap, where the vocabulary is easiest to misread. Grounded in the mapping comments on
db/data_provenance.py:collection_methodis the union of the legacy NM_AquiferLU_AltitudeMethodandLU_CoordinateMethodlookups.elevation_methodandcoordinate_methodare those two lookups on their own.origin_typeis likewise the union ofLU_DataSourceandLU_Depth_CompletionSource— both of which also exist as separate categories.casing_materialvsscreen_type: identical term sets, different well component.statusvs thestatus_type/status_valuepair used bystatus_history.country,county,statehave no seeded terms, and say so rather than implying content.release_statusdeliberately does not mentionembargoed— that term isn't in this branch's lexicon; it arrives with the access-control work and will need a follow-up line then.All 61 fit the
String(255)column.data_maturitywas left as-is.Getting it into existing databases
Two changes, because the JSON alone reaches nothing that's already seeded.
init_lexiconnow upserts. It previously inserted only categories that were missing, so a database seeded before the descriptions existed would never receive them. It now usesON CONFLICT DO UPDATEwrapped incoalesce: a NULL description is filled from the seed, and a description edited through/lexiconis left alone.A data migration for the populated databases, since neither a fresh seed nor
oco initialize-lexiconhappens on deploy. It reads the same JSON and touches only NULL descriptions.It is gated on
alembic_revision="66ac1af4ba69"— the initial migration, which created the column. This is deliberate: the runner skips any migration whose revision isn't applied, so pinning to the current head would risk it being silently skipped on an environment whose alembic state lags. The initial revision is applied everywhere.Testing
tests/test_data_migrations.py, following the file's existing pattern: a NULL description gets filled while a hand-edited one survives, and re-running is a no-op.test_data_migrations,test_data_migrations_cli,test_lexicon,test_migrations,test_cli_commands.run(): the alembic gate passed, descriptions backfilled, adata_migration_historyrow was written, and a second invocation logged "already applied" and returnedFalse. All 62 categories non-NULL afterward.black+flake8clean.Applying it
CD deliberately does not run data migrations — it runs
alembic upgrade headand only reports pending ones. So after this merges and deploys, apply it through the Data Migrations workflow (workflow_dispatch), per environment:action = statusagainststaging— confirms it is registered and unapplied.action = runwith20260901_0001_backfill_lexicon_category_descriptions.production.Safe to re-run: it only writes rows whose description is still NULL.
🤖 Generated with Claude Code