Skip to content

Handle legacy MPHeight NaNs as NULLs (skip estimator) to ensure full well migration #541

Description

@ksmuczynski

Summary

Legacy WellData.MPHeight contains NaN values. Current transfer logic treats NaN as a real value, bypasses estimation, and then fails at DB insert because MeasuringPointHistory.measuring_point_height is Numeric. We want
migration completeness without using the estimator (it’s not trustworthy right now). Proposed fix: treat NaN as missing and insert NULL MP height for those wells, with nullable history during migration.

Why this matters

  • NaN values prevent some wells from transferring.
  • Estimator currently uses early measurements and may be inaccurate.
  • We need all wells migrated even if MP height is unknown.

Proposed changes

  • Transfer logic: treat NaN as missing and do not call estimator for those records.
  • Allow MeasuringPointHistory.measuring_point_height to be NULL during migration (migration already exists).
  • Read‑path hardening: ensure API/property access handles NULL history values.

Files

  • transfers/well_transfer.py
    • Detect NaN (pd.isna) and set MP height to None. Skip estimator for NaN cases.
  • db/thing.py
    • measuring_point_height and measuring_point_description should skip null history entries.
  • schemas/thing.py
    • Ensure response types allow measuring_point_height: float | None.
  • db/measuring_point_history.py
    • Confirm column nullable matches migration.
  • (Optional) transfers/util.py
    • Guard estimator against NaN if still used elsewhere.

Acceptance criteria

  • All legacy wells transfer even when MPHeight is NaN.
  • No estimator calls for NaN MP height in well transfer.
  • API responses tolerate null MP height without errors.
  • Tests updated or added to validate null MP height behavior.

Notes

  • Do not change column type to Float; keep Numeric for integrity.
  • Estimator improvements (using most recent DTW/DTW‑BGS) are out of scope for this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions