fix(gp): bless live response oracles for 9 unblessed scripts + gate future regrowth - #229
Open
mikemcdougall wants to merge 2 commits into
Open
fix(gp): bless live response oracles for 9 unblessed scripts + gate future regrowth#229mikemcdougall wants to merge 2 commits into
mikemcdougall wants to merge 2 commits into
Conversation
…seed revision The gp-eval live smoke job defaulted to the moving `nightly-aot` tag and checked out the honua-server seed fixtures with no `ref:` (default-branch HEAD at run time), so a commit's verdict here depended on whatever honua-server merged that day and could flip red/green across re-runs with no change in this repo. Pin both halves as one identity, reusing the already-reviewed baseline conformance.yml pins (honua-server@sha256:78e3088... / f897700...) instead of promoting a second independently-moving baseline. Confirmed live: today's nightly-aot (sha256:cd12f3c...) actually 500s against the static seed SQL -- honua-server added a migration-owned-table journal guard (df25cba60, 2026-09-02) the seed does not satisfy -- while the pinned 2026-08-19 baseline starts cleanly and the full eval suite passes 50/50 against it (docs/golden-eval.md walks the exact repro). Closes #201
…uture regrowth Live smoke graded no response oracle for 9 of 25 supported honua-gp eval scripts (Describe x2, ListFields x3, InsertCursor, UpdateCursor x2), so a response-parsing regression in schema introspection or write-cursor round trips would pass the lane silently as an "unblessed" pass. Blessing the two write-cursor scripts against a fresh seed surfaced a real bug rather than a missing oracle: honua_gp.da._values_for_row/_extract_oid only recognized a feature's .attributes, but the live SDK's honua_sdk.models.QueryFeature exposes GeoJSON-shaped .properties/.id instead -- SearchCursor/UpdateCursor silently returned None for every field (OID included) against a real server. Stub-mode CI never caught this because the eval stub's _StubFeature happens to have .attributes. Fixed both helpers to read the real feature shape (preferring the SDK's already-resolved .id for OID@, since the seed's object-id field is lower-case objectid and never matched the existing OBJECTID/oid/FID key list), added regression tests against the actual QueryFeature type, and re-blessed against a fresh stack -- update_cursor_close_status now genuinely proves an update round-trip (update_count: 1) instead of masking the bug behind a vacuous zero. Describe/ListFields (schema fetch) and InsertCursor were already correct; those got a schema/apply-edits fingerprint oracle. MakeTableView makes no server round trip at all (a client-side session alias registration) -- no response to fingerprint, so it goes in a new UNBLESSED_ALLOWLIST.json with the reason instead of a silent pass. run_eval.py now fails a live-mode supported script that has no response oracle and no UNBLESSED_ALLOWLIST.json entry, so this set cannot regrow unnoticed (verified: an emptied allow-list correctly fails make_table_view.py with the new reason). Closes #202
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
mikemcdougall
had a problem deploying
to
staging
September 10, 2026 21:39 — with
GitHub Actions
Failure
|
You have reached your Codex usage limits for security reviews. Please try again later. |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #228 (needed so
ephemeral-server-smokeis green against a working baseline instead of the currently-brokennightly-aotpin) — the diff will shrink to just this PR's commit once #228 merges and this branch is rebased.Problem
ephemeral-server-smokegraded no response oracle for 9 of the 25 supportedhonua-gpeval scripts (describe_segments[_fields],list_fields_segments[_filtered|_wildcard],insert_cursor_append_rows,update_cursor_close_status,update_cursor_delete_closed,make_table_view). An un-oracled supported script graded as a silent "unblessed" pass, so a response-parsing regression in schema introspection (Describe/ListFields) or the write-cursor round trip (InsertCursor/UpdateCursor) could sail through this lane undetected.What I found while blessing them
Blessing the two write-cursor scripts against a fresh seed didn't just fill in a missing value — it surfaced a real bug.
honua_gp.da._values_for_row/_extract_oidonly recognized a feature's.attributes, but the live SDK'shonua_sdk.models.QueryFeature(whatSource.iter_featuresactually yields against a real server) exposes GeoJSON-shaped.properties/.idinstead — there is no.attributes. Every liveSearchCursor/UpdateCursorfield read (OID included) silently degraded toNone. Stub-mode CI never caught this because the eval stub's_StubFeaturehappens to define.attributes.Fixed both helpers to read the real feature shape, preferring the SDK's already-resolved
.idforOID@(the seed's object-id field is the lower-caseobjectid, which never matched the existingOBJECTID/oid/FIDkey list either). Added two regression tests against the actualQueryFeaturetype (verified they fail against the pre-fix code, restored, verified they pass with the fix). Re-blessed against a fresh stack:update_cursor_close_statusnow genuinely proves an update round-trip (update_count: 1) instead of masking the bug behind a vacuousupdate_count: 0.Describe/ListFields(schema fetch, a different code path) andInsertCursor(write-only) were already correct — those simply needed a schema / apply-edits fingerprint oracle.Non-capturable case:
make_table_viewMakeTableView/MakeFeatureLayerresolve tohonua_gp.management._make_layer_handler, which only registers a client-side session alias — no honua-server HTTP round trip at all. There's no response to fingerprint. Documented in a neweval/UNBLESSED_ALLOWLIST.jsonwith the reason, per the issue's acceptance criteria ("say so explicitly rather than leaving it implicit").Anti-regrowth gate
run_eval.pynow fails a live-mode supported script that has no response oracle and noUNBLESSED_ALLOWLIST.jsonentry, instead of passing it as "unblessed". Verified by emptying the allow-list and confirmingmake_table_view.pyfails with the expected reason, then restoring it.Acceptance criteria (from #202)
UNBLESSED_ALLOWLIST.json+ a gate that fails a new/unblessed supported script.Test plan
python3 -m pytest packages/honua-gp/tests -q: 197 passed (added regression tests confirmed to fail pre-fix, pass post-fix).python3 -m honua_gp._cli matrix --check(no drift).run_eval.py: 50/50.run_eval.py(HONUA_GP_EVAL_USE_STUB=0) against three independent fresh seeded stacks: 50/50 each, confirming the blessed oracles are deterministic.make_table_view.pywhen its allow-list entry is removed, with the documented reason.Closes #202