Skip to content

test(kubernetes): retain bounded dependency failure diagnostics - #254

Merged
yanpgwang merged 3 commits into
mainfrom
codex/kubernetes-failure-diagnostics
Oct 5, 2026
Merged

yanpgwang merged 3 commits into
mainfrom
codex/kubernetes-failure-diagnostics

Conversation

@yanpgwang

@yanpgwang yanpgwang commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Problem and result

The Kubernetes fixture previously reported downstream Temporal connection errors and a generic wait timeout without retaining the external dependency's exit/OOM facts before teardown. One main-branch recovery run exposed this gap. This change records fixed lifecycle fields for the fixture's source and restore services, including stopped containers, before disconnecting their network and removing their state.

The probe has a nine-second command budget and 250ms maximum inherited-pipe drain, keeping it below ten seconds so it cannot prevent cleanup. Raw inspect/error/healthcheck contents and unknown health strings stay private. An intentionally stopped source service remains distinguishable from a failed restore service.

Scope and unresolved condition

This improves failure diagnosis; it does not claim to fix the original restore failure. Integration37337738012 lost its restored Temporal endpoint after initially connecting. The same c56fc17 actual-artifact candidate passed on Linux AMD64, and an unchanged macOS ARM64 source journey passed in 232.20s with no observed OOM/nonzero exit. Those passes do not establish the original root cause.

No runtime, HTTP/OpenAPI, schema, SDK, timeout, resource or release-support change. This is an independently testable contributor diagnostic slice; no hosted credentials or new dependency are needed.

Acceptance and validation

  • Independent raw Docker-State regression observed RED before the helper; race/offline tests and incremental lint passed. It checks private Error/Health.Log/unknown fields, absent Health, unrecognized health text and invalid JSON.
  • A deliberate SIGKILL of this test's own restored Temporal exercises the failure callback and checks retained exit_code=137, oom_killed=false, followed by removal of its containers, volumes, network and kind cluster. The injection passed: the normal 90-second wait failed as expected, lifecycle facts were retained, and all owned resources were removed. This deliberately induced failure is not evidence of the original failure's cause.
  • A reviewer identified inherited Compose output pipes that could outlive a cancelled Docker parent. An independently authored child-process regression failed with cancellation alone; bounded WaitDelay now releases those pipes, and the test cleans its owned child. Final race tests passed; both independent reviewers approved the fix with no remaining findings.
  • Both independent subagent reviewers approved final f91e28db2ea5dfc2c97e0c19689a1afac2da5dc3 with no remaining findings; all 12 required CI checks passed at that head, including the source Kubernetes journey. After merge, the final source must also pass manual actual-artifact candidate acceptance before publication. Alpha publication uses only the later reviewed merged revision; original failure evidence remains preserved.

@yanpgwang
yanpgwang merged commit 6f363ba into main Oct 5, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant