Skip to content

[G17] Qualify crash recovery, soak and resource budgets #17

Description

@jjangg96

Goal

Establish release-candidate reliability and measured resource limits through deterministic faults and a real mixed-workload soak.

Create one active Codex goal from the statement above when this issue is dispatched. The Project Goal field is a work specification; it does not start an agent. Do not invent a token budget.

Execution contract

Field Value
Goal key G17
Stage M3 - Reliability qualification
Initial status Backlog
Primary agent gpt-5.6-luna / max
Priority / risk P0 / High
Test profiles offline, trusted-runtime, trusted-live-github

Use one issue branch/worktree and one focused PR. Independent Luna max review is required for authentication, protocol, concurrency, resource ownership, cleanup or service identity boundaries; other changes need independent contract review. Model fields are routing instructions, not GitHub user assignments.

Dependencies

Dependencies must be Done before implementation begins. A new issue is not blocked simply because its future evidence has not been collected.

Scope

24-hour/100-job proposed release gate, fault matrix, performance baseline and decision report. Do not promote if required evidence is missing.

TDD and failure evidence

  1. Kill daemon at all identified barriers, interrupt network/runtime, rotate tokens and simulate disk pressure.
  2. Run queue bursts and repeated drain/scale transitions with both orgs and OS pools.
  3. Measure manager CPU/RSS, API traffic, admission/ready latency, queue time and disk growth on fixed hardware.
  4. Control host reboot and classify expected interrupted jobs separately from manager recovery faults.
  5. Simulate stale runner/image and urgent required update during busy work; prove paused new admission, safe refresh/drain and no rollback to forbidden stale version.

Capture a meaningful failing case before the implementation, then green evidence and relevant refactor checks. Tooling/prose-only work uses appropriate negative checks without artificial application tests. Live/runtime profiles require reviewed commits, a dedicated trusted test environment and explicit authorization for the concrete experiment. Public PR CI uses hosted environments without credentials. Planned or skipped tests never count as passed.

Acceptance criteria

  • No unexplained duplicates/orphans/cap violations/busy kills/secret exposure.
  • Set recommended counts from measured headroom, not host marketing specs.
  • Document SDK preview limits and residual trust/platform limitations.
  • Independent Luna max reliability verdict; incomplete soak remains incomplete.
  • Record exact validation commands, actual results, skipped/live-test gaps and applicable rollback notes in the PR.
  • Independent review is resolved and the focused PR is merged under the repository execution policy.
  • Update issue/Project accurately; mark the active goal complete only after all required evidence and work are complete.

Safety invariants

Preserve existing manual runners; no global Docker prune/context switching, broad process kill, implicit App enrollment, busy-job cancellation during ordinary scale-down or transparent workflow replay. Use only verifiably owned resources. Keep management credentials and raw secret-bearing SDK errors out of worker environments, logs, fixtures and commits; per-worker JIT transport follows G01. Native pools remain trusted-only.

Design references

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:astra-xhighPrimary implementer: gpt-6-astra, xhigh reasoningpriority:P0Critical contract, security or reliability dependencyrelease:distributionApproved delivery sequencing; does not change acceptance or dependency gatesrisk:highIndependent Astra review on risky boundariestype:gateEvidence gate before dependent work

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions