Skip to content

[PR-007] Validate the release candidate #7

Description

@ilyar

Validate the release candidate

Outcome

The release candidate has independent functional, security, browser, and public endpoint evidence.

Scope

  • Product: product/proof-runner
  • Allowed paths: tests, QA evidence, release and demo documentation

Acceptance

  • Run 10 broken and 10 fixed demo verifications with expected verdicts.
  • Test invalid host, redirect, missing lockfile, timeout, oversized input, oversized output, full queue, idempotent replay, cleanup, restart, and modified receipt.
  • Verify postinstall never executes, install egress is allowlisted, and build/test have no network.
  • Verify API-runner authentication, lease expiry, cancellation, and incompatible-contract handling.
  • Confirm timeout, registry failure, and runner failure produce INCONCLUSIVE rather than FAIL.
  • Check Chrome, Firefox, and mobile viewport.
  • Verify keyboard flow, visible focus, WCAG AA contrast, LCP below 2.5 seconds, and CLS below 0.1.
  • Verify /skill.md, /llms.txt, PASS/FAIL examples, receipt JSON, and Open Graph previews from a clean external machine.
  • Probe public health, inspect, verify, receipt, and result URLs from a clean environment.
  • Confirm responses, logs, screenshots, and artifacts contain no secrets.
  • Record all commands, run IDs, timings, and screenshots.
  • Record ten target-worker timings and confirm p95 is at most 90 seconds.
  • Reviewer returns APPROVE; then committer creates the release commit.

Constraints

  • Andrei (@Glazlk) is the accountable QA owner and does not implement the primary API, runner, receipt, or web code under test. Ilyar (@ilyar) and Marks (@ppmarkek) provide their implementation evidence but do not accept their own work.
  • Tester and reviewer must be independent from the implementing agent.
  • Do not mark done from narrative progress alone.

Evidence

  • Commit: pending
  • Checks: pending
  • Review: pending

Blocker

Source task: https://github.com/ever-guild/proof-runner-agent/blob/main/tasks/pr-007-release-qa.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions