Skip to content

Make pre-merge evidence atomic - #1555

Open
Chris0Jeky wants to merge 3 commits into
mainfrom
issue-1547/atomic-evidence
Open

Make pre-merge evidence atomic#1555
Chris0Jeky wants to merge 3 commits into
mainfrom
issue-1547/atomic-evidence

Conversation

@Chris0Jeky

@Chris0Jeky Chris0Jeky commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • adds a two-phase pre-merge evidence collector that binds an explicit PR or the current branch to a clean exact head/base before checks, then revalidates the closing identity
  • captures cursor-complete review threads/comments, top-level comments, review summaries, and CI states twice; normalized drift invalidates the packet
  • emits Secrets scan: CLEAN only for a stable successful Secret Scan / Gitleaks Scan from the completed CI Actions run at .github/workflows/ci-required.yml, bound to the exact PR head/base
  • routes contributor and Codex worker handoffs to the canonical review-and-ship pipeline instead of a local self-review workflow

Parked disposition

The automatic Codex review arrived after the bounded two review/fix rounds and identified two confirmed HIGH blockers. This PR is parked with those canonical threads unresolved; do not merge this head.

  • the skill's separate executable blocks lose the opening state and PR-number variables, so the documented start/diff/finish sequence cannot complete in ordinary separate shell calls
  • the CLEAN verdict binds run metadata but not the PR-controlled caller/reusable workflow and scan configuration to trusted exact-base bytes

Successor acceptance criteria and the non-blocking portability follow-ups are recorded in #1547. The contributor-onramp P2 is recorded in #1544.

Verification

  • Git-for-Windows Bash syntax for both collector scripts
  • mocked collector contract: 11/11 canaries passed
  • live read-only GraphQL pagination probes exercised outer review-thread and nested comment cursors
  • node scripts/check-docs-governance.mjs
  • node scripts/check-golden-principles.mjs
  • node scripts/check-github-ops-governance.mjs
  • static Taskdeck hook-absence invariant, TOML parse, and Claude skill frontmatter/body parse
  • git diff --check origin/main...HEAD
  • DCO sign-offs: 3/3 commits
  • hosted exact-head Required CI / CI Extended / CodeQL: 26 pass, 11 intentional skips, no pending/failure/cancellation
  • bounded independent review completed; late automatic review superseded the earlier clean checkpoint and left two confirmed HIGH blockers
  • generic Codex quick_validate.py for the Claude skill — NOT VERIFIED because that validator rejects the pre-existing Claude-supported user-invocable field before body validation
  • product backend/frontend/Playwright suites not run: no product runtime or UI seam changed

Documentation

  • docs/TESTING_GUIDE.md documents the focused syntax/canary commands
  • contributor and Codex autonomy routing now point to the canonical review pipeline
  • docs/STATUS.md / docs/IMPLEMENTATION_MASTERPLAN.md not changed: shipped product behavior and roadmap priority did not change

Tracking

CI Workflow Validation

  • CI Extended passed because this PR changes scripts/**

Risk Notes

Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>
Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>
Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 87530636a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cleanup_pre_merge_state() {
rm -f "$evidence_state"
}
trap cleanup_pre_merge_state EXIT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the evidence state across skill steps

In ordinary Claude Code execution, this Step 1 Bash block ends before the separately documented test and diff-inspection phases, so its EXIT trap immediately deletes evidence_state and the shell also loses evidence_state and pr_number. Consequently Step 3 cannot expand pr_number, and Step 4's finish "$evidence_state" cannot reach the opening snapshot; the gate needs to persist the state path across tool calls or execute both collector phases inside one actual shell invocation rather than merely instructing that separate steps share a session.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed HIGH: the documented fenced steps do not actually share shell variables, and Step 1's EXIT trap deletes the opening state before later calls. The bounded two review/fix rounds are exhausted, so PR #1555 is parked and this canonical thread remains unresolved. Successor acceptance criteria are recorded in #1547: #1547 (comment)

Comment thread .claude/skills/pre-merge-gate/SKILL.md
--arg baseRef "$opening_base_ref" '
.id == $runId and
.name == $workflowName and
.path == $workflowPath and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Verify the scan definition before emitting CLEAN

When a PR changes .github/workflows/ci-required.yml or its local reusable-gitleaks.yml, the pull-request run executes that PR-controlled definition while still reporting the same workflow name and .path. A no-op replacement that preserves CI and Secret Scan / Gitleaks Scan therefore satisfies this metadata predicate and emits Secrets scan: CLEAN; bind the attestation to a trusted base revision of the caller and reusable workflow, or downgrade the verdict whenever the scan definition differs.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed HIGH. Live run 30705275967 resolved the reusable workflow from the PR merge ref, so run name/path/head/base metadata does not prove that the PR-controlled scan definition remained trusted. The two review/fix rounds are exhausted; PR #1555 is parked, this thread remains unresolved, and exact-base definition binding plus false-clean canaries are tracked in #1547: #1547 (comment)

Comment thread .claude/skills/pre-merge-gate/SKILL.md
Comment thread scripts/github/collect-pre-merge-evidence.sh
Comment thread CONTRIBUTING.md
Comment thread scripts/github/collect-pre-merge-evidence.sh

@Chris0Jeky Chris0Jeky left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review triage reply.

cleanup_pre_merge_state() {
rm -f "$evidence_state"
}
trap cleanup_pre_merge_state EXIT

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed HIGH: the documented fenced steps do not actually share shell variables, and Step 1's EXIT trap deletes the opening state before later calls. The bounded two review/fix rounds are exhausted, so PR #1555 is parked and this canonical thread remains unresolved. Successor acceptance criteria are recorded in #1547: #1547 (comment)

@Chris0Jeky Chris0Jeky left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate triage reply.

Comment thread .claude/skills/pre-merge-gate/SKILL.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Blocked

1 participant