Skip to content

ci: audit the signed tip of every default branch - #19

Closed
jdwillmsen wants to merge 1 commit into
mainfrom
chore/verify-main-signatures
Closed

ci: audit the signed tip of every default branch#19
jdwillmsen wants to merge 1 commit into
mainfrom
chore/verify-main-signatures

Conversation

@jdwillmsen

Copy link
Copy Markdown
Member

Relates to JDWLABS-230.

What

Adds .github/workflows/verify-main-signatures.yml — a daily (plus manual) job that reads the tip commit of main in apps, platform, infrastructure, deployments and .github through the API and fails if any of them is unsigned. No checkout, no secret: all five repos are public, so GITHUB_TOKEN can read their commits. Documented in the workflows README.

Why

required_signatures is active on main in three of the five repos and has been reporting as enforced for three weeks while nothing landed verified. This closes the detection half of that ticket only — it does not fix the mechanism.

Root cause this makes visible

The signature is present before the merge and destroyed by it. Same change, platform PR #167:

$ gh api repos/jdwlabs/platform/pulls/167/commits --jq '...'
dbdce291 verified=true  reason=valid    committer=Jake Willmsen   <- on the PR branch

$ gh api "repos/jdwlabs/platform/commits?sha=main&per_page=1" --jq '...'
bb20b3f1 verified=false reason=unsigned committer=Jake Willmsen   <- same change on main

All four code repos are allow_rebase_merge=true with merge-commit and squash disabled, so every merge takes the path that re-creates commits server-side without re-signing them. Local signing is not the problem — commit.gpgsign=true and key 949C342C7907CC24 in every clone, and every commit that ever verified on main arrived by direct push (6a02f89c, 7d6f29e1, a33d1ae9, c903346d, all NOPR), not through a merge.

Verification

actionlint clean:

$ actionlint .github/workflows/verify-main-signatures.yml
actionlint: exit 0 clean

Script body dry-run against the live API from a workstation, unchanged from what the job runs:

apps             99df4f41 verified=false reason=unsigned
platform         bb20b3f1 verified=false reason=unsigned
infrastructure   d7759228 verified=false reason=unsigned
deployments      20865a70 verified=false reason=unsigned
.github          87437b45 verified=false reason=unsigned

5 of 5 default branches have an unsigned tip commit.
EXIT=1

Reviewer note — this lands red

5 of 5 tips are unsigned right now, so the first scheduled run fails. That is the finding, not a bug in the check: it stays red until the merge method changes (enabling squash merge, which GitHub signs, or dropping rebase-merge) or the org decides the unsigned range is acceptable. Merge it now and the red run tracks the open decision; hold it and the decision comes first. Either is defensible — but re-signing history without changing the mechanism will erode again within a day.

🤖 Generated with Claude Code

@jdwillmsen

Copy link
Copy Markdown
Member Author

Holding this open deliberately rather than merging.

The workflow is correct — I ran its check by hand against all five default branches just now:

apps           d41e3bea verified=false reason=unsigned
platform       e146e3fe verified=false reason=unsigned
infrastructure f98621a9 verified=false reason=unsigned
deployments    4ecd8f67 verified=false reason=unsigned
.github        d73af58c verified=false reason=unsigned

5 of 5 unsigned, so merging this today buys a cron job that is red on its first run and every run after until the merge mechanism itself is fixed. A permanently-failing daily audit stops being read within a week, and then it cannot do the job it was written for.

Gate: merge once JDWLABS-230 is resolved and the tips actually verify, so the first run is green and any red afterwards is real news.

@jdwillmsen
jdwillmsen marked this pull request as draft July 31, 2026 21:10
@jdwillmsen

jdwillmsen commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Recommendation: close this, and file the replacement fresh rather than repurposing the branch

Not merging or closing it here — this is the reasoning, the call is the maintainer's.

The replacement control is open as #25 (plus four caller PRs), which enforces signatures on the pull request branch and lets required_signatures come off main.

Why auditing main tips for signatures stops measuring anything

  1. It measures a design property, not a regression. Under rebase-merge every main tip produced by a merge is unsigned by construction. Once that is the accepted, documented outcome rather than a defect, a job that goes red for it is reporting a constant.

  2. The signal inverts. The evidence already gathered on this ticket is that every commit that ever verified on main in these repos arrived by direct push, never through a merge. After the decision, a verified main tip is the anomaly worth investigating — it means something bypassed the pull request path. A check that fails on unsigned would pass on exactly the case that now warrants a look.

  3. It flags sanctioned automation as a fault. The apps release job and the deployments promotion push straight to main as github-actions[bot], unsigned, on their own schedule. The tip alternates between merge results and bot pushes depending on what ran last, so the audit would flap red on a cadence nobody can predict — the "permanently-failing job stops being read" failure this PR's own reviewer note is trying to avoid.

What would measure a real control

The gap the branch check genuinely cannot cover is anything reaching main without passing through a pull request — which is precisely what the OrganizationAdmin: always and Integration: always ruleset bypasses make possible today, and it is measurable from the API: for each recent commit on main, does GET /repos/{owner}/{repo}/commits/{sha}/pulls return an associated pull request?

That is not hypothetical. Sampled the recent history of main in all five repos:

apps            11 of 60 commits have no associated pull request
platform         0 of 60
infrastructure   0 of 60
deployments      8 of 60
.github          8 of 31

A daily job over those numbers is reporting live bypass usage, and it starts from a real, non-zero, repo-varying baseline the maintainer can decide to drive down or accept explicitly — unlike a signature audit whose answer is a constant.

Note that the org audit log is not an available substitute here: GET /orgs/jdwlabs/audit-log returns 404 on this plan, so bypass usage cannot be read from it programmatically. The commits-without-a-pull-request query is the practical mechanism.

Why fresh, not repurposed in place

The file name (verify-main-signatures.yml), the check body, the README entry and this PR's whole narrative are about signatures on main. Rewriting all of it on this branch produces a pull request whose review history argues for a premise that was rejected, and whose diff no longer matches its own title. A new pull request is cheaper to review, and closing this one leaves its evidence intact as the record of why the signature-on-main approach was abandoned — which is worth keeping.

The required_signatures rule has been active on main in three repositories the
whole time nothing has landed verified. A signature that verifies on the pull
request branch is gone from main because rebase merging re-creates each commit
server-side without re-signing it, and no gate notices: the rule reports as
enforced, no merge is rejected, and the only way the lapse surfaced was querying
commit verification by hand sixteen days later.

This job reads the tip of main in all five repositories daily and fails when any
of them is unsigned, so the next lapse is a red run rather than an archaeology
exercise. It will fail on its first run — 5 of 5 tips are unsigned today — which
is the state it exists to report, not a defect in the check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jdwillmsen
jdwillmsen force-pushed the chore/verify-main-signatures branch from 6bc5de2 to c520c21 Compare August 1, 2026 20:43
@jdwillmsen

Copy link
Copy Markdown
Member Author

Closing: the premise this workflow tests no longer holds.

This job fails when the tip of main is unsigned. As of JDWLABS-230 (merged today) those tips are unsigned by designrequired_signatures was removed from main in all five repos because it is unsatisfiable under rebase-only merges, and signature enforcement moved to the PR branch via verify-pr-signatures.yml, now a required check (signatures / signatures) in all five.

Merging this as-is would buy a job that goes red every single day, forever, reporting a condition that is the intended state. That is the same 'control reports a problem that is not one' shape the ticket set out to remove.

The detection gap this was reaching for is real but different: commits that reach main with no PR at all are still ungated. That is JDWLABS-210's scope, and a successor check belongs there — 'every commit on main is PR-attributable', not 'every commit on main is signed'. Recorded on that ticket.

Reopenable if the premise changes: if GitHub ever ships rebase-merge signing, the original conflict dissolves and this workflow becomes valid as written.

@jdwillmsen jdwillmsen closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant