ci: block releases whose action.yml cannot be loaded - #7
Open
SafeEval wants to merge 1 commit into
Open
Conversation
v1.0.0 and, until recently, v1 both named an action.yml that did not parse, so the action could not be loaded by anyone referencing them. The release job gained a YAML parse check, but two gaps remained: the immutable tag was pushed by the version job before anything validated it, and parsing alone does not prove an action is usable — a file can be valid YAML and still lack runs:, or carry a run: step with no shell:, either of which makes GitHub reject the whole action. Add validate-action.py, which checks the structure GitHub actually requires, and run it at three points: in CI on every PR, in the version job before the tag is pushed, and in the release job before the major alias moves. A tag is public and immutable the moment it exists, so validating after the push would be too late — v1.0.0 will always name a broken action. Tests cover 12 cases in both directions, including the exact dedented-heredoc shape that shipped as v1.0.0 and the YAML-valid-but-unloadable forms a parse check misses. They run in CI, since a guardrail that cannot fail is not one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VSrrciEDBTuFNMtKocScML
SafeEval
marked this pull request as ready for review
July 30, 2026 20:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First: the reported
v1issue is already fixedThe other session's follow-up said
v1still points at84abbefwith the brokenaction.yml. That was true when it was written, but is stale now. Verified:v1was moved by hand earlier, and the release workflow now moves it automatically on every release.@v1resolves to working code. No action needed there.The real gap: two holes in the existing guard
The release job had a YAML parse check, but:
1. The immutable tag was pushed before anything validated it. The
versionjob tagged and pushed with no check; only the alias move was guarded. A tag is public and permanent the instant it exists — validating afterwards can't recall it. That's whyv1.0.0will always name a broken action.2. Parsing isn't loadability. A file can be valid YAML and still be rejected by GitHub. All of these parse fine and are unusable:
The guardrail
.github/validate-action.pychecks the structure GitHub actually requires —name/description/runs,runs.using, non-empty composite steps, every step havinguses:orrun:,run:steps havingshell:, and inputs being well-formed mappings with descriptions. It reports every problem, not just the first.Wired at three points, each strictly before a tag write:
action.ymlreachingmainversionjob, beforegit push origin $nextreleasejob, before the alias moves@v1ever pointing at something unloadableI verified the ordering programmatically rather than by reading — validation precedes the tag push and the alias move in both jobs.
Tests
12 cases, both directions, run in CI — a guardrail that can't fail isn't one:
v1.0.0, plus missingruns:,run:withoutshell:, empty steps, steps with neitheruses:norrun:,runs:withoutusing:, inputs missing descriptions, non-mapping documents, and unreadable filesuses:stepConfirmed against the real artifacts: current
action.ymlpasses,v1.0.0's fails withnot valid YAML: mapping values are not allowed here.actionlintclean; all three YAML files parse.Note
This is a
ci:commit, so merging deliberately cuts no release — the guardrail lands without a version bump. The nextfix:/feat:will be the first release to pass through it.🤖 Generated with Claude Code
https://claude.ai/code/session_01VSrrciEDBTuFNMtKocScML