Skip to content

feat(scripts): script the release notes and Release creation (release:notes) - #2591

Merged
cliffhall merged 5 commits into
v2/mainfrom
v2/chore/2550-script-release-notes
Oct 5, 2026
Merged

cliffhall merged 5 commits into
v2/mainfrom
v2/chore/2550-script-release-notes

Conversation

@cliffhall

Copy link
Copy Markdown
Member

Closes #2550

What changed

scripts/release-notes.mjs (npm run release:notes) replaces the release skill's step 3a shell recipe. It assembles the four parts of a release's notes and creates the Release:

  1. What's Changed from POST releases/generate-notes, from the previous stable tag to main. It skips -rc.N, -hotfix, -amended and v2-alpha-1 tags.
  2. The smoke-ledger line: **Smoke test ledger for milestone branch**: [<branch>](<url>).
  3. ## Known issue / ## Known issues, from repeatable --known-issue "<markdown>" arguments. This part stays a maintainer judgment.
  4. ## Thanks for helping us improve: the authors of every issue a listed PR closes, from closingIssuesReferences (paginated with pageInfo/endCursor) plus closing keywords in the PR body. Maintainers (admin/maintain/write), bots, deleted accounts, PR numbers and other repos' issues are excluded. Each person gets one line, most issues first, and the section is omitted when nobody remains.

Modes. A default run is a preview that prints the notes and creates nothing. --draft creates a draft Release, and --publish creates it and publishes it as latest. Both use the bare x.y.z tag with --target main, and both refuse any version other than origin/main's. --version/--previous-tag regenerate an older release's notes, in preview only.

Fail fast. This follows the two requirements from #2555's review recorded on the issue. Every gh/git call is checked, and any non-zero exit, GraphQL errors, missing node, or permission value outside the known set throws. A failed permission lookup can never read as "community", and a rate-limited lookup can never drop a PR or issue. Either way the run aborts before anything is created.

Release skill. Step 3a now uses the helper: preview, then --draft. Step 3b is publishing the draft as the deliberate maintainer action. The re-cut step and the skill description are updated to match, and release:notes is added to the scripts/ entries in the README, AGENTS.md and project-structure trees.

Verification

  • Dry run against 2.9.0 reproduces its published notes. I ran node scripts/release-notes.mjs --version 2.9.0 --merge-branch v2/chore/milestone-merge-v2.9.0 --ledger-url https://claude.ai/artifact/3cWteCtohesQrCSgujccvn --known-issue "<2.9.0's known-issue paragraph>", which reported 2.8.0 → 2.9.0 and 44 PRs. Its output is byte-identical to the published 2.9.0 body once the CRLF line endings and trailing blank lines the UI editor adds are normalized, including all six Thanks lines in order.
  • scripts/release-notes.test.mjs: 23 tests covering the issue-author mapping and every exclusion, keyword parsing, pagination across three pages, permission caching, the note layout, a failed lookup at each stage aborting with nothing created, an unknown permission aborting, and the exact gh release create argv for --draft and --publish.
  • npm run local:gate: everything up to coverage:web passed. That covers local:validate (all 914 test:scripts tests, format, lint, typecheck, every verify:* guard) and verify:skills:cli. coverage:web then failed on a single test, the already-tracked flake Flaky integration test: inspectorClient-timeout-diagnostics 'emits connectionDiagnosticsChange' reads an undefined GET log entry #2580 (inspectorClient-timeout-diagnostics › "emits connectionDiagnosticsChange…"; 8,782 of 8,783 passed). It failed 1 in 4 to 1 in 6 runs when I ran it alone, and this diff doesn't touch it. Because the chain stops at the first red stage, the stages after coverage:web (cli/tui/launcher coverage, build-gate, bundle-externals, smokes, Storybook) did not run locally. Nothing in this diff reaches them (a standalone scripts/ helper plus docs), and CI runs them all.
    • The run had to go without NODE_USE_ENV_PROXY: in this sandbox it sends Node's fetch to the integration tests' local servers through the egress proxy, which returned 502 and failed 18 unrelated tests.

🤖 Generated with Claude Code

…:notes)

scripts/release-notes.mjs replaces the release skill's step 3a shell
recipe. It generates the What's Changed list with releases/generate-notes
(previous stable tag to main), appends the smoke-ledger line, any known
issues the maintainer passes in, and a "Thanks for helping us improve"
section crediting the community authors of every issue the listed PRs
close (paginated closingIssuesReferences plus closing keywords in the
body), excluding maintainers by permission and bots.

Every gh/git call is checked and any failure aborts, so a failed
permission lookup can never credit a maintainer and a rate limit can
never publish a partial Thanks list. The default run is a preview;
--draft creates a draft Release and --publish publishes, both with the
bare x.y.z tag and --target main, and both refuse any version other
than origin/main's. A preview of 2.9.0 reproduces its published notes.

Closes #2550

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Publishing currently accepts the preview-only --previous-tag override, allowing incorrectly scoped release notes.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds a tested release-notes workflow that generates notes, credits reporters, and creates draft or published GitHub Releases.

Changes:

  • Adds release-note assembly and release creation.
  • Adds comprehensive root-script tests.
  • Updates release procedures and command documentation.
File Description
scripts/​release-notes.mjs Implements note generation and Release creation.
scripts/​release-notes.test.mjs Tests formatting, API handling, and release modes.
package.json Adds release:notes.
.claude/​skills/​release/​SKILL.md Replaces the manual recipe with the helper.
.claude/​skills/​project-structure/​SKILL.md Documents the new helper.
README.md Lists the new command.
AGENTS.md Updates the repository structure reference.

Comment thread scripts/release-notes.mjs
A created Release always starts from the previous stable tag; a typo or
stale override in --draft/--publish would ship the wrong range of
changes and reporter credits. Make it preview-only, like --version,
and say so in the release skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 1: one finding, fixed in f792cb2.

  • Reject --previous-tag outside preview mode: done. --draft/--publish now refuse --previous-tag, with test coverage for both modes, and the release skill is updated to match. Re-checked with validate:guards + validate:core (all 914 script tests, format, lint, every verify:* guard).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The release skill contains an invalid shell command because its ledger URL placeholder is unquoted.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Low severity Quote ledger artifact URL to prevent shell redirection

.claude/​skills/​release/​SKILL.md:232

This command is not valid shell: the unquoted <ledger artifact URL> is parsed as redirection, so copying the documented release procedure fails before release:notes runs. Quote the placeholder (or use a quoted example URL).

An unquoted <ledger artifact URL> parses as a shell redirection, so the
documented command failed when copied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 2: one finding, in the review body's Previously missed block (no thread to reply into), fixed in 637f88b.

  • Quote ledger artifact URL to prevent shell redirection (.claude/skills/release/SKILL.md:232): done. Both placeholders in the ARGS=(…) line are now quoted, and the URL placeholder is shaped like a real artifact link ("https://claude.ai/artifact/<id>"). Checked with bash -n.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The raw Markdown keyword scan can incorrectly credit issues mentioned only in code examples or blockquotes.

Review effort: Balanced
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Medium severity Ignore closing keywords in Markdown code and blockquotes

scripts/​release-notes.mjs:161

This scans raw Markdown, so closing-keyword examples inside fenced/inline code or blockquotes are treated as real closures even though GitHub ignores keywords in those contexts. For example, a PR documenting `Closes #41` causes issue #41's author to be credited. Mask or parse non-prose Markdown before applying the keyword regex, and add cases for code fences, inline code, and quotes.

GitHub does not treat a closing keyword inside an HTML comment, fenced
or inline code, or a blockquote as a closure, so neither should the
reporter credit: a PR documenting `Closes #41`, or a template's leftover
<!-- Closes #… --> comment, must not credit #41's author. Mask those
before the keyword scan. A preview of 2.9.0 still reproduces its notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 3: one finding, in the review body's Previously missed block (no thread to reply into), fixed in c78e764.

  • Ignore closing keywords in Markdown code and blockquotes (scripts/release-notes.mjs): done. A new proseOf masks HTML comments, fenced code (``````` and ~~~), inline code spans and blockquote lines before the keyword scan. I added HTML comments beyond the suggestion because a PR template's leftover `` is the likeliest real case. A new test covers each context, including an unterminated fence or comment. The 2.9.0 preview is still byte-identical to the published notes.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Markdown parsing can credit unrelated issues, and the publish procedure can select a stale release version.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Previously missed (1)

In code that hasn't changed since last review

Low severity Use FETCH_HEAD instead of potentially stale origin/main

.claude/​skills/​release/​SKILL.md:280

origin/main is not the reliable result of the preceding git fetch origin main; this repository already calls that out in scripts/release-tag.mjs:54-62 and reads FETCH_HEAD instead. If the tracking ref remains stale, this command can edit the previous release rather than publish the draft just created. Fetch immediately here and derive the version from FETCH_HEAD.

Comment thread scripts/release-notes.mjs Outdated
A closing fence takes only trailing whitespace and may be longer than
the opening one; a fence line with text after it is still code, so a
keyword below it stays masked. The 3b publish snippet now fetches and
reads FETCH_HEAD rather than a possibly stale origin/main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 4: two findings, both answered in b2e156d.

  • Parse GFM code blocks correctly (inline thread): the closing-fence rule now follows GFM, with regression cases. Masking four-space indented code is declined, with the reason in the thread: it can't be done without a full Markdown parser and would hide real Closes lines in list items.
  • Use FETCH_HEAD instead of potentially stale origin/main (SKILL.md 3b, in Previously missed, no thread): fixed. The publish snippet now runs git fetch origin main and reads the version from FETCH_HEAD, as release-tag.mjs does.

The 2.9.0 preview is still byte-identical, and validate:guards + validate:core pass.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The Markdown fence parser can omit valid issue-closing references and reporter credits.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread scripts/release-notes.mjs
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 5: one finding, declined with the reason in its thread: a backtick fence with a backtick in its info string. It's a GFM corner case that doesn't occur in real PR bodies, and exact fidelity would need a CommonMark parser, beyond #2550's scope. Round 4's fence fix shows as resolved.

Review loop closed. The round held only a finding declined as out of scope, so another round would only re-argue the same scope. Over five rounds, four real defects were fixed (--previous-tag outside preview, the unquoted skill placeholder, keywords in code/quotes/comments, the closing-fence rule plus a stale-ref publish snippet), and indented code blocks and this info-string case were declined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v2 Issues and PRs for v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Script the release notes (generated list, ledger, known issues, reporter thanks) and release creation in the release skill

2 participants