diff --git a/.github/BRIEF.md b/.github/BRIEF.md index a6ffd42..139562c 100644 --- a/.github/BRIEF.md +++ b/.github/BRIEF.md @@ -23,6 +23,13 @@ verify the changes and repository protections from concrete evidence. and artifact; spec PRs show compatibility failures; unrun checks are labeled as such. - Idempotency: refresh tests prove unchanged bytes preserve the document and date, including repeated proposals; one fixed bot branch prevents daily duplicate PRs. +- Release correctness: semantic spec fixtures distinguish contract changes from + annotations. All four native jobs pass project checks and the standalone smoke + harness. Artifact tests reject wrong identity, digest, file type, archive paths + and incomplete platform sets before the writer can upload. +- Release recovery: HTTP integration tests prove draft-only creation, exact asset + verification, resumption of matching uploads, conflict rejection and preservation + of published releases. Published releases are never an automation output. ## Oracle @@ -58,3 +65,9 @@ and manual dependency patch workflow retain the exceptions documented in SPEC.md reopens the draft PR to trigger pull-request CI. Manual dispatch is diagnostic; acceptance requires all seven checks attached to the current PR revision. Evidence: [repository verification](SPEC.md#decisions). + +2026-09-09, ratified by request: spec PRs propose version/changelog changes and +reviewed versions produce four-platform verified draft releases. Public +publication of v0.2.0 is authorized; future drafts require a maintainer to publish. +Release builds use fresh native hosted VMs with read-only repository tokens; +the separate draft writer never installs dependencies or executes artifacts. diff --git a/.github/SPEC.md b/.github/SPEC.md index b0261de..435fdb7 100644 --- a/.github/SPEC.md +++ b/.github/SPEC.md @@ -16,8 +16,9 @@ updating the default branch. platform limitation, not an authorization granted by this repository. - REQ-REPO-003: Workflows default to read-only contents permissions, hosted runners, bounded timeouts, commit-pinned actions, and checkout without persisted credentials. - Fork workflows receive no write tokens or secrets. Only the spec PR job receives - contents and pull-requests write permissions; it does not install dependencies + Fork workflows receive no write tokens or secrets. The spec PR job receives + contents and pull-requests write permissions; the release draft job receives + contents write permission. Neither writer installs dependencies or execute project checks or the refreshed spec. Its builtin-only validator runs from the trusted main revision selected when the workflow started. - REQ-REPO-004: The default branch requires a PR, one code-owner approval from @@ -35,10 +36,10 @@ updating the default branch. manual review. Invalid responses fail before writing files. - REQ-REPO-007: One dedicated branch, `automation/brale-spec`, holds the latest spec proposal. A repeated upstream revision preserves its prior fetch date. - Only the vendored document and pin file enter its PR. The updater reports + Only the vendored document, pin, CLI package version and changelog enter its PR. The updater reports compatibility checks and opens a draft even when those checks fail, so upstream changes remain visible. Fetch/preparation failures do not publish a proposal. - Package versions and releases are separate maintainer decisions. + Version proposals require maintainer review; public releases require publication. - REQ-REPO-008: Preparation installs no project dependencies and uploads only raw spec bytes and revision metadata. Compatibility runs in a separate job. The publisher downloads the preparation artifact by immutable ID, rejects @@ -51,14 +52,44 @@ updating the default branch. from tracked working-tree files without Git metadata or local credentials. Untrusted logs cannot issue Actions workflow commands. Exit status determines success. The action-manifest audit requires a separate networked container. +- REQ-REPO-010: Validated spec proposals include a CLI version and changelog + proposal when the semantic contract changes. Whitespace and known annotation + changes alone do not bump versions. Schema property names and payload values + remain meaningful. Automated proposals conservatively increment the minor + version before 1.0 and the major version thereafter; reviewers can adjust the + proposal after assessing compatibility. The writer derives these files from + trusted main, never accepts executable files or release metadata from upstream, + and repeated proposals against the same base are identical. +- REQ-REPO-011: A release workflow on canonical-main pushes and manual dispatch + checks the CLI version for a pending release. Four separate read-only jobs build + and smoke-test darwin-arm64, darwin-x64, linux-arm64 and linux-x64 natively, + with a locked dependency install, Bun 1.3.3, and synthetic credentials. Failed + builds or smoke tests prevent draft creation. Native release jobs run on fresh + hosted VMs and are outside the Docker isolation of REQ-REPO-009. +- REQ-REPO-012: A separate trusted writer validates all four archives, exact file + names, regular file types, bounded sizes, archive contents, checksums, source + revision and version before uploading. It installs no project dependencies and + never executes a downloaded binary. Drafts contain four archives, SHA256SUMS, + and build-info.json. Only artifacts from the same successful workflow run enter + this writer. +- REQ-REPO-013: Automation creates draft releases only. Repeated runs preserve + complete releases; interrupted draft uploads reconcile matching assets and fail + on conflicts rather than replacing bytes. A published version is never changed. + Client PRs include reviewed version/changelog changes when a release is needed; + multiple changes may share one release. Maintainers attach every asset before + publishing, because repository immutability locks future published releases. +- REQ-REPO-014: The v0.2.0 release contains the reviewed onboarding commands and + refreshed contract. All four exact shipped binaries pass the smoke harness; + release metadata identifies their source revision and checksums. Publication + of this version is authorized by the user; future publications remain manual. ## Invariants and non-goals Workflow jobs never receive Brale credentials or move funds. External contributors cannot initiate this repository's CI; workflows run independently in their own forks are outside this repository's control. Maintainers inspect outside changes -before bringing them onto a repository branch for CI. There is no package -publication or deployment workflow. Repository visibility and licensing do not +before bringing them onto a repository branch for CI. There is no npm package +publication or automatic public release workflow. Repository visibility and licensing do not change as part of these protections. The isolation assumes reviewed workflow, runner and publisher definitions. The @@ -91,9 +122,30 @@ REQ-REPO-009; maintainers review its main-branch inputs before running it. - [ ] REQ-REPO-009: The Docker adversarial verifier proves credential/host isolation, blocked outbound access, working loopback, skipped lifecycle scripts and failure propagation. Compatibility passes inside the same container runner. +- [x] REQ-REPO-010: Semantic projection and trusted proposal integration tests pass, + including annotation-only changes, schema field names, bounds and repeatability. +- [x] REQ-REPO-012/013: Archive validation and HTTP integration tests pass; existing + lightweight and annotated tags must resolve to the build commit; conflicting + release assets cause zero replacement writes. Script typechecks run in CI. +- [x] REQ-REPO-011/013: Workflow policy tests and actionlint verify native runner + mappings, main-only execution, successful-build dependencies, stable artifact + names across failed-job reruns and draft writer isolation. +- [ ] REQ-REPO-011/014: All four exact v0.2.0 binaries pass standalone smoke; + the source revision, archive digests and published immutability are verified. ## Decisions +- 2026-09-09, ratified by request: implement spec version/changelog proposals, + verified four-platform builds and draft releases; deliver v0.2.0. Future public + releases are manual. Repository release immutability and non-updatable, + non-deletable v* tags are active. Existing mutable releases remain unchanged. +- 2026-09-09, provisional: release artifacts use one stable name per platform per + workflow run. Rebuilding a leg replaces its workflow artifact; successful legs + remain available across failed-job reruns. Each artifact is immutable once + uploaded and draft release assets are never overwritten. Conflicts stop for + inspection. An existing version tag is independently resolved before and after + uploads; an absent tag remains valid for a draft until manual publication. + - 2026-09-08, ratified by request: daily spec checking and automated PR creation are authorized; PR approval and merging remain human decisions. - 2026-09-08, provisional: native `GITHUB_TOKEN` avoids a new long-lived secret; diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..15b9cd5 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,124 @@ +name: Prepare Release + +on: + push: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: release-preparation + cancel-in-progress: false + +jobs: + candidate: + name: Check pending version + if: github.repository == '0xsend/bralecli' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + prepare: ${{ steps.inspect.outputs.prepare }} + version: ${{ steps.inspect.outputs.version }} + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ github.sha }} + persist-credentials: false + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version-file: .node-version + - name: Inspect pending release + id: inspect + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + printf '%s' "$GH_TOKEN" | node scripts/release-draft.ts inspect > "$RUNNER_TEMP/release-candidate.json" + jq -r '"prepare=\(.prepare)\nversion=\(.version)"' "$RUNNER_TEMP/release-candidate.json" >> "$GITHUB_OUTPUT" + cat "$RUNNER_TEMP/release-candidate.json" >> "$GITHUB_STEP_SUMMARY" + + build: + name: Build and smoke (${{ matrix.platform }}) + needs: candidate + if: needs.candidate.outputs.prepare == 'true' + runs-on: ${{ matrix.runner }} + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + include: + - platform: darwin-arm64 + runner: macos-15 + - platform: darwin-x64 + runner: macos-15-intel + - platform: linux-arm64 + runner: ubuntu-24.04-arm + - platform: linux-x64 + runner: ubuntu-24.04 + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ github.sha }} + persist-credentials: false + - uses: ./.github/actions/setup-nub + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + with: + bun-version: 1.3.3 + - name: Check, compile and smoke-test native executable + env: + PLATFORM: ${{ matrix.platform }} + VERSION: ${{ needs.candidate.outputs.version }} + shell: bash + run: | + set -euo pipefail + stop_token=$(openssl rand -hex 32) + echo "::stop-commands::$stop_token" + trap 'echo "::$stop_token::"' EXIT + nub ci --ignore-scripts + nub run check + nub run build + nub run knip + nub run jscpd + bun --no-env-file scripts/build-binaries.mjs "$RUNNER_TEMP/binaries" "$PLATFORM" + python3 scripts/smoke-binary.py "$RUNNER_TEMP/binaries/bralecli-$PLATFORM" "$VERSION" + node scripts/release-artifacts.ts package "$RUNNER_TEMP/binaries" "$PLATFORM" "$RUNNER_TEMP/package" + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + # Stable names retain successful legs when only failed jobs are rerun. + # A rebuilt leg replaces its workflow artifact, never a release asset. + name: release-${{ github.run_id }}-${{ matrix.platform }} + overwrite: true + path: ${{ runner.temp }}/package/ + if-no-files-found: error + retention-days: 14 + + draft: + name: Validate and prepare draft + needs: [candidate, build] + runs-on: ubuntu-latest + timeout-minutes: 15 + permissions: + contents: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ github.sha }} + persist-credentials: false + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version-file: .node-version + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + pattern: release-${{ github.run_id }}-* + merge-multiple: true + path: ${{ runner.temp }}/release-inputs + - name: Validate all artifacts and reconcile draft assets + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + printf '%s' "$GH_TOKEN" | node scripts/release-draft.ts upload "$RUNNER_TEMP/release-inputs" "$RUNNER_TEMP/verified-release" > "$RUNNER_TEMP/release-result.json" + cat "$RUNNER_TEMP/release-result.json" >> "$GITHUB_STEP_SUMMARY" + echo 'Draft only. Review its source, changelog, checksums and all four assets before publishing. Publication locks assets under repository release immutability.' >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/update-spec.yml b/.github/workflows/update-spec.yml index d9f9c28..5d0e025 100644 --- a/.github/workflows/update-spec.yml +++ b/.github/workflows/update-spec.yml @@ -122,7 +122,7 @@ jobs: artifact-ids: ${{ needs.prepare.outputs.artifact-id }} merge-multiple: true path: ${{ runner.temp }}/spec-update - - name: Validate data and reconstruct trusted pin + - name: Validate data and propose version from trusted source run: node scripts/apply-spec-proposal.ts --proposal-dir "$RUNNER_TEMP/spec-update" - name: Create or update draft PR id: proposal @@ -133,13 +133,21 @@ jobs: add-paths: | packages/brale/openapi/brale.json packages/brale/src/spec.ts + apps/cli/package.json + apps/cli/CHANGELOG.md commit-message: 'chore(spec): refresh Brale OpenAPI contract' title: 'chore(spec): refresh Brale OpenAPI contract' draft: always-true maintainer-can-modify: false body: | Brale's upstream OpenAPI bytes changed. This PR updates the vendored - document and its SHA-256/fetch-date pin; package versions are unchanged. + document and its SHA-256/fetch-date pin. A semantic contract change + also proposes a CLI version and changelog entry from trusted main. + Documentation-only or formatting-only updates keep the version unchanged. + Contract changes conservatively propose a minor release before 1.0, + and a major release from 1.0 onward; review the diff and adjust the + proposed version and release note when appropriate before merging. + A merged version change enters the draft-release build workflow. Upstream SHA-256: `${{ needs.prepare.outputs.hash }}` @@ -152,8 +160,9 @@ jobs: Preparation installs no project dependencies. Compatibility runs on a separate runner in a credential-free, offline container. The publisher - validates the preparation artifact and reconstructs the pin from trusted - source; compatibility code cannot replace the proposed files. + validates the preparation artifact and reconstructs the pin, version, + and changelog from trusted source; compatibility code cannot replace + the proposed files. The artifact contains no package or changelog files. Review the contract diff, then close and reopen this draft PR as a maintainer to trigger pull-request CI under this repository's actor diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 47df78d..86da06c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -71,6 +71,11 @@ metadata. A separate job checks compatibility in the CI container. The publisher validates the immutable preparation artifact and reconstructs the pin from its trusted main checkout, then opens or updates the draft PR on `automation/brale-spec`. It never accepts executable source from the artifact. +When the contract changes, the trusted publisher also proposes a CLI version and +changelog entry. Formatting and known documentation annotations alone do not +trigger a release. Because an upstream contract change may be breaking, automatic +proposals increment the minor version before 1.0 and the major version thereafter. +Reviewers assess compatibility and can adjust the proposed bump before merging. Failed compatibility checks remain visible in the PR and do not suppress it. To activate the workflow after merging it to `main`: @@ -102,4 +107,45 @@ Verify the checks attached to the PR itself. The bot replaces its dedicated branch on later upstream changes. Make fixes on a separate branch, and close an obsolete proposal if upstream reverts to the -vendored revision. The updater does not bump package versions or publish a release. +vendored revision. Its version and changelog changes remain proposals until merged. + +## Releases + +Include a CLI version bump in `apps/cli/package.json` and an entry in +`apps/cli/CHANGELOG.md` when a client change needs to ship. Multiple changes may +share a version. Spec PRs propose these files automatically. Repository-only +changes do not require a release. + +**Prepare Release** runs on maintainer pushes to `main`, including reviewed PR +merges, and supports manual dispatch on `main`. An already completed version is a +no-op. A pending version runs locked installs and project checks, then builds and +smoke-tests each executable on its native macOS/Linux and ARM64/x64 runner. These +jobs are read-only, skip install scripts, and use synthetic credentials; they run +on fresh hosted VMs rather than the CI Docker sandbox. + +A separate writer downloads artifacts from that successful run, validates the +archive contents, source revision, version and checksums, then creates a draft +containing four archives, `SHA256SUMS` and `build-info.json`. It never executes +downloaded binaries or installs dependencies. Failed uploads leave a draft; +reruns resume matching assets and reject conflicts without replacing bytes. + +Review the draft's source revision, changelog and all four build/smoke results. +Confirm all six assets are attached before publishing. Repository release +immutability locks the assets and tag when the draft is published. Corrections +ship under a new version. Automation never publishes a draft or approves/merges +a PR. Existing mutable releases are not changed by this workflow. + +For local recovery, use a clean checkout of the exact intended source commit: + +```sh +nub ci --ignore-scripts +bun --no-env-file scripts/build-binaries.mjs /absolute/binaries +# Run scripts/smoke-binary.py against each target on a compatible machine. +nub scripts/release-artifacts.ts package /absolute/binaries darwin-arm64 /absolute/package-darwin-arm64 +# Repeat packaging for the other three platforms; collect their files in inputs. +gh auth token | nub scripts/release-draft.ts upload /absolute/inputs /absolute/new-verified-output +``` + +Packaging does not replace the required smoke tests. The upload command validates +the inputs and leaves the release as a draft. Pass tokens only through stdin; +never place them in command arguments or files. diff --git a/README.md b/README.md index 6547f8c..b763e16 100644 --- a/README.md +++ b/README.md @@ -284,8 +284,9 @@ The **Update Brale Spec** workflow checks this endpoint daily at 08:23 UTC and can also be run manually on `main`. It compares raw bytes, including changes that leave `info.version` unchanged. A changed document opens or updates one draft PR on `automation/brale-spec`, including its checksum and fetch date. -The same upstream revision keeps its original proposal date. Package versions -are not bumped automatically. +The same upstream revision keeps its original proposal date. Contract changes +also propose a CLI version and changelog update for maintainer review; +documentation-only changes do not propose a new release. The workflow uses `nub scripts/refresh-spec.ts --update-pin --format json` to prepare the proposal. `--previous-pin ` reads an earlier proposal's pin @@ -294,7 +295,13 @@ continues to leave pin updates for the maintainer. Invalid downloads fail before writing the spec; compatibility failures are reported in the draft PR and run logs. The bot owns its branch, so make follow-up fixes on a separate branch. -Activation and the maintainer CI handoff are described in +After a reviewed version update reaches `main`, **Prepare Release** builds and +smoke-tests all four standalone executables and prepares a draft release. +Maintainers review and publish the completed draft; publication locks its assets +under repository release immutability. Installed executables retain their embedded +contract until users install a new release. + +Activation, the maintainer CI handoff and release operation are described in [CONTRIBUTING.md](CONTRIBUTING.md#daily-spec-updates). ## Development diff --git a/apps/cli/CHANGELOG.md b/apps/cli/CHANGELOG.md index e557ddc..1ce214d 100644 --- a/apps/cli/CHANGELOG.md +++ b/apps/cli/CHANGELOG.md @@ -1,5 +1,28 @@ # bralecli +## 0.2.0 + +### Added + +- Install the curated Brale operating skill for Claude Code and Codex with + `bralecli agents install`, including project/global scopes, agent selection, + conflict detection and explicit replacement. +- Embed the curated skill in standalone macOS and Linux executables. + +### Updated + +- Refresh the embedded Brale OpenAPI contract, including business address fields + and the document upload contract's reverse-side image field. +- Prepare future releases from reviewed spec version proposals, with four-platform + builds, standalone smoke checks, checksums and draft-only uploads. + +## 0.1.1 + +### Fixed + +- Restore JSON object and array flags before request validation. +- Provide standalone macOS and Linux executables for ARM64 and x64. + ## 0.1.0 ### Minor Changes diff --git a/apps/cli/package.json b/apps/cli/package.json index 5cbc349..fcc3635 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -1,6 +1,6 @@ { "name": "bralecli", - "version": "0.1.1", + "version": "0.2.0", "private": true, "bin": { "bralecli": "./src/bin.ts" diff --git a/package.json b/package.json index 52c7c5b..7bee743 100644 --- a/package.json +++ b/package.json @@ -8,7 +8,7 @@ "type": "module", "scripts": { "build": "nub run -r --filter './packages/*' build && nub run -r --filter './apps/*' build", - "typecheck": "tsc -b", + "typecheck": "tsc -b && tsc -p scripts/tsconfig.json", "lint": "oxlint .", "lint:fix": "oxlint --fix .", "format": "oxfmt --write .", diff --git a/scripts/apply-spec-proposal.test.ts b/scripts/apply-spec-proposal.test.ts index 7b423c7..ccc0f35 100644 --- a/scripts/apply-spec-proposal.test.ts +++ b/scripts/apply-spec-proposal.test.ts @@ -22,29 +22,37 @@ const proposal = { const proposedSource = baseSource .replace(proposal.previousHash, proposal.hash) .replace(/(SPEC_FETCHED_AT: string = ')[^']+/, '$12026-09-02') +const packageSource = '{"name":"bralecli","version":"0.2.7","private":true}\n' +const changelogSource = '# bralecli\n\n## 0.2.7\n\n- Existing release.\n' let root: string let proposalDirectory: string let documentPath: string let sourcePath: string +let packagePath: string +let changelogPath: string beforeEach(async () => { root = await mkdtemp(join(tmpdir(), 'brale-spec-apply-')) proposalDirectory = join(root, 'proposal') documentPath = join(root, 'packages/brale/openapi/brale.json') sourcePath = join(root, 'packages/brale/src/spec.ts') + packagePath = join(root, 'apps/cli/package.json') + changelogPath = join(root, 'apps/cli/CHANGELOG.md') await Promise.all( - ['scripts', 'packages/brale/openapi', 'packages/brale/src', 'proposal'].map((path) => - mkdir(join(root, path), { recursive: true }), + ['scripts', 'packages/brale/openapi', 'packages/brale/src', 'apps/cli', 'proposal'].map( + (path) => mkdir(join(root, path), { recursive: true }), ), ) await Promise.all([ - ...['apply-spec-proposal.ts', 'spec-proposal.ts'].map((name) => + ...['apply-spec-proposal.ts', 'spec-proposal.ts', 'spec-release.ts'].map((name) => copyFile(new URL(`./${name}`, import.meta.url), join(root, 'scripts', name)), ), writeFile(join(root, 'package.json'), '{"type":"module"}'), writeFile(documentPath, baseDocument), writeFile(sourcePath, baseSource), + writeFile(packagePath, packageSource), + writeFile(changelogPath, changelogSource), writeFile(join(proposalDirectory, 'brale.json'), proposedDocument), writeFile(join(proposalDirectory, 'revision.json'), JSON.stringify(proposal)), ]) @@ -63,12 +71,25 @@ function apply(directory = proposalDirectory): Promise<{ stdout: string; stderr: } async function rejectsUnchanged(message: string): Promise { - const before = await Promise.all([readFile(documentPath), readFile(sourcePath)]) + const paths = [documentPath, sourcePath, packagePath, changelogPath] + const before = await Promise.all(paths.map((path) => readFile(path))) await expect(apply()).rejects.toMatchObject({ stdout: '', stderr: expect.stringContaining(message), }) - expect(await Promise.all([readFile(documentPath), readFile(sourcePath)])).toEqual(before) + expect(await Promise.all(paths.map((path) => readFile(path)))).toEqual(before) +} + +async function proposeContractChange(): Promise { + const document = JSON.parse(baseDocument.toString('utf8')) + delete document.paths['/accounts/{account_id}/financial-institutions/{fi_id}/status'] + const body = Buffer.from(`${JSON.stringify(document, null, 4)}\n\n`) + await writeFile(join(proposalDirectory, 'brale.json'), body) + await writeFile( + join(proposalDirectory, 'revision.json'), + JSON.stringify({ ...proposal, hash: hash('sha256', body) }), + ) + return body } describe('trusted application of data-only spec proposals', () => { @@ -76,12 +97,49 @@ describe('trusted application of data-only spec proposals', () => { const sentinel = '\nthrow new Error("Pin source must never execute")\n' await writeFile(sourcePath, baseSource + sentinel) const result = await apply() - expect(JSON.parse(result.stdout)).toEqual(proposal) + expect(JSON.parse(result.stdout)).toEqual({ ...proposal, release: { kind: 'none' } }) expect(await readFile(documentPath)).toEqual(proposedDocument) expect(await readFile(sourcePath, 'utf8')).toBe(proposedSource + sentinel) + expect(await readFile(packagePath, 'utf8')).toBe(packageSource) + expect(await readFile(changelogPath, 'utf8')).toBe(changelogSource) + await rejectsUnchanged('previousHash') + }) + + it('proposes a reviewed release for a semantic contract change and preserves the exact spec bytes', async () => { + const body = await proposeContractChange() + const result = await apply() + expect(JSON.parse(result.stdout).release).toEqual({ + kind: 'release', + previousVersion: '0.2.7', + version: '0.3.0', + }) + expect(JSON.parse(await readFile(packagePath, 'utf8')).version).toBe('0.3.0') + expect(await readFile(changelogPath, 'utf8')).toContain('## 0.3.0\n') + expect(await readFile(changelogPath, 'utf8')).toContain(hash('sha256', body)) + expect(await readFile(documentPath)).toEqual(body) await rejectsUnchanged('previousHash') }) + it('rejects invalid trusted release metadata before changing the spec or pin', async () => { + await proposeContractChange() + await writeFile(packagePath, '{"name":"bralecli","version":"latest"}') + await rejectsUnchanged('CLI version') + }) + + it('rejects excessive document depth before changing any trusted files', async () => { + let nested: unknown = { type: 'string' } + for (let depth = 0; depth < 140; depth++) nested = { items: nested } + const document = JSON.parse(baseDocument.toString('utf8')) + document.components.schemas.Deep = nested + const body = Buffer.from(JSON.stringify(document)) + await writeFile(join(proposalDirectory, 'brale.json'), body) + await writeFile( + join(proposalDirectory, 'revision.json'), + JSON.stringify({ ...proposal, hash: hash('sha256', body) }), + ) + await rejectsUnchanged('depth limit') + }) + for (const entry of ['spec.ts', '.hidden', 'nested/file']) { it(`rejects an unexpected artifact entry ${entry} before writing`, async () => { if (entry.includes('/')) await mkdir(join(proposalDirectory, 'nested')) diff --git a/scripts/apply-spec-proposal.ts b/scripts/apply-spec-proposal.ts index a655c30..0102e46 100644 --- a/scripts/apply-spec-proposal.ts +++ b/scripts/apply-spec-proposal.ts @@ -2,8 +2,8 @@ /** Validates untrusted data, then reconstructs pins from the trusted checkout. */ import { constants } from 'node:fs' -import { lstat, open, opendir, readFile } from 'node:fs/promises' -import { join } from 'node:path' +import { lstat, mkdtemp, open, opendir, readFile, rename, rm, writeFile } from 'node:fs/promises' +import { dirname, join } from 'node:path' import { parseArgs } from 'node:util' import { @@ -18,11 +18,33 @@ import { specPath, updatePin, validateDocument, - writeProposal, type RefreshResult, } from './spec-proposal.ts' +import { planSpecRelease } from './spec-release.ts' const maxMetadataBytes = 4096 +const packagePath = new URL('../apps/cli/package.json', import.meta.url) +const changelogPath = new URL('../apps/cli/CHANGELOG.md', import.meta.url) + +async function applyFiles( + files: readonly { path: string | URL; body: Buffer | string }[], +): Promise { + const staging = await mkdtemp(join(dirname(specPath), '.apply-spec-')) + try { + await Promise.all( + files.map((file, index) => writeFile(join(staging, String(index)), file.body)), + ) + // Atomicity: each rename is atomic. Interrupted application fails the base + // hash precondition on retry. Permanent — the disposable checkout is only + // published after every write succeeds, as one Git commit. + await rename(join(staging, '0'), files[0]!.path) + await Promise.all( + files.slice(1).map((file, index) => rename(join(staging, String(index + 1)), file.path)), + ) + } finally { + await rm(staging, { recursive: true, force: true }) + } +} async function validateDirectory(path: string): Promise { const stat = await lstat(path) @@ -90,11 +112,13 @@ async function main(): Promise { const directory = values['proposal-dir'] if (!directory) throw new Error('--proposal-dir is required') await validateDirectory(directory) - const [body, metadata, current, source] = await Promise.all([ + const [body, metadata, current, source, packageSource, changelogSource] = await Promise.all([ readProposalFile(join(directory, 'brale.json'), maxDocumentBytes), readProposalFile(join(directory, 'revision.json'), maxMetadataBytes), readFile(specPath), readFile(pinPath, 'utf8'), + readFile(packagePath, 'utf8'), + readFile(changelogPath, 'utf8'), ]) validateDocument(body) const revision = parseRevision(metadata) @@ -108,8 +132,36 @@ async function main(): Promise { if (revision.hash !== sha256(body)) throw new Error('Proposal document does not match its declared hash') if (revision.hash === currentHash) throw new Error('Proposal document is unchanged') - await writeProposal(body, updatePin(source, revision)) - console.log(JSON.stringify(revision)) + const release = planSpecRelease({ + previousDocument: current, + document: body, + packageSource, + changelogSource, + revision, + }) + await applyFiles([ + { path: specPath, body }, + { path: pinPath, body: updatePin(source, revision) }, + ...(release.kind === 'release' + ? [ + { path: packagePath, body: release.packageSource }, + { path: changelogPath, body: release.changelogSource }, + ] + : []), + ]) + console.log( + JSON.stringify({ + ...revision, + release: + release.kind === 'release' + ? { + kind: release.kind, + previousVersion: release.previousVersion, + version: release.version, + } + : release, + }), + ) } try { diff --git a/scripts/build-binaries.mjs b/scripts/build-binaries.mjs index 54241d8..f32c2a3 100644 --- a/scripts/build-binaries.mjs +++ b/scripts/build-binaries.mjs @@ -1,12 +1,25 @@ -// Run after nub ci: bun --no-env-file scripts/build-binaries.mjs /absolute/output/directory +// Run after nub ci: bun --no-env-file scripts/build-binaries.mjs /absolute/output/directory [platform] // Bun 1.3.3 embeds the runtime; the checkout remains unchanged. import { mkdir, readFile, writeFile } from 'node:fs/promises' import { resolve } from 'node:path' +import { compileBinaries } from './compile-binaries.mjs' + if (Bun.version !== '1.3.3') throw new Error('This release recipe requires Bun 1.3.3') const root = process.cwd() const out = process.argv[2] if (!out?.startsWith('/')) throw new Error('Pass an absolute output directory') +const targets = { + 'darwin-arm64': 'darwin-arm64', + 'darwin-x64': 'darwin-x64-baseline', + 'linux-arm64': 'linux-arm64', + 'linux-x64': 'linux-x64-baseline', +} +const requested = process.argv[3] +if (process.argv.length > 4 || (requested !== undefined && !Object.hasOwn(targets, requested))) { + throw new Error(`Expected one platform: ${Object.keys(targets).join(', ')}`) +} +const platforms = requested === undefined ? Object.values(targets) : [targets[requested]] const head = Bun.spawnSync(['git', 'rev-parse', 'HEAD'], { cwd: root }) if (head.exitCode !== 0) throw new Error('Cannot identify source revision') const revision = head.stdout.toString().trim() @@ -66,33 +79,9 @@ const result = await Bun.build({ ], }) if (!result.success) throw new AggregateError(result.logs, 'Bundle failed') -const platforms = ['darwin-arm64', 'darwin-x64-baseline', 'linux-arm64', 'linux-x64-baseline'] -await Promise.all( - platforms.map(async (platform) => { - const filename = `bralecli-${platform.replace('-baseline', '')}` - const child = Bun.spawn( - [ - 'bun', - 'build', - intermediate, - '--compile', - `--target=bun-${platform}`, - '--no-compile-autoload-dotenv', - '--no-compile-autoload-bunfig', - '--env=disable', - '--outfile', - resolve(out, filename), - ], - { stdout: 'inherit', stderr: 'inherit' }, - ) - const timeout = setTimeout(() => child.kill(), 300_000) - const code = await child.exited - clearTimeout(timeout) - if (code !== 0) throw new Error(`Compile failed: ${platform}`) - }), -) +await compileBinaries({ compiler: process.execPath, intermediate, targets: platforms, out }) await writeFile( resolve(out, 'build-info.json'), JSON.stringify({ revision, version, bun: Bun.version }, null, 2) + '\n', ) -console.log(`Built four targets from ${revision}`) +console.log(`Built ${platforms.length} target(s) from ${revision}`) diff --git a/scripts/build-binaries.test.ts b/scripts/build-binaries.test.ts new file mode 100644 index 0000000..c569bfd --- /dev/null +++ b/scripts/build-binaries.test.ts @@ -0,0 +1,111 @@ +import { chmod, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' + +import { afterEach, beforeEach, describe, expect, it } from 'vitest' + +import { compileBinaries } from './compile-binaries.mjs' + +let root: string +let out: string +let compiler: string +const targets = ['darwin-arm64', 'darwin-x64-baseline', 'linux-arm64', 'linux-x64-baseline'] + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'brale-compiler-')) + out = join(root, 'output') + compiler = join(root, 'compiler.mjs') + await mkdir(out) +}) + +afterEach(async () => { + await rm(root, { recursive: true, force: true }) +}) + +async function fixtureCompiler(body: string) { + await writeFile( + compiler, + `#!${process.execPath}\nimport * as fs from 'node:fs';\nconst output = process.argv[process.argv.indexOf('--outfile') + 1];\n${body}\n`, + { mode: 0o755 }, + ) +} + +function compile(selectedTargets = targets) { + return compileBinaries({ + compiler, + intermediate: join(root, 'bundle.js'), + targets: selectedTargets, + out, + }) +} + +describe('standalone compiler output verification', () => { + it('rejects a compiler that exits successfully without producing its executable', async () => { + await fixtureCompiler('process.exit(0)') + await expect(compile(['darwin-x64-baseline'])).rejects.toThrow('Missing compiled executable') + expect(await readdir(out)).toEqual([]) + }) + + it('runs the four compiler processes serially and validates every final executable', async () => { + await fixtureCompiler(` +const lock = ${JSON.stringify(join(root, 'compiler.lock'))}; +fs.writeFileSync(lock, '', { flag: 'wx' }); +await new Promise(resolve => setTimeout(resolve, 50)); +fs.writeFileSync(output, 'fixture executable', { mode: 0o755 }); +fs.unlinkSync(lock); +`) + await compile() + expect((await readdir(out)).toSorted()).toEqual([ + 'bralecli-darwin-arm64', + 'bralecli-darwin-x64', + 'bralecli-linux-arm64', + 'bralecli-linux-x64', + ]) + }) + + for (const [name, body, error] of [ + ['symlink', "fs.symlinkSync('untrusted', output)", 'regular'], + ['empty file', "fs.writeFileSync(output, '', { mode: 0o755 })", 'nonempty'], + ['non-executable file', "fs.writeFileSync(output, 'fixture', { mode: 0o644 })", 'executable'], + ]) { + it(`rejects a compiler that produces a ${name}`, async () => { + await fixtureCompiler(body) + await expect(compile(['darwin-arm64'])).rejects.toThrow(error) + }) + } + + it('rejects stale output before a compiler can overwrite it', async () => { + const filename = join(out, 'bralecli-darwin-arm64') + await writeFile(filename, 'prior executable') + await chmod(filename, 0o755) + await fixtureCompiler("fs.writeFileSync(output, 'overwritten')") + await expect(compile(['darwin-arm64'])).rejects.toThrow('already exists') + expect(await readFile(filename, 'utf8')).toBe('prior executable') + }) + + it('rejects stale release metadata before compiling into an old output directory', async () => { + await writeFile(join(out, 'build-info.json'), '{"version":"0.1.1"}') + await fixtureCompiler("fs.writeFileSync(output, 'fixture', { mode: 0o755 })") + await expect(compile(['darwin-arm64'])).rejects.toThrow('already exists') + expect(await readdir(out)).toEqual(['build-info.json']) + }) + + it('rejects an unselected stale binary before a single-target build can relabel it', async () => { + const stale = join(out, 'bralecli-linux-x64') + await writeFile(stale, 'prior revision', { mode: 0o755 }) + await fixtureCompiler("fs.writeFileSync(output, 'new revision', { mode: 0o755 })") + await expect(compile(['darwin-arm64'])).rejects.toThrow('already exists') + expect(await readdir(out)).toEqual(['bralecli-linux-x64']) + expect(await readFile(stale, 'utf8')).toBe('prior revision') + }) + + it('rejects a later compiler that removes an earlier executable', async () => { + await fixtureCompiler(` +fs.writeFileSync(output, 'fixture', { mode: 0o755 }); +if (output.endsWith('darwin-x64')) fs.unlinkSync(${JSON.stringify(join(out, 'bralecli-darwin-arm64'))}); +`) + await expect(compile(['darwin-arm64', 'darwin-x64-baseline'])).rejects.toThrow( + 'Missing compiled executable', + ) + }) +}) diff --git a/scripts/compile-binaries.mjs b/scripts/compile-binaries.mjs new file mode 100644 index 0000000..cb71655 --- /dev/null +++ b/scripts/compile-binaries.mjs @@ -0,0 +1,73 @@ +import { spawn } from 'node:child_process' +import { lstat } from 'node:fs/promises' +import { resolve } from 'node:path' + +function compile(compiler, intermediate, target, output) { + return new Promise((accept, reject) => { + const child = spawn( + compiler, + [ + 'build', + intermediate, + '--compile', + `--target=bun-${target}`, + '--no-compile-autoload-dotenv', + '--no-compile-autoload-bunfig', + '--env=disable', + '--outfile', + output, + ], + { stdio: 'inherit' }, + ) + const timeout = setTimeout(() => child.kill('SIGKILL'), 300_000) + child.once('error', (error) => { + clearTimeout(timeout) + reject(error) + }) + child.once('close', (code, signal) => { + clearTimeout(timeout) + if (code === 0) accept() + else reject(new Error(`Compile failed: ${target} (exit ${code}, signal ${signal})`)) + }) + }) +} + +export async function compileBinaries({ compiler, intermediate, targets, out }) { + if (targets.length < 1 || targets.length > 4) + throw new Error('Expected one to four compile targets') + const outputs = targets.map((target) => + resolve(out, `bralecli-${target.replace('-baseline', '')}`), + ) + const existingOutputs = ['darwin-arm64', 'darwin-x64', 'linux-arm64', 'linux-x64'].map( + (platform) => resolve(out, `bralecli-${platform}`), + ) + await Promise.all( + [...existingOutputs, resolve(out, 'build-info.json')].map(async (path) => { + try { + await lstat(path) + throw new Error(`Compile output already exists: ${path}`) + } catch (error) { + if (error.code !== 'ENOENT') throw error + } + }), + ) + // Concurrent Bun 1.3.3 compilers have reported success while losing a Darwin output. + for (const [index, target] of targets.entries()) { + // eslint-disable-next-line no-await-in-loop + await compile(compiler, intermediate, target, outputs[index]) + } + await Promise.all( + outputs.map(async (path) => { + let stat + try { + stat = await lstat(path) + } catch (cause) { + if (cause.code !== 'ENOENT') throw cause + throw new Error(`Missing compiled executable: ${path}`, { cause }) + } + if (!stat.isFile()) throw new Error(`Expected a regular compiled executable: ${path}`) + if (stat.size === 0) throw new Error(`Expected a nonempty compiled executable: ${path}`) + if (!(stat.mode & 0o111)) throw new Error(`Expected an executable output: ${path}`) + }), + ) +} diff --git a/scripts/release-artifacts.test.ts b/scripts/release-artifacts.test.ts new file mode 100644 index 0000000..1b1d5a4 --- /dev/null +++ b/scripts/release-artifacts.test.ts @@ -0,0 +1,235 @@ +import { execFile } from 'node:child_process' +import { hash } from 'node:crypto' +import { + chmod, + copyFile, + mkdir, + mkdtemp, + readFile, + readdir, + rm, + stat, + symlink, + truncate, + writeFile, +} from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { promisify } from 'node:util' +import { gunzipSync, gzipSync } from 'node:zlib' + +import { afterEach, beforeEach, describe, expect, it } from 'vitest' + +const execute = promisify(execFile) +const script = fileURLToPath(new URL('./release-artifacts.ts', import.meta.url)) +const platforms = ['darwin-arm64', 'darwin-x64', 'linux-arm64', 'linux-x64'] +const revision = '0123456789abcdef0123456789abcdef01234567' +const version = '0.2.0' +const build = { revision, version, bun: '1.3.3' } +let root: string +let binaries: string +let downloaded: string +let output: string + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'brale-release-artifacts-')) + binaries = join(root, 'binaries') + downloaded = join(root, 'downloaded') + output = join(root, 'release') + await mkdir(binaries) + await mkdir(downloaded) + await writeFile(join(binaries, 'build-info.json'), JSON.stringify(build)) + await Promise.all( + platforms.map((platform) => + writeFile(join(binaries, `bralecli-${platform}`), `fixture ${platform}\n`, { + mode: 0o755, + }), + ), + ) +}) + +afterEach(async () => { + await rm(root, { recursive: true, force: true }) +}) + +function run(...args: string[]) { + return execute(process.execPath, [script, ...args], { + cwd: root, + env: {}, + timeout: 15_000, + }) +} + +async function assemble() { + await Promise.all( + platforms.map(async (platform) => { + const packaged = join(root, platform) + await run('package', binaries, platform, packaged) + await Promise.all( + (await readdir(packaged)).map((name) => + copyFile(join(packaged, name), join(downloaded, name)), + ), + ) + }), + ) +} + +async function rejectVerification(message: string) { + await expect(run('verify', downloaded, revision, version, output)).rejects.toMatchObject({ + stderr: expect.stringContaining(message), + }) + await expect(readdir(output)).rejects.toMatchObject({ code: 'ENOENT' }) +} + +describe('release archive boundary', () => { + it('packages deterministic portable executable archives and verifies all four identities without executing binaries', async () => { + await assemble() + const again = join(root, 'again') + await run('package', binaries, 'darwin-arm64', again) + const archiveName = 'bralecli-darwin-arm64.tar.gz' + expect(await readFile(join(again, archiveName))).toEqual( + await readFile(join(downloaded, archiveName)), + ) + const unpacked = join(root, 'unpacked') + await mkdir(unpacked) + await execute('/usr/bin/tar', ['-xzf', join(downloaded, archiveName), '-C', unpacked], { + timeout: 5_000, + }) + expect(await readdir(unpacked)).toEqual(['bralecli']) + expect(await readFile(join(unpacked, 'bralecli'), 'utf8')).toBe('fixture darwin-arm64\n') + expect((await stat(join(unpacked, 'bralecli'))).mode & 0o777).toBe(0o755) + await run('verify', downloaded, revision, version, output) + expect((await readdir(output)).toSorted()).toEqual([ + 'SHA256SUMS', + 'bralecli-darwin-arm64.tar.gz', + 'bralecli-darwin-x64.tar.gz', + 'bralecli-linux-arm64.tar.gz', + 'bralecli-linux-x64.tar.gz', + 'build-info.json', + ]) + const aggregate = JSON.parse(await readFile(join(output, 'build-info.json'), 'utf8')) + expect(aggregate).toMatchObject(build) + expect(aggregate.artifacts.map((artifact: { platform: string }) => artifact.platform)).toEqual( + platforms, + ) + const sums = await readFile(join(output, 'SHA256SUMS'), 'utf8') + await Promise.all( + ['build-info.json', ...platforms.map((platform) => `bralecli-${platform}.tar.gz`)].map( + async (name) => { + expect(sums).toContain(`${hash('sha256', await readFile(join(output, name)))} ${name}\n`) + }, + ), + ) + }) + + it('rejects unknown platform before publishing package output', async () => { + await expect(run('package', binaries, '../../other', output)).rejects.toMatchObject({ + stderr: expect.stringContaining('platform'), + }) + await expect(readdir(output)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('never overwrites an existing output directory', async () => { + await mkdir(output) + await writeFile(join(output, 'sentinel'), 'preserve') + await expect(run('package', binaries, 'darwin-arm64', output)).rejects.toMatchObject({ + stderr: expect.stringContaining('already exists'), + }) + expect(await readFile(join(output, 'sentinel'), 'utf8')).toBe('preserve') + }) + + it('requires executable regular bounded input binaries', async () => { + const binary = join(binaries, 'bralecli-darwin-arm64') + await chmod(binary, 0o644) + await expect(run('package', binaries, 'darwin-arm64', output)).rejects.toMatchObject({ + stderr: expect.stringContaining('executable'), + }) + await chmod(binary, 0o755) + await truncate(binary, 256 * 1024 * 1024 + 1) + await expect(run('package', binaries, 'darwin-arm64', output)).rejects.toMatchObject({ + stderr: expect.stringContaining('size limit'), + }) + }) + + for (const mutation of ['missing', 'extra', 'symlink', 'directory', 'oversized'] as const) { + it(`rejects ${mutation} downloaded archive entries without publishing partial output`, async () => { + await assemble() + const name = join(downloaded, 'bralecli-linux-x64.tar.gz') + if (mutation === 'missing') await rm(name) + if (mutation === 'extra') await writeFile(join(downloaded, 'extra.sh'), 'malicious') + if (mutation === 'symlink') { + await rm(name) + await symlink(join(binaries, 'bralecli-linux-x64'), name) + } + if (mutation === 'directory') { + await rm(name) + await mkdir(name) + } + if (mutation === 'oversized') await truncate(name, 128 * 1024 * 1024 + 1) + await rejectVerification( + mutation === 'missing' || mutation === 'extra' + ? 'exactly' + : mutation === 'oversized' + ? 'size limit' + : 'regular file', + ) + }) + } + + for (const [field, value] of [ + ['revision', '0'.repeat(40)], + ['version', '0.3.0'], + ['bun', '1.3.4'], + ['platform', 'linux-x64'], + ['archive', '../malicious'], + ['sha256', '0'.repeat(64)], + ['unexpected', true], + ]) { + it(`rejects mismatched or invalid metadata ${field}`, async () => { + await assemble() + const name = join(downloaded, 'build-info-darwin-arm64.json') + const metadata = JSON.parse(await readFile(name, 'utf8')) + await writeFile(name, JSON.stringify({ ...metadata, [field as string]: value })) + await rejectVerification('metadata') + }) + } + + for (const attack of [ + 'traversal', + 'symlink', + 'extra entry', + 'invalid gzip', + 'oversized expansion', + ] as const) { + it(`rejects ${attack} even when archive checksum is updated`, async () => { + await assemble() + const name = join(downloaded, 'bralecli-darwin-arm64.tar.gz') + let archive = await readFile(name) + const tar = gunzipSync(archive) + if (attack === 'traversal') tar.write('../evil', 0) + if (attack === 'symlink') tar.write('2', 156) + if (attack === 'extra entry') tar[1024] = 1 + if (attack === 'invalid gzip') archive = Buffer.from('invalid gzip') + else if (attack === 'oversized expansion') + archive = gzipSync(Buffer.alloc(256 * 1024 * 1024 + 2049)) + else archive = gzipSync(tar) + await writeFile(name, archive) + const metadataName = join(downloaded, 'build-info-darwin-arm64.json') + const metadata = JSON.parse(await readFile(metadataName, 'utf8')) + await writeFile( + metadataName, + JSON.stringify({ ...metadata, sha256: hash('sha256', archive) }), + ) + await rejectVerification('archive') + }, 20_000) + } + + it('rejects a symlink download directory', async () => { + await assemble() + const linked = join(root, 'linked') + await symlink(downloaded, linked) + downloaded = linked + await rejectVerification('directory') + }) +}) diff --git a/scripts/release-artifacts.ts b/scripts/release-artifacts.ts new file mode 100644 index 0000000..e1068f1 --- /dev/null +++ b/scripts/release-artifacts.ts @@ -0,0 +1,289 @@ +import { hash } from 'node:crypto' +import { constants } from 'node:fs' +import { lstat, mkdtemp, open, opendir, rename, rm, writeFile } from 'node:fs/promises' +import { basename, dirname, join, resolve } from 'node:path' +import { pathToFileURL } from 'node:url' +import { gunzipSync, gzipSync } from 'node:zlib' + +const platforms = ['darwin-arm64', 'darwin-x64', 'linux-arm64', 'linux-x64'] as const +type Platform = (typeof platforms)[number] +type Build = { revision: string; version: string; bun: '1.3.3' } +type Artifact = { platform: Platform; archive: string; sha256: string } +type Release = Build & { artifacts: Artifact[] } +const binaryLimit = 256 * 1024 * 1024 +const archiveLimit = 128 * 1024 * 1024 +const metadataLimit = 4096 +const revisionPattern = /^[a-f0-9]{40}$/ +const versionPattern = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/ + +function platformValue(value: string): Platform { + if (!platforms.includes(value as Platform)) throw new Error(`Unsupported platform: ${value}`) + return value as Platform +} + +function objectWithKeys(value: unknown, keys: string[]): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('Invalid build metadata object') + } + if (Object.keys(value).toSorted().join(',') !== keys.toSorted().join(',')) { + throw new Error('Invalid build metadata fields') + } + return value as Record +} + +function buildValue(value: Record): Build { + if (typeof value.revision !== 'string' || !revisionPattern.test(value.revision)) { + throw new Error('Invalid build metadata revision') + } + if (typeof value.version !== 'string' || !versionPattern.test(value.version)) { + throw new Error('Invalid build metadata version') + } + if (value.bun !== '1.3.3') throw new Error('Invalid build metadata Bun version') + return { revision: value.revision, version: value.version, bun: value.bun } +} + +async function regularDirectory(path: string): Promise { + const stat = await lstat(path) + if (!stat.isDirectory() || stat.isSymbolicLink()) { + throw new Error(`Expected a real directory: ${path}`) + } +} + +async function boundedFile(path: string, limit: number, executable = false): Promise { + const stat = await lstat(path) + if (!stat.isFile() || stat.isSymbolicLink()) throw new Error(`Expected a regular file: ${path}`) + const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK) + try { + const current = await file.stat() + if (!current.isFile()) throw new Error(`Expected a regular file: ${path}`) + if (current.size < 1 || current.size > limit) + throw new Error(`File exceeds size limit: ${path}`) + if (executable && !(current.mode & 0o111)) throw new Error(`Expected an executable: ${path}`) + const bytes = Buffer.alloc(current.size + 1) + let position = 0 + while (position < bytes.length) { + // Each read determines the offset for the next bounded read. + // eslint-disable-next-line no-await-in-loop + const { bytesRead } = await file.read(bytes, position, bytes.length - position, position) + if (!bytesRead) break + position += bytesRead + } + if (position !== current.size) throw new Error(`File changed while reading: ${path}`) + return bytes.subarray(0, position) + } finally { + await file.close() + } +} + +async function metadata(path: string): Promise { + const bytes = await boundedFile(path, metadataLimit) + try { + return JSON.parse(bytes.toString('utf8')) + } catch (cause) { + throw new Error(`Invalid metadata JSON: ${path}`, { cause }) + } +} + +function archiveName(platform: Platform): string { + return `bralecli-${platform}.tar.gz` +} + +// One canonical ustar entry avoids executing tar or interpreting arbitrary archive +// extensions in the writer job. uid, gid and mtime are fixed for repeatable bytes. +function tarHeader(size: number): Buffer { + const header = Buffer.alloc(512) + header.write('bralecli', 0) + header.write('0000755\0', 100) + header.write('0000000\0', 108) + header.write('0000000\0', 116) + header.write(size.toString(8).padStart(11, '0') + '\0', 124) + header.write('00000000000\0', 136) + header.fill(32, 148, 156) + header.write('0', 156) + header.write('ustar\0', 257) + header.write('00', 263) + header.write('0000000\0', 329) + header.write('0000000\0', 337) + const checksum = header.reduce((sum, value) => sum + value, 0) + header.write(checksum.toString(8).padStart(6, '0') + '\0 ', 148) + return header +} + +function packageArchive(binary: Buffer): Buffer { + const padding = Buffer.alloc(((512 - (binary.length % 512)) % 512) + 1024) + const archive = gzipSync(Buffer.concat([tarHeader(binary.length), binary, padding])) + if (archive.length > archiveLimit) throw new Error('Packaged archive exceeds size limit') + return archive +} + +function validateArchive(archive: Buffer): void { + let tar: Buffer + try { + tar = gunzipSync(archive, { maxOutputLength: binaryLimit + 2048 }) + } catch (cause) { + throw new Error('Invalid or oversized release archive', { cause }) + } + const sizeText = tar.toString('ascii', 124, 136) + if (!/^[0-7]{11}\0$/.test(sizeText)) throw new Error('Invalid archive entry size') + const size = Number.parseInt(sizeText, 8) + if (size < 1 || size > binaryLimit) throw new Error('Invalid archive binary size') + const expectedLength = 512 + Math.ceil(size / 512) * 512 + 1024 + if (tar.length !== expectedLength || !tar.subarray(0, 512).equals(tarHeader(size))) { + throw new Error('Invalid archive: expected only the canonical executable bralecli entry') + } + if (tar.subarray(512 + size).some((byte) => byte !== 0)) { + throw new Error('Invalid archive: extra entries or trailing data') + } +} + +async function newDirectory( + output: string, + populate: (stage: string) => Promise, +): Promise { + const target = resolve(output) + try { + await lstat(target) + throw new Error(`Output directory already exists: ${target}`) + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error + } + const stage = await mkdtemp(join(dirname(target), `.${basename(target)}-`)) + try { + const result = await populate(stage) + await rename(stage, target) + return result + } finally { + await rm(stage, { force: true, recursive: true }) + } +} + +function json(value: unknown): string { + return JSON.stringify(value, null, 2) + '\n' +} + +export async function packageArtifacts( + binaryDirectory: string, + target: string, + output: string, +): Promise { + const platform = platformValue(target) + await regularDirectory(binaryDirectory) + const build = buildValue( + objectWithKeys(await metadata(join(binaryDirectory, 'build-info.json')), [ + 'revision', + 'version', + 'bun', + ]), + ) + const binary = await boundedFile(join(binaryDirectory, `bralecli-${platform}`), binaryLimit, true) + const archive = packageArchive(binary) + const artifact = { + ...build, + platform, + archive: archiveName(platform), + sha256: hash('sha256', archive), + } + return newDirectory(output, async (stage) => { + await writeFile(join(stage, artifact.archive), archive, { flag: 'wx', mode: 0o644 }) + await writeFile(join(stage, `build-info-${platform}.json`), json(artifact), { + flag: 'wx', + mode: 0o644, + }) + return artifact + }) +} + +/** Validates data-only job artifacts and atomically creates the six release assets. */ +export async function verifyArtifacts( + input: string, + revision: string, + version: string, + output: string, +): Promise { + const expected = buildValue({ revision, version, bun: '1.3.3' }) + await regularDirectory(input) + const filenames = new Set( + platforms.flatMap((platform) => [archiveName(platform), `build-info-${platform}.json`]), + ) + let count = 0 + for await (const entry of await opendir(input, { bufferSize: 9 })) { + if (!filenames.has(entry.name) || ++count > filenames.size) { + throw new Error( + 'Artifact directory must contain exactly four archives and four metadata files', + ) + } + } + if (count !== filenames.size) { + throw new Error('Artifact directory must contain exactly four archives and four metadata files') + } + return newDirectory(output, async (stage) => { + const artifacts: Artifact[] = [] + // Process one archive at a time to bound memory independently of the target count. + /* eslint-disable no-await-in-loop */ + for (const platform of platforms) { + const value = objectWithKeys(await metadata(join(input, `build-info-${platform}.json`)), [ + 'revision', + 'version', + 'bun', + 'platform', + 'archive', + 'sha256', + ]) + const build = buildValue(value) + if ( + build.revision !== revision || + build.version !== version || + value.platform !== platform || + value.archive !== archiveName(platform) + ) { + throw new Error(`Build metadata identity does not match the requested release: ${platform}`) + } + const archive = await boundedFile(join(input, archiveName(platform)), archiveLimit) + const sha256 = hash('sha256', archive) + if (value.sha256 !== sha256) + throw new Error(`Archive checksum does not match metadata: ${platform}`) + validateArchive(archive) + await writeFile(join(stage, archiveName(platform)), archive, { flag: 'wx', mode: 0o644 }) + artifacts.push({ platform, archive: archiveName(platform), sha256 }) + } + /* eslint-enable no-await-in-loop */ + const release = { ...expected, artifacts } + const manifest = json(release) + const sums = + artifacts.map(({ sha256, archive }) => `${sha256} ${archive}\n`).join('') + + `${hash('sha256', manifest)} build-info.json\n` + await writeFile(join(stage, 'build-info.json'), manifest, { flag: 'wx', mode: 0o644 }) + await writeFile(join(stage, 'SHA256SUMS'), sums, { flag: 'wx', mode: 0o644 }) + return release + }) +} + +if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) { + const [command, ...args] = process.argv.slice(2) + const [first, second, third, fourth] = args + const usage = + 'Usage: release-artifacts.ts package | verify ' + try { + if (first === undefined || second === undefined || third === undefined) throw new Error(usage) + if (command === 'package' && args.length === 3) { + console.log( + json({ + status: 'packaged', + ...(await packageArtifacts(first, second, third)), + }).trimEnd(), + ) + } else if (command === 'verify' && args.length === 4 && fourth !== undefined) { + console.log( + json({ + status: 'verified', + ...(await verifyArtifacts(first, second, third, fourth)), + }).trimEnd(), + ) + } else { + throw new Error(usage) + } + } catch (error) { + console.error(error instanceof Error ? error.message : String(error)) + process.exitCode = 1 + } +} diff --git a/scripts/release-draft.test.ts b/scripts/release-draft.test.ts new file mode 100644 index 0000000..768b464 --- /dev/null +++ b/scripts/release-draft.test.ts @@ -0,0 +1,217 @@ +import { createServer } from 'node:http' +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { createHash } from 'node:crypto' +import { afterEach, expect, it } from 'vitest' + +import { githubApi, inspectRelease, uploadDraft } from './release-draft.ts' + +const version = '0.2.0' +const revision = 'a'.repeat(40) +const platforms = ['darwin-arm64', 'darwin-x64', 'linux-arm64', 'linux-x64'] +const cleanups: (() => Promise)[] = [] +afterEach(async () => { + await Promise.all(cleanups.splice(0).map((cleanup) => cleanup())) +}) + +async function fixture() { + const directory = await mkdtemp(join(tmpdir(), 'brale-release-http-')) + cleanups.push(() => rm(directory, { recursive: true, force: true })) + const files = platforms.map((platform) => ({ + name: `bralecli-${platform}.tar.gz`, + body: Buffer.from(platform), + })) + files.push({ name: 'build-info.json', body: Buffer.from(JSON.stringify({ revision, version })) }) + files.push({ name: 'SHA256SUMS', body: Buffer.from('fixture checksums') }) + await Promise.all(files.map((file) => writeFile(join(directory, file.name), file.body))) + const assets: Record[] = [] + let release: Record | undefined + const writes: { method: string; path: string; body: Buffer }[] = [] + let failUpload = false + let tag: unknown + let annotatedTag: unknown + const server = createServer(async (request, response) => { + const chunks: Buffer[] = [] + for await (const chunk of request) chunks.push(chunk) + const body = Buffer.concat(chunks) + const url = new URL(request.url!, 'http://localhost') + response.setHeader('Content-Type', 'application/json') + if (request.method === 'GET') { + if (url.pathname.includes('/git/ref/tags/')) { + response.statusCode = tag === undefined ? 404 : 200 + response.end(JSON.stringify(tag ?? { message: 'Not Found' })) + return + } + if (url.pathname.includes('/git/tags/')) { + response.end(JSON.stringify(annotatedTag)) + return + } + response.end( + JSON.stringify(url.pathname.endsWith('/releases') ? (release ? [release] : []) : release), + ) + return + } + writes.push({ method: request.method!, path: url.pathname, body }) + if (url.pathname.endsWith('/releases')) { + const data = JSON.parse(body.toString()) + release = { + ...data, + id: 123, + html_url: 'https://github.com/0xsend/bralecli/releases/tag/v0.2.0', + assets, + } + response.statusCode = 201 + response.end(JSON.stringify(release)) + return + } + if (failUpload) { + response.statusCode = 503 + response.end('{}') + return + } + const name = url.searchParams.get('name')! + assets.push({ + id: assets.length + 1, + name, + state: 'uploaded', + size: body.length, + digest: `sha256:${createHash('sha256').update(body).digest('hex')}`, + }) + response.statusCode = 201 + response.end(JSON.stringify(assets.at(-1))) + }) + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)) + cleanups.push( + () => + new Promise((resolve, reject) => + server.close((error) => (error ? reject(error) : resolve())), + ), + ) + const address = server.address() + if (!address || typeof address === 'string') throw new Error('Missing test server') + const endpoint = `http://127.0.0.1:${address.port}` + return { + api: githubApi('synthetic-test-token', { api: endpoint, uploads: endpoint }), + directory, + files, + writes, + assets, + get release() { + return release + }, + set release(value) { + release = value + }, + set failUpload(value: boolean) { + failUpload = value + }, + set tag(value: unknown) { + tag = value + }, + set annotatedTag(value: unknown) { + annotatedTag = value + }, + } +} + +it('creates only a draft, uploads the exact six files and preserves it on retry', async () => { + const test = await fixture() + expect(await inspectRelease(test.api, version)).toMatchObject({ prepare: true }) + await uploadDraft(test.api, { + version, + revision, + directory: test.directory, + notes: 'Reviewed release notes', + }) + expect(test.release).toMatchObject({ + draft: true, + tag_name: 'v0.2.0', + target_commitish: revision, + }) + expect(test.assets.map((asset) => asset.name).toSorted()).toEqual( + test.files.map((file) => file.name).toSorted(), + ) + expect(await inspectRelease(test.api, version)).toMatchObject({ prepare: false }) + const writes = test.writes.length + await uploadDraft(test.api, { + version, + revision, + directory: test.directory, + notes: 'Reviewed release notes', + }) + expect(test.writes).toHaveLength(writes) + expect(test.writes.every((entry) => entry.method === 'POST')).toBe(true) +}) + +it('resumes an incomplete matching draft after a failed upload', async () => { + const test = await fixture() + test.failUpload = true + await expect( + uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }), + ).rejects.toThrow('503') + expect(test.release?.draft).toBe(true) + expect(await inspectRelease(test.api, version)).toMatchObject({ prepare: true }) + test.failUpload = false + await uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }) + expect(test.assets).toHaveLength(6) + expect(test.writes.filter((entry) => entry.path.endsWith('/releases'))).toHaveLength(1) +}) + +it.each(['published', 'revision', 'digest', 'extra'])( + 'rejects a %s conflict before changing any asset', + async (conflict) => { + const test = await fixture() + await uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }) + if (conflict === 'published') test.release!.draft = false + if (conflict === 'revision') test.release!.target_commitish = 'b'.repeat(40) + if (conflict === 'digest') test.assets[0]!.digest = `sha256:${'f'.repeat(64)}` + if (conflict === 'extra') + test.assets.push({ + name: 'unexpected.txt', + digest: `sha256:${'f'.repeat(64)}`, + state: 'uploaded', + size: 1, + }) + const before = test.writes.length + await expect( + uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }), + ).rejects.toThrow() + expect(test.writes).toHaveLength(before) + }, +) + +it('rejects incomplete local artifacts before creating a draft', async () => { + const test = await fixture() + await rm(join(test.directory, 'SHA256SUMS')) + await expect( + uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }), + ).rejects.toThrow() + expect(test.writes).toHaveLength(0) + expect(await readFile(join(test.directory, 'build-info.json'), 'utf8')).toContain(revision) +}) + +it.each(['commit', 'tag'])( + 'rejects an existing %s tag that identifies another source commit', + async (type) => { + const test = await fixture() + test.tag = { object: { type, sha: 'b'.repeat(40) } } + test.annotatedTag = { object: { type: 'commit', sha: 'c'.repeat(40) } } + await expect( + uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }), + ).rejects.toThrow(/tag/i) + expect(test.writes).toHaveLength(0) + }, +) + +it.each(['commit', 'tag'])( + 'accepts an existing %s tag pointing at the exact source commit', + async (type) => { + const test = await fixture() + test.tag = { object: { type, sha: type === 'commit' ? revision : 'b'.repeat(40) } } + test.annotatedTag = { object: { type: 'commit', sha: revision } } + await uploadDraft(test.api, { version, revision, directory: test.directory, notes: 'Notes' }) + expect(test.release?.draft).toBe(true) + expect(test.assets).toHaveLength(6) + }, +) diff --git a/scripts/release-draft.ts b/scripts/release-draft.ts new file mode 100644 index 0000000..672b752 --- /dev/null +++ b/scripts/release-draft.ts @@ -0,0 +1,308 @@ +/** Trusted, builtin-only GitHub adapter. Public publication is deliberately absent. */ +import { createHash } from 'node:crypto' +import { execFileSync } from 'node:child_process' +import { lstat, readFile, readdir } from 'node:fs/promises' +import { join } from 'node:path' + +const repository = '0xsend/bralecli' +const releasePath = `/repos/${repository}/releases` +const platforms = ['darwin-arm64', 'darwin-x64', 'linux-arm64', 'linux-x64'] +const fileNames = [ + ...platforms.map((platform) => `bralecli-${platform}.tar.gz`), + 'build-info.json', + 'SHA256SUMS', +] +type Api = ( + method: 'GET' | 'POST', + path: string, + body?: Buffer | Record, + upload?: boolean, +) => Promise +type Asset = { name: string; size: number; digest: string; state: string } +type Release = { + id: number + tag_name: string + draft: boolean + target_commitish: string + html_url: string + assets: Asset[] +} +type Upload = { name: string; body: Buffer; digest: string } + +// Requests are ordered for pagination and retry reconciliation; file and stream +// reads are serial to keep the documented memory bounds independent of fan-out. +/* oxlint-disable no-await-in-loop */ + +function object(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +export function githubApi( + token: string, + endpoints: { api: string; uploads: string } = { + api: 'https://api.github.com', + uploads: 'https://uploads.github.com', + }, +): Api { + if (!token || /\s/.test(token)) throw new Error('A GitHub token must be provided on stdin') + return async (method, path, body, upload = false) => { + const response = await fetch(`${upload ? endpoints.uploads : endpoints.api}${path}`, { + method, + redirect: 'error', + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/vnd.github+json', + 'X-GitHub-Api-Version': '2022-11-28', + 'Content-Type': upload ? 'application/octet-stream' : 'application/json', + }, + ...(body === undefined + ? {} + : { body: Buffer.isBuffer(body) ? new Uint8Array(body) : JSON.stringify(body) }), + signal: AbortSignal.timeout(upload ? 120_000 : 30_000), + }) + if (!response.ok) { + await response.body?.cancel() + if (method === 'GET' && response.status === 404) return undefined + throw new Error(`GitHub ${method} ${path} failed: HTTP ${response.status}`) + } + const reader = response.body?.getReader() + if (!reader) throw new Error('Missing GitHub response body') + const chunks: Uint8Array[] = [] + let size = 0 + try { + while (true) { + const chunk = await reader.read() + if (chunk.done) break + size += chunk.value.length + if (size > 2 * 1024 * 1024) throw new Error('GitHub response exceeds 2 MiB') + chunks.push(chunk.value) + } + } finally { + await reader.cancel() + } + return JSON.parse(Buffer.concat(chunks).toString('utf8')) as unknown + } +} + +function parseRelease(value: unknown): Release { + if ( + !object(value) || + !Number.isSafeInteger(value.id) || + Number(value.id) <= 0 || + typeof value.tag_name !== 'string' || + typeof value.draft !== 'boolean' || + typeof value.target_commitish !== 'string' || + typeof value.html_url !== 'string' || + !Array.isArray(value.assets) || + value.assets.length > 100 + ) + throw new Error('Invalid GitHub release metadata') + for (const asset of value.assets) { + if ( + !object(asset) || + typeof asset.name !== 'string' || + typeof asset.state !== 'string' || + typeof asset.size !== 'number' || + !Number.isSafeInteger(asset.size) || + asset.size < 0 || + (asset.digest !== null && typeof asset.digest !== 'string') + ) + throw new Error('Invalid GitHub asset metadata') + } + return value as unknown as Release +} + +async function findRelease(api: Api, version: string): Promise { + if (!/^\d+\.\d+\.\d+$/.test(version)) throw new Error('Expected a stable release version') + // Tag lookup omits drafts. List authenticated releases, with a fail-closed cap. + for (let page = 1; page <= 20; page++) { + const result = await api('GET', `${releasePath}?per_page=100&page=${page}`) + if (!Array.isArray(result)) throw new Error('Expected GitHub release list') + const matching = result.filter((value) => object(value) && value.tag_name === `v${version}`) + if (matching.length > 1) throw new Error('Multiple releases share the requested version') + if (matching.length) return parseRelease(matching[0]) + if (result.length < 100) return undefined + } + throw new Error('Release search exceeds 2,000 releases') +} + +async function verifyTag(api: Api, version: string, revision: string): Promise { + let tag = await api('GET', `/repos/${repository}/git/ref/tags/v${version}`) + // GitHub creates an absent tag only when a draft is published. Existing tags + // override target_commitish, so resolve them independently of release metadata. + if (tag === undefined) return + for (let depth = 0; depth < 8; depth++) { + if ( + !object(tag) || + !object(tag.object) || + typeof tag.object.sha !== 'string' || + !/^[a-f0-9]{40}$/.test(tag.object.sha) + ) + throw new Error('Invalid release tag metadata') + if (tag.object.type === 'commit') { + if (tag.object.sha !== revision) + throw new Error('Release tag points to a different source commit') + return + } + if (tag.object.type !== 'tag') throw new Error('Release tag does not identify a commit') + tag = await api('GET', `/repos/${repository}/git/tags/${tag.object.sha}`) + } + throw new Error('Release tag exceeds eight levels of indirection') +} + +export async function inspectRelease( + api: Api, + version: string, +): Promise<{ prepare: boolean; url?: string }> { + const release = await findRelease(api, version) + if (!release) return { prepare: true } + const complete = + release.assets.length === fileNames.length && + fileNames.every((name) => + release.assets.some( + (asset) => + asset.name === name && + asset.state === 'uploaded' && + /^sha256:[a-f0-9]{64}$/.test(asset.digest), + ), + ) + return { prepare: release.draft && !complete, url: release.html_url } +} + +async function localUploads(directory: string): Promise { + if ((await lstat(directory)).isSymbolicLink()) + throw new Error('Release directory cannot be a symlink') + const entries = await readdir(directory) + if (entries.length !== fileNames.length || entries.some((name) => !fileNames.includes(name))) + throw new Error('Release output must contain exactly the six expected assets') + const uploads: Upload[] = [] + for (const name of fileNames) { + const path = join(directory, name) + const stat = await lstat(path) + if (!stat.isFile() || stat.size > 128 * 1024 * 1024) + throw new Error(`Invalid release output: ${name}`) + const body = await readFile(path) + if (body.length !== stat.size) throw new Error(`Release output changed while reading: ${name}`) + uploads.push({ + name, + body, + digest: `sha256:${createHash('sha256').update(body).digest('hex')}`, + }) + } + return uploads +} + +function reconcile(release: Release, uploads: Upload[]): Upload[] { + const remaining = new Map(uploads.map((upload) => [upload.name, upload])) + for (const asset of release.assets) { + const upload = remaining.get(asset.name) + if ( + !upload || + asset.state !== 'uploaded' || + asset.digest !== upload.digest || + asset.size !== upload.body.length + ) + throw new Error(`Existing release asset conflicts: ${asset.name}`) + remaining.delete(asset.name) + } + return [...remaining.values()] +} + +export async function uploadDraft( + api: Api, + input: { + version: string + revision: string + directory: string + notes: string + }, +): Promise<{ url: string; draft: true; assets: number }> { + if (!/^[a-f0-9]{40}$/.test(input.revision)) throw new Error('Expected an exact source commit') + const uploads = await localUploads(input.directory) + let release = await findRelease(api, input.version) + if (release && (!release.draft || release.target_commitish !== input.revision)) + throw new Error('Existing release is published or belongs to a different source revision') + await verifyTag(api, input.version, input.revision) + if (!release) { + release = parseRelease( + await api('POST', releasePath, { + tag_name: `v${input.version}`, + target_commitish: input.revision, + name: `bralecli v${input.version}`, + body: input.notes, + draft: true, + prerelease: false, + }), + ) + } + for (const upload of reconcile(release, uploads)) { + await api( + 'POST', + `${releasePath}/${release.id}/assets?name=${encodeURIComponent(upload.name)}`, + upload.body, + true, + ) + } + const verified = parseRelease(await api('GET', `${releasePath}/${release.id}`)) + await verifyTag(api, input.version, input.revision) + if ( + !verified.draft || + verified.target_commitish !== input.revision || + reconcile(verified, uploads).length + ) + throw new Error('Draft verification failed after uploading') + return { url: verified.html_url, draft: true, assets: verified.assets.length } +} + +async function main(): Promise { + const [command, inputDirectory, outputDirectory, ...extra] = process.argv.slice(2) + if ( + extra.length || + !['inspect', 'upload'].includes(command ?? '') || + (command === 'inspect' && inputDirectory !== undefined) || + (command === 'upload' && (!inputDirectory || !outputDirectory)) + ) + throw new Error( + 'Usage: release-draft.ts inspect | upload (GitHub token on stdin)', + ) + const tokenChunks: Buffer[] = [] + let tokenSize = 0 + for await (const chunk of process.stdin) { + tokenSize += chunk.length + if (tokenSize > 4096) throw new Error('Token input exceeds limit') + tokenChunks.push(chunk) + } + const api = githubApi(Buffer.concat(tokenChunks).toString('utf8').trim()) + const manifest = JSON.parse(await readFile('apps/cli/package.json', 'utf8')) as { + version: string + } + const version = manifest.version + const revision = execFileSync('git', ['rev-parse', 'HEAD'], { + encoding: 'utf8', + timeout: 10_000, + }).trim() + if (command === 'inspect') { + console.log(JSON.stringify({ version, revision, ...(await inspectRelease(api, version)) })) + return + } + const { verifyArtifacts } = await import('./release-artifacts.ts') + await verifyArtifacts(inputDirectory!, revision, version, outputDirectory!) + const changelog = await readFile('apps/cli/CHANGELOG.md', 'utf8') + const section = changelog.split(`\n## ${version}\n`) + if (section.length !== 2) throw new Error(`Missing unique changelog entry for ${version}`) + const notes = + section[1]!.split('\n## ')[0]!.trim() + + '\n\nDownload the archive matching your operating system and processor. Verify it against SHA256SUMS before installation. build-info.json records the source revision and archive digests.\n' + console.log( + JSON.stringify( + await uploadDraft(api, { version, revision, directory: outputDirectory!, notes }), + ), + ) +} + +if (import.meta.main) { + main().catch((error: unknown) => { + console.error(error instanceof Error ? error.message : 'Release preparation failed') + process.exitCode = 1 + }) +} diff --git a/scripts/repository-policy.test.ts b/scripts/repository-policy.test.ts index da8b5e7..9230712 100644 --- a/scripts/repository-policy.test.ts +++ b/scripts/repository-policy.test.ts @@ -45,13 +45,17 @@ describe('repository automation trust boundaries', () => { expect(config.permissions).toEqual({ contents: 'read' }) expect(Object.keys(config.jobs).length).toBeGreaterThan(0) for (const [id, job] of Object.entries(config.jobs)) { - expect(job['runs-on']).toBe('ubuntu-latest') + expect(job['runs-on']).toBe( + name === 'release' && id === 'build' ? '${{ matrix.runner }}' : 'ubuntu-latest', + ) expect(job['timeout-minutes']).toBeGreaterThan(0) expect(job['timeout-minutes']).toBeLessThanOrEqual(30) expect(job.permissions ?? config.permissions).toEqual( name === 'update-spec' && id === 'pull-request' ? { contents: 'write', 'pull-requests': 'write' } - : { contents: 'read' }, + : name === 'release' && id === 'draft' + ? { contents: 'write' } + : { contents: 'read' }, ) for (const step of job.steps) { if (!step.uses) continue @@ -160,7 +164,7 @@ describe('repository automation trust boundaries', () => { } }) - it('only the spec writer can publish and it handles two files without executing the artifact', () => { + it('the spec writer proposes four trusted paths without executing the artifact', () => { const job = workflow('update-spec').jobs['pull-request']! const proposal = job.steps.find((step) => step.uses?.startsWith('peter-evans/create-pull-request@'), @@ -169,7 +173,7 @@ describe('repository automation trust boundaries', () => { expect(proposal?.with?.base).toBe('main') expect(proposal?.with?.draft).toBe('always-true') expect(proposal?.with?.['add-paths']).toBe( - 'packages/brale/openapi/brale.json\npackages/brale/src/spec.ts\n', + 'packages/brale/openapi/brale.json\npackages/brale/src/spec.ts\napps/cli/package.json\napps/cli/CHANGELOG.md\n', ) expect(proposal?.with?.['branch-suffix']).toBeUndefined() expect(proposal?.with?.['maintainer-can-modify']).toBe(false) @@ -187,4 +191,55 @@ describe('repository automation trust boundaries', () => { expect(commands).not.toMatch(/\b(?:nub|npm|npx|bun|source|eval)\b|gh\s+pr\s+(merge|review)/) expect(proposal?.with?.body).toContain('${{ needs.compatibility.result }}') }) + + it('release preparation is confined to main with four native jobs before its isolated draft writer', () => { + const config = workflow('release') + expect(Object.keys(config.on).toSorted()).toEqual(['push', 'workflow_dispatch']) + expect(config.on.push).toEqual({ branches: ['main'] }) + expect(config.jobs.candidate?.if).toBe( + "github.repository == '0xsend/bralecli' && github.ref == 'refs/heads/main'", + ) + expect(config.concurrency).toEqual({ + group: 'release-preparation', + 'cancel-in-progress': false, + }) + expect(config.jobs.build?.needs).toBe('candidate') + expect(config.jobs.build?.if).toBe("needs.candidate.outputs.prepare == 'true'") + const raw = parse( + readFileSync(new URL('../.github/workflows/release.yml', import.meta.url), 'utf8'), + ) + expect(raw.jobs.build.strategy.matrix.include).toEqual([ + { platform: 'darwin-arm64', runner: 'macos-15' }, + { platform: 'darwin-x64', runner: 'macos-15-intel' }, + { platform: 'linux-arm64', runner: 'ubuntu-24.04-arm' }, + { platform: 'linux-x64', runner: 'ubuntu-24.04' }, + ]) + const build = config.jobs.build!.steps.map((step) => step.run ?? '').join('\n') + expect(build).toContain('nub ci --ignore-scripts') + expect(build).toContain('nub run check') + expect(build).toContain('scripts/smoke-binary.py') + expect(build).toContain('scripts/release-artifacts.ts package') + const job = config.jobs.draft! + expect(job.needs).toEqual(['candidate', 'build']) + expect(job.permissions).toEqual({ contents: 'write' }) + expect(job.steps.map((step) => step.run ?? '').join('\n')).toContain( + 'scripts/release-draft.ts upload', + ) + expect(job.steps.map((step) => step.run ?? '').join('\n')).not.toMatch( + /\b(?:npm|nub|bun|npx|python3)\b|--draft=false/, + ) + const download = job.steps.find((step) => step.uses?.startsWith('actions/download-artifact@')) + expect(download?.with?.pattern).toBe('release-${{ github.run_id }}-*') + const upload = config.jobs.build!.steps.find((step) => + step.uses?.startsWith('actions/upload-artifact@'), + ) + expect(upload?.with?.name).toBe('release-${{ github.run_id }}-${{ matrix.platform }}') + expect(upload?.with?.overwrite).toBe(true) + expect(download?.with?.['run-id']).toBeUndefined() + for (const current of Object.values(config.jobs)) { + expect( + current.steps.find((step) => step.uses?.startsWith('actions/checkout@'))?.with?.ref, + ).toBe('${{ github.sha }}') + } + }) }) diff --git a/scripts/spec-release.test.ts b/scripts/spec-release.test.ts new file mode 100644 index 0000000..f5273c7 --- /dev/null +++ b/scripts/spec-release.test.ts @@ -0,0 +1,139 @@ +import { readFile } from 'node:fs/promises' + +import { describe, expect, it } from 'vitest' + +import { planSpecRelease } from './spec-release.ts' + +const base = JSON.parse( + await readFile(new URL('../packages/brale/openapi/brale.json', import.meta.url), 'utf8'), +) +const packageSource = '{\n "name": "bralecli",\n "version": "0.2.7",\n "private": true\n}\n' +const changelogSource = '# bralecli\n\n## 0.2.7\n\n- Existing release.\n' +const revision = { hash: 'a'.repeat(64), fetchedAt: '2026-09-08' } +const path = '/accounts/{account_id}/financial-institutions/{fi_id}/status' + +function plan(document: unknown, previousDocument: unknown = base, version = '0.2.7') { + return planSpecRelease({ + previousDocument: Buffer.from(JSON.stringify(previousDocument)), + document: Buffer.from(JSON.stringify(document)), + packageSource: packageSource.replace('0.2.7', version), + changelogSource, + revision, + }) +} + +describe('reviewable spec release proposals', () => { + it('ignores formatting, object order, and OpenAPI documentation annotations', () => { + const document = structuredClone(base) + document.info.description = 'Reworded overview' + document.paths[path].get.summary = 'Clearer command summary' + document.paths[path].get.parameters[0].description = 'Clearer parameter text' + document.paths[path].get.responses['200'].description = 'Clearer response text' + document.paths[path].get.responses['200'].content['application/json'].examples = { + sample: { value: { status: 'active' } }, + } + document.components.schemas.UpdateExternalAddressRequest.properties.name.example = 'Example' + document.components.schemas.UpdateExternalAddressRequest.description = 'Description' + document.components.examples = { sample: { value: { status: 'active' } } } + expect(plan(document)).toEqual({ kind: 'none' }) + expect( + planSpecRelease({ + previousDocument: Buffer.from(JSON.stringify(base)), + document: Buffer.from( + JSON.stringify(Object.fromEntries(Object.entries(base).toReversed()), null, 2), + ), + packageSource, + changelogSource, + revision, + }), + ).toEqual({ kind: 'none' }) + }) + + it.each(['description', 'summary', 'example', 'examples'])( + 'retains schema property and component names called %s', + (name) => { + const document = structuredClone(base) + document.components.schemas.UpdateExternalAddressRequest.properties[name] = { type: 'string' } + expect(plan(document).kind).toBe('release') + const changed = structuredClone(document) + changed.components.schemas.UpdateExternalAddressRequest.properties[name].type = 'integer' + expect(plan(changed, document).kind).toBe('release') + document.components.schemas[name] = { type: 'integer' } + expect(plan(document).kind).toBe('release') + }, + ) + + it.each(['default', 'const', 'enum', 'x-runtime-policy', 'constructor', '__proto__'])( + 'retains arbitrary values under %s', + (key) => { + const document = structuredClone(base) + document.components.schemas.UpdateExternalAddressRequest = { + ...document.components.schemas.UpdateExternalAddressRequest, + [key]: key === 'enum' ? [{ description: 'wire' }] : { description: 'wire' }, + } + const changed = structuredClone(document) + changed.components.schemas.UpdateExternalAddressRequest = { + ...changed.components.schemas.UpdateExternalAddressRequest, + [key]: key === 'enum' ? [{ description: 'ach' }] : { description: 'ach' }, + } + expect(plan(changed, document).kind).toBe('release') + }, + ) + + it.each(['operation', 'constraint', 'schema-format', 'security'])( + 'proposes a conservative 0.x minor for a changed %s', + (change) => { + const document = structuredClone(base) + if (change === 'operation') + document.paths[path].get.operationId = 'read_financial_institution_status' + if (change === 'constraint') + document.components.schemas.UpdateExternalAddressRequest.additionalProperties = true + if (change === 'schema-format') + document.components.schemas.UpdateExternalAddressRequest.properties.name.format = 'uuid' + if (change === 'security') document.paths[path].get.security[0].oauth.push('accounts:read') + const result = plan(document) + expect(result).toMatchObject({ kind: 'release', previousVersion: '0.2.7', version: '0.3.0' }) + if (result.kind !== 'release') throw new Error('Expected a release proposal') + expect(JSON.parse(result.packageSource)).toEqual({ + name: 'bralecli', + version: '0.3.0', + private: true, + }) + expect(result.changelogSource).toContain('## 0.3.0\n') + expect(result.changelogSource).toContain('### Minor Changes\n') + expect(result.changelogSource).toContain(revision.hash) + expect(result.changelogSource).toContain('2026-09-08') + expect(result.changelogSource).toContain('## 0.2.7\n\n- Existing release.\n') + expect(plan(document)).toEqual(result) + }, + ) + + it('proposes a major after 1.0 instead of claiming a contract change is nonbreaking', () => { + const document = structuredClone(base) + delete document.paths[path] + expect(plan(document, base, '2.8.9')).toMatchObject({ kind: 'release', version: '3.0.0' }) + }) + + it.each(['latest', '0.2.7-rc.1', '9007199254740991.0.0'])( + 'rejects unsafe or unsupported version %s', + (version) => { + const document = structuredClone(base) + delete document.paths[path] + expect(() => plan(document, base, version)).toThrow('version') + }, + ) + + it('rejects excessive JSON depth with a bounded diagnostic', () => { + let nested: unknown = { type: 'string' } + for (let depth = 0; depth < 140; depth++) nested = { items: nested } + const document = structuredClone(base) + document.components.schemas.Deep = nested + expect(() => plan(document)).toThrow('depth limit') + }) + + it('rejects excessively broad JSON within the document byte limit', () => { + const document = structuredClone(base) + document.components.schemas.Wide = { enum: Array.from({ length: 1_000_000 }, () => 0) } + expect(() => plan(document)).toThrow('node limit') + }) +}) diff --git a/scripts/spec-release.ts b/scripts/spec-release.ts new file mode 100644 index 0000000..3546912 --- /dev/null +++ b/scripts/spec-release.ts @@ -0,0 +1,216 @@ +/** Pure, builtin-only release planning for already validated OpenAPI proposals. */ +import { createHash } from 'node:crypto' + +import { isObject, parseJson, type Revision } from './spec-proposal.ts' + +type ObjectContext = + | 'document' + | 'info' + | 'components' + | 'schema' + | 'pathItem' + | 'operation' + | 'parameter' + | 'header' + | 'response' + | 'requestBody' + | 'mediaType' + | 'encoding' + | 'securityScheme' + | 'server' + | 'serverVariable' + | 'tag' + | 'link' + | 'callback' +type Context = ObjectContext | `map:${ObjectContext}` | 'data' +type Rule = { omit: readonly string[]; children?: Record } +type ReleasePlan = + | { kind: 'none' } + | { + kind: 'release' + previousVersion: string + version: string + packageSource: string + changelogSource: string + } + +const rules: Record = { + document: { + omit: ['externalDocs'], + children: { + info: 'info', + paths: 'map:pathItem', + webhooks: 'map:pathItem', + components: 'components', + servers: 'server', + tags: 'tag', + }, + }, + info: { omit: ['description', 'summary'] }, + components: { + omit: ['examples'], + children: { + schemas: 'map:schema', + responses: 'map:response', + parameters: 'map:parameter', + requestBodies: 'map:requestBody', + headers: 'map:header', + securitySchemes: 'map:securityScheme', + links: 'map:link', + callbacks: 'map:callback', + pathItems: 'map:pathItem', + }, + }, + schema: { + omit: ['description', 'title', 'example', 'examples', '$comment', 'externalDocs'], + children: { + properties: 'map:schema', + patternProperties: 'map:schema', + $defs: 'map:schema', + definitions: 'map:schema', + dependentSchemas: 'map:schema', + items: 'schema', + prefixItems: 'schema', + additionalItems: 'schema', + contains: 'schema', + additionalProperties: 'schema', + unevaluatedProperties: 'schema', + unevaluatedItems: 'schema', + propertyNames: 'schema', + allOf: 'schema', + anyOf: 'schema', + oneOf: 'schema', + not: 'schema', + }, + }, + pathItem: { + omit: ['description', 'summary'], + children: { + get: 'operation', + put: 'operation', + post: 'operation', + delete: 'operation', + options: 'operation', + head: 'operation', + patch: 'operation', + trace: 'operation', + parameters: 'parameter', + servers: 'server', + }, + }, + operation: { + omit: ['description', 'summary', 'externalDocs'], + children: { + parameters: 'parameter', + requestBody: 'requestBody', + responses: 'map:response', + callbacks: 'map:callback', + servers: 'server', + }, + }, + parameter: { + omit: ['description', 'summary', 'example', 'examples'], + children: { schema: 'schema', content: 'map:mediaType' }, + }, + header: { + omit: ['description', 'summary', 'example', 'examples'], + children: { schema: 'schema', content: 'map:mediaType' }, + }, + response: { + omit: ['description', 'summary'], + children: { headers: 'map:header', content: 'map:mediaType', links: 'map:link' }, + }, + requestBody: { omit: ['description', 'summary'], children: { content: 'map:mediaType' } }, + mediaType: { + omit: ['example', 'examples'], + children: { schema: 'schema', encoding: 'map:encoding' }, + }, + encoding: { omit: [], children: { headers: 'map:header' } }, + securityScheme: { omit: ['description', 'summary'] }, + server: { omit: ['description'], children: { variables: 'map:serverVariable' } }, + serverVariable: { omit: ['description'] }, + tag: { omit: ['description', 'externalDocs'] }, + link: { omit: ['description', 'summary'], children: { server: 'server' } }, + callback: { omit: [] }, +} + +function ruleFor(context: Context): Rule | undefined { + return context === 'data' || context.startsWith('map:') + ? undefined + : rules[context as ObjectContext] +} + +function childContext(context: Context, key: string): Context { + if (context.startsWith('map:')) return context.slice(4) as ObjectContext + if (context === 'callback') return 'pathItem' + if (context === 'schema' && ['if', 'then', 'else'].includes(key)) return 'schema' + const children = ruleFor(context)?.children + return children && Object.hasOwn(children, key) ? children[key]! : 'data' +} + +function contractHash(body: Buffer): string { + const digest = createHash('sha256') + let nodes = 0 + function visit(value: unknown, context: Context, depth: number): void { + if (depth > 128) throw new Error('OpenAPI comparison exceeds its 128 level depth limit') + if (++nodes > 1_000_000) throw new Error('OpenAPI comparison exceeds its node limit') + if (Array.isArray(value)) { + digest.update('[') + for (const item of value) { + visit(item, context, depth + 1) + digest.update(',') + } + digest.update(']') + } else if (isObject(value)) { + digest.update('{') + for (const key of Object.keys(value).toSorted()) { + // Only OpenAPI annotations are ignored. Map keys and arbitrary payload + // data (defaults, enums, security scopes, extensions) remain significant. + if (ruleFor(context)?.omit.includes(key)) continue + digest.update(JSON.stringify(key)).update(':') + visit(value[key], childContext(context, key), depth + 1) + digest.update(',') + } + digest.update('}') + } else { + digest.update(JSON.stringify(value)) + } + } + visit(parseJson(body, 'OpenAPI document'), 'document', 0) + return digest.digest('hex') +} + +export function planSpecRelease(input: { + previousDocument: Buffer + document: Buffer + packageSource: string + changelogSource: string + revision: Revision +}): ReleasePlan { + if (contractHash(input.previousDocument) === contractHash(input.document)) return { kind: 'none' } + const manifest = parseJson(Buffer.from(input.packageSource), 'CLI package manifest') + if (!isObject(manifest) || manifest.name !== 'bralecli' || typeof manifest.version !== 'string') + throw new Error('Trusted CLI manifest must declare bralecli and a version') + const match = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/.exec(manifest.version) + if (!match) throw new Error('CLI version must be a stable major.minor.patch version') + const [major, minor, patch] = match.slice(1).map(Number) + if ( + ![major, minor, patch].every( + (part) => Number.isSafeInteger(part) && part! < Number.MAX_SAFE_INTEGER, + ) + ) + throw new Error('CLI version exceeds safe integer bounds') + const version = major === 0 ? `0.${minor! + 1}.0` : `${major! + 1}.0.0` + if (!input.changelogSource.startsWith('# bralecli\n')) + throw new Error('Trusted CLI changelog must start with # bralecli') + if (input.changelogSource.includes(`\n## ${version}\n`)) + throw new Error(`CLI changelog already contains proposed version ${version}`) + const entry = `\n## ${version}\n\n### ${major === 0 ? 'Minor' : 'Major'} Changes\n\n- Refresh the generated CLI for the Brale OpenAPI contract fetched on ${input.revision.fetchedAt}\n (SHA-256: \`${input.revision.hash}\`).\n- Propose a conservative ${major === 0 ? 'pre-1.0 minor' : 'major'} release because upstream contract changes may be breaking.\n Maintainers review the contract diff and may adjust this version and release note before merging.\n` + return { + kind: 'release', + previousVersion: manifest.version, + version, + packageSource: `${JSON.stringify({ ...manifest, version }, null, 2)}\n`, + changelogSource: `# bralecli\n${entry}${input.changelogSource.slice('# bralecli\n'.length)}`, + } +} diff --git a/scripts/tsconfig.json b/scripts/tsconfig.json new file mode 100644 index 0000000..12cc6c9 --- /dev/null +++ b/scripts/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2023", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "noEmit": true, + "allowImportingTsExtensions": true, + "strict": true, + "noUncheckedIndexedAccess": true, + "types": ["node"] + }, + "include": ["*.ts"], + "exclude": ["*.test.ts"] +}