Skip to content

ci: run tests and tag-only publish through the shared workflows - #120

Open
singlerider wants to merge 10 commits into
devfrom
ci/publish-on-tag
Open

singlerider wants to merge 10 commits into
devfrom
ci/publish-on-tag

Conversation

@singlerider

@singlerider singlerider commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Moves blasthttp CI onto the org's shared workflows and publishes only on a human-pushed version tag.

Shared workflows

Calls the reusable workflows in blacklanternsecurity/CLA, pinned to e532142ee9e7322888f6edc80d9a89e2f8c0d96d (branch feat/shared-workflows, blacklanternsecurity/CLA#3). No repo-local copies of test, wheel, crates, release, or SBOM steps remain.

  • tests.yml (pull_request, push to stable, workflow_call) calls rust-tests.yml (fmt, clippy, cargo test on the rust-toolchain channel) and python-tests.yml (ruff, then pytest on every CPython minor satisfying requires-python, maturin develop detected from the build backend).
  • publish.yml (v* tags only) calls release-check.yml (refuses anything but vMAJOR.MINOR.PATCH / vMAJOR.MINOR.PATCH-rc.N and any tag differing from the Cargo.toml version), tests.yml, maturin-wheels.yml (interpreters from requires-python, no more hardcoded -i list), crates.yml, then publish.yml (GitHub release, SPDX SBOM, wheels attached, --prerelease for rc tags, same run). PyPI stays a top-level job because trusted publishing does not match reusable workflows.
  • Nothing publishes or tags on a branch push. No workflow runs git tag.
  • README ## Releasing documents the procedure and links the shared release.sh.
  • Vendored OpenSSL: scripts/build-openssl.sh is the setup script for tests and crates; the wheel container setup lives in scripts/wheel-container-setup.sh. Wheels build for the linux and musllinux families, as before.
  • pyproject.toml reads its version from Cargo.toml, lets uv manage the project, and adds maturin to the dev group so the shared python workflow can sync and build.
  • release.toml (cargo-release) removed: one release tool, the shared release.sh.
  • README examples ruff formatted (enforced by ruff format --check).
  • CLA calls the shared reusable workflow.

Required checks for governance: rust / passed, python / passed.

Admin steps

  • Register a PyPI trusted publisher for workflow publish.yml, environment pypi, then delete PYPI_API_TOKEN.

Validation

$ uvx --from actionlint-py==1.7.12.24 actionlint .github/workflows/*.yml
(no findings)
$ grep -rnE 'uses: [^.].*@' .github | grep -vE '@[0-9a-f]{40}'
(none)
$ grep -rn 'ubuntu-latest\|actions-rs' .github
(none)
$ cargo clippy --all-targets --all-features --locked -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.42s
$ cargo test --all-features --locked
test result: ok. 223 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.00s
$ uv run pytest
162 passed in 17.58s
$ gh pr checks 120 -R blacklanternsecurity/blasthttp
CLAAssistant	pass
python / lint	pass
python / matrix / read	pass
python / passed	pass
python / test (3.10)	pass
python / test (3.11)	pass
python / test (3.12)	pass
python / test (3.13)	pass
python / test (3.14)	pass
rust / lint	pass
rust / passed	pass
rust / test	pass

Closes blacklanternsecurity/bbot-enterprise#139, blacklanternsecurity/bbot-enterprise#140, blacklanternsecurity/bbot-enterprise#149, blacklanternsecurity/bbot-enterprise#155, blacklanternsecurity/bbot-enterprise#156 (blasthttp part)

Move PyPI (maturin) and crates.io publishing out of python-tests.yml
and rust-tests.yml into publish.yml, triggered only by v* tags.

- version job fails unless the tag equals v + the Cargo.toml version
- test workflows run via workflow_call; every build job gates every
  registry upload so a release is never partial
- drop --skip-existing and registry already-published probes
- pyproject reads its version from Cargo via dynamic = ["version"]
- concurrency groups use a fixed per-file prefix since github.workflow
  is the caller's name under workflow_call
- pin every action to a full SHA, replace -latest runners

Part of blacklanternsecurity/bbot-enterprise#139
Part of blacklanternsecurity/bbot-enterprise#149
Part of blacklanternsecurity/bbot-enterprise#156
The publish workflow now refuses any tag that is not vMAJOR.MINOR.PATCH
or vMAJOR.MINOR.PATCH-rc.N before comparing it to Cargo.toml, so the
org tag convention is enforced rather than implied.

PyPI upload drops PYPI_API_TOKEN and relies on trusted publishing via
the job's id-token permission. Registry jobs run in the release
environment so the publish gate applies.

A final job creates the GitHub Release for the tag in the same run and
attaches an SPDX JSON SBOM from anchore/sbom-action, so a tag and its
release always exist together.

CLA calls the shared reusable workflow instead of carrying a fork.
Governance declares one gated environment named pypi across published
repos. crates and docker jobs stay in release.
@singlerider
singlerider requested a review from en0f October 3, 2026 13:59
@singlerider singlerider self-assigned this Oct 3, 2026
rust-tests.yml, python-tests.yml and the inline wheel matrix, crates
upload and release steps duplicated what every PyO3 repo carries, and
the wheel builds hardcoded the interpreter list. tests.yml now calls the
shared rust-tests.yml and python-tests.yml in blacklanternsecurity/CLA
with scripts/build-openssl.sh as setup. publish.yml calls
release-check.yml, maturin-wheels.yml (Linux families only, the
container OpenSSL build now lives in scripts/wheel-container-setup.sh),
crates.yml and publish.yml, keeping only the trusted-publishing PyPI
upload at top level.

pyproject drops managed = false and adds maturin to the dev group so
the shared workflow can uv sync and maturin develop. release.toml is
removed: cargo-release competed with the one release cut tool. README
examples are ruff formatted, which ruff format --check enforces.
Completes the previous commit, which removed the old test workflows but
missed the files replacing them. tests.yml calls the shared rust and
python test workflows with scripts/build-openssl.sh as setup. publish.yml
calls release-check, maturin-wheels (Linux families, container OpenSSL
build in scripts/wheel-container-setup.sh), crates and publish, keeping
only the trusted-publishing PyPI upload at top level. pyproject lets uv
manage the project and adds maturin to the dev group. README examples
are ruff formatted.
Releases are a pushed tag cut with the shared release.sh, which checks
the tag against the manifest. State that here instead of copying the
script.
@singlerider singlerider changed the title ci: publish only on version tags ci: run tests and tag-only publish through the shared workflows Oct 3, 2026

@en0f en0f left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. PRs into stable will be stuck. The default ruleset on stable requires checks named rust-tests and python-tests (3.10) through (3.14). This PR deletes the workflows that produce those checks. The new shared workflows report under different names (rust / passed, python / passed, python / test (3.x), and so on). Once this reaches dev, the next dev → stable PR will wait forever on checks that never run. Before merging, change the required checks in that ruleset to rust / passed and python / passed. The dev ruleset has no required checks, so this PR itself can merge.

  2. The pins point at a branch that hasn't been merged. publish.yml and tests.yml pin blacklanternsecurity/CLA@e532142…, a commit on feat/shared-workflows. That CLA PR is still open. If it's squash-merged and the branch is deleted, that commit may stop being fetchable, and tests and releases would fail to resolve the workflows. Merge the CLA PR first, then update the pins (and the README link) to the matching commit on CLA main.

Check before the first tag: the PyPI job now publishes with trusted publishing (environment: pypi) instead of PYPI_API_TOKEN. Unless PyPI already has a trusted publisher registered for blacklanternsecurity/blasthttp, workflow publish.yml, environment pypi, the first release will fail at the PyPI step. I can't see PyPI settings from here, so I couldn't confirm it.

@liquidsec liquidsec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not remove the rust tests here. They have highly customized tests that will not apply to any other repository (for example, custom openssl build)

Restore rust-tests.yml from dev and drop the shared rust job from
tests.yml. The Rust job builds a custom OpenSSL via
scripts/build-openssl.sh, caches it, and runs fmt, clippy and tests
in a way that is specific to this repo. The rust-tests job name is
kept since it is a required status check.

The rust-publish job stays removed: crates.io publishing now runs
only on version tags via publish.yml.
rust-tests.yml is now callable and publish.yml runs it alongside
tests.yml before building wheels, so a tag cannot release a crate or
wheel whose Rust tests, clippy, or fmt fail.

The concurrency group is named explicitly and only cancels on pull
requests, matching tests.yml. Inherited github.workflow would resolve
to the caller's name and collide with the publish run's own group.
@singlerider
singlerider requested review from en0f and liquidsec October 6, 2026 14:33
CLA#3 was squash-merged into main. The pinned workflow files and
scripts/release.sh are byte-identical at de82726743e51ac51a1be31196d893312a9fea59.

@en0f en0f left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found two blocking issues: one is already live, the other will hit as soon as Python 3.15.0 ships. Everything else checks out.

  1. PRs into stable will hang (still open from your earlier review). The default ruleset on stable still requires python-tests (3.10) through (3.14). This PR deletes python-tests.yml, and the shared workflow reports those results under different names (python / test (3.x) and python / passed), so the old check names never appear. Every dev → stable PR would wait on them forever. rust-tests is fine because the repo's own workflow is kept. Fix: swap those five required checks for python / passed before this lands.

  2. Python 3.15 will break PR CI and the wheel builds.

How: The shared python-versions.yml picks the Python versions from requires-python = ">=3.10" and whatever versions uv can download. Once uv ships 3.15.0 final, 3.15 joins both the test matrix and the wheel -i list.
Why it fails: The locked PyO3 (pyo3-ffi 0.27.2) caps CPython at 3.14. Both old workflows set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to get past that cap, and the new calls don't pass it.
What breaks: python / test (3.15) fails, which fails python / passed on every PR, and the publish run fails at wheels.
Timing: It isn't failing yet, because the latest uv (0.12.23) still lists only 3.15.0rc3, which the shared script skips.
Fix: In tests.yml and in the wheels job of publish.yml, pass env: '{"PYO3_USE_ABI3_FORWARD_COMPATIBILITY": "1"}'. Bumping PyO3 to a release that supports 3.15 would also work.

The locked pyo3-ffi 0.27.2 caps CPython at 3.14. The removed
workflows set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1, and the shared
python-tests and maturin-wheels callers did not, so 3.15 would fail
the test matrix and the wheel builds once uv ships it.

Pass the variable through the env input of both calls.
@singlerider
singlerider requested a review from en0f October 6, 2026 18:31

@en0f en0f left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, but @singlerider see comment on repo settings:

PRs into stable will hang. I checked the live default ruleset on stable. It still requires these checks:

rust-tests (still produced, because the repo keeps its own workflow)
python-tests (3.10) through python-tests (3.14)

This PR deletes python-tests.yml. The shared workflow reports under python / test (3.x) and python / passed instead. Once this reaches dev, the next dev → stable PR will wait forever on five checks that never run. The fix is in repo settings, not the code: swap those five required checks for python / passed before or right as this merges. The dev ruleset has no required checks, so this PR itself can merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants