Skip to content

ci: test and SBOM against requirements.lock — the versions the image ships - #146

Open
MrChengLen wants to merge 1 commit into
mainfrom
pr-ci-lockfile-parity
Open

MrChengLen wants to merge 1 commit into
mainfrom
pr-ci-lockfile-parity

Conversation

@MrChengLen

Copy link
Copy Markdown
Owner

What & why

The Docker image installs requirements.lock (pip install --require-hashes). Four workflows still installed requirements.txt, whose >= ranges resolve to the newest PyPI releases, so CI tested (and the SBOM described) versions the image does not ship. This PR fixes the three findings from the #137/#139 reviews, plus the same defect in the veraPDF gate.

Workflow Before After
ci.yml → lint-and-test pip install -r requirements-dev.txt (unpinned; SQLAlchemy 2.1.0 in CI vs 2.0.52 in the image) same, plus -c with the lockfile's pins (hashes stripped)
new deps-latest.yml — the old unpinned install + pytest, weekly (Mon) + on demand, gates nothing
sbom.yml, release.yml SBOM of the runner's Python after pip install -r requirements.txt + the generator SBOM of a fresh venv installed the Dockerfile's way; generator outside it
verapdf.yml pip install -r requirements.txt pip install --require-hashes -r requirements.lock
docker.yml committed CRLF (API commit, PR #129) LF, no content change

Details

  • Why the hashes are stripped for -c: pip turns on --require-hashes for the whole install as soon as a single constraint carries a hash, and the dev tools are unhashed (reproduced with pip 26.2.1: "In --require-hashes mode, all requirements must have their versions pinned with =="). The step fails with an ::error:: if any lockfile entry didn't become a constraint (checked against an extras line, a URL line and an empty lock), instead of letting that package install unpinned.
  • Dev-only pins vs -c: ruff, uv, aiosqlite, pytest, httpx, pip-audit, pyinstaller and bs4 aren't in the lockfile, so -c leaves them alone. A pip dry run for cp314/manylinux resolves them to the same versions as without constraints, and resolves 76 of the 77 locked packages at their locked version. The 77th, uvloop, is Linux-only and was skipped because the dry run ran on Windows. A dependency they share with the app (packaging, requests, …) stays at its locked version.
  • Side effects, documented in ci.yml and dependabot.yml:
    • A pip PR that raises a floor past the locked version now fails lint-and-test's install as well as lockfile-drift. Recompiling the lockfile fixes both.
    • If a dev tool ever needs a newer version of a locked package, lint-and-test fails while lockfile-drift stays green. The comment in ci.yml gives the uv pip compile --upgrade-package <name> recompile for that case, which doesn't change the lockfile header.
  • Why scheduled and not a non-blocking PR job: a job-level continue-on-error still puts a red check on every PR whenever upstream breaks. That's the noise this PR removes. A guard test keeps deps-latest's system packages and pytest call in step with lint-and-test, so a red weekly run really does point at upstream.
  • SBOM evidence (main @ 1d7bad6): 106 components against 77 in the lockfile. 19 of them had versions the image doesn't contain (SQLAlchemy 2.1.1 vs 2.0.52). 28 belong to the generator itself, and its install had downgraded packaging from 26.3 to 25.0.
    • cyclonedx-py requirements requirements.lock was considered. It parses the hashed lockfile fine, but its SBOM has no licences and no dependency graph (main's SBOM has 70 licensed components and 61 graph entries).
    • So environment stays, pointed at the lockfile venv. That was checked locally with cyclonedx-bom 5.5.0.
    • The new SBOM should list the 77 locked packages plus the venv's pip. pip's own version follows the runner's Python patch release.
  • Docs: development.md and third-party-licenses.md said sbom.yml builds the release SBOM, but that's release.yml since fix(ci): attach SBOM from release.yml #52. patch-policy.md now says what the SBOM covers: the app's Python dependencies at the shipped versions, not system packages.
  • Guards: tests/test_supply_chain_hygiene.py pins all of it, and each of 10 simulated reverts fails a guard.

Not in this PR (pre-existing, flagged by the security review)

  • The release job still installs the SBOM generator without hashes. The job has contents: write, restores the shared pip cache and keeps the checkout's credentials. This PR cuts the unverified install there from about 105 packages to about 28.
  • build-desktop.yml has never run: its release: published trigger doesn't fire for releases published with GITHUB_TOKEN.
  • release.yml feeds a tag-derived ${{ }} value into a shell script, and verapdf/cli:latest is unpinned.

Verification

  • Local: full suite green (see commit footer); ruff clean; python-version gate green; YAML parses; all files LF; gitleaks and the pre-commit scope guard are clean.
  • This PR's CI: lint-and-test runs the constrained install, which is also the first full test run against the locked versions in a while. verapdf runs the lockfile install.
  • sbom.yml is dispatched on this branch; its artifact gets checked against the lockfile.
  • release.yml only runs on tags. Its SBOM step runs the same commands as sbom.yml.
  • deps-latest.yml can only be dispatched once it is on main, so it gets one manual run after merge.

🤖 Generated with Claude Code

…ships

The image installs requirements.lock with --require-hashes, but four
workflows still installed requirements.txt, whose >= ranges resolve to
the newest PyPI releases. So CI tested, and the SBOM described, versions
the image does not ship.

lint-and-test: installs requirements-dev.txt with the lockfile's pins as
constraints. SQLAlchemy 2.1.0 reached CI unpinned and failed a test on
every branch while the image stayed on 2.0.52. The hashes are stripped
into $RUNNER_TEMP/constraints.txt because pip switches the whole install
to --require-hashes as soon as one constraint carries a hash (checked
with pip 26.2.1), and the dev tools are unhashed. The step fails if any
lockfile entry does not become a constraint, instead of letting it
install unpinned. The dev-only tools are not in the lockfile: a pip
dry-run for cp314/manylinux resolves them to the same pytest, ruff, uv,
aiosqlite, httpx, pip-audit, pyinstaller and bs4 versions as without
constraints, and 76 of the 77 locked packages at their locked version
(uvloop, Linux-only, was skipped on the Windows host that ran it).
A floor raised past the locked version (the usual Dependabot pip PR) now
fails this install as well as lockfile-drift; recompiling fixes both. A
dev tool that needs a newer locked package fails here with
lockfile-drift green; the ci.yml comment gives the --upgrade-package
recompile for that case.

deps-latest (new): the unpinned install plus pytest, weekly on Mondays
and on demand, gating nothing. It keeps the early warning that caught
2.1 before any lockfile bump. Not a continue-on-error job in ci.yml:
that still shows a red check on every PR whenever upstream breaks.

sbom / release: the SBOM described the runner's own Python after
`pip install -r requirements.txt` plus cyclonedx-bom. main's SBOM at
1d7bad6 had 106 components against 77 in the lockfile: 19 at versions
the image does not contain, 28 from the generator itself, and packaging
downgraded from 26.3 to 25.0 by the generator's install. Now: a fresh
venv installed the Dockerfile's way, the generator outside it, and
`cyclonedx-py environment <venv python>` (checked with cyclonedx-bom
5.5.0). Rejected `cyclonedx-py requirements requirements.lock`: it
parses the hashed lockfile fine but emits no licences and no dependency
graph (main's SBOM carries 70 licensed components, 61 graph entries).
The docs now say which workflow builds the release SBOM (release.yml,
not sbom.yml) and that system packages are not in it.

verapdf: built its PDF/A fixture with the newest pikepdf; it now
installs the lockfile the image's way (all 77 locked versions have
cp314 manylinux wheels). build-desktop.yml still installs
requirements.txt: it builds on Windows, the lockfile is Linux-only.

docker.yml: renormalized to LF. PR #129 committed it with CRLF via an
API commit despite `*.yml text eol=lf`, so every checkout showed it as
modified. No content change.

tests/test_supply_chain_hygiene.py pins all of it, including that
deps-latest keeps lint-and-test's system packages and pytest call; each
of 10 simulated reverts fails a guard.

Local suite 1239 passed, 63 skipped (native-library tests run in CI);
ruff clean; python-version gate green; i18n and pip-audit inputs
untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MrChengLen
MrChengLen force-pushed the pr-ci-lockfile-parity branch from 744d4a5 to 9a41431 Compare September 25, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant