ci(release): adopt self-hosted SIF release pipeline (Spartan CPU)#339
Merged
Conversation
Replace the GitHub-hosted release workflow with the proven self-hosted
SIF pipeline (test->build->publish->release inside the reused scitex-ci
ci-cpu.sif via apptainer exec), matching the fleet canary
(openalex-local). The bare Spartan compute node has no Python, so
actions/setup-python and the Docker-based pypa/gh-action-pypi-publish
cannot run there; publish now does MANUAL OIDC trusted publishing inside
the SIF (GitHub OIDC JWT -> PyPI mint-token -> twine upload). Trust
config on PyPI is unchanged.
- KEEP trust-bound filename pypi-publish-and-github-release-on-tag.yml
(PyPI trusted publisher is bound to the workflow filename).
- Add the 4 .github/ci/{exec,run,build,publish}-in-sif.sh helpers.
- Strip the auto sync-main job (develop->main is a deliberate PR).
- runs-on from Actions Variable CI_RUNS_ON; SIF + apptainer paths from
SCITEX_CI_SIF / SCITEX_CI_APPTAINER (same as pytest-matrix).
- --cov=src/scitex; environment url https://pypi.org/p/scitex;
unique-per-run TMPDIR; matplotlib Agg + warm font cache guarded.
- fail-loud: a missing SIF/shim/interpreter is a HARD error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Self-hosted SIF release pipeline (Spartan CPU)
Replaces the GitHub-hosted release workflow with the proven self-hosted SIF pipeline (test->build->publish->release inside the reused
scitex-cici-cpu.sifviaapptainer exec), matching the fleet canary (openalex-localdevelop).Why
The bare Spartan compute node has no Python, so
actions/setup-python@v5fails at init, and the Docker-basedpypa/gh-action-pypi-publishcannot run (no Docker on the compute nodes). Publish now does manual OIDC trusted publishing inside the SIF: GitHub OIDC JWT -> PyPI mint-token -> twine upload. PyPI trust config is unchanged (only the client changes).Changes
pypi-publish-and-github-release-on-tag.yml(the PyPI trusted publisher is bound to the workflow filename)..github/ci/{exec,run,build,publish}-in-sif.shhelpers (executable).sync-mainjob (develop->main is a deliberate, separate PR — operator policy).runs-ondriven by Actions VariableCI_RUNS_ON; SIF + apptainer paths fromSCITEX_CI_SIF/SCITEX_CI_APPTAINER(same as pytest-matrix).TMPDIR, matplotlibAgg+ warm font cache guarded.Actions Variables
CI_RUNS_ON/SCITEX_CI_SIF/SCITEX_CI_APPTAINERset on the repo.