Research inquiry: Reflex’s PyPI approval workflow

[abishekgiri]

Hello Reflex release maintainers,

Your release instructions describe building and validating packages before a human approves the pypi deployment, including prereleases. Would the person responsible for that workflow be willing to answer a brief feasibility question?

I’m investigating whether information available before approval can help predict whether the approved publish job succeeds or fails. That is an open research question, not an established capability.

Roughly how many approved publish-job executions occur monthly, and have any naturally failed or timed out in the past six months? If suitable, would you be open to discussing passive observation of approval timing and job outcomes?

This first ask is only a conversation; no installation, pipeline changes, source-code contents, secrets, or production access are requested.

Thank you,
Abishek Kumar Giri

[masenf]

I developed the reflex-release workflow that we use for changelog-based publishing.

In September we had 25 manual release dispatch operations. Of those, 3 failed for various reasons, but usually due to preflight checks determining that the selected release mode was not valid (i.e. maintainer clicked the wrong thing).

In a few cases we've had publish errors, where a new package was added but not primed for trusted publishing on PyPI yet. In those cases, we configure the new package and rerun the job and it has worked.

The nice thing about the publish-before-tag workflow is that a failed publish doesn't leave an orphan tag in the repo. We only tag and repo when we successfully publish the package to pypi. Another benefit to this workflow is that the changelog in the repo is the source of truth, so there's no way for the published package to get out of sync with the published changelog.

There are other nice aspects of the workflow, like automatic association of changelog fragments with the PR where they were committed, and the automatic lifting of dev pins in sibling packages to ensure the published packages can resolve their dependencies correctly.

[abishekgiri]

Thanks for explaining this. For my study, I’m focusing on jobs that start after approval, so I would keep preflight failures separate.
Were the trusted-publishing errors failures of the publish job after the pypi approval?
Do the 25 September dispatches correspond to 25 approved, started publish jobs, or can one dispatch publish several packages or stop before approval? A rough sense of monthly volume over April–September would help too; no export needed yet.
Would you be open to briefly discussing whether existing approval/job metadata could support passive observation, without changing your workflow or approval decisions?
Thanks,
Abishek Kumar Giri