Repository navigation
Conversation
Each published package is added by a task that holds the repository exclusively. The task was dispatched as immediate only, so a publish arriving while another one held the repository was canceled and answered with 429. npm does not retry that, so publishing several packages at once, as workspace tools do, failed. The task may now wait in the queue for a worker, and the request waits for it for up to NPM_PUBLISH_TASK_TIMEOUT seconds (30 by default) before answering 429, as the container registry API does for its uploads.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The publish endpoint (
PUT /<package>) adds each published package with a task dispatchedwith
immediate=True, deferred=Falseand the repository as an exclusive resource. Whenanother publish holds the repository at that moment, pulpcore cancels the task ("Resources
temporarily unavailable.") and the endpoint answers 429. npm, yarn and pnpm do not retry a
refused publish, so publishing several packages into one repository at the same time fails,
for example a workspace that publishes all its packages, or two pipelines that publish at
once.
Change
The task may now wait in the queue for a worker (
deferred=True), and the request waits forit for up to
NPM_PUBLISH_TASK_TIMEOUTseconds (30 by default) before answering 429. This iswhat the container registry API does for its uploads. A publish that finds the repository
free still runs immediately, as before.
NPM_PUBLISH_TASK_TIMEOUT.CHANGES/+concurrent-publishes.bugfix.Tests
test_concurrent_publishes_to_one_repository_all_succeedpublishes 8 packages at once andexpects all 201 and all 8 packages in the latest version. Without the change it fails with
429s; with it, it passes repeatedly. The rest of the functional suite passes unchanged.
The Maven plugin has the same pattern in its deploy API; pulp/pulp_maven#562 carries the
equivalent change there.