npm: read the stored publish time in the cooldown download check - #296
Merged
andrew merged 1 commit intoSep 2, 2026
Merged
Conversation
- Consult versions.published_at before fetching the packument, and persist the parsed time after the packument fallback, so each version's metadata is fetched and parsed at most once - Add DB.SetVersionPublishedAt, an upsert that writes only the publish time - Preserve a stored published_at in UpsertVersion when the incoming value is NULL, so the artifact-cache upsert cannot erase it - Add handler tests for stored-time downloads and single-fetch behavior, and a database test for preserve-on-NULL in both dialects
VictorCodesseira
marked this pull request as ready for review
September 1, 2026 18:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With cooldown enabled,
versionInCooldownruns on every npm tarball request —cache hits included — and loads the package's full packument through
FetchOrCacheMetadata, unmarshalling the entire body to read one entry of thetimemap. For packages like typescript or @swc/core that is a multi-megabytebuffer and parse per tarball, and with an S3 storage backend it is also one
storage GET per tarball.
Under a heavy load (bursts of several hundred concurrent requests) this
dominates the proxy's cost: I measured ~12-16ms of CPU per request with cooldown on versus
well under 1ms with it off, install fetch times of ~100s that dropped to ~9s
when cooldown was disabled, and higher memory usage with it enabled, causing OOM kills.
A version's publish time is immutable, so the check now reads the stored
versions.published_atfirst and only falls back to the packument for aversion the proxy has never seen, persisting the parsed time so the packument
is fetched and parsed at most once per version. Same shape as the PyPI fix
from #242.
Changes:
versionInCooldownconsultsversions.published_atbefore fetchingmetadata, and persists the parsed time after the packument fallback
DB.SetVersionPublishedAt, an upsert that writes only the publish timeso it never disturbs enrichment data on an existing row
UpsertVersionnow preserves a storedpublished_atwhen the incomingvalue is NULL (
COALESCE, both dialects). Without this, caching theartifact right after the check upserts the versions row without a publish
time and erases the value the same request just stored; it also means a
null-bearing upsert can no longer erase enrichment data
upstream (served and withheld cases); two downloads of an uncached version
fetch the packument exactly once, proving the stored time survives the
artifact-cache upsert; database test for preserve-on-NULL and
update-on-value in both dialects
go test ./...,go fmt,go vetclean.Companion PR: #297 (serving publish times without enabling cooldown).