From 32e8b6bc7fed7b2cf9b51c6811a01cbc86515f1f Mon Sep 17 00:00:00 2001 From: Antoine Date: Mon, 31 Aug 2026 14:53:41 +0200 Subject: [PATCH 1/3] fix(docker): retry go mod download and cache Go modules Why: The v1.168.5 release container job failed at `RUN go mod download` with a transient HTTP/2 stream reset from proxy.golang.org. The Docker build fetched all 198 modules uncached with no retry, so a single reset broke the release and left no image published for the tag. What: - Retry `go mod download` up to 5 times with linear backoff (5/10/15/20s). - Mount BuildKit caches for the module cache and the Go build cache. Notes: GOPROXY's `direct` fallback does not cover this error class (only 404/410), so a retry is the only way to absorb it. --- Dockerfile | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/Dockerfile b/Dockerfile index 27e6e194..ffc5af57 100644 --- a/Dockerfile +++ b/Dockerfile @@ -9,13 +9,23 @@ WORKDIR /app COPY go.mod go.sum ./ # Download dependencies -RUN go mod download +# Retried: proxy.golang.org intermittently resets HTTP/2 streams mid-download. +RUN --mount=type=cache,target=/go/pkg/mod \ + for attempt in 1 2 3 4 5; do \ + go mod download && exit 0; \ + echo "go mod download failed (attempt ${attempt}/5)"; \ + [ "${attempt}" = 5 ] && break; \ + sleep $((attempt * 5)); \ + done; \ + exit 1 # Copy the source code to the container's working directory COPY . . # Build the Go application -RUN go build -o qovery -ldflags "-X github.com/qovery/qovery-cli/utils.Version=$APP_VERSION" +RUN --mount=type=cache,target=/go/pkg/mod \ + --mount=type=cache,target=/root/.cache/go-build \ + go build -o qovery -ldflags "-X github.com/qovery/qovery-cli/utils.Version=$APP_VERSION" FROM public.ecr.aws/r3m4q3r9/pub-mirror-debian:bookworm-slim as runner From 2dcc4f607c36ed9bd8854994f0a11231b742869e Mon Sep 17 00:00:00 2001 From: Antoine Date: Mon, 31 Aug 2026 15:11:30 +0200 Subject: [PATCH 2/3] fix(docker): fail fast on non-transient go mod download errors Why: The retry loop retried every failure, including ones decidable from go.mod and go.sum alone. A go.sum checksum mismatch can never succeed on retry, so the build burned 5 download passes and ~50s of backoff before failing, and buried Go's SECURITY ERROR banner in retry noise. What: Capture stderr and short-circuit when it matches a non-transient error class (checksum mismatch, missing go.sum entry, go.mod parse or version errors). Everything else stays retried. Notes: Exit codes cannot discriminate here: network failures, checksum mismatches and unknown revisions all exit 1, and -json changes neither the exit code nor the stderr routing, so stderr matching is the only available signal. The match list is a denylist rather than an allowlist of retryable errors on purpose: an unmatched permanent error costs ~50s of CI, while an unmatched transient one would break a release. --- Dockerfile | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index ffc5af57..58687829 100644 --- a/Dockerfile +++ b/Dockerfile @@ -10,9 +10,16 @@ COPY go.mod go.sum ./ # Download dependencies # Retried: proxy.golang.org intermittently resets HTTP/2 streams mid-download. +# Errors decidable from go.mod/go.sum alone can never succeed on retry, so they +# fail fast; anything else is assumed transient and retried. RUN --mount=type=cache,target=/go/pkg/mod \ for attempt in 1 2 3 4 5; do \ - go mod download && exit 0; \ + if go mod download >/tmp/godl.log 2>&1; then exit 0; fi; \ + cat /tmp/godl.log; \ + if grep -qE 'SECURITY ERROR|checksum mismatch|missing go.sum entry|errors parsing go.mod|invalid version' /tmp/godl.log; then \ + echo "go mod download failed with a non-transient error; not retrying"; \ + exit 1; \ + fi; \ echo "go mod download failed (attempt ${attempt}/5)"; \ [ "${attempt}" = 5 ] && break; \ sleep $((attempt * 5)); \ From f322bd4ef3c5e244d2d0cf91e8419943b6df1b84 Mon Sep 17 00:00:00 2001 From: Antoine Date: Mon, 31 Aug 2026 15:26:05 +0200 Subject: [PATCH 3/3] fix(docker): widen non-transient go mod download discriminant Why: The fail-fast list missed deterministic errors, so a bad module path or an unresolvable version still burned 5 download passes and ~50s of backoff. The list was drawn on the wrong axis: local-vs-remote rather than whether the proxy actually answered. `unknown revision` is a definitive negative answer, not a failed round trip, so retrying it cannot help. What: Retry only transport failures. Add `unknown revision`, `malformed module path` and `module lookup disabled` to the non-transient set. Notes: Kept as a denylist rather than an allowlist of retryable errors. The failure that motivated this PR was `stream error: stream ID 171; INTERNAL_ERROR`, a string no hand-written transient allowlist would plausibly have contained; missing an entry there breaks a release, while missing one here costs ~50s. `no matching versions for query` is not reachable from `go mod download` with pinned versions, which report `errors parsing go.mod` or `unknown revision`. --- Dockerfile | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Dockerfile b/Dockerfile index 58687829..5190da86 100644 --- a/Dockerfile +++ b/Dockerfile @@ -10,13 +10,14 @@ COPY go.mod go.sum ./ # Download dependencies # Retried: proxy.golang.org intermittently resets HTTP/2 streams mid-download. -# Errors decidable from go.mod/go.sum alone can never succeed on retry, so they -# fail fast; anything else is assumed transient and retried. +# Only transport failures are worth retrying. If the proxy or VCS returned a +# definitive answer, or the error is in local files, another attempt cannot +# change it, so fail fast instead of sleeping through the backoff. RUN --mount=type=cache,target=/go/pkg/mod \ for attempt in 1 2 3 4 5; do \ if go mod download >/tmp/godl.log 2>&1; then exit 0; fi; \ cat /tmp/godl.log; \ - if grep -qE 'SECURITY ERROR|checksum mismatch|missing go.sum entry|errors parsing go.mod|invalid version' /tmp/godl.log; then \ + if grep -qE 'SECURITY ERROR|checksum mismatch|missing go.sum entry|errors parsing go.mod|invalid version|unknown revision|malformed module path|module lookup disabled' /tmp/godl.log; then \ echo "go mod download failed with a non-transient error; not retrying"; \ exit 1; \ fi; \