ci: retry pcre2/zlib/openssl downloads in build-windows - #389
Open
fzipi wants to merge 1 commit into
Open
Conversation
The "Set up third-party libraries" step piped wget directly into tar with no retry, so a single truncated download (gzip: stdin: unexpected end of file / tar: Child returned status 1) killed the whole job with no indication of which of the three downloads failed. Download to a file first, with wget's own retry flags, then extract once the download is complete. Piping straight into tar can't be retried safely: a retried request restarts the response from byte 0, but tar has already consumed whatever the first, truncated attempt sent through the pipe -- a retry would just append a second copy of the file after the first truncated one, corrupting the archive rather than fixing anything. A file-based retry cleanly overwrites the previous attempt instead. Fixes owasp-modsecurity#388 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #388.
The
build-windowsjob's "Set up third-party libraries" step pipedwgetdirectly intotarwith no retry:wget -q -O - https://.../pcre2-10.47.tar.gz | tar -xzf -A single truncated download (
gzip: stdin: unexpected end of file/tar: Child returned status 1) killed the whole job immediately, with-qhiding which of the three downloads actually failed.wget --tries=3 --retry-connrefused --waitretry=5 -O "$archive" "$url"), then extract, rather than piping straight intotar. This isn't just cosmetic: piping can't be retried safely in the first place — a retried request restarts the response from byte 0, buttarhas already consumed whatever the first, truncated attempt streamed through the pipe. A retry there would just append a second copy of the file after the truncated first one, corrupting the archive instead of fixing anything. A file-based retry cleanly overwrites the previous attempt.set -euo pipefailadded explicitly at the top of the step, so any failure (including fromwgetitself) is fail-fast rather than silently continuing.Test plan
shellcheck/shfmtclean on the extracted script.actionlint/zizmorshow no new findings versus the file's current state (the existing unpinned-action/artipacked/cmd-shell findings elsewhere in this file are pre-existing and out of scope for this fix).build-windowsgets past this step reliably. Note per Windows CI: flaky pcre2/zlib/openssl tarball download in test_new.yml's build-windows job #388's own writeup: this job also hits a separate, unrelated, deterministic failure further along (yajl/CMakeCMP0026incompatibility, tracked upstream as owasp-modsecurity/ModSecurity#3604), sobuild-windowswon't go fully green until that's also fixed — this PR only addresses the flaky download this step itself is responsible for.