Skip to content

Retry CI bundler installs on rubygems.org blips - #1828

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/retry-bundler-install
Oct 29, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/retry-bundler-install

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On 2026-10-29 the merge_group CI run for #1813 (run 40882178298, job hosted-e2e) failed in Setup Ruby and evicted the PR from the merge queue (removed 13:35:06, re-added 14:01:32, so about 27 minutes lost for that PR plus a rebuild of the entries behind it). It was the only merge_group failure in the last 24h (212 merge_group runs).

/opt/hostedtoolcache/Ruby/3.2.10/x64/bin/gem install bundler -v ~> 2.6.0
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
    SocketError: Failed to open TCP connection to rubygems.org:443 (https://rubygems.org/quick/Marshal.4.8/bundler-2.6.9.gemspec.rz)
##[error]The process '/opt/hostedtoolcache/Ruby/3.2.10/x64/bin/gem' failed with exit code 1

Root cause

ruby/setup-ruby's bundler: input installs bundler with a single bare gem install. RubyGems has no retry, so one failed TCP connect to rubygems.org fails the required hosted-e2e leg and, through ci-ok, evicts the queue entry. This is the same class as the PyPI blip that scripts/pip-install-retry.sh covers and the setup-python download failure #1803 fixed.

Fix

  • New scripts/gem-install-retry.sh, modelled on pip-install-retry.sh: it runs gem install <args> up to 4 times with a growing pause and passes the args through unchanged, so pins stay pinned.
  • hosted-e2e: setup-ruby now gets bundler: 'none', and a new Install Bundler step runs gem-install-retry.sh bundler -v '~> 2.6.0' --no-document. That is the same requirement setup-ruby used (see the log above). As the highest installed bundler, it is the one the bundle binstub runs, so the >= 2.6 CHECKSUMS requirement still holds. The step prints bundle --version as a check.
  • e2e: the existing Install Bundler 1.x step's bare gem install goes through the same script. Behaviour is otherwise unchanged.

No test moves or is removed, and no required-check names change.

Proof

  • I ran the real install against rubygems.org with a scratch GEM_HOME: Successfully installed bundler-2.6.9, then bundle --version gave Bundler version 2.6.9.
  • I ran the script with a fake gem on PATH:
    • fail once, then succeed: a warning on attempt 1 and exit 0 on attempt 2.
    • always fail: 4 attempts, then ::error:: and exit 1.
    • no args: usage message and exit 2.
  • python3 scripts/check-tool-pins.py .github/workflows/ci.yml: all pinned.
  • python3 -m unittest discover -s scripts/tests: 384 tests OK.
  • YAML parses. actionlint and zizmor aren't available in this container.

The bundler: input on the e2e matrix's own setup-ruby step (2.x legs) still installs without a retry. Moving it means matching setup-ruby's per-version resolution leg by leg, so I left it out of this PR to keep the change small.


Generated by Claude Code


Note

Low Risk
CI-only resilience around bundler installation; no application or release behavior changes.

Overview
Adds scripts/gem-install-retry.sh, mirroring pip-install-retry.sh: up to four gem install attempts with backoff when rubygems.org flakes, without changing version pins.

hosted-e2e no longer relies on ruby/setup-ruby’s one-shot bundler install (bundler: 'none' plus a dedicated Install Bundler step via the retry script at ~> 2.6.0). The e2e matrix Install Bundler 1.x step uses the same wrapper instead of a bare gem install.

Setup-ruby bundler installs on other e2e 2.x matrix legs are unchanged in this PR.

Reviewed by Cursor Bugbot for commit 2a2d5f5. Configure here.


Generated by Claude Code

ruby/setup-ruby's `bundler:` input installs bundler with one bare
`gem install`, and RubyGems has no retry of its own. A single failed
TCP connect to rubygems.org in hosted-e2e's Setup Ruby step failed the
merge_group run for #1813 and evicted it from the queue.

Install bundler in hosted-e2e through a new gem-install-retry.sh (the
pip-install-retry.sh pattern: 4 attempts, growing pause) instead, with
the same `~> 2.6.0` requirement setup-ruby used, and route the e2e
job's Bundler 1.x install through it too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeTA3DTuUZTYGYzv6KB
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Oct 29, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 2a2d5f5. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 29, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Burn-down agent: labeled Ready for review at 2a2d5f5dea69298dd9b7bf104644698073e19854.

  • CI: all 51 checks on this head are success/skipped; PR is mergeable (only waiting on review).
  • Bugbot: reviewed this head, no issues.
  • For reviewers: CI-only change. hosted-e2e now installs Bundler ~> 2.6.0 through the new scripts/gem-install-retry.sh (4 attempts with backoff) instead of setup-ruby's one-shot install, and the e2e Bundler 1.x step uses the same wrapper.
  • Slack announcement not sent this run (the Slack send tool was unavailable); the next run will retry.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. A single rubygems.org TCP blip during ruby/setup-ruby's one-shot gem install bundler failed hosted-e2e and evicted #1813 from the merge queue. This adds scripts/gem-install-retry.sh (a copy of the pip-install-retry.sh pattern: 4 attempts, 10/20/30 s pauses, args passed through) and routes the hosted-e2e Bundler 2.6 install and the e2e Bundler 1.x install through it. CI-only; no product code changes.

Risk. Low. Same Bundler requirement as before (~> 2.6.0, which is what bundler: '2.6' resolved to), no required-check names change, no tests moved.

Look here

Verified

  • Read the full diff (2 files, +45/-4); the script matches pip-install-retry.sh line for line apart from the command, and is committed as mode 100755.
  • On this head, hosted-e2e ran (not gated off): its Install Bundler step and both production e2e steps passed, so the installed Bundler emits CHECKSUMS as the hosted rewrite needs.
  • Ran the script locally with a fake gem that fails once: warning on attempt 1, exit 0 on attempt 2, args passed through intact.
  • CI 51/51 (32 success, 19 skipped), ci-ok and clippy green; mergeable; no review threads. Bugbot reviewed this head: no issues. CHANGELOG.md untouched.

Changes I made. None.

Open questions. None blocking. Note: the Bundler 1.x matrix legs don't run on PR CI, so that one-line change is exercised only on the next run of those legs. The 2.x e2e legs still use setup-ruby's own unretried install (called out in the description as out of scope).

Auto-merge is armed: approving sends this straight to the merge queue.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants