fix(ci): authenticate simulator-build clones with github.token - #3516
danceratopz wants to merge 2 commits into
Conversation
The eels simulator image builds clone execution-specs anonymously and GitHub rate-limits anonymous git-over-HTTPS per IP, which intermittently fails the Engine/RLP/Sync jobs during the image build before any test runs: fatal: could not read Username for 'https://github.com': No such device or address fatal: expected flush after ref listing Pass the job token via the github_token build-arg added to the eels simulator Dockerfiles in ethereum/hive#1606. The token is job-scoped and expires when the job ends. Temporarily pins the hive checkout to the ethereum/hive#1606 branch so this PR's CI exercises the authenticated clone; restore ethereum/hive@master before merge once that PR lands.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3516 +/- ##
================================================
Coverage 94.01% 94.01%
================================================
Files 624 624
Lines 36905 36905
Branches 3326 3326
================================================
Hits 34695 34695
Misses 1533 1533
Partials 677 677
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
github.token
github.tokengithub.token
The temporary pin to the ethereum/hive#1606 branch served its purpose: the Engine, RLP and Sync jobs built their simulator images with the authenticated clone and passed. Until ethereum/hive#1606 merges, the github_token build-arg is a no-op on hive master (unused build-arg warning, the clone stays anonymous); it becomes active on merge with no further changes here.
spencer-tb
left a comment
There was a problem hiding this comment.
Approving for now. I'll let you decide on whether we include danceratopz/hive#3 in your hive PR first before merging :)
If you prefer it then we need to use the environment-backed BuildKit secret interface from the hive change!
- name: Run simulator tests
if: matrix.mode == 'simulator'
env:
HIVE_GITHUB_TOKEN: ${{ github.token }}
run: |
cd hive
./hive --sim '${{ matrix.simulator }}' \
--sim.parallelism=1 \
--client ${{ env.CLIENT }} \
--client-file ../execution-specs/.github/configs/hive/${{ env.CLIENT_FILE }} \
--sim.buildarg fixtures=${{ env.FIXTURES_URL }} \|
back to draft? 👀 |
Yeah, I think we should pursue a better, all encompassing approach and distribute our simulators via a container registry: |
Passes the Actions job token to the eels simulator image builds in the Hive Consume workflow, so the
git cloneof execution-specs inside those builds authenticates instead of running anonymously.Motivation
GitHub rate-limits anonymous git-over-HTTPS per source IP, and the self-hosted runners intermittently hit the limit. When that happens the simulator image build dies before any test runs, e.g. this Engine job on forks/amsterdam:
The same failure mode has been hitting ethpandaops/hive-tests dashboard runs and other EF repo CI on the same runner pool. GitHub tightened unauthenticated rate limits in May 2025 and documents authentication as the remedy.
Changes
Add
--sim.buildarg github_token=${{ github.token }}to the simulator invocation. The build-arg is consumed by the eels simulator Dockerfiles via ethereum/hive#1606; the token is job-scoped, works for cloning public repos, expires when the job ends, and is masked by the runner in logs and summaries.Validation
An earlier revision of this PR temporarily pinned the Hive checkout to the ethereum/hive#1606 branch; the Engine, RLP and Sync jobs all built their simulator images with the authenticated clone and passed. The pin is now removed and the checkout is back on
ethereum/hive@master.Before merge
ethereum/hive@master