Skip to content

fix(ci): authenticate simulator-build clones with github.token - #3516

Draft
danceratopz wants to merge 2 commits into
ethereum:forks/amsterdamfrom
danceratopz:hive-sim-github-token
Draft

danceratopz wants to merge 2 commits into
ethereum:forks/amsterdamfrom
danceratopz:hive-sim-github-token

Conversation

@danceratopz

@danceratopz danceratopz commented Sep 3, 2026

Copy link
Copy Markdown
Member

Passes the Actions job token to the eels simulator image builds in the Hive Consume workflow, so the git clone of execution-specs inside those builds authenticates instead of running anonymously.

Motivation

GitHub rate-limits anonymous git-over-HTTPS per source IP, and the self-hosted runners intermittently hit the limit. When that happens the simulator image build dies before any test runs, e.g. this Engine job on forks/amsterdam:

Step 6/13 : RUN git clone --depth 1 https://github.com/ethereum/execution-specs.git && ...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing

The same failure mode has been hitting ethpandaops/hive-tests dashboard runs and other EF repo CI on the same runner pool. GitHub tightened unauthenticated rate limits in May 2025 and documents authentication as the remedy.

Changes

Add --sim.buildarg github_token=${{ github.token }} to the simulator invocation. The build-arg is consumed by the eels simulator Dockerfiles via ethereum/hive#1606; the token is job-scoped, works for cloning public repos, expires when the job ends, and is masked by the runner in logs and summaries.

Validation

An earlier revision of this PR temporarily pinned the Hive checkout to the ethereum/hive#1606 branch; the Engine, RLP and Sync jobs all built their simulator images with the authenticated clone and passed. The pin is now removed and the checkout is back on ethereum/hive @ master.

Before merge

The eels simulator image builds clone execution-specs anonymously and
GitHub rate-limits anonymous git-over-HTTPS per IP, which
intermittently fails the Engine/RLP/Sync jobs during the image build
before any test runs:

  fatal: could not read Username for 'https://github.com': No such device or address
  fatal: expected flush after ref listing

Pass the job token via the github_token build-arg added to the eels
simulator Dockerfiles in ethereum/hive#1606. The token is job-scoped
and expires when the job ends.

Temporarily pins the hive checkout to the ethereum/hive#1606 branch so
this PR's CI exercises the authenticated clone; restore
ethereum/hive@master before merge once that PR lands.
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.01%. Comparing base (b462ba2) to head (a285d49).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3516   +/-   ##
================================================
  Coverage            94.01%   94.01%           
================================================
  Files                  624      624           
  Lines                36905    36905           
  Branches              3326     3326           
================================================
  Hits                 34695    34695           
  Misses                1533     1533           
  Partials               677      677           
Flag Coverage Δ
unittests 94.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@danceratopz danceratopz changed the title ci(hive-consume): authenticate simulator-build clones with the job token fix(ci): authenticate simulator-build clones with the job's github.token Sep 3, 2026
@danceratopz danceratopz changed the title fix(ci): authenticate simulator-build clones with the job's github.token fix(ci): authenticate simulator-build clones with github.token Sep 3, 2026
@danceratopz
danceratopz marked this pull request as ready for review September 3, 2026 10:05
The temporary pin to the ethereum/hive#1606 branch served its purpose:
the Engine, RLP and Sync jobs built their simulator images with the
authenticated clone and passed. Until ethereum/hive#1606 merges, the
github_token build-arg is a no-op on hive master (unused build-arg
warning, the clone stays anonymous); it becomes active on merge with no
further changes here.
@danceratopz danceratopz added C-bug Category: this is a bug, deviation, or other problem A-ci Area: Continuous Integration labels Sep 3, 2026
@danceratopz
danceratopz marked this pull request as draft September 3, 2026 15:20
@danceratopz
danceratopz marked this pull request as ready for review September 3, 2026 15:26

@spencer-tb spencer-tb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving for now. I'll let you decide on whether we include danceratopz/hive#3 in your hive PR first before merging :)

If you prefer it then we need to use the environment-backed BuildKit secret interface from the hive change!

      - name: Run simulator tests
        if: matrix.mode == 'simulator'
        env:
          HIVE_GITHUB_TOKEN: ${{ github.token }}
        run: |
          cd hive
          ./hive --sim '${{ matrix.simulator }}' \
            --sim.parallelism=1 \
            --client ${{ env.CLIENT }} \
            --client-file ../execution-specs/.github/configs/hive/${{ env.CLIENT_FILE }} \
            --sim.buildarg fixtures=${{ env.FIXTURES_URL }} \

@danceratopz
danceratopz marked this pull request as draft September 3, 2026 23:11
@fselmo

fselmo commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

back to draft? 👀

@danceratopz

Copy link
Copy Markdown
Member Author

back to draft? 👀

Yeah, I think we should pursue a better, all encompassing approach and distribute our simulators via a container registry:
ethereum/hive#1606 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Area: Continuous Integration C-bug Category: this is a bug, deviation, or other problem

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants