feat(tooling,ci): publish ready-to-run docker images for the eels hive simulators - #3572
Open
danceratopz wants to merge 8 commits into
Open
danceratopz wants to merge 8 commits into
danceratopz wants to merge 8 commits into
Conversation
…lators Add packages/testing/docker/base/Dockerfile, which builds the execution-specs source tree and its synced virtualenv into an image, published as ghcr.io/ethereum/execution-specs, tagged by commit (sha-<7>) and by line: `latest` for the head of the default branch, `<name>-devnet-<n>` for the head of a devnet branch, `nightly` for the commit of a nightly fill. It is the code primitive of the hive simulator images and runs every command of the testing package on its own, t8n and statetest included. The image carries no fixtures. The commit and branch it was built from are recorded as OCI labels and as environment variables, since the copied tree has no .git directory. Dockerfile.dockerignore, read by BuildKit from beside the Dockerfile, keeps the build context to the source tree; generated fixtures, caches, worktrees and docs are excluded.
Add packages/testing/docker/fixtures: a data-only image holding one fixture format of one release under /fixtures, with .meta/index.json filtered to that format, published as ghcr.io/ethereum/execution-specs/fixtures/<format> under the release's tags, or the nightly tags for a nightly fill. Each consume hive simulator consumes a single format, so a simulator image only needs to carry that one, and the images double as a distribution channel for fixtures. build.sh prepares the build context from a release tarball URL, a tarball on disk such as the artifact of a nightly fill, or an extracted release such as the `consume cache` directory, handing docker only .meta/ and the chosen format directory. subset_index.py filters the index and recomputes test_count, fixture_formats and forks. The image is labelled with the release and the commit that produced it.
Add packages/testing/docker/tests: a data-only image holding the tests/ tree at the commit a release was built from, under /execution-specs/tests, published as ghcr.io/ethereum/execution-specs/tests under the release's tags, or the nightly tags for a nightly fill. It is to the execute simulators what the fixture images are to the consume simulators: the frozen test content of a release, which an execute image takes in place of the branch head's tests so that a tag selects the same tests on both kinds of simulator. build.sh exports the tree from a git ref with `git archive`, or takes a checked-out tests/ directory as the workflow does; the ignore file beside the Dockerfile keeps generated fixtures and caches out of a checked-out tree.
Add packages/testing/docker/hive: one image per eels hive simulator, published as ghcr.io/ethereum/execution-specs/hive/<simulator>. A tag selects the version of the tests and the simulator and framework code follows the branch head, so runner fixes such as client exception mappings reach every image without a new test release. Dockerfile builds a consume simulator from the fixture image of its format with the repository image flattened on top; the fixture image is the base layer so that it is shared by digest between every simulator image built from the same release. Dockerfile.execute builds an execute simulator from the repository image with its tests/ tree replaced by the release's tests image, the head's tree being removed first so that renamed or deleted tests do not linger. The same tag on consume-engine and execute-blobs therefore runs the same tests. entrypoint.sh, installed as eels-simulator, selects the consume or execute command from EEST_SIMULATOR and honours DISABLE_STRICT_EXCEPTION_MATCHING, an empty value exempting no client, and FORK. build.sh picks the Dockerfile and the fixture or tests image for a simulator. check.sh verifies an image before it is published: a consume image must carry a non-empty index whose entries all point to files present in the image; an execute image must collect the release's tests under the head framework, since test sources import the framework where fixtures do not. `fill --collect-only` exits 5 whenever a marker deselects tests, so the check judges by the collected count and treats any other non-zero status as a failure.
…ease in the log header The docker images built from this repository carry no .git directory, so the `consume ref` header printed "Not a git repository". Fall back to EEST_GIT_SHA, which the images set to the commit the framework was built from. Add a `fixtures release` line from EEST_FIXTURES_RELEASE for fixtures baked into a consume image, matching the line printed for downloaded releases, and give execute a header of its own: `execute ref`, and `tests release` and `tests ref` from EEST_TESTS_RELEASE and EEST_TESTS_GIT_SHA when the tests are a release's snapshot. A hive log then names the release whose tests ran and the commit of the framework that ran them.
Add execution_testing.tools.docker_images, which holds every naming rule of the published images in one place: the catalogue of images with their family and test content; the branch tag in the ecosystem's naming (devnets/glamsterdam/8 -> glamsterdam-devnet-8), checked against docker's tag syntax; the release tags, EEST's release names with "@" replaced by "-" (v20.0.2, glamsterdam-devnet-v8.1.4); which branch a release was cut from, inferred from its series; and the channel tags a release holds. A release holds its branch's channel, `latest` for the default branch or `<name>-devnet-<n>` for a devnet branch, when it is the highest release of the branch, and its series' channel, `latest` or `<name>-devnet-latest`, when it is the highest of the series. Channels follow the highest version, never the newest publication, and resolve exactly as `consume --input tests@latest` and `<name>-devnet@latest` do: the highest published release with a fixtures tarball, the GitHub pre-release flag ignored. Only the default branch owns the main line, and a release is only paired with the branch it was cut from. plan() computes what a publishing run builds so that the workflow contains no naming logic of its own: for a push, the branch's channel owner rebuilt with the head; for a release, its fixture, tests and simulator images paired with the head of its branch; for a nightly fill, every image at the fill commit under `nightly` and `nightly-<7>`. A release's commit is resolved from its git tag rather than taken from the API's target_commitish. `eest images branch-tag`, `release-branch` and `plan` expose it; the plan is JSON, optionally with tag lists joined for shell steps. Tests pin the rules, including that an older release moves nothing, that publication order does not decide the channel owner, and that a newer devnet takes the series channel but not the older devnet's own.
danceratopz
added
A-doc
Add a workflow that publishes the repository, fixture, tests and simulator images to ghcr.io. A plan job runs `eest images plan` once and every other job consumes its JSON and checks out the commit it planned. Three kinds of run: a push to a fork or devnet branch rebuilds the repository image and the simulator images of the branch's current release with the new head, so that the channel tag follows the branch; a published fixture release (`release: published`) builds its fixture images and tests image and the simulator images pairing it with the head of the branch it was cut from, which is inferred from the tag; and the scheduled nightly fill in release_fixtures.yaml calls the workflow with the fill commit, whose fixtures_<sha> artifact becomes every image under `nightly`. Runs of one line, mainnet or devnets, queue behind each other so that channel tags move in order, and a run whose branch moved on while it was building leaves the channel tags to the newer run. Every simulator image is verified with check.sh before it is pushed, and every pushed tag is recorded with its digest in the step summary and a `digests-*` artifact, since tags name sources rather than bytes. Action versions are pinned to commit SHAs.
Add a Simulator Images section to the hive documentation with a tutorial, how-to guides, a reference and an explanation, and a developer page on publishing the images. The pages lead with the rule behind every tag: a tag selects the version of the tests, the framework follows the branch head, and consume and execute simulators under the same tag run the same release. The reference holds the one canonical tag table: release tags as `consume --input` names with "@" replaced by "-", the channels `latest`, `<name>-devnet-<n>`, `<name>-devnet-latest` and `nightly`, and what each resolves to. The how-to guides cover following a devnet, running last night's fill, pinning a release, reproducing a run exactly by digest, assembling a combination that is not published from the primitive images, and using the repository image as a t8n and statetest tool image. The releasing page notes that publishing a release triggers the image workflow.
danceratopz
force-pushed
the
consume-images
branch
from
September 11, 2026 09:33
c267988 to
1379d60
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3572 +/- ##
===================================================
+ Coverage 94.01% 94.39% +0.37%
===================================================
Files 624 624
Lines 36927 36927
Branches 3326 3326
===================================================
+ Hits 34717 34857 +140
+ Misses 1533 1459 -74
+ Partials 677 611 -66
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
spencer-tb
self-requested a review
September 14, 2026 12:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Currently, the
ethereum/eels/*simulators in hive:uv sync,This happens on every build (and on every ephemeral CI runner for hive dashboards). These steps have the following cost and issues:
hive/simulators/<simulator>:latestwhatever the arguments. Switching between two releases, or between a release and a branch, therefore rebuilds the simulator from the clone onwards and overwrites the previous image; two releases cannot be kept side by side, locally or on a runner.This PR publishes ready-to-run images with only the required fixture format included, so that a hive run can pull a dedicated image and start. Releases become image tags that coexist on a machine, so switching from
v20.0.2toglamsterdam-devnet-v8.1.4reduces to about a second of overhead, where hive's own build of a simulator image previously took over 100 s. The companion hive-side PR, ethereum/hive#1609, reduces the default simulatorDockerfiles to aFROM.A simulator tag selects the tests. Consume images contain generated fixtures; execute images contain the Python test sources from the release commit. Images for a branch's current release receive framework updates on branch pushes. Older releases retain their last framework build. Nightly images use the fill commit for the tests and framework together. Pin a digest to reproduce the exact image.
Examples:
latest: the latest mainnet test release.glamsterdam-devnet-8: the latest release of Glamsterdam devnet 8.glamsterdam-devnet-latest: the latest release across Glamsterdam devnets.glamsterdam-devnet-v8.1.4: the tests from release 8.1.4.nightly: the most recent nightly fill of the main line.The image and tag reference describes the full contract.
The implementation adds:
eest images plan, used by the publishing workflow for push, release and nightly runs.See Simulator Images for usage and Publishing Simulator Images for operation and rollout.
Validation, all local:
just static, workflow validation and Markdown lint passed.No images have been published yet. Before rollout, confirm the registry choice and package permissions, allow the pinned
docker/setup-buildx-action,docker/login-actionanddocker/build-push-actionin the upstream Actions policy, and exercise release, push and nightly publication on a fork. Seed the current release with a manual release build before relying on push builds, which reuse its fixture and tests images. Roll out branch by branch as these changes are propagated; the Hive documentation should refer only to tags that have been published. Publish the images needed by the Hive companion before it merges. Registry cleanup and a retention policy remain follow-up work.Related Issues or PRs
Checklist
just static<type>(<area>): <title>, where<type>and<area>come from an appropriateC-<type>, respectivelyA-<area>, label. The title should match the target squash commit message.Cute Animal Picture