Skip to content

chore(test-forks): add descriptive Amsterdam BPO test scenarios - #3597

Open
danceratopz wants to merge 5 commits into
ethereum:forks/amsterdamfrom
danceratopz:remove-some-bpos
Open

danceratopz wants to merge 5 commits into
ethereum:forks/amsterdamfrom
danceratopz:remove-some-bpos

Conversation

@danceratopz

@danceratopz danceratopz commented Sep 16, 2026 •

Copy link
Copy Markdown
Member

Description

Warning

Post-Amsterdam BPO transitions require client changes before rollout. Clients that enforce BPO3/BPO4 activation before Amsterdam must update their fork-order handling to support these scenarios; Geth currently rejects that ordering. Hive mapper aliases alone do not fix this. Automatic fixture releases exclude the synthetic scenarios pending client support. Test releases will not include these scenarios without informing client teams first and confirming the required client support. Successful EELS filling does not establish client compatibility, and no external-client execution is claimed.

Add explicitly selected synthetic blob schedules under Amsterdam: Amsterdam → BPOIncrease → BPODecrease. The increase uses target/max 21/32 and update fraction 20609697; the decrease uses 14/21 and 13739630. Both standalone schedules active at genesis and transitions at timestamp 15,000 are covered. Existing blob-fee and excess-blob-gas tests now opt into standalone BPO coverage.

Retain numbered BPO1–BPO5 definitions and historical transitions for existing fixture releases and genesis parsing. Fork selection follows ancestry by default, so --until=Amsterdam excludes the parallel historical BPO3–BPO5 branch. Mainnet release generation stops at BPO2. Select the new scenarios explicitly with --from=BPOIncrease --until=BPODecrease; automatic releases do not include them.

Internal fill CI extends the existing Amsterdam matrix entry through BPODecrease, keeping not slow and primary_format. This selects 4,658 standalone synthetic BPO cases (356 reserve-price and 4,302 excess-blob-gas cases); synthetic transition tests remain excluded by their slow markers. These CI fixtures are not published. The release-only fork-ranges.yaml splits releases; feature.yaml keeps devnet releases capped at Amsterdam.

The Fusaka Devnet-4-specific regression case is intentionally removed. The new scenarios test Amsterdam schedule changes rather than preserving that historical devnet behavior.

Hive uses descriptive HIVE_BPO_INCREASE_* and HIVE_BPO_DECREASE_* rulesets; numbered aliases remain supported.

See synthetic blob schedule documentation for selection and compatibility details.

Validation: just static; 364 unit/plugin checks passed, 6 skipped; 6 release-matrix checks passed; 152 blockchain/Engine fixtures filled (22 transition, 120 standalone reserve-price boundary, 10 standalone excess-blob-gas cases). A separate manual reserve-price fill passed 367 primary-format cases (356 standalone, 11 transition). Collection of the BPO-enabled suites confirmed the 4,658 standalone cases selected by the amended CI range and filter; these were not all re-filled locally. just lint-actions passed. Inspected generated networks and Amsterdam headers; verified 28 EELS-to-Hive ruleset mappings across seven clients and unchanged legacy mapper output.

Related Issues or PRs

Hive mapper aliases: ethereum/hive#1611.

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

A curious cat

@danceratopz danceratopz added C-chore Category: chore A-test-forks Area: execution_testing.forks A-ci Area: Continuous Integration labels Sep 16, 2026
@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.44%. Comparing base (7942da0) to head (332512a).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3597   +/-   ##
================================================
  Coverage            94.44%   94.44%           
================================================
  Files                  624      624           
  Lines                36928    36928           
  Branches              3326     3326           
================================================
+ Hits                 34875    34876    +1     
+ Misses                1450     1449    -1     
  Partials               603      603           
Flag Coverage Δ
unittests 94.44% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@danceratopz
danceratopz marked this pull request as ready for review September 16, 2026 07:03
@danceratopz
danceratopz marked this pull request as draft September 16, 2026 08:14
@danceratopz danceratopz changed the title chore(test-forks): exclude unscheduled BPO forks from fixture generation chore(test-forks): add descriptive Amsterdam BPO test scenarios Sep 16, 2026
@danceratopz

Copy link
Copy Markdown
Member Author

Simplified fill CI to extend the existing Amsterdam matrix entry through BPODecrease and removed the separate BPO job. The existing not slow and primary_format filter selects 4,658 standalone synthetic BPO cases; synthetic transition tests remain excluded by slow. Release generation remains capped at Amsterdam for devnets and BPO2 for mainnet.

Warning

Post-Amsterdam BPO transitions require client fork-order support before rollout. Hive aliases alone do not provide it. These synthetic scenarios remain excluded from automatic releases pending client support. Test releases will not include these scenarios without informing client teams first and confirming the required client support.

Validation: just static and just lint-actions passed; collection verified the standalone selection. No optimization or slow-marker changes are included.

@danceratopz

Copy link
Copy Markdown
Member Author

@spencer-tb I'm a bit unsure about the new descriptive names. See where it sits with you. But the client-side requires a change to update them to be based on Amsterdam either way. The test IDs will be more descriptive with this change and as they won't necessary map to the actual BPO3 and BPO4 forks, a rename for such tests will be required at some point.

Btw, the Amsterdam fill job has grown quite a lot with this inclusion, presumably the Osaka matrix entry has shrunk. Happy to revisit this to speed-up CI if need be. Will let a review land in the meantime, thank you.
https://github.com/ethereum/execution-specs/pull/3597/changes#diff-245392b692a50c38ecab4381b118862db514035c10983f3bd4f4b7f1f4be4692L104

@spencer-tb

Copy link
Copy Markdown
Contributor

Thanks for working on this! The new BPOIncrease/BPODecrease testing forks are definitely the right approach :)

A few changes/additions and alignment with yesterday’s ACDT:

  1. Keep the Osaka testing range through BPO4. Use until: BPO4 for the osaka label in fork-ranges.yaml and restore --until=BPO4 in feature.yaml. Update the new release matrix assertion from BPO2 to BPO4 too.

  2. Keep the fusaka devnet-4 regression case on both transitions. Within test_blob_reserve_price_with_bpo_transitions.py. BPOIncrease uses BPO3’s schedule and Amsterdam uses BPO2’s, so they both reproduce the same excess blob gas of 0x132CF5F.

if fork in (BPO2ToBPO3AtTime15k, AmsterdamToBPOIncreaseAtTime15k):
  1. Add EELS block validation for BPOIncrease, BPODecrease and their transitions. This could follow Jochem’s transition-fork support for validate-blocks: feat(spec-tests, spec-tools): run fork-transition fixtures through EELS #3564

  2. It looks like we have some small unit test gaps, can we add BPO2ToAmsterdamAtTime15k to the test_bpo_transition_ruleset, including an assertion that Amsterdam’s blob keys are removed. Also add BPOIncrease and BPODecrease to FORK_ORDER in generate_build_matrix.py.

@spencer-tb

Copy link
Copy Markdown
Contributor

One more thought:

Could we generate BPOIncrease, BPODecrease and their transitions from the latest development fork? So its automatic. That would avoid manually updating each when the development fork changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Area: Continuous Integration A-test-forks Area: execution_testing.forks C-chore Category: chore

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants