Skip to content

Improve local benchmark backfill and upload workflow - #982

Merged
helly25 merged 4 commits into
mainfrom
improve-local-benchmark-workflow
Oct 6, 2026
Merged

helly25 merged 4 commits into
mainfrom
improve-local-benchmark-workflow

Conversation

@helly25

@helly25 helly25 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Local benchmark backfills can now be inspected, retried, and published without reconstructing hidden batch state or manually moving reports. The workflow preserves successful partial campaigns, exposes actionable failure details, and makes upload retries safe when publication dispatch previously failed.

The history output is also easier to scan: compact mode summarizes older compatible runs, full mode retains aligned detailed rows, and dataset metadata makes the selected measurement series explicit. A design document records the next fixture-layout revision without changing current CI measurements.

AG;DR

Local collection and status

  • Report status before its detail, using ok for complete observations and n/a when no useful detail exists.
  • Format merge subjects as PR NUMBER: TITLE and include time alongside the revision date.
  • Collapse older history into date/PR ranges with state counts while retaining the latest ten revisions.
  • Add --full for the complete, headed, aligned revision list and advertise it from compact output.
  • Preserve and reuse the latest nonempty historical build-library path when later batches omit it.
  • Surface captured build failure details and retain failed status independently from successful observations.
  • Keep retry planning based on missing revisions; completed observations remain reusable.

Upload and publication

  • Add an upload action to the benchmark entry point.
  • Discover eligible local datasets under the benchmark root and present a numbered picker showing series, ready/uploaded state, and completed/total revisions.
  • Treat an interactive selection as consent to validate, retain, push, and dispatch publication; cancellation performs no mutation.
  • Retain explicit repeatable --batch selection and -Y for automation.
  • Import only validated completed observations, so failed revisions do not block partial campaigns.
  • Make upload idempotent and still dispatch benchmark publication when raw observations were already pushed. This repairs the case where the initial push succeeded but GitHub authentication prevented workflow dispatch.
  • Keep source checkouts, binaries, and logs local; only frozen contracts, status, and completed reports are retained.

Dataset discovery and presentation

  • Identify the selected published dataset in measurement inventory output.
  • Improve dataset-list spacing and alignment and make metadata/list boundaries explicit.
  • Validate that compatible observations from separate local batches retain a common dataset recipe.
  • Turn the Version and Performance overlays into bold-titled disclosure cards that collapse to their title and border.
  • Move the version slider below its metadata, span the card width, and add endpoint-aware previous/next buttons.
  • Move Scale, Range, and Out of range controls into the Performance card and shorten its dynamic title to the selected scale.
  • Make threshold-plane mode persistent from the lowest level, with a full-width slider and step buttons shown only while the plane is active.

Fixture-layout design

  • Add a detailed design for versioned broad/deep fixture layouts with nested reusable count anchors.
  • Register the design document in the release-site page map so internal links resolve in published documentation.
  • Specify dataset identity, viewer boundaries, preparation lifecycle, retry/sharding keys, local-first evaluation, obsolete four-worker dataset retirement, and CI adoption gates.
  • Keep current CI layouts unchanged until measurements on the participating local machines justify a separate decision.
  • Reserve a future flat extreme without collecting or exposing an empty dataset now.

Validation

  • pre-commit run -a
  • bazel test //tools:benchmark_test //tools:benchmark_backfill_test //tools:benchmark_local_test //tools:benchmark_datasets_test
  • bazel test //tools:benchmark_landscape_test
  • Node 24 renderer build and all Playwright Chromium tests in tools/benchmark_web
  • bazel test //tools:release_site_test
  • trunk check --no-fix
  • Markdown table alignment checks for all changed benchmark/design documents.
  • git diff --check

The upload path writes to the existing coverage-pages retention branch and dispatches the trusted benchmark_pages.yml workflow from main. It does not execute code from retained measurement data.

@helly25
helly25 merged commit 103fbf6 into main Oct 6, 2026
31 checks passed
@helly25
helly25 deleted the improve-local-benchmark-workflow branch October 6, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant