Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 1 addition & 2 deletions .github/workflows/deno-deno-cargo-product.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: 'deno: Deno Cargo product run'
run-name: "${{ format('Deno Cargo fresh | {0} | CLI {1}', inputs.cache_profile ||
'cargo-product', inputs.cli_version || 'Action default') }}"
run-name: "Deno Cargo / ${{ inputs.cache_profile || 'cargo-product' }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
actions: read
contents: read
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/deno-deno-cargo-rolling-chain.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
---
name: 'deno: Deno Cargo rolling run'
run-name: "${{ github.event_name == 'push' && 'Deno Cargo rolling pinned source update'
|| format('Deno Cargo rolling {0} | {1} to {2}', inputs.cache_profile || 'cargo-product',
inputs.base_sha, inputs.head_sha) }}"
run-name: "Deno Cargo / ${{ inputs.cache_profile || 'cargo-product' }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.base_sha }} to ${{ inputs.head_sha }}"
permissions:
actions: read
contents: read
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/grpc-grpc-bazel-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
---
name: 'grpc: gRPC Bazel Benchmark'
run-name: "${{ format('Benchmark {0} | {1}', inputs.cli_version != '' && 'Canary'
|| 'Rolling', github.event_name == 'push' && 'Upstream update' || format('Manual
dispatch ({0})', github.ref_name)) }}"
run-name: "gRPC Bazel | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/grpc-grpc-bazel-fresh-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
---
name: 'grpc: gRPC Bazel Fresh'
run-name: "${{ format('Benchmark {0} | {1}', github.event_name == 'pull_request' &&
'Validation' || inputs.cli_version != '' && 'Canary' || 'Fresh', github.event_name
== 'schedule' && 'Weekly' || github.event_name == 'pull_request' && format('PR #{0}',
github.event.pull_request.number) || format('Manual dispatch ({0})', github.ref_name))
}}"
run-name: "gRPC Bazel | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/native-fresh-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: Native comparison fresh
run-name: "${{ inputs.case_id }} ${{ inputs.variant }} fresh ${{ inputs.series_id }} sample ${{ inputs.sample
}}"
run-name: "${{ inputs.case_id }}${{ inputs.variant != '' && format(' / {0}', inputs.variant) || '' }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/native-rolling-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: Native comparison rolling
run-name: "${{ inputs.case_id }} ${{ inputs.variant }} rolling ${{ inputs.series_id
}} sample ${{ inputs.sample }}"
run-name: "${{ inputs.case_id }}${{ inputs.variant != '' && format(' / {0}', inputs.variant) || '' }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/nix-fresh-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: Nix comparison fresh
run-name: "${{ inputs.case_id }} fresh ${{ inputs.series_id }} sample ${{ inputs.sample
}}"
run-name: "${{ inputs.case_id }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/nix-rolling-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: Nix comparison rolling
run-name: "${{ inputs.case_id }} rolling ${{ inputs.series_id }} sample ${{ inputs.sample
}}"
run-name: "${{ inputs.case_id }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/obs-rolling-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: OBS Studio comparison rolling
run-name: obs-studio ${{ inputs.cache_tool }} rolling ${{ inputs.series_id }} sample
${{ inputs.sample }}
run-name: "obs-studio / ${{ inputs.cache_tool || 'all' }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
'on':
workflow_dispatch:
inputs:
Expand Down
5 changes: 1 addition & 4 deletions .github/workflows/obs-studio-obs-actions-cache.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---
name: 'obs-studio: OBS Studio Actions Cache benchmark'
run-name: "${{ github.event_name == 'pull_request' && format('OBS validation | PR
#{0}', github.event.pull_request.number) || github.event_name == 'push' && 'OBS
rolling | Upstream update' || format('OBS Actions Cache | Manual dispatch ({0})',
github.ref_name) }}"
run-name: "obs-studio / ${{ inputs.cache_tool || 'all' }} / Actions Cache | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
actions: read
contents: read
Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/obs-studio-obs-boringcache.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
---
name: 'obs-studio: OBS Studio BoringCache benchmark'
run-name: "${{ github.event_name == 'pull_request' && format('OBS validation | PR
#{0}', github.event.pull_request.number) || inputs.cli_version != '' && format('OBS
canary | CLI {0}', inputs.cli_version) || github.event_name == 'push' && 'OBS rolling
| Upstream update' || format('OBS BoringCache | Manual dispatch ({0})', github.ref_name)
}}"
run-name: "obs-studio / ${{ inputs.cache_tool || 'all' }} / BoringCache | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
actions: read
contents: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/reapi-fresh-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: REAPI comparison fresh
run-name: "${{ inputs.case_id }} fresh ${{ inputs.series_id }} sample ${{ inputs.sample
}}"
run-name: "${{ inputs.case_id }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/reapi-rolling-benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: REAPI comparison rolling
run-name: "${{ inputs.case_id }} rolling ${{ inputs.series_id }} sample ${{ inputs.sample
}}"
run-name: "${{ inputs.case_id }} | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
contents: read
actions: read
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/source-case.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ permissions:
actions: read
jobs:
previous:
name: '${{ inputs.case_id }} previous rolling run'
name: Reconcile previous runs
runs-on: ubuntu-24.04
timeout-minutes: 10
permissions:
Expand Down Expand Up @@ -48,7 +48,7 @@ jobs:
if-no-files-found: ignore
retention-days: 90
inspect:
name: '${{ inputs.case_id }} source check'
name: Check source
needs: previous
if: needs.previous.outputs.proceed == 'true'
runs-on: ubuntu-24.04
Expand Down Expand Up @@ -87,7 +87,7 @@ jobs:
if-no-files-found: error
retention-days: 90
publish:
name: '${{ inputs.case_id }} queue rolling runs'
name: Prepare rolling runs
needs: [previous, inspect]
if: needs.inspect.outputs.changed == 'true'
runs-on: ubuntu-24.04
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/zed-zed-cargo-product.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: 'zed: Zed Cargo layer matrix'
run-name: "${{ format('Zed Cargo layers | CLI {0}', inputs.cli_version || 'Action default') }}"
run-name: "Zed Cargo layers | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} fresh | ${{ inputs.series_id || github.ref_name }} / sample ${{ inputs.sample || '1' }}"
permissions:
actions: read
contents: read
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/zed-zed-cargo-rolling-auto.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
name: 'zed: Zed Cargo rolling run'
run-name: "${{ format('Zed Cargo rolling {0} to {1}', inputs.base_sha, inputs.head_sha)
}}"
run-name: "Zed Cargo | ${{ startsWith(inputs.cli_version, 'vcli-canary-') && 'Canary' || 'Stable' }} rolling | ${{ inputs.base_sha }} to ${{ inputs.head_sha }}"
permissions:
actions: read
checks: read
Expand Down
4 changes: 2 additions & 2 deletions cases/obs-studio/payload/scripts/install-ccache.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail

ccache_version=4.14
ccache_version=4.14.1
ccache_archive="ccache-${ccache_version}-linux-x86_64-glibc.tar.gz"
ccache_sha256=c64760b0b85ba86068f4cd162dc42e2dc39c6f46b0cb8c1990dfccbec7a1fed0
ccache_sha256=67f8b4cd2620b0d7bc9e834b3baea76bcdca4f2f3f47fe658c343fee58f816fa
ccache_url="https://github.com/ccache/ccache/releases/download/v${ccache_version}/${ccache_archive}"

storage_version=0.10
Expand Down
1 change: 1 addition & 0 deletions cases/obs-studio/payload/scripts/write_phase_result.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
raise "Phase must be base or rolling" unless %w[base rolling].include?(phase)
raise "Continuous observations require the rolling phase" if args["continuous"] && phase != "rolling"
cache_hit = args["continuous"] ? args["cache_hit"] : (phase == "base" ? "false" : "true")
cache_hit = nil if cache_hit == ""
raise "Cache hit must be true or false" if cache_hit && !%w[true false].include?(cache_hit)
build_seconds = Float(args.fetch("build_seconds"))
raise "Build time must be finite and nonnegative" unless build_seconds.finite? && build_seconds >= 0
Expand Down
45 changes: 45 additions & 0 deletions docs/cadence-cutover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Cadence cutover

The new workflows are merged, but central activation is off. The user deferred
activation until a compatible stable CLI is released. Historical cron triggers
remain active. Do not apply the prepared patches or set the activation variable
before the release and qualification gates in [cadence.md](cadence.md) pass.

The [October 5 inventory](../migration/cadence-schedule-inventory-2026-10-05.json)
records 48 scheduled workflows in the 17 historical repositories represented by
the maintained suite. It records the default branch, observed head, workflow
blob SHA, cron expressions and remaining entrypoints. The seven snapshot cases
have no replaced BoringCache repository schedules.

[Prepared patches](../migration/cadence-cutover-patches) remove only each
inventoried workflow's `schedule` event. Each edited document was parsed and
compared with the original after removing that event; manual and PR events, jobs,
permissions and other workflow behavior remain equivalent. These patches have
not been applied. Unselected repositories, including Docker corpus, are outside
this cutover.

When release and qualification are complete:

1. Resolve the exact stable CLI and run the full 31-target dry run on `main`.
Require the REAPI capability check to pass without omissions or substitution
of a canary. Confirm hosted checks and the retained qualification records.
2. Refresh the historical inventory. Review any changed workflow blobs before
applying their patches. Keep central activation off while removing all 48
historical cron triggers. Preserve their manual and PR entrypoints.
3. Query every historical repository for queued, pending and running scheduled
or source-dispatched work. Allow those runs and downstream requests to finish;
retain their run IDs. Recheck that no replaced cron triggers remain. Do not
cancel unrelated runs or treat a stopped scheduler as a drained queue.
4. Set `BENCHMARK_CADENCE_ACTIVE=true` in `boringcache/benchmarks`. This transfers
source checks, weekly/nightly dispatch and canary monitoring to the central
suite. The first hourly source check should produce per-case observations;
changed cases should retain intent and requested run IDs independently.
5. Verify the first complete source cycle and the first weekly and nightly
dispatches. Compare each receipt with the 24-case / 31-fresh-target inventory,
then inspect leaf outcomes and comparison reports. Request acceptance alone
is not build completion.

If central ownership must be stopped, unset the activation variable first and
inspect already requested central runs. Restore historical cron triggers only
after central requests have drained, so both owners do not request the same
work. Retain source receipts and observations during either transition.
66 changes: 66 additions & 0 deletions docs/cadence-qualification-2026-10-05.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Cadence qualification — October 5, 2026

All seventeen rolling targets have passed a seed run and an actual source
advancement with both providers. Zed Nix also passed both fresh samples. This is
an operational correctness review; these single-sample pairs do not establish
comparative performance. All runs use `vcli-canary-7a5b27146ebe`.

The [machine-readable inventory](../migration/cadence-qualification-2026-10-05.json)
records frozen plans, exact sources, cache scopes and selected run IDs. Earlier
failed and cancelled cohorts remain in `results/`; corrected cohorts have new
plans. The [evidence inventory](../migration/evidence.json) links archives that
were downloaded again and verified after publication.

Large archives use a JSON index as `bundle_url`. Its `sha256` and `bundle_bytes`
describe that index; `archive_sha256` and `archive_bytes` describe the reconstructed
tar.gz file. Download and verify each listed part, concatenate the parts in order,
verify the resulting archive, then extract it. The hosted preservation workflow
performs those checks and verifies every exported file before recording success.

| Case / variant | Seed | Changed source | Result |
| --- | --- | --- | --- |
| immich | [37321739971](https://github.com/boringcache/benchmarks/actions/runs/37321739971) | [37323121773](https://github.com/boringcache/benchmarks/actions/runs/37323121773) | Passed |
| mastodon / server | [37321744434](https://github.com/boringcache/benchmarks/actions/runs/37321744434) | [37323126507](https://github.com/boringcache/benchmarks/actions/runs/37323126507) | Passed |
| mastodon / streaming | [37321748690](https://github.com/boringcache/benchmarks/actions/runs/37321748690) | [37323131290](https://github.com/boringcache/benchmarks/actions/runs/37323131290) | Passed |
| n8n / distroless | [37321753318](https://github.com/boringcache/benchmarks/actions/runs/37321753318) | [37323140444](https://github.com/boringcache/benchmarks/actions/runs/37323140444) | Passed |
| n8n / docker | [37321758035](https://github.com/boringcache/benchmarks/actions/runs/37321758035) | [37323145174](https://github.com/boringcache/benchmarks/actions/runs/37323145174) | Passed |
| n8n / runners | [37321762921](https://github.com/boringcache/benchmarks/actions/runs/37321762921) | [37323150431](https://github.com/boringcache/benchmarks/actions/runs/37323150431) | Passed |
| n8n / turbo | [37321767409](https://github.com/boringcache/benchmarks/actions/runs/37321767409) | [37323155106](https://github.com/boringcache/benchmarks/actions/runs/37323155106) | Passed |
| posthog / layers | [37321772493](https://github.com/boringcache/benchmarks/actions/runs/37321772493) | [37323164071](https://github.com/boringcache/benchmarks/actions/runs/37323164071) | Passed |
| executorch-buck2 | [37324032100](https://github.com/boringcache/benchmarks/actions/runs/37324032100) | [37324204556](https://github.com/boringcache/benchmarks/actions/runs/37324204556) | Passed |
| gogs-moon | [37325135541](https://github.com/boringcache/benchmarks/actions/runs/37325135541) | [37325282767](https://github.com/boringcache/benchmarks/actions/runs/37325282767) | Passed |
| opencut-moon | [37325140807](https://github.com/boringcache/benchmarks/actions/runs/37325140807) | [37325287254](https://github.com/boringcache/benchmarks/actions/runs/37325287254) | Passed |
| stackstorm-pants | [37324025714](https://github.com/boringcache/benchmarks/actions/runs/37324025714) | [37324228460](https://github.com/boringcache/benchmarks/actions/runs/37324228460) | Passed |
| msgpack-sbt | [37324036688](https://github.com/boringcache/benchmarks/actions/runs/37324036688) | [37324214890](https://github.com/boringcache/benchmarks/actions/runs/37324214890) | Passed |
| obs-studio / ccache | [37323465858](https://github.com/boringcache/benchmarks/actions/runs/37323465858) | [37324219008](https://github.com/boringcache/benchmarks/actions/runs/37324219008) | Passed |
| obs-studio / xcode | [37326133177](https://github.com/boringcache/benchmarks/actions/runs/37326133177) | [37326340221](https://github.com/boringcache/benchmarks/actions/runs/37326340221) | Passed |
| helix-nix | [37322604229](https://github.com/boringcache/benchmarks/actions/runs/37322604229) | [37323117664](https://github.com/boringcache/benchmarks/actions/runs/37323117664) | Passed |
| zed-nix | [37322636399](https://github.com/boringcache/benchmarks/actions/runs/37322636399) | [37323174641](https://github.com/boringcache/benchmarks/actions/runs/37323174641) | Passed |

The five REAPI comparator archives identify the intended previous run and source
in `rolling-seed.json`. Both OBS comparator archives likewise identify their
intended seed in `previous-cache-source.json`. The review checks those identities
against the frozen plans; a cache flag alone is insufficient.

The first n8n Turbo advancement retained an available BoringCache entry but
reported zero native hits. A further actual upstream revision in
[run 37325183094](https://github.com/boringcache/benchmarks/actions/runs/37325183094)
reported 70 native hits and 2 misses. Both observations remain retained. Changed
source does not guarantee a hit; absence of a hit must not be reported as reuse.

Helix Nix passed two fresh cold/warm samples and its rolling pair. Zed Nix passed both fresh cold/warm samples and its rolling pair. Each Nix run imports one checksummed
dependency store into its provider workers and checks the dependency baseline.
The measured package is excluded from that seed. A changed package derivation
may require a build even when the rolling provider cache is available.

The shared controller passed a 24-case hosted planning rehearsal, and fresh
dispatch passed the complete 31-target dry run. The
[publication rehearsal](../migration/rehearsals/mastodon-publication/review.md)
passed isolated conditional publication, dispatch, reconciliation and a forced
concurrent-write conflict. It used the operator token; the first activated cycle
must verify workflow-token writes on `main`.

Central activation remains off by request until a compatible stable CLI release.
The [cutover procedure](cadence-cutover.md) preserves manual entrypoints and
drains historical runs before central ownership begins. The prepared historical
schedule patches have not been applied.
Loading