ci: use host-local Fireactions caches#2926
Conversation
UnArbosFive
added
the
fireactions-cache-benchmark
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🛡️ AI Review — Skeptic (security review)VERDICT: SAFE VERY HIGH account-age scrutiny, mitigated by admin repository permission, substantive prior subtensor contributions, matching commit authorship, and no known Gittensor association; branch targets staging. Static analysis covered the complete current diff, including the latest E2E coverage-routing commit, protected credential boundaries, host-local cache discovery, artifact provenance and integrity checks, archive extraction, workflow permissions, and authoritative-origin fallbacks. Trusted AI-review instructions were unchanged. FindingsNo findings. ConclusionNo malicious behavior or security vulnerability was found in the reviewed changes. 🔍 AI Review — Auditor (domain review)VERDICT: 👍 Gittensor association UNKNOWN; author has repository admin permission and substantial recent subtensor contributions. The implementation matches the substantive PR description. Overlaps with #2853 and #2804 are incidental. This CI-only change targets Cache access fails open to authoritative origins, while artifact identity, digest, size, producer workflow, and publication ordering are validated. The latest E2E routing remains fail-safe for unknown or incomplete change sets. No auto-fix was needed. FindingsNo findings. ConclusionThe host-local cache pilot is ready to merge. The cache, snapshot mirroring, and selective E2E changes preserve trusted execution and artifact-integrity boundaries. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| payload="$RUNNER_TEMP/artifacts.json" | ||
| for attempt in 1 2 3 4 5; do | ||
| gh api "repos/$GITHUB_REPOSITORY/actions/runs/$PRODUCER_RUN_ID/artifacts?per_page=100" > "$payload" | ||
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") |
There was a problem hiding this comment.
[LOW] Count only artifacts the warming loop can process
This readiness count includes expired artifacts, but the processing query below excludes them. If a run contains only expired matching artifacts, the retry loop exits and the job succeeds after warming nothing. Apply the same expiry filter here so maintenance fails or retries instead of reporting a false success.
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") | |
| count=$(jq '[.artifacts[] | select(.expired == false) | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
Benchmark results on the standard 8 vCPU / 28 GiB turbo runner shape:
Exact remote compiler keys refresh every six hours. Scheduled clone snapshots are produced daily with a 36-hour freshness gate and now publish an immutable R2 mirror manifest for per-host polling. There is no random-runner local maintenance job because one selected VM cannot warm a shared-label fleet. Runs: exact-key verification https://github.com/RaoFoundation/subtensor/actions/runs/29456620982 ; paired/artifact benchmark https://github.com/RaoFoundation/subtensor/actions/runs/29456911456 ; cold + warm PR attempts https://github.com/RaoFoundation/subtensor/actions/runs/29456621901 |
| payload="$RUNNER_TEMP/artifacts.json" | ||
| for attempt in 1 2 3 4 5; do | ||
| gh api "repos/$GITHUB_REPOSITORY/actions/runs/$PRODUCER_RUN_ID/artifacts?per_page=100" > "$payload" | ||
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") |
There was a problem hiding this comment.
[LOW] Count only artifacts the warming loop can process
This count includes expired artifacts and any name with the try-runtime prefix. Such an artifact can end polling even though the loop skips it or download-artifact.sh rejects its name, producing misleading maintenance results. Match the downloader’s exact allowlist and the loop’s expiry filter.
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") | |
| count=$(jq '[.artifacts[] | select(.expired == false) | select(.name == "mainnet-snapshot" or .name == "try-runtime-snap-v0.10.1-mainnet" or .name == "try-runtime-snap-v0.10.1-testnet" or .name == "try-runtime-snap-v0.10.1-devnet")] | length' "$payload") |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
Final validation note: a subsequent run reached 597 Rust hits / 0 misses, which exposed that sccache omits the language-specific miss row at zero. The gate initially parsed that omission as empty and failed despite a perfect cache result. Commit e339b14 normalizes omitted counters to zero; the final validation is green with 596 hits / 1 miss on the fill pass and 597 hits / 0 misses (100%) after the second cargo clean. Final validation: https://github.com/RaoFoundation/subtensor/actions/runs/29458291945 |
| payload="$RUNNER_TEMP/artifacts.json" | ||
| for attempt in 1 2 3 4 5; do | ||
| gh api "repos/$GITHUB_REPOSITORY/actions/runs/$PRODUCER_RUN_ID/artifacts?per_page=100" > "$payload" | ||
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") |
There was a problem hiding this comment.
[LOW] Count only artifacts the warming loop can process
This readiness check counts expired artifacts and any name beginning with try-runtime-snap-v0.10.1-, while the loop excludes expired entries and download-artifact.sh accepts only three exact network names. An unusable artifact can therefore stop polling before a usable one exists. Apply the processing loop/downloader’s expiry and exact-name filters here.
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") | |
| count=$(jq '[.artifacts[] | select(.expired == false) | select(.name == "mainnet-snapshot" or .name == "try-runtime-snap-v0.10.1-mainnet" or .name == "try-runtime-snap-v0.10.1-testnet" or .name == "try-runtime-snap-v0.10.1-devnet")] | length' "$payload") |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
UnArbosFive
removed
the
fireactions-cache-benchmark
UnArbosFive
changed the title
| payload="$RUNNER_TEMP/artifacts.json" | ||
| for attempt in 1 2 3 4 5; do | ||
| gh api "repos/$GITHUB_REPOSITORY/actions/runs/$PRODUCER_RUN_ID/artifacts?per_page=100" > "$payload" | ||
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") |
There was a problem hiding this comment.
[LOW] Count only artifacts the warming loop can process
This count still includes expired artifacts and any name beginning with the try-runtime prefix, while the loop excludes expired entries and download-artifact.sh accepts only three exact try-runtime names. An expired or similarly prefixed artifact can make the retry loop stop despite having nothing usable, or make the warming step fail. Apply the same expired == false and exact-name allowlist predicates here and in the selection at line 69.
| count=$(jq '[.artifacts[] | select(.name == "mainnet-snapshot" or (.name | startswith("try-runtime-snap-v0.10.1-")))] | length' "$payload") | |
| count=$(jq '[.artifacts[] | select(.expired == false) | select(.name == "mainnet-snapshot" or .name == "try-runtime-snap-v0.10.1-mainnet" or .name == "try-runtime-snap-v0.10.1-testnet" or .name == "try-runtime-snap-v0.10.1-devnet")] | length' "$payload") |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
| publisher.put( | ||
| f"{MIRROR_PREFIX}/{args.artifact_name}/latest.json", |
There was a problem hiding this comment.
[LOW] Prevent stale writers from replacing the latest manifest
Both the scheduled snapshot producer and per-PR prewarm job call this publisher. If a prewarm selects artifact A, then a scheduled refresh publishes newer artifact B before the prewarm finishes, this unconditional final PUT replaces latest.json with A and gives it a new published_at. Hosts can consequently prefetch an older snapshot until another publication. Preserve monotonic freshness—for example, publish latest.json only from the scheduled producer, or conditionally replace it only when the existing manifest has an older artifact/production timestamp.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👍 |
1 participant
What changed
node-subtensor-runtimeCargo feature graph aftercargo cleanWhy
Fireactions VMs are ephemeral while their physical hosts persist. Immutable compiler objects and large snapshot artifacts can therefore be reused locally instead of being transferred again for every VM, while R2 and GitHub Actions remain authoritative origins.
A GitHub Actions maintenance job can warm only the physical host selected for that job, so it cannot maintain a fleet sharing one production label. Snapshot freshness is now published through R2 for each host to poll. Compiler objects remain content-addressed: a cold local lookup falls through to a remote cache hit and then remains local for later VMs.
The former warm workflow compiled the whole workspace, but representative CI selected only
node-subtensor-runtime. Cargo feature unification produced different rustc/sccache keys. The exact-key warmer now cleans the job-local target directory and runs the same command as CI.Security and failure behavior
Validation