Skip to content

Harden agent workflow bootstrap and tool setup - #396

Merged
HardMax71 merged 3 commits into
mainfrom
aw-artifact-cache
Aug 13, 2026
Merged

HardMax71 merged 3 commits into
mainfrom
aw-artifact-cache

Conversation

@HardMax71

@HardMax71 HardMax71 commented Aug 12, 2026 •

Copy link
Copy Markdown
Owner

The lane installers die when GitHub's release CDN 503s, and replay proved the outage sticks around (#392, #393). This makes the review lane's copilot install survive it.

Both engine blocks pin copilot 1.0.79, so the installer asks for one fixed artifact. A cache block keeps the toolcache copy of that version across runs, a post-install step seeds the toolcache from wherever the installer put the binary, and the detection job restores the same cache before its own installer runs. The installer's toolcache lookup finds the seeded copy and skips the download, and since the agent job's cache save lands before detection starts, even the first run's detection leg is covered.

The awf bundle still comes off the release CDN every run, since its installer has no toolcache lookup to feed. That half needs an upstream change. The mention and triage lanes keep downloading too; the same pattern extends to them once this proves out.

On the tool bench, scripts/tools.sh is gone. Every consumer queries cargo metadata with jq directly, the shared tool cache keys off hashFiles on the manifest, and taiki-e installs carry fallback none so a bad pin fails loudly instead of quietly building from source. The contract tests check the cargo form in the composite, the platform lane, and both ratchet hooks, and refuse the shim coming back.


Summary by cubic

Hardens the agent review lane against GitHub Releases CDN 503s and makes tool setup deterministic. The review lane now pins Copilot CLI 1.0.79, restores it from the runner toolcache, warms the build cache, and extends the timeout from 20 to 30 minutes.

  • Details

    • Pins the engine and installers to 1.0.79; exports COPILOT_CLI_VERSION and GH_AW_ENGINE_VERSION to keep one version across the lane and lock.
    • Restores Copilot CLI from ${{ runner.tool_cache }} via actions/cache/restore, seeds ${{ runner.tool_cache }}/copilot-cli/1.0.79/<arch>/bin/copilot, and saves with actions/cache/save before the agent runs; detection restores the same cache and skips downloads.
    • Adds shared-key: gates to Swatinem/rust-cache to warm the lane build cache from the gates job (applied to review, mention, triage, and CI).
    • Replaces the scripts/tools.sh shim with direct cargo metadata --locked --format-version 1 --no-deps --offline queries; the tools composite keys its cache with hashFiles('Cargo.toml') + runner.arch and sets fallback: none in taiki-e/install-action.
    • CI reads the cargo-nextest pin from cargo metadata and installs with fallback: none; pre-commit hooks read the similarity-rs pin from the source table.
    • Contract tests assert cargo metadata usage, cache keying, absence of the shim, and one Copilot version across the lane and lock. AWF binary downloads remain unchanged.
  • Migration

    • Replace any scripts/tools.sh usage with cargo metadata --locked --format-version 1 --no-deps --offline | jq ... as shown in the tools composite and hooks.

Written for commit cc55370. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Improvements

    • Tool versions are now sourced consistently from the project’s Cargo metadata.
    • CI and development checks use locked tool definitions with stricter installation behavior.
    • Caching for review tooling and command-line utilities has been improved.
    • Review engines and related tooling are pinned to a consistent version for more predictable results.
  • Documentation

    • Updated development guidance to reflect the streamlined tool-version configuration and update process.

Copilot AI lite review requested due to automatic review settings August 12, 2026 22:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c23f320b-d80a-4304-a7cf-eac12b65a1e6

📥 Commits

Reviewing files that changed from the base of the PR and between a604d65 and cc55370.

📒 Files selected for processing (8)
  • .github/workflows/agent-mention.lock.yml
  • .github/workflows/agent-mention.md
  • .github/workflows/agent-review.lock.yml
  • .github/workflows/agent-review.md
  • .github/workflows/agent-triage.lock.yml
  • .github/workflows/agent-triage.md
  • .github/workflows/ci.yml
  • crates/amiss/tests/documentation_contracts/tools.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • .github/workflows/ci.yml
  • .github/workflows/agent-review.md
  • .github/workflows/agent-review.lock.yml
  • crates/amiss/tests/documentation_contracts/tools.rs

📝 Walkthrough

Walkthrough

The change removes scripts/tools.sh as the tool metadata reader. CI, hooks, actions, tests, and documentation now use locked Cargo metadata. Agent workflows pin Copilot CLI 1.0.79, configure shared Rust caching, and cache the architecture-specific CLI toolcache.

Changes

Cargo metadata tool resolution

Layer / File(s) Summary
Cargo metadata consumers
.github/actions/tools/action.yml, .github/workflows/ci.yml, .pre-commit-config.yaml, Cargo.toml
Tool lists and versions now come from locked offline Cargo metadata. Prebuilt installation disables fallback. Cache keys hash Cargo.toml.
Metadata contract validation
crates/amiss/tests/documentation_contracts/tools.rs
Contract tests validate metadata structure, projected tool data, workflow queries, cache inputs, ratchet hooks, and Copilot CLI version consistency.
Metadata documentation
docs/src/development.md
Development documentation identifies manifest metadata as the source for tool versions.

Versioned Copilot CLI caching

Layer / File(s) Summary
Agent workflow configuration
.github/workflows/agent-review.md, .github/workflows/agent-review.lock.yml, .github/workflows/agent-mention.*, .github/workflows/agent-triage.*
Agent workflows use Copilot CLI 1.0.79 where configured, shared Rust cache key gates, updated metadata hashes, and 30-minute agent timeouts.
Copilot CLI cache and toolcache setup
.github/workflows/agent-review.md, .github/workflows/agent-review.lock.yml
The review workflow restores and saves versioned CLI caches, installs Copilot CLI 1.0.79, and seeds the architecture-specific runner toolcache.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Mergeability Score: ⚪ Minimal · up to cc553

The workflow and tooling changes introduce no identified merge-blocking risk at the current head; the PR is merge-ready after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant AgentReview
  participant ActionsCache
  participant RunnerToolcache
  participant CopilotCLI
  AgentReview->>ActionsCache: Restore versioned CLI cache
  AgentReview->>CopilotCLI: Install Copilot CLI 1.0.79
  AgentReview->>RunnerToolcache: Seed architecture-specific executable
  AgentReview->>ActionsCache: Save the CLI toolcache
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes to harden agent workflow bootstrap and tool setup.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch aw-artifact-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/agent-review.md:
- Around line 30-41: Update the restore steps in
.github/workflows/agent-review.md (lines 30-41) and
.github/workflows/agent-review.lock.yml (lines 457-462 and 1314-1318) to use
continue-on-error: true, then add an explicit actions/cache/save step after the
Copilot CLI tool cache is seeded so saving does not depend on job success.
Regenerate .github/workflows/agent-review.lock.yml from the updated workflow.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c6887a02-b0d7-4fdd-b0e0-247eeb785e02

📥 Commits

Reviewing files that changed from the base of the PR and between 2689f66 and a604d65.

📒 Files selected for processing (9)
  • .github/actions/tools/action.yml
  • .github/workflows/agent-review.lock.yml
  • .github/workflows/agent-review.md
  • .github/workflows/ci.yml
  • .pre-commit-config.yaml
  • Cargo.toml
  • crates/amiss/tests/documentation_contracts/tools.rs
  • docs/src/development.md
  • scripts/tools.sh
💤 Files with no reviewable changes (1)
  • scripts/tools.sh

Comment thread .github/workflows/agent-review.md Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread .github/workflows/agent-review.md Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Tool bootstrap is right and the copilot cache works under the current pin, but the version literal is written six times per workflow and a partial bump would silently disable the cache.

Question: the copilot version literal 1.0.79 is hardcoded in six places in agent-review.md (engine.version line 19, cache key/path 31-32, detection restore key/path 40-41, seed target line 58, safe-outputs engine line 72) and re-emitted in the compiled lock; a bump that edits only engine.version silently disables the new cache — should the key and path derive the version from it?

Because the cache key and the seed target also hardcode the version, a partial bump makes the seed step copy the new binary under the old version's toolcache directory, while the installer's exact-version lookup (install_copilot_cli.sh v0.86.2) can never match the new version: the restore keeps hitting the stale key and the installer downloads on every run — the CDN-503 failure mode this PR removes — leaving a wrong-version binary under the old label.

confirmation: simulated the seed step with a version mismatch — the new binary landed at copilot-cli/1.0.79/x64/bin/copilot, and the installer's own lookup shape (find -maxdepth 4 -path '*/bin/copilot' -path '*/1.0.80/*') returned zero results.

Nit: the seed step reimplements the uname -m → x64|arm64 mapping the installer already owns and exit 1s on unknown arches; both maps agree today but can drift — agent-review.md:52 vs install_copilot_cli.sh lines 105-113. Reading the arch from ${{ runner.arch }} would keep a single source.

confirmation: diffed both maps — identical four-name mapping today.

Session details

run — cargo nextest run --workspace --locked (1364 passed, 3 skipped), amiss check --repo . --object-format sha1 --profile enforce (exit 0), the six documentation_contracts tools tests, all three cargo metadata jq queries, taiki-e manifest version checks at both pinned SHAs, gh-aw-actions v0.86.2 installer reading, and a sandbox simulation of the seed step.

Comment thread .github/workflows/agent-review.md Outdated
Comment thread .github/workflows/agent-review.md Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Everything holds — the copilot pin, the toolcache seeding, and the cargo-metadata queries all verify against the tree and the pinned upstream installer.

No findings were confirmed. The load-bearing claims were checked against the tree, the pinned action sources, and live release metadata.

  • The seeded layout ${RUNNER_TOOL_CACHE}/copilot-cli/1.0.79/${arch}/bin/copilot matches what gh-aw's installer (install_copilot_cli.sh at the pinned sha) scans: find -maxdepth 4 -path '*/bin/copilot', exact-version match, arch names x64/arm64, and the cached-copy activation prepends the bin dir to PATH, so the seed guard short-circuits on a hit. The v prefix the installer adds to the version makes the download URL point at the real tag.
  • The v1.0.79 release of github/copilot-cli publishes copilot-linux-x64.tar.gz; the exact pinned artifact exists.
  • The three jq projections (the tools composite, the platform-tests nextest pin, and both ratchet hooks) emit exactly what the deleted scripts/tools.sh produced; hashFiles('Cargo.toml') covers every pin edit since the pins live in the root manifest.
  • fallback: none is a documented input of the pinned taiki-e/install-action revision.
  • The toolcache restore in the detection leg lands after the agent job's save step (save runs before the agent, detection needs: agent), so even the first run's detection leg hits the cache.
Session details

run

Ran: cargo nextest run --locked -p amiss --test documentation_contracts (32/32 passed, including all seven tools tests), a simulation of the installer's toolcache scan against the seeded layout (exact match, no download), the three jq expressions against the live manifest, and a check of the v1.0.79 release assets and both pinned installers' sources.

@HardMax71
HardMax71 merged commit e06f444 into main Aug 13, 2026
22 checks passed
@HardMax71
HardMax71 deleted the aw-artifact-cache branch August 13, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants