Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions .github/workflows/ci-failure-fix.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,11 +181,15 @@ Persist each KBE's dedup verdict to `/tmp/gh-aw/agent/dedup/<kbe>.txt` so later
For each surviving KBE, locate the failure and a likely cause.

1. **Reproduce the signature.** From the `Build:` link, walk the AzDO timeline (`/builds/{id}/timeline?api-version=7.1`, reconstruct via `parentId`), find the failing task/Helix work item, and `curl -s "$log_url" | tee /tmp/gh-aw/agent/failure_<kbe>.log`. Confirm the KBE's `Error Message` substring actually appears in the log. If it does not, the KBE may be stale or already fixed -> `-> skipped: signature no longer reproduces in cited build`.
2. **Locate the failing test/source.** From the test name or compile error, find the owning file(s) at `HEAD`. Read them. For build breaks, read the failing compile unit and the cited error code/line.
3. **Identify the regression-introducing change (attribution gates).** Prefer the *first-bad-commit* range over raw `git blame`:
2. **Freshness and live-leg gate (mandatory, before any fix attempt).** A KBE only deserves a fix if the failure still happens, on a leg that still runs, with the bug still present at `HEAD`. Run all three checks; the first one that trips ends this KBE with the recorded skip — no PR, no loop-in comment.
- **Recent occurrence.** List recent completed builds of the KBE's definition (`/builds?definitions=<id>&statusFilter=completed&%24top=20&api-version=7.1`) and check the signature against the newest ones, alongside any Build Analysis occurrence data already in the KBE body. If the most recent occurrence is older than 14 days, the failure was almost certainly fixed or retired already -> `-> skipped: no occurrence in last 14d, likely already fixed or retired`.
- **Live leg.** Confirm the leg from `Build error leg or test failing:` still runs: it appears in the timeline of the most recent completed build of that definition, and its job/queue/platform is still defined in the pipeline YAML at `HEAD` (`eng/pipelines/**`). Retired legs (removed Mono mobile or wasm legs, deleted queues, dropped OS images) cannot be fixed by a PR -> `-> skipped: failing leg retired, no longer runs at HEAD`.
- **Fix already landed.** Read the failing source at `HEAD` and `git log --oneline --since=<KBE creation date> -- <failing file(s)>`. If a merged change already removed the faulty code path, or the KBE cites a fix PR that has since merged, the KBE is stale -> `-> skipped: fix already present at HEAD; KBE stale`.
3. **Locate the failing test/source.** From the test name or compile error, find the owning file(s) at `HEAD`. Read them. For build breaks, read the failing compile unit and the cited error code/line.
4. **Identify the regression-introducing change (attribution gates).** Prefer the *first-bad-commit* range over raw `git blame`:
- Anchor on the KBE's `First build it occurred` commit/date. `git log --oneline --since=<a few days before first-occurrence> -- <failing file>` to find candidate PRs that touched the failing file/function/test before the first failing build.
- **Exclude** formatting-only, bulk-rename, file-move, dependency-bump, container-digest, and bot/codeflow commits (authors like `dotnet-maestro[bot]`, `github-actions[bot]`, codeflow merges) unless clearly causal.
- Require **>= 2 evidence points** before attributing to an author: (a) the failure first appears after their merge, (b) their change touched the failing file/function/test, (c) the change is topically related to the failure (same API/area).
- Require **>= 2 evidence points** before attributing to an author: (a) the failure first appears after their merge, (b) their change touched the failing file/function/test, (c) the change is topically related to the failure (same API/area).
- If confidence is high, record at most ONE likely author handle. If confidence is low, record the candidate as a "possible related PR" link with NO author handle.

### Step 5 — Decide: confident fix, help-wanted PR, or loop-in comment
Expand Down Expand Up @@ -285,7 +289,7 @@ Per KBE, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`:

`<outcome>` is one of: `fix-PR #aw_<id>` (confident), `help-PR #aw_<id>` (needs review), `loopin-comment #<kbe>`, `skipped: <reason>`.

Recognized skip reasons (reuse these phrasings so the feedback workflow's aggregation stays stable): `not yet area-labeled`, `KBE too fresh, defer to next run`, `open fix PR #<n> already exists`, `fix PR #<n> already merged; KBE may be stale`, `prior fix PR #<n> closed without merge within 30d`, `human PR #<n> already addressing`, `author already engaged on #<kbe>`, `loop-in comment already posted`, `signature no longer reproduces in cited build`, `candidate fix already present in source`, `no producible diff (JIT/GC/security/API/infra) — comment`, `cap reached`, `integrity-filtered candidate, needs human review`. The list is non-exhaustive but additions SHOULD reuse one of these phrasings.
Recognized skip reasons (reuse these phrasings so the feedback workflow's aggregation stays stable): `not yet area-labeled`, `KBE too fresh, defer to next run`, `open fix PR #<n> already exists`, `fix PR #<n> already merged; KBE may be stale`, `prior fix PR #<n> closed without merge within 30d`, `human PR #<n> already addressing`, `author already engaged on #<kbe>`, `loop-in comment already posted`, `signature no longer reproduces in cited build`, `no occurrence in last 14d, likely already fixed or retired`, `failing leg retired, no longer runs at HEAD`, `fix already present at HEAD; KBE stale`, `candidate fix already present in source`, `no producible diff (JIT/GC/security/API/infra) — comment`, `cap reached`, `integrity-filtered candidate, needs human review`. The list is non-exhaustive but additions SHOULD reuse one of these phrasings.

At end of run, print this table to the agent log:

Expand Down