Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions .github/workflows/impl-generate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1016,6 +1016,11 @@ jobs:
# Pass via env (not template-interpolated into the script) so hints
# containing quotes / $ / backticks can't break shell parsing.
CHANGE_REQUEST: ${{ inputs.change_request }}
# How far back a failure marker still counts toward the 3-attempt cap.
# Covers a whole campaign (a generate→review→repair→merge cycle runs
# well under an hour) with room for a stalled tail, without letting
# last month's failures veto today's retries.
CAMPAIGN_WINDOW_H: '12'
run: |
echo "::notice::Handling generation failure for $LIBRARY/$SPEC_ID"

Expand All @@ -1035,11 +1040,25 @@ jobs:
# (the old code broke on issues with >100 comments for this reason).
# pipefail (Actions bash default) makes a gh failure fail the whole
# pipeline, so the fail-closed else-branch still triggers.
#
# CAMPAIGN-SCOPED: only markers younger than CAMPAIGN_WINDOW_H count.
# The markers are the permanent audit trail — nothing deletes them —
# so counting all of them made the "3 attempts" cap per-issue-lifetime
# instead of per-generation-run. Any (spec, library) pair that ever
# failed twice then got exactly ONE attempt on every future dispatch,
# and a single hit of the ~9%/run "agent reports success but writes no
# file" flake parked it under `impl:<lib>:failed`, which nothing
# retries (the watchdog reports it as "needs manual attention").
# Measured 2026-08-24: counts of 3 and 4 on pairs that then succeeded
# on the very next manual dispatch, and 87 pairs parked repo-wide.
# A generation campaign is minutes long, so a window of hours is
# generous while still letting stale history age out on its own.
MARKER="<!-- impl-fail:${SPEC_ID}:${LIBRARY} -->"
CAMPAIGN_CUTOFF=$(date -u -d "${CAMPAIGN_WINDOW_H} hours ago" +%Y-%m-%dT%H:%M:%SZ)
if FAILURE_COUNT=$(gh api --paginate "repos/${{ github.repository }}/issues/${ISSUE}/comments?per_page=100" \
--jq "[.[] | select(.body != null and (.body | contains(\"$MARKER\")))] | length" \
--jq "[.[] | select(.body != null and (.body | contains(\"$MARKER\")) and .created_at > \"$CAMPAIGN_CUTOFF\")] | length" \
| awk '{ sum += $1 } END { print sum + 0 }'); then
echo "::notice::Previous failures for ${LIBRARY}/${SPEC_ID}: $FAILURE_COUNT"
echo "::notice::Previous failures for ${LIBRARY}/${SPEC_ID} since ${CAMPAIGN_CUTOFF}: $FAILURE_COUNT"
else
echo "::warning::Failure-count API call failed — failing closed (treating retry cap as reached, no auto-retry)"
FAILURE_COUNT=999
Expand All @@ -1056,7 +1075,13 @@ jobs:
if [ "$FAILURE_COUNT" -eq 999 ]; then
CAP_REASON="the failure counter could not be read (failed closed — auto-retry disabled)"
else
CAP_REASON="3 generation attempts"
# State the real total, not a flat "3 attempts": a pair capped by
# stale history had ONE attempt today, and reading the old wording
# as "tried three times, must be a capability gap" is exactly how
# recoverable pairs got written off (2026-08-24).
# ATTEMPT, not FAILURE_COUNT: the latter excludes the failure being
# handled right now, so at the cap it would under-report by one.
CAP_REASON="${ATTEMPT} failed attempt(s) in the last ${CAMPAIGN_WINDOW_H}h (cap: 3 per campaign)"
fi
echo "::warning::Marking $LIBRARY as failed: $CAP_REASON"

Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,22 @@ aggregate instead: an italic *Catalog* line at the end of the version section an

### Fixed

- **The generation retry cap counted a library's entire failure history, not the current
run** — `impl-generate.yml` derives its "3 attempts" budget from hidden marker comments
on the spec issue, and nothing ever deletes those markers. A `(spec, library)` pair that
failed twice at any point in the past was therefore capped forever: every later dispatch
got a single attempt, and one hit of the intermittent "agent reports success but writes
no implementation file" failure (8 of 85 generate runs on 2026-08-24) parked it under
`impl:<library>:failed`, which nothing retries — the watchdog only reports it as needing
manual attention. Repo-wide that state had accumulated on **87 `(spec, library)` pairs**.
Marker counting is now scoped to a 12-hour campaign window, so stale history ages out and
a fresh dispatch gets its full three attempts. The fail-closed behaviour on an unreadable
counter (issue #1010: ~1,200 runs in 38 h) is untouched (#10627).
- **A failed library no longer claims three attempts it never made** — the cap message read
`Marking <lib> as failed: 3 generation attempts` even when the run made exactly one, which
reads as "tried repeatedly, must be a genuine capability gap". It now names the counted
failures and the window. Three pairs written off under the old wording on 2026-08-24
succeeded on the very next dispatch (#10627).
- **Image structured data carries the licensing fields Google recommends** — Search
Console flagged every implementation page's `ImageObject` for missing `creator`,
`copyrightNotice`, `creditText`, and `acquireLicensePage`. The bot-page JSON-LD now
Expand Down
Loading