From b0c031b6488c20a8b7c166f37e16f1f478df9f34 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 21:04:21 +0100 Subject: [PATCH 1/6] fix(triage): verify a comment or body marker landed before calling a run successful Claude Code exiting cleanly only proves the process ran; nothing in this composite action ever checked whether triage's own promised output -- a new issue comment, or a body edit carrying the claude-triage:start marker -- actually landed on GitHub. Both are ordinary tool calls the model makes from its own prompt, so a run that legitimately completes without calling either still reported steps.claude.outcome == 'success', which made the progress-comment step delete the one visible trace of the run and made the composite action's own overall conclusion green. Confirmed happening for real on three separate triage runs (novus-power/hive#1490, #1657, #1661): each reported success and left nothing behind. Adds a step that re-fetches the issue's own comments and body after a triage run reports success, and folds a genuine mismatch into the run's own outcome the same way turn-limit wrap-up already does, rather than trusting the model's self-report. --- action.yml | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/action.yml b/action.yml index b1fc0f8..02f0b0a 100644 --- a/action.yml +++ b/action.yml @@ -2110,6 +2110,33 @@ runs: path: ~/.claude/projects key: ${{ steps.cache-scope.outputs.cache_key }} + # Closes a real, observed gap: "Resolve Claude Code result" above only confirms Claude Code itself ran (a real conclusion was produced), not that triage's own promised output -- a new issue comment, or a body edit carrying the `claude-triage:start` marker (prompts/triage/base.md's own "append a short triage summary to the issue's own description" section) -- actually landed on GitHub. Both are ordinary tool calls the model makes from its own prompt, not something this composite action enforces; a run that legitimately completes without ever calling either still reports steps.claude.outcome == 'success', which "Update progress comment" below already treats as "real output now exists elsewhere" and deletes the one visible trace (the progress comment itself) on that basis -- and, worse, that success also becomes this composite action's own overall conclusion, so even the Actions tab shows green. Confirmed happening for real, not just theoretically: three separate triage runs (novus-power/hive#1490, #1657, #1661) reported success and left zero comments and no marker, with the progress comment deleted and nothing to show for the run at all. This step re-checks GitHub's own state rather than trusting the model's self-report, the same principle "Structured review summary" above already applies to review mode by re-fetching what was actually submitted. Placed after every optional post-run step (Headroom reporting, turn-limit wrap-up, session save) rather than right after "Resolve Claude Code result", so a real failure here -- deliberately with no continue-on-error, for the same reason "Wrap up on turn limit" has none -- never starves an unrelated step of the implicit success() every unadorned `if:` in this file still carries; "Update progress comment" below is the one step built to run regardless (`if: always()`), which is exactly why this sits immediately before it. + - name: Verify triage output landed + id: verify-triage-output + if: ${{ inputs.mode == 'triage' && steps.claude.outcome == 'success' && steps.progress-comment.outputs.comment_id != '' }} + shell: bash + env: + GH_TOKEN: ${{ inputs.github_token || github.token }} + REPOSITORY: ${{ github.repository }} + ISSUE_NUMBER: ${{ inputs.issue_number || github.event.issue.number }} + PROGRESS_COMMENT_ID: ${{ steps.progress-comment.outputs.comment_id }} + run: | + set -euo pipefail + + # A genuine new comment posted after this run's own progress comment is real output -- the progress comment's own id is this run's "before" marker, since it was the most recent comment on the issue at the moment this run started. + NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate -q "[.[] | select(.id > ${PROGRESS_COMMENT_ID})] | length") + + # The marker's mere presence is enough: a re-triage run that replaces an existing block still counts as real output, and there is no cheap way to tell "replaced" from "left untouched" without a pre-run body snapshot this step does not have. + HAS_MARKER=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}" --jq '.body // "" | contains("")') + + if [ "$NEWER_COMMENTS" -gt 0 ] || [ "$HAS_MARKER" = "true" ]; then + echo "Verified: triage left real output (new comments: ${NEWER_COMMENTS}, marker present: ${HAS_MARKER})." + exit 0 + fi + + echo "::error::Claude Code reported success, but triage left no new comment and no marker on issue #${ISSUE_NUMBER}. Treating this run as failed rather than deleting the progress comment on an unverified success." >&2 + exit 1 + # Counterpart to "Post progress comment" above -- resolves the SAME comment (by its captured id) once the whole run has finished, including any optional wrap-up/fix/summary pass above. if: always() so it still fires when Run Claude Code itself failed outright -- exactly when a plain visible status marker matters most, since nothing else in this composite action guarantees any comment lands when a mode's own model-driven output (a review, a triage comment, an interactive reply) never got produced. - name: Update progress comment if: ${{ always() && inputs.post_progress_comment == 'true' && steps.progress-comment.outputs.comment_id != '' }} @@ -2124,6 +2151,8 @@ runs: CLAUDE_OUTCOME: ${{ steps.claude.outcome }} # "Wrap up on turn limit" is the one later step with no continue-on-error and no prior id (unlike the fix pass / structured-summary steps, which are all continue-on-error: true and so can never make an already-successful primary run look failed here) -- if it runs and itself fails, the job outcome is failure even though steps.claude.outcome is still the earlier, successful "success". Folded in below rather than trusted on its own, since it reports "skipped" (not "success") whenever detect-turn-limit didn't fire, which must not be read as a failure. WRAP_UP_OUTCOME: ${{ steps.wrap-up.outcome }} + # "Verify triage output landed" is the other step with no continue-on-error -- same reasoning, folded in the same way. It only ever runs (rather than reporting "skipped") for triage mode, so the "skipped" exemption below applies identically for every other mode. + VERIFY_TRIAGE_OUTCOME: ${{ steps.verify-triage-output.outcome }} RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} run: | set -euo pipefail @@ -2131,8 +2160,11 @@ runs: if [ "$OUTCOME" = "success" ] && [ -n "${WRAP_UP_OUTCOME:-}" ] && [ "$WRAP_UP_OUTCOME" != "success" ] && [ "$WRAP_UP_OUTCOME" != "skipped" ]; then OUTCOME="$WRAP_UP_OUTCOME" fi + if [ "$OUTCOME" = "success" ] && [ -n "${VERIFY_TRIAGE_OUTCOME:-}" ] && [ "$VERIFY_TRIAGE_OUTCOME" != "success" ] && [ "$VERIFY_TRIAGE_OUTCOME" != "skipped" ]; then + OUTCOME="$VERIFY_TRIAGE_OUTCOME" + fi - # Real output now exists elsewhere (a submitted review, a triage comment, an interactive reply -- every mode always leaves one on success, per each mode's own prompt), so this placeholder has nothing left to say. Deleting it outright, rather than editing it to a "finished" message, is what stops repeat runs on the same pull request/issue from leaving a trail of near-identical status pings behind. + # Real output now exists elsewhere (a submitted review, a triage comment, an interactive reply -- every mode always leaves one on success, per each mode's own prompt), so this placeholder has nothing left to say. Deleting it outright, rather than editing it to a "finished" message, is what stops repeat runs on the same pull request/issue from leaving a trail of near-identical status pings behind. For triage specifically this is no longer a bare assumption -- "Verify triage output landed" above already re-checked it, and OUTCOME above already reflects the real answer. if [ "$OUTCOME" = "success" ]; then for attempt in 1 2 3; do if gh api -X DELETE "repos/${REPOSITORY}/issues/comments/${COMMENT_ID}"; then From 833054dbfc4c0a6cb47a25cbdbd0a033ee80d9db Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 21:16:42 +0100 Subject: [PATCH 2/6] fix(triage): count real comments across pages and exclude Headroom's own comment --paginate -q filters the jq expression per page rather than across the combined result, so an issue with more prior comments than fit on one page left the comment count as a multi-line value. Separately, "Post Headroom savings comment" posts its own comment unconditionally whenever Headroom proxied anything, under the shipped default configuration, and that comment's id is always greater than the progress comment's -- counting it made the check report success even when triage itself left nothing. Slurps every page into one array before filtering, and excludes any comment carrying the Headroom marker. Both gh api calls in this step are now retried three times like every other one in this file that isn't already guarded by an outer continue-on-error. --- action.yml | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/action.yml b/action.yml index 02f0b0a..4e45fd7 100644 --- a/action.yml +++ b/action.yml @@ -2123,11 +2123,37 @@ runs: run: | set -euo pipefail - # A genuine new comment posted after this run's own progress comment is real output -- the progress comment's own id is this run's "before" marker, since it was the most recent comment on the issue at the moment this run started. - NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate -q "[.[] | select(.id > ${PROGRESS_COMMENT_ID})] | length") + MARKER_HEADROOM="" + + # A genuine new comment posted after this run's own progress comment is real output -- the progress comment's own id is this run's "before" marker, since it was the most recent comment on the issue at the moment this run started. Two things a naive `--paginate -q` gets wrong here: `-q` filters PER PAGE, not across the combined result (confirmed empirically -- a paginated endpoint with `-q '. | length'` returns one count per page, not a single total), so an issue with more prior comments than fit on one page would leave NEWER_COMMENTS as a multi-line value; and "Post Headroom savings comment" above posts its own comment unconditionally (if: always(), regardless of steps.claude.outcome) whenever Headroom actually proxied anything, which is the shipped default (headroom_enabled/headroom_show_savings both default "true") -- its comment id is always greater than PROGRESS_COMMENT_ID, so counting it here would report "verified" even when triage itself produced nothing. `--slurp` combines every page into one array first, fixing the first problem; it can't be passed to `-q`/`--jq` directly (confirmed: gh api rejects the combination), so the result is piped into a separate jq call instead, flattening the array-of-pages with `.[][]` and excluding the Headroom marker explicitly, fixing the second. Retried like every other gh api call in this composite action that isn't already guarded by an outer continue-on-error (see "Post progress comment", "Update progress comment") -- this step's whole purpose is avoiding a mis-reported outcome, so a single transient API hiccup must not itself become one. + NEWER_COMMENTS="" + for attempt in 1 2 3; do + if NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate --slurp | jq "[.[][] | select(.id > ${PROGRESS_COMMENT_ID} and ((.body // \"\") | contains(\"${MARKER_HEADROOM}\") | not))] | length"); then + break + fi + NEWER_COMMENTS="" + echo "Failed to list issue comments (attempt ${attempt}/3)." + [ "$attempt" -lt 3 ] && sleep 2 + done + if [ -z "$NEWER_COMMENTS" ]; then + echo "::error::Could not list issue comments after 3 attempts; cannot verify triage output." >&2 + exit 1 + fi # The marker's mere presence is enough: a re-triage run that replaces an existing block still counts as real output, and there is no cheap way to tell "replaced" from "left untouched" without a pre-run body snapshot this step does not have. - HAS_MARKER=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}" --jq '.body // "" | contains("")') + HAS_MARKER="" + for attempt in 1 2 3; do + if HAS_MARKER=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}" --jq '.body // "" | contains("")'); then + break + fi + HAS_MARKER="" + echo "Failed to fetch issue body (attempt ${attempt}/3)." + [ "$attempt" -lt 3 ] && sleep 2 + done + if [ -z "$HAS_MARKER" ]; then + echo "::error::Could not fetch issue body after 3 attempts; cannot verify triage output." >&2 + exit 1 + fi if [ "$NEWER_COMMENTS" -gt 0 ] || [ "$HAS_MARKER" = "true" ]; then echo "Verified: triage left real output (new comments: ${NEWER_COMMENTS}, marker present: ${HAS_MARKER})." From 3c95258b65fba56e74ecc04b8059629914c9da13 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 21:27:00 +0100 Subject: [PATCH 3/6] fix(triage): verify against a run-start timestamp, not the progress comment's id The verification step's if: required steps.progress-comment.outputs.comment_id, which is empty whenever post_progress_comment is false -- a supported, documented input combination this repeats through a different door. In that configuration verification was skipped entirely and steps.claude.outcome flowed straight through as this composite action's own overall conclusion, unverified. Adds a "Record run start time" step, unconditional and first in the job, and switches the comment check to created_at >= that timestamp instead of id > the progress comment's id. The progress comment (when it exists) and Headroom's own savings comment are still excluded explicitly, by id and by marker respectively, so neither is miscounted as real triage output. --- action.yml | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/action.yml b/action.yml index 4e45fd7..fdd9657 100644 --- a/action.yml +++ b/action.yml @@ -414,6 +414,12 @@ outputs: runs: using: "composite" steps: + # Unconditional and first in the job: "Verify triage output landed" further down needs a "before this run touched anything" timestamp that exists regardless of whether post_progress_comment is enabled -- the progress comment's own id was the original baseline, but that comment might never be posted at all under a supported, documented input combination. A plain UTC timestamp captured before any other step runs is a baseline every mode can use without depending on another feature being on. + - name: Record run start time + id: run-start + shell: bash + run: echo "started_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT" + - name: Validate inputs shell: bash env: @@ -2110,25 +2116,28 @@ runs: path: ~/.claude/projects key: ${{ steps.cache-scope.outputs.cache_key }} - # Closes a real, observed gap: "Resolve Claude Code result" above only confirms Claude Code itself ran (a real conclusion was produced), not that triage's own promised output -- a new issue comment, or a body edit carrying the `claude-triage:start` marker (prompts/triage/base.md's own "append a short triage summary to the issue's own description" section) -- actually landed on GitHub. Both are ordinary tool calls the model makes from its own prompt, not something this composite action enforces; a run that legitimately completes without ever calling either still reports steps.claude.outcome == 'success', which "Update progress comment" below already treats as "real output now exists elsewhere" and deletes the one visible trace (the progress comment itself) on that basis -- and, worse, that success also becomes this composite action's own overall conclusion, so even the Actions tab shows green. Confirmed happening for real, not just theoretically: three separate triage runs (novus-power/hive#1490, #1657, #1661) reported success and left zero comments and no marker, with the progress comment deleted and nothing to show for the run at all. This step re-checks GitHub's own state rather than trusting the model's self-report, the same principle "Structured review summary" above already applies to review mode by re-fetching what was actually submitted. Placed after every optional post-run step (Headroom reporting, turn-limit wrap-up, session save) rather than right after "Resolve Claude Code result", so a real failure here -- deliberately with no continue-on-error, for the same reason "Wrap up on turn limit" has none -- never starves an unrelated step of the implicit success() every unadorned `if:` in this file still carries; "Update progress comment" below is the one step built to run regardless (`if: always()`), which is exactly why this sits immediately before it. + # Closes a real, observed gap: "Resolve Claude Code result" above only confirms Claude Code itself ran (a real conclusion was produced), not that triage's own promised output -- a new issue comment, or a body edit carrying the `claude-triage:start` marker (prompts/triage/base.md's own "append a short triage summary to the issue's own description" section) -- actually landed on GitHub. Both are ordinary tool calls the model makes from its own prompt, not something this composite action enforces; a run that legitimately completes without ever calling either still reports steps.claude.outcome == 'success', which "Update progress comment" below already treats as "real output now exists elsewhere" and deletes the one visible trace (the progress comment itself) on that basis -- and, worse, that success also becomes this composite action's own overall conclusion, so even the Actions tab shows green. Confirmed happening for real, not just theoretically: three separate triage runs (novus-power/hive#1490, #1657, #1661) reported success and left zero comments and no marker, with the progress comment deleted and nothing to show for the run at all. This step re-checks GitHub's own state rather than trusting the model's self-report, the same principle "Structured review summary" above already applies to review mode by re-fetching what was actually submitted. Placed after every optional post-run step (Headroom reporting, turn-limit wrap-up, session save) rather than right after "Resolve Claude Code result", so a real failure here -- deliberately with no continue-on-error, for the same reason "Wrap up on turn limit" has none -- never starves an unrelated step of the implicit success() every unadorned `if:` in this file still carries; "Update progress comment" below is the one step built to run regardless (`if: always()`), which is exactly why this sits immediately before it. Not conditioned on `steps.progress-comment.outputs.comment_id != ''` (unlike an earlier revision of this fix) -- that would skip this whole check when `post_progress_comment` is `false`, a supported and documented input combination, reopening the exact gap this step exists to close through a different door. "Record run start time" at the very top of the job exists so this has a baseline regardless of whether a progress comment was ever posted. - name: Verify triage output landed id: verify-triage-output - if: ${{ inputs.mode == 'triage' && steps.claude.outcome == 'success' && steps.progress-comment.outputs.comment_id != '' }} + if: ${{ inputs.mode == 'triage' && steps.claude.outcome == 'success' }} shell: bash env: GH_TOKEN: ${{ inputs.github_token || github.token }} REPOSITORY: ${{ github.repository }} ISSUE_NUMBER: ${{ inputs.issue_number || github.event.issue.number }} + START_TIME: ${{ steps.run-start.outputs.started_at }} PROGRESS_COMMENT_ID: ${{ steps.progress-comment.outputs.comment_id }} run: | set -euo pipefail MARKER_HEADROOM="" + # 0 never matches a real comment id -- the fallback for when post_progress_comment is false and no progress comment (and therefore no id to exclude) exists at all. + PROGRESS_COMMENT_ID_ARG="${PROGRESS_COMMENT_ID:-0}" - # A genuine new comment posted after this run's own progress comment is real output -- the progress comment's own id is this run's "before" marker, since it was the most recent comment on the issue at the moment this run started. Two things a naive `--paginate -q` gets wrong here: `-q` filters PER PAGE, not across the combined result (confirmed empirically -- a paginated endpoint with `-q '. | length'` returns one count per page, not a single total), so an issue with more prior comments than fit on one page would leave NEWER_COMMENTS as a multi-line value; and "Post Headroom savings comment" above posts its own comment unconditionally (if: always(), regardless of steps.claude.outcome) whenever Headroom actually proxied anything, which is the shipped default (headroom_enabled/headroom_show_savings both default "true") -- its comment id is always greater than PROGRESS_COMMENT_ID, so counting it here would report "verified" even when triage itself produced nothing. `--slurp` combines every page into one array first, fixing the first problem; it can't be passed to `-q`/`--jq` directly (confirmed: gh api rejects the combination), so the result is piped into a separate jq call instead, flattening the array-of-pages with `.[][]` and excluding the Headroom marker explicitly, fixing the second. Retried like every other gh api call in this composite action that isn't already guarded by an outer continue-on-error (see "Post progress comment", "Update progress comment") -- this step's whole purpose is avoiding a mis-reported outcome, so a single transient API hiccup must not itself become one. + # A genuine new comment created at or after this run's own start time is real output. `created_at` compares correctly as a plain string here because GitHub always renders it in the same fixed-width ISO 8601 UTC form ("2026-09-10T20:17:02Z") this step's own START_TIME is captured in, so lexical and chronological order agree. Excludes two comments that would otherwise be miscounted as "new" even on a run that produced nothing real: the progress comment itself (posted, if post_progress_comment is enabled, after START_TIME but before Claude Code even runs) by id, and "Post Headroom savings comment"'s own comment (posted unconditionally -- if: always(), regardless of steps.claude.outcome -- whenever Headroom actually proxied anything, which is the shipped default) by its own marker. Two things a naive `--paginate -q` gets wrong here, independent of the filter itself: `-q` filters PER PAGE, not across the combined result (confirmed empirically -- a paginated endpoint with `-q '. | length'` returns one count per page, not a single total), so an issue with more prior comments than fit on one page would leave NEWER_COMMENTS as a multi-line value. `--slurp` combines every page into one array first; it can't be passed to `-q`/`--jq` directly (confirmed: gh api rejects the combination), so the result is piped into a separate jq call instead, flattening the array-of-pages with `.[][]`. Retried like every other gh api call in this composite action that isn't already guarded by an outer continue-on-error (see "Post progress comment", "Update progress comment") -- this step's whole purpose is avoiding a mis-reported outcome, so a single transient API hiccup must not itself become one. NEWER_COMMENTS="" for attempt in 1 2 3; do - if NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate --slurp | jq "[.[][] | select(.id > ${PROGRESS_COMMENT_ID} and ((.body // \"\") | contains(\"${MARKER_HEADROOM}\") | not))] | length"); then + if NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate --slurp | jq --arg start "$START_TIME" --argjson pcid "$PROGRESS_COMMENT_ID_ARG" --arg marker "$MARKER_HEADROOM" '[.[][] | select(.created_at >= $start and .id != $pcid and ((.body // "") | contains($marker) | not))] | length'); then break fi NEWER_COMMENTS="" From 1364283ac01b63b851a5f8d5c15821bd1bf5e28b Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 21:40:23 +0100 Subject: [PATCH 4/6] fix(triage): filter counted comments by this run's own actor identity Any comment landing in the run's time window -- a human reply, a different bot, another automation entirely -- satisfied the created_at/id/marker filters and counted as real triage output, even when Claude's own comment never landed. Filters by the identity GH_TOKEN actually authored as: read straight from the progress comment's own API response when one exists (post_progress_comment enabled, the default), falling back to the documented default GITHUB_TOKEN identity when it doesn't and no custom github_token is set. The one combination neither can resolve -- no progress comment and a custom github_token -- omits the author filter rather than guessing or failing a valid input combination outright. "Post progress comment" now captures the real poster login from its own API response alongside the comment id, rather than discarding it. --- action.yml | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/action.yml b/action.yml index fdd9657..bd62a44 100644 --- a/action.yml +++ b/action.yml @@ -743,20 +743,22 @@ runs: BODY=$(printf 'šŸ”„ Claude is %sā€¦ [View job run](%s)' "$VERB" "$RUN_URL") # Three attempts with a short backoff before falling back on continue-on-error above -- a transient rate limit or network blip shouldn't cost the comment even though a sustained failure (bad permissions, a real outage) still must not fail the job. - COMMENT_ID="" + COMMENT_JSON="" for attempt in 1 2 3; do - if COMMENT_ID=$(gh api "repos/${REPOSITORY}/issues/${ENTITY_NUMBER}/comments" -f body="$BODY" -q .id); then + if COMMENT_JSON=$(gh api "repos/${REPOSITORY}/issues/${ENTITY_NUMBER}/comments" -f body="$BODY"); then break fi - COMMENT_ID="" + COMMENT_JSON="" echo "Failed to post progress comment (attempt ${attempt}/3)." [ "$attempt" -lt 3 ] && sleep 2 done - if [ -z "$COMMENT_ID" ]; then + if [ -z "$COMMENT_JSON" ]; then echo "Giving up on posting a progress comment after 3 attempts." exit 1 fi - echo "comment_id=${COMMENT_ID}" >> "$GITHUB_OUTPUT" + echo "comment_id=$(jq -r '.id' <<<"$COMMENT_JSON")" >> "$GITHUB_OUTPUT" + # The actual identity GH_TOKEN just posted as, read straight from this real API response rather than assumed -- "Verify triage output landed" further down uses this (when it exists) as the authoritative answer to "which comments are this run's own output", since it's the one point in the job that observes the true poster identity firsthand. + echo "actor_login=$(jq -r '.user.login' <<<"$COMMENT_JSON")" >> "$GITHUB_OUTPUT" # Threaded through to "Update progress comment" below as the single source of truth for the marker text, rather than a second hardcoded literal that has to be kept in sync by hand. echo "marker_failed=${MARKER_FAILED}" >> "$GITHUB_OUTPUT" echo "Posted progress comment ${COMMENT_ID}." @@ -2127,6 +2129,8 @@ runs: ISSUE_NUMBER: ${{ inputs.issue_number || github.event.issue.number }} START_TIME: ${{ steps.run-start.outputs.started_at }} PROGRESS_COMMENT_ID: ${{ steps.progress-comment.outputs.comment_id }} + PROGRESS_ACTOR_LOGIN: ${{ steps.progress-comment.outputs.actor_login }} + CUSTOM_GITHUB_TOKEN: ${{ inputs.github_token }} run: | set -euo pipefail @@ -2134,10 +2138,20 @@ runs: # 0 never matches a real comment id -- the fallback for when post_progress_comment is false and no progress comment (and therefore no id to exclude) exists at all. PROGRESS_COMMENT_ID_ARG="${PROGRESS_COMMENT_ID:-0}" - # A genuine new comment created at or after this run's own start time is real output. `created_at` compares correctly as a plain string here because GitHub always renders it in the same fixed-width ISO 8601 UTC form ("2026-09-10T20:17:02Z") this step's own START_TIME is captured in, so lexical and chronological order agree. Excludes two comments that would otherwise be miscounted as "new" even on a run that produced nothing real: the progress comment itself (posted, if post_progress_comment is enabled, after START_TIME but before Claude Code even runs) by id, and "Post Headroom savings comment"'s own comment (posted unconditionally -- if: always(), regardless of steps.claude.outcome -- whenever Headroom actually proxied anything, which is the shipped default) by its own marker. Two things a naive `--paginate -q` gets wrong here, independent of the filter itself: `-q` filters PER PAGE, not across the combined result (confirmed empirically -- a paginated endpoint with `-q '. | length'` returns one count per page, not a single total), so an issue with more prior comments than fit on one page would leave NEWER_COMMENTS as a multi-line value. `--slurp` combines every page into one array first; it can't be passed to `-q`/`--jq` directly (confirmed: gh api rejects the combination), so the result is piped into a separate jq call instead, flattening the array-of-pages with `.[][]`. Retried like every other gh api call in this composite action that isn't already guarded by an outer continue-on-error (see "Post progress comment", "Update progress comment") -- this step's whole purpose is avoiding a mis-reported outcome, so a single transient API hiccup must not itself become one. + # Every comment this composite action's own steps post (the progress comment, the Headroom savings comment) and every comment Claude Code itself posts via its own gh tool calls authenticate as whatever identity GH_TOKEN resolves to -- so that identity, not a hardcoded name, is the one whose comments actually count as "this run's own output". Without this, ANY comment landing in the run's time window -- a human reply, a different bot, another automation entirely -- would satisfy the check and mask a run that produced no real triage output of its own. Resolved from real observed evidence, not guessed: "Post progress comment" already captured the true poster login straight from the API response that created it, which is authoritative whenever it exists (post_progress_comment enabled, the default). When it doesn't (post_progress_comment disabled) and no custom github_token was supplied, the ambient default GITHUB_TOKEN's authored identity is a documented GitHub platform constant, not something that needs discovering at runtime. The one combination this can't resolve -- post_progress_comment disabled AND a custom github_token supplied -- has no comment response to read and no documented default to fall back on; rather than guess (or hard-fail a valid, if rare, input combination), the author filter is simply omitted for that one case, falling back to the same start-time/id/marker filtering the previous revision of this fix already used. + if [ -n "$PROGRESS_ACTOR_LOGIN" ]; then + ACTOR_LOGIN="$PROGRESS_ACTOR_LOGIN" + elif [ -z "$CUSTOM_GITHUB_TOKEN" ]; then + ACTOR_LOGIN="github-actions[bot]" + else + ACTOR_LOGIN="" + echo "No progress comment to read an actor identity from, and a custom github_token is set -- verifying by timestamp/id/marker only, without an author filter." + fi + + # A genuine new comment created at or after this run's own start time, authored by this run's own identity (when known -- see above), is real output. `created_at` compares correctly as a plain string here because GitHub always renders it in the same fixed-width ISO 8601 UTC form ("2026-09-10T20:17:02Z") this step's own START_TIME is captured in, so lexical and chronological order agree. Excludes two comments that would otherwise be miscounted as "new" even on a run that produced nothing real: the progress comment itself (posted, if post_progress_comment is enabled, after START_TIME but before Claude Code even runs) by id, and "Post Headroom savings comment"'s own comment (posted unconditionally -- if: always(), regardless of steps.claude.outcome -- whenever Headroom actually proxied anything, which is the shipped default) by its own marker -- both belt-and-braces on top of the author filter, since either could in principle share this run's own identity. Two things a naive `--paginate -q` gets wrong here, independent of the filter itself: `-q` filters PER PAGE, not across the combined result (confirmed empirically -- a paginated endpoint with `-q '. | length'` returns one count per page, not a single total), so an issue with more prior comments than fit on one page would leave NEWER_COMMENTS as a multi-line value. `--slurp` combines every page into one array first; it can't be passed to `-q`/`--jq` directly (confirmed: gh api rejects the combination), so the result is piped into a separate jq call instead, flattening the array-of-pages with `.[][]`. Retried like every other gh api call in this composite action that isn't already guarded by an outer continue-on-error (see "Post progress comment", "Update progress comment") -- this step's whole purpose is avoiding a mis-reported outcome, so a single transient API hiccup must not itself become one. NEWER_COMMENTS="" for attempt in 1 2 3; do - if NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate --slurp | jq --arg start "$START_TIME" --argjson pcid "$PROGRESS_COMMENT_ID_ARG" --arg marker "$MARKER_HEADROOM" '[.[][] | select(.created_at >= $start and .id != $pcid and ((.body // "") | contains($marker) | not))] | length'); then + if NEWER_COMMENTS=$(gh api "repos/${REPOSITORY}/issues/${ISSUE_NUMBER}/comments" --paginate --slurp | jq --arg start "$START_TIME" --arg actor "$ACTOR_LOGIN" --argjson pcid "$PROGRESS_COMMENT_ID_ARG" --arg marker "$MARKER_HEADROOM" '[.[][] | select(.created_at >= $start and ($actor == "" or .user.login == $actor) and .id != $pcid and ((.body // "") | contains($marker) | not))] | length'); then break fi NEWER_COMMENTS="" From ef6183b564c2e9091960b8d112e094c419931198 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 21:52:03 +0100 Subject: [PATCH 5/6] fix(triage): use claude[bot] as the expected identity when github_token is empty "Run Claude Code" passes inputs.github_token straight through to upstream with no fallback, unlike GH_TOKEN elsewhere in this file (github_token || github.token). With the documented default -- github_token left empty, as every claude-triage.yml example in this repository does -- that asymmetry means Claude's own comment authenticates via an upstream-minted GitHub App token (claude[bot]), while the progress comment authenticates as the ambient github-actions[bot]: two different identities on the same run. Confirmed directly against real production comments on novus-power/hive, which uses exactly this configuration: the progress comment is authored by github-actions[bot], genuine triage comments by claude[bot]. The previous revision would have made every real triage run there report a false failure, having filtered out Claude's own comment by author. Checks for an empty github_token first and uses claude[bot] in that case, before falling back to the progress comment's own observed identity (valid only when both paths share the same real token) or no author filter at all. --- action.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/action.yml b/action.yml index bd62a44..1cc148d 100644 --- a/action.yml +++ b/action.yml @@ -2138,11 +2138,11 @@ runs: # 0 never matches a real comment id -- the fallback for when post_progress_comment is false and no progress comment (and therefore no id to exclude) exists at all. PROGRESS_COMMENT_ID_ARG="${PROGRESS_COMMENT_ID:-0}" - # Every comment this composite action's own steps post (the progress comment, the Headroom savings comment) and every comment Claude Code itself posts via its own gh tool calls authenticate as whatever identity GH_TOKEN resolves to -- so that identity, not a hardcoded name, is the one whose comments actually count as "this run's own output". Without this, ANY comment landing in the run's time window -- a human reply, a different bot, another automation entirely -- would satisfy the check and mask a run that produced no real triage output of its own. Resolved from real observed evidence, not guessed: "Post progress comment" already captured the true poster login straight from the API response that created it, which is authoritative whenever it exists (post_progress_comment enabled, the default). When it doesn't (post_progress_comment disabled) and no custom github_token was supplied, the ambient default GITHUB_TOKEN's authored identity is a documented GitHub platform constant, not something that needs discovering at runtime. The one combination this can't resolve -- post_progress_comment disabled AND a custom github_token supplied -- has no comment response to read and no documented default to fall back on; rather than guess (or hard-fail a valid, if rare, input combination), the author filter is simply omitted for that one case, falling back to the same start-time/id/marker filtering the previous revision of this fix already used. - if [ -n "$PROGRESS_ACTOR_LOGIN" ]; then + # Every comment Claude Code itself posts via its own gh tool calls authenticates as whichever identity the "Run Claude Code" steps' own github_token resolves to -- and that is NOT always the same identity "Post progress comment"'s GH_TOKEN resolves to, even though both read from inputs.github_token. "Run Claude Code" passes inputs.github_token straight through with no fallback; GH_TOKEN elsewhere in this file reads inputs.github_token || github.token. When inputs.github_token is genuinely empty (the documented default for the direct/composite-step form every example in this repository's own claude-triage.yml uses), that asymmetry matters: "Run Claude Code" hands upstream an empty token, and upstream mints its own separate GitHub App token instead (see README: "the underlying action will try to mint a GitHub App token") -- which authenticates as the Claude Code GitHub App's own well-known, documented identity, claude[bot] (named as such elsewhere in this same README) -- while GH_TOKEN elsewhere falls through to the ambient github.token, github-actions[bot]. Confirmed directly against real production comments on novus-power/hive, which uses exactly this configuration: the progress comment there is authored by github-actions[bot], Claude's own genuine triage comments by claude[bot] -- two different accounts on the same run. So when inputs.github_token is empty, claude[bot] is the correct identity regardless of what the progress comment observed, checked first, before falling back to a custom token's own real observed identity (when both this composite action's own steps and "Run Claude Code" share the same real token, and therefore the same real identity) or omitting the filter entirely when neither is resolvable. + if [ -z "$CUSTOM_GITHUB_TOKEN" ]; then + ACTOR_LOGIN="claude[bot]" + elif [ -n "$PROGRESS_ACTOR_LOGIN" ]; then ACTOR_LOGIN="$PROGRESS_ACTOR_LOGIN" - elif [ -z "$CUSTOM_GITHUB_TOKEN" ]; then - ACTOR_LOGIN="github-actions[bot]" else ACTOR_LOGIN="" echo "No progress comment to read an actor identity from, and a custom github_token is set -- verifying by timestamp/id/marker only, without an author filter." From 4ef350b1008c9e9aeaea1d3793d16be382a82609 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Thu, 10 Sep 2026 22:00:27 +0100 Subject: [PATCH 6/6] fix(examples): grant id-token: write in the direct-form triage example examples/direct/claude-triage.yml left github_token unset (the App-identity path) but never declared id-token: write, so the OIDC exchange that mints the Claude Code App token had nothing to authenticate with -- every triage run using this exact shipped example, unmodified, would fail outright at setupGitHubToken() with "Could not fetch an OIDC token", matching examples/direct/claude-review.yml's own already-correct configuration and the same permission novus-power/hive's real workflow had already added independently. Corrected the "Verify triage output landed" comment that cited this example as already granting id-token: write, which overstated what was actually shipped before this fix. --- action.yml | 2 +- examples/direct/claude-triage.yml | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/action.yml b/action.yml index 1cc148d..32e9b45 100644 --- a/action.yml +++ b/action.yml @@ -2138,7 +2138,7 @@ runs: # 0 never matches a real comment id -- the fallback for when post_progress_comment is false and no progress comment (and therefore no id to exclude) exists at all. PROGRESS_COMMENT_ID_ARG="${PROGRESS_COMMENT_ID:-0}" - # Every comment Claude Code itself posts via its own gh tool calls authenticates as whichever identity the "Run Claude Code" steps' own github_token resolves to -- and that is NOT always the same identity "Post progress comment"'s GH_TOKEN resolves to, even though both read from inputs.github_token. "Run Claude Code" passes inputs.github_token straight through with no fallback; GH_TOKEN elsewhere in this file reads inputs.github_token || github.token. When inputs.github_token is genuinely empty (the documented default for the direct/composite-step form every example in this repository's own claude-triage.yml uses), that asymmetry matters: "Run Claude Code" hands upstream an empty token, and upstream mints its own separate GitHub App token instead (see README: "the underlying action will try to mint a GitHub App token") -- which authenticates as the Claude Code GitHub App's own well-known, documented identity, claude[bot] (named as such elsewhere in this same README) -- while GH_TOKEN elsewhere falls through to the ambient github.token, github-actions[bot]. Confirmed directly against real production comments on novus-power/hive, which uses exactly this configuration: the progress comment there is authored by github-actions[bot], Claude's own genuine triage comments by claude[bot] -- two different accounts on the same run. So when inputs.github_token is empty, claude[bot] is the correct identity regardless of what the progress comment observed, checked first, before falling back to a custom token's own real observed identity (when both this composite action's own steps and "Run Claude Code" share the same real token, and therefore the same real identity) or omitting the filter entirely when neither is resolvable. + # Every comment Claude Code itself posts via its own gh tool calls authenticates as whichever identity the "Run Claude Code" steps' own github_token resolves to -- and that is NOT always the same identity "Post progress comment"'s GH_TOKEN resolves to, even though both read from inputs.github_token. "Run Claude Code" passes inputs.github_token straight through with no fallback; GH_TOKEN elsewhere in this file reads inputs.github_token || github.token. When inputs.github_token is genuinely empty AND id-token: write is granted in the calling job's own permissions -- exactly examples/direct/claude-triage.yml's own configuration, which was itself missing that permission until this same change added it -- that asymmetry matters: "Run Claude Code" hands upstream an empty token, upstream's own setupGitHubToken() successfully exchanges the OIDC token for a separate GitHub App token instead (see README: "the underlying action will try to mint a GitHub App token"), which authenticates as the Claude Code GitHub App's own well-known, documented identity, claude[bot] (named as such elsewhere in this same README) -- while GH_TOKEN elsewhere falls through to the ambient github.token, github-actions[bot]. Confirmed directly against real production comments on novus-power/hive, whose own claude-triage.yml independently added the same id-token: write permission this shipped example was missing: the progress comment there is authored by github-actions[bot], Claude's own genuine triage comments by claude[bot] -- two different accounts on the same run. Without id-token: write granted at all, the OIDC exchange itself fails and "Run Claude Code" errors outright (steps.claude.outcome becomes 'failure'), so this whole step never runs -- the claude[bot] branch below is only ever reached on a real, successful App-token mint, never a guess about what an errored run might have done. So when inputs.github_token is empty, claude[bot] is the correct identity regardless of what the progress comment observed, checked first, before falling back to a custom token's own real observed identity (when both this composite action's own steps and "Run Claude Code" share the same real token, and therefore the same real identity) or omitting the filter entirely when neither is resolvable. if [ -z "$CUSTOM_GITHUB_TOKEN" ]; then ACTOR_LOGIN="claude[bot]" elif [ -n "$PROGRESS_ACTOR_LOGIN" ]; then diff --git a/examples/direct/claude-triage.yml b/examples/direct/claude-triage.yml index 6c87b02..94a7415 100644 --- a/examples/direct/claude-triage.yml +++ b/examples/direct/claude-triage.yml @@ -15,6 +15,7 @@ on: permissions: contents: read issues: write + id-token: write # the OIDC exchange that mints the Claude Code App token needs this -- without it, an empty github_token (below, left unset deliberately) makes every run fail outright at setupGitHubToken() with "Could not fetch an OIDC token" jobs: triage: