Skip to content

build(workflows): stream PR list when checking for duplicate PRs - #14850

Draft
Planeshifter wants to merge 1 commit into
developfrom
philipp/ci-fix-check-duplicate-prs-timeout-2026-08-31
Draft

build(workflows): stream PR list when checking for duplicate PRs#14850
Planeshifter wants to merge 1 commit into
developfrom
philipp/ci-fix-check-duplicate-prs-timeout-2026-08-31

Conversation

@Planeshifter

@Planeshifter Planeshifter commented Aug 31, 2026

Copy link
Copy Markdown
Member

Description

What is the purpose of this pull request?

This pull request:

  • streams the open pull request list through a single jq invocation in .github/workflows/scripts/check_duplicate_prs/run, instead of indexing into the accumulated JSON array once per pull request.

Failing run: https://github.com/stdlib-js/stdlib/actions/runs/33352880096 (nightly check_duplicate_prs, develop).

Symptom: the "Check for duplicate PRs" step aborts with

##[error]The action 'Check for duplicate PRs' has timed out after 15 minutes.

shortly after logging Processed 900 PRs.... Four of the last five nightly runs have failed this way (runs 503, 502, 500, 499; run 501 passed).

Root cause: main() pages every open pull request into one JSON array and then walks that array with a C-style for loop over its indices, reading each element back out with jq -c ".[$i]". Every one of those jq invocations re-parses the whole array, which is roughly 4 MB at the current ~950 open pull requests, so the loop is quadratic in the number of open pull requests. Job logs show a flat 47 seconds per 50 pull requests from index 650 onward, which puts a full pass just past the step's timeout-minutes: 15 budget. Nothing else in the step is close to the budget; the workflow started flapping as the open pull request count crossed the threshold, and now fails most nights.

Fix: drive the loop with while IFS= read -r pr and feed it from a single streaming jq -c '.[]' invocation via process substitution — the same idiom the file already uses for the labeled-PR loop directly above. The array is parsed once rather than once per pull request. Process substitution rather than a pipe keeps the loop body in the current shell, so the accumulated issue_prs_keys / issue_prs_values arrays and processed_count are unaffected.

timeout-minutes is deliberately left at 15 — raising it would defer the failure rather than remove it.

Related Issues

Does this pull request have any related issues?

This pull request has no related issues.

Questions

Any questions for reviewers of this pull request?

No.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

Validation

  • bash -n on the modified script.
  • Behavioural equivalence: the pre-fix and post-fix scripts were both run end-to-end against a mocked GitHub API and a 122-element pull request fixture covering normal bodies, null bodies, empty bodies, both the Closes issue-URL form and the fixes #n form, a null array element, and an element whose number is null. stdout and stderr are byte-identical, including the ordering of the label add/remove calls, and both exit 0.
  • Scaling: against a 950-element, 4 MB fixture the post-fix script completes the full pass in 24 seconds locally. The pre-fix script did not finish the same fixture inside a two-minute budget; an isolated micro-benchmark of its loop measured 5.8 seconds for the first 50 pull requests, consistent with the ~15 minutes observed in CI.

Reviewer notes (non-blocking)

  • Two jq invocations per pull request remain (.number and .body), each parsing a single small object. These are linear overall (~30 seconds at current scale) and were left alone to keep the diff minimal; folding them into the single streaming pass would buy further headroom if ever needed.
  • If jq itself fails, the loop yields zero records and the script goes on to strip the Potential Duplicate label from every currently-labeled pull request. The pre-fix code had the identical exposure — a failing per-index jq made every iteration continue — so this is not a regression, but neither version distinguishes "no duplicates found" from "enumeration failed".
  • shellcheck is not available in the environment used to prepare this change, so no shellcheck run was performed.

Process disclosure: the routine that produced this change normally validates each fix with three independent reviewer sub-agents. Sub-agent spawning was unavailable in this environment, so the correctness, regression-scope, and style/conventions reviews were instead carried out as three separate self-review passes against the same briefs. All three concluded "approve" with no blocking findings, but they were not independent agents, and reviewers should weight them accordingly.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

  • Yes
  • No

If you answered "yes" above, how did you use AI assistance?

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

This pull request was written primarily by Claude Code as part of an automated CI-failure triage routine. The root cause was identified from GitHub Actions job logs, and the fix was validated locally against a mocked GitHub API as described above. See the process disclosure in the "Other" section regarding how the change was reviewed.


@stdlib-js/reviewers

The `check_duplicate_prs` workflow has failed on four of the last five
nightly runs against `develop`. The "Check for duplicate PRs" step
aborts with `The action 'Check for duplicate PRs' has timed out after
15 minutes.` shortly after reporting `Processed 900 PRs...`.

The script accumulates every open pull request into a single JSON
array and then indexes into that array once per pull request via
`jq -c ".[$i]"`. Each of those invocations re-parses the entire array,
which is roughly 4 MB at the current ~950 open pull requests, making
the loop quadratic. Job logs show a steady 47 seconds per 50 pull
requests, which puts a full pass just past the step's 15 minute
budget.

Stream the array through a single `jq -c '.[]'` invocation and drive
the loop with `read` instead, so that the list is parsed once rather
than once per pull request. Process substitution keeps the loop body
in the current shell, so the accumulated issue-to-PR arrays are
unaffected. Against a mocked API and a 122 pull request fixture
covering null bodies, empty bodies, a null array element, and a null
pull request number, the script's stdout and stderr are byte-identical
to before the change.

Ref: https://github.com/stdlib-js/stdlib/actions/runs/33352880096
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants