Skip to content

fix(github): ignore 404 when collecting unfinished pull requests (#9140) - #9145

Merged
klesh merged 3 commits into
apache:mainfrom
Thundercloud12:fix/github-pr-collector-404
Sep 18, 2026
Merged

klesh merged 3 commits into
apache:mainfrom
Thundercloud12:fix/github-pr-collector-404

Conversation

@Thundercloud12

Copy link
Copy Markdown
Contributor

Summary

When collecting unfinished pull request details (CollectUnfinishedDetails in pr_collector.go), DevLake iterates over previously collected open PRs (state != 'closed' OR merged = false) and queries:

GET /repos/{owner}/{repo}/pulls/{number}

If a PR has since been deleted, scrubbed (e.g., credential leak removal), or made inaccessible on GitHub, GitHub returns HTTP 404 Not Found.

Without an AfterResponse hook, ApiAsyncClient retries the endpoint 3 times and then terminates the entire subtask with:

Retry exceeded 3 times...

Consequently, all subsequent syncs fail on the same unavailable PR, causing data collection for newer PRs to halt and repository dashboards to become stale.

This PR adds AfterResponse: ignoreHTTPStatus404 to CollectUnfinishedDetails in pr_collector.go. The hook returns api.ErrIgnoreAndContinue when GitHub responds with HTTP 404, allowing the collector to gracefully skip unavailable PRs and continue collecting the remaining pull requests.

Does this close any open issues?

Closes #9140

Screenshots

N/A

Other Information

  • Follows the existing convention used across DevLake collectors (e.g., cicd_job_collector.go).
  • Added unit tests in plugins/github/tasks/shared_test.go covering ignoreHTTPStatus404 and ignoreHTTPStatus422.

@Thundercloud12

Copy link
Copy Markdown
Contributor Author

@klesh hey so this is the one issue that you have assigned to me, applied the approach we discussed earlier in that thread, hope this helps!!

@Thundercloud12

Copy link
Copy Markdown
Contributor Author

hey @klesh sorry for asking it here but the link to the slack discussion is expired i think can you guide me towards a working one

@klesh
klesh merged commit e744aef into apache:main Sep 18, 2026
10 checks passed
@klesh

klesh commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Good work. Thanks for your contribution.

@klesh

klesh commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

hey @klesh sorry for asking it here but the link to the slack discussion is expired i think can you guide me towards a working one

Try the following one please:

https://join.slack.com/t/devlake-io/shared_invite/zt-47ktd6ws4-BdpD0M4W4eaJl8tGwH7Clw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][Github] GitHub PR collection stops on 404 for stale cached pull request

2 participants