Skip to content

ci: govulncheck retry self-test races on the tee process substitution #1047

Closed

Description

@cristim

Problem

scripts/test-govulncheck-retry.sh (run in CI's "Build Docker Image" job, step "Run govulncheck retry self-tests") fails intermittently, which fails CI Success on unrelated PRs. Seen twice on 2026-10-16/17:

Main's Docker build passes on re-run, so this is a race, not a regression.

Likely cause

scripts/govulncheck-retry.sh copies stderr into $errfile through a process substitution, 2> >(tee "$errfile" >&2), then runs a bare wait before grepping the file. A bare wait is not guaranteed to wait for that process substitution (bash only tracks the last one, and only through $!). When tee has not flushed, grep 'fetching vulnerabilities' "$errfile" sees an empty file, the failure is classed as non-transient and the retry loop stops early. The fewer calls (2 instead of 3) match that.

The same race can affect the real CI run: a genuine transient DB-fetch failure may not be retried.

Fix direction

Write stderr straight to the file and replay it (govulncheck "$@" 2>"$errfile" || rc=$?; cat "$errfile" >&2), or wait on the process substitution's own PID. Prove it by running the self-test in a loop (for example 200 iterations) before and after.

Activity

  1. added a commit that references this issue on Oct 17, 2026
    74baca8
  2. added a commit that references this issue on Oct 17, 2026
    db6d4a6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions