Skip to content

test(markdown-format): SessionStart probe case failed once on Windows with only the jq notice #6011

Description

@kyle-sexton

Problem

The markdown-format suite's "SessionStart probe with the kill switch unset" case (plugins/markdown-format/hooks/markdown-format.test.sh:1979-1990) failed once on test-windows on a PR that did not touch markdown-format, then passed on the next head of the same branch. It expects the probe's notice to name markdownlint-cli2; the failing output named only jq.

Evidence

  • Failure: run 37094085446, job 111120317567, head 12532bb89 of feat(effort): phase effort advice, event-log content opt-in and typed OTEL columns, with the upstream record #5992, step "Test the markdown-format hook on Windows": PASS=166 FAIL=1 SKIPPED=4. Output: {"hookSpecificOutput":{... "additionalContext":"markdown-format: jq was not found on PATH. Hooks that need it will skip until it is installed. ..."}}, with no markdownlint-cli2 line. The line just before reads bash.exe: warning: could not find /tmp, please create!.
  • The next head, a65512646, passed test-windows. git diff --stat 12532bb89 a65512646 -- plugins/markdown-format is empty.
  • Flake rate: over the 44 most recent completed, non-cancelled test-windows runs (PR and push, through 2026-10-03T04:46Z), the markdown-format step failed in 1 and passed in 43. No other failure in that window.

Likely cause (not confirmed)

run_probe_launcher runs the probe under env -i PATH="$PROBE_BIN", which removes TMP/TEMP/TMPDIR; hence the /tmp warning. probe-prerequisite.sh reports each missing tool through hook::notice_once "$plugin-$name", keyed on session s1, which every probe case reuses. If notice-once state lands in a location shared across cases when the temp variables are gone, an earlier case can mark markdownlint-cli2 as already noticed, leaving only jq (which is never in PROBE_BIN on Windows because the test links only /usr/local/bin, /usr/bin and /bin). Where that state is written was not checked; #4612 (notice_once keys on agent) is in the same area.

Impact

Low: 1 in 44 runs, and test-windows is not a required check. It still turns a PR's checks red and sends an agent to debug a plugin the PR did not change.

Proposed fix options

  1. Give each probe case its own notice-once state (pass a fresh TMPDIR and whatever state root hook::notice_once reads into the env -i call) and a unique session_id.
  2. Assert on what the case is about (the markdownlint line is present when it is missing) and stop depending on which other tools happen to be absent from PROBE_BIN.

Recommendation: confirm where hook::notice_once writes under env -i on Git Bash, then apply option 1. Basis: the failing output and the /tmp warning above; the cause is a hypothesis until that check is done.

Acceptance criteria

  • The probe cases pass TMPDIR (or the notice state root) and a per-case session id inside env -i.
  • The /tmp warning no longer appears in the step log.
  • No markdown-format failure in the next 50 test-windows runs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.priority: lowNice-to-have, cosmetic, or speculative; opportunistic.status: needs-decisionAwaiting a human or maintainer judgment call.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions