Skip to content

run_all_tests.sh should hold the per-tree build lock for the whole run; a concurrent make between sections re-points src/eigenscript (the #681 guard aborts the suite) #1986

Closed

Description

@InauguralPhysicist

The per-tree build lock (#1940, #1941/#1955/#1970) serializes concurrent make runs. A suite run is not a make, though: tests/run_all_tests.sh holds no lock between the makes it triggers. So a concurrent make in the same tree, such as the Stop hook's tools/stop_gate.sh or another session, can run BETWEEN sections, re-point the src/eigenscript hard link to a different variant, and the #681 fingerprint guard then aborts the suite: "src/eigenscript changed during the run (rebuilt mid-suite) — results are invalid".

Measured on 2026-10-26 (phase2 sync #9 train, ~/src/wt/es-sync9):

  • the extensions suite aborted at [83] and the zlib lane at [0f], both with that error;
  • the builder reproduced it: an independent Stop hook selected the release variant and re-linked the alias while the suite left the lock free between auxiliary makes;
  • with the train holding the lock across the whole suite run, the rerun recorded one binary identity across all 319 section seams, and both suites completed.

The guard did its job (it failed loud), but each abort cost a full suite.

Done when

  • tests/run_all_tests.sh acquires the tree's build lock at start, reusing tools/tree_lock.sh rather than a second implementation, and holds it until it exits. It exports EIGS_TREE_LOCK_HELD=1, so the suite's own inner makes do not self-deadlock.
  • A concurrent make in the same tree, started mid-suite, waits or skips: stop_gate.sh already try-locks and skips. It must not re-point the alias. A test plants a concurrent make mid-run and shows the suite completes with one binary identity at every seam.
  • tools/make_lock_check.sh gains that case, and its self-test pin moves with it.

Activity

  1. added a commit that references this issue on Oct 26, 2026
  2. added a commit that references this issue on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:gatesSubsystem: gateskind:flakeIntermittent under CI/timing; reproduce before fixing

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions