Add SarTest: headless, hardware-free test harness

[eiz-claude]

Tooling for automated SAR testing on a machine with no audio hardware and nobody at the keyboard, plus the CI change the test machine needs. Built to check #145 and the start-up hang reported against many-endpoint setups; it has since run the full scenario matrix against master and #145 on a disposable Windows 11 VM.

Branch includes the one-line commit from #146; rebasing after that merges drops it.

What's in it

SarTestClock.dll, a minimal ASIO driver that ticks from a timer thread (480 frames at 48 kHz by default). SarAsio never starts SarClient without an inner driver, so without a software clock there are no SAR endpoints at all. It also times each callback into the host, which includes SarAsio's whole tick, so a blocking call inside the tick shows up as a number.

SarTest.exe:

• install creates the SAR software device node, installs the driver package from a CI-built INF non-interactively, registers the clock, and writes default.json for an endpoint layout.
• host loads SarAsio.dll through DllGetClassObject and loops each playback endpoint back to its recording twin, channel for channel.
• wasapi streams a self-describing signal (channel id plus sequence number per sample) through every endpoint and verifies it on the capture side. Runs of undecodable frames are classified by what borders them: next to silence they are the engine fading a stream in or out, between valid frames they are corruption. Dropouts and sequence jumps are timed so gaps can be lined up across endpoints. Invalidated streams are reopened the way a real client would, and counted.
• run does both, repeating the create/stream/tear-down cycle.
• race starts and stops the host back to back while threads keep opening, starting, holding and closing streams on its endpoints, which is what applications with a SAR endpoint as their default device do while a DAW starts. Every WASAPI and ASIO call runs under a watchdog.

tools/test/Invoke-SarTest.ps1 runs the scenarios on a prepared machine (-Scenarios matrix,race,kill): 1, 4, 8 and 16 endpoint pairs with three start/stop iterations each, the race on the largest layout, and the ASIO host killed while streams are open followed by a recovery check. One JSON and log per scenario plus summary.json.

CI: both projects build in the usermode job and ship with tools/test in sar-asio-<platform>. The driver job also publishes sar-driver-<platform>-testsigned, a copy of the package signed with a throwaway certificate generated for that build, with the certificate as testsign.cer, so a test machine with test signing enabled can install the CI build with no manual step. The unsigned package is unchanged.

tools/test/README.md covers the design, commands, pass criteria and how to prepare a test machine.

What running it found

Setup: Windows 11 25H2 guest, test signing on, Driver Verifier standard flags (deadlock detection, IRQL checking, special pool, I/O verification) on SynchronousAudioRouter.sys, kernel dumps on, restored to a clean checkpoint before every run.

• It found a BSOD on master. The start-up race bugchecked the master driver: 0x3B in SarKsPinRtGetBufferCore, where master dereferences the endpoint before its NULL check and the compiler drops the check. Fix double IRP completion, filter factory error handling, and other bugs from patch review #145 fixes it and ran the same suite without a crash; the dump analysis is in the Fix double IRP completion, filter factory error handling, and other bugs from patch review #145 comment. It is a separate bug from the reported start-up freeze, which an immediate crash does not explain.
• SarAsio's format-change broadcast invalidates streams. Every SAR endpoint activation makes SarAsio send KSEVENT_PINCAPS_FORMATCHANGE. Its comment says brand-new endpoints are skipped, but the code doesn't skip them. So any stream open while the host starts is invalidated, repeatedly for seconds at 16 endpoint pairs, and Initialize sometimes fails with AUDCLNT_E_UNSUPPORTED_FORMAT because the mix format changed underneath it. The harness reopens streams the way a real client would and counts all of it.
• Stream setup stalls after repeated host restarts, on master too. After the race's 30 host restarts, killing the host and starting a new one leaves IAudioClient::Activate on SAR endpoints blocked for 4 s to 36 s, with many streams released at the same instant; stopping the host during the race also occasionally takes 6 to 7 s while a concurrent Initialize blocks for as long. A master build with only Fix double IRP completion, filter factory error handling, and other bugs from patch review #145's NULL-check fix does the same, so it predates Fix double IRP completion, filter factory error handling, and other bugs from patch review #145. A plain kill and restart doesn't trigger it. This is the closest match to the reported start-up freeze so far; the harness names the blocked call but not yet what it waits on inside the audio stack.
• Dropouts line up across endpoints. Gaps of 8 to 40 ms hit many endpoints at the same instant, which points at a shared stall. SarAsio's own tick stayed at or under about 20 ms, so the stall isn't only in its callback.
• Nothing else tripped. No hang, and no Driver Verifier report other than the crash above, on either build in any scenario, including the host killed mid-stream.

Final runs, each from a clean checkpoint:

Scenario	master	master + #145's NULL check only	#145
Matrix, 1 to 16 endpoint pairs, 3 start/stop cycles each	passed	not run	passed
Start-up race, 30 host restarts	bugcheck, 2 of 2 runs	passed	passed, 2 of 2 runs
Host killed with streams open	passed	passed	passed
Recovery after the kill, run on its own	passed	not run	passed
Recovery after the kill, following the race	not reached (crashed in the race)	fails: setup stalls up to 36 s	fails: setup stalls up to 22–38 s

Not covered yet

Exclusive-mode WASAPI, application routing (the registry filter), and driver installation from the x86 binary. Dropout and invalidation counts vary between runs on a VM, so they are reported for comparison with a baseline rather than gated on.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Ae1KbSGxTuh8F6i85MGxp