ci(tests): cut idle waits from the python suite and skip OS-independent tests on windows

[Pfannkuchensack]

Summary

The windows python-tests job takes 27–28 minutes and sets the CI wall clock. Most of that time is not test work. Fixtures wait out idle intervals, bcrypt hashes at cost 12, a gc pass walks the whole heap, and a fresh interpreter is started for each of 16 parametrized cases. Tests that cannot behave differently by OS also run three times.

• bcrypt cost 4 in tests. An autouse fixture swaps password_utils.pwd_context for a cost-4 copy. The cost-factor test and the slow auth benchmarks request production_password_hashing, which restores the production context. Cost-12 hashes stored elsewhere still verify.
• Service polls no longer delay teardown. The download and installer queue polls are now module constants (_QUEUE_POLL_SECONDS, _INSTALL_QUEUE_POLL_SECONDS). The root conftest shortens them and the image index's existing _POLL_SECONDS, so stop() in a fixture teardown returns at once instead of after up to 1 s per service. Production values are unchanged.
• Move backoff only where failures are scripted. _move_with_retries reads _MOVE_RETRY_DELAY_SECONDS. Only the installer tests that fake a failing move zero it, through _fake_move. A real Windows lock keeps its full recovery window.
• gc.freeze() in the RAM-budget cache tests, so their 33 gc.collect() calls scan only the test's own objects.
• Legacy-module symbol checks run in one subprocess. They now run inside the existing alias test instead of 16 cold interpreters.
• New marker platform_independent on 47 tests: import-graph boundaries in a fresh interpreter, static source scans, scheduler visit counts and VAE tile-shape sweeps. Windows installs the linux dependency set and deselects them. macOS runs a different torch and keeps them. Numerics, process-spawning behavior, paths and locking, and dependency smoke tests (umap/numba, PiD chunking) stay unmarked.
• Open-GOP trim test targets the I-frames. test_open_gop_source_trim_keeps_every_frame no longer trims from all 40 starts in the last two GOPs. It reads the source's key frames, trims only around each open-GOP I-frame, and asserts that read_h264_layout offers none of them as a cut point. That assertion is new and matters: with the IDR filter removed, the old loop still passed, because the node's join validation re-encoded instead.
• Lent-GPU scheduling test no longer flakes. test_worker_leaves_items_alone_while_its_gpu_is_lent_and_resumes_on_release also failed on main. A woken worker runs a full gc.collect() before taking its item, and late in an xdist session that can outlast the test's 2 s join. Reproduced locally by inflating the heap: a 2.1 s collection fails it. The test now polls every 60 s with a 10 s deadline, so a pass still requires the release listener.
• tests/AGENTS.md now covers test cost and when the marker applies.

Measured locally (CPU only, 8 xdist workers, OMP_NUM_THREADS=1), worker time on the files present in both runs:

	before	after
whole suite, same files	2418 s	1518 s (−37 %)
windows selection (not slow and not platform_independent)	2419 s	~1285 s (−47 %)
test_model_install.py	304 s	57 s
test_image_index_service.py	206 s	21 s
test_module_compatibility.py	230 s	69 s
test_multiuser_authorization.py	117 s	26 s
test_model_cache_ram_budget.py	80 s	6 s
test_open_gop_source_trim_keeps_every_frame (alone)	~60 s	16 s

Related Issues / Discussions

None.

QA Instructions

• uv run --no-sync pytest -n 8 on the full suite passes. The local HF_ENDPOINT was pointed at huggingface.co, and the LFS fixtures were pulled.
• The affected areas (model install, download, image index, auth, users, routers, model cache, module compatibility, VAE tiling scope) pass after the review fixes: 2146 passed.
• pytest --collect-only -m platform_independent collects 47 tests.
• A probe confirmed that hashes default to cost 04, that production_password_hashing yields 12, and that the cost-4 context verifies a cost-12 hash.
• Mutation checks:
  • Renaming one representative symbol fails test_every_legacy_module_imports_and_aliases_canonical.
  • Dropping the IDR filter in read_h264_layout fails the new open-GOP test; the old version passed it.
  • test_video_frame_extract_range.py: 27 passed.
• ruff check and ruff format --check are clean on the changed files.
• On the hosted runners, this PR's windows job ran 13641 passed in 14:02 min for the pytest step. Recent main runs took 27–28 min.
• Lent-GPU test: with a heap inflated to a 2.3 s gc.collect() it passes. With the release listener disabled it still fails, after the 10 s deadline.

Review

• An earlier draft also shortened the installer's move backoff for every test. That would have removed the retry window that real Windows file locks rely on, so it is now limited to tests that script the failure.
• An earlier draft also deselected the marker on macOS. macOS runs a different torch without bitsandbytes, so its import graph can differ; it now keeps those tests. By the local numbers, macOS stays below windows.
• No material findings remain.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Approving at d213bf5. I tried to break each mechanism and found no blockers.

Verified

• Pinned to 2 cores (taskset -c 0,1, -n 2, OMP_NUM_THREADS=1) to approximate the windows runner, the image_index, model_install, download, auth, routers and RAM-budget cache tests passed twice: 1931 passed, 2 skipped each run. test_module_compatibility.py and test_video_frame_extract_range.py: 32 passed. --collect-only -m platform_independent collects 47.
• Marker override: the command-line -m comes after addopts, so linux and macOS still select exactly not slow.
• Password context: nothing imports pwd_context by name; the only users are hash_password/verify_password. Nothing calls needs_update, so no hash is rehashed. Cost-12 hashes from module-scoped fixtures still verify, and the cost-factor guard still reads the real production context.
• Move backoff: _move_with_retries reads _MOVE_RETRY_DELAY_SECONDS when it is called, and no caller passed delay=. Every failing-move patch goes through _fake_move; the remaining one (move_then_scan) doesn't fail.
• platform_independent: torch/bitsandbytes markers match for win32 and linux x86_64, and none of invokeai's sys.platform/os.name branches run at import time.
• gc.freeze(): it freezes uncollected garbage from earlier tests, including caches whose deferred workers are still alive. test_dropped_cache_is_collectable_and_its_worker_exits compares thread identity, not count, so it is unaffected. Budgets are per-test objects.
• Open-GOP test: once sync_samples == (0,) holds, every start the old loop also covered takes the same full re-encode path, so narrowing the starts loses nothing.

Non-blocking

1. _fast_service_polling docstring: it says the fixture "only removes the idle wait". image_index_default._POLL_SECONDS is also the base of _backoff_seconds() (it starts at 0.02 s instead of 1 s and reaches the 60 s cap after about 13 failures). It is also the generation-idle poll in _wait_for_idle_generation, which now checks the session queue 50 times a second. The outage tests use >= assertions and pass; the docstring should say this.
2. Workflow -m fallback: matrix.pytest-markers || 'not slow' duplicates the addopts default. If addopts -m is ever extended, linux and macOS CI would silently drop the change. Passing -m only when the matrix sets it avoids that.
3. Folded symbol checks: inside the shared subprocess, the representative-symbol check is mostly implied by old is new; it still catches a missing symbol. What the old parametrized tests also gave was importing each legacy module first in a fresh interpreter. That gap is small but worth noting in the PR.
4. Possible follow-up: the four _fake_move tests whose move fails permanently still take about 2.2 s each. A profile puts that in _safe_rmtree's unconditional gc.collect() (about 0.37 s per call, 7 calls), not the backoff. That collect only matters on Windows, so restricting it to win32 would remove most of the cost. This is out of scope here.