Merge pull request #848 from dkackman/develop

Release 0.13.0

Author: dkackman

.env.example

@@ -70,7 +70,7 @@
 
 # --- Hugging Face Hub (all of diffusers and transformers download through it) --
 
-# Token for gated models such as FLUX.1-dev (or run `huggingface-cli login` once).
+# Token for gated models such as FLUX.1-dev (or run `hf auth login` once).
 # HF_TOKEN=hf_...
 
 # Where models are cached (default ~/.cache/huggingface). Point HF_HUB_CACHE

CLAUDE.md

@@ -21,7 +21,7 @@ python -m dw.run workflows/templates/text-to-image.json
 python -m dw.run workflows/templates/text-to-image.json prompt="a cat" num_images_per_prompt=4
 
 # A gated model (e.g. workflows/models/flux-dev.json) needs Hugging Face auth
-# first: huggingface-cli login
+# first: hf auth login
 
 # Validate a workflow against schema
 python -m dw.validate workflows/models/z-image.json

README.md

@@ -63,7 +63,7 @@ family that knows which workflow fits a request and the rules that bite:
 ```
 
 Most of the shipped workflows (Flux, LTX-2, MiniMax...) use **gated** models.
-Request access on the model's Hugging Face page, then `huggingface-cli login`
+Request access on the model's Hugging Face page, then `hf auth login`
 once; without it the run fails partway through with a 401/403 from the Hub.
 
 > **GPU on another machine?** Start the engine there with `--mcp` and connect

constraints-openapi.txt

@@ -4,5 +4,5 @@
 # scripts/dump_openapi.py refuses to write under anything else. To move
 # them: bump both lines, install them, run the dump and `npm run gen:api`,
 # and commit the four files together.
-fastapi==0.142.2
-pydantic==2.13.5
+fastapi==0.143.0
+pydantic==2.14.1

docs/ARCHITECTURE.md

@@ -150,10 +150,10 @@ one to open.
 | App factory | `dw/server/app.py`: `create_app` | `create_app` builds the JobManager and `app.state`, installs the middleware and registers the routes, and all state lives in the JobManager. | — |
 | Routers | `dw/server/routes/*.py`, `dw/server/routes/__init__.py`: `ROUTERS`, `include_routers`, `include_file_routes` | Routers register in `ROUTERS` order, because a greedy `{name:path}` route must come after its specific siblings. The files router comes later, after `/mcp`. | `tests/test_server_downloads.py::test_download_workflow_sets_content_disposition_attachment` |
 | Job queue | `dw/server/jobs.py`: `JobManager`; `dw/server/job_results.py`: `consume_results` | A dispatcher thread hands queued jobs, in queue order, to the pool's workers (`Worker pool dispatch`); each worker runs one job at a time, and `consume_results` folds that worker's replies into the job, naming files relative to the job's own `output_dir`. A job carries its own `output_dir`, `asset_dir` and `workflow_dir`, so it stays in its workspace. | `tests/test_server_workspaces.py::TestRunning::test_rerun_stays_in_the_workspace_it_ran_in` (FIFO: —) |
-| Worker pool dispatch | `dw/server/jobs.py`: `JobManager._next_dispatch`; `dw/server/pool.py`: `WorkerSlot`, `check_fits`, `largest_ceiling_gb`; `dw/vram_estimate.py`: `required_vram_gb`; `dw/server/admission.py`: `pool_capacity` | `--devices` runs one worker per card; a queued job goes to a free card that fits what it needs, a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, and cancel or a crash touches only the worker running that job. On a backend no `cost` entry describes, a declared `vram_estimate` is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. | `tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once`, `tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest`, `tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job`, `tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job`, `tests/test_worker_pool.py::TestOomRanking::test_the_later_started_worker_is_ranked_first`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted` |
+| Worker pool dispatch | `dw/server/jobs.py`: `JobManager._next_dispatch`; `dw/server/pool.py`: `WorkerSlot`, `check_fits`, `largest_ceiling_gb`; `dw/vram_estimate.py`: `required_vram_gb`; `dw/server/admission.py`: `pool_capacity` | `--devices` runs one worker per card; a queued job goes to a free card that fits what it needs (none starts while a cache clear holds dispatch, `dw disk caches`), a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, and cancel or a crash touches only the worker running that job. On a backend no `cost` entry describes, a declared `vram_estimate` is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. | `tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once`, `tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest`, `tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job`, `tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job`, `tests/test_worker_pool.py::TestOomRanking::test_the_later_started_worker_is_ranked_first`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted` |
 | Card affinity | `dw/server/pool.py`: `choose_slot`; `dw/server/jobs.py`: `JobManager.route`; `dw/worker_protocol.py`: `workflow_identity` | Among the free cards a job fits it goes to the rerun's original card, else the card whose worker last ran its workflow identity (`workflow_identity`, the one rule the worker caches by too), else one with no worker running, else the first in `--devices` order; a cache probe and `plan.estimate` (`priced_for`) ask about the card `route` names, and memory is read and cleared per card. | `tests/test_worker_pool.py::TestIdentityAffinity::test_a_second_run_lands_on_the_card_that_ran_the_workflow_first`, `tests/test_worker_pool.py::TestRerunAffinity::test_a_rerun_lands_on_the_original_card_when_it_is_free`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle` |
 | Per-card memory and cache | `dw/server/worker_memory.py`: `memory_status`, `clear_memory`, `probe_cache` | Each card keeps its own last memory reading; an idle card's worker is asked live, a busy one answers with the run's last report and says why it is not live, and a clear leaves a card running a job alone. Every request takes the card's lock with a bounded wait, and a card's lock is never taken while the JobManager's lock is held. | `tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_server.py::test_memory_says_why_a_reading_is_not_the_worker_s` |
-| dw disk caches | `dw/disk_caches.py` | Members `inductor`, `triton`, `quantized`. Clear only a root carrying the `.dw-cache` marker, never while a job is running or queued (`quantized` also not during a model download); an entry delete takes only a 64-hex id, checked before any path is built; clear renames into `.clearing-*` then deletes. | `tests/test_disk_caches.py` |
+| dw disk caches | `dw/disk_caches.py`; `dw/server/jobs.py`: `JobManager.hold_dispatch`; `dw/server/routes/system.py`: `clear_cache`, `delete_cache_entry` | Members `inductor`, `triton`, `quantized`. Clear only a root carrying the `.dw-cache` marker, never while a job is running or queued (`quantized` also not during a model download); an entry delete takes only a 64-hex id, checked before any path is built; clear renames into `.clearing-*` then deletes. The busy check and the dispatch hold are one step in `JobManager.hold_dispatch`, kept for the whole clear or entry delete: a job submitted meanwhile is accepted but stays queued until it ends. | `tests/test_disk_caches.py`, `tests/test_disk_caches.py::test_job_submitted_during_a_clear_waits_for_it` |
 | Job-to-run link | `dw/server/job_record.py`, `dw/server/jobs.py`: `JobManager.realized` | A job records `run_id`, `run_dir` and `run_version`, and `JobManager.realized` reads that run's `workflow.json`, confined to the output root. | `tests/test_server_jobs.py::test_realized_reads_the_file_the_run_wrote`, `tests/test_server_jobs.py::test_realized_refuses_a_run_dir_that_escapes_the_output_root` |
 | Job history | `dw/server/job_history.py`: `JobHistory` | Finished jobs persist in `jobs.sqlite`, in WAL mode, so the Jobs view survives a restart. | `tests/test_server.py::test_job_history_survives_restart_and_reruns`, `tests/test_job_history_wal.py::test_the_jobs_database_uses_wal_mode` |
 | HTTP security | `dw/server/http_security.py`: `install_middleware`, `query_token_ok` | Every API route needs the token, and only a route marked `query_token_ok` takes it as `?token=`. | `tests/test_security_auth.py::TestTheTokenGate::test_every_api_spelling_needs_the_token`, `tests/test_security_auth.py::TestTheTokenGate::test_the_query_token_is_refused_where_it_is_not_allowed` |

docs/DEPENDENCIES.md

@@ -76,10 +76,10 @@ bash ./install.sh
 
 ## Troubleshooting
 
-**Hugging Face authentication (401/403 downloading a model):** Most workflows under `workflows/` (Flux, LTX-2, MiniMax...) point at **gated** models on the Hub — repos that require the owner to approve your account before you can download them. A run against one of these fails with an actionable error naming the repo and `huggingface-cli login` (mapped from the Hub's 401/403 in `load_component()`, `dw/pipeline_processors/components.py`) — request access on the model's page (e.g. [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)) and then log in once, locally:
+**Hugging Face authentication (401/403 downloading a model):** Most workflows under `workflows/` (Flux, LTX-2, MiniMax...) point at **gated** models on the Hub — repos that require the owner to approve your account before you can download them. A run against one of these fails with an actionable error naming the repo and `hf auth login` (mapped from the Hub's 401/403 in `load_component()`, `dw/pipeline_processors/components.py`) — request access on the model's page (e.g. [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)) and then log in once, locally:
 
 ```bash
-huggingface-cli login
+hf auth login
 ```
 
 `workflows/templates/text-to-image.json` uses an ungated model and needs no login - start there if you just want to confirm the install works.