You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9eb535e
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: docs/ACCELERATION.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -198,8 +198,9 @@ Compile a component once it is fully configured - the graph captures final dtype
198
198
199
199
Typical gains are 1.3-1.5x on diffusion transformers, and compilation stacks with the caches above. Notes:
200
200
201
-
-**First run pays the compile cost.** The [server](SERVER.md)'s persistent worker keeps compiled pipelines loaded between runs, so the cost is paid once per session rather than once per generation. A new session still retraces, but the kernels it built come back from the on-disk cache, which dw keeps across reboots (`TORCHINDUCTOR_CACHE_DIR`, see [Environment Defaults](#environment-defaults)).
201
+
-**First run pays the compile cost.** The [server](SERVER.md)'s persistent worker keeps compiled pipelines loaded between runs, so the cost is paid once per session rather than once per generation. A new session still retraces, but the kernels it built come back from the on-disk cache, which dw keeps across reboots (`TORCHINDUCTOR_CACHE_DIR`, see [Environment Defaults](#environment-defaults)). SDNQ's `triton.jit`/autotune kernels are cached separately by Triton itself (`TRITON_CACHE_DIR`, else `$TRITON_HOME/.triton/cache`, else `~/.triton/cache`); dw sets no `TRITON_*` variable and leaves that cache where it is, shared with any other Triton user on the machine. Both caches' sizes show in `get_server_info`'s `caches` and `GET /api/caches`, and `DELETE /api/caches/{name}` empties one ([SERVER.md](SERVER.md)); the next first step of each shape pays the compile or re-autotune again (100-300 s per shape on MiniMax-H3 for Triton).
202
202
-**Pin the attention backend** on a compiled component (`"attention_backend"` in the same `components` entry) rather than using the pipeline-level per-call context manager, which forces recompiles.
203
+
-**Quantize-on-load time is a separate cost.** An SDNQ on-the-fly component re-reads and re-quantizes its bf16 weights on every cold load; `DW_QUANTIZED_CACHE=1` saves the quantized result and reads it back instead - see [QUANTIZATION.md](QUANTIZATION.md#cached-on-the-fly-dw_quantized_cache).
203
204
-**Composes with offloading**: apply `group_offload` and `compile` on the same component and the offload hooks are installed first, as required. Skipped with a warning on MPS.
204
205
-**Don't combine `fullgraph` with a `cache`**: the cache hooks decide skip-or-compute per step, a data-dependent branch diffusers wraps in `torch.compiler.disable` - it needs the graph break that `fullgraph: true` forbids. Compile with the default (partial) graph mode when a cache is active.
205
206
-**TorchAO quantization needs compile to be fast** - see [QUANTIZATION.md](QUANTIZATION.md#torchao).
@@ -295,7 +296,7 @@ Set automatically at import unless already present in the environment (export yo
295
296
|`PYTORCH_CUDA_ALLOC_CONF`|`expandable_segments:True`| Lets the CUDA allocator grow segments instead of fragmenting fixed-size ones. Multi-step workflows churn differently-shaped allocations (generate, upscale, interpolate); fragmentation is what OOMs a card that nominally has room. |
|`PYTORCH_MPS_HIGH_WATERMARK_RATIO`|`0.0`| MPS only - use all available unified memory. |
298
-
|`TORCHINDUCTOR_CACHE_DIR`|`$XDG_CACHE_HOME/dw/torchinductor` (`~/.cache/dw/torchinductor`) | Keeps `torch.compile`'s on-disk cache across reboots - Inductor's own default is under `/tmp`. A new worker still retraces, but a cache hit skips codegen and kernel builds, the bulk of a cold compile. Entries are per shape, so a new size or frame count compiles once more. |
299
+
|`TORCHINDUCTOR_CACHE_DIR`|`$XDG_CACHE_HOME/dw/torchinductor` (`~/.cache/dw/torchinductor`) | Keeps `torch.compile`'s on-disk cache across reboots - Inductor's own default is under `/tmp`. A new worker still retraces, but a cache hit skips codegen and kernel builds, the bulk of a cold compile. Entries are per shape, so a new size or frame count compiles once more. Listed as `inductor` by `GET /api/caches` and cleared with `DELETE /api/caches/inductor`. Triton's cache is not set by dw and stays at its own default (`~/.triton/cache`) - see the compile notes above. |
299
300
300
301
For faster model downloads, optionally `pip install hf_transfer` and set `HF_HUB_ENABLE_HF_TRANSFER=1`. Not enabled automatically - it bypasses the Python HTTP stack and breaks some proxy setups.
Copy file name to clipboardExpand all lines: docs/ARCHITECTURE.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,7 @@ one to open.
28
28
| Device translation |`dw/pipeline_processors/placement.py`: `place_component`| A device is translated by `resolve_device()` before anything reads its backend, so the MPS accommodations also fire for a translated device. |`tests/test_device_portability.py::TestPlacement::test_a_translated_device_gets_its_backends_offload_downgrade`|
29
29
| On-demand wrappers |`dw/pipeline_processors/placement.py`: `apply_on_demand_placement`| The on-demand wrappers keep the wrapped signature, because H3's denoiser reads `signature(transformer.forward)`. |`tests/test_modular_pipeline.py::TestOnDemandResidency::test_the_wrapped_signature_survives`|
30
30
| On-demand and offload |`dw/pipeline_processors/placement.py`: `place_component`, `has_component_group_offload`| On-demand residency and `group_offload` cannot both be set on one component, and either one stops the whole pipeline from being moved to the device. |`tests/test_modular_pipeline.py::TestOnDemandResidency::test_group_offload_and_on_demand_together_are_rejected`, `tests/test_modular_pipeline.py::TestOffloadDevice::test_component_group_offload_is_not_moved_to_device`|
31
+
| Quantized component cache |`dw/quantized_cache.py`: `load_components`, `from_pretrained`, `cache_key`| A key that can't pin the weights isn't cached, the cache never fails a job, and the save comes before pruning, so an entry is the whole component. |`tests/test_quantized_cache.py`|
31
32
| LoRAs |`dw/pipeline_processors/adapters.py`: `active_loras`, `load_loras`| A `loras` entry whose `model_name` is null is switched off. |`tests/test_lora_disable.py::TestLoadLoras::test_an_all_null_entry_loads_nothing`|
32
33
| Step progress |`dw/pipeline_processors/progress.py`: `reported_progress_bars`| A pipeline with no step callback reports its denoise steps through its progress bars. |`tests/test_modular_progress.py::test_every_advance_of_the_bar_is_reported`|
33
34
| Community pipelines |`dw/community_pipelines/` (e.g. `pipeline_ltx2_refine.py`: `LTX2RefinePipeline`), `dw/introspection.py`: `community_pipelines`, `load_allowed_class`| A pipeline dw ships is named by its dotted path (`dw.community_pipelines.<module>.<Class>`), loads untrusted because it is under `dw`, and is listed by `list_pipelines` and resolved by `get_pipeline_signature` - only the modules in that directory, never a caller-chosen import. `LTX2RefinePipeline` VAE-encodes `video` at `width` x `height` and passes it unnormalized as `latents`, because `LTX2Pipeline.prepare_latents` normalizes 5-D latents itself. |`tests/test_ltx2_refine_pipeline.py::TestIntrospection::test_only_the_shipped_modules_resolve`, `tests/test_ltx2_refine_pipeline.py::TestEncode::test_the_latents_are_left_unnormalized_for_the_parent`|
@@ -125,7 +126,7 @@ one to open.
125
126
| Opening media |`dw/media.py`| This is the only module that calls `av.open`. It imports no torch and no step types. |`tests/test_media_layering.py::test_av_open_appears_only_in_media`, `tests/test_media_layering.py::test_media_imports_no_step_types` (no torch: —) |
126
127
| Signal processing |`dw/dsp.py`| It measures and transforms waveforms, decides nothing, and imports nothing from `dw`. |`tests/test_media_layering.py::test_dsp_imports_nothing_from_dw`|
127
128
| Assessment probes |`dw/tasks/assess.py`, `dw/assessment_rules.py`| Exactly three registered commands are assessment probes. Each measures a finished file, and every rule names a real probe field. |`tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_assessment_is_exactly_the_three_probes`, `tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_every_rule_names_a_registered_json_command`|
128
-
| Script check | `dw/tasks/script_check.py`: `check`; `dw/script_lines.py`: `parse_lines`, `parse_shots`, `shot_names_error` | `check_script` aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with `assessment_rules.finding()` and reads the guard's floor and window (`DEAD_AIR_FLOOR_DBFS`, `DEAD_AIR_WINDOW`) rather than copying them. It is not an assessment probe and adds nothing to `RULES`. Its literal `lines` is refused at validation through `task_problems` (`script_lines_errors`), which calls the same `parse_lines` the task runs, and a literal `shots` through the same `parse_shots` and `shot_names_error`; the parsers live in `dw/script_lines.py`, not the task, so that `task_problems` reaches them without importing `script_check` (the `import_cycles` ratchet in `scripts/arch_metrics.py`). Its shot map comes from `assess.resolve_shots`, the one the probes use, never a copy; it places each shot on the soundtrack with `assess.sample_span`, decides repeated shot names with `shots.duplicate_shot_names`, and asks `locations.is_http_url` whether the take is a URL. | `tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag`, `tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found`, `tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference` |
129
+
| Script check | `dw/tasks/script_check.py`: `check`; `dw/script_lines.py`: `parse_lines`, `parse_shots`, `shot_names_error` | `check_script` aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with `assessment_rules.finding()` and reads the guard's floor and window (`DEAD_AIR_FLOOR_DBFS`, `DEAD_AIR_WINDOW`) rather than copying them; the loudest-window measure is `dsp.loudest_window_dbfs` (`script_check.word_level` wraps it), which `transcribe_audio`'s `suspect: "silent"` mark reads with the same floor and window; the repetition-loop detector (`script_lines.looped_texts`, `REPEAT_RUN_MIN`, `REPEAT_MAX_PERIOD`) lives in `dw/script_lines.py` and backs both `check_script`'s `repetition` discard and `transcribe_audio`'s `suspect: "repetition"`, and `script_lines.suspect_runs`/`SUSPECT_LEFT_OUT` are what `plan_cuts` groups and leaves out suspect chunks by - so neither task imports the other. It is not an assessment probe and adds nothing to `RULES`. Its literal `lines` is refused at validation through `task_problems` (`script_lines_errors`), which calls the same `parse_lines` the task runs, and a literal `shots` through the same `parse_shots` and `shot_names_error`; the parsers live in `dw/script_lines.py`, not the task, so that `task_problems` reaches them without importing `script_check` (the `import_cycles` ratchet in `scripts/arch_metrics.py`). Its shot map comes from `assess.resolve_shots`, the one the probes use, never a copy; it places each shot on the soundtrack with `assess.sample_span`, decides repeated shot names with `shots.duplicate_shot_names`, and asks `locations.is_http_url` whether the take is a URL. | `tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag`, `tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found`, `tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference` |
129
130
| Audio sync against a source slice |`dw/tasks/measure_sync.py`: `measure_sync`; `dw/dsp.py`: `envelope_lag`|`measure_sync` cross-correlates the onset envelopes of a shot's audio and its source slice and answers the lag and a 0-1 confidence. A positive lag means the audio is LATE (its events fall after the reference's), negative early; `lag_seconds` is None for a silent or flat track. It builds its findings (`sync_unmeasurable`, `sync_low_confidence`, `audio_out_of_sync`) with `assessment_rules.finding()` over module-local rule dicts and adds nothing to `RULES`; it reads the silence floor from `beats.SILENT_DBFS`. `envelope_lag` imports nothing from `dw`. |`tests/test_measure_sync.py::TestLag::test_late_audio_is_a_positive_lag`, `tests/test_measure_sync.py::TestLag::test_early_audio_is_a_negative_lag`, `tests/test_measure_sync.py::TestUnmeasurable::test_a_silent_reference_has_no_lag`|
130
131
131
132
## Security and architecture guardrails
@@ -152,6 +153,7 @@ one to open.
152
153
| Worker pool dispatch | `dw/server/jobs.py`: `JobManager._next_dispatch`; `dw/server/pool.py`: `WorkerSlot`, `check_fits`, `largest_ceiling_gb`; `dw/vram_estimate.py`: `required_vram_gb`; `dw/server/admission.py`: `pool_capacity` | `--devices` runs one worker per card; a queued job goes to a free card that fits what it needs, a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, and cancel or a crash touches only the worker running that job. On a backend no `cost` entry describes, a declared `vram_estimate` is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. | `tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once`, `tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest`, `tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job`, `tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job`, `tests/test_worker_pool.py::TestOomRanking::test_the_later_started_worker_is_ranked_first`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted` |
153
154
| Card affinity |`dw/server/pool.py`: `choose_slot`; `dw/server/jobs.py`: `JobManager.route`; `dw/worker_protocol.py`: `workflow_identity`| Among the free cards a job fits it goes to the rerun's original card, else the card whose worker last ran its workflow identity (`workflow_identity`, the one rule the worker caches by too), else one with no worker running, else the first in `--devices` order; a cache probe and `plan.estimate` (`priced_for`) ask about the card `route` names, and memory is read and cleared per card. |`tests/test_worker_pool.py::TestIdentityAffinity::test_a_second_run_lands_on_the_card_that_ran_the_workflow_first`, `tests/test_worker_pool.py::TestRerunAffinity::test_a_rerun_lands_on_the_original_card_when_it_is_free`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle`|
154
155
| Per-card memory and cache |`dw/server/worker_memory.py`: `memory_status`, `clear_memory`, `probe_cache`| Each card keeps its own last memory reading; an idle card's worker is asked live, a busy one answers with the run's last report and says why it is not live, and a clear leaves a card running a job alone. Every request takes the card's lock with a bounded wait, and a card's lock is never taken while the JobManager's lock is held. |`tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_server.py::test_memory_says_why_a_reading_is_not_the_worker_s`|
156
+
| dw disk caches |`dw/disk_caches.py`| Members `inductor`, `triton`, `quantized`. Clear only a root carrying the `.dw-cache` marker, never while a job is running or queued (`quantized` also not during a model download); an entry delete takes only a 64-hex id, checked before any path is built; clear renames into `.clearing-*` then deletes. |`tests/test_disk_caches.py`|
155
157
| Job-to-run link |`dw/server/job_record.py`, `dw/server/jobs.py`: `JobManager.realized`| A job records `run_id`, `run_dir` and `run_version`, and `JobManager.realized` reads that run's `workflow.json`, confined to the output root. |`tests/test_server_jobs.py::test_realized_reads_the_file_the_run_wrote`, `tests/test_server_jobs.py::test_realized_refuses_a_run_dir_that_escapes_the_output_root`|
156
158
| Job history |`dw/server/job_history.py`: `JobHistory`| Finished jobs persist in `jobs.sqlite`, in WAL mode, so the Jobs view survives a restart. |`tests/test_server.py::test_job_history_survives_restart_and_reruns`, `tests/test_job_history_wal.py::test_the_jobs_database_uses_wal_mode`|
157
159
| HTTP security |`dw/server/http_security.py`: `install_middleware`, `query_token_ok`| Every API route needs the token, and only a route marked `query_token_ok` takes it as `?token=`. |`tests/test_security_auth.py::TestTheTokenGate::test_every_api_spelling_needs_the_token`, `tests/test_security_auth.py::TestTheTokenGate::test_the_query_token_is_refused_where_it_is_not_allowed`|
0 commit comments