Skip to content

Windows CUDA ops and 5090 low-VRAM load path - #23

Draft
JonEcho wants to merge 1 commit into
jd-opensource:mainfrom
JonEcho:windows-cuda-ops-lowvram
Draft

Windows CUDA ops and 5090 low-VRAM load path#23
JonEcho wants to merge 1 commit into
jd-opensource:mainfrom
JonEcho:windows-cuda-ops-lowvram

Conversation

@JonEcho

@JonEcho JonEcho commented Aug 30, 2026

Copy link
Copy Markdown

Scope

Windows load path for JoyAI-Video-Edit on an RTX 5090 (32 GB).

  • Build joyomni_ops with MSVC and CUDA 12.8 (deploy/build_joyomni_ops.ps1, deploy/install_vs2022_cpp.cmd, deploy/requirements-windows.txt).
  • Skip Windows VAE inductor compile unless JOYOMNI_VAE_COMPILE=1.
  • Cap the CUDA allocator with JOYOMNI_VRAM_CAP_GB.
  • Fall back when CUTLASS FP8 GEMM is missing or broken so 720p DiT still loads.
  • Use asyncio instead of uvloop on win32.
  • Raise holder idle timeout to 1800s so a long restyle clip does not drop the websocket.

Stack

This is PR 1 of 2.

Child: JonEcho#1 (file-restyle-loop-splice; base is this branch).

Verification

  • git diff --stat origin/main...HEAD shows 13 files, +241 / -65.
  • Local code_rules_gate --staged reported 25 introduced hits on this fork. Those hits match existing JoyAI style (imports inside functions, print(), large create_app). This contribution does not rewrite that style.
  • This session did not re-run GPU serve. A prior 720p restyle on this machine used JOYOMNI_VRAM_CAP_GB=26 and VAE on CPU.

Risks

  • FP8 fallback is slower than a working CUTLASS kernel.
  • Idle timeout 1800s applies on every platform, not only Windows.
  • build_joyomni_ops.ps1 still assumes VS 2022 Community vcvars64.bat and a repo-root .venv.

Follow-up

Merge stays with the maintainer. This pull request is draft.

Build joyomni_ops on MSVC, skip VAE inductor compile on Windows, cap the allocator, and fall back from broken FP8 GEMM to bf16 dequant so 720p DiT load fits a 32 GB card.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant