Skip to content

Normalize GPU VRAM against registered capacities - #279

Open
peterschmidt85 wants to merge 2 commits into
mainfrom
pr_normalize_gpu_memory
Open

peterschmidt85 wants to merge 2 commits into
mainfrom
pr_normalize_gpu_memory

Conversation

@peterschmidt85

@peterschmidt85 peterschmidt85 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Offers can report VRAM that differs from the GPU SKU's known capacity. This makes expected-VRAM filters unreliable and splits equivalent offers into separate groups when grouping by GPU.

  • Normalize full-GPU, per-device VRAM against KNOWN_ACCELERATORS, accepting a correction only when exactly one capacity matches the existing ±7% window.
  • Keep reported VRAM for unknown models or unmatched/ambiguous capacities, with warnings identifying the offer and expected capacities. Partial allocations keep their assigned VRAM.
  • Correct DigitalOcean's combined-VRAM handling and the MI325X registry capacity, and add common RTX models found in real offers.

Examples, in GiB per GPU:

GPU Backend Before After
B300 AWS 268.59 270
B300 OCI 262.5 270
B300 RunPod 288 270
B300 Verda 268 270
L40S AWS 44.70 48
MI325X, 8 GPUs DigitalOcean 2,048 256

Compatibility:

  • SSH GPU discovery uses the existing helper; shim formats and configuration stay unchanged.
  • Kubernetes continues using the registry for GPU memory; MI325X detection now reports 256GB instead of 288GB. No dstack code or configuration changes are required.
  • Slurm syntax and P100/A100 mappings are unchanged. In gpu_partitions, gpu: MI325X:288GB will fail to resolve; change it to gpu: MI325X:256GB. Bare gpu: MI325X needs no edit and now resolves to 256GB.

@peterschmidt85
peterschmidt85 requested a review from jvstme October 26, 2026 11:18
@peterschmidt85
peterschmidt85 marked this pull request as ready for review October 26, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant