The current milestone is one complete inference path: SDXL 1.0 base at 512×512 with Euler Ancestral sampling.
Pipelines::SDXL::TextToImage.from_single_file validates the safetensors header and canonical SDXL shapes before allocating a model. It then constructs and strictly loads four components in sequence:
- SDXL UNet: 1,680 tensors
- CLIP-L text encoder: 196 tensors
- OpenCLIP bigG text encoder: 389 tensors
- VAE decoder and post-quant convolution: 140 tensors
UNet and text-encoder layers are converted incrementally to the selected inference dtype during construction. The VAE stays float32 because SDXL decoding requires the additional precision. Every component is put in evaluation mode and gradients are disabled. On accelerators, components are staged onto the resolved device only while needed: text encoders for conditioning, the UNet for denoising, and the VAE for decode.
The prompt encoder tokenizes to 77 tokens independently for CLIP-L and bigG. It concatenates their penultimate hidden states and uses bigG's projected EOS-pooled output. An omitted negative prompt produces zero embeddings. Classifier-free guidance is enabled only when guidance is greater than 1.0, using sequential unconditional and conditional UNet calls to reduce peak memory.
Euler A uses the SDXL scaled-linear beta schedule, leading timestep spacing with offset 1, and epsilon prediction. Scheduler arithmetic is float32. One persistent CPU Torch::Generator produces both the initial latent and ancestral noise; generated noise is transferred to MPS or CUDA afterward.
Native Torch::NN.scaled_dot_product_attention is used in UNet, CLIP, OpenCLIP, and VAE attention. At 512×512, this and sequential guidance are the key constraints that make the pipeline practical on a 16 GB M1 Pro. The implementation exposes 1024×1024 dimensions but does not currently claim that size as a memory-safe acceptance target.
Model-ID and Hub checkpoint loading, the SDXL refiner, LoRA, ControlNet, image-to-image, and additional schedulers remain outside this milestone.