coreai-opt provides implementations of popular model optimizations such as quantization, palettization (codebook-based compression), and pruning, for PyTorch models, customized for deployment on Apple Silicon via Core AI.
Jump to: Getting Started · Documentation · Contributing · Support · License · Related projects
Install the latest release from PyPI:
pip install coreai-optOr with uv:
uv pip install coreai-optThis project uses uv for environment management. Install uv by following the official installation guide.
To set up the environment from a checkout:
make envThis creates a project-specific virtual environment .venv and installs all dependencies. Activate it in a new terminal session with:
source .venv/bin/activateimport torch
from coreai_opt.quantization import Quantizer, QuantizerConfig
from torch import nn
# A simple model and example input.
model = nn.Sequential(nn.Linear(128, 64), nn.ReLU(), nn.Linear(64, 10)).eval()
example_inputs = (torch.randn(1, 128),)
# Apply INT8 weight-only quantization using a built-in preset.
config = QuantizerConfig.presets.w8()
quantizer = Quantizer(model, config)
prepared_model = quantizer.prepare(example_inputs)
# Finalize for Core AI export.
finalized_model = quantizer.finalize()For APIs, options, and detailed workflows, see the hosted documentation at apple.github.io/coreai-optimization.
Contributions are welcome within a defined scope. Please read CONTRIBUTING.md before opening a pull request or issue, particularly the section on contribution scope.
- GitHub Issues — Bug reports, feature requests, and questions
This project is licensed under the BSD-3-Clause license.
- Core AI — Apple's on-device AI inference stack
- Core AI Torch — converts PyTorch models to Core AI format; the upstream step before optimization
- Core AI Models — ready-to-run optimized models, Python reproduction scripts, and Swift utilities for on-device integration