From 8bce5b5b3d3579667e847762821015904750893f Mon Sep 17 00:00:00 2001 From: john-rocky Date: Wed, 9 Sep 2026 17:48:44 +0900 Subject: [PATCH 1/2] docs: add Apple Core AI cookbook and Agent Skill for MiniCPM5-2B Adds docs/deployment/coreai.md and skills/minicpm5-deploy-coreai/SKILL.md (community Core AI .aimodel conversion of MiniCPM5-2B, iPhone / iPad / Mac in Swift), routes "iPhone / iOS / Swift / Xcode" to the new skill in minicpm5-deploy, and lists the backend in the MiniCPM5-2B deployment table of README.md and README-cn.md. --- README-cn.md | 1 + README.md | 1 + docs/deployment/coreai.md | 113 +++++++++++++++++++++++++ skills/minicpm5-deploy-coreai/SKILL.md | 103 ++++++++++++++++++++++ skills/minicpm5-deploy/SKILL.md | 5 ++ 5 files changed, 223 insertions(+) create mode 100644 docs/deployment/coreai.md create mode 100644 skills/minicpm5-deploy-coreai/SKILL.md diff --git a/README-cn.md b/README-cn.md index 7d0273e..4a07448 100644 --- a/README-cn.md +++ b/README-cn.md @@ -298,6 +298,7 @@ MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**，主流推理引擎 | Ollama | GGUF，本地端侧运行 | [ollama.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/ollama.md) | [minicpm5-deploy-ollama](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-ollama/SKILL.md) | | LM Studio | GGUF，Mac 桌面应用与 OpenAI server | [lmstudio.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/lmstudio.md) | [minicpm5-deploy-lmstudio](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-lmstudio/SKILL.md) | | MLX | MLX / 4bit，Apple Silicon 本地推理 | [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md) | [minicpm5-deploy-mlx](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-mlx/SKILL.md) | +| Core AI | Apple `.aimodel`，Swift 应用内端侧推理：iPhone / iPad / Mac（社区转换） | [coreai.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/coreai.md) | [minicpm5-deploy-coreai](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-coreai/SKILL.md) | | ArcLight | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) | | vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) | diff --git a/README.md b/README.md index 2b6e0f2..eea8222 100644 --- a/README.md +++ b/README.md @@ -296,6 +296,7 @@ MiniCPM5-2B uses the **standard `LlamaForCausalLM` architecture**, so mainstream | Ollama | GGUF local on-device runtime | [ollama.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/ollama.md) | [minicpm5-deploy-ollama](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-ollama/SKILL.md) | | LM Studio | GGUF Mac desktop app and OpenAI server | [lmstudio.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/lmstudio.md) | [minicpm5-deploy-lmstudio](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-lmstudio/SKILL.md) | | MLX | MLX / 4bit local inference on Apple Silicon | [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md) | [minicpm5-deploy-mlx](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-mlx/SKILL.md) | +| Core AI | Apple `.aimodel`, on-device in Swift apps: iPhone / iPad / Mac (community conversion) | [coreai.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/coreai.md) | [minicpm5-deploy-coreai](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-coreai/SKILL.md) | | ArcLight | GGUF local on-device, CPU, Desktop & Server | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) | | vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) | diff --git a/docs/deployment/coreai.md b/docs/deployment/coreai.md new file mode 100644 index 0000000..d945555 --- /dev/null +++ b/docs/deployment/coreai.md @@ -0,0 +1,113 @@ +# Deploy MiniCPM5-2B with Apple Core AI (iPhone / iPad / Mac) + +[Core AI](https://developer.apple.com/documentation/coreai) is Apple's on-device ML runtime in iOS 27 / macOS 27. A model is exported once to an `.aimodel` bundle and then runs inside a Swift app on iPhone, iPad and Apple Silicon Mac, with no Python at runtime. The bundle below runs on the GPU through Core AI's pipelined engine. Use this path when the target is an **app** (Swift / Xcode). For Python on a Mac, use [MLX](./mlx.md). + +The bundle below is a **community conversion**, maintained in the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Its Hugging Face card carries the conversion recipe and the measurements quoted here. This page covers MiniCPM5-2B; the zoo's MiniCPM5-1B bundle is not listed here yet. + +## TL;DR + +```bash +# Requires macOS 27 + Xcode 27 (CoreAIKit 0.4.1 targets Xcode 27 beta 5). +git clone https://github.com/john-rocky/coreai-kit +cd coreai-kit/Examples/ChatDemo + +# Headless run on the Mac (downloads the bundle on first use, then loads from cache): +swift run -c release chat-cli --model minicpm5-2b --prompt "1+1=?" +# → 1 + 1 = 2. + +# On an iPhone: open ChatDemo.xcodeproj, pick your device as the run destination, +# set your signing team, Run, then choose "MiniCPM5 2B" in the model picker. +``` + +## Pre-converted bundle + +| Model | Hugging Face repo | Quantization | Size | Numerics vs HF fp32 | +| --- | --- | --- | --- | --- | +| MiniCPM5-2B | [mlboydaisuke/MiniCPM5-2B-CoreAI](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) | int8 weight-only, per-block-32 | 2.7 GB | 24/24 + 24/24 greedy tokens exact on iPhone, 16/16 on Mac | + +It is a dynamic-shape bundle: one file runs unchanged on macOS and iOS through Core AI's pipelined engine. SDPA, RoPE and RMSNorm stay in full precision; the chat template's end-of-turn token `<|im_end|>` (id 130073) is set as the bundle's `eos_token`, and the model stops there (checked on `1+1=?` with both runtimes below). + +## Measured (greedy, 128-token random prompt, Release build) + +| | iPhone 17 Pro decode | iPhone 17 Pro prefill | M4 Max decode | +| --- | --- | --- | --- | +| MiniCPM5-2B int8 | 22.4 tok/s | 27.3 tok/s | 127.6 tok/s | + +Per-block-32 int8 lands on the Mac GPU's quantized-matmul path; the per-channel int8 sibling of this bundle decoded 5× slower on the Mac (25.6 tok/s) and the same on the phone, which is why the bundle ships per-block-32 scales. + +## Swift API (CoreAIKit) + +[CoreAIKit](https://github.com/john-rocky/coreai-kit) is a community Swift package over Core AI: it downloads a catalog model from Hugging Face on first use, caches it in Application Support, and exposes a chat session. Add the package in Xcode (**File → Add Package Dependencies…**, exact version `0.4.1`, product **CoreAIKit**), or in `Package.swift`: + +```swift +.package(url: "https://github.com/john-rocky/coreai-kit", exact: "0.4.1") +// target dependency: +.product(name: "CoreAIKit", package: "coreai-kit") +``` + +```swift +import CoreAIKit + +let chat = try await ChatSession(catalog: "minicpm5-2b") +let reply = try await chat.respond(to: "1+1=?") +print(reply) // final answer only + +// Streaming: answer and reasoning arrive as separate events. +for try await event in await chat.streamResponse(to: "Explain GQA in one sentence.") { + switch event { + case .response(let delta): print(delta, terminator: "") + case .thinking(let delta): _ = delta // the trace, if you want it + default: break + } +} +``` + +- The session keeps the conversation history; call `respond(to:)` again for the next turn. +- `ChatSession.Configuration`: `temperature` (default 0.7; `nil` = greedy), `maxResponseTokens` (default 2048), `systemPrompt`. Version 0.4.1 exposes temperature only, not top-p. +- Thinking is on by default for MiniCPM5, as in the released chat template. The trace is surfaced as `.thinking` events and on `Message.thinking`; `respond(to:)` returns the answer without it. The trace alone can run several hundred tokens, so keep `maxResponseTokens` generous. +- `KitLanguageModel(model:)` plugs the same bundles into Apple's `FoundationModels` `LanguageModelSession`, with tool calling on ChatML-style models. See the kit README. + +## Apple's own runtime package (no CoreAIKit) + +The bundle also loads with Apple's Swift package from [apple/coreai-models](https://github.com/apple/coreai-models). Download the bundle folder, then: + +```bash +hf download mlboydaisuke/MiniCPM5-2B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-2B-CoreAI +# ./MiniCPM5-2B-CoreAI/int8/ holds metadata.json, the .aimodel, and tokenizer/ +``` + +```swift +import FoundationModels +import CoreAILanguageModels + +let model = try await CoreAILanguageModel(resourcesAt: bundleFolderURL) // the int8/ folder +let session = LanguageModelSession(model: model) +print(try await session.respond(to: "1+1=?")) +``` + +The same repo's CLI runs the folder on a Mac: `swift run -c release llm-runner --model ./MiniCPM5-2B-CoreAI/int8 --prompt "1+1=?"`. + +## Building the bundle from your own checkpoint (advanced) + +The export uses Apple's `coreai-torch` (`coreai.llm.export`) with two adjustments that the zoo's recipe applies for you: + +- MiniCPM5 is a plain `LlamaForCausalLM`; the exporter's Mistral graph builder is architecturally identical (GQA, no qkv bias, no qk-norm, explicit `head_dim`), so the recipe maps `llama → mistral`. +- int8 weight-only symmetric quantization via `--compression-config` (per-block-32 scales), and `eos_token` set to `<|im_end|>` in the bundle's tokenizer. + +```bash +git clone https://github.com/john-rocky/coreai-model-zoo && cd coreai-model-zoo +python3 conversion/zoo_convert.py show minicpm5-2b # prints the exact export command +python3 conversion/zoo_convert.py run minicpm5-2b # export_minicpm5.py --hf-id openbmb/MiniCPM5-2B --qconfig minicpm5_int8sym_b32.yaml +python3 cli/coreai_verify.py -n 16 # greedy token check against the fp32 HF reference +``` + +Point `--hf-id` at a local fine-tuned checkpoint to convert your own weights. Full notes: [`models/minicpm5-2b`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/minicpm5-2b/README.md) in the zoo. + +## Common pitfalls + +- **Toolchain**: macOS 27 / iOS 27 and Xcode 27 are required. CoreAIKit 0.4.1 is pinned to Xcode 27 beta 5; if `swift` picks another Xcode, export `DEVELOPER_DIR=/Applications/Xcode-27.0.0-Beta.5.app/Contents/Developer` first. +- **First launch on iPhone**: the 2.7 GB bundle specializes on the phone once (28.9 s on iPhone 17 Pro), then the cache persists. This needs about 3 GB of free storage and the `com.apple.developer.kernel.increased-memory-limit` entitlement (ChatDemo has it). A full phone fails with `No space left on device`. +- **Context on iPhone**: prompt + generated tokens must stay under 1024 on iOS (the shipped pipelined engine caps growing-KV capacity there). Trim or chunk the history on the phone; macOS has no cap. +- **Debug builds** are about 3× slower per token on host-side work. Measure in Release. +- **macOS App Sandbox**: enable **Outgoing Connections (Client)** for the first-use download. +- **Model runs past the end of turn**: not expected with this bundle, since `eos_token` is `<|im_end|>` and the stop was checked end to end. If you re-export yourself, keep that setting and check a short no-think prompt (`1+1=?` should end within a few tokens). diff --git a/skills/minicpm5-deploy-coreai/SKILL.md b/skills/minicpm5-deploy-coreai/SKILL.md new file mode 100644 index 0000000..ea8d902 --- /dev/null +++ b/skills/minicpm5-deploy-coreai/SKILL.md @@ -0,0 +1,103 @@ +--- +name: minicpm5-deploy-coreai +description: Run MiniCPM5-2B on-device inside a Swift app with Apple Core AI (iOS 27 / macOS 27). Use when the user targets an iPhone, iPad or Mac app, or says "iOS", "iPhone", "Swift", "Xcode", "Core AI", "CoreAIKit", ".aimodel", "on-device in my app", or wants a Foundation Models LanguageModelSession backed by MiniCPM5-2B. +--- + +# Deploy MiniCPM5-2B with Apple Core AI (iPhone / iPad / Mac) + +Apple's on-device runtime in iOS 27 / macOS 27. The model is an `.aimodel` bundle loaded from Swift; no Python at runtime. The bundle is a community conversion from the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Covers MiniCPM5-2B only. Human-readable reference: [`docs/deployment/coreai.md`](../../docs/deployment/coreai.md). + +## Required input + +| Var | Example | Default | +| --- | --- | --- | +| `MODEL` | `minicpm5-2b` (CoreAIKit catalog id) | `minicpm5-2b` | +| `TARGET` | `mac` (headless CLI) or `iphone` (Xcode app) | `mac` | +| `PROMPT` | `1+1=?` | `1+1=?` | + +Bundle: [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) (int8, 2.7 GB). Downloads on first use. + +## Steps + +### 0. Check the toolchain (once) + +macOS 27 and Xcode 27 are required; CoreAIKit 0.4.1 is pinned to Xcode 27 beta 5. iPhone runs need iOS 27 on the device. + +```bash +xcodebuild -version # expect Xcode 27 +# If another Xcode is selected: +export DEVELOPER_DIR=/Applications/Xcode-27.0.0-Beta.5.app/Contents/Developer +``` + +### 1. Get the runner + +```bash +git clone https://github.com/john-rocky/coreai-kit +cd coreai-kit/Examples/ChatDemo +``` + +### 2A. `TARGET=mac`: headless run + +```bash +swift run -c release chat-cli --model "${MODEL}" --prompt "${PROMPT}" +``` + +Progress (download, load) goes to stderr; stdout carries only the final answer, so assert on stdout. First run downloads the bundle into Application Support; later runs load from cache. + +### 2B. `TARGET=iphone`: run the app on the device + +```bash +open ChatDemo.xcodeproj +``` + +In Xcode: choose the iPhone as the run destination, set your signing team under **Signing & Capabilities**, Run, then pick "MiniCPM5 2B" in the model picker. The app already carries the `com.apple.developer.kernel.increased-memory-limit` entitlement the bundle needs. + +### 2C. Integrate into your own app + +Add the package (Xcode: **File → Add Package Dependencies…**, `https://github.com/john-rocky/coreai-kit`, exact `0.4.1`, product **CoreAIKit**), then: + +```swift +import CoreAIKit + +let chat = try await ChatSession(catalog: "minicpm5-2b") +let reply = try await chat.respond(to: "1+1=?") // answer only; thinking is separate +``` + +`chat.streamResponse(to:)` yields `.response` and `.thinking` deltas. `ChatSession.Configuration` has `temperature` (default 0.7; `nil` = greedy), `maxResponseTokens` (default 2048) and `systemPrompt`; 0.4.1 exposes no top-p. + +### 3. Validate + +stdout (or the app's reply) contains `2` for `1+1=?`, and generation stops on its own. Measured on an M4 Max with the bundle already cached: `1 + 1 = 2.` in 18 s wall time including engine load; Apple's `llm-runner` on the same bundle thinks, answers `2`, and stops at `<|im_end|>`. + +## Without CoreAIKit (Apple's own package) + +```bash +hf download mlboydaisuke/MiniCPM5-2B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-2B-CoreAI +``` + +```swift +import FoundationModels +import CoreAILanguageModels // Swift package: https://github.com/apple/coreai-models + +let model = try await CoreAILanguageModel(resourcesAt: folderURL) // the int8/ folder +let session = LanguageModelSession(model: model) +print(try await session.respond(to: "1+1=?")) +``` + +Mac CLI from the same repo: `swift run -c release llm-runner --model ./MiniCPM5-2B-CoreAI/int8 --prompt "1+1=?"`. + +## Common pitfalls + +- **Wrong Xcode selected**: `swift` builds against whichever Xcode `xcode-select` points at. Set `DEVELOPER_DIR` as in step 0. +- **First launch on iPhone**: one-time on-device specialization of the 2.7 GB bundle (28.9 s on iPhone 17 Pro), then cached. Needs about 3 GB free storage; a full phone fails with `No space left on device`. +- **Context on iPhone**: prompt + generated tokens must stay under 1024 on iOS. Trim or chunk history on the phone; macOS has no cap. +- **Thinking is on by default** (as in the released chat template) and can run several hundred tokens before the answer. Keep `maxResponseTokens` generous; read the trace from `.thinking` events if needed. +- **Debug builds** are about 3× slower per token. Measure in Release (`-c release`). +- **macOS App Sandbox**: enable **Outgoing Connections (Client)** for the first-use download. + +## When NOT to use + +- Python on a Mac, no app → `minicpm5-deploy-mlx` (fastest on Apple Silicon in Python) +- Desktop GUI, no code → `minicpm5-deploy-lmstudio` +- Linux / Windows / CPU-only → `minicpm5-deploy-llama-cpp` +- Server with an OpenAI-compatible endpoint → `minicpm5-deploy-vllm` or `minicpm5-deploy-sglang` diff --git a/skills/minicpm5-deploy/SKILL.md b/skills/minicpm5-deploy/SKILL.md index 301369e..69cc73e 100644 --- a/skills/minicpm5-deploy/SKILL.md +++ b/skills/minicpm5-deploy/SKILL.md @@ -24,6 +24,7 @@ Before picking a backend, you MUST know: | **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` | | GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` | | MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` | +| Core AI `.aimodel` (iPhone / iPad / Mac apps, Swift; community conversion, MiniCPM5-2B only) | [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) | `minicpm5-deploy-coreai` | If the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout). @@ -39,6 +40,7 @@ If the user has a local copy, accept any directory path that contains `config.js | "Ollama" / "ollama run" / "Modelfile" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** | | "LM Studio" / "desktop GUI" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** | | "MLX" / "Apple Silicon native" / "fastest on Mac" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** | +| "iPhone" / "iOS app" / "Swift" / "Xcode" / "Core AI" | iPhone / iPad / Apple Silicon Mac | Core AI `.aimodel` (MiniCPM5-2B) | **`minicpm5-deploy-coreai`** | If the user **has not specified** any of the above and asks "how do I run this?": @@ -46,6 +48,7 @@ If the user **has not specified** any of the above and asks "how do I run this?" - **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`. - **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`. - **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest). +- **iPhone / iPad, or a Swift / Xcode app on any Apple device**: pick `minicpm5-deploy-coreai` (MiniCPM5-2B). - **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M). ## 3. Invocation contract @@ -70,6 +73,8 @@ curl http://localhost:PORT/v1/chat/completions \ Expected: HTTP 200 with `choices[0].message.content` containing `"2"`. +`minicpm5-deploy-coreai` runs in-process with no HTTP server; its own step 3 (`chat-cli --prompt "1+1=?"`, stdout contains `2`) is the equivalent check. + ## 5. Known cross-backend pitfalls These are common to multiple backends — surface to the user up front: From d057b91a83dc0f821ea13dc5ce1f5aecc66128b8 Mon Sep 17 00:00:00 2001 From: john-rocky Date: Thu, 10 Sep 2026 00:37:46 +0900 Subject: [PATCH 2/2] docs: add MiniCPM5-1B to the Apple Core AI cookbook, Agent Skill and router MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cookbook and skill now cover both sizes. README.md / README-cn.md get a Core AI row in the MiniCPM5-1B deployment table (8 backends) and `coreai` in the router's route list. The 1B bundle is the per-block-32 rebuild (revision b8a6ac39…); the earlier per-channel bundle did not stop at the end of a turn. --- README-cn.md | 5 +++-- README.md | 5 +++-- docs/deployment/coreai.md | 28 +++++++++++++++++--------- skills/minicpm5-deploy-coreai/SKILL.md | 20 +++++++++--------- skills/minicpm5-deploy/SKILL.md | 6 +++--- 5 files changed, 38 insertions(+), 26 deletions(-) diff --git a/README-cn.md b/README-cn.md index 4a07448..973c334 100644 --- a/README-cn.md +++ b/README-cn.md @@ -562,7 +562,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可 | 顶层 Skill | 作用 | 路由到 | | --- | --- | --- | -| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | 推理路由 | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` | +| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | 推理路由 | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` · `coreai` | | **[`minicpm5-finetune`](./skills/minicpm5-finetune/SKILL.md)** | 微调路由 | `trl` · `llamafactory` · `ms-swift` · `unsloth` · `xtuner` | 在 Cursor / Claude Code 中可以这样调用：agent 会读取顶层 Skill，并根据目标后端、硬件和数据路径选择对应的子 Skill 与 cookbook，再执行命令并回报结果。 @@ -574,7 +574,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可 下方按推理后端和微调框架列出对应的 cookbook 与子 Skill。量化模型不单独作为后端列出，而是在可加载对应格式的推理后端中说明。 -**推理部署**（7 个后端） +**推理部署**（8 个后端） | 后端 | 适用模型 / 场景 | Cookbook | 对应 Agent Skill | | --- | --- | --- | --- | @@ -586,6 +586,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可 | [LM Studio](https://lmstudio.ai) | GGUF，Mac 桌面应用与 OpenAI server | [`docs/deployment/lmstudio.md`](./docs/deployment/lmstudio.md) | [`minicpm5-deploy-lmstudio`](./skills/minicpm5-deploy-lmstudio/SKILL.md) | | [MLX](https://github.com/ml-explore/mlx-lm) | MLX / 4bit，Apple Silicon 本地推理 | [`docs/deployment/mlx.md`](./docs/deployment/mlx.md) | [`minicpm5-deploy-mlx`](./skills/minicpm5-deploy-mlx/SKILL.md) | | [ArcLight](https://github.com/OpenBMB/ArcLight) | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [`docs/deployment/arclight.md`](./docs/deployment/arclight.md) | [`minicpm5-deploy-arclight`](./skills/minicpm5-deploy-arclight/SKILL.md) | +| Core AI | Apple `.aimodel`，Swift 应用内端侧推理：iPhone / iPad / Mac（社区转换） | [`docs/deployment/coreai.md`](./docs/deployment/coreai.md) | [`minicpm5-deploy-coreai`](./skills/minicpm5-deploy-coreai/SKILL.md) | **微调**（5 个框架） diff --git a/README.md b/README.md index eea8222..17aa744 100644 --- a/README.md +++ b/README.md @@ -560,7 +560,7 @@ The two top-level skills cover deployment and fine-tuning: | Top-level skill | What it does | Routes to | | --- | --- | --- | -| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | Inference router | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` | +| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | Inference router | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` · `coreai` | | **[`minicpm5-finetune`](./skills/minicpm5-finetune/SKILL.md)** | Fine tuning router | `trl` · `llamafactory` · `ms-swift` · `unsloth` · `xtuner` | In Cursor / Claude Code, you can call them like this: the agent reads the top-level skill, selects the matching sub-skill and cookbook based on the target backend, hardware, and data path, then runs the command and reports back. @@ -572,7 +572,7 @@ In Cursor / Claude Code, you can call them like this: the agent reads the top-le The tables below list the cookbook and sub-skill for each inference backend and fine-tuning framework. Quantized models are not listed as standalone backends; they are described under the inference backend that can load each format. -**Inference Deployment** (7 backends) +**Inference Deployment** (8 backends) | Backend | Model format / use case | Cookbook | Paired Agent Skill | | --- | --- | --- | --- | @@ -583,6 +583,7 @@ The tables below list the cookbook and sub-skill for each inference backend and | [LM Studio](https://lmstudio.ai) | GGUF Mac desktop app and OpenAI server | [`docs/deployment/lmstudio.md`](./docs/deployment/lmstudio.md) | [`minicpm5-deploy-lmstudio`](./skills/minicpm5-deploy-lmstudio/SKILL.md) | | [MLX](https://github.com/ml-explore/mlx-lm) | MLX / 4bit local inference on Apple Silicon | [`docs/deployment/mlx.md`](./docs/deployment/mlx.md) | [`minicpm5-deploy-mlx`](./skills/minicpm5-deploy-mlx/SKILL.md) | | [ArcLight](https://github.com/OpenBMB/ArcLight) | GGUF local on-device, CPU, Desktop & Server | [`docs/deployment/arclight.md`](./docs/deployment/arclight.md) | [`minicpm5-deploy-arclight`](./skills/minicpm5-deploy-arclight/SKILL.md) | +| Core AI | Apple `.aimodel`, on-device in Swift apps: iPhone / iPad / Mac (community conversion) | [`docs/deployment/coreai.md`](./docs/deployment/coreai.md) | [`minicpm5-deploy-coreai`](./skills/minicpm5-deploy-coreai/SKILL.md) | **Fine tuning** (5 frameworks) diff --git a/docs/deployment/coreai.md b/docs/deployment/coreai.md index d945555..d5c6728 100644 --- a/docs/deployment/coreai.md +++ b/docs/deployment/coreai.md @@ -1,8 +1,8 @@ -# Deploy MiniCPM5-2B with Apple Core AI (iPhone / iPad / Mac) +# Deploy MiniCPM5-2B / MiniCPM5-1B with Apple Core AI (iPhone / iPad / Mac) [Core AI](https://developer.apple.com/documentation/coreai) is Apple's on-device ML runtime in iOS 27 / macOS 27. A model is exported once to an `.aimodel` bundle and then runs inside a Swift app on iPhone, iPad and Apple Silicon Mac, with no Python at runtime. The bundle below runs on the GPU through Core AI's pipelined engine. Use this path when the target is an **app** (Swift / Xcode). For Python on a Mac, use [MLX](./mlx.md). -The bundle below is a **community conversion**, maintained in the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Its Hugging Face card carries the conversion recipe and the measurements quoted here. This page covers MiniCPM5-2B; the zoo's MiniCPM5-1B bundle is not listed here yet. +The bundles below are **community conversions**, maintained in the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Each Hugging Face card carries the conversion recipe and the measurements quoted here. This page covers MiniCPM5-2B and MiniCPM5-1B; the catalog ids are `minicpm5-2b` and `minicpm5-1b`. ## TL;DR @@ -14,9 +14,10 @@ cd coreai-kit/Examples/ChatDemo # Headless run on the Mac (downloads the bundle on first use, then loads from cache): swift run -c release chat-cli --model minicpm5-2b --prompt "1+1=?" # → 1 + 1 = 2. +swift run -c release chat-cli --model minicpm5-1b --prompt "1+1=?" # the 1B: same command, 1.1 GB # On an iPhone: open ChatDemo.xcodeproj, pick your device as the run destination, -# set your signing team, Run, then choose "MiniCPM5 2B" in the model picker. +# set your signing team, Run, then choose "MiniCPM5 2B" (or "MiniCPM5 1B") in the model picker. ``` ## Pre-converted bundle @@ -24,16 +25,20 @@ swift run -c release chat-cli --model minicpm5-2b --prompt "1+1=?" | Model | Hugging Face repo | Quantization | Size | Numerics vs HF fp32 | | --- | --- | --- | --- | --- | | MiniCPM5-2B | [mlboydaisuke/MiniCPM5-2B-CoreAI](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) | int8 weight-only, per-block-32 | 2.7 GB | 24/24 + 24/24 greedy tokens exact on iPhone, 16/16 on Mac | +| MiniCPM5-1B | [mlboydaisuke/MiniCPM5-1B-CoreAI](https://huggingface.co/mlboydaisuke/MiniCPM5-1B-CoreAI) (revision b8a6ac397ccd5fb815f97336f8a8b1800b110da1 or newer) | int8 weight-only, per-block-32 | 1.1 GB | 24/24 greedy tokens exact + the stop on iPhone, 16/16 on Mac | -It is a dynamic-shape bundle: one file runs unchanged on macOS and iOS through Core AI's pipelined engine. SDPA, RoPE and RMSNorm stay in full precision; the chat template's end-of-turn token `<|im_end|>` (id 130073) is set as the bundle's `eos_token`, and the model stops there (checked on `1+1=?` with both runtimes below). +Both are dynamic-shape bundles: one file runs unchanged on macOS and iOS through Core AI's pipelined engine. SDPA, RoPE and RMSNorm stay in full precision; the chat template's end-of-turn token `<|im_end|>` (id 130073) is set as the bundle's `eos_token`, and the model stops there (checked on `1+1=?` with both runtimes below, for both sizes). + +Use the 1B revision named above or newer: the earlier 1B revision (`5ad650f`, per-channel int8) did not stop at the end of a turn, and its card says why. ## Measured (greedy, 128-token random prompt, Release build) | | iPhone 17 Pro decode | iPhone 17 Pro prefill | M4 Max decode | | --- | --- | --- | --- | | MiniCPM5-2B int8 | 22.4 tok/s | 27.3 tok/s | 127.6 tok/s | +| MiniCPM5-1B int8 | 61.7 tok/s | 65.6 tok/s | 246.6 tok/s | -Per-block-32 int8 lands on the Mac GPU's quantized-matmul path; the per-channel int8 sibling of this bundle decoded 5× slower on the Mac (25.6 tok/s) and the same on the phone, which is why the bundle ships per-block-32 scales. +Per-block-32 int8 lands on the Mac GPU's quantized-matmul path; the per-channel int8 sibling of the 2B decoded 5× slower on the Mac (25.6 tok/s) and the same on the phone, which is why both bundles ship per-block-32 scales. ## Swift API (CoreAIKit) @@ -48,7 +53,7 @@ Per-block-32 int8 lands on the Mac GPU's quantized-matmul path; the per-channel ```swift import CoreAIKit -let chat = try await ChatSession(catalog: "minicpm5-2b") +let chat = try await ChatSession(catalog: "minicpm5-2b") // or "minicpm5-1b" let reply = try await chat.respond(to: "1+1=?") print(reply) // final answer only @@ -74,6 +79,7 @@ The bundle also loads with Apple's Swift package from [apple/coreai-models](http ```bash hf download mlboydaisuke/MiniCPM5-2B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-2B-CoreAI # ./MiniCPM5-2B-CoreAI/int8/ holds metadata.json, the .aimodel, and tokenizer/ +# 1B: hf download mlboydaisuke/MiniCPM5-1B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-1B-CoreAI ``` ```swift @@ -96,18 +102,20 @@ The export uses Apple's `coreai-torch` (`coreai.llm.export`) with two adjustment ```bash git clone https://github.com/john-rocky/coreai-model-zoo && cd coreai-model-zoo -python3 conversion/zoo_convert.py show minicpm5-2b # prints the exact export command +python3 conversion/zoo_convert.py show minicpm5-2b # prints the exact export command (minicpm5-1b for the 1B) python3 conversion/zoo_convert.py run minicpm5-2b # export_minicpm5.py --hf-id openbmb/MiniCPM5-2B --qconfig minicpm5_int8sym_b32.yaml python3 cli/coreai_verify.py -n 16 # greedy token check against the fp32 HF reference +python3 cli/coreai_verify.py --chat no-think --prompt "Reply with only the number: 1+1=?" -n 16 --must-stop-within 16 +# ^ the end-of-turn check: the fp32 reference answers `2` and stops; a bundle that runs past that stop fails ``` -Point `--hf-id` at a local fine-tuned checkpoint to convert your own weights. Full notes: [`models/minicpm5-2b`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/minicpm5-2b/README.md) in the zoo. +Point `--hf-id` at a local fine-tuned checkpoint to convert your own weights. Keep per-block-32 scales: a per-channel int8 export of the 1B produced a bundle whose LM head scored every vocab id above ~65024 at zero through the engine, `<|im_end|>` included, while a short greedy parity check still passed — the stop check above is what catches it. Full notes: [`models/minicpm5-2b`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/minicpm5-2b/README.md) and [`models/minicpm5-1b`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/minicpm5-1b/README.md) in the zoo. ## Common pitfalls - **Toolchain**: macOS 27 / iOS 27 and Xcode 27 are required. CoreAIKit 0.4.1 is pinned to Xcode 27 beta 5; if `swift` picks another Xcode, export `DEVELOPER_DIR=/Applications/Xcode-27.0.0-Beta.5.app/Contents/Developer` first. -- **First launch on iPhone**: the 2.7 GB bundle specializes on the phone once (28.9 s on iPhone 17 Pro), then the cache persists. This needs about 3 GB of free storage and the `com.apple.developer.kernel.increased-memory-limit` entitlement (ChatDemo has it). A full phone fails with `No space left on device`. +- **First launch on iPhone**: the bundle specializes on the phone once (2B: 28.9 s on iPhone 17 Pro; 1B: 7.3 s), then the cache persists. The 2B needs about 3 GB of free storage and the `com.apple.developer.kernel.increased-memory-limit` entitlement (ChatDemo has it). A full phone fails with `No space left on device`. - **Context on iPhone**: prompt + generated tokens must stay under 1024 on iOS (the shipped pipelined engine caps growing-KV capacity there). Trim or chunk the history on the phone; macOS has no cap. - **Debug builds** are about 3× slower per token on host-side work. Measure in Release. - **macOS App Sandbox**: enable **Outgoing Connections (Client)** for the first-use download. -- **Model runs past the end of turn**: not expected with this bundle, since `eos_token` is `<|im_end|>` and the stop was checked end to end. If you re-export yourself, keep that setting and check a short no-think prompt (`1+1=?` should end within a few tokens). +- **Model runs past the end of turn**: not expected with these bundles (the 1B from revision b8a6ac397ccd5fb815f97336f8a8b1800b110da1 on), since `eos_token` is `<|im_end|>` and the stop was checked end to end. If you re-export yourself, keep that setting, keep per-block-32 scales, and run the stop check above (`Reply with only the number: 1+1=?` should end at `2`). diff --git a/skills/minicpm5-deploy-coreai/SKILL.md b/skills/minicpm5-deploy-coreai/SKILL.md index ea8d902..38d6a71 100644 --- a/skills/minicpm5-deploy-coreai/SKILL.md +++ b/skills/minicpm5-deploy-coreai/SKILL.md @@ -1,21 +1,21 @@ --- name: minicpm5-deploy-coreai -description: Run MiniCPM5-2B on-device inside a Swift app with Apple Core AI (iOS 27 / macOS 27). Use when the user targets an iPhone, iPad or Mac app, or says "iOS", "iPhone", "Swift", "Xcode", "Core AI", "CoreAIKit", ".aimodel", "on-device in my app", or wants a Foundation Models LanguageModelSession backed by MiniCPM5-2B. +description: Run MiniCPM5-2B or MiniCPM5-1B on-device inside a Swift app with Apple Core AI (iOS 27 / macOS 27). Use when the user targets an iPhone, iPad or Mac app, or says "iOS", "iPhone", "Swift", "Xcode", "Core AI", "CoreAIKit", ".aimodel", "on-device in my app", or wants a Foundation Models LanguageModelSession backed by MiniCPM5. --- -# Deploy MiniCPM5-2B with Apple Core AI (iPhone / iPad / Mac) +# Deploy MiniCPM5-2B / MiniCPM5-1B with Apple Core AI (iPhone / iPad / Mac) -Apple's on-device runtime in iOS 27 / macOS 27. The model is an `.aimodel` bundle loaded from Swift; no Python at runtime. The bundle is a community conversion from the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Covers MiniCPM5-2B only. Human-readable reference: [`docs/deployment/coreai.md`](../../docs/deployment/coreai.md). +Apple's on-device runtime in iOS 27 / macOS 27. The model is an `.aimodel` bundle loaded from Swift; no Python at runtime. The bundles are community conversions from the [Core AI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) (not an OpenBMB or Apple release). Covers MiniCPM5-2B and MiniCPM5-1B. Human-readable reference: [`docs/deployment/coreai.md`](../../docs/deployment/coreai.md). ## Required input | Var | Example | Default | | --- | --- | --- | -| `MODEL` | `minicpm5-2b` (CoreAIKit catalog id) | `minicpm5-2b` | +| `MODEL` | `minicpm5-2b` or `minicpm5-1b` (CoreAIKit catalog id) | `minicpm5-2b` | | `TARGET` | `mac` (headless CLI) or `iphone` (Xcode app) | `mac` | | `PROMPT` | `1+1=?` | `1+1=?` | -Bundle: [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) (int8, 2.7 GB). Downloads on first use. +Bundles: [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) (int8, 2.7 GB) and [`mlboydaisuke/MiniCPM5-1B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-1B-CoreAI) (int8, 1.1 GB; revision b8a6ac397ccd5fb815f97336f8a8b1800b110da1 or newer). Download on first use. ## Steps @@ -50,7 +50,7 @@ Progress (download, load) goes to stderr; stdout carries only the final answer, open ChatDemo.xcodeproj ``` -In Xcode: choose the iPhone as the run destination, set your signing team under **Signing & Capabilities**, Run, then pick "MiniCPM5 2B" in the model picker. The app already carries the `com.apple.developer.kernel.increased-memory-limit` entitlement the bundle needs. +In Xcode: choose the iPhone as the run destination, set your signing team under **Signing & Capabilities**, Run, then pick "MiniCPM5 2B" or "MiniCPM5 1B" in the model picker. The app already carries the `com.apple.developer.kernel.increased-memory-limit` entitlement the bundle needs. ### 2C. Integrate into your own app @@ -59,7 +59,7 @@ Add the package (Xcode: **File → Add Package Dependencies…**, `https://githu ```swift import CoreAIKit -let chat = try await ChatSession(catalog: "minicpm5-2b") +let chat = try await ChatSession(catalog: "minicpm5-2b") // or "minicpm5-1b" let reply = try await chat.respond(to: "1+1=?") // answer only; thinking is separate ``` @@ -67,12 +67,13 @@ let reply = try await chat.respond(to: "1+1=?") // answer only; th ### 3. Validate -stdout (or the app's reply) contains `2` for `1+1=?`, and generation stops on its own. Measured on an M4 Max with the bundle already cached: `1 + 1 = 2.` in 18 s wall time including engine load; Apple's `llm-runner` on the same bundle thinks, answers `2`, and stops at `<|im_end|>`. +stdout (or the app's reply) contains `2` for `1+1=?`, and generation stops on its own. Measured on an M4 Max with the bundle already cached: the 2B answers `1 + 1 = 2.` in 18 s wall time including engine load; Apple's `llm-runner` on the same bundles thinks, answers `2`, and stops at `<|im_end|>` (2B after 190 tokens, 1B after 171). ## Without CoreAIKit (Apple's own package) ```bash hf download mlboydaisuke/MiniCPM5-2B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-2B-CoreAI +# 1B: hf download mlboydaisuke/MiniCPM5-1B-CoreAI --include "int8/*" --local-dir ./MiniCPM5-1B-CoreAI ``` ```swift @@ -89,7 +90,8 @@ Mac CLI from the same repo: `swift run -c release llm-runner --model ./MiniCPM5- ## Common pitfalls - **Wrong Xcode selected**: `swift` builds against whichever Xcode `xcode-select` points at. Set `DEVELOPER_DIR` as in step 0. -- **First launch on iPhone**: one-time on-device specialization of the 2.7 GB bundle (28.9 s on iPhone 17 Pro), then cached. Needs about 3 GB free storage; a full phone fails with `No space left on device`. +- **First launch on iPhone**: one-time on-device specialization (2B: 28.9 s on iPhone 17 Pro; 1B: 7.3 s), then cached. The 2B needs about 3 GB free storage; a full phone fails with `No space left on device`. +- **1B revision**: use `b8a6ac397ccd5fb815f97336f8a8b1800b110da1` or newer. The earlier 1B revision (`5ad650f`) did not stop at the end of a turn. - **Context on iPhone**: prompt + generated tokens must stay under 1024 on iOS. Trim or chunk history on the phone; macOS has no cap. - **Thinking is on by default** (as in the released chat template) and can run several hundred tokens before the answer. Keep `maxResponseTokens` generous; read the trace from `.thinking` events if needed. - **Debug builds** are about 3× slower per token. Measure in Release (`-c release`). diff --git a/skills/minicpm5-deploy/SKILL.md b/skills/minicpm5-deploy/SKILL.md index 69cc73e..f7f6306 100644 --- a/skills/minicpm5-deploy/SKILL.md +++ b/skills/minicpm5-deploy/SKILL.md @@ -24,7 +24,7 @@ Before picking a backend, you MUST know: | **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` | | GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` | | MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` | -| Core AI `.aimodel` (iPhone / iPad / Mac apps, Swift; community conversion, MiniCPM5-2B only) | [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) | `minicpm5-deploy-coreai` | +| Core AI `.aimodel` (iPhone / iPad / Mac apps, Swift; community conversion, MiniCPM5-2B and MiniCPM5-1B) | [`mlboydaisuke/MiniCPM5-2B-CoreAI`](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI) | `minicpm5-deploy-coreai` | If the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout). @@ -40,7 +40,7 @@ If the user has a local copy, accept any directory path that contains `config.js | "Ollama" / "ollama run" / "Modelfile" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** | | "LM Studio" / "desktop GUI" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** | | "MLX" / "Apple Silicon native" / "fastest on Mac" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** | -| "iPhone" / "iOS app" / "Swift" / "Xcode" / "Core AI" | iPhone / iPad / Apple Silicon Mac | Core AI `.aimodel` (MiniCPM5-2B) | **`minicpm5-deploy-coreai`** | +| "iPhone" / "iOS app" / "Swift" / "Xcode" / "Core AI" | iPhone / iPad / Apple Silicon Mac | Core AI `.aimodel` (MiniCPM5-2B / 1B) | **`minicpm5-deploy-coreai`** | If the user **has not specified** any of the above and asks "how do I run this?": @@ -48,7 +48,7 @@ If the user **has not specified** any of the above and asks "how do I run this?" - **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`. - **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`. - **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest). -- **iPhone / iPad, or a Swift / Xcode app on any Apple device**: pick `minicpm5-deploy-coreai` (MiniCPM5-2B). +- **iPhone / iPad, or a Swift / Xcode app on any Apple device**: pick `minicpm5-deploy-coreai` (MiniCPM5-2B or MiniCPM5-1B). - **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M). ## 3. Invocation contract