Skip to content

docs: LiteRT-LM cookbook and Agent Skill for MiniCPM5-2B / MiniCPM5-1B (Android / iOS / desktop) #375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hansjohn merged 7 commits into from
Sep 10, 2026
+250 −4
Merged

docs: LiteRT-LM cookbook and Agent Skill for MiniCPM5-2B / MiniCPM5-1B (Android / iOS / desktop) #375

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c9a84b2
docs: LiteRT-LM cookbook and Agent Skill for MiniCPM5-2B / MiniCPM5-1…
john-rocky Sep 10, 2026
f92d4fa
docs: drop the community-conversion wording from the LiteRT-LM cookbo…
john-rocky Sep 10, 2026
2be21b4
docs: link LiteRT, drop the pointers to other backends, label the bun…
john-rocky Sep 10, 2026
140620a
docs: int8 on iOS is an entitlement setting, not a wall
john-rocky Sep 10, 2026
4f07606
docs: trim the comments in the code blocks
john-rocky Sep 10, 2026
ed6a888
docs: IoT in the LiteRT-LM rows, OpenAI-compatible server section and…
john-rocky Sep 10, 2026
d860be8
docs: three wording fixes from a final read (sampler default, output …
john-rocky Sep 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README-cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,7 @@ MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**，主流推理引擎
| LM Studio | GGUF，Mac 桌面应用与 OpenAI server | [lmstudio.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/lmstudio.md) | [minicpm5-deploy-lmstudio](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-lmstudio/SKILL.md) |
| MLX | MLX / 4bit，Apple Silicon 本地推理 | [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md) | [minicpm5-deploy-mlx](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-mlx/SKILL.md) |
| ArcLight | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) |
| LiteRT-LM | `.litertlm` 端侧运行时：Android / iOS / 桌面 / IoT，CPU + GPU | [litert.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/litert.md) | [minicpm5-deploy-litert](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-litert/SKILL.md) |
| vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) |

#### 微调
Expand Down Expand Up @@ -561,7 +562,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可

| 顶层 Skill | 作用 | 路由到 |
| --- | --- | --- |
| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | 推理路由 | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` |
| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | 推理路由 | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` · `litert` |
| **[`minicpm5-finetune`](./skills/minicpm5-finetune/SKILL.md)** | 微调路由 | `trl` · `llamafactory` · `ms-swift` · `unsloth` · `xtuner` |

在 Cursor / Claude Code 中可以这样调用：agent 会读取顶层 Skill，并根据目标后端、硬件和数据路径选择对应的子 Skill 与 cookbook，再执行命令并回报结果。
Expand All @@ -573,7 +574,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可

下方按推理后端和微调框架列出对应的 cookbook 与子 Skill。量化模型不单独作为后端列出，而是在可加载对应格式的推理后端中说明。

**推理部署**（7 个后端）
**推理部署**（9 个后端）

| 后端 | 适用模型 / 场景 | Cookbook | 对应 Agent Skill |
| --- | --- | --- | --- |
Expand All @@ -584,6 +585,7 @@ MiniCPM5-1B 使用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可
| [Ollama](https://github.com/ollama/ollama) | GGUF，本地端侧运行 | [`docs/deployment/ollama.md`](./docs/deployment/ollama.md) | [`minicpm5-deploy-ollama`](./skills/minicpm5-deploy-ollama/SKILL.md) |
| [LM Studio](https://lmstudio.ai) | GGUF，Mac 桌面应用与 OpenAI server | [`docs/deployment/lmstudio.md`](./docs/deployment/lmstudio.md) | [`minicpm5-deploy-lmstudio`](./skills/minicpm5-deploy-lmstudio/SKILL.md) |
| [MLX](https://github.com/ml-explore/mlx-lm) | MLX / 4bit，Apple Silicon 本地推理 | [`docs/deployment/mlx.md`](./docs/deployment/mlx.md) | [`minicpm5-deploy-mlx`](./skills/minicpm5-deploy-mlx/SKILL.md) |
| [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | `.litertlm` 端侧运行时：Android / iOS / 桌面 / IoT，CPU + GPU | [`docs/deployment/litert.md`](./docs/deployment/litert.md) | [`minicpm5-deploy-litert`](./skills/minicpm5-deploy-litert/SKILL.md) |
| [ArcLight](https://github.com/OpenBMB/ArcLight) | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [`docs/deployment/arclight.md`](./docs/deployment/arclight.md) | [`minicpm5-deploy-arclight`](./skills/minicpm5-deploy-arclight/SKILL.md) |

**微调**（5 个框架）
Expand Down
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ MiniCPM5-2B uses the **standard `LlamaForCausalLM` architecture**, so mainstream
| LM Studio | GGUF Mac desktop app and OpenAI server | [lmstudio.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/lmstudio.md) | [minicpm5-deploy-lmstudio](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-lmstudio/SKILL.md) |
| MLX | MLX / 4bit local inference on Apple Silicon | [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md) | [minicpm5-deploy-mlx](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-mlx/SKILL.md) |
| ArcLight | GGUF local on-device, CPU, Desktop & Server | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) |
| LiteRT-LM | `.litertlm` on-device runtime: Android / iOS / desktop / IoT, CPU + GPU | [litert.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/litert.md) | [minicpm5-deploy-litert](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-litert/SKILL.md) |
| vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) |

#### Fine-tuning
Expand Down Expand Up @@ -559,7 +560,7 @@ The two top-level skills cover deployment and fine-tuning:

| Top-level skill | What it does | Routes to |
| --- | --- | --- |
| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | Inference router | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` |
| **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | Inference router | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` · `litert` |
| **[`minicpm5-finetune`](./skills/minicpm5-finetune/SKILL.md)** | Fine tuning router | `trl` · `llamafactory` · `ms-swift` · `unsloth` · `xtuner` |

In Cursor / Claude Code, you can call them like this: the agent reads the top-level skill, selects the matching sub-skill and cookbook based on the target backend, hardware, and data path, then runs the command and reports back.
Expand All @@ -571,7 +572,7 @@ In Cursor / Claude Code, you can call them like this: the agent reads the top-le

The tables below list the cookbook and sub-skill for each inference backend and fine-tuning framework. Quantized models are not listed as standalone backends; they are described under the inference backend that can load each format.

**Inference Deployment** (7 backends)
**Inference Deployment** (8 backends)

| Backend | Model format / use case | Cookbook | Paired Agent Skill |
| --- | --- | --- | --- |
Expand All @@ -581,6 +582,7 @@ The tables below list the cookbook and sub-skill for each inference backend and
| [Ollama](https://github.com/ollama/ollama) | GGUF local on-device runtime | [`docs/deployment/ollama.md`](./docs/deployment/ollama.md) | [`minicpm5-deploy-ollama`](./skills/minicpm5-deploy-ollama/SKILL.md) |
| [LM Studio](https://lmstudio.ai) | GGUF Mac desktop app and OpenAI server | [`docs/deployment/lmstudio.md`](./docs/deployment/lmstudio.md) | [`minicpm5-deploy-lmstudio`](./skills/minicpm5-deploy-lmstudio/SKILL.md) |
| [MLX](https://github.com/ml-explore/mlx-lm) | MLX / 4bit local inference on Apple Silicon | [`docs/deployment/mlx.md`](./docs/deployment/mlx.md) | [`minicpm5-deploy-mlx`](./skills/minicpm5-deploy-mlx/SKILL.md) |
| [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | `.litertlm` on-device runtime: Android / iOS / desktop / IoT, CPU + GPU | [`docs/deployment/litert.md`](./docs/deployment/litert.md) | [`minicpm5-deploy-litert`](./skills/minicpm5-deploy-litert/SKILL.md) |
| [ArcLight](https://github.com/OpenBMB/ArcLight) | GGUF local on-device, CPU, Desktop & Server | [`docs/deployment/arclight.md`](./docs/deployment/arclight.md) | [`minicpm5-deploy-arclight`](./skills/minicpm5-deploy-arclight/SKILL.md) |

**Fine tuning** (5 frameworks)
Expand Down
Loading