docs: LiteRT-LM cookbook and Agent Skill for MiniCPM5-2B / MiniCPM5-1B (Android / iOS / desktop) - #375
Merged
Merged
Conversation
…B (Android / iOS / desktop) - docs/deployment/litert.md: the litert-lm CLI (download from the Hub, CPU / GPU, thinking on / off / budget, sampling), Android through the AI Edge Gallery app and the Kotlin API, an iOS pointer, the measured tables from the litert-community card, pitfalls. Every command was run with litert-lm 0.17.0; the Kotlin path on a Galaxy S26 with litertlm-android 0.17.0. - skills/minicpm5-deploy-litert/SKILL.md: the paired Agent Skill (same shape as minicpm5-deploy-mlx). - skills/minicpm5-deploy/SKILL.md: one checkpoint row, one decision-matrix row, one default pick, one line on the sanity check. - README.md / README-cn.md: one row in each deployment table, `litert` in the router's route list, backend count 7 -> 8.
…ok, skill and README rows
smilingday
reviewed
Sep 10, 2026
smilingday
reviewed
Sep 10, 2026
smilingday
reviewed
Sep 10, 2026
smilingday
reviewed
Sep 10, 2026
smilingday
reviewed
Sep 10, 2026
smilingday
reviewed
Sep 10, 2026
… skill step, thinking and iOS wording from the sources
…cap scope, thinking flag in the skill)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for the per-backend cookbooks and Agent Skills. The MiniCPM5-2B card on Hugging Face already lists the LiteRT-LM variant, but the cookbook table and the
minicpm5-deployrouter have no route to it, so "run MiniCPM5 on an Android phone" or "in an on-device app" has nowhere to go. This adds LiteRT-LM (Google's on-device runtime, formerly TensorFlow Lite) in the shape the table uses: a cookbook and a paired skill.What it adds:
docs/deployment/litert.mdandskills/minicpm5-deploy-litert/SKILL.md: thelitert-lmCLI on a desktop (download from the Hub, CPU / GPU, thinking on / off / budget), Android through the AI Edge Gallery app and the Kotlin API, an iOS pointer, the measured tables, and the pitfalls.minicpm5-deploy: one checkpoint row, one decision-matrix row, one default pick, one line on the sanity check.litertin the router's route list.The 2B bundles (int4 1.55 GB, int8 2.60 GB) are my conversions, hosted in litert-community/MiniCPM5-2B with the recipe and the measurements in the card; one file runs on CPU and GPU (M4 Max, Galaxy S26; int4 also on iPhone 17 Pro). The 1B rows use the three files already in litert-community/MiniCPM5-1B. Every command on the page was run today with litert-lm 0.17.0, and the Kotlin path was run on a Galaxy S26 with litertlm-android 0.17.0 (every node delegated to OpenCL, reasoning on the thought channel). The numbers are the card's: int4 decodes 92.8 tok/s on the M4 Max GPU and 16–19 tok/s on the S26 GPU; GSM8K with thinking off is 91 % (int8) and 86–87 % (int4) against 92 % for bf16.
One thing worth knowing: with thinking on, the int4 file can fail to close its reasoning (0/10 on the CPU on the card's thinking-on GSM8K subset, where int8 closes 9/10 like bf16). The cookbook says so and points to
--thinking falseor the int8 file.#369 adds a row to the same tables; whichever merges first, I will rebase the other. If any wording or placement should be different for this repo, tell me and I will change it. Thanks again.
cc @smilingday