Summary
The sampling table in skills/minicpm5-deploy-llama-cpp/SKILL.md (and the matching guidance in docs/deployment/llama_cpp.md) lists only --temp 1.0 and --top-p 0.95 for MiniCPM5-2B Think. With exactly those settings and nothing else, I see a very high rate of generations where the thinking channel collapses into repetition and never terminates.
Adding a single flag, --repeat-penalty, changes the outcome dramatically. Everything else was held constant across all ten runs below; only that one value changed.
HumanEval+ (164 tasks), RTX 3060 12GB
| repeat-penalty |
Q8_0 score |
Q8_0 runaway rate |
Q4_K_M score |
Q4_K_M runaway rate |
| 1.00 (as documented) |
43.3 |
55.5% |
6.1 |
92.1% |
| 1.05 |
86.0 |
7.9% |
34.1 |
61.6% |
| 1.10 |
90.9 |
4.9% |
67.7 |
25.6% |
| 1.15 |
92.1 |
1.8% |
71.3 |
11.0% |
| 1.20 |
87.1 |
5.5% |
60.1 |
7.9% |
Both quants peak at 1.15 and regress at 1.20, so this is not simply "more is better" — 1.15 looks like a real optimum rather than an artifact.
Two things stand out:
- At the documented setting, Q4_K_M is effectively unusable for coding — 92% of tasks never produce an answer at all. The quant table in the same SKILL.md describes Q4_K_M as a "small drop, ideal for laptops", which is fair at 1.15 but very misleading at 1.00.
- The lower quant is far more sensitive to this. Q8_0 recovers almost fully in a single step from 1.00 to 1.05, while Q4_K_M needs the whole sweep. So the omission hurts exactly the users the recommended quant is aimed at.
Setup
- llama.cpp served through llama-swap (
ghcr.io/mostlygeek/llama-swap:unified-cuda), RTX 3060 12GB
- Official GGUFs from
openbmb/MiniCPM5-2B-GGUF, both Q8_0 and Q4_K_M
-ngl 99 -c 131072 -fa on --jinja -np 4 -ctk f16 -ctv f16 -b 2048 -ub 1024 --temp 1.0 --top-p 0.95- Thinking enabled. My harness runs one repair round, so these scores are not directly comparable to standard pass@1 numbers — the relative movement is the point, not the absolute values.
- "Runaway rate" is my harness killing a generation once it detects repetition. That is my own definition, not a MiniCPM concept.
Question
Was repeat-penalty left out deliberately — for example because it costs something on a capability I did not measure? If not, would you consider adding a recommended value to the sampling table for MiniCPM5-2B? I only tested HumanEval+ on one machine, so a value validated against your own eval suite would be much better than mine.
Related: #360 covered a different gap in the same sampling guidance, so it may be worth reviewing the documented profiles as a whole rather than patching one value at a time.
Summary
The sampling table in
skills/minicpm5-deploy-llama-cpp/SKILL.md(and the matching guidance indocs/deployment/llama_cpp.md) lists only--temp 1.0and--top-p 0.95for MiniCPM5-2B Think. With exactly those settings and nothing else, I see a very high rate of generations where the thinking channel collapses into repetition and never terminates.Adding a single flag,
--repeat-penalty, changes the outcome dramatically. Everything else was held constant across all ten runs below; only that one value changed.HumanEval+ (164 tasks), RTX 3060 12GB
Both quants peak at 1.15 and regress at 1.20, so this is not simply "more is better" — 1.15 looks like a real optimum rather than an artifact.
Two things stand out:
Setup
ghcr.io/mostlygeek/llama-swap:unified-cuda), RTX 3060 12GBopenbmb/MiniCPM5-2B-GGUF, both Q8_0 and Q4_K_M-ngl 99 -c 131072 -fa on --jinja -np 4 -ctk f16 -ctv f16 -b 2048 -ub 1024 --temp 1.0 --top-p 0.95Question
Was
repeat-penaltyleft out deliberately — for example because it costs something on a capability I did not measure? If not, would you consider adding a recommended value to the sampling table for MiniCPM5-2B? I only tested HumanEval+ on one machine, so a value validated against your own eval suite would be much better than mine.Related: #360 covered a different gap in the same sampling guidance, so it may be worth reviewing the documented profiles as a whole rather than patching one value at a time.