|
| 1 | +# LevelCode — Kimi K3 support — scope & implementation plan |
| 2 | + |
| 3 | +**Goal:** make Moonshot's **Kimi K3** a first-class model in LevelCode — both to catch the current |
| 4 | +wave of K3 interest and as an adoption lever ("LevelCode runs Kimi K3"). |
| 5 | + |
| 6 | +**The happy news:** the provider layer is explicitly *"many providers, little code"* — a new |
| 7 | +OpenAI-compatible model is a **data row, not new code** (`providers/openaiCompat.js:2-8`). The whole |
| 8 | +client-side integration is ~4 small edits along the existing **DeepSeek** template. |
| 9 | + |
| 10 | +**The one catch:** K3 is an **always-on reasoning model**, and the agent's tool loop **drops** |
| 11 | +`reasoning_content`. LevelCode already had to disable the agent for exactly one model for this reason |
| 12 | +(`deepseek-reasoner`, `tools: false`). Whether K3 survives the loop is unknown and **must be validated |
| 13 | +before shipping** — there is no non-reasoning K3 to fall back to. This is §3, and it gates everything. |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +## 1. The two paths (recommend both, sequenced) |
| 18 | + |
| 19 | +| | **Path A — BYOK direct provider** | **Path B — Cloud gateway flagship** | |
| 20 | +|---|---|---| |
| 21 | +| What | User brings their own Moonshot key; LevelCode calls `api.moonshot.ai` directly | User just signs in; LevelCode Cloud proxies to K3, billed in LevelCode credits | |
| 22 | +| Where | This repo only (`extensions/levelcode-ai/`) | thin.ly backend roster + billing **and** a client constant flip | |
| 23 | +| Size | ~4 small edits + tests | Backend work (roster, cost multipliers, capacity) — out of this repo | |
| 24 | +| Friction to **use** | High — needs a Moonshot account, key, billing | **None** — the real advertising vehicle | |
| 25 | +| Advertisable as | "LevelCode now supports Kimi K3 (BYOK)" — to the K3-enthusiast crowd who already have keys | "Kimi K3, built in — just sign in" — to everyone | |
| 26 | + |
| 27 | +**Recommendation:** ship **Path A first** (small, low-risk, independently shippable, immediately |
| 28 | +advertisable), then do **Path B** as the mass-adoption follow-up. Both are gated by the §3 spike. |
| 29 | +The gateway already ships a Moonshot flagship today — `GATEWAY_PRO_MODEL = 'moonshotai/kimi-k2.7-code'` |
| 30 | +(`extension.js:174-175`), the default of `levelcode.ai.cloudModel` (`package.json:317`) — so Path B is |
| 31 | +"upgrade the flagship to K3," not "wire Moonshot from scratch." |
| 32 | + |
| 33 | +--- |
| 34 | + |
| 35 | +## 2. The Kimi K3 API (verified against Moonshot's own docs, 2026-07-20) |
| 36 | + |
| 37 | +| | | |
| 38 | +|---|---| |
| 39 | +| Base URL | `https://api.moonshot.ai/v1` — **OpenAI-compatible** | |
| 40 | +| Auth | `Authorization: Bearer $MOONSHOT_API_KEY` | |
| 41 | +| Model id | **`kimi-k3`** (canonical; "K3 Max" / "Swarm Max" are product names, not API strings) | |
| 42 | +| Streaming | Yes — emits **separate `reasoning_content` and `content`** deltas | |
| 43 | +| Tools | Yes — official tool-calling + dynamic-loading guides exist | |
| 44 | +| Context | 1,000,000 tokens | |
| 45 | +| Pricing | ~$3 / M input, $15 / M output, $0.30 / M cached input *(secondary sources — confirm on the official pricing page before Path B billing)* | |
| 46 | + |
| 47 | +**Two corrections to watch for:** |
| 48 | +- The API host is **`api.moonshot.ai`**, not `platform.kimi.ai` (that's the docs/console host — it would 404). |
| 49 | +- There is **no Anthropic-compatible endpoint** (confirmed against Moonshot's official `llms.txt` index). |
| 50 | + K2 had one; K3 does not. So this does **not** reuse the native Anthropic provider — it goes through the |
| 51 | + OpenAI-compat adapter. |
| 52 | + |
| 53 | +Sources: `platform.kimi.ai/docs/guide/kimi-k3-quickstart`, `platform.kimi.ai/docs/api/overview`, |
| 54 | +`platform.kimi.ai/docs/llms.txt`. |
| 55 | + |
| 56 | +--- |
| 57 | + |
| 58 | +## 3. The gate — validate the reasoning/tool loop FIRST (½ day, blocks everything) |
| 59 | + |
| 60 | +**Why this is the whole risk.** K3 streams its thinking as a separate `reasoning_content` field. The |
| 61 | +universal adapter reads **only** `d.content` and `d.tool_calls` and never `reasoning_content` |
| 62 | +(`openaiCompat.js:72-77`, `:229-235`). So the stored transcript carries no reasoning, and the next |
| 63 | +turn's request (rebuilt by `translate.toOpenAIMessages`) echoes none back. For most reasoning models |
| 64 | +that is fine — reasoning is output-only. But **`deepseek-reasoner` is marked `tools: false`** |
| 65 | +(`providers/index.js:73-83`, `catalog.js`) with the note *"its tool loop needs reasoning_content |
| 66 | +round-tripping the boundary doesn't carry."* If K3 has the same requirement, it breaks in the agent — |
| 67 | +and because K3's thinking is **always on**, there is no `kimi-k3-chat` to fall back to. |
| 68 | + |
| 69 | +**The test — run before writing any provider row.** With a Moonshot key, drive a **multi-step** |
| 70 | +tool-calling conversation directly against `https://api.moonshot.ai/v1/chat/completions`, `model: |
| 71 | +"kimi-k3"`, `stream: true`, sending `tools`, and on the second turn send the assistant's prior |
| 72 | +`tool_calls` + the `tool` result **without** any `reasoning_content` (exactly what the transcript |
| 73 | +boundary produces). Three outcomes: |
| 74 | + |
| 75 | +- **Works** → Path A is a pure data-row addition. Proceed to §4. |
| 76 | +- **400 / rejects the turn** (like deepseek-reasoner) → K3 cannot be an agent model via the naive |
| 77 | + adapter. Options: (a) teach `streamOpenAIAgentTurn` to capture `reasoning_content` into a block and |
| 78 | + `translate.toOpenAIMessages` to re-emit it (real adapter work, no longer a data row); or (b) ship K3 |
| 79 | + as **chat-only** (`tools: false`) and say so. Decide then. |
| 80 | +- **Works but pollutes** — leaks `<think>…</think>` into `content` → already handled: the agent strips |
| 81 | + inline think tags from stored turns (`agent.js:621-631`). Just confirm it's clean. |
| 82 | + |
| 83 | +Also confirm two body details during the spike: Moonshot accepts `max_tokens` (the adapter sends |
| 84 | +`max_tokens`, not `max_completion_tokens`, because `kimi-k3` doesn't match the `o[1-9]` reasoning |
| 85 | +heuristic at `openaiCompat.js:42-44`), and it accepts a `temperature` (K3 "thinking always on" endpoints |
| 86 | +sometimes reject it). The quickstart's bare `model + messages` call implies defaults are fine; verify |
| 87 | +under `tools`. |
| 88 | + |
| 89 | +**Runnable version:** `extensions/levelcode-ai/scripts/kimi-k3-spike.js` does exactly this — it calls the |
| 90 | +*real* `streamOpenAIAgentTurn` twice, building turn 2's transcript the way `agent.js` does (assistant |
| 91 | +`tool_use` with reasoning already dropped, then a `tool_result`), and prints a PASS/FAIL/INCONCLUSIVE |
| 92 | +verdict. It defaults to the Path-B route (OpenRouter, `moonshotai/kimi-k3`) so it validates production: |
| 93 | +```bash |
| 94 | +export OPENROUTER_API_KEY=… # thin.ly: `rails credentials:show` → openrouter_api_key, or the EB env |
| 95 | +node extensions/levelcode-ai/scripts/kimi-k3-spike.js |
| 96 | +``` |
| 97 | +PASS → keep `tools:true`; FAIL → set the catalog `kimi-k3` to chat-only or add reasoning capture to the |
| 98 | +adapter. Runs server-side with the gateway's own key — no user key, no editor build. |
| 99 | + |
| 100 | +--- |
| 101 | + |
| 102 | +## 4. Path A — BYOK direct Moonshot provider (client only) |
| 103 | + |
| 104 | +Copy the **DeepSeek** row — the closest analog (dedicated key, own `api.*/v1`, curated model list). No |
| 105 | +adapter, dispatch, key-storage, picker, streaming, or tool-translation code changes: all are |
| 106 | +provider-agnostic and key on the row's `kind: 'openai'`. |
| 107 | + |
| 108 | +**A1 — registry row** · `providers/index.js` (alongside `deepseek`, ~`:73`): |
| 109 | +```js |
| 110 | +moonshot: { |
| 111 | + id: 'moonshot', kind: 'openai', label: 'Moonshot (Kimi)', |
| 112 | + baseURL: 'https://api.moonshot.ai/v1', keyId: 'moonshot', |
| 113 | + caps: { tools: true }, // ← set tools:false here if §3 says chat-only |
| 114 | + models: [ |
| 115 | + { id: 'kimi-k3', label: 'Kimi K3', detail: 'Moonshot · 1M ctx, reasoning' } |
| 116 | + ] |
| 117 | +}, |
| 118 | +``` |
| 119 | +Follow DeepSeek exactly: no `vision` (image wire-format unverified), no `fast` (K3 is a large, slow |
| 120 | +reasoning model — a poor ghost-text completer), no `agent: true` (appears claude-specific). |
| 121 | + |
| 122 | +**A2 — per-model caps** · `providers/catalog.js` `CAPS` table (~`:27-56`): |
| 123 | +```js |
| 124 | +'kimi-k3': { context: 1000000, tools: true, reasoning: true }, |
| 125 | +``` |
| 126 | +`context` feeds the context meter; without it the agent falls back to 200 000 (`agent.js:618`) and would |
| 127 | +badly under-report K3's 1M window. Do **not** add `kimi-k3` to `FAST_COMPLETION` (`catalog.js:110-116`). |
| 128 | + |
| 129 | +**A3 — settings enum** · `package.json`, `levelcode.ai.provider` (`:274-304`): append `"moonshot"` to |
| 130 | +**both** the `enum` array and the parallel `enumDescriptions` array (they must stay index-aligned). The |
| 131 | +QuickPick lists built-in models automatically, but the settings-UI dropdown reads this enum. |
| 132 | + |
| 133 | +**A4 — tests** · add `'moonshot'` to the `kind === 'openai'` id list (`test/providers.test.js:77`) and |
| 134 | +the `listProviders` presence list (`:97`); assert `secretStorageKey('moonshot') === |
| 135 | +'levelcode.ai.key.moonshot'`. Add the `'kimi-k3'` caps-resolution + `supportsToolsForModel('moonshot', |
| 136 | +'kimi-k3')` cases to `test/catalog.test.js`. Run: `for t in test/*.test.js; do node "$t"; done`. |
| 137 | + |
| 138 | +**A5 — (optional) show K3's reasoning.** By default the separate `reasoning_content` is dropped, so users |
| 139 | +see answers but not thinking — fine, and less clutter. If we want "watch Kimi think" as a demo hook, |
| 140 | +teach `deltaFromEvent` / `streamOpenAIAgentTurn` to surface `d.reasoning_content` into a collapsible |
| 141 | +reasoning block. Real adapter work; defer unless §3 forces it or marketing wants it. |
| 142 | + |
| 143 | +**Verification (Path A):** unit tests green; then a live smoke test — set a Moonshot key via |
| 144 | +`AI: Set API Key…`, pick **Moonshot (Kimi) → Kimi K3**, and (1) send a plain chat message, (2) run a |
| 145 | +real agent task that edits a file and runs a command (the §3 loop, now end-to-end in the editor), (3) |
| 146 | +confirm the context pill shows the 1M window. No key must ever touch settings/logs — it's in |
| 147 | +SecretStorage (`extension.js:128-152`). |
| 148 | + |
| 149 | +--- |
| 150 | + |
| 151 | +## 5. Path B — Kimi K3 in the Cloud gateway ← **CHOSEN** |
| 152 | + |
| 153 | +The zero-friction, mass-adoption path. It turned out **far simpler** than "stand up backend billing": |
| 154 | +the gateway is driven by **one table** — `Levelcode::ModelCatalog::MODELS` in |
| 155 | +`app/services/levelcode/model_catalog.rb` — and everything (entitlement, pricing, the derived multiplier, |
| 156 | +the editor picker, the pricing page) falls out of it. And Kimi is reached **via OpenRouter**, not a direct |
| 157 | +Moonshot key: the direct `MoonshotAdapter` is a disabled stub, and any `moonshotai/*` slug routes through |
| 158 | +`OpenRouterAdapter` on the existing `OPENROUTER_API_KEY` (`ai_router.rb:49-56`). **Verified 2026-07-20:** |
| 159 | +`moonshotai/kimi-k3` is **live on OpenRouter** — 1,048,576 ctx, $3/$15 in/out, cache $0.19–0.30. |
| 160 | + |
| 161 | +### It ships without an editor release |
| 162 | +The client picker reads the roster **live** from `GET /account/models` (`extension.js:210-233`). A new |
| 163 | +`status: :confirmed` catalog row appears in every entitled user's picker **with no client update**. That |
| 164 | +is the entire "make it available" feature. |
| 165 | + |
| 166 | +### Two shapes — very different blast radius |
| 167 | + |
| 168 | +**B1 — Add K3 as a *selectable* Pro model (recommended first).** One row in `model_catalog.rb` (after the |
| 169 | +K2.7 entry, ~`:35`): |
| 170 | +```ruby |
| 171 | +"moonshotai/kimi-k3" => { |
| 172 | + label: "Kimi K3", provider: "openrouter", |
| 173 | + input: 3.00, cached_input: 0.30, output: 15.00, # OpenRouter list $/M; ROUTING_FEE 1.055 added on top |
| 174 | + context: 1_048_576, min_tier: :pro, status: :confirmed |
| 175 | +}, |
| 176 | +``` |
| 177 | +Server-only, ships instantly, doesn't touch anyone's default or existing cost. `status: :assumption` |
| 178 | +first if you want it shown-but-not-billable during the spike. Reversible (flip status / delete the row). |
| 179 | +Its **derived multiplier is ≈ 4×** the current K2.7 baseline (`REFERENCE_TURN` cost ratio) — the picker |
| 180 | +shows that honestly, so users opt into the cost knowingly. |
| 181 | + |
| 182 | +**B2 — Make K3 THE default flagship (separate, riskier — do NOT bundle with B1).** Also change |
| 183 | +`DEFAULT_MODEL` (`lib/levelcode.rb:90`), the client `GATEWAY_PRO_MODEL` + label (`extension.js:174-175`), |
| 184 | +and `levelcode.ai.cloudModel` default (`package.json:317`). This **re-bases every multiplier** (they're |
| 185 | +relative to `DEFAULT_MODEL`), rewrites the multiplier specs, and — because K3 is ~4× the cost — **quarters |
| 186 | +every paid user's effective turns-per-plan** silently. Plan budgets are revenue-based and model-independent |
| 187 | +(`lib/levelcode.rb`), so margins are safe, but the *user-visible* `turns_left` drops ~4×. Defer until |
| 188 | +capacity + cost are proven. |
| 189 | + |
| 190 | +### Tests (B1) |
| 191 | +- `spec/models/levelcode_model_catalog_spec.rb` — add K3 to the `rate_table` (`:16-19`), the multiplier |
| 192 | + map (`:30-36`), and confirmed-status list (`:7-14`); keep `MODELS` ordered so the monotonic-multiplier |
| 193 | + assertion (`:38-42`) still holds. |
| 194 | +- `spec/requests/api/levelcode/v1/account_spec.rb:59-82` — assert K3 appears with `live: true`. |
| 195 | +- Optional request spec: a Pro user may select `moonshotai/kimi-k3`; a free user may not. |
| 196 | + |
| 197 | +### No changes needed |
| 198 | +No adapter/key/routing edit (rides OpenRouter), no migration, no plan/budget change, no streaming-proxy |
| 199 | +change — the proxy re-emits chunks **verbatim**, so K3's `reasoning_content` already passes straight |
| 200 | +through to the editor (`ai_controller.rb:332-334`). |
| 201 | + |
| 202 | +### The §3 spike, now server-side |
| 203 | +Because thin.ly holds the `OPENROUTER_API_KEY`, the always-on-reasoning validation runs **without any user |
| 204 | +key** — drive a multi-step agent turn through the gateway (or hit OpenRouter directly with that key) and |
| 205 | +confirm K3 survives the tool loop with no `reasoning_content` echoed back. OpenRouter bills the hidden |
| 206 | +reasoning trace at the full $15/M output rate, so watch real cost during the spike, not just correctness. |
| 207 | + |
| 208 | +--- |
| 209 | + |
| 210 | +## 6. Risks |
| 211 | + |
| 212 | +- **Always-on reasoning in the tool loop (§3)** — the one that can sink it. No non-reasoning K3 fallback. |
| 213 | +- **Model-id drift** — Moonshot may also expose a dated snapshot (`kimi-k3-0716`-style, as K2 did). We seed |
| 214 | + the stable `kimi-k3`; the picker's "Browse all models (live)" (`extension.js:1230-1250` → `GET /v1/models`) |
| 215 | + surfaces snapshots without a code change. |
| 216 | +- **Data governance** — Moonshot is a PRC company; `api.moonshot.ai` is the international endpoint but some |
| 217 | + users/orgs will care where prompts go. For BYOK it's the user's own call; for the **gateway default** |
| 218 | + (Path B) it's LevelCode's, so it's a positioning decision, not just an engineering one. Worth a line in |
| 219 | + the model's `detail` and the marketing. |
| 220 | +- **Advertising accuracy** — K3's own launch materials put it behind the current Claude/GPT flagships on |
| 221 | + overall quality (ahead on some coding/agent benches). Advertise it as "supported / strong at coding," |
| 222 | + not "best model," so the claim survives contact with users. |
| 223 | +- **OpenRouter capacity (blocks B2, not B1)** — OpenRouter's own K3 page warns *"upstream capacity is |
| 224 | + currently limited, may return frequent 429 errors"* (4 days post-launch). As a *selectable* model (B1) |
| 225 | + a 429 hits only the user who chose it; as the *default* (B2) it would hit the entire paid base. Wait for |
| 226 | + capacity to settle before B2. Confirm the gateway degrades a 429 gracefully (`classify_upstream`, |
| 227 | + `ai_controller.rb:406-425`). |
| 228 | +- **Cost, ~4× and hidden** — K3 is *always-on reasoning with no non-thinking mode*, and OpenRouter bills |
| 229 | + the reasoning trace at the full **$15/M output** rate. So every turn costs more than its visible answer |
| 230 | + implies, and K3's turn is ~4× the current flagship's. Fine as an opt-in (B1, multiplier shown); a silent |
| 231 | + ~4× budget burn if made default (B2). |
| 232 | + |
| 233 | +--- |
| 234 | + |
| 235 | +## 7. Exit test |
| 236 | + |
| 237 | +1. **Spike (§3)** passes, or the fallback (chat-only / adapter work) is chosen deliberately. |
| 238 | +2. **Path A:** unit suites green (14 → still green with the new cases); live smoke test — chat + a |
| 239 | + real file-editing, command-running agent task on `kimi-k3` — succeeds; context pill shows ~1M. |
| 240 | +3. **Path B (if pursued):** signed-in user with zero keys selects Kimi K3, runs an agent task, and the |
| 241 | + credits meter decrements at the expected multiplier. |
| 242 | + |
| 243 | +## 8. Not doing (yet) |
| 244 | + |
| 245 | +- Self-hosting the open weights (released ~2026-07-27) — out of scope; BYOK/gateway only. |
| 246 | +- Vision/image input — K3 is multimodal, but the editor's image path isn't verified against Moonshot; |
| 247 | + ship text/code first, revisit `caps.vision` later. |
| 248 | +- A bespoke reasoning-display UI (A5) — only if the spike or marketing calls for it. |
0 commit comments