Longer context by default for the 27B and the 35B MoE (a context ladder, capped at full speed)

CHANGELOG.md

@@ -4,6 +4,9 @@ All notable changes to Pooled (called SwarmLLM before September 2026). Format fo
 
 ## [Unreleased]
 
+### Changed
+- **The 27B and the 35B MoE open at a long context by default; `?ctx=` is no longer needed** (#421). Each has a ladder of lengths checked correct on it (27B: 64K, 32K, 16K, 8K; MoE: 128K, 64K, 32K, 16K, 8K), and a room opens at the longest step its memory holds, past the old default (16K / 32K) only at full speed: no expert offloaded to RAM or disk and no more devices holding layers than at the default (a longer window alone costs memory, not decode speed: identical tokens and tok/s at 32K and 128K on the GB10). A room short even for the default now opens at a shorter step (down to 8K) and says so, instead of not starting. The picker's **Context** row (8K/16K/32K/64K on the 27B, 8K/32K/64K/128K on the MoE) shows each length's memory in the room and picks another; Room settings changes it while the model runs. The room page, `pooled host` / `pooled serve`'s room and the OpenClaw plugin pick the same context (`room/ctxpick.js` `roomCtxPlan`); room nodes no longer open the MoE at 128K whatever the room holds. `?ctx=N` / `--ctx N` still ask for an exact length. The 1.7B and Flash are unchanged.
+
 ## [2.0.0] - 2026-10-16
 
 **Run a 125B model from your SSD, in a browser tab, and pool devices to make it faster.** Pooled 2.0 runs Qwen3.8 Flash (**experimental**), a 125B mixture-of-experts model with about 6B active per token. It is far bigger than most GPUs' memory, so each device keeps its share of the experts on its SSD (browser storage for a tab, the model files for `pooled join`), caches recently used experts on its GPU (routing repeats: about 17% of a layer's experts serve about 77% of lookups), and reads the rest from disk as tokens need them. Disk is the slow part; every device that joins holds part of the model in its memory, so less comes from disk and it gets faster: a Mac Studio alone in Chrome decodes about 8 tok/s, and with an RTX 5070 PC joined about 10-12 tok/s, depending on the network. Flash rooms run at up to 64K context (128K is in progress). 2.0 also brings the system-RAM and SSD expert tiers in the terminal, `pooled serve`, channel-bound join auth and the default TURN relay. Release notes and the demo video: [docs/releases/2.0.md](docs/releases/2.0.md).

README.md

@@ -43,8 +43,8 @@ Models in the room today:
 
 | Model | Size | Notes |
 |---|---|---|
-| Qwen 3.8 27B | Q4_0, needs ~17 GB across the room | hybrid Gated DeltaNet + attention, built-in draft layer for speculative decoding, 16K context (up to 64K) |
-| Qwen 3.6 35B MoE | Q4_0, needs ~22.5 GB across the room | 256 experts, 8 active per token, so it decodes several times faster than the 27B; 32K context (up to 128K) |
+| Qwen 3.8 27B | Q4_0, needs ~17 GB across the room | hybrid Gated DeltaNet + attention, built-in draft layer for speculative decoding; opens at the longest context the room holds at full speed, up to 64K (16K at ~17 GB, 64K at ~20 GB) |
+| Qwen 3.6 35B MoE | Q4_0, needs ~22.5 GB across the room | 256 experts, 8 active per token, so it decodes several times faster than the 27B; opens at the longest context the room holds at full speed, up to 128K (32K at ~22.5 GB, 128K at ~23.5 GB) |
 | Qwen3 1.7B | Q8_0, needs ~5.6 GB across the room for 16K context, ~4 GB for 8K | small and quick, for rooms of phones and light laptops; 16K context, 8K when the room is short of memory |
 | Qwen3.8 Flash (**experimental**) | Q4_0, 125B (6B active), ~99 GB on disk | 512 experts, 10 active per token; keeps most of itself on disk, so it needs far less GPU memory than its size; 64K context. See below |
 

cli/README.md

@@ -179,7 +179,7 @@ ANTHROPIC_BASE_URL=http://127.0.0.1:8080 ANTHROPIC_API_KEY=pooled \
   CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 claude --model pooled
 ```
 
-`CLAUDE_CODE_MAX_CONTEXT_TOKENS` is the room's context (the banner prints this line for your room): Claude Code does not know a model called `pooled`, and without it never compacts before the room refuses a prompt. `CLAUDE_CODE_MAX_OUTPUT_TOKENS` (a quarter of the context, at most 32000): Claude Code keeps its output limit (32000) free in the window, so on a room under about 32 k without it, it refuses its own prompt ("Prompt is too long") before sending anything. `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` keeps its side requests (titles) from interleaving with the agent's and costing a re-prefill. Its prompt is 15-20 k tokens before any history, so use a room with a long context: the 35B MoE (32 k by default, up to 64 k with `?ctx=65536`) or the 27B (up to 32 k). The 1.7B's 16 k leaves about 1 k tokens after Claude Code's prompt: a file read can already overflow it, and the 1.7B cannot write the compaction summary, so use a bigger model for Claude Code. Each step after the first reuses the room's caches and prefills only the new tool results. MCP servers are fine: up to 1024 tools per request.
+`CLAUDE_CODE_MAX_CONTEXT_TOKENS` is the room's context (the banner prints this line for your room): Claude Code does not know a model called `pooled`, and without it never compacts before the room refuses a prompt. `CLAUDE_CODE_MAX_OUTPUT_TOKENS` (a quarter of the context, at most 32000): Claude Code keeps its output limit (32000) free in the window, so on a room under about 32 k without it, it refuses its own prompt ("Prompt is too long") before sending anything. `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` keeps its side requests (titles) from interleaving with the agent's and costing a re-prefill. Its prompt is 15-20 k tokens before any history, so use a room with a long context: the 35B MoE or the 27B, which a room opens at the longest context it holds at full speed (up to 128 k and 64 k; at least 32 k and 16 k when the room holds those; `?ctx=` or `--ctx` asks for an exact length). The 1.7B's 16 k leaves about 1 k tokens after Claude Code's prompt: a file read can already overflow it, and the 1.7B cannot write the compaction summary, so use a bigger model for Claude Code. Each step after the first reuses the room's caches and prefills only the new tool results. MCP servers are fine: up to 1024 tools per request.
 
 ### opencode
 

cli/lib/hostrun.js

@@ -19,7 +19,7 @@ export async function runHostInteractive(opts, { prepared, version = "" }) {
   const { rn, loader, rule, mem, ram, disk } = prepared;
   const lib = { MODELS: rn.MODELS, FILES: rn.FILES, NEED_GB: rn.NEED_GB, roomBytes: rn.roomBytes, roomFit: rn.roomFit, shortNote: rn.shortNote,
     shortBy: rn.shortBy, gbUp: rn.gbUp, pledgeGB: rn.pledgeGB, nodeCtxFor: rn.nodeCtxFor, dealRoom: rn.dealRoom,
-    pickCtx: rn.pickCtx, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor, ngramFor: rn.ngramFor, ngramOf: rn.ngramOf };
+    pickCtx: rn.pickCtx, roomCtxPlan: rn.roomCtxPlan, maxSeqFor: rn.maxSeqFor, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor, ngramFor: rn.ngramFor, ngramOf: rn.ngramOf };
   const dir = opts.modelDir;
   const keys = hostable(rn.MODELS);
   const pulled = new Set(keys.filter((k) => modelState(dir, k, rn.MODELS, rn.FILES, rn.LOCAL).pulled));

cli/lib/hostui.js

@@ -31,8 +31,10 @@ export function deviceKind(meta = {}, self = false) {
 // (roomBytes), else room/models.js NEED_GB for a model without a SHAPE
 // rounded up to a tenth of a GiB, as room/models.js NEED_GB (the room page's number for the same model)
 const needAt = (lib, key, ctx) => { const rb = lib.roomBytes(key, ctx, "f16"); return rb ? Math.ceil((rb.L * rb.layerBytes + rb.hostBytes) / GiB * 10 - 1e-9) / 10 : null; };
+// (without --ctx: at the model's default, CTX def, as the room page's NEED_GB: a model with a ladder opens longer only
+// when the room holds that at full speed, room/ctxpick.js roomCtxPlan)
 export function modelNeedGB(lib, key, ctxAsk = 0) {
-  return needAt(lib, key, lib.nodeCtxFor(key, ctxAsk)) ?? lib.NEED_GB[key] ?? null;
+  return needAt(lib, key, lib.maxSeqFor ? lib.maxSeqFor(key, ctxAsk) : lib.nodeCtxFor(key, ctxAsk)) ?? lib.NEED_GB[key] ?? null;
 }
 // The context a node falls back to for `key` when the room is short for its default (room/models.js
 // pickCtx: the 1.7B opens at 8K when 16K doesn't fit), and what the model needs there; null when it
@@ -41,7 +43,11 @@ export function modelFallback(lib, key, ctxAsk = 0) {
   if (!lib.pickCtx) return null;
   const want = lib.nodeCtxFor(key, ctxAsk);
   const p = lib.pickCtx(key, { want, ask: ctxAsk, fitsAt: () => false });
-  return p.ctx < want ? { ctx: p.ctx, needGB: needAt(lib, key, p.ctx) } : null;
+  if (!(p.ctx < want)) return null;
+  // (a fallback that saves under 1 GB is not worth a word: the 27B's and the MoE's 8K, ~0.5 GB under their default; the
+  // room still opens there when that is all it holds)
+  const needGB = needAt(lib, key, p.ctx), def = modelNeedGB(lib, key, ctxAsk);
+  return needGB != null && def != null && def - needGB < 1 ? null : { ctx: p.ctx, needGB };
 }
 const kOf = (n) => `${Math.round(n / 1024)}K`;
 // "6 GB" or "6 GB (4 GB at 8K)": a model's need in words, whole GB
@@ -74,11 +80,15 @@ export function roomFitNow(lib, { model, devices, ctxAsk = 0, spareGB = [] }) {
   }
   const off = (rb) => (lib.offloadFor ? lib.offloadFor(devices.map((d) => d.meta), rb.experts || rb.expertBytes || 0, model) : null);
   // (rb.layerCost: Qwen3.8-Flash-Next's per-layer bytes, room/models.js flashLayerCost, which roomFit takes in place of one size)
-  const fitAt = (c) => { const rb = lib.roomBytes(model, c, "f16"); return lib.roomFit(rb.L, pl.map((g) => g * GiB), rb.layerCost || rb.layerBytes, rb.hostBytes, off(rb)); };
-  const pick = lib.pickCtx ? lib.pickCtx(model, { want, ask: ctxAsk, fitsAt: (c) => fitAt(c).fits }) : { ctx: want, want, fellBack: false };
+  // (lib.roomCtxPlan: the room node's own plan, room/ctxpick.js, so the screen opens the room where Start will: the top of
+  // the model's ladder down to what fits, past its default only at full speed)
+  const plan = lib.roomCtxPlan ? lib.roomCtxPlan(model, { metas: devices.map((d) => d.meta), pledges: pl.map((g) => g * GiB), ask: ctxAsk }) : null;
+  const fitAt = plan ? (c) => plan.fitAt(c).fit
+    : (c) => { const rb = lib.roomBytes(model, c, "f16"); return lib.roomFit(rb.L, pl.map((g) => g * GiB), rb.layerCost || rb.layerBytes, rb.hostBytes, off(rb)); };
+  const pick = plan || (lib.pickCtx ? lib.pickCtx(model, { want, ask: ctxAsk, fitsAt: (c) => fitAt(c).fits }) : { ctx: want, want, fellBack: false });
   const fb = modelFallback(lib, model, ctxAsk);
-  const base = { needGB, minGB: fb?.needGB ?? null, haveGB, ctx: pick.ctx, want, fellBack: !!pick.fellBack,
-    ctxNote: pick.fellBack && lib.ctxShortNote ? lib.ctxShortNote(label, pick.ctx, want) : "" };
+  const base = { needGB, minGB: fb?.needGB ?? null, haveGB, ctx: pick.ctx, want: pick.want ?? want, fellBack: !!pick.fellBack,
+    ctxNote: pick.fellBack && lib.ctxShortNote ? lib.ctxShortNote(label, pick.ctx, pick.want ?? want) : "" };
   const fit = fitAt(pick.ctx);
   if (fit.fits) return { fits: true, ...base, shortGB: 0, note: "", offload: !!fit.offload, disk: !!fit.disk };
   const shortGB = lib.gbUp(lib.shortBy ? lib.shortBy(fit, spareGB) : fit.short);

cli/lib/lend.js

@@ -126,9 +126,9 @@ export const HELP_HOST = `Usage
 
 More options
   --code <CODE>     the room code to use (default: a random one of six characters)
-  --ctx <n>         the context to ask for, in tokens (default: the longest the model takes in a
-                    room: 16k on the 1.7B, 8k when the room is short of memory for 16k; 64k on
-                    the 27B, 128k on the MoE)
+  --ctx <n>         the context to ask for, in tokens (default: the longest the room holds at full
+                    speed: up to 64k on the 27B and 128k on the MoE, 16k on the 1.7B; shorter
+                    when the room is short of memory)
   --devices <n>     the same as --wait (its older name)
   --signal <spec>   PeerJS signaling server(s), as pooled join
   --no-check        skip the test allocation that confirms the memory is there

cli/lib/lendrun.js

@@ -472,7 +472,7 @@ async function runHost(opts, out, prepared = null) {
   // --here: lend what the model needs on this computer alone and start; --pool: spread over the room
   if (opts.mode === "here") {
     const { hereGB, hereWhy, modelNeedGB, modelFallback } = await import("./hostui.js");
-    const lib = { MODELS: rn.MODELS, NEED_GB: rn.NEED_GB, roomBytes: rn.roomBytes, roomFit: rn.roomFit, shortNote: rn.shortNote, shortBy: rn.shortBy, gbUp: rn.gbUp, pledgeGB: rn.pledgeGB, nodeCtxFor: rn.nodeCtxFor, pickCtx: rn.pickCtx, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor };
+    const lib = { MODELS: rn.MODELS, NEED_GB: rn.NEED_GB, roomBytes: rn.roomBytes, roomFit: rn.roomFit, shortNote: rn.shortNote, shortBy: rn.shortBy, gbUp: rn.gbUp, pledgeGB: rn.pledgeGB, nodeCtxFor: rn.nodeCtxFor, pickCtx: rn.pickCtx, roomCtxPlan: rn.roomCtxPlan, maxSeqFor: rn.maxSeqFor, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor };
     const max = Math.max(rule.gb, memoryRule(mem, { max: true }, { capGB: rn.maxPledgeGB(opts.model) }).gb || rule.gb);
     // a MoE model this GPU can't hold whole: with --ram (a discrete GPU's default), its experts in RAM, lending the rule's GB
     // --disk: the model's files will be here (pulled below), unless --no-pull streams a model that is not
@@ -598,7 +598,7 @@ async function runHost(opts, out, prepared = null) {
   // model, by the room page's math (room/plan.js roomFit); a terminal otherwise waits for Enter
   const auto = !tty || opts.start || opts.devices > 0;
   const lib = { MODELS: rn.MODELS, NEED_GB: rn.NEED_GB, roomBytes: rn.roomBytes, roomFit: rn.roomFit, shortNote: rn.shortNote, shortBy: rn.shortBy,
-    gbUp: rn.gbUp, pledgeGB: rn.pledgeGB, nodeCtxFor: rn.nodeCtxFor, pickCtx: rn.pickCtx, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor };
+    gbUp: rn.gbUp, pledgeGB: rn.pledgeGB, nodeCtxFor: rn.nodeCtxFor, pickCtx: rn.pickCtx, roomCtxPlan: rn.roomCtxPlan, maxSeqFor: rn.maxSeqFor, ctxShortNote: rn.ctxShortNote, offloadFor: rn.offloadFor };
   let lastWhy = "";
   const maybeDeal = () => {
     if (!auto || starting || leaving || node.ai.engine || node.ai.starting) return;