Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,6 @@ All notable changes to Pooled (called SwarmLLM before September 2026). Format fo

## [Unreleased]

## [2.0.1] - 2026-10-16

### Changed
- **The 27B and the 35B MoE open at a long context by default; `?ctx=` is no longer needed** (#421). Each has a ladder of lengths checked correct on it (27B: 64K, 32K, 16K, 8K; MoE: 128K, 64K, 32K, 16K, 8K), and a room opens at the longest step its memory holds, past the old default (16K / 32K) only at full speed: no expert offloaded to RAM or disk and no more devices holding layers than at the default (a longer window alone costs memory, not decode speed: identical tokens and tok/s at 32K and 128K on the GB10). A room short even for the default now opens at a shorter step (down to 8K) and says so, instead of not starting. The picker's **Context** row (8K/16K/32K/64K on the 27B, 8K/32K/64K/128K on the MoE) shows each length's memory in the room and picks another; Room settings changes it while the model runs. The room page, `pooled host` / `pooled serve`'s room and the OpenClaw plugin pick the same context (`room/ctxpick.js` `roomCtxPlan`); room nodes no longer open the MoE at 128K whatever the room holds. `?ctx=N` / `--ctx N` still ask for an exact length. The 1.7B and Flash are unchanged.

## [2.0.0] - 2026-10-16

**Run a 125B model from your SSD, in a browser tab, and pool devices to make it faster.** Pooled 2.0 runs Qwen3.8 Flash (**experimental**), a 125B mixture-of-experts model with about 6B active per token. It is far bigger than most GPUs' memory, so each device keeps its share of the experts on its SSD (browser storage for a tab, the model files for `pooled join`), caches recently used experts on its GPU (routing repeats: about 17% of a layer's experts serve about 77% of lookups), and reads the rest from disk as tokens need them. Disk is the slow part; every device that joins holds part of the model in its memory, so less comes from disk and it gets faster: a Mac Studio alone in Chrome decodes about 8 tok/s, and with an RTX 5070 PC joined about 10-12 tok/s, depending on the network. Flash rooms run at up to 64K context (128K is in progress). 2.0 also brings the system-RAM and SSD expert tiers in the terminal, `pooled serve`, channel-bound join auth and the default TURN relay. Release notes and the demo video: [docs/releases/2.0.md](docs/releases/2.0.md).
Expand Down Expand Up @@ -80,6 +75,7 @@ All notable changes to Pooled (called SwarmLLM before September 2026). Format fo
- **In the room**: API clients get their own card ("API client · Continue") and are not counted as devices; the host can disconnect one or switch API clients off; the black **Serve API** button in the room's header (the one with the dots) opens the Serve API page: this device's layers and passes beside the command for the room, the base URLs, examples and who is connected; once a client is connected the steps fold under "How to connect" and the page leads with what it serves ("Serving Continue via API · 12 requests answered"). API exchanges show in the chat marked "via API · not part of this chat's memory" and follow the room's visibility (the asking client always gets its whole answer). Protocol: new fields on existing messages, `PROTOCOL` stays 4 ([docs/protocol.md](docs/protocol.md#api-clients)).

### Changed
- **The 27B and the 35B MoE open at a long context by default; `?ctx=` is no longer needed** (#421). Each has a ladder of lengths checked correct on it (27B: 64K, 32K, 16K, 8K; MoE: 128K, 64K, 32K, 16K, 8K), and a room opens at the longest step its memory holds, past the old default (16K / 32K) only at full speed: no expert offloaded to RAM or disk and no more devices holding layers than at the default (a longer window alone costs memory, not decode speed: identical tokens and tok/s at 32K and 128K on the GB10). A room short even for the default now opens at a shorter step (down to 8K) and says so, instead of not starting. The picker's **Context** row (8K/16K/32K/64K on the 27B, 8K/32K/64K/128K on the MoE) shows each length's memory in the room and picks another; Room settings changes it while the model runs. The room page, `pooled host` / `pooled serve`'s room and the OpenClaw plugin pick the same context (`room/ctxpick.js` `roomCtxPlan`); room nodes no longer open the MoE at 128K whatever the room holds. `?ctx=N` / `--ctx N` still ask for an exact length. The 1.7B and Flash are unchanged.
- **MoE prompt processing ~2x** (#314): GB10 Deno at 2,048 tokens 402 -> 861 tok/s; OpenClaw's cold turn on one device 30.9 -> 14.7 s. DP4a on the MoE is now on by default on NVIDIA (`DP4A_DEFAULT.moe = true`; `?dp4a=0` turns it off; never on Apple).
- **MoE decode +11-16% in Chrome** (#315): DeltaNet update in two sweeps, the verify in one compute pass and decode-ahead; GB10 plain 55.1 -> 63.9 tok/s at 1K, 52.0 -> 59.6 at 8K, 45.0 -> 50.1 at 32K, same bits.
- **DP4a prefill is the dense default on NVIDIA** (~1.8x faster 27B prompt processing on the GB10; `?dp4a=0` turns it off; the MoE followed in #314, so `DP4A_DEFAULT` is now `{ dense: true, moe: true }`): it passed a pre-registered accuracy evaluation against llama.cpp on the 27B and Qwen3.5-2B (KL and top-1 agreement over 256-token generations up to 8K-token prompts, 40 scored downstream items: same answers as the f32 path) (#309).
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ abstract: "Pooled runs one open language model across the devices in a room, in
repository-code: "https://github.com/Nehanth/pooled"
url: "https://pooled.run"
license: MIT
version: 2.0.1
version: 2.0.0
date-released: 2026-10-16
10 changes: 7 additions & 3 deletions docs/releases/2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,13 @@ Greedy output matches llama.cpp's.

**Demo video:** [flash-demo.mp4](https://pooled.run/docs/media/flash-demo.mp4) (recorded October 13, 2026, 2:13, subtitles): Qwen3.8 Flash in a Chrome tab on a Mac Studio, then an RTX 5070 PC joins the room and it gets 1.4x faster. It is also on the docs' [Pick a model](https://pooled.run/docs/models#qwen38-flash-experimental) page.

## Long context: up to 64K
## Long context by default

Flash rooms run at up to 64K tokens of context (#396), checked against llama.cpp at 2K, 8K, 16K, 32K and 64K. The picker offers 8K, 32K and 64K, and 64K is the default; a room short of memory opens at the longest length it holds. 128K is in progress.
Rooms now open at a long context without any `?ctx=` in the address, and the picker has a Context row for each big model:

- **Qwen3.8 Flash:** up to 64K tokens (#396), checked against llama.cpp at 2K, 8K, 16K, 32K and 64K. The picker offers 8K, 32K and 64K; 64K is the default. 128K is in progress.
- **Qwen3.6 35B MoE:** up to 128K (was 32K by default), and **Qwen3.8 27B:** up to 64K (was 16K) (#421). A room goes past the old default only when it stays at full speed (no experts moved to RAM or disk, no extra devices holding layers); otherwise it stays at 32K / 16K, with longer one click away.
- A room short of memory opens at the longest length it holds, down to 8K, instead of not starting. `?ctx=` and `--ctx` still ask for an exact length.

## What else is new

Expand All @@ -37,7 +41,7 @@ Flash rooms run at up to 64K tokens of context (#396), checked against llama.cpp
- The speed panel shows live tok/s, each device, the time per hop and the expert cache's hit rate (#353, #358).

### Other models
- Qwen3 1.7B opens at 16K context (8K when the room is short); the 35B MoE goes up to 128K, the 27B up to 64K.
- Qwen3 1.7B opens at 16K context (8K when the room is short).
- The tokenizer uses Qwen's own pre-tokenizer split, checked against llama.cpp.

### Experts in RAM and on SSD (terminal)
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "pooled",
"version": "2.0.1",
"version": "2.0.0",
"description": "Peer-to-peer LLM inference in the browser: each device holds some of the layers, tabs talk over WebRTC, and our own WebGPU kernels do the math.",
"homepage": "https://pooled.run",
"repository": {
Expand Down
Loading