Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ All notable changes to Pooled (called SwarmLLM before September 2026). Format fo

**Run a 125B model from your SSD, in a browser tab, and pool devices to make it faster.** Pooled 2.0 runs Qwen3.8 Flash (**experimental**), a 125B mixture-of-experts model with about 6B active per token. It is far bigger than most GPUs' memory, so each device keeps its share of the experts on its SSD (browser storage for a tab, the model files for `pooled join`), caches recently used experts on its GPU (routing repeats: about 17% of a layer's experts serve about 77% of lookups), and reads the rest from disk as tokens need them. Disk is the slow part; every device that joins holds part of the model in its memory, so less comes from disk and it gets faster: a Mac Studio alone in Chrome decodes about 8 tok/s, and with an RTX 5070 PC joined about 10-12 tok/s, depending on the network. Flash rooms run at up to 64K context (128K is in progress). 2.0 also brings the system-RAM and SSD expert tiers in the terminal, `pooled serve`, channel-bound join auth and the default TURN relay. Release notes and the demo video: [docs/releases/2.0.md](docs/releases/2.0.md).

Not in 2.0: the CLI on npm (`@pooled/cli` 0.4.1) doesn't have Flash's newest code yet; to lend or host Flash from the terminal, run the CLI from the repo ([cli/README.md](cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo)). Flash MTP speculative decoding (#352) is not part of 2.0.
`@pooled/cli` 2.0.0 on npm lends to and hosts Flash rooms ([cli/README.md](cli/README.md#qwen38-flash-experimental-from-the-terminal)). Flash MTP speculative decoding (#352) is not part of 2.0.

### Upgrade notes
- **`@pooled/openclaw` 0.3.2 bundles the room node from before these changes**: an OpenClaw gateway can't hold Qwen3.8-Flash-Next layers, use `--disk` or the TURN relay until a new plugin release.
- **To lend a GPU to Qwen3.8 Flash from the terminal, run the CLI from the repo for now.** `@pooled/cli` 0.4.1 (on npm) and older don't know the model: a Flash start fails with "couldn't load its layers (unknown model qwen3.8-flash)", and a 0.4.1 node refuses a Flash offload (the host then deals again without it, #376). `pooled serve` / `pooled chat` 0.4.x keep working unchanged, and the room protocol is still 4.
- **To lend a GPU to Qwen3.8 Flash from the terminal, use `@pooled/cli` 2.0.0.** `@pooled/cli` 0.4.1 and older don't know the model: a Flash start fails with "couldn't load its layers (unknown model qwen3.8-flash)", and a 0.4.1 node refuses a Flash offload (the host then deals again without it, #376). `pooled serve` / `pooled chat` 0.4.x keep working unchanged, and the room protocol is still 4.
- **Flash room checkpoints have a new signature (v2, #379).** Copies saved on disk by an older build count as missing: the first question on those prefixes reads the prompt again, and the old copies are evicted under the cap.
- A v1.0.0 browser tab joining a host with "Ask before new devices join" on (the default) is turned away with "this tab runs an older Pooled"; a reload fixes it.
- `pooled serve` can't use a v1.0.0 host (v1.0.0 has no API) and says so.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ It is **experimental**:

Demo video: [Qwen3.8 Flash in a Chrome tab on a Mac Studio, then an RTX 5070 PC joins](https://pooled.run/docs/media/flash-demo.mp4) (recorded October 13, 2026, 2:13, subtitles; the room gets 1.4x faster when the PC joins).

Pick it in the room's model picker. To lend a computer from the command line, run the CLI from this repo for now, not from npm: [cli/README.md](cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo).
Pick it in the room's model picker. To lend a computer from the command line: `npx @pooled/cli join <ROOM CODE>` ([cli/README.md](cli/README.md#qwen38-flash-experimental-from-the-terminal)).

Details: [docs/architecture.md](docs/architecture.md), [docs/kernels.md](docs/kernels.md), [docs/protocol.md](docs/protocol.md), [docs/models.md](docs/models.md).

Expand Down
16 changes: 6 additions & 10 deletions cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -418,21 +418,17 @@ room HJQ-N44 · online · 2 devices · layers 14-27 of qwen3-1.7b · 56.4 tok/s

Not included yet: running as a background service (it is a foreground command), and phones (they join from the room page).

### Qwen3.8 Flash (experimental): run the CLI from the repo
### Qwen3.8 Flash (experimental) from the terminal

Qwen3.8 Flash is **experimental**, and the `@pooled/cli` package on npm doesn't have its newest code yet. To lend a computer to a Flash room, or host one, run the CLI from a clone of the repo (Node 22 or newer):
Qwen3.8 Flash is **experimental**. `@pooled/cli` 2.0.0 and newer lend to a Flash room and host one (Node 22 or newer):

```bash
git clone https://github.com/Nehanth/pooled.git
cd pooled/cli && npm install
(cd ../packages/room-node && npm install) # the room node and Dawn (WebGPU), for join and host

node bin/pooled.js join <ROOM CODE> # lend this computer to a Flash room
node bin/pooled.js pull qwen3.8-flash # optional: download its 93.7 GB first
POOLED_DEV=1 node bin/pooled.js host qwen3.8-flash # open a Flash room from this computer
npx @pooled/cli join <ROOM CODE> # lend this computer to a Flash room
npx @pooled/cli pull qwen3.8-flash # optional: download its 93.7 GB first
POOLED_DEV=1 npx @pooled/cli host qwen3.8-flash # open a Flash room from this computer
```

Every option above works the same way (`--gb`, `--ram`, `--disk`, `--here`). `git pull` in the clone picks up fixes. Once a release of `@pooled/cli` carries Flash, `npx @pooled/cli join` does the same.
Every option above works the same way (`--gb`, `--ram`, `--disk`, `--here`). An older CLI (0.4.x) can join a room running another model, but can't load Flash: update with `npx @pooled/cli@latest`.

## Development

Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Qwen3.8 Flash is a 125B mixture-of-experts model that uses about 6B of its param

Reading from disk is the slow part. Every device that joins holds part of the model in its own memory, so less comes from disk and the room gets faster. A Mac Studio running it alone in a Chrome tab decodes about 8 tokens/s; with an RTX 5070 PC joined, about 10-12 tokens/s, depending on the network between them.

It is **experimental**. Context goes up to 64K tokens (the picker offers 8K, 32K and 64K), tested against llama.cpp; 128K is in progress. To lend or host it from the terminal, run the CLI from the repo for now: the CLI on npm doesn't have Flash's newest code yet (see the [CLI README](https://github.com/Nehanth/pooled/blob/main/cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo)).
It is **experimental**. Context goes up to 64K tokens (the picker offers 8K, 32K and 64K), tested against llama.cpp; 128K is in progress. To lend or host it from the terminal, use `@pooled/cli` 2.0.0 or newer: `npx @pooled/cli join <ROOM CODE>` (see the [CLI README](https://github.com/Nehanth/pooled/blob/main/cli/README.md#qwen38-flash-experimental-from-the-terminal)).

:::tip
Only give memory the device really has free. On a Mac, the GPU shares memory with everything else that is open.
Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/terminal.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ qwen3.8-27b Qwen3.8 27B · Q4
qwen3.6-35b-moe Qwen3.6 35B MoE · Q4
```

With `POOLED_DEV=1`, the list also has `qwen3.8-flash` (Qwen3.8-Flash-Next · Q4, a 100.6 GB download in three files: 71.8 GB of layers and a 28.8 GB n-gram table). It is experimental (the CLI calls it a preview): rooms run it at up to 64K tokens of context. The CLI on npm doesn't have Flash's newest code yet, so run the CLI from the repo to lend or host it (see the [CLI README](https://github.com/Nehanth/pooled/blob/main/cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo)). `pooled join` needs no flag to lend to a room that runs it, and `pooled pull qwen3.8-flash` / `pooled rm qwen3.8-flash` take its full name without `POOLED_DEV=1`.
With `POOLED_DEV=1`, the list also has `qwen3.8-flash` (Qwen3.8-Flash-Next · Q4, a 100.6 GB download in three files: 71.8 GB of layers and a 28.8 GB n-gram table). It is experimental (the CLI calls it a preview): rooms run it at up to 64K tokens of context. Lending to or hosting it needs `@pooled/cli` 2.0.0 or newer (see the [CLI README](https://github.com/Nehanth/pooled/blob/main/cli/README.md#qwen38-flash-experimental-from-the-terminal)). `pooled join` needs no flag to lend to a room that runs it, and `pooled pull qwen3.8-flash` / `pooled rm qwen3.8-flash` take its full name without `POOLED_DEV=1`.

### pooled host --help

Expand Down
2 changes: 1 addition & 1 deletion docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Pooled ships no weights. Browsers fetch tensors by HTTP range request from publi
| Qwen 3.8 27B | GGUF Q4_0 (~15 GB), includes the `nextn` draft layer | `Qwen35Engine` | 64 layers: 48 Gated DeltaNet + 16 attention; MTP speculation; 16K context by default, up to 64K |
| Qwen 3.6 35B MoE (Qwen3.6-35B-A3B) | GGUF Q4_0 (bartowski) | `Qwen35Engine` (`qwen35moe`) | 256 experts, 8 active per token (~3B of 35B); router, experts and shared expert on the GPU; 32K context by default, up to 128K |
| Qwen3.5-122B-A10B MoE | GGUF Q4_0 (bartowski, two files, 72.5 GB) | `Qwen35Engine` (`qwen35moe`) | **Developer only, hidden from users**: in the room page's `?dev=1` select and, with `POOLED_DEV=1`, in `pooled list` / `host` / `pull` (pull / rm take its full key without it); `tests/test_moe.js MODEL=122b`. 256 experts, 8 active (~10B of 122B); 32K context by default, up to 128K |
| Qwen3.8 Flash Next (**experimental**) | GGUF Q4_0 (bartowski, three files, ~100.6 GB, of which a 28.8 GB n-gram table) | `Qwen4ExpEngine` (`engine/qwen4exp.js`, rooms via `room/flash.js`) | 125B, 512 experts, 10 active per token (~6B); most experts stay on disk and stream into a GPU cache (disk tier); context up to 64K, tested against llama.cpp; 128K in progress; from the command line, run the CLI from the repo ([cli/README.md](../cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo)); in the room picker |
| Qwen3.8 Flash Next (**experimental**) | GGUF Q4_0 (bartowski, three files, ~100.6 GB, of which a 28.8 GB n-gram table) | `Qwen4ExpEngine` (`engine/qwen4exp.js`, rooms via `room/flash.js`) | 125B, 512 experts, 10 active per token (~6B); most experts stay on disk and stream into a GPU cache (disk tier); context up to 64K, tested against llama.cpp; 128K in progress; from the command line, `@pooled/cli` 2.0.0+ ([cli/README.md](../cli/README.md#qwen38-flash-experimental-from-the-terminal)); in the room picker |
| Qwen3 4B / 1.7B / 0.6B | GGUF Q4_0 / Q8_0 | `DenseEngine` | dense; 0.6B is the golden-test model |
| SmolLM2 135M | safetensors f32 | `DenseEngine` | smallest demo; quantized to Q8 at load |

Expand Down
2 changes: 1 addition & 1 deletion docs/releases/2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Full list: [CHANGELOG.md](../../CHANGELOG.md).
## Compatibility and upgrading

- **The room protocol is unchanged (4).** 1.x and 2.0 tabs can share a room, with two limits: a 2.0 creator refuses a Start pressed on a v1.0.0 tab (reload it to press Start), and a v1.0.0 tab turned away with "this tab runs an older Pooled" only needs a reload.
- **To lend a computer to Flash from the terminal, run the CLI from the repo for now.** The CLI on npm (`@pooled/cli` 0.4.1) doesn't have Flash's newest code yet: [cli/README.md](../../cli/README.md#qwen38-flash-experimental-run-the-cli-from-the-repo). An older `pooled join` can still join a 2.0 room and run the other models; a Flash start on it fails with "couldn't load its layers (unknown model qwen3.8-flash)", and a 0.4.1 node dealt a Flash offload refuses it (the host deals again without it). `pooled serve` / `pooled chat` 0.4.x keep working.
- **To lend a computer to Flash from the terminal, use `@pooled/cli` 2.0.0** (`npx @pooled/cli join <ROOM CODE>`): [cli/README.md](../../cli/README.md#qwen38-flash-experimental-from-the-terminal). An older `pooled join` can still join a 2.0 room and run the other models; a Flash start on it fails with "couldn't load its layers (unknown model qwen3.8-flash)", and a 0.4.1 node dealt a Flash offload refuses it (the host deals again without it). `pooled serve` / `pooled chat` 0.4.x keep working.
- `pooled serve` can't use a v1.0.0 host (it has no API) and says so.
- Flash checkpoints saved on disk by an older build count as missing (#379): the first question on them reads the prompt again.
- Behaviour changes: the layer split defaults to "For speed"; MoE DP4a prefill is on by default on NVIDIA (`?dp4a=0`); `--ram` is on by default when a room is short; iPhone pledges are capped at 1 GB; scripts that join a gated room need the invite link or `--allow-all`.
Expand Down
Loading