doppler-gpu

Doppler turns inference engineering into a verifiable search problem. Optimizers edit RDRR manifests, execution plans, and kernels; parity and benchmark gates accept or reject each candidate against correctness and speed.

Today engineers use these contracts and gates to tune a JavaScript/WGSL WebGPU runtime for supported RDRR artifacts in browser and Node. Doppler runs text generation, embeddings, and reranking locally, with CLI and OpenAI-compatible server entry points. Bun WebGPU support is experimental.

Try the live demo | npm | docs

npx doppler-gpu

Evidence

Doppler has lower steady-state inference latency than Transformers.js in each of the comparable browser WebGPU results indexed below.

For text, faster means lower decode ms/token; for retrieval it means lower ms/embedding or ms/rerank. Every comparison passes its declared workload correctness gate. Model loading is separate: Transformers.js loads the Vulkan embedding and reranker artifacts faster. The competition scoreboard links every receipt, and the methodology defines the gates.

Why these lanes are faster

Doppler authors the WGSL path and pins it in RDRR. Runtime profiles pin session cadence. The measured wins come from different mechanisms in different phases:

Lever	Affected phase	Measured receipt
Fused Q4_K projection and FFN work removes separate dispatches	Text decode	Qwen 3.5 0.8B Metal
Fixed head_dim=128 attention avoids the generic prefill path	Retrieval attention	embedding, reranking
Batch-four decode amortizes one readback across four tokens	Submit and map waits	Qwen 3.5 2B Vulkan
INT4-PLE and Q4_K layouts lower projection traffic	Text decode	Gemma 4 Vulkan

The Gemma 4 comparison uses its declared product-format output policy. It does not claim exact greedy token parity.

Parity and comparable benchmark gates decide what is retained. The challenger framework records accepted and rejected trials, and each verdict's receipt binds it to a model, profile, workload, and device; the negative-results findings keep the dead ends.

Long-term direction

Humans run this loop today. The WGSL-distillation workload is experimental. Automated kernel generation and autotuning are directions, not shipped product paths. Doppler aims to reduce the manual steps between a new checkpoint or GPU and a verified runtime lane. Automated proposals will face the same parity and benchmark gates.

How it works

+----------+     +------------------+     +--------------+     +-----------------+     +------------------+
| Contract | --> | Candidate        | --> | Parity gate  | --> | Benchmark gate  | --> | Retain or reject |
| RDRR /   |     | model / plan /   |     | correctness  |     | comparable      |     | receipts /       |
| request  |     | kernel edit      |     | vs reference |     | speed vs prior  |     | findings         |
+----------+     +------------------+     +--------------+     +-----------------+     +--------+---------+
                                                                                               |
                                                                            next search <------+

The full resolve, load, bind, dispatch, and readback flow lives in the architecture document. Unsupported paths fail closed. Doppler owns artifact and execution contracts; applications own policy.

New model families need RDRR conversion and may need tokenizer, graph, or kernel support.

Ouroboros/Reploid keeps orchestration above this boundary. Program Bundles preserve program identity for downstream backends.

Quick start

Browser

The live demo runs locally and works offline after its first model download.

CLI

npx doppler-gpu "Summarize WebGPU in one sentence"
npx doppler-gpu --model qwen3-0.8b --prompt "Write a haiku about GPUs"
npx doppler-gpu --list-models

Root API

The dr facade is the primary app API. Advanced APIs use package subpaths.

import { dr } from 'doppler-gpu';

// Stream tokens
const model = await dr.load('qwen3-0.8b');
for await (const token of model.generate('Describe WebGPU briefly')) {
  process.stdout.write(token);
}

// One-shot
const text = await model.generateText('Explain WebGPU in one sentence');

OpenAI-compatible server

npx doppler-serve --model qwen3-0.8b --port 8080

Point an OpenAI client at http://localhost:8080/v1:

import OpenAI from 'openai';
const client = new OpenAI({ baseURL: 'http://localhost:8080/v1', apiKey: 'unused' });
const response = await client.chat.completions.create({
  model: 'qwen3-0.8b',
  messages: [{ role: 'user', content: 'Hello' }],
});

Registry IDs resolve to hosted RDRR artifacts from Clocksmith/rdrr by default. See the Root API guide.

Start here

Reader	Entry points
Application developers	Getting started, Root API, and the OpenAI-compatible server
Model integrators	RDRR format, model support, and Program Bundles
Runtime and kernel engineers	Architecture, kernel optimization, and the challenger framework
Evidence reviewers	Competition scoreboard, benchmark methodology, and release matrix

The docs index owns the complete model, subsystem, API, architecture, and operator inventory.

Environment requirements

WebGPU is required. Use a current Chromium browser; Node installs the webgpu provider as an optional dependency.

License

Apache License 2.0 (Apache-2.0). See LICENSE and NOTICE.