Models, servers, benchmarks, and papers that work with ThinkThen. ThinkThen turns a decision model's three endpoints into ten typed functions for the shell, libraries, and databases. Each model has a guide that shows how to use it with ThinkThen 0.2.
An entry joins this list after the ThinkThen compliance check passes against it, and each entry names the check's result and the date it ran. To add something, open an issue. Pull requests are not accepted.
ThinkThen 0.2.0 ran every check below on 2026-10-13, built from commit 65ce9d5af unless the entry names another.
- Jev: the decision model ThinkThen was built on, from TypeSafe. Check passed, critical 0, warning 0, 2026-10-13; first recorded 2026-09-23.
- Liquid d1: a decision foundation model from an MIT spin-off, at
api.liquid.ai. Check passed, critical 0, warning 0, 2026-10-13. The free model rate-limits hard; its guide says how to run it. - pplx-decider: Perplexity's decision model,
pplx-decider-v1-27b. Check passed, critical 0, warning 0, 2026-10-13. - Jev through OpenRouter: Jev resold by OpenRouter as
typesafe/jev-1.13. Check passed, critical 0, warning 0, 2026-10-13. - OpenAI Decisions API: OpenAI's decision API, in public beta, with the model
gpt-6-luna. ThinkThen speaks its own shape. Check passed, critical 0, warning 0, 2026-10-13, under commitccb2b579f.
ThinkThen 0.2.0 ran every check below on 2026-10-13, built from commit 65ce9d5af, on a MacBook Pro with an Apple M5 Max chip and 128 GB of memory. Each guide holds the exact install, start and check commands.
- d3-flash: the 9B model of Decision 3.0, the open decision models of vLLM Semantic Router, served by llama.cpp. Check passed, critical 0, warning 0, 2026-10-13.
- Clef-Flash: Cloudflare's 9B decision model with Apache 2.0 weights, served by llama.cpp. Check passed, critical 0, warning 0, 2026-10-13.
- nimble: Bespoke Labs' 9B decision model, served by Ollama. Check passed, critical 0, warning 3, 2026-10-13; the warnings mark ThinkThen's text form for Ollama's descriptions.
- Kev-4B: the 4B size of Kev, Qwen-based decision models from 0.8B to 27B with their own server; llama.cpp serves it too. Check passed on both servers, critical 0, warning 0, 2026-10-13; first recorded 2026-09-29.
- tev1: Together AI's experimental decision models in 4B and 0.8B, served by Ollama. Check passed for both sizes, critical 0, warning 3, 2026-10-13.
- Strands Decider 2B: Strands Labs' decision server under MLX, the target of ThinkThen's
mlxbuilt-in. Check passed, critical 0, warning 3, 2026-10-13; the warnings mark the built-in's text form for descriptions. - Laya: ModernBERT-large under MLX, about 421M parameters (weights). Served on System One by stuntdouble's sidecar,
sidecars/laya.py. Check passed through that wrapper, critical 0, warning 0, 2026-10-13; first recorded 2026-09-30.
typesafe/jev-routeron OpenRouter: a chat-completions router, not a decision model. OpenRouter's System One endpoint answers that it does not exist, and the check of 2026-10-13 failed with critical 14. Its notes sit in the OpenRouter guide.
- Beatles Bench: knowledge judgments over Beatles songs, run through ThinkThen. The leaderboard below scores every backend we test on it. The bench's main score measures what a model remembers about Beatles songs, a trivia test, not how well it decides questions about text you hand it, so treat it as one data point, not a ranking of decision quality.
- JevBench, by Benchmark Heaven: the typed-decision board. 106 systems as of 2026-09-30, scored on a composite of intelligence, calibration, speed, and cost. Harness.
- JevBench.dev: a different project that shares the name and says so itself; it ranks models by verified game wins in StarCraft II and Minecraft. The two boards are unaffiliated, and neither's scores convert to the other's.
A model earns an entry here by being tested, not by being ranked, so the open-model half of the Benchmark Heaven board is not copied into this list.
The bench's main score measures what a model remembers about Beatles songs, a trivia test, not how well it decides questions about text you hand it, so treat it as one data point, not a ranking of decision quality.
The ten best decision models we have run on Beatles Bench, best first. The large language model sits below the line as the yardstick, with reasoning off. Every run, including Strands Decider 2B, tev1 0.8B and Laya, is in runs.md, and every tested model has a guide in models/.
| # | System | Where | Full dataset, 1,501 | Hard set, 505 | Report |
|---|---|---|---|---|---|
| 1 | OpenAI Decisions API | hosted | 88.2% | 83.2% | 2026-10-13 |
| 2 | Jev through OpenRouter | hosted | 70.6% | 56.2% | 2026-10-13 |
| 3 | Jev | hosted | 69.8% | 55.7% | 2026-10-13 |
| 4 | Liquid d1 | hosted | 69.3% | 56.8% | 2026-10-13 |
| 5 | pplx-decider | hosted | 65.4% | 54.1% | 2026-10-13 |
| 6 | d3-flash | local | 60.6% | 49.1% | 2026-10-13 |
| 7 | Clef-Flash | local | 53.4% | 43.4% | 2026-10-13 |
| 8 | nimble | local | 52.6% | 43.5% | 2026-10-13 |
| 9 | Kev-4B | local | 47.9% | 39.8% | 2026-10-13 |
| 10 | tev1 4B | local | 43.6% | 33.7% | 2026-10-13 |
| — | GLM-5.3 Flash, reasoning off | hosted | 96.7% | 95.4% | 2026-09-23 |
The hard set is the 505 questions from the seven categories where Jev scores worst. Each row is its newest run, under ThinkThen 0.2. Local rows ran on the M5 Max with ThinkThen on the same machine. The OpenAI Decisions API refused 9 questions, scored as wrong. The three rows after it differ by less than their 95% intervals, about 2.3 points either way on the full dataset, so the order among them is not settled.
The catalog is papers.md, and the papers themselves sit in papers/ as Markdown, fetched from arXiv with markxiv.
The decision APIs this list tracks live in apis/, one file each:
- System One: TypeSafe's decision API, the shape Jev, Liquid d1, Kev, and Ollama's decision models speak. Supported today, with the published schema and the dialects each backend speaks.
- Decisions API: OpenAI's decision API, in public beta. ThinkThen 0.2 speaks it through the
openaibackend, and its check passed on 2026-10-13.