From 651262ee340f540c57377e5d6b3afc9b103bc722 Mon Sep 17 00:00:00 2001 From: mattgodbolt-molty Date: Thu, 30 Jul 2026 04:39:09 -0500 Subject: [PATCH] Record model-choice decisions in CLAUDE.md Haiku 4.5 rejection (d188a44) previously lived only in a commit message, which nearly caused a redundant re-evaluation. The 2026-07 fast-model bake-off findings and the parked fast-draft idea live in issue #31; point at both so neither gets re-litigated without new data. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_012jJRqmqhE11VAc3biKbxUY --- CLAUDE.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 752a180..69caa04 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -64,6 +64,12 @@ Pre-commit hooks may modify files (e.g. ruff format); re-`git add` if a hook rep architecture (Lambda response streaming, SQS poll, etc.). - **Reviewer thinking is on by default** (`--reviewer-thinking adaptive`). It catches factual errors the no-think reviewer misses but adds ~70% to review cost; pass `off` for cheap comparative runs. +- **Cheaper/faster models: evaluated and rejected for the default path.** Haiku 4.5 was the original explainer, + replaced for factual errors on complex optimisations (commit `d188a44`, 2025-12). A 2026-07 multi-model + bake-off (gpt-oss-120b on Cerebras/Groq, gemini-3.5-flash, deepseek-v4-flash, via OpenRouter) found the only + fast option 10x quicker but similarly error-prone; full data, a parked "fast draft tier" idea, and the privacy + implications of any multi-provider routing are in issue #31. Don't propose down-tiering the default model + without rerunning that eval. - **Prompt caching: evaluated 2026-07 and rejected at current traffic.** ~104 fresh Claude calls/day (CloudWatch, 14-day window), only ~35 hours/fortnight above 12 calls/hour, against a 5-minute cache TTL and a prefix fragmented by language/arch/audience/type. Generous math: ~$0.40 saved per fortnight of ~$22 spend,