The four max_tokens budgets in packages/outpost/ai/src/config.ts are sized for models that answer without thinking:
| Setting |
Value |
Call |
maxResponseTokens |
2048 |
generator.generate / generateStream |
maxClassifierTokens |
512 |
classifier.classify |
maxSentimentTokens |
512 |
analyzeSentiment |
maxConfidenceTokens |
256 |
confidence.score |
Every one of those models is env-overridable (AI_RESPONSE_MODEL, AI_CLASSIFIER_MODEL, AI_SENTIMENT_MODEL, AI_CONFIDENCE_MODEL), and on Claude Opus 5 and Fable 5 thinking is on by default. A thinking turn consumes the budget before the answer starts, so 256 and 512 are below what one costs.
Why this is the gating item rather than a tuning nit
Two changes have now removed the loud failures on that swap and left this one:
- #240 stopped
temperature being sent to models that reject it, so the swap no longer 400s on every call and publishes the apology fallback.
- The same PR made an empty extraction a failure at all three secondary call sites, so a thinking-only response degrades visibly instead of returning a fabricated value with
degraded: false.
So after #240 the swap is possible and observable. These budgets are what make it safe: with them unchanged, a thinking-default model spends the whole allowance thinking, returns no text, and every classify / sentiment / confidence call degrades to its heuristic on every request. Not broken, not silent — just uniformly useless, which is a worse outcome to debug than a hard failure.
The generator's 2048 is the least alarming of the four and still wants checking, since a truncated answer at stop_reason: max_tokens publishes mid-sentence today — see #231.
What settling it involves
Not just raising the numbers. It needs a decision per call site:
- Should the cheap calls think at all? For classify, sentiment and confidence the answer is probably no — they emit small JSON.
thinking: { type: 'disabled' } with output_config.effort: 'low' is valid on Opus 5 and cheaper than a raised ceiling, but it is a 400 on Fable 5, where thinking is always on. So "disable thinking" is not portable and the budget has to cover a thinking turn wherever it is not.
- What does the generator's ceiling become, given a thinking turn plus a full support answer, and what does that do to cost per ticket.
- Where the numbers live. They are constants today; if they have to vary by model, that is a per-model table rather than four integers.
Raising them blindly costs money on every ticket for headroom most requests do not use, which is why this wants an owner and a decision rather than a quick bump.
Origin
Flagged from two directions during review of #240: from the empty-response side (a thinking-only reply is what the new guard catches) and from the budget side (why that reply happens). Filed rather than mentioned a third time in a PR body.
The four
max_tokensbudgets inpackages/outpost/ai/src/config.tsare sized for models that answer without thinking:maxResponseTokensgenerator.generate/generateStreammaxClassifierTokensclassifier.classifymaxSentimentTokensanalyzeSentimentmaxConfidenceTokensconfidence.scoreEvery one of those models is env-overridable (
AI_RESPONSE_MODEL,AI_CLASSIFIER_MODEL,AI_SENTIMENT_MODEL,AI_CONFIDENCE_MODEL), and on Claude Opus 5 and Fable 5 thinking is on by default. A thinking turn consumes the budget before the answer starts, so 256 and 512 are below what one costs.Why this is the gating item rather than a tuning nit
Two changes have now removed the loud failures on that swap and left this one:
temperaturebeing sent to models that reject it, so the swap no longer 400s on every call and publishes the apology fallback.degraded: false.So after #240 the swap is possible and observable. These budgets are what make it safe: with them unchanged, a thinking-default model spends the whole allowance thinking, returns no text, and every classify / sentiment / confidence call degrades to its heuristic on every request. Not broken, not silent — just uniformly useless, which is a worse outcome to debug than a hard failure.
The generator's 2048 is the least alarming of the four and still wants checking, since a truncated answer at
stop_reason: max_tokenspublishes mid-sentence today — see #231.What settling it involves
Not just raising the numbers. It needs a decision per call site:
thinking: { type: 'disabled' }withoutput_config.effort: 'low'is valid on Opus 5 and cheaper than a raised ceiling, but it is a 400 on Fable 5, where thinking is always on. So "disable thinking" is not portable and the budget has to cover a thinking turn wherever it is not.Raising them blindly costs money on every ticket for headroom most requests do not use, which is why this wants an owner and a decision rather than a quick bump.
Origin
Flagged from two directions during review of #240: from the empty-response side (a thinking-only reply is what the new guard catches) and from the budget side (why that reply happens). Filed rather than mentioned a third time in a PR body.