Problem
llm_classifier classifies a conversation once and, with session_affinity, holds that decision forever. Both halves are wrong for long agentic sessions:
- Classify every turn and a tool chain switches tier mid-flight — the weak tier picks up a conversation the strong tier was three tool calls into, with no idea what it was doing.
- Classify once (
session_affinity) and the opening verdict goes stale. A session that opened "summarise this file" becomes a hard debugging session ten turns later and is still served by the tier chosen for the summary.
The unit that should drive a routing decision is not the request and not the session — it is the user turn. Tool results are the agent continuing work the user already asked for.
Proposal
A followup table that re-classifies when the user asks something new and pins that target across the tool-call turns in between:
[routes.assistant]
type = "llm_classifier"
followup = { refresh = "on_user_turn", recent_turn_window = 4 }
| Key |
Default |
Meaning |
refresh |
on_user_turn |
on_user_turn re-classifies on each new user message; pin_once holds the first decision |
recent_turn_window |
unset |
Trailing turns the judge sees. At least 1 |
fail_open_tier |
capable |
Tier served when the judge yields nothing usable and nothing is pinned. Mode capability only |
abstain_selector |
unset |
JSON Pointer to a boolean abstain flag in the verdict. Mode custom only |
Available on modes capability and custom. Rejected alongside session_affinity, whose latch is first-wins and would prevent every refresh — silently turning on_user_turn into pin_once; use refresh = "pin_once" if that is what you want.
The deliberate asymmetry, which is the part worth reviewing:
- an unusable verdict, or an unreachable judge, holds the pinned target. No evidence is not evidence of change. It falls open only when nothing is pinned.
- in mode
custom, an explicit abstain outranks the pin — the judge declining to decide is itself a decision. Mode capability has no equivalent, because its packaged rubric answers with a solve probability and carries no abstain field.
- neither path retries the judge.
Relationship to #298 and #308
These overlap and the difference is intentional. #298's RoutingIdentity and #308's Harbor proxy_x_session_id normalisation both improve who a conversation is; this is about when its tier is re-decided. The pin lives in per-session State rather than the affinity map precisely because AffinityRouter's assignment is first-wins and process-local, so it can never be refreshed — which is the behaviour this issue exists to change.
Where a request carries no session id the pin has nothing to persist in, so it can optionally be keyed on a hash of the first user message, the same fallback session_affinity already offers.
Rebased onto current main and green. Depends on nothing else; sibling issue for the graded-verdict labels map filed separately.
Problem
llm_classifierclassifies a conversation once and, withsession_affinity, holds that decision forever. Both halves are wrong for long agentic sessions:session_affinity) and the opening verdict goes stale. A session that opened "summarise this file" becomes a hard debugging session ten turns later and is still served by the tier chosen for the summary.The unit that should drive a routing decision is not the request and not the session — it is the user turn. Tool results are the agent continuing work the user already asked for.
Proposal
A
followuptable that re-classifies when the user asks something new and pins that target across the tool-call turns in between:refreshon_user_turnon_user_turnre-classifies on each new user message;pin_onceholds the first decisionrecent_turn_window1fail_open_tiercapablecapabilityonlyabstain_selectorcustomonlyAvailable on modes
capabilityandcustom. Rejected alongsidesession_affinity, whose latch is first-wins and would prevent every refresh — silently turningon_user_turnintopin_once; userefresh = "pin_once"if that is what you want.The deliberate asymmetry, which is the part worth reviewing:
custom, an explicit abstain outranks the pin — the judge declining to decide is itself a decision. Modecapabilityhas no equivalent, because its packaged rubric answers with a solve probability and carries no abstain field.Relationship to #298 and #308
These overlap and the difference is intentional. #298's
RoutingIdentityand #308's Harborproxy_x_session_idnormalisation both improve who a conversation is; this is about when its tier is re-decided. The pin lives in per-sessionStaterather than the affinity map precisely becauseAffinityRouter's assignment is first-wins and process-local, so it can never be refreshed — which is the behaviour this issue exists to change.Where a request carries no session id the pin has nothing to persist in, so it can optionally be keyed on a hash of the first user message, the same fallback
session_affinityalready offers.Rebased onto current
mainand green. Depends on nothing else; sibling issue for the graded-verdictlabelsmap filed separately.