feat(routing): add validated routing policy profiles - #1011
Merged
Conversation
Contributor
|
Warning Review limit reached
Next review available in: 14 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughChangesRouting policy profiles
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant RoutePolicyCLI
participant ManagementAPI
participant RoutingProfileRoutes
participant PolicyEvaluator
RoutePolicyCLI->>ManagementAPI: Submit policy dry-run
ManagementAPI->>RoutingProfileRoutes: Validate profile and evidence
RoutingProfileRoutes->>PolicyEvaluator: Evaluate candidates
PolicyEvaluator-->>RoutingProfileRoutes: Return selection and decision trace
RoutingProfileRoutes-->>RoutePolicyCLI: Return dry-run result
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This was referenced Aug 4, 2026
Wibias
force-pushed
the
feat/ri-04-policy-profile-core
branch
from
August 5, 2026 00:04
00e1c4a to
6392449
Compare
Wibias
marked this pull request as ready for review
August 5, 2026 00:11
… commands (RI-04)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 63924495e7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 25
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devlog/_plan/260804_router_intelligence/001_pr_stack_status.md`:
- Around line 13-14: Synchronize the RI-03 and RI-04 entries across the stack
table and acceptance log: update every related SHA, PR number, and status field
so merged/open/pending states and RI-04’s base SHA agree with the recorded stack
history, including the sections around the acceptance entries and lines 148–160.
If historical values must remain, label them explicitly as historical rather
than presenting them as current.
In `@docs-site/src/content/docs/reference/configuration/routing.md`:
- Around line 88-100: Update the routing policy profiles documentation to state
that policy IDs and aliases are virtual model identifiers and do not rewrite
explicit provider-qualified model IDs, combo resolution, account namespaces,
default-provider resolution, credential routing, account pools, cooldowns, or
session affinity. Clarify that every candidate is an explicitly configured {
provider, model } pair.
- Around line 100-105: Expand the routing configuration documentation around the
candidates definition to state that provider and model are separate fields, with
model preserving the complete configured native ID including any “/” characters.
Document that candidate evaluation uses the selected provider’s adapter,
authentication mode, model metadata, and reasoning-capability constraints, and
clarify that dry-run only evaluates and exposes the decision without bypassing
adapter validation or transforming upstream requests.
- Line 14: Update the model-resolution order in the routing documentation to
include the policy-profile resolver at its actual precedence position. Document
canonical-ID-first resolution and note that both policy/fast and its ocx/fast
alias are valid identifiers, covering the corresponding resolution-order section
and repeated configuration example.
- Around line 88-91: Update the routing policy documentation around the profile
activation and route-decision trace descriptions to distinguish current RI-04
behavior from future RI-05–RI-08 work. State that production requests do not
currently route through profiles, profile decisions are exposed through dry-run
APIs only, and requested policy IDs do not yet activate production routing.
- Line 86: Update the ja, ko, ru, and zh-cn routing documentation pages to match
the English routing.md coverage, adding equivalent translated sections for
config.routingProfiles, policy/<id>, candidate allowlists, and ocx route policy
dry-run while preserving each locale’s existing formatting and terminology.
In `@src/cli/route-policy.ts`:
- Around line 81-89: Update the destructuring in handleRoutePolicyCommand to
default sub to "list" when argv is empty, matching handleComboCommand behavior.
Preserve the existing list, show, dry-run, and unknown-subcommand dispatch
logic.
- Around line 51-79: Update dryRun to accept a candidate-evidence input option,
parse it, and forward the resulting candidates array in the runtimeRequest
payload so requirements can be evaluated against capabilities. Add the option to
USAGE and validate malformed input consistently with other CLI arguments.
Replace the non-JSON raw result dump with a concise human-readable
eligibility/selection summary while preserving the complete result for --json.
Ensure the candidate evidence can represent false-valued capabilities needed by
negative requirements.
In `@src/routing/evaluator.ts`:
- Line 223: Move requirements from the trace-level construction around
candidates.flatMap into per-candidate RouteCandidateTrace and
TraceCandidateInput data, preserving candidate ownership. Apply the existing
per-candidate cap pattern used by MAX_EXCLUSIONS_PER_CANDIDATE, including a
budget.truncation flag when requirements are truncated, and remove the
cross-candidate slice(0, 16) behavior.
- Around line 67-82: Document the boolean requirement semantics in
booleanRequirement and OcxRoutingProfileRequirements: required false is a
negative assertion that rejects candidates reporting true, while omitting the
key means “don’t care.” Update the routing configuration page in docs-site to
state the same behavior consistently.
- Around line 219-221: Update the requestedModel field in
buildRouteDecisionTrace to use the shared policyModelId helper with profile.id
instead of the inline template and caller-supplied profileId. Preserve
consistency with profile.revision and the returned profileId; if requestedModel
represents the client-facing request, use policyPublicModelId so aliases are
retained.
- Around line 159-217: Update evaluatePolicyProfile to incorporate
requestEvidence when deriving each candidate’s effective requirements: raise
minContextWindow to at least requestEvidence.contextWindow and enforce the
request’s toolsRequired, imageRequired, and structuredOutputRequired flags
alongside profile requirements. Use these effective requirements for
unsatisfied/unknown checks and add a focused regression test varying only
requestEvidence to verify eligibility changes, without including request-derived
values in profileRevision.
In `@src/routing/profile.ts`:
- Around line 401-403: Update listRoutingProfileIds to replace localeCompare
with an explicit deterministic code-unit comparison, ordering IDs by their
character code values without locale-dependent collation. Preserve the existing
key extraction and sorted string-array behavior.
- Around line 363-381: Update normalizeRoutingProfile to fall back to
DEFAULT_PROFILE_WEIGHTS whenever the merged optimize weight sum is not positive,
then normalize those fallback weights so the result always sums to 1. Also
update routingProfileIssues to reject an optimize block whose weights are all
zero, while preserving acceptance of valid non-negative weight configurations.
In `@src/routing/trace.ts`:
- Around line 458-462: Update the serviceTier handling in the trace
normalization logic so it sets caps.strings = true whenever a non-unknown
serviceTier exceeds MAX_TRACE_STRING and is truncated. Preserve the existing
"unknown" behavior and normal assignment for values within the limit.
- Line 219: Update buildCandidate so input.score is normalized through the same
bounding and validation logic used by parseCandidate via parseScore. Extract the
shared score-normalization logic into a reusable helper if needed, then use that
helper in both paths to preserve bounded components and reject or safely handle
non-finite totals.
- Line 49: Update the serviceTier type and related routing logic to represent
missing evidence by omitting serviceTier rather than using the string "unknown".
In parseCapability, preserve any non-empty raw tier string—including
"unknown"—and leave serviceTier unset when evidence is missing; update
requirementFor and profile validation as needed so legitimate tiers or
requirements named "unknown" remain satisfiable.
In `@src/server/management/routing-profile-routes.ts`:
- Around line 31-44: Update parseEvidence to treat an absent or undefined raw
value as valid empty evidence, returning an empty PolicyRequestEvidence with ok:
true. Preserve ok: false for non-object malformed evidence so the route
continues rejecting values such as strings, while keeping the existing field
validation unchanged.
- Around line 58-63: Correct the comment above the capability, health, quota,
and cost mappings to state that these caller-supplied plain objects are echoed
without trace-normalizer validation; do not claim they are re-bounded. Keep the
existing isPlainRecord checks and structural casts unchanged.
- Around line 72-77: Update both success responses in the routing-profile
handlers: in src/server/management/routing-profile-routes.ts lines 72-77, pass
req and config to jsonResponse for the profiles response; in lines 106-107, pass
req and config to jsonResponse for result. Preserve the existing status code and
response payloads.
In `@tests/routing-profile.test.ts`:
- Around line 120-124: Update the collision test around routingProfileIssues so
it uses the bare alias "free" with the existing baseConfig combo, and assert the
resulting issue message contains "combo selector" to cover the resolveComboId
collision branch. Keep a separate test for the "combo/free" alias if the
reserved-prefix branch also needs coverage, asserting "reserved" there.
- Around line 246-248: Extend the list response type and assertion in the
routing-profile list test around listBody and the profiles[0] expectation to
include the model field. Assert that model equals the configured public alias,
ocx/fast, so the API contract verifies alias exposure rather than only id and
revision.
- Around line 93-110: Add a regression case to the existing “weights default and
normalize deterministically” test covering a routing profile whose optimize
block sets latency, health, cost, and quota to zero. Assert that the normalized
optimize weights satisfy the same sum-to-one invariant, alongside the existing
weight assertions, using getRoutingProfile and baseConfig.
- Around line 225-238: Rename the existing test to describe that it verifies the
highest-scoring candidate wins, since configuredPriority produces different
scores. Do not label it as a tie-break test; adding a genuine equal-score case
is optional and should only use an existing repeatable score component or
directly exercise the selection comparison in evaluatePolicyProfile.
- Around line 201-207: Preserve the literal value types of unknownEvidence in
the dry-run evaluator test by applying a const assertion or the appropriate
OcxRoutingProfileUnknownEvidence annotation. Update the TypeScript configuration
or typecheck setup so tests/routing-profile.test.ts is included in typechecking,
while retaining the existing test behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 41f68c1e-efc1-4cc1-a7c5-808a6e52260c
📒 Files selected for processing (12)
devlog/_plan/260804_router_intelligence/001_pr_stack_status.mddocs-site/src/content/docs/reference/configuration/routing.mdsrc/cli/index.tssrc/cli/route-policy.tssrc/config.tssrc/routing/evaluator.tssrc/routing/profile.tssrc/routing/trace.tssrc/server/management-api.tssrc/server/management/routing-profile-routes.tssrc/types.tstests/routing-profile.test.ts
Collaborator
Author
|
[GD] Addressed feedback feedbacks:
commit: dd10c9a |
…uirements (RI-04)
Collaborator
Author
|
[GD] Addressed feedback feedbacks:
commit: d313d2b |
Collaborator
Author
[GD] Verdict: approve-commentTLDR
Full verdictSemantic propagation
UsefulnessRI-04 is the configuration/evaluation foundation for the Router Intelligence programme: a validated, fail-closed Bugs / correctness
Security
Spec / standards
Reviews
Base / CI
Simplification
Gatenone — draft cleared during review; ship-gate Bottom lineUseful, well-scoped RI-04 core with all review findings fixed or explicitly documented. Merging. |
Collaborator
Author
[GD] Merge readyPR: #1011 — feat(routing): add validated routing policy profiles Reviews
Tip freshness
Checks
Why it helps: this is the RI-04 foundation of the routing control plane — a validated, fail-closed Ready to merge. |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RI-04 of the Router Intelligence / Routing Control Plane programme. Adds the
routing policy profile core: a validated top-level
routingProfilesconfig schema, deterministic normalization + revision digest, collision
validation, a dry-run evaluator, management API, CLI, and documentation.
Policy routing is inert in this PR: no production request uses a profile
yet (that is RI-05). Everything here is configuration, evaluation-on-demand,
and dry-run.
Scope
src/types.ts-OcxRoutingProfileConfig(candidates allowlist, alias,require,optimize,limits,unknownEvidence) +routingProfilesonOcxConfig.src/routing/profile.ts- validation (routingProfileIssues), aliascollision rules (providers, combos, codex account namespaces, sibling
profiles, reserved native families,
policy/+combo/namespaces),normalization (
normalizeRoutingProfile), SHA-256 revision digest(16-hex),
resolvePolicyProfileId/parsePolicyModelId/policyPublicModelId.src/routing/evaluator.ts- deterministic dry-run evaluator: hardcapability requirements (min context window, tools, image input, structured
output, reasoning effort, service tier, local/remote, encrypted Codex
tasks) are evaluated against supplied evidence; unknown-capability handling
follows the profile's
unknownEvidence; scoring is deterministicconfigured priority only (capability/health/quota/cost components arrive
with RI-05..08). Produces a full RI-01 decision trace with
profile.id/profile.revision.src/config.ts-routingProfilesvalidated insidevalidateConfigCandidate(broken profiles fail closed like broken combos).GET /api/routing-profiles,POST /api/routing-profiles/dry-run.ocx route policy list | show <id> | dry-run <id> --model-context <tokens> --tools [--image] [--structured-output].routingProfilesschemaand the combos-vs-profiles distinction (locale sync completes in RI-10).
tests/routing-profile.test.ts- 12 tests.Unknown is not zero
unknownEvidenceper profile controls unknown capability evidence:exclude(default),penalize, orallow. Unknown prices/health/quota stayunknown; nothing converts unknown to zero or free.
No implicit routing
Existing model ids, combos, account namespaces, and default-provider
resolution are byte-for-byte unchanged. Profiles only activate for an
explicitly requested
policy/<id>or alias (RI-05 wires execution).Compatibility
routingProfilesload unchanged./api/logs,/api/request-history,/api/routing-analyticsunchanged.
Privacy / security
bun run privacy:scanpasses.Dependency
dev.devheada594938c5(parents' commitsdropped; single RI-04 commit
63924495e), then the review round landedd478b393(request-evidence wiring),8e1f1c3d(alias namespace check),aa9212fa(CLI cleanup). Current head:aa9212fa3.Non-goals
Local verification (exact)
bun x tsc --noEmit-> PASSED (0 errors)bun run test tests/routing-profile.test.ts-> 13/13 pass (12 + request-evidence regression)request-history-index, combos, codex-routing, internal-cli-dispatch
bun run privacy:scan-> passedtests/config.test.ts: 109/115 pass; the 6 symlink failures reproduceidentically on the pristine base (Windows symlink EPERM, environmental)
Review round (2026-08-05)
structured output, reasoning effort, service tier, encrypted Codex tasks)
against candidate capabilities together with the profile
requireblock; theCLI dry-run flags are live.
<provider>/<model>routing namespaceof a configured provider.
routingProfileIssuesForConfigexport; simplified CLI outputcalls and candidate scoring.
Notes for reviewers
tests):
optimize: { latency: 1, cost: 3 }merges health 0.25 and quota0.1 before normalizing (sum 4.35).
JSON), so configs that differ only in formatting share a revision.
Summary by CodeRabbit
route policycommands to list, inspect, and dry-run routing profiles.