feat(gateway): serve Claude Opus 5 on Pro + fix Opus 4.8's under-reserving context window#390
Conversation
There was a problem hiding this comment.
Pull request overview
Adds the anthropic/claude-opus-5 model to LevelCode’s server-driven model roster (and updates pricing/plan copy + specs accordingly), while also introducing a production Sidekiq Web mount gated by Devise admin (and optional email allowlist).
Changes:
- Add
anthropic/claude-opus-5as a confirmed, Pro-entitled OpenRouter model with 1M context inModelCatalog::MODELS. - Update LevelCode plan marketing copy and multiple specs to include/validate Opus 5 availability and multiplier parity with Opus 4.8.
- Mount Sidekiq::Web at
/web_sidekiqin all environments behind Devise admin + optionalSIDEKIQ_WEB_EMAILSallowlist, with request specs.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| app/services/levelcode/model_catalog.rb | Adds Opus 5 model entry (confirmed pricing, 1M context, Pro+) adjacent to Opus 4.8. |
| lib/levelcode.rb | Updates plan features strings to name Opus 5 in pricing copy. |
| spec/requests/api/levelcode/v1/account_spec.rb | Extends account/models request spec to assert Opus 5 appears live with expected multiplier. |
| spec/models/levelcode_model_catalog_spec.rb | Adds Opus 5 to catalog confirmation + multiplier expectations and entitlement assertions. |
| spec/models/levelcode_plans_spec.rb | Updates allowed_models roster expectations to include Opus 5 for Pro. |
| config/routes.rb | Mounts Sidekiq Web at /web_sidekiq behind admin/authenticate + optional email allowlist. |
| spec/requests/sidekiq_web_spec.rb | Adds request-level coverage ensuring Sidekiq Web is reachable only when properly entitled/allowlisted. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Gated on the Devise session + the `admin?` role — the same gate the admin dashboard API uses, so | ||
| # there is one notion of "who is staff" rather than two. Devise's `authenticate` runs through Warden, | ||
| # so a signed-out (or non-admin) visitor never reaches the Rack app at all: the route simply does not | ||
| # match for them, and /web_sidekiq 404s rather than advertising that a console lives there. |
There was a problem hiding this comment.
This one is moot here — config/routes.rb came from the unpushed 74d3a28 that I'd accidentally branched over, and it's no longer in this PR after the rebase. The point looks correct though (the HTML catch-all before not_found means a non-admin gets the SPA, not a 404), so it's worth keeping for whoever lands the Sidekiq change.
| describe "SIDEKIQ_WEB_EMAILS allowlist" do | ||
| let!(:admin) { create(:user, role: "admin") } | ||
|
|
||
| it "keeps an admin OFF the list out, even though they are an admin" do |
There was a problem hiding this comment.
Also moot here — spec/requests/sidekiq_web_spec.rb came from the same accidentally-included commit and is out of this PR now. Worth carrying over to the Sidekiq PR.
| # Sidekiq console — mounted in EVERY environment, including production, because that is where you | ||
| # actually need it: the 2026-07-22 worker outage left 141 jobs queued and 8 dead with no way to see | ||
| # the queue without shelling onto the box. |
There was a problem hiding this comment.
You were right, and this was my error — thank you. I branched off the local develop, which had an unpushed commit (74d3a28, the Sidekiq console work) that I then carried into this PR unintentionally. The PR description was accurate about my intent and wrong about its contents.
Rebased onto origin/develop (now at 48efb77), so this PR is back to exactly what it claims: 2 commits, 5 files, all LevelCode roster. config/routes.rb and spec/requests/sidekiq_web_spec.rb are no longer in the diff. The Sidekiq commit is untouched on the local branch and can go up as its own PR — which is what you suggested.
One catalog row is the whole feature — it goes live on DEPLOY with no editor release, because the client reads the roster live from GET /account/models. Confirmed against the OpenRouter models API (2026-07-24): anthropic/claude-opus-5 — $5/M in · $25/M out · $0.50/M cached read, 1M context Notes on the two things that are easy to get wrong here: - **Placement.** The row sits immediately after Opus 4.8 because the rates are IDENTICAL, so the two share a 6.67x multiplier. The monotonic-multiplier spec orders by COST, not recency; equal neighbours keep it satisfied. Appending it at the end (or grouping it by family) would have broken that invariant. - **status: :confirmed.** `allowed_models` filters on it, so an :assumption row would render in the picker and then silently fall back to the plan default when selected — no error to debug from. The price is a live quote, so :confirmed. Pro's economics are unchanged: same price sheet as Opus 4.8 means the same multiplier and the same ~39 turns on a $20 plan. What Pro gains is the newer model and a 1M window. The 1M context is load-bearing beyond display — estimate_cost_micros clamps the admission reservation against it, so understating it would refuse long-context turns the budget can afford. Deliberately NOT changed: the plan default stays Kimi K2.7 Code. Opus 5 costs 6.67x per turn, so defaulting to it would burn a Pro budget ~7x faster — it is an opt-in pick, same call as the K3 rollout. Marketing copy now names Opus 5 (served verbatim by /pricing, so the site picks it up on deploy). Opus 4.8 stays on the roster and selectable. Verified: Pro/Pro+/Max/Ultra all reach the id; free still downgrades to gpt-oss; rate_for resolves; ANTHROPIC_FAMILY matches so prompt caching engages. 234 LevelCode examples pass.
…r-reserving
The catalog listed anthropic/claude-opus-4-8 at 200_000, the pre-1M default.
Every OpenRouter endpoint for that model — Anthropic first-party, Bedrock, Azure,
Google — advertises 1M/128K (checked 2026-07-24).
This was not cosmetic. estimate_cost_micros clamps its input estimate DOWN to the
context window:
est_input = [ est_input, ctx ].min if ctx.positive?
so a stale window is a ceiling on what the admission guard will set aside. A
600k-token turn was being reserved as if it were 200k — roughly a third of what
it can actually cost — which is the opposite of the "reserve high, reconcile
down on settle" contract the guard is built on. Raising the window makes the
reservation honest; it never over-bills, because the ledger still meters real
usage afterwards.
Also corrects a comment I got backwards on the Opus 5 row in the previous commit:
understating `context` under-reserves (waves through turns that overrun), it does
not "refuse turns the budget can afford". The clamp is a floor on permissiveness,
not a gate.
Two specs, both of which fail against the old 200K value (verified by reverting):
· roster integrity pins every row's window and forbids a nil/0 one, since a
zero disables the clamp entirely;
· an estimate_cost_micros test that pins the DIRECTION — a 600k body must
reserve more than the old 200k clamp would, while a 10M body still clamps to
the window rather than running past it.
236 LevelCode examples pass.
5b868b4 to
48efb77
Compare
Two changes to the gateway roster. One catalog row is the whole first feature — it goes live on deploy with no editor release, since the client reads the roster from
GET /account/models.1. Opus 5, entitled from Pro up
Price confirmed against the OpenRouter models API (2026-07-24):
anthropic/claude-opus-5Identical to Opus 4.8's sheet — so Pro's per-turn economics don't move (same 6.67× multiplier, same ~39 turns on $20). What Pro gains is the newer model and the 1M window.
Two things that fail quietly if you get them wrong:
status:.allowed_modelsfilters on:confirmed. An:assumptionrow renders in the picker and then silently falls back to the plan default when selected, with no error.2. Opus 4.8's context window was 200K — it should be 1M
Not cosmetic.
estimate_cost_microsclamps its input estimate down to the window:So the window is a ceiling on what the admission guard reserves. A 600k-token turn on Opus 4.8 was reserved as if it were 200k — about a third of what it can cost — the opposite of the "reserve high, reconcile down on settle" contract. Raising it makes reservations honest. It never over-bills: the ledger still meters real usage afterwards.
Every OpenRouter endpoint for 4.8 (Anthropic first-party, Bedrock, Azure, Google) advertises 1M/128K.
This also corrects a comment I had backwards on the Opus 5 row: understating
contextunder-reserves, it does not "refuse turns the budget can afford."Deliberately not changed
Pro's default stays
moonshotai/kimi-k2.7-code. Opus 5 is 6.67× per turn; defaulting would drain a Pro budget ~7× faster. Opt-in pick, same call as the K3 rollout. Opus 4.8 stays on the roster.Verification
Runtime behavior (a wrong id here downgrades silently rather than erroring):
236 LevelCode examples pass. The two new context specs were checked as a negative control — reverting the window to 200K fails both, so they genuinely guard the bug rather than just restating the constant.
Marketing
features:copy now names Opus 5; it's served verbatim by/api/levelcode/v1/pricing, so the pricing page updates on deploy with no frontend change.Editor companion: levelcode #42 (BYOK picker + the same context correction in the local caps table).