Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,18 @@ qbraid-code --model gpt-5.6-sol
qbraid-code --model gpt-5.4-mini -p "explain this error"
```

Claude Code speaks the Anthropic API and the gateway serves GPT only on its
OpenAI-compatible surface, so the first GPT request starts a small local
translation proxy (CLIProxyAPI, loopback only, installed by the installer).
Claude models never touch it. `qbraid-code --stop` shuts it down.

Two caveats: GPT models accept at most 128 tools, so with many MCP servers
add `--strict-mcp-config`; and the `/model` picker inside a session lists
Claude models only — choose a GPT model at launch with `--model`.
Every model — Claude and GPT — is served through one local endpoint: a small
translation proxy (CLIProxyAPI, loopback only, started on demand). Claude
models pass through to the gateway untouched; GPT models are translated to its
OpenAI-compatible surface. Because it is one endpoint, `/model <name>` works
for **any** of the models mid-session, e.g. `/model gpt-5.6-sol`.

If the proxy is missing, Claude models automatically fall back to the gateway
directly — they can never break because of it. `qbraid-code --stop` shuts the
proxy down.

One caveat: GPT models accept at most 128 tools, so with many MCP servers add
`--strict-mcp-config`.

A session looks like this:

Expand Down
120 changes: 120 additions & 0 deletions docs/research/2026-08-20-model-picker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Can the `/model` picker list custom (qBraid gateway GPT) models?

Date: 2026-08-20. Claude Code v2.1.238 (native binary, `~/.local/share/claude/versions/2.1.238`).

## Question

Can the in-session `/model` picker show custom models — the qBraid gateway's GPT models (`gpt-5.6-sol` etc.) — when Claude Code runs against a third-party `ANTHROPIC_BASE_URL`?

## TL;DR verdict

**Yes — two supported, documented mechanisms exist.**

1. `ANTHROPIC_CUSTOM_MODEL_OPTION` (+ `_NAME`, `_DESCRIPTION`) adds **one** unvalidated custom row to the picker. Any model ID works, including `gpt-5.6-sol`.
2. `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` makes Claude Code fetch `GET {ANTHROPIC_BASE_URL}/v1/models?limit=1000` at startup and add the results to the picker — **but only IDs containing `claude` or `anthropic`** (case-insensitive, substring). `gpt-5.6-sol` is silently dropped unless the proxy serves it under an alias that passes the filter (e.g. `anthropic-compat/gpt-5.6-sol`).

A third lever, the `ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL` family (+ `_NAME`, `_DESCRIPTION`), remaps the built-in alias rows to arbitrary IDs with custom labels — up to 4 more slots, at the cost of changing what `opus`/`sonnet`/`haiku`/`fable` mean session-wide.

The `~/.claude.json` caches are readable and unsigned, but seeding them is unsupported and fragile (overwritten by bootstrap). `availableModels` in settings only restricts; it cannot add non-Claude IDs.

## Evidence

### 1. Binary (primary source)

All byte offsets are into `/Users/belazy/.local/share/claude/versions/2.1.238` (306 MB Mach-O with embedded JS; app source region ≈ bytes 283 M–306 M).

**Picker option builder** (`_7b`, offset ≈ 285,100,020 = extract 1,984,500):

```js
function _7b(e,t){
let r=h7b(e), // built-in tier rows
n=V.ANTHROPIC_CUSTOM_MODEL_OPTION;
if(n&&!r.some((c)=>c.value===n))
r.push({value:n,
label:V.ANTHROPIC_CUSTOM_MODEL_OPTION_NAME??n,
description:V.ANTHROPIC_CUSTOM_MODEL_OPTION_DESCRIPTION??`Custom model (${n})`});
for(let c of tDn()) ... // gateway-discovered rows
if(o==="firstParty"||o==="gateway"){ for(let u of wJe()) ... } // additionalModelOptionsCache rows
let{availableModels:i}=Na()??{}; // settings allowlist: adds only anthropic.* (Mantle) or claude-* ids
...
}
```

- `ANTHROPIC_CUSTOM_MODEL_OPTION` pushes exactly **one** row, no name filter, no validation. Env registry (offset ≈ 281,631,458) defines only the three vars — no plural form.
- `availableModels` entries can only *add* rows when they start with `anthropic.` (Bedrock Mantle) or match `/^claude-[a-z0-9-]+$/` and contain opus/sonnet/haiku. A `gpt-*` entry never adds a row.

**Gateway discovery for custom base URL** (`Vna`/`dCd`/`tDn`, offsets ≈ 283,557,5xx = extract 442–443 k):

```js
function Vna(){
if(!V.CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY)return!1;
if(ro()!=="firstParty")return!1; // no CLAUDE_CODE_USE_* provider set
if(Nm())return!1; // Nm(): base URL host is api.anthropic.com
if(!V.ANTHROPIC_BASE_URL)return!1;
return!0
}
// dCd(): GET `${ANTHROPIC_BASE_URL}/v1/models?limit=1000`
// auth: Bearer ANTHROPIC_AUTH_TOKEN, else x-api-key; ANTHROPIC_CUSTOM_HEADERS honored
// redirect:"error", timeout lTb=3000 ms
// FILTER: l.data.data.filter((p)=>/(claude|anthropic)/i.test(p.id))
// 0 survivors -> "[gatewayDiscovery] 0 usable models after filter" -> nothing cached
// cache: <config>/cache/gateway-models.json {baseUrl, fetchedAt, models}
// tDn(): reads cache, requires baseUrl === current ANTHROPIC_BASE_URL,
// maps to {value:id, label:display_name||id, description:"From gateway"}
```

`dCd(t)` runs once at startup (init sequence, offset ≈ extract 23,555,000). `Nm()` (extract 195,805): returns true only for host `api.anthropic.com` or `_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL`. So qbraid-code's loopback proxy qualifies for discovery.

**Cloud-gateway variant** (`ugT`/`pgT`, offsets ≈ 295,005,000): when signed in through a Claude apps gateway (`ro()==="gateway"`, JWT-based), the same env flag gates a `/v1/models` fetch with an extra family filter. Not the qbraid case; noted for completeness.

**Built-in rows and pinned aliases** (`h7b`, extract 1,980,354): each tier row can be replaced by a pinned override (`vep`/`_ep`/`Eep`/`Sep` — the `ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL` values with `_NAME`/`_DESCRIPTION` labels).

**`availableModels` setting** (offset ≈ 86,300,350): schema description reads "Allowlist of models that users can select… If undefined, all models are available." It is a restriction, enforced client-side; it also bounds what discovery and the custom option can add.

### 2. `~/.claude.json` caches (read-only inspection)

| Key | Shape on this machine | Verdict on seeding |
|---|---|---|
| `additionalModelOptionsCache` | `[{value:"claude-fable-5[1m]", label:"Fable", description:"…"}]` | Read unsigned by `wJe()` (extract 1,044,105) with shape validation only; rows shown when provider is firstParty/gateway. **But** it is a server bootstrap cache: `vvt()` (offset ≈ 295,007,700) overwrites it from `{base}/api/claude_cli/bootstrap` responses. Seeding works mechanically, survives only until the next successful bootstrap. Unsupported. |
| `additionalModelCostsCache` | `{}` | Pricing display only. |
| `modelAccessCache` | `[]` | Entitlements `[{apiName, entitled}]` (`bHr`); used to gate/disable rows, not add them. |
| `orgModelDefaultCache` | `null` | Org default model; not additive. |
| `customApiKeyResponses` | `{approved:[], rejected:["6cvg…"]}` | API-key trust prompts. Nothing to do with models. |

No signing or expiry on any of them. `~/.claude/cache/gateway-models.json` does not exist yet on this machine (discovery flag never set).

### 3. Official docs

- `code.claude.com/docs/en/model-config#add-a-custom-model-option`: "Use `ANTHROPIC_CUSTOM_MODEL_OPTION` to add a single custom entry to the `/model` picker… Claude Code skips validation for the model ID… For LLM gateway deployments, Claude Code can populate the picker from the gateway's `/v1/models` endpoint when `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` is set."
- `code.claude.com/docs/en/llm-gateway-protocol#model-discovery`: request is `GET /v1/models?limit=1000`, 3 s timeout, redirects = failure; sends exactly one credential header (`ANTHROPIC_AUTH_TOKEN` bearer preferred); "Claude Code keeps an entry when its `id` contains `claude` or `anthropic` anywhere in the string, matched case-insensitively, and ignores the rest" (substring match since v2.1.223; prefix-only before). Rows are labeled "From gateway"; duplicates fold into built-in alias rows (v2.1.197+); cache at `~/.claude/cache/gateway-models.json`, refreshed each startup, stale cache used on fetch failure. "If your gateway serves Claude models under aliases that don't match the discovery filter, developers can add those aliases manually with the model configuration variables."
- `code.claude.com/docs/en/env-vars`: documents all of `ANTHROPIC_CUSTOM_MODEL_OPTION{,_NAME,_DESCRIPTION}`, `ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL{,_NAME,_DESCRIPTION}`, `ANTHROPIC_CUSTOM_HEADERS`. (`CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY` is documented on the protocol page, not the env-vars page.)
- `model-config`: `availableModels` "restricts which named models users can select"; the only additive exception is Mantle `anthropic.*` IDs. Discovery is "off by default so that gateways backed by a shared API key don't surface every model the key can access to every user."

## Mechanisms evaluated

| Mechanism | Adds GPT models to picker? | Evidence |
|---|---|---|
| `ANTHROPIC_CUSTOM_MODEL_OPTION` (+`_NAME`,`_DESCRIPTION`) | **Yes — one model, any ID, no validation** | `_7b` @ extract 1,984,500; docs model-config#add-a-custom-model-option |
| `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` + proxy `/v1/models` | **Yes, many — but only IDs containing `claude`/`anthropic`**; plain `gpt-5.6-sol` filtered out | `dCd` filter @ extract 442,9xx; docs llm-gateway-protocol#model-discovery |
| `ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL` (+`_NAME`,`_DESCRIPTION`) | **Yes — remaps up to 4 built-in rows** to arbitrary IDs with custom labels; changes alias meaning session-wide (haiku slot also serves background traffic) | `h7b` pinned overrides @ extract 1,980,354; docs env-vars |
| Seed `additionalModelOptionsCache` in `~/.claude.json` | Works mechanically (unsigned, shape-checked only) but **unsupported**; overwritten by server bootstrap | `wJe` @ extract 1,044,105; `vvt` cache write @ 295,007,700 |
| `availableModels` settings key | **No** — allowlist only; additive only for `anthropic.*` (Mantle) and `claude-*` IDs | schema @ 86,300,350; docs model-config#restrict-model-selection |
| `modelAccessCache` / `orgModelDefaultCache` / `customApiKeyResponses` | No — entitlements / org default / API-key trust | `bHr`; jq inspection |
| A `--models` flag | Does not exist | `claude --help`; env registry |

## Recommendation for qbraid-code

Layered, all from the installer/wrapper env — no config-file writes:

1. **Today, zero-risk:** export `ANTHROPIC_CUSTOM_MODEL_OPTION="gpt-5.6-sol"`, `ANTHROPIC_CUSTOM_MODEL_OPTION_NAME="GPT-5.6 Sol"`, `ANTHROPIC_CUSTOM_MODEL_OPTION_DESCRIPTION="qBraid gateway · OpenAI GPT-5.6"` in the `qbraid-code` launcher before exec'ing `claude`. One GPT model becomes selectable in `/model` immediately. If the wrapper knows which GPT model the user launched with (`--model gpt-…`), set the custom option to that model so the active model always has a picker row.
2. **For the full GPT roster:** implement `GET /v1/models` on the loopback proxy (:8320) and export `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1`. To pass the filter, serve the GPT models under IDs containing `anthropic` or `claude` — e.g. `id: "anthropic-compat/gpt-5.6-sol"`, `display_name: "GPT-5.6 Sol"` — and have the proxy accept those IDs on `/v1/messages` (strip the prefix before translating). Serve the endpoint directly: no redirect, respond < 3 s, accept the same `ANTHROPIC_AUTH_TOKEN` bearer. Rows appear labeled "From gateway".
3. **Optional:** repurpose alias rows via `ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-5.6-sol` + `_NAME`/`_DESCRIPTION` for up to 4 labeled rows. Only do this if changing what `opus`/`sonnet`/`haiku` mean (subagents, background haiku traffic) is acceptable — probably not as a default.
4. Do **not** seed `~/.claude.json` caches; do not rely on `availableModels` to add models.
5. Upstream ask, if the prefix trick feels too ugly: a feature request to relax the discovery filter (opt-in "trust my gateway's full model list") — the filter exists only to hide shared-key upsell, and the docs already acknowledge the alias workaround.

## Open questions

- Filter-passing alias format: `anthropic-compat/gpt-5.6-sol` vs `qbraid-anthropic/gpt-5.6-sol` — pick one, proxy must accept it on inference.
- Does picking a "From gateway" GPT row keep `ANTHROPIC_SMALL_FAST_MODEL` / subagent model routing sane? (Picker only sets main model; verify proxy handles haiku background calls.)
- Multiple `ANTHROPIC_CUSTOM_MODEL_OPTION` rows: only one env slot exists; >1 static GPT rows need the discovery path or alias-slot remaps.
- Discovery + claude.ai-subscription auth mix: qbraid-code sets `ANTHROPIC_AUTH_TOKEN`, so bearer path is used; untested whether an active claude.ai login alongside changes bootstrap overwrite timing of `additionalModelOptionsCache` (irrelevant unless seeding, which is not recommended).
15 changes: 13 additions & 2 deletions install.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,9 @@ if ($ProxyBin) {
Write-RawText $keyFile (($bytes | ForEach-Object { $_.ToString('x2') }) -join '')
}
$localKey = (Get-Content $keyFile -Raw).Trim()
# One proxy, every model: Claude passthrough + GPT translation, so one
# endpoint lists and serves all of them.
$claudeModels = @(Get-Prop $list 'data' | ForEach-Object { Get-Prop $_ 'id' } | Where-Object { $_ -like 'claude-*' })
$yaml = @()
$yaml += '# Generated by the qbraid-code installer. Loopback only.'
$yaml += 'host: "127.0.0.1"'
Expand All @@ -383,8 +386,16 @@ if ($ProxyBin) {
$yaml += ' allow-remote: false'
$yaml += ' disable-control-panel: true'
$yaml += 'debug: false'
$yaml += 'claude-api-key:'
$yaml += " - api-key: `"$ApiKey`""
$yaml += " base-url: `"$GatewayUrl`""
$yaml += ' models:'
foreach ($cm in $claudeModels) {
$yaml += " - name: `"$cm`""
$yaml += " alias: `"$cm`""
}
$yaml += 'openai-compatibility:'
$yaml += ' - name: "qbraid-gateway"'
$yaml += ' - name: "qbraid-gateway-gpt"'
$yaml += " base-url: `"$GatewayUrl`""
$yaml += ' api-key-entries:'
$yaml += " - api-key: `"$ApiKey`""
Expand All @@ -395,7 +406,7 @@ if ($ProxyBin) {
}
Write-RawText (Join-Path $HomeDir 'proxy-config.yaml') (($yaml -join "`n") + "`n")
New-Item -ItemType Directory -Force -Path (Join-Path $HomeDir 'proxy-auth') | Out-Null
Ok "proxy configured for $($gptModels.Count) GPT models (starts on demand)"
Ok "proxy configured: all $($gptModels.Count + $claudeModels.Count) models on one endpoint (starts on demand)"
}
} else {
Warn 'CLIProxyAPI unavailable - GPT models will not work; Claude models are unaffected.'
Expand Down
21 changes: 19 additions & 2 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,11 @@ if [ -n "$PROXY_BIN" ]; then
umask "$OLD_UMASK"
fi
PROXY_LOCAL_KEY=$(cat "$HOME_DIR/proxy.key")
# One proxy, every model: Claude models pass through to the Anthropic
# surface untouched (claude-api-key with a custom base-url), GPT models are
# translated to the OpenAI surface. The proxy's /v1/models then lists all
# of them, and one base URL serves any `--model`.
CLAUDE_MODELS=$(set +o pipefail; printf '%s' "$API_BODY" | grep -o '"id":"claude-[^"]*"' | sed 's/"id":"//; s/"$//')
OLD_UMASK=$(umask); umask 077
{
cat <<PEOF
Expand All @@ -505,22 +510,34 @@ remote-management:
allow-remote: false
disable-control-panel: true
debug: false
claude-api-key:
- api-key: "$API_KEY"
base-url: "$GATEWAY_URL"
models:
PEOF
printf '%s\n' "$CLAUDE_MODELS" | while IFS= read -r cm; do
[ -n "$cm" ] || continue
printf ' - name: "%s"\n alias: "%s"\n' "$cm" "$cm"
done
cat <<PEOF
openai-compatibility:
- name: "qbraid-gateway"
- name: "qbraid-gateway-gpt"
base-url: "$GATEWAY_URL"
api-key-entries:
- api-key: "$API_KEY"
models:
PEOF
printf '%s\n' "$GPT_MODELS" | while IFS= read -r gm; do
[ -n "$gm" ] || continue
printf ' - name: "%s"\n alias: "%s"\n' "$gm" "$gm"
done
} > "$HOME_DIR/proxy-config.yaml"
chmod 600 "$HOME_DIR/proxy-config.yaml"
umask "$OLD_UMASK"
mkdir -p "$HOME_DIR/proxy-auth"
GPT_COUNT=$(printf '%s\n' "$GPT_MODELS" | wc -l | tr -d ' ')
ok "proxy configured for $GPT_COUNT GPT models (starts on demand)"
CLAUDE_COUNT=$(printf '%s\n' "$CLAUDE_MODELS" | wc -l | tr -d ' ')
ok "proxy configured: all $((GPT_COUNT + CLAUDE_COUNT)) models on one endpoint (starts on demand)"
fi
else
warn "CLIProxyAPI unavailable — GPT models will not work; Claude models are unaffected."
Expand Down
37 changes: 25 additions & 12 deletions qbraid-code
Original file line number Diff line number Diff line change
Expand Up @@ -195,21 +195,34 @@ command -v claude >/dev/null 2>&1 || {
RUN_MODEL=$(requested_model "$@")
RUN_BASE="$BASE_URL"
RUN_TOKEN="$TOKEN"
case "$RUN_MODEL" in
gpt-*)
# GPT route: loopback proxy translates Anthropic Messages to the
# gateway's OpenAI surface. Azure caps `tools` at 128 — with many MCP
# servers Claude Code exceeds it; --strict-mcp-config avoids that.
start_proxy || exit 1
if [ -n "$PROXY_BIN" ] && [ -x "$PROXY_BIN" ] && [ -s "$HOME_DIR/proxy-config.yaml" ]; then
# Unified route: the proxy serves EVERY model on one endpoint — Claude
# models pass through to the Anthropic surface untouched, GPT models are
# translated to the OpenAI surface — so /v1/models lists all of them and
# `/model <name>` works for any of them mid-session. Azure caps `tools` at
# 128 for GPT models; with many MCP servers use --strict-mcp-config.
if start_proxy; then
RUN_BASE="$PROXY_URL"
RUN_TOKEN=$(cat "$HOME_DIR/proxy.key")
;;
esac
else
case "$RUN_MODEL" in
gpt-*) exit 1 ;; # start_proxy already explained
*) echo "qbraid-code: proxy unavailable — using the gateway directly (Claude models only)." >&2 ;;
esac
fi
else
case "$RUN_MODEL" in
gpt-*)
echo "qbraid-code: GPT models need the local proxy, which is not installed." >&2
echo "Re-run the installer: curl -fsSL https://qbraid.com/code.sh | bash" >&2
exit 1 ;;
esac
fi

# ANTHROPIC_AUTH_TOKEN sends `Authorization: Bearer <key>`, which the gateway
# accepts. ANTHROPIC_API_KEY would work too, but it makes Claude Code ask the
# user to approve a custom API key on first run — a prompt with no good answer
# for someone who just wants to start working.
# ANTHROPIC_AUTH_TOKEN sends `Authorization: Bearer <key>`, which both the
# gateway and the proxy accept. ANTHROPIC_API_KEY would work too, but it makes
# Claude Code ask the user to approve a custom API key on first run — a prompt
# with no good answer for someone who just wants to start working.
#
# MAX_THINKING_TOKENS=0: recent Claude Code sends `thinking: {type: "adaptive"}`,
# which the gateway's Anthropic surface rejects (it accepts enabled|disabled —
Expand Down
18 changes: 12 additions & 6 deletions qbraid-code.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -55,15 +55,21 @@ for %%a in (%*) do (

set "RUNBASE=%QBRAID_CODE_BASE_URL%"
set "RUNTOKEN=%QBRAID_CODE_TOKEN%"
if /i not "%RUNMODEL:~0,4%"=="gpt-" goto :direct
rem GPT route: the loopback proxy translates Anthropic Messages to the
rem gateway's OpenAI surface. Azure caps tools at 128; with many MCP servers
rem use --strict-mcp-config.
rem Unified route: the proxy serves every model on one endpoint (Claude
rem passthrough, GPT translated). Fall back to the direct gateway for Claude
rem models when the proxy is unavailable; GPT models require it.
if not exist "%QC_HOME%\proxy-config.yaml" goto :noproxy
powershell -NoProfile -ExecutionPolicy Bypass -File "%QC_HOME%\qbraid-proxy.ps1" ensure
if errorlevel 1 exit /b 1
if errorlevel 1 goto :noproxy
set /p RUNTOKEN=<"%QC_HOME%\proxy.key"
set "RUNBASE=http://127.0.0.1:8320"
:direct
goto :routed
:noproxy
if /i "%RUNMODEL:~0,4%"=="gpt-" (
echo qbraid-code: GPT models need the local proxy. Re-run: irm https://qbraid.com/code.ps1 ^| iex 1>&2
exit /b 1
)
:routed

where claude >nul 2>&1
if errorlevel 1 (
Expand Down
Loading