Skip to content

Pin tool-capable OpenRouter free models in OpenCode review pool #619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
seonghobae merged 18 commits into from
Jul 24, 2026
+30 −14
Merged

Pin tool-capable OpenRouter free models in OpenCode review pool #619

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
7218e9e
Add OpenRouter free model for Strix and OpenCode
seonghobae Jul 23, 2026
d474490
Fix Strix GitHub Models fallback API base selection
seonghobae Jul 23, 2026
23b732b
Bump pyasn1 in Strix CI lockfile
seonghobae Jul 23, 2026
2c06c11
Restore Strix provider key-route contract checks
seonghobae Jul 23, 2026
4e97a13
Pin pyasn1 exactly in Strix requirements
seonghobae Jul 23, 2026
272e8ba
Trim Strix API key file inputs
seonghobae Jul 23, 2026
2e96191
Add temporary OpenRouter diagnostic workflow
seonghobae Jul 23, 2026
4627e50
Trigger OpenRouter diagnostic on push
seonghobae Jul 23, 2026
cde25f3
Add tool-call probe to OpenRouter diagnostic
seonghobae Jul 23, 2026
093a905
Add OpenCode CLI probe to OpenRouter diagnostic
seonghobae Jul 23, 2026
5265f10
Match review flags in OpenRouter diagnostic probe
seonghobae Jul 23, 2026
32ad887
Bound OpenCode diag probe with timeout
seonghobae Jul 23, 2026
1f7a842
Bisect opencode flag combinations in OpenRouter diagnostic
seonghobae Jul 23, 2026
7a92991
Remove temporary OpenRouter diagnostic workflow
seonghobae Jul 23, 2026
0a86f4f
Revert "Remove temporary OpenRouter diagnostic workflow"
seonghobae Jul 23, 2026
2026040
Pin tool-capable OpenRouter free models in review pool
seonghobae Jul 24, 2026
d8e6a5b
Merge origin/main, keep pinned OpenRouter free models
seonghobae Jul 24, 2026
6e6bd6c
Raise pinned OpenRouter model output limits to provider caps
seonghobae Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 17 additions & 7 deletions .github/workflows/opencode-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3299,12 +3299,20 @@ jobs:
"apiKey": "{env:OPENROUTER_API_KEY}"
},
"models": {
"openrouter/free": {
"name": "OpenRouter Free",
"nvidia/nemotron-3-ultra-550b-a55b:free": {
"name": "NVIDIA Nemotron 3 Ultra 550B (free)",
"tool_call": true,
"limit": {
"context": 128000,
"output": 4096
"context": 1000000,
"output": 65536
}
Comment thread
seonghobae marked this conversation as resolved.
},
"poolside/laguna-m.1:free": {
"name": "Poolside Laguna M.1 (free)",
"tool_call": true,
"limit": {
"context": 262144,
"output": 32768
}
}
}
Expand Down Expand Up @@ -3537,14 +3545,16 @@ jobs:
NO_COLOR: "1"
# High-sensitivity review candidates only. DeepSeek V3 has been the
# most reliable first-pass reviewer in the org queue, then the pool
# falls through to the direct GPT-5.6 Luna slot, OpenRouter's free
# compatibility endpoint, then the full-size GPT-4.1 long-context
# falls through to the direct GPT-5.6 Luna slot, pinned OpenRouter
# free models (the openrouter/free router routed to weak models that
# never produced a control block, so tool-capable free models are
# pinned explicitly), then the full-size GPT-4.1 long-context
# endpoint and provider-specific GPT/o3 fallbacks.
# The direct-OpenAI slot runs GPT-5.6 Luna: the newest family's
# cost-efficient tier, cheaper than the legacy gpt-5 it replaced
# ($1/$6 vs $1.25/$10 per 1M tokens) so the org OpenAI budget
# stretches further between top-ups.
OPENCODE_MODEL_CANDIDATES: "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/openrouter/free github-models/openai/gpt-4.1 github-models/openai/gpt-5 github-models/openai/gpt-5-chat github-models/openai/o3 github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1"
OPENCODE_MODEL_CANDIDATES: "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/nvidia/nemotron-3-ultra-550b-a55b:free openrouter/poolside/laguna-m.1:free github-models/openai/gpt-4.1 github-models/openai/gpt-5 github-models/openai/gpt-5-chat github-models/openai/o3 github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1"
# One attempt per model, then fall through to the next model. Retrying
# the SAME model 5x let a rate-limited/hung leader consume the whole
# step, so the pool never reached a healthy fallback model.
Expand Down
6 changes: 3 additions & 3 deletions scripts/ci/test_strix_quick_gate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,7 +700,7 @@ assert_opencode_review_uses_codegraph_and_gpt5_fallback() {
assert_file_contains "$workflow_file" 'OPENCODE_MODEL_ATTEMPTS: "1"' "opencode fallback tries the catalog promptly instead of spending the entire review on one model"
assert_file_contains "$workflow_file" "Run OpenCode PR Review model pool" "opencode review includes a broad catalog fallback pool"
assert_file_not_contains "$workflow_file" "steps.opencode_review_model_pool.outcome == 'success'" "opencode approval gate still runs after model pool failure to publish a reason"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/openrouter/free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review starts with DeepSeek V3 before full-size GPT fallbacks"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/nvidia/nemotron-3-ultra-550b-a55b:free openrouter/poolside/laguna-m.1:free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review starts with DeepSeek V3 before full-size GPT fallbacks"
assert_file_contains "$workflow_file" "The publish gate re-runs source-backed validation against PR-head data" "opencode review publish gate validates model output against the PR-head worktree"
assert_file_contains "$workflow_file" '"openai/o3"' "opencode config declares OpenAI o3 fallback"
assert_file_contains "$workflow_file" '"openai/o4-mini"' "opencode config declares OpenAI o4-mini fallback"
Expand Down Expand Up @@ -848,7 +848,7 @@ assert_opencode_review_uses_codegraph_and_gpt5_fallback() {
assert_file_contains "$workflow_file" 'OPENCODE_RUN_TIMEOUT_SECONDS: "5400"' "opencode catalog fallback preserves legitimate full-hour provider sessions"
assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" "OpenCode %s attempt %s/%s failed" "opencode catalog fallback records per-model retry failures"
assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" "exponential backoff" "opencode model retry paths use exponential backoff instead of fixed sleeps"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/openrouter/free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review tries DeepSeek V3 before OpenAI fallbacks"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/nvidia/nemotron-3-ultra-550b-a55b:free openrouter/poolside/laguna-m.1:free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review tries DeepSeek V3 before OpenAI fallbacks"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1" "opencode review keeps DeepSeek reasoning fallback coverage after OpenAI candidates"
assert_file_contains "$workflow_file" "coverage-source-tree:" "opencode workflow materializes coverage source before running PR-head tests"
assert_file_contains "$workflow_file" "coverage-evidence:" "opencode workflow measures coverage before review"
Expand Down Expand Up @@ -1165,7 +1165,7 @@ assert_opencode_review_uses_codegraph_and_gpt5_fallback() {
assert_file_contains "$workflow_file" 'GH_TOKEN: ${{ secrets.OPENCODE_APPROVE_TOKEN || steps.review_read_app_token.outputs.token || github.token }}' "opencode manual dispatch uses the cross-repo approval token for target PR evidence lookups with app-token fallback"
assert_file_contains "$workflow_file" 'repos/${GH_REPOSITORY}' "opencode review workflow uses env-backed repository context in shell commands"
assert_file_contains "$workflow_file" "Run OpenCode PR Review model pool" "opencode review starts the central model pool"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/openrouter/free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review starts with DeepSeek V3 before full-size GPT fallbacks"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/nvidia/nemotron-3-ultra-550b-a55b:free openrouter/poolside/laguna-m.1:free github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review starts with DeepSeek V3 before full-size GPT fallbacks"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-r1-0528" "opencode review keeps a reachable DeepSeek R1 reasoning fallback model"
assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324" "opencode review has a reachable DeepSeek V3 fallback model"
assert_file_contains "$workflow_file" "github-models/openai/gpt-5" "opencode review still has a bounded GPT-5 fallback model"
Expand Down
14 changes: 10 additions & 4 deletions tests/test_opencode_agent_contract.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,8 @@ def test_opencode_model_pool_sets_high_effort_for_capable_candidates():
assert candidate_pairs == [
["github-models", "deepseek/deepseek-v3-0324"],
["openai", "gpt-5.6-luna"],
["openrouter", "openrouter/free"],
["openrouter", "nvidia/nemotron-3-ultra-550b-a55b:free"],
["openrouter", "poolside/laguna-m.1:free"],
["github-models", "openai/gpt-4.1"],
["github-models", "openai/gpt-5"],
["github-models", "openai/gpt-5-chat"],
Expand All @@ -117,7 +118,10 @@ def test_opencode_model_pool_sets_high_effort_for_capable_candidates():
["github-models", "deepseek/deepseek-r1"],
]
assert direct_openai_models == ["gpt-5.6-luna"]
assert openrouter_models == ["openrouter/free"]
assert openrouter_models == [
"nvidia/nemotron-3-ultra-550b-a55b:free",
"poolside/laguna-m.1:free",
]
assert set(github_candidate_models).issubset(set(github_models))
assert github_candidate_models == [
"deepseek/deepseek-v3-0324",
Expand Down Expand Up @@ -1099,7 +1103,8 @@ def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
assert (
'OPENCODE_MODEL_CANDIDATES: "github-models/deepseek/deepseek-v3-0324 '
"openai/gpt-5.6-luna "
"openrouter/openrouter/free "
"openrouter/nvidia/nemotron-3-ultra-550b-a55b:free "
"openrouter/poolside/laguna-m.1:free "
"github-models/openai/gpt-4.1 "
"github-models/openai/gpt-5 "
"github-models/openai/gpt-5-chat "
Expand Down Expand Up @@ -1223,7 +1228,8 @@ def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
assert (
'OPENCODE_MODEL_CANDIDATES: "github-models/deepseek/deepseek-v3-0324 '
"openai/gpt-5.6-luna "
"openrouter/openrouter/free "
"openrouter/nvidia/nemotron-3-ultra-550b-a55b:free "
"openrouter/poolside/laguna-m.1:free "
"github-models/openai/gpt-4.1 "
"github-models/openai/gpt-5 "
"github-models/openai/gpt-5-chat "
Expand Down
Loading