Repository navigation
feat: adopt Sol scanner policy with compatible Responses adapters - #68
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
clawsweeper
Bot
added
clawsweeper:intake:feature
|
Codex review: needs maintainer review before merge. What this changesThis PR adopts Sol scanner and judge defaults, adds Responses compatibility patches, and records configured judge-model provenance while preserving operator overrides. Example: A custom SkillSpector profile sets gpt-6.1-sol and medium reasoning, omits the provider, and runs with OPENAI_API_KEY.
Review scores
ProductKind: Feature · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review Keep open: this maintainer-authored PR delivers useful work absent from main; the previous provider-inference finding is fixed, and no new actionable defect was found. Priority: P2 Before mergeNone. FindingsNone. Agent review detailsHow this fits togetherProfile resolution passes non-secret defaults into the runner, which launches sandboxed scanners and an external judge and preserves their JSON evidence in run artifacts. flowchart LR
A[CLI and profile] --> B[Runner defaults]
B --> C[Sandboxed scanners]
C --> D[Patched Responses adapters]
D --> E[Raw scanner evidence]
E --> F[External Codex judge]
F --> G[Run artifact]
Technical reviewBest possible solution: Land the reviewed source with its matching patched runtime and retain the documented separation between source adoption and consumer rollout. Do we have a high-confidence way to reproduce the issue? The earlier provider-ordering failure follows directly from the previous source; the current source fixes it, and the posted executed CLI trace exercises the corrected custom-profile case. Is this the best way to solve the issue? The change keeps profile defaults in the existing runner and confines transport compatibility to pinned upstream adapters, with conservative configured-request provenance and preserved overrides. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 148c420335e3. Provenance checked
TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Likely related people:
LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (1 earlier review cycle)
Reviewed October 19, 2026, 10:59 PM ET / October 20, 2026, 02:59 UTC (Revision 2). |
|
The reported P2 and both requested proof gaps are addressed at head The provider-inference regression reproduced before the fix and passes now. The effective provider is selected after explicit profile-provider settings but before provider-specific defaults; explicit environment settings and non-OpenAI providers remain authoritative. Both bundled Codex judges also explicitly request The actual v0.2.0 binary (source This is actual CLI/profile/environment/subprocess routing proof with secretless synthetic scanner/judge probes, not provider wire or verdict-quality proof. The checked-in reproducible harness is Native image proof passed for both Linux amd64 and arm64, including export/load, architecture/OS assertions and The source stamp in those artifacts is GitHub synthetic merge
Artifacts Fresh full Go tests/vet, docs build, harness lint/format and Sol/high/Standard P0–P3 review passed. Parent independently reran the focused Go regressions. Provider-observed identity is not claimed by Raw executed upgrade trace{
"kind": "clawscan-model-policy-upgrade-v1",
"previous": {
"sha256": "014ee7bca4fdd0cab6368b27e8265a7bc51eccfbc3c90cc0dec033686bd2dda3"
},
"candidate": {
"sha256": "3468554b2e3965755db68e38f9a73b98ac9fcf3e1c0712133278eaa34bc60d0c"
},
"execution": "real CLI binaries with synthetic subprocess probes; no provider calls",
"cases": [
{
"case": "previous-bundled",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"DEFAULT_MODEL": "gpt-5.5",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "aig-skill-scan",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"DEFAULT_MODEL": "gpt-5.5",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "codex",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"DEFAULT_MODEL": "gpt-5.5",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
},
"model": "gpt-5.5",
"config": [
"approval_policy=never",
"model_reasoning_effort=high",
"service_tier=fast",
"shell_environment_policy.inherit=\"core\"",
"shell_environment_policy.ignore_default_excludes=false"
],
"ignoreUserConfig": true
}
],
"scanners": [
"aig",
"clawscan-static",
"skillspector"
],
"judgeConfiguredModel": null
},
{
"case": "candidate-bundled",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "aig-skill-scan",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "codex",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
},
"model": "gpt-6.1-sol",
"config": [
"approval_policy=never",
"model_reasoning_effort=high",
"service_tier=\"default\"",
"shell_environment_policy.inherit=\"core\"",
"shell_environment_policy.ignore_default_excludes=false"
],
"ignoreUserConfig": true
}
],
"scanners": [
"aig",
"clawscan-static",
"skillspector"
],
"judgeConfiguredModel": "gpt-6.1-sol"
},
{
"case": "candidate-clawhub-aig",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "aig-skill-scan",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
}
},
{
"probe": "codex",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium",
"DEFAULT_MODEL": "gpt-6.1-sol",
"REASONING_EFFORT": "medium",
"DEFAULT_BASE_URL": "https://api.openai.com/v1"
},
"model": "gpt-6.1-sol",
"config": [
"approval_policy=never",
"model_reasoning_effort=high",
"service_tier=\"default\"",
"shell_environment_policy.inherit=\"core\"",
"shell_environment_policy.ignore_default_excludes=false"
],
"ignoreUserConfig": true
}
],
"scanners": [
"aig",
"skillspector"
],
"judgeConfiguredModel": "gpt-6.1-sol"
},
{
"case": "previous-overrides",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
}
},
{
"probe": "aig-skill-scan",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
}
},
{
"probe": "codex",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
},
"model": "gpt-6-luna",
"config": [
"model_reasoning_effort=low",
"service_tier=\"priority\""
],
"ignoreUserConfig": false
}
],
"scanners": [
"aig",
"clawscan-static",
"skillspector"
],
"judgeConfiguredModel": null
},
{
"case": "previous-project-shadow",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai"
}
}
],
"scanners": [
"skillspector"
],
"judgeConfiguredModel": null
},
{
"case": "candidate-overrides",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
}
},
{
"probe": "aig-skill-scan",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
}
},
{
"probe": "codex",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic",
"SKILLSPECTOR_MODEL": "operator-model",
"SKILLSPECTOR_REASONING_EFFORT": "high",
"DEFAULT_MODEL": "operator-aig",
"REASONING_EFFORT": "low",
"DEFAULT_BASE_URL": "https://provider.example/v1"
},
"model": "gpt-6-luna",
"config": [
"model_reasoning_effort=low",
"service_tier=\"priority\""
],
"ignoreUserConfig": false
}
],
"scanners": [
"aig",
"clawscan-static",
"skillspector"
],
"judgeConfiguredModel": "gpt-6-luna"
},
{
"case": "candidate-project-shadow",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai"
}
}
],
"scanners": [
"skillspector"
],
"judgeConfiguredModel": null
},
{
"case": "candidate-inferred-provider",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "openai",
"SKILLSPECTOR_MODEL": "gpt-6.1-sol",
"SKILLSPECTOR_REASONING_EFFORT": "medium"
}
}
],
"scanners": [
"skillspector"
],
"judgeConfiguredModel": null
},
{
"case": "candidate-custom-other-provider",
"observed": [
{
"probe": "skillspector",
"controls": {
"SKILLSPECTOR_PROVIDER": "anthropic"
}
}
],
"scanners": [
"skillspector"
],
"judgeConfiguredModel": null
}
]
} |
|
@clawsweeper re-review The P2 fix and upgrade/image proof are ready at exact head |
|
🦞👀 Re-review progress:
|
clawsweeper
Bot
added
the
clawsweeper:intake:feature
|
Fresh current-head runtime proof is now complete: https://github.com/openclaw/clawscan/actions/runs/39361404351. Both native architecture jobs passed export/load, identity assertions and execution of all three installed CLIs. No provider credentials/network were used. Artifacts stamp GitHub synthetic merge amd64: arm64: Actual CLI help output and |
1 participant
Summary
judge.configuredModelrequest provenance; model-authored identity remains untrusted.Verification
go test -count=1 ./...,go vet ./..., docs build and 20 Node tests passed.service_tier: defaultobserved. Unpatched SkillSpector tool calls failed HTTP 400; patched Responses calls succeeded.efedcc6c67111eebd45862143b613d05ad7fa4a0. Fresh exact-head run: https://github.com/openclaw/clawscan/actions/runs/39361404351.Rollout boundary