Repository navigation
Benchmarks: against OpenCode 2.0 - #23
Conversation
OpenCode 2.0.27 against Pi-Bolt 0.8.5 on Linux and macOS with the README's tools: bench/opencode.sh runs it from Pi's arguments against the local model (--standalone, compaction off), and the harness waits for OpenCode's command menu before the Enter of /quit. Raw data in bench/results/2026-10-12-*-opencode-2.0.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02636aa56b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Pi-Bolt 0.8.5 in a run of its own on the same machines, with the same tools and the same local model. `bench/opencode.sh` turns | ||
| the tools' Pi arguments into OpenCode's and writes its configuration: an OpenAI-compatible provider pointing at the local model, | ||
| and automatic compaction off, so that a long session sends the whole conversation as the others do. OpenCode runs with | ||
| `--standalone`, a server of its own started by the process, so that the server's CPU is counted (`wait4` includes it) and each run |
There was a problem hiding this comment.
The wait4 claim does not hold for the published long-answer and long-session CPU figures. long_answer.py and long_session.py calculate deltas with cpu_ms(tty.pid) while OpenCode is running, and that function reads only the main PID's threads; the server subprocess is therefore omitted. Although Tty.quit() later returns wait4 usage, neither script uses it for these per-answer or per-prompt metrics. The OpenCode values in rows 255, 257, and 260 are consequently understated; sample the entire process tree for these live deltas before publishing them.
Useful? React with 👍 / 👎.
|
|
||
|
|
||
| <h2>Against PiG and oh-my-pi</h2> | ||
| <h2>Against PiG, oh-my-pi and OpenCode</h2> |
There was a problem hiding this comment.
Renaming this heading changes the generated site anchor from #against-pig-and-oh-my-pi to #against-pig-oh-my-pi-and-opencode, but docs-site/pages/faq.html and the generated FAQ still link to the old anchor. Those FAQ links now open the benchmarks page without navigating to the comparison section. Preserve the old explicit ID or update all inbound links.
Useful? React with 👍 / 👎.
Adds OpenCode 2.0.27 to the README tables, BENCHMARKS.md and the site's Benchmarks page, measured next to Pi-Bolt 0.8.5 on the same Linux and macOS machines with the same tools. bench/opencode.sh runs OpenCode from the tools' Pi arguments (--standalone, so its server's CPU is counted; compaction off, so a long session sends the whole conversation); the harness waits for OpenCode's command menu before the Enter of /quit. OpenCode's memory figures leave out its server process, and it has no tmux figures (it did not finish its answers there); both are noted in the tables.