Skip to content

docs(benchmark): refresh task-aware measurements - #143

Merged
SarthakWade merged 1 commit into
mainfrom
docs/refresh-benchmark-50
Aug 27, 2026
Merged

docs(benchmark): refresh task-aware measurements#143
SarthakWade merged 1 commit into
mainfrom
docs/refresh-benchmark-50

Conversation

@SarthakWade

Copy link
Copy Markdown
Collaborator

Summary

  • refresh the canonical benchmark with five fresh task-aware Linux ARM64 runs
  • update the README, benchmark methodology, roadmap, and quality contract with the current medians and honest point-in-time caveat
  • validate README claims and every methodology-table median against the generated benchmark document during web lint

Results

  • Headless warm: 174 estimated tokens, 3,239 ms wall time, 1,218 ms CPU, 379 MiB peak memory
  • Selenium: 410 estimated tokens, 2,828 ms wall time, 1,788 ms CPU, 378 MiB peak memory
  • Puppeteer: 499 estimated tokens, 2,400 ms wall time, 1,849 ms CPU, 367 MiB peak memory

Puppeteer remains fastest and lowest-memory. The refreshed claim is limited to the measured agent surface and median CPU result.

Validation

  • pnpm test
  • pnpm test:runtime
  • pnpm test:npm
  • pnpm --dir apps/web lint
  • pnpm --dir apps/web build
  • pnpm test:e2e:linux
  • ./apps/headless/build.sh
  • shell syntax checks and QA evidence checksum verification
  • Prettier 3.6.2 check for all changed files

Closes #50

@SarthakWade
SarthakWade merged commit aa57b3c into main Aug 27, 2026
8 checks passed
@SarthakWade
SarthakWade deleted the docs/refresh-benchmark-50 branch August 27, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

F4 — Refresh or caveat the stale benchmark numbers on the site

1 participant