Capture Copilot assistant usage events#21
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the pull-request-dashboard telemetry scripts to capture additional Copilot token-usage metrics (cache read, cache write, and reasoning tokens) and to recognize the Copilot CLI's assistant.usage JSONL event as a usage source. Previously these tokens and the dedicated usage event were not captured, so the dashboard under-reported token totals. The change is well-contained: it adds the three new fields wherever existing token fields are handled, keeping the ingest (classification.py) and aggregation/reporting (dashboard.py) layers in sync.
Changes:
- Added
cache_read_tokens,cache_write_tokens, andreasoning_tokensto the usage field aliases and normalization inclassification.py, and mirrored them in the dashboard's empty/aggregate/print helpers indashboard.py. - Reworked
parse_copilot_jsonlto readevt["data"]once and treat anassistant.usageevent'sdataas the usage payload, with a fallback todata["usage"].
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
.github/scripts/pull-request-dashboard/classification.py |
Adds new token-field aliases, normalizes them, and parses assistant.usage events as a usage source. |
.github/scripts/pull-request-dashboard/dashboard.py |
Mirrors the three new token fields in the usage initializer, aggregator, and summary printer. |
I did not identify concrete, objective defects in the diff: the new fields are consistently threaded through both files, missing keys are guarded with or 0, and per-event usage selection avoids within-event double counting. The one aspect I could not verify is the external contract of the Copilot CLI's assistant.usage event (e.g., whether it is emitted as a cumulative running total or alongside per-message usage), which the cross-event summation in parse_copilot_jsonl depends on for correct totals.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Last PR didn't report everything.
Capture token usage from Copilot CLI assistant.usage JSONL events, including cached and reasoning token fields, so the pull request dashboard reports exact token totals when the CLI emits its current usage event shape.