feat: add experimental coverage command - #2993
Open
ariesclark wants to merge 18 commits into
Open
Conversation
🦋 Changeset detectedLatest commit: 5990dea The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Author
|
It really is endless. 😓 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit dc7f6b4. Configure here.
Contributor
|
Thanks for the contribution |
adamaltman
reviewed
Jul 31, 2026
Comment on lines
+22
to
+24
| This is the opposite direction from [`drift`](./drift.md). | ||
| `drift` judges the traffic against the description and reports what disagrees; it is silent about a description that is never put to the test. | ||
| A `drift` run with no findings is only as meaningful as the share of the description the traffic actually covered, and that share is what `coverage` measures. |
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What/Why/How?
Note
This was heavily assisted by Anthropic's Opus 5. I've reviewed the code to the best of my ability, but if there's any obvious issues I didn't catch, let me know.
driftreports where traffic and the description disagree, but says nothing about the parts no traffic reached. A cleandriftrun means only that nothing was wrong in the portion the traffic covered.coveragemeasures that portion.The command parses a traffic log, matches each exchange to a documented operation through
drift's matcher, walks each body against the schema that describes it, and reports what nothing reached: operations, properties,oneOf/anyOfbranches, and component schemas.It adds no runtime dependencies. Spec loading reuses
@redocly/openapi-core; traffic parsing, operation matching, and mime selection come fromdrift.It bundles the description a second time with
dereference: false.drift's loader dereferences, deep-cloning every$reftarget, so nothing underpathsshares identity withcomponents.schemas. Without that identity a value cannot be traced back to the component it came from, and coverage is reported per component.Reference
Stacked on #2992, #2994. This branch includes that commit until it merges.
Testing
Unit tests cover the engine:
$refresolution, declared properties, union-site enumeration, branch matching, walking, and summarizing. E2E runs the built CLI over a HAR fixture in stylish, JSON,--all, and--schemamodes, plus the missing-traffic-path case. The fixture composesUserwithallOfso inherited properties are credited to the schema that declares them.Check yourself
Security
The command reads traffic logs and a description, both already handled as untrusted input by
drift, and writes the report only to the path given by--output.Note
Medium Risk
Large new experimental command reuses drift matching and shared path-regex logic, so regressions could affect drift routing; behavior is heavily tested but the feature surface (schema walking, unions, dual spec load) is non-trivial.
Overview
Introduces the experimental
coveragecommand: it ingests the same traffic formats asdrift, matches exchanges viadrift’s matcher, walks request/response bodies against documented schemas (withdereference: falsebundling so coverage can attribute hits to component schemas), and reports unexercised operations, parameters, responses, properties, union branches, and schemas in stylish or JSON output (--schema,--all,--output).driftpath compilation now supports segments that mix literals and multiple parameters (e.g./instances/{worldId}:{instanceId}), with scoring so more specific templates win—fixing missed matches for bothdriftandcoverage.respect --har-outputnow populates HARpostData(andbodySize) for string/URLSearchParamsbodies viabuildPostData, so replays can validate request bodies.Docs, changesets, unit/e2e tests, and a root
/coverage/.gitignoreentry (so Vitest reports don’t hide the command source tree) round out the change;driftloader helpers are exported forcoveragereuse.Reviewed by Cursor Bugbot for commit 9f52750. Bugbot is set up for automated code reviews on this repo. Configure here.