From 458fa8149557f70bda55bffbe1a1cb826986f495 Mon Sep 17 00:00:00 2001 From: Walter Pinto Date: Sat, 3 Oct 2026 14:17:54 +0100 Subject: [PATCH] docs: add the raw lcov coverage report contract Document where each test rule keeps its raw lcov report, the format, what each rule set provides, how to classify covered, uncovered and partly covered lines, and how to read and merge the files. Phase 3 of #46 (the contract; the consumer lives in Veritas). Refs #46 Co-Authored-By: Claude Sonnet 5.5 --- docs/README.md | 3 ++ docs/coverage-lcov.md | 117 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+) create mode 100644 docs/coverage-lcov.md diff --git a/docs/README.md b/docs/README.md index 5a3d77c..4401734 100644 --- a/docs/README.md +++ b/docs/README.md @@ -134,3 +134,6 @@ please-rules/ - **[Veritas Code Quality & Telemetry](veritas.md)**: Local static analysis and automated test telemetry configuration. +- **[Raw lcov Coverage Reports](coverage-lcov.md)**: where each test rule keeps + its function and branch coverage, the format, per-language limits, and how to + classify covered, uncovered and partly covered lines. diff --git a/docs/coverage-lcov.md b/docs/coverage-lcov.md new file mode 100644 index 0000000..1678294 --- /dev/null +++ b/docs/coverage-lcov.md @@ -0,0 +1,117 @@ +# Raw lcov Coverage Reports + +`plz cover` reduces every language's coverage to per-line hit counts in +`plz-out/log/coverage.xml` and `coverage.json`. The test rules of this +repository also keep the coverage tool's own report as **lcov**, which carries +what the merged report drops: function records and branch records. This page is +the contract for consumers, such as an annotated source view or a coverage +dashboard. + +--- + +## Where the files are + +Each test target writes one file, declared as a test output: + +```text +plz-out/bin//.lcov +``` + +| Rule | Language | Example | +| :------------------------ | :--------- | :---------------------------------------------------- | +| `ts_test` | TypeScript | `plz-out/bin/test/ts/lib/branches_test.lcov` | +| `kotlin_test` (`kt_test`) | Kotlin | `plz-out/bin/test/kotlin/branches/branches_test.lcov` | +| `rust_test` | Rust | `plz-out/bin/test/rust/branches/branches_test.lcov` | +| `swift_test` | Swift | `plz-out/bin/test/swift/branches/branches_test.lcov` | + +Find them with `plz-out/bin/**/*.lcov`. Go and Python tests come from the +upstream `go-rules` and `python-rules` plugins, which do not write lcov: Go +writes its native block profile and Python writes line-only Cobertura XML to +`$COVERAGE_FILE`, where Please reads it. + +### Lifetime + +- The file is only filled when the test runs under `plz cover`. A plain + `plz test` leaves an empty file. +- It reflects the last test run that actually executed. Produce fresh data with + `./pleasew cover --rerun ` and read the files right after. +- Do not assume the file is recreated when Please reuses a cached result: a + deleted file stays deleted. Treat a missing or empty file as "no data for this + target", not as "no coverage". + +--- + +## Format + +Standard lcov tracefile, one record per source file: + +| Record | Meaning | +| :--------------------- | :-------------------------------------------------------------------------------------- | +| `SF:` | Source file, **repository-relative** and slash-separated | +| `FN:,` | Function `n` starting at `line` | +| `FNDA:,` | Function `n` ran `c` times | +| `DA:,` | Executable line ran `c` times (`0` = never) | +| `BRDA:,,,` | Branch arm `a` of block `b` on line `l`, taken `t` times, or `-` if the block never ran | + +Lines without a `DA` record are not executable. Summary records (`LF`, `LH`, +`FNF`, `FNH`, `BRF`, `BRH`) are written for convenience and can be recomputed. + +### What each rule set provides + +| Rule set | Line counts | Functions | Branches | +| :---------------- | :---------- | :--------------------------------------------------- | :------------------------------------------------------------------------------------------- | +| ts (Deno, Vitest) | real counts | `FN`/`FNDA` | real `BRDA` from V8 coverage | +| kotlin | 0 or 1 | one per method, `..` | per line counts of arms that ran and did not; arms are anonymous (JaCoCo does not say which) | +| rust | real counts | `FN`/`FNDA`, v0-mangled names (`_R...`) | synthesized for partly executed lines (see below) | +| swift | real counts | `FN`/`FNDA`, Swift-mangled names, closures separate | synthesized for partly executed lines (see below) | + +**Synthesized branches (Rust, Swift).** Neither compiler emits branch regions on +stable, so the branch records are derived from `llvm-cov`'s sub-line regions: a +line that ran but contains a region that never did (the right-hand side of +`a && b`, the body of a one-line `if`) gets one `BRDA` arm per region starting +on that line, in column order. This is a heuristic. A branch whose arms are on +separate lines shows up as an uncovered line, not as a partial one, and lines +whose sub-line regions all ran carry no `BRDA` record. + +**Do not compare branch arm numbers across tools.** Two tools number the arms of +the same branch point differently (the Deno and Vitest runners do). Merge +records of the same file only when they come from the same tool. + +--- + +## Classifying a line + +For an annotated source view, classify each source line from its records: + +| Status | Rule | +| :----------------- | :------------------------------------------------------------------ | +| **Not executable** | no `DA` record for the line | +| **Uncovered** | `DA` count is `0` | +| **Partial** | `DA` count is above `0` and a `BRDA` arm on the line has `0` or `-` | +| **Covered** | `DA` count is above `0` and no arm on the line was missed | + +Function coverage comes from `FNDA` (above `0` = ran); for tools without +function records, join the line statuses with the function's line range. + +Show the source of the same revision the tests ran on, otherwise the line +numbers drift. + +--- + +## Reading and merging + +The reference implementation is the Go package +[`tools/common/lcov`](../tools/common/lcov), used by the test runners to write +these files. It provides: + +- `lcov.Parse` and `Report.Write`: read and write tracefiles (output is sorted + and deterministic; summary records are recomputed); +- `lcov.Merge`: sum the counts of several reports, for example the reports of + several test targets. Lines match by number, functions by name, branches by + `(line, block, arm)`. A line is covered after merging if any target ran it, + and the merged report cannot say which target covered it; +- `File.LineStatus`: the classification above; +- `RelPath` and `Report.NormalizePaths`: repository-relative paths. + +The format is plain lcov, so a consumer outside this repository can use any lcov +parser and apply the rules above.