Skip to content

Commit 3247ce4

Browse files
committed
docs(qa): finalize log-volume census — 72/72 packages, ratio does not hold
All 72 workspace packages with a test script measured, all green. Final totals: 61,980 structured / 17,428 console / 993 reporter / 80,401 total lines — structured share of total 77.1% (comparable to the earlier five-suite framing, whose own `console` bucket absorbed what this instrument tracks separately as `reporter`). The five-suite ~45% figure does not hold, and not toward more noise: it moved to 77.1% structured, decomposed in "The answer" section as two effects — (1) two commits (#13985, #14016) that landed on main between the earlier reading and this one already took the original five suites from ~45% to 86.6% structured on their own, by cutting a `[Registry]` debug-console population; (2) extending to the other 67 packages pulls that back down to 77.1% (not further), since that population's own structured share (57.7%) is lower than the five heaviest suites' but is also a minority (30.3%) of total test-run volume, and most of it (35/72 packages under 30 lines total, 22 of those exactly 13 -- pure reporter boilerplate) never boots a kernel at all. No seam added -- per triage's ruling (comment 5489397004), this document is the measurement; which seam (if either) to build is triage's call. Merged origin/main (dispatch-gates warned STALE TREE, 19 commits behind) -- no relevant file (logger.ts, harness.ts, log-level.ts, vitest configs, package.json test scripts) changed across that range, so the merge does not invalidate any measurement already taken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68
1 parent 782c724 commit 3247ce4

1 file changed

Lines changed: 92 additions & 27 deletions

File tree

docs/audits/2026-09-test-log-volume-census.md

Lines changed: 92 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Test-run log volume census — both channels, all 72 packages
22

3-
**Measured on `origin/main` `PLACEHOLDER_SHA`, 2026-09-03.** Instrument:
3+
**Measured on `origin/main` `b1d49b394`, 2026-09-03T11:41 UTC.** Instrument:
44
`scripts/qa/log-volume-census.mjs` (this document states nothing the instrument
55
cannot be re-run to produce).
66

@@ -12,9 +12,16 @@ stating before any number:
1212
> a test run writes — hold across the other 67 packages?
1313
1414
**Answer: no, and not in the direction the phrasing suggests.** Across all 72
15-
packages the structured logger is not ~45% of the output. It is
16-
`PLACEHOLDER_SHARE`% — and the reason is that the *other* half moved. See
17-
[The answer](#the-answer).
15+
packages the structured logger is **77.1%** of the output (61,980 of 80,401
16+
lines) — not lower than the five-suite reading, higher. The other 67 packages
17+
are not what moved it: two commits that landed on `main` in the three days
18+
between the earlier reading and this one already took the *original five*
19+
suites from ~45% to ~87% structured on their own, by cutting a debug-only
20+
`console.log` population out of four of them. Extending to all 72 packages
21+
pulls that back down to 77.1%, not because the extra 67 reverse the direction,
22+
but because they are collectively a smaller, more console-leaning slice of
23+
total volume than the five heaviest suites already were. See
24+
[The answer](#the-answer) for the arithmetic.
1825

1926
## What is NOT claimed here
2027

@@ -232,13 +239,7 @@ instrument drift.
232239

233240
## The census
234241

235-
**IN PROGRESS — 68/72 measured, this is a checkpoint commit, not the
236-
final reading.** Continuing under the shared verify lock with tighter
237-
per-batch budgets so the lock cycles for other agents on this container;
238-
the ledger this table is generated from is
239-
`/tmp/os-log-volume-census/ledger.json`, one row per package written the
240-
moment that package finishes, so no completed measurement is lost if this
241-
run is interrupted.
242+
**All 72/72 packages measured.**
242243

243244
| package | console | structured | reporter | total | exit |
244245
|---|---:|---:|---:|---:|---|
@@ -309,28 +310,92 @@ run is interrupted.
309310
| `packages/services/service-sms` | 0 | 0 | 13 | 13 | ok |
310311
| `packages/services/service-storage` | 113 | 218 | 13 | 344 | ok |
311312
| `packages/spec` | 15 | 0 | 17 | 32 | ok |
312-
| `packages/triggers/trigger-api` | | | | | **NOT MEASURED** |
313-
| `packages/triggers/trigger-record-change` | | | | | **NOT MEASURED** |
314-
| `packages/triggers/trigger-schedule` | | | | | **NOT MEASURED** |
315-
| `packages/types` | | | | | **NOT MEASURED** |
313+
| `packages/triggers/trigger-api` | 0 | 0 | 13 | 13 | ok |
314+
| `packages/triggers/trigger-record-change` | 401 | 0 | 13 | 414 | ok |
315+
| `packages/triggers/trigger-schedule` | 0 | 0 | 13 | 13 | ok |
316+
| `packages/types` | 1 | 0 | 13 | 14 | ok |
316317
| `packages/verify` | 221 | 3,171 | 13 | 3,405 | ok |
317-
| **total, 68/72 suites measured** | **17,026** | **61,980** | **941** | **79,947** | |
318+
| **total, 72/72 suites measured** | **17,428** | **61,980** | **993** | **80,401** | |
318319

319-
structured share of total (structured / (structured+console+reporter)): 77.5%
320+
structured share of total, structured / (structured+console+reporter) —
321+
comparable to the 5-suite framing, whose own `console` already absorbed what
322+
this instrument tracks separately as `reporter`: **77.1%**
320323

321-
structured share of console+structured (comparable to the 5-suite framing): 78.4%
324+
structured share of console+structured alone, reporter set aside: 78.1%
322325

323-
measured: 68/72 — NOT MEASURED (4): `packages/triggers/trigger-api`, `packages/triggers/trigger-record-change`, `packages/triggers/trigger-schedule`, `packages/types`
326+
measured: 72/72
324327

325328
non-zero exit: 0
326329

327330
## The answer
328331

329-
Not yet — see "IN PROGRESS" above. Provisional read at 68/72 (do not cite):
330-
structured share of console+structured is ~78%, nowhere near the
331-
five-suite ~45%, in the direction the earlier reading's own explanation
332-
predicts (see "What is NOT claimed here" / the five-suite table above): most
333-
of the *unmeasured* population never boots a kernel at all, so it was always
334-
going to skew toward `console`-only or near-silent, not toward more structured
335-
share. The final numbers replace this paragraph once all 72 (or a declared
336-
NOT-MEASURED-with-reason subset) are in.
332+
**No — the ratio does not hold, and it moved in the opposite direction from
333+
the one "the other 67 packages are noisier" would predict.**
334+
335+
All 72 packages, all green (exit 0), totals from the ledger:
336+
337+
| | console | structured | reporter | total |
338+
|---|---:|---:|---:|---:|
339+
| **all 72 packages** | 17,428 | 61,980 | 993 | 80,401 |
340+
341+
- **structured share of total** (the metric comparable to the earlier
342+
reading's own convention, where `console + structured == total` with no
343+
separate reporter bucket — i.e. `structured / (structured + console +
344+
reporter)`): **77.1%**.
345+
- **structured share of console+structured alone** (reporter's own ~1% of
346+
total set aside): 78.1%. The two are close because `reporter` is a small,
347+
closed vocabulary (993 of 80,401 lines, 1.2%) — see Method.
348+
349+
**Why it moved this far, decomposed:**
350+
351+
1. **The original five suites, re-measured on today's tree, are already at
352+
86.6% structured** (48,426 structured / 55,925 console+structured — from
353+
the "Reproduction" table above), not ~45%. Two commits explain essentially
354+
all of that move: `b79ddf17d` (#13985) and `5e2c04da7` (#14016) each
355+
declared `OS_REGISTRY_LOG=warn` in a suite's vitest harness, cutting a
356+
`[Registry]` debug-`console.log` population that had been the majority of
357+
`console` output in dogfood, objectql, runtime and verify. `packages/rest`
358+
— the one suite of the five nothing touched — moved only +5.8% on
359+
`console`, which is what makes the other four attributable to those two
360+
commits rather than to this instrument reading differently than the
361+
original one did.
362+
2. **Extending to the other 67 packages pulls the number back down, from
363+
86.6% to 77.1%, but not remotely far enough to reverse it.** Those 67
364+
packages contribute 24,396 of the 80,401 total lines (30.3%) — collectively
365+
a minority of test-run volume — and most of them are near-silent either
366+
way: **35 of the 72 packages write fewer than 30 lines total**, and **22
367+
of the 72 write exactly 13** — the reporter's own fixed banner with
368+
nothing else at all (a package whose suite has no tests that emit
369+
anything, structured or console). Structured lines require a kernel boot
370+
(`ObjectLogger`'s INFO-level startup chatter); a package whose suite never
371+
constructs one — most of `packages/services/*`, `packages/triggers/*`,
372+
`packages/drivers/*` (besides `driver-memory`), and several thin
373+
`plugins/*` — writes at or near zero of both.
374+
3. **The marginal 67 packages' own structured share is 57.7%** (13,554
375+
structured / 23,483 console+structured among just that group) — lower
376+
than the reproduced five-suite figure, higher than the original ~45%
377+
reading. The largest genuinely console-**majority** contributors outside
378+
the original five (`console > structured`, sorted by `console`):
379+
`packages/plugin-auth` (2,678 console / 2,149 structured — its own suite
380+
boots a kernel and logs per-request auth denials), `packages/plugin-sharing`
381+
(633 / 149), `examples/app-todo` (482 / 0), `packages/plugin-approvals`
382+
(428 / 300) and `packages/trigger-record-change` (401 / 0). `packages/cli`
383+
has the second-highest raw `console` count outside the original five
384+
(1,608) but is itself majority-**structured** (4,926) — it belongs to the
385+
`console`-heavy-in-absolute-terms group, not the console-majority one.
386+
387+
**Reading the two effects together:** the five-suite figure this card cited
388+
(~45%) was a snapshot from *before* #13985/#14016 landed. Re-running the same
389+
five suites today already answers most of the question — the ratio was never
390+
stable at 45%, because the population it measured moved out from under it in
391+
three days. The extension to all 72 packages is the second, smaller
392+
correction, and it is a real one: the untouched 67 packages are more
393+
console-leaning on average (57.7%) than the five heaviest suites (86.6%), so a
394+
full-repo reading is not simply "the five-suite number, unchanged." But at no
395+
point does the combined population cross back toward parity, let alone toward
396+
`console` being the majority — it stays firmly structured-dominated (77.1%)
397+
throughout.
398+
399+
**No seam was added.** Per triage's ruling, this document is the measurement
400+
only; which of the two candidate seams (if either) to build is triage's call,
401+
made with this table in hand.

0 commit comments

Comments
 (0)