Repository navigation
Reconcile the documentation with the tree, and unbreak CI - #1
Merged
Merged
Conversation
The conformance job asserted the pinned version by reading /usr/local/lib/node_modules/harper/package.json. That path is npm's default prefix, not a guarantee: the runner resolves somewhere else, so every run since the repo went public has failed at the version check with MODULE_NOT_FOUND — which reads as "Harper is missing" rather than "we looked in the wrong place". `npm root -g` is the question we actually meant to ask. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It listed "Correctness guards" under what it implements, on the grounds that the quote is deterministic. Determinism is necessary and is checked, but it is not a ground-truth check and this file was the one place in the repo saying otherwise — docs/plan.md has had it right all along: a consistently wrong answer is deterministic too. Moved to the not-implemented list, with what closing it takes. Three other gaps were listed as missing but have since been built: the containerized path, trial restore, and the cold/warm/hot split all exist in the orchestrator. Replaced with the gap that is actually left — one target means "identical for every target" is vacuous, not satisfied — and a note that running run.mjs directly gets you none of it. Dropped the ladder table. It described a dataset an order of magnitude smaller than the current one and a capacity an order of magnitude higher, because it was a second copy of numbers whose first copy moved. Observations now live only in docs/plan.md, dated and next to what may not be claimed from them. Row counts go the same way: MANIFEST.json is the contract, and a second copy of a contract is just a second thing to be wrong. Recorded the third time the generator was the bottleneck. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AGENTS.md says to keep this file current. It was four days and five commits behind: it reported the conformance suite as mostly stubs after the stubs were eliminated, listed filling it as the next task, called FRESH_MS "deliberately TBD" after SPEC.md set it provisionally, carried loyaltyBalance as unapplied after QUOTE-004 was rewritten around it, and said CI had never run while CI was failing on every push since the repo went public. One open item was not merely stale but backwards: it said the quote endpoint emits only `total` while the product endpoint decomposes. It is the other way round, and the real finding is sharper — the product aggregate omits the compute phase that OBS-001 requires of both endpoints, and the gate passes anyway because the quote's test also names OBS-001. Recorded as what it is: the coverage gate satisfied by the letter rather than the requirement. Also recorded: scripts/verify-run-record.mjs declares coverage of the run-shaped requirements and nothing executes it, so those requirements currently pass on a comment. Review-gate status was tracked both here and in SPEC.md §8, and the two disagreed in both directions. SPEC.md §8 is now the only place it lives. Counts are gone — of requirements, of tests, of stubs, of rows. Every one of them was wrong within days of being written, and every one has a source that cannot drift. Measurements keep their numbers; tallies do not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README said the conformance suite was incomplete after it was completed, stated a requirement count that had moved, and opened its quickstart by telling every newcomer to regenerate the committed dataset in place — two sections above the line explaining that implementations do not re-derive it, and in a repo whose CONTRIBUTING calls regenerating it a heavy change. The dataset is committed; there is nothing to generate. data-model.md's schema block had drifted out of agreement with its own prose: it explained at length why promotions carry ['*'] sentinel key arrays, then listed a Promotion type without them, with @indexed still on the authoritative arrays that are deliberately not indexed. It also omitted @Sealed throughout and omitted ProductView entirely — the cache is the one piece of the Harper design SPEC.md does not describe, so this document is the only place it was going to be described, and it wasn't. The block now matches schemas/store.graphql field for field. structure.md called the repo root the Next.js app root, and asserted that measurement scaffolding "does not live here at all" with bench/ and containers/ sitting beside it. Both corrected; the second kept as a recorded decision with the reason it changed, since the original intent still holds and the location is the part that moved. Its layout block had also fallen behind the tree. future-work.md said the Next.js wiring was retained but inert; structure.md said it was removed. Removed is right — there is no next.config.ts and nothing in package.json. CONTRIBUTING counted the checks and got a number that is no longer true. It now says what they check instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Several documents explained a design by narrating the thing it replaced: "an earlier version drew keys uniformly", "that single fact broke this twice", "the first version scheduled every arrival up front". A reader who was not here for that learns a sequence of states and has to work out which one is current. Each is now stated as the constraint it is. The uniform-draw story becomes why the access distribution is skewed; the two broken promotion lookups become the two obvious implementations and why neither works; the harness's three memory failures become the three properties that keep the generator out of the measurement. Nothing substantive is dropped — the reasoning is the point, and it reads better without the chronology. data-model.md no longer opens by relitigating a reversed decision: resolve-on- read is argued on its own merits, with the condition that would change it. structure.md no longer justifies the e2e exclusion by a Next.js version that is no longer in the tree. Optional peer dependencies defeating --omit=dev is the durable fact, and it holds for any framework that declares one. future-work.md described deferred scope as things that "was" — now it says what each is and what decision applies when it returns, which is what the file is actually for. plan.md drops its Done list. The status table above it already states what state every piece is in, and a changelog beside a status table is two places to disagree. Git has the history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We pushed to the public repo at a decent point in the implementation, so I did a pass over every document in it to check what still matched reality. Four things, in order of how much they cost us.
CI has been red since the repo went public
The conformance job asserted Harper's pinned version by reading
/usr/local/lib/node_modules/harper/package.json. That is npm's default prefix, not a guarantee — the runner resolves elsewhere, so every run failed at the version check withMODULE_NOT_FOUND, which reads as "Harper is missing" rather than "we looked in the wrong place". It now asksnpm root -g.No document knew this.
docs/plan.mdstill listed the GitHub repo as a blocker and CI as "written, never executed".The harness README claimed a guard the harness does not have
It listed Correctness guards under what it implements, on the grounds that the quote is deterministic. Determinism is necessary and is checked, but it is not a ground-truth check, and
docs/plan.mdhas had this right all along: a consistently wrong answer is deterministic too. For a repo whose whole discipline is not claiming what it cannot support, this was the one that mattered. Moved to the not-implemented list with what closing it takes.Three other bullets listed gaps that have since been built — the containerized path, trial restore, and the cold/warm/hot split all exist in the orchestrator. Replaced with the gap actually left: one target means "containerized identically for every target" is vacuous rather than satisfied.
Also dropped its ladder table, which described a dataset an order of magnitude smaller than the current one and a capacity an order of magnitude higher. It was a second copy of numbers whose first copy moved.
The plan was four days and five commits behind
AGENTS.mdsays to keep it current. It reported the conformance suite as mostly stubs after the stubs were gone, listed filling it as the next task, calledFRESH_MS"deliberately TBD" after SPEC.md set it provisionally, and carriedloyaltyBalanceas unapplied after QUOTE-004 was rewritten around it.One item was backwards rather than merely stale, and the corrected version is a real finding: the product endpoint omits the compute phase
OBS-001requires of both endpoints, and the coverage gate passes anyway because the quote's test also namesOBS-001. That is the gate satisfied by the letter rather than the requirement, which is the failure mode the gate exists to prevent.Related, and also now recorded:
scripts/verify-run-record.mjsdeclares coverage of the run-shaped requirements and nothing executes it — notnpm run check, not CI, notrun-benchmark.sh. The gate scans itsCOVERS:line statically, so those requirements currently pass on a comment. The split is right; the wiring is missing.Remaining drift
docs/data-model.mdexplained at length why promotions carry['*']sentinel key arrays, then listed aPromotiontype without them, with@indexedstill on the authoritative arrays that are deliberately not indexed. It also omitted@sealedthroughout and omittedProductViewentirely — the cache is the one piece of the Harper design SPEC.md does not describe, so this document was the only place it was going to be described, and it wasn't. The block now matchesschemas/store.graphqlfield for field.docs/structure.mdcalled the repo root the Next.js app root, and asserted measurement scaffolding "does not live here at all" withbench/andcontainers/sitting beside it.docs/future-work.mdsaid the Next.js wiring was retained but inert;structure.mdsaid it was removed. Removed is right.On counts
I took tallies out of the docs — requirements, tests, stubs, checks, row totals — and pointed each at a source that cannot drift:
SPEC.mdand the registry,npm run check, each dataset'sMANIFEST.json. Every one of those numbers was wrong within days of being written, and thedevrow count had drifted silently when the generator was fixed.Measurements keep their numbers: the ladder latencies, the 2.5% → 77.7% cache-hit finding, the scrypt memory table. Those are dated experimental records, not status.
Two things I did not decide
npm run checkpasses.🤖 Generated with Claude Code