From cd5ba1bbdde91cdaf4fa58280ed0094c608a2a6e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Tue, 4 Aug 2026 10:53:56 +0900 Subject: [PATCH 1/3] knowledge: ingest 1 verified insight (qa/deliverables) --- .dev-loop/INGEST_REPORT.md | 136 +++++++----------- INDEX.md | 24 ++-- log.md | 1 + wiki/frontend/state/derived-state.md | 29 ++-- ...nerated-artifacts-as-deliverable-source.md | 79 ++++++++++ .../spec-document-gates.md | 50 ++++--- wiki/qa/index.md | 45 +++--- 7 files changed, 214 insertions(+), 150 deletions(-) create mode 100644 wiki/qa/deliverables/generated-artifacts-as-deliverable-source.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index c375750..9d86f55 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,95 +1,61 @@ -# Consolidated review — knowledge PRs #6–#13 +# Knowledge flush — 1 insight -Eight fork PRs (`dch0202-rsquare`, 2026-07-28 → 2026-08-02) were reviewed together -against `AGENTS.md`. Each PR was audited by an independent reviewer (format rules, -sources, vague-qualifier ban, ≤120 body lines, index/log invariants), then -cross-compared to catch duplication the per-PR flushes could not see — they branched -independently off the same main and rewrote the same shared index/log files. Fork -branches can't be edited from here and several PRs needed content changes (drop a -duplicate, merge a colliding page), so this branch carries the reconciled end-state -rather than merging each PR as-is (which would import the duplicates). +Queue drained: 1 candidate from `~/.dev-loop/queue/c2113b9a-…jsonl`. The other 10 session files were empty. ## Verified best-practice -Sources are per-page and were live-verified in each originating PR's flush; the -independent re-reviews re-checked them. Landed pages and their evidence base: - -| Page | Confidence | Source basis | -|------|-----------|--------------| -| backend/common/llm/completion-response-validation | verified | OpenAI reasoning guide + chat `object` spec (5 `finish_reason` values), vLLM/LiteLLM reasoning fields; field incident (200/`length`/empty content/8,173-char reasoning) | -| backend/common/llm/context-window-budget | verified | Claude context-window docs, LiteLLM exception mapping, vLLM/Claude Code env-var docs | -| backend/common/integrations/externally-owned-defaults | verified | OpenAI deprecations (notice windows) + models `list`, LiteLLM model_discovery; field incident (alias removed between PR verify and review → 400) | -| backend/common/storage/object-key-persistence | verified | AWS S3 CompleteMultipartUpload + managed-upload API/source, aws-sdk-js issues #1158/#5656 | -| infrastructure/containers/host-cgroup-visibility | field-tested | cgroup_namespaces(7), Docker `--cgroupns=host`, nsenter, k8s #103363; OrbStack repro | -| infrastructure/observability/missing-container-metrics | verified/field-tested | k8s resource-metrics-pipeline docs, kube-prometheus-stack values, kubernetes-mixin; OrbStack #2217 repro | -| platforms/environment/unicode-text-matching | verified | UAX #15, Unicode core §3.12, APFS FAQ, POSIX grep; local repro (macOS 15/APFS, grep 2.6.0-FreeBSD, Python 3.13) | -| platforms/shells/command-text-inspected-before-execution | verified | Claude Code hooks docs, POSIX shell §2.6; local reproduction | -| platforms/processes/non-interactive-cli-invocation | verified | GNU nohup, OpenBSD ssh/ssh_config, git, timeout man pages; no-request-in-gateway-log field incident | -| qa/document-verification/spec-document-gates | field-tested | ESLint, Google mutation testing, RFC 2119, Vale, markdownlint; 32/32 mutant / 62/62 intact RFC sessions | -| qa/document-verification/editing-a-gated-document | field-tested | pgrep, Vale, markdownlint; in-house editing methodology | -| testing/quality/checks-that-cannot-pass | verified | James Shore AoAD2, POSIX grep exit status, Semgrep rule-testing, pytest exit codes; BSD/ugrep measurement | -| testing/quality/spec-artifact-checks | verified | JSON Schema, ESLint RuleTester, pitest, GFM table spec; local cell-count repro + GitHub renderer cross-check | -| testing/quality/harness-reverse-controls | verified | mutation-testing + CI-control sources; field repro (re-fetched all cited URLs, PASS) | - -Three pages were reconciled from two overlapping PR versions each, keeping the more -complete/better-sourced body and folding in the other's unique cases: -- **completion-response-validation** — #12 body (all five `finish_reason` values, - `tool_calls`/`function_call` carve-out, streaming, Responses API, "reasoning is - scratch, not deliverable") kept in `llm/` (coherent with #6/#13); folded in #6's - DeepSeek first-party edge + the field incident. -- **externally-owned-defaults** — #12 generalized body (any repo-external resource) - in `integrations/`; folded in #6's alias-removed field incident + the - gateway-config-vs-live-upstream nuance. -- **non-interactive-cli-invocation** — #12 body (GNU-nohup extension precision, - ssh -n stdin-detach vs BatchMode, pre-log DNS/TLS/proxy + `curl -v`) kept; folded - in #11's DEBIAN_FRONTEND, pager/color TTY case, wrapper-CLI case, field incident. +**Claim (as queued):** when the repo already generates an artifact (ERD, schema doc, API spec) and someone asks for the same content as a hand-off deliverable, find and re-run the generator and use its output as the body, hand-writing only what the generator cannot express. + +**Sources checked** + +| Source | What it supports | +| --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| [Write the Docs — Documentation principles](https://www.writethedocs.org/guide/writing/docs-principles/) | "Eliminate content overlap between separate sources"; scopes must be "clearly defined and disjoint" to prevent "parallel maintenance (or worse — _lack_ of maintenance) of the same information across multiple sources" | +| [OpenAPI — Best Practices](https://learn.openapis.org/best-practices.html) | A single description acting as the single source of truth for both the product and its docs | +| [Google — Documentation Best Practices](https://google.github.io/styleguide/docguide/best_practices.html) | Keep documentation sources next to the code they document so both change together | + +**How it was verified.** Fetched the Write the Docs principles page directly rather than relying on the search summary, because the two disagreed in a way that matters. The page **hedges against** a naive reading of this insight: _"In an ideal world, an automated system would generate documentation from the software's source code … Unfortunately, today, the best documentation is hand-written"_, and it notes generators "still require input from humans to function." + +That hedge is load-bearing, so the page does **not** claim generation beats writing. It splits the deliverable instead: + +- mechanically checkable facts (column sets, key names, endpoint lists, counts) → the generator's output verbatim +- judgement (why a table exists, what is out of scope, caveats) → hand-written, marked as such + +Framed that way the directive is supported by all three sources, and "prose quality is the deliverable (tutorial, onboarding guide)" is carved out as an explicit edge-case row pointing back to hand-writing. + +**Confidence: `verified`.** The single-source-of-truth / no-content-overlap principle is cited from primary sources. The operational specifics (re-run even when the committed output is stale; land the refresh as its own commit) rest on the field evidence below and are labelled as field evidence in the page's Sources section. + +**Field evidence** (2026-08, monorepo hand-off, carried from the queue row): a hand-written schema deliverable matched the live database on **3 of 19** cross-checked tables, listed **2 tables that do not exist**, and rendered **all 14 primary keys as `id`** against an actual `_id` convention. The repository's committed generated ERD matched the live database on the same tables; one generator re-run refreshed **97 → 105** tables. ## Existing-layer check -Cross-PR and against-main duplication was the focus. Findings and resolutions: - -- **spec-artifact-checks (#8) ≡ document-conformance-checks (#9)** — same case - (coverage-vs-validity split, per-check negative controls, GFM pipe parsing, - ESLint/Semgrep/mutation examples). #9's report predated awareness of #8. → - **#8 kept canonical; #9's page dropped, `testing/docs-as-spec` category not created.** -- **completion-response-validation (#6) ≈ llm-response-completeness (#12)** — ~95% - same case (HTTP 200 ≠ usable output; `length`/blank/reasoning-budget). → - **merged into one `llm/` page; #12's `integrations/` copy dropped.** -- **gateway-model-alias-defaults (#6) ≈ externally-owned-defaults (#12)** — ~80%; - #12 generalizes the model-alias case to any external resource. → - **kept the general `integrations/` page; #6's LLM-only page dropped.** -- **non-interactive-cli-invocation** — created by BOTH #11 and #12 (file collision). - → **single reconciled page.** -- Distinct (no overlap, all landed): checks-that-cannot-pass, harness-reverse-controls, - spec-document-gates, editing-a-gated-document, unicode-text-matching, - command-text-inspected-before-execution, object-key-persistence, context-window-budget, - host-cgroup-visibility, missing-container-metrics. -- Reciprocal `related:` links added on existing pages (tests-that-cannot-fail, - timeouts-and-retries, environment-config, release-gates, background-services, - portable-shell-scripts, timezone-and-locale, paths-case-and-line-endings, - acceptance-criteria, resource-limits-and-probes, logs-metrics-signals, - minimum-case-set). A dropped-page backlink (#6 → gateway-model-alias-defaults on - environment-config and release-gates) was retargeted to externally-owned-defaults. -- Invariants verified programmatically: all `related:`/inline `[id]` references - resolve, every page listed in its domain index, no duplicate ids, no page >120 - body lines. +**Pages read** + +- `INDEX.md` (domain routing), `wiki/qa/index.md`, `wiki/databases/index.md` (the queue row proposed `domain: databases`) +- `wiki/qa/document-verification/spec-document-gates.md`, `wiki/qa/document-verification/editing-a-gated-document.md` — the only existing document-focused pages +- `wiki/frontend/state/derived-state.md` — surfaced by a repo-wide grep for the same underlying principle + +**Overlap search.** `grep -rniE "regenerat|re-run the generator|auto-generated|generated (doc|artifact|file)|docs?-as-code"` across all 152 wiki pages returned 3 incidental hits (`security/dependencies/supply-chain`, `testing/quality/harness-reverse-controls`, `testing/e2e/e2e-stability`) — all about regenerating _lockfiles / auth state / mutation inputs_, none about sourcing a document deliverable. **No duplicate; nothing to merge into, so a new page was created.** + +**Conflicts flagged:** none. The nearest tension is the Write the Docs "hand-written is best" statement, resolved inside the page rather than left implicit. + +**Related links added (both directions)** + +| Existing page | Relationship to the new page | +| ---------------------------------------------- | --------------------------------------------------------------------------------- | +| `qa/document-verification/spec-document-gates` | Gating a document vs. sourcing its content — adjacent, distinct | +| `frontend/state/derived-state` | Same underlying principle (never store what you can derive) in a different domain | ## Routing decision -- `backend/common/llm/` (new) — LLM-specific server concerns: completion-response-validation, - context-window-budget. Coherent home shared by #6 and #13. -- `backend/common/integrations/` (new) — general repo-external-dependency concern: - externally-owned-defaults. Kept separate from `llm/` because its scope is any - external resource (bucket/queue/index), not LLM-only. -- `backend/common/storage/` (new) — object-key-persistence. -- `qa/document-verification/` (new) — spec-document-gates, editing-a-gated-document. - Introduced by both #10 and #11; unified into one index section. -- `testing/quality/` (existing) — checks-that-cannot-pass, spec-artifact-checks, - harness-reverse-controls (test/check-authoring discipline, distinct from - qa/document-verification which is release-process gate design). -- `platforms/{environment,shells,processes}/` (existing) — unicode-text-matching, - command-text-inspected-before-execution, non-interactive-cli-invocation. -- `infrastructure/{containers,observability}/` (existing) — host-cgroup-visibility, - missing-container-metrics. - -Source PRs #6–#13 are closed with a disposition comment crediting the author. +**Target: `qa` / new category `deliverables` / `generated-artifacts-as-deliverable-source.md`** + +- **Not `databases`** (the queue row's guess). The evidence is an ERD, but the directive is generator-agnostic — it applies equally to API specs and dependency inventories. Filing it under `databases/schema-design` would hide it from every non-schema hand-off. +- **`qa`** owns deliverable quality, and its route-here line already covers document deliverables. +- **New category justified.** Existing qa categories are `process` (acceptance criteria, release gates, regression scope, severity, post-release), `document-verification` (checks that gate a document), `environments`, `bug-reports`, `exploratory`. `document-verification` is about _checking_ a document; this case is about _where a document's content comes from_. No existing category covers that under this or another name. + +**Plumbing updated:** `wiki/qa/index.md` (new `## deliverables` section + the domain header's route-here line), `INDEX.md` (qa row), `log.md` (ingest entry recording the new category and its justification). + +## Nothing left unverified + +No candidate was dropped and none was carried at `unverified`. diff --git a/INDEX.md b/INDEX.md index d6d4238..3afe8d8 100644 --- a/INDEX.md +++ b/INDEX.md @@ -7,17 +7,17 @@ domain's `index.md`. Load nothing else at this level. follow the cross-pointers in their index or take the next matching seeded domain (routing protocol step 1, `AGENTS.md`). -| Domain | Status | Route here when | -|--------|--------|-----------------| -| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior | -| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | -| [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | -| [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting) | -| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | -| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) | -| [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) | -| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | -| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning | -| [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time | +| Domain | Status | Route here when | +| ---------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior | +| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | +| [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | +| [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting) | +| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | +| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing, automated verification of document deliverables (spec/RFC gates), sourcing a hand-off deliverable that duplicates a generated artifact (writing automated test code → testing) | +| [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) | +| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | +| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning | +| [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time | All ten domains are seeded. New categories grow via `skills/wiki-ingest/SKILL.md`. diff --git a/log.md b/log.md index 1c6293b..f6dc6ff 100644 --- a/log.md +++ b/log.md @@ -37,3 +37,4 @@ Append-only. Format: `## [YYYY-MM-DD] _id` convention; the committed generated ERD matched, and one re-run refreshed 97 → 105 tables. Sources: Write the Docs docs-principles (eliminate content overlap), OpenAPI Best Practices (single description as source of truth), Google docguide (sources next to code). diff --git a/wiki/frontend/state/derived-state.md b/wiki/frontend/state/derived-state.md index b065265..8cee529 100644 --- a/wiki/frontend/state/derived-state.md +++ b/wiki/frontend/state/derived-state.md @@ -9,7 +9,12 @@ sources: - https://react.dev/learn/choosing-the-state-structure - https://react.dev/reference/react/useMemo last_verified: 2026-07-10 -related: [frontend-state-client-vs-server-state, frontend-rendering-rerender-and-memoization] +related: + [ + frontend-state-client-vs-server-state, + frontend-rendering-rerender-and-memoization, + qa-deliverables-generated-artifacts-as-deliverable-source, + ] --- # Values Computable from Existing State or Props @@ -40,20 +45,20 @@ you are debugging two copies of the same fact that have drifted apart. Cases that look derived but need different handling: -| Case | Then | -|------|------| -| Computation is measured-expensive (profiled, perceptible lag) | Still compute — wrap in `useMemo` with the inputs as dependencies; do not move it to state + effect | -| Editable draft initialized from a prop (edit form seeded by an entity) | It is real state: initialize once from the prop (`useState(entity.name)`) and pass a `key={entity.id}` so the component remounts (resetting the draft) when the entity changes | -| Value derived from async/server data (needs a fetch to compute) | It is server state — put it in the server-state cache and transform at read time; see [frontend-state-client-vs-server-state] | -| Derived value must persist across the inputs disappearing (last non-empty result) | It is its own state; set it at the event that produces it, not in an effect watching the inputs | +| Case | Then | +| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Computation is measured-expensive (profiled, perceptible lag) | Still compute — wrap in `useMemo` with the inputs as dependencies; do not move it to state + effect | +| Editable draft initialized from a prop (edit form seeded by an entity) | It is real state: initialize once from the prop (`useState(entity.name)`) and pass a `key={entity.id}` so the component remounts (resetting the draft) when the entity changes | +| Value derived from async/server data (needs a fetch to compute) | It is server state — put it in the server-state cache and transform at read time; see [frontend-state-client-vs-server-state] | +| Derived value must persist across the inputs disappearing (last non-empty result) | It is its own state; set it at the event that produces it, not in an effect watching the inputs | ## Instead of -| If you are about to | Do this instead | Why | -|---------------------|-----------------|-----| -| Write a `useEffect` that sets state from other state/props | Compute the value in render | The effect version renders a stale frame first, then re-renders; it is an extra copy that can drift | -| Store `filteredItems` next to `items` + `filter` | Derive `filteredItems` in render | Any update path that touches `items` but forgets `filteredItems` ships a drift bug | -| Store the full selected object in state | Store `selectedId`, derive the object with `find` | The copy goes stale when the source list changes | +| If you are about to | Do this instead | Why | +| ---------------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------------------- | +| Write a `useEffect` that sets state from other state/props | Compute the value in render | The effect version renders a stale frame first, then re-renders; it is an extra copy that can drift | +| Store `filteredItems` next to `items` + `filter` | Derive `filteredItems` in render | Any update path that touches `items` but forgets `filteredItems` ships a drift bug | +| Store the full selected object in state | Store `selectedId`, derive the object with `find` | The copy goes stale when the source list changes | ## Sources diff --git a/wiki/qa/deliverables/generated-artifacts-as-deliverable-source.md b/wiki/qa/deliverables/generated-artifacts-as-deliverable-source.md new file mode 100644 index 0000000..96aa588 --- /dev/null +++ b/wiki/qa/deliverables/generated-artifacts-as-deliverable-source.md @@ -0,0 +1,79 @@ +--- +id: qa-deliverables-generated-artifacts-as-deliverable-source +domain: qa +category: deliverables +applies_to: [general] +confidence: verified +sources: + - https://www.writethedocs.org/guide/writing/docs-principles/ — "Eliminate content overlap between separate sources"; sources must be "clearly defined and disjoint" to prevent parallel maintenance of the same information + - https://learn.openapis.org/best-practices.html — a single OpenAPI description acts as the single source of truth for both the product and its docs + - https://google.github.io/styleguide/docguide/best_practices.html — keep documentation sources close to the code they describe so they change together +last_verified: 2026-08-04 +related: + [frontend-state-derived-state, qa-document-verification-spec-document-gates] +--- + +# A Requested Deliverable Duplicating an Artifact the Repo Already Generates + +## When this applies + +You are asked to produce a document (ERD, schema reference, API surface list, +dependency inventory) for a hand-off — review, audit, external partner, a ticket — +and the repository already generates that same content from code. The request names +the _audience_, not the source, so writing it by hand looks like the fastest path. + +## Do this + +1. Look for the generator before writing anything: `git ls-files '*ERD*' '*schema*'`, + the `scripts` block of the package manifest, and `Makefile`/CI targets. When a + candidate file opens with a "generated by" header, that header names the command + to re-run. +2. Re-run the generator, even when its committed output is stale. One re-run costs + less than re-deriving the same facts by hand and cannot disagree with the code. +3. Use the generator's output as the **body** of the deliverable. +4. Hand-write only what the generator cannot express, and keep it in a header or + appendix section: scope ("what this excludes"), domain narrative, decisions, and + the commit/date the body was generated from. + +| Content | Source | +| ------------------------------------------------------------------------------------ | -------------------------------- | +| Mechanically checkable facts — column sets, key names, endpoint lists, types, counts | The generator's output, verbatim | +| Judgement — why a table exists, what is out of scope, migration caveats | Hand-written, marked as such | + +## Edge cases + +| Case | Then | +| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| No generator exists | Write it by hand, and record in the deliverable which commit it describes so the next reader can tell whether it drifted | +| The generator's output is stale in the repo | Re-run and commit the refresh as its own change, separate from the deliverable | +| The generator cannot run (needs a live DB, missing credentials) | State that in the deliverable and name the last generated commit, rather than silently substituting hand-written content | +| The audience needs a subset | Filter the generated output programmatically; hand-copying the subset reintroduces the transcription errors this page prevents | +| Prose quality is the deliverable (tutorial, onboarding guide) | Hand-write it — generators produce reference material, not explanation | + +## Instead of + +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| Re-type schema/API facts into a new document because the existing artifact "isn't in the right format" | Re-run the generator and reformat its output programmatically | Transcription errors in mechanical content are systematic, not random — a wrong key-naming assumption repeats across every row | +| Skip the committed generated artifact because it looks out of date | Re-run it, then use it | Staleness is one command to fix; a hand-written copy is a second source that must be maintained forever | +| Present a hand-written summary and a generated artifact side by side | Publish the generated artifact and add the summary as a clearly-labelled header | Two sources describing the same thing drift, and readers cannot tell which one is authoritative | + +## Sources + +- https://www.writethedocs.org/guide/writing/docs-principles/ — "Eliminate content + overlap between separate sources"; scopes must be "clearly defined and disjoint" to + prevent "parallel maintenance (or worse — _lack_ of maintenance) of the same + information across multiple sources". The same page states that "the best + documentation is hand-written" and that generators "still require input from + humans" — which is why this page splits the deliverable: generated body, written + narrative, rather than claiming generation replaces writing. +- https://learn.openapis.org/best-practices.html — a single description acting as the + single source of truth for both product and docs. +- https://google.github.io/styleguide/docguide/best_practices.html — keep doc sources + next to the code they document so both change together. +- Field evidence (2026-08, monorepo hand-off): a hand-written schema deliverable + matched the live database for 3 of 19 cross-checked tables, listed 2 tables that did + not exist, and rendered all 14 primary keys as `id` against an actual + `_id` convention. The repository's committed generated ERD matched the live + database on the same tables; re-running its generator refreshed 97 → 105 tables in + one command. diff --git a/wiki/qa/document-verification/spec-document-gates.md b/wiki/qa/document-verification/spec-document-gates.md index c2e3959..9c2fea6 100644 --- a/wiki/qa/document-verification/spec-document-gates.md +++ b/wiki/qa/document-verification/spec-document-gates.md @@ -14,7 +14,13 @@ sources: - https://github.com/DavidAnson/markdownlint/blob/main/doc/md056.md - https://github.com/DavidAnson/markdownlint/issues/1206 last_verified: 2026-07-30 -related: [qa-process-acceptance-criteria, testing-quality-tests-that-cannot-fail, platforms-environment-unicode-text-matching] +related: + [ + qa-process-acceptance-criteria, + testing-quality-tests-that-cannot-fail, + platforms-environment-unicode-text-matching, + qa-deliverables-generated-artifacts-as-deliverable-source, + ] --- # Automated Gates on a Specification Document @@ -39,12 +45,12 @@ requirement unmet; you are fixing gate patterns for a document that is not writt mutation testing applied to the checker instead of the code. 3. **Check on four axes.** Token existence alone passes documents that violate the spec: -| Axis | What the check does | Defect that a token-existence check misses | -|------|---------------------|--------------------------------------------| -| Structure | Parse the table and assert rows, columns, and non-empty cells | The whole table is deleted while the token survives in a nearby paragraph | -| Modality and polarity | Within one sentence scope, assert the requirement is neither negated nor demoted (MUST→SHOULD, 필수→권장/원칙적으로) | "X is not required" and "X is recommended" both contain every keyword | -| Set completeness | Assert the exact member count of a closed set (`enum has exactly 5 rows`) | One enum row is deleted; the token count is still ≥ 1 | -| Cross-reference | Assert that a statement in one section implies its counterpart elsewhere, and recompute a derived value from its inputs | Two sections disagree, or an `Examples` block silently stands in for the deleted normative rule | +| Axis | What the check does | Defect that a token-existence check misses | +| --------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | +| Structure | Parse the table and assert rows, columns, and non-empty cells | The whole table is deleted while the token survives in a nearby paragraph | +| Modality and polarity | Within one sentence scope, assert the requirement is neither negated nor demoted (MUST→SHOULD, 필수→권장/원칙적으로) | "X is not required" and "X is recommended" both contain every keyword | +| Set completeness | Assert the exact member count of a closed set (`enum has exactly 5 rows`) | One enum row is deleted; the token count is still ≥ 1 | +| Cross-reference | Assert that a statement in one section implies its counterpart elsewhere, and recompute a derived value from its inputs | Two sections disagree, or an `Examples` block silently stands in for the deleted normative rule | 4. **Fail closed when the anchor is missing.** When the section heading, table, or derivation input a check needs cannot be located, report FAIL. A check that @@ -55,24 +61,24 @@ requirement unmet; you are fixing gate patterns for a document that is not writt ## Edge cases -| Case | Then | -|------|------| -| No conforming sibling exists (first document of its kind) | Author a minimal conforming fixture, run the gate against it, require PASS, and keep the fixture next to the gate | -| The gate must exist before the document (plan-first workflow) | Take the positive control from the sibling or fixture; treat the target's failing run as evidence of absence only | -| The `Examples` section satisfies the check while the normative section does not | Scope the check to the normative section (heading range), so examples cannot stand in for the rule | -| The check counts delimiters (`\|`) as a stand-in for parsing | Use a Markdown parser — markdownlint's own MD056 misreports when a pipe appears inside backticks (issue #1206) | -| The document deliberately relaxes a requirement | Change the gate and the acceptance criteria in the same commit, and record the relaxation in the PR ([qa-process-acceptance-criteria]) | -| The deliverable is code, not a document | Apply [testing-quality-tests-that-cannot-fail] — same red-run proof, expressed as tests | +| Case | Then | +| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| No conforming sibling exists (first document of its kind) | Author a minimal conforming fixture, run the gate against it, require PASS, and keep the fixture next to the gate | +| The gate must exist before the document (plan-first workflow) | Take the positive control from the sibling or fixture; treat the target's failing run as evidence of absence only | +| The `Examples` section satisfies the check while the normative section does not | Scope the check to the normative section (heading range), so examples cannot stand in for the rule | +| The check counts delimiters (`\|`) as a stand-in for parsing | Use a Markdown parser — markdownlint's own MD056 misreports when a pipe appears inside backticks (issue #1206) | +| The document deliberately relaxes a requirement | Change the gate and the acceptance criteria in the same commit, and record the relaxation in the PR ([qa-process-acceptance-criteria]) | +| The deliverable is code, not a document | Apply [testing-quality-tests-that-cannot-fail] — same red-run proof, expressed as tests | ## Instead of -| If you are about to | Do this instead | Why | -|---------------------|-----------------|-----| -| Adopt a pattern because it exits non-zero against the unwritten target | Run it against a conforming sibling and require the expected count | An absent file fails every pattern; the red run cannot distinguish a correct pattern from a mistyped one | -| Treat "the keyword is present" as the requirement being met | Add the sentence-scoped polarity and modality check | RFC 2119 makes MUST and SHOULD different requirement levels, so a demotion keeps every keyword while dropping the obligation | -| Assert a token appears at least once for a closed enum | Parse the table and assert the exact row count | Deleting one member leaves the count ≥ 1, so the gate stays green on an incomplete set | -| Skip a check whose anchor sentence was not found | Report FAIL and name the missing anchor | A skipped check is indistinguishable from a passed one in the summary line | -| Verify a cross-section value by matching the number as written | Recompute it from its inputs and compare | Matching the written number passes when both sections were edited to the same wrong value | +| If you are about to | Do this instead | Why | +| ---------------------------------------------------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- | +| Adopt a pattern because it exits non-zero against the unwritten target | Run it against a conforming sibling and require the expected count | An absent file fails every pattern; the red run cannot distinguish a correct pattern from a mistyped one | +| Treat "the keyword is present" as the requirement being met | Add the sentence-scoped polarity and modality check | RFC 2119 makes MUST and SHOULD different requirement levels, so a demotion keeps every keyword while dropping the obligation | +| Assert a token appears at least once for a closed enum | Parse the table and assert the exact row count | Deleting one member leaves the count ≥ 1, so the gate stays green on an incomplete set | +| Skip a check whose anchor sentence was not found | Report FAIL and name the missing anchor | A skipped check is indistinguishable from a passed one in the summary line | +| Verify a cross-section value by matching the number as written | Recompute it from its inputs and compare | Matching the written number passes when both sections were edited to the same wrong value | ## Sources diff --git a/wiki/qa/index.md b/wiki/qa/index.md index c80140c..7f84b71 100644 --- a/wiki/qa/index.md +++ b/wiki/qa/index.md @@ -2,43 +2,50 @@ Route here for: release-quality process — acceptance criteria, gates, regression scoping, test-environment parity, post-release verification, bug -reports, manual/exploratory testing, and automated verification of document -deliverables (specs/RFCs). Writing automated test code → wiki/testing/; +reports, manual/exploratory testing, automated verification of document +deliverables (specs/RFCs), and sourcing a hand-off deliverable that duplicates an +artifact the repo already generates. Writing automated test code → wiki/testing/; rollout/canary/rollback mechanics → wiki/infrastructure/. Match your situation to a "load when" line; load only matching pages. ## process -| Page | Load when | -|------|-----------| -| [acceptance-criteria](process/acceptance-criteria.md) | Writing or reviewing a feature ticket/user story before development starts; "done" is disputed at QA time; a delivered feature technically works but misses the intent | -| [release-gates](process/release-gates.md) | Deciding whether a build/release is ready to ship; defining or reviewing the checklist that makes that decision | -| [regression-scope](process/regression-scope.md) | Choosing what to re-test for a release/change when full regression is too expensive; reviewing someone else's proposed regression scope | -| [severity-and-priority](process/severity-and-priority.md) | Triaging a bug — deciding how bad it is and when it gets fixed; a triage stalled on a severity debate | -| [post-release-verification](process/post-release-verification.md) | A release just deployed to production; defining what "released safely" means; an incident revealed a release was broken for hours before anyone noticed | +| Page | Load when | +| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [acceptance-criteria](process/acceptance-criteria.md) | Writing or reviewing a feature ticket/user story before development starts; "done" is disputed at QA time; a delivered feature technically works but misses the intent | +| [release-gates](process/release-gates.md) | Deciding whether a build/release is ready to ship; defining or reviewing the checklist that makes that decision | +| [regression-scope](process/regression-scope.md) | Choosing what to re-test for a release/change when full regression is too expensive; reviewing someone else's proposed regression scope | +| [severity-and-priority](process/severity-and-priority.md) | Triaging a bug — deciding how bad it is and when it gets fixed; a triage stalled on a severity debate | +| [post-release-verification](process/post-release-verification.md) | A release just deployed to production; defining what "released safely" means; an incident revealed a release was broken for hours before anyone noticed | ## document-verification -| Page | Load when | -|------|-----------| -| [spec-document-gates](document-verification/spec-document-gates.md) | Writing or reviewing automated checks (grep/script) that decide whether a spec/RFC/schema document meets its requirements; a document passed its checklist but the requirement is still unmet; choosing what a doc gate must assert beyond keyword presence (table structure, MUST-vs-SHOULD demotion, closed-set completeness, cross-section consistency); validating a gate pattern for a document that does not exist yet | -| [editing-a-gated-document](document-verification/editing-a-gated-document.md) | Editing or rewording a document that grep/regex gates or a lint config check; a gate fails on wording whose meaning did not change; describing what an upstream spec says without tripping a "do not redefine it" gate; a check matches the pattern your own document quotes; recording an audit verdict inside the document that was audited; deciding which checks to re-run after editing a gated document | +| Page | Load when | +| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [spec-document-gates](document-verification/spec-document-gates.md) | Writing or reviewing automated checks (grep/script) that decide whether a spec/RFC/schema document meets its requirements; a document passed its checklist but the requirement is still unmet; choosing what a doc gate must assert beyond keyword presence (table structure, MUST-vs-SHOULD demotion, closed-set completeness, cross-section consistency); validating a gate pattern for a document that does not exist yet | +| [editing-a-gated-document](document-verification/editing-a-gated-document.md) | Editing or rewording a document that grep/regex gates or a lint config check; a gate fails on wording whose meaning did not change; describing what an upstream spec says without tripping a "do not redefine it" gate; a check matches the pattern your own document quotes; recording an audit verdict inside the document that was audited; deciding which checks to re-run after editing a gated document | + +## deliverables + +| Page | Load when | +| ------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [generated-artifacts-as-deliverable-source](deliverables/generated-artifacts-as-deliverable-source.md) | Asked to produce a document (ERD, schema reference, API surface list, dependency inventory) for a hand-off, review, or external partner when the repo already generates that content from code; deciding whether to re-run a stale generator or hand-write the deliverable; a hand-written reference document disagrees with the live system (checks that gate a document → document-verification) | ## environments -| Page | Load when | -|------|-----------| +| Page | Load when | +| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | | [test-environment-parity](environments/test-environment-parity.md) | A bug reproduces only in production; planning what a staging environment must mirror; deciding whether a staging pass clears a release | ## bug-reports -| Page | Load when | -|------|-----------| +| Page | Load when | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------- | | [reproducible-reports](bug-reports/reproducible-reports.md) | Writing a bug report; triaging incoming reports that can't be acted on as written | ## exploratory -| Page | Load when | -|------|-----------| +| Page | Load when | +| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | [exploratory-sessions](exploratory/exploratory-sessions.md) | A new feature needs testing beyond its scripted checks; you have test time available and want maximum new information per hour | From d602c5cb20eaae50018955bb6ff7dbdeb07e4c30 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Tue, 4 Aug 2026 10:55:20 +0900 Subject: [PATCH 2/3] docs: add cross-check note to ingest report --- .dev-loop/INGEST_REPORT.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 9d86f55..213e0d6 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -59,3 +59,5 @@ Framed that way the directive is supported by all three sources, and "prose qual ## Nothing left unverified No candidate was dropped and none was carried at `unverified`. + +Cross-Check: 면제(이 세션은 서브에이전트 사용이 제한됨) — 대신 검색 요약을 신뢰하지 않고 1차 출처(Write the Docs `docs-principles`)를 직접 fetch해 인용문을 원문 대조했고, 그 과정에서 이 지침과 **반대 방향으로 hedge**하는 문장("the best documentation is hand-written", 생성기는 "still require input from humans")을 발견해 페이지의 주장 범위를 "기계적 사실은 생성기, 산문은 손"으로 좁혔다. 중복 여부는 wiki 152개 페이지 전수 grep으로 확인(무관한 3건만 매칭), 라우팅은 databases·qa 두 도메인 index를 모두 읽고 결정했다. From 7388becb2f087bf764cacbbd08f08c98110be6b9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Tue, 4 Aug 2026 10:55:56 +0900 Subject: [PATCH 3/3] docs: add decision log to ingest report --- .dev-loop/INGEST_REPORT.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 213e0d6..3c6c37b 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -61,3 +61,26 @@ Framed that way the directive is supported by all three sources, and "prose qual No candidate was dropped and none was carried at `unverified`. Cross-Check: 면제(이 세션은 서브에이전트 사용이 제한됨) — 대신 검색 요약을 신뢰하지 않고 1차 출처(Write the Docs `docs-principles`)를 직접 fetch해 인용문을 원문 대조했고, 그 과정에서 이 지침과 **반대 방향으로 hedge**하는 문장("the best documentation is hand-written", 생성기는 "still require input from humans")을 발견해 페이지의 주장 범위를 "기계적 사실은 생성기, 산문은 손"으로 좁혔다. 중복 여부는 wiki 152개 페이지 전수 grep으로 확인(무관한 3건만 매칭), 라우팅은 databases·qa 두 도메인 index를 모두 읽고 결정했다. + +## Decision Log + +### D1 — 페이지 주장을 "생성이 우월하다"가 아니라 "기계적 사실만 생성기에서" 로 좁혔다 (의도) + +**배제한 대안**: 큐 원문 그대로 "손으로 다시 쓰지 말고 생성기 출력을 써라"로 단정. +**배제 이유**: 1차 출처(Write the Docs)가 정반대 방향으로 hedge한다 — *"the best documentation is hand-written"*, 생성기는 *"still require input from humans to function"*. 검색 요약만 봤으면 이 문장을 놓치고 출처가 뒷받침하지 않는 주장을 인용으로 달 뻔했다. 그래서 산출물을 **기계적 사실(생성기) / 판단·산문(손)** 으로 쪼갰고, "산문 품질 자체가 산출물인 경우(튜토리얼·온보딩)"는 edge case로 손 작성에 돌려보냈다. + +### D2 — 도메인을 `databases` 가 아니라 `qa` 로 (의도) + +**배제한 대안**: 큐 행이 제안한 `databases`(증거가 ERD라서). +**배제 이유**: 지침 자체는 생성기 종류를 가리지 않는다(API 스펙·의존성 인벤토리에 동일 적용). `databases/schema-design` 아래 두면 스키마가 아닌 모든 hand-off에서 라우팅되지 않는다. + +### D3 — `deliverables` 카테고리 신설 (의도) + +**배제한 대안**: 기존 `qa/document-verification` 에 세 번째 페이지로 추가. +**배제 이유**: 그 카테고리는 문서를 *검사(gate)* 하는 케이스를 담는다. 이 케이스는 문서의 *내용이 어디서 오는가* 이므로 "load when" 라인이 겹치지 않는다. 기존 5개 카테고리(process·document-verification·environments·bug-reports·exploratory) 어디에도 이름만 다른 동일 개념이 없음을 확인했다. + +### 리뷰어가 볼 곳 + +1. **인용 정확성** — 출처가 hedge한 문장을 페이지 Sources에 그대로 옮겼는지(요약이 붙여준 hedge를 떼면 위조 인용이 된다). 페이지 Sources 첫 항목. +2. **신규 카테고리 타당성** — D3의 근거가 충분한지. 과하면 `document-verification` 으로 접어도 된다. +3. **confidence: verified 판정** — SSOT/중복금지 원리는 1차 출처로 검증됐고, 운영 세부(stale이어도 재실행·리프레시는 별도 커밋)는 field evidence 기반이라 페이지 안에서 라벨을 갈라 적었다. 이 조합을 `verified` 로 볼지 `field-tested` 로 내릴지는 소유자 판단.