diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..77806ef 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,288 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 4 insights -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 4 pending candidates across 3 session files +(`9dab7c31…` ×2, `a26ea793…` ×1, `df9561a2…` ×1). All 4 ingested; 0 dropped. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. Classify a surviving mutant before writing a test for it (`testing`) + +**Claim.** When a mutant survives, decide whether it is a missing test, an +_equivalent_ mutant, or an uncovered line before changing anything. When it is +equivalent, the branch it mutates is redundant — delete it and correct the +comment that justified it, rather than adding a test. + +**Sources checked (all opened this session).** + +- + — "There is no definitive way for Stryker to find and ignore them"; the + documented remedy is "by finding these by hand, which is time consuming and + try to rewrite the code so it won't occur, or accept that you won't make + 100%". This is the primary support, and both halves are load-bearing: the + docs name rewriting the code and accepting a classified survivor as the two + outcomes — neither of them is "add a test for it". +- — "Not all mutations will + behave differently than the unmutated class. These mutants are referred to as + **equivalent mutations**"; "The resulting mutant behaves in exactly the same + way as the original"; and the two distinct verdicts "Survived: The mutation + was not detected by the covering test" vs "No coverage: The same as Survived + except there were no tests that exercised the line of code where the mutation + was created" — which is the three-way split the page's step-1 table encodes. +- + — the mutant state set and `detected / valid` scoring. +- — inserting + faults and requiring failure is the measurement. + +**How verified.** The mutation-testing docs substantiate the classification and +both remedies directly. The comment-correction step is the session's field +observation, recorded as a dated field-measurement line in the page's Sources +rather than attributed to a doc. After the adversarial pass, the page no longer +lets one hand-run input establish equivalence: the same Stryker sentence that +supports the remedy ("no definitive way … to find and ignore them") is what +makes a domain argument the required evidence for deleting a branch. + +**Confidence: verified** (classification + both remedies doc-backed; the +comment-correction step carries its field measurement inline). + +### 2. Anchor source-text wiring assertions per site instead of counting (`testing`) + +**Claim.** A guard that asserts by regex that a call is present, using +`toHaveLength(n)` or `>= n` over match count, stays green when one of the N call +sites is deleted. Bind each occurrence to its own context — a bounded order +anchor `A[\s\S]{0,N}B` whose anchor occurs **exactly once** in the file, or a +function-body slice — and prove each by deleting only its own site. + +**Sources checked (all opened this session).** + +- + — the Block Statement mutator "removes the content of every block statement". + Corrected after the adversarial pass: this empties a whole block rather than + removing one call, so the per-site deletion is a **hand-seeded** mutation and + the page now says so instead of claiming tool support it does not have. +- — "'Survived' means the + mutation was not detected by the covering test": the per-site deletion that + leaves the suite green is exactly this verdict. +- + — documents `{min,max}` as bounded repetition and `?` as the non-greedy form + that "will try to match as few times as possible". Corrected after the + adversarial pass: an earlier draft presented MDN's `{min,max}` **table** as a + prose quotation, and claimed the lazy form limits the anchor's reach. Neither + holds — see the Node measurement below. +- — `toHaveLength` compares a `.length` value; on + a match array it is a total and carries no per-site information. +- + — cited for the change-detector category (the refactor cost a source-text + guard accepts), **without a quotation**: the sentence an earlier draft + attributed to this article is a reader comment, and the body was not + retrievable in full. See the cross-check table, row 1. + +**How verified.** The "a lower bound survives deleting one of N" property is +arithmetic and is stated as such. The regex mechanism was **measured**, not +assumed: in Node, `/ANCHOR\([\s\S]{0,20}CALL\(/` and its lazy variant return +identical verdicts on four inputs (in range, call-before-anchor only, beyond the +bound, and call on both sides of the anchor), so the bound plus a once-occurring +anchor is what constrains the match. The concrete red/green pair (count +assertion green vs anchored assertion red on the same mutant, comment-only +control green) is the session's field measurement, dated in the page's Sources. + +**Confidence: verified.** + +### 3. `data === undefined` is not "loading" in TanStack Query (`frontend`) + +**Claim.** A component contract of `data | undefined` collapses two orthogonal +axes. A disabled (`enabled: false`) or offline-paused query is `status: 'pending'` +with `isLoading === false` and `isError === false` and `data === undefined`, so +"undefined means loading" renders a spinner no fetch will resolve. + +**Sources checked (all opened this session).** + +- — + "The `status` gives information about the `data`: Do we have any or not? The + `fetchStatus` gives information about the `queryFn`: Is it running or not?"; + "Background refetches and stale-while-revalidate logic make all combinations + for `status` and `fetchStatus` possible"; the value definitions including + `paused`: "The query wanted to fetch, but it is paused". +- — + `isLoading` "Is `true` whenever the first fetch for a query is in-flight. Is + the same as `isFetching && isPending`"; `data` "Defaults to `undefined`". +- + — a disabled query with no cached data is "status === 'pending' and + fetchStatus === 'idle'"; "Lazy queries will be in `status: 'pending'` right + from the start because `pending` means that there is no data yet … you likely + cannot use this flag to show a loading spinner"; the `skipToken`/`refetch` + incompatibility quoted in the page's edge table. +- — + "Queries can be in `state: 'pending'`, but `fetchStatus: 'paused'` if they are + mounting for the first time, and you have no network connection"; "it might + not be enough to check for `pending` state to show a loading spinner". + +**How verified.** Docs above, plus a local source check of the shipped build: +`@tanstack/query-core@5.100.14`, `build/modern/queryObserver.js` line 308 +`const isPending = status === "pending"`, line 310 +`const isLoading = isPending && isFetching`, line 332 +`isPaused: newState.fetchStatus === "paused"`. The derivation in the shipped +code matches the reference, so `pending` + non-`fetching` yields +`isLoading === false` with `data === undefined`. + +**Confidence: verified.** + +### 4. Prove a Python `encoding=` fix with `EncodingWarning`, not byte round-trip (`backend/python`) + +**Claim.** On a UTF-8 locale, removing `encoding="utf-8"` from `open()` produces +byte-identical output, so a round-trip regression test is green on the defect. +Run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` +and assert zero warning lines naming the file under test. Scope correction from +the adversarial pass: this replaces the round-trip only for an *omitted* +argument — `EncodingWarning` never fires on an explicitly wrong value, so the +page keeps a value assertion (run under a non-UTF-8 locale) for the encodings +you set on purpose. + +**Sources checked (all opened this session).** + +- — `EncodingWarning` "is emitted when the + `encoding` argument to `open()` is omitted and the default locale-specific + encoding is used"; "The `-X warn_default_encoding` option and the + `PYTHONWARNDEFAULTENCODING` environment variable are added. They are used to + enable `EncodingWarning`"; "Developers using macOS or Linux may forget that + the default encoding is not always UTF-8". +- — UTF-8 mode by default targets Python + 3.15; "many Python developers using Unix forget that the default encoding is + platform dependent … Inconsistent default encoding causes many bugs"; "this + change mostly affects Windows users". This is the mechanism for "invisible on + your machine". +- — `open()`: "The default + encoding is platform dependent (whatever `locale.getencoding()` returns)"; + "For reading and writing raw bytes use binary mode and leave _encoding_ + unspecified". + +**How verified.** Reproduced locally this session (CPython 3.14.6, macOS, +`locale.getpreferredencoding(False) == 'UTF-8'`): a script containing one +`open(p, "w")` and one `open(p, "w", encoding="utf-8")` produced byte-identical +output — the round-trip assertion cannot distinguish them. Running +`python3 -X warn_default_encoding -W always::EncodingWarning script.py out.txt` +emitted exactly one line, naming the unencoded call by file and line number; the +same run without the flag emitted nothing (which is why the page requires proving +the harness reddens on a deliberately unencoded `open()`). + +**Confidence: verified.** + +## Adversarial cross-check (run before this PR was opened) + +Cross-Check: independent `claude` CLI (headless, `--permission-mode plan`) reviewed the wiki diff for fabricated citations, overreach, internal contradiction, bare prohibitions, and vague qualifiers — it returned 18 findings (4 critical, 9 warning, 5 info); every critical was re-verified by me against the primary source or by measurement, and all 18 were fixed before this PR was created. + +The four criticals were real, and two of them were citation defects: + +| # | Finding | Verified how | Fix | +|---|---|---|---| +| 1 | `source-text-wiring-assertions` quoted "you cannot safely refactor code if you know you need to adapt the tests afterwards…" as the Google Testing Blog article's own sentence | Re-fetched the page: the sentence is from a **reader comment dated 2015-02-04**, and its wording differs ("refactor **stuff**", "know **for sure**"). The article body was not retrievable in full, so no sentence from it is quotable | Quotation removed; the URL is now cited for the change-detector *category* only, with a note that nothing is quoted from it. **The same fabricated quote exists in the already-merged `testing-quality-guard-shape-vs-consequence`** — I inherited it from there rather than opening the source. That bullet is corrected in this PR with the correction stated inline | +| 2 | Step 4 claimed deleting one call site is Stryker's Block Statement mutator | The doc says it "removes the content of every block statement" — it empties a whole block, not one call | Reworded: the per-site deletion is a hand-seeded mutation, and the doc is cited for why tools do not generate it | +| 3 | The order-anchor rationale claimed a lazy quantifier limits the anchor's reach and that the call could "not [appear] anywhere else in the file" | Measured in Node: `{0,20}` and `{0,20}?` return identical verdicts on all four inputs, and a call appearing both before and after the anchor still matches | Rewritten around what actually constrains the match: the bound `N` **and an anchor that occurs exactly once**. Added an occurrence-count step, a "no unique anchor" row, and the measurement as a source. A separate MDN pseudo-quote (a table rendered as prose) was also removed | +| 4 | `surviving-mutant-equivalence-triage` authorised **deleting production code** on the basis of one hand-run input, while its own cited source says "There is no definitive way for Stryker to find and ignore them" | Read against the cited Stryker page | Step 1 now starts from the tool's `No coverage` verdict; step 2 requires a stated argument over the branch's whole input domain and routes to the missing-test row when that argument cannot be written; step 5 splits a moved pass count into behavior-test vs implementation-test causes | + +Warnings fixed: selective half-quote of the Stryker remedy; the universal claim that any change in pass count means misclassification; the missing `success` + `paused` cell (with a `status: 'error'` row that had collapsed the fetch axis into `any`, contradicting the page's own premise) in a table that step 5 makes a coverage contract; `applies_to: general` on a page whose every field name is TanStack-specific (now `[react, tanstack-query]`, with an edge row for single-axis caches); the "keep the flag on the test invocation only" step contradicting the next step's "enable it repo-wide"; and the claim that a round-trip test "can only fail on a machine you are not testing on" — plus the gap it hid, that `EncodingWarning` fires only on an *omitted* argument and says nothing about an explicitly wrong one (new step 6 keeps a value assertion for those). + +Info fixed: banned qualifiers "usually", "commonly", "generally" removed from directive sentences; a "sixth combination" ordinal that did not match its own table. + +Re-checked after the fixes: 183 pages / 0 duplicate ids, 0 unresolved `[page-id]` refs in the new pages, 0 broken index links, all four sections present in each page, bodies 83–94 lines (limit 120). ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed each candidate via `INDEX.md` → domain `index.md`, then read every page +whose "load when" line overlapped. + +Pages read: testing-quality-tests-that-cannot-fail, testing-quality-harness-reverse-controls, testing-quality-behavior-not-implementation, testing-quality-guard-shape-vs-consequence, frontend-data-fetching-async-ui-states, frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions, platforms-environment-timezone-and-locale, backend-python-language-mutable-state-traps, backend-python-language-bytecode-cache-staleness + +**Overlaps found and what was done.** + +| Existing page | Overlap | Action | +| --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `testing-quality-tests-that-cannot-fail` | Its whole-suite edge row read _every_ surviving mutant in changed code as "missing or defective tests" — the exact naive reading insight 1 corrects. Page is already ~100 body lines, so appending a triage procedure would break the ≤120 rule | **Refined, not overwritten.** The row now routes surviving mutants through classification first. New page created for the procedure; `related:` both ways | +| `testing-quality-harness-reverse-controls` | Already covers equivalent mutants — but as a _deliberately constructed_ no-op control whose correct verdict is "survived". Different trigger (building/citing a harness vs triaging one live mutant) | Kept separate; `related:` both ways, and the new page routes uniform-verdict cases to it | +| `testing-quality-behavior-not-implementation` | Source-text assertions are implementation-coupled, which this page governs | Kept as the upstream decision ("should you assert on source at all"); insight 2's page opens by routing there, and links back | +| `testing-quality-guard-shape-vs-consequence` | Also about scanning-guard design, but its trigger is a repo-wide guard over _shipped artifacts_ going red on a legitimate one — the opposite failure (false positive) from insight 2 (false negative) | Kept separate; `related:` both ways | +| `frontend-data-fetching-async-ui-states` | Owns the loading/error/empty/data design and mentions `isLoading` vs `isFetching` in one line. It has no coverage of the status × fetchStatus product, and its four-state model has no cell for disabled/paused | **Merged where it fit** (+1 edge row routing the disabled/paused case) + new page for the mechanism; `related:` both ways | +| `frontend-state-client-vs-server-state`, `frontend-data-fetching-race-conditions` | Grepped for `isPending`/`fetchStatus`/`isLoading`/`isFetching`: zero hits. No overlap | No change | +| `platforms-environment-timezone-and-locale` | Owns locale as a hidden input generally, and pins `TZ` for tests. Says nothing about text-encoding defaults or `EncodingWarning` (repo-wide grep for `EncodingWarning`/`warn_default_encoding`/`getpreferredencoding`/`cp949`: 0 hits before this PR) | New page in `backend/python/language`; `related:` both ways, new page routes upward for the general case | +| `backend-python-language-mutable-state-traps` | Same category, unrelated trigger (state leaking across calls) | No change | +| `backend-python-language-bytecode-cache-staleness` | Adjacent: it governs mutation harnesses that rewrite `.py` files, which insight 4's harness does | `related:` both ways; new page carries an edge row routing to it | + +**Conflicts flagged:** none. The one directive that needed adjusting +(`tests-that-cannot-fail`'s whole-suite row) was incomplete rather than +contradictory, so it was refined in place and routed onward, per +`wiki-ingest` step 4. + +**Health checks run on the checkout after the edits:** 183 pages, 0 duplicate +ids; 0 unresolved `[page-id]` references introduced (the 3 the scan reports are +pre-existing false positives — a regex character class in two testing pages and +the literal `[openai-compatible]` in an LLM page); 0 broken relative links from +any index; all 4 new pages 75–80 body lines (limit 120). + +## Open-PR check + +Listed with +`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"`. +Four open heads: + +| PR | Head | Wiki paths touched | Overlap with this batch | +| --- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| #51 | `knowledge/dch0202-20260806-183029` | backend/{api-design,change-impact}, backend/index, debugging/hypothesis-testing, infrastructure/agent-orchestration ×2, qa/deliverables + qa/index, security/data/commit-identity-in-public-repos + security/index | None — no testing/quality, frontend, or python/language page in common | +| #50 | `knowledge/dch0202-20260806-172420` | testing/index, testing/quality/policy-at-several-return-sites (new), backend/change-impact/call-site-enumeration | **Nearest neighbour — examined in full.** Same underlying defect shape ("one of N sites silently stops being covered"), different subject: #50 covers _behavioral_ tests, one per success-return site of a handler, proven by reverting one site. Insight 2 covers _source-text_ regex guards where no behavioral seam is reachable, and the count-vs-anchor pattern choice. Neither carries the other's content | +| #49 | `knowledge/dch0202-rsquare-20260806-142309` (head ref deleted on remote; diffed via `refs/pull/49/head`) | testing/index + testing/quality ×8 (new: stale-artifact-baselines, unasserted-return-fields, value-preserving-refactor-assertions) | Same category, no shared trigger: baselines/return-field assertions/refactor-value assertions vs mutant triage and source-text wiring | +| #47 | `knowledge/dch0202-20260806-130040` | infrastructure/agent-orchestration, platforms/filesystems, testing/index, testing/quality/{guard-shape-vs-consequence, tests-that-cannot-fail} amendments | Touches two of the pages I amend. My edits are additive and in different regions (a `related:` id and one edge-row rewording); noted as a possible textual conflict for the owner to resolve at merge, not a content duplicate | + +**Per-candidate verdict:** 1 = **new**, 2 = **new**, 3 = **new**, 4 = **new**. +No candidate was folded or dropped — no open PR carries any of these four +insights. + +Note for the owner: #50, #49 and this PR all add pages under +`wiki/testing/quality` and all append a row to `wiki/testing/index.md`, so +whichever merges second will need the index rows rebased. The page files +themselves do not collide. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| # | Insight | Target | New category? | +| --- | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| 1 | Surviving-mutant triage | `testing / quality` → `testing-quality-surviving-mutant-equivalence-triage` (new page) | No — `quality` already owns assertion strength and mutation verification | +| 2 | Source-text wiring assertions | `testing / quality` → `testing-quality-source-text-wiring-assertions` (new page) | No | +| 3 | Query state vs fetch state | `frontend / data-fetching` → `frontend-data-fetching-query-state-vs-fetch-state` (new page) + 1 edge row and a `related:` link on `frontend-data-fetching-async-ui-states` | No — `data-fetching` already owns in-UI fetching states | +| 4 | Locale-default text I/O encoding | `backend / python / language` → `backend-python-language-default-encoding-in-text-io` (new page) | No — `python/language` is described in `INDEX.md` as the home for "language traps" | + +**Why insight 4 went to `backend` and not `testing` or `platforms`.** The +directive changes Python source (`encoding=` at every text-mode call site) and +its test is a property of that language's tooling, so the routing protocol's +"own the artifact you will change" rule puts it in `backend/python`. It routes +upward to `platforms-environment-timezone-and-locale` for the general +hidden-environment-input case and to `testing-quality-tests-that-cannot-fail` +for proving the harness reddens. + +**Plumbing updated:** `wiki/testing/index.md` (+2 rows), +`wiki/frontend/index.md` (+1 row), `wiki/backend/python/index.md` (+1 row), +`log.md` (+1 ingest entry). `INDEX.md` unchanged — no new domain, and every +target domain's "route here when" line already covers these cases. + +## Decision Log (AI 생성) + +### 의도 — 무엇을 / 왜 + +- `~/.dev-loop/queue` 에 쌓인 pending 후보 4건을 검증 통과시켜 wiki 에 편입하는 것이 목적. 각 후보를 1차 출처(공식 문서)로 확인하고, 2건은 로컬에서 재현(CPython 3.14.6 EncodingWarning, `@tanstack/query-core@5.100.14` queryObserver.js)해 `confidence: verified` 근거를 만들었다. +- 4건 모두 **신규 페이지**로 라우팅했다. merge-before-create 를 먼저 적용했으나, 가장 가까운 기존 페이지(`tests-that-cannot-fail`)가 이미 ~100 body line 이라 절차를 덧붙이면 ≤120 규칙을 깬다. 대신 그 페이지의 "surviving mutant = missing test" edge 행을 **분류 우선**으로 정정하고 새 페이지로 라우팅하도록 고쳤다. +- PR 직전 독립 적대검증을 1회 돌렸고, 18건 전건을 반영했다. 특히 위조 인용 1건은 **기존 wiki 페이지에서 물려받은 것**이라 그 원본(`guard-shape-vs-consequence`)까지 같은 PR 에서 정정했다 — 알면서 거짓 귀속을 남길 수 없다고 판단. + +### 배제한 대안 — 무엇을 안 했나 / 왜 + +- **열려 있는 PR #50 에 fold 하지 않음.** #50 은 한 핸들러의 여러 success-return 지점을 *행위 테스트*로 덮는 내용이고, 이번 insight 2 는 행위 seam 이 없을 때 쓰는 *소스 텍스트* 가드의 count↔anchor 선택이다. 겹치는 것은 결함 형태("N개 중 하나가 조용히 빠짐")이지 기법이 아니라 별도 페이지로 두고 보고서에 근거를 남겼다. +- **insight 4 를 `testing/` 이 아니라 `backend/python/language/` 로.** 지시가 바꾸는 산출물이 Python 소스이고 판별자가 그 언어 도구의 성질이라, 라우팅 프로토콜의 "바꿀 artifact 를 소유한 도메인" 규칙을 따랐다. +- **round-trip 단정을 전면 금지하지 않음.** 적대검증이 짚은 대로 `EncodingWarning` 은 *인자 누락*만 잡는다 — 명시했지만 틀린 값(`encoding="latin-1"`)은 값 단정이 아니면 아무도 못 잡으므로 둘을 병행하게 했다. +- **merge 하지 않음.** knowledge-flush 는 PR-only 이고 승인은 레포 오너 몫이다. +- **[추정] 커밋 아이덴티티**: skill 지시(ambient git identity 상속)와 직전 flush(PR #49)의 선례에 맞춰 `최영기 ` 로 커밋했다. 이 레포는 public 이고, 마침 열려 있는 PR #51 이 "public 레포에는 forge no-reply 주소를 쓰라"는 페이지를 추가하는 중이라 상충 소지가 있다 — 바꿀지는 작성자 판단. + +### 리뷰어가 볼 곳 — 신뢰성 판단 포인트 + +- `wiki/testing/quality/surviving-mutant-equivalence-triage.md:49` (step 2) — 이 단계가 **운영 코드 분기 삭제**를 승인하는 게이트다. 도메인 논증 요구가 충분한 강도인지 봐 달라. +- `wiki/testing/quality/source-text-wiring-assertions.md:39` (step 2~3) — anchor 유일성 + bound N. 적대검증 전 버전은 lazy quantifier 가 reach 를 제한한다고 **틀리게** 적었다가 실측으로 뒤집힌 자리다. +- `wiki/testing/quality/guard-shape-vs-consequence.md` (Sources 마지막 bullet) — 이번 PR 범위 밖이지만 위조 인용을 정정한 out-of-band 수정. 되돌릴지 판단 필요. +- `wiki/frontend/data-fetching/query-state-vs-fetch-state.md` (step 2 표) — step 5 가 이 표를 테스트 커버리지 계약으로 못박으므로 빠진 셀이 곧 커버리지 구멍이다. 8행이 status × fetchStatus 를 다 덮는지 확인해 달라. +- `wiki/testing/index.md` — #50, #49 도 같은 파일에 행을 추가한다. 두 번째로 머지되는 쪽이 이 행을 rebase 해야 한다(페이지 파일 자체는 충돌 없음). + +> [추정] 표시 항목은 세션에 명시 근거가 없어 사후 재구성한 의도임 — 검증 필요 diff --git a/log.md b/log.md index c930fc2..d66c0cc 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD] = n` / `toHaveLength(n)` count assertion over a call appearing at several sites stays green when the one site the guard was written for is deleted — enumerate the sites and bind each to a bounded order anchor that occurs exactly once in the file, or to a function-body slice; prove each by deleting only its own site, and run a reformat control. Measured: greedy vs lazy quantifiers give identical verdicts, so the bound and the anchor's uniqueness are what constrain the match), frontend/data-fetching/query-state-vs-fetch-state (a `data | undefined` component prop collapses TanStack Query's two orthogonal axes; a disabled or offline-paused query is `status: pending` with `isLoading === false` and `isError === false`, so "undefined means loading" renders a spinner no fetch will resolve — pass status+fetchStatus or an explicit union and test one case per cell), backend/python/language/default-encoding-in-text-io (a byte round-trip cannot prove an `encoding=` fix on a UTF-8 locale — run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` and assert zero warning lines naming that file). Merged into existing: tests-that-cannot-fail (whole-suite edge row now routes surviving mutants through classification instead of reading them all as missing tests), harness-reverse-controls, behavior-not-implementation, guard-shape-vs-consequence, async-ui-states (+disabled/paused edge row), bytecode-cache-staleness, timezone-and-locale — related links both ways. All cited URLs opened this session; two local reproductions (CPython 3.14.6 EncodingWarning discriminator vs byte-identical round-trip; `@tanstack/query-core@5.100.14` queryObserver.js:308-332 `isLoading = isPending && isFetching`). +## [2026-08-07] revise | testing/quality/guard-shape-vs-consequence — corrected a misattributed citation found by the pre-PR adversarial pass: the sentence "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" was presented as the Google Testing Blog article's own, but re-fetching the page shows it is a reader comment (2015-02-04) with different wording ("refactor stuff", "know for sure"). The article body was not retrievable in full, so the bullet now cites the URL for the change-detector category without quoting it, and states the correction inline. The same quote had been copied into a new page in this flush before verification — the lesson being that a citation already present in the wiki is not a verified citation. diff --git a/wiki/backend/python/index.md b/wiki/backend/python/index.md index c0473f2..f9d1a25 100644 --- a/wiki/backend/python/index.md +++ b/wiki/backend/python/index.md @@ -30,3 +30,4 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [mutable-state-traps](language/mutable-state-traps.md) | State persists or leaks across calls/requests in a long-lived Python process — one user's data appears for another, values "remembered" between calls; loop-built callbacks all use the last value; reviewing function signatures (mutable defaults), class bodies (class attributes), or module-level objects for hidden sharing; choosing contextvars vs thread-locals for request context | | [bytecode-cache-staleness](language/bytecode-cache-staleness.md) | A script or harness rewrites `.py` files and re-runs them in a loop (mutation testing, edit/test/revert, codegen check, bisect) and the result stops tracking what is on disk — a revert that `git diff` reports clean still fails, or an injected change has no effect; choosing between clearing `__pycache__`, refreshing mtime, and hash-based `.pyc` (PEP 552); designing byte-length-preserving mutations | +| [default-encoding-in-text-io](language/default-encoding-in-text-io.md) | Python opens a text file without `encoding=` (`open`, `Path.read_text`, `subprocess` text mode) and you are adding the argument or writing the regression test that keeps it there; a file-writing bug reproduces on Windows, a `LANG=C` container, or a cp949/cp932 desktop but not on your machine; choosing a test discriminator that does not depend on the runner's locale | diff --git a/wiki/backend/python/language/bytecode-cache-staleness.md b/wiki/backend/python/language/bytecode-cache-staleness.md index c911edd..a7ddf1f 100644 --- a/wiki/backend/python/language/bytecode-cache-staleness.md +++ b/wiki/backend/python/language/bytecode-cache-staleness.md @@ -9,7 +9,7 @@ sources: - https://peps.python.org/pep-0552/ - https://docs.python.org/3/library/py_compile.html last_verified: 2026-08-04 -related: [testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, backend-python-language-mutable-state-traps] +related: [backend-python-language-default-encoding-in-text-io, testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, backend-python-language-mutable-state-traps] --- # Edited Python Source the Interpreter Keeps Ignoring diff --git a/wiki/backend/python/language/default-encoding-in-text-io.md b/wiki/backend/python/language/default-encoding-in-text-io.md new file mode 100644 index 0000000..e3b6917 --- /dev/null +++ b/wiki/backend/python/language/default-encoding-in-text-io.md @@ -0,0 +1,107 @@ +--- +id: backend-python-language-default-encoding-in-text-io +domain: backend +category: language +applies_to: [python] +confidence: verified +sources: + - https://peps.python.org/pep-0597/ + - https://peps.python.org/pep-0686/ + - https://docs.python.org/3/library/functions.html +last_verified: 2026-08-07 +related: + [ + backend-python-language-bytecode-cache-staleness, + platforms-environment-timezone-and-locale, + testing-quality-tests-that-cannot-fail, + testing-quality-minimum-case-set, + ] +--- + +# Text I/O Whose Encoding Comes from the Machine's Locale + +## When this applies + +Python code opens a text file without `encoding=` — `open(p)`, `open(p, "w")`, +`Path.read_text()`, `csv`/`json` wrappers built on them — and you are adding the +argument, or writing the regression test that keeps it there. Also when a +file-writing bug reproduces on one machine (Windows, a `LANG=C` container, a +cp949/cp932 desktop) and not on yours. + +Timezone and locale as hidden inputs across dates and text → +[platforms-environment-timezone-and-locale]. + +## Do this + +1. **Pass `encoding=` at every text-mode call site.** The default is the + machine's: "The default encoding is platform dependent (whatever + `locale.getencoding()` returns)". Choose the value from what the file is: + +| The file is | Pass | +| --------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| A format with a defined encoding (JSON, TOML, YAML, Markdown, source) | `encoding="utf-8"` | +| Written and read only by this program | `encoding="utf-8"` | +| Produced by a tool bound to the OS console encoding, deliberately | `encoding=locale.getencoding()`, stated explicitly so the dependency is visible | +| Raw bytes | Binary mode with no `encoding` — "For reading and writing raw bytes use binary mode and leave _encoding_ unspecified" | + +2. **Make the regression test run the real entry point under the interpreter's + own diagnostic, and assert zero warnings naming the file you fixed:** + + ```sh + python3 -X warn_default_encoding -W always::EncodingWarning + ``` + + `EncodingWarning` "is emitted when the `encoding` argument to `open()` is + omitted and the default locale-specific encoding is used", and the flag (or + `PYTHONWARNDEFAULTENCODING`) is what enables it. Filter the captured stderr + to the file under test by name, so unfixed call sites elsewhere in the + codebase do not redden this test. + +3. **Assert on the warning lines, not on the produced bytes.** The warning is + emitted at the call site regardless of what the locale happens to be, so it + is the same verdict on your laptop and in CI. + +4. **Prove the check reddens before trusting it.** Without `-X + warn_default_encoding` the warning is silent, so a runner that drops the flag + reports green on the reintroduced defect and looks identical to a pass. Seed a + deliberately unencoded `open()` in the file under test, require red, then + restore ([testing-quality-tests-that-cannot-fail]). + +5. **Widen the flag from the one test invocation to the whole CI run once every + call site is clean**, so a new omission is caught where it is written rather + than at the next locale change. Until then the filter in step 2 is what keeps + the unfixed sites from reddening this test. + +6. **Keep a value assertion for the encodings you set explicitly.** + `EncodingWarning` fires only on an *omitted* argument, so it says nothing about + `encoding="latin-1"` or a deliberate `encoding=locale.getencoding()`. For those + call sites, assert the bytes the file should contain, and run that assertion + under a non-UTF-8 locale (`LANG=C`, or a cp949/cp932 job) where a wrong value + changes the output. + +## Edge cases + +| Case | Then | +| ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The entry point is a library function, not a script | Run it through a one-line driver under the same flags; the warning is attributed to the frame that called `open()`, so the driver's own lines do not mask it | +| A dependency emits `EncodingWarning` from its own files | Filter by filename as in step 2 and record the dependency in the test's name, so the filter states what it is excluding | +| The code targets Python 3.15 or later, where UTF-8 mode is on by default (PEP 686) | Keep the explicit `encoding=`: the argument states the file's contract and is what makes the call correct under an inherited `PYTHONUTF8=0` or an older runtime | +| Running under `PYTHONUTF8=1` / UTF-8 mode already | The warning still fires on the omitted argument, so the test keeps working; the mode changes the value used, not whether the argument was passed | +| `subprocess` output is being decoded | The same default applies to its text mode — pass `encoding="utf-8"` there, and include it in the call-site sweep | +| The harness rewrites the file between runs to seed the missing-`encoding` mutation | Clear the bytecode cache between iterations ([backend-python-language-bytecode-cache-staleness]) | + +## Instead of + +| If you are about to | Do this instead | Why | +| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Prove an omitted-`encoding` fix with a round-trip assertion alone (write non-ASCII text, read it back, compare) | Assert zero `EncodingWarning` lines naming the file, under `-X warn_default_encoding`, and keep the round-trip for the explicitly-set encodings (step 6) | On a UTF-8 locale the encoded bytes are identical with and without the argument, so the round-trip is green on the defect — it discriminates only on a runner whose locale encoding is not UTF-8, which is not the default on macOS or on most Linux CI images | +| Assert the output file's declared charset (``, an XML declaration) | Assert the warning count | A declaration is a literal in the template — it is written correctly by code that encoded the body wrongly | +| Set `LANG`/`PYTHONUTF8` in the test environment to make the behavior deterministic | Fix the call sites and assert the warning | Pinning the environment makes the test pass by removing the input the defect depends on, so the defect ships and fails on the machines that do not inherit that environment | +| Read "it works on macOS and Linux" as evidence the encoding is right | Run the warning check | PEP 686: "many Python developers using Unix forget that the default encoding is platform dependent … Inconsistent default encoding causes many bugs"; "this change mostly affects Windows users" | + +## Sources + +- https://peps.python.org/pep-0597/ — `EncodingWarning` "is emitted when the `encoding` argument to `open()` is omitted and the default locale-specific encoding is used"; "The `-X warn_default_encoding` option and the `PYTHONWARNDEFAULTENCODING` environment variable are added. They are used to enable `EncodingWarning`"; "When the flag is set, `io.TextIOWrapper()`, `open()` and other modules using them will emit `EncodingWarning` when the `encoding` argument is omitted"; "Developers using macOS or Linux may forget that the default encoding is not always UTF-8" +- https://peps.python.org/pep-0686/ — enabling UTF-8 mode by default targets Python 3.15; "many Python developers using Unix forget that the default encoding is platform dependent. They omit to specify `encoding="utf-8"` … Inconsistent default encoding causes many bugs"; "Most Unix systems use UTF-8 locale … So this change mostly affects Windows users" +- https://docs.python.org/3/library/functions.html — `open()`: "The default encoding is platform dependent (whatever `locale.getencoding()` returns)"; "In text mode, if _encoding_ is not specified the encoding used is platform-dependent"; "For reading and writing raw bytes use binary mode and leave _encoding_ unspecified" +- Reproduction 2026-08-07 (CPython 3.14.6, macOS, `locale.getpreferredencoding(False) == 'UTF-8'`): a script with one `open(p, "w")` and one `open(p, "w", encoding="utf-8")` produced byte-identical output — a round-trip assertion cannot distinguish them. `python3 -X warn_default_encoding -W always::EncodingWarning script.py out.txt` emitted exactly one line, naming the unencoded call by file and line number; the same run without the flag emitted nothing diff --git a/wiki/frontend/data-fetching/async-ui-states.md b/wiki/frontend/data-fetching/async-ui-states.md index 636aaf1..9a10001 100644 --- a/wiki/frontend/data-fetching/async-ui-states.md +++ b/wiki/frontend/data-fetching/async-ui-states.md @@ -11,7 +11,7 @@ sources: - https://tanstack.com/query/latest/docs/framework/react/guides/optimistic-updates - https://react.dev/reference/react/Component last_verified: 2026-07-10 -related: [frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions] +related: [frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions, frontend-data-fetching-query-state-vs-fetch-state] --- # Designing Loading, Error, Empty, and Data States for an Async View @@ -59,6 +59,7 @@ Then apply these to the transitions between states: | List is empty because the user's filters excluded everything | Say so, and offer "clear filters" — the generic empty state ("add your first item") misleads | | Response resolves fast enough that the skeleton only flashes | Keep the reserved space but suppress indicator animation for sub-second responses — feedback that fast is distraction, not information | | Mutation has no inverse (send email, submit payment) | No optimistic update — render an explicit pending state until the server confirms | +| The query can be disabled (`enabled: false`) or paused offline, so it has no data and is not fetching | The four states above do not cover it — branch on the cache's status/fetchStatus pair ([frontend-data-fetching-query-state-vs-fetch-state]) | ## Instead of diff --git a/wiki/frontend/data-fetching/query-state-vs-fetch-state.md b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md new file mode 100644 index 0000000..66ce45d --- /dev/null +++ b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md @@ -0,0 +1,102 @@ +--- +id: frontend-data-fetching-query-state-vs-fetch-state +domain: frontend +category: data-fetching +applies_to: [react, tanstack-query] +confidence: verified +sources: + - https://tanstack.com/query/latest/docs/framework/react/guides/queries + - https://tanstack.com/query/latest/docs/framework/react/reference/useQuery + - https://tanstack.com/query/latest/docs/framework/react/guides/disabling-queries + - https://tanstack.com/query/latest/docs/framework/react/guides/network-mode +last_verified: 2026-08-07 +related: + [ + frontend-data-fetching-async-ui-states, + frontend-state-client-vs-server-state, + frontend-data-fetching-race-conditions, + testing-mocking-what-to-mock, + ] +--- + +# A Server-State Query That Has No Data and Is Not Loading + +## When this applies + +You are defining what a component receives from a TanStack Query hook and are +about to treat `data === undefined` as "loading". +Also when a view shows a permanent spinner with no error and no retry, or the +query it renders can be disabled (`enabled: false`, `skipToken`) or paused by +the network mode. + +Designing the loading / error / empty / data renderings themselves → +[frontend-data-fetching-async-ui-states]. + +## Do this + +1. **Take two independent inputs, not one.** TanStack Query exposes them as + separate fields for this reason: "The `status` gives information about the `data`: Do + we have any or not? The `fetchStatus` gives information about the `queryFn`: + Is it running or not?" — and "all combinations for `status` and `fetchStatus` + [are] possible". A component prop of `data | undefined` collapses both axes + into one bit and cannot recover them. + +2. **Branch on the combination, and give every cell a rendering:** + +| status | fetchStatus | What it means | Render | +| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| `pending` | `fetching` | First fetch in flight — this is `isLoading`, defined as "`isFetching && isPending`" | Skeleton | +| `pending` | `idle` | Disabled/lazy query: "status === 'pending' and fetchStatus === 'idle'" with `enabled: false` | The pre-request state — the prompt, the disabled form, the "select a row" placeholder | +| `pending` | `paused` | Wanted to fetch, has no connection: "state: 'pending', but fetchStatus: 'paused' if they are mounting for the first time, and you have no network connection" | Offline notice plus a retry affordance | +| `error` | `idle` | The attempt failed and nothing is retrying | Error message plus a retry affordance ([frontend-data-fetching-async-ui-states]) | +| `error` | `fetching` | A retry is already running over the failed state | Keep the error message, disable the retry affordance while it runs | +| `error` | `paused` | Failed, and the retry is waiting for a connection | Offline notice — a retry affordance here cannot run | +| `success` | `fetching` | Background refresh over existing data | The data, plus a subtle refresh indicator | +| `success` | `paused` | Data is on screen and a background refetch is waiting for a connection | The data, plus an offline/stale indicator instead of the refresh indicator | +| `success` | `idle` | Settled | The data (or the empty state when it is an empty collection) | + +3. **Pass the discriminator down.** Give the presentational component either the + two fields or an explicit union (`{kind: 'idle' | 'loading' | 'paused' | +'error' | 'ready', …}`) built at the boundary that holds the query. The union + makes every unhandled state a type error instead of a blank screen. + +4. **Use `isLoading` for spinners and `isPending` for "no data yet".** The docs + state the split directly: lazy queries "will be in `status: 'pending'` right + from the start because `pending` means that there is no data yet … you likely + cannot use this flag to show a loading spinner". + +5. **Cover the disabled and paused cells in tests explicitly.** A test that mocks + the query hook supplies the flags by hand and therefore only ever produces + combinations its author already thought of — so the combination that ships the + bug is the one the suite never constructs. Write one case per row of the step-2 + table, taking the flag values from that table rather than from the component + ([testing-mocking-what-to-mock]). + +## Edge cases + +| Case | Then | +| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The query has `initialData` or `placeholderData` | It starts at `status: 'success'`, so the pending rows never render — assert which data the user is seeing before treating success as authoritative | +| A disabled query has cached data from an earlier mount | It initializes as "status === 'success' or isSuccess" rather than pending; the idle-pre-request rendering does not apply | +| The component must trigger the fetch itself | Keep `enabled: false` rather than `skipToken`: "`refetch` from `useQuery` will not work with `skipToken`. Calling `refetch()` on a query that uses `skipToken` will result in a `Missing queryFn` error" | +| `select` narrows the data and returns `undefined` for a valid response | That is `status: 'success'` with `data === undefined` — a case outside the status × fetchStatus grid that the one-bit contract also loses; assert on `status`, not on the value | +| The paused state is unreachable because `networkMode: 'always'` is set | Drop the paused row for that query and state the mode in the component's contract, so a later mode change re-opens the row deliberately | +| The cache in use is SWR, RTK Query, or Apollo rather than TanStack Query | Map its fields onto the two axes before applying the table — where a cache exposes no separate fetch axis, build the explicit union of step 3 from the fields it does expose, and keep the pre-request case distinct from loading | +| Several queries feed one view | Combine on the axes, not the values: pending if any is pending, paused if any is paused — a merged `data === undefined` cannot distinguish them | + +## Instead of + +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Type the child's prop as `data \| undefined` and read `undefined` as "loading" | Pass `status` and `fetchStatus`, or an explicit state union built at the query boundary | A disabled or paused query is `pending` with `data === undefined`, `isLoading === false` and `isError === false`, so the child renders a spinner that no fetch will ever resolve | +| Show the spinner on `isPending` | Show it on `isLoading` (`isPending && isFetching`) and render the idle case separately | A lazy query is `pending` from the first render, so `isPending` puts a spinner on a query that was never requested | +| Add a timeout that turns a long spinner into an error | Render the `pending`/`idle` and `pending`/`paused` cells | The spinner is not slow, it is terminal — a timeout converts a missing state into a wrong one | +| Test the component by mocking the hook with `{data: undefined, isLoading: true}` and `{data: X}` | Drive one case per row of the step-2 table | Hand-written mocks reproduce the author's model of the states, so the combination that causes the bug is the one never constructed | + +## Sources + +- https://tanstack.com/query/latest/docs/framework/react/guides/queries — the `status` values (`pending` "The query has no data yet", `error`, `success`) and `fetchStatus` values (`fetching`, `paused` "The query wanted to fetch, but it is paused", `idle`); "Background refetches and stale-while-revalidate logic make all combinations for `status` and `fetchStatus` possible"; "The `status` gives information about the `data` … The `fetchStatus` gives information about the `queryFn`" +- https://tanstack.com/query/latest/docs/framework/react/reference/useQuery — `isLoading` "Is `true` whenever the first fetch for a query is in-flight. Is the same as `isFetching && isPending`"; `data` "Defaults to `undefined`"; `status` is `pending` "if there's no cached data and no query attempt was finished yet" +- https://tanstack.com/query/latest/docs/framework/react/guides/disabling-queries — a disabled query with no cached data is "status === 'pending' and fetchStatus === 'idle'"; "Lazy queries will be in `status: 'pending'` right from the start because `pending` means that there is no data yet … you likely cannot use this flag to show a loading spinner"; the `skipToken`/`refetch` incompatibility +- https://tanstack.com/query/latest/docs/framework/react/guides/network-mode — "Queries can be in `state: 'pending'`, but `fetchStatus: 'paused'` if they are mounting for the first time, and you have no network connection"; "it might not be enough to check for `pending` state to show a loading spinner" +- Source verification 2026-08-07 (`@tanstack/query-core@5.100.14`, `build/modern/queryObserver.js`): line 308 `const isPending = status === "pending"`, line 310 `const isLoading = isPending && isFetching`, line 332 `isPaused: newState.fetchStatus === "paused"` — the shipped derivation matches the reference, so `pending` + non-`fetching` yields `isLoading === false` with `data === undefined` diff --git a/wiki/frontend/index.md b/wiki/frontend/index.md index 4ca6c93..291a511 100644 --- a/wiki/frontend/index.md +++ b/wiki/frontend/index.md @@ -34,6 +34,7 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [race-conditions](data-fetching/race-conditions.md) | Repeated fetches with changing params can overlap (search-as-you-type, rapid tab/filter switches); UI intermittently shows results for a previous input; mutations race refetches | | [async-ui-states](data-fetching/async-ui-states.md) | Building any view backed by async data; users see blank screens, eternal spinners, or dead-end errors; reviewing loading/error/empty handling in UI code; deciding on skeletons vs spinners, retry affordances, empty states, background-refresh indication, or optimistic updates | +| [query-state-vs-fetch-state](data-fetching/query-state-vs-fetch-state.md) | Defining what a component receives from a server-state cache (TanStack Query and equivalents) and about to treat `data === undefined` as "loading"; a view shows a permanent spinner with no error and no retry; the query can be disabled (`enabled: false`, `skipToken`) or paused by the network mode; deciding what the presentational component's state prop should be | | [infinite-scroll](data-fetching/infinite-scroll.md) | Implementing infinite scroll or a load-more feed; an existing feed loses scroll position on back-navigation, duplicates/skips items, or spams page requests; choosing between infinite scroll and a load-more button | ## performance diff --git a/wiki/platforms/environment/timezone-and-locale.md b/wiki/platforms/environment/timezone-and-locale.md index 729de7c..618c01d 100644 --- a/wiki/platforms/environment/timezone-and-locale.md +++ b/wiki/platforms/environment/timezone-and-locale.md @@ -14,7 +14,7 @@ sources: - https://unicode.org/faq/casemap_charprop.html - https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html last_verified: 2026-07-10 -related: [databases-schema-design-column-data-types, platforms-processes-background-services, platforms-environment-unicode-text-matching] +related: [backend-python-language-default-encoding-in-text-io, databases-schema-design-column-data-types, platforms-processes-background-services, platforms-environment-unicode-text-matching] --- # Timezone and Locale as Hidden Inputs to Date and Text Code diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 5f3ed78..fb1d36e 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -34,6 +34,8 @@ Match your situation to a "load when" line; load only matching pages. | [spec-artifact-checks](quality/spec-artifact-checks.md) | Writing or reviewing an automated check that a mapping table covers every rule/field/enum case, or that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo | | [schema-additions-under-a-golden-gate](quality/schema-additions-under-a-golden-gate.md) | Adding a node kind, variant, discriminator value, or field to a document format (IR, JSON Schema, spec artifact) whose only automated gate builds its negatives by mutating one committed golden example; the gate or the whole suite comes back green right after a schema change; deciding which negative each new schema keyword needs, and whether a green suite that never loads the schema is evidence at all | | [harness-reverse-controls](quality/harness-reverse-controls.md) | You built a harness that scores how well something is verified (mutation run, doc/spec gate suite, CI check matrix) and are about to cite its score in a commit, PR, README, or report; its verdicts come out uniform (every case caught, or every case green); deciding what control run proves the harness discriminates, how to score errored/never-ran cases, and what the harness's isolated working tree must contain | +| [surviving-mutant-equivalence-triage](quality/surviving-mutant-equivalence-triage.md) | A mutation run (PIT, Stryker, or a hand-seeded mutation) left a mutant alive on code you own and you are deciding what to change; a reviewer asks for a test covering a specific surviving mutant; a defensive branch carries a comment explaining why it is needed and its mutant survives; separating a missing test from an equivalent mutant from an uncovered line | +| [source-text-wiring-assertions](quality/source-text-wiring-assertions.md) | A test reads a source file as a string and asserts by regex that a call is present (cleanup in every handler, logging after each branch, teardown in each exit path) because the behavior has no reachable seam; such a guard is green while one of the call sites is gone; choosing between a count assertion, an order anchor, and a function-body slice | ## data diff --git a/wiki/testing/quality/behavior-not-implementation.md b/wiki/testing/quality/behavior-not-implementation.md index e1181df..7cbe4d0 100644 --- a/wiki/testing/quality/behavior-not-implementation.md +++ b/wiki/testing/quality/behavior-not-implementation.md @@ -9,7 +9,7 @@ sources: - https://abseil.io/resources/swe-book/html/ch12.html - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html last_verified: 2026-07-10 -related: [testing-quality-minimum-case-set, testing-mocking-what-to-mock, testing-quality-guard-shape-vs-consequence, backend-common-change-impact-call-site-enumeration] +related: [testing-quality-minimum-case-set, testing-mocking-what-to-mock, testing-quality-guard-shape-vs-consequence, testing-quality-source-text-wiring-assertions, backend-common-change-impact-call-site-enumeration] --- # Asserting Behavior Through the Public Interface diff --git a/wiki/testing/quality/guard-shape-vs-consequence.md b/wiki/testing/quality/guard-shape-vs-consequence.md index e3875e8..6ffea97 100644 --- a/wiki/testing/quality/guard-shape-vs-consequence.md +++ b/wiki/testing/quality/guard-shape-vs-consequence.md @@ -8,7 +8,7 @@ sources: - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html - https://pitest.org/ last_verified: 2026-08-04 -related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope] +related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-source-text-wiring-assertions, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope] --- # A Repo-Wide Guard That Fires on a Legitimate Artifact @@ -78,6 +78,6 @@ Reviewing a guard that has never been red → [testing-quality-tests-that-cannot ## Sources -- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27): "Change-detector tests do not add clarity, and you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again." A shape-only guard that must be exempted for each new legitimate artifact is this failure mode at repo scope +- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27), cited for the change-detector category: a shape-only guard that must be exempted for each new legitimate artifact is that failure mode at repo scope. Correction 2026-08-07: an earlier revision of this bullet presented "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" as the article's own sentence. Re-fetching the page shows it is from a reader comment dated 2015-02-04 and its wording differs ("refactor stuff", "know for sure"); the article body was not retrievable in full, so nothing is quoted from it here - https://pitest.org/ — "Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived" — the basis for step 4's required-red fixture - Field evidence (linkly #35, 2026-08-04): `test_no_shipped_example_has_a_guarded_repository_call` asserted that no shipped `.lnpl` example contained a repository call under a guard. `examples/checkout.lnpl` legitimately added a `create` under `when stock > 0` — the issue's own reproduction shape — turning the guard permanently red. Re-expressing it as "a guarded call that could actually fail", with the conflict/miss decision taken from the production `_lnpl_ops` derivation via `seeded_entities`/`repository_calls`, returned the suite to `Ran 518 tests / OK` while a fixture holding a guarded-and-can-fail create still drove the guard red diff --git a/wiki/testing/quality/harness-reverse-controls.md b/wiki/testing/quality/harness-reverse-controls.md index aa62831..74965f5 100644 --- a/wiki/testing/quality/harness-reverse-controls.md +++ b/wiki/testing/quality/harness-reverse-controls.md @@ -13,7 +13,7 @@ sources: - https://testing.googleblog.com/2021/04/mutation-testing.html - https://docs.python.org/3/library/unittest.mock.html last_verified: 2026-08-04 -related: [testing-quality-tests-that-cannot-fail, testing-quality-minimum-case-set, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, backend-python-language-bytecode-cache-staleness, qa-exploratory-override-control-pairs] +related: [testing-quality-tests-that-cannot-fail, testing-quality-surviving-mutant-equivalence-triage, testing-quality-minimum-case-set, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, backend-python-language-bytecode-cache-staleness, qa-exploratory-override-control-pairs] --- # Citing a Verification Harness's Own Score diff --git a/wiki/testing/quality/source-text-wiring-assertions.md b/wiki/testing/quality/source-text-wiring-assertions.md new file mode 100644 index 0000000..5f5640f --- /dev/null +++ b/wiki/testing/quality/source-text-wiring-assertions.md @@ -0,0 +1,116 @@ +--- +id: testing-quality-source-text-wiring-assertions +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ + - https://pitest.org/quickstart/basic_concepts/ + - https://jestjs.io/docs/expect + - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier + - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html +last_verified: 2026-08-07 +related: + [ + testing-quality-tests-that-cannot-fail, + testing-quality-behavior-not-implementation, + testing-quality-guard-shape-vs-consequence, + testing-quality-harness-reverse-controls, + testing-quality-surviving-mutant-equivalence-triage, + backend-common-change-impact-call-site-enumeration, + ] +--- + +# Asserting on Source Text That a Wiring Call Still Exists + +## When this applies + +A test reads a source file as a string and asserts by regex that some call is +present — a cleanup call in every handler, a logging call after each branch, a +teardown in each exit path — because the behavior has no reachable seam at this +test level. You are choosing the assertion shape, or such a guard is green while +one of the call sites is gone. + +Deciding whether a source-text assertion is warranted at all → +[testing-quality-behavior-not-implementation]. Guards that scan every shipped +artifact for a structural shape → [testing-quality-guard-shape-vs-consequence]. + +## Do this + +1. **Enumerate the call sites the guard is meant to protect, from the source, + before writing the pattern.** The defect this class of guard exists to catch + is "one of N sites was dropped", so the site list is the assertion's real + subject ([backend-common-change-impact-call-site-enumeration]). Write the list + down: the assertion count comes from it, and step 6 re-runs against it. + +2. **Give each site an anchor that occurs exactly once in the file, and assert + one pattern per site.** A regex is satisfied by _any_ anchor–call pair that + fits its bound, so an anchor appearing at two sites lets either one satisfy + the other's assertion. Count the anchor's occurrences before using it: + +| How the sites are separated | Assertion shape | +| -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Each site follows a call or branch condition whose text appears once in the file | Bounded order anchor: `/[\s\S]{0,N}/`, after confirming the anchor's occurrence count is 1 | +| Each site lives in a separate named function | Slice the source to that function's body (from its declaration to the next top-level declaration) and assert the call inside the slice — the slice bounds the search without needing a unique anchor | +| Sites share the callee and differ only by argument | Assert the full call text including the argument, inside the slice or after the unique anchor — the callee name alone is satisfied by any site | +| No anchor is unique and the sites are not separable into slices | The file gives the guard nothing to bind to: extract the sites into named functions first, or test the behavior at a level that reaches it | + +3. **Set the bound N from the distance the anchor and call actually have in the + current source, plus the length of one statement**, so a legitimately inserted + line does not redden the guard. The bound is what limits the anchor's reach: + measured 2026-08-07 in Node, `{0,20}` and `{0,20}?` return the same verdict on + every input — greedy versus lazy changes which match is reported, not whether + one exists, so a lazy quantifier adds no constraint. + +4. **Prove each assertion by deleting exactly its own site and requiring exactly + that assertion to redden**, leaving the other sites intact. This is a + hand-seeded mutation: Stryker's nearest operator, Block Statement, "removes + the content of every block statement" — it empties a whole block rather than + one call, so tools do not generate this edit for you. + +5. **Run a semantics-preserving control and require green** — change a comment or + reformat the file. A source-text pattern is one whitespace assumption away + from asserting formatting, and the control is what separates "the guard reads + the wiring" from "the guard reads the layout" + ([testing-quality-harness-reverse-controls]). + +6. **Re-run step 1 whenever the enclosing function grows a branch.** A per-site + guard has no signal for a site that was never enumerated, so the site list — + not the assertions — is what has to be kept current. + +7. **Name each test after its site**, so a reviewer reading a failure knows which + call went missing rather than that "the count changed". + +## Edge cases + +| Case | Then | +| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The behavior is reachable through the public interface after all | Assert the behavior and delete the source-text guard — a text assertion passes on a call that is present and broken | +| The sites are generated from a template or macro | Assert on the generator's input at its one site, and add one behavior test on the generated output; per-site text assertions on generated files re-assert the generator | +| The anchor call itself is renamed in a refactor | The guard reddens on correct code — that is the coupling this guard buys; update anchor and call together, and re-run step 4 for each site | +| Two sites legitimately share one anchor (a branch and its else) | Anchor on each arm's own branch-condition text, or slice per arm; a shared anchor makes the two assertions interchangeable | +| The anchor's occurrence count rises from 1 to 2 in a later change | Both assertions became satisfiable by either site — re-run step 2 and pick a new anchor, then re-prove with step 4 | +| The pattern must survive a formatter that reflows lines | Match on the token sequence with `[\s\S]{0,N}` between tokens rather than on a literal multi-line string, and keep the step-5 reformat control | +| A site's mutant survives despite the assertion | Classify it before strengthening the pattern ([testing-quality-surviving-mutant-equivalence-triage]) — the call may be redundant at that site | + +## Instead of + +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Assert `matches.length >= n` or `toHaveLength(n)` for a call that appears at several sites | Assert one anchored or sliced pattern per enumerated site | A lower bound is satisfied by the surviving sites, so deleting the one site the guard was written for keeps it green — the exact defect the guard exists to catch passes it | +| Raise the count assertion to an exact `toHaveLength(n)` after finding this gap | Keep the per-site assertions and re-run the enumeration (step 6) when the function grows a branch | An exact count reddens on legitimate additions and names no site; the per-site guard names the site, and step 6 is what covers additions | +| Match the call anywhere in the file (`/setPendingEntry\(null\)/`) | Bind it to a once-occurring anchor or to the enclosing function slice | An unbound match is satisfied by any one of the sites, making N sites indistinguishable from one | +| Add `?` to the quantifier to keep the anchor from reaching a later site | Set the bound N, and confirm the anchor occurs once | Laziness changes which match is reported, not whether the pattern matches; the reach is decided by N and by the anchor's uniqueness | +| Ship the anchored guard because the suite is green | Delete each site once and require its own assertion red | Narrowing a pattern is the easiest way to narrow it to nothing; a pattern that matches nothing and one that matches everything both read as green | +| Use a source-text guard as the primary coverage for the handler's logic | Keep it as a wiring check and test the behavior at the level that can reach it | Text guards fail on refactors that preserve behavior and pass on a call whose implementation broke | + +## Sources + +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the Block Statement mutator "removes the content of every block statement"; it empties a block rather than removing one call, which is why the per-site deletion in step 4 is hand-seeded rather than tool-generated +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; a per-site deletion that leaves the suite green is exactly this verdict for the site the guard names +- https://jestjs.io/docs/expect — `toHaveLength` asserts a `.length` value; applied to a match array it compares a total and carries no information about which element is missing +- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier — documents `{min,max}` as a bounded repetition and `?` as the non-greedy form that "will try to match as few times as possible"; non-greediness governs how much the quantifier consumes, not whether the overall pattern matches (measured, step 3) +- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27), cited for the change-detector category itself: a test coupled to source shape fails on behavior-preserving edits. The article body was not retrievable in full on 2026-08-07, so nothing here is quoted from it +- Measurement 2026-08-07 (Node): `/ANCHOR\([\s\S]{0,20}CALL\(/` and its lazy form `{0,20}?` returned identical verdicts on four inputs (anchor-then-call in range, call-before-anchor only, call beyond the bound, call both before and after the anchor) — the bound decides reach, and a call elsewhere in the file neither blocks nor is excluded by the pattern +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail panel): `setPendingEntry(null)` appears at four sites (close, list-select, post-resolve, error branch). A `>= 3` count assertion stayed green after a mutant deleted the post-resolve site. Binding it to the once-occurring anchor `resolveBuildingDetailEntry\(` within a 200-character bound produced red on that same mutant, and a comment-only edit kept it green diff --git a/wiki/testing/quality/surviving-mutant-equivalence-triage.md b/wiki/testing/quality/surviving-mutant-equivalence-triage.md new file mode 100644 index 0000000..286b9ab --- /dev/null +++ b/wiki/testing/quality/surviving-mutant-equivalence-triage.md @@ -0,0 +1,109 @@ +--- +id: testing-quality-surviving-mutant-equivalence-triage +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ + - https://pitest.org/quickstart/basic_concepts/ + - https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/ + - https://testing.googleblog.com/2021/04/mutation-testing.html +last_verified: 2026-08-07 +related: + [ + testing-quality-tests-that-cannot-fail, + testing-quality-harness-reverse-controls, + testing-quality-minimum-case-set, + testing-quality-behavior-not-implementation, + testing-quality-source-text-wiring-assertions, + backend-common-change-impact-call-site-enumeration, + ] +--- + +# A Surviving Mutant Before You Write a Test for It + +## When this applies + +A mutation run (PIT, Stryker, or a hand-seeded mutation) left a mutant alive on +code you own, and you are deciding what to change. Also when a reviewer asks for +a test to cover a specific surviving mutant, or a defensive branch you added has +a comment explaining why it is needed and its mutant survives. + +Building the mutation harness itself, or citing its score → +[testing-quality-harness-reverse-controls]. + +## Do this + +1. **Classify the survivor before writing anything.** A live mutant is one of + three things, and only one of them is a missing test. Start from the tool's + own verdict, then decide the remaining split by argument over the input + domain, not by trying one value: + +| Signal | Class | Do | +| --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| The tool reports `No coverage` — "there were no tests that exercised the line of code where the mutation was created" | Uncovered line | Add a test that reaches the line, then re-run the mutation; the kill/survive question is not answerable until it is covered | +| The line is covered, and some input in the branch's domain makes original and mutant differ | Missing or weak test | Add that input as a case ([testing-quality-minimum-case-set]), then re-run the mutation and require red | +| The line is covered, and step 2 produces a proof that no input in the branch's domain makes them differ | Equivalent mutant — the branch is redundant | Steps 3–5 | + +2. **Prove equivalence over the domain, not over one input.** Name the condition + elsewhere in the code that absorbs the mutated branch — a later comparison, a + type coercion, a caller-side check — and state why it covers the branch's + whole input set (`Number(s) === 0` for every `s` the branch accepts is + rejected by a following `parsed > 0`). One value that agrees is consistent + with equivalence and does not establish it. When you cannot write that + argument, treat the mutant as the missing-test row and add the case: Stryker + states "There is no definitive way for Stryker to find and ignore them", so + the burden of proof sits on the deletion, not on keeping the branch. + +3. **Delete the redundant branch and keep the absorbing condition from step 2 as + the single decision point.** Stryker's guidance names two acceptable + outcomes — "The only solution is by finding these by hand, which is time + consuming and try to rewrite the code so it won't occur, or accept that you + won't make 100%" — so recording the mutant as a classified survivor is the + correct alternative when the branch stays for a reason in the edge table. + +4. **Correct the justification comment in the same edit, using the argument from + step 2.** When the branch removed in step 3 carries a comment saying why it + exists, that comment asserted a mechanism the equivalence proof contradicts, + so leaving it in place moves a false premise onto whichever condition remains. + Replace it with the absorbing condition you named, or delete it when that + condition is self-evident. + +5. **Re-run the full suite and read a changed pass count by what moved:** + +| After the deletion | Read it as | Do | +| --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| Same pass count | The branch had no observable consequence the suite asserts | Keep the deletion | +| A behavior test reddens (asserting an input/output pair) | Step 2's domain argument is wrong — the branch is live | Restore it and re-classify with the input that failed | +| Only a test that names the branch itself reddens (source-shape guard, branch-coverage threshold, path snapshot) | The deletion is correct and the test asserted the implementation | Update that test to the new shape ([testing-quality-behavior-not-implementation]) | + +## Edge cases + +| Case | Then | +| ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The branch is unreachable through the public interface but reachable through another entry point | Not equivalent — it is uncovered from this level. Move the test to the level that reaches it ([testing-quality-behavior-not-implementation]) rather than deleting the branch | +| The mutated behavior differs only in a dimension the suite is not meant to cover (logging, metrics, timing) | PIT's second undetectable class — exclude that region from the mutation set instead of adding a test to chase it | +| The redundant branch exists for readability at a trust boundary (validating external input twice) | Keep it and record why in the comment as a deliberate defense-in-depth, not as a correctness claim; the mutant stays a classified survivor | +| The equivalence holds only for the current caller set | Treat it as coverage, not equivalence: enumerate the call sites ([backend-common-change-impact-call-site-enumeration]); when a future caller could pass the absorbed input, the branch is live | +| Several mutants survive in the same function | Classify each one separately — one verdict covering all of them hides whichever is the other kind | +| The tool reports a 100% kill rate with no survivors at all | Read that as a harness signal, not a code signal, and run the no-op control ([testing-quality-harness-reverse-controls]) | +| The survivor is on a wiring call asserted by a source-text regex rather than by behavior | The count-style assertion is what let it live → [testing-quality-source-text-wiring-assertions] | + +## Instead of + +| If you are about to | Do this instead | Why | +| -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Read every surviving mutant as a test gap and write a case for it | Classify it against the step-1 table first | An equivalent mutant cannot be killed by any correct test, so the case you add asserts a behavior the code does not have and passes for every implementation | +| Declare equivalence because one input produced the same result | Write the step-2 domain argument, or classify it as a missing test | Agreement on one value is what both classes look like; the deletion in step 3 changes production code, so it needs the stronger claim | +| Chase a 100% mutation score by testing the survivors that resist | Take one of the two documented outcomes: rewrite the code so the mutant cannot arise, or record the survivor as classified and accept the score | Stryker states there is no definitive way to detect equivalent mutants and names accepting a sub-100% score as an acceptable outcome | +| Delete a redundant branch and leave its explanatory comment on the remaining condition | Replace the comment with the absorbing condition from step 2 | The comment stated why the deleted branch was necessary; the equivalence proof contradicts that claim, and the next reader refactors the surviving condition against it | +| Suppress or ignore the mutant in the tool's config to get the run green | Record the classification in the code (step 4) and leave the mutant visible | A suppression carries no reason, so the next person re-derives the same analysis; a corrected comment carries it | + +## Sources + +- https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ — "There is no definitive way for Stryker to find and ignore them"; "The only solution is by finding these by hand, which is time consuming and try to rewrite the code so it won't occur, or accept that you won't make 100%" — both halves of that sentence are load-bearing here: rewriting is one outcome, a classified survivor is the other +- https://pitest.org/quickstart/basic_concepts/ — "Not all mutations will behave differently than the unmutated class. These mutants are referred to as **equivalent mutations**"; "The resulting mutant behaves in exactly the same way as the original"; and the distinct verdicts "Survived: The mutation was not detected by the covering test" vs "No coverage: The same as Survived except there were no tests that exercised the line of code where the mutation was created" — the split in step 1 +- https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/ — the mutant state set and `detected / valid` scoring, which is what makes classifying a survivor a prerequisite to reporting the number +- https://testing.googleblog.com/2021/04/mutation-testing.html — inserting faults and requiring test failure is the measurement; a fault that changes no observable behavior is not one +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail URL parsing): a mutant that deleted the empty-string guard on `?buildingId=` survived. The domain argument was that the guard's whole input set is strings that `Number()` maps to `0` or `NaN`, both of which the following `parsed > 0` rejects — so no accepted input distinguishes the two. The branch's comment claimed "an empty string is otherwise read as 0", which that argument contradicts. Deleting the branch and rewriting the comment left all 49 tests passing at the same count diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index b0534af..3bfa6cc 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -16,7 +16,7 @@ sources: - https://git-scm.com/docs/git-checkout - https://git-scm.com/docs/git-restore last_verified: 2026-08-05 -related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] +related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-surviving-mutant-equivalence-triage, testing-quality-source-text-wiring-assertions, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] --- # Proving a Test Can Fail @@ -83,7 +83,7 @@ suite reported as covered, or you are auditing a suspiciously green suite. | Case | Then | |------|------| | Mutating the code under test is impractical right now (slow build, shared branch) | Invert the expected value in the assertion instead and require red — this proves the assertion executes and compares, though not which code defects it catches | -| Auditing a whole suite, not one test | Run an automated mutation-testing tool (PIT, Stryker) and treat surviving mutants in changed code as missing or defective tests | +| Auditing a whole suite, not one test | Run an automated mutation-testing tool (PIT, Stryker), then classify each surviving mutant in changed code before writing a test for it — a missing test, an equivalent mutant, or an uncovered line ([testing-quality-surviving-mutant-equivalence-triage]) | | The mutation run is your own script rather than PIT/Stryker | Prove the harness discriminates before citing its score — a semantics-preserving no-op must survive ([testing-quality-harness-reverse-controls]) | | A test intentionally has no outcome assertion (smoke test: module loads, page renders) | Keep it only when the regression it guards manifests as a throw; name it as a smoke test so reviewers do not count it as behavior coverage | | The always-green test is a snapshot approved without reading | Snapshot rules → [testing-quality-behavior-not-implementation] |