diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..672c233 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,349 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 4 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: `1717316a-…jsonl` (2 rows), `ab5516dc-…jsonl` (2 rows) — **4** +candidates, all ingested as new pages, none dropped. + +> Correction, recorded because it nearly cost a candidate: the first three sections +> of this report were written for 3 insights. I had counted the queue with +> `wc -l`, which reported 3 because one file's final line carried no trailing +> newline. The 4th row surfaced only when the retirement step parsed the files as +> JSON and moved 4. It had been marked `ingested` by that step while never having +> been researched or routed; it was then processed in full (insight 4 below) and +> folded into this same PR rather than left with a false status. Row counts now +> come from a JSON parse, not from `wc -l`. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. A spy that captures one argument leaves the same call's siblings unasserted + +**Claim.** When a review flags one argument of one wiring call and you hold the fix +with a spy, record the whole call and assert every caller-decided argument; +asserting the constant's value is a different claim from asserting that the call +site passes it on, and extracting a resolver moves that gap up a layer. + +**Sources checked.** + +- https://docs.python.org/3/library/unittest.mock.html — `assert_called_with` is + "a convenient way of asserting that the last call has been made in a particular + way" (whole-call, last-call only); a *spec*'d mock "will introspect the + specification object's signature when matching calls … regardless of whether + they were passed positionally or by name", and "using autospec will catch + mistakes where the mock is called with the wrong signature". +- https://jestjs.io/docs/expect — `.toHaveBeenCalledWith` checks arguments "with + the same algorithm that `.toEqual` uses"; `expect.objectContaining` matches "a + received object which contains properties that are present in the expected + object" — a **subset** match, which is why omitted keys stay unasserted (this + became an edge-case row rather than a recommendation). +- https://github.com/mockito/mockito/blob/main/mockito-core/src/main/java/org/mockito/ArgumentMatchers.java + — "If you are using argument matchers, **all arguments** have to be provided by + matchers." +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation + was not detected by the covering test" (how step 5 reads a green per-argument run). + +**How verified.** Reproduced both halves locally this session (Python 3, +`unittest.mock`), not just cited: + +| Run | Correct call | Mutated call | +|---|---|---| +| partial-capture stub (stores only `kw["port"]`) | passes | **passes** — `host` `0.0.0.0`→`127.0.0.1` undetected | +| `Mock(spec=…)` + `assert_called_with(host=…, port=…, tls=…)` | passes | fails | + +The green cell on the correct call is the no-op control: the stronger assertion +discriminates rather than always failing. A second run held +`DEFAULT_PORT == 8914` green across three call-site variants (reads the constant, +reads an extracted `resolve_port()`, hardcodes `8770`) while the recorded-call +assertion was green for the first two and red only for the hardcoded one — the +"value vs wiring" split, and the evidence that the wiring assertion survives the +refactor. + +**Confidence: verified** (official docs for every API claim + local reproduction). +The field evidence behind the candidate (a 6-round audit where `port`, then the +extracted resolver, then `host` each survived in turn, the last being a change +whose only failure surface is a Kubernetes readiness probe) is recorded in the +page's Sources as a field measurement, kept distinct from the reproduction. + +### 2. pg_trgm degenerates to a full-index scan below three characters + +**Claim.** A `LIKE`/`ILIKE` wildcard segment of fewer than three characters yields +no extractable trigrams, so a pg_trgm GIN/GiST index is scanned in full and the +cost moves into the heap recheck — while the plan still reads `Bitmap Index Scan`. + +**Sources checked.** + +- https://www.postgresql.org/docs/current/pgtrgm.html — "For both `LIKE` and + regular-expression searches, keep in mind that a pattern with no extractable + trigrams will degenerate to a full-index scan." Also the padding rule ("Each + word is considered to have two spaces prefixed and one space suffixed…"), which + I checked precisely because it is a trap: `show_trgm('cat')` returns four + trigrams, so "short strings have no trigrams" is **wrong** as stated — padding + applies to a *word being indexed*, while a `%…%` pattern asserts no word + boundary to pad against. The page states it that way. +- https://postgrespro.com/list/thread-id/1821635 — Amit Langote, pgsql list + (2013-05-31): "get_wildcard_trigrams return no trigrams for wildcard part 'st' + since charlen < 3"; "Hence, GIN_SEARCH_MODE_ALL mode is used and results in full + index scan instead of trigrams being used." This is what lets the page state the + rule per wildcard-delimited segment rather than per pattern. +- https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — 2-gram index; + its comparison table rates "Full text search with 1-2 characters keyword" as + "Fast" vs pg_trgm's "Slow (\*2)", **and** lists pg_bigm's operators as "LIKE + only" vs pg_trgm's "LIKE (~~), ILIKE (~~*), ~, ~*". That constraint corrects the + candidate, which suggested pg_bigm without noting it is not a drop-in for an + `ILIKE` workload; the page splits those into two decision rows. Footnote (\*2) + turned out to state the mechanism independently of the PostgreSQL docs — + "Because, in this search, only sequential scan or index full scan (not normal + index scan) can run" — so the central claim now has two unrelated primary + sources. + +**How verified.** Doc quotes fetched and read this session. The quantitative half +is the candidate's own field `EXPLAIN (ANALYZE, BUFFERS)` on a 4.64M-row table +(3-char 17 ms / 4 buffers vs 2-char 18,789 ms / 121,837 buffers, +`Rows Removed by Index Recheck: 4,640,486`, 3 rows matched) — **not** re-run here: +no PostgreSQL was reachable in this environment (`psql` absent, Docker daemon +down, no postgres pod in the local cluster). It is labelled a field measurement +with its date and table size, and no claim in the page depends on my having +re-run it. + +**Confidence: verified** (mechanism doc-sourced; magnitude field-measured). + +### 3. `@Transactional(timeout=N)` does not reach a raw JdbcTemplate path by itself + +**Claim.** The declared timeout reaches `JdbcTemplate` only through a +`ConnectionHolder` bound by `JpaTransactionManager`; when that bind is skipped the +raw path runs unbounded, and the two paths raise **different** Spring exceptions. + +**Sources checked (source read at pinned tags, not from memory).** + +- `JpaTransactionManager` javadoc — "To be able to register a DataSource's + Connection for plain JDBC code, this instance needs to be aware of the + DataSource (`setDataSource(DataSource)`)"; "will autodetect the DataSource used + as the connection factory of the EntityManagerFactory, so you usually don't need + to explicitly specify the 'dataSource' property"; "this requires a + vendor-specific `JpaDialect` to be configured". +- `JpaTransactionManager.java` @ v6.2.0 — `conHolder.setTimeoutInSeconds(...)` + sits inside `if (getDataSource() != null)` **and** requires + `getJpaDialect().getJdbcConnection(em, …) != null`, else it logs "Not exposing + JPA transaction … does not support JDBC Connection retrieval". + `DefaultJpaDialect.getJdbcConnection` returns `null`. This is a **correction**: + the candidate named only the DataSource wiring, so a Boot app (where the + DataSource is autodetected) would have looked exempt; the dialect branch and the + DataSource-instance-identity branch are separate failure modes, and the debug + log line is a checkable diagnostic. The page's step 2 is a 4-row table because + of this. +- `DataSourceUtils` javadoc + `DataSourceUtils.java` @ v6.2.0 — `applyTimeout` + applies "the current transaction timeout, **if any**"; it looks the holder up by + the `DataSource` instance and otherwise falls back to the passed timeout only + `if (timeout >= 0)`. `JdbcTemplate.applyStatementSettings` calls it with + `getQueryTimeout()`, whose field default is `private int queryTimeout = -1` — + so with no holder, nothing is set at all. +- Exception split: Hibernate `PostgreSQLDialect.java` maps SQLState `"57014"` → + `org.hibernate.QueryTimeoutException`, and `HibernateJpaDialect.java` @ v6.2.0 + converts that to `org.springframework.dao.QueryTimeoutException`. On the raw + path, PgJDBC's `PSQLException extends SQLException` with + `PSQLState.QUERY_CANCELED = "57014"`, so `SQLExceptionSubclassTranslator`'s + `instanceof SQLTimeoutException` branch misses and its + `SQLStateSQLExceptionTranslator` fallback maps class `57` + (`Set.of("08","53","54","57","58")`) → `DataAccessResourceFailureException`. +- **Version boundary found while verifying, which the candidate did not know.** + `main` special-cases `"57014".equals(sqlState)` → `QueryTimeoutException`. I + fetched the file at seven released tags to find where it starts: + +| Tag | `"57014".equals` present | +|---|---| +| v5.3.31, v6.0.0, v6.2.0, v6.2.1, v6.2.3, v6.2.5, v6.2.8 | no | +| v7.0.0, `main` | yes | + +So the candidate's exception claim is correct for Spring Framework ≤ 6.2.x and +inverts at 7.0.0. The page states both, and an `Instead of` row requires pinning +the framework version any single-branch handler assumes. + +**How verified.** Every quote above was fetched this session; the version table +came from downloading the same file at each tag and grepping it. The timing half +(Hibernate cancelled at 10,012 ms → 400 vs raw JdbcTemplate 151,558 ms / +163,489 ms → 500 on one annotated endpoint, 11/11 errors over 30 days matching the +per-path status split) is the candidate's production p6spy measurement, labelled as +such. + +**Confidence: verified** (framework behaviour read from pinned source + javadoc; +production magnitudes field-measured). + +### 4. A constructor default is unasserted by both usual test shapes + +**Claim.** A default named as a number in a spec (`ttl_s=600`, `max_tokens=256`) is +guarded by neither a mechanism test that passes the value in nor a test that +constructs with defaults without exercising them. Push the default to its +observable point and require red in **both** mutation directions. + +**Sources checked.** + +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation + was not detected by the covering test"; a green suite under a changed default is + that verdict for the default. +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — + the published mutator set is operator/literal/block based, so a tool generates + one variant of a literal rather than both directions; the page states both + directions explicitly instead of relying on the tool. +- https://docs.python.org/3/reference/compound_stmts.html — "Default parameter + values are evaluated from left to right when the function definition is + executed", the basis for the shared-mutable-default edge case. + +**How verified.** Reproduced (Python 3, a TTL + cap store with four existing tests: +two constructing with defaults, two passing values in explicitly): + +| Mutant | Existing suite | With the added default tests | +|---|---|---| +| baseline (`ttl 600`, `cap 256`) | GREEN | GREEN *(control)* | +| `ttl_s 600 → 1` | **GREEN** | RED | +| `max_tokens 256 → 1` | RED *(incidental)* | RED | +| `ttl_s → 6000`, `max_tokens → 9999` | **GREEN** | RED | + +This **sharpened the candidate**, which stated the survival as a flat property. It +is direction-dependent: shrinking a cap below what an existing test happens to +exercise is caught incidentally (row 3), while the **grow** direction was +uncatchable by the existing suite in every configuration tried — a test that issues +N items passes for every cap ≥ N, and one that consumes immediately passes for every +TTL > 0. The page leads with that asymmetry, and the baseline-GREEN row is the +control showing the added boundary cases are not simply always-failing. + +**Confidence: verified** (mutation semantics + language behaviour doc-sourced; +mechanism reproduced locally; the CSRF-store magnitudes kept as field measurement). ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via `INDEX.md` → the three domain indexes, then read every page whose "load +when" line overlapped. Full-body reads: `testing-quality-tests-that-cannot-fail`, +`testing-mocking-what-to-mock`, `databases-indexing-index-selection`. Targeted +reads (grep for timeout/JdbcTemplate/trigram/LIKE/arg-capture terms, to establish +absence of coverage): the remaining ids below. + +Pages read: testing-quality-minimum-case-set, testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, testing-mocking-what-to-mock, testing-quality-behavior-not-implementation, backend-common-change-impact-call-site-enumeration, databases-indexing-index-selection, databases-query-optimization-reading-execution-plans, databases-indexing-partial-and-expression-indexes, databases-indexing-covering-indexes, backend-common-orm-transaction-boundaries, backend-common-reliability-timeouts-and-retries, backend-common-errors-exception-handling, backend-java-jpa-persistence-context, backend-java-spring-proxy-pitfalls + +**Overlaps found, and why each is composition rather than duplication.** + +| Existing page | Overlap | Resolution | +|---|---|---| +| `testing-mocking-what-to-mock` | Its step 1 last row and step 2 already say to "assert the **outbound contract**: which command, with what arguments", and one edge row says to "deep-equal the full stub-recorded call sequence" | Closest neighbour, but its subject is *whether* to replace a dependency — folding assertion-completeness in would break "one case per page". Kept separate; added the new id to its `related:` and a pointer on the outbound-contract row | +| `testing-quality-minimum-case-set` | Owns boundary-value selection for a function's inputs (60 body lines, room to merge) | Kept separate: its subject is the case set for the *behaviour*, while insight 4's is a shipped **default** whose trigger is a spec-named number plus an existing suite, with its own bidirectional procedure. Reciprocal `related:` added | +| `testing-quality-harness-reverse-controls` | Owns "prove the harness discriminates" | Cited from insight 4's step 5 — boundary cases one unit from a limit are where a test-side off-by-one imitates a caught mutant | +| `testing-quality-tests-that-cannot-fail` | Owns per-assertion mutation granularity and the "testing the mock instead of the code" row | The new page cites it for the mutation step instead of restating it; added the new id to its `related:` | +| `databases-indexing-index-selection` | Line 51 routes `LIKE '%term%'` to "a trigram or full-text index type" | That advice has an unstated precondition — exactly the new page. Extended the row to carry the minimum-keyword-length pointer, plus a `related:` link | +| `databases-query-optimization-reading-execution-plans` | Owns plan reading generally | Cited from "When this applies"; the new page adds only the trigram-specific counter to read (`Rows Removed by Index Recheck`) | +| `backend-common-reliability-timeouts-and-retries` | One row: "Dependency is a DB with its own driver timeout — set both the driver statement timeout and your outer deadline" | Consistent, and the new page is the Spring-specific mechanism for why the driver timeout is silently absent. Reciprocal `related:` added | +| `backend-common-orm-transaction-boundaries` | Transaction scope; no timeout content (grep: only an external-API-in-transaction row) | Reciprocal `related:` added | +| `backend-java-spring-proxy-pitfalls` | Owns "the annotation had no effect at all" | Cited as the upstream check in step 2's table, so the two failure modes stay distinguishable | +| `backend-java-jpa-persistence-context`, `backend-common-errors-exception-handling`, `databases-indexing-partial-and-expression-indexes`, `databases-indexing-covering-indexes` | No overlapping directive | Linked where relevant (case-folded expression index; exception handling) | + +**Conflicts flagged:** none. No existing page states a contradicting directive. + +**Coverage gaps confirmed by grep before creating:** `trgm|trigram|ILIKE` matches +exactly one file in the whole wiki (`index-selection.md`, the one row above); +`call_args|assert_called_with|toHaveBeenCalledWith|argument captor` matches exactly +one (`what-to-mock.md`); no page mentions `statement_timeout`, `JdbcTemplate`, or +`QueryTimeout`. For insight 4, `default value|defaults|default argument` across +`wiki/testing/**` returned six hits, every one incidental (a factory filling +fixture defaults, a permissive-schema note, a `f(x=None)` trap) — no page states a +directive about a shipped default's own test. + +**Format invariants checked mechanically after writing:** body lines 94 / 75 / 90 +(limit 120); every `related:` id and inline `[page-id]` reference resolves to a +page in this checkout (16/16); no banned vague qualifier in any directive (the two +`usually`/`Consider` hits were a verbatim Spring javadoc quote in Sources, left +intact, and one `Instead of` anti-pattern label, reworded); every prohibition word +occurs only inside an `Instead of` row or a quoted source. + +## Open-PR check + +Listed all 17 open `knowledge/*` heads. Three of them (**#72, #52, #49**) produced +suspiciously **empty** `wiki/` diffs on a first pass, because `git fetch origin +` and `repos/choiyounggi/dev-loop/git/refs/heads/` both 404 for +them. Rather than read an empty diff as "no overlap", I re-read all three through +`refs/pull//head`, and then established the actual cause: those heads live on +the contributor fork `dch0202-rsquare/dev-loop` (this flush's own account), not on +upstream — all three refs resolve there (`6a3ff08`, `bd03fbe`, `346dd95`). They are +alive and pushable, so `fold` was a genuinely available verdict for them; it was +not taken for the content reasons below. This PR is likewise opened from the fork. + +Files touched by each open head, matched against the three candidates: + +| Candidate | Overlapping open PRs | Verdict | +|---|---|---| +| 1 — captured call arguments (testing/mocking) | #52 adds `testing/quality/source-text-wiring-assertions.md`; #49 adds `testing/quality/value-preserving-refactor-assertions.md` + `unasserted-return-fields.md`; #47/#52 modify `tests-that-cannot-fail.md` | **new** | +| 4 — default values under test (testing/quality) | Same three heads as candidate 1, plus #49's `unasserted-return-fields` and #47/#52's edits to `tests-that-cannot-fail.md` | **new** | +| 2 — pg_trgm short patterns (databases/indexing) | none — zero open heads touch `wiki/databases/**` | **new** | +| 3 — raw JDBC in a JPA transaction (backend/java/jpa) | none — the backend-touching heads (#68, #58, #56, #55, #51, #50, #72) are all under `backend/common/**` or `backend/python/**`; zero touch `wiki/backend/java/**` | **new** | + +**Why candidate 1 is `new` and not `fold`**, having read all three in-flight pages +in full or in relevant part: + +- `#52 source-text-wiring-assertions` — same *word* "wiring", different subject: it + is about asserting by **regex over source text** when no seam exists (anchor + uniqueness, comment stripping, bound sizing). The new page is about the case + where a seam **does** exist and a recorder captured the call. Its own "Instead + of" even routes away from text guards when the behaviour is reachable, which is + the situation the new page occupies. No directive is duplicated. +- `#49 value-preserving-refactor-assertions` — nearest in spirit (a literal + replaced by a config read; sentinel substitution to prove the dependency). Its + trigger is a *value-preserving refactor of a source of truth*; the new page's is + *a reviewed fix to one argument of one call*, and its distinct content is + argument-set completeness across one call — the thing #49 does not address. +- `#49 unasserted-return-fields` — the mirror direction (fields a function + **returns** that no assertion reads). The new page is the call/argument + direction. Deliberately kept as siblings. + +**Why candidate 4 is `new`:** `#49 unasserted-return-fields` is the closest +in-flight page — it also turns on "a value no assertion reads" — but its subject is +fields a function **returns** on a given call, and its remedy is per-field +assertions plus cross-field relations. Candidate 4's subject is a **default that is +never supplied**, and its remedy is bidirectional mutation at the default's +observable point; the grow-direction asymmetry has no counterpart there. +`#49 value-preserving-refactor-assertions` covers a literal moved behind a config +read, the opposite direction of travel (the value is already asserted; the question +is whether the caller reads it). + +No candidate is a pending duplicate, so nothing was dropped and no sibling +duplicate PR is opened here. + +**Merge-order note for the owner:** this branch adds one id to +`wiki/testing/quality/tests-that-cannot-fail.md`'s `related:` list, a file #47 and +#52 also modify. It is a single-line frontmatter addition. If any of #47/#49/#52 +merge first, the reciprocal links to `testing-quality-source-text-wiring-assertions`, +`-value-preserving-refactor-assertions` and `-unasserted-return-fields` become +resolvable and are worth adding to the new page then — they are intentionally +omitted now because AGENTS.md invariant 4 requires every `related:` id to resolve, +and those pages do not exist on `main`. + +## Citation audit (post-write) + +Cross-Check: mechanical citation audit rather than an LLM second opinion — every +source-derived claim in the three new pages was re-grepped against the bytes +actually fetched this session (21 checks: Spring at 9 tags, Hibernate +`PostgreSQLDialect`, `HibernateJpaDialect`, PgJDBC `PSQLException`/`PSQLState`, +Mockito `ArgumentMatchers`, pg_bigm docs). 19 confirmed; one intended-absence check +confirmed absent (`"57014".equals` not in v6.2.8); **one failed and was fixed** — +the pg_bigm sentence had been taken from a fetch summary and did not match the raw +file byte-for-byte (the source reads `**2-gram**` with a line break and +`[PostgreSQL](…)` link, and the comparison cell is "Slow (\*2)", not "slow"). The +quote and the table cells were re-derived from the raw file cell by cell, which +also surfaced footnote (\*2) — a second independent statement of the mechanism — +and footnote (\*1), now recorded as an edge case. No claim was left resting on a +summarizer's paraphrase. Structural re-validation after insight 4 landed: all +`related:`/inline ids across the whole wiki resolve (the only two regex hits were a +literal `[a-z0-9]` character class in two pre-existing pages); the four new pages +are 94 / 75 / 90 / 77 body lines against the 120 limit; every page appears in its +domain index; and no index table row is malformed — one row was caught missing its +closing pipe and fixed. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Domain / category | Page | New category? | +|---|---|---|---| +| 1 | `testing` / `mocking` | `wiki/testing/mocking/captured-call-arguments.md` (`testing-mocking-captured-call-arguments`) | No — `mocking` is the category that owns stub/spy mechanics; `quality` owns whether a test can fail (already cited), and the subject here is what the double records | +| 2 | `databases` / `indexing` | `wiki/databases/indexing/trigram-index-short-patterns.md` (`databases-indexing-trigram-index-short-patterns`) | No — `indexing` owns index-type suitability; the case is a precondition on one index type, and `query-optimization/reading-execution-plans` stays the owner of plan reading | +| 3 | `backend` / `java` → `jpa` | `wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md` (`backend-java-jpa-raw-jdbc-inside-a-jpa-transaction`) | No — the mechanism is `JpaTransactionManager`/`JpaDialect`, so it belongs in the `jpa` category rather than `spring` (which owns proxy-level "the annotation did nothing") or `backend/common` (language-agnostic principles; this is stack-specific source behaviour) | + +| 4 | `testing` / `quality` | `wiki/testing/quality/default-values-under-test.md` (`testing-quality-default-values-under-test`) | No — `quality` owns assertion sufficiency and mutation proof, which is what the case is about; `mocking` owns doubles and would misfile a page whose subject needs no double | + +Plumbing updated: `wiki/testing/index.md`, `wiki/databases/index.md`, +`wiki/backend/java/index.md` each +1 "load when" row (testing +2 — one per testing +page); `log.md` +1 ingest entry. +`INDEX.md` unchanged — all three domains are already listed and their "route here +when" lines already cover these cases. diff --git a/log.md b/log.md index c930fc2..8a77674 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD] 0. Kept separate from minimum-case-set (case set for the behaviour, not for a shipped default) and from the in-flight #49 unasserted-return-fields (return direction) — reciprocal related links added to minimum-case-set, tests-that-cannot-fail and captured-call-arguments; testing index +1 row. diff --git a/wiki/backend/common/orm/transaction-boundaries.md b/wiki/backend/common/orm/transaction-boundaries.md index d4d067f..c3dc57d 100644 --- a/wiki/backend/common/orm/transaction-boundaries.md +++ b/wiki/backend/common/orm/transaction-boundaries.md @@ -9,7 +9,7 @@ sources: - https://docs.spring.io/spring-framework/reference/data-access/transaction/declarative/tx-propagation.html - https://vladmihalcea.com/spring-transaction-best-practices/ last_verified: 2026-07-10 -related: [backend-common-jobs-idempotent-handlers, backend-common-errors-exception-handling, databases-transactions-isolation-level-selection] +related: [backend-java-jpa-raw-jdbc-inside-a-jpa-transaction, backend-common-jobs-idempotent-handlers, backend-common-errors-exception-handling, databases-transactions-isolation-level-selection] --- # Transaction Boundaries in Application Code diff --git a/wiki/backend/common/reliability/timeouts-and-retries.md b/wiki/backend/common/reliability/timeouts-and-retries.md index f176013..e15dc46 100644 --- a/wiki/backend/common/reliability/timeouts-and-retries.md +++ b/wiki/backend/common/reliability/timeouts-and-retries.md @@ -9,7 +9,7 @@ sources: - https://sre.google/sre-book/addressing-cascading-failures/ - https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ last_verified: 2026-08-05 -related: [backend-common-api-design-idempotency, backend-common-llm-completion-response-validation, backend-common-reliability-client-side-rate-limiting, debugging-concurrency-intermittent-failures] +related: [backend-java-jpa-raw-jdbc-inside-a-jpa-transaction, backend-common-api-design-idempotency, backend-common-llm-completion-response-validation, backend-common-reliability-client-side-rate-limiting, debugging-concurrency-intermittent-failures] --- # Calling Another Service over the Network: Timeouts, Retries, Backoff diff --git a/wiki/backend/java/index.md b/wiki/backend/java/index.md index 8ac2e14..27e8dfb 100644 --- a/wiki/backend/java/index.md +++ b/wiki/backend/java/index.md @@ -15,6 +15,7 @@ load it alongside the stack page here — these pages link the exact ids. | Page | Load when | |------|-----------| | [entity-mapping](jpa/entity-mapping.md) | Writing or reviewing JPA entity classes/associations — fetch types (to-one EAGER default), bidirectional sync helpers, entity equals/hashCode; debugging `LazyInitializationException`, `MultipleBagFetchException`, entities vanishing from Sets, or unexpected joins/queries traced to mappings; deciding DTO projection vs entity for read-only endpoints; evaluating Open Session in View | +| [raw-jdbc-inside-a-jpa-transaction](jpa/raw-jdbc-inside-a-jpa-transaction.md) | One query in a JPA/Hibernate service was dropped to `JdbcTemplate`/`NamedParameterJdbcTemplate` and `@Transactional(timeout = N)` is what bounds it; one endpoint's slow queries are cancelled at N seconds on some paths and run for minutes on others; the same timeout surfaces as a 4xx on one path and a 5xx on another; writing the handler branch for a query-cancellation exception, or deciding where to put a statement timeout | | [persistence-context](jpa/persistence-context.md) | Debugging changes saved without calling save (dirty checking), stale reads within one transaction (first-level cache), flush timing surprises around queries, detached-entity errors (merge vs persist, lost updates after merge); designing or fixing slow/memory-hungry JPA batch inserts; choosing IDENTITY vs SEQUENCE id generation for batch-heavy tables | ## spring diff --git a/wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md b/wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md new file mode 100644 index 0000000..8ebcf66 --- /dev/null +++ b/wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md @@ -0,0 +1,112 @@ +--- +id: backend-java-jpa-raw-jdbc-inside-a-jpa-transaction +domain: backend +category: jpa +applies_to: [java, spring, jpa, hibernate] +confidence: verified +sources: + - https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/orm/jpa/JpaTransactionManager.html + - https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/datasource/DataSourceUtils.html + - https://github.com/spring-projects/spring-framework/blob/v6.2.0/spring-orm/src/main/java/org/springframework/orm/jpa/JpaTransactionManager.java + - https://github.com/spring-projects/spring-framework/blob/v6.2.0/spring-jdbc/src/main/java/org/springframework/jdbc/support/SQLStateSQLExceptionTranslator.java + - https://github.com/hibernate/hibernate-orm/blob/main/hibernate-core/src/main/java/org/hibernate/dialect/PostgreSQLDialect.java +last_verified: 2026-08-10 +related: + [ + backend-common-orm-transaction-boundaries, + backend-common-reliability-timeouts-and-retries, + backend-common-errors-exception-handling, + backend-java-spring-proxy-pitfalls, + ] +--- + +# A Raw JdbcTemplate Query Inside a JPA-Managed Transaction + +## When this applies + +A Spring service is JPA/Hibernate-backed, and one query was dropped to +`JdbcTemplate`/`NamedParameterJdbcTemplate` for performance while +`@Transactional(timeout = N)` is what is supposed to bound its runtime. Also when +one endpoint's slow queries are cancelled at N seconds through some paths and run +for minutes through others, or when the same timeout surfaces as a 4xx on one +path and a 5xx on another. + +Where the transaction boundary belongs → [backend-common-orm-transaction-boundaries]. +The annotation having no effect at all (self-invocation, non-public method) → +[backend-java-spring-proxy-pitfalls]. + +## Do this + +1. **Measure the raw path's actual cancellation point before trusting the + declared timeout.** Run a query you know exceeds N through that exact method + and record the elapsed time (a JDBC-level logger such as p6spy, or the + database's own cancellation message). The declared timeout and the applied + timeout are separate facts, and nothing logs the gap between them. + +2. **Trace the deadline's path.** `JdbcTemplate` ends every statement setup with + `DataSourceUtils.applyTimeout(stmt, getDataSource(), getQueryTimeout())`, which + applies "the current transaction timeout, **if any**". It looks the deadline up + as a `ConnectionHolder` bound to *its own* `DataSource` instance; with no + holder it falls back to the template's own `queryTimeout`, which defaults to + `-1` and so sets nothing. Check each link: + +| Link | Check | If it fails | +|------|-------|-------------| +| The transaction manager knows the DataSource | `JpaTransactionManager` binds a `ConnectionHolder` only inside `if (getDataSource() != null)`; it "will autodetect the DataSource used as the connection factory of the EntityManagerFactory" | Set it explicitly (`setDataSource`), matching the EntityManagerFactory's DataSource | +| The JpaDialect can expose the JDBC connection | The bind is skipped when `getJpaDialect().getJdbcConnection(...)` returns `null` — `DefaultJpaDialect`'s implementation returns `null`; enable debug logging on the manager and look for "Not exposing JPA transaction … because JpaDialect … does not support JDBC Connection retrieval" | Configure the vendor dialect (`HibernateJpaDialect`), which the docs state "requires a vendor-specific `JpaDialect` to be configured" | +| The template uses the same DataSource instance | The lookup key is the `DataSource` object the template holds; a second `DataSource` bean, or one wrapped after the manager captured it, is a different key | Build the template from the same bean, or from a `TransactionAwareDataSourceProxy` | +| The timeout is declared where the proxy sees it | `timeout` on the annotation only reaches `doBegin` when that call is proxied | → [backend-java-spring-proxy-pitfalls] | + +3. **Give the raw path a deadline that does not depend on that chain.** Set + `setQueryTimeout(N)` on the template, or set the database's own statement + timeout for the connection, so the bound is present whether or not the holder + is. `applyTimeout` prefers the transaction's remaining time when a holder + exists, so an explicit value is a floor, not a conflict. + +4. **Handle both timeout exception types before shipping the fix.** Cancellation + raises different Spring exceptions per path, so a handler branch written for + the JPA path does not cover the raw one: + +| Path | Chain | Spring exception | +|------|-------|------------------| +| Hibernate/JPA query | PostgreSQL SQLState `57014` → `org.hibernate.QueryTimeoutException` → `HibernateJpaDialect` | `org.springframework.dao.QueryTimeoutException` | +| Raw JdbcTemplate, Spring Framework ≤ 6.2.x | `PSQLException` (a plain `SQLException`, so the JDBC-4 `SQLTimeoutException` branch does not match) → SQLState class `57` | `DataAccessResourceFailureException` | +| Raw JdbcTemplate, Spring Framework ≥ 7.0.0 | Same chain, plus a `"57014".equals(sqlState)` check ahead of the class-57 mapping | `org.springframework.dao.QueryTimeoutException` | + +5. **Keep the timeout's response status the same on both paths, and fix the + missing bound as its own change.** A path that now runs for minutes and a path + cancelled at N seconds are one defect wearing two status codes; the status + difference is the symptom that leads to it. + +## Edge cases + +| Case | Then | +|------|------| +| A server-side `statement_timeout` is also set | It bounds every path independently of Spring, which makes it the cheapest guard to add; keep the application timeout below it so the application's own error is what surfaces | +| The method is `@Transactional(readOnly = true)` | The timeout is unaffected by `readOnly` — both are set on the same transaction object — so a read-only annotation is not evidence the deadline applies | +| The raw query runs outside any transaction (no annotation on the path) | There is no holder to carry a deadline at all, so the explicit `setQueryTimeout` from step 3 is the only bound | +| The team's first fix is to map the new exception to a 4xx so the alerts stop | Map it after the bound exists — otherwise minute-long queries leave the 5xx alerting entirely and the remaining defect has no signal | +| The database is MySQL rather than PostgreSQL | The class-57 mapping is PostgreSQL's SQLState; Spring's fallback translator also returns `QueryTimeoutException` when the driver's exception class name contains "Timeout", which is the MySQL path — confirm which branch your driver takes before writing the handler | +| A `sql-error-codes.xml` file sits at the classpath root | The template then uses `SQLErrorCodeSQLExceptionTranslator` instead of the subclass/state chain above, so re-derive the exception type for your file's mappings | +| The same value is also enforced by an HTTP or gateway timeout | The client-visible failure comes from whichever fires first; order them so the database cancellation happens first, or the query keeps running after the response is gone ([backend-common-reliability-timeouts-and-retries]) | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Read `@Transactional(timeout = 10)` on the class as evidence every query inside is bounded at 10s | Measure one deliberately-slow query per access path | Measured on one endpoint: the Hibernate path cancelled at 10,012 ms while raw-JdbcTemplate calls on the same annotated path ran 151,558 ms and 163,489 ms | +| Treat the new `DataAccessResourceFailureException` as a newly-broken dependency | Read it as the same timeout arriving through the other translator branch | SQLState class `57` is in Spring's `DATA_ACCESS_RESOURCE_FAILURE_CODES` through 6.2.x; the message still carries the cancellation text | +| Add the raw path's exception to the 4xx branch and close the incident | Add the missing query timeout, then align the status | The status mapping removes the alert; the unbounded query is what the alert was pointing at | +| Wrap the raw call in an application-level watchdog (a future with a timeout) | Set the statement timeout so the database cancels the work | A cancelled wrapper returns control while the query keeps running and holds its connection | +| Assume the DataSource is wired because Spring Boot auto-configured the manager | Verify the holder exists — check the manager's debug log line, or assert that a `@Transactional(timeout = 1)` method's raw query fails | Autodetection covers the DataSource, and the bind still fails silently when the dialect cannot expose the connection | +| Rely on the Spring 7 mapping and write one handler branch for `QueryTimeoutException` | Pin the framework version the branch assumes, and keep the `DataAccessResourceFailureException` branch while any service is on 6.2.x or earlier | The `"57014"` check exists in 7.0.0 and is absent in 6.2.8 and every earlier tag checked | + +## Sources + +- https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/orm/jpa/JpaTransactionManager.html — "This transaction manager also supports direct DataSource access within a transaction (i.e. plain JDBC code working with the same DataSource)"; "To be able to register a DataSource's Connection for plain JDBC code, this instance needs to be aware of the DataSource (`setDataSource(DataSource)`)"; "This transaction manager will autodetect the DataSource used as the connection factory of the EntityManagerFactory, so you usually don't need to explicitly specify the 'dataSource' property"; and "Note that this requires a vendor-specific `JpaDialect` to be configured" +- https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/datasource/DataSourceUtils.html — `applyTimeout` "Apply the specified timeout - overridden by the current transaction timeout, if any - to the given JDBC Statement object"; `applyTransactionTimeout` "Apply the current transaction timeout, **if any**, to the given JDBC Statement object" — the "if any" is the silent branch +- https://github.com/spring-projects/spring-framework/blob/v6.2.0/spring-orm/src/main/java/org/springframework/orm/jpa/JpaTransactionManager.java — `doBegin` sets `conHolder.setTimeoutInSeconds(timeoutToUse)` only inside `if (getDataSource() != null)` and only when `getJpaDialect().getJdbcConnection(em, …)` returned non-null, otherwise logging "Not exposing JPA transaction … because JpaDialect … does not support JDBC Connection retrieval". `DefaultJpaDialect.getJdbcConnection` returns `null`. Read at tag v6.2.0 +- https://github.com/spring-projects/spring-framework/blob/v6.2.0/spring-jdbc/src/main/java/org/springframework/jdbc/support/SQLStateSQLExceptionTranslator.java — `DATA_ACCESS_RESOURCE_FAILURE_CODES` is `Set.of("08", "53", "54", "57", "58")` and class `57` returns `new DataAccessResourceFailureException(...)` with no timeout special-case; the only `QueryTimeoutException` route is `ex.getClass().getName().contains("Timeout")` (commented "For MySQL"). Verified 2026-08-10 across tags: `"57014".equals(sqlState)` is absent in v5.3.31, v6.0.0, v6.2.0, v6.2.1, v6.2.3, v6.2.5 and v6.2.8, and present in v7.0.0 and `main` (as `indicatesQueryTimeout`, documented "with SQL state 57014 as a specific indication") +- `SQLExceptionSubclassTranslator` (same tag) maps `ex instanceof SQLTimeoutException` to `QueryTimeoutException` and constructs `setFallbackTranslator(new SQLStateSQLExceptionTranslator())`; `JdbcAccessor` documents it as the default "as of 6.0" unless a user-provided `sql-error-codes.xml` is on the classpath. PgJDBC's `PSQLException extends SQLException` and its `PSQLState.QUERY_CANCELED` is `"57014"`, so the subclass branch does not match and the state fallback decides — read from https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/util/PSQLException.java and `PSQLState.java` +- https://github.com/hibernate/hibernate-orm/blob/main/hibernate-core/src/main/java/org/hibernate/dialect/PostgreSQLDialect.java — SQLState `"57014"` maps to `org.hibernate.QueryTimeoutException`, which `HibernateJpaDialect` converts to `org.springframework.dao.QueryTimeoutException` (https://github.com/spring-projects/spring-framework/blob/v6.2.0/spring-orm/src/main/java/org/springframework/orm/jpa/vendor/HibernateJpaDialect.java) — the JPA half of the split in step 4 +- Field measurement 2026-08-10 (production endpoint, p6spy JDBC timing, `@Transactional(readOnly = true, timeout = 10)` declared, no server-side `statement_timeout`): the Hibernate path was cancelled at 10,012 ms and surfaced as HTTP 400; two raw `JdbcTemplate` calls on the same endpoint ran 151,558 ms and 163,489 ms and surfaced as HTTP 500. Over 30 days, 11 recorded errors matched the per-path status split with no exceptions diff --git a/wiki/databases/index.md b/wiki/databases/index.md index 66083a7..49b2310 100644 --- a/wiki/databases/index.md +++ b/wiki/databases/index.md @@ -13,6 +13,7 @@ Match your situation to a "load when" line; load only matching pages. | [composite-index-column-order](indexing/composite-index-column-order.md) | Creating a multi-column index; choosing column order for equality + range/sort queries | | [covering-indexes](indexing/covering-indexes.md) | A query already served by an index still reads the table (heap) heavily; deciding whether to add INCLUDE/covering columns | | [partial-and-expression-indexes](indexing/partial-and-expression-indexes.md) | Queries always filter a fixed rare condition (status, deleted_at) or a function of a column (lower(email)); a uniqueness rule applies only to a subset of rows (e.g. live rows) | +| [trigram-index-short-patterns](indexing/trigram-index-short-patterns.md) | A `LIKE`/`ILIKE '%keyword%'` search on a PostgreSQL `pg_trgm` GIN/GiST index is fast for ordinary words and slow for one- or two-character keywords; `EXPLAIN` shows a `Bitmap Index Scan` on the trigram index and the query is still slow; choosing a minimum search-keyword length, or deciding between pg_trgm, pg_bigm, and a driver index for another condition | | [index-write-cost](indexing/index-write-cost.md) | Adding indexes to write-heavy tables; bulk loads; auditing for unused/redundant indexes | ## query-optimization diff --git a/wiki/databases/indexing/index-selection.md b/wiki/databases/indexing/index-selection.md index f5a92c8..f27fe56 100644 --- a/wiki/databases/indexing/index-selection.md +++ b/wiki/databases/indexing/index-selection.md @@ -8,7 +8,7 @@ sources: - https://www.postgresql.org/docs/current/indexes.html - https://use-the-index-luke.com/ last_verified: 2026-07-10 -related: [databases-indexing-composite-index-column-order, databases-indexing-index-write-cost, databases-query-optimization-reading-execution-plans] +related: [databases-indexing-composite-index-column-order, databases-indexing-index-write-cost, databases-query-optimization-reading-execution-plans, databases-indexing-trigram-index-short-patterns] --- # Deciding Whether a Column Needs an Index @@ -48,7 +48,7 @@ designing a new table and choosing initial indexes. |------|------| | Table is small (fits in a few pages) | Planner will sequential-scan regardless; skip the index until the table grows | | Column has few distinct values but you always query one rare value | Partial index on that value beats a full index | -| Text search / `LIKE '%term%'` | B-tree cannot serve infix matches; use a trigram or full-text index type instead of adding a useless B-tree | +| Text search / `LIKE '%term%'` | B-tree cannot serve infix matches; use a trigram or full-text index type instead of adding a useless B-tree — and set the minimum keyword length that index type needs ([databases-indexing-trigram-index-short-patterns]) | | Write-heavy table, marginal read gain | Weigh maintenance cost first ([databases-indexing-index-write-cost]) | | Creating the index on a large live table | PostgreSQL: `CREATE INDEX CONCURRENTLY` — no long write-block, cannot run inside a transaction, and a failed build leaves an `INVALID` index (drop it, retry). MySQL 8.0: online DDL (`ALGORITHM=INPLACE, LOCK=NONE`) | diff --git a/wiki/databases/indexing/trigram-index-short-patterns.md b/wiki/databases/indexing/trigram-index-short-patterns.md new file mode 100644 index 0000000..1dd0581 --- /dev/null +++ b/wiki/databases/indexing/trigram-index-short-patterns.md @@ -0,0 +1,95 @@ +--- +id: databases-indexing-trigram-index-short-patterns +domain: databases +category: indexing +applies_to: [postgresql] +confidence: verified +sources: + - https://www.postgresql.org/docs/current/pgtrgm.html + - https://postgrespro.com/list/thread-id/1821635 + - https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md +last_verified: 2026-08-10 +related: + [ + databases-indexing-index-selection, + databases-query-optimization-reading-execution-plans, + databases-indexing-partial-and-expression-indexes, + databases-indexing-covering-indexes, + ] +--- + +# Substring Search on a Trigram Index When the Keyword Is Shorter Than Three Characters + +## When this applies + +A `LIKE`/`ILIKE '%keyword%'` search is served by a PostgreSQL `pg_trgm` GIN or +GiST index, and the keyword comes from a user — so it can be one or two +characters. Also when such a search is fast for ordinary words and slow for short +ones, or when `EXPLAIN` shows a `Bitmap Index Scan` on the trigram index and the +query is still slow, or when you are choosing the minimum length for a search +input. + +Reading the plan that shows this → [databases-query-optimization-reading-execution-plans]. +Choosing the index type in the first place → [databases-indexing-index-selection]. + +## Do this + +1. **Count the characters between wildcards, not the characters the user typed.** + The index is searched by extracting trigrams from the pattern, and "a pattern + with no extractable trigrams will degenerate to a full-index scan". A + wildcard-delimited segment of fewer than three characters yields none: + `get_wildcard_trigrams` "return[s] no trigrams for wildcard part 'st' since + charlen < 3", so "GIN_SEARCH_MODE_ALL mode is used and results in full index + scan instead of trigrams being used". `show_trgm('cat')` returning four + trigrams does not contradict this — that padding applies to a *word* being + indexed, and a `%…%` pattern asserts no word boundary to pad against. + +2. **Read the cost from the recheck counters, not from the scan node's name.** + The plan still reads `Bitmap Index Scan` on the trigram index; what changes is + that the candidate bitmap becomes every row, so the work moves into the heap + recheck. `EXPLAIN (ANALYZE, BUFFERS)` is what shows it — compare + `Rows Removed by Index Recheck` against the table's row count and the buffer + count against the short and long pattern. + +3. **Pick the branch by whether short keywords are a supported input:** + +| Situation | Do | +|-----------|-----| +| The search field has no other selective filter and short keywords are optional | Enforce a minimum keyword length at the API boundary and return a stated validation error, so the cost is refused rather than paid | +| The same query carries another selective condition (owner, department, tenant, date range) | Give that condition its own index and let it produce the bitmap, then let the substring match run as a heap filter — this bounds the scan by the selective condition instead of the pattern | +| Short keywords must return results and the operator is `LIKE` | Evaluate `pg_bigm`, which "allows a user to create **2-gram** (bigram) index", and whose own comparison rates "Full text search with 1-2 characters keyword" as "Fast" against pg_trgm's "Slow" — footnoted with the same mechanism, "only sequential scan or index full scan (not normal index scan) can run" | +| Short keywords must return results and the query needs `ILIKE`, `~`, or `~*` | Keep pg_trgm and normalize instead — index and query one case-folded expression ([databases-indexing-partial-and-expression-indexes]) — because pg_bigm's index supports "LIKE only" while pg_trgm supports "LIKE (~~), ILIKE (~~*), ~, ~*" | +| The short keyword is a prefix, not an infix (`'ab%'`) | Serve it from a B-tree on the column (or its case-folded expression) — a left-anchored pattern needs no trigrams | + +4. **Verify the chosen branch on production-scale data before shipping it.** The + degeneration is invisible at small row counts, where a full index scan is + cheap; measure at the table's real size. + +## Edge cases + +| Case | Then | +|------|------| +| The other WHERE conditions appear under `Bitmap Heap Scan` as `Filter` rather than as `Index Cond` | They are not reducing the scan — they are applied after the rows are read, so the plan is still paying the full recheck. Add the index that lets one of them drive the bitmap | +| The query returns very few rows, so the result looks cheap | Read the cost from buffers and recheck counts, not from the row count — the scan reads the whole index and rechecks the whole heap whichever way the match comes out | +| Only some of the keyword's wildcard segments are short (`'%ab%defg%'`) | The pattern has extractable trigrams from the longer segment, so the index search works; the short segment contributes nothing and is checked on recheck | +| The column is searched with both a short and a long keyword in one `OR` | The short branch degenerates independently; split the branches so the long one keeps its index path, or apply the length rule per branch | +| The table is small today and the search is new | Record the row count at which the branch was chosen — the same query flips from acceptable to a full-table recheck with growth, and nothing in the plan's shape changes when it does | +| A GiST trigram index is used instead of GIN | The same extraction rule governs it: with no extractable trigrams there is nothing to look up, and the docs' degeneration statement covers "both `LIKE` and regular-expression searches" | +| The workload is non-alphabetic text (Japanese, Chinese, Korean) | The same comparison lists pg_trgm's full text search for non-alphabetic language as "Not supported", so the 3-character rule bites ordinary two-character words — treat pg_bigm as the default candidate rather than the fallback. Its footnote records the alternative, "commenting out KEEPONLYALNUM macro variable in contrib/pg_trgm/pg_trgm.h and rebuilding pg_trgm module", which makes the choice a build-vs-extension decision rather than a capability wall | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Read `Bitmap Index Scan` on the trigram index as proof the index is doing the work | Compare `Rows Removed by Index Recheck` with the table's row count | The node name is the same in both cases; the recheck count is what separates a lookup from a full scan | +| Add a second trigram index, or `REINDEX`, because the short-keyword query is slow | Apply the length rule from step 3 | The index is being scanned in full by design for a pattern with no trigrams; another copy of it is scanned in full too | +| Raise `work_mem` or add heap-side tuning to make the short query fit | Refuse the short pattern at the boundary or give the query a selective driver index | The cost is proportional to the table, not to the memory available for the bitmap | +| Assume a two-character search is cheap because it returns three rows | Measure buffers for a two-character and a three-character pattern on the same index | Measured 2026-08-10 on a 4.64M-row table: a 3-character `ILIKE` ran 17 ms / 4 buffers, the 2-character one 18,789 ms / 121,837 buffers, with `Rows Removed by Index Recheck: 4,640,486` and 3 rows matched | +| Swap pg_trgm for pg_bigm to fix a slow `ILIKE` | Decide the case-folding strategy first, then choose | pg_bigm's operator support is "LIKE only"; an `ILIKE` workload has to be rewritten to a normalized expression either way | + +## Sources + +- https://www.postgresql.org/docs/current/pgtrgm.html — "For both `LIKE` and regular-expression searches, keep in mind that a pattern with no extractable trigrams will degenerate to a full-index scan"; "The index search works by extracting trigrams from the search string and then looking these up in the index. The more trigrams in the search string, the more effective the index search is"; "A trigram is a group of three consecutive characters taken from a string"; and the padding rule — "Each word is considered to have two spaces prefixed and one space suffixed when determining the set of trigrams contained in the string" — which is why `show_trgm` on a short *word* still returns trigrams while a `%…%` pattern yields none +- https://postgrespro.com/list/thread-id/1821635 — Amit Langote, pgsql list thread (2013-05-31): "When I debugged a partial match case such as 'column like '%st%'', it appears that get_wildcard_trigrams return no trigrams for wildcard part 'st' since charlen < 3"; "Hence, GIN_SEARCH_MODE_ALL mode is used and results in full index scan instead of trigrams being used". This is the mechanism behind the docs' one-sentence statement, and it is stated in terms of the wildcard-delimited segment rather than the whole pattern +- https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — "The pg_bigm module provides full text search capability in [PostgreSQL]. This module allows a user to create **2-gram** (bigram) index for faster full text search." Its pg_trgm comparison table (verified against the raw file 2026-08-10, cell by cell) reads: "Phrase matching method for full text search" 3-gram vs 2-gram; "Available text search operators" "LIKE (~~), ILIKE (~~*), ~, ~*" vs "LIKE only"; "Full text search for non-alphabetic language (e.g., Japanese)" "Not supported (\*1)" vs "Supported"; "Full text search with 1-2 characters keyword" "Slow (\*2)" vs "Fast"; "Available index" "GIN and GiST" vs "GIN only". Footnote (\*2) gives the mechanism independently of the PostgreSQL docs — "Because, in this search, only sequential scan or index full scan (not normal index scan) can run" — and footnote (\*1) records that pg_trgm's non-alphabetic limit is liftable "by commenting out KEEPONLYALNUM macro variable … and rebuilding pg_trgm module". The operator row is the constraint that decides step 3's last two rows +- Field measurement 2026-08-10 (PostgreSQL, 4,640,489-row table, `gin(tip_ctn gin_trgm_ops)`, `EXPLAIN (ANALYZE, BUFFERS)`): a 3-character `ILIKE '%…%'` ran 17 ms reading 4 buffers; a 2-character `ILIKE '%TI%'` on the same index and column ran 18,789 ms reading 121,837 buffers with `Rows Removed by Index Recheck: 4,640,486` and 3 rows actually matching. Both plans showed a `Bitmap Index Scan` on the trigram index, and the query's other conditions appeared as `Filter` on the `Bitmap Heap Scan`, reducing nothing diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 5f3ed78..41af489 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -27,6 +27,7 @@ Match your situation to a "load when" line; load only matching pages. | [guard-shape-vs-consequence](quality/guard-shape-vs-consequence.md) | A repo-wide guard asserting that no shipped artifact (example, config, migration, fixture) has a structural shape has gone red on a legitimate new artifact; authoring such a scanning guard; deciding between exempting an artifact, deleting the guard, and sharpening it; an existing guard has accumulated an exemption/allow list | | [injected-clock-duration-assertions](quality/injected-clock-duration-assertions.md) | Asserting an elapsed duration between two readings of an injected/fake float clock (rate-limit interval, backoff, debounce, TTL); choosing that fake clock's start value; a single duration test fails on correct code by a margin in the far decimal places; choosing a comparison tolerance, or deciding between float seconds and integer nanoseconds | | [write-path-assertions](quality/write-path-assertions.md) | Writing an HTTP-level test for an endpoint that persists something (form submit, create/update, onboarding step) and choosing what to assert beyond the status code; such a test is green while the records are empty or defaulted; sending repeated form fields from a client (httpx/TestClient) and deciding the `data=` shape | +| [default-values-under-test](quality/default-values-under-test.md) | A constructor/factory/config default is named as a number in a spec, plan, or measurement record (`ttl_s=600`, `max_tokens=256`, `retries=3`) and you are judging whether it is guarded; a mutation of that default left the suite green; deciding between a mechanism test that passes the value in and a test of the shipped default; choosing the boundary cases and the two mutation directions for a default | | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate | | [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect | @@ -48,6 +49,7 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [destructive-operations-on-shared-daemons](mocking/destructive-operations-on-shared-daemons.md) | The code under test enumerates and deletes a machine-wide daemon's resources by name/pattern (tmux sessions, docker containers, systemd units, namespaces) and that daemon runs on the test machine; proving a sweep deletes the targets and spares bystanders; keeping a scope bug from destroying the dev environment instead of failing the test; giving a shell script a substitution seam for the tool it shells out to | | [what-to-mock](mocking/what-to-mock.md) | Deciding whether to mock/stub/fake a dependency or use the real one; mocks breaking on refactors; testing handling of a third-party's failure modes; the same mock setup is copy-pasted across tests | +| [captured-call-arguments](mocking/captured-call-arguments.md) | Writing the spy/stub test that holds a fix to one argument of one wiring call (constructor, factory, server startup); such a test is green while a mutation of a *different* argument of the same call survives; the fix extracted the value into a resolver and you are choosing what to assert; deciding between asserting a constant's value and asserting that the call site passes it on; choosing how to record an argument you deliberately leave unpinned | ## flaky diff --git a/wiki/testing/mocking/captured-call-arguments.md b/wiki/testing/mocking/captured-call-arguments.md new file mode 100644 index 0000000..d1d2fb1 --- /dev/null +++ b/wiki/testing/mocking/captured-call-arguments.md @@ -0,0 +1,116 @@ +--- +id: testing-mocking-captured-call-arguments +domain: testing +category: mocking +applies_to: [general] +confidence: verified +sources: + - https://docs.python.org/3/library/unittest.mock.html + - https://jestjs.io/docs/expect + - https://github.com/mockito/mockito/blob/main/mockito-core/src/main/java/org/mockito/ArgumentMatchers.java + - https://pitest.org/quickstart/basic_concepts/ +last_verified: 2026-08-10 +related: + [ + testing-mocking-what-to-mock, + testing-quality-default-values-under-test, + testing-quality-tests-that-cannot-fail, + testing-quality-behavior-not-implementation, + backend-common-change-impact-call-site-enumeration, + ] +--- + +# Asserting a Wiring Call Through What the Stub Captured + +## When this applies + +A review, audit, or mutation run flagged one argument of one call — a port, a +host, a flag, an id passed at a constructor, factory, or server-startup site — +you fixed that argument, and you are writing the spy/stub test that holds the fix. +Also when such a test is green and a mutation of a *different* argument of the +same call survives, or when the fix was to extract the value into a resolver +function and you are choosing what the test asserts. + +Deciding whether to stub this dependency at all → +[testing-mocking-what-to-mock]. + +## Do this + +1. **Record the whole call, then assert every argument whose value the caller + decides.** The defect this test exists to catch is "this call site passes the + wrong value", and that defect has one instance per argument. Assert the full + argument list in one assertion rather than reading one key out of a capture + dict: `assert_called_with(host=…, port=…, tls=…)`, + `toHaveBeenCalledWith(…)` — Jest's compares the arguments "with the same + algorithm that `.toEqual` uses" — or Mockito's `verify(mock).f(eq(…), any(), …)`, + whose rule is that "**all arguments** have to be provided by matchers". + +2. **Diff the stub's signature against what it stores, and treat the difference + as the unasserted part of the contract.** A stub written as + `def runner(**kw): captured["port"] = kw["port"]` accepts `host` and `tls` and + keeps neither, so no assertion can ever read them. Replace hand-rolled capture + dicts with the framework's recorder (`Mock(spec=…)`, `jest.fn()`, + `ArgumentCaptor`), which stores the call whole. + +3. **Bind the stub to the real signature so positional and keyword forms are the + same claim.** With a spec, unittest.mock "will introspect the specification + object's signature when matching calls … regardless of whether they were + passed positionally or by name", and autospec "will catch mistakes where the + mock is called with the wrong signature". Without it, moving an argument from + keyword to positional flips a passing assertion to failing with no behavior + change. + +4. **Separate the two claims a constant carries, and write one test for each:** + +| Claim | Assert | What it catches | +|-------|--------|-----------------| +| The constant holds the right value | The constant, or the resolver's return, equals the expected value | An edit to the constant's own definition | +| The call site passes that constant on | The spy's recorded call carries the constant's current value (read the constant in the assertion, not a literal) | A call site that computes, hardcodes, or defaults the value instead of reading the constant | + +5. **Prove each argument's assertion with its own mutation, and keep the other + arguments intact.** Change that one argument at the call site and require the + test to redden; a green run is the *survived* verdict — "the mutation was not + detected by the covering test" — for that argument specifically + ([testing-quality-tests-that-cannot-fail]). + +6. **Re-run step 5 after extracting the value into a resolver.** Replacing + `port=DEFAULT_PORT` with `port=resolve_port()` moves the gap rather than + closing it: a wiring assertion that reads the constant stays green across the + extraction and still reddens on a hardcoded value, while a test that only + exercises `resolve_port()` says nothing about whether the caller calls it. + +7. **Enumerate the other call sites of the same callee before finishing.** The + spy proves one site; the sibling sites are a separate list + ([backend-common-change-impact-call-site-enumeration]). + +## Edge cases + +| Case | Then | +|------|------| +| An argument is a large object or one the caller does not decide (a logger, a session) | Assert it with a placeholder matcher — `expect.anything()`, `any()`, `ANY` — so the argument stays named in the assertion while its value is out of scope; an omitted argument and a deliberately-unpinned one read the same in review otherwise | +| The argument is an options object and only some keys matter | Name the keys that matter and state the rest as out of scope in the test name: `expect.objectContaining` matches "a received object which contains properties that are present in the expected object", so the keys you omit are unasserted by construction | +| The call happens more than once (retry, per-item loop) | Assert the recorded call list, not the last call — `assert_called_with` "is a convenient way of asserting that the **last** call has been made in a particular way", so an extra later call with different arguments can satisfy it | +| The value is only reachable by editing source (a module-local constant with no injection seam) | Assert the wiring at the level that reads it — a startup/integration test that boots the component and reads back the effective value — since no unit-level substitution can vary it | +| The mutated argument has a default in the callee, so the mutation changes nothing observable | Classify it before strengthening the assertion: an argument whose two values behave identically at this level needs the assertion at the level where they diverge | +| The stub is a fake with behavior, not a recorder | Keep the recording separate from the behavior: a fake that computes a result and also stores its inputs tends to store only the inputs it computes from | +| The argument is a deployment-visible value (bind host, port, path) | Add one assertion at the level the platform reads it — a bind host mutated from `0.0.0.0` to `127.0.0.1` passes every in-process test and fails only a container's readiness probe | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Capture just the argument the review flagged (`captured["port"] = kw["port"]`) | Record the call whole and assert every caller-decided argument | Reproduced 2026-08-10: with only `port` captured, mutating `host` from `0.0.0.0` to `127.0.0.1` at the same call left the assertion green; `assert_called_with(host=…, port=…, tls=…)` reddened on it and stayed green on the unmutated call | +| Assert `DEFAULT_PORT == 8914` as the regression test for a call site that was passing the wrong port | Assert the spy's recorded argument equals `DEFAULT_PORT` | Reproduced 2026-08-10: the constant's own assertion stayed green while the caller passed a hardcoded `8770`; only the recorded-call assertion reddened | +| Call the gap closed once the value is extracted into `resolve_port()` | Assert that the caller's recorded argument matches the resolver's value | The extraction adds a second unasserted hop — the caller can bypass the resolver, and the resolver's own unit test cannot see that | +| Write the expected value as a literal in the assertion (`port=8914`) | Read the constant in the assertion (`port=DEFAULT_PORT`) | A literal makes the test fail on every legitimate change to the constant, which trains the next author to update the literal rather than to read the failure | +| Drop an argument from the assertion because its value is uninteresting | Keep it with a placeholder matcher | A dropped argument and an unpinned one are indistinguishable later; the placeholder records that the omission was a decision | +| Accept a green suite as proof the fixed argument is now guarded | Mutate that argument alone and require the owning test red | A test can be green because it never reads the argument; the red run is what distinguishes the two | + +## Sources + +- https://docs.python.org/3/library/unittest.mock.html — `assert_called_with` is "a convenient way of asserting that the last call has been made in a particular way" (whole-call, and last-call only); `call_args` exposes `.args`/`.kwargs` for the last call; a mock created with a *spec* "will introspect the specification object's signature when matching calls to the mock … regardless of whether they were passed positionally or by name", and "using autospec will catch mistakes where the mock is called with the wrong signature" +- https://jestjs.io/docs/expect — `.toHaveBeenCalledWith` checks arguments "with the same algorithm that `.toEqual` uses"; `expect.anything()` "matches anything but `null` or `undefined`" and is usable "inside `toEqual` or `toHaveBeenCalledWith` instead of a literal value"; `expect.objectContaining(object)` matches "a received object which contains properties that are present in the expected object" — a subset match, which is why omitted keys stay unasserted +- https://github.com/mockito/mockito/blob/main/mockito-core/src/main/java/org/mockito/ArgumentMatchers.java — "If you are using argument matchers, **all arguments** have to be provided by matchers", with `verify(mock).someMethod(anyInt(), anyString(), eq("third argument"))` shown as the correct form; the API's own rule is that a verified call is specified argument-complete +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; step 5 reads a per-argument green run as this verdict for that argument +- Reproduction 2026-08-10 (Python 3, `unittest.mock`): a stub storing only `kw["port"]` reported `assertion_passes=True` both for the correct call and for one whose `host` was mutated `0.0.0.0` → `127.0.0.1`; a `Mock(spec=…)` with `assert_called_with(host=…, port=…, tls=…)` reported `True` for the correct call and `False` for the mutated one — the no-op control that shows the stronger assertion discriminates rather than always failing. A second run held `DEFAULT_PORT == 8914` green across three call-site variants (reads the constant, reads an extracted `resolve_port()`, hardcodes `8770`) while the recorded-call assertion was green for the first two and red only for the hardcoded one +- Field measurement 2026-08-10 (a Python service's startup wiring, 6-round audit): a `DEFAULT_PORT` value assertion left `main()`'s `port = resolve_port()` free — a `8770` mutation survived; after extracting the resolver, the same mutation survived again because no assertion said `main()` calls it; after adding the wiring assertion for `port`, the same call's `host` argument was still unasserted and `"0.0.0.0"` → `"127.0.0.1"` survived, a change whose only failure surface is a Kubernetes readiness probe. Each surviving mutant sat inside the previous round's own fix diff --git a/wiki/testing/mocking/what-to-mock.md b/wiki/testing/mocking/what-to-mock.md index 6372ebd..17f7fa2 100644 --- a/wiki/testing/mocking/what-to-mock.md +++ b/wiki/testing/mocking/what-to-mock.md @@ -11,7 +11,7 @@ sources: - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import - https://nodejs.org/api/test.html last_verified: 2026-08-06 -related: [testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-tests-that-cannot-fail] +related: [testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-tests-that-cannot-fail, testing-mocking-captured-call-arguments] --- # Deciding Whether to Mock, Fake, or Use the Real Dependency @@ -34,7 +34,7 @@ where mocks are breaking on refactors. | Nondeterminism sources: clock, randomness, UUID generation | Inject them and substitute a fixed clock / seeded generator in tests | | Your own DB, when the test's subject is query behavior (SQL shape, mapping, constraints) | Real test database — a mocked DB asserts your assumption of the contract, not the contract ([testing-strategy-test-level-choice]) | | A dependency that is real-capable but too slow/stateful for every unit test (your DB behind a repository, a queue) | An in-memory **fake** implementing the same interface, kept honest by running the contract's own integration tests against the real one | -| Command sent to an external boundary is itself the behavior (charge card, publish event) | Mock the owned boundary interface and assert the **outbound contract**: which command, with what arguments — not internal call sequences leading up to it | +| Command sent to an external boundary is itself the behavior (charge card, publish event) | Mock the owned boundary interface and assert the **outbound contract**: which command, with what arguments — not internal call sequences leading up to it. Assert the recorded call whole, so no argument of it stays unchecked ([testing-mocking-captured-call-arguments]) | 2. Stub **queries**, assert **commands**: for data the dependency returns, a stub with canned answers is enough — assert the subject's resulting diff --git a/wiki/testing/quality/default-values-under-test.md b/wiki/testing/quality/default-values-under-test.md new file mode 100644 index 0000000..c1d0d74 --- /dev/null +++ b/wiki/testing/quality/default-values-under-test.md @@ -0,0 +1,97 @@ +--- +id: testing-quality-default-values-under-test +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://pitest.org/quickstart/basic_concepts/ + - https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ + - https://docs.python.org/3/reference/compound_stmts.html +last_verified: 2026-08-10 +related: + [ + testing-quality-minimum-case-set, + testing-quality-tests-that-cannot-fail, + testing-mocking-captured-call-arguments, + testing-quality-harness-reverse-controls, + ] +--- + +# The Default Value of a Constructor or Factory Parameter + +## When this applies + +A constructor, factory, or config object has a default that a spec document, plan, +or measurement record names as a number (`ttl_s=600`, `max_tokens=256`, +`retries=3`), the class already has tests, and you are judging whether that default +is guarded — or a mutation run changed the default and the suite stayed green. + +Choosing the case set for the behaviour itself → [testing-quality-minimum-case-set]. +Asserting that a *caller* passes a constant on → [testing-mocking-captured-call-arguments]. + +## Do this + +1. **Separate the two subjects and give the default its own test.** A test that + passes the value in (`cls(ttl_s=0)`) pins the *mechanism* and says nothing about + the shipped default; a test that constructs with defaults pins the default only + if it *exercises* it. Both shapes are normal and both are needed — the gap is + that neither is the default's test. + +2. **Push the default to the point where it is observable, and assert from both + sides of it.** The observable point is the behaviour the number decides: + +| Default's role | Assert | +|---|---| +| A duration or TTL | Consumption just inside the boundary succeeds, and just outside it fails | +| A cap, limit, or pool size | Exactly the cap's worth of operations succeeds, and the next one is refused | +| A retry or attempt count | Exactly that many attempts are observed at the boundary the retries drive | +| A threshold or ratio | One case each side of the threshold, taken from the default's own value | +| An enum or mode | The behaviour that distinguishes this mode from the adjacent one | + +3. **Require red in both directions before believing the test.** Shrinking the + default and growing it are different mutants, and the growing direction is the + one no incidental test catches: a test that issues N items passes for every cap + ≥ N, and a test that consumes immediately passes for every TTL > 0. Run both + mutants and require your new test red for each. + +4. **Read the default's value from the code in the assertion, and assert the value + itself once.** `assert store.ttl_s == 600` next to the boundary case makes the + spec's number checkable at one place; deriving the boundary from + `store.ttl_s` keeps the boundary cases correct when the default legitimately + changes. + +5. **Run the unmutated suite and require green.** The boundary cases in step 2 sit + one unit from a limit, which is where an off-by-one in the *test* looks exactly + like a caught mutant ([testing-quality-harness-reverse-controls]). + +## Edge cases + +| Case | Then | +|------|------| +| The default is a duration long enough that exercising it would slow the suite | Reach the boundary by controlling the clock the code reads — an injected clock, or seeding the stored timestamp — rather than by shortening the default for the test, which turns it back into a mechanism test | +| An existing test happens to catch the shrink direction | Keep it and still add the grow direction: reproduced 2026-08-10, an existing test that issued 2 items reddened a `cap 256 → 1` mutant incidentally while `cap → 9999` stayed green in the same suite | +| The default is evaluated once at definition time (a Python mutable or computed default) | Assert the shared-state consequence as its own case — the language evaluates the default expression once when the function is defined, so two instances observe one object | +| The default is supplied by a framework or config layer, not by the signature | Assert the effective value after the layer resolves it, at the level that layer runs; a signature default the framework always overrides is not the shipped default | +| The spec document and the code disagree about the number | Fix the disagreement before writing the test, and record which one was authoritative — a test written against the wrong one locks the drift in | +| The default is deliberately unspecified (the caller is expected to always pass it) | Assert that omitting it is refused, so "no default" is itself the guarded behaviour | +| The value's only consequence is operational (a bind address, a timeout that only a probe observes) | Add one assertion at the level that observes it; the in-process suite cannot distinguish the values ([testing-mocking-captured-call-arguments]) | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Count a test that constructs with defaults as the default's coverage | Check whether that test reaches the default's observable point | Reproduced 2026-08-10: a suite whose "defaults" tests constructed with defaults and consumed immediately stayed **green** on a `ttl_s 600 → 1` mutant; the mechanism tests passed `ttl_s` in, so nothing read the default | +| Pass the value explicitly everywhere for determinism and leave it there | Keep those tests and add one default-valued test per number | Explicit passing is the right call for the mechanism, and it is exactly what makes the shipped default unasserted | +| Prove the default with one mutation in the direction that seems risky | Mutate it both smaller and larger | The grow direction survives every test that stays under the limit; reproduced, `ttl 6000 / cap 9999` was green against the whole existing suite | +| Shorten the default in a fixture so the boundary is quick to reach | Control the clock or the stored timestamp and keep the default | Changing the default for the test removes the subject; the test then proves the mechanism a second time | +| Write the boundary as a literal (`599`, `257`) | Derive it from the default read off the object | A literal boundary and a literal default drift apart, and the pair passes while neither matches the spec | +| Treat a green run after adding the test as proof it works | Require red on each mutant and green on the unmutated suite | A boundary case built one unit off reads as a caught mutant on every run, including the honest one | + +## Sources + +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; a changed default that leaves the suite green is that verdict for the default specifically, and PIT attributes a kill to the covering test rather than to the file +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the published mutator set operates on operators, literals, and blocks; a parameter default's value is reached by literal mutation, which is why step 3 states both directions explicitly rather than relying on a tool's single generated variant +- https://docs.python.org/3/reference/compound_stmts.html — "Default parameter values are evaluated from left to right when the function definition is executed", so a mutable or computed default is shared across calls; the basis for the shared-state edge case +- Reproduction 2026-08-10 (Python 3, a TTL + cap store, four existing tests: two constructing with defaults, two passing the values in explicitly): baseline green. `ttl_s 600 → 1` — existing suite **GREEN**, added bidirectional default tests RED. `max_tokens 256 → 1` — existing suite RED (caught incidentally, because one existing test issued two items), added tests RED. `ttl_s → 6000` and `max_tokens → 9999` together — existing suite **GREEN**, added tests RED. The unmutated run was green with the added tests, which is the control showing the boundary cases are not simply always-failing. The grow direction was uncatchable by the existing suite in every configuration tried +- Field measurement 2026-08-10 (a Python service's CSRF store, 8-round audit): `CsrfStore(ttl_s=600, max_tokens=256)` was named in the plan document, and `ttl_s=1` / `max_tokens=1` mutants passed all 65 existing cases. Three defaults-constructing tests consumed immediately or issued at most two tokens, and the two TTL/cap mechanism tests passed the values in. The production consequence of the surviving `ttl_s=1` was that any form taking longer than a second to fill would fail every submission while the page still rendered normally. Two bidirectional default tests turned all four mutants red diff --git a/wiki/testing/quality/minimum-case-set.md b/wiki/testing/quality/minimum-case-set.md index 1540584..403ab09 100644 --- a/wiki/testing/quality/minimum-case-set.md +++ b/wiki/testing/quality/minimum-case-set.md @@ -9,7 +9,7 @@ sources: - https://abseil.io/resources/swe-book/html/ch12.html - https://martinfowler.com/bliki/TestDrivenDevelopment.html last_verified: 2026-07-10 -related: [testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-checks-that-cannot-pass, qa-exploratory-guard-true-path-coverage] +related: [testing-quality-default-values-under-test, testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-checks-that-cannot-pass, qa-exploratory-guard-true-path-coverage] --- # Selecting the Minimum Case Set for a Function or Endpoint diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index b0534af..ca2aace 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -16,7 +16,7 @@ sources: - https://git-scm.com/docs/git-checkout - https://git-scm.com/docs/git-restore last_verified: 2026-08-05 -related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] +related: [testing-quality-default-values-under-test, testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, testing-mocking-captured-call-arguments, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] --- # Proving a Test Can Fail