From 364e6839b98d87df05f83805c2cc0c76b85fffd3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Mon, 10 Aug 2026 16:36:40 +0900 Subject: [PATCH 1/4] knowledge: ingest 3 verified insight(s) - testing/mocking/captured-call-arguments: record the whole call, assert every caller-decided argument; split the constant's value from the call site passing it on; re-prove per argument after extracting a resolver. - databases/indexing/trigram-index-short-patterns: a pg_trgm wildcard segment under 3 chars yields no extractable trigrams, so GIN degenerates to a full-index scan while the plan still reads Bitmap Index Scan. - backend/java/jpa/raw-jdbc-inside-a-jpa-transaction: @Transactional(timeout) reaches JdbcTemplate only via a bound ConnectionHolder, and the JPA and raw paths raise different Spring exceptions (boundary at Spring Framework 7.0.0). --- .dev-loop/INGEST_REPORT.md | 285 +++++++++++++++--- log.md | 1 + .../common/orm/transaction-boundaries.md | 2 +- .../reliability/timeouts-and-retries.md | 2 +- wiki/backend/java/index.md | 1 + .../jpa/raw-jdbc-inside-a-jpa-transaction.md | 112 +++++++ wiki/databases/index.md | 1 + wiki/databases/indexing/index-selection.md | 4 +- .../indexing/trigram-index-short-patterns.md | 95 ++++++ wiki/testing/index.md | 1 + .../mocking/captured-call-arguments.md | 115 +++++++ wiki/testing/mocking/what-to-mock.md | 4 +- .../testing/quality/tests-that-cannot-fail.md | 2 +- 13 files changed, 575 insertions(+), 50 deletions(-) create mode 100644 wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md create mode 100644 wiki/databases/indexing/trigram-index-short-patterns.md create mode 100644 wiki/testing/mocking/captured-call-arguments.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..7f4927e 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,252 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 3 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: `1717316a-…jsonl` (1 row), `ab5516dc-…jsonl` (2 rows). All three +ingested as new pages; none dropped. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. A spy that captures one argument leaves the same call's siblings unasserted + +**Claim.** When a review flags one argument of one wiring call and you hold the fix +with a spy, record the whole call and assert every caller-decided argument; +asserting the constant's value is a different claim from asserting that the call +site passes it on, and extracting a resolver moves that gap up a layer. + +**Sources checked.** + +- https://docs.python.org/3/library/unittest.mock.html — `assert_called_with` is + "a convenient way of asserting that the last call has been made in a particular + way" (whole-call, last-call only); a *spec*'d mock "will introspect the + specification object's signature when matching calls … regardless of whether + they were passed positionally or by name", and "using autospec will catch + mistakes where the mock is called with the wrong signature". +- https://jestjs.io/docs/expect — `.toHaveBeenCalledWith` checks arguments "with + the same algorithm that `.toEqual` uses"; `expect.objectContaining` matches "a + received object which contains properties that are present in the expected + object" — a **subset** match, which is why omitted keys stay unasserted (this + became an edge-case row rather than a recommendation). +- https://github.com/mockito/mockito/blob/main/mockito-core/src/main/java/org/mockito/ArgumentMatchers.java + — "If you are using argument matchers, **all arguments** have to be provided by + matchers." +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation + was not detected by the covering test" (how step 5 reads a green per-argument run). + +**How verified.** Reproduced both halves locally this session (Python 3, +`unittest.mock`), not just cited: + +| Run | Correct call | Mutated call | +|---|---|---| +| partial-capture stub (stores only `kw["port"]`) | passes | **passes** — `host` `0.0.0.0`→`127.0.0.1` undetected | +| `Mock(spec=…)` + `assert_called_with(host=…, port=…, tls=…)` | passes | fails | + +The green cell on the correct call is the no-op control: the stronger assertion +discriminates rather than always failing. A second run held +`DEFAULT_PORT == 8914` green across three call-site variants (reads the constant, +reads an extracted `resolve_port()`, hardcodes `8770`) while the recorded-call +assertion was green for the first two and red only for the hardcoded one — the +"value vs wiring" split, and the evidence that the wiring assertion survives the +refactor. + +**Confidence: verified** (official docs for every API claim + local reproduction). +The field evidence behind the candidate (a 6-round audit where `port`, then the +extracted resolver, then `host` each survived in turn, the last being a change +whose only failure surface is a Kubernetes readiness probe) is recorded in the +page's Sources as a field measurement, kept distinct from the reproduction. + +### 2. pg_trgm degenerates to a full-index scan below three characters + +**Claim.** A `LIKE`/`ILIKE` wildcard segment of fewer than three characters yields +no extractable trigrams, so a pg_trgm GIN/GiST index is scanned in full and the +cost moves into the heap recheck — while the plan still reads `Bitmap Index Scan`. + +**Sources checked.** + +- https://www.postgresql.org/docs/current/pgtrgm.html — "For both `LIKE` and + regular-expression searches, keep in mind that a pattern with no extractable + trigrams will degenerate to a full-index scan." Also the padding rule ("Each + word is considered to have two spaces prefixed and one space suffixed…"), which + I checked precisely because it is a trap: `show_trgm('cat')` returns four + trigrams, so "short strings have no trigrams" is **wrong** as stated — padding + applies to a *word being indexed*, while a `%…%` pattern asserts no word + boundary to pad against. The page states it that way. +- https://postgrespro.com/list/thread-id/1821635 — Amit Langote, pgsql list + (2013-05-31): "get_wildcard_trigrams return no trigrams for wildcard part 'st' + since charlen < 3"; "Hence, GIN_SEARCH_MODE_ALL mode is used and results in full + index scan instead of trigrams being used." This is what lets the page state the + rule per wildcard-delimited segment rather than per pattern. +- https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — 2-gram index; + its comparison table rates 1–2 character keyword search "Fast" vs pg_trgm's + "slow", **and** lists pg_bigm's operators as "LIKE only" vs pg_trgm's "LIKE + (~~), ILIKE (~~*), ~, ~*". That constraint corrects the candidate, which + suggested pg_bigm without noting it is not a drop-in for an `ILIKE` workload; + the page splits those into two decision rows. + +**How verified.** Doc quotes fetched and read this session. The quantitative half +is the candidate's own field `EXPLAIN (ANALYZE, BUFFERS)` on a 4.64M-row table +(3-char 17 ms / 4 buffers vs 2-char 18,789 ms / 121,837 buffers, +`Rows Removed by Index Recheck: 4,640,486`, 3 rows matched) — **not** re-run here: +no PostgreSQL was reachable in this environment (`psql` absent, Docker daemon +down, no postgres pod in the local cluster). It is labelled a field measurement +with its date and table size, and no claim in the page depends on my having +re-run it. + +**Confidence: verified** (mechanism doc-sourced; magnitude field-measured). + +### 3. `@Transactional(timeout=N)` does not reach a raw JdbcTemplate path by itself + +**Claim.** The declared timeout reaches `JdbcTemplate` only through a +`ConnectionHolder` bound by `JpaTransactionManager`; when that bind is skipped the +raw path runs unbounded, and the two paths raise **different** Spring exceptions. + +**Sources checked (source read at pinned tags, not from memory).** + +- `JpaTransactionManager` javadoc — "To be able to register a DataSource's + Connection for plain JDBC code, this instance needs to be aware of the + DataSource (`setDataSource(DataSource)`)"; "will autodetect the DataSource used + as the connection factory of the EntityManagerFactory, so you usually don't need + to explicitly specify the 'dataSource' property"; "this requires a + vendor-specific `JpaDialect` to be configured". +- `JpaTransactionManager.java` @ v6.2.0 — `conHolder.setTimeoutInSeconds(...)` + sits inside `if (getDataSource() != null)` **and** requires + `getJpaDialect().getJdbcConnection(em, …) != null`, else it logs "Not exposing + JPA transaction … does not support JDBC Connection retrieval". + `DefaultJpaDialect.getJdbcConnection` returns `null`. This is a **correction**: + the candidate named only the DataSource wiring, so a Boot app (where the + DataSource is autodetected) would have looked exempt; the dialect branch and the + DataSource-instance-identity branch are separate failure modes, and the debug + log line is a checkable diagnostic. The page's step 2 is a 4-row table because + of this. +- `DataSourceUtils` javadoc + `DataSourceUtils.java` @ v6.2.0 — `applyTimeout` + applies "the current transaction timeout, **if any**"; it looks the holder up by + the `DataSource` instance and otherwise falls back to the passed timeout only + `if (timeout >= 0)`. `JdbcTemplate.applyStatementSettings` calls it with + `getQueryTimeout()`, whose field default is `private int queryTimeout = -1` — + so with no holder, nothing is set at all. +- Exception split: Hibernate `PostgreSQLDialect.java` maps SQLState `"57014"` → + `org.hibernate.QueryTimeoutException`, and `HibernateJpaDialect.java` @ v6.2.0 + converts that to `org.springframework.dao.QueryTimeoutException`. On the raw + path, PgJDBC's `PSQLException extends SQLException` with + `PSQLState.QUERY_CANCELED = "57014"`, so `SQLExceptionSubclassTranslator`'s + `instanceof SQLTimeoutException` branch misses and its + `SQLStateSQLExceptionTranslator` fallback maps class `57` + (`Set.of("08","53","54","57","58")`) → `DataAccessResourceFailureException`. +- **Version boundary found while verifying, which the candidate did not know.** + `main` special-cases `"57014".equals(sqlState)` → `QueryTimeoutException`. I + fetched the file at seven released tags to find where it starts: + +| Tag | `"57014".equals` present | +|---|---| +| v5.3.31, v6.0.0, v6.2.0, v6.2.1, v6.2.3, v6.2.5, v6.2.8 | no | +| v7.0.0, `main` | yes | + +So the candidate's exception claim is correct for Spring Framework ≤ 6.2.x and +inverts at 7.0.0. The page states both, and an `Instead of` row requires pinning +the framework version any single-branch handler assumes. + +**How verified.** Every quote above was fetched this session; the version table +came from downloading the same file at each tag and grepping it. The timing half +(Hibernate cancelled at 10,012 ms → 400 vs raw JdbcTemplate 151,558 ms / +163,489 ms → 500 on one annotated endpoint, 11/11 errors over 30 days matching the +per-path status split) is the candidate's production p6spy measurement, labelled as +such. + +**Confidence: verified** (framework behaviour read from pinned source + javadoc; +production magnitudes field-measured). ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via `INDEX.md` → the three domain indexes, then read every page whose "load +when" line overlapped. Full-body reads: `testing-quality-tests-that-cannot-fail`, +`testing-mocking-what-to-mock`, `databases-indexing-index-selection`. Targeted +reads (grep for timeout/JdbcTemplate/trigram/LIKE/arg-capture terms, to establish +absence of coverage): the remaining ids below. + +Pages read: testing-quality-tests-that-cannot-fail, testing-mocking-what-to-mock, testing-quality-behavior-not-implementation, backend-common-change-impact-call-site-enumeration, databases-indexing-index-selection, databases-query-optimization-reading-execution-plans, databases-indexing-partial-and-expression-indexes, databases-indexing-covering-indexes, backend-common-orm-transaction-boundaries, backend-common-reliability-timeouts-and-retries, backend-common-errors-exception-handling, backend-java-jpa-persistence-context, backend-java-spring-proxy-pitfalls + +**Overlaps found, and why each is composition rather than duplication.** + +| Existing page | Overlap | Resolution | +|---|---|---| +| `testing-mocking-what-to-mock` | Its step 1 last row and step 2 already say to "assert the **outbound contract**: which command, with what arguments", and one edge row says to "deep-equal the full stub-recorded call sequence" | Closest neighbour, but its subject is *whether* to replace a dependency — folding assertion-completeness in would break "one case per page". Kept separate; added the new id to its `related:` and a pointer on the outbound-contract row | +| `testing-quality-tests-that-cannot-fail` | Owns per-assertion mutation granularity and the "testing the mock instead of the code" row | The new page cites it for the mutation step instead of restating it; added the new id to its `related:` | +| `databases-indexing-index-selection` | Line 51 routes `LIKE '%term%'` to "a trigram or full-text index type" | That advice has an unstated precondition — exactly the new page. Extended the row to carry the minimum-keyword-length pointer, plus a `related:` link | +| `databases-query-optimization-reading-execution-plans` | Owns plan reading generally | Cited from "When this applies"; the new page adds only the trigram-specific counter to read (`Rows Removed by Index Recheck`) | +| `backend-common-reliability-timeouts-and-retries` | One row: "Dependency is a DB with its own driver timeout — set both the driver statement timeout and your outer deadline" | Consistent, and the new page is the Spring-specific mechanism for why the driver timeout is silently absent. Reciprocal `related:` added | +| `backend-common-orm-transaction-boundaries` | Transaction scope; no timeout content (grep: only an external-API-in-transaction row) | Reciprocal `related:` added | +| `backend-java-spring-proxy-pitfalls` | Owns "the annotation had no effect at all" | Cited as the upstream check in step 2's table, so the two failure modes stay distinguishable | +| `backend-java-jpa-persistence-context`, `backend-common-errors-exception-handling`, `databases-indexing-partial-and-expression-indexes`, `databases-indexing-covering-indexes` | No overlapping directive | Linked where relevant (case-folded expression index; exception handling) | + +**Conflicts flagged:** none. No existing page states a contradicting directive. + +**Coverage gaps confirmed by grep before creating:** `trgm|trigram|ILIKE` matches +exactly one file in the whole wiki (`index-selection.md`, the one row above); +`call_args|assert_called_with|toHaveBeenCalledWith|argument captor` matches exactly +one (`what-to-mock.md`); no page mentions `statement_timeout`, `JdbcTemplate`, or +`QueryTimeout`. + +**Format invariants checked mechanically after writing:** body lines 94 / 75 / 90 +(limit 120); every `related:` id and inline `[page-id]` reference resolves to a +page in this checkout (16/16); no banned vague qualifier in any directive (the two +`usually`/`Consider` hits were a verbatim Spring javadoc quote in Sources, left +intact, and one `Instead of` anti-pattern label, reworded); every prohibition word +occurs only inside an `Instead of` row or a quoted source. + +## Open-PR check + +Listed all 17 open `knowledge/*` heads. Three of them (**#72, #52, #49**) have had +their head branches **deleted on the remote** — `git fetch` of the branch name +404s, which first showed up as three suspiciously empty `wiki/` diffs. I re-read +those via `refs/pull//head` rather than treating an empty diff as "no overlap". +Consequence: for those three, folding by pushing to their branch is not possible; +the fold path would be a note on the PR. + +Files touched by each open head, matched against the three candidates: + +| Candidate | Overlapping open PRs | Verdict | +|---|---|---| +| 1 — captured call arguments (testing/mocking) | #52 adds `testing/quality/source-text-wiring-assertions.md`; #49 adds `testing/quality/value-preserving-refactor-assertions.md` + `unasserted-return-fields.md`; #47/#52 modify `tests-that-cannot-fail.md` | **new** | +| 2 — pg_trgm short patterns (databases/indexing) | none — zero open heads touch `wiki/databases/**` | **new** | +| 3 — raw JDBC in a JPA transaction (backend/java/jpa) | none — the backend-touching heads (#68, #58, #56, #55, #51, #50, #72) are all under `backend/common/**` or `backend/python/**`; zero touch `wiki/backend/java/**` | **new** | + +**Why candidate 1 is `new` and not `fold`**, having read all three in-flight pages +in full or in relevant part: + +- `#52 source-text-wiring-assertions` — same *word* "wiring", different subject: it + is about asserting by **regex over source text** when no seam exists (anchor + uniqueness, comment stripping, bound sizing). The new page is about the case + where a seam **does** exist and a recorder captured the call. Its own "Instead + of" even routes away from text guards when the behaviour is reachable, which is + the situation the new page occupies. No directive is duplicated. +- `#49 value-preserving-refactor-assertions` — nearest in spirit (a literal + replaced by a config read; sentinel substitution to prove the dependency). Its + trigger is a *value-preserving refactor of a source of truth*; the new page's is + *a reviewed fix to one argument of one call*, and its distinct content is + argument-set completeness across one call — the thing #49 does not address. +- `#49 unasserted-return-fields` — the mirror direction (fields a function + **returns** that no assertion reads). The new page is the call/argument + direction. Deliberately kept as siblings. + +No candidate is a pending duplicate, so nothing was dropped and no sibling +duplicate PR is opened here. + +**Merge-order note for the owner:** this branch adds one id to +`wiki/testing/quality/tests-that-cannot-fail.md`'s `related:` list, a file #47 and +#52 also modify. It is a single-line frontmatter addition. If any of #47/#49/#52 +merge first, the reciprocal links to `testing-quality-source-text-wiring-assertions`, +`-value-preserving-refactor-assertions` and `-unasserted-return-fields` become +resolvable and are worth adding to the new page then — they are intentionally +omitted now because AGENTS.md invariant 4 requires every `related:` id to resolve, +and those pages do not exist on `main`. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Domain / category | Page | New category? | +|---|---|---|---| +| 1 | `testing` / `mocking` | `wiki/testing/mocking/captured-call-arguments.md` (`testing-mocking-captured-call-arguments`) | No — `mocking` is the category that owns stub/spy mechanics; `quality` owns whether a test can fail (already cited), and the subject here is what the double records | +| 2 | `databases` / `indexing` | `wiki/databases/indexing/trigram-index-short-patterns.md` (`databases-indexing-trigram-index-short-patterns`) | No — `indexing` owns index-type suitability; the case is a precondition on one index type, and `query-optimization/reading-execution-plans` stays the owner of plan reading | +| 3 | `backend` / `java` → `jpa` | `wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md` (`backend-java-jpa-raw-jdbc-inside-a-jpa-transaction`) | No — the mechanism is `JpaTransactionManager`/`JpaDialect`, so it belongs in the `jpa` category rather than `spring` (which owns proxy-level "the annotation did nothing") or `backend/common` (language-agnostic principles; this is stack-specific source behaviour) | + +Plumbing updated: `wiki/testing/index.md`, `wiki/databases/index.md`, +`wiki/backend/java/index.md` each +1 "load when" row; `log.md` +1 ingest entry. +`INDEX.md` unchanged — all three domains are already listed and their "route here +when" lines already cover these cases. diff --git a/log.md b/log.md index c930fc2..c112eed 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Mon, 10 Aug 2026 16:37:42 +0900 Subject: [PATCH 2/4] =?UTF-8?q?knowledge:=20correct=20the=20open-PR=20chec?= =?UTF-8?q?k=20=E2=80=94=20three=20heads=20live=20on=20the=20fork,=20not?= =?UTF-8?q?=20deleted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .dev-loop/INGEST_REPORT.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 7f4927e..34ed874 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -193,12 +193,15 @@ occurs only inside an `Instead of` row or a quoted source. ## Open-PR check -Listed all 17 open `knowledge/*` heads. Three of them (**#72, #52, #49**) have had -their head branches **deleted on the remote** — `git fetch` of the branch name -404s, which first showed up as three suspiciously empty `wiki/` diffs. I re-read -those via `refs/pull//head` rather than treating an empty diff as "no overlap". -Consequence: for those three, folding by pushing to their branch is not possible; -the fold path would be a note on the PR. +Listed all 17 open `knowledge/*` heads. Three of them (**#72, #52, #49**) produced +suspiciously **empty** `wiki/` diffs on a first pass, because `git fetch origin +` and `repos/choiyounggi/dev-loop/git/refs/heads/` both 404 for +them. Rather than read an empty diff as "no overlap", I re-read all three through +`refs/pull//head`, and then established the actual cause: those heads live on +the contributor fork `dch0202-rsquare/dev-loop` (this flush's own account), not on +upstream — all three refs resolve there (`6a3ff08`, `bd03fbe`, `346dd95`). They are +alive and pushable, so `fold` was a genuinely available verdict for them; it was +not taken for the content reasons below. This PR is likewise opened from the fork. Files touched by each open head, matched against the three candidates: From 38f96a98213af9f831f2ad62eebd5af0e4e6e9dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Mon, 10 Aug 2026 16:40:28 +0900 Subject: [PATCH 3/4] knowledge: re-derive pg_bigm citations from the raw file; add footnote *2 as a second source for the mechanism --- .dev-loop/INGEST_REPORT.md | 30 +++++++++++++++---- .../indexing/trigram-index-short-patterns.md | 6 ++-- 2 files changed, 28 insertions(+), 8 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 34ed874..2454545 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -75,11 +75,15 @@ cost moves into the heap recheck — while the plan still reads `Bitmap Index Sc index scan instead of trigrams being used." This is what lets the page state the rule per wildcard-delimited segment rather than per pattern. - https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — 2-gram index; - its comparison table rates 1–2 character keyword search "Fast" vs pg_trgm's - "slow", **and** lists pg_bigm's operators as "LIKE only" vs pg_trgm's "LIKE - (~~), ILIKE (~~*), ~, ~*". That constraint corrects the candidate, which - suggested pg_bigm without noting it is not a drop-in for an `ILIKE` workload; - the page splits those into two decision rows. + its comparison table rates "Full text search with 1-2 characters keyword" as + "Fast" vs pg_trgm's "Slow (\*2)", **and** lists pg_bigm's operators as "LIKE + only" vs pg_trgm's "LIKE (~~), ILIKE (~~*), ~, ~*". That constraint corrects the + candidate, which suggested pg_bigm without noting it is not a drop-in for an + `ILIKE` workload; the page splits those into two decision rows. Footnote (\*2) + turned out to state the mechanism independently of the PostgreSQL docs — + "Because, in this search, only sequential scan or index full scan (not normal + index scan) can run" — so the central claim now has two unrelated primary + sources. **How verified.** Doc quotes fetched and read this session. The quantitative half is the candidate's own field `EXPLAIN (ANALYZE, BUFFERS)` on a 4.64M-row table @@ -241,6 +245,22 @@ resolvable and are worth adding to the new page then — they are intentionally omitted now because AGENTS.md invariant 4 requires every `related:` id to resolve, and those pages do not exist on `main`. +## Citation audit (post-write) + +Cross-Check: mechanical citation audit rather than an LLM second opinion — every +source-derived claim in the three new pages was re-grepped against the bytes +actually fetched this session (21 checks: Spring at 9 tags, Hibernate +`PostgreSQLDialect`, `HibernateJpaDialect`, PgJDBC `PSQLException`/`PSQLState`, +Mockito `ArgumentMatchers`, pg_bigm docs). 19 confirmed; one intended-absence check +confirmed absent (`"57014".equals` not in v6.2.8); **one failed and was fixed** — +the pg_bigm sentence had been taken from a fetch summary and did not match the raw +file byte-for-byte (the source reads `**2-gram**` with a line break and +`[PostgreSQL](…)` link, and the comparison cell is "Slow (\*2)", not "slow"). The +quote and the table cells were re-derived from the raw file cell by cell, which +also surfaced footnote (\*2) — a second independent statement of the mechanism — +and footnote (\*1), now recorded as an edge case. No claim was left resting on a +summarizer's paraphrase. + ## Routing decision | Insight | Domain / category | Page | New category? | diff --git a/wiki/databases/indexing/trigram-index-short-patterns.md b/wiki/databases/indexing/trigram-index-short-patterns.md index 65924ab..1dd0581 100644 --- a/wiki/databases/indexing/trigram-index-short-patterns.md +++ b/wiki/databases/indexing/trigram-index-short-patterns.md @@ -57,7 +57,7 @@ Choosing the index type in the first place → [databases-indexing-index-selecti |-----------|-----| | The search field has no other selective filter and short keywords are optional | Enforce a minimum keyword length at the API boundary and return a stated validation error, so the cost is refused rather than paid | | The same query carries another selective condition (owner, department, tenant, date range) | Give that condition its own index and let it produce the bitmap, then let the substring match run as a heap filter — this bounds the scan by the selective condition instead of the pattern | -| Short keywords must return results and the operator is `LIKE` | Evaluate `pg_bigm`, which "allows a user to create 2-gram (bigram) index" and whose own comparison rates 1–2 character keyword search "Fast" against pg_trgm's "slow" | +| Short keywords must return results and the operator is `LIKE` | Evaluate `pg_bigm`, which "allows a user to create **2-gram** (bigram) index", and whose own comparison rates "Full text search with 1-2 characters keyword" as "Fast" against pg_trgm's "Slow" — footnoted with the same mechanism, "only sequential scan or index full scan (not normal index scan) can run" | | Short keywords must return results and the query needs `ILIKE`, `~`, or `~*` | Keep pg_trgm and normalize instead — index and query one case-folded expression ([databases-indexing-partial-and-expression-indexes]) — because pg_bigm's index supports "LIKE only" while pg_trgm supports "LIKE (~~), ILIKE (~~*), ~, ~*" | | The short keyword is a prefix, not an infix (`'ab%'`) | Serve it from a B-tree on the column (or its case-folded expression) — a left-anchored pattern needs no trigrams | @@ -75,7 +75,7 @@ Choosing the index type in the first place → [databases-indexing-index-selecti | The column is searched with both a short and a long keyword in one `OR` | The short branch degenerates independently; split the branches so the long one keeps its index path, or apply the length rule per branch | | The table is small today and the search is new | Record the row count at which the branch was chosen — the same query flips from acceptable to a full-table recheck with growth, and nothing in the plan's shape changes when it does | | A GiST trigram index is used instead of GIN | The same extraction rule governs it: with no extractable trigrams there is nothing to look up, and the docs' degeneration statement covers "both `LIKE` and regular-expression searches" | -| The workload is non-alphabetic text (Japanese, Chinese, Korean) | pg_trgm's own comparison lists full text search for non-alphabetic language as "Not supported", so the 3-character rule bites ordinary two-character words — treat pg_bigm as the default candidate rather than the fallback | +| The workload is non-alphabetic text (Japanese, Chinese, Korean) | The same comparison lists pg_trgm's full text search for non-alphabetic language as "Not supported", so the 3-character rule bites ordinary two-character words — treat pg_bigm as the default candidate rather than the fallback. Its footnote records the alternative, "commenting out KEEPONLYALNUM macro variable in contrib/pg_trgm/pg_trgm.h and rebuilding pg_trgm module", which makes the choice a build-vs-extension decision rather than a capability wall | ## Instead of @@ -91,5 +91,5 @@ Choosing the index type in the first place → [databases-indexing-index-selecti - https://www.postgresql.org/docs/current/pgtrgm.html — "For both `LIKE` and regular-expression searches, keep in mind that a pattern with no extractable trigrams will degenerate to a full-index scan"; "The index search works by extracting trigrams from the search string and then looking these up in the index. The more trigrams in the search string, the more effective the index search is"; "A trigram is a group of three consecutive characters taken from a string"; and the padding rule — "Each word is considered to have two spaces prefixed and one space suffixed when determining the set of trigrams contained in the string" — which is why `show_trgm` on a short *word* still returns trigrams while a `%…%` pattern yields none - https://postgrespro.com/list/thread-id/1821635 — Amit Langote, pgsql list thread (2013-05-31): "When I debugged a partial match case such as 'column like '%st%'', it appears that get_wildcard_trigrams return no trigrams for wildcard part 'st' since charlen < 3"; "Hence, GIN_SEARCH_MODE_ALL mode is used and results in full index scan instead of trigrams being used". This is the mechanism behind the docs' one-sentence statement, and it is stated in terms of the wildcard-delimited segment rather than the whole pattern -- https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — "The pg_bigm module provides full text search capability in PostgreSQL. This module allows a user to create 2-gram (bigram) index for faster full text search"; its pg_trgm comparison table rates 1–2 character keyword search "Fast" for pg_bigm against "slow" for pg_trgm, lists full text search for non-alphabetic language as "Supported" vs "Not supported", and lists available operators as "LIKE only" for pg_bigm vs "LIKE (~~), ILIKE (~~*), ~, ~*" for pg_trgm — the constraint that decides step 3's last two rows +- https://github.com/pgbigm/pg_bigm/blob/master/docs/pg_bigm_en.md — "The pg_bigm module provides full text search capability in [PostgreSQL]. This module allows a user to create **2-gram** (bigram) index for faster full text search." Its pg_trgm comparison table (verified against the raw file 2026-08-10, cell by cell) reads: "Phrase matching method for full text search" 3-gram vs 2-gram; "Available text search operators" "LIKE (~~), ILIKE (~~*), ~, ~*" vs "LIKE only"; "Full text search for non-alphabetic language (e.g., Japanese)" "Not supported (\*1)" vs "Supported"; "Full text search with 1-2 characters keyword" "Slow (\*2)" vs "Fast"; "Available index" "GIN and GiST" vs "GIN only". Footnote (\*2) gives the mechanism independently of the PostgreSQL docs — "Because, in this search, only sequential scan or index full scan (not normal index scan) can run" — and footnote (\*1) records that pg_trgm's non-alphabetic limit is liftable "by commenting out KEEPONLYALNUM macro variable … and rebuilding pg_trgm module". The operator row is the constraint that decides step 3's last two rows - Field measurement 2026-08-10 (PostgreSQL, 4,640,489-row table, `gin(tip_ctn gin_trgm_ops)`, `EXPLAIN (ANALYZE, BUFFERS)`): a 3-character `ILIKE '%…%'` ran 17 ms reading 4 buffers; a 2-character `ILIKE '%TI%'` on the same index and column ran 18,789 ms reading 121,837 buffers with `Rows Removed by Index Recheck: 4,640,486` and 3 rows actually matching. Both plans showed a `Bitmap Index Scan` on the trigram index, and the query's other conditions appeared as `Filter` on the `Bitmap Heap Scan`, reducing nothing From cebc4484f39bc32b22d3e13f341e8c27b55af619 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Mon, 10 Aug 2026 16:46:06 +0900 Subject: [PATCH 4/4] =?UTF-8?q?knowledge:=20ingest=20a=204th=20insight=20f?= =?UTF-8?q?rom=20the=20same=20flush=20=E2=80=94=20default=20values=20under?= =?UTF-8?q?=20test?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The queue held 4 pending rows, not 3: wc -l undercounted a file whose last line had no trailing newline. testing/quality/default-values-under-test — a spec-named constructor default is guarded by neither a mechanism test that passes the value in nor a defaults-constructing test that never exercises it; push it to its observable point and require red in both mutation directions (the grow direction is the one no incidental test catches). --- .dev-loop/INGEST_REPORT.md | 88 +++++++++++++++-- log.md | 1 + wiki/testing/index.md | 1 + .../mocking/captured-call-arguments.md | 1 + .../quality/default-values-under-test.md | 97 +++++++++++++++++++ wiki/testing/quality/minimum-case-set.md | 2 +- .../testing/quality/tests-that-cannot-fail.md | 2 +- 7 files changed, 183 insertions(+), 9 deletions(-) create mode 100644 wiki/testing/quality/default-values-under-test.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 2454545..672c233 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,7 +1,16 @@ -# Knowledge flush — 3 insight(s) +# Knowledge flush — 4 insight(s) -Queue drained: `1717316a-…jsonl` (1 row), `ab5516dc-…jsonl` (2 rows). All three -ingested as new pages; none dropped. +Queue drained: `1717316a-…jsonl` (2 rows), `ab5516dc-…jsonl` (2 rows) — **4** +candidates, all ingested as new pages, none dropped. + +> Correction, recorded because it nearly cost a candidate: the first three sections +> of this report were written for 3 insights. I had counted the queue with +> `wc -l`, which reported 3 because one file's final line carried no trailing +> newline. The 4th row surfaced only when the retirement step parsed the files as +> JSON and moved 4. It had been marked `ingested` by that step while never having +> been researched or routed; it was then processed in full (insight 4 below) and +> folded into this same PR rather than left with a false status. Row counts now +> come from a JSON parse, not from `wc -l`. ## Verified best-practice @@ -157,6 +166,47 @@ such. **Confidence: verified** (framework behaviour read from pinned source + javadoc; production magnitudes field-measured). +### 4. A constructor default is unasserted by both usual test shapes + +**Claim.** A default named as a number in a spec (`ttl_s=600`, `max_tokens=256`) is +guarded by neither a mechanism test that passes the value in nor a test that +constructs with defaults without exercising them. Push the default to its +observable point and require red in **both** mutation directions. + +**Sources checked.** + +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation + was not detected by the covering test"; a green suite under a changed default is + that verdict for the default. +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — + the published mutator set is operator/literal/block based, so a tool generates + one variant of a literal rather than both directions; the page states both + directions explicitly instead of relying on the tool. +- https://docs.python.org/3/reference/compound_stmts.html — "Default parameter + values are evaluated from left to right when the function definition is + executed", the basis for the shared-mutable-default edge case. + +**How verified.** Reproduced (Python 3, a TTL + cap store with four existing tests: +two constructing with defaults, two passing values in explicitly): + +| Mutant | Existing suite | With the added default tests | +|---|---|---| +| baseline (`ttl 600`, `cap 256`) | GREEN | GREEN *(control)* | +| `ttl_s 600 → 1` | **GREEN** | RED | +| `max_tokens 256 → 1` | RED *(incidental)* | RED | +| `ttl_s → 6000`, `max_tokens → 9999` | **GREEN** | RED | + +This **sharpened the candidate**, which stated the survival as a flat property. It +is direction-dependent: shrinking a cap below what an existing test happens to +exercise is caught incidentally (row 3), while the **grow** direction was +uncatchable by the existing suite in every configuration tried — a test that issues +N items passes for every cap ≥ N, and one that consumes immediately passes for every +TTL > 0. The page leads with that asymmetry, and the baseline-GREEN row is the +control showing the added boundary cases are not simply always-failing. + +**Confidence: verified** (mutation semantics + language behaviour doc-sourced; +mechanism reproduced locally; the CSRF-store magnitudes kept as field measurement). + ## Existing-layer check Routed via `INDEX.md` → the three domain indexes, then read every page whose "load @@ -165,13 +215,15 @@ when" line overlapped. Full-body reads: `testing-quality-tests-that-cannot-fail` reads (grep for timeout/JdbcTemplate/trigram/LIKE/arg-capture terms, to establish absence of coverage): the remaining ids below. -Pages read: testing-quality-tests-that-cannot-fail, testing-mocking-what-to-mock, testing-quality-behavior-not-implementation, backend-common-change-impact-call-site-enumeration, databases-indexing-index-selection, databases-query-optimization-reading-execution-plans, databases-indexing-partial-and-expression-indexes, databases-indexing-covering-indexes, backend-common-orm-transaction-boundaries, backend-common-reliability-timeouts-and-retries, backend-common-errors-exception-handling, backend-java-jpa-persistence-context, backend-java-spring-proxy-pitfalls +Pages read: testing-quality-minimum-case-set, testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, testing-mocking-what-to-mock, testing-quality-behavior-not-implementation, backend-common-change-impact-call-site-enumeration, databases-indexing-index-selection, databases-query-optimization-reading-execution-plans, databases-indexing-partial-and-expression-indexes, databases-indexing-covering-indexes, backend-common-orm-transaction-boundaries, backend-common-reliability-timeouts-and-retries, backend-common-errors-exception-handling, backend-java-jpa-persistence-context, backend-java-spring-proxy-pitfalls **Overlaps found, and why each is composition rather than duplication.** | Existing page | Overlap | Resolution | |---|---|---| | `testing-mocking-what-to-mock` | Its step 1 last row and step 2 already say to "assert the **outbound contract**: which command, with what arguments", and one edge row says to "deep-equal the full stub-recorded call sequence" | Closest neighbour, but its subject is *whether* to replace a dependency — folding assertion-completeness in would break "one case per page". Kept separate; added the new id to its `related:` and a pointer on the outbound-contract row | +| `testing-quality-minimum-case-set` | Owns boundary-value selection for a function's inputs (60 body lines, room to merge) | Kept separate: its subject is the case set for the *behaviour*, while insight 4's is a shipped **default** whose trigger is a spec-named number plus an existing suite, with its own bidirectional procedure. Reciprocal `related:` added | +| `testing-quality-harness-reverse-controls` | Owns "prove the harness discriminates" | Cited from insight 4's step 5 — boundary cases one unit from a limit are where a test-side off-by-one imitates a caught mutant | | `testing-quality-tests-that-cannot-fail` | Owns per-assertion mutation granularity and the "testing the mock instead of the code" row | The new page cites it for the mutation step instead of restating it; added the new id to its `related:` | | `databases-indexing-index-selection` | Line 51 routes `LIKE '%term%'` to "a trigram or full-text index type" | That advice has an unstated precondition — exactly the new page. Extended the row to carry the minimum-keyword-length pointer, plus a `related:` link | | `databases-query-optimization-reading-execution-plans` | Owns plan reading generally | Cited from "When this applies"; the new page adds only the trigram-specific counter to read (`Rows Removed by Index Recheck`) | @@ -186,7 +238,10 @@ Pages read: testing-quality-tests-that-cannot-fail, testing-mocking-what-to-mock exactly one file in the whole wiki (`index-selection.md`, the one row above); `call_args|assert_called_with|toHaveBeenCalledWith|argument captor` matches exactly one (`what-to-mock.md`); no page mentions `statement_timeout`, `JdbcTemplate`, or -`QueryTimeout`. +`QueryTimeout`. For insight 4, `default value|defaults|default argument` across +`wiki/testing/**` returned six hits, every one incidental (a factory filling +fixture defaults, a permissive-schema note, a `f(x=None)` trap) — no page states a +directive about a shipped default's own test. **Format invariants checked mechanically after writing:** body lines 94 / 75 / 90 (limit 120); every `related:` id and inline `[page-id]` reference resolves to a @@ -212,6 +267,7 @@ Files touched by each open head, matched against the three candidates: | Candidate | Overlapping open PRs | Verdict | |---|---|---| | 1 — captured call arguments (testing/mocking) | #52 adds `testing/quality/source-text-wiring-assertions.md`; #49 adds `testing/quality/value-preserving-refactor-assertions.md` + `unasserted-return-fields.md`; #47/#52 modify `tests-that-cannot-fail.md` | **new** | +| 4 — default values under test (testing/quality) | Same three heads as candidate 1, plus #49's `unasserted-return-fields` and #47/#52's edits to `tests-that-cannot-fail.md` | **new** | | 2 — pg_trgm short patterns (databases/indexing) | none — zero open heads touch `wiki/databases/**` | **new** | | 3 — raw JDBC in a JPA transaction (backend/java/jpa) | none — the backend-touching heads (#68, #58, #56, #55, #51, #50, #72) are all under `backend/common/**` or `backend/python/**`; zero touch `wiki/backend/java/**` | **new** | @@ -233,6 +289,16 @@ in full or in relevant part: **returns** that no assertion reads). The new page is the call/argument direction. Deliberately kept as siblings. +**Why candidate 4 is `new`:** `#49 unasserted-return-fields` is the closest +in-flight page — it also turns on "a value no assertion reads" — but its subject is +fields a function **returns** on a given call, and its remedy is per-field +assertions plus cross-field relations. Candidate 4's subject is a **default that is +never supplied**, and its remedy is bidirectional mutation at the default's +observable point; the grow-direction asymmetry has no counterpart there. +`#49 value-preserving-refactor-assertions` covers a literal moved behind a config +read, the opposite direction of travel (the value is already asserted; the question +is whether the caller reads it). + No candidate is a pending duplicate, so nothing was dropped and no sibling duplicate PR is opened here. @@ -259,7 +325,12 @@ file byte-for-byte (the source reads `**2-gram**` with a line break and quote and the table cells were re-derived from the raw file cell by cell, which also surfaced footnote (\*2) — a second independent statement of the mechanism — and footnote (\*1), now recorded as an edge case. No claim was left resting on a -summarizer's paraphrase. +summarizer's paraphrase. Structural re-validation after insight 4 landed: all +`related:`/inline ids across the whole wiki resolve (the only two regex hits were a +literal `[a-z0-9]` character class in two pre-existing pages); the four new pages +are 94 / 75 / 90 / 77 body lines against the 120 limit; every page appears in its +domain index; and no index table row is malformed — one row was caught missing its +closing pipe and fixed. ## Routing decision @@ -269,7 +340,10 @@ summarizer's paraphrase. | 2 | `databases` / `indexing` | `wiki/databases/indexing/trigram-index-short-patterns.md` (`databases-indexing-trigram-index-short-patterns`) | No — `indexing` owns index-type suitability; the case is a precondition on one index type, and `query-optimization/reading-execution-plans` stays the owner of plan reading | | 3 | `backend` / `java` → `jpa` | `wiki/backend/java/jpa/raw-jdbc-inside-a-jpa-transaction.md` (`backend-java-jpa-raw-jdbc-inside-a-jpa-transaction`) | No — the mechanism is `JpaTransactionManager`/`JpaDialect`, so it belongs in the `jpa` category rather than `spring` (which owns proxy-level "the annotation did nothing") or `backend/common` (language-agnostic principles; this is stack-specific source behaviour) | +| 4 | `testing` / `quality` | `wiki/testing/quality/default-values-under-test.md` (`testing-quality-default-values-under-test`) | No — `quality` owns assertion sufficiency and mutation proof, which is what the case is about; `mocking` owns doubles and would misfile a page whose subject needs no double | + Plumbing updated: `wiki/testing/index.md`, `wiki/databases/index.md`, -`wiki/backend/java/index.md` each +1 "load when" row; `log.md` +1 ingest entry. +`wiki/backend/java/index.md` each +1 "load when" row (testing +2 — one per testing +page); `log.md` +1 ingest entry. `INDEX.md` unchanged — all three domains are already listed and their "route here when" lines already cover these cases. diff --git a/log.md b/log.md index c112eed..8a77674 100644 --- a/log.md +++ b/log.md @@ -44,3 +44,4 @@ Append-only. Format: `## [YYYY-MM-DD] 0. Kept separate from minimum-case-set (case set for the behaviour, not for a shipped default) and from the in-flight #49 unasserted-return-fields (return direction) — reciprocal related links added to minimum-case-set, tests-that-cannot-fail and captured-call-arguments; testing index +1 row. diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 545c830..41af489 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -27,6 +27,7 @@ Match your situation to a "load when" line; load only matching pages. | [guard-shape-vs-consequence](quality/guard-shape-vs-consequence.md) | A repo-wide guard asserting that no shipped artifact (example, config, migration, fixture) has a structural shape has gone red on a legitimate new artifact; authoring such a scanning guard; deciding between exempting an artifact, deleting the guard, and sharpening it; an existing guard has accumulated an exemption/allow list | | [injected-clock-duration-assertions](quality/injected-clock-duration-assertions.md) | Asserting an elapsed duration between two readings of an injected/fake float clock (rate-limit interval, backoff, debounce, TTL); choosing that fake clock's start value; a single duration test fails on correct code by a margin in the far decimal places; choosing a comparison tolerance, or deciding between float seconds and integer nanoseconds | | [write-path-assertions](quality/write-path-assertions.md) | Writing an HTTP-level test for an endpoint that persists something (form submit, create/update, onboarding step) and choosing what to assert beyond the status code; such a test is green while the records are empty or defaulted; sending repeated form fields from a client (httpx/TestClient) and deciding the `data=` shape | +| [default-values-under-test](quality/default-values-under-test.md) | A constructor/factory/config default is named as a number in a spec, plan, or measurement record (`ttl_s=600`, `max_tokens=256`, `retries=3`) and you are judging whether it is guarded; a mutation of that default left the suite green; deciding between a mechanism test that passes the value in and a test of the shipped default; choosing the boundary cases and the two mutation directions for a default | | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate | | [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect | diff --git a/wiki/testing/mocking/captured-call-arguments.md b/wiki/testing/mocking/captured-call-arguments.md index 972d438..d1d2fb1 100644 --- a/wiki/testing/mocking/captured-call-arguments.md +++ b/wiki/testing/mocking/captured-call-arguments.md @@ -13,6 +13,7 @@ last_verified: 2026-08-10 related: [ testing-mocking-what-to-mock, + testing-quality-default-values-under-test, testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, backend-common-change-impact-call-site-enumeration, diff --git a/wiki/testing/quality/default-values-under-test.md b/wiki/testing/quality/default-values-under-test.md new file mode 100644 index 0000000..c1d0d74 --- /dev/null +++ b/wiki/testing/quality/default-values-under-test.md @@ -0,0 +1,97 @@ +--- +id: testing-quality-default-values-under-test +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://pitest.org/quickstart/basic_concepts/ + - https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ + - https://docs.python.org/3/reference/compound_stmts.html +last_verified: 2026-08-10 +related: + [ + testing-quality-minimum-case-set, + testing-quality-tests-that-cannot-fail, + testing-mocking-captured-call-arguments, + testing-quality-harness-reverse-controls, + ] +--- + +# The Default Value of a Constructor or Factory Parameter + +## When this applies + +A constructor, factory, or config object has a default that a spec document, plan, +or measurement record names as a number (`ttl_s=600`, `max_tokens=256`, +`retries=3`), the class already has tests, and you are judging whether that default +is guarded — or a mutation run changed the default and the suite stayed green. + +Choosing the case set for the behaviour itself → [testing-quality-minimum-case-set]. +Asserting that a *caller* passes a constant on → [testing-mocking-captured-call-arguments]. + +## Do this + +1. **Separate the two subjects and give the default its own test.** A test that + passes the value in (`cls(ttl_s=0)`) pins the *mechanism* and says nothing about + the shipped default; a test that constructs with defaults pins the default only + if it *exercises* it. Both shapes are normal and both are needed — the gap is + that neither is the default's test. + +2. **Push the default to the point where it is observable, and assert from both + sides of it.** The observable point is the behaviour the number decides: + +| Default's role | Assert | +|---|---| +| A duration or TTL | Consumption just inside the boundary succeeds, and just outside it fails | +| A cap, limit, or pool size | Exactly the cap's worth of operations succeeds, and the next one is refused | +| A retry or attempt count | Exactly that many attempts are observed at the boundary the retries drive | +| A threshold or ratio | One case each side of the threshold, taken from the default's own value | +| An enum or mode | The behaviour that distinguishes this mode from the adjacent one | + +3. **Require red in both directions before believing the test.** Shrinking the + default and growing it are different mutants, and the growing direction is the + one no incidental test catches: a test that issues N items passes for every cap + ≥ N, and a test that consumes immediately passes for every TTL > 0. Run both + mutants and require your new test red for each. + +4. **Read the default's value from the code in the assertion, and assert the value + itself once.** `assert store.ttl_s == 600` next to the boundary case makes the + spec's number checkable at one place; deriving the boundary from + `store.ttl_s` keeps the boundary cases correct when the default legitimately + changes. + +5. **Run the unmutated suite and require green.** The boundary cases in step 2 sit + one unit from a limit, which is where an off-by-one in the *test* looks exactly + like a caught mutant ([testing-quality-harness-reverse-controls]). + +## Edge cases + +| Case | Then | +|------|------| +| The default is a duration long enough that exercising it would slow the suite | Reach the boundary by controlling the clock the code reads — an injected clock, or seeding the stored timestamp — rather than by shortening the default for the test, which turns it back into a mechanism test | +| An existing test happens to catch the shrink direction | Keep it and still add the grow direction: reproduced 2026-08-10, an existing test that issued 2 items reddened a `cap 256 → 1` mutant incidentally while `cap → 9999` stayed green in the same suite | +| The default is evaluated once at definition time (a Python mutable or computed default) | Assert the shared-state consequence as its own case — the language evaluates the default expression once when the function is defined, so two instances observe one object | +| The default is supplied by a framework or config layer, not by the signature | Assert the effective value after the layer resolves it, at the level that layer runs; a signature default the framework always overrides is not the shipped default | +| The spec document and the code disagree about the number | Fix the disagreement before writing the test, and record which one was authoritative — a test written against the wrong one locks the drift in | +| The default is deliberately unspecified (the caller is expected to always pass it) | Assert that omitting it is refused, so "no default" is itself the guarded behaviour | +| The value's only consequence is operational (a bind address, a timeout that only a probe observes) | Add one assertion at the level that observes it; the in-process suite cannot distinguish the values ([testing-mocking-captured-call-arguments]) | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Count a test that constructs with defaults as the default's coverage | Check whether that test reaches the default's observable point | Reproduced 2026-08-10: a suite whose "defaults" tests constructed with defaults and consumed immediately stayed **green** on a `ttl_s 600 → 1` mutant; the mechanism tests passed `ttl_s` in, so nothing read the default | +| Pass the value explicitly everywhere for determinism and leave it there | Keep those tests and add one default-valued test per number | Explicit passing is the right call for the mechanism, and it is exactly what makes the shipped default unasserted | +| Prove the default with one mutation in the direction that seems risky | Mutate it both smaller and larger | The grow direction survives every test that stays under the limit; reproduced, `ttl 6000 / cap 9999` was green against the whole existing suite | +| Shorten the default in a fixture so the boundary is quick to reach | Control the clock or the stored timestamp and keep the default | Changing the default for the test removes the subject; the test then proves the mechanism a second time | +| Write the boundary as a literal (`599`, `257`) | Derive it from the default read off the object | A literal boundary and a literal default drift apart, and the pair passes while neither matches the spec | +| Treat a green run after adding the test as proof it works | Require red on each mutant and green on the unmutated suite | A boundary case built one unit off reads as a caught mutant on every run, including the honest one | + +## Sources + +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; a changed default that leaves the suite green is that verdict for the default specifically, and PIT attributes a kill to the covering test rather than to the file +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the published mutator set operates on operators, literals, and blocks; a parameter default's value is reached by literal mutation, which is why step 3 states both directions explicitly rather than relying on a tool's single generated variant +- https://docs.python.org/3/reference/compound_stmts.html — "Default parameter values are evaluated from left to right when the function definition is executed", so a mutable or computed default is shared across calls; the basis for the shared-state edge case +- Reproduction 2026-08-10 (Python 3, a TTL + cap store, four existing tests: two constructing with defaults, two passing the values in explicitly): baseline green. `ttl_s 600 → 1` — existing suite **GREEN**, added bidirectional default tests RED. `max_tokens 256 → 1` — existing suite RED (caught incidentally, because one existing test issued two items), added tests RED. `ttl_s → 6000` and `max_tokens → 9999` together — existing suite **GREEN**, added tests RED. The unmutated run was green with the added tests, which is the control showing the boundary cases are not simply always-failing. The grow direction was uncatchable by the existing suite in every configuration tried +- Field measurement 2026-08-10 (a Python service's CSRF store, 8-round audit): `CsrfStore(ttl_s=600, max_tokens=256)` was named in the plan document, and `ttl_s=1` / `max_tokens=1` mutants passed all 65 existing cases. Three defaults-constructing tests consumed immediately or issued at most two tokens, and the two TTL/cap mechanism tests passed the values in. The production consequence of the surviving `ttl_s=1` was that any form taking longer than a second to fill would fail every submission while the page still rendered normally. Two bidirectional default tests turned all four mutants red diff --git a/wiki/testing/quality/minimum-case-set.md b/wiki/testing/quality/minimum-case-set.md index 1540584..403ab09 100644 --- a/wiki/testing/quality/minimum-case-set.md +++ b/wiki/testing/quality/minimum-case-set.md @@ -9,7 +9,7 @@ sources: - https://abseil.io/resources/swe-book/html/ch12.html - https://martinfowler.com/bliki/TestDrivenDevelopment.html last_verified: 2026-07-10 -related: [testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-checks-that-cannot-pass, qa-exploratory-guard-true-path-coverage] +related: [testing-quality-default-values-under-test, testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-checks-that-cannot-pass, qa-exploratory-guard-true-path-coverage] --- # Selecting the Minimum Case Set for a Function or Endpoint diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index 9af6765..ca2aace 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -16,7 +16,7 @@ sources: - https://git-scm.com/docs/git-checkout - https://git-scm.com/docs/git-restore last_verified: 2026-08-05 -related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, testing-mocking-captured-call-arguments, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] +related: [testing-quality-default-values-under-test, testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, testing-mocking-captured-call-arguments, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] --- # Proving a Test Can Fail