choiyounggi · choiyounggi · Aug 5, 2026 · Aug 5, 2026
@@ -39,3 +39,5 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
 ## [2026-08-03] dedup | Dropped 3 candidate pages as duplicates/superseded during the #6–#13 consolidation: testing/docs-as-spec/document-conformance-checks (#9 — same case as testing/quality/spec-artifact-checks from #8: coverage-vs-validity split, per-check negative controls, GFM pipe parsing; #8 kept as canonical, docs-as-spec category not created); backend/common/llm/gateway-model-alias-defaults (#6 — subsumed by the generalized integrations/externally-owned-defaults; the model-alias case is one instance); backend/common/integrations/llm-response-completeness (#12 — folded into llm/completion-response-validation, kept in llm/ for category coherence with context-window-budget).
 ## [2026-08-04] ingest | knowledge-flush of 3 queued insights. New: testing/quality/schema-additions-under-a-golden-gate (a new node kind absent from the golden example is untouched by every mutant derived from it — commit a minimal conforming fixture, one negative per keyword the new branch adds, restore each and require red; judge the suite by whether anything loads the schema at all), backend/common/change-impact/call-site-enumeration (NEW category change-impact: enumerate by callee name, treat a parameter-name search as a partial index — positional-or-keyword is Python's default parameter kind and ast.Call keeps args/keywords in separate fields; re-run the enumeration after the edit). Merged: testing/data/test-data-and-isolation +1 Do row / +1 edge case / +1 Instead-of (a factory whose shape depends on a value the test also passes to the code under test takes it as a parameter, never as a module-level default). All cited URLs live-verified this session; one local reproduction (Python 3.14.6: keyword search 1 hit vs AST callee enumeration 4).
 ## [2026-08-04] ingest | knowledge-flush: 3 queued session insights merged into existing pages (no new pages — merge-before-create). platforms/environment/path-resolution +keg-only case: a package manager reports a tool installed but `command -v`/`which` find nothing because keg-only/unlinked formulae (llvm, openssl, curl) are deliberately off PATH — run from `$(brew --prefix)/opt/<f>/bin` (verified: `which mlir-opt` not found vs `/opt/homebrew/opt/llvm/bin/mlir-opt` → LLVM 22.1.8; source docs.brew.sh/FAQ). platforms/processes/non-interactive-cli-invocation +bracketed-paste edge: injecting a long/multiline prompt into a REPL (tmux `send-keys -l`) stalls at `❯ [Pasted text #1]` because the input is one bracketed-paste block (ESC[200~…201~) whose embedded newline is not submit — send Enter as a separate keystroke a beat later (source en.wikipedia.org/wiki/Bracketed-paste + claude-code#43169). testing/strategy/test-level-choice +import-side-effect edge/instead-of: a "pure" function's test is not dependency-free if its module runs I/O at import; `@pytest.mark.skipif` evaluates after the module import so it can't gate it — use `importorskip`/`skip(allow_module_level=True)` or move the function to a side-effect-free module (source docs.pytest.org skipping). Harvested "infrastructure" hint for the keg-only insight re-routed to platforms/environment (dedicated PATH page). Confidence: keg-only & pytest verified vs official docs; bracketed-paste mechanism doc-verified, the claude-CLI submit specifics field-tested.
+## [2026-08-05] ingest | knowledge-flush of 3 queued insights (qa +3, all verified). New: process/scope-purity-checks (default porcelain collapses an entirely-untracked directory to `?? dir/` so per-file path filters false-flag it — pass `-uall` explicitly since `status.showUntrackedFiles` can shift the ambient default; rename lines carry two paths; `-z` for quoting; validated by scratch-repo reproduction on git 2.50.1 + git-scm docs), exploratory/override-control-pairs (before trusting a `--field k=v`/env override matrix into an ignore-unknown-keys consumer, run a control pair that flips an observable; uniform output = lever not connected, not stable behavior — reproduced against a workflow runner where bare key names were silently ignored at exit 0 and the canonical dotted name flipped the guard; pydantic extra-ignored default cited as mechanism prevalence), exploratory/guard-true-path-coverage (run every guard's true and false direction once and assert on the executed-step list — when compile/validation do not resolve cross-node references, the guard-true run is the only reference check; reproduced: undeclared event emit passed compile 0-errors + validate PASS and failed only in runs where the guard let emit execute; ISTQB branch coverage). Reverse related-links added to checks-that-cannot-pass, harness-reverse-controls, minimum-case-set, exploratory-sessions.
+## [2026-08-05] ingest | knowledge-flush round 2, same PR: 2 more queued insights (security +1, qa +1, both verified by local reproduction). New: security/data/masking-verification (masking is per-channel — enumerate every output channel and grep the planted raw secret in each, paired with an unmasked negative-control field; reproduced: one lnpl `--json` run held the raw card number at result.bindings while trace/logs showed `***`, and the platform's differential check compares only masked-clean channels), qa/exploratory/lowered-declaration-survival (a DSL accepting stacked declarations at exit 0 can silently keep only the last — count survivors in the lowered IR/AST by condition content, not node id, before trusting runtime; reproduced: two stacked `when` guards lowered to a single Guard node carrying the first guard's id and the second guard's condition). Queue hint `debugging` for the second re-routed to qa/exploratory (pre-failure QA verification, not failure diagnosis). Backlinks: pii-handling, guard-true-path-coverage, override-control-pairs.
@@ -8,7 +8,7 @@ sources:
   - https://www.satisfice.com/download/session-based-test-management
   - https://developsense.com/blog/2009/08/testing-vs-checking
 last_verified: 2026-07-10
-related: [qa-bug-reports-reproducible-reports, qa-process-release-gates]
+related: [qa-bug-reports-reproducible-reports, qa-process-release-gates, qa-exploratory-guard-true-path-coverage]
 ---
 
 # Running a Chartered Exploratory Testing Session

@@ -0,0 +1,69 @@
+---
+id: qa-exploratory-guard-true-path-coverage
+domain: qa
+category: exploratory
+applies_to: [general]
+confidence: verified
+sources:
+  - https://istqb-glossary.page/branch-coverage/
+  - "Field reproduction (lnpl 0.2.0, 2026-08-05): undeclared event reference passed compile (0 errors) and IR validation (PASS); failed only in runs that executed the guarded emit step; the one run where a presence guard skipped the step exited 0"
+last_verified: 2026-08-05
+related: [qa-exploratory-override-control-pairs, testing-quality-minimum-case-set, qa-exploratory-exploratory-sessions, qa-exploratory-lowered-declaration-survival]
+---
+
+# Executing Guard-True Paths When Static Stages Skip Reference Resolution
+
+## When this applies
+
+You are QA-ing a program or workflow whose steps hide behind guards (`when` /
+`until` conditions, optional branches) in a pipeline whose compile or schema
+validation does not resolve cross-node references; or a guarded step has never
+executed in any green run you are about to cite.
+
+## Do this
+
+1. **Enumerate every guard and run each direction at least once** — one run
+   where the guard is true and one where it is false — and record which run
+   exercised which direction in a bidirectional contrast table (guard, true
+   run, false run, observed signal on each side). This is branch coverage
+   ("the percentage of branches that have been exercised by a test suite" —
+   ISTQB) applied at the whole-program QA level.
+2. **Treat exit 0 from a run that skipped a guarded step as evidence about the
+   skip path only.** The guarded step's body is unexecuted code of unknown
+   validity; repeated green runs that all skip it accumulate no evidence about
+   it.
+3. **Establish what the static stages actually check.** When compile and
+   validation do not resolve references between nodes, a dangling reference
+   inside a guarded step (an undeclared event, a missing target id) survives
+   every static stage and surfaces only when the guard lets the step run — so
+   the guard-true run is the *only* reference check that exists. Run it before
+   calling the artifact shippable.
+4. **For loop guards (`until`, retry conditions), the zero-iteration case is
+   also a branch**: include one run where the condition is satisfied
+   immediately, alongside the run that iterates.
+5. **Assert on the executed-step list, not on skip markers alone.** Observed
+   asymmetry in the field reproduction: a zero-round `until` loop was absent
+   from the skipped list even though its body never ran — skip markers and
+   execution records can disagree.
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| A guard-true state is unreachable through the program's inputs | Record it explicitly as a coverage gap in the QA report; do not claim full guard coverage silently |
+| Two guards cannot both be true in one run | Cover them in separate runs; one row per guard in the contrast table, each with its own true/false pair |
+| The guard-true run fails late in QA | That is the mechanism working — the error was latent behind the guard; fix, then re-run both directions of that guard |
+| Static validation claims to check references | Verify with a planted dangling reference that it actually fails validation; if it passes, treat reference resolution as runtime-only |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Ship after N green runs that all skipped a guarded step | Force one guard-true run per guard first | Compile 0 errors + validation PASS + exit 0 never executed the step body; an error inside it stays latent until production hits the guard-true state |
+| Cite validator PASS as proof references resolve | Check whether the validator resolves references; if not, count only guard-true runs as reference checks | Validation scope and reference resolution are separate concerns; PASS on the former says nothing about the latter |
+| Infer skipped steps from a skipped-markers list | Assert on the executed-step list | Zero-iteration loops can be missing from both lists — only the execution record is authoritative |
+
+## Sources
+
+- https://istqb-glossary.page/branch-coverage/ — "The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage"
+- Field reproduction (2026-08-05, lnpl 0.2.0 workflow runner): an `emit` referencing an undeclared event passed compile (0 errors) and IR validation (PASS); 6 of 7 runs failed at runtime at the emit step ("EventEmit references undeclared event"), while the single run where a presence guard skipped emit exited 0 — a guard-skipping input masks the defect indefinitely. Raw run outputs archived alongside the QA case
@@ -0,0 +1,64 @@
+---
+id: qa-exploratory-lowered-declaration-survival
+domain: qa
+category: exploratory
+applies_to: [general]
+confidence: verified
+sources:
+  - "Local reproduction (lnpl 0.2.0, 2026-08-05): two stacked `when` guards compiled rc=0 with zero diagnostics; Semantic IR and MLIR both held exactly one Guard node — the first guard's id carrying the second guard's condition"
+last_verified: 2026-08-05
+related: [qa-exploratory-guard-true-path-coverage, qa-exploratory-override-control-pairs]
+---
+
+# Counting Stacked Declarations in the Lowered Artifact
+
+## When this applies
+
+A compiler/DSL accepts multiple stacked declarations (consecutive guards,
+policies, annotations) with exit 0 and no diagnostic, and you are about to
+trust runtime behavior that depends on all of them; or a run honored only one
+of several declared rules.
+
+## Do this
+
+1. **Dump the lowered artifact and count that every declared item survived**,
+   before any runtime trust: semantic IR (compile stage output), MLIR/AST dump,
+   or the generated config — whichever lowered form the toolchain exposes. Key
+   the count to source constructs: one node per declared guard, each with its
+   own condition text.
+2. **Read "exit 0 + no diagnostic" as absence of rejection, not as
+   acceptance.** Lowering can keep only the last of consecutive declarations
+   silently: in the reproduction, two stacked `when` guards produced a single
+   Guard node, and the loss was invisible in every compiler message.
+3. **Count by content, not by id presence.** The surviving node can carry the
+   *first* item's id with the *second* item's payload (observed:
+   `…guard.1` holding the second guard's condition) — so "an id exists per
+   declaration" is a weaker check than "each declared condition string appears
+   once".
+4. **When a declaration is missing from the artifact, probe runtime with an
+   input only the dropped rule would reject** to size the impact (in the
+   originating case, runtime approved amounts `0` and `-1` that the dropped
+   guard existed to block), then report the silent drop as a finding
+   ([qa-exploratory-guard-true-path-coverage] owns exercising the surviving
+   guards).
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| The CLI has no explicit IR-dump flag | Use whatever the build leaves behind — a compile stage that prints IR JSON, `.mlir`/`.ll` files in the build workdir — any lowered form supports the count |
+| The language documents merge semantics for stacked declarations | Check the documented merge rule first and verify the merged node matches it; an undocumented merge or last-wins is the finding |
+| Same-kind declarations live in different scopes (per-service, per-workflow) | Count within each scope separately — cross-scope totals hide a drop in one scope |
+| The artifact nests or renames declarations during lowering | Match on the invariant part (condition text, rule value), not on source-level names |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Trust exit 0 as proof all declarations took effect | Count survivors in the lowered artifact first | Last-wins lowering drops predecessors with no warning; the loss surfaces only when an input the dropped rule guarded slips through |
+| Verify the declarations with one runtime input | Pick one input per declared rule, including inputs only a dropped rule would reject | A single input can satisfy the surviving rule and still say nothing about the dropped ones |
+| Check that a node id exists per declaration | Check each declared condition/value appears in the artifact | The surviving node can reuse a dropped declaration's id |
+
+## Sources
+
+- Local reproduction (2026-08-05, lnpl 0.2.0): a workflow with `when approval.amount > 100` directly followed by `when approval.amount < 0` compiled with rc=0 and zero diagnostics; the Semantic IR JSON and the lowered MLIR each contained exactly one Guard node, `wf.approve.refund.guard.1`, with condition `approval.amount < 0` — the second declaration's condition under the first declaration's id. The originating QA session observed the runtime consequence: amounts `0` and `-1` were approved past the dropped guard
@@ -0,0 +1,66 @@
+---
+id: qa-exploratory-override-control-pairs
+domain: qa
+category: exploratory
+applies_to: [general]
+confidence: verified
+sources:
+  - https://pydantic.dev/docs/validation/latest/concepts/models/
+  - "Field reproduction (lnpl 0.2.0, 2026-08-05): 5-run override matrix flat under bare key names, exit 0, no warning; canonical dotted name flipped the guarded step — raw traces archived with the QA case"
+last_verified: 2026-08-05
+related: [qa-exploratory-guard-true-path-coverage, testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, qa-exploratory-lowered-declaration-survival]
+---
+
+# A Control Pair Before Trusting a Value-Override Matrix
+
+## When this applies
+
+You are measuring branch/guard behavior by feeding a matrix of values through
+name-based runtime injection — CLI `--field key=value`, environment overrides,
+config overlays — into a consumer whose policy for unknown keys is "ignore".
+Also whenever every variant in such a matrix returns identical observations.
+
+## Do this
+
+1. **Before running the matrix, run one control pair**: two runs whose injected
+   values are chosen to flip a concrete observable (a guard fires in one and
+   skips in the other). Require the flip. Until the flip is observed, no run of
+   the matrix measures anything.
+2. **Read uniform output across all variants as "lever not connected", not
+   "behavior stable".** Ignore-unknown-keys is a common default — pydantic, for
+   example: "By default, Pydantic models won't error when you provide extra
+   data, and these values will simply be ignored" — so a mis-named key produces
+   the default-value branch on every run, with exit 0 and no warning.
+3. **Look up the canonical internal key name** (normalized dotted path, IR
+   field id) from the tool's trace output, IR dump, or docs — not from the
+   spelling you used in source. A bare name that normalizes differently lands
+   in the "not compared" bucket silently.
+4. **When the consumer offers a strict mode** (error/warn on unknown keys —
+   pydantic `extra='forbid'`, schema `additionalProperties: false`), turn it on
+   for the measurement run so a mis-named key fails loudly instead of
+   defaulting.
+5. **Do not count exit codes as evidence the values landed.** The failure mode
+   is exit 0 across the whole matrix.
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| The tool's help text documents the ignore policy | The run is still silent — read the key-handling policy before building the matrix, and pick key names from the documented canonical form |
+| The chosen observable is insensitive to the value (aggregate status, summary count) | Switch to an observable the value provably drives: executed-step list, trace line, emitted record — then re-run the control pair |
+| No injected value flips anything observable | Instrument first (verbose/trace mode, debug output) before measuring; a matrix without any observable lever produces only noise |
+| The control pair flips but a later matrix cell looks impossible | Re-run that cell's control neighbor — key handling can differ per field (compared vs non-compared fields) |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Conclude "behavior is stable" from identical outputs across a value matrix | Run a control pair and require the observable to flip first | Silently ignored keys make misconfiguration masquerade as coverage — every run exercised the same default branch |
+| Trust the key spelling you wrote in source files | Find the normalized internal name from trace/IR output | Normalized (dotted) and bare names diverge; the bare name is classified as a field the run does not compare on |
+| Cite the matrix's uniform exit 0 as passing evidence | Cite the flipped observable from the control pair, then per-cell observations | Exit status stays 0 while the injection is ignored |
+
+## Sources
+
+- https://pydantic.dev/docs/validation/latest/concepts/models/ — "By default, Pydantic models won't error when you provide extra data, and these values will simply be ignored"; `ConfigDict(extra=...)` values `ignore` (default) / `allow` / `forbid` — the ignore-unknown default this page defends against
+- Field reproduction (2026-08-05, lnpl 0.2.0 workflow runner): five `--field value=N` runs produced byte-identical step traces (guard always false, exit 0, no diagnostics); the tool's own help stated "Fields the workflow does not compare on are ignored"; switching to the canonical dotted name `measurement.value` flipped the guarded create step. Raw run outputs archived alongside the QA case
+- [testing-quality-harness-reverse-controls] — the same principle applied to harnesses that *score* verification: a uniform verdict is a property of the instrument, proven otherwise only by a control run