From 008a738685c820c1bddabf5385fb782d769304aa Mon Sep 17 00:00:00 2001 From: dch0202 Date: Wed, 5 Aug 2026 13:14:55 +0900 Subject: [PATCH] knowledge: ingest 9 verified insight(s) 5 new pages, 2 existing pages extended. New: - databases/schema-design/verifying-additive-migrations - platforms/processes/driving-a-tui-in-a-tmux-pane - platforms/processes/harness-tool-result-interception - testing/mocking/destructive-operations-on-shared-daemons - backend/common/reliability/client-side-rate-limit-pacing Extended: - platforms/shells/portable-shell-scripts (POSIX set -- scope, payload quoting) - platforms/shells/command-text-inspected-before-execution (blocked vs silent success) Five claims reproduced locally (tmux 3.7b, /bin/sh, dash, zsh); the rest sourced to SQLAlchemy, PostgreSQL, POSIX, tmux.1, Claude Code hooks, and Okta/Auth0/GitHub rate-limit docs. One page kept at field-tested because its provider quota could not be sourced. --- .dev-loop/INGEST_REPORT.md | 265 ++++++++++++------ INDEX.md | 8 +- log.md | 2 + .../client-side-rate-limit-pacing.md | 80 ++++++ wiki/backend/index.md | 1 + wiki/databases/index.md | 1 + .../verifying-additive-migrations.md | 82 ++++++ wiki/platforms/index.md | 6 +- .../processes/driving-a-tui-in-a-tmux-pane.md | 76 +++++ .../harness-tool-result-interception.md | 68 +++++ ...command-text-inspected-before-execution.md | 27 +- .../shells/portable-shell-scripts.md | 31 +- wiki/testing/index.md | 1 + ...estructive-operations-on-shared-daemons.md | 80 ++++++ 14 files changed, 634 insertions(+), 94 deletions(-) create mode 100644 wiki/backend/common/reliability/client-side-rate-limit-pacing.md create mode 100644 wiki/databases/schema-design/verifying-additive-migrations.md create mode 100644 wiki/platforms/processes/driving-a-tui-in-a-tmux-pane.md create mode 100644 wiki/platforms/processes/harness-tool-result-interception.md create mode 100644 wiki/testing/mocking/destructive-operations-on-shared-daemons.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index c375750..1634b31 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,95 +1,190 @@ -# Consolidated review — knowledge PRs #6–#13 +# Knowledge flush — 9 insight(s) -Eight fork PRs (`dch0202-rsquare`, 2026-07-28 → 2026-08-02) were reviewed together -against `AGENTS.md`. Each PR was audited by an independent reviewer (format rules, -sources, vague-qualifier ban, ≤120 body lines, index/log invariants), then -cross-compared to catch duplication the per-PR flushes could not see — they branched -independently off the same main and rewrote the same shared index/log files. Fork -branches can't be edited from here and several PRs needed content changes (drop a -duplicate, merge a colliding page), so this branch carries the reconciled end-state -rather than merging each PR as-is (which would import the duplicates). +Drained 9 pending candidates from 7 session files in `~/.dev-loop/queue`. Result: +**5 new pages, 2 existing pages extended, 0 dropped.** No candidate was accepted on +assertion alone — each was re-derived from primary docs, reproduced locally, or +demoted to `field-tested` with the reason stated. ## Verified best-practice -Sources are per-page and were live-verified in each originating PR's flush; the -independent re-reviews re-checked them. Landed pages and their evidence base: - -| Page | Confidence | Source basis | -|------|-----------|--------------| -| backend/common/llm/completion-response-validation | verified | OpenAI reasoning guide + chat `object` spec (5 `finish_reason` values), vLLM/LiteLLM reasoning fields; field incident (200/`length`/empty content/8,173-char reasoning) | -| backend/common/llm/context-window-budget | verified | Claude context-window docs, LiteLLM exception mapping, vLLM/Claude Code env-var docs | -| backend/common/integrations/externally-owned-defaults | verified | OpenAI deprecations (notice windows) + models `list`, LiteLLM model_discovery; field incident (alias removed between PR verify and review → 400) | -| backend/common/storage/object-key-persistence | verified | AWS S3 CompleteMultipartUpload + managed-upload API/source, aws-sdk-js issues #1158/#5656 | -| infrastructure/containers/host-cgroup-visibility | field-tested | cgroup_namespaces(7), Docker `--cgroupns=host`, nsenter, k8s #103363; OrbStack repro | -| infrastructure/observability/missing-container-metrics | verified/field-tested | k8s resource-metrics-pipeline docs, kube-prometheus-stack values, kubernetes-mixin; OrbStack #2217 repro | -| platforms/environment/unicode-text-matching | verified | UAX #15, Unicode core §3.12, APFS FAQ, POSIX grep; local repro (macOS 15/APFS, grep 2.6.0-FreeBSD, Python 3.13) | -| platforms/shells/command-text-inspected-before-execution | verified | Claude Code hooks docs, POSIX shell §2.6; local reproduction | -| platforms/processes/non-interactive-cli-invocation | verified | GNU nohup, OpenBSD ssh/ssh_config, git, timeout man pages; no-request-in-gateway-log field incident | -| qa/document-verification/spec-document-gates | field-tested | ESLint, Google mutation testing, RFC 2119, Vale, markdownlint; 32/32 mutant / 62/62 intact RFC sessions | -| qa/document-verification/editing-a-gated-document | field-tested | pgrep, Vale, markdownlint; in-house editing methodology | -| testing/quality/checks-that-cannot-pass | verified | James Shore AoAD2, POSIX grep exit status, Semgrep rule-testing, pytest exit codes; BSD/ugrep measurement | -| testing/quality/spec-artifact-checks | verified | JSON Schema, ESLint RuleTester, pitest, GFM table spec; local cell-count repro + GitHub renderer cross-check | -| testing/quality/harness-reverse-controls | verified | mutation-testing + CI-control sources; field repro (re-fetched all cited URLs, PASS) | - -Three pages were reconciled from two overlapping PR versions each, keeping the more -complete/better-sourced body and folding in the other's unique cases: -- **completion-response-validation** — #12 body (all five `finish_reason` values, - `tool_calls`/`function_call` carve-out, streaming, Responses API, "reasoning is - scratch, not deliverable") kept in `llm/` (coherent with #6/#13); folded in #6's - DeepSeek first-party edge + the field incident. -- **externally-owned-defaults** — #12 generalized body (any repo-external resource) - in `integrations/`; folded in #6's alias-removed field incident + the - gateway-config-vs-live-upstream nuance. -- **non-interactive-cli-invocation** — #12 body (GNU-nohup extension precision, - ssh -n stdin-detach vs BatchMode, pre-log DNS/TLS/proxy + `curl -v`) kept; folded - in #11's DEBIAN_FRONTEND, pager/color TTY case, wrapper-CLI case, field incident. +### 1. `create_all()` + hand-written ALTER — testing additive migrations +**Claim:** a test that calls `init_db()` twice does not test the migration; only a +run that starts from the *previous* schema does. +**Checked:** SQLAlchemy [metadata](https://docs.sqlalchemy.org/en/20/core/metadata.html) +— `create_all()` "will issue queries that first check for the existence of each +individual table, and if not found will issue the CREATE statements", and altering +constructs "via the ALTER statement … is outside of the scope of SQLAlchemy itself". +SQLAlchemy [defaults](https://docs.sqlalchemy.org/en/20/core/defaults.html) — +`server_default` "gets placed in the CREATE TABLE statement", and neither default +backfills existing rows. PostgreSQL [ddl-alter](https://www.postgresql.org/docs/current/ddl-alter.html) +— "the default value will be returned the next time the row is accessed". +**Correction applied:** the candidate credited the surviving-row values to the model's +`server_default`. The docs place the backfill on the `DEFAULT` clause of the `ALTER` +instead; the page now says so, and adds `ADD COLUMN` without a `DEFAULT` as the defect +the assertion catches. → **verified** + +### 2. tmux pane-diff is not delivery evidence +**Claim:** a busy pane echoes typed characters, so a `capture-pane` diff reports +"delivered" for input that was never consumed. +**Reproduced (tmux 3.7b, macOS):** sent `echo SECOND_PROMPT` to a pane running +`sleep 6` — pane content changed (naive diff → delivered) while the command's own +output count stayed **0**, becoming **1** only after the sleep drained. Mechanism is +terminal `ECHO` in canonical mode +([POSIX chap11](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap11.html)). +→ **verified** + +### 3. `tmux send-keys` payloads need a `--` separator +**Reproduced:** `tmux send-keys -t S -l "-n hello"` → `command send-keys: unknown flag -n`, +exit 1; the identical call with `-- "-n hello"` → exit 0. +[POSIX Guideline 10](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html) +gives the mechanism. Noted honestly in the page: [tmux.1](https://man.openbsd.org/tmux.1) +documents `-l` but does **not** document `--`; the behaviour follows getopt convention. +→ **verified** + +### 4. Backticks survive double quotes and gut a CLI text payload +**Reproduced** — in zsh, with the command example backquoted inside double quotes: + +``` +$ echo "run: `pip install -e .[dev]` first" +zsh:1: no matches found: .[dev] +run: first # exit 0, message gutted +``` + +bash substituted the command's output instead; the single-quoted form printed the +text verbatim. +[POSIX 2.2.3](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html): +the backquote "shall retain its special meaning introducing the other form of command +substitution" inside double quotes. → **verified** + +### 5. POSIX `set --` inside a helper function is discarded +**Reproduced under `/bin/sh` and `dash`:** `parse_flags "$@"` left the caller with +`argc=4` still containing `--dry-run`; the identical loop inline gave `argc=3` with the +space-bearing operand intact. `DRY=1` in **both** — the flag is detected either way, so +only the operand list is wrong and nothing errors. POSIX 2.9.5: on return "the value of +the special parameter `#` and the positional parameters shall be restored to the values +they had before the function was executed". → **verified** + +### 6. A gate-blocked command is indistinguishable from a silent success +**Claim:** a `worktree_escape` guardrail blocked the orchestrator's own +`status-update.sh`; empty stdout read as success while no status file was written. +**Verified in-session:** the status directory was empty afterwards, and a Write-tool +call to the same tree succeeded — proving the block is command-text-scoped, not a +filesystem permission. Consistent with the page's already-cited +[hooks doc](https://code.claude.com/docs/en/hooks) (exit 2 blocks; the reason goes to +stderr, not stdout). → **verified** + +### 7. A harness hook can replace a tool's result +**Claim:** a session-memory plugin returned only line 1 of a file plus a remediation +note that did not work. +**Checked:** the [hooks doc](https://code.claude.com/docs/en/hooks) documents +`PostToolUse` `updatedToolOutput` — it "replaces the tool's result" — and names +transformation of "inbound tool results" as an intended use. The *mechanism* is +therefore verified; the specific plugin's behaviour is dated field context in the page, +and the page's rule is "size-check with `wc` first, because interception and a +genuinely small file look identical". → **verified** (mechanism) with dated field context + +### 8. PATH-injected recording fake for destructive daemon sweeps +**Reproduced:** with a fake `tmux` prepended to `PATH`, a prefix sweep over fixtures +`run-1, run-2, mydev` logged exactly `KILL run-1` / `KILL run-2`, **zero** bystander +lines, and an **empty** log for a non-matching prefix — while the machine's 8 real tmux +sessions, 3 of which matched the pattern under test, were untouched. The seam is +[POSIX chap08](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap08.html) +`PATH` search order; fake-vs-mock framing from +[Fowler](https://martinfowler.com/articles/mocksArentStubs.html). → **verified** + +### 9. Auth/token requests bypass a method-level rate-limit throttle +**Claim:** a throttle on the client's public methods misses the token POST issued +inside `_headers()`, so two requests leave in the same second. +**Checked:** [Okta](https://developer.okta.com/docs/reference/rate-limits/) documents +per-endpoint rate-limit buckets covering OAuth token endpoints; +[Auth0](https://auth0.com/docs/troubleshoot/customer-support/operational-policies/rate-limit-policy) +publishes an `/oauth/token` limit; [GitHub](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api) +states OAuth-app requests "count towards" the user's limit. Together these source the +general rule (token issuance is metered) and the remedy (throttle at the lowest +HTTP-issuing layer). +**Honest limit:** the specific 2-requests/second quota and the timestamped log evidence +come from one provider's mock environment and were **not** confirmed against that +provider's published spec — no primary doc stating the number was found, and no URL was +invented for it. The page is therefore **field-tested**, not verified, with the +provider-specific numbers confined to a "Field context" note. ## Existing-layer check -Cross-PR and against-main duplication was the focus. Findings and resolutions: - -- **spec-artifact-checks (#8) ≡ document-conformance-checks (#9)** — same case - (coverage-vs-validity split, per-check negative controls, GFM pipe parsing, - ESLint/Semgrep/mutation examples). #9's report predated awareness of #8. → - **#8 kept canonical; #9's page dropped, `testing/docs-as-spec` category not created.** -- **completion-response-validation (#6) ≈ llm-response-completeness (#12)** — ~95% - same case (HTTP 200 ≠ usable output; `length`/blank/reasoning-budget). → - **merged into one `llm/` page; #12's `integrations/` copy dropped.** -- **gateway-model-alias-defaults (#6) ≈ externally-owned-defaults (#12)** — ~80%; - #12 generalizes the model-alias case to any external resource. → - **kept the general `integrations/` page; #6's LLM-only page dropped.** -- **non-interactive-cli-invocation** — created by BOTH #11 and #12 (file collision). - → **single reconciled page.** -- Distinct (no overlap, all landed): checks-that-cannot-pass, harness-reverse-controls, - spec-document-gates, editing-a-gated-document, unicode-text-matching, - command-text-inspected-before-execution, object-key-persistence, context-window-budget, - host-cgroup-visibility, missing-container-metrics. -- Reciprocal `related:` links added on existing pages (tests-that-cannot-fail, - timeouts-and-retries, environment-config, release-gates, background-services, - portable-shell-scripts, timezone-and-locale, paths-case-and-line-endings, - acceptance-criteria, resource-limits-and-probes, logs-metrics-signals, - minimum-case-set). A dropped-page backlink (#6 → gateway-model-alias-defaults on - environment-config and release-gates) was retargeted to externally-owned-defaults. -- Invariants verified programmatically: all `related:`/inline `[id]` references - resolve, every page listed in its domain index, no duplicate ids, no page >120 - body lines. +Routed via `INDEX.md`, then read every domain index whose route line overlapped, then +every page whose "load when" line could collide. + +**Pages read in full:** `platforms/shells/portable-shell-scripts`, +`platforms/shells/command-text-inspected-before-execution`, +`platforms/processes/non-interactive-cli-invocation`, +`platforms/processes/background-services` (its tmux mention), +`databases/schema-design/online-schema-changes`, `testing/mocking/what-to-mock`, +`backend/common/reliability/timeouts-and-retries`. Plus a repo-wide grep for +`tmux|orchestrat|rate limit|throttl|quota` to catch coverage the index lines hide. + +**Merged rather than duplicated (2):** + +| Candidate | Merged into | What was added | +|-----------|-------------|----------------| +| POSIX `set --` in a function; backticks in a double-quoted payload | `platforms/shells/portable-shell-scripts` | Step 5 extended with the POSIX-sh in-place `"$@"` reordering idiom + the inline-only rule; new step 6 on single- vs double-quoting a text payload; 3 edge cases, 3 Instead-of rows, 3 sources | +| Guardrail blocks a script, silent failure | `platforms/shells/command-text-inspected-before-execution` | New steps 6–7 (verify the artifact, not the silence; hand a blocked signal back as un-emitted); 2 edge cases, 2 Instead-of rows, dated field context | + +Both pages' `last_verified` bumped to 2026-08-05; both stay under the 120-line cap +(92 and 107 body lines). + +**Overlap examined and rejected as a merge (4):** + +- `timeouts-and-retries` covers outbound-call reliability (timeouts, retry-by-failure-type, + backoff, concurrency caps) but never client-side pacing to a provider quota — a + repo-wide grep found no `throttl|rate limit` page outside edge/WAF contexts in + `security/`. New page; cross-linked for the 429/`Retry-After` path. +- `what-to-mock` decides *whether* to substitute an in-process dependency at an + interface. The daemon case has a different seam (`PATH`, not an interface) and a + different stake (the test can destroy the developer's environment). New page, + `related`-linked. +- `online-schema-changes` owns ALTER **lock** behaviour, not proving that a hand-rolled + migration ran. New sibling page in the same category, cross-linked for the + volatile-default rewrite case. +- `non-interactive-cli-invocation` owns *starting* a prompt-capable CLI; driving an + *already-running* TUI through a pty is a distinct case. New page, `related`-linked. + +**Conflicts flagged:** none. No new directive contradicts an existing one. + +**Related-links added:** every new page links back into the existing graph +(`online-schema-changes`, `nullability-and-defaults`, `test-data-and-isolation`, +`tests-that-cannot-fail`, `what-to-mock`, `path-resolution`, +`command-text-inspected-before-execution`, `non-interactive-cli-invocation`, +`background-services`, `timeouts-and-retries`, `jwt-server-side`, `hypothesis-testing`). +An invariant pass confirms every `related:` id and inline `[page-id]` reference +resolves, all 7 touched pages appear in their domain index, and no page exceeds 120 +body lines. ## Routing decision -- `backend/common/llm/` (new) — LLM-specific server concerns: completion-response-validation, - context-window-budget. Coherent home shared by #6 and #13. -- `backend/common/integrations/` (new) — general repo-external-dependency concern: - externally-owned-defaults. Kept separate from `llm/` because its scope is any - external resource (bucket/queue/index), not LLM-only. -- `backend/common/storage/` (new) — object-key-persistence. -- `qa/document-verification/` (new) — spec-document-gates, editing-a-gated-document. - Introduced by both #10 and #11; unified into one index section. -- `testing/quality/` (existing) — checks-that-cannot-pass, spec-artifact-checks, - harness-reverse-controls (test/check-authoring discipline, distinct from - qa/document-verification which is release-process gate design). -- `platforms/{environment,shells,processes}/` (existing) — unicode-text-matching, - command-text-inspected-before-execution, non-interactive-cli-invocation. -- `infrastructure/{containers,observability}/` (existing) — host-cgroup-visibility, - missing-container-metrics. - -Source PRs #6–#13 are closed with a disposition comment crediting the author. +| # | Insight | Target | New/merge | +|---|---------|--------|-----------| +| 1 | Testing an additive migration under `create_all()` + hand-written ALTER | `databases/schema-design/verifying-additive-migrations` | **new** | +| 2+3 | Confirming a keystroke reached a TUI in a tmux pane; `--` for `send-keys` payloads | `platforms/processes/driving-a-tui-in-a-tmux-pane` | **new** (one page — both are the same operation) | +| 7 | A harness hook substituted a tool's result | `platforms/processes/harness-tool-result-interception` | **new** | +| 8 | Sweep-tests against a live shared daemon | `testing/mocking/destructive-operations-on-shared-daemons` | **new** | +| 9 | Token requests bypassing a client throttle | `backend/common/reliability/client-side-rate-limit-pacing` | **new** | +| 4+5 | POSIX `set --` scope; backticks in a double-quoted payload | `platforms/shells/portable-shell-scripts` | merge | +| 6 | Gate-blocked command reads as a silent success | `platforms/shells/command-text-inspected-before-execution` | merge | + +**No new categories were created.** Each new page landed in an existing category, and +in the two places a new category was tempting the closest fit was taken instead: + +- Insight 1 could have opened `databases/migrations/`, but `schema-design/` already + holds `online-schema-changes`, which is migration machinery. A category holding one + page next to its sibling adds a routing hop for no discrimination. +- Insights 2, 3 and 7 are agent/orchestration concerns that could have opened + `platforms/orchestration/`. `platforms/processes/` already covers "keeping processes + alive as services" and "invoking prompt-capable CLIs non-interactively"; driving and + observing another process is the same concern, so all three went there. + +Root `INDEX.md` route lines were extended for all four touched domains (databases, +backend, testing, platforms); `log.md` gained one `ingest` and one `revise` entry. + +**Reviewer's attention is best spent on:** the `field-tested` rating on +`client-side-rate-limit-pacing` (§9 — the provider-specific quota is unsourced by +design), and on whether insights 2, 3 and 7 belong under `platforms/processes/` or +justify an `orchestration` category once more agent-harness pages accumulate. diff --git a/INDEX.md b/INDEX.md index d6d4238..2760c97 100644 --- a/INDEX.md +++ b/INDEX.md @@ -9,15 +9,15 @@ follow the cross-pointers in their index or take the next matching seeded domain | Domain | Status | Route here when | |--------|--------|-----------------| -| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior | -| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | +| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior, verifying that a hand-rolled additive migration reaches an already-deployed database | +| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references, pacing calls under a provider rate limit) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting) | -| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | +| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests, testing code that bulk-deletes resources from a machine-wide daemon (release-process quality → qa) | | [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) | | [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) | | [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | -| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning | +| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, driving a TUI in a tmux pane, agent-harness hooks that replace a tool's result, toolchain version pinning | | [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time | All ten domains are seeded. New categories grow via `skills/wiki-ingest/SKILL.md`. diff --git a/log.md b/log.md index 1c6293b..a2ce152 100644 --- a/log.md +++ b/log.md @@ -37,3 +37,5 @@ Append-only. Format: `## [YYYY-MM-DD] ` + for each new column, then call `init_db()`, then assert. Put the restore in a + `finally` block so a failed assertion does not leave the database half-migrated + for the rest of the suite. + +4. **Assert against the catalog, not the ORM.** Query `information_schema.columns` + for `data_type`, `is_nullable`, and `column_default` per column. The model object + reports what you declared; only the catalog reports what the database has. + +5. **Assert what pre-existing rows read back.** Insert a row before the drop, and + after `init_db()` assert its value for each new column. Postgres fills existing + rows from the `DEFAULT` in the `ADD COLUMN` clause — "the default value will be + returned the next time the row is accessed" — so a constant default is instant + and correct. An `ADD COLUMN` written without a `DEFAULT` leaves those rows NULL, + which is the defect this assertion exists to catch. + +## Edge cases + +| Case | Then | +|------|------| +| The new column is `NOT NULL` | `ADD COLUMN ... NOT NULL` against a non-empty table needs the `DEFAULT` in the same statement, or it fails outright — assert `is_nullable='NO'` *and* the surviving row's value | +| The default is volatile (`clock_timestamp()`), generated, or identity | Each existing row is updated at `ALTER` time — a rewrite under `ACCESS EXCLUSIVE`; size it as a maintenance operation ([databases-schema-design-online-schema-changes]) | +| Tests run on SQLite while production is Postgres | `information_schema` does not exist and `ADD COLUMN` default semantics differ — run this test against the production engine or it proves nothing about the deploy | +| The suite shares one database across tests | The drop/restore window is visible to anything running concurrently — serialize this test or give it its own schema ([testing-data-test-data-and-isolation]) | +| CI always starts from an empty database | Both paths ship, so keep a fresh-create assertion too; the drop-and-restore test covers only the upgrade path | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Call `init_db()` twice and call that the migration test | Drop the new columns, re-run `init_db()`, assert catalog + existing rows | Two calls against an already-correct schema exercise only the no-op path | +| Declare `server_default=` on the model and expect deployed tables to pick it up | Put the `DEFAULT` in the `ALTER TABLE ADD COLUMN` statement and assert an old row's value | `server_default` shapes CREATE TABLE only; SQLAlchemy states ALTER "is outside of the scope of SQLAlchemy itself" | +| Assert the new column exists by selecting it through the ORM | Query `information_schema.columns` for type, nullability, and default | A successful SELECT proves presence, not type/nullability/default — the parts a hand-written ALTER gets wrong | +| Leave the dropped columns in place when an assertion fails | Restore in `finally` | A failed run otherwise poisons every later test against that database | + +## Sources + +- https://docs.sqlalchemy.org/en/20/core/metadata.html — `create_all()` "will issue queries that first check for the existence of each individual table, and if not found will issue the CREATE statements"; altering constructs "via the ALTER statement … is outside of the scope of SQLAlchemy itself" (Alembic is the recommended tool) +- https://docs.sqlalchemy.org/en/20/core/defaults.html — `Column.server_default` "gets placed in the CREATE TABLE statement during a `Table.create()` operation"; `Column.default` is applied client-side at INSERT. Neither backfills existing rows +- https://www.postgresql.org/docs/current/ddl-alter.html — "Adding a column with a constant default value does not require each row of the table to be updated … Instead, the default value will be returned the next time the row is accessed"; volatile defaults update every row at `ALTER TABLE` time +- Field context: a 2026-08 schema task added six columns; the drop-and-restore test re-checked `data_type` / `is_nullable='NO'` / `column_default` per column from `information_schema` and asserted a pre-existing row read back its defaults, in a 397-test suite diff --git a/wiki/platforms/index.md b/wiki/platforms/index.md index b97fa71..d434095 100644 --- a/wiki/platforms/index.md +++ b/wiki/platforms/index.md @@ -15,8 +15,8 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [portable-shell-scripts](shells/portable-shell-scripts.md) | Writing a shell script that must run on more than one machine/OS/shell or in CI; a script that works locally fails elsewhere; choosing a shebang (bash vs sh); a bash script misbehaves in zsh or vice versa (unquoted vars, `=word`, array indexing); deciding how `set -euo pipefail` protects (and doesn't); building argument lists safely | -| [command-text-inspected-before-execution](shells/command-text-inspected-before-execution.md) | A hook, policy gate, allow-list, or audit rule blocked a command that is correct as written; composing a command that must satisfy such a gate first try; deciding whether to write a path literally or as `"$VAR"` in an inspected argument; a gate reports an argument missing or a file nonexistent though both are right; a gate must read a file your command creates; prose containing a dangerous-looking command (release notes, docs, fixtures) trips a text scanner | +| [portable-shell-scripts](shells/portable-shell-scripts.md) | Writing a shell script that must run on more than one machine/OS/shell or in CI; a script that works locally fails elsewhere; choosing a shebang (bash vs sh); a bash script misbehaves in zsh or vice versa (unquoted vars, `=word`, array indexing); deciding how `set -euo pipefail` protects (and doesn't); building argument lists safely; reordering `"$@"` in POSIX sh to accept position-independent flags; a parsed flag still showing up among the operands; choosing single vs double quotes for a text payload that quotes a command; passing an argument that begins with `-` | +| [command-text-inspected-before-execution](shells/command-text-inspected-before-execution.md) | A hook, policy gate, allow-list, or audit rule blocked a command that is correct as written; composing a command that must satisfy such a gate first try; deciding whether to write a path literally or as `"$VAR"` in an inspected argument; a gate reports an argument missing or a file nonexistent though both are right; a gate must read a file your command creates; prose containing a dangerous-looking command (release notes, docs, fixtures) trips a text scanner; a gated side-effect command produced no output and you must tell a block from a silent success; a gate blocked a script the harness itself prescribed, leaving a consumer waiting on a signal that was never emitted | ## tools @@ -45,6 +45,8 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [background-services](processes/background-services.md) | Something must run persistently or on a schedule on a dev machine or server (daemon, watcher, cron-style job); a "started" process dies when the terminal/SSH/agent session ends; choosing nohup vs LaunchAgent vs systemd unit vs cron/timer; a job works in the terminal but fails under cron/launchd (minimal environment); wiring service logs and restart policy | | [non-interactive-cli-invocation](processes/non-interactive-cli-invocation.md) | Calling a tool that can prompt (agent CLI, ssh, git, package manager) from a script, CI step, hook, or agent session, including with its own `-p`/`--print`/`--yes` flag; such a call produced no output and never returned; deciding whether a hang belongs to the client, the network, or the far-side service; choosing the stdin/timeout/fail-fast switches for an unattended call; a TTY-detecting tool changes its output format under automation | +| [driving-a-tui-in-a-tmux-pane](processes/driving-a-tui-in-a-tmux-pane.md) | Sending prompts or keystrokes into a long-lived interactive program in a tmux pane (`send-keys`) and needing to know it was consumed; a `capture-pane` before/after diff as delivery evidence; a payload that begins with `-` or comes from a variable; choosing between polling pane text and an out-of-band status artifact | +| [harness-tool-result-interception](processes/harness-tool-result-interception.md) | A plugin or hook in your agent harness returned substitute content for a built-in tool (truncated read, redaction, a note telling you to call something else); deciding whether a short result is interception or a genuinely small/empty file; the hook's suggested workaround failed too; briefing spawned worker sessions about a known-degraded tool | ## toolchains diff --git a/wiki/platforms/processes/driving-a-tui-in-a-tmux-pane.md b/wiki/platforms/processes/driving-a-tui-in-a-tmux-pane.md new file mode 100644 index 0000000..b053088 --- /dev/null +++ b/wiki/platforms/processes/driving-a-tui-in-a-tmux-pane.md @@ -0,0 +1,76 @@ +--- +id: platforms-processes-driving-a-tui-in-a-tmux-pane +domain: platforms +category: processes +applies_to: [macos, linux, tmux] +confidence: verified +sources: + - https://man.openbsd.org/tmux.1 + - https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html + - https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap11.html +last_verified: 2026-08-05 +related: [platforms-processes-non-interactive-cli-invocation, platforms-processes-background-services, platforms-shells-portable-shell-scripts] +--- + +# Sending Input to a TUI Running in a tmux Pane + +## When this applies + +A script, orchestrator, or agent sends text or keystrokes into a long-lived +interactive program running in a tmux pane (`tmux send-keys`) and must know +whether the program actually consumed them. Also when the payload is arbitrary +text from a variable rather than a fixed literal. + +## Do this + +1. **Pass the payload after a `--` separator**: `tmux send-keys -t "$pane" -l -- "$text"`. + tmux parses its own arguments getopt-style, so a payload beginning with `-` is + read as a flag and the whole command is rejected — quoting does not help, + because the problem is tmux's argument parsing, not shell word splitting. + POSIX reserves `--` as "the first argument that … delimit[s] the end of the + options"; `send-keys` honours it even though its man page does not list it. + +2. **Send the payload and the newline as separate calls**: `-l` "disables key name + lookup and processes the keys as literal UTF-8 characters", so a trailing + newline in the payload is not a key press. Follow with + `tmux send-keys -t "$pane" Enter`. + +3. **Confirm delivery by the target's own state, never by pane content changing.** + Run the checks in this order and stop at the first that matches: + +| Check | What it proves | +|-------|----------------| +| The program's own busy/queued indicator in the last N non-empty lines of `capture-pane` | The program has the text but has not consumed it — treat as **not yet delivered** and wait | +| An effect only the program can produce (its output line, a status file it writes, a marker it prints) | Consumed | +| `capture-pane` output differs from before the send | **Nothing.** The tty line discipline echoes typed characters back to the pane while the foreground process is busy, so the pane changes for input that was never read | + +4. **Make anything the caller must act on out-of-band.** Have the target write a + status file and poll that file, rather than parsing the pane. Pane text is a + rendering — it repaints, scrolls, and wraps. + +5. **Treat `send-keys` exit 0 as "tmux accepted the keys"**, not as "the program + read them". The two are separated by the pty buffer. + +## Edge cases + +| Case | Then | +|------|------| +| The pane repainted between send and capture | The echoed characters are gone, so absence of the echo is not evidence of consumption either — fall back to the program's own effect (step 3, row 2) | +| The payload contains a literal newline and must arrive as one paste | Use `tmux load-buffer -` + `paste-buffer -t "$pane"`; `send-keys -l` delivers the newline as a character, which many TUIs treat as submit | +| Delivery must be confirmed but the program has no busy indicator and no artifact | Add one: have the wrapper echo a unique marker after processing, and search for that marker rather than for the prompt text | +| The pane's process has exited (shell prompt only) | The keys land on the shell and run as commands — check `#{pane_dead}` / the pane's current command before sending | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Diff `capture-pane` before/after and call a difference "delivered" | Check the program's busy indicator first, then an effect only it can produce | The tty echoes keystrokes while the process is busy, so the diff reports success for exactly the queued case the check was written to catch | +| Interpolate a variable straight into `send-keys -l "$text"` | Add the `--` separator before the payload | A payload starting with `-` is parsed as a tmux flag and the send fails with exit 1 | +| Treat `send-keys` exit 0 as proof the prompt was answered | Poll a status artifact the target writes | Exit 0 means the keys reached the pty, which is upstream of the program reading them | + +## Sources + +- https://man.openbsd.org/tmux.1 — `send-keys [-FHKlMRX] … [key ...]`; "The `-l` flag disables key name lookup and processes the keys as literal UTF-8 characters" +- https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html — Utility Syntax Guideline 10: `--` delimits the end of options, after which arguments are operands +- https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap11.html — terminal `ECHO` in canonical mode echoes input characters back to the terminal, independently of whether the reading process has consumed them +- Field context (tmux 3.7b, macOS, 2026-08-05): `tmux send-keys -t S -l "-n hello"` → `command send-keys: unknown flag -n`, exit 1; the same call with `-- "-n hello"` → exit 0. Sending `echo SECOND_PROMPT` to a pane running `sleep 6` changed the pane content (a naive diff reads "delivered") while the command's own output count stayed 0, becoming 1 only after the sleep drained diff --git a/wiki/platforms/processes/harness-tool-result-interception.md b/wiki/platforms/processes/harness-tool-result-interception.md new file mode 100644 index 0000000..cd0a17d --- /dev/null +++ b/wiki/platforms/processes/harness-tool-result-interception.md @@ -0,0 +1,68 @@ +--- +id: platforms-processes-harness-tool-result-interception +domain: platforms +category: processes +applies_to: [general] +confidence: verified +sources: + - https://code.claude.com/docs/en/hooks +last_verified: 2026-08-05 +related: [platforms-shells-command-text-inspected-before-execution, platforms-processes-non-interactive-cli-invocation, debugging-methodology-hypothesis-testing] +--- + +# A Harness Hook Replaced a Tool's Result with a Substitute + +## When this applies + +You are running inside an agent harness where a plugin or hook wraps a built-in +tool (file read, search, shell) and hands back substitute content — a truncated +result, a redaction, or a note suggesting you call something else instead. Also +when briefing worker sessions you spawn against the same repo. + +## Do this + +1. **Recognise the shape.** A `PostToolUse` hook's `updatedToolOutput` "replaces + the tool's result", and the docs name transformation of "inbound tool results" + as an intended use. So a tool returning one line, an empty body, or prose about + how to call it differently is a plausible harness substitution, not necessarily + the file's real content. + +2. **Separate substitution from a genuinely empty target first**, with a tool the + hook does not wrap: `wc -l ` and `ls -l `. A one-line result from a + 400-line file is interception; from a one-line file it is the truth. + +3. **Test the hook's suggested remediation exactly once**, with the narrowest + concrete call it describes (the explicit `offset`/`limit`, the alternative + fetch). The remediation text is written by the hook author, not derived from + the failure, so it can be wrong. + +4. **On the second degraded result, switch tool families for that repo** instead + of trying more variants: locate with `grep -n '' `, then read the + range with `awk 'NR>=A && NR<=B' `. Two calls settle it; enumerating + parameter combinations does not. + +5. **Record the fallback once and put it in the brief of every session you spawn** + against that repo. Otherwise each worker rediscovers it independently and pays + the same three-to-four wasted calls. + +## Edge cases + +| Case | Then | +|------|------| +| The substitution is silent — no note, just short content | Compare the tool's byte/line count against `wc -c`/`wc -l` before treating the content as complete; a truncation with no message reads exactly like a small file | +| The wrapping hook also intercepts your shell fallback | Read through a different mechanism (an editor/Write-tool round trip, `base64` of a byte range) and escalate the harness configuration to the human — a harness that blocks every read path is a configuration fault, not a puzzle to route around | +| Only some paths are affected | The hook matcher is path- or tool-scoped; establish the boundary with one probe inside and one outside it, then scope the brief's instruction to the affected paths | +| The hook is doing intended redaction (secrets, PII) | Do not route around it — take the redacted result as the answer and request the value through the sanctioned channel | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Retry the tool with the parameters its own error text suggests, repeatedly | Try the suggestion once, then switch to `grep -n` + `awk` range reads | The remediation is authored by the hook, not diagnosed from your failure; repeated variants of a wrong suggestion cost calls without new information | +| Conclude the file is empty or the symbol is absent | Confirm the size with `wc -l` before concluding anything about content | An intercepted read and an empty file are indistinguishable from the result alone | +| Let every spawned worker discover the workaround itself | State the fallback in the spawn brief | The cost is paid once per session instead of once per agent | + +## Sources + +- https://code.claude.com/docs/en/hooks — `PostToolUse` decision control: `updatedToolOutput` "replaces the tool's result"; "For redaction or transformation use cases, intercept at `PreToolUse` for outbound tool inputs and `PostToolUse` for inbound tool results" +- Field context (2026-08-05): in a repo with a session-memory plugin installed, `Read(path, offset=151, limit=120)` returned only line 1, as did plain reads of three sibling files; the plugin's own note suggested the `offset`/`limit` retry that had just failed. Two independently orchestrated worker sessions each logged the same discovery and each fell back to shell reads before the fallback was added to the spawn brief diff --git a/wiki/platforms/shells/command-text-inspected-before-execution.md b/wiki/platforms/shells/command-text-inspected-before-execution.md index e69daff..93c95b9 100644 --- a/wiki/platforms/shells/command-text-inspected-before-execution.md +++ b/wiki/platforms/shells/command-text-inspected-before-execution.md @@ -7,7 +7,7 @@ confidence: verified sources: - https://code.claude.com/docs/en/hooks - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html -last_verified: 2026-07-30 +last_verified: 2026-08-05 related: [platforms-shells-portable-shell-scripts, platforms-environment-path-resolution] --- @@ -58,11 +58,26 @@ on the first attempt. your exact command string before rewriting anything else — one run tells you whether you are in the missing-argument or nonexistent-file mode above. +6. **Verify the artifact, never the silence.** A blocked command emits nothing on + stdout, which is byte-identical to a command that ran and printed nothing. When + the command's whole purpose is a side effect (writing a status file, touching a + marker, sending a signal), `ls`/`stat` that artifact before reporting the step + done. A gate that matches on a path blocks on the path text regardless of the + command's purpose — including the harness's own scripts, invoked exactly as the + harness documents them. + +7. **Hand a blocked signal back rather than routing around it.** When the gate + escalates, report the signal as *un-emitted* to whoever is waiting on it. A + consumer polling for a file that will never appear waits forever, and an + improvised workaround defeats a control the human put there deliberately. + ## Edge cases | Case | Then | |------|------| | Blocking feedback appears without the gate's message | Exit code 2 sends the reason to **stderr**, not stdout; read stderr for the actual cause | +| A path-scoped gate blocks a script the harness itself told you to run | The gate matched the absolute path in the command text, not the script's role — the two are indistinguishable to a text rule. Report it to the human who owns the gate; do not rewrite the path to evade the match | +| The same target is writable through a file-writing tool but not through the shell | The block is command-text-scoped, not a filesystem permission. That asymmetry is the diagnostic: use it to confirm the gate rather than to bypass it | | The gate matches an intended-as-prose mention of a dangerous command (in a commit message, doc, or test fixture) | Move the text into a file and pass it by path (step 4) rather than reshaping the sentence | | Path contains a space, so quoting is unavoidable | Relocate or symlink the target to a space-free path for gated commands; a gate that excludes quote characters cannot receive a quoted path at all | | The gate needs `~` expanded | Write the absolute path; a gate that resolves `~` itself is doing so on the literal tilde, which only works if it implements the expansion | @@ -76,6 +91,8 @@ on the first attempt. | Assume a blocked command means the deliverable is wrong | Reproduce the gate's extraction pattern against your literal command string first | A quoting-level extraction failure and a genuinely incomplete deliverable produce the same refusal, so fixing content wastes the round | | Build the file the gate checks with a heredoc in the same command | Write it in a prior command and reference the path | The gate is evaluated before execution, so the file is absent at decision time | | Reword prose to get a dangerous-looking string past a scanner | Put the prose in a file and pass `--notes-file`/`--body-file` | Editing meaning to satisfy a text scanner degrades the artifact; a file is not scanned as a command | +| Treat a side-effect command's empty output as success | `ls`/`stat` the artifact it should have produced | A blocked command and a silent successful one produce identical stdout; only the artifact distinguishes them | +| Retry a gated status-emitting command with a reshaped path | Report the signal as un-emitted to its consumer | The consumer polls forever on a wrong assumption, and evading the gate removes a control the human installed | ## Sources @@ -92,3 +109,11 @@ missing), `--body-file $REPO/…` → literal `$REPO/…` (blocked as nonexisten `--body-file /abs/…` and `--body-file=/abs/…` extracted correctly. A same-command heredoc body-file was separately blocked as not-yet-existing until moved to a preceding call. + +On 2026-08-05 a worker session inside a git worktree ran the orchestrator's own +`status-update.sh` with the status directory pointing at the main checkout. A +`worktree_escape` guardrail matched the absolute main-checkout path and returned +only an escalation notice; `ls -la` on the status directory then showed it empty, +so the orchestrator would have polled indefinitely for a file that was never +written. A Write-tool call to the same tree succeeded, confirming the block was +scoped to the Bash command text rather than to filesystem permissions. diff --git a/wiki/platforms/shells/portable-shell-scripts.md b/wiki/platforms/shells/portable-shell-scripts.md index 23eed9b..edff73a 100644 --- a/wiki/platforms/shells/portable-shell-scripts.md +++ b/wiki/platforms/shells/portable-shell-scripts.md @@ -10,7 +10,7 @@ sources: - https://zsh.sourceforge.io/Doc/Release/Parameters.html - https://google.github.io/styleguide/shellguide.html - https://www.shellcheck.net/ -last_verified: 2026-07-10 +last_verified: 2026-08-05 related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management, platforms-shells-command-text-inspected-before-execution] --- @@ -51,7 +51,25 @@ non-interactive environment). | Array indexing | 0-based | 1-based (unless `KSH_ARRAYS`) | Iterate with `"${arr[@]}"`; when a numeric index is unavoidable, gate the script to one shell via the shebang | 5. Build argument lists as arrays and expand quoted: `args=(-o "$out"); cmd "${args[@]}"`. -6. Run `shellcheck` on every script before it ships or gates anything; it flags + In POSIX sh there are no arrays — reorder the positional parameters in place, and + write that loop **inline at the top level of the dispatcher**, never inside a + helper function: + + ```sh + n=$#; while [ "$n" -gt 0 ]; do a="$1"; shift + case "$a" in --dry-run) DRY=1 ;; *) set -- "$@" "$a" ;; esac + n=$((n-1)); done + ``` + + POSIX restores a function's caller positional parameters on return, so a helper's + `set --` is discarded and the caller's `"$@"` still carries the flag as an operand. +6. Choose the quoting by what the text is, not by habit. Inside double quotes the + backquote "shall retain its special meaning introducing … command substitution" + and `$` still introduces expansion, so text quoting a command runs it. Wrap + literal text — prose, error messages, anything containing a command example — in + **single** quotes, and pass long bodies via a file (`--body-file`) rather than as + an argument. +7. Run `shellcheck` on every script before it ships or gates anything; it flags unquoted expansions, bashisms under `#!/bin/sh`, and `set -e` blind spots. ## Edge cases @@ -62,6 +80,9 @@ non-interactive environment). | Critical command is in a pipeline but the interpreter is POSIX sh (no `pipefail`) | Run the critical command outside the pipeline (temp file between stages) and test `$?` directly | | Script runs via cron/CI/hooks and commands are "not found" | Non-interactive shells load no rc files — no user PATH, no version-manager shims. Call binaries by absolute path (see platforms-toolchains-version-management) | | `set -u` breaks on optional variables | Expand with an explicit default: `"${OPT:-}"` | +| A flag is parsed correctly but still appears among the operands | The reordering loop is inside a function. Flags set globals (which survive), operands are set positionally (which do not) — so detection works and the argument list stays wrong, with no error. Move the loop inline (step 5) | +| A payload argument begins with `-` | Pass it after a `--` separator (`cmd -- "$text"`); quoting does not help, because the option parser, not the shell, is what claims it | +| Message text must contain a command example | Single-quote the whole argument, or write the text to a file and pass the path — a double-quoted backtick executes and the message ships with the output spliced in | ## Instead of @@ -70,10 +91,16 @@ non-interactive environment). | Build a command string and `eval` it | Build an array and expand it: `cmd "${args[@]}"` | `eval` re-parses quotes and globs; arrays pass arguments through exactly | | Put a command plus its flags in one variable and run `$cmd` | Variable holds the binary path only; flags are separate words | zsh runs the whole value as one command name; bash re-splits and re-globs it | | Trust a trailing `echo "done"` as proof a step ran | Verify the produced state with an independent command | Inside `&&` chains and subshells, `set -e` misses failures and the echo still prints | +| Wrap POSIX-sh flag parsing in a `parse_flags "$@"` helper | Keep the `set --` reordering loop inline in the dispatcher | Positional parameters are restored when the function returns, so the caller runs with the original, unfiltered arguments | +| Accumulate POSIX-sh operands into a string to work around the missing array | Reorder `"$@"` in place with `set -- "$@" "$a"` | A string re-splits on whitespace, so a path containing a space becomes two operands | +| Double-quote a message that quotes a command | Single-quote it, or pass it with `--body-file` | `"…\`cmd\`…"` runs `cmd`, substitutes its output, and can exit 0 with the message silently gutted | ## Sources - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html — POSIX shell quoting and field splitting (sections 2.2, 2.6.5) +- https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html — 2.2.3 Double-Quotes: the backquote "shall retain its special meaning introducing the other form of command substitution" and `` "shall retain its special meaning introducing parameter expansion … command substitution … and arithmetic expansion"; 2.9.5 Function Definition Command: on return "the value of the special parameter `#` and the positional parameters shall be restored to the values they had before the function was executed" +- https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html — Utility Syntax Guideline 10: `--` delimits the end of options +- Reproduced 2026-08-05 under `/bin/sh` and `dash`: `parse_flags "$@"` left the caller with `argc=4` still containing `--dry-run`, while the identical loop inline gave `argc=3` with the space-bearing operand intact — `DRY=1` in both, so the flag was detected either way and only the operand list was wrong. In zsh, `echo "run: \`pip install -e .[dev]\` first"` printed `run: first` after a `no matches found: .[dev]` error and still exited 0; the single-quoted form printed the text verbatim - https://zsh.sourceforge.io/Doc/Release/Expansion.html — zsh: no word splitting of unquoted parameters (14.3); `=word` expansion (14.7.3) - https://zsh.sourceforge.io/Doc/Release/Parameters.html — zsh arrays numbered from 1 (KSH_ARRAYS excepted) - https://google.github.io/styleguide/shellguide.html — quote variables, prefer bash for scripts, arrays over eval diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 2cbad76..81e200a 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -38,6 +38,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| | [what-to-mock](mocking/what-to-mock.md) | Deciding whether to mock/stub/fake a dependency or use the real one; mocks breaking on refactors; testing handling of a third-party's failure modes; the same mock setup is copy-pasted across tests | +| [destructive-operations-on-shared-daemons](mocking/destructive-operations-on-shared-daemons.md) | The code under test enumerates and deletes a machine-wide daemon's resources by name/pattern (tmux sessions, docker containers, systemd units, namespaces) and that daemon runs on the test machine; proving a sweep deletes the targets and spares bystanders; keeping a scope bug from destroying the dev environment instead of failing the test; giving a shell script a substitution seam for the tool it shells out to | ## flaky diff --git a/wiki/testing/mocking/destructive-operations-on-shared-daemons.md b/wiki/testing/mocking/destructive-operations-on-shared-daemons.md new file mode 100644 index 0000000..0acc0ec --- /dev/null +++ b/wiki/testing/mocking/destructive-operations-on-shared-daemons.md @@ -0,0 +1,80 @@ +--- +id: testing-mocking-destructive-operations-on-shared-daemons +domain: testing +category: mocking +applies_to: [general] +confidence: verified +sources: + - https://martinfowler.com/articles/mocksArentStubs.html + - https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap08.html +last_verified: 2026-08-05 +related: [testing-mocking-what-to-mock, testing-data-test-data-and-isolation, testing-quality-tests-that-cannot-fail, platforms-environment-path-resolution] +--- + +# Testing Code That Bulk-Deletes Resources from a Machine-Wide Daemon + +## When this applies + +The code under test enumerates a shared daemon's resources by name or pattern +(tmux sessions, docker containers, systemd units, k8s namespaces, cron entries) +and deletes the matches — and that daemon is running on the machine executing the +test, holding the developer's own resources and often the test runner's session. + +## Do this + +1. **Never point this test at the real daemon.** The pattern-scope bug is exactly + what the test exists to catch, and its first manifestation deletes the + developer's live resources instead of producing a failure report. The failure + arrives as a destroyed environment, not as a red test. + +2. **Inject a recording fake ahead of the real binary on `PATH`** and run the + script with that `PATH`. The fake answers the enumeration call with fixture + names and appends every destructive call to a log file: + + ```sh + # fakebin/tmux + case "$1" in + list-sessions) printf '%s\n' "run-1" "run-2" "mydev" ;; + kill-session) shift; echo "KILL $2" >> "$FAKE_LOG" ;; + esac + ``` + +3. **Keep `PATH` usable as the seam.** The script under test must resolve the tool + at call time — a bare `tmux`, or `TOOL="$(command -v tmux)"` evaluated inside the + function. A hardcoded absolute path removes the seam; when the path must be + absolute, read it from one env var the test overrides + ([platforms-environment-path-resolution]). + +4. **Assert both directions.** The log contains exactly the intended targets, *and* + contains none of the bystanders. Put a realistic bystander in the fixture list — + a name shaped like the developer's own sessions — because the scope bug shows up + as an extra line, not a missing one. + +5. **Include a deletes-nothing case.** Run the sweep with a pattern that matches no + fixture and assert the log is empty. Without it, a fake that silently fails to + record makes every "no bystanders" assertion vacuous + ([testing-quality-tests-that-cannot-fail]). + +## Edge cases + +| Case | Then | +|------|------| +| The script calls the daemon for setup as well as deletion | The fake implements the read subcommands as fixtures too, so one binary serves both; keep the fixture list in the test, not in the fake | +| A test genuinely needs the real daemon (integration coverage of the daemon's own behaviour) | Give its resources a dedicated prefix the sweep pattern provably cannot match, and assert that non-match in the test itself before the destructive call runs | +| CI has no daemon installed at all | The fake is the only way this code is covered there — keep the fake test at unit level so coverage does not depend on the runner's environment | +| The script builds the deletion list and the deletion in one pipeline | Split enumeration from deletion into two functions so the scope decision is testable without any deletion call at all | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Run the sweep against the real daemon with "harmless" test resources | Inject a recording fake on `PATH` and assert its log | The bug under test is the pattern matching too much, so its first occurrence takes the resources you did not list — including the runner's own session | +| Assert only that the intended targets were deleted | Assert the bystanders' absence from the same log | Over-deletion is invisible to a test that only checks the targets are gone | +| Guard the real run with a dry-run flag and eyeball the output | Make the fake's log the assertion subject | A human reading output is not a gate; a diffed log is, and it runs on every commit | +| Trust an empty kill log as proof of correct scoping | Add a positive case that produces a non-empty log with the same fake | An empty log is also what a broken fake produces | + +## Sources + +- https://martinfowler.com/articles/mocksArentStubs.html — fakes as working implementations with a shortcut; recording interactions to verify outbound commands rather than resulting state +- https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap08.html — `PATH` is searched left to right for a command name containing no slash, which is what makes a prepended directory a substitution seam +- Field context (2026-08-05, verified by reproduction): with a fake `tmux` prepended to `PATH`, a prefix sweep over the fixture list `run-1, run-2, mydev` logged exactly `KILL run-1` / `KILL run-2`, zero bystander lines, and an empty log for a non-matching prefix — while the machine's eight real tmux sessions, three of which matched the pattern under test, were untouched