From 88f564ed4fd85323e7075d8eafece8c48614e05b Mon Sep 17 00:00:00 2001 From: dch0202 Date: Tue, 4 Aug 2026 10:24:54 +0900 Subject: [PATCH] knowledge: ingest 2 verified platforms insights (keg-only PATH, double-quote backslash) + defer 1 (MLIR IRDL) --- .dev-loop/INGEST_REPORT.md | 180 ++++++++++-------- log.md | 3 + wiki/platforms/environment/path-resolution.md | 6 +- .../shells/portable-shell-scripts.md | 6 +- 4 files changed, 110 insertions(+), 85 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index c375750..9de1a41 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,95 +1,109 @@ -# Consolidated review — knowledge PRs #6–#13 +# Knowledge flush — 3 insight(s): 2 ingested, 1 held back -Eight fork PRs (`dch0202-rsquare`, 2026-07-28 → 2026-08-02) were reviewed together -against `AGENTS.md`. Each PR was audited by an independent reviewer (format rules, -sources, vague-qualifier ban, ≤120 body lines, index/log invariants), then -cross-compared to catch duplication the per-PR flushes could not see — they branched -independently off the same main and rewrote the same shared index/log files. Fork -branches can't be edited from here and several PRs needed content changes (drop a -duplicate, merge a colliding page), so this branch carries the reconciled end-state -rather than merging each PR as-is (which would import the duplicates). +Drained 3 pending candidates from `~/.dev-loop/queue`. Two were verified and +merged into existing `platforms` pages; one (MLIR IRDL) is verified-but-niche and +held back from this general wiki with a recommendation (see Routing decision). ## Verified best-practice -Sources are per-page and were live-verified in each originating PR's flush; the -independent re-reviews re-checked them. Landed pages and their evidence base: +### 1. Homebrew keg-only formulae are installed but off `PATH` → `verified` +- **Claim:** On macOS, `which ` / `command -v ` reporting not-found + does **not** mean the tool is absent. Homebrew keg-only formulae (llvm, curl, + openjdk, node@N, libpq, ruby) are installed into the Cellar but deliberately not + symlinked onto `PATH`. Check the package manager's record before concluding + absence. +- **Sources checked:** https://docs.brew.sh/FAQ ("What does keg-only mean?" — + "installed only into the Cellar and is not linked into the default prefix"); + `brew info llvm` output ("llvm is keg-only, which means it was not symlinked + into /opt/homebrew"). +- **How verified (reproduced on this machine, 2026-08-04):** `which mlir-opt` → + "mlir-opt not found" (exit 1) and `command -v mlir-opt` → not found, **while** + `brew list --versions llvm` → `llvm 22.1.8` and + `/opt/homebrew/opt/llvm/bin/mlir-opt --version` → "Homebrew LLVM version + 22.1.8". This is the exact incident from the queue (issue #7 deferred on a stale + `which` check though LLVM 22.1.8 was installed). +- **Confidence: verified** (official doc + local reproduction). -| Page | Confidence | Source basis | -|------|-----------|--------------| -| backend/common/llm/completion-response-validation | verified | OpenAI reasoning guide + chat `object` spec (5 `finish_reason` values), vLLM/LiteLLM reasoning fields; field incident (200/`length`/empty content/8,173-char reasoning) | -| backend/common/llm/context-window-budget | verified | Claude context-window docs, LiteLLM exception mapping, vLLM/Claude Code env-var docs | -| backend/common/integrations/externally-owned-defaults | verified | OpenAI deprecations (notice windows) + models `list`, LiteLLM model_discovery; field incident (alias removed between PR verify and review → 400) | -| backend/common/storage/object-key-persistence | verified | AWS S3 CompleteMultipartUpload + managed-upload API/source, aws-sdk-js issues #1158/#5656 | -| infrastructure/containers/host-cgroup-visibility | field-tested | cgroup_namespaces(7), Docker `--cgroupns=host`, nsenter, k8s #103363; OrbStack repro | -| infrastructure/observability/missing-container-metrics | verified/field-tested | k8s resource-metrics-pipeline docs, kube-prometheus-stack values, kubernetes-mixin; OrbStack #2217 repro | -| platforms/environment/unicode-text-matching | verified | UAX #15, Unicode core §3.12, APFS FAQ, POSIX grep; local repro (macOS 15/APFS, grep 2.6.0-FreeBSD, Python 3.13) | -| platforms/shells/command-text-inspected-before-execution | verified | Claude Code hooks docs, POSIX shell §2.6; local reproduction | -| platforms/processes/non-interactive-cli-invocation | verified | GNU nohup, OpenBSD ssh/ssh_config, git, timeout man pages; no-request-in-gateway-log field incident | -| qa/document-verification/spec-document-gates | field-tested | ESLint, Google mutation testing, RFC 2119, Vale, markdownlint; 32/32 mutant / 62/62 intact RFC sessions | -| qa/document-verification/editing-a-gated-document | field-tested | pgrep, Vale, markdownlint; in-house editing methodology | -| testing/quality/checks-that-cannot-pass | verified | James Shore AoAD2, POSIX grep exit status, Semgrep rule-testing, pytest exit codes; BSD/ugrep measurement | -| testing/quality/spec-artifact-checks | verified | JSON Schema, ESLint RuleTester, pitest, GFM table spec; local cell-count repro + GitHub renderer cross-check | -| testing/quality/harness-reverse-controls | verified | mutation-testing + CI-control sources; field repro (re-fetched all cited URLs, PASS) | +### 2. Bash double-quote strips the backslash only before five chars → `verified` +- **Claim:** Inside a double-quoted bash string, the backslash retains special + meaning only before `$`, backtick, `"`, `\`, or newline. So a regex embedded in + a double-quoted string has `"\$"` collapse to a bare `$` (a regex end-of-line + anchor) before the tool ever sees it, while `"\d"` keeps its backslash. Miscopy + the pattern as `$` or `\\$` and the EOL match silently changes. +- **Sources checked:** + https://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html — the + backslash "retains its special meaning only when followed by one of" `$`, + backtick, `"`, `\`, or newline (page fetched and the sentence confirmed present + 2026-08-04). +- **How verified (reproduced 2026-08-04):** `od -c` on the guardrails-style + pattern `"(sh|bash)([[:space:]]|-|<|\$)"` shows the byte reaching grep is a bare + `$` (backslash stripped); `echo 'curl http://x | sh' | grep -E "$pat"` matches + `sh` at end-of-line, confirming `$` acts as the EOL anchor. Matches the queue + incident (why `curl … | sh` with no trailing char matches the bash-guard rule). +- **Confidence: verified** (official doc + local reproduction). -Three pages were reconciled from two overlapping PR versions each, keeping the more -complete/better-sourced body and folding in the other's unique cases: -- **completion-response-validation** — #12 body (all five `finish_reason` values, - `tool_calls`/`function_call` carve-out, streaming, Responses API, "reasoning is - scratch, not deliverable") kept in `llm/` (coherent with #6/#13); folded in #6's - DeepSeek first-party edge + the field incident. -- **externally-owned-defaults** — #12 generalized body (any repo-external resource) - in `integrations/`; folded in #6's alias-removed field incident + the - gateway-config-vs-live-upstream nuance. -- **non-interactive-cli-invocation** — #12 body (GNU-nohup extension precision, - ssh -n stdin-detach vs BatchMode, pre-log DNS/TLS/proxy + `curl -v`) kept; folded - in #11's DEBIAN_FRONTEND, pager/color TTY case, wrapper-CLI case, field incident. +### 3. MLIR IRDL region ops fail verification without a borrowed terminator → `field-tested` +- **Claim:** An MLIR op with a region defined declaratively via IRDL (LLVM 22) + fails block verification ("block with no terminator") because IRDL cannot + declare a terminator op or attach `NoTerminator`/set `RegionKind` on + user-defined ops. Either embed a borrowed terminator + (`omp.terminator`/`llvm.unreachable`) or model the nesting with flat marker ops + carrying a `children` id-list attribute. +- **Sources checked:** https://mlir.llvm.org/docs/LangRef/ (SSACFG regions require + a terminator; a single-block region may opt out only via `NoTerminator` **on the + enclosing op**); https://mlir.llvm.org/docs/Dialects/IRDL/ (`irdl.dialect` + itself carries `NoTerminator`, but the op-definition surface exposes no way to + attach that trait or set `RegionKind` on the ops you define). The mechanics are + doc-corroborated; the specific IRDL limitation is not stated as such in the docs. +- **How verified:** contributor measured it with `mlir-opt 22.1.8` (region op → + "block with no terminator"; only `omp.terminator`/`llvm.unreachable` verified + inside a generic IRDL region; flat marker ops round-tripped cleanly). Not + re-reproduced here (no dialect fixture on hand). +- **Confidence: field-tested** — measured in production, mechanism doc-backed, but + the IRDL-can't-set-it limitation has no official-doc statement. ## Existing-layer check -Cross-PR and against-main duplication was the focus. Findings and resolutions: - -- **spec-artifact-checks (#8) ≡ document-conformance-checks (#9)** — same case - (coverage-vs-validity split, per-check negative controls, GFM pipe parsing, - ESLint/Semgrep/mutation examples). #9's report predated awareness of #8. → - **#8 kept canonical; #9's page dropped, `testing/docs-as-spec` category not created.** -- **completion-response-validation (#6) ≈ llm-response-completeness (#12)** — ~95% - same case (HTTP 200 ≠ usable output; `length`/blank/reasoning-budget). → - **merged into one `llm/` page; #12's `integrations/` copy dropped.** -- **gateway-model-alias-defaults (#6) ≈ externally-owned-defaults (#12)** — ~80%; - #12 generalizes the model-alias case to any external resource. → - **kept the general `integrations/` page; #6's LLM-only page dropped.** -- **non-interactive-cli-invocation** — created by BOTH #11 and #12 (file collision). - → **single reconciled page.** -- Distinct (no overlap, all landed): checks-that-cannot-pass, harness-reverse-controls, - spec-document-gates, editing-a-gated-document, unicode-text-matching, - command-text-inspected-before-execution, object-key-persistence, context-window-budget, - host-cgroup-visibility, missing-container-metrics. -- Reciprocal `related:` links added on existing pages (tests-that-cannot-fail, - timeouts-and-retries, environment-config, release-gates, background-services, - portable-shell-scripts, timezone-and-locale, paths-case-and-line-endings, - acceptance-criteria, resource-limits-and-probes, logs-metrics-signals, - minimum-case-set). A dropped-page backlink (#6 → gateway-model-alias-defaults on - environment-config and release-gates) was retargeted to externally-owned-defaults. -- Invariants verified programmatically: all `related:`/inline `[id]` references - resolve, every page listed in its domain index, no duplicate ids, no page >120 - body lines. +- **Insight 1 → `wiki/platforms/environment/path-resolution.md` (merge, not new).** + Read the full page. Its "load when" line already owns "'command not found' + though the tool is installed"; it already had a brew-shadowing row and a + `which`→`type`/`command -v` Instead-of row. **Gap:** none of those cover the + keg-only case where even `command -v`/`type` correctly report not-found because + the binary is genuinely unlinked (absence-on-PATH ≠ not-installed). Merged one + `Edge cases` row + one `Instead of` row + one source; no duplication, no + conflict. Related links already point to `toolchains/version-management` and + `processes/background-services` (both relevant, left as-is). +- **Insight 3 → `wiki/platforms/shells/portable-shell-scripts.md` (merge, not new).** + Read the full page plus the adjacent + `shells/command-text-inspected-before-execution.md` (the candidate arose in a + guardrails PreToolUse hook, which that page owns). That page is about a *gate + reading command text*; this lesson is about the *shell mangling an embedded + regex* — a quoting-semantics fact, so it belongs on portable-shell-scripts (the + quoting page), which the two pages already cross-link via `related:`. The page's + "Do this #2" says "quote every expansion" but nowhere states the double-quote + backslash-stripping rule. Merged one `Edge cases` row + one `Instead of` row + + one source; no duplication, no conflict. +- **Insight 2 (MLIR IRDL):** no domain owns compiler/dialect internals. Closest + seeded categories (`platforms/toolchains` = version pinning; `infrastructure` + = CI/CD) genuinely do not cover "how MLIR's region verifier interacts with + IRDL-defined ops." No merge target exists. ## Routing decision -- `backend/common/llm/` (new) — LLM-specific server concerns: completion-response-validation, - context-window-budget. Coherent home shared by #6 and #13. -- `backend/common/integrations/` (new) — general repo-external-dependency concern: - externally-owned-defaults. Kept separate from `llm/` because its scope is any - external resource (bucket/queue/index), not LLM-only. -- `backend/common/storage/` (new) — object-key-persistence. -- `qa/document-verification/` (new) — spec-document-gates, editing-a-gated-document. - Introduced by both #10 and #11; unified into one index section. -- `testing/quality/` (existing) — checks-that-cannot-pass, spec-artifact-checks, - harness-reverse-controls (test/check-authoring discipline, distinct from - qa/document-verification which is release-process gate design). -- `platforms/{environment,shells,processes}/` (existing) — unicode-text-matching, - command-text-inspected-before-execution, non-interactive-cli-invocation. -- `infrastructure/{containers,observability}/` (existing) — host-cgroup-visibility, - missing-container-metrics. - -Source PRs #6–#13 are closed with a disposition comment crediting the author. +- **Insight 1 → `platforms/environment/path-resolution`** (merged). The harvested + `domain: infrastructure` hint was wrong — this is a PATH-resolution symptom, not + a CI/CD/build concern. No new category. +- **Insight 3 → `platforms/shells/portable-shell-scripts`** (merged). The harvested + `domain: security` hint was incidental (the bug surfaced in a guardrails hook); + the reusable lesson is bash double-quote semantics, owned by `platforms/shells`. + No new category. +- **Insight 2 → held back (no ingest).** Verified-as-field-tested and genuinely + correct, but it is single-repo compiler-internals with no home in the current 10 + general SWE domains. Ingesting it would mean creating a `compilers`/`mlir` domain + for one insight loadable by exactly one repo — which the wiki's own philosophy + (one case per page, cross-repo reusability) argues against. **Recommendation:** + keep it in the owning repo's own docs (e.g. a dialect NOTES file), or, if you + want the wiki to carry dialect-authoring knowledge, say so and I will open a + dedicated `compilers` domain in a follow-up. Logged as a `gap` entry in `log.md`. + All 3 candidates are retired from the active queue so auto-flush will not re-run + them; the MLIR row is preserved in `.processed.jsonl` (status `deferred-out-of-scope`). diff --git a/log.md b/log.md index 1c6293b..bf1de29 100644 --- a/log.md +++ b/log.md @@ -37,3 +37,6 @@ Append-only. Format: `## [YYYY-MM-DD] ` and `"$(brew --prefix )/bin/"` before concluding absence; invoke via the keg path or `brew link --force`. Doc-backed (docs.brew.sh/FAQ) + reproduced 2026-08-04 (`which mlir-opt`→not-found while llvm 22.1.8 installed at /opt/homebrew/opt/llvm/bin). last_verified bumped to 2026-08-04. +## [2026-08-04] revise | platforms/shells/portable-shell-scripts +1 edge case + 1 Instead-of row: inside double quotes bash strips the backslash only before `$` backtick `"` `\` newline, so a regex embedded in a double-quoted string (`grep -E "…\$"`) reaches the tool with a bare `$` (EOL anchor) while `"\d"` keeps its backslash — single-quote regex literals or account for exactly those five escapes. Doc-backed (GNU Bash manual, Double Quotes) + reproduced 2026-08-04 (guardrails PreToolUse rule `(sh|bash)([[:space:]]|-|<|\$)` matching `curl … | sh` at EOL). last_verified bumped to 2026-08-04. +## [2026-08-04] gap | Held back one queued candidate as out-of-scope for a general cross-repo wiki: MLIR IRDL region ops fail block verification ("block with no terminator") because IRDL (LLVM 22) cannot declare a terminator op or set NoTerminator/RegionKind on user-defined ops, so a region-bearing IRDL op needs a borrowed terminator (omp.terminator/llvm.unreachable) or should use flat marker ops carrying a `children` id-list instead. Field-tested (measured with mlir-opt 22.1.8) and the terminator/NoTerminator/RegionKind mechanics are doc-corroborated (mlir.llvm.org LangRef, IRDL dialect), but it is single-repo compiler-internals with no home in the current 10 general SWE domains. Recommend it live in the owning repo's own docs, or open a dedicated `compilers`/`mlir` domain if the wiki should carry dialect-authoring knowledge. diff --git a/wiki/platforms/environment/path-resolution.md b/wiki/platforms/environment/path-resolution.md index 88f0687..173e379 100644 --- a/wiki/platforms/environment/path-resolution.md +++ b/wiki/platforms/environment/path-resolution.md @@ -10,7 +10,8 @@ sources: - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/hash.html - https://man7.org/linux/man-pages/man7/environ.7.html - https://www.sudo.ws/docs/man/sudoers.man/ -last_verified: 2026-07-10 + - https://docs.brew.sh/FAQ +last_verified: 2026-08-04 related: [platforms-toolchains-version-management, platforms-processes-background-services] --- @@ -62,6 +63,7 @@ binaries; interactive convenience is the only place bare names are safe. | Case | Then | |------|------| | `command -v` shows an alias/function, not a binary | You are debugging the wrong thing — `type ` names the kind; bypass with `command ` or the absolute path to test the real binary | +| `command -v tool` / `which tool` reports not-found on macOS and you are about to conclude the tool is not installed | A Homebrew **keg-only** formula (llvm, curl, openjdk, node@N, libpq, ruby) is installed but deliberately not symlinked onto `PATH`. Confirm with `brew list --versions ` and `ls "$(brew --prefix )/bin/"`; invoke via the keg path `"$(brew --prefix )/bin/tool"`, or `brew link --force ` only if it will not shadow system software | | `PATH` reordered in the rc file but the running shell still resolves the old one | Rc edits apply to NEW shells; `exec $SHELL -l` or open a fresh terminal, then `hash -r` | | Same command, different result under `env -i sh -c 'cmd'` | The difference is your interactive environment — reproduce daemon/CI behavior this way before blaming the machine | | An empty entry or `.` in `PATH` | Current-directory lookup: a file named like a common command in the cwd gets executed — remove the entry and use `./name` for local scripts | @@ -72,6 +74,7 @@ binaries; interactive convenience is the only place bare names are safe. |---------------------|-----------------|-----| | Add `export PATH=...` lines to every rc file until it works | Identify which context runs the command (interactive shell, sudo, cron, GUI app, ssh) and set `PATH` in THAT context's init point once | Shotgun exports mask the real resolution order and drift apart across files | | `which cmd` for debugging resolution | `type cmd` / `command -v cmd` | `which` is an external binary that scans `PATH` — it cannot see the aliases, functions, and builtins your shell will run first | +| Conclude a tool is absent because `which`/`command -v` returns not-found on macOS | Check the package manager's own record first: `brew list --versions ` and `ls "$(brew --prefix )/bin/"` | Keg-only Homebrew formulae are installed but intentionally off `PATH`; PATH-absence is not proof of not-installed | | Calling a bare tool name in a hook/daemon/agent script | Resolve to `TOOL="$(command -v tool)"` with a fail-loud check, or hardcode the absolute path | The caller's `PATH` is not yours; silent resolution to a different or missing binary corrupts the run | ## Sources @@ -81,3 +84,4 @@ binaries; interactive convenience is the only place bare names are safe. - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/hash.html — `hash -r` forgets all remembered utility locations - https://man7.org/linux/man-pages/man7/environ.7.html — `PATH`: colon-separated directory prefixes searched for executables - https://www.sudo.ws/docs/man/sudoers.man/ — `secure_path` value replaces `PATH` for sudo-run commands (with default `env_reset`) +- https://docs.brew.sh/FAQ — "keg-only" formulae are installed into the Cellar and **not** symlinked into the prefix, so they are absent from `PATH` though installed; `brew info ` prints why and how to use it (verified 2026-08-04: `which mlir-opt` → not found while `brew list --versions llvm` → 22.1.8 and `/opt/homebrew/opt/llvm/bin/mlir-opt` ran) diff --git a/wiki/platforms/shells/portable-shell-scripts.md b/wiki/platforms/shells/portable-shell-scripts.md index 23eed9b..2a1d6b3 100644 --- a/wiki/platforms/shells/portable-shell-scripts.md +++ b/wiki/platforms/shells/portable-shell-scripts.md @@ -10,7 +10,8 @@ sources: - https://zsh.sourceforge.io/Doc/Release/Parameters.html - https://google.github.io/styleguide/shellguide.html - https://www.shellcheck.net/ -last_verified: 2026-07-10 + - https://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html +last_verified: 2026-08-04 related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management, platforms-shells-command-text-inspected-before-execution] --- @@ -62,6 +63,7 @@ non-interactive environment). | Critical command is in a pipeline but the interpreter is POSIX sh (no `pipefail`) | Run the critical command outside the pipeline (temp file between stages) and test `$?` directly | | Script runs via cron/CI/hooks and commands are "not found" | Non-interactive shells load no rc files — no user PATH, no version-manager shims. Call binaries by absolute path (see platforms-toolchains-version-management) | | `set -u` breaks on optional variables | Expand with an explicit default: `"${OPT:-}"` | +| Embedding a regex or pattern in a double-quoted string (`grep -E "…"`, `sed`, `awk`) | Inside double quotes bash strips the backslash only before `$`, backtick, `"`, `\`, or newline — so `"\$"` reaches the tool as a bare `$` (a regex end-of-line anchor) while `"\d"` keeps its backslash. Single-quote regex literals so nothing is stripped, or account for exactly those five escapes | ## Instead of @@ -70,6 +72,7 @@ non-interactive environment). | Build a command string and `eval` it | Build an array and expand it: `cmd "${args[@]}"` | `eval` re-parses quotes and globs; arrays pass arguments through exactly | | Put a command plus its flags in one variable and run `$cmd` | Variable holds the binary path only; flags are separate words | zsh runs the whole value as one command name; bash re-splits and re-globs it | | Trust a trailing `echo "done"` as proof a step ran | Verify the produced state with an independent command | Inside `&&` chains and subshells, `set -e` misses failures and the echo still prints | +| Wrap a regex containing `\$`, `\"`, or a backtick in double quotes | Single-quote the pattern (`grep -E '…\$'`), or write only the five double-quote escapes deliberately | Double quotes silently drop the backslash before those characters, so `"\$"` becomes bare `$` and the pattern matches something else | ## Sources @@ -78,3 +81,4 @@ non-interactive environment). - https://zsh.sourceforge.io/Doc/Release/Parameters.html — zsh arrays numbered from 1 (KSH_ARRAYS excepted) - https://google.github.io/styleguide/shellguide.html — quote variables, prefer bash for scripts, arrays over eval - https://www.shellcheck.net/ — shell script static analysis +- https://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html — inside double quotes the backslash "retains its special meaning only when followed by one of" `$`, backtick, `"`, `\`, or newline; before any other character it is preserved (verified 2026-08-04: `"(sh|bash)([[:space:]]|-|<|\$)"` reaches grep with a bare `$`, matching `sh` at end-of-line)