Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
247 changes: 204 additions & 43 deletions .dev-loop/INGEST_REPORT.md

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions log.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,6 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
## [2026-08-06] revise | Union-merged amendments from the same 15 PRs into 24 existing pages (largest: tests-that-cannot-fail — 7 PRs: per-assertion mutation granularity, restore-mechanism-by-commit-state, suite-total-drop detection; portable-shell-scripts — 8 PRs: POSIX inline set-- reordering, ${VAR:-} vs ${VAR-} colon semantics, quote-by-what-the-text-is; test-data-and-isolation — 6 PRs: env-derived write paths, absent-variable cases, exec-bit fixtures, leak attribution; change-impact/call-site-enumeration — 8 unique additions from 7 path-variants: rename-in-place, same-type reorder, *args forwarding, two-searches-agree fallacy, codemod, coverage completion check). All non-canonical cross-page ids remapped to canonical pages; every added line traced to its source PR hunk (fabricated worker output was detected by orphan-line verification and replaced with true PR content).
## [2026-08-06] dedup | Cross-PR duplicate clusters collapsed: client-side rate limiting (8 PR versions at 3 paths → 1), call-site enumeration (7 versions at 7 paths → additions folded into the already-merged canonical page), stderr/exit-0 diagnostics (4 versions at 4 paths → 1), macOS sysroot (2→1), env-var off switch (2→1), completion predicates (2→1), robots.txt (2→1), harness-mediated results (2→1), leaked test artifacts (2→1), orchestration category naming (orchestration vs agent-orchestration → agent-orchestration). Kept distinct after trigger comparison: differential-testing vs differential-run-agreement (setup vs interpretation), unset-versus-empty-parameters vs env-var-off-switches (semantics vs design, cross-linked), import-time-side-effects vs test-level-choice edge (tactics vs level choice, cross-linked). Merged-main near-dup scan (Jaccard over title+trigger, 141 pages): 0 pairs — no duplication among previously merged content.
## [2026-08-06] ingest | Reconciled post-consolidation flush PRs #42–#43 (created against pre-#44 main). Folded unique content: platforms/filesystems/permissions-and-exec-bits (+`sh "$SCRIPT"` interpreter-invocation rows — stubs need no exec bit, avoids EDR chmod+x heuristics), platforms/shells/command-text-inspected-before-execution (+gate-AUTHOR step 8: parse all three POSIX quoting forms, expand only ~/$HOME/${HOME}; red-then-green bats evidence), testing/quality/checks-that-cannot-pass (+vacuously-green pre-implementation test row), platforms/processes/tool-diagnostics-without-a-failing-exit-code (+repeat-suppression edge, +clang reproduction source), infrastructure/ci-cd/secrets-handling (+independent credential channels row: gh token vs git SSH vs API token, git push --dry-run / author:@me probes), testing/mocking/what-to-mock (+ESM read-only-bindings DI rows, +PATH-emptied tripwire negative-proof). Dropped as duplicates of the #44 state: #42's rate-limit row (already in timeouts-and-retries + client-side-rate-limiting), #42's warning-only-diagnostics page (same trigger as tool-diagnostics-without-a-failing-exit-code), #42's leak-attribution row (already in test-data-and-isolation).
## [2026-08-08] ingest | knowledge-flush of 5 queued candidates → 2 promoted, 3 dropped as in-flight duplicates. New: qa/deliverables/command-transcripts-in-a-document (re-capture a CLI transcript at paste time and give each stream its own labelled block — a `2>&1` capture records the environment's flush order, not the program's write order; local three-way reproduction on CPython 3.13/macOS: through a pipe the merged order was all-stderr-then-all-stdout, under `python -u` and on a pty it was the program's order, one unchanged binary; POSIX stdin.html — stderr "shall not be fully buffered", stdout fully buffered iff not on an interactive device; doctest captures stdout but not stderr). Merged: qa/document-verification/spec-document-gates +5th "External agreement" axis / +4 edge cases / +1 Instead-of (when a documented table's cells are copies of a code constant, resolve the owning symbol and compare per row — asserting the invariant the table itself states makes the document its own oracle, so the gate built to catch drift is what pins the stale claim green; reproduced 2026-08-08 in `linkly`: `docs/ENFORCEMENT-MATRIX.md` §C reads `warning` in all five rows while `SEVERITY_OF` grades three `info`). Reciprocal related-links added on generated-artifacts-as-deliverable-source, tool-diagnostics-without-a-failing-exit-code, non-interactive-cli-invocation.
## [2026-08-08] dedup | Dropped 3 orchestration candidates as pending duplicates of open PRs, not re-ingested: worktree_escape fires `ask` on read-only cross-worktree access and the coordinator must budget the escalation round trip → already in #51's worktree-isolated-workers hunk (identical directive, richer reproduction); the Orca dispatch-binding taxonomy (runtime_unavailable = occupied turn tail → wait and rebind; agent_unconfigured = dead agent → replace; always pass the worktree with the pane) → already in #51's pane-delivery-confirmation hunk; a tmux worker wedged on a numbered in-band chooser → already in #64's unattended-worker-questions (classify by terminal tail, unblock by allowlisted key, re-send the in-flight prompt). Merging #51 and #64 retires this recurring trio.
## [2026-08-08] lint | 0 errors fixed, 0 reported — changed-page pass over qa/deliverables/command-transcripts-in-a-document (new, 62 body lines, verified with 4 cited sources), qa/document-verification/spec-document-gates (85 body lines), and the three pages that gained reciprocal related-links. Checks run: sources-vs-confidence, prohibition-outside-Instead-of, related/inline id resolution, domain-index presence and trigger agreement, vague qualifiers, body length.
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ sources:
- https://man7.org/linux/man-pages/man1/timeout.1.html
- https://en.wikipedia.org/wiki/Bracketed-paste
last_verified: 2026-08-04
related: [platforms-processes-background-services, platforms-tools-bsd-vs-gnu-cli, platforms-shells-portable-shell-scripts, debugging-methodology-hypothesis-testing, platforms-processes-parsing-cli-structured-output, platforms-shells-env-var-off-switches, infrastructure-agent-orchestration-pane-delivery-confirmation]
related: [platforms-processes-background-services, platforms-tools-bsd-vs-gnu-cli, platforms-shells-portable-shell-scripts, debugging-methodology-hypothesis-testing, platforms-processes-parsing-cli-structured-output, platforms-shells-env-var-off-switches, infrastructure-agent-orchestration-pane-delivery-confirmation, qa-deliverables-command-transcripts-in-a-document]
---

# Invoking a Prompt-Capable CLI from a Script or Agent Harness
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ sources:
- https://www.gnu.org/software/bash/manual/bash.html#Redirections
- https://code.claude.com/docs/en/hooks
last_verified: 2026-08-06
related: [platforms-processes-non-interactive-cli-invocation, platforms-shells-command-text-inspected-before-execution, testing-quality-checks-that-cannot-pass]
related: [platforms-processes-non-interactive-cli-invocation, platforms-shells-command-text-inspected-before-execution, testing-quality-checks-that-cannot-pass, qa-deliverables-command-transcripts-in-a-document]
---

# Feeding a Tool's Warnings Back When It Exits 0
Expand Down
82 changes: 82 additions & 0 deletions wiki/qa/deliverables/command-transcripts-in-a-document.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
id: qa-deliverables-command-transcripts-in-a-document
domain: qa
category: deliverables
applies_to: [general]
confidence: verified
sources:
- https://pubs.opengroup.org/onlinepubs/9799919799/functions/stdin.html
- https://docs.python.org/3/using/cmdline.html
- https://docs.python.org/3/library/doctest.html
- https://google.github.io/styleguide/docguide/best_practices.html
last_verified: 2026-08-08
related:
[
qa-deliverables-generated-artifacts-as-deliverable-source,
qa-document-verification-spec-document-gates,
platforms-processes-tool-diagnostics-without-a-failing-exit-code,
platforms-processes-non-interactive-cli-invocation,
]
---

# A Pasted CLI Transcript Inside a Document

## When this applies

You are putting a captured command run — an RFC's worked example, a README
quickstart, a design doc's "here is what it prints" — into a document, and the
transcript shows both the program's output and its diagnostics. Also when a
reviewer re-runs a documented example and gets a different order or extra lines.

## Do this

1. **Re-capture the transcript from the current tree immediately before the
paste lands.** A transcript captured earlier in the work describes the build
at capture time; anything added since is missing from a block presented as a
measurement. Capture and paste in the same step, and record the commit and
the exact command line above the block.
2. **Give each stream its own block, labelled.** Capture them separately —
`cmd >out.txt 2>err.txt` — and paste `out.txt` and `err.txt` as two blocks.
The document then states what the program produced on each stream, which is a
property of the program.
3. **When the document's point is the ordering** (a diagnostic must appear
before/after a result), state the ordering as a sentence about the streams
and show the two blocks, rather than showing one merged block as the proof.
4. **Force unbuffered output for any transcript you do capture merged**
(`python -u`, `PYTHONUNBUFFERED=1`, `stdbuf -o0 -e0`) and say in the document
that it was captured that way, so a reader who re-runs it plainly and sees a
different order knows why.
5. **Make the transcript executable where the toolchain has a mechanism for it**
— `doctest`, rustdoc doctests, a snippet-extraction test. An executed
transcript cannot go stale silently. Note that `doctest` checks stdout only:
"Output to stdout is captured, but not output to stderr", so a merged block
is not checkable by it either.

## Edge cases

| Case | Then |
|------|------|
| The tool writes everything to one stream by design | Say so in the document and show one block; the rule is about not *merging* two streams, not about splitting one |
| The tool changes its output when it is not attached to a terminal (colour, progress bars, plain-vs-rich format) | Capture the form your reader will get, name which one it is, and keep the capture command in the document ([platforms-processes-non-interactive-cli-invocation]) |
| The transcript is long and you abridge it | Mark the elision (`…`) at the point it happens; an unmarked cut reads as complete output and is what makes a later-added block go unnoticed |
| The output contains timings, temp paths, or ids that change per run | Replace them with a stable placeholder and say the substitution was made, so a re-runner diffs the shape rather than the values |
| The document is generated from the run | Re-run the generator instead of editing the block ([qa-deliverables-generated-artifacts-as-deliverable-source]) |
| A gate greps the transcript block | Anchor it on the stream label and the command line, not on line adjacency between the two streams ([qa-document-verification-spec-document-gates]) |

## Instead of

| If you are about to | Do this instead | Why |
|---------------------|-----------------|-----|
| Paste a `cmd 2>&1` capture as the record of what the program prints | Capture the two streams separately and paste two labelled blocks | With stdout to a pipe the C runtime makes it fully buffered while stderr is not, so the merged order is the environment's flush order, not the program's write order |
| Reuse a transcript captured earlier in the same piece of work | Re-run and re-capture at paste time | Output added between capture and paste is absent from a block the document presents as measured, and nothing in the block shows the omission |
| Argue an ordering claim ("the diagnostic comes after the result") from a merged block | State the claim about the streams and show them separately | The same binary produces both orders depending on whether stdout is a pipe or a tty — the merged block cannot support either claim |
| Reproduce the transcript once by hand and call it verified | Re-run it under the reader's conditions (piped, unbuffered, and on a tty) and reconcile | A single capture cannot distinguish a program-order claim from a buffering artefact |

## Sources

- https://pubs.opengroup.org/onlinepubs/9799919799/functions/stdin.html — "When opened, `stderr` shall not be fully buffered"; "`stdout` shall be fully buffered if and only if the file descriptor associated with the stream is determined not to be associated with an interactive device" — redirecting a transcript into a pipe or file changes stdout's buffering and therefore the interleaving, while stderr's is unchanged
- https://docs.python.org/3/using/cmdline.html — `-u`: "Force the stdout and stderr streams to be unbuffered"; `PYTHONUNBUFFERED` "is equivalent to specifying the `-u` option"
- https://docs.python.org/3/library/doctest.html — doctest "executes those sessions to verify that they work exactly as shown" (the executable-transcript mechanism), and "Output to stdout is captured, but not output to stderr"
- https://google.github.io/styleguide/docguide/best_practices.html — "Change your documentation in the same CL as the code change" — the capture-at-paste-time rule applied to transcripts
- Local reproduction 2026-08-08 (CPython 3.13, macOS), one program printing `STDOUT-1, STDERR-1, STDOUT-2, STDERR-2` in that order: through a pipe (`2>&1 | cat`) the merged capture came out `STDERR-1, STDERR-2, STDOUT-1, STDOUT-2` — stdout fully buffered, flushed at exit; the same command with `python -u` gave the program's order; and with both streams on a pty (`pty.openpty()`) it also gave the program's order. Three environments, two different documented orders, one unchanged program
- Field incident 2026-08-07 (`linkly`, RFC-0022 draft): a `build --run` transcript stated the reverse of the real diagnostic/output ordering and omitted a `validation-sample-derived` block added two tasks after the capture, with no elision mark. A reviewer re-running under `python -u` and a pty found both; rewritten with the streams separated, all four transcripts in the RFC then matched their re-runs byte for byte
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ sources:
- https://google.github.io/styleguide/docguide/best_practices.html — keep documentation sources close to the code they describe so they change together
last_verified: 2026-08-04
related:
[frontend-state-derived-state, qa-document-verification-spec-document-gates]
[frontend-state-derived-state, qa-document-verification-spec-document-gates, qa-deliverables-command-transcripts-in-a-document]
---

# A Requested Deliverable Duplicating an Artifact the Repo Already Generates
Expand Down
Loading
Loading