Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 103 additions & 44 deletions .dev-loop/INGEST_REPORT.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,112 @@
# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state

The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the
harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75
unique pages, with the same insight landing at up to 3 different paths across
up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the
#6–#13 consolidation — this branch carries the reconciled end-state and the 15
PRs are closed in its favor.
# Knowledge flush — 3 candidate(s): 1 ingested, 2 dropped

## Verified best-practice

Every adopted page's sources were carried from its originating PR's flush, where
they were live-verified at flush time; no new URLs were introduced during
consolidation (checked mechanically: every `http(s)` URL in every merged page
appears in a source PR's diff; every added body line in amended pages traces to
a source PR hunk — orphan-line verification). Confidence fields were kept as the
originating flushes set them, except client-side-rate-limiting where the union
of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports
`verified` for the load-bearing claims. One subagent's fabricated content (12
files matching neither main nor any PR, with invented source URLs) was detected
by the same verification and replaced with true PR content.
### 1. A dependency command that resyncs the environment deletes undeclared packages — `confidence: verified`

**Candidate as harvested:** "run `uv add`/`uv remove` only after background jobs
finish; register dev tools like pytest/ruff in a dependency-group instead of
`uv pip install`, so they survive a sync."

**Sources checked (live this session):**

- https://docs.astral.sh/uv/concepts/projects/sync/ — "`uv sync` performs 'exact'
syncing by default, which means it will remove any packages that are not present
in the lockfile"; `--inexact` retains them; "`uv run` uses 'inexact' syncing by
default, ensuring that all required packages are installed but not removing
extraneous packages."
- https://docs.astral.sh/uv/concepts/projects/dependencies/ — PEP 735
`[dependency-groups]`; "the `dev` group is synced by default."
- https://docs.astral.sh/uv/reference/cli/ — `uv add`: "The lockfile and project
environment will be updated to reflect the added dependencies."
- https://peps.python.org/pep-0735/ — dependency groups as a standard manifest field.

**How it was verified — local reproduction, uv 0.11.5 (aarch64-apple-darwin),
throwaway project:**

1. Declared `packaging`, then installed `iniconfig` ad-hoc with `uv pip install`
(undeclared, absent from `uv.lock`).
2. `uv add typing-extensions` → `iniconfig` **survived**.
3. `uv remove packaging` → `iniconfig` **deleted** along with the intended package.
Repeated a second time with the same result.
4. `uv remove packaging --no-sync` → `iniconfig` **preserved**.
5. Flag inventory via `uv {add,remove,sync,run} --help`: only `uv sync` exposes
`--inexact`, only `uv run` exposes `--exact`; `uv add` and `uv remove` expose
neither. `uv remove --no-sync`: "Avoid syncing the virtual environment after
re-locking the project."
6. Delayed-failure mechanism: a background process that had imported `iniconfig`
before the resync kept running and still resolved the cached module, while a
later `import tomli_w` in that same process raised `ModuleNotFoundError`.

**Correction applied to the candidate.** The candidate treats `uv add` and
`uv remove` as symmetric ("uv add/remove re-syncs the whole environment"). The
reproduction refutes that: `add` is inexact and harmless to undeclared packages,
`remove` is exact and has no `--inexact` escape hatch. The ingested page carries
the measured per-command exactness table rather than the symmetric claim, so the
"wait for the job" directive is scoped to the commands that actually prune.

### 2 & 3. Dropped — see Open-PR check

No independent verification was performed on the two orchestration candidates
because both are pending duplicates; neither is being promoted to a page here.

## Existing-layer check

- Merged-main near-dup scan before consolidation: pairwise Jaccard over
title + "When this applies" across all 141 merged pages → **0 flagged pairs**;
previously merged content carries no duplication.
- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one
canonical page each (rate limiting 8→1, call-site enumeration 7→folded into
the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1,
env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1,
harness-mediated results 2→1, leaked artifacts 2→1, orchestration category
naming unified). Three near-pairs kept distinct after trigger comparison,
with mutual `related:` links (differential setup vs interpretation; expansion
semantics vs off-switch design; import-time tactics vs level choice).
- 24 existing pages received union-merged amendments; additions already present
in main (from #16/#20) were skipped, and all non-canonical `related:` ids
were remapped to canonical page ids (post-merge broken-link scan: 0).
Routed via `INDEX.md` → `platforms` ("toolchain version pinning", "background
services") rather than the harvested `infrastructure` hint, whose seeded scope is
CI/CD, containers, rollout, observability, path-valued config and multi-agent
orchestration — none of which covers a local package manager mutating a project
virtualenv. Then read the `platforms` domain index and every toolchains/processes
page whose "load when" line overlaps.

Pages read: platforms-toolchains-version-management, platforms-toolchains-compiler-sysroot-on-macos, platforms-processes-background-services, infrastructure-agent-orchestration-pane-delivery-confirmation, infrastructure-agent-orchestration-worktree-isolated-workers

**Overlaps found.** `platforms-toolchains-version-management` was the only merged
page mentioning uv at all (`uv sync --frozen`, `.python-version`). Its trigger is
*version drift across machines* and its directives are about pinning and
committing lockfiles — it never covers a sync deleting packages, nor the timing
hazard of mutating dependencies mid-run. Different trigger → new page, not a
merge (a merge would have forced two cases onto one page, against the one-case
rule).

**Conflicts flagged:** none. Nothing in the merged wiki asserts the opposite
behavior; `version-management`'s "install from lockfiles (`npm ci`, `uv sync
--frozen`)" line is consistent with the new page and is not modified.

**Created:** `wiki/platforms/toolchains/environment-resync-removes-undeclared-packages.md`
(63 body lines, within the 120 limit; positive-guidance form; the two prohibitions
appear only as paired `Instead of` rows).

**Related links added, both ways:**
`platforms-toolchains-version-management` ↔ new page (lockfile/reproducibility
adjacency), and `platforms-processes-background-services` ↔ new page (the timing
hazard is against a job running in the background).

**Plumbing:** `wiki/platforms/index.md` toolchains table gains a routing row;
`log.md` gains the dated ingest entry.

## Open-PR check

Listed with `gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"` —
10 open heads: #61, #58, #57, #56, #55, #52, #51, #50, #49, #47. Each head was
fetched and diffed against `origin/main` under `wiki/`.

| Candidate | Overlapping open head | Verdict |
|-----------|----------------------|---------|
| uv / environment resync deletes undeclared packages | none — grepping every one of the 10 heads' `wiki/` diffs for `uv (add\|remove\|sync\|pip)` and `site-packages` returned 0 hits on all 10 | **new** |
| dev-loop `worktree_escape` guardrail escalates read-only cross-worktree access | **#51** — its `infrastructure-agent-orchestration-worktree-isolated-workers` already carries this case, and its local reproduction is *more* accurate than the candidate: reads alone (`grep`, `awk`, `cat`, `git -C … status`) pass; the escalation fires only when a main-root mention survives the strip **and** a write verb or absolute-path redirect also matches | **drop** |
| dev-loop Orca dispatch binding taxonomy (busy pane, `runtime_unavailable` vs `agent_unconfigured`, `--terminal` with `--worktree`) | **#51** — its `infrastructure-agent-orchestration-pane-delivery-confirmation` carries the idle-prompt-before-binding rule and the three-stage field observation verbatim | **drop** |

Both dropped candidates are **recurring re-emissions**, not un-retired rows: they
already appear in `~/.dev-loop/queue/.processed.jsonl` 15× and 10× respectively,
and PRs #56, #57 and #58 each recorded dropping the same pair as in-flight
duplicates of #51. They will keep re-entering the queue from live orchestration
sessions until #51 merges; retiring them again here is correct and cheap.

## Routing decision

- New categories: `infrastructure/agent-orchestration` (5 pages; unified the
competing `orchestration`/`agent-orchestration` names), `databases/data-survey`
(1), `qa/deliverables` (1). All other pages route into existing categories.
- Canonical-path decisions: rate limiting → `backend/common/reliability/`
(sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr
diagnostics → `platforms/processes/` (concern spans beyond shells); leaked
artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site
enumeration → the existing `backend/common/change-impact/` page.
- All 38 new pages listed in their domain indexes (nearest-index rule; backend
routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain
summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter,
ids, related-links, index coverage, size, qualifiers, staleness → 0 findings.
| Insight | Domain / category / page | New category? |
|---------|--------------------------|---------------|
| A dependency command resyncing the environment deletes packages absent from the lockfile | `platforms` / `toolchains` / `environment-resync-removes-undeclared-packages` (new page) | No — `toolchains` already owns package/version tooling behavior (`version-management`, `compiler-sysroot-on-macos`). The harvested `infrastructure` hint was re-routed: `infrastructure`'s seeded scope is pipelines, containers, rollout, observability and orchestration, none of which covers a local package manager pruning a project virtualenv. |
| `worktree_escape` guardrail read-vs-write | already at `infrastructure/agent-orchestration/worktree-isolated-workers` in open PR #51 | Not ingested (drop) |
| Orca dispatch binding taxonomy | already at `infrastructure/agent-orchestration/pane-delivery-confirmation` in open PR #51 | Not ingested (drop) |
2 changes: 2 additions & 0 deletions log.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
## [2026-08-06] revise | Union-merged amendments from the same 15 PRs into 24 existing pages (largest: tests-that-cannot-fail — 7 PRs: per-assertion mutation granularity, restore-mechanism-by-commit-state, suite-total-drop detection; portable-shell-scripts — 8 PRs: POSIX inline set-- reordering, ${VAR:-} vs ${VAR-} colon semantics, quote-by-what-the-text-is; test-data-and-isolation — 6 PRs: env-derived write paths, absent-variable cases, exec-bit fixtures, leak attribution; change-impact/call-site-enumeration — 8 unique additions from 7 path-variants: rename-in-place, same-type reorder, *args forwarding, two-searches-agree fallacy, codemod, coverage completion check). All non-canonical cross-page ids remapped to canonical pages; every added line traced to its source PR hunk (fabricated worker output was detected by orphan-line verification and replaced with true PR content).
## [2026-08-06] dedup | Cross-PR duplicate clusters collapsed: client-side rate limiting (8 PR versions at 3 paths → 1), call-site enumeration (7 versions at 7 paths → additions folded into the already-merged canonical page), stderr/exit-0 diagnostics (4 versions at 4 paths → 1), macOS sysroot (2→1), env-var off switch (2→1), completion predicates (2→1), robots.txt (2→1), harness-mediated results (2→1), leaked test artifacts (2→1), orchestration category naming (orchestration vs agent-orchestration → agent-orchestration). Kept distinct after trigger comparison: differential-testing vs differential-run-agreement (setup vs interpretation), unset-versus-empty-parameters vs env-var-off-switches (semantics vs design, cross-linked), import-time-side-effects vs test-level-choice edge (tactics vs level choice, cross-linked). Merged-main near-dup scan (Jaccard over title+trigger, 141 pages): 0 pairs — no duplication among previously merged content.
## [2026-08-06] ingest | Reconciled post-consolidation flush PRs #42–#43 (created against pre-#44 main). Folded unique content: platforms/filesystems/permissions-and-exec-bits (+`sh "$SCRIPT"` interpreter-invocation rows — stubs need no exec bit, avoids EDR chmod+x heuristics), platforms/shells/command-text-inspected-before-execution (+gate-AUTHOR step 8: parse all three POSIX quoting forms, expand only ~/$HOME/${HOME}; red-then-green bats evidence), testing/quality/checks-that-cannot-pass (+vacuously-green pre-implementation test row), platforms/processes/tool-diagnostics-without-a-failing-exit-code (+repeat-suppression edge, +clang reproduction source), infrastructure/ci-cd/secrets-handling (+independent credential channels row: gh token vs git SSH vs API token, git push --dry-run / author:@me probes), testing/mocking/what-to-mock (+ESM read-only-bindings DI rows, +PATH-emptied tripwire negative-proof). Dropped as duplicates of the #44 state: #42's rate-limit row (already in timeouts-and-retries + client-side-rate-limiting), #42's warning-only-diagnostics page (same trigger as tool-diagnostics-without-a-failing-exit-code), #42's leak-attribution row (already in test-data-and-isolation).

## [2026-08-07] ingest | knowledge-flush of 3 queued candidates → 1 ingested, 2 dropped. New: platforms/toolchains/environment-resync-removes-undeclared-packages — a dependency command that resyncs the environment deletes packages absent from the lockfile, so an ad-hoc `uv pip install` dev tool disappears on an unrelated dependency change; declare dev tools in a PEP 735 dependency group (synced by default), and gate only the *exact* commands on background jobs finishing. The harvested candidate asserted `uv add`/`uv remove` were symmetric; local reproduction (uv 0.11.5, macOS) refuted that — `uv add` left the undeclared package installed, `uv remove` deleted it twice, `uv remove --no-sync` preserved it, and `--help` shows only `uv sync` exposes `--inexact` and only `uv run` exposes `--exact`, so `uv remove` has no exactness escape hatch. Also reproduced the delayed-failure mechanism: a process that had already imported the package kept running off the module cache while a later new import in the same process raised ModuleNotFoundError. Sources live-verified (docs.astral.sh sync/dependencies/CLI reference, PEP 735). Plumbing: platforms/index.md toolchains row + reciprocal related-links on version-management and background-services. Dropped as in-flight duplicates of open PR #51: the guardrail worktree_escape read-vs-write candidate (whose naive "reads escalate" claim #51 already corrects with a local reproduction) and the Orca dispatch-binding taxonomy candidate (covered by pane-delivery-confirmation and worktree-isolated-workers). Both are recurring re-emissions — already present in the processed queue 15x and 10x respectively.
1 change: 1 addition & 0 deletions wiki/platforms/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ Match your situation to a "load when" line; load only matching pages.
|------|-----------|
| [compiler-sysroot-on-macos](toolchains/compiler-sysroot-on-macos.md) | On macOS a non-Xcode compiler (Homebrew/MacPorts LLVM) fails with `'stdio.h' file not found`, `ld: library 'System' not found`, or a `-Wmissing-sysroot` warning naming an SDK directory that does not exist; a build works under `/usr/bin/clang` but not under the toolchain the project requires; choosing between `-isysroot`, `SDKROOT`, `CPATH`, and `LIBRARY_PATH`; separating a toolchain precondition from a code regression when only the compiled tests fail |
| [version-management](toolchains/version-management.md) | "Works on my machine" from tool-version drift; a project needs a pinned language/tool version (.nvmrc, .python-version, .tool-versions); making CI use the same versions as local; onboarding a machine reproducibly; a script/cron/CI step can't find a version-managed binary (shims absent in non-interactive shells); deciding where lockfiles fit in reproducibility |
| [environment-resync-removes-undeclared-packages](toolchains/environment-resync-removes-undeclared-packages.md) | A package that was working (pytest, ruff, a scratch library) vanished after an unrelated dependency change and imports fail across unrelated test files; adding or dropping a dependency while a long job/test run/experiment is in flight; deciding whether dev-only tools belong in a dependency group or an ad-hoc `pip install`; deciding whether a manager's command prunes packages absent from the lockfile (`uv add` vs `uv remove` vs `uv sync` vs `uv run` exactness) |

## Planned (unseeded categories)

Expand Down
2 changes: 1 addition & 1 deletion wiki/platforms/processes/background-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ sources:
- https://man7.org/linux/man-pages/man1/nohup.1.html
- https://man7.org/linux/man-pages/man1/loginctl.1.html
last_verified: 2026-08-04
related: [platforms-toolchains-version-management, platforms-shells-portable-shell-scripts, platforms-processes-non-interactive-cli-invocation, infrastructure-config-path-valued-config]
related: [platforms-toolchains-version-management, platforms-shells-portable-shell-scripts, platforms-processes-non-interactive-cli-invocation, infrastructure-config-path-valued-config, platforms-toolchains-environment-resync-removes-undeclared-packages]
---

# Keeping a Process Running Beyond the Terminal Session
Expand Down
Loading
Loading