From 05c5378d4414bd661f4e8e1f89ab29c81d3442aa Mon Sep 17 00:00:00 2001 From: Mother Seara Date: Tue, 25 Aug 2026 05:12:30 +0900 Subject: [PATCH 1/3] fix(harness): make each signal say only what the code actually checked MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A first-use field report (2026-08-24, Windows/Codex) filed 16 items. Reproducing them against the real binaries, five of the seven that reproduce share one shape: a machine-readable signal asserts something the code never tested. exit code build-handoff exited 1 on the SUCCESS path only. The last line was `[ -z "$SUMMARY" ] && echo ...`; with a closed arc found the test is false, `&&` short-circuits, and that truth value became the script's exit status. Callers could not tell success from failure. Both paths create dev-plan.md + TODO.md, so both are exit 0 and the missing arc stays a warning; real failures still exit non-zero. flag name `arc-list --all` dropped `_archive` before testing the flag, so `--all` meant "every arc except the closed ones" — on a workspace whose only arc had been closed it printed nothing. Archived rows now list, stamped status=Archived. guard loop-guard's `--tokens` defaulted to 0, so a caller that never passed it accumulated zero forever: STOP:budget could not fire while `status` printed "tokens_used=0/200000", a line shaped exactly like a measurement. Unmeasured ticks are now counted and reported with a denominator; a reported budget still stops the loop. message the close printed "install mirror-stack for sealing" after testing `command -v am`. An MCP-only install has mirror-stack present and no `am` on PATH; the text sent those users to reinstall what they already had. It now names the condition tested. instruction the JOIN prompt told every role its first action was `bin/arc-attach "./projects/..."` — two relative paths, resolvable only from the repo root, while setup/install.sh explicitly does not promise bin/ on PATH. Every role's mandated first action failed. Both paths are resolved before they are printed, and arc-open refuses to emit a command it could not confirm is executable. Also atomic scaffolding: a `/` in the topic aborted `sed "s//$TOPIC/g"` after the project tree had already been created, leaving a half-built project that made the retry fail with "already exists". The topic is now substituted literally (bash pattern replacement — no delimiter to escape, no regex, and command substitution stays text), the tree is staged and moved into place in one step, and a failing arc-open rolls the project back and propagates its real exit code. tests/test_gates.sh: 48/48, and the summary carries its denominator — "all gate tests passed" is also what a run that collected zero checks prints, so an empty run now fails. Each new check was confirmed to go red with its own fix reverted. Co-Authored-By: Claude Opus 5 (1M context) --- bin/arc-close | 8 +++- bin/arc-list | 11 ++++- bin/arc-open | 14 +++++- bin/build-handoff | 10 ++++- bin/close-project | 8 +++- bin/loop-guard | 26 +++++++---- bin/yeoul-new | 39 +++++++++++++++-- tests/test_gates.sh | 103 +++++++++++++++++++++++++++++++++++++++++++- 8 files changed, 198 insertions(+), 21 deletions(-) diff --git a/bin/arc-close b/bin/arc-close index a015702..2991aea 100755 --- a/bin/arc-close +++ b/bin/arc-close @@ -251,7 +251,13 @@ mv "$ARC_DIR" "$ARCHIVE_DIR/" SEALED="$ARCHIVE_DIR/$ARC/_SUMMARY_${ARC}.md" # ── optional: seal into append-only ledger (best-effort; skips if `am` absent) ── -SEAL_MSG="no ledger — file only (install mirror-stack for sealing)" +# 🔴 say what was actually tested. The condition below is "is the `am` CLI on PATH", +# which is not the same as "mirror-stack is not installed" — an MCP-only install has +# mirror-stack present with no `am` on PATH, and the old text sent those users to +# reinstall something they already had. +# NB: single-quoted on purpose — backticks inside double quotes are command substitution, +# which ran `am` and leaked its exit status into the close (caught by tests/test_gates.sh). +SEAL_MSG='not sealed — file only (`am` CLI not on PATH; mirror-stack may still be installed)' if command -v am >/dev/null 2>&1; then if am record --agent yeoul --action arc-close --target "$ARC" \ --payload "{\"stop_reason\":\"${STOP}\"}" \ diff --git a/bin/arc-list b/bin/arc-list index 83e5549..fb23f33 100755 --- a/bin/arc-list +++ b/bin/arc-list @@ -1,7 +1,9 @@ #!/usr/bin/env bash set -euo pipefail # arc-list [--roots="dir1 dir2"] [--all] -# Find open deliberation arcs (default status=In-Progress only; --all = every arc). _archive excluded. +# Find deliberation arcs. Default: open (status=In-Progress), _archive excluded. +# --all = EVERY arc, archived ones included (they are the closed record; excluding them made --all +# return nothing on a workspace whose only arc had been closed). Archived rows carry status=Archived. # Default roots = ./projects/*/design/arcs. Output (tab-separated): STATUS \t ARC_DIR \t TITLE \t ROLES PROJECTS_DIR="${YEOUL_PROJECTS:-./projects}" @@ -19,9 +21,14 @@ field() { grep -m1 "^$1:" "$2" 2>/dev/null | sed "s/^$1:[[:space:]]*//; s/\"//g" for root in $ROOTS; do [ -d "$root" ] || continue while IFS= read -r arcfile; do - case "$arcfile" in */_archive/*) continue ;; esac + # 🔴 `--all` has to mean all. This skip used to run BEFORE the OPEN_ONLY test, so the flag + # named a coverage it did not have: with one closed arc on disk, `--all` printed nothing. + archived=0 + case "$arcfile" in */_archive/*) archived=1 ;; esac + if [ "$archived" = "1" ] && [ "$OPEN_ONLY" = "1" ]; then continue; fi arcdir="$(cd "$(dirname "$(dirname "$arcfile")")" && pwd)" status="$(field status "$arcfile")" + if [ "$archived" = "1" ]; then status="Archived"; fi title="$(field title "$arcfile")" roles="$(field roles "$arcfile")" if [ "$OPEN_ONLY" = "1" ] && [ "$status" != "In-Progress" ]; then continue; fi diff --git a/bin/arc-open b/bin/arc-open index 73d18a4..cd60e77 100755 --- a/bin/arc-open +++ b/bin/arc-open @@ -165,6 +165,18 @@ MDEOF } > "$ARC_DIR/ROSTER.md" # ── JOIN_PROMPTS (A = agent prompt / B = paste into external tab; same body) ── +# 🔴 A join prompt is an instruction someone will RUN, from a cwd we do not control. It used to +# emit the workspace-relative `bin/arc-attach "./projects/..."`, which resolves only when the +# reader happens to sit in the repo root — and setup/install.sh explicitly does NOT promise bin/ +# on PATH ("add to PATH, or call by path"). An MCP-only install has no ./bin at all, so every +# role's mandated first action failed (field report, 2026-08-24). Emit resolved absolute paths, +# and refuse to print a command we could not confirm is executable. +ATTACH_BIN="$SCRIPT_DIR/arc-attach" +if [ ! -x "$ATTACH_BIN" ]; then + echo "arc-open: cannot find an executable arc-attach at $ATTACH_BIN" >&2 + exit 1 +fi +ARC_DIR_ABS="$(cd "$ARC_DIR" && pwd)" { echo "# 🔗 Join prompts — ${ARC}" echo @@ -177,7 +189,7 @@ MDEOF echo "You are the '${r}' role of deliberation arc ${ARC}. Topic: ${TOPIC}." echo "" echo "★ First action — run ONLY this line and act on its output (no improvisation):" - echo " bin/arc-attach \"${ARC_DIR}\" ${r}" + echo " \"${ATTACH_BIN}\" \"${ARC_DIR_ABS}\" ${r}" echo " → it assigns your role, starts the watcher, and dictates your ticket order." echo "" echo "[invariant] Do NOT edit the deliberation thread directly (relay-only). Your output = answer in tickets/${r}/ + status open→answered." diff --git a/bin/build-handoff b/bin/build-handoff index d26268f..ec55e76 100755 --- a/bin/build-handoff +++ b/bin/build-handoff @@ -73,4 +73,12 @@ echo "┌─ dev handoff ─┐" echo " project: $NAME" echo " verdict: ${VERDICT:-TBD}" echo " created: dev/dev-plan.md · dev/TODO.md (fill blanks from _SUMMARY)" -[ -z "$SUMMARY" ] && echo " ⚠️ no closed arc — usually run after a GO close" +if [ -z "$SUMMARY" ]; then + echo " ⚠️ no closed arc — usually run after a GO close" +fi +# 🔴 exit code must report whether the handoff was BUILT, not what the last test happened to +# return. This line used to be `[ -z "$SUMMARY" ] && echo ...`: with a closed arc found the test +# is false, `&&` short-circuits, and the script exited 1 *on the success path only*. Both paths +# here created dev-plan.md + TODO.md, so both are exit 0; the missing arc is a warning, not a +# failure. Real failures above (no project / already exists) still exit non-zero with a cause. +exit 0 diff --git a/bin/close-project b/bin/close-project index f7b70fc..7932b44 100755 --- a/bin/close-project +++ b/bin/close-project @@ -41,7 +41,13 @@ mkdir -p "$ARCHIVE" mv "$SRC" "$DEST" # optional: seal into append-only ledger (best-effort; content-hash = _CLOSED.md) -SEAL_MSG="no ledger — file only (install mirror-stack for sealing)" +# 🔴 say what was actually tested. The condition below is "is the `am` CLI on PATH", +# which is not the same as "mirror-stack is not installed" — an MCP-only install has +# mirror-stack present with no `am` on PATH, and the old text sent those users to +# reinstall something they already had. +# NB: single-quoted on purpose — backticks inside double quotes are command substitution, +# which ran `am` and leaked its exit status into the close (caught by tests/test_gates.sh). +SEAL_MSG='not sealed — file only (`am` CLI not on PATH; mirror-stack may still be installed)' if command -v am >/dev/null 2>&1; then if am record --agent yeoul --action close-project --target "$NAME" \ --payload "{\"open_arcs\":${OPEN},\"reason\":\"closed\"}" \ diff --git a/bin/loop-guard b/bin/loop-guard index c23054d..08cf097 100755 --- a/bin/loop-guard +++ b/bin/loop-guard @@ -10,42 +10,50 @@ ARC_DIR="${1:-}"; CMD="${2:-status}" [ -d "$ARC_DIR" ] || { echo "arc dir not found: $ARC_DIR" >&2; exit 1; } STATE="$ARC_DIR/loop_state.tsv" -MAXR=""; BUDGET=""; TOKENS=0; PROGRESS="yes" +# 🔴 TOKENS has no safe default. `--tokens` omitted used to mean 0, so a caller that never +# passed it accumulated 0 forever: STOP:budget could not fire, while `status` printed +# "tokens_used=0/200000" — a line shaped exactly like a measurement. Omission and a +# measured zero must not look the same, so track whether it was ever supplied. +MAXR=""; BUDGET=""; TOKENS=0; TOKENS_GIVEN=0; PROGRESS="yes" for a in "${@:3}"; do case "$a" in --max-rounds=*) MAXR="${a#*=}" ;; --token-budget=*) BUDGET="${a#*=}" ;; - --tokens=*) TOKENS="${a#*=}" ;; + --tokens=*) TOKENS="${a#*=}"; TOKENS_GIVEN=1 ;; --progress=*) PROGRESS="${a#*=}" ;; esac; done -ROUND=0; MR=3; TB=200000; TU=0; NP=0 +ROUND=0; MR=3; TB=200000; TU=0; NP=0; UM=0 load(){ if [ -f "$STATE" ]; then while IFS=$'\t' read -r k v; do case "$k" in round) ROUND=$v ;; max_rounds) MR=$v ;; token_budget) TB=$v ;; - tokens_used) TU=$v ;; no_progress) NP=$v ;; + tokens_used) TU=$v ;; no_progress) NP=$v ;; unmeasured) UM=$v ;; esac; done < "$STATE" fi } -save(){ printf 'round\t%s\nmax_rounds\t%s\ntoken_budget\t%s\ntokens_used\t%s\nno_progress\t%s\n' \ - "$ROUND" "$MR" "$TB" "$TU" "$NP" > "$STATE"; } +save(){ printf 'round\t%s\nmax_rounds\t%s\ntoken_budget\t%s\ntokens_used\t%s\nno_progress\t%s\nunmeasured\t%s\n' \ + "$ROUND" "$MR" "$TB" "$TU" "$NP" "$UM" > "$STATE"; } +# How much of tokens_used is actually backed by a reported count. Printed with a denominator so a +# quiet "tokens=0/200000" can never be mistaken for a measurement of zero. +warn_unmeasured(){ [ "$UM" -gt 0 ] && printf ' unmeasured=%s/%s-ticks' "$UM" "$ROUND"; return 0; } case "$CMD" in init) - ROUND=0; MR="${MAXR:-3}"; TB="${BUDGET:-200000}"; TU=0; NP=0; save + ROUND=0; MR="${MAXR:-3}"; TB="${BUDGET:-200000}"; TU=0; NP=0; UM=0; save echo "loop init: max_rounds=$MR token_budget=$TB" ;; tick) load ROUND=$((ROUND+1)); TU=$((TU+TOKENS)) + if [ "$TOKENS_GIVEN" = "0" ]; then UM=$((UM+1)); fi if [ "$PROGRESS" = "no" ]; then NP=$((NP+1)); else NP=0; fi save if [ "$ROUND" -gt "$MR" ]; then echo "STOP:max-rounds (round=$ROUND > max=$MR)" elif [ "$TU" -gt "$TB" ]; then echo "STOP:budget (used=$TU > budget=$TB)" elif [ "$NP" -ge 2 ]; then echo "STOP:no-progress (streak=$NP)" - else echo "CONTINUE (round=$ROUND/$MR tokens=$TU/$TB noprog=$NP)" + else echo "CONTINUE (round=$ROUND/$MR tokens=$TU/$TB noprog=$NP$(warn_unmeasured))" fi ;; status) - load; echo "round=$ROUND/$MR tokens_used=$TU/$TB no_progress=$NP" ;; + load; echo "round=$ROUND/$MR tokens_used=$TU/$TB no_progress=$NP$(warn_unmeasured)" ;; *) echo "usage: loop-guard init|tick|status [--max-rounds=N --token-budget=T --tokens=N --progress=yes|no]"; exit 1 ;; esac diff --git a/bin/yeoul-new b/bin/yeoul-new index 5ed0aa1..adef0c2 100755 --- a/bin/yeoul-new +++ b/bin/yeoul-new @@ -25,17 +25,32 @@ fi PROJ_DIR="$PROJECTS_DIR/$NAME" if [ -e "$PROJ_DIR" ]; then echo "already exists: $PROJ_DIR"; exit 1; fi + +# 🔴 Build the scaffold in a staging dir and move it into place only once every step succeeded. +# This used to mkdir the real tree first and then render the spec, so a topic containing `/` +# (e.g. "fast/medium/slow") blew up the sed below — `unknown option to 's'` — and left +# projects//{design/arcs,dev} plus an empty spec.md behind under `set -e`. The retry then +# hit "already exists". Staging keeps failure atomic: nothing lands unless everything worked. +mkdir -p "$PROJECTS_DIR" +STAGE="$(mktemp -d "$PROJECTS_DIR/.staging-$NAME.XXXXXX")" # same filesystem → mv is atomic +cleanup_stage() { [ -n "${STAGE:-}" ] && rm -rf "$STAGE"; } +trap cleanup_stage EXIT + DESIGN_DIR="$PROJ_DIR/design" -mkdir -p "$DESIGN_DIR/arcs" "$PROJ_DIR/dev" +S_DESIGN="$STAGE/design" +mkdir -p "$S_DESIGN/arcs" "$STAGE/dev" TEMPLATE="$SCRIPT_DIR/../templates/spec.md" if [ -f "$TEMPLATE" ]; then - sed "s//$TOPIC/g" "$TEMPLATE" > "$DESIGN_DIR/spec.md" + # 🔴 literal substitution, not sed: the topic is arbitrary user text and must never be parsed as + # a sed expression. Bash pattern replacement has no delimiter to escape and no regex to escape. + TPL="$(cat "$TEMPLATE")" + printf '%s\n' "${TPL///$TOPIC}" > "$S_DESIGN/spec.md" else - echo "# 📐 Spec — $TOPIC" > "$DESIGN_DIR/spec.md" + printf '# 📐 Spec — %s\n' "$TOPIC" > "$S_DESIGN/spec.md" fi -cat > "$PROJ_DIR/README.md" << MDEOF +cat > "$STAGE/README.md" << MDEOF # $NAME > Yeoul incubator project. Topic: $TOPIC @@ -45,6 +60,10 @@ cat > "$PROJ_DIR/README.md" << MDEOF - when mature: \`yeoul-graduate $NAME\`. MDEOF +# everything rendered; publish the staged tree in one move, then stop cleaning it up. +mv "$STAGE" "$PROJ_DIR" +STAGE=""; trap - EXIT + echo "┌─ Yeoul project created ─┐" echo " project: $NAME" echo " dirs : $PROJ_DIR/{design,dev}" @@ -53,6 +72,18 @@ echo " spec : $DESIGN_DIR/spec.md" if [ "$MAKE_ARC" -eq 1 ]; then SLUG="$(echo "$NAME" | tr ' ' '_')" echo + # 🔴 if the arc fails to open, the project must not survive half-built — same reason as above. + # 🔴 capture the real code. `if ! cmd; then rc=$?` records the *negation's* status (always 0), + # so a failing arc-open was rolled back and then reported as "exit 0" — the same class of bug + # this commit exists to remove. Caught by the rollback test below, which asserted the code. + set +e bash "$SCRIPT_DIR/arc-open" "$SLUG" --topic="$TOPIC" --roles="$ROLES" --backend="$BACKEND" \ --arcs-dir="$DESIGN_DIR/arcs" + rc=$? + set -e + if [ "$rc" -ne 0 ]; then + rm -rf "$PROJ_DIR" + echo "arc-open failed (exit $rc) — rolled back $PROJ_DIR so a retry starts clean" >&2 + exit "$rc" + fi fi diff --git a/tests/test_gates.sh b/tests/test_gates.sh index 122c662..6b4d646 100755 --- a/tests/test_gates.sh +++ b/tests/test_gates.sh @@ -10,8 +10,19 @@ BIN="$(cd "$(dirname "${BASH_SOURCE[0]}")/../bin" && pwd)" WS="$(mktemp -d)"; trap 'rm -rf "$WS"' EXIT cd "$WS"; export YEOUL_PROJECTS="$WS/projects" FAIL=0 +# 🔴 count what was collected. "all gate tests passed" is also what a run that collected ZERO +# checks prints — the summary has to carry its own denominator, and an empty run has to fail. +CHECKS=0; PASSED=0 assert() { # assert - if [ "$2" = "$3" ]; then echo " ✓ $1 (exit $3)"; else echo " ✗ $1 — expected $2, got $3"; FAIL=1; fi + CHECKS=$((CHECKS+1)) + if [ "$2" = "$3" ]; then PASSED=$((PASSED+1)); echo " ✓ $1 (exit $3)" + else echo " ✗ $1 — expected $2, got $3"; FAIL=1; fi +} +ok() { CHECKS=$((CHECKS+1)); PASSED=$((PASSED+1)); echo " ✓ $1"; } +bad() { CHECKS=$((CHECKS+1)); echo " ✗ $1"; FAIL=1; } +check() { # check — passes if the command succeeds + local d="$1"; shift + if "$@" >/dev/null 2>&1; then ok "$d"; else bad "$d"; fi } # portable in-place edit (GNU + BSD/macOS) @@ -280,6 +291,94 @@ ls -d "$WS/arcs/_archive"/*_sb >/dev/null 2>&1 \ || echo " ✓ nothing archived while the instrument was broken" rm -rf "$SBIN" +# ═══════════════════════════════════════════════════════════════════════════════ +# Signals must say only what the code checked. +# Each block below reproduces a field-reported defect (first-use report, 2026-08-24) and asserts +# the repaired behaviour. Every one of these went red against the pre-fix binary — see +# `revert-to-red` in the PR body; a test that cannot go red is not evidence. +# ═══════════════════════════════════════════════════════════════════════════════ +echo +echo "── signal/evidence agreement ──" +NEW="$WS/new"; mkdir -p "$NEW" + +# YL-01 · a `/` in the topic used to abort sed and leave a half-built project behind. +( cd "$NEW" && YEOUL_PROJECTS="$NEW/projects" "$BIN/yeoul-new" slashed --topic='fast/medium/slow' ) \ + >/dev/null 2>&1 +assert "topic containing / scaffolds" 0 $? +if grep -qF 'fast/medium/slow' "$NEW/projects/slashed/design/spec.md" 2>/dev/null; then + ok "topic substituted literally (not parsed as a sed expression)"; else bad "topic not rendered literally"; fi +# command substitution in a topic must stay text +( cd "$NEW" && YEOUL_PROJECTS="$NEW/projects" "$BIN/yeoul-new" inj --topic='$(id -u)' ) >/dev/null 2>&1 +if grep -qF '$(id -u)' "$NEW/projects/inj/design/spec.md" 2>/dev/null; then + ok "topic is not evaluated as shell"; else bad "topic was evaluated"; fi + +# YL-01 (atomicity) · a failure anywhere must leave no partial project to collide with a retry. +STUB="$WS/stubbin"; rm -rf "$STUB"; cp -r "$BIN" "$STUB" +printf '#!/usr/bin/env bash\nexit 7\n' > "$STUB/arc-open"; chmod +x "$STUB/arc-open" +ROLL="$WS/roll"; mkdir -p "$ROLL" +( cd "$ROLL" && YEOUL_PROJECTS="$ROLL/projects" "$STUB/yeoul-new" doomed --topic='a/b' ) >/dev/null 2>&1 +assert "failed scaffold propagates the real exit code" 7 $? +if [ -z "$(ls -A "$ROLL/projects" 2>/dev/null)" ]; then + ok "failed scaffold left nothing behind (retry starts clean)"; else bad "partial scaffold survived"; fi +rm -rf "$STUB" + +# YL-02 · the JOIN prompt is an instruction someone runs from an unknown cwd. +JP="$(ls "$NEW"/projects/slashed/design/arcs/*/JOIN_PROMPTS.md 2>/dev/null | head -1)" +ATTACH_LINE="$(grep -m1 'arc-attach' "$JP" 2>/dev/null | sed 's/^ *//')" +case "$ATTACH_LINE" in + /*|'"/'*) ok "JOIN prompt emits an absolute attach path" ;; + *) bad "JOIN prompt still emits a relative path: $ATTACH_LINE" ;; +esac +# the emitted line must actually run — from a cwd that is not the workspace +if ( cd / && eval "$ATTACH_LINE" ) >/dev/null 2>&1; then + ok "the emitted first-action line runs from a foreign cwd"; else bad "emitted attach line does not run from /"; fi + +# YL-03 · exit code must report whether the handoff was built. +HO="$WS/ho"; mkdir -p "$HO" +( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/yeoul-new" hp --topic='handoff path' ) >/dev/null 2>&1 +HARC="$(ls -d "$HO"/projects/hp/design/arcs/*_hp 2>/dev/null | head -1)" +( cd "$HO" && "$BIN/arc-close" "$HARC" "GO: build it" --stop=converged ) >/dev/null 2>&1 +HSUM="$(ls "$HARC"/_SUMMARY_*.md 2>/dev/null | head -1)" +sedi 's/- (fill in)/- the phase-owned runtime boundary is settled and snapshots are taken at transition edges/' "$HSUM" +( cd "$HO" && "$BIN/arc-close" "$HARC" "GO: build it" --stop=converged ) >/dev/null 2>&1 +( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/build-handoff" hp ) >/dev/null 2>&1 +assert "build-handoff exits 0 on the SUCCESS path (closed arc found)" 0 $? +if [ -f "$HO/projects/hp/dev/TODO.md" ]; then ok "build-handoff produced dev/TODO.md"; else bad "no TODO.md"; fi +( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/build-handoff" hp ) >/dev/null 2>&1 +assert "a real failure (already exists) still exits non-zero" 1 $? + +# YL-06 · `--all` has to mean all. +ARCHN="$( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/arc-list" --all 2>/dev/null | grep -c '_archive' )" +if [ "$ARCHN" -ge 1 ]; then ok "arc-list --all includes archived arcs ($ARCHN)"; else bad "--all still hides the archive"; fi +OPENN="$( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/arc-list" 2>/dev/null | grep -c '_archive' )" +if [ "$OPENN" -eq 0 ]; then ok "default listing still shows open arcs only"; else bad "default listing leaked archived arcs"; fi + +# YL-08 · omitting --tokens must not look like a measured zero. +LG="$WS/lg"; mkdir -p "$LG" +"$BIN/loop-guard" "$LG" init --max-rounds=99 --token-budget=1000 >/dev/null 2>&1 +"$BIN/loop-guard" "$LG" tick >/dev/null 2>&1 +if "$BIN/loop-guard" "$LG" status 2>/dev/null | grep -q 'unmeasured='; then + ok "a tick with no token count is flagged unmeasured (with a denominator)" +else bad "unmeasured ticks are indistinguishable from a measured zero"; fi +LG2="$WS/lg2"; mkdir -p "$LG2" +"$BIN/loop-guard" "$LG2" init --max-rounds=99 --token-budget=1000 >/dev/null 2>&1 +"$BIN/loop-guard" "$LG2" tick --tokens=400 >/dev/null 2>&1 +"$BIN/loop-guard" "$LG2" tick --tokens=400 >/dev/null 2>&1 +if "$BIN/loop-guard" "$LG2" tick --tokens=400 2>/dev/null | grep -q 'STOP:budget'; then + ok "a reported budget still stops the loop"; else bad "budget guard did not fire"; fi +if "$BIN/loop-guard" "$LG2" status 2>/dev/null | grep -q 'unmeasured='; then + bad "measured ticks wrongly flagged unmeasured"; else ok "measured ticks carry no warning"; fi + +# YL-09 · the seal message must name the condition the code actually tested. +if grep -rq 'install mirror-stack for sealing' "$BIN"/arc-close "$BIN"/close-project 2>/dev/null; then + bad "seal message still blames installation for a PATH test" +else ok "seal message names the tested condition (\`am\` on PATH), not an assumed cause"; fi + echo -if [ "$FAIL" -eq 0 ]; then echo "✅ all gate tests passed"; else echo "⛔ gate tests FAILED"; fi +if [ "$CHECKS" -eq 0 ]; then + echo "⛔ 0/0 — no checks were collected; an empty run is a failure, not a pass" + exit 1 +fi +if [ "$FAIL" -eq 0 ]; then echo "✅ $PASSED/$CHECKS gate checks passed" +else echo "⛔ gate checks FAILED — $PASSED/$CHECKS passed"; fi exit "$FAIL" From 2d350119dc7f7f672fbdea789ec9fa03d797a3b2 Mon Sep 17 00:00:00 2001 From: Mother Seara Date: Tue, 25 Aug 2026 05:15:35 +0900 Subject: [PATCH 2/3] test(gates): tag field-report checks with their item id A revert-to-red control keyed on the success wording can never match a failing line, because the pass and fail branches print different prose. That produced a false KILL on three fixes that were in fact non-vacuous. Both branches now carry the item tag, so the control keys on an identifier. Co-Authored-By: Claude Opus 5 (1M context) --- tests/test_gates.sh | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/tests/test_gates.sh b/tests/test_gates.sh index 6b4d646..76da5f3 100755 --- a/tests/test_gates.sh +++ b/tests/test_gates.sh @@ -304,34 +304,34 @@ NEW="$WS/new"; mkdir -p "$NEW" # YL-01 · a `/` in the topic used to abort sed and leave a half-built project behind. ( cd "$NEW" && YEOUL_PROJECTS="$NEW/projects" "$BIN/yeoul-new" slashed --topic='fast/medium/slow' ) \ >/dev/null 2>&1 -assert "topic containing / scaffolds" 0 $? +assert "[YL-01] topic containing / scaffolds" 0 $? if grep -qF 'fast/medium/slow' "$NEW/projects/slashed/design/spec.md" 2>/dev/null; then - ok "topic substituted literally (not parsed as a sed expression)"; else bad "topic not rendered literally"; fi + ok "[YL-01] topic substituted literally (not parsed as a sed expression)"; else bad "[YL-01] topic not rendered literally"; fi # command substitution in a topic must stay text ( cd "$NEW" && YEOUL_PROJECTS="$NEW/projects" "$BIN/yeoul-new" inj --topic='$(id -u)' ) >/dev/null 2>&1 if grep -qF '$(id -u)' "$NEW/projects/inj/design/spec.md" 2>/dev/null; then - ok "topic is not evaluated as shell"; else bad "topic was evaluated"; fi + ok "[YL-01] topic is not evaluated as shell"; else bad "[YL-01] topic was evaluated"; fi # YL-01 (atomicity) · a failure anywhere must leave no partial project to collide with a retry. STUB="$WS/stubbin"; rm -rf "$STUB"; cp -r "$BIN" "$STUB" printf '#!/usr/bin/env bash\nexit 7\n' > "$STUB/arc-open"; chmod +x "$STUB/arc-open" ROLL="$WS/roll"; mkdir -p "$ROLL" ( cd "$ROLL" && YEOUL_PROJECTS="$ROLL/projects" "$STUB/yeoul-new" doomed --topic='a/b' ) >/dev/null 2>&1 -assert "failed scaffold propagates the real exit code" 7 $? +assert "[YL-01] failed scaffold propagates the real exit code" 7 $? if [ -z "$(ls -A "$ROLL/projects" 2>/dev/null)" ]; then - ok "failed scaffold left nothing behind (retry starts clean)"; else bad "partial scaffold survived"; fi + ok "[YL-01] failed scaffold left nothing behind (retry starts clean)"; else bad "[YL-01] partial scaffold survived"; fi rm -rf "$STUB" # YL-02 · the JOIN prompt is an instruction someone runs from an unknown cwd. JP="$(ls "$NEW"/projects/slashed/design/arcs/*/JOIN_PROMPTS.md 2>/dev/null | head -1)" ATTACH_LINE="$(grep -m1 'arc-attach' "$JP" 2>/dev/null | sed 's/^ *//')" case "$ATTACH_LINE" in - /*|'"/'*) ok "JOIN prompt emits an absolute attach path" ;; - *) bad "JOIN prompt still emits a relative path: $ATTACH_LINE" ;; + /*|'"/'*) ok "[YL-02] JOIN prompt emits an absolute attach path" ;; + *) bad "[YL-02] JOIN prompt still emits a relative path: $ATTACH_LINE" ;; esac # the emitted line must actually run — from a cwd that is not the workspace if ( cd / && eval "$ATTACH_LINE" ) >/dev/null 2>&1; then - ok "the emitted first-action line runs from a foreign cwd"; else bad "emitted attach line does not run from /"; fi + ok "[YL-02] the emitted first-action line runs from a foreign cwd"; else bad "[YL-02] emitted attach line does not run from /"; fi # YL-03 · exit code must report whether the handoff was built. HO="$WS/ho"; mkdir -p "$HO" @@ -342,37 +342,37 @@ HSUM="$(ls "$HARC"/_SUMMARY_*.md 2>/dev/null | head -1)" sedi 's/- (fill in)/- the phase-owned runtime boundary is settled and snapshots are taken at transition edges/' "$HSUM" ( cd "$HO" && "$BIN/arc-close" "$HARC" "GO: build it" --stop=converged ) >/dev/null 2>&1 ( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/build-handoff" hp ) >/dev/null 2>&1 -assert "build-handoff exits 0 on the SUCCESS path (closed arc found)" 0 $? -if [ -f "$HO/projects/hp/dev/TODO.md" ]; then ok "build-handoff produced dev/TODO.md"; else bad "no TODO.md"; fi +assert "[YL-03] build-handoff exits 0 on the SUCCESS path (closed arc found)" 0 $? +if [ -f "$HO/projects/hp/dev/TODO.md" ]; then ok "[YL-03] build-handoff produced dev/TODO.md"; else bad "[YL-03] no TODO.md"; fi ( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/build-handoff" hp ) >/dev/null 2>&1 -assert "a real failure (already exists) still exits non-zero" 1 $? +assert "[YL-03] a real failure (already exists) still exits non-zero" 1 $? # YL-06 · `--all` has to mean all. ARCHN="$( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/arc-list" --all 2>/dev/null | grep -c '_archive' )" -if [ "$ARCHN" -ge 1 ]; then ok "arc-list --all includes archived arcs ($ARCHN)"; else bad "--all still hides the archive"; fi +if [ "$ARCHN" -ge 1 ]; then ok "[YL-06] arc-list --all includes archived arcs ($ARCHN)"; else bad "[YL-06] --all still hides the archive"; fi OPENN="$( cd "$HO" && YEOUL_PROJECTS="$HO/projects" "$BIN/arc-list" 2>/dev/null | grep -c '_archive' )" -if [ "$OPENN" -eq 0 ]; then ok "default listing still shows open arcs only"; else bad "default listing leaked archived arcs"; fi +if [ "$OPENN" -eq 0 ]; then ok "[YL-06] default listing still shows open arcs only"; else bad "[YL-06] default listing leaked archived arcs"; fi # YL-08 · omitting --tokens must not look like a measured zero. LG="$WS/lg"; mkdir -p "$LG" "$BIN/loop-guard" "$LG" init --max-rounds=99 --token-budget=1000 >/dev/null 2>&1 "$BIN/loop-guard" "$LG" tick >/dev/null 2>&1 if "$BIN/loop-guard" "$LG" status 2>/dev/null | grep -q 'unmeasured='; then - ok "a tick with no token count is flagged unmeasured (with a denominator)" -else bad "unmeasured ticks are indistinguishable from a measured zero"; fi + ok "[YL-08] a tick with no token count is flagged unmeasured (with a denominator)" +else bad "[YL-08] unmeasured ticks are indistinguishable from a measured zero"; fi LG2="$WS/lg2"; mkdir -p "$LG2" "$BIN/loop-guard" "$LG2" init --max-rounds=99 --token-budget=1000 >/dev/null 2>&1 "$BIN/loop-guard" "$LG2" tick --tokens=400 >/dev/null 2>&1 "$BIN/loop-guard" "$LG2" tick --tokens=400 >/dev/null 2>&1 if "$BIN/loop-guard" "$LG2" tick --tokens=400 2>/dev/null | grep -q 'STOP:budget'; then - ok "a reported budget still stops the loop"; else bad "budget guard did not fire"; fi + ok "[YL-08] a reported budget still stops the loop"; else bad "[YL-08] budget guard did not fire"; fi if "$BIN/loop-guard" "$LG2" status 2>/dev/null | grep -q 'unmeasured='; then - bad "measured ticks wrongly flagged unmeasured"; else ok "measured ticks carry no warning"; fi + bad "[YL-08] measured ticks wrongly flagged unmeasured"; else ok "[YL-08] measured ticks carry no warning"; fi # YL-09 · the seal message must name the condition the code actually tested. if grep -rq 'install mirror-stack for sealing' "$BIN"/arc-close "$BIN"/close-project 2>/dev/null; then - bad "seal message still blames installation for a PATH test" -else ok "seal message names the tested condition (\`am\` on PATH), not an assumed cause"; fi + bad "[YL-09] seal message still blames installation for a PATH test" +else ok "[YL-09] seal message names the tested condition (\`am\` on PATH), not an assumed cause"; fi echo if [ "$CHECKS" -eq 0 ]; then From 553806bbdabcc5a4b0ad203ebc83e851e9928411 Mon Sep 17 00:00:00 2001 From: Mother Seara Date: Tue, 25 Aug 2026 05:23:48 +0900 Subject: [PATCH 3/3] fix(docs,setup): state who drives the next step; scope the publish guard to what ships MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Continuation contract (docs/BOOTSTRAP_PROMPT.md). The first external user had to type "continue" at every scaffold step, and an arc was left open when they stepped away. The prompt named exactly one legitimate stop (Gate-1) and said nothing about the rest, so an agent reasonably treated each internal step as the end of a request. It now enumerates the stops — failed tool, gate refusal or loop-guard STOP, reproducible blocker, authority not granted, a decision that changes the goal, close/archive — and says everything between them continues as commentary. The contract governs sequencing, not judgment. Sealing, PASS/KILL, convergence, graduation and publishing stay with the operator, and "keep going" is never authority to decide one; convergence and the verdict are read off role evidence by the relay, not by a fixed rule. Like the rest of the prompt it is advisory, and deliberately unenforced — an enforced "keep going" would be a mechanism for skipping gates. The enforcement runs the other way, in the scripts. Publish guard (setup/pre-publish-check.sh). It walked the whole working tree, so the gitignored KNOWLEDGE_INDEX.md that every arc close appends to made it report "personalization leaks found" about a file git will never publish — red on a clean checkout, for content that cannot ship. A red that does not mean what it says gets ignored, which is worse than no guard. All three scans now run over `git ls-files`, with a walk fallback outside a work tree. Scoping a scan is how a guard goes quietly blind, so the denominator is enforced and printed: "✓ clean (41 files scanned)", and a scan over zero files fails the run. Verified by planting each violation it claims to catch — Hangul, private absolute path, over-claim copy — in a tracked file and confirming each one fires. The first attempt at this passed all three while inspecting nothing: `python3 - < --- docs/BOOTSTRAP_PROMPT.md | 24 +++++++++++++++++ setup/pre-publish-check.sh | 53 ++++++++++++++++++++++++++++---------- 2 files changed, 64 insertions(+), 13 deletions(-) diff --git a/docs/BOOTSTRAP_PROMPT.md b/docs/BOOTSTRAP_PROMPT.md index 7b3fd33..9599763 100644 --- a/docs/BOOTSTRAP_PROMPT.md +++ b/docs/BOOTSTRAP_PROMPT.md @@ -36,6 +36,23 @@ HARD RULES (do not violate — these are the point of the tool) error you can quote; say "applying the discipline, I suspect X" for your own reasoning. Never borrow the tool's credibility for a judgment call. +CONTINUATION (who drives the next step) +- Once I have approved the finalized spec, run the arc through to its next honest stopping point without + asking me to say "continue". Report progress as commentary; give a final answer only when you stop. +- Scaffolding a project, opening an arc, assigning roles, issuing and collecting tickets, running the + bounded rounds, drafting the summary, and the two-phase close are internal steps. None of them is the + end of a request. If the roles run as subagents, my approval of the spec covers spawning them — confirm + the runtime once at the interview, not once per role and not once per round. +- Stop and hand back only for: a tool or connection that actually failed; a gate refusal or loop-guard + STOP; a genuine technical blocker you can reproduce; an action needing authority you were not given + (anything outward-facing, destructive, or costly); a decision that changes the goal rather than + executing it; the arc reaching close/archive. Name which one when you stop. +- This governs sequencing, not judgment. It does NOT let you seal a pre-registration, declare PASS/KILL, + call convergence, graduate, or publish on your own — those stay mine, and "keep going" is never + authority to decide one. Convergence and the verdict are read off the role evidence by you as relay and + put to me; a fixed rule must not stand in for that reading. If continuing would require one of those + decisions, that is a stop. + Confirm you've read METHODOLOGY.md and are ready, then ask me for the seed idea (or which project to resume). ``` @@ -47,3 +64,10 @@ Confirm you've read METHODOLOGY.md and are ready, then ask me for the seed idea (blank-refusal, KILL-defense, verify-gate). If you skip a gate manually, the enforcement no longer holds — that is why the enforced pieces exist as tools, not just instructions. - Runtime-independent: any agent that can run shell commands and (optionally) call MCP tools can follow this. +- The CONTINUATION block exists because the first external user had to type "continue" at every scaffold + step, and an arc left open when they stepped away (field report, 2026-08-24). The prompt previously named + exactly one legitimate stop (Gate-1) and said nothing about the rest, so an agent reasonably treated each + step as the end of a request. Like the rest of this prompt it is **advisory**: nothing enforces it, and it + deliberately does not — an enforced "keep going" would be a mechanism for skipping the gates. What is + enforced is the opposite direction (blank-refusal, KILL-defense, verify-gate, loop-guard), which is why + continuing is safe to ask for: the stops that matter are in the scripts, not in the agent's manners. diff --git a/setup/pre-publish-check.sh b/setup/pre-publish-check.sh index c770d6e..7b2af1f 100755 --- a/setup/pre-publish-check.sh +++ b/setup/pre-publish-check.sh @@ -10,40 +10,65 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" REPO="$(cd "$SCRIPT_DIR/.." && pwd)" FAIL=0 +# 🔴 Scope the scans to what would actually ship. These used to walk/grep the whole working tree, +# so a gitignored runtime artifact (KNOWLEDGE_INDEX.md, written by every arc close) made the guard +# report "personalization leaks found" about a file git will never publish. A red that does not +# mean what it says gets ignored, which is worse than no guard. +publishable_files() { # publishable_files [-z] + if git -C "$REPO" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git -C "$REPO" ls-files ${1:+-z} + else + if [ "${1:-}" = "-z" ]; then (cd "$REPO" && find . -type f -not -path './.git/*' -print0) + else (cd "$REPO" && find . -type f -not -path './.git/*'); fi + fi +} + echo "── 1) personalization leak scan ──" # Generic de-personalization checks (no internal codenames are enumerated here, so this file ships clean): # (a) any Hangul — this repo is English-only, so any Korean text is a leak; # (b) private absolute paths (/home/... or /data/...) that must not ship. # Intentional localizations are allowed (README_KO.md, *.ko.md, docs/ko/); Hangul anywhere else is a leak. # Hangul detection via python (portable — GNU grep's -P is unavailable on macOS/BSD). -HANGUL="$(python3 - "$REPO" <<'PY' +# NB: the file list goes through a temp file, not a pipe. `python3 - < "$FILELIST" +# 🔴 the denominator, enforced in the shell. A scan over zero files reports "clean" for every +# section, so an empty list has to fail the run outright — not merely print a note that the +# surrounding `if [ -n "$LEAKS" ]` then treats as clean. +SCANNED="$(grep -c . "$FILELIST" || true)" +if [ "${SCANNED:-0}" -eq 0 ]; then + echo " ✗ scanned 0 files — an empty scan is a failure, not a pass" + FAIL=1 +fi +HANGUL="$(python3 - "$REPO" "$FILELIST" <<'PY' import os, re, sys root = sys.argv[1]; h = re.compile('[\uac00-\ud7a3]') # Hangul syllables (escaped → this file stays Hangul-free) -for dp, _, fns in os.walk(root): - if '/.git' in dp or dp.endswith('/ko') or '/ko/' in dp: continue - for f in fns: - if f.endswith(('_KO.md', '.ko.md')): continue - p = os.path.join(dp, f) - try: - if h.search(open(p, encoding='utf-8', errors='ignore').read()): print(p) - except Exception: pass +rels = [x.rstrip('\n') for x in open(sys.argv[2], encoding='utf-8') if x.strip()] +if not rels: sys.exit('pre-publish: file list is empty - refusing to report clean') +for rel in rels: + if rel.endswith(('_KO.md', '.ko.md')) or rel.startswith('ko/') or '/ko/' in rel: continue + p = os.path.join(root, rel) + try: + if h.search(open(p, encoding='utf-8', errors='ignore').read()): print(p) + except Exception: pass PY )" -PATHS="$(grep -rlE '(/home|/data)/[A-Za-z]' "$REPO" --include='*.sh' --include='*.py' --include='*.md' --include='*.json' --exclude-dir=.git 2>/dev/null || true)" +PATHS="$( (cd "$REPO" && tr '\n' '\0' < "$FILELIST" | xargs -0 -r grep -lE '(/home|/data)/[A-Za-z]' --include='*.sh' --include='*.py' --include='*.md' --include='*.json' -- ) 2>/dev/null || true)" LEAKS="$(printf '%s\n%s\n' "$HANGUL" "$PATHS" | grep -v '^$' | sort -u)" if [ -n "$LEAKS" ]; then echo "$LEAKS" | sed 's/^/ /' echo " ✗ personalization leaks found (Hangul or private absolute path)" FAIL=1 else - echo " ✓ clean" + echo " ✓ clean ($SCANNED files scanned)" fi echo "── 2) over-claim copy scan ──" # Superlatives / unfalsifiable marketing the discipline forbids in our own docs. CLAIM_RE='revolutionary|world.?first|state.of.the.art|SOTA|guarantee[sd]?|never fails|solves? (the )?reproducibility|best.in.class|game.?chang|unprecedented|10x|breakthrough' -if CLAIMS="$(grep -rInE -i "$CLAIM_RE" "$REPO" --include='*.md' --exclude-dir=.git 2>/dev/null \ - | grep -vE '/setup/pre-publish-check\.sh')"; then +if CLAIMS="$( (cd "$REPO" && tr '\n' '\0' < "$FILELIST" | xargs -0 -r grep -InE -i "$CLAIM_RE" --include='*.md' -- ) 2>/dev/null \ + | grep -vE '(^|/)setup/pre-publish-check\.sh')"; then echo "$CLAIMS" | sed 's/^/ /' echo " ✗ over-claim copy found (rewrite to bounded, honest wording)" FAIL=1 @@ -51,6 +76,8 @@ else echo " ✓ clean" fi +rm -f "$FILELIST" + echo "── 3) empty-scaffolding guard ──" if [ -d "$REPO/examples" ] && [ -n "$(find "$REPO/examples" -type f 2>/dev/null | head -1)" ]; then echo " ✓ examples/ present"