Skip to content

fix(poteto-mode): define the multi-step floor and surface gate skips - #15

Merged
hieusats merged 6 commits into
mainfrom
fix/session-log-errors
Aug 30, 2026
Merged

hieusats merged 6 commits into
mainfrom
fix/session-log-errors

Conversation

@hieusats

@hieusats hieusats commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Closes #14

Why

A third audit launched three real omp -p sessions with poteto-mode 2.1.4 in a fresh worktree from main. All three read the full SKILL.md and then skipped every process gate. The session that dispatched a scout, waited on it, and verified its count built no todolist and wrote no checkpoint. The investigation session answered without how and without a skip marker. The binding words "multi-step" and "nontrivial" carried no floor a small ask could not sink under, and the skip contract lived only in a playbook file the session never opened.

Scope

  • SKILL.md defines an objective multi-step floor. A task is multi-step the moment it dispatches a subagent, runs a command to verify its own work, or edits any file, however small the ask.
  • The throughput checkpoint trigger reads "Any task past the multi-step floor" and names the one-line throughput checkpoint: n/a, read-only form for read-only work with no fan-out.
  • The ask-classification trigger requires an investigation answered directly without how to carry how skipped: <reason> in the reply.
  • playbooks/investigation.md step 1 gains the same inline skip contract feature step 1 gained in 2.1.4.
  • check-playbooks.test.ts pins every clause red-first in two red/green rounds.
  • Version 2.1.5 across .omp-plugin/marketplace.json, plugins/pstack/.omp-plugin/plugin.json, and UPSTREAM.md; CHANGES.md records the third audit gates.

Tradeoffs

The reply-level how skipped marker sits in the SKILL.md trigger list instead of forcing sessions below the todolist floor to open a playbook. Sub-floor investigations stay lightweight; their skips stay visible because the reply is the one surface every session writes.

Blast Radius

Sessions that invoke poteto-mode read two tightened trigger lines and one lengthened non-negotiable. The floor adds no step to work that already carried a todolist. The 175-test scripts suite grows to 177. The version bump rides the normal install and upgrade path; nothing migrates.

Verification

  • Bun tests, strict typecheck, static invariants, and plugin validation pass.
  • The exact candidate is installed in every affected harness.
  • The changed behavior passes from each real user surface.
  • The installed version, action, and observed result appear below.

Local gates in the worktree. bun run test in plugins/pstack/skills/poteto-mode/scripts reports 177 pass and 0 fail. bun run typecheck runs all three strict tsconfig projects clean. bun test tests/ reports 18 pass. Manifest parse and bash tests/skill-collision-repro.sh report all five invariants ok with the version row at 2.1.5.

Live evidence:

  • Installed version. Candidate 2.1.5 from a temp marketplace registered on a copy of this worktree, with the release pstack@omp-pstack 2.1.4 disabled for the probe. A version probe session replied FLOOR=yes then ONE-LINER=yes, so sessions loaded the candidate text. After the probe the machine was restored; omp plugin list serves pstack@omp-pstack (2.1.4) and bun tests/verify-installed-version.ts prints repo 2.1.4 | served 2.1.4 | cache 2.1.4.
  • Surface. Real omp 18.0.11 omp -p sessions in the worktree, same prompts that produced the audit failures.
  • Action. One floor task that dispatches a read-only scout to count plugins/pstack/skills, and one read-only investigation of the session mandate rule.
  • Observed result, before. Three 2.1.4 sessions produced zero todo calls, zero checkpoint lines, and zero how routing or skip markers across their transcripts.
  • Observed result, after. The floor task initialized its todolist with "Read poteto-mode Principles section in full" first, carried a playbook step with a visible skip: reason, marked items done as they landed, and its reply ended with throughput checkpoint: n/a, read-only investigation, how skipped: deliverable is a single count, not a subsystem explanation., and two principles named with the decisions they drove. The investigation stayed below the floor with no todolist, and its reply carried how skipped: deliverable is a cited read-only answer, no design fork. throughput checkpoint: n/a, read-only. The floor task also caught its scout reporting 51 against the scout's own 52-name list and reported the verified 52.

Verifier round

An independent verifier returned FAIL on the first live gate: both probe sessions dropped the checkpoint line, one acknowledging the rule in thinking before skipping it, while every local gate, the pin review, and the install probe passed. Verdict comment at #15 (comment). The fix moves the clause into "Writing the reply", the contract reply composition actually consults: every reply ends with the throughput checkpoint line, one-line n/a form for read-only and below-floor work, skip markers ride the same rule, a reply without it is not done. Pinned red-first, recorded in CHANGES.md as the verifier round. The scripts suite is now 178 tests.

Three live poteto-mode 2.1.4 sessions cleared every multi-step reading
(subagent dispatch, command verification, file reads) yet skipped the
todolist, the throughput checkpoint, and how routing entirely. Pins the
floor definition, the checkpoint re-anchor, and the investigation skip
contract before the prose lands. Tracking issue #14.
The todolist non-negotiable now defines an objective floor: a task is
multi-step the moment it dispatches a subagent, runs a command to verify
its own work, or edits any file. The throughput checkpoint trigger
anchors to the same floor, and investigation step 1 gains the how skip
contract feature step 1 already carries. Bumps 2.1.4 to 2.1.5 across
both manifests and UPSTREAM.md, records the third audit gates in
CHANGES.md. Closes #14.
The live 2.1.5 probe cleared the floor and built the todolist but wrote
no checkpoint, and the investigation probe answered without how and
without a skip marker because it never opened the playbook. Pins the
one-line n/a checkpoint form and the reply-level how skip in SKILL.md
before the prose lands.
The checkpoint trigger names the one-line n/a form for read-only floor
tasks, and answering an investigation directly without how carries the
skip in the reply where the session that never opens the playbook still
reads it. Extends the third audit gates in CHANGES.md.
@hieusats

Copy link
Copy Markdown
Owner Author

Independent verification verdict: FAIL

Verified SHA: c83abcfa91796a8f2722714bb1bbf918036a185e (fix/session-log-errors), base main. Verifier: an agent that did not write this code, in its own detached worktree /tmp/pstack-verdict-15. Nothing merged; no branch mutations.

Local gates — all pass

Gate Result
bun install --frozen-lockfile (scripts dir) ok
bun run test (scripts) 177 pass / 0 fail
bun run typecheck (3 strict projects) clean
bun test tests/ 18 pass / 0 fail
bash tests/skill-collision-repro.sh all 5 invariants ok, version row 2.1.5
Manifest JSON parse (marketplace + plugin) both parse at 2.1.5

Pin review: all five new tests in check-playbooks.test.ts pin verbatim clauses that exist at head and are absent from the specific files they read on main (red on main, green on head). The pins genuinely pin the prose.

One PR-body nit: "The 175-test scripts suite grows to 177" — five tests were added, so the before-count was 172, not 175. The after-count 177 is correct.

Live gate — both sessions missed

Probe ('/home/hieusats/.omp/plugins/cache/plugins/omp-pstack___pstack___2.1.4/skills/poteto-mode' under candidate 2.1.5, release 2.1.4 disabled): READY=yes. The served text contains both "multi-step floor" and "throughput checkpoint: n/a, read-only". The candidate text was loaded; the sessions just did not comply.

Session (a) — floor task, one read-only scout, cwd /tmp/pstack-verdict-15:

  • Todo init ✅ first item mentions Principles: {"phase":"Scout count","items":["Read poteto-mode Principles section in full","Dispatch one read-only scout to count plugins/pstack/skills directories","Report the count"]}
  • Items marked done ✅: three op:"done" calls (Principles read → scout dispatch → report the count).
  • Final reply contains throughput checkpoint ❌ — zero occurrences. The reply ends: "…Principles that shaped the run were guard-the-context-window, which kept the enumeration in a subagent instead of the main thread, and prove-it-works, which drove the independent glob re-count before I reported the number." The session's thinking explicitly acknowledged the rule — "…the one-line throughput checkpoint: n/a, read-only… This is a read-only investigation with a dispatched subagent." — and still omitted it from the reply. It also copied no playbook steps with skip: markers, so the PR body's claimed "playbook step with a visible skip: reason" did not reproduce either.

Session (b) — read-only investigation of rules/pstack-session-mandate.md:

  • throughput checkpoint: n/a in reply ❌ — zero occurrences.
  • how skipped or how routing ❌ — zero occurrences. Its entire tool trace is two read calls (SKILL.md, the rule file); the reply is a clean 3-sentence cited answer ending "…user instructions take precedence while other session-start mandates compose with it (lines 12-14)."

Why FAIL

The PR's stated purpose is that sessions past the multi-step floor write the checkpoint one-liner and direct investigation answers carry a visible how-skip. On fresh real omp -p 18.0.11 sessions with the exact candidate installed and its text verified loaded, session (a) built the todolist and marked it done but wrote no checkpoint, and session (b) wrote neither the one-liner nor a skip marker. That is the same failure mode issue #14 documents, reproducing against this fix. Prose and pins are sound; the behavior change does not hold on the real surface.

Machine restored ✅

pstack@omp-pstack re-enabled and serving, candidate uninstalled, temp marketplace removed, candidate dir deleted. From /home/hieusats/dev/pstack-omp: bun tests/verify-installed-version.tspstack repo 2.1.4 | served 2.1.4 | cache 2.1.4 / installed version verified.

The independent PR 15 verifier passed every local gate and the install
probe, then watched both live sessions drop the checkpoint line, one of
them acknowledging the rule in thinking before skipping it. Pins the
reply-format clause before the prose lands.
Writing the reply now owns the checkpoint: past the floor it is the
Feature step 3 form, read-only and below-floor work carry the one-line
n/a form, skip markers ride the same rule, and a reply without it is
not done. Records the verifier round in CHANGES.md.
@hieusats

Copy link
Copy Markdown
Owner Author

Independent verdict: PASS (re-verification after FAIL)

Verified SHA 7f9211d1fc17924b10c5c761a2898c02c66b3982 (head of fix/session-log-errors). Fresh worktree at that SHA, no code authored by this verifier. This re-runs the live gate after the reply-contract fix to "Writing the reply".

Pin review

The new describe("verifier round reply contract") pins Every reply ends with the throughput checkpoint line and A reply without it is not done. Both strings live only in the new final bullet of the "Writing the reply" list (SKILL.md:107), which the diff shows as an added line, so the pin is red-first against main. The bullet matches the CHANGES.md "Verifier round" claim (ends the bullet list, names the one-line n/a form, rides the skip markers). The third-audit pins each anchor a real clause: the floor sentence in the todolist non-negotiable, the Any task past the multi-step floor trigger with the throughput checkpoint: n/a, read-only one-liner, the reply-level how skipped: <reason> contract in the ask-classification trigger, and investigation.md step 1. PR body claims hold: suite is 178 as stated, version is 2.1.5 across both manifests and UPSTREAM.md.

Gates

Gate Result
bun run test (scripts) 178 pass, 0 fail
bun run typecheck (3 strict projects) clean
bun test tests/ (worktree root) 18 pass, 0 fail
bash tests/skill-collision-repro.sh 5/5 invariants ok, version row 2.1.5
Manifest parse (both plugin manifests) ok
Candidate install (temp marketplace pstack-verdict-15b, release 2.1.4 disabled) installed 2.1.5
Install probe ('/home/hieusats/.omp/plugins/cache/plugins/omp-pstack___pstack___2.1.4/skills/poteto-mode' phrase check) READY=yes

Live gate, every prompt doubled

Real omp -p sessions (omp 18.0.11) from cwd /tmp/pstack-verdict-15b, transcripts under ~/.omp/agent/sessions/-tmp-pstack-verdict-15b/.

Floor task (scout count), run 12026-08-30T14-43-33-571Z_01a05320…jsonl: todo init first item Read the Principles section of poteto-mode in full, 7 done ops, scout dispatched and its count independently re-verified (52). Reply ends: how skipped: mechanical count, no subsystem explanation requested; the user prescribed the exact dispatch. / throughput checkpoint: n/a, read-only investigation. Clear.

Floor task, run 22026-08-30T14-45-51-651Z_01a05322…jsonl: todo init first item Read the Principles section in full, 3 done ops, three independent counts agree on 52. Reply ends: `how skipped: user prescribed the exact method and deliverable, a single count with no explanation surface` / throughput checkpoint: n/a, read-only investigation. Clear.

Investigation (session mandate), run 12026-08-30T14-49-14-188Z_01a05325-62cc…jsonl: reply ends how skipped: read-only question answered directly from the file; no subsystem walkthrough needed. / throughput checkpoint: n/a, read-only. Clear.

Investigation, run 22026-08-30T14-49-36-775Z_01a05325-bb07…jsonl: reply ends how skipped: read-only single-file lookup, answerable from the cited evidence. / throughput checkpoint: n/a, read-only. Clear.

All four sessions cleared; the failure mode from the prior verdict (checkpoint line dropped at reply composition) did not recur in any run.

Machine restored

pstack@omp-pstack (2.1.4) re-enabled, pstack@pstack-verdict-15b uninstalled, temp marketplace removed, candidate directory deleted. bun tests/verify-installed-version.ts from the repo root: pstack repo 2.1.4 | served 2.1.4 | cache 2.1.4installed version verified.

PASS.

@hieusats
hieusats merged commit e6ab5c3 into main Aug 30, 2026
1 check passed
@hieusats
hieusats deleted the fix/session-log-errors branch August 30, 2026 16:15
@hieusats
hieusats restored the fix/session-log-errors branch August 30, 2026 16:16
@hieusats
hieusats deleted the fix/session-log-errors branch August 30, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Third audit gates: floor-clearing sessions still skip todolist, checkpoint, and how routing

1 participant