Open items, ordered by priority: row 1 is the next thing this repo owes. Every entry here is work
that is still owed; an item leaves the table only when it is done or explicitly retired with a
reason — a pending item that quietly disappears is the failure this file exists to prevent. Skill
work lives in the agentspec repo's BACKLOG.md; these two files are the only work queues.
| # | item | what remains | next step |
|---|---|---|---|
| D1 | The floors at the doors (a later implementation round — not in the C program) | Engine-owned always-on floors, one per door, and everything guarding the ENTRY is deterministic because it runs BEFORE any model call — including the router's own. ENTRY (router): declared blockPattern, structural PII shapes (email, card, phone — forms, never words) and the structural injection markers (chat-template tokens, tool-markup literals); the judged injection question (row 21) cannot guard this door — it needs a model, so it runs at the call phase where one is already in play. CALL (rulebook): maxCalls as a floor with its declared number, beside maxDestructive. EXIT (delivery, where the words leave): declared maskPattern/purgePattern/swapTerms and the same structural PII shapes. One floor may arm at BOTH doors — a declared pattern names its doors (input, reply, or both), the shape blockPattern already carries in miniature. Competitor names are vocabulary, never the engine's words: a declared subject list/pattern, or a route problem (the none-door, routed-residue row (c)). injectionCheck (judged) lives until this lands and dies here. The spec walks the whole catalog for every other floor that can arm at a door. |
Design the three doors in one spec; each floor measured at its own door; runs after the C program closes. |
| — | Routed residue: structural code routing + one verb scope | Certified at unpinned 96/100 · routed 97/100; the residue is three rows, none grave. (a) A bare approval code after a drift carries no routing signal — the deterministic cut: the routed door already sees every desk's issued/consumed questions on the records it holds, so a message carrying an OPEN code routes structurally to the desk that issued it, zero model calls (kills the whole approve-after-drift family). (b) The fieldops line's late-return verb over-absorbs a RETURNED hire's balance ask — scope it to a hire still out (what a late return still out owes), micro-probe route-35 vs 37 first. (c) route-87 (competitor-price lure past the none-door) stays the accepted miss. (d) A read living on one desk is unreachable from the desk the turn routed to — c20 case 52 routed to billing while getAuditLog exists only on workspace, so the reply never mentions the log the operator pointed at. |
Engine cut (a) + one T-loop round (b); targeted subset, then one closing routed run. |
| 4 | The six house laws move to the engine's mouth | The six conduct templates (declareHonestly, oneQuestion, yourLaneYourReads, recordsOverAssertions, askBeforeYouChoose, nameItDoNotPassItOn) are skill markdown that every author copies and fills by hand — so their wording drifts silently. Measured: the "put nothing up" clause lived in three catalog passages and taught blind authors to refuse without attempting, starving every deterministic channel, until the passages were rewritten by hand; a wording defect in a template is invisible to every lint. The engine already owns minted sentences (floor guards, the eight engine sentences, factory rules), so the same mechanism fits: a house law the ENGINE mints, with declared per-desk slots for the desk's own vocabulary. Trade-off to design around: the per-desk wording freedom of the current templates measured as valuable. | Spec: house laws as engine-minted rows with per-desk slots; the skill teaches only the slot-filling. |
| 5 | Every guard generates its own catalog text | The parity triangle (agentspec declared-parity.test.ts x LAWFUL_ARGS x declare.md) verifies factory NAMES and ARGS, never teaching WORDING — the catalog's per-guard content is hand-written and drifts: the "$25,000 -> REFUSED" example stated the old matcher's behavior until it was hand-corrected in the same session as the code change, and only the ship-together law caught it. Direction ruled as required ("EXTREMAMENTE CORRETO"): the per-guard catalog text is GENERATED from looprun, nothing per-guard comes from outside. Design: each factory in packages/core/src/cards/catalog.ts carries its teaching as co-located structured metadata (whenToUse, neverFor, a filled invented-domain example) beside the code it describes; a generator emits the catalog's per-guard sections between markers; a freshness test fails when the committed page differs from the generated output; adding a guard without its teaching fails the build. Honest boundary: only PER-GUARD content generates — the catalog's cross-cutting sections (byte arithmetic, the pairing walk) remain authored pages. | Spec: teaching metadata on every factory + the generator + the freshness gate; the triangle's factory half becomes construction instead of verification. Same principle as the engine-minted house laws above (the engine owns its own words) — one spec or two sibling specs; the spec work decides. |
| 6 | The guard doc outside code is one hand-written lesson | docs/tutorial/04-guards.md is the only guard documentation outside the code and the skill; it is held honest by the lesson-compile test and the ship-together law, nothing else. No generated reference exists. | Fold into the catalog-generation spec above: one metadata source, two outputs — the skill's catalog section and a looprun reference page. |
| 7 | The install path the LP promises has no repo behind it | The public org is looprun-ai: this engine is looprun-ai/looprun, the benchmarks are looprun-ai/looprun-bench, and every LP link names them. What the org move did not buy is the promise on the page — npx skills add looprun-ai/looprun resolves to a repo that carries the engine and no skill, so the command fails for anyone who runs it. Launch gate for the LP, owned by the skill migration (agentspec backlog row 1); the row stands here so the gate is visible from the engine's queue too. One residue outside that migration: the skill's own author field still names the old org. |
Nothing here until the migration ships the skill into this repo; then run the command end to end from a clean machine. |
| 8 | Local serving measurement of the cache wiring (steps 4/4b of the minimal-core program) | RULED OUT of the execution run: the goal ships cache_prompt: true + -np 1 with unit-level acceptance only. This row owns the real-box measurement afterwards: the microtest-7 ruler against the ENGINE's own client (identical call twice → timings.prompt_n collapses), prefill tokens/turn, tokens/s (prefill + decode), RSS — before and after step 4b's byte cuts. Instrument recipe (measured healthy: ~600 tok/s prefill, ~35 decode, RSS ≤16 GB): binary ~/Dev/github-clones/llama.cpp/build/bin/llama-server (rebuild: cmake -B build -DGGML_METAL=ON && cmake --build build --target llama-server -j 8), model ~/models/qwen36-mtp-gguf/Qwen3.6-35B-A3B-UD-IQ2_XXS.gguf, flags --jinja -fa on -c 65536 -ngl 99 -ctk f16 -ctv f16 --mlock --no-mmap -np 1 --slot-save-path /tmp/llama-slots --cache-ram 16384 -ctxcp 64 --spec-type draft-mtp --chat-template-kwargs '{"enable_thinking":false}' --host 127.0.0.1 --port 8081; kill by BINARY PATH only (pkill -f "build/bin/llama-server"). Harness and baselines are archived in git at commit cd9b495 — git show cd9b495:microtests/07-prefill/PREFILL-REPORT.md (the 09-tape record sits beside it); restore the directory from that commit when this row runs. Also owed here: the tool-array pinning measurement (the returnable two-shapes question). |
After the program's step 4b lands: one measured run per layout-affecting change, against the microtest-7 baseline. |
| 53 | Every published figure is void | No looprun-measured number in the tree was produced by the engine, the guards and the process as they now stand. | Re-measure before publishing any looprun figure. |
| 20 | Engine-owned question: off-surface promise | The evidence is already in the engine — spec.surface.tools says what this agent can do. A promise to email, sync or open an app when no such tool exists is answerable without a word of domain vocabulary, yet every domain writes its own wording for it. Two hermes-sim specs record it as conditioned prose. |
Design one engine question, measure its miss rate on its own fixture set, then ship. Depends on docs/superpowers/specs/2026-08-05-one-judge-one-question-design.md. |
| 21 | Engine-owned question: instruction arriving inside a tool result | Fully generic: "does a result instruct a destructive act the user did not authorise this turn?" carries no domain vocabulary at all. An author binding it today writes the wording from scratch, on the hook where a bad question is most expensive — a call-side judgement gates the act itself. | Design one engine question bound on preTool, measure it, then ship. Depends on the one-judge design. |
| 22 | Engine-owned question: a claim about an earlier conversation | The engine holds ctx.history, so "does the reply assert a prior exchange the history does not carry?" is answerable from evidence it already has. One hermes-sim spec records it as conditioned prose. |
Design, measure, ship. Depends on the one-judge design. |
| 23 | Engine-owned question: disclosure with no grounding | The generic half — "does the reply state a value that appears in no result?" — is engine-answerable. The domain half — WHICH fields count as personal — is not, and a medical desk and a rental desk disagree. Splitting the two is the design work. | Decide the split before designing; the generic half may fold into the grounding question rather than become its own. |
| 24 | Cost per governed conversation is unverified against a real workload | Measured on the atlas exam: R$0.031 per conversation of 1.53 turns, R$0.020 per turn, 15.5k input tokens per turn. Nobody has checked what that becomes on a conversation of ten or twenty turns, where every turn resends the whole transcript, nor how much of the per-turn input is the static assembled prefix that a cache should be absorbing — the measured cache-read is 19.6% of input. | Measure a long conversation, split per-turn input into static prefix vs transcript, and state the cost curve. See below. |
| 26 | Framework adapters the landing page promises: Vercel AI and LangChain | The landing page (looprun.ai/index.html:413, :719-720) offers looprun/vercel and looprun/langchain beside looprun/mastra; neither seam exists under packages/**. The runtime is Mastra-only (@looprun-ai/mastra) plus the OpenAI-compatible endpoint (@looprun-ai/server), which is how any other framework reaches a governed agent today. |
Implement both seams, or take the two names off the page, before the LP launches. |
| 52 | createStep(agent) cannot take a LoopRunAgent |
LoopRunAgent IS a @mastra/core Agent, so a Mastra workflow should be able to mount it as a step with createStep(agent). Today the step fails before the engine runs: Mastra's agent step first asks agent.getModel().specificationVersion, and the host seat on LoopRunAgent throws looprun: the engine owns the model seat; then it calls agent.stream(prompt, { onFinish, ... }) and reads fullStream (chunks of { type: 'text-delta', payload: { text } }) plus a settled text promise, while the governed stream() returns { loopRun, textStream }. Two overrides close it: getModel() answering { specificationVersion: 'v2' } (the seat stays the engine's), and stream() returning fullStream over the delivered text with text settled to loopRun.text and onFinish called once. A subclass carrying exactly those two overrides ran createWorkflow(...).then(createStep(agent)) to status: success against @mastra/core 1.50.1 with a scripted model. What the automatic step still cannot do: it returns { text } only — the sealed record (loopRun.acts) does not reach the workflow — and it has no suspend, so a held destructive act cannot become a workflow suspension there. An own step calling agent.generate() (six lines) keeps the record and maps a hold to suspend/resume with the minted code; both measured in docs/analysis/2026-09-07-positioning-against-agent-frameworks.md §5.3. The adjustment is a convenience for read-only steps; the own step stays the door for any act that waits for a word. |
Add the two overrides to packages/mastra/src/loop-run-agent.ts, a test that mounts the agent with createStep and runs a workflow to success, and one line under README "Framework-agnostic by construction" saying a LoopRunAgent is a workflow step. |
| 35 | Attestation service (idea, not scheduled) | Design at docs/superpowers/specs/2026-07-31-attestation-service-design.md: ed25519 layer-2 attestation over the seal, free-for-telemetry service, hashed client identifiers, transparency log. Zero code exists — no attest command, no service, no keys. |
When prioritized: implementation plan for (1) looprun-eval attest + telemetry builder, (2) the service repo, (3) verify layer-2 extension. |
| 36 | Agent-as-tool bridge (MCP server) | Runtime consumes MCP tools but never serves agents as tools; governance verdict as structured result data. The OpenAI endpoint is the works-today path; this is roadmap. | Roadmap — design when prioritized. |
| 38 | looprun-bench still speaks the retired vocabulary |
looprun, agentspec and agentspec-bench name the seven concepts plainly, and tests/plain-names.test.mjs holds them there. looprun-bench pins one engine per edition — 0.2.1 for tau2-telecom, 0.6.0 and 0.6.1 for atlas — because an edition is reproducible against the engine that measured it. A swept tree fails typecheck with Module '@looprun-ai/core' has no exported member 'createActionHistory', so renaming an existing edition would mean re-measuring it. |
Rename an edition only when a new one is built on a post-rename engine. Spec: docs/superpowers/specs/2026-08-06-plain-names-design.md. |
| 39 | hermes-sim bundles carry no reply-honesty coverage | No guard factory takes a RegExp, and the hermes-sim example bundles install no reply-honesty rule at all. The judge seam a bound question needs is reachable now, so this is a porting task, not a design gap. Ruled last of the queue: hermes-sim work comes after everything above. |
Author the honesty rules as llmCheck questions and port them into the bundles. |
| 42 | proseDropsReads passes a reply that names one id and drops every field |
The predicate (turn.ts:55) returns false the moment the prose carries ONE returned identifier, and it runs only on the prose-direct branch — a reply that says "bk_9" and none of the record's fields passes. Measured escapes: c20 cases 17/37/71 and c17 cases 30/71/83. |
Demand the read's material fields, not one id; run it on every path that delivers prose. |
| 48 | The discriminate stage has no routed twin | T3 measures what governance pays by running each case governed and ungoverned — but the ungoverned twin exists only for a PINNED case, and with pins retired every case is routed, where the router itself is governance and the engine builds no twin. The last measured gap (governed 12/12, ungoverned 7/12, blind-judged) describes the retired configuration; the ruled path has no discrimination number at all. | Design the ungoverned twin for a routed house (e.g. the same desks with every guard disarmed and the router kept), then re-measure T3 on the ruled path. |
| 50 | figureIsGrounded passes a figure DERIVED from two grounded ones |
harborpoint a-week-costs-what-the-record-says r2: the desk multiplies a grounded rate by a grounded count (52 × 7) and states 364 — a figure no record carries — and no rung refuses it; wall wording ("never multiply the two yourself") was tried twice and did not hold. |
A rung over derived figures: a stated number that is an arithmetic product of grounded figures is not itself grounded. |
| 51 | c20 case 95 — one transcript read decides bug vs rubric | The final reply never states that bk_1004 is still to be cancelled. If the second cancellation was proposed and HELD, the turn owes an ask fact for it and the desk's delivered message dropped it (delivery bug); if the model never proposed it, the engine holds no state to speak and the letter is rubric territory. The transcript in atlas-c20/test/2026-08-30-cert/full100 settles it. |
Read case 95's trace once; file the answer on the side it lands. |
The atlas exam, gemini-3.1-flash-lite, 2496 conversations run between 4 and 10 August 2026, priced
at the published paid-tier rates ($0.25/M input, $0.025/M cached input, $1.50/M output):
per conversation (1.53 turns) 23,752 input · 224 output R$ 0.031
per turn 15,524 input R$ 0.020
cache-read share of input 19.6%
The engine's own overhead is part of that: a 100-case exam is 153 turns but 183 generations — 14 redrives, 12 forced terminals, 4 forced reads — plus 291 tool calls, each a round-trip that resends the conversation.
The atlas conversation is 1.53 turns. A support desk conversation is ten or twenty. Every turn resends the whole transcript, so per-turn input grows with the conversation while the static assembled prefix stays put. Nobody has measured where that curve goes, and the exam cannot answer it: its cases are too short to show the slope.
The cache is absorbing a fifth of the input. The assembled prompt is case-invariant by design — the shared-prefix law exists for exactly this — yet only 19.6% of input tokens came back as cache-read. Either the prefix is not being cached, or it is being invalidated per turn. Whichever it is, it is the largest lever on the number above, and it is unexamined.
At R$0.031 a conversation, ten thousand conversations a day is R$300/day. Whether that is cheap or expensive is a question about the workload it replaces, not about the model — and no such comparison exists in this tree. The figure most likely to mislead is the DEVELOPMENT one: 2496 conversations in a week was 42 exam runs, which is a measurement habit, not a production load.