Skip to content

fix(harness): plan-first sessions are ordinary sessions with map tools [SAP-3143] - #835

Open
gwitwer wants to merge 3 commits into
mainfrom
gwitwer/planner-sessions-are-ordinary
Open

fix(harness): plan-first sessions are ordinary sessions with map tools [SAP-3143]#835
gwitwer wants to merge 3 commits into
mainfrom
gwitwer/planner-sessions-are-ordinary

Conversation

@gwitwer

@gwitwer gwitwer commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Primary change type

  • Bug fix
  • Documentation
  • Feature
  • Tests
  • Dependency update
  • Maintenance or refactor

Problem and motivation

Opening a project's Plan Agents session on main launches Claude Code with a replacement system prompt that says "Do not act as a coding or implementation agent. Do not scaffold agents, edit application source code, run implementation tasks, or deploy software." The session Studio puts the user into cannot build an agent, so the primary flow is dead until the planner profile is removed. Maintainer ruling: there is no special planning session type; a project session is an ordinary session that also has the map tools.

Summary and scope

Stop-the-bleed. Nothing is redesigned.

  • src/server/index.ts: a map-planner session uses the served prompt (loadSystemPrompt() with the same DEFAULT_SYSTEM_PROMPT fallback) and appends the map context, instead of replacing the prompt with AGENT_MAP_PLANNER_SYSTEM_PROMPT. Appendix order: rehydration brief, map context, focused project context.
  • src/profiles/agent-map-planner.ts: prohibition sentences removed. What remains names the three map tools (read, validate, propose), says to keep the map current when work changes the architecture, says to build directly when asked, and countermands the served prompt's mandatory first-reply orientation (the session already showed its own at SessionStart). The SessionStart copy no longer says "not to implement it yet". Export names unchanged.
  • src/core/planning-session.ts, src/core/planner-greeting.ts: the non-Claude onboarding sentence and the greeting prompt say "plan and build" instead of "plan".
  • Tests re-pointed (agent-map-mcp-wiring.test.ts, claude-settings.test.ts); none deleted. The wiring stub now carries the orientation clause and asserts the override lands after it.

Out of scope: removing the planner session type, routes, and client state. That is the follow-up PR on gwitwer/no-planner-sessions, and Yash's #820/#826 stack deletes the planner wholesale; this PR keeps every export so that stack rebases cleanly.

Restrictions searched for and what was done

Location Finding Action
profiles/agent-map-planner.ts "Do not act as a coding or implementation agent. Do not scaffold agents, edit application source code, run implementation tasks, or deploy software." Removed
profiles/agent-map-planner.ts SessionStart copy "not to implement it yet ... Studio will create focused execution sessions" Rewritten: the session plans and builds
profiles/default.ts served prompt Mandatory first-reply orientation offering the sample project Countermanded by the appended map context (review round 1)
core/planning-session.ts focused context "Do not propose architecture or invoke mutation tools before the user asks you to." Kept. It sequences map mutations behind the user's request; it does not forbid implementation
core/planning-session.ts non-Claude onboarding "can plan agents ... together" Now "can plan and build"
core/planner-greeting.ts greeting prompt "will plan the agents ..." and "Do not propose an architecture ... before the user replies." Copy now "plan and build"; the one-turn greeting scope is kept
Tool allowlists None. The planner session's generated MCP config carries agent-map, sapiom, and sapiom-dev; no allowedTools or disallowedTools in planning-session.ts, planner-greeting.ts, or the server Nothing to remove

Before / after: generated system-prompt.txt for a planner session

Measured on a real server (node dist/cli/bin.js <tree> --port 4131 --state-root <scratch>), reading <state-root>/generated/<session>/system-prompt.txt after POST /api/projects/:id/planner-sessions.

Before (origin/main build, 1462 bytes, full file):

You are the project planning agent running in Agent Studio.
...
Do not act as a coding or implementation agent. Do not scaffold agents, edit
application source code, run implementation tasks, or deploy software.

<agent-map-planner-context>
...

After (this branch at 984ae91, 6772 bytes, full file):

You are the coding agent running in Agent Studio. This is not a stock coding session ...
[served authoring prompt, unchanged, ~78 lines]

This session is also the project planning agent for Agent Studio.
... agent_map_read for the current map, agent_map_validate to check a change, agent_map_propose to record one.

When the user asks you to build, scaffold, edit, run, or deploy an agent, do it
directly with the ordinary authoring tools. Keep the Agent Map current when
your work changes the architecture.

Skip the general first-reply orientation described above: this session showed
its own orientation at start, so answer the user's first message directly and
do not suggest an unrelated sample project.

<agent-map-planner-context>
...

grep -c "Do not act as a coding\|Do not scaffold" on the after file: 0.

Screenshot: Plan Agents session asked to scaffold

Playwright against the real server with page.on("pageerror") attached (zero page errors). Prompt sent: "Scaffold a new agent in this project called hello-hotfix that just logs hello world. Do it now, locally only. Do not deploy and do not run anything remotely."

Plan Agents session scaffolds hello-hotfix

The session scaffolded hello-hotfix/ through sapiom-dev, typechecked it, ran the check step, did a stub Local Run, and the new agent appeared in the rail under the project. Nothing was deployed. Taken at b93a343, before the round-1 orientation override; the prompt text change since then is shown above.

Related work

Related issue or discussion: SAP-3143. Superseded by Yash's #820 / #826 stack, which deletes the planner; this is the stopgap until that stack lands.

Validation

pnpm --filter "@sapiom/harness..." build : pass
pnpm typecheck : pass
pnpm test (vitest main) : 3304 passed, 13 failed in 7 files; perf config 10 passed
  the same 13 fail on unmodified origin/main in the same worktree (workflow-registry,
  system-graph-relationships, system-graph-watcher, agent-source-discovery, workspace-watcher,
  system-graph-freshness): filesystem-watcher tests, none touch prompts
vitest agent-map-mcp-wiring, claude-settings, planning-session, planner-greeting : 57 passed
mutation: old server code restored, wiring test fails on startsWith(codingPrompt) : killed
pnpm test:ui (VITE_MOCK=1) : 541 passed, 5 failed first run; all 5 pass on --last-failed
  (known flake band). Mock mode never launches a real session, so it cannot observe this change.
real server: planner session prompt captured before/after (above); Plan Agents session scaffolds (screenshot)

Tests and documentation

Tests re-pointed as described. Documentation: changeset only; no user docs describe the planning profile.

Compatibility and release impact

  • Breaking or externally visible changes: the 0.13.0 read-only planning profile is gone. A Plan Agents session can now edit source and deploy agents when asked. No migration; persisted planner sessions resume with the new prompt.
  • Changeset: added, @sapiom/harness minor.

Security

  • I have not included secrets, credentials, private data, or unsanitized logs.
  • This pull request does not publicly disclose a suspected vulnerability. I
    will follow the
    Security Policy for
    private reporting.

AI assistance

  • I did not use AI assistance for this change.
  • I used AI assistance and have described it below.

Claude Code (Fable 5.1) wrote the change and tests under direction, ran the real-server verification, and drafted this body. Verified by the build, typecheck, test runs, mutation check, and the real-server prompt capture and screenshot above.

Checklist

  • I read CONTRIBUTING.md, and this contribution follows the direct-PR or issue-first policy.
  • This pull request addresses one focused problem and contains no unrelated cleanup.
  • I added or updated tests, or explained above why tests are not applicable.
  • I ran the relevant build, typecheck, lint, and test commands, or explained
    any N/A checks above.
  • I updated documentation for user-facing changes, or marked it N/A above.
  • I added a Changeset for a published-package change, or explained why it is not applicable.
  • I can explain and maintain every submitted change, including any AI-assisted work.

did_not_work

  • The appended context tells the model to keep the Agent Map current when its work changes the architecture. In the real run it scaffolded the agent but did not call agent_map_propose; the map pane still read "Nothing generated yet". Prompt guidance only, not enforced.
  • pnpm test is not green on this machine even on unmodified origin/main (13 watcher tests). Reported, not investigated here.
  • Prettier flags server/index.ts and claude-settings.test.ts, but both already fail prettier --check on origin/main, so this PR does not reformat them.
  • The first wiring assertion matched a phrase that wraps across two lines in the template literal and failed; re-pointed to a single-line phrase.
  • The screenshot was not retaken after the round-1 orientation override; the prompt diff is shown instead.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DUq2aqxS5DQtG6aPvoYtn7

…s [SAP-3143]

A project's Plan Agents session launched with a replacement system prompt
that forbade scaffolding, editing, running, and deploying, so the session
Studio opened a user into could not build an agent. A project session is
an ordinary session that also has the Agent Map tools: it now loads the
served authoring prompt like every other session and appends the map
context (read, validate, propose; keep the map current; build directly
when asked). The prohibition sentences and the "not to implement it yet"
SessionStart copy are removed. Tests re-pointed to the appended shape.

Stopgap until Yash's #820/#826 stack deletes the planner profile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DUq2aqxS5DQtG6aPvoYtn7
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Review — PR #835 (round 1)

No confidentiality issues: the changeset and comments name only Sapiom and Studio surfaces.

1. The served authoring prompt forces a generic orientation into every planner session

packages/harness/src/server/index.ts:580 now prepends the served coding prompt to planner
sessions. That prompt ends with a mandatory clause (packages/harness/src/profiles/default.ts:73):

In your very first reply this session, orient the person before you get to their actual
request … Suggest ONE concrete first step … (e.g. the bundled order-triage sample project) …

The planner's own context says the opposite — buildFocusedPlannerContext
(packages/harness/src/core/planning-session.ts:205): "Let the user's first real message be the
first visible conversation turn" — and Claude Code planner sessions already show
AGENT_MAP_PLANNER_SESSION_START_MESSAGE as SessionStart orientation.

Failure: a user opens Plan Agents, reads "Use this session to plan and build", types a goal, and the
first assistant reply is the generic Studio orientation ending in an offer to inspect the bundled
order-triage sample — instead of engaging with the project map. Two orientations, one of them about
an unrelated sample project.

The tests can't catch this: agent-map-mcp-wiring.test.ts:143 mocks loadSystemPrompt with a stub
string, and the new negative assertions only target the planner profile's own phrases
("In your first response, briefly explain"), not the served prompt's "In your very first reply this session".

Fix: have AGENT_MAP_PLANNER_SYSTEM_PROMPT explicitly override the first-reply orientation (it is
appended after the served prompt, so it can), and assert it — e.g. a stub coding prompt in the test
that contains an orientation clause, with the planner appendix countermanding it.

2. Patch level reverts behavior that shipped as a Minor

packages/harness/CHANGELOG.md (0.13.0, minor) published the guarantee this PR removes:

Agent Map planners now launch with a dedicated architecture-planning profile instead of inheriting
Studio's scaffold/run/deploy coding profile.

.changeset/planner-sessions-are-ordinary.md reverses that at patch. A consumer on ~0.14 picks
this up automatically and a session they were told was planning-only can now edit source and deploy
agents. That is a capability expansion, not a bug fix — make the changeset minor and say plainly
in the body that the 0.13.0 read-only planning profile is gone, so anyone who relied on it sees it.

3. Copy did not travel: non-Claude planners still say planning-only

The new "plan and build" copy reaches Claude Code sessions only. Two surfaces still say the session
plans and does not build:

  • packages/harness/src/core/planning-session.ts:205 — the onboardOnFirstResponse sentence, used
    for every non-claude-code harness (planning-session.ts:356): "briefly explain that you and the
    user can plan agents, responsibilities, data flow, resources, and connectors together".
    A Codex planner user is told this session plans; the model has just been told it may build.
  • packages/harness/src/core/planner-greeting.ts:245-247plannerGreetingPrompt still says
    "Respond as the project planning agent" and "Explain that you and the user will plan…". This
    path is compatibility-only for persisted sessions and API clients, but it is still reachable and
    still contradicts the new behavior.

Both are the same root cause as the SessionStart change and belong in this PR.

Nits

  • PR body is Placeholder body; full body follows once media is pushed. — the deterministic labeler
    requires .github/pull_request_template.md, so this will land contribution: incomplete.
  • agent-map-mcp-wiring.test.ts:252 / :351 now assert loadSystemPrompt call counts (1, then 2).
    Counting a shared mock across two launches breaks on any unrelated added launch in this test; the
    prompt-content assertions above already cover the behavior.

Verdict: request changes — finding 1 is a user-visible regression in the flow this PR is meant
to fix, and finding 2 mislabels a capability expansion as a patch.

Review round 1: the served prompt's mandatory first-reply orientation
would have run in planner sessions on top of the SessionStart message,
so the appended map context now countermands it. The non-Claude onboarding
sentence and the greeting prompt say plan and build. Changeset is minor,
since the 0.13.0 read-only planning profile is removed. The wiring test
stub carries the orientation clause and asserts the override lands after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DUq2aqxS5DQtG6aPvoYtn7
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Review — PR #835 (round 2, follow-up)

Delta reviewed: b93a343984ae91 (one commit, 5 files).

Earlier findings — all three addressed

  1. First-reply orientation conflict — fixed. profiles/agent-map-planner.ts:23-25 now
    countermands the served prompt's orientation clause, and it is appended after it
    (server/index.ts:604-609 order: brief → map context → focused context). The wiring test's
    stub prompt now carries a real orientation clause and asserts the countermand appears later
    in the file (agent-map-mcp-wiring.test.ts:234-242) — that assertion would have failed
    before this commit.
  2. Patch masking a capability expansion — fixed. Changeset is minor and the body names
    the removal of the 0.13.0 read-only profile.
  3. Copy did not travel — fixed. planning-session.ts:206 and planner-greeting.ts:246
    both say "plan and build" now.

Nit 2 (call-count assertions) is addressed, though expect(loadSystemPrompt).toHaveBeenCalled()
at :359 is now vacuous — it was already true at :250. Harmless.

New findings

None that break or leak.

Nits

  • agent-map-planner.ts:23 asserts "this session showed its own orientation at start", but
    sessionStartSystemMessage is only set for mode === "created" && harness === "claude-code"
    (planning-session.ts:357-364) and SessionManager.resume never re-supplies it
    (session-manager.ts:830-835). A resumed planner session gets a prompt claiming an
    orientation the user never saw. Dropping the clause ("Answer the user's first message
    directly and do not suggest an unrelated sample project.") loses nothing.
  • Carried over: the PR body still does not follow .github/pull_request_template.md (no
    Primary change type / Validation / Checklist sections), so the deterministic labeler will
    flag contribution: incomplete.

Verdict: approve — the push resolved every round-1 finding; the two remaining items are
cosmetic.

…nnot make

Review round 2 nit: sessionStartSystemMessage is only set on create, so the
appended context no longer asserts the user saw an orientation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DUq2aqxS5DQtG6aPvoYtn7
@gwitwer
gwitwer marked this pull request as draft September 5, 2026 22:05
@gwitwer
gwitwer marked this pull request as ready for review September 5, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant