Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/early-maps-start.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@sapiom/harness": minor
---

Automatically run the configured planning agent to inspect existing project agents and draft a reviewable Agent Map proposal when a fresh planner opens an unstarted map. The automatic turn may consume provider credits and remains preemptible by user input.
20 changes: 15 additions & 5 deletions packages/harness/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ from what its other components do on their own (the app's product analytics, and
`npx @sapiom/mcp@latest` fetching and running the local MCP server each session):

Planner-session bootstrap makes no additional network request. Its focused
context, greeting coordination, FIFO, and lifecycle persistence stay inside the
local server. Existing outbound surfaces remain the system-prompt fetch below,
the coding agent's ordinary provider traffic, and opt-in telemetry.
context, automatic empty-map inspection turn, FIFO, and lifecycle persistence
stay inside the local server. Existing outbound surfaces remain the system-prompt
fetch below, the coding agent's ordinary provider traffic, and opt-in telemetry.

- **System prompt, on every session start** — an unauthenticated
`GET https://api.sapiom.ai/v1/harness/system-prompt`, so the Studio conventions
Expand Down Expand Up @@ -93,9 +93,19 @@ is project-scoped:
live/resumable planner or creates one. Use `{ "mode": "fresh" }` to always
create a new planner.
- `POST /api/projects/:projectId/planner-sessions/:sessionId/messages` durably
accepts planner input and releases it FIFO after greeting resolution.
accepts planner input and releases it FIFO after startup-turn resolution.
- `POST /api/projects/:projectId/planner-sessions/:sessionId/greeting/retry`
retries an eligible failed automatic greeting.
retries an eligible failed automatic startup turn.

When a newly created planner sees no confirmed revision, active proposal, or
project build plan, it dispatches one server-authored startup turn after CLI
readiness. The planner reads the authoritative map, inspects the project
read-only for existing agents and evidence-backed relationships, validates the
result, and creates a proposal for the user to review. It never confirms or
implements that proposal automatically. Live, resumed, and rehydrated sessions
preserve their prior startup state instead of replaying the turn. The automatic
turn uses the configured planning provider and may consume provider credits; a
real user message takes priority and skips or preempts unfinished startup work.

Planner metadata is part of the session registry. Its input FIFO and greeting
attempt state live at
Expand Down
188 changes: 157 additions & 31 deletions packages/harness/src/core/planner-greeting.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ import * as path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";

import type { AnalyticsEvent, HarnessSession } from "../shared/types.js";
import type { SessionManager } from "./session-manager.js";
import {
SessionInputGuardRejectedError,
type SessionManager,
} from "./session-manager.js";
import {
PlannerGreetingCoordinator,
PlannerGreetingRetryUnavailableError,
Expand Down Expand Up @@ -85,7 +88,7 @@ describe("PlannerGreetingCoordinator", () => {
await fs.rm(root, { recursive: true, force: true });
});

it("persists one ready-gated greeting, then releases accepted input FIFO", async () => {
it("lets accepted user input preempt an in-flight startup turn", async () => {
const coordinator = new PlannerGreetingCoordinator({
root,
sessionManager: manager,
Expand All @@ -98,7 +101,15 @@ describe("PlannerGreetingCoordinator", () => {

await coordinator.enqueue(session.id, "first user message");
await coordinator.enqueue(session.id, "second user message");
expect(submitted).toEqual([greeting]);
expect(submitted).toEqual([
greeting,
"first user message",
"second user message",
]);
expect(session.planning?.greeting).toEqual({
status: "skipped",
reason: "user-proceeded",
});

const localPrompt = coordinator.decorateLocalEvent(
event(session.id, "prompt.submitted", { prompt: greeting }),
Expand All @@ -107,20 +118,19 @@ describe("PlannerGreetingCoordinator", () => {
prompt: greeting,
plannerOrigin: "infrastructure",
});
expect(coordinator.redactForTelemetry(localPrompt).payload).not.toHaveProperty(
"prompt",
);
expect(coordinator.redactForTelemetry(localPrompt).payload).toMatchObject({
planner: true,
origin: "infrastructure",
});
expect(
coordinator.redactForTelemetry(localPrompt).payload,
).not.toHaveProperty("prompt");

await coordinator.onEventPersisted(
event(session.id, "turn.completed", { assistantText: "What should we build?" }),
);
expect(submitted).toEqual([
greeting,
"first user message",
"second user message",
]);
expect(session.planning).toMatchObject({
greeting: { status: "delivered", messageId: "event-turn.completed" },
greeting: { status: "skipped", reason: "user-proceeded" },
queuedInputIds: [],
});
const durable = JSON.parse(
Expand All @@ -132,6 +142,73 @@ describe("PlannerGreetingCoordinator", () => {
expect(durable.inputs).toEqual([]);
});

it("keeps an uncertain failed startup classified when the user proceeds", async () => {
let calls = 0;
manager.submitInput = async (_id, text) => {
submitted.push(text);
calls += 1;
if (calls === 1) throw new Error("uncertain PTY write");
return true;
};
const coordinator = new PlannerGreetingCoordinator({
root,
sessionManager: manager,
deliveryTimeoutMs: 60_000,
});
await coordinator.register(session, { emptyProject: true, mode: "created" });
const greeting = submitted[0]!;
expect(session.planning?.greeting).toEqual({
status: "failed",
retryable: true,
errorCode: "injection_failed",
});

await coordinator.enqueue(session.id, "continue with my request");
expect(submitted).toEqual([greeting, "continue with my request"]);
const localPrompt = coordinator.decorateLocalEvent(
event(session.id, "prompt.submitted", { prompt: greeting }),
);
expect(localPrompt.payload).toMatchObject({
prompt: greeting,
plannerOrigin: "infrastructure",
});
expect(coordinator.redactForTelemetry(localPrompt).payload).toMatchObject({
planner: true,
origin: "infrastructure",
});
expect(session.planning?.greeting).toEqual({
status: "skipped",
reason: "user-proceeded",
});
});

it("keeps a staged guard rejection classified for a delayed hook", async () => {
manager.submitInput = async (_id, text) => {
submitted.push(text);
throw new SessionInputGuardRejectedError(true);
};
const coordinator = new PlannerGreetingCoordinator({
root,
sessionManager: manager,
deliveryTimeoutMs: 60_000,
});
await coordinator.register(session, { emptyProject: true, mode: "created" });
const greeting = submitted[0]!;
expect(session.planning?.greeting).toEqual({
status: "failed",
retryable: false,
errorCode: "session_exited",
});

const localPrompt = coordinator.decorateLocalEvent(
event(session.id, "prompt.submitted", { prompt: greeting }),
);
expect(localPrompt.payload).toMatchObject({
prompt: greeting,
plannerOrigin: "infrastructure",
});
});

it("rejects a planner session identity that could escape the queue root", async () => {
session = plannerSession("../outside-planner-root");
const coordinator = new PlannerGreetingCoordinator({
Expand Down Expand Up @@ -556,7 +633,7 @@ describe("PlannerGreetingCoordinator", () => {
).toEqual({ schemaVersion: 1, inputIds: [] });
});

it("bounds pending readiness, then drains its durable FIFO when readiness arrives", async () => {
it("lets user input preempt a pending startup turn, then drains at readiness", async () => {
vi.useFakeTimers();
session.ready = false;
const coordinator = new PlannerGreetingCoordinator({
Expand All @@ -566,6 +643,10 @@ describe("PlannerGreetingCoordinator", () => {
});
await coordinator.register(session, { emptyProject: true, mode: "created" });
await coordinator.enqueue(session.id, "queued while booting");
expect(session.planning?.greeting).toEqual({
status: "skipped",
reason: "user-proceeded",
});
await vi.advanceTimersByTimeAsync(101);
await (coordinator as unknown as { writes: Map<string, Promise<unknown>> })
.writes.get(session.id);
Expand All @@ -581,6 +662,43 @@ describe("PlannerGreetingCoordinator", () => {
expect(session.planning?.queuedInputIds).toEqual([]);
});

it("uses a short readiness deadline without cutting off a longer startup mapping turn", async () => {
vi.useFakeTimers();
session.ready = false;
const coordinator = new PlannerGreetingCoordinator({
root,
sessionManager: manager,
readinessTimeoutMs: 100,
deliveryTimeoutMs: 1_000,
});
await coordinator.register(session, { emptyProject: true, mode: "created" });
await vi.advanceTimersByTimeAsync(101);
await (coordinator as unknown as { writes: Map<string, Promise<unknown>> })
.writes.get(session.id);
expect(session.planning?.greeting).toEqual({
status: "failed",
retryable: true,
errorCode: "session_not_ready",
});

session = plannerSession("session-2");
await coordinator.register(session, { emptyProject: true, mode: "created" });
expect(session.planning?.greeting.status).toBe("generating");
expect(submitted).toHaveLength(1);

await vi.advanceTimersByTimeAsync(101);
expect(session.planning?.greeting.status).toBe("generating");

await vi.advanceTimersByTimeAsync(900);
await (coordinator as unknown as { writes: Map<string, Promise<unknown>> })
.writes.get(session.id);
expect(session.planning?.greeting).toEqual({
status: "failed",
retryable: true,
errorCode: "delivery_timeout",
});
});

it("contains timer persistence rejection with only a bounded local classification", async () => {
vi.useFakeTimers();
session.ready = false;
Expand Down Expand Up @@ -940,23 +1058,31 @@ describe("PlannerGreetingCoordinator", () => {
});

describe("plannerGreetingPrompt", () => {
it("keeps the automatic greeting scoped to collaborative planning and one question", () => {
const empty = plannerGreetingPrompt(true);
const existing = plannerGreetingPrompt(false);
for (const prompt of [empty, existing]) {
expect(prompt).toContain("project planning agent");
expect(prompt).toContain("agents, responsibilities, data flow, resources, and connectors");
expect(prompt).toContain("exactly one open-ended question");
expect(prompt).toContain("Do not propose an architecture");
expect(prompt).toContain("invoke tools");
}
expect(empty).toContain(
"what kind of agent architecture the user wants to build",
);
expect(existing).toContain("current plan exists");
const attempted = plannerGreetingPrompt(true, "attempt-private-1");
expect(attempted).toContain("Internal attempt ID: attempt-private-1");
expect(attempted).toContain("Never mention this ID");
expect(empty).not.toContain("attempt-private-1");
it("keeps the automatic request single-line, human-readable, and review-only", () => {
const empty = plannerGreetingPrompt(true, 1);
expect(empty).toContain("Agent Studio automatic request");
expect(empty).toContain("Inspect this project for existing agents");
expect(empty).toContain("unconfirmed Agent Map proposal");
expect(empty).not.toContain("\n");
expect(empty).not.toContain("agent_map_read");
expect(empty).not.toContain("Internal attempt ID");
expect(empty).not.toContain("retry");
});

it("keeps the legacy existing-plan greeting conversational", () => {
const existing = plannerGreetingPrompt(false, 1);
expect(existing).toContain("current Agent Map");
expect(existing).toContain("review, extend, or change");
expect(existing).not.toContain("\n");
expect(existing).not.toContain("agent_map_propose");
});

it("uses a human-readable ordinal to keep retry prompts distinct", () => {
const initial = plannerGreetingPrompt(true, 1);
const retry = plannerGreetingPrompt(true, 2);
expect(retry).not.toBe(initial);
expect(retry).toContain("Automatic retry 1 of 2");
expect(plannerGreetingPrompt(true, 3)).toContain("Automatic retry 2 of 2");
expect(retry).not.toContain("attempt-private");
});
});
Loading
Loading