Run component tests from the Sidekick

architecture.md

@@ -6431,6 +6431,34 @@ no project fallback. The Sidekick's native provider keeps the captured project
 `agentCwd`, or the caller's explicit dependency, and its existing identity and
 configuration.
 
+### Response-owned component testing
+
+`repl/sidekick/testing.ts` executes REPL.Test documents in fresh children of the
+command host captured during profile assembly. Each child installs one real
+useTesting session and executes the finalized REPL profile, including immutable
+host declarations and the selected TypeScript loader. The CLI supplies the
+existing nested testing-host installation factory with its captured host
+settings. Response and entry bindings or live registrations never enter those
+roots. Child outcomes travel back as Results so a clean host child cannot
+reassign response failure ownership to its parent. The response owns halting
+and joining every child; document cleanup finishes before the next starts.
+
+The root's existing admission state holds a process-local test slot beside its
+entry reservation. Both submitters consult it; testing checks that reservation
+and session readiness, then takes the slot synchronously before discovery. The
+slot remains held through joined cleanup. Stop cancels response-owned tests and
+leaves admitted ordinary entries with their session owner; Exit joins both
+owners before returning.
+
+Actual Testing results, execution completion and drained output are collected
+separately. Ordered completion callbacks distinguish a valid zero-test policy
+rejection from earlier document failure without inspecting error names. Fatal
+persistence, Files, disclosure and owning-cleanup failures retain their existing
+authority. Completed detached reports use the existing designated-control
+observation records; unmatched intents remain unknown and inspection does no
+work. The exact report and status fold live in the
+[REPL testing contract](specs/repl-spec.md#testing-session-components).
+
 ### Source ownership in retained entries
 
 A source position identifies authored bytes, not which invocation used them.

packages/cli/src/cli.ts

@@ -2984,7 +2984,37 @@ function* dispatch(
       // line carried and nothing more: the profile settles the fallback, and
       // whether anybody answers to that name is the first explicit message's
       // question rather than this command's.
+      const moduleLoader = yield* ComponentModuleLoader.get();
+      const testAgentWorker = yield* readWorkerCommand();
+      const bundledPlugin = yield* BundledRunPlugin.expect();
       const profile = yield* assembleReplProfile(replStack, plugins, {
+        *testingHarness(includes) {
+          return testHarnessInstallation(
+            testingExecutionHost({
+              componentModuleLoader: moduleLoader,
+              includes: [...includes],
+              plugins: plugins.plugins,
+              pluginArgs: plugins.args,
+              secretDetection: true,
+              installService,
+              bundledPlugin,
+              testAgentWorker,
+              planDeclaration: (request) =>
+                planComponentDeclaration({
+                  surface: "component",
+                  includes,
+                  plugins,
+                  context: request.context,
+                  ...(request.planWriterRoot === undefined
+                    ? {}
+                    : { planWriterRoot: request.planWriterRoot }),
+                  host: request.host,
+                  installElicitation: request.installElicitation,
+                }),
+            }),
+            [testAgentChildDeclaration()],
+          );
+        },
         includes: command.config.include,
         ...(command.config.sidekickAgent === undefined
           ? {}

packages/cli/src/documents/sidekick/entries.md

@@ -1,7 +1,7 @@
 # Entries
 
 Read this before preparing an entry for the first time, or whenever you need the
-authoring pattern for putting source into the shared draft.
+authoring pattern for preparing, testing and reusing session components.
 
 ## Applies to
 
@@ -140,6 +140,104 @@ arguments for continuation, or `null` when there is no more. If a later read's
 cursor refuses because the answer changed, restart the read without the cursor.
 Nothing these reads return authorizes a repository change.
 
+### Test, fix and reuse a session component
+
+When asked to create and test a component, write its definition and a companion
+Markdown test through an ordinary entry. This Greeting writes `greeting.txt`
+when invoked and returns its greeting. Authoring its file performs neither:
+
+````xmd
+<Let as="code" select="code">
+```xmd
+<Let as="definition" value={"<File path=\"greeting.txt\">Hello from this session.</File>Hello from this session."} />
+<File path="components/Greeting.md">{definition}</File>
+<Let as="test" value={"<Test name=\"greeting\"><Greeting as=\"actual\" /><AssertEquals actual={actual} expected=\"Hello from this session.\" /></Test>"} />
+<File path="components/Greeting.test.md">{test}</File>
+```
+</Let>
+<REPL.EntryInput>{code}</REPL.EntryInput>
+<REPL.Submit />
+````
+
+Follow the receipt's read instruction and check the actual File outcomes. A
+completed entry alone does not prove either write succeeded. `greeting.txt`
+does not exist yet, and authoring the test has run no assertion.
+
+After successful writes, request the document explicitly:
+
+````xmd
+<REPL.Test path="components/Greeting.test.md" />
+````
+
+Feedback contains a passed document and its named assertion. Invocation created
+`greeting.txt`. Testing executes authored effects under the command's ordinary
+permissions; it does not append an entry or consume the draft.
+
+To see a real failure, replace the expectation through another ordinary entry,
+then request the directory suite:
+
+````xmd
+<Let as="code" select="code">
+```xmd
+<Let as="test" value={"<Test name=\"greeting\"><Greeting as=\"actual\" /><AssertEquals actual={actual} expected=\"wrong\" /></Test>"} />
+<File path="components/Greeting.test.md">{test}</File>
+```
+</Let>
+<REPL.EntryInput>{code}</REPL.EntryInput>
+<REPL.Submit />
+<REPL.Test path="components" as="tests" />
+````
+
+The completed report is `failed`. Read the named assertion's full diagnostic,
+actual and expected values before changing the file. Fix that expectation and
+request a fresh suite:
+
+````xmd
+<Let as="code" select="code">
+```xmd
+<Let as="test" value={"<Test name=\"greeting\"><Greeting as=\"actual\" /><AssertEquals actual={actual} expected=\"Hello from this session.\" /></Test>"} />
+<File path="components/Greeting.test.md">{test}</File>
+```
+</Let>
+<REPL.EntryInput>{code}</REPL.EntryInput>
+<REPL.Submit />
+<REPL.Test path="components" />
+````
+
+A passed report belongs to this request. Earlier failed reports stay as facts.
+You can now invoke the component in a later ordinary entry without includes,
+pasted definitions or a restart:
+
+````xmd
+<Let as="code" select="code">
+```xmd
+<Greeting />
+```
+</Let>
+<REPL.EntryInput>{code}</REPL.EntryInput>
+<REPL.Submit />
+````
+
+The entry returns `Hello from this session.`. A person can also enter
+`<Greeting />` manually. Both use the selected component; configured directories
+still precede session components during a test, including a directory suite.
+
+Use `as` when this response needs the structured report. Capturing and leaving
+it uncaptured both deliver the same feedback and publish no chat. To show it to
+the person explicitly, write:
+
+```xmd
+<REPL.Test path="components" as="tests" />
+<Output><Json value={tests} /></Output>
+```
+
+`no-tests` means no document matched, or otherwise valid documents reached no
+Test. It is not a pass. A parse or runtime error before the first Test is a
+failed document; inspect its `error` and any partial `report`. An admission or
+discovery refusal produces a diagnostic instead of a completed report. Wait for
+entry or test cleanup to finish before requesting another run. Stop cancels and
+joins response-owned tests; it leaves an already admitted ordinary entry alone.
+
 ## Common mistake
 
 Writing the entry source directly into your response, or inside a fence marked

packages/cli/src/documents/sidekick/environment.md

@@ -23,6 +23,12 @@ answer.
   `<REPL.Syntax names={["Greeting"]} />` before invoking a new component: a
   configured definition can win. A selected malformed definition fails; fix it
   rather than assuming another candidate runs.
+- Request real tests with `<REPL.Test path="components/Greeting.test.md" />`
+  or `<REPL.Test path="components" />`. Targets are relative to the session
+  workspace even inside a response Dir; absolute and escaping paths refuse.
+  A directory selects `**/*.test.md` recursively and retains configured-first
+  component lookup. Tests have effects and use the command's ordinary permissions.
+  Read `<Guide.Entries />` for the create, test, fix and reuse sequence.
 - A nested `<Dir>` selects another known, authorized directory for its content;
   later siblings return to the workspace. To copy project input, enter its
   explicitly known project path, capture the read there, then write the copy

packages/cli/src/repl-profile.ts

@@ -5,6 +5,9 @@
  * id before the includes-dependent Plan declaration and entry vocabulary exist.
  */
 
+import { replTesting } from "./repl/sidekick/testing.ts";
+import type { ReplTesting } from "./repl/sidekick/testing.ts";
+
 import type { AcpxProviderDependencies } from "@executablemd/acp";
 import { agentIdentityComponents } from "@executablemd/core";
 import type { PermissionMode } from "@executablemd/core";
@@ -52,6 +55,7 @@ export interface ReplProfileSeams {
  * parameter: a caller states what it knows and leaves out the rest.
  */
 export interface ReplProfileFacts extends ReplProfileSeams {
+  readonly testingHarness?: (includes: readonly string[]) => Operation<ExecutionInstallation>;
   readonly includes?: readonly string[];
   /**
    * The name `--sidekick-agent` carried, or none when the line named none.
@@ -93,6 +97,7 @@ export interface ReplProfileAssembly {
 
 /** One finalized execution profile, shared by validation, descriptions and entries. */
 export interface ReplExecutionProfile {
+  readonly testing?: ReplTesting;
   readonly projectCwd?: string;
   /** Where a declared name is looked for, outermost first. */
   readonly includes: readonly string[];
@@ -252,7 +257,7 @@ export function* assembleReplProfile(
         installElicitation: noElicitationOfItsOwn,
       });
 
-      return Object.freeze({
+      const profile = Object.freeze({
         includes,
         projectCwd,
         vocabulary: entryVocabulary(includes, plugins),
@@ -282,6 +287,10 @@ export function* assembleReplProfile(
         // assembly rather than a second one standing in for it.
         sidekickAssembly: sidekick,
       });
+      return Object.freeze({
+        ...profile,
+        testing: replTesting(host, profile, workspace, facts.testingHarness),
+      });
     },
   });
 }

packages/cli/src/repl/program.ts

@@ -1,3 +1,4 @@
+import { testForSidekick } from "./sidekick/admission.ts";
 /**
  * The command, as one scope.
  *
@@ -1068,6 +1069,13 @@ function* drive(
         return Ok(established);
       },
       root: sidekickRoot,
+      test: (path) =>
+        testForSidekick(path, {
+          held: admitting,
+          session: current,
+          testing: options.profile.testing,
+          ...(reopening.blocked === undefined ? {} : { blocked: reopening.blocked }),
+        }),
       submit(at, journal): Operation<Result<SubmitReceipt>> {
         return submitForSidekick(at, {
           session: {

packages/cli/src/repl/sidekick/admission.ts

@@ -49,6 +49,8 @@
  * guessed at here, and why reopening resolves before it admits anything else.
  */
 
+import type { ReplTesting, ReplTestReport } from "./testing.ts";
+
 import { ensure, Err, Ok } from "effection";
 import type { Operation, Result } from "effection";
 
@@ -76,6 +78,7 @@
  * slow append and a slow entry never hold the keyboard.
  */
 export interface Admitting {
+  testing?: boolean;
   /** The reservation a submission in flight holds, or none. */
   held: Reservation | undefined;
   /** What that work reported, for the loop to apply on its next pass. */
@@ -114,6 +117,9 @@
   slot: { readonly entry: string; readonly ordinal: number },
   held: Admitting,
 ): Result<Reservation> {
+  if (held.testing === true) {
+    return Err(new Error("session tests are running or cleaning up"));
+  }
   const taken = admissible(by, facts, held.held);
   if (!taken.ok) {
     return taken;
@@ -197,7 +203,7 @@
  * became one, and this call goes on waiting for the entry afterwards.
  */
 export function* submitForSidekick(
   at: { readonly conversation: string; readonly response: string },
   options: {
     readonly session: AdmittingSession;
     /**
@@ -226,6 +232,9 @@
   // record claiming an attempt on terms this could have asked about. It is also
   // what keeps a busy request a refusal rather than work that queues behind the
   // entry in flight: nothing below waits for that entry to finish.
+  if (held.testing === true) {
+    return Err(new Error("session tests are running or cleaning up"));
+  }
   const admits = session.admits();
   if (!admits.ok) {
     return admits;
@@ -368,7 +377,7 @@
 ): ControlJournal {
   let next = from;
   return {
     *began(occurrence, reserved): Operation<number> {
       const order = next;
       next += 1;
       yield* writer.append({
@@ -542,3 +551,34 @@
 export function blocking(error: Error): string {
   return `${error.message} Nothing can be admitted into this execution until that is resolved.`;
 }
+
+export function* testForSidekick(
+  path: string,
+  options: {
+    held: Admitting;
+    session: Pick<AdmittingSession, "admits">;
+    testing: ReplTesting | undefined;
+    blocked?: string;
+  },
+): Operation<Result<ReplTestReport>> {
+  const { held, session, testing } = options;
+  if (held.testing === true || held.held !== undefined) {
+    return Err(new Error("an entry or test is running, preparing or cleaning up"));
+  }
+  const ready = session.admits();
+  if (!ready.ok) {
+    return ready;
+  }
+  if (options.blocked !== undefined) {
+    return Err(new Error(options.blocked));
+  }
+  if (testing === undefined) {
+    return Err(new Error("this REPL host has no test runner"));
+  }
+  held.testing = true;
+  try {
+    return yield* testing(path);
+  } finally {
+    held.testing = false;
+  }
+}

packages/cli/src/repl/sidekick/controls.ts

@@ -1,3 +1,4 @@
+import type { ReplTesting, ReplTestReport } from "./testing.ts";
 /**
  * The words a reply may use, and what each of them asks the root to do.
  *
@@ -123,6 +124,7 @@ export type GuideChapter = (typeof GUIDE_CHAPTERS)[number];
  * unavailable may well ask for something else next.
  */
 export interface SidekickRequests {
+  readonly test?: ReplTesting;
   /** The acknowledged published root values, detached. */
   bindings(): Operation<Result<Json>>;
   /** The active entry vocabulary, or the lookup of particular names. */
@@ -271,6 +273,7 @@ export const DESIGNATED: ReadonlySet<string> = new Set([
   "REPL.Environment",
   "REPL.EntryInput",
   "REPL.Submit",
+  "REPL.Test",
   "REPL.EntryOutput",
   ...ENTRY_INSPECTIONS,
   ELICIT_CONTROL,
@@ -706,6 +709,27 @@ export function sidekickMutations(
         );
       },
     }),
+    writing("REPL.Test", "Test", "self-closing", {
+      kind: "function",
+      name: "REPL.Test",
+      props: {
+        type: "object",
+        properties: { path: { type: "string" } },
+        required: ["path"],
+        additionalProperties: false,
+      },
+      description:
+        'Run a session test document or directory suite with `<REPL.Test path="components" />`. Read its completed results as feedback; add Output when the person needs to see them.',
+      *fn(props: Record<string, Json>): Operation<ReplTestReport | string> {
+        const path = props.path;
+        if (typeof path !== "string") {
+          throw new Error("REPL.Test requires path");
+        }
+        return yield* inspected(journal, { component: "REPL.Test", arguments: { path } }, () =>
+          requests.test === undefined ? refusedTesting() : requests.test(path),
+        );
+      },
+    }),
     writing("REPL.Submit", "Submit", "self-closing", {
       kind: "function",
       name: "REPL.Submit",
@@ -826,3 +850,7 @@ function longestRun(text: string, character: string): number {
   }
   return longest;
 }
+
+function* refusedTesting(): Operation<Result<ReplTestReport>> {
+  return Err(new Error("this REPL host has no test runner"));
+}