Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/BABYSITTER-CATALOG-HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ repository/PR event, and call the Cloud lineage path. Cloud must recheck the
exact live `babysit` label and the bound session/head. Permission declarations
are not enforcement. Export success is byte verification, not execution approval.

The native package source is `extensions/babysitter` (see its README for the
turn contract). It is unreleased and cannot execute: #549 still refuses it,
the SDK context has no `capabilities.cloud.babysitterTurn`, and its
`compat` requires a surface release after 2.0.25 that routes `labeled`,
`unlabeled`, and `ready_for_review`. Export it only from a reviewed, merged
commit.

## Export reviewed bytes

After the native package is reviewed and committed, build the SDK and run:
Expand Down
62 changes: 62 additions & 0 deletions evidence/babysitter-native-handler/full-suite.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Full SDK suite on this branch's final bytes (local macOS, not CI)
$ (cd packages/sdk && npm test)
Test Files 23 failed | 140 passed | 3 skipped (166)
Tests 51 failed | 2556 passed | 25 skipped (2632)
exit=1
Failing files (count of failing tests):
1 tests/authored-flow-operation.test.ts
1 tests/authored-node-runtime.test.ts
2 tests/authored-root.test.ts
1 tests/budget-unmetered-live.test.ts
1 tests/build-gate.test.ts
4 tests/bundle-preflight.test.ts
3 tests/bundle.test.ts
1 tests/cli-status.test.ts
8 tests/cli-watch.test.ts
1 tests/cloud-schedule.test.ts
1 tests/communication-mixed-resume.test.ts
3 tests/deploy.test.ts
2 tests/direct-input.test.ts
3 tests/flow-executor-chain.test.ts
1 tests/flow-extension-compose.test.ts
6 tests/live-kernel.test.ts
1 tests/mcp.test.ts
3 tests/provider-trigger-executor.test.ts
5 tests/run-from-digest.test.ts
1 tests/stop-process-group.test.ts
6 tests/webhook-live.test.ts
1 tests/worker-cli.test.ts
1 tests/yaml-helper-live.test.ts
tests/babysitter-native-extension.test.ts is not among them.

# Baseline: 21 files that failed in an earlier full run, rerun at origin/main 607ac671 with this branch's changes stashed (git stash push -u)
$ (cd packages/sdk && npm run build && npx vitest run <the 21 files>)
Test Files 17 failed | 4 passed (21)
Tests 38 failed | 236 passed | 18 skipped (292)
exit=1
1 tests/authored-flow-operation.test.ts
1 tests/authored-node-runtime.test.ts
3 tests/authored-root.test.ts
2 tests/bundle.test.ts
1 tests/cli-status.test.ts
8 tests/cli-watch.test.ts
1 tests/cloud-schedule.test.ts
1 tests/communication-mixed-resume.test.ts
1 tests/direct-input.test.ts
4 tests/flow-executor-chain.test.ts
1 tests/flow-extension-compose.test.ts
3 tests/live-kernel.test.ts
1 tests/mcp.test.ts
3 tests/provider-trigger-executor.test.ts
3 tests/run-from-digest.test.ts
1 tests/stop-process-group.test.ts
6 tests/webhook-live.test.ts

# Files failing in the final full run but not in that baseline, rerun in isolation on this branch
$ (cd packages/sdk && npx vitest run tests/build-gate.test.ts tests/budget-unmetered-live.test.ts tests/deploy.test.ts tests/worker-cli.test.ts tests/yaml-helper-live.test.ts tests/bundle-preflight.test.ts)
Test Files 6 passed (6)
Tests 40 passed (40)
exit=0

Observed failure causes under full-suite load include 5000ms test timeouts, relayflowd lease expiry, ENOTEMPTY temp cleanup, and timing assertions (e.g. expected 5001 to be less than 5000).
Claim limited to: the new suite passes; the listed failures also occur at origin/main in this environment or pass in isolation. CI is the authority.
35 changes: 35 additions & 0 deletions evidence/babysitter-native-handler/mutation.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Mutation 1: delete the event/subscription binding check in extensions/babysitter/turn.ts
$ diff original mutated
75d74
< if (event.eventType !== subscription) refuse(`event ${JSON.stringify(event.eventType)} was delivered to the ${subscription} handler.`);
$ npx vitest run tests/babysitter-native-extension.test.ts -t 'another subscription'
× native Babysitter extension > refuses an event for another subscription before requesting a turn 17ms
→ promise resolved "undefined" instead of rejecting
Tests 1 failed | 30 skipped (31)
$ shasum -a 256 original restored
a5189b3b4db9cb7f4812e480f29dc33dfb5b392ff284952cefac9b5f8a18f781
a5189b3b4db9cb7f4812e480f29dc33dfb5b392ff284952cefac9b5f8a18f781
$ npx vitest run tests/babysitter-native-extension.test.ts -t 'another subscription'
✓ tests/babysitter-native-extension.test.ts (31 tests | 30 skipped) 323ms
Tests 1 passed | 30 skipped (31)

# Mutation 2: delete the fail-closed capability guard in extensions/babysitter/babysitter.flow.ts
$ diff original mutated
22,24d21
< if (typeof port !== 'object' || port === null || typeof (port as Partial<TurnQueue>).queue !== 'function') {
< throw new Error('babysitter: this runtime does not provide the cloud:babysitter-turn write.');
< }
$ npx vitest run tests/babysitter-native-extension.test.ts -t 'fails closed on'
× native Babysitter extension > fails closed on no capabilities (today's SDK context) 23ms
× native Babysitter extension > fails closed on no cloud capability 3ms
× native Babysitter extension > fails closed on a queue that is not a function 2ms
Received: "Cannot read properties of undefined (reading 'queue')"
Received: "turnQueue(...).queue is not a function"
Tests 3 failed | 28 skipped (31)
$ shasum -a 256 original restored
e1e8e9690334360612a6f6e749746e6a322eefea922f61342479c3bac18490b6
e1e8e9690334360612a6f6e749746e6a322eefea922f61342479c3bac18490b6
$ npx vitest run tests/babysitter-native-extension.test.ts -t 'fails closed on'
✓ tests/babysitter-native-extension.test.ts (31 tests | 28 skipped) 308ms
Tests 3 passed | 28 skipped (31)
Note: without the guard the run still fails, as a raw TypeError; the guard pins a clear, specific refusal.
24 changes: 24 additions & 0 deletions evidence/babysitter-native-handler/tests.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
$ (cd packages/sdk && npx vitest run tests/babysitter-native-extension.test.ts)

RUN v2.1.9 /Users/khaliqgant/Projects/AgentWorkforce/flows-worktrees/babysitter-native-handler/packages/sdk

✓ tests/babysitter-native-extension.test.ts (31 tests) 366ms

Test Files 1 passed (1)
Tests 31 passed (31)
Start at 03:11:40
Duration 5.09s (transform 1.69s, setup 0ms, collect 2.94s, tests 366ms, environment 0ms, prepare 335ms)

Exit: 0

$ (cd packages/sdk && npx vitest run tests/babysitter-catalog-export.test.ts tests/flow-extension-compose.test.ts tests/plugin-extension.test.ts tests/authored-flow.test.ts tests/preflight.test.ts) # passing lines elided

RUN v2.1.9 /Users/khaliqgant/Projects/AgentWorkforce/flows-worktrees/babysitter-native-handler/packages/sdk


Test Files 5 passed (5)
Tests 219 passed (219)
Start at 03:11:47
Duration 14.25s (transform 5.37s, setup 0ms, collect 19.35s, tests 15.13s, environment 3ms, prepare 2.28s)

Exit: 0
13 changes: 13 additions & 0 deletions evidence/babysitter-native-handler/typecheck.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
$ (cd packages/sdk && npm run typecheck)

> @relayflows/sdk@2.0.25 typecheck
> tsc --noEmit && tsc -p tsconfig.type-tests.json

Exit: 0

$ (cd packages/sdk && npm run typecheck:tests)

> @relayflows/sdk@2.0.25 typecheck:tests
> tsc -p tsconfig.tests.json

Exit: 0
41 changes: 41 additions & 0 deletions extensions/babysitter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Native Babysitter (flow extension)

A schema-2 extension on `software-factory`. Each of its eleven GitHub
subscriptions takes Cloud's normalized delivery descriptor and makes one
`cloud:babysitter-turn` queue request:

```jsonc
// in: Cloud's normalized descriptor, never a raw webhook
{ "event": { "provider": "github", "eventType": "pull_request.labeled", "deliveryId": "…" },
"pullRequest": { "host": "github", "owner": "…", "repo": "…", "number": 1, "headSha": "<40 hex, optional>" } }
// out: f.capabilities.cloud.babysitterTurn.queue(request)
{ "delivery": { "deliveryId": "…", "provider": "github", "eventType": "pull_request.labeled",
"pullRequest": { "owner": "…", "repository": "…", "number": 1 } } }
```

The delivery is an envelope, not authority. The handler never sends findings,
prose, a head, label, session, lineage, relay agent, config, merge flag, or
route; Cloud builds the prompt from the live head and the trusted binding. Cloud's
capability adapter checks the envelope against the host-verified dispatch,
loads the persisted activation, rereads the live PR (open, exact `babysit`
label, head), and resolves the one bound Codex session. It answers
`{ receiptId, status: queued | duplicate }` and the run completes `success`;
Cloud dedupes by lineage and live head, so `deliveryId` is provenance, not the
idempotency key. Every policy refusal and in-doubt transport failure rejects,
and the run fails once with no retry or fallback. Any other input or receipt,
or a runtime without the capability, fails the run.

## What this does not do

- Hosted execution is still refused. The SDK rejects every matched extension
handler with `plugin_unsupported` until manifest permissions are enforced
(#549, gate 8 / #442). This package does not change that.
- The SDK context has no `capabilities.cloud.babysitterTurn`, and Cloud's
adapter is not merged. Both must land before a turn can happen.
- It is not the legacy `examples/babysitter` (Claude review lenses, GitHub
comments, merge-gate hook). It holds no GitHub write authority.
- `compat` needs a surface release after 2.0.25: the published 2.0.25 event
registry cannot route `labeled`, `unlabeled`, or `ready_for_review`.

Catalog export follows `docs/BABYSITTER-CATALOG-HANDOFF.md`, with
`#extensions/babysitter` as the path, after human review and merge.
57 changes: 57 additions & 0 deletions extensions/babysitter/babysitter.flow.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Native Babysitter: an extension on Software Factory whose only effect is a
// `cloud:babysitter-turn` request for the PR a verified delivery names. The
// review itself happens in the original Codex session Cloud has bound to that
// PR; this entry holds no GitHub credentials, runs no agent, and writes nothing
// to GitHub. Hosted execution stays refused by the SDK (#549, gate 8 / #442).
import { flow, github, type Ctx, type TriggerSource } from '@relayflows/surface';
import { turnDelivery, turnReceipt, type BabysitterTurnDelivery, type Subscription } from './turn.ts';

interface TurnQueue {
queue(request: { readonly delivery: BabysitterTurnDelivery }): PromiseLike<unknown>;
}

/**
* The declared write, as Cloud's capability adapter spells it. The SDK `Ctx`
* has no `capabilities` field yet; a runtime without it must fail the run,
* never skip the turn silently. The adapter, not this entry, holds workspace,
* activation, and the verified delivery authority.
*/
function turnQueue(f: Ctx): TurnQueue {
const capabilities = (f as unknown as { readonly capabilities?: { readonly cloud?: { readonly babysitterTurn?: unknown } } }).capabilities;
const port = capabilities?.cloud?.babysitterTurn;
if (typeof port !== 'object' || port === null || typeof (port as Partial<TurnQueue>).queue !== 'function') {
throw new Error('babysitter: this runtime does not provide the cloud:babysitter-turn write.');
}
return port as TurnQueue;
}

function handler(subscription: Subscription) {
return async (f: Ctx, input: unknown): Promise<void> => {
const delivery = turnDelivery(subscription, input);
// A delivery-only wake: Cloud builds the prompt from the live head and the
// trusted binding, so this entry sends no findings or prose. Refusals reject.
turnReceipt(await turnQueue(f).queue({ delivery }));
f.done('success');
};
}

const handlers: readonly (readonly [TriggerSource, Subscription])[] = [
[github.pull_request('opened'), 'pull_request.opened'],
[github.pull_request('synchronize'), 'pull_request.synchronize'],
[github.pull_request('reopened'), 'pull_request.reopened'],
[github.pull_request('ready_for_review'), 'pull_request.ready_for_review'],
[github.pull_request('closed'), 'pull_request.closed'],
[github.pull_request('labeled'), 'pull_request.labeled'],
[github.pull_request('unlabeled'), 'pull_request.unlabeled'],
[github.pull_request_review({ action: 'submitted' }), 'pull_request_review.submitted'],
[github.pull_request_review({ action: 'dismissed' }), 'pull_request_review.dismissed'],
[github.check_run('completed'), 'check_run.completed'],
[github.issue_comment('created'), 'issue_comment.created'],
];

// The default body is reachable only by a direct run, whose input is
// caller-controlled JSON. It never requests a turn.
export default handlers.reduce<ReturnType<typeof flow>>(
(handle, [trigger, subscription]) => handle.on(trigger, handler(subscription)),
flow('babysitter', { budget: { dollars: 1, wallclock: '5m' } }, async f => { f.done('declined'); }),
);
28 changes: 28 additions & 0 deletions extensions/babysitter/flows-plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"schema": 2,
"kind": "flow-extension",
"name": "babysitter",
"version": "0.2.0",
"description": "Native Babysitter for Software Factory: turns a verified PR delivery into one cloud:babysitter-turn request. Cloud rechecks the live babysit label, head, and bound session; this extension holds no GitHub write authority.",
"compat": {
"surface": "^2.0.26",
"sdk": "^2.0.26",
"base": [{ "name": "software-factory", "version": "*" }]
},
"entry": "babysitter.flow.ts",
"extends": { "handlers": true, "hooks": [] },
"triggers": [
{ "provider": "github", "event": "pull_request", "actions": ["opened", "synchronize", "reopened", "ready_for_review", "closed", "labeled", "unlabeled"] },
{ "provider": "github", "event": "pull_request_review", "actions": ["submitted", "dismissed"] },
{ "provider": "github", "event": "check_run", "actions": ["completed"] },
{ "provider": "github", "event": "issue_comment", "actions": ["created"] }
],
"permissions": {
"integrations": ["github"],
"harnesses": ["codex"],
"mcp": [],
"writes": ["cloud:babysitter-turn"],
"budget": { "dollars": 1, "wallclock": "5m" }
},
"preflight": { "credentials": [], "servers": [] }
}
96 changes: 96 additions & 0 deletions extensions/babysitter/turn.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
/**
* The native Babysitter turn contract: Cloud's normalized delivery descriptor
* in, one `cloud:babysitter-turn` queue request out.
*
* Nothing here is authority. The delivery is an envelope: Cloud's capability
* adapter checks it against the host-verified dispatch, loads the persisted
* activation, rereads the live PR (open, exact `babysit` label, head), and
* resolves the bound session and lineage. The handler never names a session,
* lineage, head, label, route, or config, so a forged or stale descriptor can
* at most ask Cloud to look again.
*/

/** The subscriptions flows-plugin.json declares, as `event.action` event types. */
export const SUBSCRIPTIONS = [
'pull_request.opened', 'pull_request.synchronize', 'pull_request.reopened',
'pull_request.ready_for_review', 'pull_request.closed', 'pull_request.labeled',
'pull_request.unlabeled', 'pull_request_review.submitted', 'pull_request_review.dismissed',
'check_run.completed', 'issue_comment.created',
] as const;
export type Subscription = (typeof SUBSCRIPTIONS)[number];

export interface BabysitterTurnDelivery {
readonly deliveryId: string;
readonly provider: 'github';
readonly eventType: Subscription;
readonly pullRequest: { readonly owner: string; readonly repository: string; readonly number: number };
}

/**
* `queued`: Cloud wrote its head-bound receipt and Relay accepted the turn.
* `duplicate`: that lineage and live head were already queued. Every refusal
* and every in-doubt transport failure rejects instead of resolving.
*/
export interface BabysitterTurnReceipt {
readonly receiptId: string;
readonly status: 'queued' | 'duplicate';
}

const DELIVERY = /^[A-Za-z0-9_.:-]{1,200}$/;
const OWNER = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})$/;
const REPO = /^[A-Za-z0-9._-]{1,100}$/;
const SHA = /^[0-9a-f]{40}$/;
const RECEIPT = /^[A-Za-z0-9_.:-]{1,200}$/;

function refuse(message: string): never {
throw new Error(`babysitter: ${message}`);
}

function record(value: unknown, what: string, keys: readonly string[], required: readonly string[]): Record<string, unknown> {
if (typeof value !== 'object' || value === null || Array.isArray(value)) refuse(`${what} must be an object.`);
const object = value as Record<string, unknown>;
const extra = Object.keys(object).filter(key => !keys.includes(key));
if (extra.length > 0) refuse(`${what} has unexpected fields ${extra.join(', ')}.`);
const missing = required.filter(key => !Object.hasOwn(object, key));
if (missing.length > 0) refuse(`${what} is missing ${missing.join(', ')}.`);
return object;
}

function matches(value: unknown, pattern: RegExp, what: string): string {
if (typeof value !== 'string' || !pattern.test(value)) refuse(`${what} is malformed.`);
return value;
}

/**
* Validate the descriptor a handler received against the subscription that
* handler was registered for. Any drift, unknown field, or raw webhook shape
* is refused rather than guessed at. Cloud's `headSha` enrichment is checked
* but not forwarded: Cloud binds to the head it rereads, never to a hint.
*/
export function turnDelivery(subscription: Subscription, input: unknown): BabysitterTurnDelivery {
const top = record(input, 'input', ['event', 'pullRequest'], ['event', 'pullRequest']);
const event = record(top.event, 'event', ['provider', 'eventType', 'deliveryId'], ['provider', 'eventType', 'deliveryId']);
const pr = record(top.pullRequest, 'pullRequest', ['host', 'owner', 'repo', 'number', 'headSha'], ['owner', 'repo', 'number']);
if (event.provider !== 'github') refuse('event.provider must be github.');
if (event.eventType !== subscription) refuse(`event ${JSON.stringify(event.eventType)} was delivered to the ${subscription} handler.`);
if (pr.host !== undefined && pr.host !== 'github') refuse('pullRequest.host must be github.');
const owner = matches(pr.owner, OWNER, 'pullRequest.owner');
const repository = matches(pr.repo, REPO, 'pullRequest.repo');
if (repository === '.' || repository === '..') refuse('pullRequest.repo is malformed.');
if (typeof pr.number !== 'number' || !Number.isSafeInteger(pr.number) || pr.number <= 0) refuse('pullRequest.number must be a positive integer.');
if (pr.headSha !== undefined) matches(pr.headSha, SHA, 'pullRequest.headSha');
return Object.freeze({
deliveryId: matches(event.deliveryId, DELIVERY, 'event.deliveryId'),
provider: 'github',
eventType: subscription,
pullRequest: Object.freeze({ owner, repository, number: pr.number }),
});
}

/** Cloud's answer is data from across a boundary; an unknown shape fails the run. */
export function turnReceipt(value: unknown): BabysitterTurnReceipt {
const receipt = record(value, 'turn receipt', ['receiptId', 'status'], ['receiptId', 'status']);
const receiptId = matches(receipt.receiptId, RECEIPT, 'turn receipt receiptId');
if (receipt.status !== 'queued' && receipt.status !== 'duplicate') refuse('turn receipt status is unknown.');
return Object.freeze({ receiptId, status: receipt.status });
}
Loading
Loading