diff --git a/docs/BUILD_LOG.md b/docs/BUILD_LOG.md index 662cc85..fcaef9e 100644 --- a/docs/BUILD_LOG.md +++ b/docs/BUILD_LOG.md @@ -416,3 +416,14 @@ Evidence: `docs/v0.3/ADR-010-DUAL-SOURCE-OPERATOR-SOURCE.md` and synchronized PR - Verified the real frozen 18-packet index through CLI init/status/next, then passed `npm run check`, 114/114 runnable extension tests, production build, and VSIX packaging; two Docker-only tests remained skipped in the local Windows run. This removes clerical friction from the R7 human gate without replacing expert judgment or exposing automatic outcomes. Full R7 remains pending until two experienced TypeScript raters and the blinded adjudicator return real bundles. + +## 2026-08-01 — Standalone R7 blind kit + +- Added a deterministic offline-kit builder that copies the frozen packet bytes unchanged and prebundles the rating CLI into one Node.js file. +- Added an allowlisted `kit.json` manifest binding every file, byte size, SHA-256, packet-index hash, and the full `kitSha256`; timestamps and repository paths are excluded. +- Added standalone `verify-kit`, strict missing/extra/symlink/special-file rejection, and a protocol allowlist that still rejects `outcome.json` after a malicious manifest rehash. +- Added reproducibility, CLI-tamper, manifest-rewrite, no-overwrite, and no-checkout/no-npm execution tests. +- Built the real 18-packet kit twice on Windows with identical hash `a209f22938a3c4744a1decd80ffb14bd0681b0a111aa50fdd535f88443fbf643`, then ran its bundled verifier and `init/status` outside the repository. This is local build evidence; the hash becomes a study artifact only when the owner freezes and communicates a chosen kit out of band. +- Passed `npm run check`, 119/119 runnable extension tests, production build, and VSIX packaging; two Docker-only tests remained skipped in the local Windows run. + +The kit changes no frozen packet byte, question, rating field, compiler result, or threshold. It reduces setup and unblinding risk but cannot act as a human rater. diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md index 5d146b8..9772d54 100644 --- a/docs/PROJECT_STATE.md +++ b/docs/PROJECT_STATE.md @@ -36,6 +36,7 @@ Branch `codex/shadow-cockpit-rnd` resets the product R&D thesis around **Dual-Co - The preregistered R7 collector froze 30 eligible patches from six repositories after evaluating 457 bounded eligibility records. Manifest `a4ef6cbfa48c66cb9d384bcc2834ecbfae8ff08810abfd1863b395b8aa47d149` contains 12 development and 18 held-out patches; `docs/v0.3/results/R7_CORPUS_COLLECTION.md` reports repository and first-match exclusion counts. No compiler or human outcome influenced selection. - The frozen R7 automatic audit passed its preregistered automatic threshold: 17/18 held-out identities compiled and 16/18 were valid end-to-end. The frozen blind expert packet set and deterministic rating join exist, but two independent ratings and adjudication remain pending. Full R7 has not passed; R5/R6 stay gated. - The R7 blind-rating path now includes a local hash-verified workspace CLI. It verifies the exact packet directory, supports resumable human drafts, refuses incomplete export, and prepares a blinded adjudication workspace by copying only exact categorical consensus. It automates no causal rating and exposes no compiler outcome; the two real expert bundles and panel decision are still pending. +- A deterministic offline blind-kit builder now removes the need to share the PureFlow checkout. It copies the frozen packet bytes unchanged, prebundles the CLI, emits an allowlisted integrity manifest, and verifies the full extracted directory without npm or network access. The expected kit hash must be communicated separately; this packaging evidence does not complete either human rating. - ADR-007 proposes an Executable Operator Model and shadow-control protocol. ADR-008 adds a bidirectional Intent Ledger for artifact accountability. ADR-009 adds Decision Futures and a Takeover Envelope so an on-time pre-reveal human commitment can determine a live integrated branch while agents retain implementation. Together they cover artifact accountability, demonstrated control, and real decision authority; none is implementation evidence. - ADR-010 proposes Dual Source only as a falsifiable product metaphor: agents maintain the complete Software Source while a pure, sparse Operator Projection renders immutable human commitments and bounded control evidence. It owns no state or readiness authority and must beat the existing representation in a matched pilot before becoming core. - `docs/v0.3/R8_COMBINED_PILOT_PROTOCOL.md` now defines the draft four-condition human study needed to test the combined architecture against ordinary autonomous use and a fixed active-review comparator. It has no participants or measured outcomes and cannot be frozen until full R7 and the R5/R5.1/R5.2/R6 runtime exist. @@ -157,7 +158,7 @@ No external input blocks the repository-owned fixture R0–R4.5 mechanism in `do ## Next ordered actions -1. Use `extension/scripts/r7-rater.mjs` to obtain and adjudicate two independent blind expert ratings for packet index `54d78382b3ddbe15cba1f8153275e8149d32ddaa5192163f99ca5f43d903e8fe`. +1. Build the standalone artifact with `extension/scripts/r7-rater-kit.mjs`, communicate its printed kit hash out of band, then obtain and adjudicate two independent blind expert ratings for packet index `54d78382b3ddbe15cba1f8153275e8149d32ddaa5192163f99ca5f43d903e8fe`. 2. If full R7 passes, review ADR-007–010 and the R8 combined protocol together, then freeze the baseline ledger/model schemas and one offline Operator Source prototype. 3. Implement the local R5 evidence ledger and run Experiment 2.5 with identical evidence in both projections; make Operator Source the R6 default only if it improves causal localization per active minute without increasing false confidence. 4. Build the minimal cockpit with the surviving projection, then freeze and run the four-condition delayed-transfer pilot; run the longitudinal field pilot before making a sustained skill-retention claim. diff --git a/docs/v0.3/AGENT_EXECUTION.md b/docs/v0.3/AGENT_EXECUTION.md index 80b1e03..5c11177 100644 --- a/docs/v0.3/AGENT_EXECUTION.md +++ b/docs/v0.3/AGENT_EXECUTION.md @@ -403,6 +403,7 @@ If the fixture-only probe needs arbitrary participant code or model-generated te - Freeze both the recovery compiler and semantic-probe compiler before running the held-out set and make no per-patch code changes. - Use two independent expert raters, blind to compiler outcome, for causal relevance and expected judge result; adjudicate disagreements and report inter-rater agreement. - Use the hash-verified local rater workspace from `R7_EXPERT_RATING.md` so packet integrity, exact coverage, resume, bundle export, identity separation, and consensus-only adjudication prefill are machine-checked while causal decisions remain human. +- Deliver ratings through the deterministic standalone blind kit, not a repository checkout. Freeze and communicate its `kitSha256` out of band; the kit may contain only byte-identical frozen packets, the prebundled local CLI, instructions, and its allowlisted integrity manifest. - Compile both a recovery episode and an Explain-to-Break probe for each eligible patch. Dynamic probes implement the Phase-B `SandboxControlProbe` contract: frozen sanitized snapshot/tree, immutable command-registry hash, approved command or controller-generated deterministic oracle, read-only mounts, and selected sandbox. They may never pass model output directly into code, tests, commands, arguments, paths, mounts, or environment. - Report valid-episode rate, valid-probe rate, capsule rejection/leakage results, false-pass rate, false-fail rate, and confidence intervals separately. - Run the held-out audit on Windows as well as Linux, including paths with spaces, concurrent twins, cancellation, locked files, and cleanup. diff --git a/docs/v0.3/R7_EXPERT_RATING.md b/docs/v0.3/R7_EXPERT_RATING.md index b6e682e..3037bb6 100644 --- a/docs/v0.3/R7_EXPERT_RATING.md +++ b/docs/v0.3/R7_EXPERT_RATING.md @@ -29,20 +29,45 @@ Each rater returns one bundle containing exactly one record for every `packet-*. Use the local workspace CLI so the engineer spends time on causal judgment rather than copying packet IDs or assembling JSON. The tool reads only the frozen packet directory. It verifies the index and every packet hash, refuses unexpected files or subdirectories, stores a resumable draft outside the packet directory, and will not export until all 18 packets have valid ratings. It has no command that accepts compiler results, audit summaries, commits, transcripts, or protected repairs. -From `extension/`: +### Build the standalone blind kit + +The study owner must not give a rater a PureFlow repository checkout. Build a new offline kit containing only the byte-identical frozen packets, a prebundled CLI, a short instruction file, and `kit.json`: ```powershell +cd extension $packetDir = "..\docs\v0.3\results\held-out-rater-packets" +$kitOutput = Join-Path $env:TEMP "pureflow-r7-blind-kit" +node scripts/r7-rater-kit.mjs $packetDir $kitOutput +``` + +The output directory must not already exist. The builder prints a deterministic `kitSha256` and packet-index hash. Give the extracted kit directory to each rater and communicate the expected `kitSha256` through a separate channel. Transport may use an archive, but the extracted directory—not the archive—is the verified artifact. Do not add cover notes, outcome files, repository metadata, or rating workspaces inside it. + +Before rating, the expert runs from the extracted kit: + +```powershell +node .\r7-rater.cjs verify-kit . +``` + +The printed kit hash must equal the out-of-band value. Verification rejects a changed CLI or packet, missing file, extra file, directory, symlink, and even a recomputed manifest that tries to allow a file outside the frozen kit allowlist. Node.js 22 or newer is required; `npm install`, network access, and a PureFlow checkout are not. + +This packaging layer was added after the automatic run. It is allowed to copy and render the already frozen packet bytes and collect the already frozen rating schema only. It may not change packet order, content, questions, rating fields, eligibility, compiler outputs, or thresholds. + +### Rate the packets + +From the extracted kit: + +```powershell +$packetDir = ".\packets" $ratingRoot = Join-Path $env:TEMP "pureflow-r7-expert-a" New-Item -ItemType Directory -Path $ratingRoot $workspace = Join-Path $ratingRoot "workspace.json" $bundle = Join-Path $ratingRoot "expert-a.json" -node scripts/r7-rater.mjs init $packetDir expert-a $workspace -node scripts/r7-rater.mjs next $packetDir $workspace -node scripts/r7-rater.mjs answer $packetDir $workspace yes pass fail none 4 "Reason grounded in the visible diff" -node scripts/r7-rater.mjs status $packetDir $workspace -node scripts/r7-rater.mjs export $packetDir $workspace $bundle +node .\r7-rater.cjs init $packetDir expert-a $workspace +node .\r7-rater.cjs next $packetDir $workspace +node .\r7-rater.cjs answer $packetDir $workspace yes pass fail none 4 "Reason grounded in the visible diff" +node .\r7-rater.cjs status $packetDir $workspace +node .\r7-rater.cjs export $packetDir $workspace $bundle ``` `next` prints the next unrated packet, visible diff, and frozen questions. `answer` may be repeated for a packet before export to correct a draft. Put neither the workspace nor the exported bundle inside `$packetDir`; the exact-content check deliberately rejects that. Give each rater a separate scratch directory and stable pseudonymous ID. @@ -55,10 +80,10 @@ New-Item -ItemType Directory -Path $panelRoot $panelWorkspace = Join-Path $panelRoot "workspace.json" $adjudication = Join-Path $panelRoot "adjudication.json" -node scripts/r7-rater.mjs init-adjudication $packetDir panel-chair $panelWorkspace -node scripts/r7-rater.mjs next $packetDir $panelWorkspace -node scripts/r7-rater.mjs answer $packetDir $panelWorkspace yes pass fail none 4 "Panel reason grounded in the visible diff" -node scripts/r7-rater.mjs export $packetDir $panelWorkspace $adjudication +node .\r7-rater.cjs init-adjudication $packetDir panel-chair $panelWorkspace +node .\r7-rater.cjs next $packetDir $panelWorkspace +node .\r7-rater.cjs answer $packetDir $panelWorkspace yes pass fail none 4 "Panel reason grounded in the visible diff" +node .\r7-rater.cjs export $packetDir $panelWorkspace $adjudication ``` Resolve all pending disagreements without exposing compiler outcomes. The exported adjudication is a full 18-record third bundle with a distinct panel ID. Then run: diff --git a/extension/package.json b/extension/package.json index 479e621..08fe6f1 100644 --- a/extension/package.json +++ b/extension/package.json @@ -262,6 +262,7 @@ "r7:audit": "node scripts/r7-audit.mjs", "r7:corpus": "node scripts/r7-corpus.mjs", "r7:rate": "node scripts/r7-rater.mjs", + "r7:kit": "node scripts/r7-rater-kit.mjs", "test": "npm run fixture:runtime && vitest run", "package": "npm run build && vsce package --no-dependencies" }, diff --git a/extension/scripts/r7-rater-kit.mjs b/extension/scripts/r7-rater-kit.mjs new file mode 100644 index 0000000..63506c0 --- /dev/null +++ b/extension/scripts/r7-rater-kit.mjs @@ -0,0 +1,37 @@ +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { pathToFileURL } from "node:url"; +import { build } from "esbuild"; + +const root = await mkdtemp(join(tmpdir(), "pureflow-r7-kit-cli-")); +const rater = join(root, "r7-rater.cjs"); +const builder = join(root, "builder.cjs"); + +try { + await Promise.all([ + build({ + entryPoints: ["src/rating/workspace-cli.ts"], + bundle: true, + platform: "node", + format: "cjs", + target: "node22", + legalComments: "none", + outfile: rater, + }), + build({ + entryPoints: ["src/rating/kit-build-cli.ts"], + bundle: true, + platform: "node", + format: "cjs", + target: "node22", + legalComments: "none", + outfile: builder, + }), + ]); + process.argv.push(rater); + const loaded = await import(pathToFileURL(builder).href); + await (loaded.done ?? loaded.default?.done); +} finally { + await rm(root, { recursive: true, force: true }); +} diff --git a/extension/src/rating/kit-build-cli.ts b/extension/src/rating/kit-build-cli.ts new file mode 100644 index 0000000..5c16d18 --- /dev/null +++ b/extension/src/rating/kit-build-cli.ts @@ -0,0 +1,20 @@ +import { readFile } from "node:fs/promises"; +import { buildBlindKit } from "./kit"; + +export const done = main(); + +async function main(): Promise { + try { + const [packetDir, outputDir, cliBundlePath] = process.argv.slice(2); + if (!packetDir || !outputDir || !cliBundlePath) { + process.stderr.write("Usage: node scripts/r7-rater-kit.mjs \n"); + process.exitCode = 2; + return; + } + const manifest = await buildBlindKit(packetDir, outputDir, await readFile(cliBundlePath)); + process.stdout.write(`Built offline blind kit ${manifest.kitSha256} for packet index ${manifest.packetIndexSha256}.\n`); + } catch (error) { + process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`); + process.exitCode = 1; + } +} diff --git a/extension/src/rating/kit.ts b/extension/src/rating/kit.ts new file mode 100644 index 0000000..f5992c1 --- /dev/null +++ b/extension/src/rating/kit.ts @@ -0,0 +1,148 @@ +import { mkdir, readFile, readdir, writeFile } from "node:fs/promises"; +import { join, relative, resolve, sep } from "node:path"; +import { assertExactKeys, assertRelPath, assertSha256, canonicalHash, compareUtf8, rawSha256 } from "../rnd/canonical"; +import { loadPacketSet } from "./workspace-files"; + +export interface BlindKitManifest { + schemaVersion: 1; + protocol: "r7-blind-kit-v1"; + packetProtocol: "r7-blind-expert-v1"; + packetIndexSha256: string; + runtime: "node>=22"; + files: Array<{ path: string; bytes: number; sha256: string }>; + kitSha256: string; +} + +export async function buildBlindKit(packetDir: string, outputDir: string, cliBundle: Uint8Array): Promise { + const packetSet = await loadPacketSet(packetDir); + const output = resolve(outputDir); + await mkdir(output, { recursive: false }); + await mkdir(join(output, "packets")); + + const files = new Map(); + files.set("r7-rater.cjs", cliBundle); + files.set("README.md", Buffer.from(readme(), "utf8")); + files.set("packets/index.json", await readFile(join(packetSet.dir, "index.json"))); + for (const { filename } of packetSet.index.packets) { + files.set(`packets/${filename}`, await readFile(join(packetSet.dir, filename))); + } + + for (const [path, bytes] of [...files].sort(([left], [right]) => compareUtf8(left, right))) { + assertRelPath(path); + await writeFile(join(output, ...path.split("/")), bytes, { flag: "wx" }); + } + + const entries = [...files] + .map(([path, bytes]) => ({ path, bytes: bytes.byteLength, sha256: rawSha256(bytes) })) + .sort((left, right) => compareUtf8(left.path, right.path)); + const core = { + schemaVersion: 1 as const, + protocol: "r7-blind-kit-v1" as const, + packetProtocol: "r7-blind-expert-v1" as const, + packetIndexSha256: packetSet.index.indexSha256, + runtime: "node>=22" as const, + files: entries, + }; + const manifest = { ...core, kitSha256: canonicalHash("r7-blind-kit", core) }; + await writeFile(join(output, "kit.json"), `${JSON.stringify(manifest, null, 2)}\n`, { flag: "wx" }); + await verifyBlindKit(output); + return manifest; +} + +export async function verifyBlindKit(kitDir: string): Promise { + const root = resolve(kitDir); + const manifest = JSON.parse(await readFile(join(root, "kit.json"), "utf8")) as BlindKitManifest; + validateManifest(manifest); + const actualPaths = (await walk(root)).sort(compareUtf8); + const expectedPaths = [...manifest.files.map(({ path }) => path), "kit.json"].sort(compareUtf8); + if (actualPaths.length !== expectedPaths.length || actualPaths.some((path, index) => path !== expectedPaths[index])) { + throw new Error("Blind kit contains missing or unexpected files"); + } + + for (const entry of manifest.files) { + const bytes = await readFile(join(root, ...entry.path.split("/"))); + if (bytes.byteLength !== entry.bytes || rawSha256(bytes) !== entry.sha256) { + throw new Error(`Blind kit file mismatch: ${entry.path}`); + } + } + const packets = await loadPacketSet(join(root, "packets")); + if (packets.index.indexSha256 !== manifest.packetIndexSha256) { + throw new Error("Blind kit packet index mismatch"); + } + return manifest; +} + +function validateManifest(manifest: BlindKitManifest): void { + assertExactKeys(manifest, ["schemaVersion", "protocol", "packetProtocol", "packetIndexSha256", "runtime", "files", "kitSha256"], "blind kit manifest"); + if ( + manifest.schemaVersion !== 1 + || manifest.protocol !== "r7-blind-kit-v1" + || manifest.packetProtocol !== "r7-blind-expert-v1" + || manifest.runtime !== "node>=22" + ) { + throw new Error("Invalid blind kit manifest"); + } + assertSha256(manifest.packetIndexSha256, "packet index hash"); + assertSha256(manifest.kitSha256, "kit hash"); + if (!Array.isArray(manifest.files) || manifest.files.length < 4) throw new Error("Blind kit manifest has no files"); + const seen = new Set(); + for (const file of manifest.files) { + assertExactKeys(file, ["path", "bytes", "sha256"], "blind kit file"); + assertRelPath(file.path); + if (!/^(README\.md|r7-rater\.cjs|packets\/index\.json|packets\/packet-[0-9]{3}\.json)$/.test(file.path)) { + throw new Error(`Blind kit file is outside the allowlist: ${file.path}`); + } + if (!Number.isSafeInteger(file.bytes) || file.bytes < 1) throw new Error(`Invalid blind kit file size: ${file.path}`); + assertSha256(file.sha256, `${file.path} hash`); + const folded = file.path.toLowerCase(); + if (seen.has(folded)) throw new Error(`Duplicate blind kit path: ${file.path}`); + seen.add(folded); + } + const required = ["README.md", "r7-rater.cjs", "packets/index.json"]; + if (required.some((path) => !seen.has(path.toLowerCase()))) throw new Error("Blind kit is missing a required file"); + const { kitSha256, ...core } = manifest; + if (canonicalHash("r7-blind-kit", core) !== kitSha256) throw new Error("Blind kit manifest hash mismatch"); +} + +async function walk(root: string, dir = root): Promise { + const result: string[] = []; + for (const entry of await readdir(dir, { withFileTypes: true })) { + if (entry.isSymbolicLink()) throw new Error(`Blind kit rejects symlinks: ${entry.name}`); + const absolute = join(dir, entry.name); + if (entry.isDirectory()) { + result.push(...await walk(root, absolute)); + } else if (entry.isFile()) { + result.push(relative(root, absolute).split(sep).join("/")); + } else { + throw new Error(`Blind kit rejects special files: ${entry.name}`); + } + } + return result; +} + +function readme(): string { + return `# PureFlow R7 blind expert kit + +This directory contains only the frozen expert packets and a prebundled local CLI. Do not search commit history, execute the disclosed project code, or request compiler outcomes while rating. + +Requires Node.js 22 or newer. No npm install or network access is required. + +From this directory: + +\`\`\`powershell +node .\\r7-rater.cjs verify-kit . +$ratingRoot = Join-Path $env:TEMP "pureflow-r7-expert-a" +New-Item -ItemType Directory -Path $ratingRoot +$workspace = Join-Path $ratingRoot "workspace.json" +$bundle = Join-Path $ratingRoot "expert-a.json" + +node .\\r7-rater.cjs init .\\packets expert-a $workspace +node .\\r7-rater.cjs next .\\packets $workspace +node .\\r7-rater.cjs answer .\\packets $workspace yes pass fail none 4 "Reason grounded in the visible diff" +node .\\r7-rater.cjs status .\\packets $workspace +node .\\r7-rater.cjs export .\\packets $workspace $bundle +\`\`\` + +Keep workspaces and exported bundles outside this directory. The verifier rejects added or changed files. +`; +} diff --git a/extension/src/rating/workspace-cli.ts b/extension/src/rating/workspace-cli.ts index 83985c8..a5b7f39 100644 --- a/extension/src/rating/workspace-cli.ts +++ b/extension/src/rating/workspace-cli.ts @@ -1,4 +1,5 @@ import type { ExpertRating } from "./r7"; +import { verifyBlindKit } from "./kit"; import { adjudicationDisagreement, createAdjudicationWorkspace, createRaterWorkspace, exportRatingBundle, ratingProgress, saveRating, type RaterPacket } from "./workspace"; import { loadPacketSet, readRatingBundle, readWorkspace, replaceJson, writeNewJson } from "./workspace-files"; @@ -6,8 +7,16 @@ void main(); async function main(): Promise { try { + const nodeMajor = Number(process.versions.node.split(".")[0]); + if (!Number.isInteger(nodeMajor) || nodeMajor < 22) throw new Error("R7 blind rating requires Node.js 22 or newer"); const [command, packetDir, ...args] = process.argv.slice(2); if (!command || !packetDir) return usage(); + if (command === "verify-kit") { + if (args.length) return usage(); + const manifest = await verifyBlindKit(packetDir); + process.stdout.write(`Verified blind kit ${manifest.kitSha256} for packet index ${manifest.packetIndexSha256}.\n`); + return; + } const set = await loadPacketSet(packetDir); if (command === "init") { if (args.length !== 2) return usage(); @@ -95,6 +104,7 @@ function printProgress(total: number, complete: number, nextPacketId: string | n function usage(): void { process.stderr.write([ "Usage:", + " node r7-rater.cjs verify-kit ", " node scripts/r7-rater.mjs init ", " node scripts/r7-rater.mjs next ", " node scripts/r7-rater.mjs answer <1-5> ", diff --git a/extension/test/r7-rating-kit.test.ts b/extension/test/r7-rating-kit.test.ts new file mode 100644 index 0000000..65ae463 --- /dev/null +++ b/extension/test/r7-rating-kit.test.ts @@ -0,0 +1,146 @@ +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { execFile } from "node:child_process"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { build } from "esbuild"; +import { afterEach, describe, expect, it } from "vitest"; +import { canonicalHash, rawSha256 } from "../src/rnd/canonical"; +import { buildBlindKit, verifyBlindKit, type BlindKitManifest } from "../src/rating/kit"; +import type { RatingIndex } from "../src/rating/r7"; +import type { RaterPacket } from "../src/rating/workspace"; + +const roots: string[] = []; +const run = promisify(execFile); + +afterEach(async () => { + await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true }))); +}); + +describe("R7 standalone blind kit", () => { + it("builds reproducibly and verifies every allowed file", async () => { + const root = await scratch(); + const packets = join(root, "source-packets"); + await writePackets(packets); + const first = await buildBlindKit(packets, join(root, "kit-a"), Buffer.from("#!/usr/bin/env node\n")); + const second = await buildBlindKit(packets, join(root, "kit-b"), Buffer.from("#!/usr/bin/env node\n")); + + expect(second).toEqual(first); + expect((await verifyBlindKit(join(root, "kit-a"))).kitSha256).toBe(first.kitSha256); + expect(first.files.map(({ path }) => path)).toEqual([ + "README.md", + "packets/index.json", + "packets/packet-001.json", + "packets/packet-002.json", + "r7-rater.cjs", + ]); + }); + + it("rejects a changed bundled CLI", async () => { + const root = await scratch(); + const packets = join(root, "source-packets"); + await writePackets(packets); + await buildBlindKit(packets, join(root, "kit"), Buffer.from("original cli")); + await writeFile(join(root, "kit", "r7-rater.cjs"), "changed cli"); + + await expect(verifyBlindKit(join(root, "kit"))).rejects.toThrow("file mismatch"); + }); + + it("rejects an outcome file even when a rewritten manifest accounts for it", async () => { + const root = await scratch(); + const packets = join(root, "source-packets"); + await writePackets(packets); + const kit = join(root, "kit"); + await buildBlindKit(packets, kit, Buffer.from("original cli")); + const outcome = Buffer.from("{}\n"); + await writeFile(join(kit, "outcome.json"), outcome); + const manifest = JSON.parse(await readFile(join(kit, "kit.json"), "utf8")) as BlindKitManifest; + const core = { + ...manifest, + files: [...manifest.files, { path: "outcome.json", bytes: outcome.byteLength, sha256: rawSha256(outcome) }], + }; + const { kitSha256: _old, ...withoutHash } = core; + await writeFile(join(kit, "kit.json"), JSON.stringify({ ...withoutHash, kitSha256: canonicalHash("r7-blind-kit", withoutHash) })); + + await expect(verifyBlindKit(kit)).rejects.toThrow("outside the allowlist"); + }); + + it("refuses to overwrite an existing output directory", async () => { + const root = await scratch(); + const packets = join(root, "source-packets"); + await writePackets(packets); + const output = join(root, "kit"); + await buildBlindKit(packets, output, Buffer.from("original cli")); + + await expect(buildBlindKit(packets, output, Buffer.from("original cli"))).rejects.toThrow(); + }); + + it("runs the prebundled CLI without a repository checkout or npm install", async () => { + const root = await scratch(); + const packets = join(root, "source-packets"); + const bundled = join(root, "r7-rater.cjs"); + await writePackets(packets); + await build({ + entryPoints: ["src/rating/workspace-cli.ts"], + bundle: true, + platform: "node", + format: "cjs", + target: "node22", + legalComments: "none", + outfile: bundled, + }); + const kit = join(root, "kit"); + await buildBlindKit(packets, kit, await readFile(bundled)); + const cli = join(kit, "r7-rater.cjs"); + const verified = await run(process.execPath, [cli, "verify-kit", kit], { cwd: root, windowsHide: true }); + expect(verified.stdout).toContain("Verified blind kit"); + + const workspace = join(root, "workspace.json"); + const initialized = await run(process.execPath, [cli, "init", join(kit, "packets"), "expert-smoke", workspace], { cwd: root, windowsHide: true }); + expect(initialized.stdout).toContain("Rated 0/2"); + }); +}); + +async function scratch(): Promise { + const root = await mkdtemp(join(tmpdir(), "pureflow-r7-kit-test-")); + roots.push(root); + return root; +} + +async function writePackets(dir: string): Promise { + await mkdir(dir); + const packets = [packet("a".repeat(24), "src/a.ts"), packet("b".repeat(24), "src/b.ts")]; + const core = { + schemaVersion: 1 as const, + protocol: "r7-blind-expert-v1" as const, + cohort: "held-out" as const, + corpusManifestSha256: "d".repeat(64), + sourcePlanFileSha256: "e".repeat(64), + packetCount: packets.length, + packets: packets.map((value, index) => ({ + filename: `packet-${String(index + 1).padStart(3, "0")}.json`, + packetId: value.packetId, + packetSha256: value.packetSha256, + })), + }; + const index: RatingIndex = { ...core, indexSha256: canonicalHash("r7-rater-index", core) }; + await writeFile(join(dir, "index.json"), `${JSON.stringify(index, null, 2)}\n`); + await Promise.all(packets.map((value, index) => writeFile(join(dir, `packet-${String(index + 1).padStart(3, "0")}.json`), `${JSON.stringify(value, null, 2)}\n`))); +} + +function packet(packetId: string, path: string): RaterPacket { + const core = { + schemaVersion: 1 as const, + protocol: "r7-blind-expert-v1" as const, + packetId, + projectAlias: "project-12345678", + seam: { path, symbol: "run", kind: "function" }, + writablePaths: [path], + attributedTestPaths: [path.replace("src", "test")], + mutationRule: "Replace the changed source with its adjacent base version.", + observation: "Run the preregistered check and classify its process outcome.", + sourceAndTestDiff: `diff --git a/${path} b/${path}`, + questions: ["Is the proposed rewind causally relevant?"], + }; + return { ...core, packetSha256: canonicalHash("r7-rater-packet", core) }; +}