Skip to content

test(privacy): pin the compiler/repair no-cookie guarantee with a tripwire - #117

Open
myselfsiddharth wants to merge 2 commits into
mainfrom
track1/b5-session-material-tripwire
Open

test(privacy): pin the compiler/repair no-cookie guarantee with a tripwire#117
myselfsiddharth wants to merge 2 commits into
mainfrom
track1/b5-session-material-tripwire

Conversation

@myselfsiddharth

Copy link
Copy Markdown
Contributor

Closes #101. Follow-up to #37 (SC-04). Branched from main; no overlap with #107#110.

SC-04 was true and unpinned. src/compiler/types.ts's Trajectory / TrajectoryStep / Fingerprint and src/runner/types.ts's PageStateSnapshot have no cookie- or storage-shaped field, so session material was never representable in the compiler's input or the repair model's. Nothing made that fail loudly if it changed — which is what the issue asked for.

The obvious test does not work

Serialize a real trajectory, grep its keys, assert none is cookies. That passes for the wrong reason: adding cookies?: string[] to Fingerprint does not populate it in a committed fixture, so an instance-only check stays green through exactly the change it exists to catch. It also cannot see an additionalProperties: false that someone relaxed.

So the tripwire asserts at the three levels the guarantee actually rests on, and each catches something the others structurally cannot:

Level Catches Why the others cannot
Type surface — TypeScript compiler API, walked recursively from each root type a field declared on any reachable interface, populated or not an optional field nobody populates is erased at runtime
Contract schemaproperties names, plus additionalProperties: false on every object definition a forbidden property, or an object that stopped closing itself a TS-only change never touches the schema, and vice versa
Instances — real recordings, their compiled bundles, a live capturePageState a value that reached an artifact through an untyped path neither declaration-level check sees a runtime spread

The type-surface level is the one that answers #101's actual worry ("a future field addition to any of these types"), and it is the one an instance walk cannot substitute for.

Guard-proven, not asserted

Every level was sabotaged and the failure observed, rather than claimed:

Sabotage Result
cookies?: string[] added to Fingerprint, left unpopulated 3 type-surface cases fail; all 20 schema + instance assertions stay green
$defs.fingerprint.additionalPropertiestrue only the schema-closure case fails
storage_state property declared in the schema only the schema-property case fails
capturePageState() returns page.context().cookies() only the live-browser case fails

That first row is the whole argument for the type-surface level: 20 of 23 assertions cannot see the change at all.

Note the first row also fails via two-hop reachability — the field is on Fingerprint, but the Trajectory and TrajectoryStep cases fail too, because the walk descends Trajectory → steps[] → TrajectoryStep → pre_state → Fingerprint. There is a dedicated case pinning that the recursion actually reaches nested types, so the others cannot pass vacuously on a root they never opened.

Two details worth the review

Keys are normalized before comparison (lowercased, separators stripped), so storage_state, storageState and StorageState are one entry rather than three near-misses. A future field arrives in whichever convention its author reached for; a tripwire that knows one spelling has a gap in it. The storage_state sabotage above is caught by this and not by an exact-match list.

The schema check is scoped to nodes that define an object type ("type": "object" with properties), and my first cut got this wrong — it flagged four real nodes. cache-row.schema.json's allOf if/then pair carries properties without being an object definition, and closing one would be a bug, not a tightening: additionalProperties: false inside a then rejects every property that branch does not itself mention, i.e. the rest of the row. Called out in a comment at the check, because it is exactly the kind of thing a future reader would "tighten" into a false-positive machine.

The live-browser case sets a real cookie on the context before capturing, so "no cookies present" is not the reason it passes — and it asserts the cookie's value did not ride along inside a permitted field either.

Scope

This is not new enforcement. SC-04's status goes from "enforced by construction" to "enforced by construction + test" — the construction was already correct and is unchanged. No source file is touched; the diff is one test plus the SC-04 write-up.

tests/unit/page-state.test.ts already launches Chromium under tests/unit, so the live case follows an established convention rather than introducing browser usage to that directory.

npm run ci            # green — 197 unit, 14 integration, secret-scan clean, lint-docs clean (47 docs)
npm run test:canary   # 13 pass

🤖 Generated with Claude Code

…pwire

Closes #101.

SC-04 was true and unpinned. `src/compiler/types.ts`'s Trajectory /
TrajectoryStep / Fingerprint and `src/runner/types.ts`'s PageStateSnapshot
have no cookie- or storage-shaped field, so session material was never
representable in the compiler's input or the repair model's. Nothing made
that fail loudly if it changed.

The obvious test does not work. Serializing a real trajectory and grepping
its keys passes for the wrong reason: adding `cookies?: string[]` to
Fingerprint does not populate it in a committed fixture, so an instance-only
check stays green through exactly the change it exists to catch, and it
cannot see an `additionalProperties: false` someone relaxed.

So the tripwire asserts at the three levels the guarantee actually rests on:

- Type surface, via the TypeScript compiler API, walked recursively from each
  root type. Catches a field *declared* on any reachable interface, populated
  or not — which is erased at runtime and invisible to an instance walk.
- Contract schema. Catches a forbidden property name, and an object
  definition that stopped closing itself.
- Instances: real recordings, their compiled bundles, and a live
  capturePageState against a context that genuinely has a cookie set.

Guard-proven, not asserted. Adding an unpopulated `cookies?: string[]` to
Fingerprint fails three type-surface cases (including two-hop reachability
from Trajectory) while all twenty schema and instance assertions stay green.
Relaxing $defs.fingerprint.additionalProperties, declaring a `storage_state`
property, and making capturePageState return the context's cookies each fail
exactly their own level.

Keys are compared after normalization (lowercased, separators stripped) so
storage_state / storageState / StorageState are one entry, not three
near-misses.

The schema check is scoped to nodes that define an object type. An applicator
fragment — cache-row.schema.json's allOf if/then pair — carries `properties`
without being an object definition, and closing one would reject every
property the branch does not itself mention.

Not new enforcement: SC-04 stays "enforced by construction", now + test.

npm run ci          # green — 197 unit, 14 integration, lint-docs clean (47 docs)
npm run test:canary # 13 pass

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@myselfsiddharth
myselfsiddharth requested a review from a team as a code owner August 6, 2026 05:02
@github-actions github-actions Bot added the size/L <= 600 changed lines label Aug 6, 2026
@github-actions
github-actions Bot requested a review from OM152002 August 6, 2026 05:03
@github-actions github-actions Bot added documentation Improvements or additions to documentation privacy-boundary Touches the privacy boundary — canary is merge-blocking labels Aug 6, 2026
@myselfsiddharth

Copy link
Copy Markdown
Contributor Author

@copilot Review this PR

Co-authored-by: myselfsiddharth <99870733+myselfsiddharth@users.noreply.github.com>

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown

@copilot Review this PR

Addressed a review finding in commit 94b74f7: the new live-browser tripwire test now skips when Playwright Chromium binaries are unavailable, so the unit test file remains runnable in environments without a local browser install.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation privacy-boundary Touches the privacy boundary — canary is merge-blocking size/L <= 600 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin the compiler/repair no-cookie guarantee with a tripwire test

2 participants