Skip to content

Capture off-Space macOS windows and protect authentication handoffs - #126

Open
abrichr wants to merge 4 commits into
mainfrom
codex/capture-reliability-20260830
Open

Capture off-Space macOS windows and protect authentication handoffs#126
abrichr wants to merge 4 commits into
mainfrom
codex/capture-reliability-20260830

Conversation

@abrichr

@abrichr abrichr commented Aug 31, 2026

Copy link
Copy Markdown
Member

What changed

The Amex tax-workflow smoke run exposed two capture gaps. macOS rejected an
exact Chrome window on another Space, and an in-workflow login had no
source-time privacy boundary. This PR fixes both in openadapt-capture.

Current macOS systems now use one persistent ScreenCaptureKit stream with a
desktop-independent exact-window filter. The target can be occluded or on
another Space. Each retained frame carries its stream generation, sequence,
status, display time, pixel time, and evidence digest. Quartz and the signed
screencapture -o -l utility remain exact-window compatibility paths. A
provider that fails stays disabled for that recording.

Capture still refuses a minimized target. macOS can return stale backing pixels
for one. Accessibility must identify an off-Space window as non-minimized before
Capture accepts it. The recorder also fails if frame evidence is missing,
regresses, or becomes ambiguous.

The new authentication handoff API suppresses screen frames, native input,
accessibility lookup, window metadata, and microphone content at their sources.
The audio process writes generated silence during the interval so its clock
doesn't drift. Capture retains a clean entry frame, writes a bounded marker,
and keeps input blocked until it has stored a fresh resume frame and proof.

The marker accepts fixed method classes and booleans only. It has no field for
a provider, account identifier, password, OTP, recovery code, vault item, or
free-form note. An autofill account click uses password_manager with
saved_account_selected=True.

The same begin and end operations are available through Capture's authenticated
loopback control channel. Interval IDs make retries idempotent. A stop during an
active handoff records aborted without taking a terminal credential frame.
The sealed loader rejects open intervals, proof mismatches, retained source
events inside a protected interval, and all source events after an aborted
handoff starts.

Product boundary

Authenticate-before-record remains the default procedure. The handoff exists
for workflows where login is part of the demonstration or a session can expire.

Capture proves source suppression and exact frame reacquisition. It doesn't
store credentials or prove that the application accepted a login. This PR does
not change Flow, App Model, compiler, replay, or credential-vault behavior. The
Playwright browser recorder still owns its DOM and browser source boundary.

Windows exact-window capture still needs an unoccluded target. Linux X11 keeps
using its named-window XComposite path. macOS still needs a logged-in desktop
session and the required Screen Recording and Accessibility permissions.

Review focus

  • Verify the ScreenCaptureKit lifecycle and sticky fallback rules.
  • Check the input receipt-to-delivery lease and the audio callback barrier.
  • Check that entry and resume proofs bind the exact source ordinal and pixels.
  • Check the sealed-loader refusal rules for completed and aborted handoffs.

Validation

  • Supported test suite: 738 passed, 15 skipped.
  • Ruff checks pass for the changed package and tests.
  • The public source-boundary check passes.
  • Wheel and source archive builds pass, and the wheel imports the new API.

The test run excludes the repository-only browser bridge and browser E2E suites.
Capture already refuses that unsupported native browser path at startup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant