Skip to content

feat: functional trusted VSI binding, ladder signing and ownership hardening - #4

Draft
mstattma wants to merge 35 commits into
feat/live-video-vsifrom
feat/trusted-vsi-functional
Draft

mstattma wants to merge 35 commits into
feat/live-video-vsifrom
feat/trusted-vsi-functional

Conversation

@mstattma

@mstattma mstattma commented Sep 30, 2026 •

Copy link
Copy Markdown

Functional trusted VSI Python binding plus single-file ladder signing, paired with castlabs/c2pa-rs feat/trusted-vsi-functional.

What's in it

  • TrustedVsiSession and helpers for the trusted native ABI; exact native gate c2pa-rs/0.92.0-dev + capability mask 63 (no version range). Contract: docs/trusted-vsi-python-contract.md.

  • True merge of the ladder candidate 502b8bb2 (Builder.sign_ladder), with ladder callback errors aligned to the other Builder paths (DynamicAssertion exceptions keep identity; claim-signer interrupts propagate).

  • True merge of the generic native-ownership patch 7ba8615: consume-first FFI calls triage registry rejections by the rejected handle id, with anchored tag parsing; opaque C2paStream. Opaque-registry-only checks in tests/test_native_ownership_opaque.py.

  • True merge of its reviewed follow-up a303db8 (not the temporary CI-only commit): comments clarify deferred opaque-native drops and the sticky error slot; tests cover a real non-NULL typed pointer and distinguish the raised error from the slot after cleanup. Remaining real-native concurrency/sticky-slot checks and the stock Windows ARM64 ownership-qualification gap are tracked in docs/roadmap.md.

  • Unreleased source/qualification identity 0.37.13.dev0. Immutable dev5 release inputs, lock and tooling unchanged.

  • Paired CI native pin 6b506352 (203dc08d plus CI-only fixes in feat: functional trusted VSI (consolidated candidates + ContentAuth main) c2pa-rs#15; no C ABI change).

Qualification

  • Hosted paired Linux + Windows (trusted_vsi_only, non-publishing), run 36808706395 at 12d265db, native 6b506352, Rust 1.96.0. Both platforms: 186 focused, real-native ladder harness, 730 non-threaded, 54 threaded, 186 installed-wheel tests passed. Previous pairing: run 36793704783 at 5c64f2c, native 203dc08d, identical counts.
  • Local Linux against the exact native library: 263 focused tests and 45 subtests passed.
  • Stock c2pa-rs 0.91.0 Windows x64 ownership/unit/ladder/threaded qualification passed separately in mstattma run 36775608167. Stock Windows ARM64 ownership qualification was outside that scope.
  • Reviews: independent review of each delta; ladder owner review of callback/ownership changes.

Not in scope

No publication; the dev5 release path still refuses this source.

tmathern and others added 30 commits August 27, 2026 15:38
* chore: Update c2pa version to v0.90.16

* Bump version from 0.37.8 to 0.37.9

* Bump version from 0.37.8 to 0.37.9
…uth#320)

* build: drop wheel, setuptools and pytest from runtime dependencies

None of the three is imported anywhere under `src/`. `c2pa.py` and `lib.py`
import only the standard library; `build.py` — the `download-artifacts` console
script — imports `requests` and, lazily, `toml`. Those two stay.

They are also already classified correctly elsewhere in the repo:

* `[build-system] requires` already lists `setuptools>=68.0.0` and `wheel`, so
  the build has what it needs and the runtime entries are duplicates.
* `requirements-dev.txt` lists `wheel` and `setuptools` under
  "# Build dependencies" and `pytest` under "# Testing dependencies".
* `.github/workflows/build.yml` installs pytest explicitly (`pip install
  pytest`, lines 285 and 377), so CI does not rely on the runtime declaration
  either.

Removing them is therefore a no-op for this repo's own build and test paths,
and it keeps three packages out of every consumer's production environment.

* build: declare pytest in a PEP 735 dev dependency group

Dropping pytest from `[project.dependencies]` left it undeclared in
pyproject.toml entirely, with `requirements-dev.txt` as the only manifest
naming it. `[dependency-groups] dev` states it where it belongs: installed
for contributors (`uv sync`, `pip install --group dev`) and, unlike
`[project.optional-dependencies]`, absent from the published package
metadata — which is the separation this branch is about.

The bound matches requirements-dev.txt (`pytest>=8.1.0`) rather than the
`>=7.4.0` the runtime entry carried; nothing installs the old one.

The comment above the remaining dependencies goes with it. The rationale
for keeping `toml` and `requests` belongs in the pull request, not in a
manifest that has carried no comments so far.

* build: keep pytest declared in requirements-dev.txt only

Review feedback: the PEP 735 group restated a bound that
`requirements-dev.txt` already carries, so the two could drift — which is
what this branch set out to stop, not to reproduce one line further down.

Nothing in the repo would have read the group. The Makefile's `install-deps`
and every workflow that installs dependencies do so with `pip install -r
requirements-dev.txt` (`build.yml` lines 52, 90/93, 164/167, 490/492), and
the wheel test jobs install pytest by name. The group was a declaration with
no consumer.

The published metadata — the point of this branch — is unaffected either
way, and the diff is now purely subtractive.
* Update c2pa version from v0.90.16 to v0.90.19

* Bump version from 0.37.9 to 0.37.10

* Bump version from 0.37.9 to 0.37.10
* chore: Update C2PA version to 0.90.22

* Bump version from 0.37.10 to 0.37.11

* Bump version from 0.37.10 to 0.37.11
…t-tooling

Add reusable c2pa-rs RC-preflight workflow; fix latent test bugs
…#327)

test_sdk_version compares the loaded native library's version against
c2pa-native-version.txt, but an RC-preflight run (per
test-c2pa-rs-source-build.yml) actually builds from whatever ref is in
c2pa-rs-preflight-ref.txt instead, so the test always failed on that
one assertion during a preflight even when everything else passed.

parse_native_version() now prefers c2pa-rs-preflight-ref.txt when
present, falling back to c2pa-native-version.txt otherwise -- the same
precedence the workflow itself uses to decide what to build. Verified
both paths locally: green against the pinned 0.90.22 with no preflight
file, and green against a c2pa-rc-v0.91.0-rc.3 build with the file
present.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
contentauth#328)

contentauth#327's parse_native_version() fix preferred c2pa-rs-preflight-ref.txt
whenever it existed in the checked-out tree. That broke the *real*
build.yml jobs on contentauth#325: that PR adds the ref file to the branch, so an
ordinary tests-unix/tests-windows run (which downloads and installs
the actual pinned release, unrelated to the preflight workflow) picked
up the file too and wrongly expected the RC's version string, failing
with e.g. "'0.91.0-rc.3' not found in '0.90.22'".

The file's mere presence was never a reliable signal -- only
test-c2pa-rs-source-build.yml's own "Run tests" step actually builds
from that ref. Gate on a new C2PA_PREFLIGHT_RUN env var that only that
step sets instead.

Verified locally: with the pinned 0.90.22 installed and no env var
set, all 444 tests pass regardless of whether
c2pa-rs-preflight-ref.txt happens to exist in the tree.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Fix test_sdk_version regression: gate on an env var, not file presence

contentauth#327's parse_native_version() fix preferred c2pa-rs-preflight-ref.txt
whenever it existed in the checked-out tree. That broke the *real*
build.yml jobs on contentauth#325: that PR adds the ref file to the branch, so an
ordinary tests-unix/tests-windows run (which downloads and installs
the actual pinned release, unrelated to the preflight workflow) picked
up the file too and wrongly expected the RC's version string, failing
with e.g. "'0.91.0-rc.3' not found in '0.90.22'".

The file's mere presence was never a reliable signal -- only
test-c2pa-rs-source-build.yml's own "Run tests" step actually builds
from that ref. Gate on a new C2PA_PREFLIGHT_RUN env var that only that
step sets instead.

Verified locally: with the pinned 0.90.22 installed and no env var
set, all 444 tests pass regardless of whether
c2pa-rs-preflight-ref.txt happens to exist in the tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Point the RC preflight at c2pa-rs 0.91.0-rc.3

This is the entire diff this PR now carries on top of contentauth#328: pin the
preflight workflow at c2pa-rc-v0.91.0-rc.3 so it builds from that git
tag (no crates.io publish, no prebuilt GitHub release binaries for an
RC) and runs the full unit test suite against it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Point the RC preflight at c2pa-rs 0.91.0-rc.3

This is the entire diff this PR now carries on top of contentauth#328: pin the
preflight workflow at c2pa-rc-v0.91.0-rc.3 so it builds from that git
tag (no crates.io publish, no prebuilt GitHub release binaries for an
RC) and runs the full unit test suite against it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Preflight against c2pa-rs 0.91.0-rc.4

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* chore: bump c2pa-rs to v0.91.0

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* chore: Bump version from 0.37.11 to 0.37.12

* Bump version from 0.37.11 to 0.37.12
Register the native ladder export as optional, validate path arrays, preserve typed errors and allocation ownership, and use the modern single-sign builder lifecycle. Add focused binding tests and isolated stock/candidate native qualification lanes without changing release pins.

Co-authored-by: bibinbaby444 <bibinbaby444@gmail.com>

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Use c2pa_free for new manifest allocations, include focused ladder tests in existing CI runs, reject optimized verification harness execution, and document the modern builder lifecycle.
Include a documented synthetic fixture and offline native smoke in the existing CI selection. Enforce candidate capability on request, verify the native signature and shared manifest, and cover typed ctypes conversion without changing stock native pins or builder semantics.

Co-authored-by: bibinbaby444 <bibinbaby444@gmail.com>
…idation

# Conflicts:
#	docs/class-diagram.md
#	pyproject.toml
#	src/c2pa/c2pa.py
#	tests/test_unit_tests.py
…o 203dc08d

True merge of fix/native-resource-ownership-upstream 7ba8615. Takes its
anchored registry-tag parsing (strip one 'Other: ' prefix, match at start)
so tags quoted inside another error's payload no longer establish
ownership, its _handle_value and consume-first triage, its C2paStream
docstring, generic ownership tests and ownership documentation, keeping the
consolidation's fork content. Opaque-registry-only consumed-id checks move
to tests/test_native_ownership_opaque.py.

Paired CI now builds castlabs/c2pa-rs 203dc08d (ContentAuth main 69907b5a
merged; 0.92.0-dev, Rust 1.96.0 unchanged).
… of skipping

Require the opaque registry (odd object ids) with a fixture that fails when
C2PA_TRUSTED_VSI_ABI_REQUIRED/FUNCTIONAL_REQUIRED is set, list the new file
in the installed-wheel contract, and mark the 203dc08d pairing qualification
as pending in the contract status.
@mstattma

Copy link
Copy Markdown
Author

@BibinBaby444 requesting your review (a formal review request isn't possible: you're not a collaborator on this repository). Companion draft PRs: castlabs/c2pa-rs#15, #4, castlabs/stardustproof-keystore#13, castlabs/stardustproof-c2pa-signer#25.

@mstattma

mstattma commented Oct 1, 2026

Copy link
Copy Markdown
Author

The independently reviewed ownership follow-up a303db8 is true-merged (not cherry-picked) into functional head 5c64f2cc090eeb29506bc766faa69b959e4ed982; the CI-only commit remains excluded. The remaining native-registry checks and stock Windows ARM64 ownership qualification are in docs/roadmap.md. Paired run 36793704783 passed on Linux and Windows against native 203dc08d: 186 focused, real-native ladder harness, 730 non-threaded, 54 threaded, and 186 installed-wheel tests per platform. Stock 0.91.0 Windows x64 passed in mstattma run 36775608167. @BibinBaby444 please review the new merge abef446 and roadmap commits bc9e99d, 5c64f2c. No release or dev5 evidence was changed.

@BibinBaby444 BibinBaby444 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested. One memory-safety bug reachable from plain Python, plus one native state that's invisible to Python. Verified at 12d265db against a native built from c2pa-rs d738fbb2 (debug, unstable_live_video). Note that CI is not evidence here yet: every test job on this PR was skipped, including "Trusted VSI paired-source API"; only format and tooling checks ran.

P1: closing a session during a native call frees callbacks native is still using.

  • TrustedVsiSession._release drops _signer_callback_cb, _dynamic_assertion_cbs and _trusted_vsi_callback, and _teardown calls it before c2pa_free.
  • Native free defers the actual drop while a call holds the borrow, so the in-flight finalize_init_uuid keeps calling through the freed thunks.
  • Once the Context has been closed (which the contract allows), the session holds the only reference to the claim-signer thunk.
  • A single thread is enough. Close the session from inside its own signer_binding callback, then allocate fresh SignerCallbacks:
    finalize raised: _C2paSignature('Signature: internal error (bad parameter: signer callback returned error code -1)')
    real claim callback calls: []  impostor calls: [1876]
    
  • Native jumped through the freed claim thunk into an unrelated callback, which received the 1,876-byte claim payload. With different reuse that's a crash or a call into arbitrary code.
  • Suggest keeping the callback pins alive for the object's lifetime (don't clear them in _release), or deferring close() until in-flight calls drain.
  • The same drop-on-close pattern exists in the older live-video VSI session and Builder _releases, and the native contract text ("until the handle is freed") should say "until the deferred drop".

P2: blocked is never exposed.

  • Native C2paLiveVideoTrustedVsiStatusV1 has blocked: bool between has_exhaustion_reason and exhaustion_reason, at offset 18.
  • The ctypes struct, TrustedVsiStatus and status() all omit it. The layout still lines up, since Python treats the byte as padding, so there's no memory hazard.
  • But Python can't see the one state the contract tells adapters to act on ("After a failure once an external signing call began, the session is blocked. Discard it, construct a NEW session…"):
    before failure: native blocked byte[18] = 0  -> status() has no blocked field
    after failure:  native blocked byte[18] = 1  -> status() unchanged
    next call: _C2paOther Other: bad parameter: session is blocked after a failed external signing step; …
    
  • Add the field and a blocked attribute on TrustedVsiStatus, update the offsets pinned in tests/test_trusted_vsi_api.py, and add a paired test that asserts status().blocked after a callback failure.

P2: the stock-native lane will fail, and CI hasn't run it. The dynamic variant of test_scripted_wrappers_store_base_exceptions_return_minus_one_and_reraise calls signer.add_dynamic_assertion unguarded. The -k "not paired" lane in build.yml downloads upstream c2pa-v0.91.0 (c2pa-native-version.txt), and v0.91.0 (e7cc666c) doesn't export c2pa_signer_add_dynamic_assertion, so those four parametrizations raise NotSupported. Guard the variant on has_dynamic_assertions() (or stub the export in _ScriptedNative), then get the skipped jobs to actually run.

P3

  • Ladder error guidance drops a guarantee. docs/ladder-signing.md and the sign_ladder docstring say "Errors may leave partial newly created outputs; discard these files", but leave out native's guarantee that "Existing files, including every source, are never overwritten". The native error doesn't say which path failed, so say callers should delete only destinations they confirmed absent before the call. Otherwise a cleanup-everything handler deletes a pre-existing file.
  • The version gate is a release label, not an ABI identity. 0.92.0-dev is upstream's workspace version, so the gate effectively rests on the symbol plus mask 63. A native ABI-revision probe would make a future in-place V1 struct change detectable.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants