Add the provider documentation set and finish the decomposition by jwrosewell · Pull Request #1047 · IABTechLab/trusted-server

jwrosewell · 2026-08-19T14:50:51Z

Fifth and final PR of the stack decomposing #838 as requested in the #986 review. Stacks on #1046. Compare split/5-response-hook-docs to split/4-client-resolve to see only this PR's change.

Specs carried by this PR:

docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md, the migration and rollout draft revised against the implemented series. Its sign-off ledger is kept intact, with each row the series implements marked with its PR, so Thursday's task force can ratify rows rather than re-litigate them.
docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md, retained as the design bar for the response-header hook. The hook itself is not in this series.

What this PR does

The IntegrationResponseMutator response-header hook from the earlier draft of this PR is removed. It had no consumer, which is the spec set's own rule against speculative surface. It returns together with the first integration that needs it, and its spec stays in the tree as that design's starting bar.
Documentation for the provider model: the configuration reference for the [ec], [device], and [geo] sections (including the required default_country, the assume_single_jurisdiction acknowledgment, and the requires-signal floor on a failed lookup), the Edge Cookie guide rewritten around providers and the permission model (including the narrow withdrawal semantics, the hardened resolve endpoint, and the resolved-marker cookie), setup and error-reference updates, and the permission-model guide joining the docs navigation.
The example configuration documents every provider selector in one place, with the [geo] baseline explained and the single-jurisdiction acknowledgment shown commented next to the key it concerns.
The Axum and Cloudflare native test jobs also run on windows-latest.

What happens to #838

Once these five PRs merge, #838 is closed. It stays open as a draft reference for the review period only.

How it was verified

Full local gate on this branch, all clean. cargo test-fastly, cargo test-axum, cargo test-cloudflare, cargo test-spin, the integration parity suite, docs prettier check, cargo fmt --check, and all six per-target clippy aliases.

References #777 and #778. Decomposes #838. Spec baseline from #986.

Produced with AI assistance under James Rosewell's direction, and flagged here so reviewers know to apply the usual scrutiny.

The five-PR series (IABTechLab#1043 to IABTechLab#1047) opens the identity, device and geo seams. The nine vendor integrations already in core sit behind the integration registry instead, which is a private table, so none of them can move out until that table is opened. This spec defines the one core change that opens it: public registration builders with a second input on IntegrationRegistry, browser JavaScript carried on the registration, startup validation as a hook, the same treatment for auction providers and the bid renderer contract, and neutral replacements for the two places where a vendor reaches into core. It then sets out the migration of all nine existing integrations, one PR each. The change is complete in itself: after it, no vendor move needs a core change. Written against the series' tree with the file and line references for every claim about the current code. Documentation only.

The review of IABTechLab#1043 asked that spec changes land before the code that implements them, so a divergence is a decision taken in review rather than a ratification of something already merged. PRs IABTechLab#1043 to IABTechLab#1047 each carried the design document for their own step, and IABTechLab#1043 carried a 607-line spec describing device providers, geo providers, the permission model and the browser resolve endpoint, none of which is in that PR. Move all six series documents here, so this PR carries the complete normative set and no code: - 2026-07-30-pluggable-providers-design.md (from IABTechLab#1043) - provider-code-registry.md (from IABTechLab#1043) - 2026-07-30-permission-model-design.md (from IABTechLab#1045) - 2026-07-30-client-cycle-ec-resolve-design.md (from IABTechLab#1046, later revised by IABTechLab#1047) - 2026-07-30-integration-response-header-hook-design.md (from IABTechLab#1047) - 2026-07-30-provider-migration-rollout-design.md (from IABTechLab#1047) Each file is taken verbatim at the tip of the stack, so the later revisions are preserved: the provider-switching continuity section, the geo requires-signal floor, and the code-envelope paragraph IABTechLab#1047 added to the client-cycle spec. The revision-record tables are unchanged. No document's substance was edited. The only edits are to this spec's own status line, which said the PR adds one document and that the series specs land with IABTechLab#1047, and a revision-record row recording the move.

aram356 · 2026-08-31T18:47:55Z

The sequencing discussion for this series is on #1084. This PR is superseded rather than rejected. The design in §3.6 is accepted and most of the provider work carries over onto the reordered base. See #1084.

The four series specs (client-cycle EC resolve, permission model, pluggable providers, migration and rollout) each carried a Status line saying they were implemented. The code they describe is only in PRs IABTechLab#1043 to IABTechLab#1047 and none of those is merged, so the line read as shipped behavior. Each now says Proposed, names the PR that carries the implementation and states that it is not yet on main, keeping the existing revision dates and notes. The integration provider seam spec carried counts and line references that do not hold on main at d516a9e. Corrected against that commit: - Section 4 said migration_guards.rs embeds "the thirteen vendor files". The directory holds 23 .rs files (2 infrastructure, 6 in nextjs/, 2 in datadome/, 13 top-level integration modules), the guard embeds 20 of them and 9 of those 20 belong to the nine vendors, with osano.rs and the two datadome/ files absent. builders() registers 13 integrations, which is a different 13 from the file count. - Section 3.5 gave no counts for the prepare and finalize calls. There are nine production prepare_request call sites across the four adapters and a tenth in core, and the single production finalize_response call site is in core rather than in any adapter. - Section 8 item 3 described a proxy resolving geo twice, which does not happen on main. The real double resolution is the adapter EC context build against handle_auction on POST /auction. - Section 8 item 5 understated the Spin gap and misdescribed Cloudflare. Cloudflare covers every route it registers and has no health route, while Spin skips its first-party bindings as well as its inline admin stubs. - Line references: settings.rs:166 to :215, auction/mod.rs:49 to the list at :51 to :53, publisher.rs:4361 to :4369. Section 6 now requires the round trip to be proven on the Fastly adapter, the primary deployment target, rather than on any adapter, because Fastly has no library target and the round trip otherwise only runs on the Axum dev server.

The file header called itself a shipped demo policy, which reads as Trusted Server shipping a policy position. It never does. The builder of a deployment chooses the permissions.yaml compiled into their image, an operator overlay is the recorded follow-on, and the visitor's signals decide the rest at runtime. This copy is the repository's test policy, and the header now says exactly that.

No policy ever ships with Trusted Server, and the repository's permissions files are samples, not fixtures owned by any one consumer. They now live in config/permissions, each carrying a display name at the top (the repository sample is config/permissions/vanilla.yaml, named Vanilla), so a tool listing the directory can present them readably, and whoever builds, configures, or inspects makes an explicit choice of file. The compiled-in default follows the move, and concrete path references in the example TOML and guides are updated. All 2400 core tests pass.

A single page at tools/permissions-inspector answers the question a policy owner has about a permissions file: for a visitor in a given place, with given consent signals, which Data Uses are set? The page walks three steps, the policy baseline, the visitor's input (location, TCF consent, and Global Privacy Control on their own tabs), and what the application layer gets, with a plain-language reason per Data Use and balloon help quoting each Data Use's definition from the IAB Tech Lab Privacy Taxonomy. Step 3 is computed by trusted-server-core itself, compiled to WebAssembly by the wrapper crate in tools/permissions-inspector/wasm and built by scripts/build-inspector-wasm.sh, which a new workflow runs in CI. The wrapper exposes the production path: raw TC, GPP and US Privacy strings and GPC go through build_context_from_signals, the policy resolves through assemble_permissions, and pasted files are validated by PermissionMaps::from_yaml with the server's own error messages. The page states the trusted-server version, branch, commit and date the engine was built from, with source links, and reports how many Data Uses the page's explanatory JS mirror agrees on so drift is visible. The page lists the repository's sample permissions from config/permissions through a manifest the build script writes, labeled by each file's name line, and says plainly that no policy ever ships with Trusted Server. Hosting the page through GitHub Pages is noted in the README as the follow-on.

CI failed with permission denied because the executable bit was never recorded in git, the usual Windows-authored-script gap.

The code-scanning bot flagged the new workflow for carrying the default GITHUB_TOKEN permissions. The job only checks out and builds, so it reads contents and nothing else.

The doc comment above the include_str still said the file lives at the repository root after the move to config/permissions/vanilla.yaml.

Fourth slice of the PR 838 decomposition. A client-cycle Edge Cookie provider defers at the edge and lets the page derive the identifier in the browser; the page posts it to POST /_ts/api/v1/ec/resolve and the provider verifies it before the edge mints it as the Edge Cookie: - EdgeCookieProvider gains resolve_from_client with a no-op default, so server-side providers are untouched. ClientResolveInput carries the posted payload plus the request's resolved permissions and consent. - The resolve endpoint enforces the same rules as organic generation and several of its own. The permission gate applies unchanged. The request must carry an Origin on the publisher's domain (an identity-setting POST must not be drivable from a foreign page) and a text/plain or application/json body. A minted identifier must fit the identifier bounds (400), must not silently replace a different identity already on the request (409), and is persisted to the identity graph before the cookie is set, keyed by the provider's canonical form, so withdrawal reaches a client-set identity the same way it reaches an edge-minted one. With no graph available nothing is minted, matching the organic rule against phantom cookies; a graph write failure is 503. Every response carries Cache-Control: no-store. - The Edge Cookie stays HttpOnly. A non-HttpOnly companion marker cookie (ts-ecr=1, no identity content) tells the page script a resolve succeeded, so it does not re-post on every page view; the marker expires together with the Edge Cookie on withdrawal. A Rust test asserts the marker name and the demo's fixed word stay in sync with the page script source. - The client-fixed demonstration provider (fixed shared word, verify-before-mint) is compiled only behind the trusted-server-core client-fixed-demo cargo feature. Production builds reject the selection at startup: a fixed shared word is not an identity. - The Fastly adapter routes the endpoint and passes the same bot-gated identity graph as generation, so unrecognized clients cannot mint through resolve either. The other adapters deliberately do not route it yet, matching identify and batch-sync, which need the same platform KV wiring those adapters lack. The design spec for this slice lives at docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md, the 2026-07-31 threat-model draft revised to the implemented state, with the deferred reservation design retained verbatim as the bar for the first vendor scheme.

The spec used em dashes throughout; they are replaced with plain punctuation. The resolve handler's doc claimed every response carries Cache-Control: no-store, but a provider or configuration error leaves through the error path to the adapter's own response, so the doc now says every response the handler builds.

`ec/provider.rs` documents that core checks every provider response header against its reserved surface before applying it, so a provider may set its own cookies and headers but cannot reach into the surface core manages. The organic mint path honours that in `EcContext::generate_with_provider`. The resolve endpoint applied provider headers with no check at all, on both the 204 path where the provider mints nothing and the 200 path where it does. That let a browser-side provider set the managed `ts-ec` cookie directly and bypass the identifier bounds check, the conflict check and the row-before-cookie rule that sit a few lines below it. Removing the new check and re-running the tests shows the 204 response going out as `204` with `set-cookie: ts-ec=forged-value; Path=/` and no identity-graph row behind it, which is an external reviewer's own finding on this stack, answered on the Edge Cookie provider branch and reintroduced here. His words were that a provider can set the managed cookie "including when it returns no identifier", which is exactly that path. The check is the same call the organic path makes, placed immediately after `resolve_from_client` and before the identifier is read, so one check covers both paths for the same reason the organic one sits there: a provider can return headers with no identifier at all. A breach fails the request rather than producing a status, matching the organic path and the module's existing rule that a provider error propagates to the adapter's error response, because a provider reaching into the reserved surface has broken its contract rather than sent a bad request. Tested on both paths: a managed `ts-ec` cookie with nothing minted, and an `x-ts-ec` header on the minted path.

The two sites in the resolve handler had the same defect the EC finalization loop had: `HeaderMap::insert` replaces every existing value for a header name, so a provider setting more than one cookie kept only the last, and a provider setting a header this handler already wrote silently replaced it. On this path the replaced header is core's own rather than an origin's, and the one that matters is `Cache-Control: no-store`. Every response the handler builds carries it, because an identity response must never be cached, and an inserting write let a provider drop it. Removing the fix and re-running the new test shows both: the provider's first cookie gone, and `no-store` replaced by the provider's `max-age=600`. Both sites now go through the shared `apply_provider_response_headers`, which carries the rule and is the same helper EC finalization uses, so the two paths cannot drift.

`ProviderCode::new` is fallible and returns `Option<ProviderCode>`, but this test provider passed its result straight back from `code()`, which returns `ProviderCode`. The test module therefore did not compile. Nothing caught it because the only checks run against this branch were `cargo check`, which builds the library and not the `#[cfg(test)]` modules, so the branch looked healthy while its own tests could not be built. The macro is the established form for a fixed code and is already used by the neighbouring test provider in this same file. It resolves the code in a `const` block, so a malformed code fails the build rather than the request, and there is no `Option` left to unwrap. Addresses: crates/trusted-server-core/src/ec/resolve.rs, where the `ResolveHeaderProvider` test double could not compile.

…rface

The Edge Cookie context carried the raw injected provider so the resolve endpoint could rebuild it later. That slot is gone, so the context now carries the provider the composition root threaded, which is the one seam section 3.6 specifies.

…e resolve cycle on a provider switch Two review findings from Christian Pavilonis on IABTechLab#1084. The resolve endpoint authorized any origin that was a suffix match on the publisher domain, and it also accepted an http:// origin and discarded the port before comparing. The Edge Cookie is scoped to the parent domain, so a delegated or compromised sibling subdomain could fix an identity that lands on the apex and every sibling with it. Authorization is now a comparison of the whole serialized origin, being the scheme, the lowercased host and the effective port. The default accepted origin is https://<publisher.domain> and nothing else, and [ec] resolve_allowed_origins adds further exact origins for a publisher whose pages are served elsewhere. The resolved marker is not namespaced by the provider code the way the cookie value, the identity-graph key and withdrawal are, so after a switch between client-cycle providers it outlived the identity it was set for and the new provider's page script would skip the resolve it should perform. Core now expires the marker whenever the request carries an identifier the selected provider does not own, which does not depend on any vendor page script comparing a marker value correctly.

The client resolve endpoint compared the request Origin and each allowed origin as lowercased strings, which matched what browsers send but was not the RFC's rule, so a configured https://www.example.com:443 could never match the https://www.example.com a browser serializes. origins_match now parses both sides as RFC 6454 §6.1 serialized origins and compares the scheme, host and port triple of §4 under the §5 same-origin test, with a missing port standing for the scheme's default and the opaque null origin never matching. A unit test covers the equal and unequal cases, and the settings and example config wording cite the RFC.

The client-fixed page script posted the known word whenever the resolved marker was absent, relying on the server's gate alone. A page module is treated like any other provider: it declares the permissions it requires and checks them against the resolved state the server hands the page before it does anything. The script now declares necessary.operations.storage, the same Data Use the server-side provider requires, waits on tsjs.whenPermissions() so the check works whether the state arrived at head open or at the body seam, and posts only when that permission is set. With no permission state on the page it does not post. The server still enforces the same gate on the resolve endpoint, so this is the page's half of one decision rather than a second policy. The Rust test that keeps the script's constants in step with the provider now also asserts that REQUIRED_PERMISSION matches Permission::StoreOnDevice.as_str(). Vitest covers the permission unset, no state, and state-arrives-later cases.

Fifth and final slice of the PR 838 decomposition: - Documentation for the provider model: configuration reference for the [ec], [device], and [geo] sections (including the required default_country, the assume_single_jurisdiction acknowledgment, and the requires-signal floor on a failed lookup), the Edge Cookie guide rewritten around providers and the permission model (including the narrow withdrawal semantics, the hardened resolve endpoint, and the resolved-marker cookie), setup and error-reference updates, and the permission-model guide joins the docs navigation. - The example configuration documents every provider selector in one place, with the [geo] baseline explained and acknowledged. - The HostSignals trait moves to the top of the evidence module with its service framing documented. - The Fastly EC lifecycle integration-test job joins the test workflow. The earlier draft of this slice carried an IntegrationResponseMutator response-header hook. It shipped with no consumer, so it is not included: the hook returns together with the first integration that needs it. Two specs land with this slice: the migration and rollout spec (docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md), its sign-off ledger kept intact with each row the series implements marked with its PR, and the response-header hook spec (docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md), retained as the design bar for the hook when its first consumer arrives.

The rollout spec cited commit SHAs from a rebuilt branch, credited the series with a passphrase tightening that pre-dates it, said graph rows were keyed verbatim when the pre-series normalization already lowercased them, attributed the Fastly-only identity endpoints to PR 1046, and claimed a CLI regression test for the provider overrides that does not exist. The hook spec said the fastly device provider is startup-rejected (it ships opt-in), that integrations cannot touch response headers (request filters can name them) and that the wasm target builds with panic = abort (it has no unwinding at all). The configuration guide now lists every Edge Cookie and geo provider value, the API reference documents each resolve response and the coded identifier form, and the setup guide shows the hmac~ prefix a minted cookie carries.

The no-client-IP test now derives identity from the request query and cookies through `EvidenceCapturingProvider`, which left `NoClientIpProvider` with no remaining caller. It is deleted rather than left looking like a live fixture, and the branch builds without the dead-code warning that denied clippy. That test's comment claimed the provider receives the documented unavailable value, the empty string, but the rewrite no longer observes what the provider was handed, so nothing proved it. The comment now says only what the test asserts, which is that a provider reading other evidence still mints when the host can determine no client IP. Restoring the stronger assertion would mean recording the client IP the provider saw, which is a change to the fixture rather than to this cleanup. The `http::HeaderMap` import in the device tests was unused, because the one place needing the type names it in full. Addresses: crates/trusted-server-core/src/ec/mod.rs and crates/trusted-server-core/src/ec/device.rs, which carried a dead test provider, a comment asserting more than the test proved, and an unused import.

The rewrite that replaced the old no-client-IP fixture left its comment claiming the provider receives the documented unavailable value while nothing asserted it, so the contract went unchecked and the comment said more than the test knew. The shared evidence-capturing provider now records the client IP it was given alongside the query parameter and cookie it already recorded, and the test asserts the empty string reaches it. A provider cannot decide how to behave on a host that cannot determine a client IP without knowing what absence looks like, so that value is part of the contract rather than an implementation detail. Verified by falsifying the expectation and watching the assertion report the real value it observed rather than passing regardless. Addresses: crates/trusted-server-core/src/ec/mod.rs, a comment claiming more than its test proved

The core library is exercised only through the WebAssembly adapter targets, which build with panic=abort. Their harness stops at the first failing test and reports every later one as never run, so a red build shows one failure when there may be several, and the ones it hides are invisible until the first is fixed and the suite is run again. This adds a host-target run of the same crate to the job that already runs the CLI and codegen tests natively, so every failure is reported at once. It costs one more compilation of a crate the job has already built for other targets. Addresses: .github/workflows/test.yml, core failures hidden behind panic=abort

The permission baseline and consent handling for a request the geo provider leaves unmatched come from the top of the permissions.yaml rules tree, so the [geo] reference no longer documents a default_country field. The deployer states the policy in that file, and the software encodes no jurisdiction's law.

The paragraph still placed the permissions file at the repository root and referenced the retired default_country. It now points at the config/permissions sample and the top-of-tree fallback.

jwrosewell mentioned this pull request Aug 19, 2026

Add pluggable Edge Cookie, device, and geo providers gated by a permission model #838

Draft

aram356 assigned jwrosewell Aug 19, 2026

jwrosewell force-pushed the split/5-response-hook-docs branch 3 times, most recently from 7ebce99 to 700c913 Compare August 25, 2026 10:51

jwrosewell mentioned this pull request Aug 25, 2026

Add a pluggable Edge Cookie provider seam with the built-in HMAC provider #1043

Open

jwrosewell changed the title ~~Add the integration response-header hook and the provider documentation set~~ Add the provider documentation set and finish the decomposition Aug 25, 2026

jwrosewell force-pushed the split/5-response-hook-docs branch 4 times, most recently from c17a7ea to 5b63f48 Compare August 27, 2026 05:37

jwrosewell mentioned this pull request Aug 27, 2026

Add the integration provider seam design spec #1084

Open

jwrosewell force-pushed the split/5-response-hook-docs branch from 5b63f48 to 0bab4c0 Compare August 27, 2026 15:10

jwrosewell force-pushed the split/5-response-hook-docs branch 2 times, most recently from 3cfe393 to 45acb97 Compare August 31, 2026 12:50

jwrosewell mentioned this pull request Aug 31, 2026

Open the integration seam so a vendor module can live outside core #1094

Open

This was referenced Aug 31, 2026

Upgrade EdgeZero to v0.0.7 and complete staging runtime integration #940

Merged

Thread EC KV read through the request and recover orphaned cookies #885

Open

Honor EdgeZero app config store default #879

Open

jwrosewell force-pushed the split/5-response-hook-docs branch 6 times, most recently from 98764db to d48c98d Compare September 1, 2026 23:04

jwrosewell added 3 commits September 2, 2026 17:07

jwrosewell force-pushed the split/5-response-hook-docs branch from f54e08a to acfd31e Compare September 2, 2026 16:32

github-advanced-security Bot found potential problems Sep 2, 2026

View reviewed changes

Comment thread .github/workflows/inspector.yml Fixed

Mark the inspector build script executable

5fa76c9

CI failed with permission denied because the executable bit was never recorded in git, the usual Windows-authored-script gap.

jwrosewell force-pushed the split/5-response-hook-docs branch from acfd31e to 1fd2dc5 Compare September 2, 2026 16:39

Limit the inspector workflow token to reading contents

ae055e1

The code-scanning bot flagged the new workflow for carrying the default GITHUB_TOKEN permissions. The job only checks out and builds, so it reads contents and nothing else.

jwrosewell force-pushed the split/5-response-hook-docs branch from 1fd2dc5 to 0418d98 Compare September 2, 2026 16:55

jwrosewell added 20 commits September 2, 2026 20:35

Point the compiled-in rules doc at config/permissions

d87711f

The doc comment above the include_str still said the file lives at the repository root after the move to config/permissions/vanilla.yaml.

Documentation and house-style cleanup for the client-resolve PR

b09c7c5

Align the header-accumulation test with the reserved cache-control su…

0e4bfb8

…rface

Documentation and house-style cleanup for the response-hook and docs PR

4d7a1c6

Correct the configuration guide's permissions file location

8608f73

The paragraph still placed the permissions file at the repository root and referenced the retired default_country. It now points at the config/permissions sample and the top-of-tree fallback.

jwrosewell force-pushed the split/5-response-hook-docs branch from 0418d98 to 8608f73 Compare September 2, 2026 19:39

Navigation Menu

Uh oh!

Add the provider documentation set and finish the decomposition - #1047

Add the provider documentation set and finish the decomposition#1047
jwrosewell wants to merge 96 commits into
IABTechLab:mainfrom
jwrosewell:split/5-response-hook-docs

jwrosewell commented Aug 19, 2026 •

edited

Loading

Uh oh!

aram356 commented Aug 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jwrosewell commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

What happens to #838

How it was verified

Uh oh!

aram356 commented Aug 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jwrosewell commented Aug 19, 2026 •

edited

Loading