ci: run the full CI for the #235 → #236 → #237 stack (do not merge)

.github/workflows/ci.yml

@@ -85,7 +85,7 @@ jobs:
     permissions:
       contents: read
       # Mint a GitHub OIDC token as the write tests' caller identity — the
-      # worker verifies it against GitHub's JWKS (AUTH_ISSUER below).
+      # worker verifies it against GitHub's JWKS (PLATFORM_ISSUERS below).
       id-token: write
     steps:
       - uses: actions/checkout@v4
@@ -106,16 +106,20 @@ jobs:
         # dependency in this job can't mint federation assertions any AWS role
         # trusts. Federated end-to-end coverage lives in the deployed-environment
         # smoke tests (tests/test_federation.py, wired into staging.yml) instead.
-        # Inbound callers authenticate with GitHub Actions OIDC tokens:
-        # AUTH_ISSUER points at GitHub and AUTH_AUDIENCE matches the audience
-        # requested in "Mint caller identity token" below.
+        # Inbound callers authenticate with GitHub Actions OIDC tokens. GitHub
+        # is a platform issuer, as in production: PLATFORM_ISSUERS names the
+        # audience requested in "Mint caller identity token" below, and a
+        # token acts as an account the stub says trusts this repository. No
+        # person-issuer token exists in CI; AUTH_ISSUER names an issuer that
+        # mints nothing, and AUTH_AUDIENCE keeps /.sts from answering 501.
         run: |
           {
             openssl genpkey -algorithm RSA -out /tmp/oidc.pem -pkeyopt rsa_keygen_bits:2048
             printf 'OIDC_PROVIDER_KEY="%s"\n' "$(cat /tmp/oidc.pem)"
             echo "SESSION_TOKEN_KEY=$(openssl rand -base64 32)"
-            echo "AUTH_ISSUER=https://token.actions.githubusercontent.com"
+            echo "AUTH_ISSUER=https://auth.example.invalid"
             echo "AUTH_AUDIENCE=source-data-proxy-ci"
+            echo 'PLATFORM_ISSUERS={"https://token.actions.githubusercontent.com": ["source-data-proxy-ci"]}'
             echo "SOURCE_API_URL=http://localhost:9000"
           } > .dev.vars
       - name: Mint caller identity token (GitHub OIDC)
@@ -144,9 +148,9 @@ jobs:
           fi
           echo "::add-mask::$token"
           echo "CI_WRITE_ID_TOKEN=$token" >> "$GITHUB_ENV"
-          # A second, validly-signed token whose audience mismatches
-          # AUTH_AUDIENCE: test_writes.py asserts the /.sts aud gate rejects
-          # it (signature checks alone would let it through).
+          # A second, validly-signed token whose audience is not GitHub's in
+          # PLATFORM_ISSUERS: test_writes.py asserts the /.sts aud gate
+          # rejects it (signature checks alone would let it through).
           wrong=$(curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
             "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=not-the-data-proxy" | jq -r '.value')
           if [ -z "$wrong" ] || [ "$wrong" = "null" ]; then

.github/workflows/staging.yml

@@ -58,14 +58,12 @@ jobs:
         # A GitHub Actions OIDC token, minted per run — short-lived by design
         # and never stored, unlike a token parked in a repo secret.
         #
-        # Dormant until Source registers GitHub as a valid IdP, so that products
-        # can accept writes from GitHub Actions. Two things must land first:
-        # the deployment's AUTH_ISSUER must accept GitHub's issuer (today it is
-        # a single Ory URL — src/config.rs reads AUTH_ISSUER as one String,
-        # unlike the comma-separated AUTH_AUDIENCE, so this needs a code change
-        # too), and the audience Source expects must be set as
-        # FEDERATION_TEST_AUDIENCE. Until then no token is minted and the
-        # copy-source authz test skips; the rest of the suite is unaffected.
+        # Dormant until FEDERATION_TEST_AUDIENCE is set to the staging proxy's
+        # origin, the audience staging's PLATFORM_ISSUERS accepts for GitHub,
+        # and FEDERATION_TEST_TRUST_ACCOUNT to a staging service account that
+        # trusts this repository's workflows (ADR-014): the token acts as that
+        # account. Until then no token is minted and the copy-source authz
+        # test skips; the rest of the suite is unaffected.
         if: vars.FEDERATION_TEST_AUDIENCE != ''
         run: |
           set -euo pipefail
@@ -88,6 +86,7 @@ jobs:
           FEDERATION_WRITE_PRODUCT: ${{ vars.FEDERATION_WRITE_PRODUCT }}
           # Set by the mint step above, and only when it runs.
           CI_WRITE_ID_TOKEN: ${{ env.CI_WRITE_ID_TOKEN }}
+          CI_TRUST_ACCOUNT: ${{ vars.FEDERATION_TEST_TRUST_ACCOUNT }}
         # boto3: the copy-source authz test signs SigV4 through the AWS SDK
         # rather than hand-rolling requests.
         run: uvx --with requests --with boto3 pytest tests/test_federation.py -v

Cargo.toml

@@ -34,6 +34,10 @@ path = "tests/object_path.rs"
 name = "fixtures"
 path = "tests/fixtures.rs"
 
+[[test]]
+name = "keys"
+path = "tests/keys.rs"
+
 [dependencies]
 # Multistore
 multistore = { version = "0.7.2", features = ["azure", "gcp"] }
@@ -53,6 +57,9 @@ percent-encoding = "2"
 hmac = "0.12"
 sha2 = "0.10"
 
+# Reading a platform IdP's token before verifying it (issuer, key id)
+base64 = "0.22"
+
 # Tracing
 tracing = "0.1"
 

README.md

@@ -110,12 +110,41 @@ Set in `wrangler.toml` or via the Cloudflare dashboard:
 | ---------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
 | `SOURCE_API_URL`             | `https://source.coop`       | Source Cooperative API base URL                                                                                                    |
 | `LOG_LEVEL`                  | `WARN`                      | Tracing level (`TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`)                                                                          |
-| `AUTH_ISSUER`                | `https://auth.source.coop`  | OIDC issuer trusted for `/.sts` token exchange                                                                                     |
+| `AUTH_ISSUER`                | `https://auth.source.coop`  | The person issuer trusted for `/.sts` token exchange; its tokens act as their own subject                                          |
 | `AUTH_AUDIENCE`              | —                           | Comma-separated OAuth client ID(s) that `/.sts` subject tokens must be issued to (`aud` claim); a token is accepted if it matches any. Unset = `/.sts` token exchange is disabled (returns 501) |
+| `PLATFORM_ISSUERS`           | —                           | JSON object from each platform issuer URL to the audiences its tokens must carry, such as `{"https://token.actions.githubusercontent.com": ["https://data.source.coop"]}`. An issuer with no audience is refused. Unset = no platform issuer is trusted |
 | `OIDC_PROVIDER_ISSUER`       | `https://data.source.coop`  | Issuer URL for minted JWTs and OIDC discovery                                                                                      |
 | `OIDC_PROVIDER_KID`          | `data-proxy-1`              | Key ID for the active signing key                                                                                                  |
 | `OIDC_PROVIDER_KID_PREVIOUS` | —                           | Key ID for the previous key (during rotation)                                                                                      |
 
+### Bindings
+
+| Binding              | Kind        | Description                                                                                                                                  |
+| -------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
+| `STS_EXCHANGE_LIMIT` | `ratelimit` | Per-client-IP limit on `/.sts` exchanges that cost a Source API call: API keys (ADR-013) and platform tokens (ADR-014). Declared under `[[unsafe.bindings]]` in every `wrangler*.toml`; a deployment without it logs an error and exchanges without a limit |
+
+### API keys
+
+A service account's API key (ADR-013) is an opaque `sck_` secret that source.coop stores as a hash. It is presented at `/.sts` as `WebIdentityToken`, from a POST form body only — a key in the URL is refused, because the URL is logged. The proxy trims and format-checks it, hashes it, and asks `POST {SOURCE_API_URL}/api/v1/service-account-keys/exchanges` for its standing as itself (subject `urn:source:data-proxy`), caching the answer for 60 seconds; then it mints credentials for the account the API names, exactly as it would for an ID token. Every refusal of the key reads `API key was not accepted (request id …)`; the reason is in the log under that id.
+
+### Roles
+
+Every exchange at `/.sts`, of an ID token or an API key, names a Role in `RoleArn`, either bare or as the resource of an ARN of any partition and account (`arn:aws:iam::000000000000:role/ReadOnly`), since AWS SDKs insist on an ARN. The account matters only to a platform token (below). The Roles are hardcoded (ADR-014):
+
+| Role         | Credentials may                                          |
+| ------------ | -------------------------------------------------------- |
+| `FullAccess` | do everything the account's memberships allow            |
+| `ReadOnly`   | do the same, except write                                |
+| `_default`   | do what `FullAccess` does; the name existing clients use |
+
+Any other name is refused with `MalformedPolicyDocument`, never mapped to a default. A Role only subtracts: its ceiling is sealed into the session token and checked locally before the account's own permissions are looked up (ADR-011), and a request it refuses gets the same `AccessDenied` as any other refusal.
+
+### Platform identity providers
+
+A token from a platform issuer in `PLATFORM_ISSUERS`, such as GitHub Actions, says which workload is calling but not which account it may act as. At `/.sts` it acts as the service account in `RoleArn`, `arn:aws:iam::<owner>--<name>:role/FullAccess`, and only if that account trusts the token's issuer and subject (ADR-014); an account that is not a service account is refused before anything else. The proxy verifies the token against the issuer's JWKS, with that issuer's own audiences and a required `exp`, then, within `STS_EXCHANGE_LIMIT`, asks `POST {SOURCE_API_URL}/api/v1/accounts/{account}/trusts/exchanges` with `{"issuer", "subject"}`, as the account. Per account, issuer and subject, a yes is cached for 60 seconds and a no for 10, and the credentials' principal is the account, never the token's subject. Every refusal reads `AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity (request id …)`. A token from `AUTH_ISSUER` still acts as its own subject and ignores the account in `RoleArn`.
+
+`aws-actions/configure-aws-credentials` fails after the exchange succeeds: it checks the credentials it exports with `GetCallerIdentity`, which the proxy cannot answer until developmentseed/multistore#126 lands. Until then a workflow saves its token to a file and lets an AWS SDK exchange it, with `AWS_WEB_IDENTITY_TOKEN_FILE`, `AWS_ROLE_ARN`, `AWS_ENDPOINT_URL_STS=<proxy>/.sts`, `AWS_ENDPOINT_URL_S3=<proxy>` and `AWS_REGION`.
+
 ### Secrets
 
 **GitHub environment secrets are the source of truth.** The deploy workflow

adrs/001-s3-credentials.md

@@ -57,17 +57,17 @@ The sealed payload carries:
 | `access_key_id` | The identifier the caller signs with |
 | `secret_access_key` | The signing secret, recovered by unsealing |
 | `expiration` | Enforced at unseal time; an expired token fails closed |
-| `assumed_role_id` | The Role assumed at exchange time (currently always `_default`) |
-| `source_identity` | The original OIDC `sub` — the caller's Ory identity |
-| `allowed_scopes` | Scope ceiling sealed at mint time — currently empty, and not consulted on this path (see below) |
+| `assumed_role_id` | The Role assumed at exchange time: `_default`, `FullAccess` or `ReadOnly` (ADR-004) |
+| `source_identity` | Who the credentials act as: an Ory ID token's `sub`, or the account an API key or a trusted platform token names (ADR-013, ADR-014) |
+| `allowed_scopes` | The Role's ceiling, sealed at mint time: empty for `FullAccess` and `_default`, reads of every product for `ReadOnly` (see below) |
 | `session_token` | A discarded random placeholder. The credential set is sealed *before* this field is overwritten with the sealed blob, so the value inside the envelope is not the token itself |
 
 Key properties of this design:
 
 - **Verification is fully stateless.** The proxy decrypts the token on each request and recovers the `SecretAccessKey` directly. No database lookup, no key derivation, and no asymmetric verification on the request hot path — which matters on Workers, where in-memory state does not persist across invocations.
 - **The token is opaque to the caller.** Unlike a JWT, a client cannot read the sealed payload. Scope and identity metadata are not disclosed to whoever holds the credential.
-- **`allowed_scopes` is sealed but not enforced on this path.** Its only consumer, `multistore::auth::authorize`, has no call site in the pinned crate; the gateway delegates authorization to the bucket registry instead (ADR-005). Where scopes *are* evaluated, an empty vec means **deny-all**, not unlimited — which is why the registry overrides `authorize_key` rather than inheriting the default. The effective behaviour is "no ceiling", but by bypass rather than by an empty-means-unlimited rule. ADR-011 is where this field would become load-bearing, and wiring it up is part of that work rather than a given.
-- **`source_identity` preserves the original subject**, which is what the proxy presents to the policy store (see ADR-005).
+- **`allowed_scopes` is enforced by the bucket registry, not by multistore.** multistore's own consumer, `multistore::auth::authorize`, has no call site in the pinned crate; the gateway delegates authorization to the registry instead (ADR-005), which checks the ceiling before any lookup (ADR-011, #236). The registry reads an empty vec as **no ceiling**, the reverse of `authorize`, where empty means deny-all — which is also why the registry overrides `authorize_key` rather than inheriting the default. The only non-empty ceiling is `ReadOnly`'s: every product (`*`), read actions only.
+- **`source_identity` is the principal**, which is what the proxy presents to the policy store (see ADR-005): the original subject for an Ory ID token, never a platform token's subject.
 - **Authenticated encryption.** GCM provides integrity as well as confidentiality: a tampered token fails to decrypt rather than decoding into attacker-chosen values.
 
 ### SigV4 Verification Flow

adrs/004-sts.md

@@ -5,7 +5,7 @@
 **RFC:** RFC-001 §7
 **Depends on:** ADR-001
 **Implementation:** `src/sts.rs`, `src/lib.rs`, `src/config.rs`; `source.coop:src/lib/actions/proxy-credentials.ts`
-**Implemented by:** #116 (initial `/.sts` exchange), #163 (multiple accepted audiences), #165 (configurable max session duration), #185 (ARN-shaped `_default` alias), #196 (form-encoded POST bodies, wiring [multistore#112](https://github.com/developmentseed/multistore/pull/112)) · source.coop#283 (OIDC auth), source.coop#391 (in-browser uploads via the proxy), source.coop#402 (mid-upload credential refresh)
+**Implemented by:** #116 (initial `/.sts` exchange), #163 (multiple accepted audiences), #165 (configurable max session duration), #185 (ARN-shaped `_default` alias), #196 (form-encoded POST bodies, wiring [multistore#112](https://github.com/developmentseed/multistore/pull/112)), #236 (`FullAccess` and `ReadOnly` Roles), #237 (platform issuers) · source.coop#283 (OIDC auth), source.coop#391 (in-browser uploads via the proxy), source.coop#402 (mid-upload credential refresh)
 
 ---
 
@@ -73,10 +73,16 @@ A single built-in Role, `_default`, is served from a hardcoded registry:
 
 `RoleArn` is accepted either literally as `_default` or as an ARN-shaped alias whose resource is `role/_default` (e.g. `arn:aws:iam::000000000000:role/_default`, any partition or account ID). The alias exists because AWS SDKs validate `RoleArn` client-side — ARN shape, 20-character minimum — before the request is ever sent, so a bare `_default` cannot reach the server from unmodified tooling. The partition and account portions carry no meaning here and are ignored rather than validated.
 
+> [!NOTE]
+> Two more hardcoded Roles are served alongside it (ADR-014, #236): `FullAccess`, of which `_default` is now an alias, and `ReadOnly`, whose read-only ceiling is sealed into the session and enforced by the bucket registry (ADR-011). All three take the same ARN form.
+
 ### Trust Model — Issuer and Audience
 
 **Issuer.** `AUTH_ISSUER` names the single trusted OIDC issuer: Source Cooperative's Ory-based auth system (`https://auth.source.coop`, or the staging equivalent). A token from any other issuer is rejected before any network call.
 
+> [!NOTE]
+> Platform issuers (ADR-009) are now trusted alongside it, each with its own audiences in `PLATFORM_ISSUERS`. Their tokens take a separate path ahead of the STS route and act as the account `RoleArn` names, if that account trusts the token's issuer and subject (ADR-014, #237).
+
 **Audience.** `AUTH_AUDIENCE` is a comma-separated allowlist of OAuth client IDs; a token is accepted if its `aud` matches any entry. Production lists the web frontend and `source-coop-cli`.
 
 **The audience restriction is load-bearing and the endpoint fails closed without it.** Without it, an ID token that a user granted to *any* third-party OAuth client registered with the issuer could be exchanged for that user's proxy credentials. When `AUTH_AUDIENCE` is unset the STS route is never mounted and `/.sts` returns `501 NotImplemented`, rather than being served unrestricted.
@@ -110,7 +116,7 @@ flowchart TD
 ```
 
 > [!WARNING]
-> **`exp` is only checked when the claim is present.** Upstream, both time claims are guarded by `if let Some(..)`, so a token carrying no `exp` is accepted and never expires. A missing `aud`, by contrast, is fail-closed. This is harmless with a single trusted issuer that always sets `exp` (Ory does), but it becomes a real exposure the moment ADR-009 admits issuers we do not control — it should be fixed upstream before then.
+> **`exp` is only checked when the claim is present.** Upstream, both time claims are guarded by `if let Some(..)`, so a token carrying no `exp` is accepted and never expires. A missing `aud`, by contrast, is fail-closed. This is harmless with a single trusted issuer that always sets `exp` (Ory does), but it becomes a real exposure the moment ADR-009 admits issuers we do not control — it should be fixed upstream before then. #237 admits them and requires `exp` on their tokens itself; the upstream fix, developmentseed/multistore#146, is not yet in a release.
 
 Steps 2 and 3 reject **before any network call**: the issuer is matched and the algorithm pinned prior to fetching JWKS, so a token from an untrusted issuer costs nothing to refuse.