Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
807 changes: 543 additions & 264 deletions dist/deploy.js

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions dist/deploy.js.map

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions dist/resolve-image.js.map

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions dist/tag-image.js.map

Large diffs are not rendered by default.

63 changes: 63 additions & 0 deletions docs/pipeline-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,8 @@ services }` contracts.
3. update each service, rewriting **only** the app container (sidecars such as
the Datadog agent are preserved), re-attaching RUNTIME secrets as
`secretKeyRef:latest`, and forcing a new revision.
4. publish the IAP friendly sign-in page carried by the image, if the app has
one (see below).
- **transient-failure tolerance**: the AWS SDK clients already retry
(`maxAttempts: 5`); the Artifact Registry **REST** calls did not, and a
transient AR 503 failed a pilot rollback at the resolve step (pre-mutation — the
Expand All @@ -202,6 +204,66 @@ services }` contracts.
through. The POST tag-create retry is safe: an `alreadyExists` on a retried
create is already treated as success (`addTag`'s 409 → move handler).

### IAP friendly sign-in page (Cloud Run)

With `iap.friendly_signin`, the `gcp/cloudrun/app` module puts a GCS-hosted
sign-in page in front of an otherwise-gated app. Terraform owns the bucket and
the load-balancer routing; something has to own the bytes. Under v1 that was a
workflow in the app repo, which does not survive v2:

- **Production had no automated path.** v2 promotes an already-built image, so no
app-repo workflow fires on a prod deploy. The upload was a manual dispatch.
- **Rollback put an old app behind a new page.** The page inlines the app's own
CSS, so it is version-specific, but nothing tied it to the deployed digest.

So the page travels **inside the image**, and the deploy publishes it. Same
artifact, same digest, same guarantees as the app: `promote` and `rollback` carry
the matching page for free, and the only identity that can write the bucket is
the deployer.

**Contract with the app image** — two lines in the Dockerfile:

```dockerfile
LABEL org.cru.iap-signin=signin
COPY --from=builder /app/tmp/static-export/signin /cru/iap-signin/signin
```

The label value is the **GCS object key**, and it is the single source of truth
for the name — so the image and the bucket cannot disagree. It must equal the
module's `iap.friendly_signin.path` minus its leading slash, or the LB serves a
404 for a page that uploaded successfully.

**Detection is bucket-first.** Terraform injects `IAP_SIGNIN_BUCKET` into the app
container, so the deploy reads it off the (pre-update) service spec — no new
action input, no app-info field, and no registry read at all for the apps that
have no sign-in page. Only when a bucket is present does the deploy read the
image's labels.

**Extraction reads one blob, not the image.** `src/v2/oci.js` speaks just enough
of the Docker Registry v2 API to fetch the config blob (labels) and then scan
layers **newest-first** for the file, stopping at the first hit. A page COPYed
late in the Dockerfile is found in the first, tiny blob fetched — no docker
daemon, no `crane`, no full pull. `src/v2/tar.js` is a minimal ustar reader for
the same reason: one fixed short path, no new dependency in a shared repo.

**Failure is never fatal.** The page is static, cosmetic and pre-auth — the
module README calls a few minutes of staleness a non-outage — so failing a
production promote over it would trade a real problem for a trivial one. The
publish runs **last** (a deploy that fails partway must not leave a new page in
front of an old app) and any failure becomes a `::warning::` annotation on the
run. Two cases warn rather than error by design:

| Case | Behavior |
| ---- | -------- |
| No `IAP_SIGNIN_BUCKET` on the service | Silent no-op; the app has no sign-in page |
| Bucket present, image carries no label | Warning; existing object left alone. **Expected** when rolling back to a release built before this contract, and during the rollout |
| Label present, file missing at the conventional path | Warning; a real Dockerfile bug |
| Upload rejected (IAM, transient) | Warning; upload POSTs retry 5× first |

> **Follow-up:** the publish result is returned from `deployCloudRun` but not yet
> surfaced as an action output, so it does not reach the promote/rollback Slack
> messages. A skipped publish is currently visible only as a run annotation.

### ECS implementation notes

ECS shares the Cloud Run action contracts but derives everything from the env
Expand Down Expand Up @@ -1008,6 +1070,7 @@ actions page.
| app **prod** build SA (`github-actions@<prod-project>`) | **AR writer** on the app's `cru-shared-artifacts/<app>` repo | `build-candidate` pushes `candidate-*`/`sha-*` |
| each env's `cru-deploy@<env-project>` SA | **AR reader** on `cru-shared-artifacts/<app>` | `resolve-image` reads tags/digests |
| **prod** `cru-deploy@<prod-project>` SA | **AR writer** on `cru-shared-artifacts/<app>` | `promote` adds the `release-*` tag |
| each env's `cru-deploy@<env-project>` SA | **`roles/storage.objectAdmin`** on that env's `<project-id>-iap-signin` bucket | `deploy` publishes the IAP sign-in page out of the image. Only for apps with `iap.friendly_signin`; granted by the `gcp/cloudrun/app` module |
| `cru-deploy` control repo | `authz-token` secret (pilot: `CRU_DEVOPS_GITHUB_TOKEN`) | promote/rollback collaborator-permission check |
| `cru-deploy` control repo | `vars.GCP_WORKLOAD_IDENTITY_PROVIDER` + WIF trust so each env's `cru-deploy` SA is impersonable | GCP auth in deploy-candidate/promote/rollback |

Expand Down
31 changes: 30 additions & 1 deletion src/v2/deploy-cloudrun.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import * as core from '@actions/core'
import { cloudrunListJobs, cloudrunListServices, listSecrets, runJob, updateJob, updateService } from '../gcp'
import { RUNTIME_PARAM_TYPES } from '../ecs-config'
import { assertDigestRef, isAppContainer, parseImageRef } from './gcp'
import { publishSigninPage, signinBucket } from './signin'

// Name of the optional database-migrations Cloud Run job, created by the
// gcp/cloudrun/app terraform module when `database_migrations` is enabled.
Expand Down Expand Up @@ -80,7 +81,35 @@ export async function deployCloudRun ({ image, runtimeProject }) {
updatedServices.push(shortName(service.name))
}

return { deployedImage: image, services: updatedServices }
// Publish the IAP friendly sign-in page carried by this image, if the app has
// one. Read the bucket off the services as they were BEFORE the update above:
// Terraform injects IAP_SIGNIN_BUCKET into the app container, so its presence
// is what tells us this environment expects a page (see ./signin.js).
//
// Runs last, and never fails the deploy. The page is cosmetic, static and
// pre-auth — the module README calls a few minutes of staleness a non-outage —
// so failing a production promote over it would trade a real problem for a
// trivial one. A warning annotation makes the skew visible on the run instead.
const signin = { published: false }
const bucket = signinBucket(services, repo)
if (bucket) {
try {
Object.assign(signin, await publishSigninPage({ image, bucket }))
if (signin.published) {
core.info(`published sign-in page: gs://${bucket}/${signin.objectKey} (${signin.bytes} bytes)`)
} else {
core.warning(
`${bucket} expects a sign-in page but ${image} does not carry one; ` +
'leaving the existing object in place. Expected in a rollback to a ' +
'release built before the image carried the page.'
)
}
} catch (error) {
core.warning(`sign-in page not published (deploy unaffected): ${error.message}`)
}
}

return { deployedImage: image, services: updatedServices, signin }
}

// Update a Cloud Run job's container image and secrets in place. A job has a
Expand Down
7 changes: 5 additions & 2 deletions src/v2/gcp.js
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,11 @@ const GAXIOS_RETRY = {
}

// Obtain an authenticated Google API client (ADC / workload-identity on the
// runner). Split out so tests can mock google-auth-library.
async function authClient () {
// runner). Split out so tests can mock google-auth-library. Exported for
// src/v2/oci.js, which talks to a different Google endpoint family (the Docker
// registry API on *-docker.pkg.dev, and the GCS JSON API) with the same
// credentials and scope.
export async function authClient () {
const auth = new GoogleAuth({ scopes: ['https://www.googleapis.com/auth/cloud-platform'] })
return auth.getClient()
}
Expand Down
170 changes: 170 additions & 0 deletions src/v2/oci.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
// Read-only OCI registry client: enough of the Docker Registry v2 API to read
// an image's labels and pull one small file out of its filesystem, without a
// docker daemon and without materializing the whole image.
//
// Why not `docker pull` / `crane`: the deploy runner has neither a daemon warmed
// up nor a binary to install, and both would download every layer. The only
// consumer (src/v2/signin.js) wants a ~100KB file that the app's Dockerfile
// COPYs in a late, tiny layer, so scanning layers newest-first normally reads
// exactly one small blob.
//
// Artifact Registry serves the Docker v2 API at
// https://<location>-docker.pkg.dev/v2/<project>/<repo>/<image>/... and accepts
// a plain OAuth bearer token, so the same ADC credentials the rest of the deploy
// uses work here. Requires roles/artifactregistry.reader, which every env's
// cru-deploy SA already holds for resolve-image.
import * as core from '@actions/core'
import { gunzipSync, zstdDecompressSync } from 'node:zlib'
import { authClient, parseImageRef } from './gcp'
import { findInTar } from './tar'

// Manifest media types we can read. Both spellings of both formats, plus the
// multi-platform index/list wrappers buildx emits even for a single platform.
const MANIFEST_ACCEPT = [
'application/vnd.oci.image.manifest.v1+json',
'application/vnd.docker.distribution.manifest.v2+json',
'application/vnd.oci.image.index.v1+json',
'application/vnd.docker.distribution.manifest.list.v2+json'
].join(', ')

// Platform we deploy. Cloud Run runs linux/amd64; an index's other entries
// (notably buildx attestation manifests, which carry `unknown/unknown`) are
// skipped.
const PLATFORM = { os: 'linux', architecture: 'amd64' }

// Registry reads are pure GETs, so retrying them is unconditionally safe. Same
// tolerance the Artifact Registry REST calls in ./gcp.js use, and for the same
// reason: a transient AR 503 must not fail a deploy.
const GAXIOS_RETRY = {
retry: true,
retryConfig: {
retry: 5,
retryDelay: 500,
httpMethodsToRetry: ['GET'],
statusCodesToRetry: [[429, 429], [500, 599]]
}
}

// Split a digest-pinned reference into the pieces the v2 API needs:
// us-central1-docker.pkg.dev/cru-shared-artifacts/bills/bills@sha256:abc
// -> { host, repository: 'cru-shared-artifacts/bills/bills', reference: 'sha256:abc' }
export function parseRegistryRef (ref) {
const { name, digest, tag } = parseImageRef(ref)
const slash = name.indexOf('/')
if (slash === -1) {
throw new Error(`Image reference "${ref}" has no registry host`)
}
const reference = digest ?? tag
if (!reference) {
throw new Error(`Image reference "${ref}" is not pinned to a digest or tag`)
}
return { host: name.slice(0, slash), repository: name.slice(slash + 1), reference }
}

// GET a registry URL. `responseType` is 'text' for JSON documents (the registry
// labels them application/vnd.*+json, which gaxios will not auto-parse) and
// 'arraybuffer' for blobs.
async function registryGet ({ host, repository, kind, reference, accept, responseType }) {
const client = await authClient()
const res = await client.request({
url: `https://${host}/v2/${repository}/${kind}/${reference}`,
method: 'GET',
headers: accept ? { Accept: accept } : {},
responseType,
...GAXIOS_RETRY
})
return res.data
}

async function manifestDocument (target, reference) {
const body = await registryGet({
...target,
kind: 'manifests',
reference,
accept: MANIFEST_ACCEPT,
responseType: 'text'
})
return typeof body === 'string' ? JSON.parse(body) : body
}

// Pick this platform's manifest out of an index/list.
function selectPlatform (index) {
const candidates = (index.manifests ?? []).filter(
entry => entry.platform?.os === PLATFORM.os && entry.platform?.architecture === PLATFORM.architecture
)
if (candidates.length === 0) {
const seen = (index.manifests ?? [])
.map(entry => `${entry.platform?.os ?? '?'}/${entry.platform?.architecture ?? '?'}`)
.join(', ')
throw new Error(
`Image index has no ${PLATFORM.os}/${PLATFORM.architecture} manifest (found: ${seen || 'none'})`
)
}
return candidates[0].digest
}

// Decompress a layer blob according to its media type.
function decompressLayer (mediaType, blob) {
if (mediaType.includes('zstd')) return zstdDecompressSync(blob)
if (mediaType.includes('gzip')) return gunzipSync(blob)
return blob
}

// Layers that are not a filesystem diff we can read.
function isReadableLayer (mediaType) {
// "foreign"/"nondistributable" layers live on another host entirely.
return mediaType.includes('.tar') && !mediaType.includes('foreign') && !mediaType.includes('nondistributable')
}

/**
* Open a digest-pinned image for reading: resolves the platform manifest and
* fetches the (small) config blob so labels are available synchronously.
*
* Returns { labels, readFile(path) }. `readFile` scans layers newest-first and
* returns a Buffer, or null when no layer contains the path.
*/
export async function openImage (imageRef) {
const target = parseRegistryRef(imageRef)

let manifest = await manifestDocument(target, target.reference)
// An index/list wraps per-platform manifests; resolve one more hop.
if (manifest.manifests) {
manifest = await manifestDocument(target, selectPlatform(manifest))
}
if (!manifest.config?.digest) {
throw new Error(`Image manifest for ${imageRef} has no config descriptor`)
}

const configBody = await registryGet({
...target,
kind: 'blobs',
reference: manifest.config.digest,
responseType: 'text'
})
const config = typeof configBody === 'string' ? JSON.parse(configBody) : configBody

return {
labels: config.config?.Labels ?? {},

async readFile (path) {
// Newest layer first: a file COPYed late in the Dockerfile is found in the
// first (and typically tiny) blob we fetch, and a path rewritten by a
// later layer resolves to the version the container would actually see.
const layers = (manifest.layers ?? []).filter(layer => isReadableLayer(layer.mediaType))
for (const [index, layer] of [...layers].reverse().entries()) {
const blob = await registryGet({
...target,
kind: 'blobs',
reference: layer.digest,
responseType: 'arraybuffer'
})
const found = findInTar(decompressLayer(layer.mediaType, Buffer.from(blob)), path)
if (found) {
core.info(`found ${path} in layer ${layers.length - index}/${layers.length} (${layer.digest})`)
return found
}
}
return null
}
}
}
Loading
Loading