Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/stop-publishing-container-outside-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
"nostream": patch
"nostream-website": patch
---

fix(ci): stop building and pushing container images outside a release

Remove the container publish job from checks.yml so pushes to main no longer publish
images to GHCR. Container image builds and pushes are now restricted exclusively to
release.yml on published releases. Update deployment documentation and reference compose
stacks to target the released image tag (:latest) instead of :main.
57 changes: 0 additions & 57 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,60 +227,3 @@ jobs:
run: pnpm install --frozen-lockfile
- name: Check for changeset
run: pnpm exec changeset status --since origin/${{ github.base_ref }}

publish-container-image:
name: Build and push container image
runs-on: ubuntu-latest
needs:
- changes
- lint
- build-check
- test-units-and-cover
- test-integrations-and-cover
- post-tests
if: >-
always() && github.event_name == 'push' && github.ref == 'refs/heads/main' &&
needs.changes.result == 'success' &&
(
needs.changes.outputs.src != 'true' ||
(
needs.lint.result == 'success' &&
needs.build-check.result == 'success' &&
needs.test-units-and-cover.result == 'success' &&
needs.test-integrations-and-cover.result == 'success'
)
) &&
needs.post-tests.result == 'success'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=main
type=sha,prefix=sha-

- name: Build and push Docker image
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
34 changes: 31 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ on:
jobs:
build-and-push-image:
# Changesets tags each workspace package on publish: relay releases tag
# vX.Y.Z, website releases tag nostream-website@X.Y.Z. Only a relay tag
# should build the relay container image.
if: startsWith(github.event.release.tag_name, 'v')
# nostream@X.Y.Z, website releases tag nostream-website@X.Y.Z. Only a relay
# tag should build the relay container image. The v* branch is kept for the
# releases cut before the repo moved to changesets tagging.
if: startsWith(github.event.release.tag_name, 'nostream@') || startsWith(github.event.release.tag_name, 'v')
name: Build and push container image
runs-on: ubuntu-latest
env:
Expand All @@ -32,6 +33,33 @@ jobs:
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Stated rather than inherited from the action's defaults: the deploy
# path pulls `:latest`, so a released image must carry it. Naming the
# tags here is what makes `latest` something this repository promises
# rather than a behaviour of a third-party action.
#
# Changesets tags a relay release `nostream@3.6.2`, which is not a
# semver. A semver rule derives its input from the git ref and emits
# NOTHING for it, and with no version the `latest` rule is skipped as
# well - so a relay release published no tag at all. `prefix=` cannot
# help (it is rendered after parsing), and `value=` cannot help either
# (it replaces the whole tag, leaving no room for the prefix).
#
# So the semver-shaped tag comes from a `match` rule that extracts the
# version out of the relay tag, and the release tag itself is resolved
# from the git ref, unchanged. The semver rule runs first so the bare
# version is the primary tag; the release tag is kept beside it, which
# is what a host sees on the GitHub release and pins.
#
# Verified by running this action revision's own Meta class over this
# block: a relay release yields `3.6.2`, `nostream@3.6.2` and `latest`;
# a legacy `v3.4.1` yields `3.4.1`, `v3.4.1` and `latest`; a website
# release matches neither semver nor `match`, and the job's `if:`
# never lets that job run.
tags: |
type=match,pattern=nostream@(.+),group=1
type=ref,event=tag
type=raw,value=latest
- name: Build and push Docker image
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
Expand Down
12 changes: 6 additions & 6 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ uses a pre-built image from GHCR; migrations and default settings ship inside
that image.

This guide assumes a Linux host with Docker Engine and the Compose plugin
installed. Container images are published automatically after CI succeeds on pushes to
`main`. See [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md) for the CI/CD flow and
installed. Container images are published automatically when releases are published.
See [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md) for the CI/CD flow and
[`docs/DEPLOY-RUNBOOK.md`](../docs/DEPLOY-RUNBOOK.md) for the step-by-step update
procedure after a new image lands on the host.

Expand All @@ -33,7 +33,7 @@ chmod +x deploy/bootstrap.sh
./deploy/bootstrap.sh /opt/nostream
```

Edit `/opt/nostream/.env`, load `ghcr.io/cameri/nostream:main`, then:
Edit `/opt/nostream/.env`, load `ghcr.io/cameri/nostream:latest`, then:

```bash
cd /opt/nostream
Expand All @@ -46,7 +46,7 @@ from this repository. You only maintain `.env` and optional settings overrides.
## Prerequisites

1. [Docker Engine](https://docs.docker.com/engine/install/) and the Compose plugin
2. `ghcr.io/cameri/nostream:main` loaded on the host (see
2. `ghcr.io/cameri/nostream:latest` loaded on the host (see
[Image delivery](#image-delivery-on-restricted-networks) if `docker pull` fails)

## Server layout after bootstrap
Expand All @@ -65,10 +65,10 @@ from this repository. You only maintain `.env` and optional settings overrides.

| Service | Image | Notes |
|-------------------|--------------------------------|---------------------------------------------------------|
| nostream | ghcr.io/cameri/nostream:main | `pull_policy: never` when the image is pre-loaded |
| nostream | ghcr.io/cameri/nostream:latest | `pull_policy: never` when the image is pre-loaded |
| nostream-db | postgres:15 | |
| nostream-cache | redis:7.0.5-alpine3.16 | |
| nostream-migrate | ghcr.io/cameri/nostream:main | one-shot `knex migrate:latest`; same image as the relay |
| nostream-migrate | ghcr.io/cameri/nostream:latest | one-shot `knex migrate:latest`; same image as the relay |

The relay listens on `127.0.0.1:8008`. Expose it with a reverse proxy or
tunnel (for example Cloudflare Tunnel).
Expand Down
2 changes: 1 addition & 1 deletion deploy/bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Bootstrap complete: $TARGET

Next steps:
1. Edit $TARGET/.env (SECRET, DB_PASSWORD, REDIS_PASSWORD)
2. Load ghcr.io/cameri/nostream:main on this host
2. Load ghcr.io/cameri/nostream:latest on this host
3. cd $TARGET && docker compose up -d

Optional relay overrides:
Expand Down
4 changes: 2 additions & 2 deletions deploy/docker-compose.haproxy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ networks:
- subnet: 172.28.0.0/24

x-nostream-relay: &nostream-relay
image: ghcr.io/cameri/nostream:main
image: ghcr.io/cameri/nostream:latest
pull_policy: never
env_file: .env
environment:
Expand Down Expand Up @@ -125,7 +125,7 @@ services:
retries: 10

nostream-migrate:
image: ghcr.io/cameri/nostream:main
image: ghcr.io/cameri/nostream:latest
pull_policy: never
container_name: nostream-migrate
networks:
Expand Down
4 changes: 2 additions & 2 deletions deploy/docker-compose.prod.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
nostream:
image: ghcr.io/cameri/nostream:main
image: ghcr.io/cameri/nostream:latest
pull_policy: never
container_name: nostream
env_file: .env
Expand Down Expand Up @@ -72,7 +72,7 @@ services:
retries: 10

nostream-migrate:
image: ghcr.io/cameri/nostream:main
image: ghcr.io/cameri/nostream:latest
pull_policy: never
container_name: nostream-migrate
user: node:node
Expand Down
27 changes: 14 additions & 13 deletions docs/CI-MIGRATION.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,28 @@
# CI container publishing migration

Container publishing is now part of `.github/workflows/checks.yml`.
Container publishing is now restricted exclusively to official releases via `.github/workflows/release.yml`.

## What changed

The standalone `publish-container-image.yml` workflow, which listened for a
completed `workflow_run`, was removed. `checks.yml` now contains
`publish-container-image` as its final job.
The `publish-container-image` job was removed from `.github/workflows/checks.yml`. Container images
are no longer built or published on pushes to `main`.

## Why

Keeping publishing in the workflow that performed the checks makes its
dependencies explicit. It avoids the timing and event-context ambiguity of a
separate `workflow_run`, which could be triggered by an external pull-request
run.
Publishing images from unreleased commits on `main` caused the `main` tag (and per-commit SHA tags)
to serve as moving targets of unreleased code, creating drift between published releases and images
pulled in production. Publishing is now aligned strictly with official GitHub releases.

## New behavior

On successful pushes to `main`, CI runs linting, the build check, unit tests,
integration tests, and post-test reporting before publishing the container
image. The publish job cannot run for pull requests or manual workflow runs.
Container images are published only by `.github/workflows/release.yml` when an official relay release
is published. Changesets tags a relay release `nostream@X.Y.Z`, so that prefix is what the workflow
matches; the `vX.Y.Z` form from before the changesets migration is still accepted. Pushes to `main` and pull requests run verification checks without
publishing any container images.

## Action required

None. Developers and deployment users receive the same `main` and per-commit
container image tags automatically after CI succeeds.
Deployments tracking `ghcr.io/cameri/nostream:main` should be updated to track released versions,
such as `ghcr.io/cameri/nostream:latest` or pinned release tags (e.g. `ghcr.io/cameri/nostream:nostream@3.6.2`,
published as-is from the release tag, or the bare `ghcr.io/cameri/nostream:3.6.2`).
Comment on lines +26 to +27

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Pinned image example cannot work

ghcr.io/cameri/nostream:nostream@3.6.2 is not a valid Docker image reference. @ separates an image name from its digest; it cannot appear inside a tag. Operators copying this example get an invalid-reference error.

Use the bare :3.6.2 tag already produced by the workflow. Correct the same promise in docs/DEPLOYMENT.md and the workflow comments.

See [`deploy/README.md`](../deploy/README.md) and [`docs/DEPLOY-RUNBOOK.md`](DEPLOY-RUNBOOK.md).
45 changes: 36 additions & 9 deletions docs/DEPLOY-RUNBOOK.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Deploy runbook (sandbox / production relay)

Step-by-step procedure for updating a running nostream host after a new
`ghcr.io/cameri/nostream:main` image is available.
`ghcr.io/cameri/nostream:latest` (or release tag) image is available.

| Doc | Purpose |
|-----|---------|
Expand All @@ -11,16 +11,43 @@ Step-by-step procedure for updating a running nostream host after a new
## Image delivery vs relay recreate

```text
GitHub push to main
→ CI publishes ghcr.io/cameri/nostream:main
→ (optional) webhook pull stack loads the image on the host
GitHub Release published (tag v*)
→ CI publishes ghcr.io/cameri/nostream:latest (and the version tag)
→ the release webhook pulls :latest onto the host (see below)
Comment thread
greptile-apps[bot] marked this conversation as resolved.
→ operator runs this runbook (migrate + recreate relay(s))
→ Cloudflare Tunnel / reverse proxy keeps pointing at 127.0.0.1:8008
```

Nothing publishes an image on a push to `main` any more, so a host that still
pulls `:main` will never see a new image. The on-host pull stack must track the
release instead:

1. `NOSTREAM_TAG` in `/opt/nostream/webhook/.env` set to `latest`.
2. The GitHub webhook subscribed to **releases** as well as **push**, because
the receiver ignores anything that is not a push to the watched branch.

Both live in `containers/nostream-dev-webhook/` (the versioned source) with its
own README. Until both are done, deployments are manual: pull the tag by hand
and continue at step 2 below.

The webhook pull stack **loads images only**. It does **not** run migrations or
recreate relay containers. This runbook closes that gap.

The release event reaches the webhook at the same moment the image build starts,
so the pull fires before the tag exists. The puller retries on a growing backoff
and re-queues a message it exhausts, but **there is no signal that says the pull
landed**. Before you recreate, confirm the image is on the host:

```bash
docker images --filter=reference='ghcr.io/cameri/nostream' --format '{{.Tag}} {{.ID}}'
# expect the tag the release carried, and a digest that is not the previous one
```

A recreate with a stale `latest` starts the previous release from a brand-new
build: the migrate step runs the old schema, nothing fails, and the deploy looks
successful. When in doubt, `docker pull ghcr.io/cameri/nostream:latest` and re-run
the check above.

## Choose your stack

| Stack | Compose file | Update script | Downtime |
Expand All @@ -43,7 +70,7 @@ are documented under
- New relay image on the host (`docker pull` or `docker load`)
- Shell access (`cd /opt/nostream`)
- Before loading a new image or recreating containers, record the **running**
relay image (not `docker image inspect :main` after pull — that tag may already
relay image (not `docker image inspect :latest` after pull — that tag may already
point at the new image)

```bash
Expand Down Expand Up @@ -84,7 +111,7 @@ Existing `.env` and `.nostr/settings.yaml` are preserved.
### 2. Load the new image (skip if webhook pull already did)

```bash
docker pull ghcr.io/cameri/nostream:main
docker pull ghcr.io/cameri/nostream:latest
```

On hosts that cannot reach GHCR over IPv4, use `docker save` / `docker load`. See
Expand Down Expand Up @@ -169,10 +196,10 @@ HAProxy stack: stop the relay you just replaced; leave the healthy peer running.

```bash
PREVIOUS_IMAGE="$(cat /tmp/nostream-pre-deploy-image-id)"
docker tag "$PREVIOUS_IMAGE" ghcr.io/cameri/nostream:main
docker tag "$PREVIOUS_IMAGE" ghcr.io/cameri/nostream:latest
```

Or `docker load -i /path/to/nostream-main-backup.tar.gz`.
Or `docker load -i /path/to/nostream-latest-backup.tar.gz`.

### 3. Recreate on the old image (skip migrations)

Expand All @@ -188,7 +215,7 @@ SKIP_MIGRATE=1 /opt/nostream/recreate-relay.sh /opt/nostream
```

HAProxy: run migrate only when moving **forward** on a new image. For rollback,
retag the old image to `:main`, then recreate relays **without** migrations.
retag the old image to `:latest`, then recreate relays **without** migrations.

`rolling-relay-recreate.sh` will **not** replace a running relay unless its peer
is up and `/readyz` healthy. If step 1 left one backend stopped, running the
Expand Down
22 changes: 14 additions & 8 deletions docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,25 @@
# Deployment and container publishing

The `ghcr.io/cameri/nostream` container image is published automatically by the
CI Checks workflow after a successful push to `main`. The workflow publishes
both the `main` tag and a `sha-<commit>` tag.
Release workflow (`release.yml`) whenever an official GitHub release is published.
For release `nostream@3.6.2` the workflow publishes three tags: `latest`, the release tag itself
(`nostream@3.6.2`, which is what a host pins), and the bare version (`3.6.2`).

Publishing is the final stage of the CI dependency chain:
Publishing is triggered only on published relay releases — the changesets tag `nostream@X.Y.Z`, or the
`vX.Y.Z` form used before the changesets migration. A `nostream-website@X.Y.Z` release publishes no
image:

```text
changes → lint and build-check → unit and integration tests → post-tests → publish-container-image
release: published (nostream@X.Y.Z)
→ build-and-push-image
→ ghcr.io/cameri/nostream:{nostream@X.Y.Z, X.Y.Z, latest}
```

The publish job runs only for `refs/heads/main` and only when `post-tests`
succeeds. Pull requests and manual workflow runs do not publish images.
CI Checks (`checks.yml`) does not build or publish container images on push to
`main` or pull requests, ensuring that published images only correspond to
released versions.

Deployment hosts can pull the `main` image as described in
[`deploy/README.md`](../deploy/README.md). After the image is on the host, follow
Deployment hosts can pull the released image (`latest` or a specific release tag)
as described in [`deploy/README.md`](../deploy/README.md). After the image is on the host, follow
[`docs/DEPLOY-RUNBOOK.md`](DEPLOY-RUNBOOK.md) to migrate and recreate the relay.
There is no separate manual image publishing step.
Loading
Loading