Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions deploy/compose/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
This is the single-node/VPS deployment bundle. It is intentionally separate from
the root `docker-compose.yml`, which remains local development infrastructure.

The Provably GCP pilot runs this bundle on one Compute Engine VM. It uses local
Docker volumes for PostgreSQL, Redis, MinIO (S3-compatible media storage), and
Buzz's Git path. MinIO is the storage service in this pilot; Google Cloud Storage
is not required for the current deployment.

## Quick start

```bash
Expand All @@ -19,17 +24,17 @@ cd deploy/compose
BUZZ_COMPOSE_TLS=true ./run.sh start
```

The bootstrap script should eventually replace manual `.env` editing for normal
users. It is responsible for generating stable secrets and, optionally, an owner
keypair.
For the Provably GCP pilot, `deploy/gcp/bootstrap.sh` fetches the runtime secrets
from Google Secret Manager and writes a root-only `/opt/buzz/.env`. Do not commit
that file or copy secret values into this repository.

## Production notes

- Requires Docker Compose v2.24.4 or newer; the TLS override uses Compose's
`!reset` tag to remove the direct relay port when Caddy terminates HTTPS.
- Default `BUZZ_IMAGE` tracks `ghcr.io/block/buzz:main` for early testing. Pin it to `ghcr.io/block/buzz:sha-<7>` or a semver release tag for production once available.
- Keep `BUZZ_RELAY_PRIVATE_KEY`, `BUZZ_GIT_HOOK_HMAC_SECRET`, database/Redis,
and S3 secrets stable across restarts.
and MinIO/S3-compatible secrets stable across restarts.
- `RELAY_OWNER_PUBKEY` is intentionally not prefixed with `BUZZ_`; it must be a
64-character hex Nostr pubkey when closed relay mode is enabled.
- `BUZZ_AUTO_MIGRATE` is opt-in. Set `BUZZ_AUTO_MIGRATE=true` or run
Expand All @@ -38,6 +43,9 @@ keypair.
migrations.
- The stack uses Postgres, Redis, MinIO, and a git data volume because
those are real Buzz dependencies today. Minimal mode can simplify this later.
Treat the Git volume as scratch/cache until the pinned build's restore tests
prove otherwise; PostgreSQL and the object-store manifest are the intended
sources of truth for recovery planning.
- The bundled Compose stack fixes the relay endpoint to `http://minio:9000` and
`BUZZ_S3_ADDRESSING_STYLE=path`: Docker DNS resolves `minio`, not
`<bucket>.minio`. It is not configurable for an external S3 provider through
Expand Down
97 changes: 97 additions & 0 deletions deploy/gcp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Buzz on Google Cloud

This directory documents the ProvablyAI GCP pilot and contains the VM bootstrap
used for a future rebuild. The pilot is a single Compute Engine VM, not GKE.

## Current topology

- GCP project: `provably-buzz`
- VM: `buzz-pilot-1` in `europe-west6-a`
- Public domain: `buzz.provably.ai`
- Relay endpoint: `wss://buzz.provably.ai`
- Application image: `europe-west6-docker.pkg.dev/provably-buzz/buzz/buzz:4632c55`
- Buzz source pin: `4632c55041c5d423d572a6f6411bb7b279c26f67`

The application stack is Docker Compose under `/opt/buzz/src/deploy/compose`:

| Service | Pilot role | Persistence |
| --- | --- | --- |
| `relay` | Buzz relay and HTTP bridge | `buzz-git-data` for the Git path |
| `postgres` | Buzz operational database | `buzz-postgres-data` |
| `redis` | Relay/runtime cache and coordination | `buzz-redis-data` |
| `minio` | S3-compatible media/object storage | `buzz-minio-data` |
| `minio-init` | Creates the private media bucket | no persistent state |

The reverse proxy/TLS service is a separate Nginx Proxy Manager container, with
Cloudflare in front of the public domain. It is intentionally not part of the
application Compose file. The proxy-manager container may therefore appear as
an orphan when the application Compose project is recreated; do not remove it
unless the proxy deployment is being changed deliberately.

## Secret handling

The bootstrap reads these Secret Manager IDs using the VM runtime service
account, then writes `/opt/buzz/.env` with root ownership and mode `0600`:

```text
Buzz_RELAY_OWNER_PUBKEY
Buzz_RELAY_PRIVATE_KEY
Buzz_GIT_HOOK_HMAC_SECRET
Buzz_POSTGRES_PASSWORD
Buzz_REDIS_PASSWORD
Buzz_S3_ACCESS_KEY
Buzz_S3_SECRET_KEY
```

The application environment variable names remain compatible with Buzz. The
`Buzz_` prefix is the Secret Manager naming convention, not an instruction to
rename the application variables. Secret values, private keys, and generated
`.env` files must never be committed.

The owner public key is public identity material and is not a password. The
relay private key is an identity anchor; changing it is an identity migration,
not a routine credential rotation. PostgreSQL and Redis credentials can be
rotated with an explicit staged version and a controlled Compose recreation.

## Bootstrap and verification

`bootstrap.sh` is pinned to the tested Buzz image/source commit and performs the
following operations:

1. Installs Docker, Compose, Git, and the small bootstrap dependencies.
2. Fetches Secret Manager values through the VM metadata service account.
3. Clones Buzz and checks out the pinned source commit.
4. Pulls the pinned Artifact Registry image.
5. Links the generated environment into `deploy/compose/.env`.
6. Starts the Compose stack and waits for health.

The desktop invite flow requires all three origins in `BUZZ_CORS_ORIGINS`:

```text
https://buzz.provably.ai,http://tauri.localhost,tauri://localhost
```

After a deployment or rotation, verify:

```bash
curl -fsS https://buzz.provably.ai/_liveness
curl -fsS https://buzz.provably.ai/_readiness
```

Expected responses are `ok` and `{"status":"ready"}`. Also verify ordinary
relay connectivity, closed-membership behavior, media upload/download, Git
paths, and the desktop invite flow before broader onboarding.

## Recovery boundaries

- Back up and rehearse restore for PostgreSQL and the MinIO bucket.
- Preserve relay identity secrets and deployment configuration separately from
database/object data.
- Treat the `buzz-git-data` volume as scratch/cache unless the pinned build and
restore tests prove it is authoritative. Do not treat it as the sole backup.
- Keep a rollback environment snapshot and previous Secret Manager versions
until a rotation has passed health and functional checks.

The current pilot deliberately uses local Docker PostgreSQL, Redis, and MinIO.
Cloud SQL, Memorystore, Google Cloud Storage, GKE, and QEDB are later
hardening/integration choices, not prerequisites for this baseline.
82 changes: 82 additions & 0 deletions deploy/gcp/bootstrap.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
set -euo pipefail

# GCP single-VM bootstrap for the pinned ProvablyAI Buzz image.
# Secrets are fetched from Secret Manager at boot and written only to the
# root-owned runtime file /opt/buzz/.env; this file is never committed.

PROJECT_ID="provably-buzz"
IMAGE_REGISTRY="europe-west6-docker.pkg.dev/provably-buzz/buzz/buzz:4632c55"
LOCAL_IMAGE="provably/buzz:4632c55"
BUZZ_SHA="4632c55041c5d423d572a6f6411bb7b279c26f67"
BUZZ_DIR="/opt/buzz"
TOKEN_URL="http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token"

exec > >(tee -a /var/log/buzz-bootstrap.log | logger -t buzz-bootstrap) 2>&1

export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y ca-certificates curl docker.io docker-compose-v2 git python3
systemctl enable --now docker

install -d -m 0700 "$BUZZ_DIR"

secret() {
local name="$1"
local token
token="$(curl -fsS -H 'Metadata-Flavor: Google' "$TOKEN_URL" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')"
curl -fsS \
-H 'Metadata-Flavor: Google' \
-H "Authorization: Bearer ${token}" \
"https://secretmanager.googleapis.com/v1/projects/${PROJECT_ID}/secrets/${name}/versions/latest:access" \
| python3 -c 'import base64,json,sys; print(base64.b64decode(json.load(sys.stdin)["payload"]["data"]).decode(), end="")'
}

install -o root -g root -m 0600 /dev/null "$BUZZ_DIR/.env"
{
printf '%s\n' 'BUZZ_IMAGE=provably/buzz:4632c55'
printf '%s\n' 'BUZZ_DOMAIN=buzz.provably.ai'
printf '%s\n' 'RELAY_URL=wss://buzz.provably.ai'
printf '%s\n' 'BUZZ_MEDIA_BASE_URL=https://buzz.provably.ai/media'
printf '%s\n' 'BUZZ_MEDIA_SERVER_DOMAIN=buzz.provably.ai'
# The Tauri origins are required by the Buzz desktop invite flow.
printf '%s\n' 'BUZZ_CORS_ORIGINS=https://buzz.provably.ai,http://tauri.localhost,tauri://localhost'
printf '%s\n' 'BUZZ_REQUIRE_AUTH_TOKEN=true'
printf '%s\n' 'BUZZ_REQUIRE_RELAY_MEMBERSHIP=true'
printf '%s\n' 'BUZZ_ALLOW_NIP_OA_AUTH=true'
printf '%s\n' 'BUZZ_AUTO_MIGRATE=true'
printf '%s\n' 'BUZZ_GIT_CONFORMANCE_PROBE=true'
printf '%s\n' 'BUZZ_S3_ADDRESSING_STYLE=path'
printf '%s\n' 'BUZZ_S3_BUCKET=buzz-media'
printf '%s\n' 'BUZZ_HTTP_PORT=3000'
printf '%s\n' 'POSTGRES_DB=buzz'
printf '%s\n' 'POSTGRES_USER=buzz'
printf 'RELAY_OWNER_PUBKEY=%s\n' "$(secret Buzz_RELAY_OWNER_PUBKEY)"
printf 'BUZZ_RELAY_PRIVATE_KEY=%s\n' "$(secret Buzz_RELAY_PRIVATE_KEY)"
printf 'BUZZ_GIT_HOOK_HMAC_SECRET=%s\n' "$(secret Buzz_GIT_HOOK_HMAC_SECRET)"
printf 'POSTGRES_PASSWORD=%s\n' "$(secret Buzz_POSTGRES_PASSWORD)"
printf 'REDIS_PASSWORD=%s\n' "$(secret Buzz_REDIS_PASSWORD)"
printf 'BUZZ_S3_ACCESS_KEY=%s\n' "$(secret Buzz_S3_ACCESS_KEY)"
printf 'BUZZ_S3_SECRET_KEY=%s\n' "$(secret Buzz_S3_SECRET_KEY)"
} > "$BUZZ_DIR/.env"
chown root:root "$BUZZ_DIR/.env"
chmod 0600 "$BUZZ_DIR/.env"

install -d -m 0755 "$BUZZ_DIR/src"
if [[ ! -d "$BUZZ_DIR/src/.git" ]]; then
git clone --no-tags --depth=1 https://github.com/ProvablyAI/buzz.git "$BUZZ_DIR/src"
fi
git -C "$BUZZ_DIR/src" fetch --depth=1 origin "$BUZZ_SHA"
git -C "$BUZZ_DIR/src" checkout --detach "$BUZZ_SHA"

registry_token="$(curl -fsS -H 'Metadata-Flavor: Google' "$TOKEN_URL" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')"
printf '%s' "$registry_token" | docker login -u oauth2accesstoken --password-stdin https://europe-west6-docker.pkg.dev
docker pull "$IMAGE_REGISTRY"
docker tag "$IMAGE_REGISTRY" "$LOCAL_IMAGE"
docker logout europe-west6-docker.pkg.dev >/dev/null 2>&1 || true

ln -sfn "$BUZZ_DIR/.env" "$BUZZ_DIR/src/deploy/compose/.env"
cd "$BUZZ_DIR/src/deploy/compose"
docker compose --env-file "$BUZZ_DIR/.env" -f compose.yml up -d --wait --wait-timeout 300

printf 'Buzz bootstrap completed for commit %s\n' "$BUZZ_SHA"