diff --git a/deploy/compose/README.md b/deploy/compose/README.md index bb0e63fe15d..9371d97485c 100644 --- a/deploy/compose/README.md +++ b/deploy/compose/README.md @@ -3,6 +3,11 @@ This is the single-node/VPS deployment bundle. It is intentionally separate from the root `docker-compose.yml`, which remains local development infrastructure. +The Provably GCP pilot runs this bundle on one Compute Engine VM. It uses local +Docker volumes for PostgreSQL, Redis, MinIO (S3-compatible media storage), and +Buzz's Git path. MinIO is the storage service in this pilot; Google Cloud Storage +is not required for the current deployment. + ## Quick start ```bash @@ -19,9 +24,9 @@ cd deploy/compose BUZZ_COMPOSE_TLS=true ./run.sh start ``` -The bootstrap script should eventually replace manual `.env` editing for normal -users. It is responsible for generating stable secrets and, optionally, an owner -keypair. +For the Provably GCP pilot, `deploy/gcp/bootstrap.sh` fetches the runtime secrets +from Google Secret Manager and writes a root-only `/opt/buzz/.env`. Do not commit +that file or copy secret values into this repository. ## Production notes @@ -29,7 +34,7 @@ keypair. `!reset` tag to remove the direct relay port when Caddy terminates HTTPS. - Default `BUZZ_IMAGE` tracks `ghcr.io/block/buzz:main` for early testing. Pin it to `ghcr.io/block/buzz:sha-<7>` or a semver release tag for production once available. - Keep `BUZZ_RELAY_PRIVATE_KEY`, `BUZZ_GIT_HOOK_HMAC_SECRET`, database/Redis, - and S3 secrets stable across restarts. + and MinIO/S3-compatible secrets stable across restarts. - `RELAY_OWNER_PUBKEY` is intentionally not prefixed with `BUZZ_`; it must be a 64-character hex Nostr pubkey when closed relay mode is enabled. - `BUZZ_AUTO_MIGRATE` is opt-in. Set `BUZZ_AUTO_MIGRATE=true` or run @@ -38,6 +43,9 @@ keypair. migrations. - The stack uses Postgres, Redis, MinIO, and a git data volume because those are real Buzz dependencies today. Minimal mode can simplify this later. + Treat the Git volume as scratch/cache until the pinned build's restore tests + prove otherwise; PostgreSQL and the object-store manifest are the intended + sources of truth for recovery planning. - The bundled Compose stack fixes the relay endpoint to `http://minio:9000` and `BUZZ_S3_ADDRESSING_STYLE=path`: Docker DNS resolves `minio`, not `.minio`. It is not configurable for an external S3 provider through diff --git a/deploy/gcp/README.md b/deploy/gcp/README.md new file mode 100644 index 00000000000..b18cc890497 --- /dev/null +++ b/deploy/gcp/README.md @@ -0,0 +1,97 @@ +# Buzz on Google Cloud + +This directory documents the ProvablyAI GCP pilot and contains the VM bootstrap +used for a future rebuild. The pilot is a single Compute Engine VM, not GKE. + +## Current topology + +- GCP project: `provably-buzz` +- VM: `buzz-pilot-1` in `europe-west6-a` +- Public domain: `buzz.provably.ai` +- Relay endpoint: `wss://buzz.provably.ai` +- Application image: `europe-west6-docker.pkg.dev/provably-buzz/buzz/buzz:4632c55` +- Buzz source pin: `4632c55041c5d423d572a6f6411bb7b279c26f67` + +The application stack is Docker Compose under `/opt/buzz/src/deploy/compose`: + +| Service | Pilot role | Persistence | +| --- | --- | --- | +| `relay` | Buzz relay and HTTP bridge | `buzz-git-data` for the Git path | +| `postgres` | Buzz operational database | `buzz-postgres-data` | +| `redis` | Relay/runtime cache and coordination | `buzz-redis-data` | +| `minio` | S3-compatible media/object storage | `buzz-minio-data` | +| `minio-init` | Creates the private media bucket | no persistent state | + +The reverse proxy/TLS service is a separate Nginx Proxy Manager container, with +Cloudflare in front of the public domain. It is intentionally not part of the +application Compose file. The proxy-manager container may therefore appear as +an orphan when the application Compose project is recreated; do not remove it +unless the proxy deployment is being changed deliberately. + +## Secret handling + +The bootstrap reads these Secret Manager IDs using the VM runtime service +account, then writes `/opt/buzz/.env` with root ownership and mode `0600`: + +```text +Buzz_RELAY_OWNER_PUBKEY +Buzz_RELAY_PRIVATE_KEY +Buzz_GIT_HOOK_HMAC_SECRET +Buzz_POSTGRES_PASSWORD +Buzz_REDIS_PASSWORD +Buzz_S3_ACCESS_KEY +Buzz_S3_SECRET_KEY +``` + +The application environment variable names remain compatible with Buzz. The +`Buzz_` prefix is the Secret Manager naming convention, not an instruction to +rename the application variables. Secret values, private keys, and generated +`.env` files must never be committed. + +The owner public key is public identity material and is not a password. The +relay private key is an identity anchor; changing it is an identity migration, +not a routine credential rotation. PostgreSQL and Redis credentials can be +rotated with an explicit staged version and a controlled Compose recreation. + +## Bootstrap and verification + +`bootstrap.sh` is pinned to the tested Buzz image/source commit and performs the +following operations: + +1. Installs Docker, Compose, Git, and the small bootstrap dependencies. +2. Fetches Secret Manager values through the VM metadata service account. +3. Clones Buzz and checks out the pinned source commit. +4. Pulls the pinned Artifact Registry image. +5. Links the generated environment into `deploy/compose/.env`. +6. Starts the Compose stack and waits for health. + +The desktop invite flow requires all three origins in `BUZZ_CORS_ORIGINS`: + +```text +https://buzz.provably.ai,http://tauri.localhost,tauri://localhost +``` + +After a deployment or rotation, verify: + +```bash +curl -fsS https://buzz.provably.ai/_liveness +curl -fsS https://buzz.provably.ai/_readiness +``` + +Expected responses are `ok` and `{"status":"ready"}`. Also verify ordinary +relay connectivity, closed-membership behavior, media upload/download, Git +paths, and the desktop invite flow before broader onboarding. + +## Recovery boundaries + +- Back up and rehearse restore for PostgreSQL and the MinIO bucket. +- Preserve relay identity secrets and deployment configuration separately from + database/object data. +- Treat the `buzz-git-data` volume as scratch/cache unless the pinned build and + restore tests prove it is authoritative. Do not treat it as the sole backup. +- Keep a rollback environment snapshot and previous Secret Manager versions + until a rotation has passed health and functional checks. + +The current pilot deliberately uses local Docker PostgreSQL, Redis, and MinIO. +Cloud SQL, Memorystore, Google Cloud Storage, GKE, and QEDB are later +hardening/integration choices, not prerequisites for this baseline. diff --git a/deploy/gcp/bootstrap.sh b/deploy/gcp/bootstrap.sh new file mode 100755 index 00000000000..81afec0e21d --- /dev/null +++ b/deploy/gcp/bootstrap.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +set -euo pipefail + +# GCP single-VM bootstrap for the pinned ProvablyAI Buzz image. +# Secrets are fetched from Secret Manager at boot and written only to the +# root-owned runtime file /opt/buzz/.env; this file is never committed. + +PROJECT_ID="provably-buzz" +IMAGE_REGISTRY="europe-west6-docker.pkg.dev/provably-buzz/buzz/buzz:4632c55" +LOCAL_IMAGE="provably/buzz:4632c55" +BUZZ_SHA="4632c55041c5d423d572a6f6411bb7b279c26f67" +BUZZ_DIR="/opt/buzz" +TOKEN_URL="http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" + +exec > >(tee -a /var/log/buzz-bootstrap.log | logger -t buzz-bootstrap) 2>&1 + +export DEBIAN_FRONTEND=noninteractive +apt-get update +apt-get install -y ca-certificates curl docker.io docker-compose-v2 git python3 +systemctl enable --now docker + +install -d -m 0700 "$BUZZ_DIR" + +secret() { + local name="$1" + local token + token="$(curl -fsS -H 'Metadata-Flavor: Google' "$TOKEN_URL" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')" + curl -fsS \ + -H 'Metadata-Flavor: Google' \ + -H "Authorization: Bearer ${token}" \ + "https://secretmanager.googleapis.com/v1/projects/${PROJECT_ID}/secrets/${name}/versions/latest:access" \ + | python3 -c 'import base64,json,sys; print(base64.b64decode(json.load(sys.stdin)["payload"]["data"]).decode(), end="")' +} + +install -o root -g root -m 0600 /dev/null "$BUZZ_DIR/.env" +{ + printf '%s\n' 'BUZZ_IMAGE=provably/buzz:4632c55' + printf '%s\n' 'BUZZ_DOMAIN=buzz.provably.ai' + printf '%s\n' 'RELAY_URL=wss://buzz.provably.ai' + printf '%s\n' 'BUZZ_MEDIA_BASE_URL=https://buzz.provably.ai/media' + printf '%s\n' 'BUZZ_MEDIA_SERVER_DOMAIN=buzz.provably.ai' + # The Tauri origins are required by the Buzz desktop invite flow. + printf '%s\n' 'BUZZ_CORS_ORIGINS=https://buzz.provably.ai,http://tauri.localhost,tauri://localhost' + printf '%s\n' 'BUZZ_REQUIRE_AUTH_TOKEN=true' + printf '%s\n' 'BUZZ_REQUIRE_RELAY_MEMBERSHIP=true' + printf '%s\n' 'BUZZ_ALLOW_NIP_OA_AUTH=true' + printf '%s\n' 'BUZZ_AUTO_MIGRATE=true' + printf '%s\n' 'BUZZ_GIT_CONFORMANCE_PROBE=true' + printf '%s\n' 'BUZZ_S3_ADDRESSING_STYLE=path' + printf '%s\n' 'BUZZ_S3_BUCKET=buzz-media' + printf '%s\n' 'BUZZ_HTTP_PORT=3000' + printf '%s\n' 'POSTGRES_DB=buzz' + printf '%s\n' 'POSTGRES_USER=buzz' + printf 'RELAY_OWNER_PUBKEY=%s\n' "$(secret Buzz_RELAY_OWNER_PUBKEY)" + printf 'BUZZ_RELAY_PRIVATE_KEY=%s\n' "$(secret Buzz_RELAY_PRIVATE_KEY)" + printf 'BUZZ_GIT_HOOK_HMAC_SECRET=%s\n' "$(secret Buzz_GIT_HOOK_HMAC_SECRET)" + printf 'POSTGRES_PASSWORD=%s\n' "$(secret Buzz_POSTGRES_PASSWORD)" + printf 'REDIS_PASSWORD=%s\n' "$(secret Buzz_REDIS_PASSWORD)" + printf 'BUZZ_S3_ACCESS_KEY=%s\n' "$(secret Buzz_S3_ACCESS_KEY)" + printf 'BUZZ_S3_SECRET_KEY=%s\n' "$(secret Buzz_S3_SECRET_KEY)" +} > "$BUZZ_DIR/.env" +chown root:root "$BUZZ_DIR/.env" +chmod 0600 "$BUZZ_DIR/.env" + +install -d -m 0755 "$BUZZ_DIR/src" +if [[ ! -d "$BUZZ_DIR/src/.git" ]]; then + git clone --no-tags --depth=1 https://github.com/ProvablyAI/buzz.git "$BUZZ_DIR/src" +fi +git -C "$BUZZ_DIR/src" fetch --depth=1 origin "$BUZZ_SHA" +git -C "$BUZZ_DIR/src" checkout --detach "$BUZZ_SHA" + +registry_token="$(curl -fsS -H 'Metadata-Flavor: Google' "$TOKEN_URL" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')" +printf '%s' "$registry_token" | docker login -u oauth2accesstoken --password-stdin https://europe-west6-docker.pkg.dev +docker pull "$IMAGE_REGISTRY" +docker tag "$IMAGE_REGISTRY" "$LOCAL_IMAGE" +docker logout europe-west6-docker.pkg.dev >/dev/null 2>&1 || true + +ln -sfn "$BUZZ_DIR/.env" "$BUZZ_DIR/src/deploy/compose/.env" +cd "$BUZZ_DIR/src/deploy/compose" +docker compose --env-file "$BUZZ_DIR/.env" -f compose.yml up -d --wait --wait-timeout 300 + +printf 'Buzz bootstrap completed for commit %s\n' "$BUZZ_SHA"