diff --git a/deploy/personal-node/README.md b/deploy/personal-node/README.md index 645671c1..2351fefe 100644 --- a/deploy/personal-node/README.md +++ b/deploy/personal-node/README.md @@ -1,6 +1,6 @@ # Run a personal TinyCloud node -This ordinary-server profile runs a single node with local SQLite, local blocks, and Caddy HTTPS. It tracks the version TinyCloud itself runs in production: **1.20.6**, pinned by immutable image digest in [`compose.yaml`](compose.yaml). Caddy **2.10.2** is pinned by its multi-platform image digest there too. The prior published ordinary image 1.20.5 is used only to rehearse an upgrade and restore-based rollback. Do not use `latest` or 1.20.7: the latter was rolled back in production (TC-1176). New node-image changes remain unavailable to this published-image guide until a separately approved release ships. +This ordinary-server profile runs a single node with local SQLite, local blocks, and Caddy HTTPS. It pins the published **1.22.0** node image by immutable digest in [`compose.yaml`](compose.yaml): the multi-platform index pushed by the `docker.yml` run for the `v1.22.0` tag. Caddy **2.10.2** is pinned by its multi-platform image digest there too. The previous published image, 1.20.6, is used only to rehearse an upgrade and restore-based rollback. Pin by digest; do not use a mutable tag such as `latest`. This is **not** a TEE: the VPS operator and its host can access the filesystem and key. No hardware attestation, dstack KMS isolation, or cross-host account migration is claimed. This guide does not configure registry discovery or hosted-to-owned-node replication (TC-1280/TC-1283). TLS authenticates your domain but does not attest the machine. @@ -21,12 +21,12 @@ export NODE_HOME="$HOME/personal-tinycloud" bash deploy/personal-node/node.sh init node.example.com bash deploy/personal-node/node.sh check-missing-key bash deploy/personal-node/node.sh start -bash deploy/personal-node/node.sh check-version 1.20.6 +bash deploy/personal-node/node.sh check-version 1.22.0 ``` -Replace `node.example.com` with the domain you own. `init` writes a unique 48-byte random Static root to a `0600` private `node.env`; it is never echoed. The profile mounts a keyless TOML over the bundled image config, so even the older 1.20.6 image **fails** if the installation root is absent. `check-missing-key` must run before `start`; it requires the node's exact `Missing secret` diagnostic and rejects unrelated failures without printing their logs. Caddy obtains and renews a public certificate; do not bypass its trust verification. +Replace `node.example.com` with the domain you own. `init` writes a unique 48-byte random Static root to a `0600` private `node.env`; it is never echoed. The profile mounts a keyless TOML that replaces the config bundled in the image and fails closed when the installation root is absent. `check-missing-key` must run before `start`; it requires the node's exact `Missing secret` diagnostic and rejects unrelated failures without printing their logs. The upgrade rehearsal starts from the 1.20.6 image; CI runs the bare image check (`check-bare-image-missing-key`) on 1.22.0, starting the image with no mount, key or environment and requiring the same `Missing secret` diagnostic. Caddy obtains and renews a public certificate; do not bypass its trust verification. -The 1.20.6 config loader accepts both the legacy single-underscore nesting (`TINYCLOUD_KEYS_TYPE`, `TINYCLOUD_STORAGE_DATADIR`) and canonical double-underscore nesting (`TINYCLOUD_KEYS__TYPE`, `TINYCLOUD_STORAGE__DATADIR`). It merges canonical values last when both are set. This profile uses the canonical form; the existing dstack Compose retains its working single-underscore key-type setting. +The 1.20.6 and 1.22.0 config loaders accept both the legacy single-underscore nesting (`TINYCLOUD_KEYS_TYPE`, `TINYCLOUD_STORAGE_DATADIR`) and canonical double-underscore nesting (`TINYCLOUD_KEYS__TYPE`, `TINYCLOUD_STORAGE__DATADIR`). They merge canonical values last when both are set. This profile uses the canonical form; the existing dstack Compose retains its working single-underscore key-type setting. To prove owner-held EOA operations, place your own wallet's 32-byte hexadecimal private key (without `0x`) in a separate `0600` file outside `NODE_HOME`, then run: @@ -55,7 +55,7 @@ bash deploy/personal-node/node.sh backup /secure/path/node-backup-2026-10-16 "$B export NODE_HOME="$HOME/personal-tinycloud-restored" bash deploy/personal-node/node.sh restore /secure/path/node-backup-2026-10-16 "$BACKUP_PASSPHRASE_FILE" bash deploy/personal-node/node.sh start -bash deploy/personal-node/node.sh check-version 1.20.6 +bash deploy/personal-node/node.sh check-version 1.22.0 bash deploy/personal-node/node.sh verify /secure/path/owner.key ``` @@ -63,7 +63,7 @@ Use a fresh host/directory, not two live nodes serving the same identity and dom ## Upgrade and rollback rehearsal -CI executes [`ci.sh`](ci.sh) on a clean runner: a fresh 1.20.6 installation with missing-key denial, an unrelated-failure negative control, signed data and restart; then 1.20.5 → 1.20.6, with data checks before and after restart, fresh-volume restores, and rollback by restoring the **pre-upgrade** snapshot into another fresh instance on 1.20.5. An in-place downgrade of a migrated database is not supported. A normal new installation starts directly on 1.20.6; do not downgrade it merely to rehearse. +CI executes [`ci.sh`](ci.sh) on a clean runner: a fresh 1.22.0 installation with missing-key denial, an unrelated-failure negative control, the bare 1.22.0 image's missing-key failure, signed data and restart; then 1.20.6 → 1.22.0, with data checks before and after restart, fresh-volume restores, and rollback by restoring the **pre-upgrade** snapshot into another fresh instance on 1.20.6. An in-place downgrade of a migrated database is not supported. A normal new installation starts directly on 1.22.0; do not install 1.20.6 except to rehearse this upgrade. ```bash # Run only on a disposable isolated runner with permission to create Docker @@ -71,8 +71,16 @@ CI executes [`ci.sh`](ci.sh) on a clean runner: a fresh 1.20.6 installation with bash deploy/personal-node/ci.sh ``` -Before a future upgrade, take and verify an offline backup as above, stop the node, change the pinned image only to a reviewed released digest, start and verify identity and signed data. If verification fails, stop the candidate and restore that pre-upgrade backup to a fresh instance of the previous image; never blindly start the older binary against migrated data. The next-release upgrade step **requires the next approved production node release** and a new published-image CI run. TC-1176 prohibits promotion of that release until Sam separately clears the hold. Changes to the shipped default key in TC-1200 likewise require that next image; the mounted keyless profile is the safe 1.20.6 workaround, not proof that its bare image fails closed. +### What 1.22.0 changes for an upgrade from 1.20.6 + +- **Database migrations.** The upgrade applies seven migrations that 1.20.6 does not ship: `m20261014_000000_delegation_cid_alias`, `m20261015_000000_delegation_cursor_index`, `m20261017_000000_epoch_order_space_index` and `m20261018_000000_epoch_head` (1.21.0), `m20261020_000000_current_kv_key_c_index` (1.21.3, an index on PostgreSQL), `m20261021_000000_sql_store_manifest` (1.21.8, two empty tables) and `m20261022_000000_encryption_network_key` (1.21.9, a table and a nullable column, not yet used). 1.22.0 itself adds none. An older image refuses to start against a database that has recorded a migration it does not ship. +- **Rollback needs the restore.** Because of those migrations, rolling back from 1.22.0 means restoring the pre-upgrade backup into a fresh instance of the 1.20.6 image, as `ci.sh` does for its last instance. Take and verify that backup before the upgrade; data written after it is not in it. +- **Named keys and delegated SQL caveats are disabled.** `[global.encryption_network] named_keys_v1` and `[global.sql] delegated_caveats_enforced` default to off, and this profile does not set them. With them off, named-key routes are not mounted, a `tinycloud.sql/*` delegation carrying `tables`, `columns` or `readOnly` caveats is refused as an unsupported caveat, and `/version` does not list `encryption-named-keys-v1` or `sql-delegated-caveats-v1`. `check-version` asserts that neither feature is listed. Enabling either is outside this guide and is not exercised by CI. + +### A later upgrade + +Take and verify an offline backup as above, stop the node, change the pinned image only to a reviewed released digest, start and verify identity and signed data. If verification fails, stop the candidate and restore that pre-upgrade backup to a fresh instance of the previous image; never blindly start the older binary against migrated data. Moving the pin to a later release also means updating the previous/current pair in [`node.sh`](node.sh) and [`ci.sh`](ci.sh) and running the published-image CI job for that pair. For a Caddy update, choose a reviewed release and resolve its published multi-platform index digest (for example, from the [Docker Hub tag metadata](https://hub.docker.com/v2/repositories/library/caddy/tags/2.10.2)). Replace only the `caddy@sha256:…` reference in `compose.yaml`, run the published-image CI job and verify HTTPS certificate issuance and renewal on a staging domain before upgrading a live proxy. Retain the Caddy `/data` and `/config` volumes; do not switch to a mutable tag. -The [personal-node workflow](../../.github/workflows/personal-node.yml) is the acceptance evidence only when its published-image job is green. Record the successful run URL and exact image versions/digests in TC-1282; no CI result here authorizes a production deployment or a stronger sovereignty claim. +The [personal-node workflow](../../.github/workflows/personal-node.yml) is the acceptance evidence only when its `published-images` job is green; its `dstack-profile` and `dstack-config-source` jobs cover the separate dstack Compose profile and the shipped key configuration. Record the successful run URL and exact image versions/digests in TC-1282; no CI result here authorizes a production deployment or a stronger sovereignty claim. diff --git a/deploy/personal-node/ci.sh b/deploy/personal-node/ci.sh index c2a8157e..f6d6580e 100755 --- a/deploy/personal-node/ci.sh +++ b/deploy/personal-node/ci.sh @@ -33,17 +33,18 @@ if "$here/node.sh" check-missing-key; then fi printf '%s\n' "$saved_image" > "$NODE_HOME/image" "$here/node.sh" start -"$here/node.sh" check-version 1.20.6 +"$here/node.sh" check-version 1.22.0 +"$here/node.sh" check-bare-image-missing-key "$here/node.sh" seed "$owner" "$here/node.sh" restart "$here/node.sh" verify "$owner" "$here/node.sh" stop export NODE_HOME="$work/tc1282-a" -"$here/node.sh" init localhost test 1.20.5 +"$here/node.sh" init localhost test 1.20.6 "$here/node.sh" check-missing-key "$here/node.sh" start -"$here/node.sh" check-version 1.20.5 +"$here/node.sh" check-version 1.20.6 "$here/node.sh" seed "$owner" "$here/node.sh" restart "$here/node.sh" verify "$owner" @@ -53,10 +54,10 @@ export NODE_HOME="$work/tc1282-a" export NODE_HOME="$work/tc1282-b" "$here/node.sh" restore "$work/pre-upgrade" "$passphrase" "$here/node.sh" start -"$here/node.sh" check-version 1.20.5 +"$here/node.sh" check-version 1.20.6 "$here/node.sh" verify "$owner" "$here/node.sh" upgrade -"$here/node.sh" check-version 1.20.6 +"$here/node.sh" check-version 1.22.0 "$here/node.sh" verify "$owner" "$here/node.sh" backup "$work/post-upgrade" "$passphrase" "$here/node.sh" stop @@ -64,16 +65,17 @@ export NODE_HOME="$work/tc1282-b" export NODE_HOME="$work/tc1282-c" "$here/node.sh" restore "$work/post-upgrade" "$passphrase" "$here/node.sh" start -"$here/node.sh" check-version 1.20.6 +"$here/node.sh" check-version 1.22.0 "$here/node.sh" verify "$owner" "$here/node.sh" stop # Rollback is a restore from the pre-upgrade snapshot, not an in-place -# downgrade of a migrated database. +# downgrade of a migrated database: the upgrade applies database migrations +# that 1.20.6 does not know. export NODE_HOME="$work/tc1282-d" "$here/node.sh" restore "$work/pre-upgrade" "$passphrase" "$here/node.sh" start -"$here/node.sh" check-version 1.20.5 +"$here/node.sh" check-version 1.20.6 "$here/node.sh" verify "$owner" "$here/node.sh" stop printf '%s\n' 'Personal-node lifecycle passed: TLS, signed KV/SQL, restart, fresh restore, upgrade, rollback restore' diff --git a/deploy/personal-node/compose.yaml b/deploy/personal-node/compose.yaml index f5634001..94018d13 100644 --- a/deploy/personal-node/compose.yaml +++ b/deploy/personal-node/compose.yaml @@ -1,7 +1,7 @@ name: personal-node services: node: - image: ${NODE_IMAGE:-ghcr.io/tinycloudlabs/tinycloud-node@sha256:41667d3d133aaa12290a2f3a744afacd838988327552d159c60f04466e66f86f} + image: ${NODE_IMAGE:-ghcr.io/tinycloudlabs/tinycloud-node@sha256:2f3c65d85aa94610456bf09cd54713bab5260097bb3d56858025bc6eadb5a4a0} restart: unless-stopped env_file: - ${NODE_KEY_ENV_FILE:?Set the private node key file} @@ -10,8 +10,9 @@ services: ROCKET_ADDRESS: 0.0.0.0 volumes: - ${NODE_DATA_DIR:?Set the node data directory}:/data - # 1.20.6 still embeds a sample key in its default TOML. This clean - # config masks the image copy, even when the operator key is absent. + # This config replaces the one bundled in the image and fails closed + # when the operator key is absent. The upgrade rehearsal starts from + # the 1.20.6 image; CI runs the bare image check on 1.22.0. - ./tinycloud.toml:/tinycloud.toml:ro security_opt: - no-new-privileges:true diff --git a/deploy/personal-node/node.sh b/deploy/personal-node/node.sh index d38f0222..ec358d9f 100755 --- a/deploy/personal-node/node.sh +++ b/deploy/personal-node/node.sh @@ -4,8 +4,9 @@ umask 077 here="$(cd "$(dirname "$0")" && pwd)" home="$(realpath -m "${NODE_HOME:-./personal-node}")" +image_1220='ghcr.io/tinycloudlabs/tinycloud-node@sha256:2f3c65d85aa94610456bf09cd54713bab5260097bb3d56858025bc6eadb5a4a0' +# The previous published image, used only to rehearse the upgrade. image_1206='ghcr.io/tinycloudlabs/tinycloud-node@sha256:41667d3d133aaa12290a2f3a744afacd838988327552d159c60f04466e66f86f' -image_1205='ghcr.io/tinycloudlabs/tinycloud-node@sha256:eba68094f922b8d377d4439bfa14e7d12b875de0155f03810cf6b5d4cb3b1ab7' fail() { printf '%s\n' "$*" >&2; exit 1; } require_home() { [[ -f "$home/node.env" && -f "$home/image" && -f "$home/domain" ]] || fail 'Initialize or restore the node first'; } @@ -64,18 +65,29 @@ identity() { [[ -s "$home/current-identity" ]] || fail 'Node identity is unavailable' } +# Accept only exit 1 plus the node's own complete ` Missing secret` line. +# Other diagnostics are never echoed. +require_missing_secret() { + local status="$1" file="$2" line missing=0 + while IFS= read -r line; do + if [[ "$line" == ' Missing secret' ]]; then missing=1; fi + done < "$file" + [[ "$status" == 1 && "$missing" == 1 ]] || + fail 'Missing-key check did not report the exact missing-secret error' +} + case "${1:-}" in init) - [[ $# -ge 2 ]] || fail 'Usage: node.sh init DOMAIN [test] [1.20.5]' + [[ $# -ge 2 ]] || fail 'Usage: node.sh init DOMAIN [test] [1.20.6]' [[ ! -e "$home" ]] || fail 'Choose a fresh NODE_HOME' [[ "$2" =~ ^[a-zA-Z0-9.-]+$ ]] || fail 'Invalid domain' [[ "${3:-}" == '' || "${3:-}" == test ]] || fail 'TLS mode must be test or omitted' - [[ "${4:-}" == '' || "${4:-}" == 1.20.5 ]] || fail 'Only 1.20.5 is supported as the previous image' + [[ "${4:-}" == '' || "${4:-}" == 1.20.6 ]] || fail 'Only 1.20.6 is supported as the previous image' mkdir -p "$home/data" printf '%s\n' "$2" > "$home/domain" printf '%s\n' "${3:-production}" > "$home/tls-mode" - printf '%s\n' "$image_1206" > "$home/image" - if [[ "${4:-}" == 1.20.5 ]]; then printf '%s\n' "$image_1205" > "$home/image"; fi + printf '%s\n' "$image_1220" > "$home/image" + if [[ "${4:-}" == 1.20.6 ]]; then printf '%s\n' "$image_1206" > "$home/image"; fi printf 'TINYCLOUD_KEYS__TYPE=Static\nTINYCLOUD_KEYS__SECRET=%s\n' \ "$(openssl rand -base64 48 | tr '+/' '-_' | tr -d '=\n')" > "$home/node.env" sudo chown -R 1000:1000 "$home/data" @@ -88,14 +100,19 @@ case "${1:-}" in printf 'TINYCLOUD_KEYS__TYPE=Static\n' > "$blank" status=0 NODE_KEY_ENV_FILE="$blank" timeout 30 "$0" compose-check >"$diagnostic" 2>&1 || status=$? - missing=0 - while IFS= read -r line; do - if [[ "$line" == ' Missing secret' ]]; then missing=1; fi - done < "$diagnostic" - [[ "$status" == 1 && "$missing" == 1 ]] || - fail 'Missing-key check did not report the exact missing-secret error' + require_missing_secret "$status" "$diagnostic" printf '%s\n' 'Missing-key startup failed as required' ;; + check-bare-image-missing-key) + # The published image started as shipped: no mounted config, no key, no + # environment. Its own default configuration must refuse to start. + diagnostic="$(mktemp)" + trap 'rm -f "$diagnostic"' EXIT + status=0 + timeout 60 docker run --rm "$image_1220" >"$diagnostic" 2>&1 || status=$? + require_missing_secret "$status" "$diagnostic" + printf '%s\n' 'Bare image startup without a key failed as required' + ;; compose-check) compose run --rm --no-deps node ;; @@ -111,11 +128,11 @@ case "${1:-}" in compose stop ;; check-version) - [[ $# == 2 ]] || fail 'Usage: node.sh check-version 1.20.5|1.20.6' + [[ $# == 2 ]] || fail 'Usage: node.sh check-version 1.20.6|1.22.0' health curl -fsS "${ca_args[@]}" "$(url)/version" | - jq -e --arg version "$2" '.version == $version and .quota_url == null and .inTEE == false' >/dev/null || - fail 'Unexpected node version, hosted quota, or TEE mode' + jq -e --arg version "$2" '.version == $version and .quota_url == null and .inTEE == false and ((.features | index("encryption-named-keys-v1")) == null) and ((.features | index("sql-delegated-caveats-v1")) == null)' >/dev/null || + fail 'Unexpected node version, hosted quota, TEE mode, or enabled named-key/delegated-SQL feature' ;; verify|seed) [[ $# == 2 && -f "$2" ]] || fail "Usage: node.sh $1 OWNER_KEY_FILE" @@ -157,9 +174,9 @@ case "${1:-}" in ;; upgrade) require_home - [[ "$(<"$home/image")" == "$image_1205" ]] || fail 'Upgrade expects the previous published image' + [[ "$(<"$home/image")" == "$image_1206" ]] || fail 'Upgrade expects the previous published image' compose stop node - printf '%s\n' "$image_1206" > "$home/image" + printf '%s\n' "$image_1220" > "$home/image" compose up -d node health ;; @@ -167,5 +184,5 @@ case "${1:-}" in [[ "$(<"$home/tls-mode")" == test && "$(basename "$home")" == tc1282-* ]] || fail 'Only tc1282 test instances may be removed' compose down -v ;; - *) fail 'Usage: node.sh {init|check-missing-key|start|check-version|seed|verify|restart|stop|backup|restore|upgrade|destroy-test}' ;; + *) fail 'Usage: node.sh {init|check-missing-key|check-bare-image-missing-key|start|check-version|seed|verify|restart|stop|backup|restore|upgrade|destroy-test}' ;; esac