From 0bc61275fdd64f9fe502f9d1044c199b8093a2f0 Mon Sep 17 00:00:00 2001 From: Steven Sklar Date: Fri, 9 Oct 2026 13:52:05 -0400 Subject: [PATCH] docs(operator): document backups left by departed instances Explains why departed instances' backup prefixes are never pruned, when they are still useful, and how to clean them up safely. Co-Authored-By: Claude Opus 5.5 --- .../operations/backup-restore.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/documentation/enterprise-kubernetes-operator/operations/backup-restore.md b/documentation/enterprise-kubernetes-operator/operations/backup-restore.md index 28c785bef..70c3cdebf 100644 --- a/documentation/enterprise-kubernetes-operator/operations/backup-restore.md +++ b/documentation/enterprise-kubernetes-operator/operations/backup-restore.md @@ -318,3 +318,46 @@ a valid target. Confirm the source retention window before creating the immutable destination, then follow the same bounded watch, failure checks, and data validation as a normal restore. + +## Backups left by departed instances + +Each instance writes its backups under the backup root, in a prefix named after +its backup instance name. A new name appears whenever an instance starts on a +fresh volume — for example, a demoted primary that rejoins as a replica after a +failover. Scaling down an instance that once took backups as primary also leaves +its prefix behind. + +The engine's cleaner only prunes its own instance's prefix. Nothing prunes a +departed instance's prefix — not the engine, and not the operator, which never +touches the object store. These backup sets stay in the bucket until you remove +them. + +Don't delete them blindly. A departed instance's set can still be useful: + +- right after a failover, new replicas seed from the old primary's latest backup + until the new primary completes its first backup; +- it can be a restore or PITR point (`sourceInstanceName`). + +To clean up: + +1. Get the name of every running instance — on each pod run: + + ```sql + SELECT backup_instance_name(); + ``` + +2. Get the current seed name: + + ```sh + kubectl get questdbcluster -n \ + -o jsonpath='{.status.replication.seed.backupInstanceName}{"\n"}' + ``` + +3. List the prefixes under the backup root. A prefix is safe to delete only if + it matches none of the names above and you don't need it as a restore point. + +:::warning +Bucket lifecycle rules that expire objects by age can't tell a departed prefix +from a live instance's older backups — they can delete backups you still need. +Scope such a rule to a specific departed prefix, or delete by hand. +:::