diff --git a/docs/ENV_REFERENCE.md b/docs/ENV_REFERENCE.md index fddc211..15dc967 100644 --- a/docs/ENV_REFERENCE.md +++ b/docs/ENV_REFERENCE.md @@ -33,4 +33,7 @@ source. | `SSM_RELOAD_LOG_LEVEL` | | `INFO` | Log verbosity for the reloader's structured logs. | | `OTEL_EXPORTER_OTLP_ENDPOINT` | | — | OTLP/HTTP endpoint to opt in to OpenTelemetry event export; unset disables telemetry entirely. | | `SSM_RELOAD_PROJECTION_CONFIGS` | | (empty) | Comma-separated `project/config` pairs to project even when no container is bound to them yet — an `env_file` must exist BEFORE the first `compose up` that reads it. | +| `SSM_RELOAD_SWARM_MODE` | | `False` | Manage Docker Swarm services instead of local containers: deliver each config as a Swarm secret/config object and roll it out with `docker service update` on change. Requires the local Docker socket to belong to a swarm MANAGER node; run a single instance of ssm-reload per swarm, not one per node. | +| `SSM_RELOAD_SWARM_SECRET_KIND` | | `secret` | Swarm object kind used to deliver secrets when SSM_RELOAD_SWARM_MODE is set: `secret` (encrypted at rest, the default) or `config` (unencrypted). Ignored outside swarm mode. | +| `SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR` | | `/run/ssm` | Absolute path a Swarm `config` object is mounted under when SSM_RELOAD_SWARM_SECRET_KIND=config; ignored for the `secret` kind, whose mount directory Swarm itself fixes at `/run/secrets`. Ignored outside swarm mode. | | `DOCKER_HOST` | | local Docker socket | Standard Docker endpoint override, honored directly by the Docker SDK's `docker.from_env()` — not a `ReloadSettings` field; `ssm-reload` never reads or validates it itself. | diff --git a/docs/SECRETS_RELOADER.md b/docs/SECRETS_RELOADER.md index 19e9d73..6d2f8a5 100644 --- a/docs/SECRETS_RELOADER.md +++ b/docs/SECRETS_RELOADER.md @@ -407,6 +407,95 @@ is a no-op with zero cost. Events carry semantic-convention attributes --- +## Swarm mode + +The quick start above delivers secrets via a bind-mounted volume, which only +works on a **single Docker host**. Docker Swarm services can run their tasks +on any node in the cluster, so set `SSM_RELOAD_SWARM_MODE=true` to switch +delivery to **Docker Swarm secrets or configs** instead — cluster-wide objects +Swarm itself replicates to whichever node a task lands on. + +### What changes + +| | Default (non-swarm) mode | Swarm mode | +| --- | --- | --- | +| Unit managed | A container | A Swarm **service** | +| Opt-in label lives on | The container | The service's `deploy.labels` OR its task template's `labels:` — ssm-reload checks both | +| Delivery mechanism | A dotenv file in a shared bind-mounted volume | A Docker **secret** or **config** object, attached to the service | +| How secrets reach the process | `env_file:` — literal environment variables | A mounted FILE the process must read itself (there is no Swarm-native way to turn a secret into a literal env var) | +| Mount path | `/run/ssm/-.env` | `/run/secrets/-.env` for the `secret` kind (fixed by Swarm itself); `SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR` (default `/run/ssm`) for the `config` kind | +| Rollout mechanism | Recreate the container | `docker service update`, which Swarm rolls out to every task across the cluster using the service's own update policy | +| Where it must run | One instance per Docker host | Exactly **one instance for the whole swarm**, on a **manager** node | + +### Will it reload a running Swarm service when a secret changes? + +**Yes, automatically** — `ssm-reload` mints a new secret/config object per +revision and calls `docker service update`, and Swarm itself rolls that out to +every task of the service, on every node, following the service's own update +policy. No manual `docker service update` is required. + +The one condition that must hold for this to actually reach the running +process: **the image must read its secrets from the mounted file**, because +Swarm has no mechanism to merge a secret's bytes into a service's literal +environment the way a recreated container's env can be merged. Source the +file in the entrypoint (`. /run/secrets/web-prod.env`) or use an image that +already reads a `*_FILE`/dotenv-style secret. **An image that hard-requires +literal environment variables, and cannot be given an entrypoint wrapper, +cannot pick up a rotated secret automatically in swarm mode — it must be +restarted by hand after a rotation**, or kept on the default (non-swarm) mode +instead. + +### Requirements and constraints + +- **Manager-only, single instance.** Creating secrets/configs and updating a + service both require a swarm manager's Docker API. Point `ssm-reload`'s + socket at a manager node and run exactly one replica — two instances would + race each other minting differently-named objects for the same revision. + Pin it with `deploy.placement.constraints: [node.role == manager]`. +- **`SSM_RELOAD_SWARM_SECRET_KIND`** selects `secret` (encrypted at rest — the + default) or `config` (plain). +- **`SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR`** (default `/run/ssm`) selects where a + `config`-kind object is mounted; ignored for the `secret` kind, whose + mount directory Swarm itself fixes at `/run/secrets`. +- **Old objects are garbage-collected automatically.** Docker refuses to + delete a secret still bound to a task, so `ssm-reload` recomputes "is + anything still using this?" from live cluster state every pass and removes + what it minted once nothing does — no durable bookkeeping of its own. +- **First boot is a real, structural gap.** A service's very first + `docker stack deploy` starts before `ssm-reload` has ever discovered it, so + no secret is attached yet; an image that hard-requires the file will + crash-loop until the next poll attaches it and rolls the service. Unlike the + default mode's `SSM_RELOAD_PROJECTION_CONFIGS` bootstrap list, there is no + way to pre-attach a secret to a service that does not exist yet. + What you CAN do: pre-create a placeholder secret and reference it + statically in the service's own `secrets:` block, at the same target + ssm-reload will use. Its first successful pass replaces that reference + with its own live one at the SAME target — matched by mount path, not by + name — taking it over rather than leaving both attached (which Swarm + would reject as a duplicate mount anyway). For example: + + ```yaml + secrets: + test-prod.env: + external: true + + services: + web: + secrets: + - source: test-prod.env + target: web-prod.env # matches env_filename(project, config) + ``` + + Create the placeholder once (`printf 'PLACEHOLDER="set me"\n' | docker secret + create test-prod.env -`), then `docker stack deploy`. Skip this entirely if + the image can tolerate crash-looping for one poll interval on a true first + deploy. + +A complete, runnable example lives at +[`../ssm_reload/docker-stack.swarm.example.yml`](../ssm_reload/docker-stack.swarm.example.yml). + +--- + ## Run it anywhere, at any scale `ssm-reload` is stateless and isolated — it depends only on the SSM API and @@ -451,8 +540,9 @@ conflict and need no coordination. - **Docker only.** Kubernetes support is designed for and can be added behind the same interface later; today `ssm-reload` manages Docker - containers. The projection sink is a seam too — a Kubernetes Secret slots in - behind the same renderer. + containers, plus Docker Swarm services in [swarm mode](#swarm-mode). The + projection sink is a seam too — a Kubernetes Secret slots in behind the + same renderer. - **Pull-based.** `ssm-reload` polls and watches Docker events; it does not require the SSM server to push to it. - **The tmpfs volume is empty after a reboot.** The reloader re-renders every diff --git a/frontend/src/lib/api/me.ts b/frontend/src/lib/api/me.ts index 40ba9e6..8bf7a1a 100644 --- a/frontend/src/lib/api/me.ts +++ b/frontend/src/lib/api/me.ts @@ -15,3 +15,10 @@ export async function updateMe(input: { email?: string; fullName?: string }): Pr }); return mapMeResponseDto(meResponseDtoSchema.parse(response)); } + +export async function changeMyPassword(input: { currentPassword: string; newPassword: string }): Promise { + await apiClient('/me/password', { + method: 'PATCH', + body: JSON.stringify(input) + }); +} diff --git a/frontend/src/lib/api/team.ts b/frontend/src/lib/api/team.ts index 2978ca0..f0a2955 100644 --- a/frontend/src/lib/api/team.ts +++ b/frontend/src/lib/api/team.ts @@ -62,6 +62,7 @@ export async function updateWorkspaceMember( fullName: string; workspaceRole: WorkspaceMember['workspaceRole']; disabled: boolean; + password: string; }> ) { await apiClient(`/workspace/members/${username}`, { @@ -70,6 +71,12 @@ export async function updateWorkspaceMember( }); } +export async function deleteWorkspaceMember(username: string) { + await apiClient<{ status?: string }>(`/workspace/members/${username}`, { + method: 'DELETE' + }); +} + export async function getProjectMembers(projectSlug: string): Promise { const response = await apiClient( `/workspace/projects/${projectSlug}/members` diff --git a/frontend/src/pages/AccountPage.tsx b/frontend/src/pages/AccountPage.tsx index af59da0..8cbc463 100644 --- a/frontend/src/pages/AccountPage.tsx +++ b/frontend/src/pages/AccountPage.tsx @@ -4,7 +4,7 @@ import { toast } from 'sonner'; import { Button } from '@/components/ui/button'; import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card'; import { Input } from '@/components/ui/input'; -import { getMe, updateMe } from '../lib/api/me'; +import { changeMyPassword, getMe, updateMe } from '../lib/api/me'; import { queryKeys } from '../lib/api/queryKeys'; import { notifyApiError } from '../lib/api/errorToast'; @@ -17,6 +17,8 @@ export function AccountPage() { const [email, setEmail] = useState(''); const [fullName, setFullName] = useState(''); + const [currentPassword, setCurrentPassword] = useState(''); + const [newPassword, setNewPassword] = useState(''); useEffect(() => { if (!me) return; @@ -35,8 +37,20 @@ export function AccountPage() { } }); + const changePasswordMutation = useMutation({ + mutationFn: changeMyPassword, + onSuccess: () => { + setCurrentPassword(''); + setNewPassword(''); + toast.success('Password changed'); + }, + onError: (error) => { + notifyApiError(error, 'Failed to change password'); + } + }); + return ( -
+
Account @@ -80,6 +94,35 @@ export function AccountPage() { )} + + + + Change Password + Update your login password. + + +
+ + setCurrentPassword(event.target.value)} + /> +
+
+ + setNewPassword(event.target.value)} /> +
+
+ +
+ +
); } diff --git a/frontend/src/pages/TeamPage.tsx b/frontend/src/pages/TeamPage.tsx index 9ce5a6c..d753602 100644 --- a/frontend/src/pages/TeamPage.tsx +++ b/frontend/src/pages/TeamPage.tsx @@ -5,6 +5,15 @@ import { Button } from '@/components/ui/button'; import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card'; import { Input } from '@/components/ui/input'; import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, + DialogTrigger +} from '@/components/ui/dialog'; import { Select, SelectContent, @@ -14,6 +23,7 @@ import { } from '@/components/ui/select'; import { createWorkspaceMember, + deleteWorkspaceMember, getWorkspaceMembers, getWorkspaceSettings, updateWorkspaceMember, @@ -27,6 +37,102 @@ import { notifyApiError } from '../lib/api/errorToast'; const WORKSPACE_ROLES: WorkspaceMember['workspaceRole'][] = ['owner', 'admin', 'collaborator', 'viewer']; const PROJECT_ROLES: WorkspaceSettings['defaultProjectRole'][] = ['admin', 'collaborator', 'viewer', 'none']; +function ResetPasswordDialog({ + username, + onSubmit, + isPending +}: { + username: string; + onSubmit: (password: string) => void; + isPending: boolean; +}) { + const [open, setOpen] = useState(false); + const [password, setPassword] = useState(''); + + return ( + { + setOpen(nextOpen); + if (!nextOpen) setPassword(''); + }} + > + + + + + + Reset password for {username} + Sets a new password for this member immediately. + + setPassword(event.target.value)} + /> + + + + + + ); +} + +function DeleteMemberDialog({ + username, + onConfirm, + isPending +}: { + username: string; + onConfirm: () => void; + isPending: boolean; +}) { + const [open, setOpen] = useState(false); + + return ( + + + + + + + Delete {username}? + + This permanently removes the user, their credentials, and all workspace/project/group memberships. This + cannot be undone. + + + + + + + + ); +} + export function TeamPage() { const queryClient = useQueryClient(); const { data: me } = useQuery({ queryKey: queryKeys.me(), queryFn: getMe }); @@ -79,6 +185,24 @@ export function TeamPage() { onError: (error) => notifyApiError(error, 'Failed to update member status') }); + const resetPasswordMutation = useMutation({ + mutationFn: ({ username, password }: { username: string; password: string }) => + updateWorkspaceMember(username, { password }), + onSuccess: () => { + toast.success('Password reset'); + }, + onError: (error) => notifyApiError(error, 'Failed to reset password') + }); + + const deleteMemberMutation = useMutation({ + mutationFn: deleteWorkspaceMember, + onSuccess: () => { + queryClient.invalidateQueries({ queryKey: queryKeys.workspaceMembers() }); + toast.success('Member deleted'); + }, + onError: (error) => notifyApiError(error, 'Failed to delete member') + }); + const updateSettingsMutation = useMutation({ mutationFn: updateWorkspaceSettings, onSuccess: () => { @@ -194,6 +318,22 @@ export function TeamPage() { > {member.disabled ? 'Disabled (click to enable)' : 'Enabled (click to disable)'} + {canManageMembers && ( + + resetPasswordMutation.mutate({ username: member.username, password }) + } + /> + )} + {canManageMembers && member.username !== me?.username && ( + deleteMemberMutation.mutate(member.username)} + /> + )}
))} diff --git a/ssm_reload/AGENTS.md b/ssm_reload/AGENTS.md index 0e63447..844181c 100644 --- a/ssm_reload/AGENTS.md +++ b/ssm_reload/AGENTS.md @@ -11,7 +11,12 @@ **delivery** (render every managed config to a dotenv file consumers read at create time) and **convergence** (recreate the containers it owns when those secrets change). It is an external service: it speaks only to the SSM HTTP API -and the local Docker socket, and keeps no durable state of its own. +and the local Docker socket, and keeps no durable state of its own. In +**swarm mode** (`SSM_RELOAD_SWARM_MODE=true`) both jobs are reshaped around +Docker Swarm services instead of containers: delivery becomes a Swarm +secret/config object, and convergence becomes `docker service update`, which +Swarm itself rolls out cluster-wide — see `swarm_driver.py` and the Session +Lesson below. | Module | Answers | | --- | --- | @@ -20,12 +25,13 @@ and the local Docker socket, and keeps no durable state of its own. | `projection.py` | `Projector` — the reloader's stateful wrapper over the stateless sink: which configs are already rendered (a MISSING file forces an unconditional export), and what revision was last rendered (so a config with no container bound still gets a 304 fast path). Rendering is best-effort. | | `driver.py` | The `ReloadDriver` Protocol — the seam the loop drives, so a Kubernetes driver can replace Docker without touching `reconcile`/`client`. | | `docker_driver.py` | The Docker implementation: `discover` (opt-in containers + their **lifecycle** facts), `read_binding`, `read_managed_keys`, `ensure_volume`, `apply` (non-destructive recreate + rollback + netns convergence), and the pure `build_recreate_spec` that clones a container's full runtime spec while MERGING fresh secrets into its env. | +| `swarm_driver.py` | `SwarmDriver` — the Docker Swarm implementation of the same `ReloadDriver` seam, but for **services**, not containers: `discover` finds opted-in services (label lives in `Spec.Labels`, i.e. `deploy.labels`), `apply` mints an immutable secret/config object per revision and calls `service.update()` (`fetch_current_spec=True`, so untouched spec fields survive), and `gc()` prunes rotated objects nothing references any more, recomputed from live cluster state each pass. `read_env`/`read_managed_keys` are deliberately always-empty — see the Session Lesson below. | | `client.py` | The three-call SSM HTTP client: `conditional_export` (`If-None-Match` → 304 or 200+ETag), `report_reload` (`POST /reload/events`), and `report_status` (`POST /reload/report`, the per-cycle fleet heartbeat). Mirrors `ssm_cli/api.py` conventions but never imports `ssm_cli` (no `click`/`keyring` in a daemon). | -| `config.py` | `ReloadSettings` — the ONE validated `pydantic-settings` class holding every env var the service reads; `.load()` fails fast on start-up (wrapping the pydantic error into `SsmReloadError`). Also everything that is deliberately NOT an env var: the `com.bearlike.ssm` label constants, the external-owner labels it reads but never writes, `PROJECTION_DIR`/`PROJECTION_VOLUME`, and the tmpfs volume options. `DOCKER_HOST` is honored by the Docker SDK itself, not parsed here. | +| `config.py` | `ReloadSettings` — the ONE validated `pydantic-settings` class holding every env var the service reads; `.load()` fails fast on start-up (wrapping the pydantic error into `SsmReloadError`). Also everything that is deliberately NOT an env var: the `com.bearlike.ssm` label constants, the external-owner labels it reads but never writes, `PROJECTION_DIR`/`PROJECTION_VOLUME`, the tmpfs volume options, and `SWARM_OBJECT_PREFIX`. `DOCKER_HOST` is honored by the Docker SDK itself, not parsed here. | | `models.py` | The driver-agnostic types carried across the seam: `ConfigRef` (the ONE parser for `project/config`), `Unit` (`raw` is the driver's private handle), `Binding`, `Dependent`, and `Lifecycle` — whose fields are each a REASON TO REFUSE a recreate, and which answers the questions about them itself (`settling_reason`, `stranded_by_recreate`) rather than letting the loop reach into its state. | | `errors.py` | The `SsmReloadError` hierarchy; any one raised while talking to SSM means "do nothing for this config". | | `__main__.py` | Entry point for `python -m ssm_reload`. | -| `Dockerfile` / `docker-compose.example.yml` | How the service is packaged and deployed alongside an opted-in workload (the example is the worked gluetun + netns-dependent stack). | +| `Dockerfile` / `docker-compose.example.yml` / `docker-stack.swarm.example.yml` | How the service is packaged and deployed alongside an opted-in workload (the compose example is the worked gluetun + netns-dependent stack; the stack example is the swarm-mode equivalent). | ## Non-obvious decisions @@ -378,3 +384,63 @@ and the local Docker socket, and keeps no durable state of its own. `docker run --rm ` reaching the `Configuration error:` / `ReloadSettings.load()` stage — a `ModuleNotFoundError` before that line means a dep is missing from the image. +- **Swarm mode (`SSM_RELOAD_SWARM_MODE`) manages services, not containers — + and that forces a different delivery mechanism, not just a different + driver.** A container's env can be merged on recreate; a Swarm secret/config + cannot be merged into a service's literal env at all — Swarm only ever + mounts it as a FILE, so the workload's entrypoint must source it itself. + `SwarmDriver.read_env`/`read_managed_keys` are deliberately hard-coded empty + (never `{}`/`set()` from a real read) because there is nothing on the + service spec to compare a secret's *contents* against — the adopt-by- + comparison trick that lets container mode skip a redundant recreate simply + does not apply here; `held_revision == new_etag` (checked before adoption is + ever considered) is what keeps the steady state cheap instead. Swarm + secrets/configs are also immutable and cluster-wide (not per-host), which is + why rotation mints a brand-new object per revision instead of editing one in + place, and why exactly ONE `ssm-reload` instance — on a manager node — may + run per swarm: two instances would race to mint differently-named objects + for the same revision. Verified against docker-py 7.2.0: + `Service.update(secrets=[...], labels=...)` defaults + `fetch_current_spec=True` and merges at the FIELD level (only the keys you + pass override; everything else — image, command, mounts, healthcheck, + placement, networks — comes from the live spec), so passing only + `secrets`/`labels` is safe and does not need a hand-rolled full-spec clone + the way the container driver's `build_recreate_spec` does. Objects this + service minted are deleted only once `gc()` (called every pass, from live + `service.list()` state, never persisted) finds nothing still referencing + them — Swarm itself refuses to delete a secret still bound to a task, so + there is no safe way to prune eagerly. +- **Two swarm-mode bugs, both from trusting a Docker asymmetry that isn't + obvious until you hit it.** (1) `docker service ls`'s own `label` filter + only ever inspects a service's OWN labels (`Spec.Labels`, i.e. Compose's + `deploy.labels`) — never its task template's container labels (Compose's + plain `labels:` under a service, without `deploy:`). Filtering `discover()` + server-side on that label silently dropped every service opted in the + "wrong" way, with NO error anywhere — it just looked like ssm-reload was + ignoring the labels entirely. Fix: fetch every service unfiltered and check + BOTH label sources in Python (`_labels()` merges them, service wins on + conflict since that's where ssm-reload writes its own). (2) A Swarm + **secret**'s mount directory is hard-fixed to `/run/secrets/` — `target` + only renames the file, it never relocates it — while a **config** has no + such restriction and accepts an arbitrary absolute path. Passing the same + bare filename for both kinds (as an early draft did) meant secrets landed + at the documented `/run/secrets/` but configs landed at `/` + (root), contradicting the docs' claimed `/run/ssm/` parity with the + non-swarm mode. Fix: `_target()` branches on `secret_kind` — bare filename + for secrets, `SWARM_CONFIG_MOUNT_DIR`-prefixed for configs. Neither bug + raised an exception or showed up in logs; both were "nothing happens, + silently" failures, which is why `test_swarm_driver.py` now asserts + `discover()` finds a service via EITHER label placement, and asserts the + literal mount path for each `secret_kind`, not just that `apply()` ran. +- **First-boot bootstrap for swarm mode: match by mount TARGET, not by our + own naming prefix, when deciding what to replace.** An operator can + pre-create a placeholder secret and reference it statically in the + service's `secrets:` block so the file exists before ssm-reload's first + pass (mirrors the non-swarm mode's `SSM_RELOAD_PROJECTION_CONFIGS` + bootstrap, but there is no equivalent env-var list here since a secret + can't be attached to a service that doesn't exist yet). For that hand-off + to be safe, `_update_service`'s "which existing reference do I replace" + check keys off `File.Name` (the mount target) rather than the + `ssm---` name prefix ssm-reload itself mints — Swarm + already forbids two references at one target, so this is also strictly + more correct for the steady state, not just the bootstrap case. diff --git a/ssm_reload/README.md b/ssm_reload/README.md index ed23e18..8a663f5 100644 --- a/ssm_reload/README.md +++ b/ssm_reload/README.md @@ -118,6 +118,74 @@ There is no setting to bring the old recreate-on-drift behavior back. It raced your deploys and silently dropped compose-file-only variables — the kind of failure that only surfaces once, in production, at the worst time. +## Swarm mode + +The bind-mounted `ssm-env` volume above only works on a **single Docker +host**: a Swarm service's tasks can land on any node, and a plain Docker +volume does not follow them there. Set `SSM_RELOAD_SWARM_MODE=true` to switch +delivery to **Docker Swarm secrets or configs** instead — cluster-wide +objects Swarm itself replicates to whichever node a task runs on. + +What's different from the default mode: + +- **The unit of change is the *service*, not a container.** ssm-reload + discovers Swarm services carrying `com.bearlike.ssm.enable=true` on + EITHER a service label (`deploy.labels` in a stack file) or a container + label (plain `labels:` under the service, without `deploy:`) — Compose + puts those two in different places, so ssm-reload checks both rather than + making you get the placement right. It mints a new immutable + secret/config object per revision and calls `docker service update` to + point the service at it. Swarm turns that into a rolling replacement of + every task, on every node — **this does reload running workloads + automatically**, the same promise as the default mode. +- **Secrets are delivered as a mounted FILE, never as literal service env.** + Docker Swarm has no mechanism to merge secret bytes into a service's `Env` + the way a recreated container's environment can be merged. A Swarm + **secret**'s mount directory is fixed by Swarm itself — it always lands at + `/run/secrets/-.env` — while a Swarm **config** has no such + restriction, so it is mounted under `SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR` + instead (default `/run/ssm`, matching the non-swarm mode's path). + **Your image's entrypoint must read that file itself** — source it + (`. /run/secrets/web-prod.env`) before exec'ing the real process, or use + an image that already reads `*_FILE`/dotenv-style secrets. An image that + hard-requires literal env vars and cannot be given an entrypoint wrapper + **cannot pick up a rotated secret automatically** in this mode; stay on the + default (non-swarm) mode for it, or restart it by hand after a rotation. +- **Only a swarm MANAGER can create secrets/configs or update a service.** + Point ssm-reload's Docker socket at a manager node, and run **exactly one + instance for the whole swarm** — not one per node. Two instances would race + each other minting differently-named objects for the same revision. + `deploy.placement.constraints: [node.role == manager]` in the stack file is + how you pin it there. +- **Old objects are pruned automatically, once nothing references them.** + Docker refuses to delete a secret still bound to a task, so ssm-reload + recomputes "is anything still using this?" from live cluster state every + pass and removes what it minted once nothing does — it holds no durable + bookkeeping of its own, same as everywhere else in this service. +- **`SSM_RELOAD_SWARM_SECRET_KIND`** picks `secret` (encrypted at rest — the + default) or `config` (plain) as the object type. +- **`SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR`** (default `/run/ssm`) picks where a + `config`-kind object is mounted; ignored for the `secret` kind, whose + directory Swarm itself fixes. +- **First boot is a real gap, not a nice-to-have.** A service's very first + `docker stack deploy` starts before ssm-reload has ever seen it, so no + secret is attached yet — an image that hard-requires the file will + crash-loop until the next poll (`SSM_RELOAD_POLL_INTERVAL`) attaches it and + rolls the service. Unlike the default mode's `SSM_RELOAD_PROJECTION_CONFIGS` + bootstrap list, there is no way to pre-attach a secret to a service that + does not exist yet — this is a structural property of Swarm secrets being + service-scoped, not a missing feature. You CAN close this gap yourself: + pre-create a placeholder secret and reference it statically in the + service's own `secrets:` block, at the SAME target ssm-reload will use + (`-.env`). ssm-reload's first successful pass replaces + that reference with its own live one at the same target — matched by + mount path, not by name — so the placeholder is taken over cleanly rather + than left attached alongside it. See the bootstrap comment in + `docker-stack.swarm.example.yml` for the exact commands. + +A complete, runnable stack lives at +[`docker-stack.swarm.example.yml`](docker-stack.swarm.example.yml). + ## Documentation Full product guide — quick start, configuration reference, behavior, and diff --git a/ssm_reload/config.py b/ssm_reload/config.py index f3b1e33..6487706 100644 --- a/ssm_reload/config.py +++ b/ssm_reload/config.py @@ -81,7 +81,15 @@ "o": "size=8m,mode=0750", } +# Prefix for the Docker Swarm secret/config objects ssm-reload creates. Each +# rotation mints a NEW object (Swarm objects are immutable once created), so +# the name always carries a content hash; this is the fixed namespace for it, +# same rationale as LABEL_PREFIX -- a knob here just lets a fleet's own +# rotations collide with a hand-created object of the same short name. +SWARM_OBJECT_PREFIX = "ssm" + LogLevel = Literal["DEBUG", "INFO", "WARNING", "ERROR"] +SwarmSecretKind = Literal["secret", "config"] class ReloadSettings(BaseSettings): @@ -141,6 +149,36 @@ class ReloadSettings(BaseSettings): "BEFORE the first `compose up` that reads it." ), ) + swarm_mode: bool = Field( + default=False, + validation_alias="SSM_RELOAD_SWARM_MODE", + description=( + "Manage Docker Swarm services instead of local containers: " + "deliver each config as a Swarm secret/config object and roll " + "it out with `docker service update` on change. Requires the " + "local Docker socket to belong to a swarm MANAGER node; run a " + "single instance of ssm-reload per swarm, not one per node." + ), + ) + swarm_secret_kind: SwarmSecretKind = Field( + default="secret", + validation_alias="SSM_RELOAD_SWARM_SECRET_KIND", + description=( + "Swarm object kind used to deliver secrets when " + "SSM_RELOAD_SWARM_MODE is set: `secret` (encrypted at rest, the " + "default) or `config` (unencrypted). Ignored outside swarm mode." + ), + ) + swarm_config_mount_dir: str = Field( + default="/run/ssm", + validation_alias="SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR", + description=( + "Absolute path a Swarm `config` object is mounted under when " + "SSM_RELOAD_SWARM_SECRET_KIND=config; ignored for the `secret` " + "kind, whose mount directory Swarm itself fixes at " + "`/run/secrets`. Ignored outside swarm mode." + ), + ) @property def bootstrap_configs(self) -> tuple[ConfigRef, ...]: @@ -191,6 +229,24 @@ def _normalize_log_level(cls, value: object) -> object: return value.strip().upper() return value + @field_validator("swarm_secret_kind", mode="before") + @classmethod + def _normalize_secret_kind(cls, value: object) -> object: + if isinstance(value, str): + return value.strip().lower() + return value + + @field_validator("swarm_config_mount_dir") + @classmethod + def _validate_mount_dir(cls, value: str) -> str: + # Fail fast: a relative path here would silently mount nowhere near + # what the operator expects once it reaches the Docker API. + if not value.startswith("/"): + raise ValueError( + f"{value!r} must be an absolute path (starting with '/')" + ) + return value.rstrip("/") or "/" + @classmethod def load(cls) -> "ReloadSettings": """Build settings from the environment, failing fast and clean. diff --git a/ssm_reload/docker-stack.swarm.example.yml b/ssm_reload/docker-stack.swarm.example.yml new file mode 100644 index 0000000..fd747b7 --- /dev/null +++ b/ssm_reload/docker-stack.swarm.example.yml @@ -0,0 +1,99 @@ +# Example Docker Swarm stack: ssm-reload managing Swarm SECRETS instead of +# the bind-mounted `ssm-env` volume the single-host example uses. +# +# docker stack deploy -c ssm_reload/docker-stack.swarm.example.yml ssmdemo +# +# Why this file looks different from docker-compose.example.yml: +# +# * Swarm secrets/configs are IMMUTABLE cluster objects, delivered to a +# task as a FILE at container-create time -- there is no Swarm-native way +# to merge them into a service's literal environment the way a plain +# container's env can be merged. So the workload's entrypoint must SOURCE +# the mounted file itself (see the `web` service below); an image that +# only reads literal env vars needs a small wrapper, or must stay on the +# non-swarm (bind-mount) delivery mode instead. +# * Only a swarm MANAGER can create secrets/update services, so ssm-reload +# itself must be constrained to a manager node and run as a SINGLE +# replica -- two instances would race each other minting objects for the +# same revision. `deploy.placement.constraints` below is what enforces +# that; do not scale this service. +# * The control-plane labels (`com.bearlike.ssm.*`) are read from EITHER +# `deploy.labels` (the service's own labels) OR the plain `labels:` below +# it (the task template's container labels) -- Compose puts these two in +# different places (see the comment on `web` below), and ssm-reload +# checks both so it doesn't matter which one you use. +# +# Rollout is still automatic: when a config's secrets change, ssm-reload +# mints a new Swarm secret and calls `docker service update`, which Swarm +# itself rolls out to every task of that service, on every node, using the +# service's own update policy. Nothing needs to be restarted by hand -- AS +# LONG AS the workload reads its secrets from the mounted file rather than +# expecting them as literal environment variables. + +# OPTIONAL bootstrap secret. ssm-reload attaches its OWN secret to `web` +# dynamically via the Docker API once it discovers the service -- it does NOT +# read this block. It exists only so `web` has SOMETHING at +# /run/secrets/web-prod.env on its very FIRST boot, before ssm-reload's first +# reconcile pass has run (see "First boot" in ../docs/SECRETS_RELOADER.md). +# Create the placeholder once, then deploy: +# +# printf 'PLACEHOLDER="set me"\n' | docker secret create test-prod.env - +# docker stack deploy -c ssm_reload/docker-stack.swarm.example.yml ssmdemo +# +# On ssm-reload's first successful pass it REPLACES this reference with its +# own live secret at the SAME target (matched by mount path, not by name), so +# the placeholder is safely taken over rather than left attached alongside +# it. Skip this block entirely if `web` can tolerate crash-looping for one +# poll interval on a truly first deploy. +secrets: + test-prod.env: + external: true + +services: + ssm-reload: + image: ghcr.io/bearlike/ssm-reload:latest + environment: + SSM_BASE_URL: "http://ssm:5000/api" + SSM_TOKEN: "${SSM_RELOAD_TOKEN:?set SSM_RELOAD_TOKEN}" + SSM_RELOAD_POLL_INTERVAL: "30" + SSM_RELOAD_SWARM_MODE: "true" + # "secret" (encrypted at rest, the default) or "config" (plain). + SSM_RELOAD_SWARM_SECRET_KIND: "secret" + # Where a "config"-kind object is mounted; ignored for "secret" (Swarm + # itself fixes that directory at /run/secrets). + SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR: "/run/ssm" + volumes: + # Must be a swarm MANAGER's socket -- only a manager can create + # secrets or update services. On a multi-manager swarm, bind-mount + # whichever manager this stack is pinned to. + - /var/run/docker.sock:/var/run/docker.sock:ro + deploy: + replicas: 1 + placement: + constraints: [node.role == manager] + + web: + image: myorg/web-api:1.4.2 + # OPTIONAL: the bootstrap placeholder above, mounted at the SAME path + # ssm-reload uses (`env_filename(project, config)` = "web-prod.env", + # under Swarm's fixed `/run/secrets/` directory for the "secret" kind). + # Remove this block entirely if you skipped the top-level `secrets:` + # bootstrap section. + secrets: + - source: test-prod.env + target: web-prod.env + mode: 0440 + environment: # app-native config, stays in git + LOG_LEVEL: info + deploy: + replicas: 3 + # Service-level labels (`deploy.labels`) -- ssm-reload reads these. + # Compose's plain `labels:` directly under a service (NOT under + # `deploy:`) sets the task/container labels instead; ssm-reload reads + # those too, so either placement works -- use whichever this stack + # file already uses for its other labels, for consistency. + labels: + com.bearlike.ssm.enable: "true" # opt in — required + com.bearlike.ssm.config: "web/prod" # which project/config to track + # com.bearlike.ssm.revision is stamped by ssm-reload — do not set it. + diff --git a/ssm_reload/docker_driver.py b/ssm_reload/docker_driver.py index 457977a..1929f8a 100644 --- a/ssm_reload/docker_driver.py +++ b/ssm_reload/docker_driver.py @@ -97,6 +97,10 @@ class RecreateSpec: class DockerDriver: """Manage local Docker containers opted in via ``.enable=true``.""" + # What the event stream watches to adopt a new/redeployed unit fast. + EVENT_TYPE = "container" + EVENT_ACTIONS = ["start", "create"] + def __init__(self, *, client: Any = None) -> None: self._client = client diff --git a/ssm_reload/runner.py b/ssm_reload/runner.py index 05a0980..8c5f6e4 100644 --- a/ssm_reload/runner.py +++ b/ssm_reload/runner.py @@ -3,9 +3,10 @@ ``reconcile`` is fired on two triggers: * a periodic poll every ``SSM_RELOAD_POLL_INTERVAL`` seconds, and -* the Docker event stream (container ``start``/``create`` carrying the - fixed ``com.bearlike.ssm.enable=true`` label) for near-instant adoption - of new or redeployed containers. +* the Docker event stream (container ``start``/``create``, or in swarm mode + service ``create``/``update``, carrying the fixed + ``com.bearlike.ssm.enable=true`` label) for near-instant adoption of new or + redeployed units. Both triggers call the same idempotent :func:`reconcile`; the event stream just wakes it up sooner. The runner keeps no DURABLE state -- its @@ -21,11 +22,11 @@ import socket import threading from types import FrameType -from typing import Any +from typing import TYPE_CHECKING, Any, Mapping import ssm_telemetry from ssm_contracts import Trigger -from ssm_projection import DirectorySink +from ssm_projection import DirectorySink, ProjectionSink from ssm_reload import __version__ from ssm_reload.client import SsmClient from ssm_reload.config import ( @@ -38,6 +39,10 @@ from ssm_reload.errors import SsmReloadError from ssm_reload.projection import Projector from ssm_reload.reconcile import Reconciler +from ssm_reload.swarm_driver import SwarmDriver + +if TYPE_CHECKING: + from ssm_reload.driver import ReloadDriver logger = logging.getLogger("ssm_reload.runner") @@ -48,7 +53,7 @@ class Runner: def __init__( self, settings: ReloadSettings, - driver: DockerDriver, + driver: "ReloadDriver", reconciler: Reconciler, *, host: str | None = None, @@ -76,6 +81,13 @@ def run_once(self, trigger: Trigger = "poll") -> None: with self._reconcile_lock: try: self.reconciler.run(trigger) + # Swarm mode only: prune rotated secret/config objects no + # service references any more. Recomputed from live cluster + # state, so it is safe to run every pass -- see + # SwarmDriver.gc. + gc = getattr(self.driver, "gc", None) + if gc is not None: + gc() except SsmReloadError as exc: logger.warning("Reconcile pass skipped: %s", exc) except Exception as exc: # keep the daemon alive on failure. @@ -92,13 +104,27 @@ def _poll_loop(self) -> None: def _event_loop(self) -> None: label = f"{ENABLE_LABEL}=true" + # DockerDriver watches container start/create; SwarmDriver watches + # service create/update (a service is never "started" itself -- its + # tasks are). Read off the driver rather than branching on its type, + # so a future driver just declares its own pair. + event_type = getattr(self.driver, "EVENT_TYPE", "container") + event_actions = getattr( + self.driver, "EVENT_ACTIONS", ["start", "create"] + ) while not self._stop.is_set(): try: - events = self.driver.client.events( + # `.client` (the underlying docker-py client) is a shared + # implementation detail of both Docker-based drivers, not + # part of the driver-agnostic `ReloadDriver` seam -- read via + # getattr so a future non-Docker driver isn't forced to grow + # a mismatched attribute just to satisfy this loop. + docker_client = getattr(self.driver, "client") + events = docker_client.events( decode=True, filters={ - "type": "container", - "event": ["start", "create"], + "type": event_type, + "event": event_actions, "label": label, }, ) @@ -115,15 +141,19 @@ def _event_loop(self) -> None: def start(self) -> None: """Block running both triggers until interrupted.""" logger.info( - "ssm-reload starting: host=%s poll=%ss projecting to %s", + "ssm-reload starting: host=%s poll=%ss", self.host, self.settings.poll_interval, - PROJECTION_DIR, ) - # Best-effort: makes the volume a first-class Docker object that - # consumer stacks can declare `external: true`, and warns if an - # existing one would put projected secrets on the host's disk. - self.driver.ensure_volume(PROJECTION_VOLUME) + # Best-effort, and DockerDriver-only: makes the volume a first-class + # Docker object that consumer stacks can declare `external: true`, + # and warns if an existing one would put projected secrets on the + # host's disk. Swarm mode delivers via secret/config objects instead + # and has no local volume to create. + ensure_volume = getattr(self.driver, "ensure_volume", None) + if ensure_volume is not None: + ensure_volume(PROJECTION_VOLUME) + logger.info("Projecting to %s", PROJECTION_DIR) event_thread = threading.Thread( target=self._event_loop, name="ssm-reload-events", daemon=True ) @@ -139,17 +169,47 @@ def build_runner(settings: ReloadSettings) -> Runner: # injected, so nothing below reaches for its own dependencies. # The token is unwrapped only here, at its single point of use. client = SsmClient(settings.base_url, settings.token.get_secret_value()) - driver = DockerDriver() + driver: "ReloadDriver" + sink: ProjectionSink + if settings.swarm_mode: + driver = SwarmDriver( + secret_kind=settings.swarm_secret_kind, + config_mount_dir=settings.swarm_config_mount_dir, + ) + # Delivery IS the Swarm secret/config object in this mode; there is + # no local file for anything to read, so projecting one would only + # ever fail (no volume mounted) or mislead (a file nothing reads). + sink = _NullSink() + else: + driver = DockerDriver() + sink = DirectorySink(PROJECTION_DIR) reconciler = Reconciler( driver, client, socket.gethostname(), - Projector(DirectorySink(PROJECTION_DIR)), + Projector(sink), bootstrap_configs=settings.bootstrap_configs, ) return Runner(settings, driver, reconciler, host=reconciler.host) +class _NullSink: + """No-op :class:`ProjectionSink` for swarm mode. + + ``exists`` reports true unconditionally, so :class:`Projector` never + forces an unconditional export chasing a "missing" file that was never + meant to be written; ``write`` does nothing and says so. + """ + + def write( + self, project: str, config: str, secrets: Mapping[str, str] + ) -> str: + return "swarm secret/config object (no local file projected)" + + def exists(self, project: str, config: str) -> bool: + return True + + def main() -> int: # Build (and validate) settings first: a bad env var fails fast with a # clean SsmReloadError instead of a raw pydantic traceback. Logging is diff --git a/ssm_reload/swarm_driver.py b/ssm_reload/swarm_driver.py new file mode 100644 index 0000000..9c443d1 --- /dev/null +++ b/ssm_reload/swarm_driver.py @@ -0,0 +1,412 @@ +"""Docker Swarm implementation of the :class:`ReloadDriver` seam. + +Swarm is a genuinely different runtime shape from a plain Docker host, not +just a bigger one, and that shape drives every decision below: + +* A container's env is frozen at *create* time on a single host; a Swarm + **service**'s desired state lives on the manager and is rolled out to + whichever node a task lands on. So there is no "recreate this container" + operation here -- the unit of change is the *service*, and the mechanism is + ``docker service update``, which Swarm itself turns into a rolling + replacement of that service's tasks, across every node running one. +* Docker secrets/configs are immutable objects, encrypted at rest (secrets) + or plain (configs), and delivered to a task as a FILE at container-create + time -- there is no Swarm-native way to inject them as literal environment + variables the way a plain container's ``Env`` can be merged. Consumers must + read the mounted file themselves (source it in an entrypoint, or use an + image that supports ``*_FILE``/``*_SECRETFILE`` env-file conventions). This + is the deliberate trade for running across multiple nodes: SSM never ships + secret bytes anywhere Swarm itself would not already replicate them. +* Only a swarm MANAGER can create secrets/configs or update a service, so + this driver -- unlike :class:`~ssm_reload.docker_driver.DockerDriver`, + which is meant to run one instance per host -- must run as a SINGLE + instance with its Docker socket pointed at (or bind-mounted on) a manager + node. Two instances would race each other minting differently-named + objects for the same revision. +* Because each rotation mints a brand-new object (Swarm objects cannot be + edited in place), old objects accumulate unless something prunes them. + :meth:`SwarmDriver.gc` deletes any object under our namespace that no + running service still references -- computed fresh from live cluster + state each pass, so it holds no durable state of its own, consistent with + the rest of ssm-reload. +* A Swarm secret's mount point is FIXED at ``/run/secrets/`` -- only + the filename is customizable, not the directory. A config has no such + restriction, so its mount directory is a setting + (``SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR``, default ``/run/ssm``) rather than + hardcoded. ``apply`` picks the target accordingly per + :attr:`SwarmDriver.secret_kind`. +* ``docker service ls``'s own ``label`` filter only ever matches a + service's OWN labels (``Spec.Labels``, i.e. Compose's ``deploy.labels``), + never its task template's container labels (Compose's plain ``labels:`` + under a service). Relying on that filter at discovery time would silently + drop a service opted in the "wrong" way -- an easy mix-up, since a plain + container reads the opposite ("container-level") labels. ``discover`` + therefore fetches every service and checks BOTH label sources itself. +""" + +from __future__ import annotations + +import hashlib +import logging +from datetime import datetime, timezone +from typing import Any + +from ssm_projection import env_filename, render_dotenv +from ssm_reload.config import ( + CONFIG_LABEL, + ENABLE_LABEL, + OWNER_LABEL, + REVISION_LABEL, + SWARM_OBJECT_PREFIX, + SwarmSecretKind, +) +from ssm_reload.docker_driver import _age_seconds, _docker +from ssm_reload.errors import BindingError, DriverError +from ssm_reload.models import Binding, ConfigRef, Lifecycle, Unit + +logger = logging.getLogger("ssm_reload.swarm_driver") + + +def _object_name(ref: ConfigRef, revision: str) -> str: + """The Swarm object name for one revision of one config. + + Includes a content hash of the revision (rather than the raw ETag, + which may hold characters Swarm object names reject) so every rotation + gets its own immutable object, and the same revision always maps back + to the same name -- a retried apply reuses the object instead of + minting a duplicate. + """ + digest = hashlib.sha256(revision.encode("utf-8")).hexdigest()[:12] + return f"{SWARM_OBJECT_PREFIX}-{ref.project}-{ref.config}-{digest}" + + +def _object_prefix(ref: ConfigRef) -> str: + """Every object name ssm-reload has ever minted for this config.""" + return f"{SWARM_OBJECT_PREFIX}-{ref.project}-{ref.config}-" + + +def _target( + secret_kind: SwarmSecretKind, config_mount_dir: str, filename: str +) -> str: + """Where ``filename`` is mounted for this object kind. + + A secret's directory is fixed by Swarm itself (``/run/secrets/``); a + config has no such restriction, so it is placed under + ``config_mount_dir`` (``SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR``) instead. + """ + if secret_kind == "secret": + return filename + return f"{config_mount_dir}/{filename}" + + +class SwarmDriver: + """Manage Swarm services opted in via ``.enable=true``. + + ``secret_kind`` picks the Swarm object type used to deliver a config's + content: ``"secret"`` (encrypted at rest -- the default) or ``"config"`` + (plain). Either way the payload is the SAME dotenv document the + non-swarm driver projects to a file, so an image that already knows how + to read one can be pointed at the mounted path unchanged. + """ + + # A service is never itself "started"; its tasks are. Watch create/update + # on the service object so a redeploy is adopted near-instantly. + EVENT_TYPE = "service" + EVENT_ACTIONS = ["create", "update"] + + def __init__( + self, + *, + secret_kind: SwarmSecretKind = "secret", + config_mount_dir: str = "/run/ssm", + client: Any = None, + ) -> None: + self.secret_kind = secret_kind + self.config_mount_dir = config_mount_dir + self._client = client + + @property + def client(self) -> Any: + if self._client is None: + self._client = _docker().from_env() + return self._client + + @property + def _objects(self) -> Any: + if self.secret_kind == "secret": + return self.client.secrets + return self.client.configs + + def discover(self) -> list[Unit]: + # Fetch every service rather than filtering server-side: Docker's + # own `label` filter on this endpoint only ever inspects a + # service's OWN labels, never its task template's container labels + # -- see the module docstring. `_is_enabled` checks both. + try: + services = self.client.services.list() + except Exception as exc: + raise DriverError( + "Could not list Swarm services -- is this node's Docker " + f"socket a swarm MANAGER? {exc}" + ) from exc + now = datetime.now(timezone.utc) + return [ + Unit( + id=service.id, + name=service.name, + raw=service, + lifecycle=self._lifecycle(service, now), + ) + for service in services + if _is_enabled(service) + ] + + def _lifecycle(self, service: Any, now: datetime) -> Lifecycle: + # Services have no per-task "created but not started" state to read + # cheaply, and no netns-sharing concept between them -- overlay + # networks make that unnecessary. The settling window still guards + # against racing an operator's OWN `docker service update` / + # `docker stack deploy` that just landed on this same service. + attrs = service.attrs or {} + return Lifecycle( + owner=None, + status="running", + age_seconds=_age_seconds(attrs.get("UpdatedAt"), now), + dependents=(), + ) + + def read_binding(self, unit: Unit) -> Binding: + labels = _labels(unit.raw) + config_label = labels.get(CONFIG_LABEL) + if not config_label: + raise BindingError(f"{unit.name}: missing '{CONFIG_LABEL}' label") + try: + ref = ConfigRef.parse(config_label) + except BindingError as exc: + raise BindingError(f"{unit.name}: {CONFIG_LABEL} {exc}") from exc + return Binding( + project=ref.project, + config=ref.config, + held_revision=labels.get(REVISION_LABEL) or None, + ) + + def read_env(self, unit: Unit) -> dict[str, str]: + # Secrets are delivered as a mounted FILE, never as literal service + # env -- there is nothing here to compare against `secrets`, so this + # always reports empty. `held_revision == new_etag` (checked before + # this is ever consulted) is what makes the steady state cheap; a + # non-empty `secrets` map can never look "already current" via env, + # which is the safe direction to be wrong in. + return {} + + def read_managed_keys(self, unit: Unit) -> set[str]: + # The delivered object IS the full rendered document each rotation, + # so a key removed from a config is simply absent from the next + # object -- there is no leftover-env-var hazard to track here. + return set() + + def apply(self, unit: Unit, env: dict[str, str], revision: str) -> None: + """Roll ``unit``'s service onto a freshly minted secret/config object. + + Creates (or reuses, if a previous attempt got this far and failed + later) an immutable object named after ``revision``, then updates the + service to reference it in place of any earlier object under this + config's namespace -- which is what makes Swarm itself roll the + change out to every task, on every node, via its own rolling-update + policy. + """ + service = unit.raw + binding = self.read_binding(unit) + ref = ConfigRef(binding.project, binding.config) + filename = env_filename(ref.project, ref.config) + target = _target(self.secret_kind, self.config_mount_dir, filename) + content = render_dotenv(env).encode("utf-8") + + try: + object_id, object_name = self._ensure_object( + ref, revision, content + ) + self._update_service( + service, object_id, object_name, target, revision + ) + except DriverError: + raise + except Exception as exc: + raise DriverError( + f"{unit.name}: swarm rollout failed: {exc}" + ) from exc + + def _ensure_object( + self, ref: ConfigRef, revision: str, content: bytes + ) -> tuple[str, str]: + name = _object_name(ref, revision) + docker = _docker() + try: + created = self._objects.create( + name=name, + data=content, + labels={ + OWNER_LABEL: "ssm-reload", + CONFIG_LABEL: str(ref), + }, + ) + return created.id, name + except docker.errors.APIError as exc: + if exc.response is not None and exc.response.status_code == 409: + # Same revision, retried after a failure past this point: + # reuse the object instead of erroring on "already exists". + existing = self._objects.get(name) + return existing.id, name + raise DriverError( + f"could not create {self.secret_kind} {name}: {exc}" + ) from exc + + def _update_service( + self, + service: Any, + object_id: str, + object_name: str, + target: str, + revision: str, + ) -> None: + docker = _docker() + kind_kwarg = "secrets" if self.secret_kind == "secret" else "configs" + reference_cls = ( + docker.types.SecretReference + if self.secret_kind == "secret" + else docker.types.ConfigReference + ) + # Match on the MOUNT TARGET, not our own naming prefix: a service + # can carry at most one reference per target path anyway (Swarm + # rejects two), and this is what lets a manually pre-created + # bootstrap secret/config at the same target be taken over cleanly + # instead of erroring as a duplicate mount. + existing = _current_references(service, self.secret_kind) + kept = [ + r for r in existing if (r.get("File") or {}).get("Name") != target + ] + new_reference = reference_cls( + object_id, object_name, filename=target, mode=0o440 + ) + labels = dict(_labels(service)) + labels[REVISION_LABEL] = revision + try: + service.update( + fetch_current_spec=True, + labels=labels, + **{kind_kwarg: [*_reference_objects(kept), new_reference]}, + ) + except Exception as exc: + raise DriverError( + f"{service.name}: service update failed: {exc}" + ) from exc + + def gc(self) -> None: + """Delete rotated objects no service currently references. + + Recomputed from live cluster state every call -- never persisted -- + because an object cannot safely be deleted the moment a new one is + minted: Swarm refuses to delete a secret/config still bound to a + task that has not rolled over yet. Waiting until no service + references it at all is what makes this safe to run every pass. + """ + try: + services = self.client.services.list() + except Exception as exc: + logger.warning("gc: could not list services: %s", exc) + return + referenced: set[str] = set() + for service in services: + for kind in ("secret", "config"): + for reference in _current_references(service, kind): + ref_id = reference.get("SecretID") or reference.get( + "ConfigID" + ) + if ref_id: + referenced.add(ref_id) + + try: + objects = self._objects.list( + filters={"label": f"{OWNER_LABEL}=ssm-reload"} + ) + except Exception as exc: + logger.warning( + "gc: could not list %s objects: %s", self.secret_kind, exc + ) + return + for obj in objects: + if obj.id in referenced: + continue + try: + obj.remove() + logger.info( + "gc: removed unreferenced %s %s", + self.secret_kind, + obj.name, + ) + except Exception as exc: + logger.warning( + "gc: could not remove %s %s: %s", + self.secret_kind, + getattr(obj, "name", obj.id), + exc, + ) + + +def _service_labels(service: Any) -> dict[str, str]: + return ((service.attrs or {}).get("Spec") or {}).get("Labels") or {} + + +def _container_labels(service: Any) -> dict[str, str]: + spec = ((service.attrs or {}).get("Spec") or {}).get("TaskTemplate") or {} + return (spec.get("ContainerSpec") or {}).get("Labels") or {} + + +def _labels(service: Any) -> dict[str, str]: + """Service labels (``deploy.labels``) merged over container/task labels + (plain ``labels:``) -- either placement works, and service labels win + on a conflict since that is where ssm-reload writes its own. + """ + return {**_container_labels(service), **_service_labels(service)} + + +def _is_enabled(service: Any) -> bool: + return _labels(service).get(ENABLE_LABEL) == "true" + + +def _current_references(service: Any, kind: str) -> list[dict[str, Any]]: + spec = ((service.attrs or {}).get("Spec") or {}).get("TaskTemplate") or {} + container_spec = spec.get("ContainerSpec") or {} + key = "Secrets" if kind == "secret" else "Configs" + return list(container_spec.get(key) or []) + + +def _reference_objects(raw_refs: list[dict[str, Any]]) -> list[Any]: + """Rebuild SDK reference objects from the raw spec dicts we kept. + + ``service.update`` expects ``SecretReference``/``ConfigReference`` + instances, not the plain dicts ``docker inspect`` returns -- so a + reference we are NOT touching must still be round-tripped through the + SDK's own type rather than passed through as-is. + """ + docker = _docker() + rebuilt = [] + for raw in raw_refs: + file_ = raw.get("File") or {} + is_secret = "SecretID" in raw + cls = ( + docker.types.SecretReference + if is_secret + else docker.types.ConfigReference + ) + rebuilt.append( + cls( + raw.get("SecretID") or raw.get("ConfigID"), + raw.get("SecretName") or raw.get("ConfigName"), + filename=file_.get("Name"), + uid=file_.get("UID", "0"), + gid=file_.get("GID", "0"), + mode=file_.get("Mode", 0o444), + ) + ) + return rebuilt diff --git a/ssm_server/access/userpass.py b/ssm_server/access/userpass.py index af0bf61..f10dc93 100644 --- a/ssm_server/access/userpass.py +++ b/ssm_server/access/userpass.py @@ -104,6 +104,27 @@ def remove(self, username): result = {"status": "OK"} return result, 200 + def set_password(self, username, password): + """Overwrite an existing user's password (admin/self reset). + Args: + username (str): Username + password (str): New plaintext password + Returns: + dict : Dictionary with operation status + """ + finder = self._userpass.find_one({"username": username}) + if not finder: + return "User does not exist", 400 + if not self.p_pol.check(password): + return f"Password policy not met. {self.p_pol}", 400 + password_hash = generate_password_hash( + password, method="pbkdf2:sha256" + ) + self._userpass.update_one( + {"username": username}, {"$set": {"password": password_hash}} + ) + return {"status": "OK"}, 200 + def is_authorized(self, username, password): """Check if a userpass is valid Args: diff --git a/ssm_server/api/resources/me.py b/ssm_server/api/resources/me.py index 109b3f2..2aad2a6 100644 --- a/ssm_server/api/resources/me.py +++ b/ssm_server/api/resources/me.py @@ -15,6 +15,14 @@ "fullName", type=str, required=False, location="json" ) +password_update_parser = api.parser() +password_update_parser.add_argument( + "currentPassword", type=str, required=True, location="json" +) +password_update_parser.add_argument( + "newPassword", type=str, required=True, location="json" +) + def _serialize_me(username): actor_context = conn.rbac.resolve_personal_actor(username) @@ -70,3 +78,25 @@ def patch(self): api.abort(code, msg) return _serialize_me(username), 200 + + +@me_ns.route("/password") +class MePasswordResource(Resource): + @api.doc(security=["Bearer", "Token"], parser=password_update_parser) + @with_token + def patch(self): + username = g.actor.get("subject_user") + if not username: + api.abort(403, "Service tokens do not have a user profile") + + args = password_update_parser.parse_args() + if not conn.userpass.is_authorized(username, args["currentPassword"]): + api.abort(400, "Current password is incorrect") + + status, code = conn.userpass.set_password( + username, args["newPassword"] + ) + if code >= 400: + api.abort(code, status) + + return {"status": "OK"}, 200 diff --git a/ssm_server/api/resources/workspace/workspace_resource.py b/ssm_server/api/resources/workspace/workspace_resource.py index 58ea89d..b8d9edf 100644 --- a/ssm_server/api/resources/workspace/workspace_resource.py +++ b/ssm_server/api/resources/workspace/workspace_resource.py @@ -51,6 +51,9 @@ member_update_parser.add_argument( "disabled", type=bool, required=False, location="json" ) +member_update_parser.add_argument( + "password", type=str, required=False, location="json" +) project_member_put_parser = api.parser() project_member_put_parser.add_argument( @@ -179,6 +182,18 @@ def _update_member_disabled(username, payload): api.abort(code, msg) +def _update_member_password(username, payload): + if "password" not in payload: + return + password = payload.get("password") + if not password: + return + # Admin override: resets credentials directly, no current-password check. + status, code = conn.userpass.set_password(username, password) + if code >= 400: + api.abort(code, status) + + def _update_member_workspace_role(workspace_id, username, payload): if "workspaceRole" not in payload: return @@ -313,6 +328,7 @@ def patch(self, username): payload = _parse_member_payload() _update_member_profile(username, payload) _update_member_disabled(username, payload) + _update_member_password(username, payload) _update_member_workspace_role(workspace_id, username, payload) user = conn.users.get(username) @@ -334,19 +350,18 @@ def delete(self, username): if not user: api.abort(404, "User not found") - _, msg, code = conn.users.set_disabled(username, True) - if code >= 400: - api.abort(code, msg) - - membership = conn.memberships.get_workspace_membership( - workspace_id, username + # Hard delete: credentials, memberships, group membership, and the + # user document itself. Best-effort on the userpass entry since some + # legacy users may not have one. + conn.userpass.remove(username) + conn.memberships.remove_workspace_membership(workspace_id, username) + conn.memberships.remove_all_for_subject( + workspace_id, "user", username ) - return { - "status": "OK", - "member": _serialize_member_row( - conn.users.get(username), membership - ), - }, 200 + conn.groups.remove_user_from_all_groups(workspace_id, username) + conn.users.delete(username) + + return {"status": "OK"}, 200 @workspace_ns.route("/projects//members") diff --git a/tests/reload/test_config.py b/tests/reload/test_config.py index 4b2a7e7..e3ead2e 100644 --- a/tests/reload/test_config.py +++ b/tests/reload/test_config.py @@ -104,3 +104,27 @@ def test_stray_whitespace_is_trimmed(): settings = ReloadSettings(base_url=" http://ssm/api ", token=" tok ") assert settings.base_url == "http://ssm/api" assert settings.token.get_secret_value() == "tok" + + +def test_swarm_config_mount_dir_defaults_to_run_ssm(): + settings = ReloadSettings(base_url="http://ssm", token="t") + assert settings.swarm_config_mount_dir == "/run/ssm" + + +def test_swarm_config_mount_dir_reads_override(monkeypatch): + _clear_env(monkeypatch) + monkeypatch.setenv("SSM_BASE_URL", "http://ssm/api") + monkeypatch.setenv("SSM_TOKEN", "tok") + monkeypatch.setenv("SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR", "/etc/ssm/") + settings = ReloadSettings.load() + # Trailing slash is normalized away so concatenation never double-slashes. + assert settings.swarm_config_mount_dir == "/etc/ssm" + + +def test_swarm_config_mount_dir_must_be_absolute(monkeypatch): + _clear_env(monkeypatch) + monkeypatch.setenv("SSM_BASE_URL", "http://ssm/api") + monkeypatch.setenv("SSM_TOKEN", "tok") + monkeypatch.setenv("SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR", "relative/path") + with pytest.raises(SsmReloadError): + ReloadSettings.load() diff --git a/tests/reload/test_swarm_driver.py b/tests/reload/test_swarm_driver.py new file mode 100644 index 0000000..14fe100 --- /dev/null +++ b/tests/reload/test_swarm_driver.py @@ -0,0 +1,425 @@ +"""SwarmDriver: the Docker Swarm implementation of the ReloadDriver seam. + +Hermetic like the rest of the suite -- the Docker SDK is faked, never +imported for real. ``_docker()`` is monkeypatched to hand back a small +namespace standing in for the ``docker`` module (``types``/``errors``), +and every service/secret is a bare object carrying just the attributes +the driver reads. +""" + +from __future__ import annotations + +from types import SimpleNamespace +from typing import Any + +import pytest + +from ssm_reload.config import CONFIG_LABEL, ENABLE_LABEL, REVISION_LABEL +from ssm_reload.errors import BindingError, DriverError +from ssm_reload.models import Unit +from ssm_reload.swarm_driver import SwarmDriver, _object_name, _object_prefix + + +class FakeAPIError(Exception): + def __init__(self, status_code: int | None = None) -> None: + super().__init__("api error") + self.response = ( + SimpleNamespace(status_code=status_code) + if status_code is not None + else None + ) + + +def _reference_cls(id_key: str, name_key: str): + class _Reference(dict): + def __init__( + self, obj_id, name, filename=None, uid=None, gid=None, mode=0o444 + ): + self[id_key] = obj_id + self[name_key] = name + self["File"] = { + "Name": filename or name, + "UID": uid or "0", + "GID": gid or "0", + "Mode": mode, + } + + return _Reference + + +SecretReference = _reference_cls("SecretID", "SecretName") +ConfigReference = _reference_cls("ConfigID", "ConfigName") + +FAKE_DOCKER = SimpleNamespace( + types=SimpleNamespace( + SecretReference=SecretReference, ConfigReference=ConfigReference + ), + errors=SimpleNamespace(APIError=FakeAPIError), +) + + +class FakeObject: + def __init__(self, obj_id: str, name: str) -> None: + self.id = obj_id + self.name = name + self.removed = False + + def remove(self) -> None: + self.removed = True + + +class FakeObjectCollection: + """Stands in for ``client.secrets`` / ``client.configs``.""" + + def __init__(self) -> None: + self.created: list[dict[str, Any]] = [] + self._by_name: dict[str, FakeObject] = {} + self._next_id = 0 + self.create_error: Exception | None = None + + def create(self, *, name: str, data: bytes, labels: dict[str, str]): + if self.create_error is not None: + raise self.create_error + self._next_id += 1 + obj = FakeObject(f"obj-{self._next_id}", name) + self._by_name[name] = obj + self.created.append({"name": name, "data": data, "labels": labels}) + return obj + + def get(self, name: str): + return self._by_name[name] + + def list(self, filters=None): + return list(self._by_name.values()) + + +class FakeService: + def __init__(self, service_id: str, name: str, attrs: dict[str, Any]): + self.id = service_id + self.name = name + self.attrs = attrs + self.update_calls: list[dict[str, Any]] = [] + + def update(self, **kwargs: Any) -> None: + self.update_calls.append(kwargs) + + +class FakeServiceCollection: + def __init__(self, services: list[FakeService]) -> None: + self._services = services + + def list(self, filters=None): + return list(self._services) + + +def _service_attrs( + labels: dict[str, str], + secrets: list[dict[str, Any]] | None = None, + container_labels: dict[str, str] | None = None, +) -> dict[str, Any]: + return { + "Spec": { + "Labels": labels, + "TaskTemplate": { + "ContainerSpec": { + "Secrets": secrets or [], + "Configs": [], + "Labels": container_labels or {}, + } + }, + }, + "UpdatedAt": "2026-01-01T00:00:00.000000000Z", + } + + +def _driver( + secrets_col=None, + services=None, + secret_kind="secret", + configs_col=None, + config_mount_dir="/run/ssm", +): + client = SimpleNamespace( + secrets=secrets_col or FakeObjectCollection(), + configs=configs_col or FakeObjectCollection(), + services=FakeServiceCollection(services or []), + ) + return SwarmDriver( + secret_kind=secret_kind, + config_mount_dir=config_mount_dir, + client=client, + ) + + +@pytest.fixture(autouse=True) +def _patch_docker_sdk(monkeypatch): + monkeypatch.setattr("ssm_reload.swarm_driver._docker", lambda: FAKE_DOCKER) + + +def test_object_name_is_deterministic_per_revision(): + from ssm_reload.models import ConfigRef + + ref = ConfigRef("web", "prod") + assert _object_name(ref, "v1") == _object_name(ref, "v1") + assert _object_name(ref, "v1") != _object_name(ref, "v2") + assert _object_name(ref, "v1").startswith(_object_prefix(ref)) + + +def test_read_binding_parses_labels(): + driver = _driver() + service = FakeService( + "svc1", + "web", + _service_attrs( + { + ENABLE_LABEL: "true", + CONFIG_LABEL: "web/prod", + REVISION_LABEL: '"v1"', + } + ), + ) + unit = Unit(id=service.id, name=service.name, raw=service) + binding = driver.read_binding(unit) + assert binding.project == "web" + assert binding.config == "prod" + assert binding.held_revision == '"v1"' + + +def test_read_binding_missing_label_raises(): + driver = _driver() + service = FakeService("svc1", "web", _service_attrs({})) + unit = Unit(id=service.id, name=service.name, raw=service) + with pytest.raises(BindingError): + driver.read_binding(unit) + + +def test_discover_finds_service_via_service_level_labels(): + # Compose's `deploy.labels`. + service = FakeService( + "svc1", + "web", + _service_attrs({ENABLE_LABEL: "true", CONFIG_LABEL: "web/prod"}), + ) + driver = _driver(services=[service]) + units = driver.discover() + assert [u.id for u in units] == ["svc1"] + + +def test_discover_finds_service_via_container_level_labels_too(): + # Compose's plain `labels:` (no `deploy:`) sets the TASK TEMPLATE's + # container labels, not the service's own -- `docker service ls`'s + # label filter only sees the latter, so discovery must check both. + service = FakeService( + "svc1", + "web", + _service_attrs( + {}, + container_labels={ + ENABLE_LABEL: "true", + CONFIG_LABEL: "web/prod", + }, + ), + ) + driver = _driver(services=[service]) + units = driver.discover() + assert [u.id for u in units] == ["svc1"] + + +def test_discover_ignores_services_without_enable_label(): + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + driver = _driver(services=[service]) + assert driver.discover() == [] + + +def test_read_env_and_managed_keys_are_always_empty(): + # Secrets are delivered as a mounted file, never as literal service env, + # so there is nothing to compare -- see swarm_driver module docstring. + driver = _driver() + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + unit = Unit(id=service.id, name=service.name, raw=service) + assert driver.read_env(unit) == {} + assert driver.read_managed_keys(unit) == set() + + +def test_apply_creates_secret_and_updates_service(): + secrets_col = FakeObjectCollection() + service = FakeService( + "svc1", + "web", + _service_attrs({CONFIG_LABEL: "web/prod", ENABLE_LABEL: "true"}), + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1", "B": "2"}, '"rev1"') + + assert len(secrets_col.created) == 1 + assert secrets_col.created[0]["name"].startswith("ssm-web-prod-") + assert b'A="1"' in secrets_col.created[0]["data"] + + assert len(service.update_calls) == 1 + call = service.update_calls[0] + assert call["labels"][REVISION_LABEL] == '"rev1"' + assert len(call["secrets"]) == 1 + assert call["secrets"][0]["SecretName"] == secrets_col.created[0]["name"] + assert call["secrets"][0]["File"]["Name"] == "web-prod.env" + + +def test_apply_preserves_unrelated_secret_references(): + secrets_col = FakeObjectCollection() + other_ref = { + "SecretID": "other-id", + "SecretName": "unrelated-secret", + "File": {"Name": "unrelated", "UID": "0", "GID": "0", "Mode": 292}, + } + service = FakeService( + "svc1", + "web", + _service_attrs({CONFIG_LABEL: "web/prod"}, secrets=[other_ref]), + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1"}, '"rev1"') + + call = service.update_calls[0] + names = {ref["SecretName"] for ref in call["secrets"]} + assert "unrelated-secret" in names + assert len(call["secrets"]) == 2 + + +def test_apply_reuses_object_on_conflict(): + secrets_col = FakeObjectCollection() + secrets_col.create_error = FakeAPIError(status_code=409) + # Pre-seed the object the 409 branch expects to find already there. + from ssm_reload.models import ConfigRef + + name = _object_name(ConfigRef("web", "prod"), '"rev1"') + secrets_col._by_name[name] = FakeObject("existing-id", name) + + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1"}, '"rev1"') + + assert service.update_calls[0]["secrets"][0]["SecretID"] == "existing-id" + + +def test_apply_wraps_other_api_errors(): + secrets_col = FakeObjectCollection() + secrets_col.create_error = FakeAPIError(status_code=500) + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + unit = Unit(id=service.id, name=service.name, raw=service) + + with pytest.raises(DriverError): + driver.apply(unit, {"A": "1"}, '"rev1"') + + +def test_apply_mounts_config_kind_under_run_ssm(): + # Secrets are pinned to /run/secrets/ by Swarm itself; configs are + # not, so they use the configurable SSM_RELOAD_SWARM_CONFIG_MOUNT_DIR. + configs_col = FakeObjectCollection() + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + driver = _driver( + services=[service], secret_kind="config", configs_col=configs_col + ) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1"}, '"rev1"') + + call = service.update_calls[0] + assert call["configs"][0]["File"]["Name"] == "/run/ssm/web-prod.env" + + +def test_apply_honors_custom_config_mount_dir(): + configs_col = FakeObjectCollection() + service = FakeService( + "svc1", "web", _service_attrs({CONFIG_LABEL: "web/prod"}) + ) + driver = _driver( + services=[service], + secret_kind="config", + configs_col=configs_col, + config_mount_dir="/etc/ssm", + ) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1"}, '"rev1"') + + call = service.update_calls[0] + assert call["configs"][0]["File"]["Name"] == "/etc/ssm/web-prod.env" + + +def test_apply_takes_over_a_bootstrap_secret_at_the_same_target(): + # A placeholder secret an operator pre-created for first-boot bootstrap, + # mounted at the same target ssm-reload uses, must be REPLACED rather + # than left attached alongside ours (Swarm rejects two refs at one + # target anyway). + secrets_col = FakeObjectCollection() + placeholder = { + "SecretID": "placeholder-id", + "SecretName": "test-prod.env", + "File": { + "Name": "web-prod.env", + "UID": "0", + "GID": "0", + "Mode": 292, + }, + } + service = FakeService( + "svc1", + "web", + _service_attrs({CONFIG_LABEL: "web/prod"}, secrets=[placeholder]), + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + unit = Unit(id=service.id, name=service.name, raw=service) + + driver.apply(unit, {"A": "1"}, '"rev1"') + + call = service.update_calls[0] + assert len(call["secrets"]) == 1 + assert call["secrets"][0]["SecretID"] != "placeholder-id" + + +def test_gc_removes_only_unreferenced_owned_objects(): + secrets_col = FakeObjectCollection() + kept = secrets_col.create(name="ssm-web-prod-aaa", data=b"", labels={}) + orphan = secrets_col.create(name="ssm-web-prod-bbb", data=b"", labels={}) + + service = FakeService( + "svc1", + "web", + _service_attrs( + {CONFIG_LABEL: "web/prod"}, + secrets=[ + { + "SecretID": kept.id, + "SecretName": kept.name, + "File": { + "Name": "web-prod.env", + "UID": "0", + "GID": "0", + "Mode": 292, + }, + } + ], + ), + ) + driver = _driver(secrets_col=secrets_col, services=[service]) + + driver.gc() + + assert kept.removed is False + assert orphan.removed is True