You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6b46f69
Browse filesBrowse the repository at this point in the historyBrowse files
fix(self-hosting): stop publishing Postgres and require POSTGRES_PASSWORD (#8035)
* fix(self-hosting): stop publishing Postgres and require POSTGRES_PASSWORD
The Compose files published the db service on every host interface, and the production file fell back to the password postgres when POSTGRES_PASSWORD was unset. The db is now reachable only over the Compose network, and the production file refuses to start without POSTGRES_PASSWORD. sim-setup writes the value an install needs before bringing it up: a generated password for a new database, and the legacy one for a volume created before it was required, since Postgres ignores POSTGRES_PASSWORD on an existing data directory.
* fix(sim-setup): persist a shell POSTGRES_PASSWORD and rotate the effective role
A password exported only in the shell is now written to .env, so a later run without the export does not fall back to the legacy value. Rotation steps use the install's POSTGRES_USER and ALTER ROLE CURRENT_USER.
* fix(sim-setup): refuse an ambiguous or unstorable shell POSTGRES_PASSWORD
An empty export, one that differs from .env, or one .env cannot hold verbatim now stops setup with instructions instead of silently picking a value the database may not have been created with.
* fix(sim-setup): only persist URL-safe shell passwords; honor an empty POSTGRES_USER export
DATABASE_URL embeds the password unescaped, so a shell-only password is persisted only when it is made of URL-unreserved characters. An empty POSTGRES_USER export resolves to the Compose default, matching ${POSTGRES_USER:-postgres}.
* fix(sim-setup): leave a shell-exported POSTGRES_PASSWORD to the operator
Compose interpolates a shell value over .env, so the wizard no longer copies it into .env and no longer needs rules for values .env or DATABASE_URL would change. An empty export is still refused, since Compose would use it and refuse to start.
* docs(self-hosting): note that POSTGRES_* values must be URL-safe
docker-compose.prod.yml composes DATABASE_URL from them as written, so a value containing a URL delimiter initializes the database but breaks the connection string.
<a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Generate%20required%20secrets%20%28the%20stack%20will%20not%20start%20without%20them%29%3A%0A%20%20%20cat%20%3E%20.env%20%3C%3C%20EOF%0A%20%20%20BETTER_AUTH_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20ENCRYPTION_KEY%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20INTERNAL_API_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20CRON_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20EOF%0A%0A4.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%2C%203002%2C%20or%205432%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d"><img src="https://img.shields.io/badge/Set%20Up%20with-Cursor-E6E6E6?logo=cursor&logoColor=1A1A1A&labelColor=C3C3C3&color=E6E6E6" alt="Set Up with Cursor"></a>
10
+
<a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Generate%20required%20secrets%20%28the%20stack%20will%20not%20start%20without%20them%29%3A%0A%20%20%20cat%20%3E%20.env%20%3C%3C%20EOF%0A%20%20%20BETTER_AUTH_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20ENCRYPTION_KEY%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20INTERNAL_API_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20CRON_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20POSTGRES_PASSWORD%3D%24%28openssl%20rand%20-hex%2024%29%0A%20%20%20EOF%0A%0A4.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%20or%203002%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d"><img src="https://img.shields.io/badge/Set%20Up%20with-Cursor-E6E6E6?logo=cursor&logoColor=1A1A1A&labelColor=C3C3C3&color=E6E6E6" alt="Set Up with Cursor"></a>
Copy file name to clipboardExpand all lines: apps/docs/content/docs/platform/self-hosting/docker.mdx
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,13 +43,15 @@ EOF
43
43
44
44
<Callouttype="info">
45
45
Do not set `DATABASE_URL` or `BETTER_AUTH_URL` in `.env` — `docker-compose.prod.yml` composes both on the service definition, and a value set here is ignored. Change `POSTGRES_*` and `NEXT_PUBLIC_APP_URL` instead.
46
+
47
+
Because `DATABASE_URL` is composed from them, keep `POSTGRES_USER`, `POSTGRES_PASSWORD`, and `POSTGRES_DB` URL-safe — letters, digits, and `.``_``~``-`. They are inserted into the connection string as written, so a value containing `@`, `/`, `?`, `#`, `%`, or a space can initialize the database while leaving the app and migrations unable to connect. `openssl rand -hex` output is always safe.
46
48
</Callout>
47
49
48
50
<Callouttype="error">
49
51
Save `ENCRYPTION_KEY` and `API_ENCRYPTION_KEY` somewhere outside this server. `ENCRYPTION_KEY` encrypts workspace and personal environment variables, stored provider API keys, MCP OAuth credentials, and deployment/chat secrets; `API_ENCRYPTION_KEY` encrypts user-generated Sim API keys. Neither can be regenerated — a database restore paired with a different key leaves the data it protected permanently unreadable.
50
52
</Callout>
51
53
52
-
The compose file refuses to start if `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, or `INTERNAL_API_SECRET` is missing, rather than booting with empty values. `CRON_SECRET` is treated more gently: without it the `cron` service prints what to set and exits, leaving the rest of the stack running — so upgrading from a compose file that predates the scheduler still works.
54
+
The compose file refuses to start if `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, `INTERNAL_API_SECRET`, or `POSTGRES_PASSWORD` is missing, rather than booting with empty or well-known values. Postgres applies `POSTGRES_PASSWORD` only when it first creates the database volume — see [Postgres on Compose](/platform/self-hosting/security#postgres-on-compose) before changing it on an existing install. `CRON_SECRET` is treated more gently: without it the `cron` service prints what to set and exits, leaving the rest of the stack running — so upgrading from a compose file that predates the scheduler still works.
53
55
54
56
Images track `latest` unless you pin them. For production, see [Upgrades](/platform/self-hosting/upgrades).
55
57
@@ -65,7 +67,7 @@ Six services start:
65
67
|---|---|---|
66
68
|`simstudio`| 3000 | Main application (8 GB memory limit) |
67
69
|`realtime`| 3002 | WebSocket server (1 GB memory limit) |
68
-
|`db`|5432| PostgreSQL 17 with pgvector |
70
+
|`db`|internal| PostgreSQL 17 with pgvector — not published to the host|
69
71
|`redis`| internal | Pub/sub and shared cache — not published to the host |
70
72
|`cron`| — | Runs the [background jobs](/platform/self-hosting/background-jobs) on a schedule |
71
73
|`migrations`| — | Applies schema migrations once, then exits |
@@ -206,5 +208,5 @@ npx sim-setup update
206
208
{ question: "Do scheduled workflows work on Docker Compose?", answer: "Yes. The cron service runs the same jobs the Helm chart schedules as Kubernetes CronJobs, using the schedules in docker/crontab. It needs CRON_SECRET — without it the service prints what to set and exits, and the rest of the stack keeps running."},
207
209
{ question: "Why is there a Redis container?", answer: "Redis backs pub/sub for live Chat task status and table events, plus shared caches. Pub/sub has no fallback that works across processes, so live status would not stream without it. The port is deliberately not published so it cannot collide with a local Redis."},
208
210
{ question: "How do I back up and restore the database?", answer: "Back up with: docker compose -f docker-compose.prod.yml exec -T db pg_dump -U postgres simstudio > backup.sql. The -T matters — without it exec allocates a TTY and corrupts the redirected dump. Restore with: docker compose -f docker-compose.prod.yml exec -T db psql -U postgres simstudio < backup.sql. The database data is persisted in a Docker volume named postgres_data."},
209
-
{ question: "Can I customize the PostgreSQL credentials?", answer: "Yes. The docker-compose.prod.yml uses environment variable defaults: POSTGRES_USER (default: postgres), POSTGRES_PASSWORD (default: postgres), POSTGRES_DB (default: simstudio), and POSTGRES_PORT (default: 5432). Set these in your .env file to override them." },
211
+
{ question: "Can I customize the PostgreSQL credentials?", answer: "Yes. Set POSTGRES_USER (default: postgres) and POSTGRES_DB (default: simstudio) in .env before the first start. POSTGRES_PASSWORD has no default — the compose file will not start without it. Postgres applies all three only when it creates the database volume, so changing them later does not change the existing database; to rotate the password, see Postgres on Compose in the security guide." },
Copy file name to clipboardExpand all lines: apps/docs/content/docs/platform/self-hosting/security.mdx
+18-8Lines changed: 18 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -271,19 +271,29 @@ networkPolicy:
271
271
272
272
The service bundles ~2.2 GB of spaCy models, so first start takes around three minutes and it needs at least 4 GB of memory.
273
273
274
-
## The shipped Compose file publishes Postgres
274
+
## Postgres on Compose
275
275
276
-
<Callout type="error">
277
-
`docker-compose.prod.yml` maps the database to the host: `${POSTGRES_PORT:-5432}:5432`, with `POSTGRES_USER` and `POSTGRES_PASSWORD` both defaulting to `postgres`. A plain `docker compose up -d` against that file, on a machine with a public interface, therefore exposes an open Postgres on 5432 with credentials anyone can guess. The local and Ollama stacks map the database the same way, so apply the fix to whichever file started your install.
276
+
The Compose files do not publish the `db` service to the host: `simstudio`, `realtime`, and `migrations` reach it over the Compose network as `db:5432`, and nothing outside the stack can. `docker-compose.prod.yml` also refuses to start without `POSTGRES_PASSWORD`, the same way it refuses to start without `BETTER_AUTH_SECRET`.
278
277
279
-
The [Docker guide](/platform/self-hosting/docker#1-configure-environment) tells you to generate `POSTGRES_PASSWORD` before the first start — do that, and additionally close the port:
278
+
<Callout type="error">
279
+
Installs created from an earlier Compose file published the database on every interface of the host (`5432:5432`), and a `POSTGRES_PASSWORD` left unset fell back to `postgres`. A Docker `ports:` mapping writes its own iptables rules, so a host firewall that looks like it blocks 5432 usually does not. Update the Compose file, then check what your database was created with:
280
280
281
-
- **Do not need host access.** Delete the `ports:` block from the `db` service. Every other service reaches it over the Compose network by name.
282
-
- **Need host access.** Bind it to loopback only — `127.0.0.1:${POSTGRES_PORT:-5432}:5432` — and reach it over an SSH tunnel.
281
+
- **You set `POSTGRES_PASSWORD` before the first start.** Nothing else to do — updating the file closes the port.
282
+
- **You never set it.** Postgres applies `POSTGRES_PASSWORD` only when it creates the data volume, so the database still uses `postgres`. Set `POSTGRES_PASSWORD=postgres` in `.env` so the stack starts, then rotate it: run `docker compose -f docker-compose.prod.yml exec db psql -U postgres -c "ALTER ROLE CURRENT_USER PASSWORD '<new password>'"` (with your `POSTGRES_USER` in place of `postgres` if you set one), set `POSTGRES_PASSWORD` to the same value, and run `docker compose -f docker-compose.prod.yml up -d`. Setting a new value in `.env` alone does not change the database's password and locks the app out.
283
283
284
-
A Docker `ports:` mapping writes its own iptables rules, so a host firewall that looks like it blocks 5432 usually does not.
284
+
`npx sim-setup start`and `npx sim-setup update` write the right value for you and print the rotation steps.
285
285
</Callout>
286
286
287
+
To reach the database from the host — `psql`, a desktop client, a backup job — use `docker compose -f docker-compose.prod.yml exec db psql -U postgres simstudio`, or pass a second Compose file that publishes it on loopback only and connect over an SSH tunnel:
- Object storage buckets private, with CORS limited to your Sim origin
300
-
- Database reachable only from the deployment — on Compose, the `db` service's host `ports:` mapping removed or bound to `127.0.0.1` — with a generated `POSTGRES_PASSWORD`
310
+
- Database reachable only from the deployment — on Compose, no host `ports:` mapping on the `db` service, or one bound to `127.0.0.1` — with a generated `POSTGRES_PASSWORD`
301
311
- TLS enforced (`sslMode: require`) on an externally managed database, or on the bundled one once you have configured it for TLS — the shipped Compose database does not enable it
Copy file name to clipboardExpand all lines: apps/docs/content/docs/platform/self-hosting/upgrades.mdx
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -193,11 +193,13 @@ install uses:
193
193
194
194
| Install | What it runs |
195
195
|---|---|
196
-
|`docker-compose.prod.yml`| Refreshes its managed copy of the Compose file, then `docker compose pull` — the versions configured by `SIM_VERSION`, or `latest` when unset |
196
+
|`docker-compose.prod.yml`| Refreshes its managed copy of the Compose file, writes `POSTGRES_PASSWORD` to `.env` if it is missing, then `docker compose pull` — the versions configured by `SIM_VERSION`, or `latest` when unset |
197
197
|`docker-compose.local.yml`|`docker compose build --pull` — rebuilds from source against refreshed base images, no pull of published images |
198
198
199
199
Inspect the result with `npx sim-setup logs`, which targets whichever Compose file the install uses.
200
200
201
+
`docker-compose.prod.yml` requires `POSTGRES_PASSWORD`. If you manage the file yourself and Compose stops with `required variable POSTGRES_PASSWORD is missing a value`, your database was created with the password `postgres` — set exactly that in `.env`, not a new value, then see [Postgres on Compose](/platform/self-hosting/security#postgres-on-compose) to rotate it.
202
+
201
203
The CLI detects only those two files. An install started from `docker-compose.ollama.yml` is invisible to it: `update`, `logs`, and `status` report no install, or — in a source checkout that also carries per-application env files — report that checkout's dev install instead. Upgrade that stack directly:
- 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
- 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
- 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
113
113
depends_on:
114
114
db:
115
115
condition: service_healthy
@@ -151,14 +151,21 @@ services:
151
151
simstudio:
152
152
condition: service_healthy
153
153
154
+
# Not published to the host: app, realtime and migrations reach it over the
155
+
# Compose network as db:5432. A host `ports:` mapping binds every interface and
156
+
# Docker's own iptables rules bypass host firewalls, so it would put the
157
+
# database on the network. For host access (psql, a desktop client), use
158
+
# `docker compose exec db psql -U postgres simstudio`, or pass a second -f
159
+
# file that publishes '127.0.0.1:5432:5432' (-f disables override files).
# Postgres applies this only when it creates the data volume; later
166
+
# changes are ignored. An install created before this was required got
167
+
# the password `postgres` — set exactly that to keep it working.
168
+
- 'POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}'
0 commit comments