Skip to content

DOC-404: Document orphaned containers after Docker Compose stop - #894

Open
quetzalliwrites wants to merge 2 commits into
mainfrom
doc-404-doc-faq-addition-of-why-are-some-containers-left-behind
Open

DOC-404: Document orphaned containers after Docker Compose stop#894
quetzalliwrites wants to merge 2 commits into
mainfrom
doc-404-doc-faq-addition-of-why-are-some-containers-left-behind

Conversation

@quetzalliwrites

Copy link
Copy Markdown
Collaborator

Summary

  • Add an FAQ entry explaining why some containers (including k3d containers backing an EKS cluster) can be left behind after stopping LocalStack with Docker Compose, and how to raise stop_grace_period to avoid it.
  • Add a cross-referencing note in the EKS docs, where k3d-backed cluster creation is introduced, pointing to the new FAQ entry.

Linear ticket

https://linear.app/localstack/issue/DOC-404/doc-faq-addition-of-why-are-some-containers-left-behind-after-i-stop

Test plan

  • npx astro build completes successfully (401 pages built)
  • Link validator confirms all internal links, including the new cross-link anchor between eks.mdx and faq.mdx, are valid

Add an FAQ entry explaining that Docker Compose's default 10s
SIGTERM-to-SIGKILL grace period can be too short for LocalStack to
finish tearing down auxiliary containers (including k3d containers
backing an EKS cluster), leaving them behind as orphans, and how to
raise stop_grace_period to avoid it. The CLI is not affected since it
waits for shutdown to finish.

Cross-link the same behavior from the EKS docs where k3d-backed
cluster creation is introduced.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 25, 2026

Copy link
Copy Markdown

Deploying localstack-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: ca6574b
Status: ✅  Deploy successful!
Preview URL: https://7d4e2d3c.localstack-docs.pages.dev
Branch Preview URL: https://doc-404-doc-faq-addition-of.localstack-docs.pages.dev

View logs

@quetzalliwrites

Copy link
Copy Markdown
Collaborator Author

@nik-localstack @pinzon heads up, looks like this PR is still pending eng review

@nik-localstack nik-localstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together, the entry describes a real problem, and it came up in an internal discussion recently too.

Some things from that are worth folding in:

  • There's a second setting involved, and it isn't in our docs. Alongside docker-compose stop_grace_period, LocalStack applies its own SHUTDOWN_TIMEOUT (5 seconds by default) to its cleanup, and that one runs out first. Raising only stop_grace_period doesn't give the cleanup more time and it has a side effect worth warning about: docker compose stop then takes the full three minutes before the container exits, whereas raising both settings gets a clean shutdown in well under a minute. The inline suggestions reflect that.SHUTDOWN_TIMEOUT isn't listed in the configuration reference today, so this entry would be the first place we mention it. We should probably expose it in the configuration page.
  • It isn't only EKS. The same shutdown budget is shared by every service that manages containers. ECS is the closest example, where a task's own stop timeout can consume the entire budget by itself.

I think we need more eyes before merging: shutdown behavior is being actively discussed internally right now, and how these settings interact may change. I don't think that blocks the changes but we should consider what should be advertised as a permanent solution and what not.

cc @purcell who was involved in this discussion about shutdown

btw I have added some inline suggestions but it may not be what we want to document in the end, so feel free to disregard them.

Comment thread src/content/docs/aws/getting-started/faq.mdx Outdated
Comment thread src/content/docs/aws/getting-started/faq.mdx Outdated
Comment thread src/content/docs/aws/getting-started/faq.mdx Outdated
Comment thread src/content/docs/aws/getting-started/faq.mdx
Comment thread src/content/docs/aws/getting-started/faq.mdx
Comment thread src/content/docs/aws/getting-started/faq.mdx Outdated
@quetzalliwrites

Copy link
Copy Markdown
Collaborator Author

Hey @purcell we need confirmation from ENG about whether this docs ticket request is still needed or if it's been cancelled, per @nik-localstack earlier Q:
#894 (review)

If you're not the right person to confirm, can you let us know who is? thx !

- Generalize the FAQ entry away from EKS/k3d specifics (Lambda, ECS,
  RDS, and EKS all start auxiliary containers); keep the k3d-specific
  framing in the EKS doc's cross-referencing note instead.
- Explain both timing windows that can cause orphaned containers:
  Docker Compose's stop_grace_period and LocalStack's own
  SHUTDOWN_TIMEOUT, and note both need to be raised together.
- Clarify the CLI callout refers to lstk specifically, not the
  legacy localstack CLI.
- Fix the cleanup instruction to actually say how to find the
  containers (docker ps) before removing them.
@quetzalliwrites

quetzalliwrites commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @nik-localstack, while we wait confirmation on whether this ticket is cancelled, I still went ahead and pushed ca6574be applying all six suggestions:

  • Generalized the FAQ entry away from EKS/k3d specifics (Lambda, ECS, RDS, and EKS all start auxiliary containers); kept the k3d-specific framing in the EKS doc's cross-referencing note instead.
  • Explained both timing windows that can cause orphaned containers — Docker Compose's stop_grace_period and LocalStack's own SHUTDOWN_TIMEOUT — and noted they need to be raised together.
  • Clarified the CLI callout refers to lstk specifically, not the legacy localstack CLI.
  • Fixed the cleanup instruction to say how to find the containers (docker ps) before removing them.

One thing worth double-checking before merge (independent of the cancellation question): I couldn't find SHUTDOWN_TIMEOUT documented in src/content/docs/aws/customization/configuration-options.md to independently verify the name/default — applied your suggestion as given, but flagging in case it's worth a cross-check or a follow-up to add it to the config reference too.

@purcell purcell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all accurate and reasonable, so approving this in principle, with the few proposed edits below to fix some formatting and idioms.


Raise them together. If you raise only `stop_grace_period`, LocalStack still gives up on its cleanup after 5 seconds and the container then stays up until the grace period runs out, so every docker compose stop waits the full three minutes.

If a previous run already left containers behind, run docker ps and remove the ones LocalStack started before starting it again.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If a previous run already left containers behind, run docker ps and remove the ones LocalStack started before starting it again.
If a previous run already left containers behind, run `docker ps` and remove the ones LocalStack started before starting it again.


By default, LocalStack gets only a few seconds. Docker Compose sends `SIGTERM` to the container and follows up with `SIGKILL` once `stop_grace_period` expires (10 seconds by default), and LocalStack stops waiting for its own cleanup after `SHUTDOWN_TIMEOUT` (5 seconds by default). Whichever window runs out first, LocalStack is stopped mid-cleanup and the containers survive as orphans. Cleanup that takes longer than a few seconds is the most likely to be cut short.

To give LocalStack enough time to shut down cleanly, raise `stop_grace_period` on the LocalStack service in your `docker-compose.yml` (for example, `3m`):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To give LocalStack enough time to shut down cleanly, raise `stop_grace_period` on the LocalStack service in your `docker-compose.yml` (for example, `3m`):
To give LocalStack enough time to shut down cleanly, increase `stop_grace_period` on the LocalStack service in your `docker-compose.yml` (for example, `3m`):

# ...
```

Raise them together. If you raise only `stop_grace_period`, LocalStack still gives up on its cleanup after 5 seconds and the container then stays up until the grace period runs out, so every docker compose stop waits the full three minutes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Raise them together. If you raise only `stop_grace_period`, LocalStack still gives up on its cleanup after 5 seconds and the container then stays up until the grace period runs out, so every docker compose stop waits the full three minutes.
Increase both settings. If you increase only `stop_grace_period`, LocalStack still gives up on its cleanup after 5 seconds and the container then stays up until the grace period runs out, so every `docker compose stop` waits the full three minutes.


When LocalStack shuts down, it cleans up the auxiliary containers it started for services such as Lambda, ECS, RDS or EKS. If it is stopped before that cleanup finishes, those containers are left running.

By default, LocalStack gets only a few seconds. Docker Compose sends `SIGTERM` to the container and follows up with `SIGKILL` once `stop_grace_period` expires (10 seconds by default), and LocalStack stops waiting for its own cleanup after `SHUTDOWN_TIMEOUT` (5 seconds by default). Whichever window runs out first, LocalStack is stopped mid-cleanup and the containers survive as orphans. Cleanup that takes longer than a few seconds is the most likely to be cut short.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
By default, LocalStack gets only a few seconds. Docker Compose sends `SIGTERM` to the container and follows up with `SIGKILL` once `stop_grace_period` expires (10 seconds by default), and LocalStack stops waiting for its own cleanup after `SHUTDOWN_TIMEOUT` (5 seconds by default). Whichever window runs out first, LocalStack is stopped mid-cleanup and the containers survive as orphans. Cleanup that takes longer than a few seconds is the most likely to be cut short.
By default, LocalStack gets only a few seconds in which to shut down. Docker Compose sends `SIGTERM` to the container and follows up with `SIGKILL` once `stop_grace_period` expires (10 seconds by default), and LocalStack stops waiting for its own cleanup after `SHUTDOWN_TIMEOUT` (5 seconds by default). Whichever window runs out first, LocalStack is stopped mid-cleanup and the containers survive as orphans. Cleanup that takes longer than a few seconds is the most likely to be cut short.

services:
localstack:
image: localstack/localstack-pro:latest
stop_grace_period: 3m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
stop_grace_period: 3m
stop_grace_period: 180s

Also, writing this period as seconds makes it more obviously match the number below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants