Skip to content

perf(media-server): size direct conversions from available CPUs and enable the shutdown drain in compose - #3001

Open
07rjain wants to merge 2 commits into
CapSoftware:mainfrom
07rjain:split/media-drain-capacity
Open

07rjain wants to merge 2 commits into
CapSoftware:mainfrom
07rjain:split/media-drain-capacity

Conversation

@07rjain

@07rjain 07rjain commented Oct 23, 2026 •

Copy link
Copy Markdown

Split out of #2580 (shutdown draining and capacity).

  • Direct /video/convert concurrency defaulted to 1 on any host. It now defaults to half the usable CPUs (respecting the container CPU limit), between 1 and 4. MEDIA_SERVER_MAX_CONCURRENT_DIRECT_CONVERSIONS still overrides it.
  • Main now has its own drain (lifecycle.ts), so this PR no longer adds a second one. It only turns main's drain on in the compose files: MEDIA_SERVER_SHUTDOWN_GRACE_MS=25000 with stop_grace_period: 30s.
  • audio-levels test occupies every direct slot instead of assuming one.
  • Docs: "Media Server Tuning".

RetriggerView in GreptileConfidence Score: 4/5

Adjust the shutdown timing before merging so Docker does not cut off cancellation notifications and cleanup.

Findings

  1. P1 Docker cuts cleanup short ▶
  2. P2 CPU limits can overcount ▶
  3. P2 Compose ignores these tuning settings ▶
Fix with agent prompt
### Issue 1
docker-compose.yml:52-55
If a job outlasts the 25-second drain and its webhook is slow or unavailable, only five seconds remain before Docker kills the container. Cleanup starts after the drain, and `abortAllJobs()` can spend 16.5 seconds sending cancellation webhooks before the server reaches media and temporary-file cleanup. Docker can cut off both notifications and cleanup.

Increase `stop_grace_period` to cover both stages, or bound cleanup within the remaining time. Apply the same fix to `docker-compose.template.yml` and `docker-compose.coolify.yml`.

### Issue 2
apps/media-server/src/lib/video-capacity.ts:16
If a container has a finite CPU quota larger than its assigned CPU set, `getContainerCpuLimit() || os.availableParallelism()` ignores the smaller available count. An eight-CPU quota with a two-CPU set selects four conversions instead of one, making conversions compete for the same CPUs. Use the smaller positive quota and `os.availableParallelism()` value before halving it, and test that combination.

### Issue 3
apps/web/content/docs/self-hosting.mdx:111-116
This page tells Compose users to put custom settings in `.env`, but none of the three Compose services forwards either tuning variable or loads that file with `env_file`. Adding the shown values to `.env` therefore leaves conversion limits unchanged. Show how to add them to the media-server service's `environment` mapping and recreate the container.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Changes direct conversion defaults from one slot to half the detected CPU capacity, bounded between one and four.

  • Direct conversions and audio-level work use a CPU-sized slot limit.
  • Compose deployments give active media-server work time to drain on stop.

Reviews (1) · Last reviewed commit: "perf(media-server): size direct conversi..." · Reviewed by Greptile

… the shutdown drain in compose

Direct /video/convert work defaulted to one conversion at a time on any machine. The default is now half the usable CPUs (container limit aware), between 1 and 4, and MEDIA_SERVER_MAX_CONCURRENT_DIRECT_CONVERSIONS still overrides it. The compose files set MEDIA_SERVER_SHUTDOWN_GRACE_MS to 25s under a 30s stop_grace_period so main's drain lets active work finish on docker compose stop.
@vercel

vercel Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@07rjain is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

const MAX_DEFAULT_DIRECT_VIDEO_PROCESSES = 4;

export function getDefaultMaxConcurrentDirectVideoProcesses(
cpuCapacity: number = getContainerCpuLimit() || os.availableParallelism(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 CPU limits can overcount

If a container has a finite CPU quota larger than its assigned CPU set, getContainerCpuLimit() || os.availableParallelism() ignores the smaller available count. An eight-CPU quota with a two-CPU set selects four conversions instead of one, making conversions compete for the same CPUs. Use the smaller positive quota and os.availableParallelism() value before halving it, and test that combination.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/media-server/src/lib/video-capacity.ts
Line: 16

Comment:
**CPU limits can overcount**

If a container has a finite CPU quota larger than its assigned CPU set, `getContainerCpuLimit() || os.availableParallelism()` ignores the smaller available count. An eight-CPU quota with a two-CPU set selects four conversions instead of one, making conversions compete for the same CPUs. Use the smaller positive quota and `os.availableParallelism()` value before halving it, and test that combination.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment thread docker-compose.yml Outdated
Comment on lines +52 to +55
stop_grace_period: 30s
environment:
# Finish active work for up to 25s on stop (below stop_grace_period).
MEDIA_SERVER_SHUTDOWN_GRACE_MS: "25000"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Docker cuts cleanup short

If a job outlasts the 25-second drain and its webhook is slow or unavailable, only five seconds remain before Docker kills the container. Cleanup starts after the drain, and abortAllJobs() can spend 16.5 seconds sending cancellation webhooks before the server reaches media and temporary-file cleanup. Docker can cut off both notifications and cleanup.

Increase stop_grace_period to cover both stages, or bound cleanup within the remaining time. Apply the same fix to docker-compose.template.yml and docker-compose.coolify.yml.

Prompt To Fix With AI
This is a comment left during a code review.
Path: docker-compose.yml
Line: 52-55

Comment:
**Docker cuts cleanup short**

If a job outlasts the 25-second drain and its webhook is slow or unavailable, only five seconds remain before Docker kills the container. Cleanup starts after the drain, and `abortAllJobs()` can spend 16.5 seconds sending cancellation webhooks before the server reaches media and temporary-file cleanup. Docker can cut off both notifications and cleanup.

Increase `stop_grace_period` to cover both stages, or bound cleanup within the remaining time. Apply the same fix to `docker-compose.template.yml` and `docker-compose.coolify.yml`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +111 to +116
The media server converts uploads in parallel based on the CPUs it can use: half the available CPUs, at least 1 and at most 4. It still turns work away when CPU or memory pressure is high. Override this with:

```bash
MEDIA_SERVER_MAX_CONCURRENT_DIRECT_CONVERSIONS=2 # direct /video/convert requests
MEDIA_SERVER_MAX_CONCURRENT_VIDEO_PROCESSES=4 # all video jobs
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Compose ignores these tuning settings

This page tells Compose users to put custom settings in .env, but none of the three Compose services forwards either tuning variable or loads that file with env_file. Adding the shown values to .env therefore leaves conversion limits unchanged. Show how to add them to the media-server service's environment mapping and recreate the container.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/content/docs/self-hosting.mdx
Line: 111-116

Comment:
**Compose ignores these tuning settings**

This page tells Compose users to put custom settings in `.env`, but none of the three Compose services forwards either tuning variable or loads that file with `env_file`. Adding the shown values to `.env` therefore leaves conversion limits unchanged. Show how to add them to the media-server service's `environment` mapping and recreate the container.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…cel webhooks and cleanup

A 25s drain plus webhooks for jobs aborted at shutdown (an in-flight attempt up to 5s, then 3 x 5s with 1.5s backoff) needs 46.5s before cleanup, more than the 30s stop_grace_period, so Docker could SIGKILL mid-cleanup. Raise stop_grace_period to 60s in all three compose files and document the budget.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant