You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ExecSandbox processes keep running after execution_timeout and after the client disconnects #5042
I use OpenShell v0.1.5 as the sandbox runtime of a self-hosted agent platform I am building (Fort): one gateway as a systemd service on Ubuntu 24.04, Docker driver, a supervisor middleware and gateway interceptors on Unix sockets. I hit this while testing that setup on two VMs.
Problem Statement
A process started with ExecSandbox keeps running after its execution_timeout passes and after the exec client has disconnected. With ExecSandbox{command: ["sh","-c","sleep 600"], execution_timeout: 5s}, the client stream ends after 5 s, but sleep 600 is still running in the workload afterwards (seen 8 s after the timeout).
Impact / Why This Matters
A caller cannot rely on execution_timeout or on cancelling the stream to stop what it started. Orphaned processes keep using the sandbox's resources: in one run a hung exec'd client held about 100 TCP connections, the sandbox's network relay streams (96 per sandbox) stayed busy, and new connections from the sandbox failed for more than 10 minutes. (We saw that long outage once and could not reproduce it once the client process was able to exit.) Today we kill what we start ourselves.
Acceptance Criteria
When execution_timeout expires, the exec'd process tree is terminated.
When the exec client cancels or disconnects, the process tree is terminated, or the API documents that exec'd processes are detached.
Reproduction Steps
Create any sandbox and wait for Ready.
ExecSandbox with command: ["sh","-c","sleep 600"] and execution_timeout: 5s; let the client stream end.
List the workload's processes from the host, e.g. docker exec -u 0 <workload container> ps -eo pid,etimes,args: sleep 600 is still running.
Environment
OpenShell: v0.1.5 (bdb956f), gateway release tarball, run as a systemd service (non-root openshell user)
OS: Ubuntu 24.04 (arm64 on a Lima VM, kernel 6.8; x86_64 on a GCP VM, kernel 7.0.0-1014-gcp), Docker 29.9.2
Deployment: single gateway, Docker compute driver, supervisor middleware on HTTPS loopback, two gateway interceptors on Unix sockets (extension auth on)
Activity