What happened?
Summary
The test-node-kubernetes pod is restarting automatically without any manual intervention. The restart is observed even after increasing the liveness probe timeoutSeconds from 1 second to 10 seconds.
Environment
- Namespace:
spr-ops
- Deployment:
test-node-kubernetes
- Component: Selenium Grid Kubernetes Node
Current Status
kubectl get pods -n spr-ops | grep test
test-node-kubernetes-85485bd7fc-82sx6 1/1 Running 4 (118m ago) 15h
test-selenium-hub-84f5f85bfb-7pdtb 1/1 Running 0 9h
The pod has restarted 4 times within the last 15 hours.
Findings
1. Container Exit Code
Last State:
Terminated
Reason: Error
Exit Code: 143
Exit Code 143 indicates the container received a SIGTERM signal (128 + 15), which means Kubernetes gracefully terminated the container instead of the application crashing on its own.
2. Container Logs
The logs consistently show a graceful shutdown sequence.
WARN received SIGTERM indicating exit request
INFO waiting for selenium-grid-kubernetes to die
WARN stopped: selenium-grid-kubernetes (terminated by SIGTERM)
Shutdown complete
This indicates:
- The application is not crashing unexpectedly.
- Kubernetes is explicitly sending a SIGTERM signal.
- The application performs a graceful shutdown after receiving the termination signal.
3. Liveness Probe Configuration
Current configuration:
livenessProbe:
httpGet:
path: /readyz
port: 5555
timeoutSeconds: 1
periodSeconds: 10
failureThreshold: 3This configuration means Kubernetes:
- Performs a health check every 10 seconds.
- Waits only 1 second for the
/readyz endpoint to respond.
- Restarts the container after 3 consecutive failures (approximately 30 seconds).
This behavior is consistent with the observed SIGTERM-based shutdown.
Change Attempted
To verify whether the aggressive timeout was causing false liveness failures, the following change was made:
However, the pod continued to restart, indicating that increasing the timeout alone did not resolve the issue.
Expected Behavior
The test-node-kubernetes pod should remain healthy and continue running unless:
- the application becomes unhealthy,
- the node is drained,
- a deployment rollout occurs, or
- another Kubernetes lifecycle event intentionally restarts the pod.
Actual Behavior
The pod receives a SIGTERM from Kubernetes and restarts automatically, despite increasing the liveness probe timeout.
Impact
- Unexpected Selenium node restarts.
- Existing browser sessions may be interrupted.
- Potential instability in Selenium Grid during test execution.
Investigation Completed
- Verified restart count using
kubectl get pods.
- Verified container exit code (143 / SIGTERM).
- Reviewed container logs confirming graceful shutdown.
- Reviewed liveness probe configuration.
- Increased
timeoutSeconds from 1 to 10.
- Restart issue still persists.
Next Steps / Assistance Required
Further investigation is required to determine what is triggering the SIGTERM.
Command used to start Selenium Grid with Docker (or Kubernetes)
kubectl deploy test-selenium-hub -n spr-ops
Relevant log output
WARN received SIGTERM indicating exit request
INFO waiting for selenium-grid-kubernetes to die
WARN stopped: selenium-grid-kubernetes (terminated by SIGTERM)
Shutdown complete
Operating System
mac
Docker Selenium version (image tag)
selenium/node-kubernetes:4.46.0-20260707
Selenium Grid chart version (chart version)
No response
What happened?
Summary
The
test-node-kubernetespod is restarting automatically without any manual intervention. The restart is observed even after increasing the liveness probetimeoutSecondsfrom 1 second to 10 seconds.Environment
spr-opstest-node-kubernetesCurrent Status
The pod has restarted 4 times within the last 15 hours.
Findings
1. Container Exit Code
Exit Code 143 indicates the container received a SIGTERM signal (
128 + 15), which means Kubernetes gracefully terminated the container instead of the application crashing on its own.2. Container Logs
The logs consistently show a graceful shutdown sequence.
This indicates:
3. Liveness Probe Configuration
Current configuration:
This configuration means Kubernetes:
/readyzendpoint to respond.This behavior is consistent with the observed SIGTERM-based shutdown.
Change Attempted
To verify whether the aggressive timeout was causing false liveness failures, the following change was made:
However, the pod continued to restart, indicating that increasing the timeout alone did not resolve the issue.
Expected Behavior
The
test-node-kubernetespod should remain healthy and continue running unless:Actual Behavior
The pod receives a SIGTERM from Kubernetes and restarts automatically, despite increasing the liveness probe timeout.
Impact
Investigation Completed
kubectl get pods.timeoutSecondsfrom 1 to 10.Next Steps / Assistance Required
Further investigation is required to determine what is triggering the SIGTERM.
Command used to start Selenium Grid with Docker (or Kubernetes)
Relevant log output
Operating System
mac
Docker Selenium version (image tag)
selenium/node-kubernetes:4.46.0-20260707
Selenium Grid chart version (chart version)
No response