Summary
A learned seccomp profile can be marked completed/complete and auto-enforced onto its workload while still missing syscalls the workload genuinely needs — and once enforced, that gap can never self-heal through the normal learn → enforce pipeline, because enforcement denial happens before the same eBPF gadget's own observation hook fires. This causes silent, permanent, hard-to-diagnose functional breakage in profiled workloads.
Live reproduction
Deployed a stock docker.io/hisu/ping-app (Apache 2.4.54 + PHP 7.4.33) workload with a short custom kubescape.io/max-sniffing-time: 15m learning window. Kubescape learned and auto-enforced a Localhost seccomp profile (SCMP_ACT_ERRNO default, ~85 allowed syscalls) back onto the same deployment.
With that profile enforced, every real HTTP request to the app failed:
(1)Operation not permitted: [client ...] AH00036: access to /index.php failed (filesystem path '/var')
127.0.0.1 - - [...] "GET / HTTP/1.1" 200 0 "-" "curl/8.14.1"
Apache returns 200 but with an empty body — the failure is silent from the client's perspective. Reproduced identically on two independent clusters (different node-agent deployment modes: local CRD storage and backend-storage mode), ruling out a storage-mode-specific cause.
Root cause confirmed via kernel audit log (profile temporarily switched to SCMP_ACT_LOG for diagnosis — logs but doesn't block):
audit: type=1326 ... comm="apache2" ... syscall=6 ... (lstat)
audit: type=1326 ... comm="apache2" ... syscall=7 ... (poll)
audit: type=1326 ... comm="apache2" ... syscall=20 ... (writev)
lstat, poll, and writev are genuinely used by Apache/PHP during normal request handling but were never captured during the 15-minute learning window (most likely because whatever traffic hit the pod during learning — e.g. only TCP-level liveness probes — never exercised the real page-serving code path). The generated ApplicationProfile/SeccompProfile was nonetheless marked kubescape.io/completion: complete.
Why this can never self-heal
The advise_seccomp gadget observes syscalls via a raw_tracepoint/sys_enter hook. Per standard kernel syscall-entry ordering, __secure_computing() (where seccomp's SCMP_ACT_ERRNO denial happens) runs before trace_sys_enter() fires. A syscall denied by the enforced profile therefore never reaches the same observation mechanism that's supposed to expand/correct the profile. Once a profile with any gap gets enforced, the missing syscalls become permanently invisible to the exact pipeline that could otherwise discover and fix them — verified live: the container's cumulative captured syscall count (resource_size) continued growing normally to completion under full enforcement (202 → 322 → 325 over ~17 minutes), confirming only the specifically-denied syscalls are affected, not general observation.
Impact
completion: complete / status: completed asserts a guarantee the learning-window heuristic can't actually back up — it means the time window closed, not that every code path was exercised.- Once such a profile is enforced, the workload can suffer silent, intermittent, hard-to-diagnose failures (in our case: Apache returning empty
200 responses) with no way for the system itself to detect or repair the gap.
- Shortening the learning window (e.g. via
kubescape.io/max-sniffing-time) makes this substantially more likely to bite, but the underlying risk exists for any learned-then-enforced profile whose learning traffic didn't cover every real code path.
Suggested directions (not prescriptive)
- Don't treat "learning window closed" as equivalent to "behaviorally complete" — consider a distinct signal/gate before auto-enforcing a profile (e.g. traffic-coverage heuristics, canary/audit period before switching from
SCMP_ACT_LOG-style non-blocking mode to SCMP_ACT_ERRNO).
- Consider detecting EPERM-driven failures on an enforced workload and automatically reopening/widening the profile rather than requiring manual intervention.
- At minimum, surface this limitation prominently wherever
completion: complete is shown, so it isn't read as a functional-completeness guarantee.
Environment
Summary
A learned seccomp profile can be marked
completed/completeand auto-enforced onto its workload while still missing syscalls the workload genuinely needs — and once enforced, that gap can never self-heal through the normal learn → enforce pipeline, because enforcement denial happens before the same eBPF gadget's own observation hook fires. This causes silent, permanent, hard-to-diagnose functional breakage in profiled workloads.Live reproduction
Deployed a stock
docker.io/hisu/ping-app(Apache 2.4.54 + PHP 7.4.33) workload with a short customkubescape.io/max-sniffing-time: 15mlearning window. Kubescape learned and auto-enforced aLocalhostseccomp profile (SCMP_ACT_ERRNOdefault, ~85 allowed syscalls) back onto the same deployment.With that profile enforced, every real HTTP request to the app failed:
Apache returns
200but with an empty body — the failure is silent from the client's perspective. Reproduced identically on two independent clusters (different node-agent deployment modes: local CRD storage and backend-storage mode), ruling out a storage-mode-specific cause.Root cause confirmed via kernel audit log (profile temporarily switched to
SCMP_ACT_LOGfor diagnosis — logs but doesn't block):lstat,poll, andwritevare genuinely used by Apache/PHP during normal request handling but were never captured during the 15-minute learning window (most likely because whatever traffic hit the pod during learning — e.g. only TCP-level liveness probes — never exercised the real page-serving code path). The generatedApplicationProfile/SeccompProfilewas nonetheless markedkubescape.io/completion: complete.Why this can never self-heal
The
advise_seccompgadget observes syscalls via araw_tracepoint/sys_enterhook. Per standard kernel syscall-entry ordering,__secure_computing()(where seccomp'sSCMP_ACT_ERRNOdenial happens) runs beforetrace_sys_enter()fires. A syscall denied by the enforced profile therefore never reaches the same observation mechanism that's supposed to expand/correct the profile. Once a profile with any gap gets enforced, the missing syscalls become permanently invisible to the exact pipeline that could otherwise discover and fix them — verified live: the container's cumulative captured syscall count (resource_size) continued growing normally to completion under full enforcement (202 → 322 → 325 over ~17 minutes), confirming only the specifically-denied syscalls are affected, not general observation.Impact
completion: complete/status: completedasserts a guarantee the learning-window heuristic can't actually back up — it means the time window closed, not that every code path was exercised.200responses) with no way for the system itself to detect or repair the gap.kubescape.io/max-sniffing-time) makes this substantially more likely to bite, but the underlying risk exists for any learned-then-enforced profile whose learning traffic didn't cover every real code path.Suggested directions (not prescriptive)
SCMP_ACT_LOG-style non-blocking mode toSCMP_ACT_ERRNO).completion: completeis shown, so it isn't read as a functional-completeness guarantee.Environment
kubescape/node-agentv0.3.179 (vendored viaarmosec/private-node-agent)advise_seccompgadget (github.com/matthyx/inspektor-gadgetfork)"unknown"syscall entry — not the cause of this issue, but found during the same investigation)