Skip to content

[Telemetry] Prevent crashes in shell-less containers - #32226

Merged
bmehta001 merged 3 commits into
mainfrom
bhamehta/fix-chiseled-telemetry-init
Aug 24, 2026
Merged

[Telemetry] Prevent crashes in shell-less containers#32226
bmehta001 merged 3 commits into
mainfrom
bhamehta/fix-chiseled-telemetry-init

Conversation

@bmehta001

@bmehta001 bmehta001 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #32173.

ONNX Runtime 1.29.0 crashes during OrtEnv initialization in chiseled/distroless Linux images when POSIX telemetry is enabled. These images omit both /etc/machine-id and /bin/sh, so cpp_client_telemetry reaches its device-ID fallback and popen() fails.

The SDK currently constructs std::shared_ptr<FILE>(nullptr, pclose). A shared_ptr created with a custom deleter invokes that deleter even for a null pointer, so teardown calls pclose(nullptr) and segfaults in _IO_new_fclose.

Own the FILE* with a lambda-deleter unique_ptr. Exec() has sole ownership of the pipe, and unique_ptr naturally skips its deleter when popen() returns null, allowing the empty-result fallback to work as intended when no shell is available.

Validation

  • Reproduced the issue with the released Microsoft.ML.OnnxRuntime 1.29.0 package in the filesystem from mcr.microsoft.com/dotnet/aspnet:10.0-noble-chiseled under its default UID.
    • Telemetry enabled: deterministic SIGSEGV at OrtEnv.Instance().
    • ORT_DISABLE_TELEMETRY=1: initialization succeeds.
  • Captured the failing native backtrace: pclose(nullptr) enters _IO_new_fclose(fp=0x0) immediately after the failed popen() fallback.
  • Built the telemetry-enabled Linux shared library with this unique-ownership implementation and warnings treated as errors.
  • Replaced the native library in the same managed repro and chiseled filesystem.
    • Telemetry enabled: initialization succeeds.
    • ORT_DISABLE_TELEMETRY=1: initialization succeeds.

Upstream

The SDK fix is proposed in microsoft/cpp_client_telemetry#1523. ORT should retain this compatibility patch until it pins an SDK release containing that change.

Exec() is used only by the non-Apple legacy device-ID fallback when /etc/machine-id is unavailable. ORT replaces the SDK-generated ID with its own persistent hashed device ID after LogManager initialization, but the SDK probe currently runs before that override. A future SDK option to suppress automatic device-ID discovery would let ORT avoid this unnecessary shell probe entirely; the null-safe cleanup is still required for existing SDK consumers and versions.

cpp_client_telemetry constructs a shared_ptr with pclose as its deleter even when popen fails. Chiseled images have no /bin/sh, so telemetry initialization calls pclose(nullptr) and terminates the process.

Use unique_ptr ownership so a null pipe is not passed to pclose and telemetry can degrade gracefully.

Files changed:
- cmake/patches/cpp_client_telemetry/cpp_client_telemetry.patch: make the POSIX pipe owner null-safe.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d7d2f27a-7339-4585-ad02-9f89ce20ef40
Copilot AI balanced review requested due to automatic review settings August 24, 2026 01:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents telemetry initialization crashes in shell-less Linux containers by making failed popen() ownership null-safe.

Changes:

  • Replaces shared_ptr<FILE> with unique_ptr<FILE> and a pclose deleter.
  • Preserves the existing empty-result fallback when popen() fails.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

bmehta001 and others added 2 commits August 23, 2026 21:46
Keep the existing shared ownership shape while making its custom deleter null-safe. This matches cpp_client_telemetry#1523 and avoids the function-pointer warning that occurred under the Linux warnings-as-errors build.

Files changed:
- cmake/patches/cpp_client_telemetry/cpp_client_telemetry.patch: guard pclose in the shared_ptr deleter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d7d2f27a-7339-4585-ad02-9f89ce20ef40
The pipe returned by popen has one owner. Matching the upstream unique_ptr fix makes that lifetime explicit and naturally avoids pclose for a null handle.

Files changed:`n- cmake/patches/cpp_client_telemetry/cpp_client_telemetry.patch: use the upstream lambda-deleter unique_ptr.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>`nCopilot-Session: d7d2f27a-7339-4585-ad02-9f89ce20ef40
@bmehta001 bmehta001 self-assigned this Aug 24, 2026
@bmehta001
bmehta001 enabled auto-merge (squash) August 24, 2026 07:37
@bmehta001
bmehta001 merged commit 9aa8fb0 into main Aug 24, 2026
88 of 89 checks passed
@bmehta001
bmehta001 deleted the bhamehta/fix-chiseled-telemetry-init branch August 24, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants