Added python exception capturing logic directly on otel python sdk

.changelog/mw-exception-source.added

@@ -0,0 +1 @@
+`opentelemetry-sdk`: (fork-only, not upstream) `Span.record_exception` can optionally attach the source code of every frame in the exception's traceback to the `exception` event, as an `exception.stack_details` JSON array (each entry: `exception.file`, `exception.line`, `exception.function_name`, `exception.function_body`, `exception.start_line`, `exception.end_line`, `exception.is_file_external`), plus `exception.language`. Large functions are windowed to 10 lines above/below the failing line. Opt in with `MW_RECORD_EXCEPTION_SOURCE=true`; cap the serialized size with `MW_RECORD_EXCEPTION_SOURCE_MAX_CHARS` (default 8192). Disabled by default, so behavior is unchanged unless explicitly enabled.

.changelog/mw-vcs-resource-detector.added

@@ -0,0 +1 @@
+`opentelemetry-sdk`: (fork-only, not upstream) add a `vcs` resource detector (`VcsResourceDetector`) that sets `vcs.repository_url` / `vcs.commit_sha` resource attributes from `MW_VCS_REPOSITORY_URL` / `MW_VCS_COMMIT_SHA`, falling back to the local git checkout via the `git` CLI. Consumed by Middleware Ops AI to point generated fixes at the right commit/branch. Opt in by adding `vcs` to `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS`; registered as a standard resource detector entry point, so no changes to `Resource.create` were needed.

docker/autoinstrumentation/Dockerfile

@@ -0,0 +1,76 @@
+# Custom drop-in replacement for
+# ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-python,
+# bundling this fork's opentelemetry-api/sdk (exception source capture + VCS
+# resource detector) instead of the public PyPI releases.
+#
+# Do NOT build this Dockerfile directly against whatever commit is checked
+# out on this branch -- use docker/autoinstrumentation/build.sh instead:
+#   docker/autoinstrumentation/build.sh <registry>/autoinstrumentation-python:<tag>
+# main runs ahead of the contrib release train pinned below (e.g. main may be
+# 1.45.0.dev0 while opentelemetry-instrumentation==0.64b0 hard-pins
+# opentelemetry-semantic-conventions==0.64b0 exactly), which makes the single
+# `pip install` below fail dependency resolution outright. build.sh builds
+# from a worktree at the matching upstream tag with just this fork's small
+# diff applied on top, so the versions here always resolve cleanly. See its
+# comments for the full story, including a worse failure mode this avoids.
+#
+# The output layout under /autoinstrumentation matches what the OpenTelemetry
+# Operator's init container expects (a flat `pip install --target` dump), so
+# this image can be referenced as-is from an Instrumentation CR's
+# spec.python.image field -- no operator changes needed.
+#
+# This image is just a file source copied into the target app pod by the
+# Operator's init container -- nothing in it actually runs, so setting env
+# vars here would have no effect on the instrumented app. To turn the new
+# features on, set these on the *target application* pod instead (e.g. via
+# the Instrumentation CR's spec.env, or the pod spec directly):
+#   MW_RECORD_EXCEPTION_SOURCE=true
+#   OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=vcs
+# The vcs detector falls back to reading the local .git checkout via the
+# `git` CLI, but most production app images have neither `git` nor a `.git`
+# directory -- for those, also set MW_VCS_REPOSITORY_URL / MW_VCS_COMMIT_SHA
+# explicitly (e.g. injected from CI at build/deploy time).
+
+FROM python:3.12-slim AS build
+
+WORKDIR /operator-build
+
+# Install this fork's opentelemetry-api / opentelemetry-sdk /
+# opentelemetry-semantic-conventions together with the pinned contrib
+# instrumentation set, opentelemetry-distro, and opentelemetry-instrumentation
+# in ONE pip invocation.
+#
+# This MUST be a single `pip install` call, not split across two (as earlier
+# versions of this Dockerfile did, in either order). `opentelemetry.*` is a
+# PEP 420 implicit namespace package shared by all of these distributions.
+# `pip install --target DIR` treats the top-level `opentelemetry/` directory
+# as one opaque unit for its "already exists" check -- not per sub-package --
+# so whichever separate `pip install --target workspace ...` invocation runs
+# *second* has its `opentelemetry/*` content silently dropped for every
+# package in that command (pip still prints "Successfully installed" for
+# them; only the shared-namespace files are skipped). That includes
+# opentelemetry/instrumentation/, which contains sitecustomize.py -- the
+# actual bootstrap entrypoint the whole PYTHONPATH auto-instrumentation
+# mechanism depends on -- so a split install produces an image whose init
+# container copies files in successfully but never actually instruments or
+# exports anything. Passing every requirement to a single pip invocation
+# lets pip's resolver merge them all into `opentelemetry/` in one pass, the
+# same way the unmodified upstream Dockerfile's single install step always
+# has. Since the local paths are direct requirements (not transitive), pip
+# uses them to satisfy every other package's opentelemetry-api/sdk/
+# semantic-conventions dependency instead of fetching a PyPI version.
+COPY opentelemetry-api ./src/opentelemetry-api
+COPY opentelemetry-sdk ./src/opentelemetry-sdk
+COPY opentelemetry-semantic-conventions ./src/opentelemetry-semantic-conventions
+COPY docker/autoinstrumentation/requirements.txt ./requirements.txt
+RUN pip install --no-cache-dir --target workspace \
+    ./src/opentelemetry-api \
+    ./src/opentelemetry-sdk \
+    ./src/opentelemetry-semantic-conventions \
+    opentelemetry-distro==0.64b0 \
+    opentelemetry-instrumentation==0.64b0 \
+    -r requirements.txt
+
+FROM busybox AS autoinstrumentation-image
+COPY --from=build /operator-build/workspace /autoinstrumentation
+RUN chmod -R go+r /autoinstrumentation

docker/autoinstrumentation/build.sh

@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# Builds the custom autoinstrumentation-python image from a version-aligned
+# checkout of this fork, not from whatever commit happens to be checked out.
+#
+# Why this script exists instead of a plain `docker build .`: this repo's
+# main branch runs ahead of the contrib release train pinned in
+# requirements.txt (e.g. main is 1.45.0.dev0/0.66b0.dev while
+# opentelemetry-instrumentation==0.64b0 hard-pins
+# opentelemetry-semantic-conventions==0.64b0 exactly). Building directly from
+# main makes pip's resolver fail with ResolutionImpossible -- or, if you work
+# around that by installing api/sdk/semconv as a separate pip invocation
+# layered on top, produces a WORSE failure: `pip install --target` treats the
+# whole `opentelemetry/` namespace directory as one opaque unit for its
+# "already exists, skip" check, so whichever invocation runs second silently
+# drops every *other* package's `opentelemetry/*` files -- including
+# opentelemetry/instrumentation/sitecustomize.py, the actual bootstrap
+# entrypoint the whole PYTHONPATH auto-instrumentation mechanism depends on.
+# That produces an image that copies in fine but instruments and exports
+# nothing, with no error anywhere.
+#
+# The fix: build from a worktree checked out at the upstream tag matching
+# requirements.txt's pins, with just this fork's small diff (the two new
+# _mw_*.py files, the record_exception hook, and the entry-points line)
+# copied on top -- then a single `pip install` in the Dockerfile can resolve
+# everything together in one pass, same as an unmodified upstream build.
+#
+# Usage: docker/autoinstrumentation/build.sh [image-tag]
+# Bump UPSTREAM_TAG below in lockstep whenever requirements.txt's
+# opentelemetry-instrumentation/opentelemetry-distro pin changes.
+
+set -euo pipefail
+
+UPSTREAM_TAG="v1.43.0"
+IMAGE_TAG="${1:-mw-autoinstrumentation-python:local}"
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+WORKTREE_DIR="$(mktemp -d /tmp/mw-otel-autoinstrumentation-build.XXXXXX)"
+trap 'git -C "$REPO_ROOT" worktree remove --force "$WORKTREE_DIR" 2>/dev/null || rm -rf "$WORKTREE_DIR"' EXIT
+
+echo "==> Fetching $UPSTREAM_TAG from upstream open-telemetry/opentelemetry-python"
+git -C "$REPO_ROOT" fetch https://github.com/open-telemetry/opentelemetry-python.git "tag" "$UPSTREAM_TAG" --no-tags
+
+echo "==> Checking out a worktree at $UPSTREAM_TAG"
+git -C "$REPO_ROOT" worktree add --detach "$WORKTREE_DIR" "$UPSTREAM_TAG"
+
+echo "==> Applying this fork's diff on top"
+cp "$REPO_ROOT/opentelemetry-sdk/src/opentelemetry/sdk/trace/_mw_exception_context.py" \
+   "$WORKTREE_DIR/opentelemetry-sdk/src/opentelemetry/sdk/trace/_mw_exception_context.py"
+cp "$REPO_ROOT/opentelemetry-sdk/src/opentelemetry/sdk/resources/_mw_vcs.py" \
+   "$WORKTREE_DIR/opentelemetry-sdk/src/opentelemetry/sdk/resources/_mw_vcs.py"
+
+python3 - "$WORKTREE_DIR/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py" <<'PYEOF'
+import sys
+path = sys.argv[1]
+text = open(path).read()
+
+import_anchor = "from opentelemetry.sdk.trace._tracer_metrics import create_tracer_metrics"
+import_patch = (
+    "from opentelemetry.sdk.trace._mw_exception_context import (  # mw: fork-only exception source capture\n"
+    "    get_exception_source_attributes,\n"
+    ")\n"
+) + import_anchor
+if "_mw_exception_context" not in text:
+    assert import_anchor in text, "import anchor not found -- upstream tag structure changed"
+    text = text.replace(import_anchor, import_patch, 1)
+
+hook_anchor = "            EXCEPTION_ESCAPED: str(escaped),\n        }\n        if attributes:"
+hook_patch = (
+    "            EXCEPTION_ESCAPED: str(escaped),\n"
+    "        }\n"
+    "        # mw: fork-only addition, opt-in via MW_RECORD_EXCEPTION_SOURCE\n"
+    "        _attributes.update(get_exception_source_attributes(exception))\n"
+    "        if attributes:"
+)
+if "get_exception_source_attributes(exception)" not in text:
+    assert hook_anchor in text, "record_exception anchor not found -- upstream tag structure changed"
+    text = text.replace(hook_anchor, hook_patch, 1)
+
+open(path, "w").write(text)
+PYEOF
+
+python3 - "$WORKTREE_DIR/opentelemetry-sdk/pyproject.toml" <<'PYEOF'
+import sys
+path = sys.argv[1]
+text = open(path).read()
+anchor = 'service_instance = "opentelemetry.sdk.resources:ServiceInstanceIdResourceDetector"'
+patch = anchor + (
+    '\n# mw: fork-only, opt-in via OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=vcs\n'
+    'vcs = "opentelemetry.sdk.resources._mw_vcs:VcsResourceDetector"'
+)
+if "_mw_vcs:VcsResourceDetector" not in text:
+    assert anchor in text, "entry-points anchor not found -- upstream tag structure changed"
+    text = text.replace(anchor, patch, 1)
+open(path, "w").write(text)
+PYEOF
+
+mkdir -p "$WORKTREE_DIR/docker/autoinstrumentation"
+cp "$REPO_ROOT/docker/autoinstrumentation/Dockerfile" "$WORKTREE_DIR/docker/autoinstrumentation/Dockerfile"
+cp "$REPO_ROOT/docker/autoinstrumentation/requirements.txt" "$WORKTREE_DIR/docker/autoinstrumentation/requirements.txt"
+
+echo "==> Building $IMAGE_TAG"
+docker buildx build --platform linux/amd64,linux/arm64 -f "$WORKTREE_DIR/docker/autoinstrumentation/Dockerfile" -t "$IMAGE_TAG" "$WORKTREE_DIR"
+
+echo "==> Done: $IMAGE_TAG"

docker/autoinstrumentation/requirements.txt

@@ -0,0 +1,70 @@
+# Verbatim copy of the upstream OpenTelemetry Operator's pinned contrib
+# instrumentation set (autoinstrumentation/python/requirements.txt in
+# open-telemetry/opentelemetry-operator @ main), so the pins are guaranteed
+# to actually resolve on PyPI. Bump these together with upstream when you
+# rebase -- don't hand-edit versions from memory.
+#
+# NOTE: opentelemetry-distro, opentelemetry-instrumentation, opentelemetry-api,
+# opentelemetry-sdk, and opentelemetry-semantic-conventions are intentionally
+# NOT listed here -- they're installed from this repo's local fork checkout in
+# the Dockerfile, after this file, so the fork's wheels take precedence over
+# whatever these instrumentation packages would otherwise pull in transitively.
+
+urllib3 <2.7.1
+
+opentelemetry-exporter-otlp-proto-http==1.43.0
+opentelemetry-exporter-prometheus==0.64b0
+
+opentelemetry-propagator-b3==1.43.0
+opentelemetry-propagator-jaeger==1.43.0
+opentelemetry-propagator-aws-xray==1.0.2
+opentelemetry-propagator-ot-trace==0.64b0
+
+opentelemetry-instrumentation-aio-pika==0.64b0
+opentelemetry-instrumentation-aiohttp-client==0.64b0
+opentelemetry-instrumentation-aiohttp-server==0.64b0
+opentelemetry-instrumentation-aiokafka==0.64b0
+opentelemetry-instrumentation-aiopg==0.64b0
+opentelemetry-instrumentation-asgi==0.64b0
+opentelemetry-instrumentation-asyncio==0.64b0
+opentelemetry-instrumentation-asyncpg==0.64b0
+opentelemetry-instrumentation-aws-lambda==0.64b0
+opentelemetry-instrumentation-boto3sqs==0.64b0
+opentelemetry-instrumentation-botocore==0.64b0
+opentelemetry-instrumentation-cassandra==0.64b0
+opentelemetry-instrumentation-celery==0.64b0
+opentelemetry-instrumentation-click==0.64b0
+opentelemetry-instrumentation-confluent-kafka==0.64b0
+opentelemetry-instrumentation-dbapi==0.64b0
+opentelemetry-instrumentation-django==0.64b0
+opentelemetry-instrumentation-elasticsearch==0.64b0
+opentelemetry-instrumentation-falcon==0.64b0
+opentelemetry-instrumentation-fastapi==0.64b0
+opentelemetry-instrumentation-flask==0.64b0
+opentelemetry-instrumentation-grpc==0.64b0
+opentelemetry-instrumentation-httpx==0.64b0
+opentelemetry-instrumentation-jinja2==0.64b0
+opentelemetry-instrumentation-kafka-python==0.64b0
+opentelemetry-instrumentation-logging==0.64b0
+opentelemetry-instrumentation-mysql==0.64b0
+opentelemetry-instrumentation-mysqlclient==0.64b0
+opentelemetry-instrumentation-pika==0.64b0
+opentelemetry-instrumentation-psycopg==0.64b0
+opentelemetry-instrumentation-psycopg2==0.64b0
+opentelemetry-instrumentation-pymemcache==0.64b0
+opentelemetry-instrumentation-pymongo==0.64b0
+opentelemetry-instrumentation-pymysql==0.64b0
+opentelemetry-instrumentation-pyramid==0.64b0
+opentelemetry-instrumentation-redis==0.64b0
+opentelemetry-instrumentation-remoulade==0.64b0
+opentelemetry-instrumentation-requests==0.64b0
+opentelemetry-instrumentation-sqlalchemy==0.64b0
+opentelemetry-instrumentation-sqlite3==0.64b0
+opentelemetry-instrumentation-starlette==0.64b0
+opentelemetry-instrumentation-system-metrics==0.64b0
+opentelemetry-instrumentation-threading==0.64b0
+opentelemetry-instrumentation-tornado==0.64b0
+opentelemetry-instrumentation-tortoiseorm==0.64b0
+opentelemetry-instrumentation-urllib==0.64b0
+opentelemetry-instrumentation-urllib3==0.64b0
+opentelemetry-instrumentation-wsgi==0.64b0

opentelemetry-sdk/pyproject.toml

@@ -78,6 +78,8 @@ process = "opentelemetry.sdk.resources:ProcessResourceDetector"
 os = "opentelemetry.sdk.resources:OsResourceDetector"
 host = "opentelemetry.sdk.resources:_HostResourceDetector"
 service_instance = "opentelemetry.sdk.resources:ServiceInstanceIdResourceDetector"
+# mw: fork-only, opt-in via OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=vcs
+vcs = "opentelemetry.sdk.resources._mw_vcs:VcsResourceDetector"
 
 [project.urls]
 Homepage = "https://github.com/open-telemetry/opentelemetry-python/tree/main/opentelemetry-sdk"

opentelemetry-sdk/src/opentelemetry/sdk/resources/_mw_vcs.py

@@ -0,0 +1,85 @@
+# Copyright The OpenTelemetry Authors
+# SPDX-License-Identifier: Apache-2.0
+
+"""Middleware fork-only addition.
+
+Resolves VCS metadata (repository URL, commit SHA) consumed by Middleware's
+Ops AI to point generated fixes at the right file/line and open PRs against
+the right commit/branch:
+https://docs.middleware.io/opsai/apm_configuration/python#vcs-metadata
+
+Reads ``MW_VCS_REPOSITORY_URL`` / ``MW_VCS_COMMIT_SHA`` if set (e.g. injected
+by CI), otherwise falls back to the local ``.git`` checkout via the `git`
+CLI. Mirrors middleware-labs/agent-apm-python's `get_git_info()` (PRs #61,
+#62), but shells out to `git` instead of depending on GitPython, so this adds
+no mandatory dependency to opentelemetry-sdk.
+
+This is registered as the "vcs" resource detector entry point rather than
+wired into the default detector list, so it stays purely opt-in and requires
+no changes to `Resource.create` / `_build_resource_detectors`: enable it by
+adding "vcs" to the `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` environment
+variable (e.g. `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=vcs`).
+"""
+
+from __future__ import annotations
+
+import subprocess
+from logging import getLogger
+from os import environ
+
+from opentelemetry.sdk.resources import Resource, ResourceDetector
+
+_logger = getLogger(__name__)
+
+_GIT_TIMEOUT_SECONDS = 2
+
+VCS_REPOSITORY_URL = "vcs.repository_url"
+VCS_COMMIT_SHA = "vcs.commit_sha"
+
+
+def _run_git(*args: str) -> str | None:
+    try:
+        result = subprocess.run(
+            ["git", *args],
+            capture_output=True,
+            text=True,
+            timeout=_GIT_TIMEOUT_SECONDS,
+            check=True,
+        )
+    except Exception as error:  # git missing, not a repo, no HEAD yet, timeout, ...
+        _logger.debug("git %s failed: %s", " ".join(args), error)
+        return None
+    return result.stdout.strip() or None
+
+
+def _detect_git_info() -> tuple[str | None, str | None]:
+    # `git` walks up to find the enclosing repository on its own, so this
+    # works from any subdirectory without needing to search for `.git`.
+    commit_sha = _run_git("rev-parse", "HEAD")
+    repository_url = _run_git("config", "--get", "remote.origin.url")
+    if repository_url and repository_url.endswith(".git"):
+        repository_url = repository_url[: -len(".git")]
+    return repository_url, commit_sha
+
+
+class VcsResourceDetector(ResourceDetector):
+    """Adds `vcs.repository_url` / `vcs.commit_sha` resource attributes from
+    `MW_VCS_REPOSITORY_URL` / `MW_VCS_COMMIT_SHA`, falling back to the local
+    git checkout. Omits either attribute entirely if neither source resolves
+    it."""
+
+    def detect(self) -> Resource:
+        repository_url = environ.get("MW_VCS_REPOSITORY_URL")
+        commit_sha = environ.get("MW_VCS_COMMIT_SHA")
+
+        if not repository_url or not commit_sha:
+            git_repository_url, git_commit_sha = _detect_git_info()
+            repository_url = repository_url or git_repository_url
+            commit_sha = commit_sha or git_commit_sha
+
+        attributes = {}
+        if repository_url:
+            attributes[VCS_REPOSITORY_URL] = repository_url
+        if commit_sha:
+            attributes[VCS_COMMIT_SHA] = commit_sha
+        return Resource(attributes)

opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py

@@ -55,6 +55,9 @@
     _get_process_dependent_resource,
 )
 from opentelemetry.sdk.trace import sampling
+from opentelemetry.sdk.trace._mw_exception_context import (  # mw: fork-only exception source capture
+    get_exception_source_attributes,
+)
 from opentelemetry.sdk.trace._tracer_metrics import create_tracer_metrics
 from opentelemetry.sdk.trace.id_generator import IdGenerator, RandomIdGenerator
 from opentelemetry.sdk.util import BoundedList
@@ -1083,6 +1086,8 @@ def record_exception(
             EXCEPTION_STACKTRACE: stacktrace,
             EXCEPTION_ESCAPED: str(escaped),
         }
+        # mw: fork-only addition, opt-in via MW_RECORD_EXCEPTION_SOURCE
+        _attributes.update(get_exception_source_attributes(exception))
         if attributes:
             _attributes.update(attributes)
         self.add_event(