diff --git a/products/clickstack/integration-partners/index.mdx b/products/clickstack/integration-partners/index.mdx index 7be22c2c..315640a4 100644 --- a/products/clickstack/integration-partners/index.mdx +++ b/products/clickstack/integration-partners/index.mdx @@ -11,3 +11,4 @@ ClickStack provides the core building blocks of modern observability, bringing t | Section | Description | |------|-------------| | [Bindplane](/products/clickstack/integration-partners/bindplane) | Bindplane is an OpenTelemetry-native telemetry pipeline that provides centralized management for OpenTelemetry Collectors. | +| [Odigos](/products/clickstack/integration-partners/odigos) | Odigos combines eBPF and OpenTelemetry to deliver deeper distributed traces with dramatically lower overhead than bytecode agents. | diff --git a/products/clickstack/integration-partners/odigos.mdx b/products/clickstack/integration-partners/odigos.mdx new file mode 100644 index 00000000..2e5f51e5 --- /dev/null +++ b/products/clickstack/integration-partners/odigos.mdx @@ -0,0 +1,280 @@ +--- +slug: /use-cases/observability/clickstack/integration-partners/odigos +title: 'Sending OpenTelemetry to ClickStack with Odigos' +sidebarTitle: 'Odigos' +description: 'Auto-instrument Kubernetes workloads with Odigos and export telemetry to ClickStack over OTLP' +doc_type: 'guide' +keywords: ['Odigos', 'ClickStack', 'ClickHouse', 'OpenTelemetry', 'eBPF', 'auto-instrumentation'] +--- + +import PartnerBadge from "/snippets/components/PartnerBadge/PartnerBadge.jsx"; + +# Sending OpenTelemetry to ClickStack with Odigos {#odigos-clickstack} + + + + +**TL;DR** + +This guide shows you how to export Odigos telemetry to ClickStack. You'll learn how to: + +- Deploy Odigos on Kubernetes with Helm +- Add sources in the Odigos UI +- Add an OTLP HTTP destination pointed at ClickStack +- Verify logs, metrics, and traces in ClickStack + +Odigos auto-instruments applications without code changes nor restarts; ClickStack stores and queries the data in ClickHouse. + +Time required: 10–20 minutes + + +## What is Odigos? {#what-is-odigos} + +[Odigos](https://odigos.io/) is an instrumentation control plane for Kubernetes and VMs that instruments applications from the kernel using **eBPF**. Because collection runs in the kernel, app overhead stays low while visibility stays high. You get production-grade OpenTelemetry traces, metrics, logs, and profiles without shipping new agents in application code or waiting on library upgrades across every service. + +That eBPF layer is what makes deep, consistent telemetry possible at scale. Odigos can automatically turn on and off deeper instrumentation when needed to help debug or troubleshoot problems: + +- **Code-level context** — attributes tied to functions and runtime behavior +- **HTTP traffic** — requests and responses across your services +- **Messaging systems** — payloads and messages from Kafka and similar brokers +- **Errors in detail** — stack traces when things fail +- **Custom instrumentation** — extend coverage where auto-instrumentation stops without requiring code changes or restarts + +Behind the scenes, Odigos creates and manages a full OpenTelemetry pipeline for your cluster: collectors that scale with load, routing to the backends you choose, and pipeline logic you control in the UI. Define **sampling** to manage volume, **PII masking** to keep sensitive data out of exports, and **OTTL rules** to filter, transform, or enrich telemetry before it leaves the cluster. + +## Why Odigos + ClickStack? {#why-odigos-clickstack} + +Rolling out OpenTelemetry across many services is often time consuming and only provides surface-level visibility in applications. Odigos handles eBPF instrumentation for deeper telemetry and collector operations on Kubernetes; ClickStack provides ClickHouse-backed storage and the HyperDX UI for querying telemetry at scale. + + +**Key takeaways** + +- **Odigos** auto-instruments any Kubernetes workload without requiring restarts and manages OpenTelemetry pipelines automatically. +- **ClickStack** stores logs, metrics, and traces in ClickHouse and surfaces them in HyperDX. + + +## Prerequisites {#prerequisites} + +- **ClickStack** installed and reachable from your Kubernetes cluster. See [Getting started with open source ClickStack](/products/clickstack/getting-started/oss) or [Getting started with managed ClickStack](/products/clickstack/getting-started/managed). +- Your ClickStack **OTLP HTTP endpoint** (port `4318`) and the authentication value Odigos will pass in the `Authorization` header. With open source ClickStack this is the **API ingestion key** from **Team Settings → API Keys** in the HyperDX UI. With Managed ClickStack this is the **`OTLP_AUTH_TOKEN`** you set when starting your own standalone ClickStack collector. +- A **Kubernetes cluster** (Linux nodes with kernel 4.18 or later for eBPF instrumentation) +- **Helm**, **kubectl**, and cluster credentials to install into `odigos-system` namespace +- An **Odigos Enterprise on-prem token** — contact the [Odigos team](https://odigos.io/) for access + +## Integrate ClickStack with Odigos {#integrate-odigos-clickstack} + + + +#### Deploy Odigos with Helm {#deploy-odigos} + +Odigos Enterprise requires an on-prem license token. Export it in your shell: + +```bash +export ODIGOS_ONPREM_TOKEN="" +``` +Alternatively, you can store the token in a Kubernetes Secret named `odigos-pro` before installing. See [Odigos Enterprise installation](https://docs.odigos.io/enterprise/setup/installation). + +Add the Odigos Helm repository and install the chart into `odigos-system`: + +```bash +helm repo add odigos https://odigos-io.github.io/odigos/ +helm repo update + +helm upgrade --install odigos odigos/odigos \ + --namespace odigos-system \ + --create-namespace \ + --set onPremToken=$ODIGOS_ONPREM_TOKEN +``` + +You can pass additional configuration overrides with `--set` flags or a custom values file (`-f`). The chart's default values are in [helm/odigos/values.yaml](https://github.com/odigos-io/odigos/blob/main/helm/odigos/values.yaml) on GitHub. + +Verify Odigos pods are running: + +```bash +kubectl get pods -n odigos-system +``` + + +#### Add sources in the Odigos UI {#add-sources} + +1. Port-forward the Odigos UI service: + +```bash +kubectl port-forward svc/ui -n odigos-system 3000:3000 +``` + +2. Open [http://localhost:3000](http://localhost:3000) in your browser. +3. Go to **Sources** and select the namespaces or workloads you want to instrument. +4. Click done at the bottom, once you have marked all workloads for instrumentation. +5. Verify workloads have been instrumented successfully in the sources column. + + +#### Add ClickStack as a destination in the Odigos UI {#add-destination-ui} + +To send telemetry to ClickStack, add an **OTLP HTTP** destination in Odigos. The exact configuration depends on how ClickStack is deployed. With open source ClickStack the OpenTelemetry collector is bundled and the ingestion key is generated for you in the HyperDX UI. With Managed ClickStack you run your own standalone ClickStack collector and choose the authentication token yourself when starting the container. + + +**Alternative: write directly to ClickHouse** + +If ClickHouse is reachable from your Kubernetes cluster, you can skip the OTLP collector entirely and use Odigos's [native **ClickHouse** destination](#native-clickhouse-destination) instead. This works for both open source and Managed ClickStack. + + + + + +With open source ClickStack, for example the all-in-one image, the gateway OpenTelemetry collector is included and the ingestion API key is generated automatically by HyperDX. + +1. In the Odigos UI, click **Add Destination** and select **OTLP HTTP**. +2. Set **OTLP HTTP Endpoint** to your ClickStack collector (for example, `http://clickstack.example.com:4318`). See [Ingesting with OpenTelemetry](/products/clickstack/ingesting-data/opentelemetry#sending-data-to-collector-oss) for endpoint details. +3. Copy your API ingestion key from the ClickStack UI under **Team Settings → API Keys**. +4. In **Headers**, add: + - **Key**: `Authorization` + - **Value**: your API ingestion key +5. Enable **Logs**, **Metrics**, and **Traces**. +6. Save the destination. + + + + + +Managed ClickStack does not ship a hosted OpenTelemetry collector or surface an ingestion key in the UI. Instead, you run the [ClickStack distribution of the collector in standalone mode](/products/clickstack/ingesting-data/collector#configuring-the-collector) yourself and set the authentication token via the `OTLP_AUTH_TOKEN` environment variable when starting the container. Odigos then sends OTLP HTTP traffic to that collector with the same token in the `Authorization` header. + +1. Start the ClickStack collector in standalone mode, pointing it at your ClickHouse Cloud service and securing it with an `OTLP_AUTH_TOKEN` of your choice: + + ```shell + export CLICKHOUSE_ENDPOINT= + export CLICKHOUSE_USER= + export CLICKHOUSE_PASSWORD= + export OTLP_AUTH_TOKEN="a_very_secure_string" + + docker run \ + -e OTLP_AUTH_TOKEN=${OTLP_AUTH_TOKEN} \ + -e CLICKHOUSE_ENDPOINT=${CLICKHOUSE_ENDPOINT} \ + -e CLICKHOUSE_USER=${CLICKHOUSE_USER} \ + -e CLICKHOUSE_PASSWORD=${CLICKHOUSE_PASSWORD} \ + -p 4317:4317 \ + -p 4318:4318 \ + clickhouse/clickstack-otel-collector:latest + ``` + + For TLS, dedicated ingestion users, and other production recommendations, see [Securing the collector](/products/clickstack/ingesting-data/collector#securing-the-collector). +2. In the Odigos UI, click **Add Destination** and select **OTLP HTTP**. +3. Set **OTLP HTTP Endpoint** to the standalone collector you just started (for example, `http://my-collector.example.com:4318`). +4. In **Headers**, add: + - **Key**: `Authorization` + - **Value**: the `OTLP_AUTH_TOKEN` value you set on the collector +5. Enable **Logs**, **Metrics**, and **Traces**. +6. Save the destination. + + +**Optional: Kubernetes manifest** + +You can configure the same destination with a `Destination` manifest instead of the UI. See [Configure destinations with Kubernetes manifests](#destination-manifest) in Advanced configuration. + + + + + + + +#### Verify telemetry in ClickStack {#verify-telemetry} + +1. Open the ClickStack UI (HyperDX): + - **Open source ClickStack**: for example, `http://:8080` on the all-in-one image. + - **Managed ClickStack**: open your service in the [ClickHouse Cloud console](https://console.clickhouse.cloud), then click **Launch ClickStack**. See [Navigate to the ClickStack UI](/products/clickstack/getting-started/managed#navigate-to-clickstack-ui-cloud) for details. +2. Check **Logs**, **Metrics**, and **Traces** for data from your instrumented services. +3. Filter traces by `odigos.version` to validate end-to-end export. + +If data is missing, check collector logs: `kubectl logs deploy/odigos-gateway -n odigos-system` + + + +## Advanced configuration {#advanced-configuration} + +### HyperDX log normalizer {#hyperdx-log-normalizer} + +If you export directly to ClickHouse with Odigos's native **ClickHouse** destination (instead of OTLP HTTP to ClickStack), enable **HyperDX log normalizer** (`HYPERDX_LOG_NORMALIZER: true`). It parses JSON log bodies and normalizes attributes for better querying in the ClickStack UI. + +### Native ClickHouse destination {#native-clickhouse-destination} + +When ClickHouse is directly reachable from your cluster, you can use Odigos's native **ClickHouse** destination instead of OTLP HTTP. Configure the ClickHouse endpoint, database name, and schema options in the UI or with a manifest — see [Odigos ClickHouse destination](https://docs.odigos.io/backends/clickhouse). + +- **Production schema**: Set `CLICKHOUSE_CREATE_SCHEME` to `false` and apply your own DDL. +- **TLS / auth**: Use `CLICKHOUSE_TLS_ENABLED`, `CLICKHOUSE_USERNAME`, and a Kubernetes Secret for the password. + +### Configure destinations with Kubernetes manifests {#destination-manifest} + +**OTLP HTTP (ClickStack)** + +```yaml +apiVersion: odigos.io/v1alpha1 +kind: Destination +metadata: + name: clickstack + namespace: odigos-system +spec: + type: otlphttp + destinationName: otlphttp + signals: + - TRACES + - METRICS + - LOGS + data: + OTLP_HTTP_ENDPOINT: 'http://clickstack.example.com:4318' + # API ingestion key for open source ClickStack, or OTLP_AUTH_TOKEN for Managed ClickStack + OTLP_HTTP_HEADERS: 'Authorization:' +``` + +**ClickHouse (direct)** + +```yaml +apiVersion: odigos.io/v1alpha1 +kind: Destination +metadata: + name: clickhouse + namespace: odigos-system +spec: + type: clickhouse + destinationName: clickhouse + signals: + - TRACES + - METRICS + - LOGS + data: + CLICKHOUSE_ENDPOINT: 'http://clickstack.example.com:8123' + CLICKHOUSE_DATABASE_NAME: 'otel' + CLICKHOUSE_CREATE_SCHEME: 'true' +``` + +Apply the manifest: + +```bash +kubectl apply -f destination.yaml +``` + +### Odigos VM Agent {#odigos-vm-agent} + +The [Odigos VM Agent](https://docs.odigos.io/vmagent/overview) instruments Linux processes, systemd services, and/or docker containers using eBPF. Telemetry exports to the same destinations as cluster-based Odigos, including ClickStack over OTLP HTTP. + +The VM Agent is part of Odigos Pro. See the [VM Agent overview](https://docs.odigos.io/vmagent/overview) for setup, sources, and destination configuration. + +### Odigos Central {#odigos-central} + +[Odigos Central](https://docs.odigos.io/central/overview) is a centralized control plane for managing instrumentation, destinations, and pipeline configuration across multiple Kubernetes clusters from one UI - instead of configuring each cluster separately. + +Odigos Central is available in Odigos Enterprise. See the [Central overview](https://docs.odigos.io/central/overview) for multi-cluster management, SSO, and unified sampling rules. + +## Next steps {#next-steps} + +- **Explore traces** across instrumented services in ClickStack +- **Build dashboards** for metrics Odigos exports +- **Tune ClickHouse schema and TTL** for your retention and query patterns + +## Read more {#read-more} + +- [Odigos Enterprise installation](https://docs.odigos.io/enterprise/setup/installation) +- [Odigos ClickHouse destination](https://docs.odigos.io/backends/clickhouse) +- [Odigos VM Agent overview](https://docs.odigos.io/vmagent/overview) +- [Odigos Central overview](https://docs.odigos.io/central/overview) +- [Stop guessing in production: Full fidelity tracing at scale with ClickHouse and Odigos](https://clickhouse.com/blog/odigos-full-fidelity-tracing) +- [Getting started with open source ClickStack](/products/clickstack/getting-started/oss) diff --git a/products/clickstack/navigation.json b/products/clickstack/navigation.json index eb640abf..8e69a875 100644 --- a/products/clickstack/navigation.json +++ b/products/clickstack/navigation.json @@ -217,7 +217,8 @@ "group": "Integration partners", "root": "products/clickstack/integration-partners/index", "pages": [ - "products/clickstack/integration-partners/bindplane" + "products/clickstack/integration-partners/bindplane", + "products/clickstack/integration-partners/odigos" ] }, {