Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions docs/admin/configuration/observability/request-tagging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: request-tagging
title: LLM Request Tagging
sidebar_label: Request Tagging
sidebar_position: 3
pagination_prev: admin/configuration/observability/observability-overview
pagination_next: null
---

CodeMie automatically injects metadata headers into every outbound LLM API request so that
cloud provider logs and in-cluster observability tools can attribute traffic to the running
platform version and the active project context.

## Injected Headers

Two HTTP headers are added to outbound requests:

| Header | Value | Example |
| ------------------- | -------------------------------------------------- | ------------ |
| `X-CodeMie-Version` | Value of the `APP_VERSION` environment variable | `1.4.2` |
| `X-CodeMie-Project` | Name of the active CodeMie project at request time | `my-project` |

`X-CodeMie-Version` is always present. `X-CodeMie-Project` is included only when a project
context is active; it is omitted entirely when no project is set for the request.

## Provider Coverage

Headers are injected at the LLM client construction layer. Coverage depends on the AI
provider path used by the model:

| Provider path | Headers injected | Notes |
| ------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Azure OpenAI / DIAL | Yes | Added to `default_headers` on the `AzureChatOpenAI` client |
| Google Vertex AI | Yes | Passed via `client_options.additional_headers` on the `ChatVertexAI` client |
| Anthropic (direct) | Yes | Passed via `extra_headers` on the `ChatAnthropic` client |
| AWS Bedrock | No | Bedrock does not support custom HTTP request headers; request attribution uses request-body fields instead |
| LiteLLM proxy | No | LiteLLM receives project context via `x-litellm-tags`; the `X-CodeMie-*` headers are not forwarded to avoid double-tagging |

## Purpose and Use Cases

**Cost and traffic attribution** — cloud provider portals (Azure Monitor, Vertex AI quotas,
Anthropic usage dashboards) expose custom headers in request logs, enabling filtering and
grouping by CodeMie version or project.

**Multi-version deployments** — when multiple CodeMie versions run in parallel (canary
rollout, blue/green), `X-CodeMie-Version` identifies which instance originated a request
without requiring log correlation.

**Project-level analysis** — `X-CodeMie-Project` allows isolating LLM usage by team or
workspace in provider-side logs and in Langfuse traces, complementing the project-level
spend tracking available through the LiteLLM proxy path.

## Configuration

No explicit configuration is required. Tag injection is always active for the supported
provider paths. The values are derived from two existing environment variables:

| Variable | Description | Default |
| ------------------- | --------------------------------------------------- | -------------------------- |
| `APP_VERSION` | Platform version string used as `X-CodeMie-Version` | Set at build time |
| _(project context)_ | Resolved from the active request context at runtime | _(empty — header omitted)_ |

`APP_VERSION` is set during deployment and is not modified at runtime. See
[API Configuration](../codemie/api-configuration.md) for the full environment
variable reference.

## Verifying Header Injection

### Azure / DIAL

Request headers forwarded through Azure API Management or DIAL are visible in Azure API
Management diagnostic logs and in DIAL request traces. Filter for requests containing
`x-codemie-version` to confirm injection.

### Google Vertex AI

Enable Cloud Logging for Vertex AI API calls in the Google Cloud Console. Custom headers
appear under `httpRequest.requestHeaders` in the log entries.

### Anthropic (direct)

Anthropic's usage dashboard does not expose custom request headers. Verify injection
locally by enabling `DEBUG` log level in the CodeMie deployment and inspecting the
outbound HTTP request logs, or by routing traffic through an HTTP proxy during
development.

:::info LiteLLM path
For models routed through the LiteLLM proxy, project context reaches LiteLLM via the
`x-litellm-tags` header instead. See the
[LiteLLM Model Tagging](../codemie/api-configuration.md) section of the API
Configuration reference for the related `LITE_LLM_PROJECTS_TO_TAGS_LIST` and
`LITE_LLM_TAGS_HEADER_VALUE` environment variables.
:::

## Related Documentation

- [Observability Overview](./index.md) — infrastructure logging and Langfuse tracing
- [API Configuration](../codemie/api-configuration.md) — full environment variable
reference including `APP_VERSION` and LiteLLM tagging variables
18 changes: 18 additions & 0 deletions faq/what-http-headers-does-codemie-add-to-outbound-llm-requests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# What HTTP headers does CodeMie add to outbound LLM requests?

CodeMie injects two metadata headers into every outbound LLM API request for supported
providers:

- **`X-CodeMie-Version`** — the platform version string from the `APP_VERSION` environment
variable. Always present.
- **`X-CodeMie-Project`** — the name of the active project context at request time. Omitted
when no project is set.

These headers are added for Azure OpenAI / DIAL, Google Vertex AI, and Anthropic (direct)
provider paths. AWS Bedrock and the LiteLLM proxy path are excluded — Bedrock does not
support custom HTTP request headers, and LiteLLM receives project context via the
`x-litellm-tags` header instead.

## Sources

- [LLM Request Tagging](https://docs.codemie.ai/admin/configuration/observability/request-tagging)
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Which LLM providers receive the X-CodeMie-Version and X-CodeMie-Project headers?

The `X-CodeMie-Version` and `X-CodeMie-Project` request headers are injected for three
provider paths:

| Provider | Headers injected |
| ------------------- | --------------------------------------------------------- |
| Azure OpenAI / DIAL | Yes |
| Google Vertex AI | Yes |
| Anthropic (direct) | Yes |
| AWS Bedrock | No — Bedrock does not support custom HTTP request headers |
| LiteLLM proxy | No — project context is sent via `x-litellm-tags` instead |

No configuration is required to enable this behavior — header injection is always active
for the supported paths and derives its values from the `APP_VERSION` environment variable
and the active request's project context.

## Sources

- [LLM Request Tagging](https://docs.codemie.ai/admin/configuration/observability/request-tagging)
5 changes: 4 additions & 1 deletion sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -877,7 +877,10 @@ const sidebars: SidebarsConfig = {
id: 'admin/configuration/observability/observability-overview',
},
collapsed: true,
items: ['admin/configuration/observability/logs-retention'],
items: [
'admin/configuration/observability/logs-retention',
'admin/configuration/observability/request-tagging',
],
},
],
},
Expand Down