Skip to content

feat(client): support organization API keys with project_id - #2074

Draft
marksalpeter wants to merge 1 commit into
mainfrom
mark/lfe-18172-update-sdks-to-support-multi-org-api-tokens
Draft

marksalpeter wants to merge 1 commit into
mainfrom
mark/lfe-18172-update-sdks-to-support-multi-org-api-tokens

Conversation

@marksalpeter

Copy link
Copy Markdown

What does this PR do?

Adds SDK support for organization API keys (LFE-18172). An org key can write to any project in its organization, so every request has to name its target project in the x-langfuse-project-id header.

New option: Langfuse(project_id=...), or the LANGFUSE_PROJECT_ID env var. The argument takes precedence.

Fixes a cross-project leak. Clients used to be identified by public_key only. With one org key used for two projects, Langfuse(public_key=ORG, project_id="B") silently reused project A's client, and each client's span processor exported the other's spans. A client is now identified by (public_key, project_id) in:

  • the LangfuseResourceManager._instances registry key;
  • the tracer instrumentation-scope attributes (project_id next to public_key) and the LangfuseSpanProcessor filter;
  • get_client(public_key=..., project_id=...). If several clients share a public key and no project_id is given, it returns a disabled client and logs a warning, like today's "multiple clients, no key" case;
  • @observe(langfuse_project_id=...) and the execution-context var that nested decorated functions read.

Header: sent on OTLP span exports and on the generated LangfuseAPI / AsyncLangfuseAPI clients. It is set on the generated clients rather than the httpx client, so a user-supplied httpx_client sends it too.

Conflict check: if additional_headers or httpx_client.headers sets x-langfuse-project-id / langfuse-project-id (any case) to a different value, init raises ValueError. The same value is accepted. A header set without project_id is left alone, so the existing additional_headers workaround keeps working.

URLs: with project_id configured, _get_project_id() returns it without calling projects.get(). This covers trace URLs and experiment URLs.

Warnings and hints:

  • project_id together with a custom span_exporter logs a warning that the exporter must set the header itself.
  • A 403 span export logs a one-time hint to set LANGFUSE_PROJECT_ID / project_id.

Project API keys leave project_id unset, so their behavior is unchanged.

Notes for reviewers

  • 403 hint: the OTLP exporter doesn't return the HTTP status, so the hint comes from a logging.Filter on the exporter's logger. It matches the exporter's Failed to export ... code: 403 record, fires once per process, and never suppresses the record. If a future OTel release rewords that message, the hint stops appearing and nothing else changes.
  • auth_check() still calls projects.get(). With the header sent, the server resolves the header's project, so this should work for org keys whose role grants project:read.
  • Not covered here: the OpenAI wrapper's explicit langfuse_public_key and LangChain CallbackHandler(public_key=...) don't take a project ID yet. With an org key shared by several clients, they get a disabled client. Inside @observe they already pick the right client from context. This can be a follow-up.
  • Leak test: the new test uses real span processors on a shared TracerProvider. The existing multi_project_setup fixture mocks the processor constructor, so it doesn't exercise the scope filter. I checked that the test fails when the project_id check is removed from the filter.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

List the main commands you ran:

uv run --frozen ruff format --check .
uv run --frozen ruff check .
uv run --frozen mypy langfuse --no-error-summary
uv run --frozen pytest -n auto --dist worksteal tests/unit
bash scripts/codex/quick-check.sh

Unit tests: 1031 passed. 18 tests in tests/unit/test_prompt.py error locally because LANGFUSE_PUBLIC_KEY isn't set in the environment; they error the same way on main. I didn't run e2e locally. The only e2e change is the tests/e2e/test_decorators.py cleanup helper, updated for the new registry key shape.

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed.
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

Generated by Claude Code

Organization API keys can write to any project in their organization, so
each request has to name its target project in the x-langfuse-project-id
header. Add a project_id argument (env LANGFUSE_PROJECT_ID) to Langfuse()
and identify a client by (public_key, project_id) everywhere:

- LangfuseResourceManager instances are keyed by the pair, so a second
  client on the same org key for another project no longer silently
  reuses the first client's setup.
- Tracer scope attributes carry project_id and LangfuseSpanProcessor
  filters on both, so two clients sharing an org key never export each
  other's spans.
- get_client() and @observe accept project_id / langfuse_project_id; an
  org key shared by several clients without a project_id resolves to a
  disabled client.

The header is sent on OTLP exports and on the generated API clients (so a
caller-provided httpx_client sends it too). A conflicting project header
in additional_headers or httpx_client.headers raises ValueError at init.
A configured project_id is used for trace and experiment URLs without
calling projects.get(). A custom span_exporter with project_id warns that
the exporter must set the header, and a 403 span export logs a one-time
hint to set LANGFUSE_PROJECT_ID.

Project API keys leave project_id unset, so their behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PExPf2jnm8CPvdxKYmfqt

marksalpeter commented Oct 29, 2026 •

Copy link
Copy Markdown
Author

CLA gate was failing on aaff525 ("CLA missing or unavailable").

Resolved: the CLA is now signed (license/cla passes), and CLA gate reports "CLA verified." on aaff525. No rewrite of the commit was needed.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants