refactor(observability): move backend presets out of core

[christso]

Summary

AgentV still supports the existing --otel-backend langfuse|braintrust|confident names, but backend endpoint/header logic no longer lives in @agentv/core. Core now owns only generic OTLP exporter behavior and generic resolver/resource-attribute types; the CLI owns resolver lookup for current user-facing backend names, and Phoenix-specific endpoint/header/project routing is exported from the Phoenix adapter boundary.

Design Notes

• packages/core no longer exports OTEL_BACKEND_PRESETS or contains Langfuse, Braintrust, Confident, DeepEval, or Phoenix backend env-var routing.
• The eval CLI resolves .agentv/otel-backends/<name>.ts|js|mts|mjs first, then falls back to the existing built-in names so current --otel-backend usage keeps working.
• phoenixOtelBackend lives in packages/phoenix-adapter and maps Phoenix collector endpoint, client headers, API key auth, and PHOENIX_PROJECT_NAME to generic OTLP exporter inputs.
• Docs/examples now describe backend resolvers and generic OTLP env vars instead of core built-in presets.

Red/Green UAT

Check	Before	After
Core backend routing scan	`rg "LANGFUSE_	BRAINTRUST_
CLI help wording	`bun apps/cli/src/cli.ts eval run --help	rg -in "otel
Generic OTLP file export	Existing behavior needed to remain supported.	bun apps/cli/src/cli.ts eval run examples/features/assert/evals/dataset.eval.yaml --dry-run --test-id contains-check --otel-file /tmp/agentv-otel-uat-ClFG7d/trace.otlp.json --output /tmp/agentv-otel-uat-ClFG7d/run wrote an OTLP JSON file containing resourceSpans.

Validation

• bun test packages/core/test/observability
• bun test apps/cli/test/commands/eval
• bun test packages/phoenix-adapter/test/otel-backend.test.ts
• bun run typecheck
• bun run lint
• bun apps/cli/src/cli.ts eval --help | rg -n "otel|backend|trace" produced no matches because top-level eval help only lists subcommands.
• bun apps/cli/src/cli.ts eval run --help | rg -in "otel|backend|trace"
• bun run validate:examples

---

[christso]

Code review result: changes requested (GitHub would not allow a formal request-changes review from the PR owner account).

Findings:

1. P1 - Local .ts/.mts backend resolvers are advertised and searched first, but the published CLI is a Node 20-targeted binary. apps/cli/src/commands/eval/otel-backends.ts:22 includes .ts and .mts, and the docs tell users to create .agentv/otel-backends/<name>.ts (apps/web/src/content/docs/docs/evaluation/running-evals.mdx:218, packages/phoenix-adapter/README.md:7, apps/web/src/content/docs/docs/integrations/phoenix.mdx:96). AgentV's install docs still support npm install -g agentv on Node.js 20+ (apps/web/src/content/docs/docs/getting-started/installation.mdx:10), and the CLI build is target: 'node20' with a Node shebang (apps/cli/tsup.config.ts:11, apps/cli/src/cli.ts:1). On Node 20, a plain dynamic import(file:///.../resolver.ts) will fail before the resolver can run, so the documented Phoenix/local resolver flow is broken for supported installs. Please either make local resolver loading work under the packaged Node 20 CLI or narrow the supported/documented resolver extensions to Node-loadable files.

2. P2 - Resolver resource attributes are lost in --otel-file output. The new resolver contract feeds resourceAttributes into OtelTraceExporter (packages/core/src/observability/otel-exporter.ts:50), and Phoenix uses that for openinference.project.name (packages/phoenix-adapter/src/otel-backend.ts:29). However the OTLP JSON file exporter still writes resource: { attributes: [] } unconditionally (packages/core/src/observability/otlp-json-file-exporter.ts:57). A user running --otel-file ... --otel-backend phoenix or any local resolver with resource attributes gets a file that no longer contains the project/resource routing that live OTLP export would send. Please serialize the provider/span resource attributes into the file exporter and cover that case.

Verification:

• bun install
• bun --filter @agentv/core build
• bun test packages/core/test/observability -> 32 pass
• bun test apps/cli/test/commands/eval -> 124 pass after building core
• bun test packages/phoenix-adapter/test/otel-backend.test.ts -> 4 pass
• bun run typecheck -> pass

Other notes: I confirmed the vendor-specific backend presets are no longer in packages/core, the existing CLI backend names (langfuse, braintrust, confident) remain covered by tests, and CI is currently green on the draft PR.

[cloudflare-workers-and-pages]

Deploying agentv with    Cloudflare Pages

Latest commit:	c4c3de7
Status:	✅  Deploy successful!
Preview URL:	https://8364f287.agentv.pages.dev
Branch Preview URL:	https://av-otel-backend-resolvers.agentv.pages.dev

View logs

[christso]

documented/local .ts/.mts OTel backend resolvers must either work under packaged Node 20 CLI or docs/search extensions must be narrowed to Node-loadable files
resolver resourceAttributes must be serialized into --otel-file OTLP JSON output

Addressed in c4c3de70:

• Narrowed local OTel backend discovery to Node-loadable .mjs and .js files. Updated the CLI header, docs, ADR, and Phoenix adapter README away from .ts resolver examples. Added coverage for .mjs, CommonJS .js, and ignored .ts resolver files.
• Passed OTel resource attributes into the OTLP JSON file exporter and now serialize them under resourceSpans[0].resource.attributes, including resolver-provided attributes. Added a regression test for string and boolean resource attributes.

Validation run locally:

• bun install
• bun --filter @agentv/core build
• bun test packages/core/test/observability
• bun test apps/cli/test/commands/eval
• bun test packages/phoenix-adapter/test/otel-backend.test.ts
• bun run typecheck
• bun run lint
• bun run validate:examples
• bun apps/cli/src/cli.ts eval run --help | rg -n "otel|backend|trace"
• Source CLI smoke: local .mjs resolver + --otel-file dry run, then parsed OTLP JSON and confirmed service.name, openinference.project.name, and resolver boolean resource attributes are present.
• Packaged Node 20 resolver smoke: fnm exec --using 20.20.2 node apps/cli/dist/cli.js ... loaded both local .mjs and CommonJS .js resolver files.

CI is currently running on the pushed commit.

[christso]

CI update for c4c3de70: all PR checks are green.

Passed checks: Build, Typecheck, Lint, Test, Check Links, Validate Evals, Validate Marketplace, and Cloudflare Pages.

[christso]

Re-review result for c4c3de70: the prior blockers are resolved. No blocking findings remain.

Verified:

• Local OTel backend discovery is now limited to Node-loadable .mjs and .js (RESOLVER_EXTENSIONS = ['.mjs', '.js']), and the repo docs/ADR/README no longer advertise .ts/.mts resolver files for the packaged CLI path. The resolver test covers .mjs, CommonJS .js, and ignored .ts files.
• Packaged Node 20 smoke (fnm exec --using 20.20.2 node apps/cli/dist/cli.js ...) executed both a local .mjs resolver and a CommonJS .js resolver.
• --otel-file now serializes resource attributes at resourceSpans[0].resource.attributes. A source CLI dry-run smoke with a Phoenix-style local resolver confirmed service.name=agentv and openinference.project.name=agentv-pr1399-rereview in the OTLP JSON resource.
• Vendor-specific endpoint/header resolver logic remains outside packages/core/src/observability; the current CLI backend names langfuse, braintrust, and confident are still covered by apps/cli/test/commands/eval/otel-backends.test.ts.

Validation run:

• bun install --frozen-lockfile
• bun --filter @agentv/core build
• bun test apps/cli/test/commands/eval/otel-backends.test.ts -> 9 pass
• bun test packages/core/test/observability/file-exporters.test.ts -> 5 pass
• bun test packages/core/test/observability -> 33 pass
• bun test apps/cli/test/commands/eval -> 126 pass
• bun test packages/phoenix-adapter/test/otel-backend.test.ts -> 4 pass
• bun run typecheck -> pass
• bun --filter @agentv/phoenix-adapter build
• bun --filter agentv build
• Manual source CLI smoke: Phoenix-style resolver + --otel-file, parsed OTLP JSON resource attributes successfully.
• Manual packaged Node 20 smoke: .mjs and CommonJS .js resolvers loaded successfully.

Residual notes:

• The PR description Design Notes still say .ts|js|mts|mjs; the repository docs/code are fixed, but the PR body should be updated before marking ready.
• The packaged Node 20 smoke verified resolver loading only. In this worktree, bundled CLI dist could not see optional @opentelemetry/* packages for OTLP file writing, so file-resource serialization was verified through the source CLI path after building core/adapter.