diff --git a/research/vercel-ai-sdk.md b/research/vercel-ai-sdk.md new file mode 100644 index 0000000..0820b5a --- /dev/null +++ b/research/vercel-ai-sdk.md @@ -0,0 +1,127 @@ +--- +title: "Vercel AI SDK — Stateless, Provider-Agnostic Agent Building for TypeScript" +author: Ruben Koster (@rkoster) +date: 2026-08-10 +tags: [orchestration, inter-agent-comms, observability-governance, ecosystem-survey] +cf_areas: [] +status: draft +sources: + - https://github.com/vercel/ai + - https://ai-sdk.dev/docs/agents/overview + - https://ai-sdk.dev/docs/agents/workflows + - https://ai-sdk.dev/docs/agents/memory + - https://ai-sdk.dev/docs/ai-sdk-core/mcp-tools + - https://ai-sdk.dev/docs/ai-sdk-core/telemetry + - https://useworkflow.dev + - https://www.anthropic.com/research/building-effective-agents +--- + +## Summary + +The Vercel AI SDK (`ai` npm package, plus companion packages like `@ai-sdk/react`, +`@ai-sdk/mcp`, `@ai-sdk/otel`) is an Apache-2.0-licensed, Vercel-owned TypeScript toolkit for +building AI-powered applications and agents across Next.js, React, Svelte, Vue, and Node.js +runtimes. It has evolved from a chat-UI helper library into a full agent-building toolkit: a +unified multi-provider model interface, tool-calling, automatic multi-step tool loops, a formal +`ToolLoopAgent` class, named multi-agent workflow patterns, MCP client support, OpenTelemetry +observability, and a UI-streaming layer (`useChat`/`useCompletion`) — all designed to run +inside serverless/edge functions rather than as a persistent, always-addressable server +process. This makes it architecturally the inverse of Cloudflare's Durable-Object-based Agents +SDK (`cloudflare-agents.md`): Vercel AI SDK is a stateless request/response library that must +be paired with external state and durable-execution infrastructure, whereas Cloudflare's model +bakes persistent per-agent state and hibernation directly into the runtime primitive. + +## Key findings + +- **What it is / license / governance**: "a provider-agnostic TypeScript toolkit designed to + help you build AI-powered applications and agents," created and maintained by Vercel/Next.js + team members, Apache-2.0 licensed, governed as a corporate open-source project (no + foundation) — 26.1k GitHub stars, 4.9k forks. +- **Very heavy adoption**: the core `ai` package had ~78.3M downloads in a single trailing + month on npm — several orders of magnitude beyond niche agent frameworks, indicating it's a + de facto standard for JS/TS LLM app development. +- **Unified provider API ("AI SDK Core")**: a single interface (`generateText`, `streamText`, + etc.) works across OpenAI, Anthropic, Google, xAI and others, either through direct provider + packages or by default through the Vercel AI Gateway using plain model-string identifiers + (e.g. `'anthropic/claude-opus-4.6'`). +- **Agent loop primitives, evolved into a formal class**: agents are described as "LLMs that + use tools in a loop," and the SDK now provides a first-class `ToolLoopAgent` class + encapsulating the model, tools, context management, and stopping conditions — recommended + over manually looping `generateText`/`streamText`. A separate `HarnessAgent` class wraps + pre-built coding-agent harnesses (Claude Code, Codex). Loop control (stopping/looping + behavior) is governed by explicit `stopWhen`/step-limit conditions. +- **Named multi-agent workflow patterns**: the docs explicitly define and provide code for + five patterns adapted from Anthropic's "Building Effective Agents" guide — Sequential + Processing (Chains), Routing, Parallel Processing, Orchestrator-Worker, and + Evaluator-Optimizer — implemented directly with `generateText`/structured output rather than + a dedicated multi-agent orchestration class. Notably, Cloudflare's Agents SDK documents this + exact same pattern set (`cloudflare-agents.md`), suggesting Anthropic's framing has become a + shared vocabulary across the JS/TS agent ecosystem independent of runtime architecture. +- **MCP client support**: `@ai-sdk/mcp`'s `createMCPClient` supports HTTP (recommended for + production), SSE, and stdio (local-only) transports, OAuth-based authorization, resumable + sessions, resource/prompt/completion access, elicitation-request handling, and even a "rug + pull" tool-definition-drift detector (`fingerprintTools`/`detectToolDrift`) for MCP + tool-poisoning defense — a concrete, shipped mitigation for the tool-poisoning risk flagged + in MCP's own spec (`mcp-protocol.md`). +- **Streaming/UI integration is the SDK's core strength**: `AI SDK UI` provides + framework-agnostic hooks (`useChat`, `useCompletion`) for Next.js/React/Svelte/Vue, paired + with server helpers like `createAgentUIStreamResponse` for streaming an agent's tool calls + and text as Server-Sent-Events/data-stream protocol directly into chat UIs. +- **No built-in agent state store**: the SDK provides no native session/state persistence + layer for agent memory; the docs describe three externally-provided approaches — + provider-defined memory tools (e.g. Anthropic's memory tool), third-party "Memory Providers" + (Letta, Mem0, Supermemory, Hindsight, MongoDB Atlas-backed memory — note Letta's own memory + architecture is covered in `letta.md`), or fully custom tools — all pushing persistence to an + external store rather than the runtime itself. +- **No A2A protocol support found**: not mentioned anywhere in the docs or README reviewed — + the SDK's interoperability story is built around MCP (for tools) and its own UI + message/data-stream protocol, not the Agent2Agent protocol (`a2a-protocol.md`) — another data + point (alongside LangGraph and LlamaIndex) that A2A adoption is uneven across the ecosystem. +- **Deployment model — stateless functions, durability delegated elsewhere**: the SDK is + designed to run as request-scoped functions (Next.js API routes / Vercel Functions, or any + Node/edge runtime) with no built-in durable-object-style always-on process; long-running or + resumable execution is delegated to a separate product, the "Workflow DevKit" + (useworkflow.dev), rather than being intrinsic to the AI SDK. This is the direct + architectural inverse of Cloudflare's Durable-Object-backed Agents SDK + (`cloudflare-agents.md`), where each agent instance is a persistent, addressable, + hibernatable object by default. +- **Observability — documented OpenTelemetry integration**: telemetry is opt-out once a + `@ai-sdk/otel` `OpenTelemetry` integration is registered; it emits spans following the + OpenTelemetry GenAI Semantic Conventions (`invoke_agent`, `chat {modelId}`, + `execute_tool {toolName}` spans with `gen_ai.*` attributes, see `opentelemetry-genai.md`), + plus a legacy `ai.*`-prefixed span format and a Node.js `diagnostics_channel` tracing channel + for zero-config observability-provider hookup. +- **Notable users/adoption**: Vercel's marketing lists OpenAI, Photoroom, Leonardo.ai, and + Zapier as users of the Vercel platform/AI stack — combined with the ~78M/month npm download + volume, a strong signal of de facto standardization in the JS/TS ecosystem. + +## CF relevance + +The Vercel AI SDK and Cloudflare's Agents SDK together bracket a genuine architectural choice +for a CF-native agent runtime targeting the JS/TS ecosystem: **stateless library + external +durability service** (Vercel's model, requiring CF to provide or bind a session/memory/workflow +service) versus **stateful-by-default compute primitive** (Cloudflare's model, requiring CF to +offer something like a durably-addressable object as a platform primitive). Since the Vercel AI +SDK is overwhelmingly the more widely adopted of the two (78M+ monthly downloads vs. a +comparatively niche Durable-Object-specific SDK), a CF buildpack/service story that supports +"Vercel AI SDK + external state" is likely higher-leverage for near-term JS/TS agent-workload +support than trying to replicate Cloudflare's Durable Object model. The SDK's five named +multi-agent workflow patterns (shared with Cloudflare's docs, both citing Anthropic's framing) +suggest this vocabulary is becoming a de facto standard across the JS/TS ecosystem regardless +of runtime — worth adopting as shared terminology in any CF-facing guidance on agent design +patterns. + +## Open questions + +- Given the SDK has no built-in state/memory layer, what would a CF-native "agent session + store" service binding need to look like to serve as one of its documented external Memory + Provider options? +- Is the Workflow DevKit (Vercel's separate durable-execution product, not part of the AI SDK + itself) worth evaluating directly as another durable-execution substrate alongside Temporal + and Dapr Workflows already covered in this research set? +- Given both Vercel AI SDK and Cloudflare's Agents SDK converge on the same five Anthropic- + derived multi-agent patterns, should CF adopt this vocabulary as its own standard framing for + discussing agent orchestration patterns platform-wide, independent of any specific framework? +- The built-in "rug pull" MCP tool-drift detector is a concrete, shipped mitigation for a named + MCP security risk (`mcp-protocol.md`) — should CF require or recommend equivalent tool-drift + detection for any CF-hosted agent consuming external MCP servers, regardless of framework?