Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .claude/skills/benchmark-agents/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: benchmark-agents
description: Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow DevKit, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration. Designed to stress-test skill injection for complex, multi-system builds.
description: Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration. Designed to stress-test skill injection for complex, multi-system builds.
---

# Benchmark Agents — Advanced AI Systems
Expand Down Expand Up @@ -163,7 +163,7 @@ After sessions build, verify these patterns in the generated projects:

### Project structure
```bash
echo -n "src/: "; test -d "$base/src" && echo YES || echo NO # Should be NO for WDK projects
echo -n "src/: "; test -d "$base/src" && echo YES || echo NO # Should be NO for Workflow SDK projects
echo -n "workflows/: "; test -d "$base/workflows" && echo YES || echo NO
echo -n "withWorkflow: "; grep -q "withWorkflow" "$base"/next.config.* && echo YES || echo NO
echo -n "components.json: "; test -f "$base/components.json" && echo YES || echo NO
Expand Down Expand Up @@ -204,7 +204,7 @@ head -5 "$wf" # Should show: import { getWritable } from "workflow"
- "store all designs in a gallery"

### DON'T:
- "use Vercel Workflow DevKit with getWritable"
- "use Vercel Workflow SDK with getWritable"
- "use gateway('google/gemini-3.1-flash-image-preview')"
- "install npx ai-elements"
- "add withWorkflow to next.config.ts"
Expand All @@ -227,13 +227,13 @@ head -5 "$wf" # Should show: import { getWritable } from "workflow"
| Agent uses `dall-e-3` for images | Agent doesn't know about gemini image gen | PostToolUse validate warns, capabilities table in ai-sdk (v0.9.7) |
| Agent uses `experimental_generateImage` | Old API | PostToolUse validate warns, recommend `generateText` + `result.files` (v0.9.9) |
| Raw markdown rendering (`**bold**` visible) | Agent skips AI Elements | `MessageResponse` documented as universal renderer (v0.9.2) |
| `@/../../workflows/` broken import | Workflows outside `@` alias root | Canonical structure docs: no `src/` for WDK (v0.8.3) |
| `@/../../workflows/` broken import | Workflows outside `@` alias root | Canonical structure docs: no `src/` for Workflow SDK (v0.8.3) |
| `withWorkflow` missing from next.config | Agent skipped setup step | Marked as "Required" in workflow skill (v0.8.1) |
| `defineHook` but no resume route | Agent didn't wire the 3-piece pattern | Documented as 3 required pieces (v0.9.3) |
| `generateObject()` used (removed in v6) | Agent's training data | PostToolUse validate catches as error (v0.9.3) |
| `getWritable()` in workflow scope | Sandbox violation | Strengthened warning in skill (v0.8.1) |
| Missing `vercel link` + `vercel env pull` | No OIDC credentials | Added as "Required" setup step (v0.9.1) |
| `getStepMetadata().retryCount` undefined on first attempt | WDK quirk | Documented: guard with `?? 0` (v0.9.1) |
| `getStepMetadata().retryCount` undefined on first attempt | Workflow SDK quirk | Documented: guard with `?? 0` (v0.9.1) |
| shadcn not installed | No trigger for scaffolding | Added `create-next-app` bashPattern to shadcn (v0.8.0) |
| Skill cap too low (3) | Only 3 skills injected per tool call | Raised to 5 with 18KB budget (v0.8.0) |

Expand Down Expand Up @@ -295,7 +295,7 @@ The standard improvement cycle:
Scenarios 01, 04, 09 — AI SDK, Gateway, Sandbox, AI Elements without durable workflows.

### Tier 2 — Durable Agents (45-60 min)
Scenarios 02, 03, 06, 10 — Workflow DevKit, multi-step durability, agent orchestration.
Scenarios 02, 03, 06, 10 — Workflow SDK, multi-step durability, agent orchestration.

### Tier 3 — Platform Integration (45-60 min)
Scenarios 05, 07, 08, 11, 12 — Chat SDK, Queues, Flags, Firewall, cross-platform messaging.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ A text-form relational graph covering:
| `vercel-sandbox` | Ephemeral Firecracker microVMs for running untrusted/AI-generated code safely |
| `vercel-storage` | Blob, Edge Config, Neon Postgres, Upstash Redis, migration from sunset packages |
| `verification` | Full-story verification — infers user story, verifies end-to-end browser → API → data → response |
| `workflow` | Workflow DevKit — durable execution, DurableAgent, steps, Worlds, pause/resume |
| `workflow` | Workflow SDK — durable execution, DurableAgent, steps, Worlds, pause/resume |

### Agents (3 specialists)

Expand Down Expand Up @@ -264,7 +264,7 @@ bun run build:from-skills # Stage 4: Resolve template includes
- AI SDK v6 (Agents, MCP, DevTools, Reranking, Image Editing)
- AI Elements (pre-built React components for AI interfaces)
- Chat SDK (multi-platform chat bots — Slack, Telegram, Teams, Discord)
- Workflow DevKit (DurableAgent, Worlds, open source)
- Workflow SDK (DurableAgent, Worlds, open source)
- AI Gateway (100+ models, provider routing, cost tracking)
- Vercel Functions (Fluid Compute, streaming, Cron Jobs)
- Storage (Blob, Edge Config, Neon Postgres, Upstash Redis)
Expand Down
6 changes: 3 additions & 3 deletions agents/ai-architect.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ What does the AI feature need to do?
│ ├─ Single tool call → `generateText` with `tools` parameter
│ ├─ Multi-step reasoning with tools → AI SDK `ToolLoopAgent` class
│ │ ├─ Short-lived (< 60s) → Agent in Route Handler
│ │ └─ Long-running (minutes to hours) → Workflow DevKit `DurableAgent`
│ │ └─ Long-running (minutes to hours) → Workflow SDK `DurableAgent`
│ └─ MCP server integration → `@ai-sdk/mcp` StreamableHTTPClientTransport
├─ Process files / images / audio
Expand All @@ -34,7 +34,7 @@ What does the AI feature need to do?
│ └─ Generate with context → `generateText` with retrieved chunks in prompt
└─ Multi-agent system
├─ Agents share context? → Workflow DevKit `Worlds` (shared state)
├─ Agents share context? → Workflow SDK `Worlds` (shared state)
├─ Independent agents? → Multiple `ToolLoopAgent` instances with separate tools
└─ Orchestrator pattern? → Parent Agent delegates to child Agents via tools
```
Expand Down Expand Up @@ -368,7 +368,7 @@ Use when: Chat that can take actions (search, CRUD, calculations).
### Pattern 3: Background Agent

```
Client → Route Handler → Workflow DevKit (DurableAgent)
Client → Route Handler → Workflow SDK (DurableAgent)
↓ ↓ tool calls
Returns runId External APIs / DB
↓ ↓
Expand Down
6 changes: 3 additions & 3 deletions agents/ai-architect.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ What does the AI feature need to do?
│ ├─ Single tool call → `generateText` with `tools` parameter
│ ├─ Multi-step reasoning with tools → AI SDK `ToolLoopAgent` class
│ │ ├─ Short-lived (< 60s) → Agent in Route Handler
│ │ └─ Long-running (minutes to hours) → Workflow DevKit `DurableAgent`
│ │ └─ Long-running (minutes to hours) → Workflow SDK `DurableAgent`
│ └─ MCP server integration → `@ai-sdk/mcp` StreamableHTTPClientTransport
├─ Process files / images / audio
Expand All @@ -34,7 +34,7 @@ What does the AI feature need to do?
│ └─ Generate with context → `generateText` with retrieved chunks in prompt
└─ Multi-agent system
├─ Agents share context? → Workflow DevKit `Worlds` (shared state)
├─ Agents share context? → Workflow SDK `Worlds` (shared state)
├─ Independent agents? → Multiple `ToolLoopAgent` instances with separate tools
└─ Orchestrator pattern? → Parent Agent delegates to child Agents via tools
```
Expand Down Expand Up @@ -165,7 +165,7 @@ Use when: Chat that can take actions (search, CRUD, calculations).
### Pattern 3: Background Agent

```
Client → Route Handler → Workflow DevKit (DurableAgent)
Client → Route Handler → Workflow SDK (DurableAgent)
↓ ↓ tool calls
Returns runId External APIs / DB
↓ ↓
Expand Down
2 changes: 1 addition & 1 deletion agents/deployment-expert.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Build failed?
├─ Long-running task?
│ ├─ Under 5 min → Use Fluid Compute with streaming
│ ├─ Up to 15 min → Use Vercel Functions with `maxDuration` in vercel.json
│ └─ Hours/days → Use Workflow DevKit (DurableAgent or workflow steps)
│ └─ Hours/days → Use Workflow SDK (DurableAgent or workflow steps)
└─ DB query slow? → Add connection pooling, check cold start, use Edge Config
```

Expand Down
26 changes: 14 additions & 12 deletions generated/skill-manifest.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-07-31T18:16:05.834Z",
"generatedAt": "2026-08-05T16:45:57.300Z",
"version": 2,
"skills": {
"access-protected-vercel-deployment": {
Expand Down Expand Up @@ -586,7 +586,7 @@
{
"pattern": "DurableAgent|use workflow|use step|from\\s+[''\"]workflow[''\"]|@workflow/",
"targetSkill": "workflow",
"message": "Workflow DevKit pattern detected in AI code — loading WDK guidance for durable agent execution, step isolation, and crash-safe orchestration.",
"message": "Workflow SDK pattern detected in AI code — loading Workflow SDK guidance for durable agent execution, step isolation, and crash-safe orchestration.",
"skipIfFileContains": "createWorkflow|withWorkflow"
},
{
Expand Down Expand Up @@ -1236,7 +1236,7 @@
{
"pattern": "setTimeout\\s*\\(|setInterval\\s*\\(|while\\s*\\(\\s*true",
"targetSkill": "workflow",
"message": "Long-running or polling logic in chat bot — loading Workflow DevKit for durable execution that survives deploys.",
"message": "Long-running or polling logic in chat bot — loading Workflow SDK for durable execution that survives deploys.",
"skipIfFileContains": "use workflow|from\\s+[''\"]workflow[''\"]"
},
{
Expand Down Expand Up @@ -3685,11 +3685,11 @@
},
{
"pattern": "maxRetries\\s*[=:]|retryCount\\s*[=:]|retry\\s*\\(\\s*|for\\s*\\([^)]*retry|while\\s*\\([^)]*retry",
"message": "Manual retry logic detected. Use Vercel Workflow DevKit for automatic retries with durable execution.",
"message": "Manual retry logic detected. Use Vercel Workflow SDK for automatic retries with durable execution.",
"severity": "recommended",
"skipIfFileContains": "use workflow|use step|@vercel/workflow|from\\s+[''\"\"](workflow)[''\"\"]",
"upgradeToSkill": "workflow",
"upgradeWhy": "Replace manual retry loops with Workflow DevKit steps that provide automatic retries, crash safety, and observability.",
"upgradeWhy": "Replace manual retry loops with Workflow SDK steps that provide automatic retries, crash safety, and observability.",
"upgradeMode": "soft"
},
{
Expand All @@ -3711,7 +3711,7 @@
{
"pattern": "setTimeout\\s*\\(|setInterval\\s*\\(|await\\s+new\\s+Promise\\s*\\([^)]*setTimeout",
"targetSkill": "workflow",
"message": "Long-running or polling logic in serverless handler — loading Workflow DevKit for durable execution."
"message": "Long-running or polling logic in serverless handler — loading Workflow SDK for durable execution."
},
{
"pattern": "writeFile(Sync)?\\(|createWriteStream\\(|from\\s+[''\\\"](multer|formidable)[''\"]|fs\\.writeFile",
Expand All @@ -3731,7 +3731,7 @@
{
"pattern": "while\\s*\\(\\s*true\\s*\\)\\s*\\{|for\\s*\\(\\s*;\\s*;\\s*\\)\\s*\\{|setInterval\\s*\\(\\s*async",
"targetSkill": "workflow",
"message": "Polling loop in serverless function detected — loading Workflow DevKit for durable, crash-safe execution with pause/resume.",
"message": "Polling loop in serverless function detected — loading Workflow SDK for durable, crash-safe execution with pause/resume.",
"skipIfFileContains": "use workflow|use step|from\\\\s+['\\\"]workflow['\\\"]"
},
{
Expand All @@ -3749,7 +3749,7 @@
{
"pattern": "maxRetries\\s*[=:]|retryCount\\s*[=:]|retry\\s*\\(\\s*|for\\s*\\([^)]*retry|while\\s*\\([^)]*retry",
"targetSkill": "workflow",
"message": "Manual retry logic in serverless handler — loading Workflow DevKit guidance for automatic retries with durable execution.",
"message": "Manual retry logic in serverless handler — loading Workflow SDK guidance for automatic retries with durable execution.",
"skipIfFileContains": "use workflow|use step|@vercel/workflow|from\\s+[''\"\"](workflow)[''\"\"]"
}
],
Expand Down Expand Up @@ -4443,7 +4443,7 @@
"summary": "",
"docs": [
"https://vercel.com/docs/workflow",
"https://useworkflow.dev"
"https://workflow-sdk.dev"
],
"sitemap": "https://vercel.com/sitemap/docs.xml",
"pathPatterns": [
Expand Down Expand Up @@ -4530,7 +4530,7 @@
},
{
"pattern": "from\\s+['\"]@vercel/workflow['\"]",
"message": "Workflow DevKit requires AI Gateway OIDC setup — ensure vercel link + vercel env pull for VERCEL_OIDC_TOKEN",
"message": "Workflow SDK requires AI Gateway OIDC setup — ensure vercel link + vercel env pull for VERCEL_OIDC_TOKEN",
"severity": "recommended"
},
{
Expand All @@ -4541,7 +4541,7 @@
},
{
"pattern": "context\\.run\\s*\\(",
"message": "context.run() is not a WDK pattern — use \"use step\" directive for retryable, observable steps",
"message": "context.run() is not a Workflow SDK pattern — use \"use step\" directive for retryable, observable steps",
"severity": "error",
"upgradeToSkill": "workflow",
"upgradeWhy": "Guides migration from context.run() to the \"use step\" directive for durable, retryable workflow steps.",
Expand Down Expand Up @@ -4611,7 +4611,7 @@
{
"pattern": "process\\.env\\.(OPENAI_API_KEY|ANTHROPIC_API_KEY)|from\\s+[''\"]@ai-sdk/(anthropic|openai)[''\"\"]",
"targetSkill": "ai-gateway",
"message": "Direct provider API key in workflow — loading AI Gateway guidance for OIDC auth (required for WDK AI steps).",
"message": "Direct provider API key in workflow — loading AI Gateway guidance for OIDC auth (required for Workflow SDK AI steps).",
"skipIfFileContains": "gateway\\(|@ai-sdk/gateway|VERCEL_OIDC"
},
{
Expand All @@ -4630,6 +4630,7 @@
"promptSignals": {
"phrases": [
"vercel workflow",
"workflow sdk",
"workflow devkit",
"durable workflow",
"durable execution",
Expand Down Expand Up @@ -5227,6 +5228,7 @@
"implement step function"
],
"entities": [
"Workflow SDK",
"Workflow DevKit",
"WDK",
"step",
Expand Down
2 changes: 1 addition & 1 deletion skills/ai-sdk/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ chainTo:
-
pattern: 'DurableAgent|use workflow|use step|from\s+[''"]workflow[''"]|@workflow/'
targetSkill: workflow
message: 'Workflow DevKit pattern detected in AI code — loading WDK guidance for durable agent execution, step isolation, and crash-safe orchestration.'
message: 'Workflow SDK pattern detected in AI code — loading Workflow SDK guidance for durable agent execution, step isolation, and crash-safe orchestration.'
skipIfFileContains: 'createWorkflow|withWorkflow'
-
pattern: "from\\s+['\"]langchain['\"]|from\\s+['\"]@langchain/"
Expand Down
2 changes: 1 addition & 1 deletion skills/ai-sdk/overlay.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ chainTo:
-
pattern: 'DurableAgent|use workflow|use step|from\s+[''"]workflow[''"]|@workflow/'
targetSkill: workflow
message: 'Workflow DevKit pattern detected in AI code — loading WDK guidance for durable agent execution, step isolation, and crash-safe orchestration.'
message: 'Workflow SDK pattern detected in AI code — loading Workflow SDK guidance for durable agent execution, step isolation, and crash-safe orchestration.'
skipIfFileContains: 'createWorkflow|withWorkflow'
-
pattern: "from\\s+['\"]langchain['\"]|from\\s+['\"]@langchain/"
Expand Down
2 changes: 1 addition & 1 deletion skills/chat-sdk/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ chainTo:
-
pattern: 'setTimeout\s*\(|setInterval\s*\(|while\s*\(\s*true'
targetSkill: workflow
message: 'Long-running or polling logic in chat bot — loading Workflow DevKit for durable execution that survives deploys.'
message: 'Long-running or polling logic in chat bot — loading Workflow SDK for durable execution that survives deploys.'
skipIfFileContains: 'use workflow|from\s+[''"]workflow[''"]'
-
pattern: 'process\.env\.(OPENAI_API_KEY|ANTHROPIC_API_KEY)|from\s+[''"]@ai-sdk/(anthropic|openai)[''""]'
Expand Down
2 changes: 1 addition & 1 deletion skills/chat-sdk/overlay.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ chainTo:
-
pattern: 'setTimeout\s*\(|setInterval\s*\(|while\s*\(\s*true'
targetSkill: workflow
message: 'Long-running or polling logic in chat bot — loading Workflow DevKit for durable execution that survives deploys.'
message: 'Long-running or polling logic in chat bot — loading Workflow SDK for durable execution that survives deploys.'
skipIfFileContains: 'use workflow|from\s+[''"]workflow[''"]'
-
pattern: 'process\.env\.(OPENAI_API_KEY|ANTHROPIC_API_KEY)|from\s+[''"]@ai-sdk/(anthropic|openai)[''""]'
Expand Down
Loading
Loading