Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 39 additions & 7 deletions docs/content/docs/architecture.mdx
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
---
title: "Architecture"
description: "The states an agent moves through, and what survives sleep, crashes, and deploys."
description: "The states an agent moves through, and what survives sleep, crashes, and upgrades."
---

This page describes [`pi()`](/agents/docs/pi). A [`piDurable()`](/agents/docs/pi-durable) agent sleeps and drains the same way, and differs where noted in [Pi Durable](#pi-durable).

## States

An agent is idle until a prompt starts a run, and goes back to idle when the run ends or is cancelled. When the context fills up during a run, the agent compacts it and keeps going.
Expand Down Expand Up @@ -47,10 +49,10 @@ An agent is idle until a prompt starts a run, and goes back to idle when the run

## Sleep

An idle agent sleeps and frees its memory. The next action on its key wakes it: the agent reloads its session and reconnects to the same sandbox, so clients don't do anything different. While the agent sleeps, its sandbox is suspended.
An idle agent sleeps and frees its memory. The next action on its key wakes it: the agent reloads its session, and reconnects to the same sandbox the first time a tool needs it, so clients don't do anything different. While the agent sleeps, its sandbox is suspended.

<div style="overflow-x:auto">
<svg viewBox="0 0 640 280" role="img" aria-label="The agent loop runs in memory and saves each step to the Actor&#x27;s SQLite database, which also holds the sandbox id. On wake, the loop is restored from SQLite and reconnects to the same sandbox. Sleep, a crash, or a deploy clears memory, and SQLite is kept." style="width:100%;min-width:520px;max-width:640px;height:auto;display:block;margin:2.5rem auto;font-family:system-ui,sans-serif">
<svg viewBox="0 0 640 280" role="img" aria-label="The agent loop runs in memory and saves each step to the Actor&#x27;s SQLite database, which also holds the sandbox id. On wake, the loop is restored from SQLite and reconnects to the same sandbox. Sleep, a crash, or an upgrade clears memory, and SQLite is kept." style="width:100%;min-width:520px;max-width:640px;height:auto;display:block;margin:2.5rem auto;font-family:system-ui,sans-serif">
<defs>
<marker id="arch-durability-arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0 0 L10 5 L0 10 z" fill="rgb(var(--site-ink, 27 25 22))"/></marker>
</defs>
Expand All @@ -75,17 +77,47 @@ An idle agent sleeps and frees its memory. The next action on its key wakes it:
<text x="273" y="151.5" text-anchor="start" font-size="12" fill="#56524a">restore on wake</text>
<text x="434" y="87" text-anchor="middle" font-size="12" fill="#56524a">tool calls</text>
<text x="434" y="110" text-anchor="middle" font-size="12" fill="#56524a">reconnect by id</text>
<text x="70" y="272" text-anchor="start" font-size="12" fill="#56524a">Sleep, a crash, or a deploy clears memory. SQLite is kept.</text>
<text x="70" y="272" text-anchor="start" font-size="12" fill="#56524a">Sleep, a crash, or an upgrade clears memory. SQLite is kept.</text>
</svg>
</div>

## What survives

| | Session | Sandbox |
| --- | --- | --- |
| Sleep | Every message so far. | Reconnected on wake. |
| Crash | Every message written before the crash. | Reconnected on wake. |
| Deploy | Every message so far. Old Actors get up to 30 minutes to finish before the agent wakes on the new code. See [Versions](/docs/versions). | Reconnected on wake. |
| Sleep | Every message so far. | Reconnected on first use. |
| Crash | Every message written before the crash. A run in progress resumes on wake if Pi can continue it. | Reconnected on first use. |
| Upgrade | Every message so far. A run in progress gets time to finish first. See [Upgrades](#upgrades). | Reconnected on first use. |
| Destroy | Deleted. | Deleted. |

A model chosen with `setModel` is restored too, if it's still allowed. If the sandbox no longer exists when the agent wakes, a new one is created and the old files are lost.

Actions that don't touch the sandbox, such as `getMessages` or `setModel`, don't connect to it. They keep working while the sandbox provider is down.

## Upgrades

When you ship new code, an agent in the middle of a run lets it finish, for up to `sleepGracePeriod` (15 minutes by default), before it restarts on the new version. See [Versions](/docs/versions).

A run that is still going when the grace period ends, or that a crash cuts off, resumes when the agent wakes if Pi can continue it from the saved messages:

| Stopped while | Resumes? |
| --- | --- |
| The model was answering | Yes. The model is asked again, and the partial answer is lost. |
| Parallel tool calls ran and at least one finished | Yes. Calls that didn't finish get a "No result provided" result. |
| A tool call ran and none finished | No. |
| The run was cancelled with `abort` | No. |

A run that doesn't resume is left as if it had been cancelled, and the next prompt continues the conversation. A client that was waiting on `prompt` gets an error. It can follow the resumed run through events, or read it with `getMessages`.

Nothing runs while a crashed Actor is down. The run resumes when the Actor starts again.

## Pi Durable

A [`piDurable()`](/agents/docs/pi-durable) agent saves every step of a run, so it doesn't depend on what the saved messages look like:

- **Runs carry on from the last saved step.** Model calls, tool calls, and tasks your tools launch each save their progress. After a crash, the Actor reads that record and carries on. A plain tool call that was running isn't rerun unless it's marked `replay: "safe"`, and the model learns it never completed.
- **Long waits don't keep the Actor awake.** During a long retry or poll wait, the Actor sleeps and wakes when the wait ends.
- **Streams come back on wake.** A client that is still connected gets a fresh snapshot.
- **Destroy stops at once**, without waiting for running work.

See [Known limitations](/agents/docs/pi-durable#known-limitations) for what this doesn't cover.
2 changes: 1 addition & 1 deletion docs/content/docs/client-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ A listener added with `on("event", ...)` receives every event of the session as
## Results and errors

- The `prompt` action resolves when the run ends. Read the reply with `getLastAssistantText` or `getMessages`.
- The `prompt` action rejects with an `ActorError` when the action itself fails. For example, another prompt is already running (pass `streamingBehavior` to queue it instead), the model has no credential (`model_unavailable`), or the run passes the ten-minute action timeout (`action_timed_out`).
- The `prompt` action rejects with an `ActorError` when the action itself fails. For example, another prompt is already running (pass `streamingBehavior` to queue it instead), the model has no credential (`model_unavailable`), or the run passes the ten-minute action timeout (`action_timed_out`). A timeout only stops the wait. The run keeps going until it finishes or `abort` stops it.
- A model error doesn't reject. The run ends with an assistant message whose `stopReason` is `"error"`, and `errorMessage` explains why.

## Browsers
Expand Down
56 changes: 54 additions & 2 deletions docs/content/docs/design-patterns.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Design Patterns"
description: "Patterns for agent keys, subagents, agent-to-agent messages, workflows, schedules, and shared credentials."
description: "Patterns for agent keys, subagents, agent-to-agent messages, workflows, schedules, background jobs, and shared credentials."
---

Each agent is an Actor, so agent apps use the same building blocks as the rest of Rivet: keys pick the agent, and agents call other Actors.
Expand Down Expand Up @@ -49,6 +49,8 @@ The key decides which conversation a prompt continues. Pick it from what the age

Keys are arrays, so a channel id or repository name from a webhook can't break the key's structure. See [Actor Keys](/actors/docs/keys).

A [`pi()`](/agents/docs/pi) agent holds one conversation, so the key is the conversation. A [`piDurable()`](/agents/docs/pi-durable) agent can hold many conversations and forks, which lets related conversations share one Actor, one sandbox, and its documents.

## Lead Agent with Subagents

A lead agent hands a focused task to a specialist with its own tools and waits for its answer. The specialist starts with a fresh context, so only the answer comes back.
Expand Down Expand Up @@ -81,6 +83,8 @@ A lead agent hands a focused task to a specialist with its own tools and waits f
<CodeSnippet file="examples/docs/subagents/client.ts" title="client.ts" />
</CodeGroup>

`get-order.ts` is the `get_order` tool from [Custom Tools](/agents/docs/custom-tools).

See [Subagents](/agents/docs/subagents).

## Agents Messaging Each Other
Expand Down Expand Up @@ -134,6 +138,10 @@ When the work has fixed steps, waits, or retries, let a workflow drive the agent
</svg>
</div>

```sh
npm add @rivet-dev/workflows
```

<CodeGroup>
<CodeSnippet file="examples/docs/workflows/server.ts" title="server.ts" />
<CodeSnippet file="examples/docs/workflows/client.ts" title="client.ts" />
Expand Down Expand Up @@ -170,6 +178,43 @@ An agent can wake itself on a schedule and sleep between runs.

See [Schedules](/agents/docs/schedules).

## Background Agents That Must Finish

When nobody is watching a run, such as an agent started by a webhook, a crash must not lose the work or repeat a side effect. Use [Pi Durable](/agents/docs/pi-durable): every model request and tool call is a durable task that continues where it stopped, and a redelivered webhook is ignored because of its `requestId`.

<div style="overflow-x:auto">
<svg viewBox="0 0 600 130" role="img" aria-label="A GitHub webhook submits the issue to the fixer agent with the delivery id as requestId. The fixer agent opens a pull request, a tool that is never repeated after a crash." style="width:100%;min-width:520px;max-width:600px;height:auto;display:block;margin:2.5rem auto;font-family:system-ui,sans-serif">
<defs>
<marker id="dp-background-arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0 0 L10 5 L0 10 z" fill="rgb(var(--site-ink, 27 25 22))"/></marker>
</defs>
<rect x="20" y="40" width="150" height="52" rx="7" fill="#ffffff" stroke="#1b1916" stroke-width="1.4"/>
<rect x="240" y="40" width="150" height="52" rx="7" fill="#c7e4fb" stroke="#3d9df3" stroke-width="2"/>
<rect x="440" y="40" width="140" height="52" rx="7" fill="#ffffff" stroke="#1b1916" stroke-width="1.4"/>
<g stroke="rgb(var(--site-ink, 27 25 22))" stroke-width="1.4" fill="none">
<path d="M171 66 L238 66" marker-end="url(#dp-background-arrow)"/>
<path d="M391 66 L438 66" marker-end="url(#dp-background-arrow)"/>
</g>
<text x="95" y="63" text-anchor="middle" font-size="14" font-weight="600" fill="#1b1916">GitHub webhook</text>
<text x="95" y="80" text-anchor="middle" font-size="12" fill="#56524a">requestId: delivery</text>
<text x="315" y="63" text-anchor="middle" font-size="14" font-weight="600" fill="#1b1916">Fixer agent</text>
<text x="315" y="80" text-anchor="middle" font-size="12" fill="#56524a">key: repo, issue</text>
<text x="510" y="63" text-anchor="middle" font-size="14" font-weight="600" fill="#1b1916">Open PR</text>
<text x="510" y="80" text-anchor="middle" font-size="12" fill="#56524a">never repeated</text>
<text x="204.5" y="58" text-anchor="middle" font-size="12" fill="#56524a">submit</text>
</svg>
</div>

<CodeGroup>
<CodeSnippet file="examples/docs/design-patterns/background-job/server.ts" title="server.ts" />
<CodeSnippet file="examples/docs/design-patterns/background-job/client.ts" title="client.ts" />
</CodeGroup>

- `conversation.submit` returns once the input is saved, so the webhook answers right away while the agent works.
- `open_pull_request` has no `replay`, so a call cut off by a crash isn't repeated. The model is told the call was interrupted. Mark only tools that are safe to run twice as `replay: "safe"`.
- When you ship new code, the Actor lets the run finish, for up to `sleepGracePeriod`, before it restarts on the new version. Anything still running then carries on from its last saved step.

See [Pi Durable](/agents/docs/pi-durable).

## Credentials Shared per Tenant

Start agent keys with the tenant id, and pick the `credentials` Actor from it. Every agent in a tenant shares one set of model logins, and never sees another tenant's.
Expand Down Expand Up @@ -199,7 +244,7 @@ Start agent keys with the tenant id, and pick the `credentials` Actor from it. E

<CodeGroup>
<CodeSnippet file="examples/docs/design-patterns/tenant-credentials/server.ts" title="server.ts" />
<CodeSnippet file="examples/docs/user-subscriptions/credentials.ts" title="credentials.ts" />
<CodeSnippet file="examples/docs/design-patterns/tenant-credentials/credentials.ts" title="credentials.ts" />
<CodeSnippet file="examples/docs/design-patterns/tenant-credentials/client.ts" title="client.ts" />
</CodeGroup>

Expand All @@ -223,6 +268,12 @@ A new key creates a new agent with an empty session, so the agent forgets the co

**Solution:** Reuse the key of the conversation the message belongs to.

### Unattended Runs on `pi()`

A `pi()` agent started by a webhook or schedule has nobody to ask it to continue. If a crash cuts off a tool call, the run doesn't resume. If the webhook is redelivered, the agent is prompted twice, and a tool with side effects can run twice.

**Solution:** Use [Pi Durable](/agents/docs/pi-durable) for runs nobody watches, and pass a `requestId` with every submission. See [Background Agents That Must Finish](#background-agents-that-must-finish).

## Where to Go Next

| Goal | Read |
Expand All @@ -231,6 +282,7 @@ A new key creates a new agent with an empty session, so the agent forgets the co
| Give the agent your own APIs | [Custom Tools](/agents/docs/custom-tools) |
| Let the agent run commands and edit files | [Sandboxes](/agents/docs/sandboxes), then [Built-in Tools](/agents/docs/built-in-tools) |
| Run agents on your users' own subscriptions | [User Subscriptions](/agents/docs/user-subscriptions) |
| Make runs finish after a crash, with no one to retry them | [Pi Durable](/agents/docs/pi-durable) |
| Wait for a person before a risky action | [Human in the Loop](/agents/docs/human-in-the-loop) |
| Share a result through a link | [Scoped Access with JWTs](/agents/docs/scoped-access) |
| Talk to users in Slack, Linear, GitHub, or Discord | [Slack](/agents/docs/connectors/slack) and the other connectors |
Expand Down
4 changes: 2 additions & 2 deletions docs/content/docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Each agent on Rivet is one [Rivet Actor](/actors/docs). You define the agent onc

- **The agent loop runs in the Actor.** Model calls, retries, and the conversation run in your backend, next to your application code, where you control credentials, permissions, and logging.
- **Tools run in a separate sandbox.** File and shell tools call into a sandbox such as E2B or Daytona. Nothing the agent runs executes on your worker, and your provider keys never enter the sandbox.
- **Agents are durable.** The Actor saves the session to its SQLite database and restores it when it wakes, so an idle agent can sleep, and a crash or deploy does not lose the conversation.
- **Agents are durable.** The Actor saves the session to its SQLite database and restores it when it wakes, so an idle agent can sleep, and a crash or upgrade does not lose the conversation.

<CardGroup>
<Card title="Quickstart" href="/agents/docs/quickstart">
Expand Down Expand Up @@ -111,7 +111,7 @@ Each agent on Rivet is one [Rivet Actor](/actors/docs). You define the agent onc
Trace every agent run with OpenTelemetry.
</Card>
<Card title="Architecture" href="/agents/docs/architecture">
The states an agent moves through, and what survives sleep, crashes, and deploys.
The states an agent moves through, and what survives sleep, crashes, and upgrades.
</Card>
<Card title="Security Model" href="/agents/docs/security-model">
What reaches the sandbox, where secrets live, and who can connect to an agent.
Expand Down
Loading
Loading