Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Changelog

## 0.133.6

- The direct protected model-grant port accepts an optional caller-declared `maxTotalTokens` cap across input and output tokens.
- Bind that cap into the exact reservation response and reject a final ledger that exceeds it or omits token usage.
- Signed execution plans in the Interface 0.47 cohort remain unchanged; only the direct protected model-grant port accepts this field.

## 0.133.3

- Preserve the provider-served model snapshot in official optimizer cost receipts so Eval can match the response, receipt, and execution evidence.
Expand Down
4 changes: 4 additions & 0 deletions bench/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## 0.8.8

- Verify candidate execution against Runtime 0.133.6 and its terminal aggregate-token integrity result.

## 0.8.7

- Consume Runtime 0.132.11, Eval 0.145.2, Interface 0.47.0, Knowledge 7.2.4, and Sandbox 0.21.1 as one compatible dependency set.
Expand Down
2 changes: 1 addition & 1 deletion bench/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-bench",
"version": "0.8.7",
"version": "0.8.8",
"type": "module",
"description": "Benchmark adapters and execution for agent-runtime across coding, tool-use, RAG, memory, browser, and terminal tasks.",
"repository": {
Expand Down
1 change: 1 addition & 0 deletions bench/scripts/verify-pier-agent.mts
Original file line number Diff line number Diff line change
Expand Up @@ -512,6 +512,7 @@ ${proofArm === 'success' ? "(task / 'src/status.txt').write_text('ready\\nowner=
preparationId,
grantDigest: `sha256:${'c'.repeat(64)}`,
closed: true,
usageWithinLimits: true,
calls: [],
}),
},
Expand Down
6 changes: 6 additions & 0 deletions docs/api/candidate-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,12 @@ Re-exports [AgentCandidateProtectedModelSettlement](index.md#agentcandidateprote

***

### AgentCandidateProtectedModelSettlementCall

Re-exports [AgentCandidateProtectedModelSettlementCall](index.md#agentcandidateprotectedmodelsettlementcall)

***

### AgentCandidateProtectedRunCapture

Re-exports [AgentCandidateProtectedRunCapture](index.md#agentcandidateprotectedruncapture)
Expand Down
34 changes: 31 additions & 3 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2608,7 +2608,7 @@ Evaluator service must expire and revoke this reservation at this epoch millisec

> **enforcedLimits**: [`AgentCandidateModelLimits`](#agentcandidatemodellimits)

The gateway must stop calls before any one of these limits is exceeded.
The gateway must stop calls before any declared model limit is exceeded.

##### network

Expand Down Expand Up @@ -2646,9 +2646,15 @@ Injected only into the trusted executor after all pre-launch checks pass.

> **closed**: `true`

##### usageWithinLimits

> **usageWithinLimits**: `boolean`

Router's terminal integrity result. False must never become a receipt.

##### calls

> **calls**: readonly `AgentCandidateModelSettlementCall`[]
> **calls**: readonly [`AgentCandidateProtectedModelSettlementCall`](#agentcandidateprotectedmodelsettlementcall)[]

***

Expand Down Expand Up @@ -10597,10 +10603,32 @@ Secret-free response from the service's reservation endpoint.

### AgentCandidateModelLimits

> **AgentCandidateModelLimits** = `Pick`\<`AgentCandidateExecutionLimits`, `"maxModelCalls"` \| `"maxInputTokens"` \| `"maxOutputTokens"` \| `"maxCostUsd"`\>
> **AgentCandidateModelLimits** = `Pick`\<`AgentCandidateExecutionLimits`, `"maxModelCalls"` \| `"maxInputTokens"` \| `"maxOutputTokens"` \| `"maxCostUsd"`\> & `object`

Limits mechanically enforced by the evaluator-owned model gateway.

#### Type Declaration

##### maxTotalTokens?

> `optional` **maxTotalTokens?**: `number`

Optional caller-declared cap across input and output tokens.

***

### AgentCandidateProtectedModelSettlementCall

> **AgentCandidateProtectedModelSettlementCall** = `AgentCandidateModelSettlementCall` & `object`

Protected-port wire call with the gateway's counted input total preserved.

#### Type Declaration

##### accountedInputTokens

> **accountedInputTokens**: `number`

***

### AgentCandidateExecutorTaskOutcomeCapture
Expand Down
8 changes: 5 additions & 3 deletions docs/api/primitive-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@

# Primitive catalog — the never-stale anti-reinvention inventory

> **GENERATED** from `@tangle-network/agent-runtime@0.133.5` and `@tangle-network/agent-eval@0.145.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.
> **GENERATED** from `@tangle-network/agent-runtime@0.133.6` and `@tangle-network/agent-eval@0.145.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.

## 1. agent-runtime — own public surface

Every subpath this package declares in `package.json` `exports`. Reach for these before hand-rolling a loop, driver, conversation runner, optimizer wrapper, or observability shim.

### Root — task lifecycle, conversation, RSI verbs, observability

Import from `@tangle-network/agent-runtime` — 430 exports.
Import from `@tangle-network/agent-runtime` — 431 exports.

| Symbol | Kind | Summary |
|---|---|---|
Expand Down Expand Up @@ -241,6 +241,7 @@ Import from `@tangle-network/agent-runtime` — 430 exports.
| `AgentCandidateModelGrantRunReservationInput` | type | Reservation fields supplied by a caller before Runtime resolves the model. |
| `AgentCandidateModelLimits` | type | Limits mechanically enforced by the evaluator-owned model gateway. |
| `AgentCandidateProfileSource` | type | A complete profile that can be frozen without losing behavior. |
| `AgentCandidateProtectedModelSettlementCall` | type | Protected-port wire call with the gateway's counted input total preserved. |
| `AgentEvalErrorCode` | type | Error taxonomy for `@tangle-network/agent-eval`. |
| `AgenticGeneratorShotDisposition` | type | Worktree decision emitted before a completed shot is retried, accepted, or |
| `AgenticGeneratorShotExecution` | type | Runtime's exact terminal turn plus its complete normalized event stream. |
Expand Down Expand Up @@ -1275,7 +1276,7 @@ Import from `@tangle-network/agent-runtime/primeintellect` — 29 exports.

### Candidate execution — immutable prepare, run, grade, and receipt

Import from `@tangle-network/agent-runtime/candidate-execution` — 113 exports.
Import from `@tangle-network/agent-runtime/candidate-execution` — 114 exports.

| Symbol | Kind | Summary |
|---|---|---|
Expand Down Expand Up @@ -1351,6 +1352,7 @@ Import from `@tangle-network/agent-runtime/candidate-execution` — 113 exports.
| `AgentCandidateModelGrantRunReservationInput` | type | Reservation fields supplied by a caller before Runtime resolves the model. |
| `AgentCandidateModelLimits` | type | Limits mechanically enforced by the evaluator-owned model gateway. |
| `AgentCandidateProfileSource` | type | A complete profile that can be frozen without losing behavior. |
| `AgentCandidateProtectedModelSettlementCall` | type | Protected-port wire call with the gateway's counted input total preserved. |
| `PersistedTaskOutcomeEvidence` | type | Immutable evaluator evidence retained with a verified candidate task outcome. |
| `VerifiedAgentCandidateTaskOutcome` | type | Branded task outcome that has survived independent evaluator verification. |

Expand Down
2 changes: 1 addition & 1 deletion docs/canonical-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Generated signatures and the complete export list live in docs/api/.
Run pnpm docs:freshness after editing this file. -->

> **Version 0.133.5.**
> **Version 0.133.6.**
> [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path.
> `agent-eval` must satisfy `>=0.145.2 <0.146.0`.
> `sandbox` must satisfy `>=0.21.1 <0.22.0`.
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-runtime",
"version": "0.133.5",
"version": "0.133.6",
"description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.",
"homepage": "https://github.com/tangle-network/agent-runtime#readme",
"repository": {
Expand Down
1 change: 1 addition & 0 deletions src/candidate-execution/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ export {
type AgentCandidateProtectedModelActivation,
type AgentCandidateProtectedModelReservation,
type AgentCandidateProtectedModelSettlement,
type AgentCandidateProtectedModelSettlementCall,
type AgentCandidateProtectedRunCapture,
type AgentCandidateRepositoryPort,
type AgentCandidateRunFinalization,
Expand Down
24 changes: 20 additions & 4 deletions src/candidate-execution/model-settlement.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ const USD_NANOS = 1_000_000_000

export interface SealedAgentCandidateModelSettlement {
readonly value: AgentCandidateProtectedModelSettlement
/** `inputTokens` is the gateway's accounted input total. */
readonly usage: AgentCandidateFixedSpend
}

Expand All @@ -20,10 +21,13 @@ export function sealAgentCandidateModelSettlement(
): SealedAgentCandidateModelSettlement {
assertExactObjectKeys(
settlement,
['preparationId', 'grantDigest', 'closed', 'calls'],
['preparationId', 'grantDigest', 'closed', 'usageWithinLimits', 'calls'],
'model settlement',
)
if (settlement.closed !== true) throw new Error('protected model grant is not closed')
if (settlement.usageWithinLimits !== true) {
throw new Error('protected model settlement reports usage outside frozen limits')
}
if (settlement.grantDigest !== expected.grantDigest) {
throw new Error('protected model settlement grant digest does not match the reservation')
}
Expand All @@ -36,7 +40,7 @@ export function sealAgentCandidateModelSettlement(

const callIds = new Set<string>()
const spanIds = new Set<string>()
let inputTokens = 0
let accountedInputTokens = 0
let outputTokens = 0
let cachedInputTokens = 0
let reasoningTokens = 0
Expand All @@ -54,6 +58,7 @@ export function sealAgentCandidateModelSettlement(
'startedAtMs',
'endedAtMs',
'inputTokens',
'accountedInputTokens',
'outputTokens',
'cachedInputTokens',
'reasoningTokens',
Expand Down Expand Up @@ -87,6 +92,12 @@ export function sealAgentCandidateModelSettlement(
throw new Error(`model settlement call ${index} ended before it started`)
}
assertCount(source.inputTokens, `model settlement call ${index} inputTokens`)
assertCount(source.accountedInputTokens, `model settlement call ${index} accountedInputTokens`)
if (source.accountedInputTokens < source.inputTokens) {
throw new Error(
`model settlement call ${index} accountedInputTokens cannot be less than inputTokens`,
)
}
assertCount(source.outputTokens, `model settlement call ${index} outputTokens`)
assertCount(source.cachedInputTokens, `model settlement call ${index} cachedInputTokens`)
cachedInputTokens = safeAdd(
Expand All @@ -101,15 +112,19 @@ export function sealAgentCandidateModelSettlement(
throw new Error(`model settlement call ${index} has an invalid cost provenance`)
}
if (source.costProvenance === 'estimated') costProvenance = 'estimated'
inputTokens = safeAdd(inputTokens, source.inputTokens, 'input token total')
accountedInputTokens = safeAdd(
accountedInputTokens,
source.accountedInputTokens,
'accounted input token total',
)
outputTokens = safeAdd(outputTokens, source.outputTokens, 'output token total')
costUsdNanos = safeAdd(costUsdNanos, source.costUsdNanos, 'cost total')
return Object.freeze({ ...source })
})

const usage = Object.freeze({
costUsdNanos,
inputTokens,
inputTokens: accountedInputTokens,
outputTokens,
cachedInputTokens,
reasoningTokens,
Expand All @@ -121,6 +136,7 @@ export function sealAgentCandidateModelSettlement(
preparationId: settlement.preparationId,
grantDigest: settlement.grantDigest,
closed: true as const,
usageWithinLimits: true as const,
calls: Object.freeze(calls),
}),
usage,
Expand Down
2 changes: 2 additions & 0 deletions src/candidate-execution/outcome-evidence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ export async function persistCandidateModelSettlementEvidence(
grantDigest: settlement.value.grantDigest,
closed: true as const,
resolved: identity.resolvedModel,
// Interface 0.47 signed evidence has no protected-port proof fields.
// The direct port validates them first and carries accounted input in usage.
calls: settlement.value.calls.map((call) => ({
callId: call.callId,
generationId: call.generationId,
Expand Down
Loading