feat(sdk): implement the outbound Evaluator v2 worker runtime - #758

Open

SiddarthAA wants to merge 20 commits into

mainfrom

evaluator

Commits on Sep 1, 2026

docs(sdk): clarify evaluator v2 boundary

SiddarthAA
authored and
chhhee10
committed
2e05762
feat(sdk): add evaluator v2 worker runtime
chhhee10
committed
fa8a3d2
fix(evaluator): resume only unfinished replayed runs
chhhee10
committed
34b99cd
feat(sdk): run hosted evaluator definitions
chhhee10
committed
afa0053
fix(evaluator): harden managed source sandbox and contain poison definitions

chhhee10
and
claude
committed
e9701df
feat(evaluator): normal short polling instead of long-poll in the worker

chhhee10
and
claude
committed
b6f5eb8
fix(evaluator): sandbox managed source in a killable process (hermes SEC-001)

chhhee10
and
claude
committed
7b41346
fix(evaluator): fail closed when the fork sandbox is unavailable (hermes SEC-001)

chhhee10
and
claude
committed
97ce938
fix(evaluator): fork+exec the sandbox + clamp the timeout (hermes SEC-001)

chhhee10
and
claude
committed
bc96eda
fix(evaluator): bound the sandbox result on both sides (hermes SEC-001)

chhhee10
and
claude
committed
70b5224
fix(evaluator): bound aggregate sandbox memory (hermes SEC-001)

chhhee10
and
claude
committed
db6bd9e
Harden evaluator v2 sandbox; fix condition selection and error mirror

chhhee10
and
claude
committed
fb13990
Count sandbox-slot wait against the execution timeout (hermes SEC-001)

chhhee10
and
claude
committed
34610c3
fix(evaluator): report why source was rejected; scrub the sandbox child's env

authored and
chhhee10
committed
caf63b6
test(evaluator): give the condition compute bomb a margin that survives a fast interpreter

authored and
chhhee10
committed
fedcc75
fp-cloud-cli: mirror the new evaluations:run permission

chhhee10
and
claude
committed
bdf3a8f
sdk(evaluator): fail closed when kernel resource limits are unavailable

chhhee10
and
claude
committed
4f5a9a4
sdk(evaluator): bound the condition phase by the lease; make sync-eval thread leaks observable

chhhee10
and
claude
committed
da51e59
sdk(evaluator): let a managed comprehension read session on Python 3.10 (hermes COR-001)

chhhee10
and
claude
committed
16a85a8
deps: bump browserslist to 4.28.8 to clear GHSA-73wf-gq98-2v4g (OSV gate)

chhhee10
and
claude
committed
8cfdb59

Navigation Menu

Uh oh!

feat(sdk): implement the outbound Evaluator v2 worker runtime - #758

feat(sdk): implement the outbound Evaluator v2 worker runtime#758
SiddarthAA wants to merge 20 commits into
mainfrom
evaluator

Commits on Sep 1, 2026

docs(sdk): clarify evaluator v2 boundary

feat(sdk): add evaluator v2 worker runtime

fix(evaluator): resume only unfinished replayed runs

feat(sdk): run hosted evaluator definitions

fix(evaluator): harden managed source sandbox and contain poison definitions

feat(evaluator): normal short polling instead of long-poll in the worker

fix(evaluator): sandbox managed source in a killable process (hermes SEC-001)

fix(evaluator): fail closed when the fork sandbox is unavailable (hermes SEC-001)

fix(evaluator): fork+exec the sandbox + clamp the timeout (hermes SEC-001)

fix(evaluator): bound the sandbox result on both sides (hermes SEC-001)

fix(evaluator): bound aggregate sandbox memory (hermes SEC-001)

Harden evaluator v2 sandbox; fix condition selection and error mirror

Count sandbox-slot wait against the execution timeout (hermes SEC-001)

fix(evaluator): report why source was rejected; scrub the sandbox child's env

test(evaluator): give the condition compute bomb a margin that survives a fast interpreter

fp-cloud-cli: mirror the new evaluations:run permission

sdk(evaluator): fail closed when kernel resource limits are unavailable

sdk(evaluator): bound the condition phase by the lease; make sync-eval thread leaks observable

sdk(evaluator): let a managed comprehension read session on Python 3.10 (hermes COR-001)

deps: bump browserslist to 4.28.8 to clear GHSA-73wf-gq98-2v4g (OSV gate)