Skip to content

Commit f7f3f96

Browse files
committed
fix(provenance): let a run that never started report why it failed
A copilot-run workflow that fails before reaching the engine crossed back with no provenance, which latched the tool's registry and reduced the result to "result unavailable". The caller was told its run failed but not that the workflow was undeployed, or the input invalid, or the slot unavailable — the reasons this layer produces before any block runs, naming no secret because none had been resolved yet. The executor attaches its execution result to every throw, so the absence of one is proof that no block ran: output, logs and error are all undefined and the only content is a message this layer wrote. That is an absence, not an inability to vouch, so the crossing now carries an exact-empty envelope. The message still passes the tool boundary's egress projection against the same registry, so anything that registry knows is still redacted. A run that did execute and could not vouch hands back its incomplete envelope exactly as before, and that still latches. Make the attach total rather than conditional to keep that inference sound. A block failure is already normalized on the way in, so the old `instanceof Error` guard held in practice; what it did not give was a guarantee covering a non-Error raised by the engine's own synchronous work. toError is identity-preserving, so ordinary failures keep their type. The empty envelope moves to the registry module, which owns the vocabulary, replacing a private copy in the logging session so one definition states what "vouched for, naming nothing" is.
1 parent d7c9f51 commit f7f3f96

6 files changed

Lines changed: 149 additions & 19 deletions

File tree

‎apps/sim/executor/execution/engine.test.ts‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ import { EDGE } from '@/executor/constants'
2828
import type { DAG, DAGNode } from '@/executor/dag/builder'
2929
import type { EdgeManager } from '@/executor/execution/edge-manager'
3030
import type { NodeExecutionOrchestrator } from '@/executor/orchestrators/node'
31-
import type { ExecutionContext } from '@/executor/types'
31+
import type { ExecutionContext, ExecutionResult } from '@/executor/types'
3232
import { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry'
3333
import type { SerializedBlock } from '@/serializer/types'
3434
import { ExecutionEngine } from './engine'
@@ -275,6 +275,38 @@ describe('ExecutionEngine', () => {
275275
expect(provenance?.entries).toEqual([{ name: 'TOKEN', encryptedValue: 'ciphertext' }])
276276
})
277277

278+
/**
279+
* The crossing at the copilot boundary reads the absence of an attached result as "no block
280+
* ran". That inference is only total if every throw out of `run` carries one.
281+
*/
282+
it('attaches the execution result to every throw out of a failed run', async () => {
283+
const node = createMockNode('function-1', 'function')
284+
const registry = new ResolvedSecretTraceRegistry([
285+
{ name: 'TOKEN', plaintext: 'secret-value-1234', encryptedValue: 'ciphertext' },
286+
])
287+
registry.recordResolved('TOKEN', 'secret-value-1234')
288+
const context = createMockContext({
289+
decisions: { router: new Map(), condition: new Map() },
290+
resolvedSecretTraceRegistry: registry,
291+
})
292+
const nodeOrchestrator = createMockNodeOrchestrator()
293+
vi.mocked(nodeOrchestrator.executeNode).mockRejectedValue(new Error('block exploded'))
294+
295+
const engine = new ExecutionEngine(
296+
context,
297+
createMockDAG([node]),
298+
createMockEdgeManager(),
299+
nodeOrchestrator
300+
)
301+
302+
const thrown = await engine.run(node.id).catch((error: unknown) => error)
303+
304+
expect(thrown).toBeInstanceOf(Error)
305+
const attached = (thrown as Error & { executionResult?: ExecutionResult }).executionResult
306+
expect(attached).toBeDefined()
307+
expect(attached?.executionState?.resolvedSecretTraceProvenance).toBeDefined()
308+
})
309+
278310
/** Deriving must not weaken the guarantee: a latched registry still exports incomplete. */
279311
it('keeps the final output envelope incomplete when the registry latched', async () => {
280312
const node = createMockNode('loop-1', 'loop')

‎apps/sim/executor/execution/engine.ts‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -185,10 +185,17 @@ export class ExecutionEngine {
185185
metadata: this.context.metadata,
186186
}
187187

188-
if (error instanceof Error) {
189-
attachExecutionResult(error, executionResult)
190-
}
191-
throw error
188+
/**
189+
* Normalized first so the attach is total rather than conditional on the throw already
190+
* being an `Error`. A block failure is normalized on the way in, so the old guard held in
191+
* practice; what it did not give was a guarantee. The copilot crossing reads a missing
192+
* result as proof that no block ran, and that inference has to hold for every throw out of
193+
* here, including a non-`Error` raised by this file's own synchronous work. `toError`
194+
* returns an `Error` unchanged, so ordinary failures keep their identity and their type.
195+
*/
196+
const thrown = toError(error)
197+
attachExecutionResult(thrown, executionResult)
198+
throw thrown
192199
} finally {
193200
this.cleanup()
194201
}

‎apps/sim/executor/utils/resolved-secret-trace-registry.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,18 @@ export interface ResolvedSecretIncompletenessDiagnostics {
226226
export const ANONYMOUS_SECRET_TRACE_REPLACEMENT = OPAQUE_RESOLVED_SECRET_REPLACEMENT
227227
export const RESOLVED_SECRET_TRACE_CHECKPOINT_VERSION = 1
228228

229+
/**
230+
* The envelope for content no secret ever reached: vouched for, naming nothing.
231+
*
232+
* Distinct from an incomplete envelope, which says the opposite — that something may be carried
233+
* and cannot be named. A boundary that knows nothing was resolved should say so with this rather
234+
* than latch, since latching is the claim that redaction is impossible. Returned fresh so no
235+
* caller shares a value it may serialize or extend.
236+
*/
237+
export function emptyResolvedSecretTraceProvenance(): ResolvedSecretTraceProvenanceV1 {
238+
return { version: 1, complete: true, entries: [] }
239+
}
240+
229241
const MAX_PROVENANCE_ENTRIES = PROVENANCE_MAX_ENTRIES
230242
const MAX_SERIALIZED_PROVENANCE_BYTES = PROVENANCE_MAX_SERIALIZED_BYTES
231243
const MAX_TRACE_CATALOG_ENTRIES = PROVENANCE_MAX_ENTRIES

‎apps/sim/lib/logs/execution/logging-session.ts‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ import type { SerializableExecutionState } from '@/executor/execution/types'
4646
import type { BlockLog } from '@/executor/types'
4747
import { projectResolvedSecretDiagnosticError } from '@/executor/utils/resolved-secret-content-projection'
4848
import {
49+
emptyResolvedSecretTraceProvenance,
4950
isResolvedSecretTraceProvenanceV1,
5051
RESOLVED_SECRET_TRACE_CHECKPOINT_VERSION,
5152
type ResolvedSecretTraceProvenanceV1,
@@ -124,10 +125,6 @@ function getActiveBlockDisplayProvenance(
124125

125126
const logger = createLogger('LoggingSession')
126127

127-
function emptyResolvedSecretTraceProvenance(): ResolvedSecretTraceProvenanceV1 {
128-
return { version: 1, complete: true, entries: [] }
129-
}
130-
131128
type CompletionAttempt = 'complete' | 'error' | 'cancelled' | 'paused'
132129

133130
export interface SecretSafeDisplayContent {

‎apps/sim/lib/workflows/application/run-workflow-from-copilot.test.ts‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -391,4 +391,75 @@ describe('Copilot workflow run application commands', () => {
391391
expect(readAttemptedExecutionId(error)).toBeUndefined()
392392
})
393393
})
394+
395+
describe('failed-run provenance crossing', () => {
396+
function trackingLifecycle() {
397+
const importCrossingProvenance = vi.fn().mockResolvedValue(true)
398+
return {
399+
importCrossingProvenance,
400+
lifecycle: {
401+
resolvedSecretTraceRegistry: {
402+
exportProvenanceForValue: vi.fn(() => undefined),
403+
beginPendingActivation: vi.fn(() => vi.fn()),
404+
importCrossingProvenance,
405+
},
406+
},
407+
}
408+
}
409+
410+
async function runExpectingFailure(input: { lifecycle: unknown }) {
411+
await expect(
412+
runWorkflowFromCopilot.execute({
413+
principal,
414+
input: {
415+
workflowId: 'workflow-1',
416+
useDraftState: true,
417+
lifecycle: input.lifecycle,
418+
hasWorkflowInput: false,
419+
useMockPayload: true,
420+
},
421+
})
422+
).rejects.toThrow()
423+
}
424+
425+
/**
426+
* The executor attaches its result to every throw, so a failure without one never reached a
427+
* block. Nothing crossed, and saying so keeps the caller's tool result — and the reason its
428+
* run could not start — instead of reducing it to "result unavailable".
429+
*/
430+
it('vouches for a failure that never reached the engine', async () => {
431+
const { importCrossingProvenance, lifecycle: tracked } = trackingLifecycle()
432+
mocks.executeWorkflow.mockRejectedValueOnce(new Error('workflow is not deployed'))
433+
434+
await runExpectingFailure({ lifecycle: tracked })
435+
436+
expect(importCrossingProvenance).toHaveBeenCalledWith(
437+
{ version: 1, complete: true, entries: [] },
438+
expect.objectContaining({ thrownMessage: 'workflow is not deployed' }),
439+
expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' })
440+
)
441+
})
442+
443+
/** A run that did execute and could not vouch still hands back its incomplete envelope. */
444+
it('passes through an incomplete envelope from a run that did execute', async () => {
445+
const { importCrossingProvenance, lifecycle: tracked } = trackingLifecycle()
446+
const incomplete = { version: 1 as const, complete: false, entries: [] }
447+
const failure = Object.assign(new Error('block failed'), {
448+
executionResult: {
449+
success: false,
450+
output: { partial: true },
451+
executionState: { resolvedSecretTraceProvenance: incomplete },
452+
},
453+
})
454+
mocks.executeWorkflow.mockRejectedValueOnce(failure)
455+
456+
await runExpectingFailure({ lifecycle: tracked })
457+
458+
expect(importCrossingProvenance).toHaveBeenCalledWith(
459+
incomplete,
460+
expect.objectContaining({ output: { partial: true } }),
461+
expect.objectContaining({ origin: 'copilotWorkflowMutation.failedRunCrossing' })
462+
)
463+
})
464+
})
394465
})

‎apps/sim/lib/workflows/application/run-workflow-from-copilot.ts‎

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,14 @@ import {
2929
} from '@/lib/workflows/triggers/run-options'
3030
import type { SerializableExecutionState } from '@/executor/execution/types'
3131
import type { ExecutionResult } from '@/executor/types'
32-
import { attachAttemptedExecutionId } from '@/executor/utils/errors'
32+
import { attachAttemptedExecutionId, hasExecutionResult } from '@/executor/utils/errors'
3333

3434
const logger = createLogger('CopilotWorkflowRun')
3535

36-
import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry'
36+
import {
37+
emptyResolvedSecretTraceProvenance,
38+
type ResolvedSecretTraceRegistry,
39+
} from '@/executor/utils/resolved-secret-trace-registry'
3740

3841
export interface CopilotWorkflowRunLifecycle {
3942
billingAttribution?: BillingAttributionSnapshot
@@ -325,16 +328,24 @@ async function executeCopilotRun(params: {
325328
* as never started and invite the duplicate this id exists to prevent.
326329
*/
327330
if (registry) {
328-
const executionResult =
329-
typeof error === 'object' &&
330-
error !== null &&
331-
'executionResult' in error &&
332-
typeof error.executionResult === 'object'
333-
? (error.executionResult as ExecutionResult)
334-
: undefined
331+
const executionResult = hasExecutionResult(error) ? error.executionResult : undefined
335332
try {
333+
/**
334+
* A run that never reached the engine carried nothing back: the executor attaches its
335+
* result to every throw, so its absence means no block ran, and the three content fields
336+
* below are all undefined. The only content is `thrownMessage`, which this layer produced
337+
* — an admission, validation, or setup failure — and which the tool boundary still
338+
* projects against this registry before any of it reaches a model.
339+
*
340+
* Reporting that as unvouchable cost the caller the reason its run failed: the crossing
341+
* latched, and the tool result was reduced to "result unavailable" for a message that
342+
* named no secret because none had been resolved yet. An engine that did run and could
343+
* not vouch still hands back an incomplete envelope, and that still latches.
344+
*/
336345
await registry.importCrossingProvenance(
337-
executionResult?.executionState?.resolvedSecretTraceProvenance,
346+
executionResult
347+
? executionResult.executionState?.resolvedSecretTraceProvenance
348+
: emptyResolvedSecretTraceProvenance(),
338349
{
339350
output: executionResult?.output,
340351
logs: executionResult?.logs,

0 commit comments

Comments
 (0)