fix(provenance): classify a latched-but-empty registry's file writes as unrecorded (#7084)

* fix(provenance): classify a latched-but-empty registry's file writes as unrecorded

A copilot chat whose registry latched — the recurring producer is a
workflow run that failed before exporting provenance, crossing as
value-provenance-absent — stamped every file it later wrote with the
unknown taint, which no policy relaxes. The chat then could not read
the file back: generating a document and immediately rendering it to
show the user hard-failed with "cannot be shared safely", and healed
only when a later clean turn rewrote the sidecar. Traced in
production: the refused file's terminal sidecar is exact with zero
entries — a file with no secrets in it — and the last refusal landed
under a minute before the healthy write.

The write decision now distinguishes the two states its own policy
names. A latched registry holding no active entries is an absence: no
secret plaintext was resolved or imported in that context, so none can
be in the bytes, and the only fact is that content of unrecorded
history crossed — which is what the unrecorded status states, readable
under the fail-open policy with the audit entry naming the surface.
Taint stays reserved for a registry that holds plaintext it cannot map
to this output, and for every structural refusal below (scope, derived
representations, encryption).

Also carries the fault kind as a structured field on the trace-store
display lines, so dashboards can group without parsing messages.

* fix(provenance): relax a latch to unrecorded only when no fault is on record

Review finding, accepted with a broader gate than proposed: zero active
entries alone does not prove the context never held plaintext — a
verification or decrypt fault trips while secret material is in flight,
before anything activates — so keying the absence carve-out on the
entry count alone let a fault-latched registry stamp unrecorded.

Gate it on the one centrally maintained classification of exactly this
distinction: the originating-fault reason set that already decides the
report level. A latch relaxes to unrecorded only when nothing activated
and every recorded reason says provenance was never on offer; any fault
keeps the taint. An allow-list of the single observed reason would have
re-tainted the other genuine absences — a registry born without a
catalog, a durable row nobody recorded, a client tool that never
reported.

* fix(provenance): relax to unrecorded only for the dedicated absence reasons

Second review finding, accepted: the originating-fault set assigns
report ownership, not absence semantics, and the two are not the same
split. value-provenance-filter-incomplete is warn-level because its
underlying fault already error-logs inside the staged source registry —
yet it latches precisely when real entries were decrypted in-process
and could not be narrowed to the value, plaintext in flight that never
activated. Gating the file-write relaxation on "not a fault" let that
latch stamp unrecorded.

The registry now owns a dedicated absence set with the stronger
contract membership requires: provenance was never on offer AND no
secret material transited the latching context. An absent or
declared-incomplete envelope carries no entries by schema, a registry
born without a catalog or a persisted log never handled material, a
durable-unknown latch fires before importing anything, and the
inherited markers never occur alone. The write decision relaxes only
when nothing activated and every recorded reason is in that set; an
empty reason list keeps the taint. The fault predicate from the
previous commit is superseded and removed.

Author: icecrasher321

apps/sim/executor/utils/resolved-secret-trace-registry.test.ts

@@ -17,13 +17,45 @@ import {
   ANONYMOUS_SECRET_TRACE_REPLACEMENT,
   createIncompleteResolvedSecretTraceRegistry,
   createResolvedSecretTraceRegistry,
+  isResolvedSecretProvenanceAbsence,
   isResolvedSecretTraceProvenanceV1,
   RESOLVED_SECRET_TRACE_CHECKPOINT_VERSION,
   ResolvedSecretTraceProvenanceAccumulator,
   type ResolvedSecretTraceProvenanceV1,
   ResolvedSecretTraceRegistry,
 } from '@/executor/utils/resolved-secret-trace-registry'
 
+describe('provenance absence classification', () => {
+  it.each([
+    'value-provenance-absent',
+    'source-provenance-incomplete',
+    'constructed-incomplete',
+    'log-creation-skipped',
+    'durable-provenance-unknown',
+    'inherited-incomplete-source',
+    'inherited-incomplete-input-path',
+  ] as const)('classifies %s as an absence, since no material transited the latch', (reason) => {
+    expect(isResolvedSecretProvenanceAbsence(reason)).toBe(true)
+  })
+
+  /**
+   * Warn-level is not absence: `value-provenance-filter-incomplete` latches after a staged
+   * source registry decrypted real entries it could not narrow to the value, so plaintext was in
+   * flight without ever activating. The absence set must stay narrower than the report split.
+   */
+  it.each([
+    'value-provenance-filter-incomplete',
+    'value-provenance-import-failed',
+    'entry-decrypt-failed',
+    'projection-mismatch',
+    'untrusted-provenance',
+    'restored-provenance-untrusted',
+    'client-tool-seal-failed',
+  ] as const)('keeps %s out of the absence set', (reason) => {
+    expect(isResolvedSecretProvenanceAbsence(reason)).toBe(false)
+  })
+})
+
 describe('ResolvedSecretTraceProvenanceAccumulator', () => {
   const scope = { userId: 'user-1', workspaceId: 'workspace-1' }
 

apps/sim/executor/utils/resolved-secret-trace-registry.ts

@@ -99,6 +99,40 @@ export type ResolvedSecretIncompletenessReason =
  * inheriting a parent that reported moments earlier, or an unaudited caller taking the default
  * reason from flooding the error stream. A reason added later without thought stays quiet.
  */
+/**
+ * Reasons meaning provenance was never on offer AND no secret material transited the latching
+ * context. This is a deliberately separate, narrower set than the warn side of the report-level
+ * split below: that split assigns report ownership, and a warn-level reason can still involve
+ * plaintext in flight — `value-provenance-filter-incomplete` latches after a staged source
+ * registry decrypted real entries it then could not narrow to the value, so the plaintext existed
+ * in-process without ever activating. Membership here requires the stronger claim.
+ *
+ * The claim holds for each member: an absent or declared-incomplete envelope carries no entries
+ * (the envelope schema rejects incomplete-with-entries), so nothing was decrypted; a registry
+ * built without a catalog or without a persisted log never handled material; a durable read that
+ * latched did so before importing anything; and the inherited markers never occur alone — the
+ * source's own reasons are copied first, so they are judged by the originals they accompany.
+ *
+ * Consumers use this to separate `unrecorded` (absence — readable under the fail-open policy)
+ * from taint at a write decision. A reason outside this set keeps the taint.
+ */
+const PROVENANCE_ABSENCE_REASONS = new Set<ResolvedSecretIncompletenessReason>([
+  'value-provenance-absent',
+  'source-provenance-incomplete',
+  'constructed-incomplete',
+  'log-creation-skipped',
+  'durable-provenance-unknown',
+  'inherited-incomplete-source',
+  'inherited-incomplete-input-path',
+])
+
+/** True when {@link PROVENANCE_ABSENCE_REASONS} holds the reason; see its contract. */
+export function isResolvedSecretProvenanceAbsence(
+  reason: ResolvedSecretIncompletenessReason
+): boolean {
+  return PROVENANCE_ABSENCE_REASONS.has(reason)
+}
+
 const ORIGINATING_FAULT_REASONS = new Set<ResolvedSecretIncompletenessReason>([
   'untrusted-provenance',
   'entry-decrypt-failed',

apps/sim/lib/copilot/request/tools/files.test.ts

@@ -563,6 +563,13 @@ describe('maybeWriteOutputToFile', () => {
       .mockReturnValueOnce({ version: 1, complete: false, entries: [] })
     const registry = {
       exportCommittedProvenanceForValue,
+      /** Plaintext is in scope, so the incomplete export below is a taint, not an absence. */
+      getIncompletenessDiagnostics: vi.fn(() => ({
+        reasons: ['source-provenance-incomplete'],
+        origins: [],
+        incompleteInputPathCount: 0,
+        activeEntryCount: 1,
+      })),
     } as unknown as ResolvedSecretTraceRegistry
 
     const result = await maybeWriteOutputToFile(

apps/sim/lib/logs/execution/trace-store.ts

@@ -439,6 +439,7 @@ function reportStoredDisplayProvenanceFaults(
     if (parts.length === 0) continue
     logger[report.level](report.message, {
       ...details,
+      fault: kind,
       parts: parts.slice(0, MAX_REPORTED_PROVENANCE_FAULT_PARTS),
       partCount: parts.length,
     })

apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.test.ts

@@ -20,6 +20,7 @@ import type { DbTransaction } from '@/lib/db/types'
 import {
   areModelSafeWorkspaceFileKeys,
   copyWorkspaceFileSecretProvenanceInTx,
+  createWorkspaceFileSecretProvenanceFromRegistry,
   filterModelSafeWorkspaceFileAttachments,
   importWorkspaceFileSecretProvenanceForModelView,
   importWorkspaceFileSecretProvenanceForRuntime,
@@ -1548,3 +1549,124 @@ describe('workspace file secret provenance', () => {
     ).toEqual({ status: 'unknown' })
   })
 })
+
+describe('createWorkspaceFileSecretProvenanceFromRegistry write decision', () => {
+  const SCOPE = { userId: 'user-1', workspaceId: 'workspace-1' }
+
+  /**
+   * A registry latched with nothing resolved is an absence, not a taint: no plaintext exists in
+   * the context to be in the bytes, so the file must stay readable under the unrecorded policy.
+   * Stamping taint here made one failed workflow run hard-refuse every file its chat later wrote.
+   */
+  it('classifies a latched registry holding no active entries as unrecorded', async () => {
+    const registry = {
+      exportCommittedProvenanceForValue: vi.fn(() => ({
+        version: 1,
+        complete: false,
+        entries: [],
+      })),
+      getIncompletenessDiagnostics: vi.fn(() => ({
+        reasons: ['value-provenance-absent'],
+        origins: [],
+        incompleteInputPathCount: 0,
+        activeEntryCount: 0,
+      })),
+    } as unknown as ResolvedSecretTraceRegistry
+
+    await expect(
+      createWorkspaceFileSecretProvenanceFromRegistry(registry, 'generated content', SCOPE)
+    ).resolves.toEqual({ safe: true, provenance: { status: 'unrecorded' } })
+  })
+
+  /**
+   * Zero active entries does not prove the context never held plaintext: a verification or
+   * decrypt fault trips while secret material is in flight, before anything activates. Only a
+   * latch whose recorded reasons all belong to the registry's absence set may relax.
+   */
+  it.each([
+    ['an originating fault', 'projection-mismatch'],
+    /**
+     * Warn-level, yet plaintext-bearing: it latches after a staged source registry decrypted
+     * real entries it could not narrow to the value — the report-level split must not be the
+     * absence split.
+     */
+    ['an unnarrowable crossing', 'value-provenance-filter-incomplete'],
+  ] as const)(
+    'keeps a latch caused by %s as a taint even with no active entries',
+    async (_, reason) => {
+      const registry = {
+        exportCommittedProvenanceForValue: vi.fn(() => ({
+          version: 1,
+          complete: false,
+          entries: [],
+        })),
+        getIncompletenessDiagnostics: vi.fn(() => ({
+          reasons: [reason],
+          origins: [],
+          incompleteInputPathCount: 0,
+          activeEntryCount: 0,
+        })),
+      } as unknown as ResolvedSecretTraceRegistry
+
+      await expect(
+        createWorkspaceFileSecretProvenanceFromRegistry(registry, 'generated content', SCOPE)
+      ).resolves.toEqual({ safe: false })
+    }
+  )
+
+  /** A latched registry that recorded no reason offers nothing to vouch with; keep the taint. */
+  it('keeps a latch with an empty reason list as a taint', async () => {
+    const registry = {
+      exportCommittedProvenanceForValue: vi.fn(() => ({
+        version: 1,
+        complete: false,
+        entries: [],
+      })),
+      getIncompletenessDiagnostics: vi.fn(() => ({
+        reasons: [],
+        origins: [],
+        incompleteInputPathCount: 0,
+        activeEntryCount: 0,
+      })),
+    } as unknown as ResolvedSecretTraceRegistry
+
+    await expect(
+      createWorkspaceFileSecretProvenanceFromRegistry(registry, 'generated content', SCOPE)
+    ).resolves.toEqual({ safe: false })
+  })
+
+  it('keeps a latched registry holding plaintext it cannot map as a taint', async () => {
+    const registry = {
+      exportCommittedProvenanceForValue: vi.fn(() => ({
+        version: 1,
+        complete: false,
+        entries: [],
+      })),
+      getIncompletenessDiagnostics: vi.fn(() => ({
+        reasons: ['source-provenance-incomplete'],
+        origins: [],
+        incompleteInputPathCount: 0,
+        activeEntryCount: 1,
+      })),
+    } as unknown as ResolvedSecretTraceRegistry
+
+    await expect(
+      createWorkspaceFileSecretProvenanceFromRegistry(registry, 'generated content', SCOPE)
+    ).resolves.toEqual({ safe: false })
+  })
+
+  it('stays a taint when the incomplete export carries no diagnostics to vouch with', async () => {
+    const registry = {
+      exportCommittedProvenanceForValue: vi.fn(() => ({
+        version: 1,
+        complete: false,
+        entries: [],
+      })),
+      getIncompletenessDiagnostics: vi.fn(() => undefined),
+    } as unknown as ResolvedSecretTraceRegistry
+
+    await expect(
+      createWorkspaceFileSecretProvenanceFromRegistry(registry, 'generated content', SCOPE)
+    ).resolves.toEqual({ safe: false })
+  })
+})

apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts

@@ -20,9 +20,10 @@ import {
   PROVENANCE_MAX_ENTRIES,
   PROVENANCE_MAX_SERIALIZED_BYTES,
 } from '@/lib/execution/provenance-limits'
-import type {
-  ResolvedSecretTraceProvenanceV1,
-  ResolvedSecretTraceRegistry,
+import {
+  isResolvedSecretProvenanceAbsence,
+  type ResolvedSecretTraceProvenanceV1,
+  type ResolvedSecretTraceRegistry,
 } from '@/executor/utils/resolved-secret-trace-registry'
 
 /** Ids per statement. Bounds the query, never how many files a caller may classify. */
@@ -253,7 +254,28 @@ export async function createWorkspaceFileSecretProvenanceFromRegistry(
   const persistedProvenance = Object.is(sourceValue, persistedValue)
     ? sourceProvenance
     : registry.exportCommittedProvenanceForValue(persistedValue)
-  if (!sourceProvenance.complete || !persistedProvenance.complete) return { safe: false }
+  if (!sourceProvenance.complete || !persistedProvenance.complete) {
+    /**
+     * A latched registry is the same absence only when both hold: nothing activated, and every
+     * recorded reason is in the registry's absence set — reasons meaning provenance was never on
+     * offer and no secret material transited the latching context. Zero active entries alone does
+     * not prove that: a decrypt, verification, or filtering failure trips while plaintext is in
+     * flight, before anything activates, so any such reason keeps the taint. What remains is a
+     * registry that latched with nothing to lose (a failed workflow run crossing with no envelope
+     * is the recurring producer), which is exactly what `unrecorded` states. Stamping taint for
+     * that state made one failed run turn every file its chat later wrote into a hard refusal
+     * until the next clean write.
+     */
+    const diagnostics = registry.getIncompletenessDiagnostics()
+    if (
+      diagnostics?.activeEntryCount === 0 &&
+      diagnostics.reasons.length > 0 &&
+      diagnostics.reasons.every(isResolvedSecretProvenanceAbsence)
+    ) {
+      return { safe: true, provenance: { status: 'unrecorded' } }
+    }
+    return { safe: false }
+  }
   if (
     (sourceProvenance.entries.length > 0 &&
       !isPrivateSecretProvenanceScopeCompatible(sourceProvenance.scope, destinationScope)) ||