Skip to content

fix(project-files): requeue rendered-artifact search after commit and trim Project read transactions - #9063

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/project-file-artifact-locking
Oct 15, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/project-file-artifact-locking

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

All changes are on flag-gated Project file paths (behind the projects rollout) plus small shared helpers whose workspace behavior is unchanged.

  • Lock upgrade in Project artifact reads. readArtifact holds the Project lock and the source row shared, but its final transaction called markFileSearchArtifactReadyInTx, which takes the Project lock exclusively and file rows FOR UPDATE. Every compiled render did this (a cache hit still returns artifactKey), so concurrent generated-document reads in one Project could deadlock or wait out the 5 s Project lock timeout. The read now records the search completion and requeues after commit via requeueFileSearchForRenderedArtifact (lock-free current check, then its own transaction). That transaction now takes the owner lock under the 1 s artifact-ready bound instead of the 5 s Project lock bound; a timeout is caught and logged and never fails the read.
  • Fewer authorized transactions. Artifact, inline, and CSV preview reads authorize from their own snapshots (prepareFromSnapshot) instead of a separate preflight. Artifact inputs are snapshotted and admitted only when the source references any; the final transaction still rechecks the source, every input, and provenance after storage reads. Artifact: 4 transactions / 3 admissions → 2 / 2 without inputs, 3 / 3 with. Inline and CSV: 3 → 2. Search: drops the nested listProjectFileFolders use case.
  • Artifact inputs. Downloads run with bounded concurrency after checking declared sizes against the per-file and total budgets. Each read is capped at its declared size plus the unclaimed budget, and the first failure aborts the rest. The request signal now reaches render, downloads, and compile.
  • Project inline delivery streams instead of buffering. It fences the file head after storage opens and serves workspaceFileCacheControl(contentAddressed) like workspace inline. It no longer lists every Project folder.
  • Project search admits, withholds stale previews, and reports provenance after its snapshot commits. loadFileSearchDelivery returns an explicit current set, and provenance is classified only when a content gate or delivery observer needs it.
  • Foreign page embeds. A Project page that embeds another owner's image now fails as a missing input (not_found) instead of a 500, through a typed ProjectPageForeignEmbedError.
  • Cleanup.
    • Page inline limits are exported from page-document.server.ts and reused.
    • knip.jsonc narrows the project-files.ts entry to types.
    • SearchProjectFileContentInput is un-exported.
    • The CI isolate build step uses paths: ["."] and JOBS=4, matching docker/app.Dockerfile.

Expected rebase conflicts in the open Project file stack:

Type of Change

  • Bug fix

Testing

  • Biome on all changed files; lib/workspace-files/page-compile.test.ts.
  • New integration checks in content-rendered.integration.ts:
    • A generated artifact read beside a concurrent shared Project holder completes and requeues search. On the previous code it times out on the exclusive Project lock.
    • A contended requeue gives up within its short bound without failing the read.
    • A Project page embedding another owner's image is not_found.
  • Type-check, audits, and integration suites run in CI.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

… trim Project read transactions

- Project artifact reads no longer requeue search inside the read transaction, which held the
  Project and source row shared and then needed them exclusively; the requeue runs after commit
  with its owner lock bounded by the short artifact-ready lock timeout and is best effort
- Artifact, inline, and CSV preview reads authorize from their own snapshots instead of a separate
  preflight; artifact inputs are snapshotted only when referenced and download with bounded
  concurrency after a declared-size budget check; the request signal reaches render and compile
- Project inline delivery streams and caches content-addressed key requests like workspace inline
- Project search admits, withholds previews, and reports provenance after its snapshot commits,
  classifies provenance only when a gate or delivery observer needs it, and lists folders directly
- A Project page embedding another owner's image is a missing input instead of a 500
- Page limits are shared from page-document.server; knip and the CI isolate build are tightened
@vercel

vercel Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 15, 2026 1:43am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 13 files

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/projects/files/__integration__/content-rendered.integration.ts Outdated
@greptile-apps

greptile-apps Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The changes since the previous review appear safe to merge.

Summary

This PR moves rendered-artifact search requeues after commit and reduces transactions on Project file reads.

  • Rendered files check their inputs before and after they are built.
  • Rendered artifact reads requeue search after the read commits.
  • Project inline files stream to callers instead of buffering first.
  • CSV previews use their file snapshot for authorization.
  • Project search hides previews that no longer match current files.
  • CI builds the Project rendering addon with a fixed job limit.

No new actionable issues were found in the changes since the previous review.

Diagram

sequenceDiagram
  participant Reader as Project artifact reader
  participant DB as Database
  participant Search as Search requeue
  Reader->>DB: Check source and inputs under shared Project lock
  DB-->>Reader: Commit and release locks
  Reader->>Reader: Admit content and report delivery
  Reader->>Search: Requeue rendered artifact
  Search->>DB: Acquire exclusive Project lock with short timeout
  alt Lock acquired
    Search->>DB: Mark revision pending and enqueue search
  else Lock times out
    Search-->>Reader: Log failure without failing the read
  end
Loading

Reviews (2) · Last reviewed commit: "test(project-files): start requeue check..." · Reviewed by Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 13 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit e1451e9 into staging Oct 15, 2026
43 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/project-file-artifact-locking branch October 15, 2026 01:51

This branch was previously deployed

1 inactive deployment
Preview — 2383eb9c Deployed Oct 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant