Workflows: implement CreateBatch V2 in local-dev - #15571
Draft
Naapperas wants to merge 11 commits into
Draft
Conversation
…orkflows binding The local binding created duplicate executions for deterministic instance ids: create() never checked for an existing instance and createBatch() mapped every input through create(), so code relying on the documented idempotency contract appeared to work locally while double-executing workflow bodies. Match the documented production behavior: create() with an id that already exists throws and retains the existing instance, and createBatch() skips ids that already exist, or repeat within the batch, excluding them from the result and creating instances in batch order. Existence is decided by the engine Durable Object via a new hasInstance() method that reads INSTANCE_METADATA, which is written exactly once by the first init(), so the check is consistent across isolates rather than relying on binding-local state. Auto-generated ids skip the check. Fixes cloudflare#14836
…tic-id contract create() with a caller-provided id now throws once the instance exists, so worker scripts that re-created the same fixed id on every request switch to the pattern production code needs anyway: try create, fall back to get for the existing instance. Covers the miniflare workflows persistence spec (re-created "an-id" in a polling loop and again after a restart), the dev-registry cross-worker workflow script, and both vite-plugin workflow playgrounds (whose only repeated-create path is a vitest retry against the still-running server).
…plicate-id checks
create(): restore the destructuring defaults. The ?? refactor was not
equivalent: an explicit null id became a generated id instead of being
rejected by validation, and an explicit null params was rewritten to {}.
createBatch(): validate every id before any existence probe or create.
Probing an id constructs the engine Durable Object, whose constructor
persists storage, so an invalid id previously left a phantom instance on
disk and rejected the batch only after earlier entries were already
created. A batch with a malformed id now creates nothing.
createBatch(): skip an id that a concurrent create claims between the
existence check and create(), per the skip contract, instead of failing
the batch part way through.
Three regression tests, each verified to fail on the parent commit.
…ded form Every other instance error here goes through createWorkflowError(message, code); the duplicate-id error was a free-form sentence, so code branching on an error code would behave differently locally than against the real service. Route it through a duplicateInstanceError(id) helper with the code instance.already_exists, following the instance.not_found vocabulary, and replace createBatch's substring matcher with a shared isDuplicateInstanceError predicate so the thrower and matcher stay in sync. The race regression test throws through the same helper.
…e-id error The catch-all fallbacks treated every create() failure as "the id already exists", so a real failure (malformed id, transient RPC error) was masked by the instance.not_found error from the follow-up get(). Match the instance.already_exists code and rethrow everything else, in both playground workers and both miniflare test worker scripts.
createBatch checked every caller-provided id twice (its own loop plus the identical probe inside create()) and awaited each probe sequentially, up to 200 serialized engine round-trips for a 100-entry batch. Factor the creation body into #createUnchecked, shared by create() after its probe and by createBatch after probing each distinct id once in a concurrent round. Creations stay in batch order; since init() is dispatched fire-and-forget, the probes were the only awaited RPCs, so a batch now costs one concurrent probe round. The mid-loop duplicate catch and its regression test are removed as dead code: with a single probe there is no second check to disagree with. A concurrent external create landing between the batch probe and creation remains the documented best-effort boundary.
…eset text The uniqueness probe reads the metadata that init() commits after create() returns, so duplicate creates racing ahead of that commit all resolve successfully; the engine's init() guards keep the race from double-executing the body. State that at the check site, and name the coded error in the changeset.
…peat requests The fixture workers create fixed deterministic ids (test, something, batch-1/batch-2), so a vitest retry re-invoking an endpoint against the same wrangler session now hit the duplicate-id error or an empty createBatch result. The create endpoints fall back to get() for the instance.already_exists error only, and the createBatch endpoint reads handles for the full id set after the batch call, since existing ids are skipped and excluded from its return.
🦋 Changeset detectedLatest commit: eabcfd1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
@cloudflare/autoconfig
@cloudflare/build-output-utils
@cloudflare/codemods
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-functions
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-plugin
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes WOR-1452.
Implements local-dev behavior for the new BatchCreate V2 API being released for Workflows.
Based on top of #14847 due to it fixing the way we were rejecting duplicate IDs, which is needed for this work.
Docs pending.
A picture of a cute animal (not mandatory, but encouraged)