Repository navigation
Conversation
ceritium
force-pushed
the
defer-concurrency-lock-acquisition
branch
2 times, most recently
from
October 4, 2026 13:06
f0abcb8 to
a1e9177
Compare
When Solid Queue shares the app's connection and a concurrency-limited job is enqueued inside a transaction, acquiring the semaphore happens inside that transaction: Semaphore#wait selects the row FOR UPDATE, or inserts it, and the lock lasts until the caller's transaction ends. Until then, any other enqueue with the same key waits, and so does the worker signalling the semaphore after running a job with that key. Two transactions that enqueue jobs for two keys in opposite order deadlock, which surfaces as SolidQueue::Job::EnqueueError and aborts the caller's transaction. A job enqueued with a wait that isn't due yet when it's inserted doesn't have the problem: it's scheduled, and the dispatcher acquires its semaphore later in its own transaction. This makes that available on purpose, without a delay. A new option says when a job acquires its concurrency lock: - :on_enqueue, the default and the current behaviour. - :on_dispatch, enqueuing the job as a scheduled job that is already due, for the dispatcher to acquire the lock once it's committed. - :outside_transaction, :on_dispatch when there's an open transaction on Solid Queue's connection, :on_enqueue otherwise. It can be set globally (config.solid_queue.concurrency_lock_acquisition), per job (limits_concurrency acquire:) or per enqueue (set(acquire:)), the most specific one winning. The job row is still inserted in the caller's transaction, so it's still committed or rolled back with it. Whether there's an open transaction is checked before the job's own insert, which opens one, and only joinable transactions count, so the test fixtures' transaction doesn't. Jobs without concurrency controls have no lock to defer, so they skip the transaction check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lceyt66zciMB7d4Yv7UgDd
ceritium
force-pushed
the
defer-concurrency-lock-acquisition
branch
from
October 15, 2026 21:38
e0e0c9c to
97ef544
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When Solid Queue shares the app's database (no
connects_to) and the enqueue isn't deferred to after commit, enqueueing a due job withlimits_concurrencyacquires its semaphore (SELECT ... FOR UPDATEor insert) inside the caller's transaction. The row lock lasts until that transaction ends, so:SolidQueue::Job::EnqueueErrorand aborting the caller's transaction.Reproduction against 1.7.0 (Rails 8.1.4, PostgreSQL 17), single file: https://gist.github.com/ceritium/8d733493fef1c18f8f143fb8a2144c60
With transactions held 3 s, (1) and (2) wait ~2.5 s and (3) deadlocks.
FORK=1runs this branch, where none of them happens.The change
A job enqueued with a
wait:that isn't due yet already avoids this: it's inserted as scheduled and the dispatcher acquires the lock later, in its own short transaction. This makes that path available without a delay, through an option::on_enqueue(default): current behaviour.:on_dispatch: insert the job as scheduled and already due; the dispatcher acquires the lock on its next poll.:outside_transaction::on_dispatchwhen a transaction is open on Solid Queue's connection,:on_enqueueotherwise.The job row is still written in the caller's transaction; only the semaphore moves out. Trade-offs, documented in the README: up to
polling_intervalof extra latency, a dispatcher is required, andon_conflict: :discardhappens at dispatch (as for scheduled jobs).Related: rails#812 fixes a deadlock between dispatchers on that same path.
Tests: model tests for each mode, precedence and bulk enqueue, and an integration test per effect above showing it with
:on_enqueueand not with:outside_transaction(skipped on SQLite). Full suite green on PostgreSQL; MySQL not run.