Skip to content

Commit a902aa5

Browse files
cardmagicclaude
andcommitted
fix: report a missing migration in the doctor
A parity audit against the TypeScript branch found three gaps. The doctor passed on an instance table without `completed_idempotency_keys`, so an operator who installed the gem and skipped the migration learned about it from a worker crash rather than from `solid_objects doctor`. The TypeScript doctor verifies its whole migration list, so it already caught the equivalent. Without the column in `EXPECTED_COLUMNS` the new test fails with `Expected true to not be truthy`. The dead path wrote the instance row for every dead message, including one that carried no key. `remember_key` now returns first, which matches what the TypeScript repository does and removes a write per dead message. The complete and reject paths fold the column into an update they already perform. The changelog described neither the migration an application must run nor `SolidObjects::SchemaBootstrap`. A test proves two keyed messages in one activation pass are both remembered, because the executor reads the locked instance row rather than a claim-time copy. Validation: bundle exec rake, plus the full suite on PostgreSQL 16, mysql2, and trilogy. 783 runs, 0 failures on each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 21c35bb commit a902aa5

6 files changed

Lines changed: 52 additions & 2 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,16 @@
2828
`retained_idempotency_keys` bounds the memory and defaults to 64 keys for
2929
each actor. A lookup by request id cannot make the distinction, because the
3030
runtime, not the caller, generates a request id and no actor remembers one.
31+
- Add `db/migrate/20260923000000_add_solid_objects_completed_idempotency_keys.rb`,
32+
which adds `instances.completed_idempotency_keys` as `jsonb` on PostgreSQL and
33+
`json` elsewhere. An application installs it with
34+
`bin/rails solid_objects:install:migrations` and runs it before it upgrades a
35+
worker, because the executor writes the column on every finished turn. The
36+
doctor now reports the column as missing when it is not installed.
37+
- Apply migrations through `SolidObjects::SchemaBootstrap`, which reads
38+
`db/migrate`. Seven scripts each carried a hand-copied migration list, and
39+
three of them applied an incomplete schema. A test fails if any script names a
40+
migration class again.
3141

3242
- Retry a dead effect or broadcast. `SolidObjects.dead_letters` keeps its
3343
message meaning and answers `effects` and `broadcasts`, so the kind rides on

‎lib/solid_objects/doctor.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def to_s
6565
instances: %w[
6666
id actor_type actor_id state state_version next_message_sequence
6767
activation_owner_id activation_token activation_expires_at
68-
activation_generation
68+
activation_generation completed_idempotency_keys
6969
],
7070
messages: %w[
7171
id instance_id delivery_mode arguments sequence attempt_count request_id

‎lib/solid_objects/executor.rb‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -405,7 +405,7 @@ def fail_message(error)
405405
if error.is_a?(NonRetryableError) ||
406406
locked_message.attempt_count >= locked_message.max_attempts
407407
create_dead_letter(message: locked_message, error_details:, now:)
408-
instance.update!(completed_idempotency_keys: remembered_keys(instance, locked_message))
408+
remember_key(instance, locked_message)
409409
dead = true
410410
else
411411
ReadyMessage.create!(
@@ -475,6 +475,13 @@ def matching_claim!
475475
raise LostActivation, "message claim changed"
476476
end
477477

478+
# @rbs (Instance, Message) -> void
479+
def remember_key(instance, message)
480+
return unless message.idempotency_key
481+
482+
instance.update!(completed_idempotency_keys: remembered_keys(instance, message))
483+
end
484+
478485
# @rbs (Instance, Message) -> Array[String]
479486
def remembered_keys(instance, message)
480487
remembered = Array(instance.completed_idempotency_keys)

‎sig/generated/lib/solid_objects/executor.rbs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,9 @@ module SolidObjects
9494
# @rbs () -> ClaimedMessage
9595
def matching_claim!: () -> ClaimedMessage
9696

97+
# @rbs (Instance, Message) -> void
98+
def remember_key: (Instance, Message) -> void
99+
97100
# @rbs (Instance, Message) -> Array[String]
98101
def remembered_keys: (Instance, Message) -> Array[String]
99102

‎test/integration/doctor_test.rb‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,25 @@ class DoctorTest < ActiveSupport::TestCase
113113
assert_equal :skip, report.check(:sync_round_trip).status
114114
end
115115

116+
test "fails when a migration that a runtime path needs is missing" do
117+
installed = SolidObjects::Record.connection
118+
instances = SolidObjects.table_name(:instances)
119+
connection = Object.new
120+
connection.define_singleton_method(:data_sources) { installed.data_sources }
121+
connection.define_singleton_method(:columns) do |table|
122+
columns = installed.columns(table)
123+
next columns unless table == instances
124+
125+
columns.reject { |column| column.name == "completed_idempotency_keys" }
126+
end
127+
128+
report = SolidObjects::Doctor.new(connection:).call
129+
130+
refute report.healthy?
131+
assert_equal :fail, report.check(:schema).status
132+
assert_match(/completed_idempotency_keys/, report.check(:schema).message)
133+
end
134+
116135
test "reports live runtime roles" do
117136
now = SolidObjects.database_adapter.database_now
118137
SolidObjects::Process.create!(

‎test/integration/result_lookup_test.rb‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,17 @@ def reject_checkout
283283
assert_equal 3, SolidObjects::Instance.sole.completed_idempotency_keys.size
284284
end
285285

286+
test "remembers every key of one activation pass" do
287+
reference = CartActor.ref("alice")
288+
reference.async(idempotency_key: "first").checkout(order_id: 1)
289+
reference.async(idempotency_key: "second").checkout(order_id: 2)
290+
run_actors
291+
SolidObjects::Message.delete_all
292+
293+
assert_raises(SolidObjects::MessagePruned) { reference.find_by(idempotency_key: "first") }
294+
assert_raises(SolidObjects::MessagePruned) { reference.find_by(idempotency_key: "second") }
295+
end
296+
286297
test "remembers nothing for a message that carried no key" do
287298
reference = CartActor.ref("alice")
288299
reference.async.checkout(order_id: 1)

0 commit comments

Comments
 (0)