docs: describe retry and redrive

The roadmap still named the gaps this branch closes: no retry for a dead
effect or broadcast, one row at a time, and no record of who pressed
what. It now states what exists and what still does not, which is the
dashboard surface for the new scopes.

Operations gains the API, the idempotency rule, the batching, the
authorization resource names, and the audit row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: cardmagic

CHANGELOG.md

@@ -2,6 +2,28 @@
 
 ## Unreleased
 
+- Retry a dead effect or broadcast. `SolidObjects.dead_letters` keeps its
+  message meaning and answers `effects` and `broadcasts`, so the kind rides on
+  the receiver. `retry` returns a dead row to pending with a zero attempt count
+  and no claim, reuses the stable id so a deduplicating handler sees the same
+  key, and acts only on a dead row, so a second press cannot double-enqueue. A
+  dead transmit effect replays rather than stay lost.
+- Redrive a whole scope. `redrive` opens a durable task, returns at once, and is
+  idempotent over its scope and filters, which a dashboard button needs. A
+  unique index on the active scope enforces that in the database, so two
+  processes that start the same redrive share one task. The supervisor advances
+  one bounded batch per pass, so a redrive never holds a transaction longer than
+  one batch. `SolidObjects.redrives` reads tasks back, and `task.cancel` stops
+  one and leaves the rows it already moved.
+- Record who pressed what. Every retry and every redrive transition writes one
+  row to `solid_objects_administration_events`. The identity comes from the
+  authorization context through a new `administration_identity` hook.
+- Add `redrive_batch_size`, which defaults to 100, and `redrive_batch_pause`,
+  which defaults to 0.05 seconds.
+- Add two tables, `solid_objects_administration_events` and
+  `solid_objects_redrives`. Run `bin/rails solid_objects:install:migrations` and
+  migrate.
+
 - Select a wake-up adapter automatically. `config.wake_up_adapter` now takes a
   name or an adapter, as `config.cache_store` and
   `config.active_job.queue_adapter` do, and defaults to `:automatic`. Selection

docs/dashboard.md

@@ -149,6 +149,16 @@ actor class that no longer exists, a full mailbox, a payload over the cap. The
 dashboard renders the dead letter again with the reason and a 422 status,
 rather than failing the request.
 
+Dead effects and broadcasts have the same API, which the dashboard does not yet
+surface. `SolidObjects.dead_letters.effects` and
+`SolidObjects.dead_letters.broadcasts` read and retry their own kind, and
+`redrive` moves a whole scope as a durable task. See
+[Operations](operations.md) for both.
+
+Every retry and every redrive transition writes one row to
+`solid_objects_administration_events`, holding the action, the kind, the
+subject, and the identity that asked for it.
+
 **Pause an instance** sets `paused_at`, and the activation manager stops
 claiming that identity. Two consequences matter:
 

docs/operations.md

@@ -282,6 +282,83 @@ interval, and current interval. The polling-only warning is also emitted as
 adapters should return `true` for a notification and `false` for a timeout; an
 older adapter that returns `nil` remains compatible and keeps the fast cadence.
 
+## Dead letters, retry, and redrive
+
+A message that exhausts its attempts becomes a dead letter. An effect or a
+broadcast that exhausts its attempts stays in its own table with
+`status = 'dead'`. All three are read and retried through one receiver, which
+carries the kind:
+
+```ruby
+SolidObjects.dead_letters.all(authorization_context: current_admin)
+SolidObjects.dead_letters.retry(dead_letter_id, authorization_context: current_admin)
+
+SolidObjects.dead_letters.effects.all(authorization_context: current_admin)
+SolidObjects.dead_letters.effects.retry(effect_id, authorization_context: current_admin)
+SolidObjects.dead_letters.broadcasts.retry(broadcast_id, authorization_context: current_admin)
+```
+
+An effect or broadcast retry returns the row to pending with a zero attempt
+count, no claim, and immediate availability. It keeps the stable id, so a
+handler that deduplicates on `effect_id` still sees the same key. An effect is
+at-least-once by contract, so a retried effect can run twice.
+
+Retry acts only on a dead row. A row that is pending, processing, or completed
+comes back unchanged, so pressing a button twice cannot double-enqueue and
+cannot take a row away from a worker that holds it.
+
+An incident produces dead rows in the hundreds, so a scope also answers
+`redrive`:
+
+```ruby
+task = SolidObjects.dead_letters.effects.redrive(
+  actor_type: "payments",
+  failed_after: 6.hours.ago,
+  limit: 5_000,
+  authorization_context: current_admin
+)
+
+task.id         # => "redrive_..."
+task.status     # => "running"
+task.moved      # => 412
+task.remaining  # => 4_588
+
+task.cancel(authorization_context: current_admin)
+```
+
+`redrive` returns at once. The task is durable, and the supervisor advances one
+bounded batch per pass, so a redrive of thousands of rows never holds a
+transaction longer than one batch. `redrive_batch_size` defaults to 100 and
+`redrive_batch_pause` to 0.05 seconds.
+
+A redrive is idempotent over its scope and its filters. Starting the same one
+while it runs returns the running task rather than a second one, which a
+dashboard button an operator can press twice needs. A different scope or a
+different filter starts its own task, and the same scope can be redriven again
+once the first task finishes.
+
+Read tasks back with `SolidObjects.redrives`:
+
+```ruby
+SolidObjects.redrives.find(task.id, authorization_context: current_admin)
+SolidObjects.redrives.all(status: :running, authorization_context: current_admin)
+```
+
+A running task reports what is left to move rather than a stored estimate,
+because rows die and are retried while it runs.
+
+Retry, redrive, and cancel each go through `authorize_administration` under
+their own resource name: `dead_letters`, `effect_dead_letters`,
+`broadcast_dead_letters`, and `redrives`. Every retry and every task transition
+writes one row to `solid_objects_administration_events`, holding the action, the
+kind, the subject, the identity, and when it happened. The identity comes from
+`administration_identity`, which receives the authorization context the caller
+passed and defaults to its `to_s`. A refused caller writes nothing.
+
+Automatic redrive on a schedule is deliberately absent. A dead row means a
+person decided something, and these APIs give that person an alternative to an
+`UPDATE` against a runtime table.
+
 ## Graceful shutdown
 
 The supervisor requests shutdown, stops new claims, lets active loops return,

docs/roadmap.md

@@ -178,15 +178,18 @@
   or turn off. Every route declares its
   own administration policy and a route declared without one raises at load
   time, so the deny-by-default posture is enforced by construction rather than
-  by remembering to add a check. It changes only two things: an idempotent dead
-  letter retry and instance pause/resume. What does not exist is audit records
-  of who pressed what, and bulk-safe tools: retry is one dead letter at a time,
-  because `DeadLetterManager` exposes no bulk operation. Pause is an operator
-  brake and not a stop, since a pass already in flight finishes its turn and a
-  synchronous caller waiting on a paused instance times out. Retry also only
-  exists for message dead letters: a dead effect or broadcast has no retry
-  API, which matters for transmit effects because a dead one is a lost
-  replay until an operator returns its row to pending. The page cost was
+  by remembering to add a check. It changes only three things: an idempotent dead
+  letter retry, a redrive, and instance pause/resume. Retry covers all three
+  kinds. `SolidObjects.dead_letters` keeps its message meaning and answers
+  `effects` and `broadcasts`, so a dead effect or broadcast returns to pending
+  through an API rather than an operator's `UPDATE`, and a dead transmit effect
+  is no longer a lost replay. `redrive` moves a whole scope as a durable task
+  that is idempotent over its filters, cancellable, and advanced in bounded
+  batches by the supervisor. Every retry and task transition writes one row to
+  `solid_objects_administration_events`, so who pressed what is recorded. Pause
+  is an operator brake and not a stop, since a pass already in flight finishes
+  its turn and a synchronous caller waiting on a paused instance times out. The
+  dashboard does not yet surface the scopes or redrive; the API does. The page cost was
   reasoned about rather than measured: the summary bar issues a fixed set of
   indexed aggregate queries per page, which is why `HEAD /` exists for uptime
   monitors, but no dashboard latency has been benchmarked against a large