Smartstack: add the polling stack worker and supervisor

[timbeccue]

Summary

This is PR 6 of 8 completing the v1 banzai implementation of Smartstacking at site.

PR1 (#469): combine math
PR2 (#470): JPEG utilities
PR3 (#471): DB updates
PR4 (#473): shipper contract
PR5 (#474): smartstack data products
PR6 (#475): stack worker polling      <-- you are here
PR7 (#476): e2e tests, compose cleanup
PR8 (#477): logging

PR1 combine math ───────┐
PR2 JPEG utilities ─────┤
PR3 DB updates ─────────┼──> PR5 data products --> PR6 stack workers --> PR7 E2E --> PR8 logs/docs
PR4 shipper contract ───┘

PRs 1–4 provide the independent foundations for combining images, rendering JPEGs, storing stack state, and shipper integration (this PR). PR5 builds the data products, this PR adds the polling worker, crash-only supervisor, and site deployment configuration. PR7 adds cross-service E2E and recovery proof; PR8 adds structured lifecycle telemetry and final documentation.

Stack Worker Lifecycle behavior

The runtime starts one stateless polling process per camera. Each poll:

• Loads active stacks and their ordered members from PostgreSQL.
• Finalizes a stack when all expected members have arrived or an is_last member is present.
• Gives completion priority when a stack is both complete and past its timeout.
• Defines timeout as silence since the latest accepted reduced member.
• Produces a partial final product when an incomplete stack times out.
• Renders previews only when enabled, membership has grown, and logical frame 1 is present so product names remain stable.

A failed preview does not advance the stored preview count, allowing a later poll to retry it.

Finalization and retries

Finalization follows a deliberate order:

claim attempt → write products → publish paths → mark terminal

The attempt and backoff are persisted before product work begins. A build or publish failure therefore leaves the stack active for a bounded retry, while exhausted attempts end in error.

Publishing happens before the terminal database update. A failure between those steps can produce a duplicate notification with identical paths, but cannot leave a stack marked complete without having notified the shipper.

Process recovery

Ordinary exceptions are contained inside each worker loop. If a child dies violently—for example from an OOM kill—the supervisor exits and Docker restarts the service.

Because all lifecycle, retry, and preview state lives in PostgreSQL, workers restart without reconstructing process-local state. Claiming before product work also prevents a consistently failing stack from causing an endless restart loop.

Scope

This PR adds the polling worker, crash-only supervisor, and site deployment configuration. PR7 adds cross-service E2E and recovery proof; PR8 adds structured lifecycle telemetry and final documentation.

Focused tests cover completion and timeout precedence, preview gating, finalization ordering, retries, publish failures, attempt exhaustion, worker-loop resilience, and supervisor behavior.