feat(agents): releases — a digest of what the model is given, cut on deploy, pinned to every trace, run and evaluation

[TonsOfFun]

Summary

Agents get releases, and every trace, run and evaluation run says which one it ran under — so "6/8" is a statement about a specific release of an agent, a trace can be filtered by the deploy that produced it, and a regression is attributable to the instruction or tool change that caused it. Cut on deploy by a rake task; shown in the dashboard's existing Versions tab.

Framework (activeagent)

• ActiveAgent::Release, included in Base: release_manifest is everything about the class that shapes a generation — provider and model, generation options minus credentials (keys matching key|token|secret|password|credential are dropped at any depth), the public actions, every prompt template under the agent's view prefixes (path → content digest), and the tools and delegations the class declares. release_digest is the first 12 hex of its SHA-256: two deploys of the same agent share a digest, any change to those inputs is a new one, nobody bumps a number. Memoized per class; a dev reload replaces the class, so it recomputes. reset_release! for hosts that edit templates at runtime.
• ActiveAgent::Release.revision — the deploy itself (a git SHA or release label). Set from an initializer (value or proc), else read from SERVICE_VERSION, GIT_SHA, KAMAL_VERSION, SOURCE_VERSION, HEROKU_SLUG_COMMIT.
• Telemetry: the generation's root span carries agent.version (the digest) and agent.revision; every root span carries service.version (Telemetry.configuration.service_version, defaulting to the release revision).

Dashboard (actionagent)

• Agent#record_release!(digest:, manifest:, revision:, released_by:) cuts an AgentVersion carrying release_digest and revision, with the manifest under configuration_snapshot["release"] so the Versions tab diffs releases like edits. Idempotent per digest — a redeploy of an unchanged agent is not a new version — so it belongs in the deploy. AgentVersion#release?, .releases, Agent#latest_release.
• ActionAgent::AgentRelease.call(revision:) does that for every record naming a host class (agent_class_name — the host's sync owns the mirror; this reads it back), skipping and reporting a class that no longer resolves.
• rake action_agent:agents:release[REVISION] (also REVISION= / RELEASED_BY= env) and rake action_agent:agents:versions.
• Pinning: TelemetryTrace#attach_agent_version! at ingest — the trace's agent.version meets the version with that digest; a dashboard run's trace (no digest) takes the agent's latest. AgentRun and EvaluationRun take the agent's latest version when created. agent_version_id on all three, belongs_to :agent_version.
• Attribution fix that this depends on: AgentRegistrar matched a trace to an agent only on service + class + action — all three — so a trace from a class the host mirrors (class name set, no service/action) never matched and registered an observed per-action twin beside the synced record. Seen on support-hub today: the agent count went 6 → 7 after two code-path TicketAgent asks. A non-observed record naming the class now wins for any action of it; observed records are still matched on all three.
• Migration add_agent_releases (guarded column by column; the install generator emits it for any install lacking it); dummy app migration + schema. Version JSON gains release, release_digest, revision.

Not in this PR

• Frontend: the Versions tab already renders change_summary (Release 1a2b3c4d5e6f · <sha>: templates, tools), so releases show without a bundle change. A version filter on Traces and a release chip on trace/run/eval views would use the new fields — follow-up.
• A generic mirror of host classes into records. Hosts sync their own (support-hub's AI::DashboardAgentSync carries the hub's delegations as a flat roster on purpose); the release reads what the sync set.

Tests

• test/release_test.rb — manifest contents and exclusions, digest shape/stability/uniqueness, a template edit is a new digest, revision resolution, service_version fallback. 5 runs.
• actionagent/test/agent_release_test.rb — one version per digest with redeploy no-op, AgentRelease cut/unchanged/skipped, ingest pinning by digest / latest / unknown, runs and evaluation runs pinned, a real mock generation attributed to the mirror (not a twin) and stamped end to end. 7 runs.
• Neighbours green: framework telemetry suites, telemetry_trace, telemetry_correlation, agent_tool_roster, agent_execution_service, evaluation runner/run suites. bin/lint clean.

Deploying it (host side, for reference)

bin/rails ai:agents:sync                      # the host's own mirror
bin/rails action_agent:agents:release[$GIT_SHA]

🤖 Generated with Claude Code

[TonsOfFun]

Reviewed this alongside #452 and #454 as candidates for the next cut from release/1.6.0. All three merge cleanly onto the release branch together, and the full suite on that integration branch runs 2014 runs, 2 failures, 34 errors against a baseline of 1994 runs, 0 failures, 34 errors on bare release/1.6.0 (the 34 errors are the pre-existing local-environment RubyLLMProviderTest / RubyLLMIntegrationTest failures; same set on both). The two new failures are the one CI flagged here plus a sibling:

• DashboardEngineApiTest#test_agents_are_listed_with_their_scorecards — Expected: [4] Actual: [5, 3, 4, 2, 1]
• DashboardEngineApiTest#test_metrics_and_traces_read_the_same_store_the_ingest_endpoint_writes — Expected: 1 Actual: 3

Both pass when engine_api_test.rb runs alone (29 runs, 0 failures). A smaller reproduction fails deterministically:

bin/test actionagent/test/agent_release_test.rb test/release_test.rb actionagent/test/engine_api_test.rb   # TESTOPTS=--seed=1, 2 and 3
→ DashboardOwnershipTest#test_an_unresolved_owner_sees_nothing_once_an_owner_model_is_configured  Expected: 1  Actual: 2

and it passes once an extra query is added to that assertion's message, so the extra rows arrive on a timing boundary rather than in a fixed order. Nothing persists in storage/test.sqlite3 after agent_release_test.rb runs alone, so the rows are committed inside some later test's transaction, not outside it.

What the two telemetry tests in agent_release_test.rb leave behind, and never undo in ensure:

1. ActiveAgent::Base.include(Instrumentation) + instrument_telemetry! — GenerationInstrumentation stays prepended on Base for every test that follows (the dummy boots with telemetry off, so in the baseline it is never installed).
2. ActiveAgent::Telemetry.tracer — memoized with a reporter that was built while local_storage was on.
3. That reporter's threads — BatchingReporter#report starts the periodic flusher and deliver_batch delivers on Thread.new when async; flush drains the buffer but does not join in-flight sends, and shutdown is never called. Delivery writes through Configuration#dashboard_store → TelemetryTrace.create_from_payload → AgentRegistrar, which is exactly what mints observed agents and traces in a test that expected none.

A teardown that reverses those — ActiveAgent::Telemetry.tracer.reporter.shutdown, drop the memoized tracer, reset_configuration! — or driving TelemetryTrace.create_from_payload with the generation's attributes instead of running a real generation, should make the suite order-independent again. Happy to re-run the integration branch once it's updated.

[TonsOfFun]

Follow-up with CI evidence from the 1.6.2 preview (#455), which carries this branch merged with #452, #454 and #398:

• 3.4 / rails8: green (the seed happened to dodge it).
• 3.3 / rails8: the same two DashboardEngineApiTest failures (unresolved owner … Expected: 1 Actual: 3).
• 3.2 / rails7: Expected: 1 Actual: 14 on that test, plus 8 errors of ActiveRecord::StatementInvalid: SQLite3::BusyException: database is locked spread across RunLifecycleTest, EngineIntegrationTest, EvaluationEvidenceTest, EngineAnalyticsApiTest, ObservedAgentExecutionTest — and this branch's own AgentReleaseTest#a_release_cuts_one_version_per_digest.

The lock errors are the same cause seen from the other side: on that stack the reporter's delivery thread gets its own connection rather than the test's pinned one, so instead of writing stray rows into another test's transaction it contends with it. Shutting the reporter down and removing the instrumentation in ensure should clear both symptoms.