Repository navigation
fix(dispatch): carry inter-agent chain depth through loops, schedule triggers, session turns and events (#2973) - #3120
Merged
Conversation
…triggers, session turns and events (#2973) Follow-up to #2806. Loop starts, manual schedule triggers, chat-session turns and agent event emits by an agent principal now inherit the caller's chain depth and are refused past inter_agent_max_chain_depth with the named 403 inter_agent_depth_exceeded, instead of starting a fresh depth-0 root. - Loops: depth stored on agent_loops.chain_depth (SQLite migration + Alembic 0084) and stamped on every iteration row. - Schedule trigger: depth forwarded to the scheduler; retries keep it. - Session turns: depth passed through run_resumable_turn to execute_task. - Events: depth signed into the EVT-001 loopback JWT (chain_depth claim), minted whether or not the source is vouched; task-completion events carry the finished row's depth + 1 (the max when the row is unreadable); _chain_caller falls back to vouched_source_agent. - Per source->subscriber hourly event-dispatch budget in trigger_subscription (ops setting event_dispatch_max_fires_per_hour, default 120, one alert per window, fail-open on Redis errors). - App-level handler maps InterAgentDepthExceeded to the #2806 403 body. - MCP run_agent_loop, trigger_agent_schedule and emit_event render the refusal as a non-retryable result. Mutation-checked: reverting each fix turns its test red in tests/unit/test_2973_depth_new_roots.py, test_2973_event_dispatch_budget.py and src/mcp-server/src/tools/depth-refusal.test.ts. Webhook tokens, agent-created cron schedules and self-reminders remain roots; tracked in #3116. Fixes #2973 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
obasilakis
force-pushed
the
feature/2973-chain-depth-laundering
branch
from
September 30, 2026 13:25
3d4278c to
6b02181
Compare
…ons, budget alert write (#2973) Review follow-ups on #3120: the cold retry in run_resumable_turn keeps chain_depth (mutation-checked), a missing session answers 404 before the depth 403, and the budget alert writes a high-priority notification on the subscriber. Doc counts updated for the four covered paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6 tasks done
Contributor
|
merge-train: this PR is on today's train with #3122, #3119 and #3085. The one Tier 1 blocker was the Two gaps to consider. Neither blocks this train. Each was found by a mutation that stayed green:
Also worth a decision, from This PR goes in ahead of #3085, and #3085 will re-parent onto your |
This was referenced Sep 30, 2026
vybe
pushed a commit
that referenced
this pull request
Sep 30, 2026
…5_ent720_email_identity (#3085) #3120 landed 0084_agent_loops_chain_depth off the same parent as this PR's 0084_ent720_email_identity. Disjoint tables, so the revision is re-parented: 0085_ent720_email_identity <- 0084_agent_loops_chain_depth. SQLite list keeps both entries, dev's first — the same resolution train #3126 was gated on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trinity-ability
pushed a commit
that referenced
this pull request
Oct 9, 2026
…#3432) The hand-written fixture DDL lacked the chain_depth column that #3120 added to SchedulerDatabase.create_execution, so every agent-path INSERT in the scheduler tier raised OperationalError (70 failed / 173 passed). With the column: 3 failed / 240 passed; the remaining three are a separate drift (claim_token), left out of this fixture-only commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trinity-ability
pushed a commit
that referenced
this pull request
Oct 9, 2026
…#3432) The hand-written fixture DDL lacked the chain_depth column that #3120 added to SchedulerDatabase.create_execution, so every agent-path INSERT in the scheduler tier raised OperationalError (70 failed / 173 passed). With the column: 3 failed / 240 passed; the remaining three are a separate drift (claim_token), left out of this fixture-only commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vybe
added a commit
that referenced
this pull request
Oct 11, 2026
* test(scheduler): conftest schedule_executions DDL carries chain_depth (#3432) The hand-written fixture DDL lacked the chain_depth column that #3120 added to SchedulerDatabase.create_execution, so every agent-path INSERT in the scheduler tier raised OperationalError (70 failed / 173 passed). With the column: 3 failed / 240 passed; the remaining three are a separate drift (claim_token), left out of this fixture-only commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scheduler): remove the process-schedule remnant from the scheduler source (#3432) The scheduler still created process_schedules / process_schedule_executions at boot, re-read them every sync tick, and executed rows by POSTing to /api/processes/{id}/execute, a route the backend lost in #430. Nothing can create a row. Removed: the boot DDL, the snapshot + sync, the job add/remove/execute path, the skipped-execution recorder and its max-instances branch, the database section, the two models, the stranded _INTEGRITY_ERRORS import. The two tables are left in place on existing installs (no migration). The agent-schedule path, its lock-denial audit (#1969/#1808) and reminder machinery (#1296) are unchanged; the _on_job_max_instances unknown-prefix warning stays (reminder jobs share the listener). Red-first tests: test_legacy_process_schedule_tables_are_ignored (an enabled legacy row registered process_schedule_ps-legacy at HEAD) and test_on_job_max_instances_legacy_process_prefix_is_unknown (the legacy id reached _record_skipped_process_schedule at HEAD). Tests that exercised the removed identifiers are deleted or re-pinned in this commit: test_1994 deleted, process cases cut from test_skipped_executions / test_1474 / test_sync_loop, test_1296's ensure_process_schedules_table call dropped, test_1969's slice end re-anchored on `async def _execute_schedule(`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scheduler): test docstrings stop naming the removed process-schedule writer (#3432) test_1832 narrated three duration writers (one was the deleted process_schedule_executions finalizer); test_1823 pointed at the deleted test_1994 as its structural model, now test_1969. Docstrings only; the scanner logic is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scheduler): docs, run-core.sh and nightly workflows drop the process-schedule remnant (#3432) - tests/run-core.sh and both nightly workflows drop --ignore=process_engine (the directory no longer exists); the workflow comments stop citing a run-core.sh line number. - test_1896's literal --ignore=process_engine assertion becomes a parity check: the workflow's --ignore set must equal run-core.sh's (red with run-core.sh edited alone, green once both sides match). - scheduler-service.md: Flow 3 (process execution) removed and Flows 4-10 renumbered 3-9; process sync, events, max-instances branch, DB operations and method rows removed; revision history entry added. - requirements/scheduling.md, architecture/execution.md: the scheduler has one duration finalizer, not two. - architecture/database.md: the two tables are recorded as orphans left on upgraded installs (no migration). - architecture.md Invariant #16: "+ process-schedule variants" removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(scheduler): conftest schedule_executions DDL carries claim_token (#3433) The hand-written fixture DDL lacked the claim_token column (TEXT, as in db/schema.py) that SchedulerDatabase's require_unclaimed guard (#2514) reads, leaving three scheduler tests red: test_1472::test_a2_dispatch_exception_advances_next_run and two in test_async_dispatch. With the column: 243 passed, 0 failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(testing): strategy doc stops listing the removed process_engine exclusion (#3432) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Trinity Agent (trinity) <trinity-agent@ability.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: trinity-ability <309458136+trinity-ability@users.noreply.github.com>
vybe
added a commit
that referenced
this pull request
Oct 11, 2026
…references (#3433) (#3472) * test(scheduler): conftest schedule_executions DDL carries chain_depth (#3432) The hand-written fixture DDL lacked the chain_depth column that #3120 added to SchedulerDatabase.create_execution, so every agent-path INSERT in the scheduler tier raised OperationalError (70 failed / 173 passed). With the column: 3 failed / 240 passed; the remaining three are a separate drift (claim_token), left out of this fixture-only commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(scheduler): conftest schedule_executions DDL carries claim_token (#3433) The hand-written fixture DDL lacked the claim_token column (TEXT, as in db/schema.py) that SchedulerDatabase's require_unclaimed guard (#2514) reads, leaving three scheduler tests red: test_1472::test_a2_dispatch_exception_advances_next_run and two in test_async_dispatch. With the column: 243 passed, 0 failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(services): remove dead service symbols, checkpoint A1 (#3433) Removes symbols with no production caller on the live tree (re-verified over src, docker, scripts, tests, docs/memory and the enterprise submodule immediately before deletion): - ssh_service: cleanup_expired_keys, list_active_keys, cleanup_agent_keys (cleanup_agent_credentials itself is kept; see #3433 adjacent finding) - telegram_media: process_photo, process_document (+ os/tempfile imports) - github_service: get_github_service, GitHubAuthError, GitHubPermissionError - monitoring_alerts: alert_stuck_execution - subscription_auto_switch: handle_rate_limit_error - template_service: get_name_from_template - skill_service: _persisted_commit - settings_service: clear_template_registry_lkg - system_agent_service: is_registered - slot_service: is_at_capacity + get_active_count (its only caller) - slack_service: verify_slack_signature (+ get_slack_signing_secret import) - heartbeat_service: heartbeat_status (T1: 2 unique tests re-pointed to heartbeat_status_bulk, 4 duplicates deleted) Tests that only exercised a removed symbol are deleted; memory docs that described them are corrected. Targeted unit files: 164 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(services): remove dead service symbols, checkpoint A2 (#3433) Removes symbols with no production caller on the live tree (re-verified over src, docker, scripts, tests, docs/memory and the enterprise submodule immediately before deletion): - dispatch_breaker: get_all_dispatch_states (live reader is get_dispatch_states_for; its Redis-None fail-open is already pinned by test_states_for_no_redis_is_all_closed) - idempotency_service: derive_schedule_key (scheduler builds sched:{id}) - agent_mcp_key_service: build_mcp_key_env_overrides (test re-pointed to _mint_and_build_env) - mcp_tool_names: compute_tool_name (3 slug tests re-pointed to compute_tool_names) - agent_client.http_pool: _get_http_client (tests use _acquire_client) - agent_call_limiter: _set_cancel_sync - a2a_payment_gate: AllowlistRefused - a2a_gate: get_skills_provider (T6) - compatibility.static_checks: _is_skill_md - git_service.trinity_files: _persistent_state_for (4 fallback tests re-pointed to _read_trinity_yaml_list over the persistent-state file) - skill_sets: split_set_ref - event_bus: _serialize - backlog_service: get_backlog_service + the _backlog_service global redelivery_governor.distinct_failing_count is kept (T2: declared test/observability accessor). Targeted unit files: 374 passed, 32 skipped (lupa-less fakeredis + a pre-existing #1609 skip). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(adapters): remove dead adapter, db and utils symbols, checkpoint B (#3433) Removes symbols with no production caller on the live tree (re-verified over src, docker, scripts, tests, docs/memory and the enterprise submodule; DatabaseManager delegates by name, no __getattr__ passthrough): - message_router: _get_channel_timeout + _DEFAULT_CHANNEL_TIMEOUT (caller removed on purpose in #221; the channel_timeout_seconds setting has no other reader, and the docs now say so), _format_file_size, _sanitize_filename (+ _FILENAME_MAX_LENGTH, _FILENAME_SAFE_CHARS_RE and the os/unicodedata/format_file_size imports only it used). The copy was code-identical to upload_service.sanitize_filename, the live path; its 8 tests are re-pointed there (T4), as is the file-size test. - voip_audio: pop_frames - db.schedules.webhooks: set_webhook_enabled, _decrypt_webhook_secret - db.slack_channels: unbind_channel - db.telegram_channels: delete_groups_for_binding - db.role_readiness: delete_role_readiness (+ the delete import) - db.agent_cleanup: find_orphan_agent_names - utils.password_validation: is_password_weak - scheduler database: should_freeze_schedules (T3: asserts re-pointed to sync_freeze_reason) agent_cleanup.registered_columns is kept (T2: parity-guard accessor). Targeted unit files: 146 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scheduler): remove dead scheduler symbols, checkpoint C (#3433) Removes scheduler symbols with no production caller (re-verified over src, docker, scripts, compose, tests, docs/memory and the enterprise submodule; the backend's update_schedule and asyncio's loop.run_forever are unrelated same-name symbols): - service: add_schedule, remove_schedule, update_schedule, reload_schedules (the dead push-API section; the scheduler syncs by polling) and run_forever. The container runs `python -m scheduler.main` -> SchedulerApp.start() -> _run_until_shutdown(), which carries its own heartbeat + sync loop; no compose command override, no script calls it. - database: get_recent_executions - locking: is_schedule_locked - models: TriggerSource, AgentTaskResponse + AgentTaskMetrics Tests: add/remove job tests re-pointed to _add_job/_remove_job; the disabled-schedule test is deleted with add_schedule (its enabled gate existed only there); update/reload tests and the reload reminder-reclaim test deleted (reclaim stays covered via _sync_schedules/_reconcile_reminders); the skipped-execution probe now reads the row with a direct SELECT. No process-schedule symbol is touched (#3432 owns those); reload_schedules is deleted whole, so a textual conflict with #3432's cut inside it resolves by taking the deletion (T5). scheduler_tests tier: 235 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(heartbeat): bulk test pins the payload fields the single-agent test carried (#3433) A1 deleted test_status_alive with heartbeat_status. Its unique asserts (heartbeat_memory_mb, heartbeat_active_executions, the last-heartbeat age bound) now live on test_bulk_mixed_states_single_round_trip, the live path. Mutation-checked: nulling the memory_mb / active_executions pass-through in _compute_status turns this test red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(scheduler): sync loop refuses and drops disabled schedules (#3433) The deleted test_add_disabled_schedule only exercised the dead add_schedule push API. Pin the live sync path instead: a disabled row (present at boot or inserted between ticks) never becomes a job, and a schedule disabled between ticks is removed on the next sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: ssh cleanup and scheduler client snippets describe the code as it is (#3433) - ssh-access / async-docker-operations: cleanup_agent_credentials() exists but no stop/delete path calls it; say so instead of claiming it runs at agent stop/delete (follow-up bug). - scheduling / execution-queue / model-selection / scheduler-service: src/scheduler/agent_client.py was removed in v0.7.0. The scheduler dispatches through _call_backend_execute_task -> POST /api/internal/execute-task; the backend's TaskExecutionService POSTs the agent's /api/task via agent_post_with_retry. Replace the snippets and drop the file-table rows that named the missing module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(services): drop the empty section headers the dead-symbol removal left behind (#3433) slack_service.py kept a 'Request Verification' banner and a module-docstring bullet for the removed verify_slack_signature (the live check is adapters/transports/slack_webhook.py::_verify_signature); github_service.py kept a trailing 'Factory Function' banner for the removed get_github_service. Comment-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Trinity Agent (trinity) <trinity-agent@ability.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: trinity-ability <309458136+trinity-ability@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
inter_agent_max_chain_depthcould reset its depth to 0 by starting a loop, manually triggering a schedule, driving a chat-session turn, or emitting an event its subscription answers. All four now inherit the caller's depth and are refused at the max with the same named 403inter_agent_depth_exceeded.chain_depthclaim in the EVT-001 loopback JWT, so an emit on another agent's behalf and systemagent.task.*completions (finished row's depth + 1) carry it too._chain_callerfalls back tovouched_source_agent.event_dispatch_max_fires_per_hour, default 120, one high-priority notification per window, fail-open on Redis errors.Changes
routers/{loops,schedules,sessions,event_subscriptions}.py— callenforce_inter_agent_depth; refusal propagates to the new app-level handler (error_handlers.inter_agent_depth_exceeded, registered inmain.py).services/loop_service.py,db/loops.py+agent_loops.chain_depth(SQLite migrationloop_chain_depth, Alembic0084,schema.py,tables.py).src/scheduler/*—ExecutionOrigin.chain_depth(validated 1–1000), stamped on insert, kept on retry.services/event_dispatch_service.py— loopback claim, terminal-event depth, dispatch budget;dependencies.py/models.User.loopback_chain_depth;services/dispatch_admission_service.py— claim read before the root early-return.services/task_execution_service.execute_task(chain_depth=).services/monitoring_alerts.py— budget alert;config.py/settings_service.py— new ops setting.client.ts::depthRefusalFromError;run_agent_loop,trigger_agent_schedule,emit_eventreturn the refusal as a non-retryable result.requirements/core-agent.md§9.1.1,requirements/mcp.md,architecture/{execution,database,api-endpoints}.md, feature flows, IA-04 gaps, learnings fragment, CSO diff report.Test Plan
cd tests && pytest unit/test_2973_depth_new_roots.py unit/test_2973_event_dispatch_budget.py -v— every path driven through its real route with a minted agent bearer and a non-default max (34 + 10 pass)src/mcp-server/src/tools/depth-refusal.test.tsnode --test600/600;tsc --noEmitcleancheck_alembic_heads.pysingle head;check_alembic_parity.pyPASSDeferred: webhook tokens, agent-created cron schedules and self-reminders still start a depth-0 root — #3116 (needs a design call on whether a stored trigger carries its creator's depth forever).
Fixes #2973
🤖 Generated with Claude Code