Repository navigation
feat(sync-health): sync health on fleet health, the fleet audit over MCP, and the agent card (trinity-enterprise#707) - #3036
Merged
Merged
Conversation
…-enterprise#706) Amends §11.8 (health now measures agreement with the repository, not only push success) and adds §11.19: the four persisted columns, the one policy module and its scheduler mirror, the work-agent predicate, the state rules, the freshness-guarded freeze and the one-item-per-episode alert. Refs Abilityai/trinity-enterprise#706
…ty-enterprise#706) services/sync_freeze_policy.py::classify decides an agent's sync state (green / yellow / red / unknown), the operator-readable reason, the recommendation and whether its schedules freeze. Stdlib only, never raises, vendored byte-identically to src/scheduler/ (the #1088 precedent) with a parity test. The scheduler's SYNC_FAILURE_FREEZE_THRESHOLD now comes from the mirror, so there is one number. Nothing calls classify yet; the poller, the gates and the surfaces follow. Refs Abilityai/trinity-enterprise#706
…count, last push (trinity-enterprise#706) agent_sync_state gains diverged_since, dirty_files, dirty_since and last_successful_push_at, on both tracks: the SQLite migration agent_sync_state_divergence and Alembic 0080_agent_sync_state_divergence (parent 0079_telegram_group_context, the live head), plus schema.py and tables.py. All nullable, no backfill: the clocks start at the first poll after upgrade, which is the 24 h soak before any divergence freeze. The upsert takes a KEEP sentinel for the two clocks so a caller can say clear (None) as distinct from unchanged, which _merged() cannot express. Refs Abilityai/trinity-enterprise#706
…d the last push (trinity-enterprise#706) On every poll, inside the leased cycle: diverged_since is set once when the working tuple (origin on the agent's own branch) shows ahead or behind, cleared when both are known 0 (or ahead is 0 with no upstream), and kept when ahead could not be computed. dirty_files comes from the changes_count the status call already makes, with dirty_since as its own episode clock. last_successful_push_at is the later of the heartbeat's #3011 record, an operator Push and the stored value; an image older than #3011 falls back to its last success cycle. Agent timestamps are validated (explicit offset, not in the future) before they can win a max(). Refs Abilityai/trinity-enterprise#706
…ce freeze (trinity-enterprise#706)
When a divergence freeze is in force, the poller (inside its leader lease)
raises one sync_diverged item with the deterministic id
sync-diverged-{agent}-{diverged_since}. create_item's (agent_name,
request_id) conflict target makes repeat polls, a restart and a fail-open
double leader converge on one row; a new episode has a new id. The prefix
is platform-reserved so an agent cannot pre-create the id and silence the
alarm about itself. No item for red-but-not-frozen; not auto-resolved. The
context carries counts and the recommendation, no agent-authored error text.
Refs Abilityai/trinity-enterprise#706
… agents (trinity-enterprise#706)
Both enforcement points now decide with the one policy module:
- src/scheduler/database.py: sync_freeze_reason() returns why an agent's
cron firing is frozen (or None), reading the divergence inputs in the same
fail-open query; should_freeze_schedules() is its bool view. A reason
string, not a (bool, reason) tuple, because a tuple is truthy and a caller
on the old shape would freeze the fleet.
- src/scheduler/service.py: the skipped row's skip_reason carries the
reason ("Git sync frozen: diverged 0 behind / 7 ahead for 26h"), and the
comment no longer calls the row count bounded.
- routers/internal.py: sync-health-status keeps its keys and adds
freeze_reason, divergence_age_s and work_agent via the new
services/sync_health_view.py, which the poller now uses too.
Refs Abilityai/trinity-enterprise#706
…verdict (trinity-enterprise#706)
GET /api/agents/sync-health and GET /api/agents/{name}/git/sync-state add
state, reason, recommendation, binding, freeze, the divergence age and the
new columns; every pre-existing key is unchanged. MCP get_git_sync_state
passes the backend JSON through, so it carries them too with no TS change.
The batch now reads through one shared query, db.list_sync_health_rows
(agent_git_config joined to live ownership, left-joined to
agent_sync_state), so the dot and the poller see the same set of agents.
Refs Abilityai/trinity-enterprise#706
…(trinity-enterprise#706) utils/syncHealth.js no longer owns a threshold: classifySyncHealth maps the backend's state (unknown -> gray) and syncHealthLabel shows its reason and recommendation on hover. The 24 h / 7 d / behind-is-red rules are gone from the frontend, so the dot cannot disagree with the scheduler's freeze. Visible change: an agent ahead of origin with auto-sync off used to render gray (it had never synced); it now renders the backend's yellow or red. Refs Abilityai/trinity-enterprise#706
…freeze (trinity-enterprise#706) Architecture (agent-lifecycle Git Sync Health, database DDL, background services, backend catalog, api-endpoints), the git-sync-health flow (data model, poller, the new §2b policy and freeze, the dot, tests, controls, known limitations), the github-sync journey note, the feature-flows index row, the RD-03 invariant note, and the user docs: what the sync state means, when schedules pause and resume, the new Operating Room entry. The MCP get_git_sync_state description names the new fields (the tool is a pass-through; no behaviour change). Refs Abilityai/trinity-enterprise#706
…nterprise#706) When the agent cannot compute its working tuple (a git timeout sends ahead_working/behind_working as null), the poller correctly KEEPS diverged_since, but _coerce_counter still stored the unknown counts as 0. The row then read "diverged 25 h" next to 0/0: the freeze fired with the self-contradicting reason "diverged 0 behind / 0 ahead" and the episode's one sync_diverged item carried ahead 0 / behind 0. Pass None (upsert's "keep the prior value") for the two working counts exactly when the clock is kept on an unknown tuple. Scoped to that case: with no episode to keep, an unknown tuple is still stored as 0 (#2827). Found by /review on #3035. The new test was red before the fix (assert (0, 0) == (7, 31)); the control arm is green before and after.
…trinity-enterprise#706)
…the agent card (trinity-enterprise#707) The requirement for the #707 read surfaces, written before the code: a per-agent sync block and fleet totals on /api/monitoring/status, the recommendation in issues[], the fleet sync audit's new columns and its MCP tool, the agent-card sync chip, access scoping, and no raw git error text in any surface's reason. Refs Abilityai/trinity-enterprise#707
…trinity-enterprise#707) GET /api/monitoring/status annotates every agent with a sync block read through the shared reader and the one policy: binding, auto_sync_enabled, ahead/behind on the agent's own branch, dirty_files, last_successful_push_at, divergence_age_s, state, reason, recommendation and frozen, or null for an agent with no git binding. The response gains fleet totals in sync_summary, and a red agent gains one 'sync: <reason> — <recommendation>' issue. Sync is an annotation like the heartbeat layer: status, the sort and the health counts are unchanged. The merge is computed before it is applied and degrades on its own to null on a fault, never into the aggregation-failed payload. sync_view now takes reason and freeze_reason from the verdict computed with the git error withheld, so no surface's reason carries the agent-written stderr (this also covers the sync_diverged item's question). The recommendation still reads the error. The scheduler's skip row, which calls the policy directly, is unchanged. Refs Abilityai/trinity-enterprise#707
…rdict (trinity-enterprise#707) build_fleet_sync_audit keeps every pre-#707 key and adds, per agent, ahead, behind, dirty_files, diverged_since, divergence_age_s, last_successful_push_at, state, reason, recommendation, binding, auto_sync_enabled and frozen from the same sync_view call every other surface uses. dirty_tree is now real (dirty_files > 0) instead of a hard-coded false, so the summary's in_sync and dirty are real too; the summary adds diverged, frozen, auto_sync_off and red. Refs Abilityai/trinity-enterprise#707
… (trinity-enterprise#707) get_fleet_health passes each agent's sync block (null without a git binding), the fleet sync_summary and the issues strings through, next to the kept issues_count, so an orchestrator sees the red-sync recommendation. New get_fleet_sync_audit reads GET /api/fleet/sync-audit with the caller's key and returns the audit unchanged. No agent target, so its policy row is 'none': the backend scopes the rows through accessible_agent_names, and an agent key sees its owner's set (decision D12), as get_fleet_health already does. routers/fleet.py's '# mcp:' header now names it (Invariant #13). Refs Abilityai/trinity-enterprise#707
…st push (trinity-enterprise#707) The Fleet tile carried 'sync failing ×N' (with the raw git error as its tooltip) and a bare 'git ✓'; an agent 15 commits ahead with auto-sync off showed neither. It now carries one sync chip: the kind follows the backend's state (red crit, yellow warn, green calm, nothing for unknown), the text is '↑7 ↓0 · 12 dirty · pushed 3h ago', and the tooltip is the reason and the recommendation plus the absolute push time. One chip replaces two, so the strip does not grow; a problem leads it and a calm fact trails it. Agent Detail → Overview kept only consecutive_failures and said 'ok' for every agent that had not failed a push, including agents with no git binding. It now keeps the whole /git/sync-state payload and renders the same numbers in the state colour; it sits on a chrome fill, so status ink steps up one tier (800 light / 300 dark: the 700/400 tier measured 4.47:1 and 3.73:1 there in the browser), '—' with no observation or after a failed read. The attention count still counts failed syncs (D13). The formatter is its own module, utils/syncSummary.js, taking 'now' from the caller: syncHealth.js keeps its #706 guarantee of no clock and no counts. Semantic tokens only; per-file raw-colour and loading-gate counts unchanged. Specs mount AgentTile and OverviewPanel. Refs Abilityai/trinity-enterprise#707
… and the agent card (trinity-enterprise#707) Architecture: the monitoring.ts row gains get_fleet_sync_audit and the sync pass-through (130 tools); api-endpoints gains the /api/monitoring/status row and the sync-audit additions. Feature flows: git-sync-health (§4 audit, §5 the card, files, tests, the dirty_tree limitation closed, the scheduler skip row's error excerpt recorded), agent-monitoring (the sync merge), dashboard-grid-view (the chip) and a dated index row. User docs: monitoring, managing agents, GitHub sync. Refs Abilityai/trinity-enterprise#707
AndriiPasternak31
added
the
ui
#3005 (skill sets) landed 0080_agent_skill_sets on 0079. This branch's Alembic revision becomes 0081_agent_sync_state_divergence on top of it, and the SQLite entry follows agent_skill_sets in MIGRATIONS. No change to what the migration does; docs updated to the new revision id.
|
✅ Alembic head check clear — merging this PR into Previously flagged; resolved. Evaluated on the version line only — this PR also conflicts with Advisory — this check does not block merge. · head_sha: |
…kes the state tint (trinity-enterprise#707) From manual sign-off on the sign-off stack: - The tile chip's ⟳ read as a dot at 9.5px. It is now its own aria-hidden element at 2x, in a fixed 11px box so the chip keeps its 12.22px height, and nudged up 0.09em: the fallback font draws its ink 1.5px below the box centre (measured in the browser; 0.12px after the nudge). - The Overview 'Sync:' pill takes the state tint as a whole on red and yellow (status-* 50 / 900/30 fill, inset ring so it keeps its siblings' height). Green and unknown keep the neutral chrome of the sibling pills.
Contributor
Author
|
Manual sign-off done on a local stack, with seeded agents in the red (frozen), yellow and green states plus one with no git binding. Grid, List and Agent Detail → Overview were checked in the browser. Two changes came out of it, in
Tests: new specs for the pill tint and the glyph element, and the two chip-text assertions updated because the glyph is no longer followed by a space character. Still a draft: it's stacked on #3035 and gets rebased onto |
#3005 (skill sets) landed 0080_agent_skill_sets on 0079. The ent#706 Alembic revision this branch carries becomes 0081_agent_sync_state_divergence on top of it, and the SQLite entry follows agent_skill_sets in MIGRATIONS -- the same resolution as #3035's merge 9229f08. Docs name the new revision id.
6 tasks done
|
Resolve by running |
|
Resolve by merging |
Alembic 0081_agent_sync_state_divergence -> 0082, down_revision 0081_portal_messages_unread_idx; SQLite entry after portal_messages_unread_index. Docs/registry conflicts keep both sides.
…07: take the 0082 re-chain
…06-707 # Conflicts: # tests/registry.json
AndriiPasternak31
marked this pull request as ready for review
September 29, 2026 18:46
Contributor
Author
…to AndriiPasternak31/ent707
…06-707 # Conflicts: # docs/memory/feature-flows.md
# Conflicts: # docs/memory/feature-flows.md
# Conflicts: # docs/memory/feature-flows.md # docs/memory/feature-flows/git-sync-health.md # docs/memory/requirements/github.md # docs/user-docs/integrations/github-sync.md # src/backend/services/sync_health_view.py # tests/registry.json
Contributor
|
merge-train: this PR is on the train. /validate-pr and /review both came back READY with no criticals, and the tests run the real routes and mounted components. I made one mechanical edit: the body now says Notes, not blocking:
|
vybe
approved these changes
Sep 30, 2026
vybe
left a comment
vybe
left a comment
Contributor
There was a problem hiding this comment.
merge-train: validated (validate-pr + review, mechanical fixes noted on the PR); merged individually — batch under 4, no train assembled.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync health on the surfaces people and orchestrators actually read: fleet health, the fleet audit over MCP, and the agent card
Fixes abilityai/trinity-enterprise#707 (PR-B of two).
Problem
#706 made the backend decide sync health, but the surfaces an operator or an orchestrator looks at still didn't show it:
get_fleet_health/GET /api/monitoring/statusreported containers and context only.# mcp: none), and itsdirty_treewas hard-codedfalse.So the 2026-09-24 audit (49 agents with auto-sync off, 40 unpushed commits, 791 dirty files) had to be assembled by hand inside containers.
What changes
Read path only. No schema, poller, scheduler or base-image change.
GET /api/monitoring/status, MCPget_fleet_health)syncblock:binding,auto_sync_enabled,ahead,behind(origin on the agent's own branch),dirty_files,last_successful_push_at,divergence_age_s,state,reason,recommendationandfrozen. An agent with no git binding getssync: null.sync_summary:{git_bound, diverged, frozen, auto_sync_off, dirty, red, yellow}.issues[]entry:sync: <reason> — <recommendation>.status, the sort, or the healthy/degraded counts (D11).sync: null/sync_summary: null, never to the "aggregation failed" payload.GET /api/fleet/sync-audit)ahead,behind,dirty_files,diverged_since,divergence_age_s,last_successful_push_at,state,reason,recommendation,binding,auto_sync_enabledandfrozen.dirty_treeis now real (dirty_files > 0), so the summary'sin_syncanddirtyare real too.diverged,frozen,auto_sync_offandred.get_fleet_sync_audit, which returns the audit unchanged (client.ts,tools/monitoring.ts, and theaccess.tspolicy row).get_fleet_healthpassessync,sync_summaryand theissuesstrings through, next to the keptissues_count.routers/fleet.py's header is now# mcp: monitoring.ts (get_fleet_sync_audit).accessible_agent_names, so admins see all and others see their owned and shared agents. An agent-scoped key resolves to its owner and sees the owner's set, because an orchestrator's sweep runs as an agent. The policy row isnone(no agent target), the same asget_fleet_health.sync_viewnow computesreason/freeze_reasonwith the agent-writtenlast_error_summarywithheld, while state, freeze and the recommendation still read it (a denied push still sayscredential is read-only). This settles the question feat(sync-health): divergence age is the signal — red and frozen at >24h, persisted on the row (trinity-enterprise#706) #3035's body left to bug: workspace canvas panel flickers + charts broken in update_panel #707 (D12), for every backend surface: fleet health, the audit, the dashboard batch,/git/sync-stateand feat(sync-health): divergence age is the signal — red and frozen at >24h, persisted on the row (trinity-enterprise#706) #3035'ssync_divergeditem question. The raw text stays only in the explicitlast_error_summaryfield of the two per-agent reads it was already on.AgentTile.vue): one sync chip replacessync failing ×Nandgit ✓, so the strip doesn't grow.↑7 ↓0 · 12 dirty · pushed 3h ago.Scheduled runs are paused until it syncswhen a freeze is in force./git/sync-statepayload and renders the same line in the state colour.—when there is no observation or the read failed. It used to sayokfor every agent that hadn't failed a push, including agents with no git binding.utils/syncSummary.js: display only, withnowpassed in.Behaviour changes to know before merging
Sync: —instead ofSync: okfor an agent with no observation or no git binding.reasonon the dashboard dot tooltip and/git/sync-stateno longer includes the first 120 characters of the git error (e.g.last sync failed (seen on 3 polls)). The recommendation still names the fix. The error itself is still inlast_error_summaryon those per-agent reads.skip_reasonfor async_failingfreeze still carries the error excerpt. The scheduler calls the vendored policy directly and is out of this read-only PR's scope. This is recorded under Known Limitations.Evidence
test_ent707_fleet_health_sync.py10/10 red → green;test_fleet_sync_audit.py5 new red (9 old green) → 14/14; MCPmonitoring.test.ts7/7 red → green; frontend specs 12 red → green; the I2 tests (tile + helper) red → greenf800e91b1)npm test550 → 557 passed andtsc --noEmitclean; frontendvitest3693 → 3728 passedTOOL_ACCESS_POLICYtotality)npm run test:unitnpm run check:tokens/npm run buildsyncSummary.js0/0/0 (semantic only). Loading gates 70 → 70tests/lint_sys_modules.pycd tests; pytest unit -m "not slow", py3.13): basef800e91b1vs tip/verify-local(global,VERIFY_PYTHON=python3.13, foreground, sibling stack,--skip-agentper plan: no base-image change)--skip-unit: build+import-smoke OK, boot+health OK, integration 70 passed / 13 skipped → RESULT: PASSdb)/api/monitoring/status: thesyncblock, the issue string andsync_summaryexactly as specified, withstatusunchanged;/api/fleet/sync-audit:dirty_tree: true, new keys, summary; error marker absent from both payloadsget_fleet_sync_auditadvertised and returns the audit;get_fleet_healthcarriessync,sync_summaryandissues--gv-*blocks lifted fromFleetGrid.vue).claude/src/backend/enterprisegitlinks547b39fc/f769d07d), equal toorigin/devDecisions (posted on trinity-enterprise#706 on 2026-09-27; none overruled as of this writing)
AgentHealthSummary.statusget_fleet_sync_auditfor agent keysconsecutive_failuresNot verified
/sync-fleet-to-heador corbin's sweep running on the new surface against a realgithub:agent (the numbers matching in-container git within one poll) was not run. The live checks used seeded rows, not a real repository.--skip-agent; this PR has no base-image change).Reviewer notes
src/frontend/, so it needs theuilabel forfrontend-e2e.architecture/mcp-server.md).requirements/github.md§11.19 (the fleet-surfaces block),requirements/infrastructure.md§12.8,architecture/{api-endpoints,mcp-server}.md,feature-flows/{git-sync-health,agent-monitoring,dashboard-grid-view}.md, a datedfeature-flows.mdindex row, and the user docs for monitoring, managing agents and GitHub sync.🤖 Generated with Claude Code