fix(agents): derive worker executing state from lane and lease facts, not Todo timestamps

[songoow]

Why

Roadmap R2 requires distinguishing registered, addressable, bound, launchable, executing and blocked workers. _agent_state returned executing whenever the current Todo had been updated in the last eight hours, with no execution fact behind it. Peer activation admission counts executing as active, so a dead peer with a fresh Todo was admitted.

What changed

• Lock-free lane liveness. lock_holder_liveness in the file-lock owner and turn_lane_liveness on top of it classify a lane's last Turn from its holder record alone: released, dead (same host, pid gone), foreign_host, unreadable, live (same host, pid alive) or absent. The probe never touches the kernel lock; a probe that took it for an instant would make a racing real run-once --execute fail with turn_lane_in_flight. The holder host label is single-sourced so writer and readers agree on "this machine".
• Execution facts. agents/execution_facts.py is a read-only collector over facts LoopX already keeps: Turn lane liveness per Goal and agent, the delegation worker's operation lock, and the task lease on each open claimed Todo. Status collection passes them to the projection, each agent row carries them as execution, and status names the sources in a typed execution_facts summary.
• Derivation order. blocked, monitoring/waiting, then executing only when the lane is live or a delegation worker holds its lock, then the new unknown (holder on another host, unreadable record, or active lease expired with nothing live), then bound/launchable/addressable/registered. EXECUTING_ACTIVITY_THRESHOLD_HOURS is removed; Todo timestamps stay only in last_activity_at and stale_claim_hint.
• Consumers. Peer activation admission is an allow-list: unknown and any unlisted state fail closed as not active. The peer directory re-projects the row facts and drops lease_state_not_projected when facts are present.
• Census. The status registry read moved; the manifest is regenerated.

Behavior change disclosure

Open work updated within eight hours no longer reads as executing. Without a live lane or worker it is bound or launchable, and a holder that cannot be checked is unknown, which peer admission treats as not active. Lanes affected: status agent rows, peer directory, peer activation admission. The lifecycle tests and examples/worker-lifecycle-state-smoke.py encode the new rule.

Entry points

No frontend change is needed: the dashboard renders delegation preflight states (delegation-preflight-status.tsx, goal-team-work.tsx), a different vocabulary, and does not label worker lifecycle states. CLI and status JSON carry the new state value.

Checks run on this head

Check	Result
pytest tests/control_plane/test_agent_lifecycle_state.py tests/test_turn_lane_fence.py tests/control_plane/test_peer_agent_directory.py	passed, including the concurrent probe-versus-Turn test
pytest tests/architecture/	821 passed
node --test tests/control_plane_ts/peer_orchestration.test.ts	4 passed, 0 failed
npm run typecheck:control-plane	ok
examples/worker-lifecycle-state-smoke.py	all 6 lifecycle states verified
examples/semantic-vocabulary-drift-smoke.py	ok
loopx canary premerge --from-git-diff	passed: tier=standard, changed_files=14, surfaces=control_plane/public_boundary/python; selected=18, failures=0

Not run: a real multi-host fleet; foreign_host is covered by fixtures only.

Coordination with other open PRs

• codex/delegation-stop is stacked on this PR's lane-liveness commit and reuses lock_holder_host_label() and the lock-free lane read instead of a lock-taking probe. That commit drops out of the stop PR once either merges.
• fix(work-items): bind agent-originated team-plan settlement to the same basis and fail all-gap plans #5302 and feat(collaboration): wake the requesting lead once a delegated result is accepted #5304 also regenerate loopx/semantics/project_registry_io_manifest_v1.json; whichever merges later regenerates it again.

Rollback

Revert the PR. The collector only reads existing facts and writes nothing, so no state needs migration.

Bounded future-facing refactor

Applied: the holder host label and lane liveness read are single-sourced in the file-lock owner, so the stop PR reuses them instead of adding a second helper. Deferred: a presence provider for remote hosts; foreign_host stays unknown until R6 supplies authenticated host identity.

This is a control-plane change; it is left for maintainer review and merge.

🤖 Generated with Claude Code

[huangruiteng]

English verdict: REQUEST_CHANGES — the delegated-worker observation can attribute another worker's execution slot to a historical result reader.

Reviewed exact head: 3bbe2dfdb31b5fdb117d1b04c002b40771f90e5b, against immutable merge base 9c29941559cff92f675c37cc80c4cf87a23f8448. This review follows the current LoopX PR-review capability execution contract.

[P2] 同一个 Todo 的两个 live holder 不能证明旧 operation 的 Agent 正在执行

位置：execution_facts.py:122–132。_delegation_worker_agents 分别检查 operation lock 和 Todo execution slot 是否 live，却没有核对它们是否属于同一次执行，也没有排除已 accepted 的历史 operation。

独立复现：同一 Todo 的旧 operation 属于 old-peer 且已 accepted；current-peer 的新 operation 正在运行。子进程用真实文件锁持有新 operation 和共享 Todo slot，父进程仅持有旧 operation 的读取锁；两个 holder 的 PID 确实不同。collector 把两个 Agent 都标为 active，management projection 把 old-peer 也投影为 executing。同一输入通过真实 loopx status 入口，在 base 中 old-peer 为 registered，在本 head 中错误地变为 executing。新 worker 正确的执行状态不应成为旧结果读取者的执行证据。

最小修复：关联 operation 与 slot 的执行身份，并区别终态结果读取和实际 worker；保留无锁、只读观测，不为 status 新增 lease 或执行权。请在现有 lifecycle 测试中增加上述双进程负例，同时保留当前 worker 为 executing 的正例，覆盖 Todo 重用/重新绑定后的历史结果读取。

动机

这是 roadmap R2 中执行状态真实性的有价值修正：Todo 最近更新只说明有人写过任务，不说明进程仍在运行；反过来，长时间运行的 worker 也不应因为 Todo 时间旧而失去执行状态。用户需要从状态入口分清可启动、绑定、执行中和不可验证，而不是根据时间戳猜测。这一目标已在主路径得到改善，但上述历史 operation 反例仍违反同一验收边界。

改动思路

改动复用既有 status collection、management projection、peer directory 和文件锁 owner；新增 execution facts reader 只读取实际 lane、delegation slot 与 task lease。随后由现有投影优先级决定状态，TS peer admission 不再把 fresh Todo 当作 executing 的依据。位置总体合理，未新增用户激活按钮或 CLI 参数，也未新增平行任务调度器。需要补齐的是事实之间的关联，而非增加更多布尔标记或另一个运行时状态源。

具体改动

共 12 个文件，新增 1052 行、删除 81 行：生产读模型及锁观测、既有 TS admission 的说明、lifecycle/lane 回归、薄 smoke 和派生 registry I/O census。完整 diff 和未改动调用方均已阅读；测试数量不是完成 R2 的证明。

关键代码讲解

• lock_holder_liveness（loopx/file_lock.py:325）将 holder 观测分类为 live、dead、absent、foreign、unreadable；不抢锁、不授予权限。跨主机和不可读状态保留 unknown，PID 重用仍是已披露的观测限制。
• _delegation_worker_agents（execution_facts.py:99）从 requester journal 读取绑定 Agent，并与 Todo slot 组合。这正是阻塞点：两条独立 live 事实还不是同一个 worker 的事实。
• collect_agent_execution_facts（execution_facts.py:190）汇总 lane、worker 和 open Todo lease，canonical lease authority 不可用时不回退到已被替代的本地来源。
• _agent_state（management_projection.py:515）消费事实而非 Todo 新鲜度；blocked、monitoring、waiting 保持既有优先级。directory 再消费已有 projection，避免第二次投影丢失 execution facts。

验证是在各自 source-checkout 环境执行，而非使用另一安装版本。focused Python 70 项、文件锁/跨进程 18 项、TS peer orchestration 4 项、architecture 28 项通过，TS typecheck 通过。相同 fixture 的真实 CLI base/head 对比确认：fresh Todo 无 lane 从 executing 改为 launchable，旧 Todo 有 live lane 从 launchable 改为 executing，foreign holder 从基于时间戳的 executing 改为 unknown；registry/state 文件不被读取命令修改。双进程历史读取反例独立失败 1 项，是本次 REQUEST_CHANGES 的依据。

对主干的风险

这是默认读模型变化，影响 CLI/status、directory 以及消费同一投影的管理界面；不是默认关闭能力，故不能用 opt-in 隔离为它免责。现有 typed 状态枚举和 TS eligibility owner 被保留，无 substring denylist 或产品专属 obligation。错误的 executing 会误导 operator 对运行与恢复的判断；本评审没有据此声称它绕过原生执行授权。不能因为单个锁的进程活着，就把另一操作的生命周期也判为执行中。

语义与 CI 对齐

不查询、不轮询远端 CI。风险 canary 执行了 16 个 selected 检查及 5 个 direct 检查，唯一失败为 semantic twin budget：44 independently maintained py/ts twins; budget is 43。同一个原始检查在不可变 base 和本 head 上具有完全相同的失败细节，预算和因果路径未改动；census 全量再生成与提交内容字节一致，错误行位置的 mutation 仍被拒绝。因此该既有红检查单独留给其 owner，不要求本 PR 修改预算，也不作为此次拒绝理由。上述新引入的错误状态则有独立真实锁、CLI 和投影证据。

我的整体评价

方向和大部分入口修正有效，尚不能宣称 executing 总是由对应 worker 的事实支持。bounded future-facing pass：继续复用现有 projection/lock owner 是正确的；应在本 PR 内补齐身份关联和终态读取负例，不需要广泛 TS 迁移或新状态框架。未运行已安装 App 的端到端展示、跨主机执行或 live provider，不将本地投影验证扩大成这些声明。修复后需重新验证完整 exact head；这是 control-plane 变更，仍由 maintainer 决定合并。

[songoow]

New exact head: 4a71c3b4e63320edebf8cfcc041564b38f1da993 (was 3bbe2dfd). One blocking finding, addressed; merge of upstream/main included.

English verdict: FIXED — a delegation slot now attributes to one in-flight operation, not to whoever holds a lock for the same Todo.

[P2] The delegated-worker observation could attribute another worker's execution slot

Root cause: _delegation_worker_agents read two independent live facts — the operation lock, and the Todo's execution slot — without asking whether they belong to the same execution. Commit a40909c15.

Change in loopx/control_plane/agents/execution_facts.py: a slot now attributes to an operation only when all of the following hold, and the read stays lock-free and grants nothing:

• the row's status is in flight (prepared, running, turn_returned per the transitions in loopx/control_plane/collaboration/delegation.ts); accepted and rejected are final, so their holder is a reader;
• the operation holder and the slot holder are the same host and pid — the worker takes both in one process, readers do not;
• the operation was taken before the slot, because the slot is acquired nested inside it;
• when several in-flight operations of the slot holder qualify, only the latest is the worker; a tie names nobody (fail closed).

Also: a40909c15 initially kept a local [a-f0-9]{64} pattern for journal filenames. main now pins one owner for that whole-value shape, so 6da1f3bf7 borrows BARE_SHA256_PATTERN (as delegation_inventory does) and the reader is pinned in the digest-owner manifest.

Negative cases added to tests/control_plane/test_agent_lifecycle_state.py

Two-process historical reader (accepted old operation, live new worker) · reader that locked before the worker took the slot · settled operation under the slot holder · released and crashed records · reused pid, and the tie/ordering fallbacks · one agent's live lane not another's execution · stale lease still attributed to its own owner.

Evidence

Check	Result
pytest tests/control_plane/test_agent_lifecycle_state.py tests/test_turn_lane_fence.py tests/control_plane/test_peer_agent_directory.py tests/architecture/test_content_digest_single_owner.py	289 passed
pytest tests/architecture	1 failed, 1042 passed. The failure is test_turn_contract_generation.py::test_new_independent_twin_cannot_hide_behind_generated_pair (twin budget), reproduced identically on a clean upstream/main worktree — pre-existing, same raw failure you reported in the review
scripts/generate_project_registry_io_manifest.py --check	current, 260 sites
node --test tests/control_plane_ts/peer_orchestration.test.ts	4 passed
examples/worker-lifecycle-state-smoke.py / semantic-vocabulary-drift-smoke.py	6 states verified / ok
loopx canary premerge --from-git-diff --git-diff-base upstream/main	passed: tier=standard, direct 5/5, selected 16/16, failures=0

Mutation check — revert each rule, its test fails, restore: same-process → test_a_reader_that_locked_first_...; in-flight status → test_a_settled_operation_...; ordering, latest-operation, tie fail-closed → test_a_crashed_operation_whose_pid_was_reused_...; reverting both original rules → 3 failures; digest-owner fix → 2 failures in test_content_digest_single_owner.py.

End-to-end through the real loopx status entry point, one frozen fixture (open Todo on peer, an accepted operation for old-peer, a live worker process holding the new operation lock plus the Todo's slot):

PR head 3bbe2dfdb   old-peer -> executing {delegation_worker_active: True}
                    peer     -> executing
new head 4a71c3b4e   old-peer -> registered
                    peer     -> executing {delegation_worker_active: True}

Not covered: a real multi-host fleet and live providers; foreign_host and pid reuse remain fixture-only, and read()'s own worker_active field (same lock, different surface) keeps its existing meaning in this PR.

6d29e97fa merges upstream/main; the earlier 37c13859c merge carried the registry-IO census regeneration. Please re-review the exact head.

[huangruiteng]

本轮对 4a71c3b4e63320edebf8cfcc041564b38f1da993 的完整 base-to-head 改动重新评审，结论：没有阻塞性发现，APPROVE。上一轮的“历史 operation 读取者借用当前 worker 的 Todo slot”问题已通过独立真实进程/锁验证修复；不是沿用先前结论。

动机

Todo 最近被编辑，只能说明有过活动，不能说明执行器仍在运行。原规则会把没有进程的成员标为 executing，误导目录和 peer admission。当前改动交付的是 R2 的有界生命周期观测修复，不是远程 presence、完整多机执行或整个路线图验收。

改动思路

复用已有 Turn lane、delegation journal/operation lock、Todo slot 和 lease owner，生成只读投影；不新增手工维护的运行状态。先保留 blocked/monitoring 等既有优先级，再用可核对的执行事实识别 executing；foreign/unreadable holder 和无活体的过期 active lease 表示 unknown。时间戳只保留为诊断提示。

仅检查两个 lock 都“live”还不够：它们必须属于同一 host/pid，operation 必须仍有后续状态转换，并且是在 slot 之前取得的最新匹配 operation；不同成员并列时不猜测归属。这个规则同时避免旧结果读取者和不完整候选列表误认 worker。

具体改动

关键代码讲解

• lock_holder_liveness：输入既有 holder record，先区分 released/不可读/异机，再检查本机 PID。输出观测状态和过滤后的 holder，不获取 kernel lock；因此状态查询不会短暂抢占真实 Turn。异机不假装可验证，实际锁权威仍由原 writer 保持。
• _delegation_worker_agents：从当前 Goal/requester 的 canonical journal 读取 prepared/running/turn_returned operation，再与 slot 的 host/pid/取得顺序匹配。accepted/rejected 是终态，不是 worker；最新候选有跨成员歧义时返回不了该 worker，不把“读取历史结果”当“执行当前任务”。
• collect_agent_execution_facts：按注册成员和 open Todo 发现相关 Goal，合并 lane/worker/真实 lease owner 的事实。promoted authority 不回退到被取代的本地 lease。它没有领取、续租或改写状态的副作用，status 和 directory 复用同一投影。
• _agent_state：消费上述事实，保持当前任务的 blocked/monitoring 优先级；无执行事实时走 bound/launchable/addressable/registered。unknown 不在现有 TS peer admission 的 active allow-list 内。fresh Todo 不再构成 executing 证据，旧 lease 也不会转移给新 Todo owner。

独立验证使用相同合成 registry/Todo、真实子进程与 operation/slot 文件锁，经过公开 CLI status：base 的 fresh Todo/no-process 为 executing，head 为 launchable；旧 accepted operation 的活体读取者在 head 为 registered，当前真实 worker 为 executing。删除 Todo 时间戳、加入 350 条无关完成记录和反转这些记录顺序，均不改变 worker 归属；释放进程后恢复 launchable，registry/Todo 内容未被 status 改写。把上一轮历史错误规则临时代入同一 CLI 后，旧读取者再次被误报 executing，确认该回归检查能识别错误。

对主干的风险

• 已运行：306 项聚焦 Python 测试、4 项 peer-orchestration TS 测试、control-plane typecheck、worker lifecycle smoke、semantic vocabulary/census checks、完整 diff hygiene，以及风险选择的 premerge canary（16 项 selected checks，0 failures）。公开边界检查通过。
• 另跑 tests/architecture/test_turn_contract_generation.py：immutable base 和两个 exact heads 都在同一个测试 test_new_independent_twin_cannot_hide_behind_generated_pair 因 generated == 1 实际为 2 而失败。测试/生成 owner 的因果路径没有被这两项改动改变，changed invariant 有独立通过证据。这是既有验证问题，不要求本 PR 修改无关预算或测试，也不据此 request changes；合并准备仍须与代码评审结论分开判断。没有查询、轮询或等待远程 CI。
• 明确未证明：真实多机 fleet、Windows 实机，以及全部 PID incarnation/crash 时序；foreign host 只验证了合成 holder。这里的 executing 是本机现有 holder/PID 契约下的观测，不是授权或完备远程 presence。Promoted lease authority 使用原读取 owner，未改造 PostgreSQL 存储，也未声称验证 PostgreSQL 部署。
• 没有新增前端 lifecycle 编辑器：核对当前 dashboard 后，它消费的是 delegation preflight 状态，不直接编辑这里的 worker lifecycle。回滚可以 revert 本 PR，无新持久事实迁移。

我的整体评价

动机成立，读模型归属清楚，当前完整改动与最初误报问题成比例。上一轮 blocker 被真正修复；不存在“修了最后一处，就继承批准”的捷径。future-facing pass 已落实到共享 file-lock host/liveness owner 和公共 SHA256 matcher，没有再添加一个观察决策源；远程 presence 保留给已有 owning boundary，当前不扩张。

APPROVE；无阻塞性发现。最强剩余验证是部署级多机/process-incarnation 观测，不影响本次本地事实投影修复的批准。这是控制面改动，留给 maintainer 合并；本轮没有执行 merge。

English verdict: APPROVE