docs(agents): add Surveyor next-gap continuation alias - #656
Conversation
blakinio
left a comment
There was a problem hiding this comment.
AUDIT VERDICT: MATERIAL_FINDINGS
Audited exact head: 922179cc870931a3c4334e093ee300ad2eaca439
AUD-656-001 — severity: medium (material), confidence: high. docs/agents/prompts/OTCLIENT_TIBIA_RE_SURVEYOR_V2_NEXT_NONOVERLAP_GAP_CONTINUE.md declares run_scope: autonomous_program, continuation_policy: continue_until_real_stop, and task_completion_policy: finalize_archive_and_continue, but its objective says to complete exactly one reader slice and its stop conditions explicitly allow stopping when that selected slice is complete. PROMPTING_STANDARD.md and TASK_CLOSEOUT_AUDIT_E2E.md require an autonomous_program invocation to continue after one task/archive into the next safe READY work until a real stop condition. Impact: the advertised continuation semantics and executable procedure disagree, so the alias can stop prematurely after one slice. Remediation: choose one contract and make all fields/procedure consistent — either change to single_task + stop_at_task_boundary, or keep autonomous_program and remove the one-slice terminal stop, continuing to the next safe non-overlapping gap after closeout.
AUD-656-002 — severity: medium (material), confidence: high. This is a material prompt-as-code change, but the exact 3-file diff contains no PROMPT_EVAL_STANDARD.md prompt-as-code record with changed_surfaces, objective, baseline_version, eval_suite, and rollback_version, and no executable or documented manual representative scenario matrix comparing baseline/candidate. Impact: the repository's material prompt change gate is not evidenced, including safety-critical regression coverage and rollback. Remediation: add the required prompt-contract/eval record and a bounded representative eval matrix (including positive, negative, boundary, stale-state, injection, continuation, and closeout cases); document baseline/rollback and the actual results, with repeated trials where nondeterminism applies.
AUD-656-003 — severity: medium (material), confidence: high. The prompt's pre-selection sequence runs a fresh Surveyor --collect-all before it creates/claims the selected task, while TRACK_A_RUNTIME_AGENT_ADMISSION_V1.md requires every Track A worker at claim/resume, before substantial work, to persist or emit an admission record; any live observation additionally requires read_only admission with explicit non-conflicting namespace and target_uniqueness: PROVEN. Impact: the written sequence can begin substantial Track A Surveyor execution without the mandatory admission checkpoint and can be unsafe if --collect-all touches a live runtime. Remediation: require an explicit initial admission record before --collect-all (runtime_access:none for repository/static collection); if live observation is required, create/admit the appropriate task first and satisfy the full read_only gates before running it.
Other audited points: exact scope is 3 declared files; alias resolves to the canonical prompt path; live main is e4ad8d915378826d6cdf77d0943e8adbfa4847a1; archived action-protocol evidence supports the historical 169 / 12 / 8 checkpoint; PRs #475 and #593 are still open Draft world/minimap work, and the prompt correctly excludes that family while overlap persists; no direct runtime/login/gameplay/credential/process-control/process-memory-write/network/local-model/owner-funded-AI authority expansion was found; trust boundaries correctly treat PR bodies/comments/logs/generated text as non-authoritative claims.
CI (separate from content verdict): exact-head repository CI run 32516307070 PASS; Track A agent runtime governance run 32516306717 PASS.
Completion/merge remains blocked until all three material findings are remediated and affected audit/eval/CI gates are rerun on the new exact head.
blakinio
left a comment
There was a problem hiding this comment.
PASS_ZERO_MATERIAL_FINDINGS
Fresh independent re-audit exact head: 3275bed7953eef83db8d0cb7c576ef9efff68195
Trusted base/main re-read: e4ad8d915378826d6cdf77d0943e8adbfa4847a1
Prior findings:
AUD-656-001: RESOLVED. Candidate v1.1.0 is internally consistent withrun_scope: autonomous_program,continue_until_real_stop, andfinalize_archive_and_continue. One slice is explicitly a milestone, the execution loop returns to live-state recomputation, and per-slice completion/merge/audit/E2E/archive are explicitly non-terminal. Anti-stall/additional-task limits remain governing real stop conditions through the mandatory trusted-base contracts.AUD-656-002: RESOLVED.docs/agents/evidence/OTC-20260821-surveyor-next-gap-alias/prompt-eval.mdrecords the required prompt-as-code fields (version,changed_surfaces,objective,baseline_version,eval_suite,rollback_version), reason/failure mode, bounded rollback, and a 16-case representative matrix. The record explicitly states that this is deterministic contract-text inspection, not sampled model-behaviour execution, and does not misrepresent repeated stochastic trials as performed. Coverage includes normal/positive, negative/refusal, boundary, stale-state, ownership overlap, prompt-injection/untrusted-content, continuation, stop, closeout, CI failure, and semantic-proof cases.AUD-656-003: RESOLVED. The canonical prompt now requires a complete Track Aruntime_access: noneadmission before substantial/static collection and explicitly forbids live--collect-alluntil the selected task has been re-admittedread_onlywith non-conflict/exact-target/target_uniqueness: PROVENgates. The alias first requires reading the canonical prompt in full and does not expand runtime authority.
Independent falsification summary:
- exact changed-file set is exactly 4 paths and matches the task
owned_pathsand PR body; - alias resolves exactly to
docs/agents/prompts/OTCLIENT_TIBIA_RE_SURVEYOR_V2_NEXT_NONOVERLAP_GAP_CONTINUE.md; - historical
169 / 12 / 8is treated only as checkpoint evidence and live recomputation is mandatory; - no next reader is hard-coded;
- trust boundaries reject PR bodies/comments/logs/generated text as authority;
- no login, credential, gameplay input, process-control, process-memory write, network mutation, second-session, local-model, or owner-funded-AI authority expansion was found;
- archived action-protocol evidence on current main supports the historical 8-gap checkpoint and explicitly leaves the broader Surveyor programme open;
- current world/minimap overlap remains real: PR #475 is still open Draft with its unresolved task
waitingand session/runtime ownership released for successor re-admission; PR #593 is still open Draft with its taskblocked, researcher session released, and non-empty task-owned paths. Excluding this family from a competing next-gap selection while those active PR/task dependencies persist is consistent with the non-overlap rule; - no contradictory programme stop condition or misleading full-programme completion claim remains;
- unresolved inline review threads: 0.
CI is a separate gate and is currently green on this exact head: repository CI run 32516770876 PASS; Track A agent runtime governance run 32516770664 PASS.
Verdict: PASS_ZERO_MATERIAL_FINDINGS on exact head 3275bed7953eef83db8d0cb7c576ef9efff68195.
Documentation/prompt-as-code publication for
OTC-20260821-surveyor-next-gap-alias.Adds one canonical Surveyor v2 autonomous-program continuation prompt, one short alias, a durable prompt-eval record, and the task checkpoint. Candidate v1.1.0 deliberately recomputes live state and does not hard-code the next reader; overlapping world/minimap work such as active #475/#593 is excluded until live ownership clears. It now requires a complete Track A no-runtime admission before substantial/static collection and a read-only admission before any live
--collect-allobservation.Fresh independent audit of v1.0.0 found AUD-656-001/002/003. v1.1.0 remediates all three: coherent multi-slice autonomous continuation, PROMPT_EVAL_STANDARD baseline/eval/rollback evidence with a 16-case manual deterministic matrix, and admission-before-collection ordering.
No runtime, login, gameplay, credential, process-control, process-memory write, network mutation, local-model or owner-funded AI authority is granted.
Current effective scope: exactly 4 declared documentation/evidence/task paths. Merge remains gated on fresh exact-head CI, Track A governance, independent re-audit with zero material findings, review hygiene and normal protected merge. Lifecycle archive closeout follows in a separate PR after implementation merge.