Skip to content

修复抓取收据验证边界并接入 Autopilot lineage - #2

Merged
estelledc merged 28 commits into
mainfrom
codex/tracefetch-receipt-open-lineage
Aug 1, 2026
Merged

修复抓取收据验证边界并接入 Autopilot lineage#2
estelledc merged 28 commits into
mainfrom
codex/tracefetch-receipt-open-lineage

Conversation

@estelledc

@estelledc estelledc commented Aug 1, 2026

Copy link
Copy Markdown
Owner

What

  • 接入 TraceFetch 的 Project Development Autopilot 配置与 901–906 冻结 TaskSpec lineage
  • 让 crawl verifier 重新计算顶层状态,拒绝与页面记录不一致的 receipt
  • 把 malformed / invalid UTF-8 / unreadable crawl receipt 和损坏 SQLite 状态收敛为稳定、有限、不可反射输入的验证失败
  • 使用 O_NOFOLLOW | O_NONBLOCK、单一文件描述符、16 MiB 上限、完整 chunk 检查和前后 fstat,避免 symlink 跟随、FIFO open 阻塞、无界读取和路径替换窗口
  • 当平台缺少 O_NOFOLLOWO_NONBLOCK 时 fail closed,不静默削弱安全语义
  • 补齐 CLI 与共享验证层回归,保留有效 crawl 摘要和普通 evidence bundle 验证行为

Why

原有 crawl verification 有四个相连缺口:

  1. receipt 顶层 status 可与页面投影矛盾但仍通过;
  2. malformed receipt 会在 payload 二次解析时逃逸为 traceback,并可能放大或反射验证错误;
  3. receipt 使用 Path.read_text() 整体读取,缺少输入上限和同一 open handle 的稳定性检查;
  4. 仅用 O_NOFOLLOW 打开后再检查 regular file,FIFO 可在 fstat 前阻塞;缺失 safe-open flag 还会逃逸 AttributeError

这条 lineage 通过 frozen candidate review 收敛这些边界。SDW-AUTO-903 因未处理 read OSError 和非空短读被正式拒绝;905 修正后,post-push review 又发现 open 阶段阻塞风险,最终由 SDW-AUTO-906 封闭。

Impact and risk

  • 新增 crawl receipt 的 16 MiB 验证上限;恰好 16 MiB 不按 oversized 拒绝
  • receipt symlink、special file、open/read/fstat 错误统一返回 crawl receipt is unreadable
  • 读取过程中任何非完整 chunk 返回 crawl receipt changed during verification
  • 平台没有 O_NOFOLLOWO_NONBLOCK 时 crawl receipt verification fail closed
  • malformed receipt CLI 继续 exit 6、stdout 为空、stderr 为单个 tracefetch.error.v1 对象且不回显输入
  • 有效 receipt 即使 SQLite 状态缺失、损坏或为 symlink,仍保留真实 crawl ID 和 complete-page count
  • autopilot/tasks/ 保存历史冻结 TaskSpec;它们是 lineage 输入,不表示可以在新的 HEAD 上重放

Checks

  • GitHub Actions push CI run 30690203375:Python 3.11 / 3.12 / 3.13 全部成功
  • 三个版本均为 135 passed;coverage 83.04%;Ruff、mypy strict、schema/version、wheel/sdist build 和 installed-product smoke 全部通过
  • uv run --locked pytest tests/test_cli.py:16 passed
  • uv run --locked pytest tests/test_crawl.py:29 passed
  • 真实 POSIX FIFO probe:共享 verifier 约 0.13 ms 返回固定 unreadable payload,无阻塞或 traceback
  • SDW-AUTO-906:CLOSED_SNAPSHOT,closed evidence 已通过离线 verify-export
  • 产品提交与远端 branch ref 的 delivery observation 已通过离线 verify-delivery

Evidence boundary

Autopilot evidence 证明 frozen snapshot、结构化 review decision、产品直接后继 commit、diff match 和一次远端 ref-match observation 的内部一致性;它不证明 reviewer 身份、PR 已创建、merge、release、上线或外部业务结果。


Note

High Risk
Touches security-sensitive local file verification (symlinks, FIFO blocking, size limits, TOCTOU) and changes public crawl verify failure semantics and CLI exit behavior; ordinary evidence bundle verification is unchanged.

Overview
Adds Autopilot project config (autopilot/project.json), frozen SDW-AUTO-901–906 task specs, and ignores .autopilot/ runtime output.

Crawl verification in verify.py is refactored around a shared _crawl_verification_result path so verify_crawl_bundle and crawl_verification_payload use one receipt load. Top-level status is recomputed from page records and rejected when it disagrees (crawl status does not match page records).

Receipt loading replaces Path.read_text() with a bounded O_RDONLY | O_NOFOLLOW | O_NONBLOCK open-handle read: 16 MiB cap, chunked reads with exact-length checks, pre/post fstat identity/size, and fixed messages for oversized, changed-during-read, unreadable, invalid UTF-8, and schema failures—without echoing input or tracebacks. Missing O_NOFOLLOW/O_NONBLOCK fails closed.

SQLite state errors (sqlite3.Error, bad row conversion, close failures) map to crawl SQLite state is invalid; valid modeled receipts still expose real crawl_id and verified-page counts when state is bad.

Tests in test_cli.py and test_crawl.py cover malformed receipts (exit 6, bounded stderr), size/truncation/read/FIFO/safe-open flags, status tampering, and corrupt SQLite rows.

Reviewed by Cursor Bugbot for commit 49e0fd0. Bugbot is set up for automated code reviews on this repo. Configure here.

@estelledc
estelledc merged commit 868f340 into main Aug 1, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant