Skip to content

feat(DIARCHERS-1620): paginate MCP list endpoints and cap log size - #632

Merged
liuwei08 merged 3 commits into
masterfrom
feat/DIARCHERS-1620-mcp-pagination
Aug 5, 2026
Merged

feat(DIARCHERS-1620): paginate MCP list endpoints and cap log size#632
liuwei08 merged 3 commits into
masterfrom
feat/DIARCHERS-1620-mcp-pagination

Conversation

@Jiachen0715

Copy link
Copy Markdown
Contributor

Summary

Several MCP read-only endpoints returned unbounded results — all rows /
the full log in a single response — which could overflow the client/model
context, truncate output, or hang on large data (big matrix expansions,
large test suites, long build logs). This PR bounds every such endpoint by
default while still allowing callers to page through the full set.

Changes

  • New src/api/handlers/mcp/pagination.py — shared helper:
    • parse_pagination() reads/validates ?limit=&offset= (default limit
      100, max 500; 400 on invalid values); the default is applied even
      when the caller passes nothing, so a bare call is always bounded.
    • page(items, total, limit, offset) wraps results in a standard
      {items, total, limit, offset} envelope.
  • jobs.py:
    • list_jobs — paginated + optional ?state= filter (exact match,
      e.g. failure). Not filtered by default.
    • get_job_test_results — paginated (test suites can be very large).
    • list_job_artifacts — paginated by slicing the in-memory archive
      jsonb array.
    • get_job_log — bounded by bytes: returns the last 1 MB by default
      (where failures live), max_bytes to change the cap (max 5 MB), or
      offset+length for a ranged read. Returns
      {log, total_bytes, offset, length, truncated}.
  • projects.py:
    • list_projects — paginated; the three access paths (scoped MCP token /
      unscoped token / session user) collapsed into one shared WHERE clause.

Impact

  • ⚠️ Breaking response shape. List endpoints now return
    {items, total, limit, offset} instead of a bare array; get_job_log
    returns an object instead of raw text. The MCP client is updated in the
    companion PR.
  • Internal cascade queries are unaffected: restart_job/rerun_job still
    read the full build_jobs set (no LIMIT), so downstream propagation is
    not truncated.
  • list_builds already had LIMIT 50; this brings the rest in line.

Testing

  • Python syntax validated locally (py_compile).
  • Unit / integration tests: TODO (not yet added in this PR).

@liuwei08
liuwei08 merged commit 7f20247 into master Aug 5, 2026
2 checks passed
@liuwei08
liuwei08 deleted the feat/DIARCHERS-1620-mcp-pagination branch August 5, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants