Skip to content

feat(eval): read-only batch-evaluation TUI (list + get) - #1938

Open
jariy17 wants to merge 2 commits into
feat/eval-batch-evaluation-readonlyfrom
feat/eval-batch-evaluation-tui
Open

feat(eval): read-only batch-evaluation TUI (list + get)#1938
jariy17 wants to merge 2 commits into
feat/eval-batch-evaluation-readonlyfrom
feat/eval-batch-evaluation-tui

Conversation

@jariy17

@jariy17 jariy17 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What

Adds the read-only TUI for agentcore eval batch-evaluation, wiring it into the eval TUI alongside evaluator and online-eval:

  • list — a paginated, searchable table of batch evaluations (BatchEvaluationPicker over PaginatedTablePicker). Enter on a row opens that job.
  • get — a raw-JSON detail screen (JsonDetail). No metadata hub: the value is the full response (job metadata + per-session results), which the JSON shows cleanly. Unwraps getBatchEvaluation's { detail } (metadata + merged CloudWatch results).
  • menuBatchEvaluationScreen (RouterScreen). This route is mandatory: the eval menu already lists batch-evaluation, and without a route the entry falls through to the HelpScreen catch-all and exits the app.

A bare agentcore eval batch-evaluation now opens the TUI (renderTui + withTuiOnEmptyFlagsAndArgs) instead of printing help.

Demo

batch-evaluation-tui-demo

Files

  • src/components/BatchEvaluationPicker.tsx — list body (columns/toRow/loadPage), mirrors OnlineEvalPicker.
  • src/handlers/eval/batch-evaluation/list/screen.tsxBatchEvaluationListScreen; enter → get.
  • src/handlers/eval/batch-evaluation/get/screen.tsxBatchEvaluationGetJsonScreen + useBatchEvaluationDetail.
  • src/handlers/eval/batch-evaluation/screen.tsxBatchEvaluationScreen menu.
  • index.tsx re-wired to renderTui; Root.tsx gets the menu + list + bare-get-redirect + get/:id routes.

Testing

  • batch-evaluation.screen.test.tsx — menu (get/list), picker (rows, Core-options, bare-get redirect, selection → JSON, empty), detail (full JSON incl. merged results, CloudWatch-failure-degrades, retry).
  • Full suite: 752 pass, typecheck + lint clean.

@jariy17
jariy17 marked this pull request as ready for review August 6, 2026 21:29
jariy17 added 2 commits August 7, 2026 19:07
Wire batch-evaluation into the eval TUI, mirroring evaluator/online-eval:
- BatchEvaluationPicker (PaginatedTablePicker over listBatchEvaluations) +
  BatchEvaluationListScreen; enter on a row opens the job.
- BatchEvaluationGetJsonScreen — raw JSON only (no metadata hub); unwraps
  getBatchEvaluation's { detail } (metadata + merged CloudWatch results).
- BatchEvaluationScreen menu (RouterScreen); the menu route is mandatory or the
  eval menu entry falls through to the HelpScreen catch-all and exits the app.
- index.tsx now opens the TUI on a bare invocation (renderTui +
  withTuiOnEmptyFlagsAndArgs) instead of printing help.
- Root.tsx: menu + list + bare-get-redirect + get/:id routes.

Screen tests cover menu, picker (rows/options/redirect/select/empty), and
detail (full JSON, CloudWatch-failure-degrades, retry). 752 pass.
Align the list column with every other picker (harness, runtime, online-eval,
evaluator, memory, …): show updatedAt as 'updated UTC' rather than createdAt as
'created UTC'.
@jariy17
jariy17 force-pushed the feat/eval-batch-evaluation-tui branch from 3ec32c9 to 30585a8 Compare August 7, 2026 19:08
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.81%. Comparing base (fbb0cee) to head (30585a8).

Additional details and impacted files
@@                           Coverage Diff                           @@
##           feat/eval-batch-evaluation-readonly    #1938      +/-   ##
=======================================================================
+ Coverage                                95.78%   95.81%   +0.02%     
=======================================================================
  Files                                      296      300       +4     
  Lines                                    16431    16546     +115     
=======================================================================
+ Hits                                     15738    15853     +115     
  Misses                                     693      693              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants