Skip to content

feat(search): whole-topic background message search (UI, REST, CLI, MCP) - #4

Merged
ritikbansod merged 1 commit into
mainfrom
feat/topic-search
Sep 28, 2026
Merged

ritikbansod merged 1 commit into
mainfrom
feat/topic-search

Conversation

@ritikbansod

Copy link
Copy Markdown
Owner

Implements F2 — whole-topic message search from the deep analysis: the biggest explorer gap vs Redpanda Console / AKHQ. Today's filters only apply inside a single browse window; this adds a bounded background job that scans every partition of a topic and finds the needle. Stacked on #3 (extends MutationRequests read-shaped POSTs there) — merge #1 → #2 → #3 → this.

API

  • POST /api/clusters/{clusterId}/topics/{topic}/search — start: filters (valueRegex, valueContains, keyContains), optional fromTs/toTs epoch-ms bounds, limit, optional single partition → {searchId}.
  • GET .../searches/{searchId} — progress (partitionsDone/Total, scanned, matched, state RUNNING|DONE|CANCELLED|FAILED) plus the matches collected so far.
  • DELETE .../searches/{searchId} — cancel (consumer wakeup).

Read-only contract (same as the browser): per-job throwaway group, assign() + seeks, never commits, never triggers a rebalance. Matches reuse MessageBrowserService.toView, so schema-registry payloads come back decoded and feed the existing message detail modal + one-click reproduce flow in the UI.

Bounded by design: max 1000 results, 2M scanned messages per job, 3 concurrent jobs (400 when full), jobs expire from memory after 30 minutes. The scan stops at a snapshot end offset, so results are deterministic for the window scanned.

Role model: the search POST is classified read-shaped, so a read-only role can search; it writes nothing (and the mutation audit trail therefore does not log searches).

Surfaces

  • Web UI: new Search tab in the data explorer — regex/contains/timestamp form, live progress bar, results table wired into the message detail modal.
  • CLI: kview search <topic> [--value-regex RE] [--from-ts MS|ISO] [--limit N] [--wait SEC] (start + wait + print matches) and kview search-status <id>.
  • MCP: kview_search_messages (start + wait, returns matches with progress; partial results carry the searchId for polling) and kview_search_status — the server now serves 18 tools.

Verification

  • 4 new integration tests against an embedded broker (3 partitions): regex search finds exactly ORD-70..79 across all partitions with full scan accounting; limit stops the scan early with limitReached; fromTs in the future yields zero matches while a window covering production yields all 90; unknown searchId → 404-shaped NoSuchElementException.
  • Full suite: 53/53 green; CLI --version/--help and MCP tools/list (18 tools incl. both search tools) smoke-checked.
  • Not covered: cancel mid-scan is exercised only indirectly (wakeup path); UI polling verified by code review + existing patterns, not browser automation.

Implements F2 from the deep analysis — the biggest explorer gap: finding a
specific message deep in a topic. A bounded background job scans every
partition (or one) forward from a timestamp or the beginning, collecting
matches — read-only like the browser (throwaway group, never commits).

- POST /api/clusters/{id}/topics/{topic}/search starts a job (regex /
  value-contains / key-contains filters, optional from/to timestamp bounds,
  limit); GET .../searches/{id} polls progress + results; DELETE cancels.
  Bounded: max 1000 results, 2M scanned messages, 3 concurrent jobs, jobs
  expire from memory after 30 minutes.
- Matches carry the same schema-decoded view as the browser (shared
  MessageBrowserService.toView), so results feed the existing message
  detail modal and reproduce flow.
- Web UI: new Search tab in the data explorer with progress polling.
- CLI: kview search <topic> [...] and kview search-status <id>.
- MCP: kview_search_messages (start + wait for completion) and
  kview_search_status — 18 tools now.
- Role model: search POSTs are read-shaped, so a read-only role may search;
  the mutation audit trail intentionally does not log searches (they read,
  they write nothing).
- tests: 4 integration tests against an embedded broker (regex across 3
  partitions, early stop at limit, timestamp bounds, unknown id). Full
  suite 53/53.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant