Flaky test: EmbeddedKafkaSupervisorTest.test_runKafkaSupervisorWithHeaderFiltering #20518

This issue was generated automatically by Claude Code (Anthropic's AI coding agent) running a scheduled CI-triage routine on behalf of @FrankChen021. Analysis and suggested fixes are AI-produced; please verify before acting on them.

Status: Open: no fix PR
Subject: EmbeddedKafkaSupervisorTest.test_runKafkaSupervisorWithHeaderFiltering (extensions-core/kafka-indexing-service, unit tests (25, E*,G*,F*))
Failures: 0 · First seen: 2026-10-06 · Last seen: 2026-10-07

No master job has failed yet because the surefire retry passes. But the first attempt has failed in 6 of 7 master runs since #18525 added the test.

Root cause

The test suspends the supervisor as soon as the broker emits any metric for the datasource (line 209). It doesn't wait for the 10 produced records to be read. If the suspend lands before the task reaches the last production record, the task publishes only what it has read. ingest/handoff/count then never reaches 4, and the wait at line 220 times out after the default 10 s. test_runKafkaSupervisor in the same class avoids this race: it waits for ingest/events/processed ≥ the expected count before suspending. This is a race in the test, not in the commits it failed on.

Suggested fix

Before suspending, wait for every record to be consumed: either ingest/events/processed sum ≥ 4 (the 4 kept records), or processed plus filtered ≥ 10, as test_runKafkaSupervisor does at line 155. Optionally give the handoff wait an explicit, longer timeout.

Occurrences

Failed push-triggered master jobs only. The daily triage routine adds one row per new failed job. Rows marked (passed on retry) did not fail the job and are not counted.