Skip to content

fix: bad downcast of ArrowResultCollector in plan mapper re-enables skipped arrow/CSR tests (#881) - #884

Merged
adsharma merged 1 commit into
mainfrom
fix/881-arrow-collector-runtime-checks
Sep 1, 2026
Merged

fix: bad downcast of ArrowResultCollector in plan mapper re-enables skipped arrow/CSR tests (#881)#884
adsharma merged 1 commit into
mainfrom
fix/881-arrow-collector-runtime-checks

Conversation

@adsharma

@adsharma adsharma commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #881.

Root cause

ArrowResultCollector and the regular ResultCollector both declare
static constexpr PhysicalOperatorType type_ = PhysicalOperatorType::RESULT_COLLECTOR.
In PlanMapper::getPhysicalPlan() the plan-root marker introduced by #870/#877 work was applied via an operator-type check:

if (root->getOperatorType() == PhysicalOperatorType::RESULT_COLLECTOR) {
    root->ptrCast<ResultCollector>()->setResultExposedToClient();
}

For every arrow query (resultType=ARROW), the newly created ArrowResultCollector matched that check and was then downcast to ResultCollector, which it is not:

  • With RUNTIME_CHECKS (or any !NDEBUG build), dynamic_cast_checked asserts and throws. Every queryAsArrow() execution failed and returned an error MaterializedQueryResult whose iterator is null. Tests calling getNextArrowChunk() without checking isSuccess() then SIGSEGV in FactorizedTableIterator::hasNext — the Fatal signal 11 in the linux minimal-test job. The remaining tests fail on dynamic_cast<ArrowQueryResult*>(result.get()) == nullptr and entry.relCsrResults[0] == nullptr (PROJECT_GRAPH materialization also runs through queryAsArrow).
  • With NDEBUG (plain release), the checked cast is a reinterpret_cast and setResultExposedToClient() writes a byte at ResultCollector::internalResultTable's offset (192), which in an ArrowResultCollector object is localState.batchIndex — UB on every arrow query.

The minimal-test job builds with -DENABLE_RUNTIME_CHECKS=1, which made the failure deterministic across all 10 arrow/CSR tests — while local (non-RUNTIME_CHECKS) runs looked healthy, which is why the issue looked like a timing-dependent race.

Fix

Mark the client-facing result table on the freshly created regular ResultCollector inside the non-arrow branch of the if/else, instead of matching on the shared operator-type enum afterwards.

Also re-enables the ten tests skipped in 520abda.

Note on the earlier investigation

The "corrupted task clone" evidence in the issue (this == sharedState, garbage batch indices, CSR metadata present for CSR-free queries) was an artefact of the temporary LBUG_ARROW_DEBUG tracing: its fprintf macro appended the thread-id string after the caller's varargs while the format consumed %s first, shifting every printed value by one slot. The deterministic "reproduction with instrumentation" was the tracing itself crashing in strlen() on an integer consumed as %s. With corrected tracing, all collector state is healthy up to the crash.

Validation

  • Full api_test (304 tests) passes with -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_RUNTIME_CHECKS=1 (the minimal-test configuration), plain RelWithDebInfo, and Debug builds.
  • TSAN run of the arrow/CSR suite is clean.

…881)

Root cause of the linux minimal-test failures tracked in #881:
ArrowResultCollector and ResultCollector both declare
PhysicalOperatorType::RESULT_COLLECTOR. In
PlanMapper::getPhysicalPlan(), the plan-root marker
setResultExposedToClient() was applied via an operator-type check:

    if (root->getOperatorType() == PhysicalOperatorType::RESULT_COLLECTOR) {
        root->ptrCast<ResultCollector>()->setResultExposedToClient();
    }

For every arrow query (resultType=ARROW), the newly created
ArrowResultCollector matched the type check and was then downcast to
ResultCollector, which it is not:

- With RUNTIME_CHECKS (or any !NDEBUG build) dynamic_cast_checked
  asserts and throws, so every queryAsArrow execution failed and
  returned an error MaterializedQueryResult with a null iterator.
  Tests that call getNextArrowChunk() without checking isSuccess()
  then SIGSEGV in FactorizedTableIterator::hasNext (the 'Fatal signal
  11' in the linux minimal-test job), and the remaining tests fail on
  dynamic_cast<ArrowQueryResult*>(result.get()) == nullptr and
  entry.relCsrResults[0] == nullptr (PROJECT_GRAPH materialization
  also goes through queryAsArrow).
- With NDEBUG the checked cast is a reinterpret_cast and
  setResultExposedToClient() writes one byte at
  ResultCollector::internalResultTable's offset (192), which in an
  ArrowResultCollector is localState.batchIndex - undefined behaviour
  on every arrow query.

Fix: mark the client-facing result table on the freshly created
regular ResultCollector inside the non-arrow branch of the if/else,
instead of matching on the shared operator-type enum afterwards.

Also re-enable the ten arrow/CSR tests that were skipped for #881:
ArrowTest.queryAsArrow, getArrowResult,
queryAsArrowDirectCSRRowIDProjection (+WithFourThreads),
queryAsArrowTracksCSRMetadataWithoutRelIDs /
WithRelIDsAndExtraColumns / DoesNotTrackCSRMetadataForNonCSRShape,
ProjectGraphCsrTest.materializesArrowCsr /
materializedCsrSurvivesConsumingQueries,
ReadOnlyTest.ProjectGraphOnReadOnlyDatabase.

Note on the earlier investigation: the 'corrupted task clone' evidence
in the issue (this == sharedState, garbage batch indices, csr metadata
present for CSR-free queries) was an artefact of the temporary
LBUG_ARROW_DEBUG tracing, whose fprintf macro appended the thread-id
string after the format's varargs, shifting every printed value by one
slot; the deterministic 'reproduction with instrumentation' was the
tracing itself crashing in strlen() on an integer consumed as %s.

Validation: full api_test passes with
-DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_RUNTIME_CHECKS=1 (the
minimal-test configuration) as well as plain RelWithDebInfo and Debug;
TSAN run of the arrow/CSR suite is clean.
@adsharma
adsharma merged commit f0356f0 into main Sep 1, 2026
4 checks passed
@adsharma
adsharma deleted the fix/881-arrow-collector-runtime-checks branch September 1, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SIGSEGV race: worker threads execute a corrupted task clone in the arrow result collector path (ArrowTest/CSR tests fail in CI)

1 participant