Skip to content

test(storage): start the ART reproducer from the real pre-v10 schema - #150

Merged
Fl0p merged 2 commits into
mainfrom
flo-1058-art-repro-pre-v10-fixture
Oct 13, 2026
Merged

Fl0p merged 2 commits into
mainfrom
flo-1058-art-repro-pre-v10-fixture

Conversation

@Fl0p

@Fl0p Fl0p commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

What

The ART reproducer's fixture claimed to be spans at schema version 2 but was missing that era's duration_ms — the VIRTUAL generated column mid-table that only v9 → v10 removed. The column goes back in verbatim from ddb4a2b^:internal/storage/schema.sql.

Why it mattered: schema.sql's v10 statement is ALTER TABLE spans DROP COLUMN IF EXISTS duration_ms, so over a table that never had the column it is a silent no-op. The only ALTER the previous runs exercised under a filled, indexed table was v2 → v3 — which in production ran the day after the database was created. v10 is the one that ran on a long-lived, filled, four-index table six weeks before the open aborted. And without the column the fixture carried none of the logical/physical column skew of ADR-0013, so the indexes were built over a layout production never had.

Changes

  • preUserIDSchema carries duration_ms again, verbatim from v2.
  • The pass asserts the migration ran: duration_ms present before storage.Open, gone after. A no-op cannot return unnoticed in either direction.
  • The probe pair is renamed to what it measures. No SELECT over spans is planned as an ART index scan on the linked engine — measured for a bare constant, a bound parameter, an equality on the primary key, and the textbook case of a unique match in a 100 000-row table with a fresh index on exactly the probed column, with index_scan_max_count/index_scan_percentage raised to admit everything and with and without the storage_compatibility_version pin. So "an indexed probe agrees with an unindexed one" was two sequential scans agreeing. What the pair does compare is filter pushdown against no pushdown, which is where ADR-0013's wrong answers came from, and the test now asserts the two probes still plan differently so the comparison cannot degenerate into one path checked twice.
  • Unique-constraint enforcement is the one path that provably traverses an ART index, so each restart also re-inserts a stored span_id (must be rejected, with the error checked to be a constraint error) and inserts an unused one (must be accepted, then removed).
  • requireNoWAL tolerates a present-but-empty WAL file: an engine that truncates instead of unlinking has folded everything, and failing on the inode would report the next such engine as a reproduction of this defect on the very assertion that guards the fold.
  • docs/operations/duckdb-recovery.md: the fixture description, the assertion table (two new rows, the index-scan claim replaced with the honest one plus the measurement that forced it), and two added rows in "What has been run" beside the earlier two — with a Fixture column saying what each pair of runs did and did not exercise. CHANGELOG's unreleased entry for the reproducer is corrected to match.

Verified

On linux/arm64 under golang:1.24, CGO_ENABLED=1, against duckdb-go/v2 v2.10506.0 (SELECT version() → v1.5.6):

Parameters Result
defaults (20 000 spans, 2 cycles) did not reproduce, 4.1 s
COTEL_ART_REPRO_SPANS=200000 COTEL_ART_REPRO_CYCLES=6 did not reproduce, 2 m 58 s — 8 opens, 14 checkpoints, table peaking at 407 956 rows

Also go vet ./internal/storage/ clean and the whole internal/storage package green (10.1 s).

Nothing reproduced, so no escalation path was triggered and the forensic volume stays held under the unchanged release condition: not shown to reproduce at our scale on 1.5.6.

🤖 Generated with Claude Code

The reproducer's fixture described itself as `spans` at schema version 2
but was missing that era's `duration_ms` - a VIRTUAL generated column in
the middle of the table, which only v9 -> v10 removed. Two things follow
from its absence, and both weaken the negative result the recovery page
now rests a release condition on.

`schema.sql`'s v10 statement is `ALTER TABLE spans DROP COLUMN IF EXISTS
duration_ms`, so over a table that never had the column it is a silent
no-op: the only ALTER the runs exercised under a filled, indexed table
was v2 -> v3, which in production ran the day after the database was
created. v10 is the one that ran on a long-lived table six weeks before
the open aborted. And without the column the fixture carries none of the
logical/physical column skew of ADR-0013, so the indexes were built over
a layout production never had.

The column goes back in verbatim, and the pass now asserts the migration
actually ran: `duration_ms` present before `storage.Open`, gone after. A
no-op cannot return unnoticed in either direction.

The probe pair is renamed to what it measures. No SELECT over `spans` is
planned as an ART index scan on the linked engine - measured for a bare
constant, a bound parameter, an equality on the primary key, and the
textbook case of a unique match in a 100 000-row table with a fresh
index on the probed column, with the index_scan thresholds raised to
admit everything and with and without the storage_compatibility_version
pin - so "an indexed probe agrees with an unindexed one" was two
sequential scans agreeing. What the pair does compare is filter pushdown
against no pushdown, which is where ADR-0013's wrong answers came from,
and the test now asserts the two probes still plan differently so the
comparison cannot degenerate into one path checked twice. Unique-
constraint enforcement is the one path that provably traverses an ART
index, so each restart also re-inserts a stored span_id (must be
rejected) and inserts an unused one (must be accepted).

requireNoWAL now tolerates a present-but-empty WAL file: an engine that
truncates instead of unlinking has folded everything, and failing on the
inode would report the next such engine as a reproduction of this defect
on the very assertion that guards the fold.

Re-run on the corrected fixture, linux/arm64 under golang:1.24 against
duckdb-go/v2 v2.10506.0 (v1.5.6): did not reproduce at 20 000 spans /
2 cycles (4.1 s) or at 200 000 spans / 6 cycles (2 m 58 s, 8 opens, 14
checkpoints, the table peaking at 407 956 rows). Both rows are added to
the recovery page's run table beside the earlier two rather than
replacing them, with what each fixture did and did not exercise.

Co-Authored-By: Wayland <wayland@agents.flopbut.local>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 40e0a6ff-5dc4-4193-9a57-eba965830717
📥 Commits

Reviewing files that changed from the base of the PR and between 0cf0969 and 610d042.

📒 Files selected for processing: 3
  • CHANGELOG.md
  • docs/operations/duckdb-recovery.md
  • internal/storage/art_repro_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The defaults line still carried 4.7 s, measured on the fixture that had no
duration_ms; the run table two sections down in the recovery page reports
4.1 s for the corrected fixture, so the release notes contradicted it.

Co-Authored-By: Daedalus <daedalus@agents.flopbut.local>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Fl0p
Fl0p merged commit f10624a into main Oct 13, 2026
11 checks passed
@Fl0p
Fl0p deleted the flo-1058-art-repro-pre-v10-fixture branch October 13, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant