Skip to content

fix: give prepared compaction the next merge admission turn - #324

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/compact-commit-fairness
Oct 6, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/compact-commit-fairness

Conversation

@beinan

@beinan beinan commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

A persistent publisher can repeatedly claim another merge pass before a prepared compaction obtains commit ownership, causing the compactor to exhaust its wait budget and rebuild files. Publish a commit-ready hint only after nonempty output is prepared; subsequent merge claims yield to the matching live preparer.

The hint shares the preparation claim lease and is checked in the existing admission snapshot and final claim CAS. Preparation stays concurrent, admitted merges are not cancelled, and unresolved merge executions can still be claimed for storage recovery. Normal exclusive commit ownership and the existing wait budget remain in force.

Validation: formatting/diff checks passed. All three local real-etcd tests passed for native pass handoff, lease-loss cleanup/stale publisher rejection, and recovery bypass. Combined with #321, #322 and #323: all 170 master library tests passed, including real-etcd tests. All 11 CI checks passed before merge. Every scheduling/catchup contender needs the updated binary to honor this fairness hint; production rollout is separate.

@beinan
beinan marked this pull request as ready for review October 6, 2026 06:41
beinan added a commit that referenced this pull request Oct 6, 2026
Keep explicitly configured owned tables under continuous master catch-up
coverage, including low-count tails after successful slices. Persist the
canonical target set in the shared policy so failover cannot silently
discard desired coverage; fresh stats, cooldowns, fencing and global
capacity still gate every new Job.

If a previously confirmed Job disappears, exact terminal child-container
evidence can retire its inventory reservation as a failure. Missing or
running children and identity mismatches remain unresolved.
Execution/claim/target-lock records are retained for normal storage
recovery before successor writes.

Rebased onto merged #321; this PR contains only the continuous coverage
and missing-Job changes. New cases cover continued service after
restart, mismatched replica policy, stale post-success stats and
missing-Job identity/terminal checks. Formatting passed. Local catchup
suite: 33/33 passed with real etcd and native storage. Combined with
#321, #322 and #324: all 170 master library tests passed, including
real-etcd tests. CI remains in progress. Existing external publisher
adoption and production activation remain separate, qualified per-target
migrations; no owner keys are reset by this change.

Co-authored-by: Beinan Wang <>
@beinan
beinan merged commit cb529d8 into lance-format:main Oct 6, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant