perf: rewrite fair claim as LATERAL top-K per company#1
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the global
ROW_NUMBER()window over every eligible row with a LATERALtop-K-per-company candidate selection, eliminating the sort that dominated the old
plan (82k rows at 100k scale, 830k rows + 32 MB on-disk merge at 1M) without
regressing any concurrency, fairness, or correctness property.
Plan comparison (old vs new, verbatim plan lines)
Workload identical for both queries at each scale: seed 42, 100k / 1M tasks, 50
companies, 80% flooder, 20% future
run_after. EXPLAIN (ANALYZE, BUFFERS, VERBOSE)with
track_io_timing=on, warmup discarded, 3 steady runs.81.647/85.534/99.678 ms47.258/53.010/50.109 ms1140.892/1041.530/1108.504 ms580.859/557.606/562.137 mstemp read=1307 written=1308temp read=16068 written=16082ix_tasks_pending_run_afterused?Index Scan using ix_tasks_company_idBitmap Index Scan on ix_tasks_company_idIndex Scan using ix_tasks_company_idBitmap Index Scan on ix_tasks_company_idSort Methodon finalORDER BY c.rn, c.created_atquicksort Memory: 122kBquicksort Memory: 130kBquicksort Memory: 122kBquicksort Memory: 130kBWhat changed: the old plan sorts the whole eligible set for the window — 1M:
Incremental Sort (... rows=830496 ...) ... Peak Disk: 32552kBwithtemp read=16098 written=16112. The new plan runs the window over onlycompanies×cap rows —
WindowAgg (... rows=500 ...)fed bySort ... quicksort Memory: 56kB, in memory, no temp — with per-company work bounded byLimit (... rows=10 ... loops=50) -> top-N heapsort Memory: 26kB -> Bitmap Heap Scan on public.tasks tasks_1. Net: ~1.7× faster at 100k, ~2× at 1M, disk sorteliminated at both scales.
Invariants confirmed (enforcing SQL quoted)
WHEREontrepeatsthe eligibility predicate exactly, so a row concurrently flipped between
candidate-selection and lock is dropped by Postgres's row re-fetch:
Index Scan using tasks_pkey on public.tasks t ... Filter: ...). Proven bytest_lateral_high_contention_no_duplicates(20 iterations × 10 threads),test_no_duplicate_concurrent_processing,test_concurrent_claim_no_duplicates.FOR UPDATE OF t SKIP LOCKEDon the OUTERtasksalias — not the LATERALsubquery / CTE alias.
LockRowssits directly aboveIndex Scan using tasks_pkey on public.tasks tin every plan.processingbranch carries norun_afterfilter in any of its three locations (eligible_companies, LATERALinner, outer
WHERE):OR (status = 'processing' AND locked_at < :stale_before). Proven bytest_lateral_stale_recovery_bypasses_run_after(stale-locked task with
run_after30 days out is still claimed).LIMIT :batch_sizeretained;LATERAL
ORDER BY created_at LIMIT :per_company_capbound from newsettings.per_company_claim_cap(default 10). Proven bytest_lateral_respects_per_company_cap.Full suite: 30 passed. ruff clean, mypy clean. No existing test relaxed.
Honest note: the partial index remains unused
ix_tasks_pending_run_after(onrun_after, partialWHERE status='pending') isnot used by the new query. The LATERAL inner scan keys on
company_idequality, so the planner picks
ix_tasks_company_idand applies the eligibilitypredicate as a Filter + in-memory top-N heapsort on
created_at. The index thatwould actually serve this access pattern is a composite like
(company_id, created_at)scoped to eligible rows — deferred as separate work with its ownmigration and test, out of scope for this PR. The remaining hot cost is now the
eligible_companiesfull-table Seq Scan + HashAggregate and the per-company bitmapheap loops, not sorting; the 1M new plan also spends ~120 ms in JIT the old plan
didn't trigger (still ~2× faster net).