feat: CursorPaginator for keyset pagination#55
Open
klaidliadon wants to merge 3 commits into
Open
Conversation
Sibling of Paginator[T] for ordering-stable pagination under concurrent writes, where offset is too expensive or visibly skips/duplicates rows. - EncodeCursor / DecodeCursor + ErrInvalidCursor (opaque base64+JSON, unsigned — callers re-apply tenant scope per request) - CursorPaginator[T, C] mirrors the offset Paginator[T] lifecycle (PrepareQuery -> GetAll -> PrepareResult) so call sites stay uniform - Page gains Cursor / NextCursor on the shared type so callers can swap paginators without changing the Page type - Caller owns ORDER BY; applyCursor must match it (documented + tested) - nil applyCursor / cursorFromRow panic on entry to fail on the first request rather than the second
Replace the two callback positional args on PrepareQuery / PrepareResult with a Cursor[Self, Row] interface constraint, mirroring the self-pointer convention pgkit already uses on Table[T, P, I] -> Record[T, I]. - Cursor[Self, Row] interface: *Self + Apply + From, hangs methods on the pointer so From can populate the value and Apply can read it - CursorPaginator gains a third type parameter PC Cursor[C, T]; callers spell it like NewCursorPaginator[*Row, Cursor, *Cursor](...) - Drops the nil-check + panic on applyCursor / cursorFromRow — missing methods are now a compile error - Cursor behavior co-located on the cursor type (Apply, From) instead of split between the type + a free function at the call site
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keyset / cursor pagination as a sibling to the existing offset
Paginator[T]. Came up while reviewing a downstream impl (0xPolygon/omsx#1233) that re-implemented the same encode/decode + LIMIT n+1 pattern locally — the primitives are generic enough to live in pgkit and the lifecycle fits the existingPrepareQuery/PrepareResultshape exactly.Offset pagination is fine for stable, small result sets, but on append-only / time-ordered tables under concurrent writes it visibly skips or duplicates rows as new ones land between page fetches. Keyset (cursor) pagination uses the last row of the previous page as a
WHEREpredicate, so newly-inserted rows don't shift the next page.What's added
cursor.go— three pieces:EncodeCursor[C any](C) (string, error)/DecodeCursor[C any](string) (*C, error)— opaque, URL-safe base64+JSON. Empty input →(nil, nil)so callers compose with a nil-check. Malformed →ErrInvalidCursorso callers can map to 400, not 500.CursorPaginator[T any, C any]— sibling ofPaginator[T], same lifecycle:Trow,Ccursor) so the WHERE clause and serialized cursor are both typed.page.go—PagegainsCursorandNextCursor(bothstring). Unused by the offset paginator; shared on the type so callers can swap paginators without changing the request shape.Design choices
T,C)applyCursorget a typed cursor argument instead ofanysmuggling.ORDER BYWHEREis order-dependent — only the caller knows which columns to compare. Mismatched ORDER BY and applyCursor silently skips or duplicates rows (documented).PaginatorSettings/WithDefaultSize/WithMaxSizeWithSort/WithColumnFuncare no-ops here because the caller owns ORDER BY.Page, no parallel typeColumn/Sortalready coexist onPagewithout enforcement.nil applyCursor/cursorFromRowpanic at function entrymorefirst fires).HMACCursorvariant can be added later if needed.Tests
Pure-unit, no DB needed thanks to the PrepareQuery/PrepareResult split. Coverage:
LIMIT n+1, capacity-correct pre-allocated sliceapplyCursorinvoked, WHERE clause appendedErrInvalidCursorpropagated throughPrepareQueryPrepareResultno-more case (noNextCursor,More=false)PrepareResulthas-more case (trims to limit,NextCursorround-trips back to last surviving row)PrepareResultpropagatescursorFromRowerrorsPrepareQueryandPrepareResultExisting
TestPagination*tests unaffected.Not in this PR (intentionally)
Table.ListCursorconvenience. The PrepareQuery/PrepareResult split is enough; a table-level wrapper can come later if usage warrants it.WithCursorSigner(...)) once a caller needs it. Skipped to keep v1 minimal.Happy to split this into smaller commits, rename, or restructure if the shape isn't what you'd prefer.