Skip to content

feat(store): add key_prefix_from cursor to Query for efficient prefix scans - #108

Open
royalcala wants to merge 2 commits into
n0-computer:mainfrom
royalcala:feat/query-key-prefix-from
Open

royalcala wants to merge 2 commits into
n0-computer:mainfrom
royalcala:feat/query-key-prefix-from

Conversation

@royalcala

Copy link
Copy Markdown

Summary

Adds an optional from cursor field to the Query struct that enables B-tree-level seeks during prefix scans.

Motivation

The current key_prefix filter always starts iteration from the beginning of the prefix range. For applications maintaining a cursor position (e.g., syncing event logs), this forces O(n) client-side skipping via offset. With this change, the B-tree seeks directly to the cursor position in O(log n).

Changes

  • Add from: Option<Bytes> field to Query and QueryBuilder (store.rs)
  • Add key_prefix_from(prefix, from) builder method (store.rs)
  • Pass cursor through IndexKind::KeyAuthor to ByKeyBounds::new (util.rs, bounds.rs, query.rs)
  • When from is set with a Prefix filter, the lower bound becomes the cursor position instead of the prefix start

Backward Compatibility

Fully backward-compatible. When from is None (default), behavior is identical to current key_prefix. The wire format adds an optional field that defaults to None.

Usage

// Iterate from a known cursor position
let stream = doc.get_many(
    Query::all()
        .sort_by(SortBy::KeyAuthor, SortDirection::Asc)
        .key_prefix_from("evt:", "evt:01JABC...")
        .limit(100)
).await?;

Adds an optional  field to the Query struct that serves as a cursor
position for prefix scans. When set together with a KeyFilter::Prefix,
the B-tree seek starts at the cursor position instead of the prefix start,
enabling efficient iteration over large datasets from a known position.

- Add  to Query and QueryBuilder
- Add  builder method
- Pass cursor through IndexKind::KeyAuthor to ByKeyBounds
- Use cursor as lower bound for B-tree range scan (O(log n))
- Add unit test for ByKeyBounds with cursor lower bound
- Add debug_assert that cursor starts with prefix
- Document SortBy::KeyAuthor requirement in doc comment
- Document that from is ignored on AuthorKey index path
@n0bot n0bot Bot added this to iroh Jun 15, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh Jun 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🚑 Needs Triage

Development

Successfully merging this pull request may close these issues.

1 participant