Skip to content

Polars assisted sampling for KumoRelational - #1026

Draft
alan-kumo wants to merge 1 commit into
NVIDIA:mainfrom
alan-kumo:feat/parquet-amazon-example
Draft

alan-kumo wants to merge 1 commit into
NVIDIA:mainfrom
alan-kumo:feat/parquet-amazon-example

Conversation

@alan-kumo

Copy link
Copy Markdown

"add new example to KumoRelational ... using Polars assisted sampling" — requested workflow.

Adds a reusable Parquet-backed relational sampler and a self-contained rel-amazon user-churn example. The sampler retains relationship keys and timestamps in CPU memory, uses SDM's existing neighbor sampler, and reads selected feature rows from Parquet with Polars. The sampled RelatedTables feed directly into KumoRelational.

The example ran against 23,218,245 source rows on a 15 GiB RAM machine. With 512 context and 64 query task rows, it sampled 5,018 and 1,019 related rows, reached 4.95 GiB peak process RAM, and produced 0.7344 query accuracy. The focused sampler test checks exact sampled-table parity with the in-memory sampler; the example documents the remaining graph metadata held in RAM.

Validated with eight focused relational tests, the end-to-end rel-amazon run, and pre-commit run --all-files.

@copy-pr-bot

copy-pr-bot Bot commented Sep 29, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant