Skip to content

feat: Raw api polars integration#2678

Open
christoffer-hk wants to merge 8 commits into
masterfrom
raw-api-polars-integration
Open

feat: Raw api polars integration#2678
christoffer-hk wants to merge 8 commits into
masterfrom
raw-api-polars-integration

Conversation

@christoffer-hk

@christoffer-hk christoffer-hk commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Description

Started on an integration with Polars, as an alternative to Pandas. Beginning with RAW rows (RowCore and RowListCore) and a retrieve_polars_dataframe function in the RawRowsAPI. Currently implemented the bare minimum to have mostly feature parity with how we integrate with Pandas.

This only scratches the surface with what is technically possible to do with Polars. I believe there is a lot of potential in using Polars, especially now in version 8> with running things async. A lot of cool topics are mentioned here in a comparison written by Polars https://docs.pola.rs/user-guide/migration/pandas/ .

Checklist:

  • Tests added/updated.
  • Documentation updated. Documentation is generated from docstrings - these must be updated according to your change.
    If a new method has been added it should be referenced in cognite.rst in order to generate docs based on its docstring.
  • The PR title follows the Conventional Commit spec.

@christoffer-hk christoffer-hk requested review from a team as code owners June 9, 2026 14:35

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds Polars DataFrame support to the raw rows API and data classes. The review feedback highlights several instances where unpacking row columns could raise a TypeError if the columns are None, suggesting defensive unpacking with **(columns or {}).

Comment thread cognite/client/_api/raw/rows.py
Comment thread cognite/client/data_classes/raw.py Outdated
Comment thread cognite/client/data_classes/raw.py
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.65%. Comparing base (5309e35) to head (77744ac).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2678      +/-   ##
==========================================
- Coverage   93.65%   93.65%   -0.01%     
==========================================
  Files         498      498              
  Lines       50342    50460     +118     
==========================================
+ Hits        47148    47258     +110     
- Misses       3194     3202       +8     
Files with missing lines Coverage Δ
cognite/client/_api/raw/rows.py 91.37% <100.00%> (+0.36%) ⬆️
cognite/client/_sync_api/raw/rows.py 100.00% <100.00%> (ø)
cognite/client/data_classes/_base.py 93.79% <100.00%> (+0.01%) ⬆️
cognite/client/data_classes/raw.py 96.47% <100.00%> (+0.23%) ⬆️
tests/tests_unit/test_api/test_raw.py 100.00% <100.00%> (ø)

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@christoffer-hk

Copy link
Copy Markdown
Contributor Author

A couple comments from myself.

  • The sync-client-codegen does not add the polars import properly into the _sync_api; it falls outside of the TYPE_CHECKING check.
  • I have not ensured symmetry on purpose, by not making an implementation for something like insert_polars_dataframe to keep the size of the PR low.
  • I have to use the runtime compatibility version of polars (rtcompat) due to running under rosetta. While gemini suggests it should be ok for other devs to work with that, it'd be nice if someone could verify. Otherwise I suppose we need a different dev-dependency group to handle the different versions of polars.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant