fix(streaming): exact large integers and non-string column labels

[kevincostner17]

Summary

#208, streaming part: StreamingCleaner no longer casts nullable Int*/UInt* columns to float64 when filling them from running statistics. That cast changed present values beyond 2**53.

• Those columns keep their integer dtype and are filled with the rounded running statistic. The report says kept Int64 so values beyond 2**53 stay exact.
• Running state is float64 by design, so the fill value is only as precise as float64. Present values stay exact.
• Columns within ±2**53 keep the existing float64 behaviour.

#209: StreamingCleaner.clean_batch and fd.clean_timeseries no longer raise KeyError on non-string column labels.

• The str form of a label now only keys the running state and report entries. Frame access uses the original label.
• That covers _impute/_impute_column, detect_drift, and the time-series interpolation, seasonal and anomaly steps. last_numeric_cols and report columns stay strings.

Issue repros on this branch

Output is identical on Python 3.12 / numpy 2.5 and Python 3.9 / pandas 1.5.3:

#208 streaming: Int64 [9007199254740993, 9007199254740992, 9007199254740995, 9007199254740993]
#209 clean_batch: {0: [1.0, 3.0, 3.0, 4.0], 1: [1.0, 2.0, 3.0, 4.0]}
#209 clean_timeseries: ['a', 'a', 'a', 'a']

Tests

• tests/test_streaming_cleaner.py: integer labels, including a second batch that runs drift detection; an Int64 fill beyond 2**53 keeps the present values and the dtype.
• tests/test_streaming_timeseries.py: integer labels through fd.clean_timeseries, with interpolation reported under column "8".

Local lane pytest -m "not online and not large" on Python 3.12 and 3.9: all pass except the five sandbox tests that also fail locally on main (they need an installed package). ruff check . and mypy src/freshdata pass.

Closes #208
Closes #209

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 60649b9c-72c1-4f98-bfdb-adf223e29c91

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

FreshData benchmark report — performance

• freshdata: ?
• python: ?
• platform: ?

fixture	n_rows	n_cols	p50 s	p95 s	peak MB	repair %	false-repair %	preserve %	trust	monotonic	export %

Authored-code reduction (Metric 6)