Skip to content

Commit 538394d

Browse files
committed
docs: changelog entry for zero-copy Arrow boundary
1 parent 31925e8 commit 538394d

2 files changed

Lines changed: 12 additions & 8 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Changelog
22

3+
## UNRELEASED
4+
5+
### Changed
6+
7+
- Data now crosses the Python↔Rust boundary via the zero-copy Arrow C Data Interface instead of Arrow IPC serialization, reducing memory copies and peak RSS for large results (on a 1M-row × 50-col register+read microbenchmark: ~5.7× faster, ~1 GB lower peak RSS). Set `GGSQL_ARROW_IPC=1` to restore the previous IPC behavior. The minimum pyarrow version is now 15.
8+
39
## 0.5.2
410

511
### Added

‎src/lib.rs‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -272,14 +272,12 @@ fn py_to_df_stream(py: Python<'_>, obj: &Bound<'_, PyAny>) -> PyResult<DataFrame
272272
))
273273
})?;
274274

275-
let batches: Vec<_> = reader
276-
.collect::<Result<Vec<_>, _>>()
277-
.map_err(|e| {
278-
PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
279-
"Failed to read RecordBatch from Arrow C stream: {}",
280-
e
281-
))
282-
})?;
275+
let batches: Vec<_> = reader.collect::<Result<Vec<_>, _>>().map_err(|e| {
276+
PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
277+
"Failed to read RecordBatch from Arrow C stream: {}",
278+
e
279+
))
280+
})?;
283281

284282
if batches.is_empty() {
285283
return Ok(DataFrame::empty());

0 commit comments

Comments
 (0)