Skip to content

[Parquet][C++] Optimize RecordReader for flat optional #50628

Closed

Description

@AntoinePrv

Describe the enhancement requested

When reading flat optional values from a Parquet file, the definition levels correspond to Arrow validity bitmap.
We can speed up RecordReader for this common case by decoding the levels directly as the validity bitmap.

Open questions:

  • Should int16_t* RecordReader::def_levels() return nullptr (break) or make an effort to still provide some levels (additional complexity or cost). The new faster breaking behavior could also be enabled my a ctor flag.

Component(s)

C++, Parquet

Activity

  1. changed the title [-][Parquet][C++][/-] [+][Parquet][C++] Optimize `RecordReader` for flat optional[/+] on Jul 24, 2026
  2. added a commit that references this issue on Oct 28, 2026
  3. added this to the 27.0.0 milestone on Oct 28, 2026
  4. pitrou commented on Oct 28, 2026

    @pitrou
    Member

    Issue resolved by pull request 50629
    #50629

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions