Skip to content

Fix reading of LZMA data bigger than one LZMA2 chunk - #431

Merged
linev merged 2 commits into
root-project:devfrom
Prafyl:fix-lzma2-chunks
Oct 22, 2026
Merged

linev merged 2 commits into
root-project:devfrom
Prafyl:fix-lzma2-chunks

Conversation

@Prafyl

@Prafyl Prafyl commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

LZMA data in ROOT files is stored as LZMA2, which is split into chunks of at most 2 MiB unpacked or 64 KiB packed. decompress() skips a fixed 29 byte header and then decodes the rest as one stream, so it only works when everything fits in the first chunk. At the second chunk it throws corrupted input.

For objects this means readObject fails, for example a 1000x1000 TH2D cant be opened. For trees its worse, because the error is caught inside treeProcess and it just returns fewer entries without any error. A tree with 600k entries in one big basket gives 0 entries.

I changed the decoder to read the xz block header and loop over the chunks, resetting the decoder for each one. The output matches python lzma byte for byte.

I also added a test, demo/node/tree_lzma_test, with a small file made with ROOT 6.40 and added it to the CI jobs. It fails on dev and passes with the fix.

@linev
linev self-requested a review October 22, 2026 07:24
@linev linev self-assigned this Oct 22, 2026
@linev

linev commented Oct 22, 2026

Copy link
Copy Markdown
Member

@Prafyl

Thanks for patching the code.

Did you use some fork of https://github.com/LZMA-JS/LZMA-JS or patch code yourself?
Original repository not updated since many years.

Can you use ROOT::CompressionSettings(ROOT::kLZMA, 9) in the ROOT script for TFile creation?

Is code AI-assisted? If yes - please put correspondent marks in commits description.

@Prafyl

Prafyl commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks! i patched lzma.mjs in jsroot directly, there without fork because LZMA-JS , it has no LZMA2/xz support, so there was nothing upstream to take.

I have changed the macro to use ROOT::CompressionSettings(ROOT::kLZMA, 9)

I used claude to write the code. sometimes when i have my claude limit left i try to use my limit by contribute to open source projects before time is up. i have added the assisted by AI line to my commits.

@linev linev added the bug fix label Oct 22, 2026
@linev
linev merged commit da179f8 into root-project:dev Oct 22, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants