Repository navigation
Fix reading of LZMA data bigger than one LZMA2 chunk - #431
Conversation
|
Thanks for patching the code. Did you use some fork of https://github.com/LZMA-JS/LZMA-JS or patch code yourself? Can you use Is code AI-assisted? If yes - please put correspondent marks in commits description. |
Assisted by AI
Assisted by AI
830fdb6 to
819fc8d
Compare
|
Thanks! i patched lzma.mjs in jsroot directly, there without fork because LZMA-JS , it has no LZMA2/xz support, so there was nothing upstream to take. I have changed the macro to use ROOT::CompressionSettings(ROOT::kLZMA, 9) I used claude to write the code. sometimes when i have my claude limit left i try to use my limit by contribute to open source projects before time is up. i have added the assisted by AI line to my commits. |
LZMA data in ROOT files is stored as LZMA2, which is split into chunks of at most 2 MiB unpacked or 64 KiB packed.
decompress()skips a fixed 29 byte header and then decodes the rest as one stream, so it only works when everything fits in the first chunk. At the second chunk it throwscorrupted input.For objects this means readObject fails, for example a 1000x1000 TH2D cant be opened. For trees its worse, because the error is caught inside treeProcess and it just returns fewer entries without any error. A tree with 600k entries in one big basket gives 0 entries.
I changed the decoder to read the xz block header and loop over the chunks, resetting the decoder for each one. The output matches python lzma byte for byte.
I also added a test, demo/node/tree_lzma_test, with a small file made with ROOT 6.40 and added it to the CI jobs. It fails on dev and passes with the fix.