Found by a check that asks every format claiming it can reclaim space to prove it — and to prove the files survived it.
Ext: 'OTHER4.BIN' came back with different bytes from the reclaimed volume
first difference at 1024: got 0, want 7
The failure
Only when both switches are on:
plain 4.194.304 bytes, 0 wrong
sparse 4.194.304 bytes, 0 wrong
dedup 4.194.304 bytes, 0 wrong
sparse+dedup 4.194.304 bytes, 3 wrong
Root cause
Which blocks of a file are holes is looked up by the position the file was registered at:
var holes = this.HoleMap(fileOrdinal, blockSize);
But fileOrdinal counts along fileInodes — the files that ended up with an inode of their own — while HoleMap and ContentKey index the full registered list. Deduplication is precisely what makes those two stop running in step: the moment a copy is folded into an earlier inode, every file behind it is written with a hole map belonging to some other file.
Fixed by carrying the registered position into fileInodes rather than counting positions in it.
Why it is worth a look beyond the fix
The damage is silent and one-directional. A block that should have held data is left unallocated, so the file comes back exactly the right length and reads perfectly well, with zeros where its bytes were. Nothing throws, no checksum notices, and e2fsck is happy — the volume is entirely well-formed. It is just missing some of its contents.
Neither switch alone misbehaves, so every existing test passed: the sparse tests do not deduplicate, and the deduplication tests are not sparse. It took a check that turns on everything a format says it can do at once.
This is the same shape as the block-mover rule already written down here — an index resolved against a list that is not the one it came from.
Found by a check that asks every format claiming it can reclaim space to prove it — and to prove the files survived it.
The failure
Only when both switches are on:
Root cause
Which blocks of a file are holes is looked up by the position the file was registered at:
But
fileOrdinalcounts alongfileInodes— the files that ended up with an inode of their own — whileHoleMapandContentKeyindex the full registered list. Deduplication is precisely what makes those two stop running in step: the moment a copy is folded into an earlier inode, every file behind it is written with a hole map belonging to some other file.Fixed by carrying the registered position into
fileInodesrather than counting positions in it.Why it is worth a look beyond the fix
The damage is silent and one-directional. A block that should have held data is left unallocated, so the file comes back exactly the right length and reads perfectly well, with zeros where its bytes were. Nothing throws, no checksum notices, and
e2fsckis happy — the volume is entirely well-formed. It is just missing some of its contents.Neither switch alone misbehaves, so every existing test passed: the sparse tests do not deduplicate, and the deduplication tests are not sparse. It took a check that turns on everything a format says it can do at once.
This is the same shape as the block-mover rule already written down here — an index resolved against a list that is not the one it came from.