Fix NTFS $MFTMirr to stay in sync with $MFT

[devedse]

Problem

MasterFileTable.WriteRecord serialized each MFT record twice — once into the $MFT stream and, for records 0-3, once again into the $MFTMirr copy:

record.ToStream(_recordStream, RecordSize);   // $MFT copy
...
record.ToStream(s, RecordSize);               // $MFTMirr copy

FixupRecordBase.ToBytes calls ProtectBuffer, which does UpdateSequenceNumber++ on every serialization. So the mirror copy was always written with a USN one higher than the $MFT copy. The two copies therefore differ at:

• the USN field (offset 0x30 in the record header), and
• the fixup bytes (the last two bytes of every 512-byte sector of the record).

NTFS implementations compare $MFTMirr to $MFT verbatim at mount time, so:

• ntfs-3g refuses to mount: "$MFTMirr does not match $MFT (record 0)".
• chkdsk reports corruption.

Any operation that grows the MFT rewrites the MFT's own record 0 and triggers this.

Wrong vs. right (record 0 header, USN field at offset 0x30)

Wrong (before — mirror serialized separately, USN bumped again):

$MFT     record 0 @0x30:  05 00        (USN = 5)
$MFTMirr record 0 @0x30:  06 00        (USN = 6)   <- mismatch

Right (after — one serialization, identical bytes to both):

$MFT     record 0 @0x30:  05 00        (USN = 5)
$MFTMirr record 0 @0x30:  05 00        (USN = 5)   <- identical

Fix

Serialize the record exactly once into a buffer and write those same bytes to both streams, keeping the existing method structure (main write → MftRecordIsDirty handling → mirror write):

byte[] allocated = null;
var buffer = RecordSize <= 1024
    ? stackalloc byte[RecordSize]
    : (allocated = ArrayPool<byte>.Shared.Rent(RecordSize)).AsSpan(0, RecordSize);
try
{
    buffer.Clear();
    record.ToBytes(buffer);                 // serialized once -> single USN

    _recordStream.Position = record.MasterFileTableIndex * RecordSize;
    _recordStream.Write(buffer);            // $MFT
    _recordStream.Flush();
    ...
    s.Write(buffer);                        // $MFTMirr - identical bytes
}
finally { if (allocated != null) ArrayPool<byte>.Shared.Return(allocated); }

How to reproduce / test

Unit test added (MftMirrorStaysInSyncWithMft in Tests/LibraryTests/Ntfs/NtfsFileSystemTest.cs): formats a 64 MB NTFS volume, reopens it, creates a directory and 400 files of 4096 bytes (which grows the MFT and rewrites record 0), then parses the boot sector to locate $MFT and $MFTMirr and byte-compares records 0-3 of each, asserting they are identical.

The test fails on the unpatched library with exactly:

$MFTMirr does not match $MFT for record 0

(Verified by stashing the MasterFileTable.cs change, running the single test, observing the record-0 failure, then unstashing.)

Manual reproduction with ntfs-3g tools (Linux):

# Produce an image with the library after the 400-file workload, then:
ntfsfix -n disk.img
# Before: reports $MFTMirr does not match $MFT
# After:  "Volume is clean" / mounts OK with nothing to correct

[devedse]

@Olof-Lagerkvist wait one moment with merging this PR untill I've finished my last tests

[Olof-Lagerkvist]

Thanks a lot! I'll just push a small change first that should avoid the errors you got here. After that, I'll merge it.

[devedse]

External validation with ntfs-3g ✅

I generated a real 64 MB NTFS image with the patched library (format → 400 files of 4096 bytes to grow the MFT and rewrite record 0) and ran ntfsfix -n inside a clean mcr.microsoft.com/dotnet/sdk:10.0 Linux container with ntfs-3g installed:

--- ntfsfix -n ---
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /tmp/disk.img was processed successfully.

$MFT and $MFTMirr now match, so ntfs-3g mounts the volume cleanly. Without the fix, the same workload produces a mirror whose USN (offset 0x30) and per-sector fixup bytes are one ahead of $MFT, and the added MftMirrorStaysInSyncWithMft unit test fails with $MFTMirr does not match $MFT for record 0.

Rebased onto Minor optimization prep for pull #72

I rebased this branch onto the latest LTRData.DiscUtils-initial, which includes your prep commit changing _recordStream (and Wipe) from Stream to SparseStream. The fix applies cleanly on top of it; builds pass on net46/net48/netstandard2.0/net10.0 and the full test suite is green (582 passed, 1 skipped, 0 failed).

[Olof-Lagerkvist]

Yes, for some reason _recordStream was declared as Stream instead of SparseStream so it did not have Write(ReadOnlySpan<byte>) overloads on old frameworks. But that was really easy to fix and also good for performance as it reduces the override lookups a bit.

[devedse]

Yeah so again most of this stuff just came up with AI, so I depend mainly on your knowledge of this to verify the changes :)

-Comment not made by AI

[devedse]

Before/after ntfsfix -n comparison

To confirm the fix actually resolves the corruption (not just that the patched image happens to be clean), I ran the identical 400-file workload against both the unpatched library (upstream base 4fc4173a, which still serializes each record twice) and the patched library, then ran ntfsfix -n on each image in a clean dotnet/sdk:10.0 container with ntfs-3g.

Before the fix — mount fails, records 0–3 diverge:

$MFTMirr does not match $MFT (record 0).
Mounting volume... FAILED
Attempting to correct errors...
$MFTMirr does not match $MFT (record 0).
Processing $MFT and $MFTMirr...
Comparing $MFTMirr to $MFT... FAILED
Correcting differences in $MFTMirr record 0...OK
Correcting differences in $MFTMirr record 1...OK
Correcting differences in $MFTMirr record 2...OK
Correcting differences in $MFTMirr record 3...OK
Remount failed: Input/output error

After the fix — mounts cleanly, nothing to correct:

Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /tmp/disk.img was processed successfully.

So the divergence is real and reproducible on the unpatched code, and this PR eliminates it.

[devedse]

I think I need to remove the dependency on:
using DiscUtils.Streams.Compatibility;

Let me verify that, I'll let you know once it's ready to merge from my side.

[Olof-Lagerkvist]

I think I need to remove the dependency on: using DiscUtils.Streams.Compatibility;

Let me verify that, I'll let you know once it's ready to merge from my side.

Yes that is not needed now.

[devedse]

It should be okay from my side now. If you can do one more review after the builds complete then hopefully it's ready to merge 😄.