Add Mach-O support

doc/readme.md

@@ -5,6 +5,7 @@ This is the manual of LibObjectFile with the following API covered:
 - [ELF Object File Format](#elf-object-file-format) via the `ElfFile` API
 - [Archive ar File Format](#archive-ar-file-format) via the `ArArchiveFile` API
 - [PE File Format](#pe-object-file-format) via the `PEFile` API
+- [Mach-O File Format](#mach-o-object-file-format) via the `MachOFile` API
 
 ## ELF Object File Format
 
@@ -545,3 +546,150 @@ Assert.AreEqual(156, process.ExitCode);
 ### Links
 
 - [PE and COFF Specification](https://docs.microsoft.com/en-us/windows/win32/debug/pe-format)
+
+## Mach-O Object File Format
+
+### Overview
+
+The main entry-point for reading/writing a Mach-O file is the [`MachOFile`](https://github.com/xoofx/LibObjectFile/blob/master/src/LibObjectFile/MachO/MachOFile.cs) class.
+
+This class is the equivalent of the Mach-O header and contains the load commands, which describe everything else in the image, along with the content those commands point at.
+
+#### Content
+
+Everything in a Mach-O image is a `MachOContent` in the ordered `MachOFile.Content` list: the header, the load command table, the padding after it, the bytes of each section, each table in `__LINKEDIT`, and any alignment padding in between. Every byte of the file belongs to one of them, so writing the list back out reproduces the file.
+
+Padding is kept as the bytes that were read rather than regenerated, because a linker pads executable sections with `nop` rather than zeros.
+
+#### Addresses and what a layout may move
+
+A section's address is its segment's address plus the section's distance from the segment's file offset:
+
+```
+section.Address - segment.VmAddress == section.FileOffset - segment.FileOffset
+```
+
+Moving a section in the file therefore moves it in memory, and every instruction and relocation referring to it becomes wrong. Content carrying an address is `IsPositionPinned` and a layout leaves it where it is. Only what nothing addresses, which in practice is `__LINKEDIT` and the file tail, is free to move.
+
+This is why a `MachOSegment` describes a mapping rather than owning bytes, and why adding a load command is bounded by the padding the linker left after the command table. `MachOFile.AvailableLoadCommandSpace` reports how much of it is left.
+
+#### Universal binaries
+
+A universal binary holds one image per architecture and is read with [`MachOFatFile`](https://github.com/xoofx/LibObjectFile/blob/master/src/LibObjectFile/MachO/MachOFatFile.cs). Use `MachOFatFile.IsFat` to tell the two apart:
+
+```csharp
+if (MachOFatFile.IsFat(inputStream))
+{
+    var fat = MachOFatFile.Read(inputStream);
+    foreach (var slice in fat.Slices)
+    {
+        Console.WriteLine($"{slice.CpuType}: {slice.File!.FileType}");
+    }
+}
+```
+
+Its header and slice table are stored big-endian whatever the architectures inside are, which is the one place the format departs from the image's own byte order.
+
+### Reading a Mach-O File
+
+The Mach-O API allows to read from a `System.IO.Stream` via the method `MachOFile.Read`:
+
+```csharp
+MachOFile machO = MachOFile.Read(inputStream);
+foreach (var segment in machO.Segments)
+{
+    Console.WriteLine($"{segment.Name} at 0x{segment.VmAddress:X}");
+}
+```
+
+Load commands this library does not model are kept as a `MachOUnknownLoadCommand` and written back verbatim, so an image built by a newer linker still round-trips.
+
+The symbol table, the indirect symbol table and the relocations of a section are decoded on demand:
+
+```csharp
+foreach (var symbol in machO.ReadSymbolTable())
+{
+    Console.WriteLine($"{symbol.Name} {symbol.Kind}");
+}
+```
+
+### Writing a Mach-O File
+
+A `MachOFile` is written back with `MachOFile.Write`:
+
+```csharp
+machO.Write(outputStream);
+```
+
+Content is written at the offset recorded for it, so an image that has not been edited comes back byte for byte. `Write` verifies first and throws if anything is wrong; `TryWrite` reports through a `DiagnosticBag` instead. `MachOFile.Verify` checks an image on its own.
+
+A universal binary is written the same way through `MachOFatFile.Write` or `MachOFatFile.TryWrite`, which lay out each slice before placing it, so a slice that changed size is recorded at the size it actually writes.
+
+The reader requires the load commands to end exactly where the header's `sizeofcmds` says. dyld is looser, stopping once it has walked `ncmds` commands and ignoring any slack after them. The stricter reading is deliberate: writing an image back means reproducing that slack, and an image whose two counts disagree is one where it is not clear which the loader will believe.
+
+### Editing load commands
+
+`MachOFile` offers the operations `install_name_tool` provides, appending to the padding the linker left so that no content moves:
+
+```csharp
+machO.AddLoadDylib("@executable_path/../Frameworks/mylib.dylib");
+machO.AddRPath("@executable_path/../Frameworks");
+machO.RemoveRPath("@loader_path/../Frameworks");
+machO.ChangeDylibName("/usr/lib/libfoo.dylib", "@rpath/libfoo.dylib");
+machO.SetInstallName("@rpath/libmylib.dylib");
+```
+
+`SetInstallName` writes the `LC_ID_DYLIB` name a library reports for itself, which is what the images linking against it record. It applies to a dylib rather than an executable.
+
+A dependency is appended rather than inserted, because dyld identifies a library by the position of its command among the others and the symbol table binds against that number. Removing a dependency is not offered for the same reason. A run path carries no such numbering, so `RemoveRPath` is available.
+
+When the padding runs out these throw, naming the shortfall, rather than moving content and invalidating the image. There is no way around that short of relinking with `-headerpad_max_install_names`, which is also why `install_name_tool` fails in the same situation.
+
+### Code signing
+
+Apple Silicon refuses to execute an unsigned image, so an edited `arm64` binary has to be signed again to stay runnable:
+
+```csharp
+machO.AddRPath("@executable_path/../Frameworks");
+machO.AdHocSign("mytool");
+machO.Write(outputStream);
+```
+
+An ad-hoc signature carries no certificate. It states only that the image hashes to what its code directory says, which is what lets the kernel give the process a stable identity.
+
+Signing has to be the last thing done before writing. Editing afterwards leaves the signature covering bytes that are no longer there, so `MachOFile.IsCodeSignatureStale` records it and writing fails until the image is signed again.
+
+#### Gatekeeper
+
+There are two checks on the way to running a binary, and an ad-hoc signature only gets past one of them.
+
+The kernel is the first. On Apple Silicon it will not execute an unsigned image, and an ad-hoc signature is enough to satisfy it. That is the check an edited `arm64` binary needs signing for.
+
+Gatekeeper is the second, and ad-hoc will not get past it. There is no Developer ID behind the signature and it cannot be notarized. Anything downloaded carries a quarantine attribute, and Gatekeeper refuses it however it was signed here. Clearing the attribute is what lets it launch:
+
+```sh
+xattr -d com.apple.quarantine MyApp.app
+```
+
+One thing worth knowing before reaching for this on a shipping app: re-signing throws away any notarization it came with, and signing it again will not bring that back.
+
+#### Sealing a bundle
+
+`AdHocSign` signs a single Mach-O image. A bundle seals the rest of its files separately, in `Contents/_CodeSignature/CodeResources`, and this library does not touch that seal.
+
+Whether that matters depends on what you edited.
+
+- **The main executable.** It is not in the seal, because its own signature already covers it. Re-sign it and the bundle stays consistent.
+- **A nested framework or dylib.** These are in the seal, by hash. Edit one and the seal goes stale, and the bundle will fail verification until Apple's `codesign` reseals it.
+
+### Printing a Mach-O File
+
+`MachOFile.Print` writes the header and every load command in a form close to `otool -h -l`:
+
+```csharp
+machO.Print(Console.Out);
+```
+
+### Links
+
+- [OS X ABI Mach-O File Format Reference](https://github.com/aidansteele/osx-abi-macho-file-format-reference)

readme.md

@@ -8,10 +8,11 @@ LibObjectFile is a .NET library to read, manipulate and write linker and executa
 >
 > - **PE** image file format (Portable Executable / DLL)
 > - **ELF** object-file format
+> - **Mach-O** object-file format, including universal binaries
 > - **DWARF** debugging format (version 4)
 > - **Archive `ar`** file format (Common, GNU and BSD variants)
 >
-> There is a longer term plan to support other file formats (e.g COFF, MACH-O, .lib) but as I don't 
+> There is a longer term plan to support other file formats (e.g COFF, .lib) but as I don't 
 > have a need for them right now, it is left as an exercise for PR contributors! ;)
 
 ## Usage
@@ -54,6 +55,16 @@ elf.Write(outStream);
   - Program headers with or without sections
   - `ElfFile.AddNeededLibrary` injects a `DT_NEEDED` dependency into an existing image without moving any section (address-preserving, `patchelf`-style)
   - Print with `readelf` similar output
+- Good support for the **Mach-O file format**:
+  - Support byte-to-byte roundtrip
+  - Read and write from/to a `System.IO.Stream`
+  - 32 and 64 bit, `i386`, `x86_64` and `arm64`
+  - Universal (fat) binaries via `MachOFatFile`
+  - The load commands a linked image is built from are decoded; the rest round-trip verbatim
+  - Symbol table, indirect symbol table and relocations
+  - `MachOFile.AddLoadDylib`, `AddRPath`, `RemoveRPath`, `ChangeDylibName` and `SetInstallName` edit an existing image without moving any section (address-preserving, `install_name_tool`-style)
+  - `MachOFile.AdHocSign` writes an ad-hoc code signature, which Apple Silicon requires in order to execute an image
+  - `MachOFile.Print` to print the content of a Mach-O file with `otool` similar output
 - Support for **DWARF debugging format**:
   - Partial support of Version 4 (currently still the default for GCC)
   - Support for the sections: `.debug_info`, `.debug_line`, `.debug_aranges`, `.debug_abbrev` and `.debug_str` 

src/LibObjectFile.Tests/LibObjectFile.Tests.csproj

@@ -25,6 +25,15 @@
     <None Remove="PE\NativeLibraryWin64.dll" />
     <None Remove="PE\RawNativeConsoleWin64.exe" />
     <None Remove="small.cpp" />
+    <None Remove="MachO\chainedfixups_arm64" />
+    <None Remove="MachO\dyldinfo_i386" />
+    <None Remove="MachO\helloworld_arm64" />
+    <None Remove="MachO\helloworld_fat" />
+    <None Remove="MachO\helloworld_x86_64" />
+    <None Remove="MachO\helloworld_x86_64.o" />
+    <None Remove="MachO\libhelloworld_x86_64.dylib" />
+    <None Remove="MachO\unixthread_i386" />
+    <None Remove="MachO\unixthread_i386_rpath" />
   </ItemGroup>
 
   <ItemGroup>
@@ -84,6 +93,35 @@
   <ItemGroup>
     <None Include="$(ProjectDir)TestFiles\**" CopyToOutputDirectory="PreserveNewest" LinkBase="TestFiles\" />
   </ItemGroup>
+  <ItemGroup>
+    <Content Include="MachO\chainedfixups_arm64">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\dyldinfo_i386">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\helloworld_arm64">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\helloworld_fat">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\helloworld_x86_64">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\helloworld_x86_64.o">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\libhelloworld_x86_64.dylib">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\unixthread_i386">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="MachO\unixthread_i386_rpath">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+  </ItemGroup>
 
   <ItemGroup>
     <PackageReference Include="MSTest" />