diff --git a/website/docs/doc/GeneralUpdate.Differential.md b/website/docs/doc/GeneralUpdate.Differential.md index 71cd3b0..04678d2 100644 --- a/website/docs/doc/GeneralUpdate.Differential.md +++ b/website/docs/doc/GeneralUpdate.Differential.md @@ -4,445 +4,447 @@ sidebar_position: 6 # GeneralUpdate.Differential -## 组件概览 +`GeneralUpdate.Differential` 是 GeneralUpdate 的二进制差分组件,专注解决“一个旧文件 + 一个补丁文件 = 一个新文件”的问题。它提供可替换的文件级差分算法、补丁压缩抽象和 BSDIFF 兼容补丁读写能力;目录级对比、批量补丁生成、并行调度、删除文件处理和更新流程编排由 `GeneralUpdate.Core` 的 `DiffPipeline` 或 `GeneralUpdate.Tools` 承担。 -**GeneralUpdate.Differential** 是 GeneralUpdate 框架中负责二进制差异更新的核心组件。该组件提供了强大的差异算法,可以精确识别两个版本之间的文件变化,生成高效的增量补丁包,并支持补丁还原操作。通过使用差异更新,可以显著减少更新包的大小和下载时间,特别适合频繁发布更新的应用场景。 +**命名空间:** `GeneralUpdate.Differential`、`GeneralUpdate.Differential.Differ`、`GeneralUpdate.Differential.Abstractions` -**命名空间:** `GeneralUpdate.Differential` -**程序集:** `GeneralUpdate.Core.dll` +**主要入口:** `IBinaryDiffer`、`BsdiffDiffer`、`StreamingHdiffDiffer` -```csharp -public sealed class DifferentialCore +**NuGet 包:** `GeneralUpdate.Differential` + +```bash +dotnet add package GeneralUpdate.Differential ``` ---- +## 文档大纲与知识点导航 {#knowledge-map} -## 核心特性 +如果你第一次阅读 Differential 文档,可以先看这个导航,再跳到对应知识点。本文按照“能力边界 -> 文件级 API -> 算法选择 -> 压缩格式 -> 与 Core/Tools 集成 -> 性能与扩展”的顺序组织。 -### 1. 增量识别 -- 精确识别新增、修改、删除的文件 -- 智能文件版本对比 -- 支持跳过指定文件和格式 +| 你想了解什么 | 推荐阅读 | +| --- | --- | +| Differential 到底负责什么、不负责什么 | [组件能力边界](#组件能力边界) | +| `Clean` / `Dirty` 是什么含义 | [Clean 与 Dirty 语义](#clean-与-dirty-语义) | +| 如何给单个文件生成并应用补丁 | [单文件快速开始](#单文件快速开始) | +| 使用 Core 时是否还要手动集成 Differential | [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系) | +| 当前有哪些差分算法,如何选择 | [差分算法选择](#差分算法选择) | +| BSDIFF 补丁格式和压缩字节怎么工作 | [补丁格式与压缩 Provider](#补丁格式与压缩-provider) | +| 如何在 Core 更新流程里启用目录级差分 | [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系) | +| Tools 构建差分包时用了什么能力 | [与 GeneralUpdate.Tools 的关系](#与-generalupdatetools-的关系) | +| 下载和差分是否可以多线程并行 | [并发模型与性能建议](#并发模型与性能建议) | +| 大型项目如何提升差分构建效率 | [大型项目并行差分](#大型项目并行差分) | +| 如何接入自定义差分算法或压缩方式 | [扩展点](#扩展点) | -### 2. 二进制补丁生成 -- 高效的二进制差异算法 -- 最小化补丁文件大小 -- 快速补丁生成速度 +## 组件能力边界 -### 3. 补丁还原 -- 安全的补丁应用流程 -- 自动处理文件依赖关系 -- 完整性验证机制 +Differential 是底层文件补丁库,不是完整的更新编排器。理解这个边界可以避免把旧文档里的 `DifferentialCore`、黑名单、目录批量处理等概念误认为当前组件 API。 -### 4. 黑名单支持 -- 文件级黑名单 -- 格式级黑名单 -- 灵活的过滤规则 +| 能力 | Differential 是否负责 | 说明 | +| --- | --- | --- | +| 单文件二进制补丁生成 | 是 | 通过 `IBinaryDiffer.CleanAsync(oldFile, newFile, patchFile)` 完成。 | +| 单文件二进制补丁应用 | 是 | 通过 `IBinaryDiffer.DirtyAsync(oldFile, outputNewFile, patchFile)` 完成。 | +| 差分算法实现 | 是 | 当前主要实现为 `BsdiffDiffer` 和 `StreamingHdiffDiffer`。 | +| 补丁数据压缩/解压 | 是 | 通过 `ICompressionProvider` 抽象,内置 BZip2、Deflate,源码中预留 .NET 6+ Brotli。 | +| 目录级新旧版本对比 | 否 | 由 `GeneralUpdate.Core.Pipeline.DiffPipeline` 的 matcher 负责。 | +| 新增文件复制、删除清单、批量 patch 命名 | 否 | 由 `DiffPipeline` 负责生成 `.patch` 文件、复制新增文件和写入 `generalupdate.delete.json`。 | +| 更新包生成工具 | 否 | 推荐由 `GeneralUpdate.Tools` 调用 Core 差分管道生成发布产物。 | +| 下载、校验、解压、版本回写、重启 | 否 | 这些属于 `GeneralUpdate.Core` 更新流程。 | ---- +> 当前源码中没有旧文档提到的 `DifferentialCore` 单例。直接使用 Differential 组件时,请面向 `IBinaryDiffer` 和具体 differ 实现编程;需要目录级能力时使用 Core 的 `DiffPipeline`。 -## 快速开始 +## Clean 与 Dirty 语义 {#clean-与-dirty-语义} -### 安装 +Differential 沿用了 GeneralUpdate 差分流程中的两个术语: -通过 NuGet 安装 GeneralUpdate.Differential(包含在 Core 包中): +| 术语 | 方法 | 输入 | 输出 | 常用位置 | +| --- | --- | --- | --- | --- | +| `Clean` | `CleanAsync` | 旧文件、新文件、补丁路径 | `.patch` 补丁文件 | 构建/发布阶段 | +| `Dirty` | `DirtyAsync` | 旧文件、输出新文件路径、补丁路径 | 还原后的新文件 | 客户端升级阶段 | -```bash -dotnet add package GeneralUpdate.Core -``` +文件级补丁应用不会直接覆盖旧文件,而是把还原结果写到你传入的 `newFilePath`。Core 的 `DiffPipeline` 在目录级更新时会先写临时文件,成功后再替换原文件,从而避免补丁应用失败时破坏原文件。 -### 初始化与使用 +## 单文件快速开始 -以下示例展示了如何使用 DifferentialCore 进行增量识别和补丁操作: +下面示例只演示 Differential 的底层单文件能力。如果你已经在使用 `GeneralUpdate.Core`,Core 默认已经集成 Differential,不需要为了正常更新流程再手动集成或直接调用本组件。如果你要比较两个目录、生成一批 `.patch`、复制新增文件或处理删除文件,请直接看 [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系)。 ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -// 增量识别并生成二进制补丁 -var sourcePath = @"D:\packet\app"; // 旧版本路径 -var targetPath = @"D:\packet\release"; // 新版本路径 -var patchPath = @"D:\packet\patch"; // 补丁输出路径 +IBinaryDiffer differ = new BsdiffDiffer(); -await DifferentialCore.Instance?.Clean(sourcePath, targetPath, patchPath); +var oldFile = @"D:\releases\1.0.0\app.dll"; +var newFile = @"D:\releases\1.0.1\app.dll"; +var patchFile = @"D:\patches\app.dll.patch"; +var outputFile = @"D:\restore\app.dll"; -// 应用补丁(还原) -await DifferentialCore.Instance?.Dirty(sourcePath, patchPath); -``` +// 生成补丁:oldFile + newFile -> patchFile +await differ.CleanAsync(oldFile, newFile, patchFile); ---- +// 应用补丁:oldFile + patchFile -> outputFile +await differ.DirtyAsync(oldFile, outputFile, patchFile); +``` -## 核心 API 参考 +`CleanAsync` 和 `DirtyAsync` 都支持 `CancellationToken`。当前实现会在任务开始和 Core 管道调度点观察取消请求;单个算法内部不是每一个字节循环都检查取消,因此大文件取消可能会等到当前文件处理结束后才完全停下。 -### DifferentialCore 类 +## 核心 API -#### Instance 属性 +### IBinaryDiffer -获取 DifferentialCore 的单例实例。 +`IBinaryDiffer` 是所有文件级差分算法的统一抽象,也是 Core 差分管道接入自定义算法的关键接口。 ```csharp -public static DifferentialCore Instance { get; } +public interface IBinaryDiffer +{ + Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); + + Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); +} ``` -#### Clean 方法 +| 参数 | 含义 | +| --- | --- | +| `oldFilePath` | 旧版本文件路径。生成补丁和应用补丁时都需要。 | +| `newFilePath` | `CleanAsync` 中表示新版本源文件;`DirtyAsync` 中表示还原后的输出文件。 | +| `patchFilePath` | 补丁文件路径。`CleanAsync` 写入它,`DirtyAsync` 读取它。 | -执行增量识别、删除文件识别,并生成二进制补丁文件。 +### BsdiffDiffer -**方法签名:** +`BsdiffDiffer` 实现 BSDIFF 4.0 文件级二进制差分算法。它会把旧文件和新文件读入内存,通过后缀排序寻找匹配块,再输出控制段、差异段和额外段。 ```csharp -public async Task Clean(string sourcePath, string targetPath, string patchPath = null) +using GeneralUpdate.Differential.Differ; + +var differ = new BsdiffDiffer(); +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -**参数:** -- `sourcePath`: 旧版本文件夹路径 -- `targetPath`: 新版本文件夹路径 -- `patchPath`: 补丁文件输出目录(可选) +| 特性 | 说明 | +| --- | --- | +| 默认压缩 | `BZip2CompressionProvider`,兼容历史 BSDIFF 补丁。 | +| 可替换压缩 | 构造函数接受 `ICompressionProvider`。 | +| 补丁兼容 | 支持 32 字节旧 BSDIFF 头,也支持 33 字节扩展头。 | +| 适用场景 | 追求兼容性、补丁体积稳定、单文件体积可控的场景。 | +| 资源特征 | 生成补丁时会读入旧文件和新文件,单文件很大时需要关注内存占用。 | -**功能说明:** -1. 对比 sourcePath 和 targetPath 两个目录 -2. 识别新增、修改、删除的文件 -3. 为修改的文件生成二进制差异补丁 -4. 将补丁和新增文件保存到 patchPath +`BsdiffDiffer` 也保留了 `Clean(...)` 和 `Dirty(...)` 方法;新代码建议优先面向 `IBinaryDiffer.CleanAsync` / `DirtyAsync`,便于切换算法。 + +### StreamingHdiffDiffer + +`StreamingHdiffDiffer` 是当前源码中的另一种 differ 实现。它使用块级 FNV-1a 哈希索引预筛候选位置,再进行字节级扩展匹配,输出 BSDIFF 兼容的补丁结构。 -**示例:** ```csharp -// 生成从 v1.0.0 到 v1.1.0 的补丁包 -var source = @"D:\MyApp\v1.0.0"; -var target = @"D:\MyApp\v1.1.0"; -var patch = @"D:\MyApp\patches\v1.1.0"; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; + +var differ = new StreamingHdiffDiffer( + compressionProvider: new DeflateCompressionProvider(optimalLevel: true), + blockSize: 64 * 1024, + maxWindowSize: 128 * 1024 * 1024); -await DifferentialCore.Instance.Clean(source, target, patch); -// 结果:patch 目录包含所有必要的增量更新文件 +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -#### Dirty 方法 +| 特性 | 说明 | +| --- | --- | +| 默认压缩 | `DeflateCompressionProvider`。 | +| 块大小 | `BlockSize` 默认 64 KB,用于建立旧文件块哈希索引。 | +| 窗口预算 | `MaxWindowSize` 默认 128 MB,影响生成补丁时参与计算的内存窗口。 | +| 应用补丁 | `DirtyAsync` 委托给 `BsdiffDiffer` 的补丁应用逻辑。 | +| 适用场景 | 需要更快候选匹配、希望与 Core `DiffPipeline` 默认算法保持一致的目录级差分构建。 | -应用补丁,将旧版本文件更新到新版本。 +需要注意的是,当前实现不是完整外存流式差分:当单个文件超过 `MaxWindowSize` 时,算法只会读取预算窗口参与计算。对超大单文件,请在业务侧验证补丁还原结果,或调大 `MaxWindowSize`,或改用 `BsdiffDiffer` 等更适合当前文件规模的实现。 -**方法签名:** +## 差分算法选择 -```csharp -public async Task Dirty(string appPath, string patchPath) -``` +当前 Differential 内置两种文件级差分算法。它们都输出 BSDIFF 兼容补丁结构,但生成补丁时的匹配方式、默认压缩、性能侧重点不同。 -**参数:** -- `appPath`: 客户端应用程序目录(当前版本) -- `patchPath`: 补丁文件路径 +| 对比项 | `BsdiffDiffer` | `StreamingHdiffDiffer` | +| --- | --- | --- | +| 核心思路 | 经典 BSDIFF 4.0,基于后缀排序寻找旧文件和新文件之间的最长匹配。 | 使用块级 FNV-1a 哈希建立旧文件索引,先用哈希快速筛选候选块,再做字节级扩展匹配。 | +| 默认压缩 | BZip2 (`0x00`)。 | Deflate (`0x01`)。 | +| 补丁应用 | 自己实现 BSDIFF Dirty 逻辑。 | `DirtyAsync` 委托给 `BsdiffDiffer`,因此应用阶段和 BSDIFF 补丁兼容。 | +| 生成效率 | 匹配更精细,局部或分散变化下补丁生成表现稳定;但后缀排序和全量读入会带来 CPU/内存开销。 | 块命中效果好时生成更快;如果变化分散、块哈希命中少,生成可能变慢。 | +| 客户端应用性能 | 默认 BZip2 解压成本更高,客户端应用大量补丁时耗时可能更明显。 | 默认 Deflate 解压更快,更适合客户端批量应用补丁。 | +| 补丁体积倾向 | 通常更追求细粒度匹配,补丁体积明显更稳定。 | 速度优先,补丁体积与文件变化分布、块大小、窗口预算强相关;块命中差时可能接近完整文件。 | +| 内存特征 | 生成阶段读取旧文件和新文件,单个大文件需要关注内存峰值。 | 通过 `BlockSize` 和 `MaxWindowSize` 控制匹配窗口,超大单文件需要额外验证或调参。 | +| 兼容性 | 最适合需要兼容旧 BSDIFF/BZip2 补丁的场景。 | 适合新项目、目录级批量差分和 Core `DiffPipeline` 默认构建。 | -**功能说明:** -1. 读取 patchPath 中的补丁文件 -2. 将补丁应用到 appPath 中的对应文件 -3. 处理新增文件的复制 -4. 处理删除文件的移除 +可以简单理解为:`BsdiffDiffer` 更偏“兼容和补丁体积稳定”,`StreamingHdiffDiffer` 更偏“客户端应用速度和可调参数”。如果项目非常在意补丁体积或文件变化较分散,优先考虑 `BsdiffDiffer`;如果项目更在意客户端应用速度,并且经过压测确认补丁体积可接受,可以考虑 `StreamingHdiffDiffer`。 -**示例:** -```csharp -// 将补丁应用到当前应用程序 -var appDir = AppDomain.CurrentDomain.BaseDirectory; -var patchDir = Path.Combine(appDir, "temp", "patches"); +### 参考基准数据 {#benchmark-reference} -await DifferentialCore.Instance.Dirty(appDir, patchDir); -// 结果:应用程序更新到新版本 -``` +下面数据来自当前源码的一组本地微基准,用于给开发者判断量级,不是跨所有项目的性能承诺。测试环境为 Windows x64、.NET Release 构建,使用 2-4 MB 合成文件;真实结果会受 CPU、磁盘、文件类型、变化比例、压缩级别和并行度影响。 ---- +| 场景 | `BsdiffDiffer` 生成 | `StreamingHdiffDiffer` 生成 | `BsdiffDiffer` 应用 | `StreamingHdiffDiffer` 应用 | `BsdiffDiffer` 补丁体积 | `StreamingHdiffDiffer` 补丁体积 | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| 2 MB 文本,少量行变更/插入 | 484 ms | 2059 ms | 55 ms | 36 ms | 0.05% | 3.50% | +| 4 MB 二进制,连续局部块变更 | 1030 ms | 318 ms | 55 ms | 11 ms | 2.58% | 100.04% | +| 4 MB 二进制,随机分散字节变更 | 757 ms | 4176 ms | 70 ms | 30 ms | 2.18% | 100.27% | -## 实际使用示例 +从这组数据可以得到几个实用预估: -### 示例 1:基本补丁生成 +| 指标 | 参考结论 | +| --- | --- | +| 补丁体积 | `BsdiffDiffer` 在测试场景中约为新文件的 0.05%-2.58%;`StreamingHdiffDiffer` 约为 3.50%-100%。如果补丁包大小是第一优先级,优先测试 `BsdiffDiffer`。 | +| 客户端应用速度 | `StreamingHdiffDiffer` 默认 Deflate,在测试中应用补丁约快 1.5-5 倍。大量文件批量应用时,这个差距会更明显。 | +| 生成速度 | 没有绝对赢家:连续局部二进制变更中 `StreamingHdiffDiffer` 约快 3.2 倍;文本和随机分散变更中 `BsdiffDiffer` 约快 4.3-5.5 倍。 | +| 大型项目选择 | 大型项目建议同时看“补丁总体积 + 构建耗时 + 客户端应用耗时”。如果大量文件可以并行,`WithParallelism(...)` 往往比单个 differ 的微小差距更影响总体耗时。 | -```csharp -using GeneralUpdate.Differential; +> 这组数据的重点是帮助判断方向:`BsdiffDiffer` 通常更容易得到小补丁,`StreamingHdiffDiffer` 的应用阶段更快,但补丁体积和生成速度对文件变化形态非常敏感。正式发布前建议用自己项目的真实产物做一次压测。 + +推荐选择: + +| 场景 | 建议 | +| --- | --- | +| 只需要低层单文件补丁,并希望最大兼容 | 使用 `new BsdiffDiffer()`。 | +| 通过 Core `DiffPipeline` 批量生成目录级补丁 | 先用默认配置跑基准;若补丁体积偏大,可显式切换到 `BsdiffDiffer`;再结合 `WithParallelism(...)` 提升吞吐。 | +| 客户端解压性能更敏感 | 优先选择 Deflate 补丁,即 `StreamingHdiffDiffer` 默认配置,或 `new BsdiffDiffer(new DeflateCompressionProvider())`。 | +| 历史补丁仍是旧 BSDIFF/BZip2 | 使用 `BsdiffDiffer` 应用;32 字节头会按 BZip2 处理。 | +| 大型项目包含大量 DLL、资源文件、插件文件 | 使用 Core `DiffPipeline` 做文件级并行,避免自己逐个文件串行调用 Differential。 | + +## 补丁格式与压缩 Provider {#补丁格式与压缩-provider} -public async Task GeneratePatchAsync() +Differential 生成的是 BSDIFF 风格补丁。当前实现写入 33 字节扩展头: + +| 偏移 | 长度 | 含义 | +| --- | --- | --- | +| `0` | 8 | 魔数 `"BSDIFF40"`。 | +| `8` | 8 | 压缩后控制段长度。 | +| `16` | 8 | 压缩后差异段长度。 | +| `24` | 8 | 新文件长度。 | +| `32` | 1 | 压缩格式版本。 | + +应用补丁时也兼容 32 字节旧头:如果没有第 33 个格式字节,就按 BZip2 旧补丁处理。 + +### ICompressionProvider + +`ICompressionProvider` 负责把控制段、差异段和额外段包装成压缩流。 + +```csharp +public interface ICompressionProvider { - try - { - // 版本路径 - var oldVersion = @"D:\MyApp\1.0.0"; - var newVersion = @"D:\MyApp\1.0.1"; - var patchOutput = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("开始生成补丁..."); - - // 生成补丁 - await DifferentialCore.Instance.Clean(oldVersion, newVersion, patchOutput); - - Console.WriteLine($"补丁生成完成!输出目录:{patchOutput}"); - - // 显示补丁信息 - var patchFiles = Directory.GetFiles(patchOutput, "*.*", SearchOption.AllDirectories); - Console.WriteLine($"生成了 {patchFiles.Length} 个补丁文件"); - - long totalSize = patchFiles.Sum(f => new FileInfo(f).Length); - Console.WriteLine($"总补丁大小:{totalSize / 1024.0:F2} KB"); - } - catch (Exception ex) - { - Console.WriteLine($"补丁生成失败:{ex.Message}"); - } + byte FormatVersion { get; } + + Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default); + + Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default); } ``` -### 示例 2:应用补丁 +| Provider | 格式字节 | 当前可用性 | 说明 | +| --- | --- | --- | --- | +| `BZip2CompressionProvider` | `0x00` | 可用 | `BsdiffDiffer` 默认值,兼容旧 BSDIFF 补丁。 | +| `DeflateCompressionProvider` | `0x01` | 可用 | BCL `DeflateStream`,解压速度更适合客户端更新。 | +| `BrotliCompressionProvider` | `0x02` | 源码中以 `NET6_0_OR_GREATER` 条件编译预留 | 当前 `GeneralUpdate.Differential` 项目目标为 `netstandard2.0`,并且补丁读取逻辑当前只识别 `0x00` / `0x01`,不要把 Brotli 作为生产更新包格式。 | + +自定义压缩时,生成补丁和应用补丁必须使用能被补丁读取逻辑识别的格式字节。当前生产建议只使用 BZip2 或 Deflate。 ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -public async Task ApplyPatchAsync() -{ - try - { - // 应用程序目录 - var appDirectory = @"D:\MyApp\current"; - // 补丁目录 - var patchDirectory = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("开始应用补丁..."); - - // 验证补丁存在 - if (!Directory.Exists(patchDirectory)) - { - throw new DirectoryNotFoundException($"补丁目录不存在:{patchDirectory}"); - } - - // 应用补丁 - await DifferentialCore.Instance.Dirty(appDirectory, patchDirectory); - - Console.WriteLine("补丁应用成功!应用程序已更新到新版本。"); - } - catch (Exception ex) - { - Console.WriteLine($"补丁应用失败:{ex.Message}"); - } -} +var differ = new BsdiffDiffer( + new DeflateCompressionProvider(optimalLevel: false)); + +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -### 示例 3:完整的补丁流程 +## 与 GeneralUpdate.Core 的关系 {#与-generalupdatecore-的关系} -```csharp -using GeneralUpdate.Differential; -using System.IO.Compression; +`GeneralUpdate.Core` 在 Differential 之上提供目录级差分管道 `DiffPipeline`。它会负责: -public class PatchManager -{ - // 生成并打包补丁 - public async Task CreatePatchPackageAsync( - string oldVersionPath, - string newVersionPath, - string outputPath) - { - try - { - // 1. 生成补丁文件 - var tempPatchDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempPatchDir); - - Console.WriteLine($"正在生成补丁..."); - await DifferentialCore.Instance.Clean(oldVersionPath, newVersionPath, tempPatchDir); - - // 2. 压缩补丁文件 - var patchZipPath = Path.Combine(outputPath, "patch_1.0.1.zip"); - Console.WriteLine($"正在打包补丁..."); - - if (File.Exists(patchZipPath)) - File.Delete(patchZipPath); - - ZipFile.CreateFromDirectory(tempPatchDir, patchZipPath, - CompressionLevel.Optimal, false); - - // 3. 清理临时文件 - Directory.Delete(tempPatchDir, true); - - var patchSize = new FileInfo(patchZipPath).Length; - Console.WriteLine($"补丁包创建成功:{patchZipPath}"); - Console.WriteLine($"补丁包大小:{patchSize / 1024.0:F2} KB"); - - return patchZipPath; - } - catch (Exception ex) - { - Console.WriteLine($"创建补丁包失败:{ex.Message}"); - throw; - } - } - - // 解压并应用补丁 - public async Task ApplyPatchPackageAsync(string appPath, string patchZipPath) - { - try - { - // 1. 解压补丁包 - var tempExtractDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempExtractDir); - - Console.WriteLine($"正在解压补丁包..."); - ZipFile.ExtractToDirectory(patchZipPath, tempExtractDir); - - // 2. 应用补丁 - Console.WriteLine($"正在应用补丁..."); - await DifferentialCore.Instance.Dirty(appPath, tempExtractDir); - - // 3. 清理临时文件 - Directory.Delete(tempExtractDir, true); - - Console.WriteLine($"补丁应用成功!"); - } - catch (Exception ex) - { - Console.WriteLine($"应用补丁包失败:{ex.Message}"); - throw; - } - } -} +1. 对比旧目录和新目录。 +2. 找出发生变化的文件并调用 `IBinaryDiffer.CleanAsync` 生成 `.patch`。 +3. 复制新增文件到补丁目录。 +4. 生成 `generalupdate.delete.json` 记录删除文件。 +5. 客户端应用补丁时并行调用 `IBinaryDiffer.DirtyAsync`,先写临时文件,成功后替换原文件。 + +如果你在应用更新流程中使用 `GeneralUpdate.Core`,Core 默认已经集成 Differential 并内置差分管道。也就是说,常规更新接入时不需要额外安装、初始化或手动调用 `GeneralUpdate.Differential`;只要使用 Core 的更新流程,并按业务需要启用补丁更新能力,Core 会在内部完成 differ 创建、补丁应用和目录级编排。 + +只有在你想替换默认差分算法、调整并行度、改变错误策略或接入自定义 matcher 时,才需要通过 `UseDiffPipeline` 做高级配置: -// 使用示例 -var manager = new PatchManager(); - -// 创建补丁包 -var patchZip = await manager.CreatePatchPackageAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\releases" -); - -// 应用补丁包 -await manager.ApplyPatchPackageAsync( - @"D:\MyApp\current", - patchZip -); +```csharp +using GeneralUpdate.Core; +using GeneralUpdate.Core.Models; +using GeneralUpdate.Differential.Differ; + +await new GeneralUpdateBootstrap() + .SetSource( + updateUrl: "https://update.example.com/api/upgrade/verification", + appSecretKey: "your-app-secret") + .SetOption(Option.AppType, AppType.Client) + .SetOption(Option.PatchEnabled, true) + .UseDiffPipeline(builder => builder + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .WithStopOnFirstError(true)) + .LaunchAsync(); ``` -### 示例 4:带进度显示的补丁操作 +当前源码里有两个默认层级需要区分: + +| 使用方式 | 默认 differ | +| --- | --- | +| 直接 `new DiffPipeline()` 或 `new DiffPipelineBuilder().Build()` | `StreamingHdiffDiffer` | +| `GeneralUpdateBootstrap` 未显式调用 `UseDiffPipeline(...)` 时内部构建 | `BsdiffDiffer`,并行度 2,带 `DiffProgressReporter` | + +因此,普通用户可以把 Differential 看作 Core 已经带好的底层能力,不需要特地集成;只有希望 Core 更新流程明确使用某个算法或自定义差分行为时,才建议显式调用 `UseDiffPipeline(...)`。 + +## 与 GeneralUpdate.Tools 的关系 {#与-generalupdatetools-的关系} + +`GeneralUpdate.Tools` 面向发布侧,帮助开发者构建更新产物。当前 `DiffService` 会创建 `new DiffPipeline()`,再调用: ```csharp -using GeneralUpdate.Differential; +await pipeline.CleanAsync(oldDir, newDir, patchDir); +``` -public class ProgressivePatchManager -{ - public async Task GeneratePatchWithProgressAsync( - string sourcePath, - string targetPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("开始扫描文件差异..."); - - // 在实际场景中,可以在 Clean 前后添加进度报告 - await DifferentialCore.Instance.Clean(sourcePath, targetPath, patchPath); - - progress?.Report("补丁生成完成!"); - - // 统计信息 - var files = Directory.GetFiles(patchPath, "*.*", SearchOption.AllDirectories); - progress?.Report($"共生成 {files.Length} 个补丁文件"); - } - catch (Exception ex) - { - progress?.Report($"错误:{ex.Message}"); - throw; - } - } - - public async Task ApplyPatchWithProgressAsync( - string appPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("开始应用补丁..."); - - await DifferentialCore.Instance.Dirty(appPath, patchPath); - - progress?.Report("补丁应用成功!"); - } - catch (Exception ex) - { - progress?.Report($"错误:{ex.Message}"); - throw; - } - } -} +也就是说,Tools 生成目录级差分包时,本质上使用的是 Core 的 `DiffPipeline`,而 `DiffPipeline` 再调用 Differential 的 `IBinaryDiffer` 生成每个变更文件的补丁。对大多数开发者来说,推荐路径是: + +1. 用 Tools 对比旧版本目录和新版本目录,生成补丁目录和清单产物。 +2. 用 Core 在客户端检查版本、下载补丁包、应用补丁。 +3. 只有在需要自定义差分算法、压缩格式或单文件补丁实验时,才直接使用 Differential。 + +这种分层可以让业务代码保持简单:Tools 负责构建,Core 负责更新,Differential 负责底层文件差分。 + +## 并发模型与性能建议 -// 使用示例 -var manager = new ProgressivePatchManager(); -var progress = new Progress(msg => Console.WriteLine($"[{DateTime.Now:HH:mm:ss}] {msg}")); +Differential 的单个 differ 实例没有保存某次补丁任务的可变共享状态。只要传入的 `ICompressionProvider` 是线程安全的,内置 differ 可以被 Core 管道并发调用;内置 BZip2、Deflate provider 都会为每次调用创建新的压缩流,适合并发使用。 -await manager.GeneratePatchWithProgressAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\patches\1.0.1", - progress -); +真正的“多线程差分”通常发生在 Core `DiffPipeline` 层: + +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .Build(); + +await pipeline.CleanAsync(oldDir, newDir, patchDir); ``` ---- -## 注意事项与警告 +### 大型项目并行差分 {#大型项目并行差分} + +大型桌面项目通常不是“一个超大文件”,而是由主程序、多个 DLL、插件、资源文件、运行时文件和配置文件组成。Core `DiffPipeline` 会把目录对比结果拆成文件级任务,每个变更文件独立调用 `IBinaryDiffer.CleanAsync` 生成补丁,因此可以通过 `WithParallelism(...)` 同时处理多个文件。 + +这种并行模型对大型项目很重要: -### ⚠️ 重要提示 +1. 构建侧可以同时为多个变更文件生成 `.patch`,缩短发布包构建时间。 +2. 客户端应用补丁时也可以并行处理多个文件,减少升级窗口。 +3. 新增文件复制、删除清单处理和差分补丁生成由 Core 管道统一编排,开发者不需要手写多线程调度。 +4. 并行度可以按机器能力调整,构建机可以设置更高,低配置客户端可以保持较低。 -1. **文件名限制** - - 不能包含同名但扩展名不同的文件(如 file.txt 和 file.log) - - 建议使用唯一的文件名命名规则 +| 参数/策略 | 建议 | +| --- | --- | +| `WithParallelism(1)` | 资源敏感、机械硬盘、低内存环境。 | +| `WithParallelism(2)` | 默认平衡值,适合多数桌面应用。 | +| `WithParallelism(4-8)` | 多核 CPU、SSD、构建机或发布服务器。 | +| BZip2 | 补丁兼容性好,但客户端解压成本更高。 | +| Deflate | 解压速度更友好,适合客户端大批量应用补丁。 | +| 大文件 | 先压测补丁生成耗时、内存峰值和还原结果,不要只看补丁体积。 | -2. **目录结构** - - 源目录和目标目录的相对结构应保持一致 - - 补丁生成时会保留目录层次关系 +并行差分适合“文件数量多、每个文件可独立处理”的大型项目。需要注意的是,单个超大文件内部仍由具体 differ 算法处理,不会因为 `WithParallelism(8)` 就把一个文件拆成 8 份并行计算;并行度提升的是多个文件之间的吞吐。 -3. **磁盘空间** - - 确保有足够的磁盘空间存储补丁文件 - - 二进制差异补丁通常比完整文件小,但仍需要临时空间 +下载与差分可以在上层更新流程中并行:Core 下载阶段可以并发拉取多个资源,差分应用阶段也可以按文件并行处理补丁。Differential 只负责单个文件的补丁计算,不直接管理网络下载线程。 -4. **文件占用** - - 应用补丁时,确保目标文件没有被其他进程占用 - - 建议在应用程序关闭后应用补丁 +## 扩展点 -5. **备份建议** - - 在应用补丁前建议备份原始文件 - - 可以使用 Core 的 BackUp 选项自动备份 +### 自定义差分算法 -### 💡 最佳实践 +实现 `IBinaryDiffer` 后即可接入 Core 管道。适合接入其他算法、调用原生库,或对特定文件类型做特殊优化。 -- **版本管理**:为每个版本维护独立的补丁包,便于版本追踪和回滚 -- **补丁验证**:生成补丁后进行验证测试,确保补丁可以正确应用 -- **增量更新**:优先使用差异更新而非全量更新,可节省 50%-90% 的下载量 -- **错误处理**:实现完整的异常捕获和错误恢复机制 -- **性能优化**:对于大文件,差异算法的性能优势更加明显 +```csharp +using GeneralUpdate.Differential.Abstractions; -### 🔍 工作原理 +public sealed class MyBinaryDiffer : IBinaryDiffer +{ + public Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Generate patchFilePath from oldFilePath and newFilePath. + throw new NotImplementedException(); + } -**Clean 方法工作流程:** -1. 扫描源目录和目标目录中的所有文件 -2. 比较文件的MD5哈希值以识别变化 -3. 对于修改的文件,使用二进制差异算法生成补丁 -4. 对于新增文件,直接复制到补丁目录 -5. 记录删除的文件列表 + public Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Restore newFilePath from oldFilePath and patchFilePath. + throw new NotImplementedException(); + } +} +``` -**Dirty 方法工作流程:** -1. 读取补丁目录中的所有文件 -2. 对于补丁文件,应用到对应的原文件上 -3. 对于新增文件,直接复制到应用目录 -4. 根据删除列表移除相应文件 -5. 验证更新完整性 +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new MyBinaryDiffer()) + .WithParallelism(4) + .Build(); +``` ---- +自定义算法需要保证 `CleanAsync` 产出的补丁能被同一算法的 `DirtyAsync` 正确应用;如果补丁要交给 Core 客户端使用,发布侧和客户端必须使用同一套 differ 实现。 -## 适用平台 +### 自定义压缩 Provider -| 产品 | 版本 | -| ------------------ | ----------------- | -| .NET | 5, 6, 7, 8, 9, 10 | -| .NET Framework | 4.6.1 | -| .NET Standard | 2.0 | -| .NET Core | 2.0 | +如果仍使用 BSDIFF 兼容补丁结构,只想替换控制段、差异段和额外段的压缩方式,可以实现 `ICompressionProvider`。 ---- +```csharp +using GeneralUpdate.Differential.Abstractions; + +public sealed class MyCompressionProvider : ICompressionProvider +{ + public byte FormatVersion => 0x01; + + public Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + output, + CompressionLevel.Optimal, + leaveOpen: true); + } + + public Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + input, + CompressionMode.Decompress, + leaveOpen: true); + } +} +``` + +不要随意分配新的 `FormatVersion`。当前 `BsdiffDiffer.DirtyAsync` 只识别 BZip2 (`0x00`) 和 Deflate (`0x01`);如果你引入新格式,也需要同步扩展补丁读取逻辑,否则客户端无法应用补丁。 + +## 实战建议 -## 相关资源 +| 场景 | 推荐做法 | +| --- | --- | +| 普通应用发布差分更新 | 使用 `GeneralUpdate.Tools` 生成产物,客户端使用 Core。 | +| 需要控制目录级并行、错误策略和进度 | 使用 Core `DiffPipelineBuilder`。 | +| 只验证某个文件的补丁效果 | 直接使用 `IBinaryDiffer`。 | +| 对补丁体积和应用速度都敏感 | 对同一组文件分别测试 BZip2、Deflate 和不同算法后再定默认策略。 | +| 更新包需要长期兼容旧客户端 | 保守使用 `BsdiffDiffer` + BZip2,或确保客户端已支持 Deflate 扩展头。 | -- **示例代码**:[查看 GitHub 示例](https://github.com/GeneralLibrary/GeneralUpdate-Samples/tree/main/src/Diff) -- **主仓库**:[GeneralUpdate 项目](https://github.com/GeneralLibrary/GeneralUpdate) -- **打包工具**:GeneralUpdate.PacketTool 项目依赖此组件实现差异打包 +Differential 的价值在于把复杂的二进制差分能力收敛成稳定的文件级抽象。上层开发者可以把重点放在“什么时候更新、下载什么、如何提示用户”上,把具体补丁生成和应用交给 Core/Tools/Differential 的组合完成。 diff --git a/website/i18n/en/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md b/website/i18n/en/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md index 781c7ec..c13391f 100644 --- a/website/i18n/en/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md +++ b/website/i18n/en/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md @@ -4,446 +4,447 @@ sidebar_position: 6 # GeneralUpdate.Differential -## Component Overview +`GeneralUpdate.Differential` is the binary differential component in GeneralUpdate. It focuses on one low-level problem: old file + patch file = new file. It provides replaceable file-level differ algorithms, patch compression abstractions, and BSDIFF-compatible patch read/write support. Directory comparison, batch patch generation, parallel scheduling, deleted-file handling, and update orchestration are handled by `GeneralUpdate.Core` through `DiffPipeline`, or by `GeneralUpdate.Tools` on the publishing side. -**GeneralUpdate.Differential** is the core component responsible for binary differential updates in the GeneralUpdate framework. This component provides powerful differential algorithms that can accurately identify file changes between two versions, generate efficient incremental patch packages, and support patch restoration operations. By using differential updates, you can significantly reduce update package size and download time, making it particularly suitable for scenarios with frequent update releases. +**Namespaces:** `GeneralUpdate.Differential`, `GeneralUpdate.Differential.Differ`, `GeneralUpdate.Differential.Abstractions` -**Namespace:** `GeneralUpdate.Differential` -**Assembly:** `GeneralUpdate.Core.dll` +**Main entry points:** `IBinaryDiffer`, `BsdiffDiffer`, `StreamingHdiffDiffer` -```csharp -public sealed class DifferentialCore +**NuGet package:** `GeneralUpdate.Differential` + +```bash +dotnet add package GeneralUpdate.Differential ``` ---- +## Documentation outline and topic navigation {#knowledge-map} -## Core Features +If this is your first time reading the Differential documentation, start with this map and jump to the topic you need. The page is organized as "boundaries -> file-level API -> algorithm selection -> compression format -> Core/Tools integration -> performance and extension points". -### 1. Incremental Identification -- Accurately identify added, modified, and deleted files -- Intelligent file version comparison -- Support skipping specified files and formats +| What you want to learn | Recommended section | +| --- | --- | +| What Differential is responsible for | [Component boundaries](#component-boundaries) | +| What `Clean` and `Dirty` mean | [Clean and Dirty semantics](#clean-and-dirty-semantics) | +| How to generate and apply a patch for one file | [Single-file quick start](#single-file-quick-start) | +| Whether Core users need to integrate Differential manually | [Relationship with GeneralUpdate.Core](#relationship-with-generalupdatecore) | +| Which algorithms exist and how to choose one | [Differ algorithm selection](#differ-algorithm-selection) | +| How the BSDIFF header and compression byte work | [Patch format and compression providers](#patch-format-and-compression-providers) | +| How directory-level differential updates are enabled in Core | [Relationship with GeneralUpdate.Core](#relationship-with-generalupdatecore) | +| What Tools uses when building differential packages | [Relationship with GeneralUpdate.Tools](#relationship-with-generalupdatetools) | +| Whether downloads and diff work can run in parallel | [Concurrency model and performance guidance](#concurrency-model-and-performance-guidance) | +| How large projects can improve differential build throughput | [Parallel differential work for large projects](#parallel-differential-work-for-large-projects) | +| How to plug in a custom differ or compression provider | [Extension points](#extension-points) | -### 2. Binary Patch Generation -- Efficient binary differential algorithm -- Minimize patch file size -- Fast patch generation speed +## Component boundaries -### 3. Patch Restoration -- Safe patch application process -- Automatic handling of file dependencies -- Integrity verification mechanism +Differential is a low-level file patching library, not a complete update orchestrator. This boundary is important because older documentation mentioned concepts such as `DifferentialCore`, blacklists, and directory batch processing, but those are not the current public API of this component. -### 4. Blacklist Support -- File-level blacklist -- Format-level blacklist -- Flexible filtering rules +| Capability | Owned by Differential | Notes | +| --- | --- | --- | +| Generate a binary patch for one file | Yes | Done through `IBinaryDiffer.CleanAsync(oldFile, newFile, patchFile)`. | +| Apply a binary patch for one file | Yes | Done through `IBinaryDiffer.DirtyAsync(oldFile, outputNewFile, patchFile)`. | +| Differ algorithm implementations | Yes | Current main implementations are `BsdiffDiffer` and `StreamingHdiffDiffer`. | +| Compress and decompress patch data | Yes | Abstracted by `ICompressionProvider`; BZip2 and Deflate are available, with .NET 6+ Brotli reserved in source. | +| Compare old and new directories | No | Handled by matchers in `GeneralUpdate.Core.Pipeline.DiffPipeline`. | +| Copy new files, write delete manifests, name batch patches | No | Handled by `DiffPipeline`, which writes `.patch` files, copies new files, and writes `generalupdate.delete.json`. | +| Build update packages | No | Prefer `GeneralUpdate.Tools`, which calls the Core differential pipeline. | +| Download, verify, unzip, write back versions, restart apps | No | These belong to the `GeneralUpdate.Core` update flow. | ---- +> The current source does not contain the old `DifferentialCore` singleton mentioned by previous docs. When using Differential directly, program against `IBinaryDiffer` and concrete differ implementations. For directory-level behavior, use Core's `DiffPipeline`. -## Quick Start +## Clean and Dirty semantics {#clean-and-dirty-semantics} -### Installation +Differential follows the two terms used by the GeneralUpdate differential flow: -Install GeneralUpdate.Differential via NuGet (included in Core package): +| Term | Method | Input | Output | Common location | +| --- | --- | --- | --- | --- | +| `Clean` | `CleanAsync` | Old file, new file, patch path | `.patch` file | Build/publishing side | +| `Dirty` | `DirtyAsync` | Old file, output new-file path, patch path | Restored new file | Client upgrade side | -```bash -dotnet add package GeneralUpdate.Core -``` +File-level patch application does not overwrite the old file directly. It writes the restored result to the `newFilePath` you pass in. At the directory level, Core's `DiffPipeline` writes to a temporary file first and replaces the original only after the patch is applied successfully, avoiding corruption if patch application fails. -### Initialization and Usage +## Single-file quick start -The following example demonstrates how to use DifferentialCore for incremental identification and patch operations: +This example only demonstrates the low-level single-file capability. If you already use `GeneralUpdate.Core`, Core integrates Differential by default, so you do not need to manually integrate or directly call this component for the normal update flow. If you need to compare two directories, generate many `.patch` files, copy added files, or handle deleted files, go to [Relationship with GeneralUpdate.Core](#relationship-with-generalupdatecore). ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -// Identify increments and generate binary patches -var sourcePath = @"D:\packet\app"; // Old version path -var targetPath = @"D:\packet\release"; // New version path -var patchPath = @"D:\packet\patch"; // Patch output path +IBinaryDiffer differ = new BsdiffDiffer(); -await DifferentialCore.Instance?.Clean(sourcePath, targetPath, patchPath); +var oldFile = @"D:\releases\1.0.0\app.dll"; +var newFile = @"D:\releases\1.0.1\app.dll"; +var patchFile = @"D:\patches\app.dll.patch"; +var outputFile = @"D:\restore\app.dll"; -// Apply patches (restoration) -await DifferentialCore.Instance?.Dirty(sourcePath, patchPath); -``` +// Generate patch: oldFile + newFile -> patchFile +await differ.CleanAsync(oldFile, newFile, patchFile); ---- +// Apply patch: oldFile + patchFile -> outputFile +await differ.DirtyAsync(oldFile, outputFile, patchFile); +``` -## Core API Reference +Both `CleanAsync` and `DirtyAsync` accept a `CancellationToken`. The current implementations observe cancellation when work starts and at Core pipeline scheduling points; the inner algorithm loops do not check cancellation for every byte, so canceling a large file may complete only after the current file finishes processing. -### DifferentialCore Class +## Core API -#### Instance Property +### IBinaryDiffer -Get the singleton instance of DifferentialCore. +`IBinaryDiffer` is the shared abstraction for all file-level differ algorithms, and it is the key interface used by Core's differential pipeline. ```csharp -public static DifferentialCore Instance { get; } +public interface IBinaryDiffer +{ + Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); + + Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); +} ``` -#### Clean Method +| Parameter | Meaning | +| --- | --- | +| `oldFilePath` | Path to the old-version file. Required when generating and applying patches. | +| `newFilePath` | In `CleanAsync`, the source new-version file; in `DirtyAsync`, the restored output file. | +| `patchFilePath` | Path to the patch file. `CleanAsync` writes it; `DirtyAsync` reads it. | -Perform incremental identification, deleted file identification, and generate binary patch files. +### BsdiffDiffer -**Method Signature:** +`BsdiffDiffer` implements the BSDIFF 4.0 file-level binary diff algorithm. It reads the old and new files into memory, uses suffix sorting to find matching blocks, then writes control, diff, and extra sections. ```csharp -public async Task Clean(string sourcePath, string targetPath, string patchPath = null) +using GeneralUpdate.Differential.Differ; + +var differ = new BsdiffDiffer(); +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -**Parameters:** -- `sourcePath`: Path to the old version folder -- `targetPath`: Path to the new version folder -- `patchPath`: Directory to store discovered incremental update files (optional) +| Feature | Notes | +| --- | --- | +| Default compression | `BZip2CompressionProvider`, for compatibility with legacy BSDIFF patches. | +| Replaceable compression | The constructor accepts an `ICompressionProvider`. | +| Patch compatibility | Supports legacy 32-byte BSDIFF headers and the current 33-byte extended header. | +| Best fit | Compatibility-focused scenarios where file size is manageable and stable patch size matters. | +| Resource profile | Patch generation reads both old and new files, so large single files require memory planning. | -**Function Description:** -1. Compare sourcePath and targetPath directories -2. Identify added, modified, and deleted files -3. Generate binary differential patches for modified files -4. Save patches and added files to patchPath +`BsdiffDiffer` also keeps `Clean(...)` and `Dirty(...)` methods. New code should prefer `IBinaryDiffer.CleanAsync` and `DirtyAsync` so the algorithm can be swapped later. + +### StreamingHdiffDiffer + +`StreamingHdiffDiffer` is the other differ implementation in the current source. It builds a block-level FNV-1a hash index to pre-filter candidate positions, extends matches at byte level, and writes a BSDIFF-compatible patch structure. -**Example:** ```csharp -// Generate patch package from v1.0.0 to v1.1.0 -var source = @"D:\MyApp\v1.0.0"; -var target = @"D:\MyApp\v1.1.0"; -var patch = @"D:\MyApp\patches\v1.1.0"; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; + +var differ = new StreamingHdiffDiffer( + compressionProvider: new DeflateCompressionProvider(optimalLevel: true), + blockSize: 64 * 1024, + maxWindowSize: 128 * 1024 * 1024); -await DifferentialCore.Instance.Clean(source, target, patch); -// Result: patch directory contains all necessary incremental update files +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -#### Dirty Method +| Feature | Notes | +| --- | --- | +| Default compression | `DeflateCompressionProvider`. | +| Block size | `BlockSize` defaults to 64 KB and is used to build the old-file block hash index. | +| Window budget | `MaxWindowSize` defaults to 128 MB and affects the memory window used during patch generation. | +| Patch application | `DirtyAsync` delegates to `BsdiffDiffer` patch application logic. | +| Best fit | Faster candidate matching and directory-level builds that should align with `DiffPipeline` defaults. | -Apply patches to update old version files to the new version. +The current implementation is not a full external-memory streaming differ. If a single file exceeds `MaxWindowSize`, only the budgeted window participates in the calculation. For very large files, validate restoration results in your publishing pipeline, increase `MaxWindowSize`, or choose `BsdiffDiffer` or another implementation that better fits the file size. -**Method Signature:** +## Differ algorithm selection -```csharp -public async Task Dirty(string appPath, string patchPath) -``` +Differential currently includes two file-level differ algorithms. Both write a BSDIFF-compatible patch structure, but their match strategy, default compression, and performance priorities are different. -**Parameters:** -- `appPath`: Client application directory (current version) -- `patchPath`: Path to the patch files +| Comparison | `BsdiffDiffer` | `StreamingHdiffDiffer` | +| --- | --- | --- | +| Core idea | Classic BSDIFF 4.0; uses suffix sorting to find long matches between old and new files. | Builds an old-file index with block-level FNV-1a hashes, filters candidate blocks quickly, then extends matches at byte level. | +| Default compression | BZip2 (`0x00`). | Deflate (`0x01`). | +| Patch application | Implements BSDIFF Dirty logic directly. | `DirtyAsync` delegates to `BsdiffDiffer`, so the apply phase remains BSDIFF-compatible. | +| Generation efficiency | More fine-grained matching and stable behavior for localized or dispersed changes, but suffix sorting and full file loading cost CPU and memory. | Faster when block matches are effective; can become slower when changes are dispersed and block-hash hits are rare. | +| Client-side apply performance | Default BZip2 decompression is more expensive when many patches are applied. | Default Deflate decompression is faster and friendlier for applying many client patches. | +| Patch-size tendency | Usually aims for fine-grained matches and much more stable patch size. | Speed-first; patch size strongly depends on change distribution, block size, and window budget. When block hits are poor, the patch can approach the full file size. | +| Memory profile | Reads the old and new files during generation, so large single files need memory planning. | Uses `BlockSize` and `MaxWindowSize` to control the matching window; very large single files need validation or tuning. | +| Compatibility | Best when legacy BSDIFF/BZip2 compatibility matters. | Best for new projects, directory-level batch diffs, and Core `DiffPipeline` default builds. | -**Function Description:** -1. Read patch files from patchPath -2. Apply patches to corresponding files in appPath -3. Handle copying of added files -4. Handle removal of deleted files +As a rule of thumb, `BsdiffDiffer` leans toward compatibility and stable patch size, while `StreamingHdiffDiffer` leans toward faster client-side apply and tunable parameters. If patch size matters most or changes are dispersed, start with `BsdiffDiffer`. If client-side apply speed matters more and your own benchmarks show acceptable patch size, consider `StreamingHdiffDiffer`. -**Example:** -```csharp -// Apply patches to current application -var appDir = AppDomain.CurrentDomain.BaseDirectory; -var patchDir = Path.Combine(appDir, "temp", "patches"); +### Benchmark reference {#benchmark-reference} -await DifferentialCore.Instance.Dirty(appDir, patchDir); -// Result: application updated to new version -``` +The following numbers come from a local microbenchmark against the current source. They are meant to give developers an order-of-magnitude reference, not a performance guarantee across all projects. The test used Windows x64, a .NET Release build, and synthetic 2-4 MB files. Real results depend on CPU, disk, file type, change ratio, compression level, and parallelism. ---- +| Scenario | `BsdiffDiffer` clean | `StreamingHdiffDiffer` clean | `BsdiffDiffer` dirty | `StreamingHdiffDiffer` dirty | `BsdiffDiffer` patch size | `StreamingHdiffDiffer` patch size | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| 2 MB text, small line changes/inserts | 484 ms | 2059 ms | 55 ms | 36 ms | 0.05% | 3.50% | +| 4 MB binary, contiguous local block change | 1030 ms | 318 ms | 55 ms | 11 ms | 2.58% | 100.04% | +| 4 MB binary, dispersed random byte changes | 757 ms | 4176 ms | 70 ms | 30 ms | 2.18% | 100.27% | -## Practical Usage Examples +Practical estimates from this benchmark: -### Example 1: Basic Patch Generation +| Metric | Reference conclusion | +| --- | --- | +| Patch size | `BsdiffDiffer` produced patches around 0.05%-2.58% of the new file in these scenarios; `StreamingHdiffDiffer` produced around 3.50%-100%. If package size is the top priority, test `BsdiffDiffer` first. | +| Client-side apply speed | `StreamingHdiffDiffer` defaults to Deflate and applied patches about 1.5-5x faster in this benchmark. The difference matters more when many files are applied. | +| Generation speed | There is no absolute winner: `StreamingHdiffDiffer` was about 3.2x faster for the contiguous binary block change, while `BsdiffDiffer` was about 4.3-5.5x faster for text and dispersed random changes. | +| Large project choice | For large projects, evaluate total patch size, build time, and client apply time together. If many files can be processed independently, `WithParallelism(...)` often affects total time more than small differences in a single differ. | -```csharp -using GeneralUpdate.Differential; +> The key takeaway is directional: `BsdiffDiffer` more often produces small patches, while `StreamingHdiffDiffer` applies patches faster, but patch size and generation speed are very sensitive to the shape of file changes. Before production release, benchmark with your own application artifacts. + +Recommended choices: + +| Scenario | Recommendation | +| --- | --- | +| Low-level single-file patching with maximum compatibility | Use `new BsdiffDiffer()`. | +| Directory-level batch patches through Core `DiffPipeline` | Benchmark the default configuration first; if patch size is too large, explicitly switch to `BsdiffDiffer`; then combine it with `WithParallelism(...)` to improve throughput. | +| Client-side decompression performance is more important | Prefer Deflate patches: `StreamingHdiffDiffer` defaults, or `new BsdiffDiffer(new DeflateCompressionProvider())`. | +| Existing patches are legacy BSDIFF/BZip2 | Apply them with `BsdiffDiffer`; 32-byte headers are treated as BZip2. | +| Large projects with many DLLs, resources, or plugin files | Use Core `DiffPipeline` for file-level parallelism instead of calling Differential file by file in a serial loop. | + +## Patch format and compression providers {#patch-format-and-compression-providers} -public async Task GeneratePatchAsync() +Differential writes BSDIFF-style patches. The current implementation writes a 33-byte extended header: + +| Offset | Length | Meaning | +| --- | --- | --- | +| `0` | 8 | Magic string `"BSDIFF40"`. | +| `8` | 8 | Compressed control-section length. | +| `16` | 8 | Compressed diff-section length. | +| `24` | 8 | New-file length. | +| `32` | 1 | Compression format version. | + +Patch application is also compatible with legacy 32-byte headers. If the 33rd format byte is absent, the patch is treated as a BZip2 legacy patch. + +### ICompressionProvider + +`ICompressionProvider` wraps the control, diff, and extra sections in compression streams. + +```csharp +public interface ICompressionProvider { - try - { - // Version paths - var oldVersion = @"D:\MyApp\1.0.0"; - var newVersion = @"D:\MyApp\1.0.1"; - var patchOutput = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("Starting patch generation..."); - - // Generate patches - await DifferentialCore.Instance.Clean(oldVersion, newVersion, patchOutput); - - Console.WriteLine($"Patch generation complete! Output directory: {patchOutput}"); - - // Display patch information - var patchFiles = Directory.GetFiles(patchOutput, "*.*", SearchOption.AllDirectories); - Console.WriteLine($"Generated {patchFiles.Length} patch files"); - - long totalSize = patchFiles.Sum(f => new FileInfo(f).Length); - Console.WriteLine($"Total patch size: {totalSize / 1024.0:F2} KB"); - } - catch (Exception ex) - { - Console.WriteLine($"Patch generation failed: {ex.Message}"); - } + byte FormatVersion { get; } + + Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default); + + Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default); } ``` -### Example 2: Apply Patches +| Provider | Format byte | Current availability | Notes | +| --- | --- | --- | --- | +| `BZip2CompressionProvider` | `0x00` | Available | Default for `BsdiffDiffer`; compatible with legacy BSDIFF patches. | +| `DeflateCompressionProvider` | `0x01` | Available | BCL `DeflateStream`; friendlier decompression speed for client updates. | +| `BrotliCompressionProvider` | `0x02` | Reserved in source behind `NET6_0_OR_GREATER` | The current `GeneralUpdate.Differential` project targets `netstandard2.0`, and the patch reader currently recognizes only `0x00` and `0x01`, so do not use Brotli for production update packages. | + +When customizing compression, generated patches must use a format byte that the patch reader can recognize. For production use today, prefer BZip2 or Deflate. ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -public async Task ApplyPatchAsync() -{ - try - { - // Application directory - var appDirectory = @"D:\MyApp\current"; - // Patch directory - var patchDirectory = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("Starting patch application..."); - - // Verify patch exists - if (!Directory.Exists(patchDirectory)) - { - throw new DirectoryNotFoundException($"Patch directory not found: {patchDirectory}"); - } - - // Apply patches - await DifferentialCore.Instance.Dirty(appDirectory, patchDirectory); - - Console.WriteLine("Patch applied successfully! Application updated to new version."); - } - catch (Exception ex) - { - Console.WriteLine($"Patch application failed: {ex.Message}"); - } -} +var differ = new BsdiffDiffer( + new DeflateCompressionProvider(optimalLevel: false)); + +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -### Example 3: Complete Patch Workflow +## Relationship with GeneralUpdate.Core {#relationship-with-generalupdatecore} -```csharp -using GeneralUpdate.Differential; -using System.IO.Compression; +`GeneralUpdate.Core` builds directory-level differential updates on top of Differential through `DiffPipeline`. It is responsible for: -public class PatchManager -{ - // Generate and package patches - public async Task CreatePatchPackageAsync( - string oldVersionPath, - string newVersionPath, - string outputPath) - { - try - { - // 1. Generate patch files - var tempPatchDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempPatchDir); - - Console.WriteLine($"Generating patches..."); - await DifferentialCore.Instance.Clean(oldVersionPath, newVersionPath, tempPatchDir); - - // 2. Compress patch files - var patchZipPath = Path.Combine(outputPath, "patch_1.0.1.zip"); - Console.WriteLine($"Packaging patches..."); - - if (File.Exists(patchZipPath)) - File.Delete(patchZipPath); - - ZipFile.CreateFromDirectory(tempPatchDir, patchZipPath, - CompressionLevel.Optimal, false); - - // 3. Clean up temp files - Directory.Delete(tempPatchDir, true); - - var patchSize = new FileInfo(patchZipPath).Length; - Console.WriteLine($"Patch package created successfully: {patchZipPath}"); - Console.WriteLine($"Patch package size: {patchSize / 1024.0:F2} KB"); - - return patchZipPath; - } - catch (Exception ex) - { - Console.WriteLine($"Patch package creation failed: {ex.Message}"); - throw; - } - } - - // Extract and apply patches - public async Task ApplyPatchPackageAsync(string appPath, string patchZipPath) - { - try - { - // 1. Extract patch package - var tempExtractDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempExtractDir); - - Console.WriteLine($"Extracting patch package..."); - ZipFile.ExtractToDirectory(patchZipPath, tempExtractDir); - - // 2. Apply patches - Console.WriteLine($"Applying patches..."); - await DifferentialCore.Instance.Dirty(appPath, tempExtractDir); - - // 3. Clean up temp files - Directory.Delete(tempExtractDir, true); - - Console.WriteLine($"Patch applied successfully!"); - } - catch (Exception ex) - { - Console.WriteLine($"Patch package application failed: {ex.Message}"); - throw; - } - } -} +1. Comparing the old and new directories. +2. Finding changed files and calling `IBinaryDiffer.CleanAsync` to generate `.patch` files. +3. Copying added files into the patch directory. +4. Writing `generalupdate.delete.json` for deleted files. +5. Applying patches on the client by calling `IBinaryDiffer.DirtyAsync` in parallel, writing temporary files first, then replacing originals after success. + +If you use `GeneralUpdate.Core` in an application update flow, Core integrates Differential by default and includes the differential pipeline out of the box. In other words, normal update integration does not require extra installation, initialization, or direct calls to `GeneralUpdate.Differential`. As long as you use the Core update flow and enable patch update behavior as needed, Core creates the differ, applies patches, and performs directory-level orchestration internally. + +Use `UseDiffPipeline` only for advanced configuration, such as replacing the default differ algorithm, tuning parallelism, changing error behavior, or plugging in custom matchers: -// Usage example -var manager = new PatchManager(); - -// Create patch package -var patchZip = await manager.CreatePatchPackageAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\releases" -); - -// Apply patch package -await manager.ApplyPatchPackageAsync( - @"D:\MyApp\current", - patchZip -); +```csharp +using GeneralUpdate.Core; +using GeneralUpdate.Core.Models; +using GeneralUpdate.Differential.Differ; + +await new GeneralUpdateBootstrap() + .SetSource( + updateUrl: "https://update.example.com/api/upgrade/verification", + appSecretKey: "your-app-secret") + .SetOption(Option.AppType, AppType.Client) + .SetOption(Option.PatchEnabled, true) + .UseDiffPipeline(builder => builder + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .WithStopOnFirstError(true)) + .LaunchAsync(); ``` -### Example 4: Patch Operations with Progress Display +There are two default layers in the current source: + +| Usage | Default differ | +| --- | --- | +| Direct `new DiffPipeline()` or `new DiffPipelineBuilder().Build()` | `StreamingHdiffDiffer` | +| `GeneralUpdateBootstrap` without an explicit `UseDiffPipeline(...)` | `BsdiffDiffer`, parallelism 2, with `DiffProgressReporter` | + +For regular users, Differential can be treated as a built-in Core capability that does not need separate integration. Call `UseDiffPipeline(...)` only when you want Core to use a specific algorithm or custom differential behavior. + +## Relationship with GeneralUpdate.Tools {#relationship-with-generalupdatetools} + +`GeneralUpdate.Tools` targets the publishing side and helps developers build update artifacts. The current `DiffService` creates `new DiffPipeline()` and calls: ```csharp -using GeneralUpdate.Differential; +await pipeline.CleanAsync(oldDir, newDir, patchDir); +``` -public class ProgressivePatchManager -{ - public async Task GeneratePatchWithProgressAsync( - string sourcePath, - string targetPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("Starting file difference scan..."); - - // In actual scenarios, you can add progress reports before and after Clean - await DifferentialCore.Instance.Clean(sourcePath, targetPath, patchPath); - - progress?.Report("Patch generation complete!"); - - // Statistics - var files = Directory.GetFiles(patchPath, "*.*", SearchOption.AllDirectories); - progress?.Report($"Generated {files.Length} patch files"); - } - catch (Exception ex) - { - progress?.Report($"Error: {ex.Message}"); - throw; - } - } - - public async Task ApplyPatchWithProgressAsync( - string appPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("Starting patch application..."); - - await DifferentialCore.Instance.Dirty(appPath, patchPath); - - progress?.Report("Patch applied successfully!"); - } - catch (Exception ex) - { - progress?.Report($"Error: {ex.Message}"); - throw; - } - } -} +In other words, Tools uses Core's `DiffPipeline` to build directory-level differential packages, and `DiffPipeline` calls Differential's `IBinaryDiffer` for each changed file. For most developers, the recommended path is: + +1. Use Tools to compare the old and new version directories and generate the patch directory plus manifest artifacts. +2. Use Core on the client to check versions, download packages, and apply patches. +3. Use Differential directly only when experimenting with custom algorithms, compression formats, or single-file patching. + +This layering keeps business code simple: Tools builds artifacts, Core runs updates, and Differential handles low-level file diffs. + +## Concurrency model and performance guidance -// Usage example -var manager = new ProgressivePatchManager(); -var progress = new Progress(msg => Console.WriteLine($"[{DateTime.Now:HH:mm:ss}] {msg}")); +A Differential differ instance does not store mutable shared state for a specific patch operation. As long as the supplied `ICompressionProvider` is thread-safe, the built-in differ implementations can be called concurrently by Core's pipeline. The built-in BZip2 and Deflate providers create a new compression stream for each call and are suitable for concurrent use. -await manager.GeneratePatchWithProgressAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\patches\1.0.1", - progress -); +Real "multi-threaded diff" normally happens at the Core `DiffPipeline` layer: + +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .Build(); + +await pipeline.CleanAsync(oldDir, newDir, patchDir); ``` ---- +### Parallel differential work for large projects {#parallel-differential-work-for-large-projects} -## Notes and Warnings +Large desktop projects are usually not one huge file. They are often composed of the main executable, many DLLs, plugins, resource files, runtime files, and configuration files. Core `DiffPipeline` splits the directory comparison result into file-level tasks, and each changed file independently calls `IBinaryDiffer.CleanAsync` to produce a patch. That means `WithParallelism(...)` can process multiple files at the same time. -### ⚠️ Important Notes +This parallel model matters for large projects: -1. **File Name Restrictions** - - Cannot include files with the same name but different extensions (e.g., file.txt and file.log) - - Recommended to use unique file naming conventions +1. The publishing side can generate `.patch` files for multiple changed files at once, shortening package build time. +2. The client side can also apply multiple file patches in parallel, reducing the upgrade window. +3. Core orchestrates new-file copying, delete manifest handling, and differential patch generation, so developers do not need to write custom thread scheduling. +4. Parallelism can be tuned for the machine: higher on build servers, lower on resource-sensitive clients. -2. **Directory Structure** - - The relative structure of source and target directories should remain consistent - - Directory hierarchy is preserved during patch generation +| Parameter/strategy | Guidance | +| --- | --- | +| `WithParallelism(1)` | Resource-sensitive devices, HDDs, or low-memory environments. | +| `WithParallelism(2)` | Balanced default for most desktop applications. | +| `WithParallelism(4-8)` | Multi-core CPUs, SSDs, build machines, or publishing servers. | +| BZip2 | Better compatibility, but higher client-side decompression cost. | +| Deflate | Friendlier decompression speed for applying many client patches. | +| Large files | Measure generation time, memory peak, and restoration correctness; do not look only at patch size. | -3. **Disk Space** - - Ensure sufficient disk space for storing patch files - - Binary differential patches are usually smaller than complete files, but still require temporary space +Parallel differential work is best for large projects with many independently processable files. A single very large file is still handled internally by the selected differ algorithm; `WithParallelism(8)` does not split one file into eight parallel chunks. It improves throughput across multiple files. -4. **File Locking** - - When applying patches, ensure target files are not locked by other processes - - Recommended to apply patches after the application is closed +Downloads and differential work can run in parallel at the upper update-flow level: Core can download multiple resources concurrently, and the patch application phase can process files in parallel. Differential itself only computes the patch for one file and does not manage network download threads. -5. **Backup Recommendations** - - Recommended to backup original files before applying patches - - Can use Core's BackUp option for automatic backup +## Extension points -### 💡 Best Practices +### Custom differ algorithm -- **Version Management**: Maintain separate patch packages for each version for easier version tracking and rollback -- **Patch Validation**: Perform validation testing after generating patches to ensure they can be applied correctly -- **Incremental Updates**: Prioritize differential updates over full updates, which can save 50%-90% of download size -- **Error Handling**: Implement complete exception capture and error recovery mechanisms -- **Performance Optimization**: For large files, the performance advantage of the differential algorithm is more pronounced +Implement `IBinaryDiffer` to plug another algorithm into Core. This is useful when integrating a native library, optimizing specific file types, or replacing the built-in algorithms. -### 🔍 How It Works +```csharp +using GeneralUpdate.Differential.Abstractions; -**Clean Method Workflow:** -1. Scan all files in source and target directories -2. Compare MD5 hash values of files to identify changes -3. For modified files, use binary differential algorithm to generate patches -4. For added files, copy directly to patch directory -5. Record list of deleted files +public sealed class MyBinaryDiffer : IBinaryDiffer +{ + public Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Generate patchFilePath from oldFilePath and newFilePath. + throw new NotImplementedException(); + } + + public Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Restore newFilePath from oldFilePath and patchFilePath. + throw new NotImplementedException(); + } +} +``` -**Dirty Method Workflow:** -1. Read all files in patch directory -2. For patch files, apply to corresponding original files -3. For added files, copy directly to application directory -4. Remove corresponding files according to deletion list -5. Verify update integrity +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new MyBinaryDiffer()) + .WithParallelism(4) + .Build(); +``` ---- +Your custom algorithm must ensure that patches produced by `CleanAsync` can be applied correctly by the same algorithm's `DirtyAsync`. If the patch will be consumed by Core clients, the publishing side and client side must use the same differ implementation. -## Applicable Platforms +### Custom compression provider -| Product | Version | -| -------------- | ------------- | -| .NET | 5, 6, 7, 8, 9, 10 | -| .NET Framework | 4.6.1 | -| .NET Standard | 2.0 | -| .NET Core | 2.0 | +If you want to keep the BSDIFF-compatible patch structure but change how the control, diff, and extra sections are compressed, implement `ICompressionProvider`. ---- +```csharp +using GeneralUpdate.Differential.Abstractions; + +public sealed class MyCompressionProvider : ICompressionProvider +{ + public byte FormatVersion => 0x01; + + public Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + output, + CompressionLevel.Optimal, + leaveOpen: true); + } + + public Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + input, + CompressionMode.Decompress, + leaveOpen: true); + } +} +``` + +Do not allocate a new `FormatVersion` casually. Current `BsdiffDiffer.DirtyAsync` recognizes only BZip2 (`0x00`) and Deflate (`0x01`). If you introduce a new format, you must also extend the patch reader, otherwise clients cannot apply the patch. + +## Practical guidance -## Related Resources +| Scenario | Recommended approach | +| --- | --- | +| Regular application differential updates | Use `GeneralUpdate.Tools` to build artifacts and Core on the client. | +| Need control over directory-level parallelism, error handling, and progress | Use Core's `DiffPipelineBuilder`. | +| Need to validate one file's patch behavior | Use `IBinaryDiffer` directly. | +| Patch size and application speed both matter | Benchmark BZip2, Deflate, and different algorithms on the same file set before choosing a default. | +| Packages must remain compatible with older clients | Conservatively use `BsdiffDiffer` + BZip2, or ensure clients already support the Deflate extended header. | -- **Sample Code**: [View GitHub Examples](https://github.com/GeneralLibrary/GeneralUpdate-Samples/tree/main/src/Diff) -- **Main Repository**: [GeneralUpdate Project](https://github.com/GeneralLibrary/GeneralUpdate) -- **Packaging Tool**: The GeneralUpdate.PacketTool project depends on this component for differential packaging +Differential's value is that it compresses complex binary diff behavior into a stable file-level abstraction. Application developers can focus on when to update, what to download, and how to inform users, while Core, Tools, and Differential handle patch generation and application together. diff --git a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md index 71cd3b0..04678d2 100644 --- a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md +++ b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/doc/GeneralUpdate.Differential.md @@ -4,445 +4,447 @@ sidebar_position: 6 # GeneralUpdate.Differential -## 组件概览 +`GeneralUpdate.Differential` 是 GeneralUpdate 的二进制差分组件,专注解决“一个旧文件 + 一个补丁文件 = 一个新文件”的问题。它提供可替换的文件级差分算法、补丁压缩抽象和 BSDIFF 兼容补丁读写能力;目录级对比、批量补丁生成、并行调度、删除文件处理和更新流程编排由 `GeneralUpdate.Core` 的 `DiffPipeline` 或 `GeneralUpdate.Tools` 承担。 -**GeneralUpdate.Differential** 是 GeneralUpdate 框架中负责二进制差异更新的核心组件。该组件提供了强大的差异算法,可以精确识别两个版本之间的文件变化,生成高效的增量补丁包,并支持补丁还原操作。通过使用差异更新,可以显著减少更新包的大小和下载时间,特别适合频繁发布更新的应用场景。 +**命名空间:** `GeneralUpdate.Differential`、`GeneralUpdate.Differential.Differ`、`GeneralUpdate.Differential.Abstractions` -**命名空间:** `GeneralUpdate.Differential` -**程序集:** `GeneralUpdate.Core.dll` +**主要入口:** `IBinaryDiffer`、`BsdiffDiffer`、`StreamingHdiffDiffer` -```csharp -public sealed class DifferentialCore +**NuGet 包:** `GeneralUpdate.Differential` + +```bash +dotnet add package GeneralUpdate.Differential ``` ---- +## 文档大纲与知识点导航 {#knowledge-map} -## 核心特性 +如果你第一次阅读 Differential 文档,可以先看这个导航,再跳到对应知识点。本文按照“能力边界 -> 文件级 API -> 算法选择 -> 压缩格式 -> 与 Core/Tools 集成 -> 性能与扩展”的顺序组织。 -### 1. 增量识别 -- 精确识别新增、修改、删除的文件 -- 智能文件版本对比 -- 支持跳过指定文件和格式 +| 你想了解什么 | 推荐阅读 | +| --- | --- | +| Differential 到底负责什么、不负责什么 | [组件能力边界](#组件能力边界) | +| `Clean` / `Dirty` 是什么含义 | [Clean 与 Dirty 语义](#clean-与-dirty-语义) | +| 如何给单个文件生成并应用补丁 | [单文件快速开始](#单文件快速开始) | +| 使用 Core 时是否还要手动集成 Differential | [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系) | +| 当前有哪些差分算法,如何选择 | [差分算法选择](#差分算法选择) | +| BSDIFF 补丁格式和压缩字节怎么工作 | [补丁格式与压缩 Provider](#补丁格式与压缩-provider) | +| 如何在 Core 更新流程里启用目录级差分 | [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系) | +| Tools 构建差分包时用了什么能力 | [与 GeneralUpdate.Tools 的关系](#与-generalupdatetools-的关系) | +| 下载和差分是否可以多线程并行 | [并发模型与性能建议](#并发模型与性能建议) | +| 大型项目如何提升差分构建效率 | [大型项目并行差分](#大型项目并行差分) | +| 如何接入自定义差分算法或压缩方式 | [扩展点](#扩展点) | -### 2. 二进制补丁生成 -- 高效的二进制差异算法 -- 最小化补丁文件大小 -- 快速补丁生成速度 +## 组件能力边界 -### 3. 补丁还原 -- 安全的补丁应用流程 -- 自动处理文件依赖关系 -- 完整性验证机制 +Differential 是底层文件补丁库,不是完整的更新编排器。理解这个边界可以避免把旧文档里的 `DifferentialCore`、黑名单、目录批量处理等概念误认为当前组件 API。 -### 4. 黑名单支持 -- 文件级黑名单 -- 格式级黑名单 -- 灵活的过滤规则 +| 能力 | Differential 是否负责 | 说明 | +| --- | --- | --- | +| 单文件二进制补丁生成 | 是 | 通过 `IBinaryDiffer.CleanAsync(oldFile, newFile, patchFile)` 完成。 | +| 单文件二进制补丁应用 | 是 | 通过 `IBinaryDiffer.DirtyAsync(oldFile, outputNewFile, patchFile)` 完成。 | +| 差分算法实现 | 是 | 当前主要实现为 `BsdiffDiffer` 和 `StreamingHdiffDiffer`。 | +| 补丁数据压缩/解压 | 是 | 通过 `ICompressionProvider` 抽象,内置 BZip2、Deflate,源码中预留 .NET 6+ Brotli。 | +| 目录级新旧版本对比 | 否 | 由 `GeneralUpdate.Core.Pipeline.DiffPipeline` 的 matcher 负责。 | +| 新增文件复制、删除清单、批量 patch 命名 | 否 | 由 `DiffPipeline` 负责生成 `.patch` 文件、复制新增文件和写入 `generalupdate.delete.json`。 | +| 更新包生成工具 | 否 | 推荐由 `GeneralUpdate.Tools` 调用 Core 差分管道生成发布产物。 | +| 下载、校验、解压、版本回写、重启 | 否 | 这些属于 `GeneralUpdate.Core` 更新流程。 | ---- +> 当前源码中没有旧文档提到的 `DifferentialCore` 单例。直接使用 Differential 组件时,请面向 `IBinaryDiffer` 和具体 differ 实现编程;需要目录级能力时使用 Core 的 `DiffPipeline`。 -## 快速开始 +## Clean 与 Dirty 语义 {#clean-与-dirty-语义} -### 安装 +Differential 沿用了 GeneralUpdate 差分流程中的两个术语: -通过 NuGet 安装 GeneralUpdate.Differential(包含在 Core 包中): +| 术语 | 方法 | 输入 | 输出 | 常用位置 | +| --- | --- | --- | --- | --- | +| `Clean` | `CleanAsync` | 旧文件、新文件、补丁路径 | `.patch` 补丁文件 | 构建/发布阶段 | +| `Dirty` | `DirtyAsync` | 旧文件、输出新文件路径、补丁路径 | 还原后的新文件 | 客户端升级阶段 | -```bash -dotnet add package GeneralUpdate.Core -``` +文件级补丁应用不会直接覆盖旧文件,而是把还原结果写到你传入的 `newFilePath`。Core 的 `DiffPipeline` 在目录级更新时会先写临时文件,成功后再替换原文件,从而避免补丁应用失败时破坏原文件。 -### 初始化与使用 +## 单文件快速开始 -以下示例展示了如何使用 DifferentialCore 进行增量识别和补丁操作: +下面示例只演示 Differential 的底层单文件能力。如果你已经在使用 `GeneralUpdate.Core`,Core 默认已经集成 Differential,不需要为了正常更新流程再手动集成或直接调用本组件。如果你要比较两个目录、生成一批 `.patch`、复制新增文件或处理删除文件,请直接看 [与 GeneralUpdate.Core 的关系](#与-generalupdatecore-的关系)。 ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -// 增量识别并生成二进制补丁 -var sourcePath = @"D:\packet\app"; // 旧版本路径 -var targetPath = @"D:\packet\release"; // 新版本路径 -var patchPath = @"D:\packet\patch"; // 补丁输出路径 +IBinaryDiffer differ = new BsdiffDiffer(); -await DifferentialCore.Instance?.Clean(sourcePath, targetPath, patchPath); +var oldFile = @"D:\releases\1.0.0\app.dll"; +var newFile = @"D:\releases\1.0.1\app.dll"; +var patchFile = @"D:\patches\app.dll.patch"; +var outputFile = @"D:\restore\app.dll"; -// 应用补丁(还原) -await DifferentialCore.Instance?.Dirty(sourcePath, patchPath); -``` +// 生成补丁:oldFile + newFile -> patchFile +await differ.CleanAsync(oldFile, newFile, patchFile); ---- +// 应用补丁:oldFile + patchFile -> outputFile +await differ.DirtyAsync(oldFile, outputFile, patchFile); +``` -## 核心 API 参考 +`CleanAsync` 和 `DirtyAsync` 都支持 `CancellationToken`。当前实现会在任务开始和 Core 管道调度点观察取消请求;单个算法内部不是每一个字节循环都检查取消,因此大文件取消可能会等到当前文件处理结束后才完全停下。 -### DifferentialCore 类 +## 核心 API -#### Instance 属性 +### IBinaryDiffer -获取 DifferentialCore 的单例实例。 +`IBinaryDiffer` 是所有文件级差分算法的统一抽象,也是 Core 差分管道接入自定义算法的关键接口。 ```csharp -public static DifferentialCore Instance { get; } +public interface IBinaryDiffer +{ + Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); + + Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default); +} ``` -#### Clean 方法 +| 参数 | 含义 | +| --- | --- | +| `oldFilePath` | 旧版本文件路径。生成补丁和应用补丁时都需要。 | +| `newFilePath` | `CleanAsync` 中表示新版本源文件;`DirtyAsync` 中表示还原后的输出文件。 | +| `patchFilePath` | 补丁文件路径。`CleanAsync` 写入它,`DirtyAsync` 读取它。 | -执行增量识别、删除文件识别,并生成二进制补丁文件。 +### BsdiffDiffer -**方法签名:** +`BsdiffDiffer` 实现 BSDIFF 4.0 文件级二进制差分算法。它会把旧文件和新文件读入内存,通过后缀排序寻找匹配块,再输出控制段、差异段和额外段。 ```csharp -public async Task Clean(string sourcePath, string targetPath, string patchPath = null) +using GeneralUpdate.Differential.Differ; + +var differ = new BsdiffDiffer(); +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -**参数:** -- `sourcePath`: 旧版本文件夹路径 -- `targetPath`: 新版本文件夹路径 -- `patchPath`: 补丁文件输出目录(可选) +| 特性 | 说明 | +| --- | --- | +| 默认压缩 | `BZip2CompressionProvider`,兼容历史 BSDIFF 补丁。 | +| 可替换压缩 | 构造函数接受 `ICompressionProvider`。 | +| 补丁兼容 | 支持 32 字节旧 BSDIFF 头,也支持 33 字节扩展头。 | +| 适用场景 | 追求兼容性、补丁体积稳定、单文件体积可控的场景。 | +| 资源特征 | 生成补丁时会读入旧文件和新文件,单文件很大时需要关注内存占用。 | -**功能说明:** -1. 对比 sourcePath 和 targetPath 两个目录 -2. 识别新增、修改、删除的文件 -3. 为修改的文件生成二进制差异补丁 -4. 将补丁和新增文件保存到 patchPath +`BsdiffDiffer` 也保留了 `Clean(...)` 和 `Dirty(...)` 方法;新代码建议优先面向 `IBinaryDiffer.CleanAsync` / `DirtyAsync`,便于切换算法。 + +### StreamingHdiffDiffer + +`StreamingHdiffDiffer` 是当前源码中的另一种 differ 实现。它使用块级 FNV-1a 哈希索引预筛候选位置,再进行字节级扩展匹配,输出 BSDIFF 兼容的补丁结构。 -**示例:** ```csharp -// 生成从 v1.0.0 到 v1.1.0 的补丁包 -var source = @"D:\MyApp\v1.0.0"; -var target = @"D:\MyApp\v1.1.0"; -var patch = @"D:\MyApp\patches\v1.1.0"; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; + +var differ = new StreamingHdiffDiffer( + compressionProvider: new DeflateCompressionProvider(optimalLevel: true), + blockSize: 64 * 1024, + maxWindowSize: 128 * 1024 * 1024); -await DifferentialCore.Instance.Clean(source, target, patch); -// 结果:patch 目录包含所有必要的增量更新文件 +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -#### Dirty 方法 +| 特性 | 说明 | +| --- | --- | +| 默认压缩 | `DeflateCompressionProvider`。 | +| 块大小 | `BlockSize` 默认 64 KB,用于建立旧文件块哈希索引。 | +| 窗口预算 | `MaxWindowSize` 默认 128 MB,影响生成补丁时参与计算的内存窗口。 | +| 应用补丁 | `DirtyAsync` 委托给 `BsdiffDiffer` 的补丁应用逻辑。 | +| 适用场景 | 需要更快候选匹配、希望与 Core `DiffPipeline` 默认算法保持一致的目录级差分构建。 | -应用补丁,将旧版本文件更新到新版本。 +需要注意的是,当前实现不是完整外存流式差分:当单个文件超过 `MaxWindowSize` 时,算法只会读取预算窗口参与计算。对超大单文件,请在业务侧验证补丁还原结果,或调大 `MaxWindowSize`,或改用 `BsdiffDiffer` 等更适合当前文件规模的实现。 -**方法签名:** +## 差分算法选择 -```csharp -public async Task Dirty(string appPath, string patchPath) -``` +当前 Differential 内置两种文件级差分算法。它们都输出 BSDIFF 兼容补丁结构,但生成补丁时的匹配方式、默认压缩、性能侧重点不同。 -**参数:** -- `appPath`: 客户端应用程序目录(当前版本) -- `patchPath`: 补丁文件路径 +| 对比项 | `BsdiffDiffer` | `StreamingHdiffDiffer` | +| --- | --- | --- | +| 核心思路 | 经典 BSDIFF 4.0,基于后缀排序寻找旧文件和新文件之间的最长匹配。 | 使用块级 FNV-1a 哈希建立旧文件索引,先用哈希快速筛选候选块,再做字节级扩展匹配。 | +| 默认压缩 | BZip2 (`0x00`)。 | Deflate (`0x01`)。 | +| 补丁应用 | 自己实现 BSDIFF Dirty 逻辑。 | `DirtyAsync` 委托给 `BsdiffDiffer`,因此应用阶段和 BSDIFF 补丁兼容。 | +| 生成效率 | 匹配更精细,局部或分散变化下补丁生成表现稳定;但后缀排序和全量读入会带来 CPU/内存开销。 | 块命中效果好时生成更快;如果变化分散、块哈希命中少,生成可能变慢。 | +| 客户端应用性能 | 默认 BZip2 解压成本更高,客户端应用大量补丁时耗时可能更明显。 | 默认 Deflate 解压更快,更适合客户端批量应用补丁。 | +| 补丁体积倾向 | 通常更追求细粒度匹配,补丁体积明显更稳定。 | 速度优先,补丁体积与文件变化分布、块大小、窗口预算强相关;块命中差时可能接近完整文件。 | +| 内存特征 | 生成阶段读取旧文件和新文件,单个大文件需要关注内存峰值。 | 通过 `BlockSize` 和 `MaxWindowSize` 控制匹配窗口,超大单文件需要额外验证或调参。 | +| 兼容性 | 最适合需要兼容旧 BSDIFF/BZip2 补丁的场景。 | 适合新项目、目录级批量差分和 Core `DiffPipeline` 默认构建。 | -**功能说明:** -1. 读取 patchPath 中的补丁文件 -2. 将补丁应用到 appPath 中的对应文件 -3. 处理新增文件的复制 -4. 处理删除文件的移除 +可以简单理解为:`BsdiffDiffer` 更偏“兼容和补丁体积稳定”,`StreamingHdiffDiffer` 更偏“客户端应用速度和可调参数”。如果项目非常在意补丁体积或文件变化较分散,优先考虑 `BsdiffDiffer`;如果项目更在意客户端应用速度,并且经过压测确认补丁体积可接受,可以考虑 `StreamingHdiffDiffer`。 -**示例:** -```csharp -// 将补丁应用到当前应用程序 -var appDir = AppDomain.CurrentDomain.BaseDirectory; -var patchDir = Path.Combine(appDir, "temp", "patches"); +### 参考基准数据 {#benchmark-reference} -await DifferentialCore.Instance.Dirty(appDir, patchDir); -// 结果:应用程序更新到新版本 -``` +下面数据来自当前源码的一组本地微基准,用于给开发者判断量级,不是跨所有项目的性能承诺。测试环境为 Windows x64、.NET Release 构建,使用 2-4 MB 合成文件;真实结果会受 CPU、磁盘、文件类型、变化比例、压缩级别和并行度影响。 ---- +| 场景 | `BsdiffDiffer` 生成 | `StreamingHdiffDiffer` 生成 | `BsdiffDiffer` 应用 | `StreamingHdiffDiffer` 应用 | `BsdiffDiffer` 补丁体积 | `StreamingHdiffDiffer` 补丁体积 | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| 2 MB 文本,少量行变更/插入 | 484 ms | 2059 ms | 55 ms | 36 ms | 0.05% | 3.50% | +| 4 MB 二进制,连续局部块变更 | 1030 ms | 318 ms | 55 ms | 11 ms | 2.58% | 100.04% | +| 4 MB 二进制,随机分散字节变更 | 757 ms | 4176 ms | 70 ms | 30 ms | 2.18% | 100.27% | -## 实际使用示例 +从这组数据可以得到几个实用预估: -### 示例 1:基本补丁生成 +| 指标 | 参考结论 | +| --- | --- | +| 补丁体积 | `BsdiffDiffer` 在测试场景中约为新文件的 0.05%-2.58%;`StreamingHdiffDiffer` 约为 3.50%-100%。如果补丁包大小是第一优先级,优先测试 `BsdiffDiffer`。 | +| 客户端应用速度 | `StreamingHdiffDiffer` 默认 Deflate,在测试中应用补丁约快 1.5-5 倍。大量文件批量应用时,这个差距会更明显。 | +| 生成速度 | 没有绝对赢家:连续局部二进制变更中 `StreamingHdiffDiffer` 约快 3.2 倍;文本和随机分散变更中 `BsdiffDiffer` 约快 4.3-5.5 倍。 | +| 大型项目选择 | 大型项目建议同时看“补丁总体积 + 构建耗时 + 客户端应用耗时”。如果大量文件可以并行,`WithParallelism(...)` 往往比单个 differ 的微小差距更影响总体耗时。 | -```csharp -using GeneralUpdate.Differential; +> 这组数据的重点是帮助判断方向:`BsdiffDiffer` 通常更容易得到小补丁,`StreamingHdiffDiffer` 的应用阶段更快,但补丁体积和生成速度对文件变化形态非常敏感。正式发布前建议用自己项目的真实产物做一次压测。 + +推荐选择: + +| 场景 | 建议 | +| --- | --- | +| 只需要低层单文件补丁,并希望最大兼容 | 使用 `new BsdiffDiffer()`。 | +| 通过 Core `DiffPipeline` 批量生成目录级补丁 | 先用默认配置跑基准;若补丁体积偏大,可显式切换到 `BsdiffDiffer`;再结合 `WithParallelism(...)` 提升吞吐。 | +| 客户端解压性能更敏感 | 优先选择 Deflate 补丁,即 `StreamingHdiffDiffer` 默认配置,或 `new BsdiffDiffer(new DeflateCompressionProvider())`。 | +| 历史补丁仍是旧 BSDIFF/BZip2 | 使用 `BsdiffDiffer` 应用;32 字节头会按 BZip2 处理。 | +| 大型项目包含大量 DLL、资源文件、插件文件 | 使用 Core `DiffPipeline` 做文件级并行,避免自己逐个文件串行调用 Differential。 | + +## 补丁格式与压缩 Provider {#补丁格式与压缩-provider} -public async Task GeneratePatchAsync() +Differential 生成的是 BSDIFF 风格补丁。当前实现写入 33 字节扩展头: + +| 偏移 | 长度 | 含义 | +| --- | --- | --- | +| `0` | 8 | 魔数 `"BSDIFF40"`。 | +| `8` | 8 | 压缩后控制段长度。 | +| `16` | 8 | 压缩后差异段长度。 | +| `24` | 8 | 新文件长度。 | +| `32` | 1 | 压缩格式版本。 | + +应用补丁时也兼容 32 字节旧头:如果没有第 33 个格式字节,就按 BZip2 旧补丁处理。 + +### ICompressionProvider + +`ICompressionProvider` 负责把控制段、差异段和额外段包装成压缩流。 + +```csharp +public interface ICompressionProvider { - try - { - // 版本路径 - var oldVersion = @"D:\MyApp\1.0.0"; - var newVersion = @"D:\MyApp\1.0.1"; - var patchOutput = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("开始生成补丁..."); - - // 生成补丁 - await DifferentialCore.Instance.Clean(oldVersion, newVersion, patchOutput); - - Console.WriteLine($"补丁生成完成!输出目录:{patchOutput}"); - - // 显示补丁信息 - var patchFiles = Directory.GetFiles(patchOutput, "*.*", SearchOption.AllDirectories); - Console.WriteLine($"生成了 {patchFiles.Length} 个补丁文件"); - - long totalSize = patchFiles.Sum(f => new FileInfo(f).Length); - Console.WriteLine($"总补丁大小:{totalSize / 1024.0:F2} KB"); - } - catch (Exception ex) - { - Console.WriteLine($"补丁生成失败:{ex.Message}"); - } + byte FormatVersion { get; } + + Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default); + + Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default); } ``` -### 示例 2:应用补丁 +| Provider | 格式字节 | 当前可用性 | 说明 | +| --- | --- | --- | --- | +| `BZip2CompressionProvider` | `0x00` | 可用 | `BsdiffDiffer` 默认值,兼容旧 BSDIFF 补丁。 | +| `DeflateCompressionProvider` | `0x01` | 可用 | BCL `DeflateStream`,解压速度更适合客户端更新。 | +| `BrotliCompressionProvider` | `0x02` | 源码中以 `NET6_0_OR_GREATER` 条件编译预留 | 当前 `GeneralUpdate.Differential` 项目目标为 `netstandard2.0`,并且补丁读取逻辑当前只识别 `0x00` / `0x01`,不要把 Brotli 作为生产更新包格式。 | + +自定义压缩时,生成补丁和应用补丁必须使用能被补丁读取逻辑识别的格式字节。当前生产建议只使用 BZip2 或 Deflate。 ```csharp -using GeneralUpdate.Differential; +using GeneralUpdate.Differential.Abstractions; +using GeneralUpdate.Differential.Differ; -public async Task ApplyPatchAsync() -{ - try - { - // 应用程序目录 - var appDirectory = @"D:\MyApp\current"; - // 补丁目录 - var patchDirectory = @"D:\MyApp\patches\1.0.1"; - - Console.WriteLine("开始应用补丁..."); - - // 验证补丁存在 - if (!Directory.Exists(patchDirectory)) - { - throw new DirectoryNotFoundException($"补丁目录不存在:{patchDirectory}"); - } - - // 应用补丁 - await DifferentialCore.Instance.Dirty(appDirectory, patchDirectory); - - Console.WriteLine("补丁应用成功!应用程序已更新到新版本。"); - } - catch (Exception ex) - { - Console.WriteLine($"补丁应用失败:{ex.Message}"); - } -} +var differ = new BsdiffDiffer( + new DeflateCompressionProvider(optimalLevel: false)); + +await differ.CleanAsync(oldFile, newFile, patchFile); +await differ.DirtyAsync(oldFile, outputFile, patchFile); ``` -### 示例 3:完整的补丁流程 +## 与 GeneralUpdate.Core 的关系 {#与-generalupdatecore-的关系} -```csharp -using GeneralUpdate.Differential; -using System.IO.Compression; +`GeneralUpdate.Core` 在 Differential 之上提供目录级差分管道 `DiffPipeline`。它会负责: -public class PatchManager -{ - // 生成并打包补丁 - public async Task CreatePatchPackageAsync( - string oldVersionPath, - string newVersionPath, - string outputPath) - { - try - { - // 1. 生成补丁文件 - var tempPatchDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempPatchDir); - - Console.WriteLine($"正在生成补丁..."); - await DifferentialCore.Instance.Clean(oldVersionPath, newVersionPath, tempPatchDir); - - // 2. 压缩补丁文件 - var patchZipPath = Path.Combine(outputPath, "patch_1.0.1.zip"); - Console.WriteLine($"正在打包补丁..."); - - if (File.Exists(patchZipPath)) - File.Delete(patchZipPath); - - ZipFile.CreateFromDirectory(tempPatchDir, patchZipPath, - CompressionLevel.Optimal, false); - - // 3. 清理临时文件 - Directory.Delete(tempPatchDir, true); - - var patchSize = new FileInfo(patchZipPath).Length; - Console.WriteLine($"补丁包创建成功:{patchZipPath}"); - Console.WriteLine($"补丁包大小:{patchSize / 1024.0:F2} KB"); - - return patchZipPath; - } - catch (Exception ex) - { - Console.WriteLine($"创建补丁包失败:{ex.Message}"); - throw; - } - } - - // 解压并应用补丁 - public async Task ApplyPatchPackageAsync(string appPath, string patchZipPath) - { - try - { - // 1. 解压补丁包 - var tempExtractDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()); - Directory.CreateDirectory(tempExtractDir); - - Console.WriteLine($"正在解压补丁包..."); - ZipFile.ExtractToDirectory(patchZipPath, tempExtractDir); - - // 2. 应用补丁 - Console.WriteLine($"正在应用补丁..."); - await DifferentialCore.Instance.Dirty(appPath, tempExtractDir); - - // 3. 清理临时文件 - Directory.Delete(tempExtractDir, true); - - Console.WriteLine($"补丁应用成功!"); - } - catch (Exception ex) - { - Console.WriteLine($"应用补丁包失败:{ex.Message}"); - throw; - } - } -} +1. 对比旧目录和新目录。 +2. 找出发生变化的文件并调用 `IBinaryDiffer.CleanAsync` 生成 `.patch`。 +3. 复制新增文件到补丁目录。 +4. 生成 `generalupdate.delete.json` 记录删除文件。 +5. 客户端应用补丁时并行调用 `IBinaryDiffer.DirtyAsync`,先写临时文件,成功后替换原文件。 + +如果你在应用更新流程中使用 `GeneralUpdate.Core`,Core 默认已经集成 Differential 并内置差分管道。也就是说,常规更新接入时不需要额外安装、初始化或手动调用 `GeneralUpdate.Differential`;只要使用 Core 的更新流程,并按业务需要启用补丁更新能力,Core 会在内部完成 differ 创建、补丁应用和目录级编排。 + +只有在你想替换默认差分算法、调整并行度、改变错误策略或接入自定义 matcher 时,才需要通过 `UseDiffPipeline` 做高级配置: -// 使用示例 -var manager = new PatchManager(); - -// 创建补丁包 -var patchZip = await manager.CreatePatchPackageAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\releases" -); - -// 应用补丁包 -await manager.ApplyPatchPackageAsync( - @"D:\MyApp\current", - patchZip -); +```csharp +using GeneralUpdate.Core; +using GeneralUpdate.Core.Models; +using GeneralUpdate.Differential.Differ; + +await new GeneralUpdateBootstrap() + .SetSource( + updateUrl: "https://update.example.com/api/upgrade/verification", + appSecretKey: "your-app-secret") + .SetOption(Option.AppType, AppType.Client) + .SetOption(Option.PatchEnabled, true) + .UseDiffPipeline(builder => builder + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .WithStopOnFirstError(true)) + .LaunchAsync(); ``` -### 示例 4:带进度显示的补丁操作 +当前源码里有两个默认层级需要区分: + +| 使用方式 | 默认 differ | +| --- | --- | +| 直接 `new DiffPipeline()` 或 `new DiffPipelineBuilder().Build()` | `StreamingHdiffDiffer` | +| `GeneralUpdateBootstrap` 未显式调用 `UseDiffPipeline(...)` 时内部构建 | `BsdiffDiffer`,并行度 2,带 `DiffProgressReporter` | + +因此,普通用户可以把 Differential 看作 Core 已经带好的底层能力,不需要特地集成;只有希望 Core 更新流程明确使用某个算法或自定义差分行为时,才建议显式调用 `UseDiffPipeline(...)`。 + +## 与 GeneralUpdate.Tools 的关系 {#与-generalupdatetools-的关系} + +`GeneralUpdate.Tools` 面向发布侧,帮助开发者构建更新产物。当前 `DiffService` 会创建 `new DiffPipeline()`,再调用: ```csharp -using GeneralUpdate.Differential; +await pipeline.CleanAsync(oldDir, newDir, patchDir); +``` -public class ProgressivePatchManager -{ - public async Task GeneratePatchWithProgressAsync( - string sourcePath, - string targetPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("开始扫描文件差异..."); - - // 在实际场景中,可以在 Clean 前后添加进度报告 - await DifferentialCore.Instance.Clean(sourcePath, targetPath, patchPath); - - progress?.Report("补丁生成完成!"); - - // 统计信息 - var files = Directory.GetFiles(patchPath, "*.*", SearchOption.AllDirectories); - progress?.Report($"共生成 {files.Length} 个补丁文件"); - } - catch (Exception ex) - { - progress?.Report($"错误:{ex.Message}"); - throw; - } - } - - public async Task ApplyPatchWithProgressAsync( - string appPath, - string patchPath, - IProgress progress) - { - try - { - progress?.Report("开始应用补丁..."); - - await DifferentialCore.Instance.Dirty(appPath, patchPath); - - progress?.Report("补丁应用成功!"); - } - catch (Exception ex) - { - progress?.Report($"错误:{ex.Message}"); - throw; - } - } -} +也就是说,Tools 生成目录级差分包时,本质上使用的是 Core 的 `DiffPipeline`,而 `DiffPipeline` 再调用 Differential 的 `IBinaryDiffer` 生成每个变更文件的补丁。对大多数开发者来说,推荐路径是: + +1. 用 Tools 对比旧版本目录和新版本目录,生成补丁目录和清单产物。 +2. 用 Core 在客户端检查版本、下载补丁包、应用补丁。 +3. 只有在需要自定义差分算法、压缩格式或单文件补丁实验时,才直接使用 Differential。 + +这种分层可以让业务代码保持简单:Tools 负责构建,Core 负责更新,Differential 负责底层文件差分。 + +## 并发模型与性能建议 -// 使用示例 -var manager = new ProgressivePatchManager(); -var progress = new Progress(msg => Console.WriteLine($"[{DateTime.Now:HH:mm:ss}] {msg}")); +Differential 的单个 differ 实例没有保存某次补丁任务的可变共享状态。只要传入的 `ICompressionProvider` 是线程安全的,内置 differ 可以被 Core 管道并发调用;内置 BZip2、Deflate provider 都会为每次调用创建新的压缩流,适合并发使用。 -await manager.GeneratePatchWithProgressAsync( - @"D:\MyApp\1.0.0", - @"D:\MyApp\1.0.1", - @"D:\MyApp\patches\1.0.1", - progress -); +真正的“多线程差分”通常发生在 Core `DiffPipeline` 层: + +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new StreamingHdiffDiffer()) + .WithParallelism(4) + .Build(); + +await pipeline.CleanAsync(oldDir, newDir, patchDir); ``` ---- -## 注意事项与警告 +### 大型项目并行差分 {#大型项目并行差分} + +大型桌面项目通常不是“一个超大文件”,而是由主程序、多个 DLL、插件、资源文件、运行时文件和配置文件组成。Core `DiffPipeline` 会把目录对比结果拆成文件级任务,每个变更文件独立调用 `IBinaryDiffer.CleanAsync` 生成补丁,因此可以通过 `WithParallelism(...)` 同时处理多个文件。 + +这种并行模型对大型项目很重要: -### ⚠️ 重要提示 +1. 构建侧可以同时为多个变更文件生成 `.patch`,缩短发布包构建时间。 +2. 客户端应用补丁时也可以并行处理多个文件,减少升级窗口。 +3. 新增文件复制、删除清单处理和差分补丁生成由 Core 管道统一编排,开发者不需要手写多线程调度。 +4. 并行度可以按机器能力调整,构建机可以设置更高,低配置客户端可以保持较低。 -1. **文件名限制** - - 不能包含同名但扩展名不同的文件(如 file.txt 和 file.log) - - 建议使用唯一的文件名命名规则 +| 参数/策略 | 建议 | +| --- | --- | +| `WithParallelism(1)` | 资源敏感、机械硬盘、低内存环境。 | +| `WithParallelism(2)` | 默认平衡值,适合多数桌面应用。 | +| `WithParallelism(4-8)` | 多核 CPU、SSD、构建机或发布服务器。 | +| BZip2 | 补丁兼容性好,但客户端解压成本更高。 | +| Deflate | 解压速度更友好,适合客户端大批量应用补丁。 | +| 大文件 | 先压测补丁生成耗时、内存峰值和还原结果,不要只看补丁体积。 | -2. **目录结构** - - 源目录和目标目录的相对结构应保持一致 - - 补丁生成时会保留目录层次关系 +并行差分适合“文件数量多、每个文件可独立处理”的大型项目。需要注意的是,单个超大文件内部仍由具体 differ 算法处理,不会因为 `WithParallelism(8)` 就把一个文件拆成 8 份并行计算;并行度提升的是多个文件之间的吞吐。 -3. **磁盘空间** - - 确保有足够的磁盘空间存储补丁文件 - - 二进制差异补丁通常比完整文件小,但仍需要临时空间 +下载与差分可以在上层更新流程中并行:Core 下载阶段可以并发拉取多个资源,差分应用阶段也可以按文件并行处理补丁。Differential 只负责单个文件的补丁计算,不直接管理网络下载线程。 -4. **文件占用** - - 应用补丁时,确保目标文件没有被其他进程占用 - - 建议在应用程序关闭后应用补丁 +## 扩展点 -5. **备份建议** - - 在应用补丁前建议备份原始文件 - - 可以使用 Core 的 BackUp 选项自动备份 +### 自定义差分算法 -### 💡 最佳实践 +实现 `IBinaryDiffer` 后即可接入 Core 管道。适合接入其他算法、调用原生库,或对特定文件类型做特殊优化。 -- **版本管理**:为每个版本维护独立的补丁包,便于版本追踪和回滚 -- **补丁验证**:生成补丁后进行验证测试,确保补丁可以正确应用 -- **增量更新**:优先使用差异更新而非全量更新,可节省 50%-90% 的下载量 -- **错误处理**:实现完整的异常捕获和错误恢复机制 -- **性能优化**:对于大文件,差异算法的性能优势更加明显 +```csharp +using GeneralUpdate.Differential.Abstractions; -### 🔍 工作原理 +public sealed class MyBinaryDiffer : IBinaryDiffer +{ + public Task CleanAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Generate patchFilePath from oldFilePath and newFilePath. + throw new NotImplementedException(); + } -**Clean 方法工作流程:** -1. 扫描源目录和目标目录中的所有文件 -2. 比较文件的MD5哈希值以识别变化 -3. 对于修改的文件,使用二进制差异算法生成补丁 -4. 对于新增文件,直接复制到补丁目录 -5. 记录删除的文件列表 + public Task DirtyAsync( + string oldFilePath, + string newFilePath, + string patchFilePath, + CancellationToken cancellationToken = default) + { + // Restore newFilePath from oldFilePath and patchFilePath. + throw new NotImplementedException(); + } +} +``` -**Dirty 方法工作流程:** -1. 读取补丁目录中的所有文件 -2. 对于补丁文件,应用到对应的原文件上 -3. 对于新增文件,直接复制到应用目录 -4. 根据删除列表移除相应文件 -5. 验证更新完整性 +```csharp +var pipeline = new DiffPipelineBuilder() + .UseDiffer(new MyBinaryDiffer()) + .WithParallelism(4) + .Build(); +``` ---- +自定义算法需要保证 `CleanAsync` 产出的补丁能被同一算法的 `DirtyAsync` 正确应用;如果补丁要交给 Core 客户端使用,发布侧和客户端必须使用同一套 differ 实现。 -## 适用平台 +### 自定义压缩 Provider -| 产品 | 版本 | -| ------------------ | ----------------- | -| .NET | 5, 6, 7, 8, 9, 10 | -| .NET Framework | 4.6.1 | -| .NET Standard | 2.0 | -| .NET Core | 2.0 | +如果仍使用 BSDIFF 兼容补丁结构,只想替换控制段、差异段和额外段的压缩方式,可以实现 `ICompressionProvider`。 ---- +```csharp +using GeneralUpdate.Differential.Abstractions; + +public sealed class MyCompressionProvider : ICompressionProvider +{ + public byte FormatVersion => 0x01; + + public Stream CreateCompressStream( + Stream output, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + output, + CompressionLevel.Optimal, + leaveOpen: true); + } + + public Stream CreateDecompressStream( + Stream input, + CancellationToken cancellationToken = default) + { + return new DeflateStream( + input, + CompressionMode.Decompress, + leaveOpen: true); + } +} +``` + +不要随意分配新的 `FormatVersion`。当前 `BsdiffDiffer.DirtyAsync` 只识别 BZip2 (`0x00`) 和 Deflate (`0x01`);如果你引入新格式,也需要同步扩展补丁读取逻辑,否则客户端无法应用补丁。 + +## 实战建议 -## 相关资源 +| 场景 | 推荐做法 | +| --- | --- | +| 普通应用发布差分更新 | 使用 `GeneralUpdate.Tools` 生成产物,客户端使用 Core。 | +| 需要控制目录级并行、错误策略和进度 | 使用 Core `DiffPipelineBuilder`。 | +| 只验证某个文件的补丁效果 | 直接使用 `IBinaryDiffer`。 | +| 对补丁体积和应用速度都敏感 | 对同一组文件分别测试 BZip2、Deflate 和不同算法后再定默认策略。 | +| 更新包需要长期兼容旧客户端 | 保守使用 `BsdiffDiffer` + BZip2,或确保客户端已支持 Deflate 扩展头。 | -- **示例代码**:[查看 GitHub 示例](https://github.com/GeneralLibrary/GeneralUpdate-Samples/tree/main/src/Diff) -- **主仓库**:[GeneralUpdate 项目](https://github.com/GeneralLibrary/GeneralUpdate) -- **打包工具**:GeneralUpdate.PacketTool 项目依赖此组件实现差异打包 +Differential 的价值在于把复杂的二进制差分能力收敛成稳定的文件级抽象。上层开发者可以把重点放在“什么时候更新、下载什么、如何提示用户”上,把具体补丁生成和应用交给 Core/Tools/Differential 的组合完成。