Release 2.0.0b22

[ThomasWaldmann]

borgbackup 2.0.0b22 beta release - do not use this on production backup repositories.

Big changes, this is the first release with PACKS! So, despite the "beta" release number, rather expect alpha quality / stability.

Read the full story about packs there: https://mrraj.dev/borg

Long changelog:
https://borgbackup.readthedocs.io/en/2.0.0b22/changes.html#version-2-0-0b22-2026-07-22

If you use pip to install this, use: pip install "borgbackup==2.0.0b22"

Docs: https://borgbackup.readthedocs.io/en/2.0.0b22/

---

This discussion was created from the release Release 2.0.0b22.

[PhrozenByte]

Just wanted to leave some ❤️ here: b22 is an amazing release! It's a huge step forward and resolves a whole bunch of major, long-standing issues. Huge thanks to everyone who made it happen! 🎉

Special thanks to @ThomasWaldmann for his countless improvements and fixes, @mr-raj12 for implementing packs, all the major work around them and various other improvements, @Goddesen for interval-based pruning, and @c-herz for the date: matcher - these are the contributions I followed most closely and that have the biggest impact on my own use cases, but the same thanks goes to everyone else whose small and big contributions I may have missed. You all did an incredible job, thank you! 🫶

[callegar]

Out of curiosity: can packs be expected to lay the conditions for having a chunk size sweet spot going back to something smaller than 2MB or the overhead per chunk remains too large for that?

[ThomasWaldmann]

• the chunks index having one entry per chunk in a hashtable in RAM still means one should be thoughtful about the total amount of chunks in a repository (especially if repo is big and RAM is small)
• borg2 is doing more client-side (because there is no borg on the server-side when using e.g. cloud or sftp), so the chunks index the client loads into memory now has more information than in borg 1.x (think about the new chunks index being a combination of the old chunks index + the old repository index)
• the new borghash hash table implementation works a bit more efficient as far as the hashtable part of it is concerned: it only stores integer indexes into a kv-array, so the overhead of free hashtable entries (which a HT needs to operate quickly) is far less. the keys and values in the kv-array need a lot of space though (chunkid256 -> flags32, size32, packid256, obj_offset32, obj_size32) - so that is 80 bytes per chunk in the kv array. plus the space needed for the HT, that makes ~100 bytes per chunk.
• there are ideas on the issue tracker to reduce RAM needs (e.g. borg create does not need to know packid, obj_offset, obj_size of existing chunks - it only needs to know that they exist in the repo)

[ThomasWaldmann]

PSA for b22: use BORG_REMOTE_PATH env var instead of --remote-path cli option.

[ThomasWaldmann]

Everybody please critically test the fastcdc chunker:

borg create --chunker-params=fastcdc,19,23,21,2 ...

I am considering to make this the default chunker for borg2 because it is faster than buzhash64 and buzhash and also implements normalized chunking. Should get serious beta-testing before that can be done, though.

#9957

[PhrozenByte]

Interesting… I always assumed that changing --chunker-params would effectively "reset" deduplication across existing archives, i.e. that new archives would mostly have to store their data again because the old chunks could no longer be reused.

I was therefore quite surprised how little impact switching from the default buzhash,19,23,21,4095 to fastcdc,19,23,21,2 had. For the first backup using the new chunker I got:

Original size: 415.54 GB
Deduplicated size: 2.08 GB

Because of that assumption, I was initially a bit hesitant to test it since I have borg2 writing to a spare drive with limited free space. Fortunately, that turned out to be a non-issue. At least in my case, other beta testers with a similar setup probably don't need to worry about running out of the space they reserved for borg2 testing either.

Do you have any suggestions for verifying that everything is working correctly with fastcdc? At the moment I'm thinking about simply comparing the contents of random borg2 archives created with fastcdc against the matching production borg1 archives (they backup the same filesystem snapshots), but I'm not sure whether there's a better way to validate the new chunker and borg2.

[ThomasWaldmann]

The fastcdc chunker for sure cuts chunks differently than buzhash, but:

• small files < 512kiB do not get chunked at all (result in 1 whole-file "chunk")
• due to files cache, it does not re-read (re-chunk, re-hash) unmodified files, but just reuses the cached chunks list

Guess a good test would be a fresh repo only used with fastcdc, making sha256sums before backup and either after extract, mount or using borg list --format so it computes the sha256 there. Ignore small files, only > 512kiB is interesting.

Also keep an eye on deduplication of subsequent backups.