Borg 1.1

Date:
7th October 2017

This marks the first stable release in the Borg 1.1 series. Many improvements and new features were incorporated into Borg 1.1 since the 1.0 release of Borg in early 2016.

More than sixty people contributed code to this release — see Contributors on this page — and 2016-2017 has been a fantastic time for the Borg project.

Since Borg 1.1 is now stable, it will only receive fixes and minor additions, no larger new features. Principal development continues in the Borg 1.2 series.

Changelog summary

This is only a summary of the changes between 1.0 and 1.1. Check the full changelog to see all changes.

Major new features in the 1.1 release series are:

  • borg recreate: remove files from existing archives and re-chunk or re-compress them. This finally allows deduplication between legacy Attic / Borg 0.xx archives and Borg 1.x archives. This is an experimental feature.

  • borg diff: show differences between two archives.

  • borg mount: show all recorded versions of files with the "versions view".

  • borg list: more formatting options, including generation of hashes and per-file chunk statistics.

  • borg create: automatic compression mode (-C auto,zlib/lz4/...), faster handling of many files, experimental support for inclusion patterns (--patterns, --patterns-from), files cache mode control (mtime/ctime)

  • borg export-tar: streaming export of GNU tar compatible archives.

  • Archive comments

  • BLAKE2b256-based encryption modes, and "authenticated" modes that provide data integrity without encryption. These provide better performance than the existing SHA-256-based modes on most hardware, except where the x86 SHA extensions are available (e.g. AMD Ryzen).

  • Repository indices and the Borg cache now use checksums to detect bitrot and other corruption.

  • A documented JSON API has been added to the most vital commands (borg list, borg info and borg create).

  • Structured JSON logging is available for all commands.

  • --prefix has been supplemented with --glob-archives (-a), --sort-by, --last and --first options. These can be used for borg list, borg info, borg mount, borg check, borg delete and borg prune.

Quality of life improvements:

  • options that imply output (--show-rc, --show-version, --list, --stats, --progress) don't need -v/--info to have that output displayed any more.

  • borg check is silent by default (finally!).

  • borg can now checkpoint within (big) files, not only between files.

  • borg delete can delete multiple archives by giving their names.

  • Automatic removal of stale locks, which should make "borg break-lock" essentially superfluous. This is enabled by default, see BORG_HOSTNAME_IS_UNIQUE.

  • Answers to prompts like "Accessing previously unknown repository" are now saved immediately.

  • Cache synchronization and "borg info" are now faster.

  • Reduced space usage of chunks.archive.d in the cache by 30-40 %. Existing caches are migrated during a cache sync.

  • The cache used for remote cache syncs and mounting remote repositories does not grow indefinitely any more, but adapts to the available space. Good riddance, TMP=/var/tmp!

  • BORG_PASSCOMMAND makes using key rings and hardware keys much easier.

Documentation improvements:

  • The online command reference has been significantly revamped: better presentation of options, commands are now separate pages (improves search, and old links still work), many formatting improvements, revised examples, a new theme, ...

  • New deployment guides

  • The internals docs saw many corrections and extensions.

  • man pages are now provided in source releases.

  • --help uses ANSI sequence on terminals for basic markup

  • New asciinema screencasts.

Compatibility notes for upgrading from Borg 1.0 to Borg 1.1:

  • No explicit "borg upgrade" is required.

  • Borg 1.1 uses some new data structures which are backwards-compatible with Borg 1.0.4 and newer.

  • Borg 1.0 can't make use of Borg 1.1's "compact cache" and will silently ignore it.

  • The default compression has been changed from "none" to "lz4".

  • Repositories in the "repokey" and "repokey-blake2" modes with an empty passphrase are now treated as unencrypted repositories for security checks (e.g. BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK).

  • borg init:

    • -e/--encryption is now a mandatory option with no default value. Previously the default value was "repokey".

    • the short form of --append-only (-a) has been removed.

    • running "borg init" via a "borg serve --append-only" server will not create an append-only repository any more. Use "borg init --append-only" to initialize an append-only repository.

  • borg create:

    • A new option --files-cache MODE (default MODE is: ctime,size,inode) has been introduced and also the default cache MODE has changed from "mtime,size,inode" in 1.0.x to "ctime,size,inode" in 1.1.x.

    • Be prepared that the first "borg create" after you upgrade to 1.1.x will run significantly slower than usual. This is due to a complete cache rebuild, caused by the changed cache mode. Passing explicitly --files-cache with the old default: "mtime,size,inode" would avoid the slowdown, but is not advisable as "ctime" is better as it reflects both file contents and metadata changes, while "mtime" reflects only file contents changes.

    • the --exclude-if-present option now supports tagging a folder with any file system object type (file, folder, etc.), instead of accepting only files as tags.

  • borg upgrade: the short form of --inplace (-i) has been removed.

  • borg delete: the short form of --cache-only (-c) has been removed.

  • "borg migrate-to-repokey" has been renamed to "borg key migrate-to-repokey" with no deprecation notice, since it is rarely and only manually used.

Deprecated options and commands in Borg 1.1, which will be removed in Borg 1.2:

  • "borg change-passphrase" is deprecated, use "borg key change-passphrase" instead. "borg change-passphrase" will be removed in Borg 1.2.

  • borg create: the --keep-tag-files option has been deprecated in favour of the new --keep-exclude-tags option. Both options have the same effect. --keep-tag-files will be removed in Borg 1.2.

  • borg create: the --ignore-inode option has been deprecated in favour of --files-cache=... modes without "inode". --ignore-inode will be removed in Borg 1.2.

  • borg create: the --no-files-cache option has been deprecated in favour of --files-cache=disabled. --no-files-cache will be removed in Borg 1.2.

Contributors

We'd like to thank everyone who contributed to Borg, be it code, bug reports, testing, documentation, creating packages or spreading the word.

Contributors to the main repository for this release were:

Abdel-Rahman A. · Aleksander Charatonik · Alexander 'Leo' Bergolth · Alexander Meshcheryakov · Alexander-N · Alexander Pyhalov · anarcat · Andrea Gelmini · Andrew Engelbrecht · Andrew Skalski · Antonio Larrosa · Ben Creasy · Benedikt Heine · Benedikt Neuffer · Benjamin Pereto · Björn Ketelaars · Carlo Teubner · Chris Lamb · Dan Christensen · Dan Helfman · Daniel Reichelt · Danny Edel · Ed Blackman · edgimar · Félix Sipma · Florent Hemmi · Florian Klink · Frank Sachsenheim · Fredrik Mikker · Hans-Peter Jansen · Hartmut Goebel · infectormp · James Clarke · Janne K · Jens Rantil · Joachim Breitner · Johann Bauer · Johannes Wienke · Jonathan Zacsh · Julian Andres Klode · klemens · kmq · Lauri Niskanen · Lee Bousfield · Leo Antunes · lfam · Lukas Fleischer · Marian Beermann · Mark Edgington · Markus Engelbrecht · Martin Hostettler · Michael Gajda · Michael Herold · Milkey Mouse · Mitch Bigelow · Narendra Vardi · Nathan Musoke · Oleg Drokin · ololoru · Pankaj Garg · Patrick Goering · philippje · Radu Ciorba · Robert Marcano · Ronny Pfannschmidt · rugk · schuft69 · sherbang · Simon Frei · Simon Heath · Stefano Probst · Stefan Tatschner · Steve Groesz · sven · textshell · Thomas Kluyver · Thomas Waldmann · TuXicc · Wladimir Palant · wormingdead · Zhuoyun Wei

Special thanks also go to everyone and every organization donating funds to support development and maintainers:

storcium · IT Service Group of the Department of Computer Science, ETH Zürch · TheVillux · Daniel Parks · Dave Barker · Roland Moriz · alraban · level323 · Magnus Månsson · Bluebeep · William Weiskopf · kleptos · lf · rmiceli · Kirrus · DrTyrell · Thomas Waldmann · stevesbrain · martin · neutrinus · Jeremy Audet (=lchimonji10) · DrupaListo · mario · Jason Harris · iamnumbersix · (unknown) · kwaa · Michael Gajda · Twilek · lazlor · Christopher Lijlenstolpe · Marian Beermann · twink0r · Andreas Schamanek · Abdel-Rahman A. · multikatt · kiz · jgtimm · infectormp · Paolo Dina · Aravindh · Quallenauge · reyman