Synchronization and backup programs - ArchWiki

# Synchronization and backup programs

From ArchWiki

Related articles

- System backup
- Disk cloning
- List of applications#File sharing
- System maintenance#Backup
- Dotfiles#Version control
- File recovery

This page lists and compares applications that synchronize data between two or more locations, and those that build on top of such functionality to make incremental copies of important data for backup purposes. Because of their relationship, the two groups share several traits that justify describing them in the same article.

## Backup overview

Having backups of important data is a necessary measure to take, since human and machine processing errors are very likely to generate corruption as time passes, and also the physical media where the data is stored is inevitably destined to fail. In order to choose the best program for one's own needs, the following aspects should be considered:

- The type of backup medium that is going to store the data, e.g. CD, DVD, remote server, external hard drive, etc.
- The planned frequency of backups, e.g. daily, weekly, monthly, etc.
- The features expected from the backup solution, e.g. compression, encryption, handles renames, etc.
- The planned method to restore backups if needed.

## Data synchronization

These applications simply keep directories synchronized between multiple locations/machines, in a "mirror" fashion. Nonetheless, most of them still allow storing and reverting to old revisions of modified or deleted files.

See also Wikipedia:Comparison of file synchronization software.

Legend:

- Name: the application name, linking to the ArchWiki article or the official website.
- Installation: a link to the package.
- Implementation: the programming language, library, or utility that the application is based on.
- Delta transfer: only the modified parts of files are transferred.
- Encrypted transfer: data is encrypted by default when transferred over the network.
- FS metadata: file system permissions and attributes are synchronized.
- Resumable: the synchronization can be resumed if interrupted.
- Handles renames: moved/renamed files are detected and not stored or transferred twice. It typically means that a checksum of files or its chunks is computed.
- Version control: the old version of files are backed up (reverse incremental backup).
- Change propagation: specifies in how many directions changes can be propagated.
  - unidirectional means one-way synchronization of two locations,
  - bidirectional means two-way synchronization of two locations and
  - multidirectional means full synchronization of more than two locations.
- Conflict resolution: the application handles file conflicts, either automatically or interactively, i.e. it does not silently discard conflicting files. This attribute does not apply to applications that only propagate changes in one direction.
- FS monitoring: the application listens to file system events to trigger the synchronization.
- CLI: the application is command-line driven, i.e. it is scriptable.
- Other interfaces: the application has the specified user interfaces, e.g. GUI, TUI, or web-based.
- License: the license of the server and client applications.
- Other platforms: supported operating systems other than Linux.
- Specificity: brief notes about special features that notably set the application apart from the others.

Name | Installation | Implementation | Delta transfer | Encrypted transfer | FS metadata | Resumable | Handles renames | Version control | Change propagation | Conflict resolution | FS monitoring | CLI | Other interfaces | License | Other platforms | Maintained | Specificity
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
Resilio Sync | rslsyncAUR | ? (closed source) | Yes | Yes | ? | Yes | ? | Yes | multidirectional | ? | ? | No | Web | Proprietary freemium | FreeBSD, Windows, macOS, Android, iOS, Windows Phone, Amazon Kindle Fire | Yes | P2P sync
FreeFileSync | freefilesyncAUR | C++ | ? | SFTP [1] | ? | ? | Yes [2] | Yes [3] | unidirectional / multidirectional | Yes | ? | No | Yes | GPL | Windows, macOS | Yes |
git-annex | git-annex | Haskell, git | rsync [4] | rsync [5] | ? | ? | ? | Yes | multidirectional; with git remotes [6] | renames conflicting files [7] | ? | Yes | git-annex assistant | GPLv3 | macOS, Android | Yes | Manage files with git
osync.sh | osyncAUR | Bash, based on rsync | rsync | rsync | ? | Yes | No | Yes | bidirectional | keeps multiple versions of a file [8] | optional [9] | Yes | No | BSD |  | Yes |
rdiff-backup | rdiff-backup | Python 2, librsync | rsync | rsync | Yes | ? | No | Yes | unidirectional | No | Yes | No | GPL | Win32 | ? |
rsync | rsync | C | Yes | SSH or native protocol | Yes | Yes | No | `--link-dest` with hard links [10] `--backup` | unidirectional | No | Yes | Rsync#Front-ends | GPLv3 | Win32 | Yes | Standard tool present on all Linux distributions.
SparkleShare | sparkleshare | C#, git | Yes | ? | ? | ? | Yes | Yes | ? | ? | ? | No | Yes | GPLv3 | Windows, macOS | Yes |
Syncthing | syncthing | Go | Yes [11] | Yes [12] | partial [13] | Yes | ? | Yes [14], previous versions moved to archive folder | multidirectional | renames one file [15] | Yes with syncthing-inotify | Yes | Web, GTK | MPL v2 | BSD, Windows, macOS, Android, Kindle Paperwhite | Yes | P2P sync
Synkron | synkronAUR | C++ | ? | ? | ? | ? | ? | ? | multidirectional | ? | ? | No | Qt | GPLv2 | Windows, macOS | No |
taskd | taskdAUR | C++, Python | Yes | Yes | ? | Yes | ? | ? | multidirectional | ? | No | Yes | No | MIT | Android | Yes |
Unison | unison | OCaml | Yes | Yes | partial [16] | optional [17] | No | Yes [18] | bidirectional | interactive | No | Yes | GTK2 | GPL | FreeBSD, Windows, macOS, Android | Yes [19] |

## Incremental backups

Applications that can do incremental backups remember and take into account what data has been backed up during the last run (so-called "diffs") and eliminate the need to have duplicates of unchanged data. Restoring the data to a certain point in time would require locating the last full backup and all the incremental backups from then to the moment when it is supposed to be restored. This sort of backup is useful for those who do it very often.

See also Dotfiles#Version control.

Legend:

- Name: the application name, linking to the official website.
- Installation: a link to the main ArchWiki article, if existing, or directly to the package pages.
- Implementation: the programming language, library, or utility that the application is based on.
- Compressed storage: compression is used for storage.
- Encrypted storage: encryption is used for storage.
- Delta transfer: only the modified parts of files are transferred.
- Encrypted transfer: data is encrypted by default when transferred over a network.
- FS metadata: file system permissions and attributes are backed up.
- Easy access: the backup is stored plainly in the file system, or is mountable as such.
- Resumable: the backup can be resumed without restarting it if interrupted.
- Handles renames: moved/renamed files are detected and not stored or transferred twice; it typically means that a checksum is computed for files or chunks thereof.
- CLI: the application is command-line driven, i.e. it is scriptable.
- Other interfaces: the application has the specified user interfaces, e.g. GUI, TUI, or web-based.
- Licence: the licence of the server and client applications.
- Other platforms: supported operating systems other than Linux.
- Specificity: brief notes about special features that notably set the application apart from the others.

### Single machine

These applications are aimed at backing up data from the machine they are installed on, although the backup destination can be located on an external machine or storage media.

#### Chunk-based increments

If a file is modified, these applications store only its changed parts at the next snapshot. Compared to #File-based increments applications, these are more space-efficient, especially when large files receive small modifications; on the other hand, the archived snapshots have to be opened with the backup application that created them, since the files have to be reconstructed from the stored binary diffs.

Name | Installation | Implementation | Compressed storage | Encrypted storage | Delta transfer | Encrypted transfer | FS metadata | Easy access | Resumable | Handles renames | CLI | Other interfaces | Licence | Other platforms | Maintained | Specificity
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
Areca Backup | arecaAUR | Java | Zip, Zip64 | AES128, AES256 | Yes | Yes | Yes | No | Pausing only | No | Yes | Yes | GPLv2 | Windows | Yes |
BorgBackup | borg | Python (Attic fork) | lz4, zlib, lzma | AES256 | Yes | SSH | Yes[20] | Yes[21] | Yes[22] | Yes | Yes | third party | BSD | *BSD, macOS | Yes | Deduplication based on variable length chunks
btar | btarAUR[broken link: archived in aur-mirror] | C | Yes | Yes | Yes | Yes | ? | No | ? | ? | Yes | No | GPLv3 |  | Yes | Redundancy, indexed extraction, multicore compression, input and output serialisation, tolerance to partial archive errors.
bup | bup bup-gitAUR | C, Python, git | Yes | No | Yes | Yes | Immature | Yes[23] | pick up where you left off [24] | Yes | Yes | third party | GPLv2 | NetBSD, Windows, macOS | Yes | Same storage format as git
bups | bupsAUR | bup frontend | Yes | No | Yes | Yes | Immature | Yes | pick up where you left off [25] | Yes | Yes | GTK 3 | MIT |  | Yes |
Déjà Dup | Déjà Dup | duplicity front-end | Yes | Yes | Yes | Yes | ? | No | Yes | No | Yes | GTK+ | GPLv3 |  | Yes | Integrated into GNOME Files.
Duplicati | duplicati-latestAUR | C# | Yes | Yes | Yes | Yes | scheduled for 2.0 release | No | Pausing only | No | Yes | Yes | LGPL | Windows | Yes |
Duplicity | Duplicity | librsync | gzip | gpg | Yes | Yes | ? | No | Yes | No | Yes | Déjà Dup | GPL |  | Yes |
Duply | Duply | duplicity front-end | Yes | Yes | Yes | Yes | ? | No | Yes | No | Yes | No | GPLv2 |  | Yes |
Kup Backup System | kup | rsync, bup front-end | Yes | Yes | Yes | Yes | Immature | Yes | No | Yes | bup | Qt | GPLv2 |  | Yes |
obnam | obnamAUR | Python | Yes | GnuPG | Yes | Yes | ? | Yes | checkpoints every 100MB | ? | Yes | No | GPLv3 |  | No |
restic | resticAUR restic-gitAUR | Go | No [26] | AES-256 [27] | Yes | Yes | ? | Yes[28] | ? | Yes | Yes | ? | BSD |  | Yes |
ZBackup | zbackupAUR | C++ | LZMA, LZO | AES | Yes | Yes | ? | planned [29] | No | Kinda through tar | Yes | No | GPLv2 |  | Yes | Repository consists of immutable files.

#### File-based increments

If a file is modified, these applications store its new version entirely at the next snapshot. Compared to #Chunk-based increments applications, these are less space-efficient, especially when large files receive small modifications; on the other hand, often the archived snapshots can be opened without the need to have the backup application installed.

Specific legend:

- Hard links: whether unmodified files are stored as hard links to previous versions.

Name | Installation | Implementation | Compressed storage | Encrypted storage | Delta transfer | Encrypted transfer | FS metadata | Easy access | Resumable | Handles renames | Hard links | CLI | Other interfaces | Licence | Other platforms | Maintained | Specificity
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
Back In Time | Back In Time | Python, rsync, diff | No | No | rsync | rsync | rsync | Yes | No | No | Yes [30] | Yes | Qt | GPLv2 |  | Yes |
DAR (Disk ARchive) | darAUR | C++ | special archive format | Yes | Yes | Yes | ? | ? | ? | ? | No [31] | Yes | DarGUI | GPL | FreeBSD, NetBSD, Windows, macOS | Yes | Automatic backup using cron is possible with sarabAUR[broken link: archived in aur-mirror].
DarGUI | darguiAUR | DAR front-end | Yes | Yes | Yes | Yes | ? | ? | ? | ? | No [32] | No | GTK | GPL | Windows | ? |
hdup[dead link 2016-07-11] | hdupAUR | C | bzip, gzip, lzop | gpg | ? | SSH | ? | No | No | No | No | Yes | No | GPLv2 |  | No | Multiple backup targets.
Link-Backup | link-backupAUR | Python | No | No | ? | SSH | ? | ? | Yes | Yes | No [33] | Yes | No | MIT |  | No | It copies itself to the server.
rdup | rdupAUR | C | tar.gz | gpg, blowfish and others | ? | ? | ? | Yes | ? | No | Yes | Yes | No | GPLv3 |  | No | Set of command-line tools.
rsnapshot | rsnapshot | rsync | No | No | Yes | Yes | ? | ? | ? | ? | Yes [34] | Yes | No | GPLv2 | Win32 | Yes |
sbackup | sbackupAUR | Python | gzip, bzip2 | No | ? | SSH | ? | No | No | No | No | No | GTK | GPLv3 |  | Yes |
TimeShift | timeshiftAUR | rsync | No | No | rsync | rsync | ? | ? | ? | ? | Yes | No | GTK | GPLv3 | Designed for full-system backups to dedicated devices. | Yes |

### Network oriented

These applications have been designed to centralize the backup of several machines connected to a network, through a server-client model. In general they are more complicated to deploy, compared to #Single machine solutions.

Specific legend:

- Control direction: Pull: server logs into client. Push: client initiates backup session.
- Increment type: the strategy used to reduce used space by deduplicating data (i.e., besides compression).
  - file-based: if a file is modified, the entire new version is stored at each snapshot.
    - hard-links: whether unmodified files are stored as hard links to previous versions.
  - chunk-based: only the modified parts of files are stored at each snapshot.

Name | Installation | Implementation | Control direction | Compressed storage | Encrypted storage | Delta transfer | Encrypted transfer | FS metadata | Easy access | Resumable | Handles renames | Increment type | CLI | Other interfaces | Licence | Other platforms | Maintained | Specificity
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
BackupPC | BackupPC | Perl | Pull | Yes | No | Yes | Yes | Yes | No | Yes | ? | file-based, hard links [35] | No | Web | GPLv2 | Any (no client needed) | Yes | Identical files across backups of the same or different clients are stored only once.
Bacula | bacula* in AUR | C++ | Pull | Yes | Yes | ? | Yes | ? | ? | Yes | ? | file-based [36] | Yes | GUI, Web | AGPLv3 | Windows, macOS | Yes |
burp | burp-backupAUR | librsync | Push | Yes | Yes | Yes | Yes | ? | ? | ? | ? | chunk-based [37] | Yes | burp-ui | AGPLv3 | Windows | Yes |
SafeKeep | safekeepAUR | rdiff-backup | Pull | No | No | ? | Yes | ? | ? | ? | ? | chunk-based [38] | Yes | Yes | GPL |  | No | Integrates with LVM and databases to create consistent backups. Bandwidth throttling.
Snebu | snebuAUR[broken link: archived in aur-mirror] | C | Push or Pull | Yes | No | ? | Yes | ? | ? | ? | ? | file-based [39] | Yes | No | GPLv3 |  | ? | Supports arbitrary retention schedules.
Synbak | synbak | Multitool wrapper | ? | Yes | No | Yes | Yes | Yes | ? | ? | ? | ? | No | Web | GPLv3 |  | Yes | Unifies several backup methods.
UrBackup | urbackup* in AUR | C++ | Pull | No | No | Yes | Internet transfers only | Yes | Yes | Yes | Yes | file-based,hard-links and symlinks[40]/chunk-based CoW-Snapshots[41] | Yes (client) | GUI, Web | AGPLv3+ | Windows, macOS | Yes | Identical files across backups of the same or different clients are stored only once. Integrates with LVM, dattobd and btrfs for file system snapshots.

## Cloud storage

### Third-party services

This article or section is a candidate for merging with List of applications#File sharing.

Notes: Remove the extra bullet points: the Wikipedia comparison table, and the respective webpages and Wikipedia articles do a better job at describing them. (Discuss in Talk:Synchronization and backup programs#)

See also Wikipedia:Comparison of online backup services and List of applications#Cloud synchronization clients.

Tip: cryptomatorAUR is an open-source, multi-platform program designed to add client-side transparent encryption on cloud-shared files.

- Amazon S3 — Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable object storage. Amazon S3 is easy to use, with a simple web service interface to store and retrieve any amount of data from anywhere on the web. With Amazon S3, you pay only for the storage you actually use. There is no minimum fee and no setup cost.

  http://aws.amazon.com/s3/ || s3cmd

- CrashPlan — An online/offsite backup solution.
  - Unlimited online space for very reasonable pricing.
  - Automatic and incremental backups to multiple destinations.
  - Intuitive GUI.
  - Offers encryption and de-duplication.
  - Software is free for local use.
  - Restore prevents simultaneous backing up

  http://www.crashplan.com/ || crashplanAUR

- Dropbox — A popular file-sharing service.
  - A daemon monitors a specified directory, and uploads incremental changes to dropbox.com.
  - Changes automatically show up on your other computers.
  - Includes file sharing and a public directory.
  - You can recover deleted files.
  - Community written add-ons.
  - Free accounts have 2GB storage.

  http://www.dropbox.com || dropboxAUR nautilus-dropboxAUR

- Google Drive — A file storage and synchronization service provided by Google.
  - Provides cloud storage, file sharing and collaborative editing.
  - Multiple clients are available.

  https://drive.google.com || google-drive-ocamlfuseAUR (free), driveAUR (free), griveAUR (free), gdrivefsAUR (free), insync (non-free), gdriveAUR (free), drivesync-gitAUR (free)

- iDrive — Universal Online Backup.
  - Multiple Device Backup.
  - Online File Sync.
  - Real-Time Backup.
  - Backup and Access from Mobile Devices.
  - Remote Manage.
  - No GUI Front end for Linux, command line based. A wrapper script is available to make it easier to use.

  https://www.idrive.com/ || idevsutilAUR[broken link: archived in aur-mirror], idrive-wrapperAUR[broken link: archived in aur-mirror]

- Jungle Disk — An online backup tool that stores its data in Amazon S3 or Rackspace Cloud Files.
  - A GNOME Files extension.
  - Only paid plans available.

  http://www.jungledisk.com/ || nautilus-junglediskAUR[broken link: archived in aur-mirror]

- MEGA — Successor to the MegaUpload file-sharing service.
  - Free accounts are 50GB with paid plans available for more space.
  - Offers encryption and de-duplication.
  - Usually accessed through its web interface but other tools exist.

  https://mega.co.nz || megatoolsAUR, megasyncAUR, megafuse-gitAUR

- Nutstore — A cloud service that lets you sync and share files anywhere.
  - Multiple file folders sync.
  - Service for Chinese users.

  http://jianguoyun.com/ || nutstoreAUR

- OneDrive — Microsoft cloud service.

  https://onedrive.com || onedrive-gitAUR

- rsync.net — Cloud storage for offsite backups.
  - ZFS filesystem, accessible with any SSH/SFTP/SCP tool, running on a UNIX system.
  - Simple rsync synchronization with daily automatic ZFS snapshots.
  - Special discounted price if used with borg or attic.

  http://www.rsync.net/ || rsync/SSH, borg/atticAUR[broken link: package not found]

- SpiderOak — An online backup tool for Windows, Mac and Linux users to back up, share, sync, access and store their data.
  - Free account holds 2GB as a 60-day trial.
  - Includes file sharing and a public directory.
  - Incremental backup and sync are both supported.

  https://spideroak.com/ || spideroak-oneAUR

- Storage Made Easy — Provides unified access to numerous cloud storage services, as well as its own storage.
  - Free and paid version available.
  - Free account holds 5GB and allows access to up to three other cloud storage providers.
  - Supports local directory via fuse, as well as web access.
  - Supports many cloud storage services, such as Box, Dropbox, Google Drive, Onedrive, and others.

  http://storagemadeeasy.com/ || smestorageAUR[broken link: archived in aur-mirror]

- Tarsnap — A secure online backup service for Linux, BSD, macOS and Windows (through Cygwin).
  - Compressed encrypted backups to Amazon S3 Servers.
  - Automate via cron.
  - Incremental backups.
  - Backup any files or directories.
  - Command line only client.
  - Pay only for usage (bandwidth and storage).

  http://www.tarsnap.com || tarsnap

- Yandex Disk — Free cloud storage service created by Yandex.ru that gives you access to your photos, videos and documents from any internet-enabled device.

  https://disk.yandex.ru/ || yandex-diskAUR

#### Multi-service clients

- Déjà Dup — A simple GTK+ backup program. It hides the complexity of doing backups the 'right way' (encrypted, off-site, and regular) and uses duplicity as the backend.
  - Automatic, timed backup configurable in GUI.
  - Restore wizard.
  - Integrated into the GNOME Files file manager.
  - Inherits several features of duplicity.

  https://launchpad.net/deja-dup || deja-dup

- Duplicati — Backup client that securely stores encrypted, incremental, compressed backups on cloud storage services and remote file servers. It works with Amazon S3, Windows Live SkyDrive, Google Drive (Google Docs), Rackspace Cloud Files or WebDAV, SSH, FTP (and many more). Duplicati is open source and free.

  http://www.duplicati.com/ || duplicati-latestAUR

- Duplicity — A simple command-line utility which allows encrypted compressed incremental backup to nearly any storage.
  - Supports gpg encryption and signing.
  - Supports gzip compression.
  - Supports full or incremental backups, incremental backup stores only difference between new and old file.
  - Supports pushing over many protocols.

  http://www.nongnu.org/duplicity/ || duplicity

- Duply — Front-end for duplicity which simplifies running it by:
  - keeping recurring settings in profiles per backup job;
  - automated import/export of keys between profile and keyring;
  - enabling batch operations eg. backup_verify_purge;
  - executing pre/post scripts;
  - precondition checking for flawless duplicity operation.

  http://www.duply.net/ || duplyAUR

- rclone — Rclone is a command line program to sync files and directories to and from Google Drive, Amazon S3, Openstack Swift / Rackspace cloud files / Memset Memstore, Dropbox, Google Cloud Storage and The local filesystem.

  http://rclone.org/ || rclone

- Syncany — Cloud storage and filesharing application with a focus on security and abstraction of storage.

  https://www.syncany.org/ || syncanyAUR

### Custom infrastructure

See List of applications/Internet#Cloud storage servers

## Version control systems

This article or section is a candidate for merging with List_of_applications#Work_environment.

Notes: Also drop the extra bullets, the Wikipedia comparison table is much better than this. (Discuss in Talk:Synchronization and backup programs#)

These are traditionally used for keeping track of software development; but if you want to have a simple way to manage your config files in one directory, it might be a good solution.

See also Wikipedia:Comparison of revision control software.

- Bazaar — A distributed version control system that helps you track project history over time and to collaborate easily with others.
  - Similar commands to Subversion.
  - Supports working with or without a central server.
  - Support for working with some other revision control systems
  - Complete Unicode support.

  https://bazaar.canonical.com/en/ || bzr

- Darcs — A distributed revision control system that was designed to replace traditional, centralized source control systems such as CVS and Subversion.
  - Offline mode.
  - Easy branching and merging.
  - Written in Haskell.
  - Not very fast.

  http://darcs.net/ || darcs

- Git — A distributed revision control and source code management system with an emphasis on speed.
  - Very easy creation, merging, and deletion of branches.
  - Nearly all operations are performed locally, giving it a huge speed advantage on centralized systems.
  - Has a "staging area" or "index", this is an intermediate area where commits can be formatted and reviewed before completing the commit.
  - Does not handle binary files very well.

  https://git-scm.com/ || git

- Mercurial — A distributed version control system written in Python and similar in many ways to Git.
  - Platform independent.
  - Support for extensions.
  - A set of commands consistent with Subversion.
  - Supports tags.

  https://www.mercurial-scm.org/ || mercurial

- Subversion — A full-featured centralized version control system originally designed to be a better CVS.
  - Renamed/copied/moved/removed files retain full revision history.
  - Native support for binary files, with space-efficient binary-diff storage.
  - Costs proportional to change size, not to data size.
  - Allows arbitrary metadata ("properties") to be attached to any file or directory.

  https://subversion.apache.org/ || subversion

## See also

- Wikipedia:List of backup software
- Wikipedia:Comparison of backup software
- Exhaustive list of backup solutions for Linux
- Backing up Linux and other Unix(-like) systems
- Mirroring an Entire Site using Rsync over SSH
- Performance comparison of five remote incremental backup tools: Rsync, Rdiff-backup, Duplicity, Areca and Link-Backup
- rsync-snapshot.sh — Local and remote snapshot backup using rsync with hard links

Categories:

- Data compression and archiving
- System recovery

Hidden categories:

- Pages with broken package links
- Pages with dead links
- Pages or sections flagged with Template:Merge