Dave Horner's Website - Yet another perspective on things...
Home Tech Talk Unix/Linux/BSD/etc rdiff-backup - remote incremental backup
168 guests
Rough Hits : 3106336
moon and stars
how did u find my site?





 
answer random online questions?


 
"For every complex problem, there is a solution that is simple, neat, and wrong."
-- H.L. Mencken
\begin{bmatrix} 1 & 0 & \ldots & 0 \\ 0 & 1 & 0 & \vdots \\ \vdots & 0 & \ddots & 0\\ 0 & \ldots & 0 & 1_{n} \end{bmatrix}

rdiff-backup - remote incremental backup

Monday, 25 April 2005 00:00
Backup is one of the most important things for people who work with computers and it isn't the easiest thing to do!  Tape backups are painfully slow, hard to manage, and easily corrupted.  Backup to disk is good, but you've got to constantly swap disks and move off site to make it worth anything.  Then dropping a drive is also a very sad occasion too. :) The best solution is remote network backup that is incremental.
Using Rdiff-backup with SSHFS - Linode Library

I've found rdiff-backup to be my backup tool of choice.  It provide remote incremental backup and is very bandwidth efficient because it uses the rsync protocal (and like I said before, its incremental).

Using rdiff-backup is very easy and provides great flexability, you can restore a version of your file from any time in the past. rdiff-backup is written in python.

Backup
rdiff-backup /user/user /backup/user
This would backup /user/user to the directory /backup/user.
Local backup allows you to restore a version of the file from the past.

rdiff-backup /some/local-dir auser@ahost::/remote-dir
Remote backup /some/local-dir to the ahost machine using auser as a username.
Data will be stored in the /remote-dir and transmission will take place over SSH.

You can also backup a source directory to a local directory like so/:
rdiff-backup This e-mail address is being protected from spambots. You need JavaScript enabled to view it ::/remote-dir local-dir

rdiff-backup Restore

Since rdiff-backup stores the most current version of the file in the backup tree, simply copy the file!
If you want to get a version of the file from 10 days ago simply use:
rdiff-backup -r 10D host.net::/remote-dir/file /tmp/file


Unattended backup

Well you don't want to have to remember to login every day and run the backup.  So you've got to setup a cron job to actually run the scripts.  The best way to do this is to create a single use ssh key that allows your server to login to the mirror.  Setup a cron job to execute the rdiff-backup using the identity.
rdiff-backup --remote-schema 'ssh -i <identity> %s rdiff-backup' <rdiff params>
(or the better way of doing it is to create a session in your .ssh/config that uses the correct identity file)

If you add a public key to a remote server, you might want to limit the key to only run rdiff-backup, read only access,from a specific host, and not allow any forwarding.  Add the following in front of the public key in the authorized_keys file.
command="rdiff-backup --server --restrict-read-only /",from="servername",no-port-forwarding,no-X11-forwarding,no-pty

Since this article is not to be a complete manual for rdiff-backup, please visit:
rdiff-backup docs

Here is how to quickly setup a back server.
-------------------------------------------
Login to the server that we will be backing up.
-------------Create Key and Identity--------------------------
[user@backup_src]$ su -; #You'll want the cron job to run as root.
[root@backup_src]# cd $HOME/.ssh;
[root@backup_src]# ssh-keygen -t rsa -f ~/.ssh/backup_dest;
[root@backup_src]# ssh-copy-id user@remotemachine # some machines have ssh-copy-id which does as the command below. [root@backup_src]# cat backup_dest.pub |ssh user@remotemachine 'sh -c "cat - >>~/.ssh/authorized_keys;chmod 600 ~/.ssh/authorized_keys;"';

You should now be able to test the key using:
[root@backup_src]# ssh -i backup_dest user@remotemachine

Create identity using:
[root@backup_src]# echo "host identityname
hostname remotemachine.somenetwork.com
user auser
identityfile ~/.ssh/backup
compression no
cipher blowfish
protocol 2" >> ~/.ssh/config; chmod 600 ~/.ssh/config;

You should now be able to test the identity and key using:
[root@backup_src]# ssh identityname

----------RESTRICT KEYS AND CREATE DEST FOLDER!-----------------------------------------------------

Login to the remote backup_dest machine.

If you'd like you can restrict the key given to backup_src to read only priv. or maybe to a certain portion of your disk.
If you would like to do this:
[user@backup_dest]$ vi ~/authorized_keys;
Then prepend this on your key like so:
command="rdiff-backup --server --restrict /var/backups/",no-port-forwarding,no-X11-forwarding ssh-rsa XXXXXXXXXXXXXXXXX= root@backup_src

Then make sure the create the folders and make sure the user has permission to write in those folders.
[user@backup_dest]$ cd /var/backups/;
[user@backup_dest]$ mkdir backup_src;

-----------CREATE BACKUP SCRIPT AND TELL IT WHAT TO BACKUP-----------------------

Log back into the server we are backing up.
[user@backup_src]$ su -; #You'll want the cron job to run as root.
[root@backup_src]# cd $HOME/;
[root@backup_src]# vi backuplist.txt; chmod 600 backuplist.txt;
files
/anddirectories
/to/include
- /
[root@backup_src]# vi backup.sh;chmod 700 backup.sh;
#!/bin/bash
echo 'Backup starting at: '; date;
rdiff-backup --force --print-statistics --include-globbing-filelist /root/backuplist.txt / identityname::/var/backups/backup_src;
echo 'Backup starting at: '; date;

--------------------CREATE CROND ENTRY TO RUN EVERY NIGHT --------------------------
[root@backup_src]# ln -s /root/backup.sh /etc/cron.daily/
------------------QUICKLY INSALL--------------
cd /usr/local/src;
mkdir rdiffbackup;
cd rdiffbackup
wget http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.0.0.tar.gz
wget http://puzzle.dl.sourceforge.net/sourceforge/librsync/librsync-0.9.7.tar.gz
tar xfvz rdiff-backup-1.0.0.tar.gz;
tar xfvz librsync-0.9.7.tar.gz;
cd librsync-0.9.7;
./configure;
make; make install;
cd ..
cd rdiff-backup-1.0.0;
python setup.py install

Web Interfaces

rdiffWeb
- I'm very excited to see a web gui for the rdiff-backup tool. I'll let you know how it goes.

SafeKeep Home
rBackup - rdiff-backup = rbackup ! - a php cli over rdiff-backup, provides /etc/rbackup.conf to configure backups. rdiff-backup and svnsync.

rdiff-backup / rsnapshot / duplicity comparisons

rsnapshot vs rdiff-backup
backup - what is the difference between rsnapshot and rdiffbackup? - Super User
Backup on Linux: rsnapshot vs. rdiff-backup (vs. Time Machine) « SaltyCrane Blog
Choosing a new backup solution, Duplicity, rdiff-backup or Rsnapshot


rsnapshot

written in perl.
rsnapshot - another snapshot utility based on rsync.
Easy Automated Snapshot-Style Backups with Rsync (rsnapshot apparently based on this article)
rsnapshot - remote filesystem snapshot utility- man page
rsnapshot HOWTO
Using Rsnapshot and SSH
Backing Up Multiple Servers with Rsnapshot | Derek Simkowiak
Using rsnapshot for Redundant Backups | EVBackup
Backuping using rsnapshot made my life easier « Remi Bergsma's blog
Set Up Rsnapshot, Archiving Of Snapshots And Backup Of MySQL Databases On Debian | HowtoForge - Linux Howtos and Tutorials

Unison

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.
Unison File Synchronizer
Synchronize files with Unison - Linode Library
Last Updated on Wednesday, 12 June 2013 18:05