amuck-landowner

How to do encrypted backups with duply/duplicity

tchen

New Member
Duply is a thin command line wrapper around duplicity, handling configuration, keys and pre-post scripts.  You can still pass in duplicity options directly, but this just makes things so much nicer.

This guide assumes

  1. You are running Debian
  2. You want to backup selective parts of your system to remote storage
  3. The remote archive store is relatively unsecured
  4. A desire to be able to verify/restore files in time
  5. Backup purging

Installation

Start by installing the required packages



sudo apt-get install –y duplicity duply ncftp

Duplicity, and duply as a result, can have targets that use rsync, ftp, s3, scp, webdav.  I’ll use FTP as that’s most easily available.

Encryption and Signature Keys

Duply is pretty flexible with how you setup encryption.  For our purposes, we have the following requirements:

  1. Protect a cluster’s backup from a compromised origin server
  2. Have the ability to open any backup with a master key
  3. Have the ability to automatically verify a backup set from the server
To accomplish this, each server will use the master encryption key along with its own public key to encrypt the backup.  The server key will also act as the signature key for the backup set.

Let’s start by creating a master key



gpg --gen-key

Accept the default key type, bits, and expiry.  Fill in the name, email as you see fit.  Use a passphrase and note it down somewhere very safe.  At the end, it’ll create a key and automatically add it to your keyring.  For example,



gpg: key 3B70D27E marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   2048R/3B70D27E 2013-10-31
      Key fingerprint = 476A 1D6E 2C3E B4AC 0D94  5EA2 1A3D F71A 3B70 D27E
uid                  Bob Marley <[email protected]>
sub   2048R/317C2259 2013-10-31

Note the ID next to the pub line.  3B70D27E is the key ID we’ll be using in this example.  We’ll need to export this public key



gpg ––armor ––export 3B70D27E > master-archive-enc.pub

We’ll also export the private key and store it in a very secure place – preferably offline on a CD/DVD.  Delete it from the ring too just to be sure.  You really don’t need this private key except in extreme situations so feel free to dust off those old ICBM launch code plastic cases and place it in there.



gpg ––armor ––export-secret-keys 3B70D27E > master-archive-enc.key
gpg ––delete-secret-keys 3B70D27E

On the servers you wish to backup, import your master public key and trust it



gpg --import master-archive-enc.pub
gpg --edit-key 3B70D27E
> Trust
> 5 = I trust ultimately
> quit

Then generate the encryption key for the server



gpg --gen-key

We don’t need to export these.  Leave both public and private keys in the GPG keyring.

Profiles

Create a profile using the command



/usr/bin/duply myapp create

This will create a profile folder in /etc/duply/myapp with a skeleton conf file.  There’s a lot to take in, so we’ll outline a simpler configuration file.  Let’s start with declaring our encryption keys.



# MINIMAL DUPLY CONFIGURATION
# file: /etc/duply/myapp/conf

GPG_KEY='25CAF99E,3B70D27E'
GPG_PW='serverpassphrase'

In this case, 25CAF99E is the server key with the ever-secret passphrase ‘serverpassphrase’.  The 3B70D27E specifies our master key which will also be used as an encryption key.  The backup sets can be opened by anyone who has either of the two corresponding private keys.

The next section defines what to backup and where to do it.



SOURCE='/'
TARGET='ftp://127.0.0.20/backups/myapp'
TARGET_USER='bobmarley'
TARGET_PASS='isthislove'

SOURCE can be set to a specific directory and doesn’t have to be root.  Here, we use root and rely on an exclusion file to select which directories are actually included.  Use this if you want to backup across multiple directories like var and etc.



# DUPLY EXCLUSION CONFIGURATION
# file: /etc/duply/myapp/exclude

- **/.svn
- **/.subversion
+ /var/myapp
+ /etc/myapp
- **

Going back to the conf file, we specify what to do during a duply purge command. 



# Keep for...
MAX_AGE=6M

# Number of full backups to keep.  Must issue one at least every month.
MAX_FULL_BACKUPS=10
MAX_FULLBKP_AGE=1M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "

During the first backup, duply will export the server keys into the profile directory.  It might be a good idea to back this folder up onto CD/DVD and store it safe as well.  However, if you ever lose it – or you just have way too many servers to track these - you can still restore from the master key.

Cron

Setup an automated schedule.  Given our MAX_FULLBKP_AGE, we’ll prune about once a month.



10 15 1 * * /usr/bin/duply myapp backup_ purge --force 2>&1 >> /var/log/duply/myapp.log

10 3 * * * /usr/bin/duply myapp backup 2>&1 >> /var/log/duply/myapp.log
Be sure to setup a logrotate as the output can get quite verbose.

Pre and Post Scripts

To dump a database before the actual backup is run, duply has a pre script



#!/bin/sh
# file: /etc/duply/myapp/pre

echo 'Some very important mysql stuff'

Listing and Restoring Files

You can list the contents of the set.  For the most recent



duply myapp list

or if you want something older, say from a week ago



duply myapp list 7D

Restoring is as easy as



duply myapp restore ~/restored

or if you want something older, say from a week ago



duply myapp restore ~/restored 7D
Notes

Sadly, a compromised host will mean its target bucket is also vulnerable.  There's really nothing to do other than periodically archiving the backup sets to an offline storage (or at least semi-offline like Amazon glacier).

I also suggest separate FTP accounts for each server to protect them from each other.
 
Top
amuck-landowner