Taking Regular & Secure backups on your server

[splitice]

Should you be taking backups of more things than you currently are? Well now there is no excuse. Just use this 50 odd line script and receive your encrypted backups attached via email.

Download:

The script is available at Github - https://github.com/splitice/encrypted-email-backup

Features:

• Supports any backup format, just pipe it into the backup script. Use one script for all your backups.
• Backups are stored in a remote location, a remote mail server. This is unlikely to fail at the same time as your server being backed up, and should be secured with a different password to prevent simultaneous compromise.
• Backups are encrypted with AES-256 encryption with a key that is not transmitted to the remote location. This encryption is
• An email is sent on backup failure, successful and failed backups are easily distinguishable through the subject prefix. These can also be easily used in automatic filing/filtering system's to ensure it comes to an administrators attention.
• All the software used in the process is Opensource, this means it is unlikely to be backdoored (atleast not by me), that it is an open format and that it is likely available in the package manager of your distribution.

Backup Setup Procedure:

• Create the backup encryption key. Create a file at an appropriate location, this key can be of any length (longer better) and should be backed up in a different location in case of data-loss. If you loose this key you will be unable to decrypt your backups. You can use your favorite text editor to create this key file.
  nano /opt/backup-script/backup.key
  
• From your backup user (i.e www-data) open the crontab to add a cron job
  

  crontab -e

• Add the backup cron job, to run it daily use @daily, to run it hourly @hourly etc. This script will be similar to the following, you will need to pipe your own backup script (tar of files, mysqldump etc) into the backup script. Be sure to supply your own email address, and your own encryption key.
  

  @daily /opt/backup-script/take_backup | bash /opt/backup-script/backup-sql-encrypt.sh backup /opt/backup-script/backup.key "backup" "[email protected]" "Backup"

Restoration Procedure:

• Download Backup Email attachment on to a linux server with a copy of the backup.sh script, and the key file.
• Execute the backup script in decryption mode, pipe in the file, pipe out the backup
  

  cat backup.enc | backup.sh decrypt /opt/backup-script/backup.key > backup

• Restore the backup as appropriate for your software. See the examples section for examples using MySQL.

Examples:
Backup a MySQL Database

mysqldump -u [username] -p[password] [database_name] | bash /opt/backup-script/backup-sql-encrypt.sh backup /opt/backup-script/backup.key "backup.sql" "[email protected]" "Database Backup"
 Decrypt the MySQL Database backup to file

cat backup.sql.gz.enc | ./backup.sh decrypt /opt/backup-script/backup.key > backup.sql
 Decrypt the MySQL Database backup to a MySQL Database

cat backup.sql.gz.enc | ./backup.sh decrypt /opt/backup-script/backup.key | mysql -u root -p[root_password] [database_name]
Backup a folder on the filesystem

cd /var/www && tar c test-site-folder.com | bash /opt/backup-script/backup-sql-encrypt.sh backup /opt/backup-script/backup.key "backup.tar" "[email protected]" "Site Data Backup"

Restore a folder on the filesystem

cd /var/www && cat backup.tar.enc > bash /opt/backup-script/backup-sql-encrypt.sh restore /opt/backup-script/backup.key | tar xc

Additional Information
Why encrypt your backups?

Do you trust your email provider (Google, Microsoft etc)? Even if you do (damn!) just having your backups in your email provides another possible avenue of data-leakage (if your email is compromised, so are your site backups). Encrypted backups are useless without a copy of your encryption key which should not be available in your email.

Why receive your backups via email?

Most people check their emails frequently, hell - for most of us it is our primarily channel of communication in the connected world. By receiving your backups via email you are forced to check them daily, and will be quick to notice if they stop arriving.

Why send an email on failure?

Why not, its important you know when something changes and the backup was unable to be run. Often these scripts are the kind you run for years without modification. Its important to know if they stop running, and hence your system ceases to be backed up regularly.

How frequently should I back up?

This depends on your system. Think about it in terms of how much can you data can you afford to loose.

What about my mail server disk space?

Delete old backups. We do this via a cusomized version of the automation script found here: http://www.labnol.org/internet/gmail-auto-purge/27605/

What if my backup is already compressed?

Remove the gzip executions from the script, or else there will be a small overhead for double compression.

I want to do something slightly differently...

You are welcome to... the script is Open Source. I encourage you to fork it on Github. I want no rights. If your changes are high quality and useful for others, submit a pull request and I'll take a look and see about merging.

 

[MannDude]

Thanks for the write-up, looks good! Will need to test it out sometime as my backup script is pretty basic.

 

[willie]

Interesting.  I'd suggest using a public encryption key, so there's no need to keep a secret key at either end.  You only need the decryption key if you actually want to restore from the backup.

 

[splitice]

@willie, are you concerned regarding the security of your restoration server? Personally I dont see the use-case for public key encryption. Just so we are clear public key encryption would work as follows:

Backup Server: Has Private Key, can write and read backups

Restoration Server / During Restoration: Has a public key, can read backups.

I fail to see any security benefit to this, the most likely server to be compromised is the backup server (especially if the restoration server is only configured on as needed basis). The backup server has the private key, it can create new public keys etc. Furthermore asymmetric cryptology is more complex to use and setup. And given that symmetric cryptography is computationally much less resource-intensive, its faster to restore in an emergency (particularly AES which is often accelerated by instruction sets like AES-NI). Personally I think the much simpler symmetric encryption is a very good fit, however you can use assymetric encryption with only minimal modification to the supplied script. Here is a quick overview of the commands required:

Create your key-pair:

openssl req -x509 -nodes -newkey rsa:2048 -keyout private.pem -out public_nopass.pem

Encrypt your file:

openssl smime -encrypt -aes256 -in file -binary -outform DEM -out file.enc public.pem

Decrypt your file:

openssl smime -decrypt -in file.enc -binary -inform DEM -inkey privatekey.pem -out file

 

[willie]

Splitice, sorry for slow reply, just saw this.

1) the reason for using public key is to avoid having to store any private keys on any of the servers.  The private key is supplied to the backup server during a restore, but it's not present at any other time.

2) The way public key crypto is done in practice is you create a temporary random symmetric key in ram (this is called a session key), use the session key to encrypt the backup (with AES-NI if you want), encrypt the session key using the public key and store the encrypted session key with the backup, and dispose of the session key (it is never written to disk).  You do only one public key encryption, and it's of a very small piece of data (the session key), so if it takes a few milliseconds instead of microseconds/nanoseconds that's fine.

 

[splitice]

@willie Let me try and explain this.

1) In all cases when performing asymmetric encryption a key to use as a "private key" is required. It can be either a "private key" or (most of the time) a "public key", that is to say a public key used as a private key. The names are just semantics based on their roles you can just call them Ka and Kb. Many encryption algorithms are what are called "trapdoor permutations". This means:

• If you encrypt with a public key, you can decrypt with the private key (and also the public key).
• If you encrypt with a private key, you can decrypt with a public key (and also the private key).

 If you want to know the openssl function to decrypt with a private key in PHP is openssl_private_decrypt feel free to try it out.

Generally speaking the strength of Public Key cryptology is that:

• it is possible to generate a Kprivate and Kpublic they can both be used to decrypt the same crypt text.
• it is computationally infeasible to generate Kprivate from Kpublic (ensure identity / MITM protection). 
• Numerous other derived benefits (signing etc)

There is as of yet no such thing (unless you consider quantum encryption) as encryption that can be encrypted with Ka but not read by Ka. The same process used to encrypt can always be reversed.  Well unless you get into certain methods of key exchange (secrets forming keys not transmitted, only known locally for a session at each end). Key exchange with PFS would be impossible over email. Its the key exchange in SSL/TLS that serves to prevent eavesdropping, not just the encryption.

Now this is my understanding. It is not my area of expertise. But I have worked in this field before (in industry). I am 95% certain I haven't made any major mistakes above.

2) Yes you are correct you can use asymmetric encryption over a symmetric key. My statement was a generalization of the simplest method. You can of course encrypt a session key.

Now, feel free to try and prove me wrong. I am fairly confident that any mathematical/logical transformation (i.e encryption) can be reversed with access to the key. The only way around this is through complex secret exchanges which even if you could get to work with a email based backup system (wow PFS!) would be far beyond the scope of a simple backup system implemented in a 50 line bash script.

 

[GS-Dylan]

Looks like a great script, haven't had time to try it out. One concern I see with it however is that depending on how yur server is setup, your email could very well be hosted on the same server your backing up.. Unless your actually downloading the attachment every day from the email. Also depending on the size of the backups sending as an email isn't going to be very efficient. 

Not trying to bash your script(punny right?) just pointing out a few things I noticed, I think the script is simple and perfect for small simple backups, but not really a enterprise solution.

 

[splitice]

Yes, Dont use this for backing up your email server. I don't know of many "enterprises" keeping their primary mail server on a server with shared responsibilities (i.e web/database) requiring backup. Generally speaking its one of the first thing you seperate out for security of private communications (or you just grow big enough to justify it).

Email is a simple receiving method suitable for many Low End backups. While it may not be "enterprise" due to its inherent use of email as a storage medium, "enterprise" usually use and can afford commercial software (and require verifitcation of data etc) usually using specific servers and resources for backup. Thats outside the scope of the intent of such a script.

You would be suprised at how "efficient" (time, cost, etc) storage to email is, not to mention efficient in terms of storage representation (cheap database).

 

[Abdussamad]

@willie Let me try and explain this.

• If you encrypt with a public key, you can decrypt with the private key (and also the public key).
• If you encrypt with a private key, you can decrypt with a public key (and also the private key).

Hello you are wrong about the parts in the brackets. The entire idea is that what is encrypted with the public key can only be decrypted with the private key and vice versa. So you can make your public key public. People encrypt with it and transmit that encrypted data along a public medium like the web or email secure in the knowledge that only the person with the private key can decrypt it.

Message signing is taking a hash of a message and encrypting that hash with your private key and attaching it along with the message. The recipient also takes a hash of the message and then he tries to decrypt your encrypted hash using your public key. If the hashes match then it shows that 1. the message is unaltered and 2. that you sent it because only you have your private key and can encrypt the hash in such a way that it would decrypt with your public key.

If you want to both sign and encrypt you sign with your private key and encrypt with the recipient's public key.

 

[drmike]

This script truly is good...  One big thing it needs is simply the forgoing email entirely.  Backup say via keyless to remote.

 

[splitice]

Yes there is a mistake in my post, I got confused in regards to Key Exchange. There is no need for it (keys are transferred out of band), which makes it possible to use asymmetric public key encryption. I might consider making a flag for the script to support it if I can do it without adding significant complexity.

@drmike You can easily swap in scp/sftp (we do for some of our backups). Email was chosen for ease of demonstration (and because for small-medium backups it is suitable).

 

[splitice]

Just finished a large upgrade. Asymmetric encryption, slight performance improvement (thanks PeterMosmans) and custom upload functions.

You can now specify custom upload methods, just define a upload_$NAME function in upload/upload_$NAME.sh. Then all you need to do is change $BACKUP_MODE to the $NAME of your method.

Your upload method might look something like:

function upload_scp {
    scp $1 [email protected]:/some/remote/directory
    FILE=$(basename "$1")
    echo "remotehost.edu:/some/remote/directory/$FILE"
}

Pull requests welcome.

 

[tk-hassan]

Thank you for sharing this useful post. Cheers

 

[Wambo]

Thank you! I know this is a kinda old post but this helped me a lot!

 

[drmike]

@MannDude can we make sure this is socked away in the Library. 

We have lots of gems like this that get lost in the mass of how forums operate.