Secured Data Storage

[splitice]

After the GVH "hack" where Gov IDs were leaked I started working on this tutorial. It details a similar system to what we use to store client SSL certificates for the interface and deployment system. The system described is complex to setup, but does not compromise in security. If you store sensitive information on your clients, you should be doing so in a responsible manner (don't just store it in an attachments directory on the web server)

Overview

This tutorial details how to set up a service for the storage of sensitive files (i.e personal IDs, client SSL certificates, etc) where you need certain machines (i.e web workers) to have certain access (i.e privileges to upload) while maintaining overall security (i.e web nodes can't view / download the data). This restriction of access ensures that a compromised web node does not result in the dumping of all sensitive files, etc.

To achieve this the following technologies will be used:

• Nginx, OpenSSL and Lua (or LuaJIT) used to host the service
• IPSec (via Strongswan or optionally Openswan) used to encrypt data transferred and provide a level of protection to the service
• IPTables used to restrict access to the service only to IPSec connections

Some would consider this is quite an advanced tutorial. I have tried to reduce the steps as much as possible to keep this from becoming a small book.
The System Design

Description:

• Authorization to specific service functionality is provided through client SSL keys.
• Encryption is performed by both SSL (HTTPS) and IPSec
• Physical (bare metal) Separation is recommended between servers to prevent compromise at the physical node level.

Of course you access needs may vary, this is primarily an example. Modify as needed. Data Processing could even be an non automated action, such as access for an administrator to retrieve files etc.
This tutorial is based off Debian, commands may / will differ for CentOS and others (however the concept remains the same).

Instructions

Step 1 - Compile and Install Nginx

Nginx needs to be compiled with support for LUA, WebDav and SSL. To compile nginx I will share the build environment and script we use. You can of course do this manually.

You can download the build environment here: https://dl.dropboxus...nginx_build.rar

Then just download, build and install nginx with:


bash build.sh installStep 2 - Create Certificates for Server & Client Authentication

For this tutorial certificates should be placed in /etc/nginx/ssl/.

Create the CA Key and Certificate for signing Certificates


openssl genrsa -des3 -out ca.key 4096
openssl req -new -x509 -days 3650 -key ca.key -out ca.crt
Create the Server Key, CSR, and Certificate. Common Name for this certicate should be your servers domain name.

openssl genrsa -des3 -out server.key 1024
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

For each client generate a unique client certification. Our authentication method relies apon certain strings existing in the Common Name, i.e a common name containing the string "Upload" (and signed by the CA) has upload privileges.

openssl genrsa -des3 -out client.key 1024
openssl req -new -key client.key -out client.csr
openssl x509 -req -days 3650 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

You may also wish to remove the pass-phrases on the keys (less secure, more practical).

openssl rsa -in certificate.key -out certificate.key

Step 3 - Nginx / Service Configuration
To /etc/nginx.conf add


worker_processes 1;

error_log /var/log/nginx/error.log info;

#pid logs/nginx.pid;
user www-data;

events {
worker_connections 1024;
}


http {
include mime.types;
default_type application/octet-stream;

#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';

#access_log logs/access.log main;

sendfile on;
#tcp_nopush on;

#keepalive_timeout 0;
keepalive_timeout 65;

gzip off;

# HTTPS server
#
server {
listen 443 ssl;
server_name ssl-store.x4b.org;

ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
ssl_client_certificate /etc/nginx/ssl/ca.crt;
ssl_verify_client on;

ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;

ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

location ~ ^/api/upload(?.+) {
if ($ssl_client_s_dn !~* "Upload") {
return 403;
}

alias /var/ssl-store$file_path;

create_full_put_path on;
dav_access user:rw;

dav_methods PUT;

limit_except PUT {
deny all;
}
}

location ~ ^/api/delete(?.+) {
if ($ssl_client_s_dn !~* "Delete") {
return 403;
}

alias /var/ssl-store$file_path;

create_full_put_path on;
dav_access user:rw;

dav_methods DELETE;

limit_except DELETE {
deny all;
}
}

location = /api/md5 {
if ($ssl_client_s_dn !~* "MD5|Upload") {
return 403;
}

content_by_lua '
local lfs = require "lfs"

function readAll(file)
local f = io.open(file, "rb")
local content = f:read("*all")
f:close()
return content
end

function explode(div,str) -- credit: http://richard.warburton.it
if (div=="") then return false end
local pos,arr = 0,{}
-- for each divider found
for st,sp in function() return string.find(str,div,pos,true) end do
table.insert(arr,string.sub(str,pos,st-1)) -- Attach chars left of current divider
pos = sp + 1 -- Jump past current divider
end
table.insert(arr,string.sub(str,pos)) -- Attach chars right of last divider
return arr
end

local files

if ngx.var.request_method == "GET" then
files = explode(",",ngx.var.arg_files)
else
ngx.req.read_body()
local args, err = ngx.req.get_post_args()
files = explode(",",args.files)
end

for i=1,#files do
local fn = "/var/ssl-store/" .. files

ngx.print(files, ":", ngx.md5(readAll(fn)), "\\n")
end

';
}

location ~ ^/api/md5(?.+) {
if ($ssl_client_s_dn !~* "MD5|Upload") {
return 403;
}

content_by_lua '
local lfs = require "lfs"

function readAll(file)
local f = io.open(file, "rb")
local content = f:read("*all")
f:close()
return content
end

local fn = "/var/ssl-store" .. ngx.var.file_path
local dir;
local status, error = pcall(function()
dir = lfs.dir(fn)
end);

local not_a_directory = false
if not status then
not_a_directory = string.match(error, "Not a directory")
end

if not_a_directory then
ngx.print(ngx.md5(readAll(fn)))
else
for i in lfs.dir(fn) do
if i ~= "." and i ~= ".." then
ngx.print(i, ":", ngx.md5(readAll(fn.."/"..i)), "\\n")
end
end
end

';
}

location ~ ^/api/download(?.+) {
alias /var/ssl-store$file_path;

if ($ssl_client_s_dn !~* "Download") {
return 403;
}

limit_except GET {
deny all;
}
}
}
}
These APIs exposed are examples, MD5 for example is what we use internally to ensure integrity you may wish to use a different algorithm or not use it all. You may also wish to further extend this with more complex processing, LUA is a very powerful language.
Step 4 - Setup IPSec

Install strongswan. Openswan will also work, however is not recommended.


apt-get install strongswan
To /etc/ipsec.conf on the Secured Storage Server add

conn ssl-ipsec
      authby=secret
      left=**SERVER-IP**
      right=%any
      keyexchange=ikev2
      auto=start
      type=transport

To /etc/ipsec.conf on the Web Server add. %defaultroute ensures compatibility if you have multiple interfaces.

conn ssl-ipsec
      authby=secret
      left=%defaultroute
      right=**SECURED-SERVER-IP**
      keyexchange=ikev2
      auto=start
      type=transport

To /etc/ipsec.secrets on both the Web Server and the Secured Storage Server add

: PSK "a super secret string!!!"

Restart both servers (or load the kernel modules, and restart ipsec) to activate IPSec.
To verify IPsec is running correctly at one end execute:


tcpdump -n esp
At the other end, ping the end running tcpdump. If you see packets flowing, then its working e.g:

09:49:37.837908 IP 178.62.xxx.xxx > 95.85.xxx.xxx: ESP(spi=0xc3d1xxxx,seq=0x10), l

Step 5 - IPTables traffic restriction
Install an IPTables rule that will restrict access to the HTTPS server (port 443) for any traffic that was not received via IPSec.


iptables -A INPUT -p tcp -m tcp --dport 443 -m policy --pol none --dir in -j REJECT
To ensure this rule is loaded on system start up it can be placed into /etc/rc.local
Protections Afforded

• Web facing server has no access to data uploaded. This server is both the most likely to be compromised (exploit PHP, Apache, nginx etc) and the most exposed (internet)
• Access to functionality is restricted via the use of authorized certificates, not passwords (far more secure). Provides both authorization and authentication.
• Data between Secured storage server is encrypted with SSL, the SSL data is then encrypted with IPSec. This protects against any vulnerabilities in OpenSSL / nginx resulting in data breach.
• Access to service is restricted to those who first secure an ipsec tunnel using the Pre-Shared secret (out of band). This hardens the service against vulnerabilities in technologies used (OpenSSL, nginx, etc) and provides an additional layer of security.
• Sensitive Data can be stored in a physically separate location, possibly on a private network behind extensive firewalling and in a secured area of the datacenter if appropriate (and required).

This solution explained does not for example consider disk theft as a large enough risk to warrant additional security (i.e disk encryption).
Steps for additional security, and recommendations

Of course you should consider your risks and plan accordingly. If you need more security:

• Implement CRL to allow for the revocation of certificates (i.e compromised machines)
• Rate limiting and quotas for data retrieval and deletion
• Ensure software is kept up to date
• Ensure no other services which are not required are installed on the secured storage server
• Encrypt disk / storage
• If client IP addresses change infrequently additional security can be gained through whitelisting IP addresses.
• More complex method of verifying access permissions using client certificates (i.e database of authorization)
• Policies and Procedures for disaster handling

Notes
When I have time I can also post a PHP class for interfacing with the example API shown if anyone wants it, its not too difficult (just curl).

Please note differences exist between the tutorial steps and the steps we use, the setup also differs a bit too. I don't think I have made any mistakes in the simplification, but yeah.

 

[HalfEatenPie]

 

[bigcat]

Seems fairly complex for an objective that can be achieved cheaper, using disk encryption and security policy? (though I admit I'm fairly dumb on network security)

Eg: Implement EncFS(audit report), limit access using AppArmor/SELinux and now you're PCI/DSS & HIPA/HITECH compliance. Add tripwire just for fun.

inb4 setenforce 0

#yolo

Spoiler

 

[splitice]

@bigcat

Obvious troll is obvious? Given that the first instruction to anyone hoping to achieve PCI compliance is to Install and keep updated a firewall between the public network and the payment card data. I guess so.

What you describe would meet none of the intended protection aims. Having compromised the web node the attacker can easily dump the mounted filesystem. Octal permissions could be used for a limited effect, however nothing to this level (MD5? File Listing? Who owns, the web user?). Nor does is it accessible to multiple servers (how do you do that? NFS share? encryption?).

Maintaining a high level of security is preferable to falling foul of a security breach, and the best way to do that is through a well thought out solution.