Running your own mail server

This tutorial is about setting up a mailserver. It is based on my own efforts to not use gmail any longer.
I want to keep my own emails and I want to keep my own backups.

Running your own mail server is pain. This is something you have to monitor all the time. And as long as your server is not encrypted (and even if) you should use a client based enryption for your email client.

It is also a good idea to separate this from your other services. I am using a small 128 MB vps that is only running the mail server. It is fine for 5 users. If you want to run additional services like clamav and if you have more than 5 users you should use 512 MB of RAM.

This post will be updated once a week. This topic is quite huge so I will need some iterations to complete this tutorial. I also want to include all feedback to ensure that this tutorial is up to date.

Because email servers can have a lot of features like:

• marking spam
• graylisting
• virus scanning
• virtual mappings
• etc

I will mark every step that is just adding a feature as OPTIONAL.

So let's start with the preparations:

1. Setup your DNS:

Create an A record that is pointing to your vps (which should run the mailserver).


mailserver 55.55.55.55 A 1800
mailserver 2500:f5f5:25::b25f:2525 AAAA 1800

I use a service oriented nameing shema so in my case: "mailserver.domain.com". This name is quite important because it is used in a lot of different places.

You can add additional cnames to ensure that all the mail clients find the correct ips:


pop3 mailserver.domain.com. CNAME 1800
pop mailserver.domain.com. CNAME 1800
imap mailserver.domain.com. CNAME 1800
smtp mailserver.domain.com. CNAME 1800

And the AAAA records too if you want to support IPv6.

Create a MX record for your domain and subdomains


@ mailserver.domain.com. MX 10 3600

"@" is an alias for you domain. So all email for your domain should be sent to "mailserver.domain.com"

You have to create records for your subdomains too:


mysubdomain mailserver.mydomain.com. MX 10 3600

And to ensure that SPF gets more support add this TXT DNS entry too:


@ IN TXT "v=spf1 mx -all"

This adds the additional security that you say that only your MX entries are allowed to send emails for your domains. Quite obvious but you can add other ips too:


@ IN TXT "v=spf1 mx ip4:11.22.33.44 a:mail.company.com -all"

This states that all MX servers, the IP 11.22.33.44 and the mail.company.com are allowed to send emails for your domains.

Why? Because sometimes (e.g. for forums/mailing lists) an external company is sending emails for your domains. This is a way to approve them.

2. Setup your rDNS:
Go to your vps control panel and add the DNS record "mailserver.domain.com" to your IP address 55.55.55.55.

3. Setup your mailname in /etc


sudo nano /etc/mailname
Add "mailserver.mydomain.com"

4. Setup your iptables rules
You find the rules here. But I add them here too:


# allow SMTP
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 25 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 25 -j ACCEPT

# allow SMTPS
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 465 -j ACCEPT
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 587 -j ACCEPT

ip6tables -A INPUT -i $device -p tcp --dport 465 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 587 -j ACCEPT

# allow POP3
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 110 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 110 -j ACCEPT

# allow POP3S
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 995 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 995 -j ACCEPT

# allow IMAP
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 143 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 143 -j ACCEPT

# allow IMAPS
iptables -A INPUT -i $device -m state --state NEW -p tcp --dport 993 -j ACCEPT
ip6tables -A INPUT -i $device -p tcp --dport 993 -j ACCEPT

If you run this server with SSL certs you can disable the POP3 and IMAP rules.

A lot of email clients first try the non-SSL ports and will therefore suggest a not secured connection.

Keep in mind that all communication without SSL is not save.

5. Setup your mail server
Installation is really simple because of the great package: dovecot-postfix


sudo apt-get install dovecot-postfix

After using some other mail daemons for a while I do prefer the postfix/dovecot combo.

Postfix is mail daemon like sendmail but whith a real nice pipe framework. It is really easy to tunnel email through different modules. Dovecot is a daemon that provides pop3 and imap access to the mail accounts.

6. Config dovecot
Dovecot is providing access to your emails via pop/imap.
Only file to edit is /etc/dovecot/conf.d/10-auth.conf


nano /etc/dovecot/conf.d/10-auth.conf

Remove the # of the line "disable_plaintext_auth = yes"

7. Config postfix
Edit the file /etc/postfix/main.cf


nano /etc/postfix/main.cf

Things to edit:


mydomain = domain.com
myorigin = domain.com
myhostname = mailserver.domain.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
local_recipient_maps = $alias_maps
mydestination = domain.com, mailserver.domain.com, subdomain.domain.com, localhost

smtpd_recipient_restrictions =
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_unauth_pipelining,
permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination,
reject_rbl_client zen.spamhaus.org,
reject_rbl_client bl.spamcop.org

smtpd_data_restrictions =
reject_unauth_pipelining,
reject_multi_recipient_bounce,
permit

So what I am doning here?

I define the domain and origin of the mail server. He should handle the domain "domain.com".

I define an alias map to map different email accounts to different linux users.

And I define all allowed destinations - including all subdomains. And of course "localhost" for all my scripts.

All mail for a different domain will be rejected.

"smtpd_recipient_restrictions" is a list of filters to ensure that we did not get spammed.

"reject_rbl_client" is a referrer to one of the spam lists provided by different groups. I do like spamhaus and spamcop.

"check_policy_service" is used for my favorit greylister. Greylister do something very bad. They dismiss emails - out of the fact that real and good mail servers will try again. A lot of spammers don't have time to wait for any retries.

I know that this is ... still in discussion ... but for a private mail server it just saves a lot of time.

Another time saver is local_recipient_maps. So every email like "[email protected]" will be rejected if the mail address is not within the alias map. You do only receive emails to mailboxes you added.

Edit the file /etc/postfix/master.cf


nano /etc/postfix/master.cf

Things to edit:


submission inet n - - - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATING

dovecot unix - n n - - pipe
flags=DRhu argv=/usr/lib/dovecot/deliver -f ${sender} -d ${recipient}

Be carefull ... spaces etc. do count!

So what did I change?

I have added an output pipe for spam assassin and one for dovecot.

Postfix is recieving all mails and has to forward them to dovecot. Additionally a content_filter is set (defined at the end of the file) to ensure that the mails are sent to spamassassin and afterwards to dovecot.

8. Config aliases

I do prefer aliases instead of virtual mappings. I don't want to run a MySQL server to choose who is getting what email. There are a lot of tutorial on how to use virtual mappings. I stick to simple config files.

There are some DDOS attacks too that aim at a high load of MySQL querries too.

Now edit the file /etc/aliases


nano /etc/aliases

Content:


#
# Mail aliases for sendmail
#
# You must run newaliases(1) after making changes to this file.
#

# Required aliases
postmaster: wlanboy
MAILER-DAEMON: postmaster

# Common aliases
abuse: postmaster
spam: postmaster

# Other aliases
webmaster: wlanboy
contact: wlanboy
root: wlanboy
user1: user1
wlanboy: wlanboy

Double check that you do not map circles like: postmaster -> spam -> wlanboy -> admin -> admin -> postmaster.

You wont have any chance to see the cause in the logfiles.

On the left side are email addresses like "[email protected]".

On the rigth side there are linux users which will receive the emails.

Afterwards you have to run "newaliases" to generate the alias map file.


sudo newaliases

To add a new user just type:


sudo useradd -m -s /bin/false [username]
passwd [username]

This ensures that this user can only login into your mail server and not use any other services like ssh/scp/rsync.

9. OPTIONAL: Install postgrey


sudo apt-get install postgrey
sudo nano /etc/default/postgrey
add:
POSTGREY_OPTS="--inet=127.0.0.1:10023 --delay=55"

The OPTS I add are:

• listening to localhost only
• add a delay of 55 seconds

Afterwards you have to edit the file /etc/postfix/main.cf
 


nano /etc/postfix/main.cf

Add the check_policy_service inet:127.0.0.1:10023 to the smtpd_recipient_restrictions.

10. OPTIONAL: Install spam assassine

Enhance postfix configuration:


nano /etc/postfix/master.cf

Things to edit:


smtp inet n - - - - smtpd
-o content_filter=spamassassin

submission [.....]
dovecot [.....]

spamassassin unix - n n - - pipe
user=spamd argv=/usr/bin/spamc -f -e
/usr/sbin/sendmail -oi -f ${sender} ${recipient}

And now install spam assassine


sudo apt-get install libnet-dns-perl pyzor razor libdigest-sha-perl libencode-detect-perl libdbi-perl libgeo-ipfree-perl libnet-ident-perl
sudo apt-get install spamassassin
sudo adduser --system --no-create-home spamd

Afterwards you have to activate it:


sudo nano /etc/default/spamassassin
change to: ENABLED=1
sudo service spamassassin restart

Update spam asssasin rules:


cd /etc/spamassassin/
wget http://yerp.org/rules/GPG.KEY
sa-update --import GPG.KEY
sa-update --gpgkey 6C6191E3 --channel sought.rules.yerp.org

sa-update -D -v

11. Install fail2ban


sudo apt-get install fail2ban

You have to configure the services fail2ban has to check:


sudo nano /etc/fail2ban/jail.conf

Things to edit:


bantime = 3600
maxretry = 2

action = %(action_mw)s

[pam-generic]
enabled = true

[postfix]
enabled = true
port = smtp,ssmtp
filter = postfix
logpath = /var/log/mail.log

[sasl]
enabled = true
port = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter = sasl
# You might consider monitoring /var/log/mail.warn instead if you are
# running postfix since it would provide the same log lines at the
# "warn" level but overall at the smaller filesize.
logpath = /var/log/mail.log

[dovecot]
enabled = true
port = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter = dovecot
logpath = /var/log/mail.log

What did I do?

Set the bantime to 1 hour and the number of retries before ban to 2.

And enabled the observation of pam-generic, postfix, sasl and dovecot.

So all mail related login actions are checked.

12. OPTIONAL: Own SSL configuration

The package is creating self signed certificates. So if you want to change them because you want to use official ssl certs edit following lines:

• /etc/postfix/main.cf
• /etc/dovecot/conf.d/10-ssl.conf
• /etc/dovecot/conf.d/01-mail-stack-delivery.conf

/etc/postfix/main.cf:smtpd_tls_cert_file = /etc/ssl/certs/ssl-mail.pem
/etc/postfix/main.cf:smtpd_tls_key_file = /etc/ssl/private/ssl-mail.key

/etc/dovecot/conf.d/10-ssl.conf:ssl_cert = </etc/ssl/certs/dovecot.pem
/etc/dovecot/conf.d/10-ssl.conf:ssl_key = </etc/ssl/private/dovecot.pem
/etc/dovecot/conf.d/01-mail-stack-delivery.conf:ssl_cert = </etc/ssl/certs/ssl-mail.pem
/etc/dovecot/conf.d/01-mail-stack-delivery.conf:ssl_key = </etc/ssl/private/ssl-mail.key

13. OPTIONAL: Set rate limits

If your mail server is used by yourself ... you do not need to limit the number of emails a user can send.

Edit /etc/postfix/main.cf


sudo nano /etc/postfix/main.cf

And add these lines at the end of file:


smtpd_client_event_limit_exceptions = $mynetworks
#Clients that are excluded from connection count
anvil_rate_time_unit = 60s
#The time unit over which client connection rates and other rates are calculated.
anvil_status_update_time = 120s
#How frequently the server logs peak usage information.
smtpd_client_message_rate_limit=5
#The maximal number of message delivery requests that any client is allowed to make to this service per time unit.

So each client - not connected through $mynetworks - is only able to send 5 emails per 60 seconds.

14. OPTIONAL: DKIM

Well ....

DomainKeys Identified Mail (DKIM) is a method for E-mail authentication, allowing a person who receives email to verify that the message actually comes from the domain that it claims to have come from. The need for this type of authentication arises because spam often has forged headers.

Click to expand...

So your mail server can sign your emails to ensure that someone can check if the emails are from your approved mail servers.

Installation is quite easy:


sudo apt-get install opendkim opendkim-tools

For 12.04 you have to use backports:


sudo apt-get install opendkim/precise-backports
sudo apt-get install opendkim-tools/precise-backports

Configuration is done on two files:


/etc/opendkim.conf
/etc/default/opendkim
Things you have to change:


nano /etc/opendkim.conf
UserID 105 # 'id postfix' in your shell
Domain domain.com
KeyFile /etc/mail/dkim.key

Now we have to tell postfix to use this service:


nano /etc/postfix/main.cf
# DKIM
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

To generate the key run following command:


cd /etc/mail
sudo opendkim-genkey -t -s mail -d domain.com
cp mail.private /etc/mail/dkim.key

The DNS TXT entry should be a copy&paste of mail.txt


mail._domainkey.domain.com. IN TXT "v=DKIM1; g=*; k=rsa; p=openssl_public_key";

15. OPTIONAL: Add backup MX

First of all you have to add an additional MX record with a higher priority:


@ mailserver.domain.com. MX 10 3600
@ backupmailserver.domain.com. MX 20 3600

Everyone is first trying to send the email to mailserver.domain.com, if it is not reachable backupmailserver.domain.com is used.

The higher the priority the lower the chance that someone is using the MX server.

Next change of the backup mail server is the main.cf:


relay_domains = $mydestination, hash:/etc/postfix/relay_domains
transport_maps = hash:/etc/postfix/relay_transport

Now we have to define the relay domains:


nano /etc/postfix/relay_domains

Content:


domain1.com anything
domain2.com anything
domain3.com anything

A backup MX can be responsible for more than one domain.

Now we have to define what the backup mail server should do with an incoming email:


nano /etc/postfix/relay_transport

Conent:


domain1.com relay:mailserver.domain1.com
domain2.com relay:mailserver.domain2.com
domaint3.com relay:mailserver.domain3.com

Just forward them to the real mail servers.

Last step is to map both files:


cd /etc/postfix
postmap relay_domains
postmap relay_transport

If you want you can add a time to live for the emails too:


maximal_queue_lifetime = 60d

So the backup server will store the mails for 60 days. Hopefully your main mail server will not be offline for more than 60 days.

16. Restart you vps

Done.

Comments:

We now have a smtp/pop3/imap server that uses graylisting, spam assassine and a white list of mail addresses to ensure that you only receive mails you want.

Additionally fail2ban bans everyone trying to get access to your mail server.

Postfix and dovecot are by default supporting IPv6. You only have to add the AAAA records to ensure you mail server is accessable via IPv6.

You can even decide to drop the iptables/ip6tables rules for SMTP/POP3/IMAP to ensure every client is only using SSL secured connections.

If you need a GUI for this mail server:

Use a second vps with webserver and php and install roundcube.

Additional notes:

Yup you are right there is not a fancy clicky GUI to add new mailboxes. But I like the idea to keep things simple.

Adding a user to a linux system (one without a console access) is dead simple. Adding an alias for him too.

This is a private server - so you will not add new users every minute.

A lot of things a file based so the whole system is not using a lot of resources. A real low end mail server running on a 128 MB vps:


free
total used free shared buffers cached
Mem: 131072 82560 48512 0 0 28564
-/+ buffers/cache: 53996 77076
Swap: 131072 15808 115264

This is the state without any active connections. Dovecot is using some RAM for each logged-in user.

This tutorial is quite long and as you can see most of the steps is about securing your mail server. Mail servers are still targets of quite a lot of attacks. These attacks are simple and don't need a lot of time of traffic. So an easy target. The ports cannot be changed too so you know that port 25 is listening.

So a lot of arguments to run your mail server on another vps.

Multiple Domain instructions are here: http://vpsboard.com/topic/1506-running-your-own-mail-server/page-2#entry28355

Wow, there goes my upcoming weekend  You are most excellent @wlanboy!

Someone give this tutorial a spin and see how it goes.  

buffalooed said:

Wow, there goes my upcoming weekend  You are most excellent @wlanboy!

Someone give this tutorial a spin and see how it goes.  

Click to expand...

I'll tinker with it this weekend. Got an idle VPS sitting around.

I hesitated for weeks if I should write this tutorial.

You can get a lot of trouble if the security of your mail server breaks. You can loose your mails too if you do not run backups.

I am running my mail server for about 1 1/2 years. Everything is fine - but mabe this is based on luck.

wlanboy said:

You can get a lot of trouble if the security of your mail server breaks. You can loose your mails too if you do not run backups.

Click to expand...

I'd rather have security issues and something break than continue to feed the monster systems directly and willingly.

Hopefully, others chime in with recommendations to build upon this tutorial.

buffalooed said:

I'd rather have security issues and something break than continue to feed the monster systems directly and willingly.

Click to expand...

Totally agree. Same reason why I am running my own mail server.

You are surely aware of that fact that you are feeding the monsters already by sending and receiving unencrypted eMail alone. But I guess you meant the corporation monsters and not the government monsters... However, I was was always looking for a good step-by-step tutorial for setting up my own mailserver (without having to use a panel) and I am VERY thankful for your posting, wlanboy!

Like others already said: There goes my weekend! ;-)

Amitz said:

You are surely aware of that fact that you are feeding the monsters already by sending and receiving unencrypted eMail alone

Click to expand...

Oh no doubt.  

But, I use PGP to encrypt my real emails with some people.  That's when I actually use email.  

I'll be interested in seeing how your install goes @Amitz.  Everyone report back when they get this running or don't.

Amitz said:

You are surely aware of that fact that you are feeding the monsters already by sending and receiving unencrypted eMail alone.

Click to expand...

Yup, but encryption should not be on server side! It should be client based. Even if someone is taking the whole disk or tracing tcp packets- they cannot read anything without the GnuPG keys on my local disk.

buffalooed said:

But, I use PGP to encrypt my real emails with some people.  That's when I actually use email.  

Click to expand...

Me too.

Looking to the RAM usage per login:


23957 ? S 0:00 0 0 5180 2528 1.9 dovecot/imap-login
23958 ? S 0:00 0 0 4412 2212 1.6 dovecot/imap
23960 ? S 0:00 0 0 5172 2536 1.9 dovecot/imap-login
23961 ? S 0:00 0 0 4292 2080 1.5 dovecot/imap

5 MB of RAM per user should be sufficient.

Thank you wlanboy. Do you know how to limit the sending rate? I'd like to set up a send-only server for multi users.

buffalooed said:

Everyone report back when they get this running or don't.

Click to expand...

After buying a new vps from ramnode (NL) I have setup my brand new mail server for my domain wlanboy.com (was once hosted at hotmail).

You can test my mail server by sending an email to [test][at][wlanboy].[com]

Good tutorial! I'm about to repurpose another VPS so will use this instead of an utosetup with a panel and see how it goes... I'll be using Centos, but config files are config files.

tdc-adm said:

Thank you wlanboy. Do you know how to limit the sending rate? I'd like to set up a send-only server for multi users.

Click to expand...

This is done in /etc/postfix/main.cf

Anyone interested in server side virus scanner?

Server side virus scanner for the email?  Well, good for the WIndozer users I guess.   I don't need it personally

Good howto!

buffalooed said:

Server side virus scanner for the email?  Well, good for the WIndozer users I guess.   I don't need it personally

Click to expand...

Its good for spam reduction Those spammers be always trying to target the lowest denominator.

But seriously good howto, but I think I will stick with Zimbra. Never been willing to accept the risk myself. It would be nice to see integrated backup into the tutorial (e.g to Backupsy), I hate to say it but its a must in this day and age.

Added SPF and DKIM to the tutorials. Later just for the sake of having it.

Excellent tutorial / walk through wlanboy! Though i use iredmail this style is my preferd for low ram use / leb's

thanks, been waiting for such tutorial