Running Your Own Email Server on a VPS: Postfix and Dovecot Guide

Self-hosting your email gives you complete control over your data, eliminates per-user licensing costs, and removes dependency on third-party providers who may scan your messages for advertising purposes. A VPS with full root access, a dedicated IP address, and reliable uptime is the ideal platform for running a production email server. This guide walks through every step of setting up Postfix for SMTP delivery and Dovecot for IMAP mailbox access on a Linux VPS, including SSL/TLS encryption, DNS authentication records, spam filtering, and firewall hardening.

Prerequisites

Before you begin, ensure you have the following:

• A Linux VPS running Ubuntu 22.04 LTS or later with at least 2 GB RAM and 20 GB NVMe storage
• A registered domain name (e.g., example.com) with access to DNS records
• A dedicated IPv4 address that is not on any major email blacklists (check at mxtoolbox.com)
• Root or sudo access to the VPS
• A valid FQDN (Fully Qualified Domain Name) configured as the server's hostname, e.g., mail.example.com

Important: Email deliverability depends heavily on IP reputation. A clean IP address from a reputable hosting provider like MassiveGRID significantly improves your chances of landing in recipients' inboxes rather than spam folders.

Step 1: Configure DNS Records

Proper DNS configuration is essential for email delivery and authentication. Before installing any software, set up the following DNS records for your domain:

The PTR (reverse DNS) record must be configured through your hosting provider's control panel. On MassiveGRID, you can set the rDNS record directly from the client portal. This record is critical because many receiving mail servers reject email from IP addresses whose PTR record does not match the server's FQDN.

Step 2: Set the Server Hostname

Your server's hostname must match the FQDN used in your MX and PTR records:

sudo hostnamectl set-hostname mail.example.com

Verify the hostname is correctly set:

hostname -f
# Output: mail.example.com

Edit /etc/hosts to include the FQDN mapping:

127.0.0.1   localhost
YOUR_SERVER_IP   mail.example.com   mail

Step 3: Install Postfix and Dovecot

Update the system packages and install the required software:

sudo apt update && sudo apt upgrade -y
sudo apt install -y postfix dovecot-core dovecot-imapd dovecot-lmtpd

During the Postfix installation, select "Internet Site" when prompted for the mail server configuration type, and enter your domain name (example.com) as the system mail name.

Step 4: Obtain SSL/TLS Certificates

Encrypted connections are mandatory for modern email. Use Let's Encrypt to obtain free TLS certificates:

sudo apt install -y certbot
sudo certbot certonly --standalone -d mail.example.com

The certificates will be saved to /etc/letsencrypt/live/mail.example.com/. Set up automatic renewal:

sudo certbot renew --dry-run

Add a cron job to restart Postfix and Dovecot after certificate renewal:

echo '0 3 * * * root certbot renew --quiet --post-hook "systemctl restart postfix dovecot"' | sudo tee /etc/cron.d/certbot-mail

Step 5: Configure Postfix (SMTP)

Edit the main Postfix configuration file at /etc/postfix/main.cf. Replace the default contents with the following configuration:

# Basic settings
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
mydestination = $myhostname, $mydomain, localhost
mynetworks = 127.0.0.0/8 [::1]/128

# TLS parameters
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_security_level = may

# SASL authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous

# Restrictions
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination,
    reject_rbl_client zen.spamhaus.org

# Delivery via Dovecot LMTP
mailbox_transport = lmtp:unix:private/dovecot-lmtp

# Message size limit (25 MB)
message_size_limit = 26214400

Enable submission (port 587) for authenticated clients by editing /etc/postfix/master.cf. Uncomment and modify the submission section:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

Step 6: Configure Dovecot (IMAP)

Edit the Dovecot configuration files. First, /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

Configure authentication in /etc/dovecot/conf.d/10-auth.conf:

disable_plaintext_auth = yes
auth_mechanisms = plain login

Set up TLS in /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem
ssl_min_protocol = TLSv1.2

Configure the LMTP socket and SASL authentication for Postfix in /etc/dovecot/conf.d/10-master.conf:

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Step 7: Configure DKIM Signing

DKIM (DomainKeys Identified Mail) adds a cryptographic signature to outgoing emails, proving they were sent from your authorized server. Install OpenDKIM:

sudo apt install -y opendkim opendkim-tools

Generate a DKIM key pair:

sudo mkdir -p /etc/opendkim/keys/example.com
sudo opendkim-genkey -b 2048 -d example.com -D /etc/opendkim/keys/example.com -s mail -v
sudo chown -R opendkim:opendkim /etc/opendkim

Configure OpenDKIM in /etc/opendkim.conf:

Syslog          yes
Domain          example.com
Selector        mail
KeyFile         /etc/opendkim/keys/example.com/mail.private
Socket          inet:8891@localhost
Canonicalization relaxed/simple
Mode            sv

Add the DKIM integration to Postfix by appending to /etc/postfix/main.cf:

# DKIM
milter_protocol = 6
milter_default_action = accept
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Publish the DKIM public key as a DNS TXT record. View the key:

sudo cat /etc/opendkim/keys/example.com/mail.txt

Add the output as a TXT record for mail._domainkey.example.com in your DNS zone.

Step 8: Install SpamAssassin

Protect your inbox from spam with SpamAssassin:

sudo apt install -y spamassassin spamc
sudo systemctl enable spamassassin
sudo systemctl start spamassassin

Integrate SpamAssassin with Postfix by editing /etc/postfix/master.cf. Modify the smtp line and add a SpamAssassin transport:

smtp      inet  n       -       y       -       -       smtpd
  -o content_filter=spamassassin

spamassassin unix  -       n       n       -       -       pipe
  user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Adjust SpamAssassin's scoring thresholds in /etc/spamassassin/local.cf:

required_score 5.0
rewrite_header Subject [SPAM]
report_safe 0
use_bayes 1
bayes_auto_learn 1

Step 9: Configure the Firewall

Open only the ports required for email services and secure everything else:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp     # SSH
sudo ufw allow 25/tcp     # SMTP
sudo ufw allow 587/tcp    # Submission (authenticated SMTP)
sudo ufw allow 993/tcp    # IMAPS (encrypted IMAP)
sudo ufw allow 80/tcp     # HTTP (for Let's Encrypt renewal)
sudo ufw allow 443/tcp    # HTTPS
sudo ufw enable

Do not open port 143 (unencrypted IMAP). All IMAP connections should use port 993 with TLS encryption. The security infrastructure provided by MassiveGRID adds an additional layer of protection with network-level DDoS mitigation that keeps your mail server accessible even under attack.

Step 10: Create Email Accounts and Test

Create a system user for each email account:

sudo adduser --disabled-login --gecos "" user1
sudo passwd user1

Restart all services:

sudo systemctl restart postfix dovecot opendkim spamassassin

Test your configuration by sending a test email and checking the headers for SPF, DKIM, and DMARC results. You can also use external tools like mail-tester.com to score your email deliverability.

VPS Requirements for Email Hosting

Email servers have modest resource requirements but demand consistent availability and clean IP reputation. Here are the recommended specifications:

MassiveGRID's Cloud VPS plans start at $1.99/month and include dedicated IP addresses, NVMe storage, and the high availability architecture that ensures your mail server remains accessible even during hardware failures. The Ceph-replicated storage protects your mailbox data against disk failures, while Proxmox HA ensures automatic failover if the host server encounters issues.

Conclusion

Running your own email server on a VPS gives you complete sovereignty over your communications data, eliminates recurring per-user licensing costs, and provides flexibility that hosted email services cannot match. The combination of Postfix, Dovecot, DKIM, SPF, DMARC, and SpamAssassin creates a production-grade email stack that rivals commercial offerings.

The key to success is choosing a VPS provider with clean IP addresses, reliable uptime, and responsive support. MassiveGRID VPS plans provide the NVMe performance, HA reliability, and global datacenter locations needed to run a mail server that delivers consistently to recipients' inboxes.