Private Email Server

Overview

This project is a fully self-hosted email infrastructure using Postfix as the SMTP server and Dovecot for IMAP/POP3 access, deployed on a VPS with the custom domain tyfsadik.org. The goal was to eliminate reliance on Gmail or Proton while maintaining full deliverability to major providers.

Deliverability is achieved through a complete set of DNS-based email authentication records: DKIM (DomainKeys Identified Mail) signs every outbound message with an RSA key, SPF (Sender Policy Framework) authorizes the server's IP, and DMARC enforces policy alignment and provides aggregate reporting. TLS is enforced on all connections via Let's Encrypt certificates. The configuration scored 10/10 on mail-tester.com and successfully delivers to Gmail, Outlook, and Yahoo inboxes without hitting spam folders.

Architecture

Tech Stack

• Postfix — MTA handling SMTP reception (port 25) and submission (port 587)
• Dovecot — IMAP/POP3 server with Maildir storage and SASL authentication
• OpenDKIM — DKIM signing milter integrated with Postfix
• SPF / DMARC — DNS TXT records for sender authorization and policy enforcement
• Let's Encrypt / Certbot — TLS certificates for SMTP and IMAP
• Linux (Debian 12) — VPS host operating system
• Thunderbird / mobile clients — IMAP clients for reading and sending mail

Build Process

apt install postfix -y
# Select "Internet Site" during setup, set domain to tyfsadik.org

# /etc/postfix/main.cf key settings:
myhostname = mail.tyfsadik.org
mydomain = tyfsadik.org
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, $mydomain
home_mailbox = Maildir/
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.tyfsadik.org/fullchain.pem
smtpd_tls_key_file  = /etc/letsencrypt/live/mail.tyfsadik.org/privkey.pem
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

# Enable submission port in /etc/postfix/master.cf:
submission inet n - y - - smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

systemctl restart postfix

apt install dovecot-core dovecot-imapd -y

# /etc/dovecot/conf.d/10-mail.conf
mail_location = maildir:~/Maildir

# /etc/dovecot/conf.d/10-auth.conf
disable_plaintext_auth = yes
auth_mechanisms = plain login

# /etc/dovecot/conf.d/10-ssl.conf
ssl = required
ssl_cert = 

apt install certbot -y

certbot certonly --standalone \
  -d mail.tyfsadik.org \
  --agree-tos --email [email protected]

# /etc/letsencrypt/renewal-hooks/deploy/restart-mail.sh
#!/bin/bash
systemctl restart postfix dovecot

chmod +x /etc/letsencrypt/renewal-hooks/deploy/restart-mail.sh

# Test renewal
certbot renew --dry-run

apt install opendkim opendkim-tools -y

# Generate key pair
mkdir -p /etc/opendkim/keys/tyfsadik.org
opendkim-genkey -s mail -d tyfsadik.org \
  -D /etc/opendkim/keys/tyfsadik.org/
chown opendkim:opendkim /etc/opendkim/keys/tyfsadik.org/mail.private

# /etc/opendkim.conf (key settings)
Domain                  tyfsadik.org
KeyFile                 /etc/opendkim/keys/tyfsadik.org/mail.private
Selector                mail
Socket                  inet:12301@localhost

# /etc/postfix/main.cf (add milter)
milter_protocol = 6
milter_default_action = accept
smtpd_milters = inet:localhost:12301
non_smtpd_milters = $smtpd_milters

# Publish the DNS TXT record from:
cat /etc/opendkim/keys/tyfsadik.org/mail.txt

systemctl enable opendkim && systemctl restart opendkim postfix

# DNS records to add in domain registrar / DNS panel:

# MX record
tyfsadik.org.      IN  MX  10  mail.tyfsadik.org.

# A record for mail subdomain
mail.tyfsadik.org. IN  A   x.x.x.x

# SPF TXT (authorize only this server's IP)
tyfsadik.org.      IN  TXT "v=spf1 ip4:x.x.x.x -all"

# DKIM TXT (copy public key from mail.txt)
mail._domainkey.tyfsadik.org. IN TXT "v=DKIM1; k=rsa; p=MIIBIj..."

# DMARC TXT (report only first, then reject)
_dmarc.tyfsadik.org. IN TXT "v=DMARC1; p=reject; rua=mailto:[email protected]"

# PTR record (set at VPS provider control panel)
x.x.x.x  -->  mail.tyfsadik.org

apt install swaks -y

# Send a test email via the submission port with auth
swaks --to [email protected] \
  --from [email protected] \
  --server mail.tyfsadik.org:587 \
  --auth LOGIN \
  --auth-user [email protected] \
  --auth-password 'yourpassword' \
  --tls

# Verify DKIM header in received message
# Look for: DKIM-Signature: v=1; a=rsa-sha256; d=tyfsadik.org; s=mail;

# Check mail logs for delivery status
tail -f /var/log/mail.log

Request Flow

Challenges & Solutions

• Mail landing in spam at Gmail: Initial deliveries scored 7/10 on mail-tester.com due to a missing PTR record. The VPS provider's reverse DNS panel was used to set the PTR for the server's IP to mail.tyfsadik.org. After that change, the score reached 10/10 and Gmail inbox delivery was consistent.
• Dovecot SSL certificate permission errors: Postfix could not read the Let's Encrypt private key because its process runs as the postfix user, which lacked read access to the /etc/letsencrypt/live/ directory. Resolved by adding the postfix user to the ssl-cert group and adjusting the cert directory permissions with chmod 750.
• DKIM signing failing silently: Messages were sent without DKIM headers because the milter socket was not accessible to Postfix. Fixed by changing the milter socket from a Unix socket to a TCP socket (inet:12301@localhost) which is accessible regardless of socket file permissions.
• Port 25 blocked by VPS provider: Many VPS providers block outbound port 25 by default to prevent spam. Resolved by submitting a ticket to the provider to unblock port 25 for the server IP after demonstrating legitimate email server configuration.

What I Learned

• SMTP protocol flow: EHLO, STARTTLS, AUTH, DATA, and QUIT sequence
• Email authentication stack: SPF authorization, DKIM cryptographic signing, DMARC policy enforcement
• PTR record importance for SMTP reputation and inbox delivery
• Postfix queue management, delivery logs, and bounce handling
• Dovecot SASL integration with Postfix for submission authentication
• TLS certificate lifecycle management for non-Nginx services using deploy hooks