Email Protocols

// Chapter 1

The Oldest Living Protocol

Email delivery involves three protocols working together: SMTP for sending and relaying, IMAP for synchronized multi-device access, and POP3 for legacy download-and-delete access. The security stack — SPF, DKIM, and DMARC — was retrofitted decades later to address the authentication problems SMTP's designers never anticipated.

// Chapter 2

SMTP: The Sending Protocol

SMTP (Simple Mail Transfer Protocol) is a text-based, session-oriented protocol for sending and relaying email. Client and server exchange commands and multi-digit response codes in a defined dialogue. Every SMTP response code has three digits: first digit indicates class (2xx success, 3xx intermediate, 4xx transient failure, 5xx permanent failure).

Port Architecture

• Port 25: MTA-to-MTA server relay. No authentication required. Blocked by residential ISPs and most cloud providers to prevent botnet spam. Direct submission to port 25 is not permitted from most IP ranges.

• Port 587: Mail submission with required AUTH. This is the correct port for all application email sending. Always uses STARTTLS for TLS negotiation.

• Port 465: Legacy implicit TLS (SMTPS). The original SSL port, briefly deprecated then restored by RFC 8314. Some providers use this instead of 587 for implicit TLS.

4xx vs 5xx: Retry vs Bounce

The 4xx / 5xx distinction is operationally critical. A 4xx response (transient failure) means: "I can't accept this right now — try again later." The sending MTA queues the message and retries with exponential backoff, typically for 4–5 days before generating a bounce notification (NDR). A 5xx response (permanent failure) means: "this will never succeed." The message is immediately bounced back to the sender. Sending to a non-existent address returns 5xx; a temporarily overloaded server returns 4xx.

// Chapter 3

Email Delivery Architecture

Mail Agent Roles

• MUA (Mail User Agent): Client — Outlook, Thunderbird, Apple Mail, Gmail. Submits via SMTP/587, retrieves via IMAP/993.

• MSA (Mail Submission Agent): Receives from MUAs (port 587), enforces policies, signs DKIM, forwards to MTA.

• MTA (Mail Transfer Agent): Routes between servers (port 25). Examples: Postfix, Exim, Sendmail, Exchange.

• MDA (Mail Delivery Agent): Delivers to local mailboxes. Examples: Dovecot, procmail.

• MRA (Mail Retrieval Agent): IMAP/POP3 server serving client requests. Often same as MDA.

MX Record Routing

When delivering to user@other.com, the sending MTA queries DNS for MX records at other.com. MX records have priority values — lower number is preferred. If the primary MX is unavailable, the MTA tries lower-priority alternatives. If all are unreachable, the message is queued and retried. After the queue lifetime (typically 4–5 days), an NDR (Non-Delivery Receipt) is sent to the original sender with the failure reason. Never return a 5xx to a message you intend to deliver later — use 4xx for transient issues.

// Chapter 4

IMAP: Synchronized Multi-Device Access

IMAP (Internet Message Access Protocol, RFC 3501) keeps messages on the server. Clients synchronize state — read/unread flags, folder structure, deleted messages — rather than downloading and removing from the server. Every client (phone, laptop, web UI) always sees the same mailbox state because the server is the single source of truth.

IMAP Commands and State Machine

IMAP connections progress through states: Not Authenticated → Authenticated (after LOGIN or AUTHENTICATE) → Selected (after SELECT mailboxname). Commands are tagged with client-assigned identifiers (A001, A002...) — the server's response references each tag, enabling pipelining. A client can send multiple commands without waiting for each response.

Critical commands: SELECT INBOX (open folder, returns message count and recent flags), SEARCH UNSEEN (server-side search returning message IDs), FETCH 1:* ENVELOPE (bulk fetch headers without body), STORE +FLAGS (\Seen) (mark as read), EXPUNGE (actually delete \Deleted messages).

IMAP IDLE: Push Notifications

The IDLE command keeps the IMAP connection open in a waiting state. When new mail arrives, the server sends an unsolicited EXISTS response — the client immediately receives the notification without polling. This is the mechanism behind "push email" on mobile devices. The client sends IDLE\r\n, server responds + idling, and both sides wait. Client terminates with DONE\r\n. Apple's iOS and Android both use IMAP IDLE for email push notification.

# IMAP session (manual telnet example)
openssl s_client -connect imap.gmail.com:993

# After TLS handshake:
A001 LOGIN user@gmail.com apppassword
A002 LIST "" "*"                    # list all folders
A003 SELECT INBOX                   # open inbox
A004 SEARCH UNSEEN                  # find unread messages
A005 FETCH 1 BODY[HEADER.FIELDS (FROM SUBJECT DATE)]
A006 FETCH 1 BODY[TEXT]             # fetch body
A007 STORE 1 +FLAGS (Seen)         # mark as read
A008 LOGOUT

# Check SMTP queue and delivery (Postfix)
mailq                      # show queued messages
postqueue -f               # flush queue
postcat -q <QUEUEID>       # inspect message

// Chapter 5

Email Security: SPF, DKIM, DMARC

DMARC Alignment: The Critical Concept

DMARC alignment is what ties the three mechanisms together. For a DMARC check to pass, at least one of SPF or DKIM must (1) pass the authentication check AND (2) align — the verified domain must match the visible From: header domain. An attacker could set up evil.com with valid SPF and DKIM, then put From: ceo@legitimate.com in the message header. SPF passes for evil.com; DKIM passes for evil.com — but DMARC alignment fails because evil.com does not match legitimate.com. The DMARC policy (none/quarantine/reject) then determines what the receiving server does.

Two alignment modes: relaxed (default) — subdomain OK (mail.example.com aligns with example.com). strict — exact domain match required. Strict alignment breaks mailing lists and legitimate subdomains, so relaxed is almost always appropriate.

// Chapter 6

DKIM Deep Dive

DKIM (DomainKeys Identified Mail, RFC 6376) adds a cryptographic signature to every outbound message. The signature is stored in a DKIM-Signature: header. The receiving server verifies it using a public key from DNS. The signature covers specified headers plus a hash of the body.

Reading a DKIM-Signature Header

A typical DKIM-Signature: v=1; a=rsa-sha256; d=example.com; s=google2024; h=from:to:subject:date; bh=ABC...==; b=XYZ...==

• a=: signing algorithm (rsa-sha256 or ed25519-sha256)

• d=: signing domain (must align with From: for DMARC)

• s=: selector (key identifier — fetch public key from google2024._domainkey.example.com)

• h=: signed headers list (changing any of these invalidates the signature)

• bh=: SHA-256 hash of the canonicalized body

• b=: the actual RSA/Ed25519 signature

Key Rotation with Selectors

The selector enables zero-downtime key rotation: generate new key pair → publish public key at new selector in DNS → configure signing server to use new private key → wait for old key TTL to expire → remove old DNS record → optionally revoke old key by setting p= empty in the old selector's DNS TXT record (signals "revoked" to receivers). Ed25519 DKIM is preferred for new deployments: 68-character vs 392-character public key, faster verification, no known quantum vulnerability.

// Chapter 7

SMTP TLS: STARTTLS, MTA-STS, DANE

SMTP port 25 between servers can use STARTTLS to upgrade to TLS — but it is opportunistic. If the receiving server doesn't advertise STARTTLS in its EHLO capabilities, the sending server falls back to plaintext. A MITM attacker can trivially strip the STARTTLS capability from the greeting, forcing plaintext delivery. This downgrade attack was widely used by nation-state actors for years.

MTA-STS: Mandatory TLS for Inbound Delivery

MTA-STS (RFC 8461) publishes a policy specifying that TLS is mandatory for delivering to a domain. Setup requires two components: (1) a policy file at https://mta-sts.yourdomain.com/.well-known/mta-sts.txt listing required MX hostnames and mode (enforce/testing/none), and (2) a DNS TXT record at _mta-sts.yourdomain.com with a policy ID. Sending MTAs fetch the policy and refuse to deliver in plaintext or with invalid TLS certificates — they queue instead.

DANE: Certificate Pinning via DNSSEC

DANE (RFC 7672) uses DNSSEC-signed TLSA records to pin TLS certificates for mail servers. Instead of trusting public CAs, the domain owner publishes the certificate (or CA) hash directly in DNS. A DANE-supporting sending MTA fetches the TLSA record and verifies the server's certificate against it — preventing MITM even with a rogue CA certificate. Requires DNSSEC on the target domain; without DNSSEC, the TLSA records themselves could be spoofed.

# MTA-STS policy file
# Host at: https://mta-sts.yourdomain.com/.well-known/mta-sts.txt
version: STSv1
mode: enforce
mx: mail.yourdomain.com
max_age: 86400

# MTA-STS DNS TXT record
# _mta-sts.yourdomain.com TXT "v=STSv1; id=20241201"

# DANE TLSA record format: usage selector matching-type hash
# _25._tcp.mail.yourdomain.com TLSA 3 1 1 <SHA256-of-SPKI>
# Generate hash:
openssl x509 -in cert.pem -pubkey -noout | openssl pkey -pubin -outform DER | openssl dgst -sha256 -binary | xxd -p -c 256

# Test SMTP TLS
openssl s_client -connect mail.yourdomain.com:25 -starttls smtp
# Check for: Verify return code: 0 (ok)

// Chapter 8

Email Headers: The Audit Trail

Email headers record a complete delivery history. Every SMTP server that handles a message prepends a Received: header. Reading headers bottom-up traces the path from sender to recipient. Each Received header includes the sending server's claimed identity, the receiving server's identity, the protocol used, a message queue ID, and a timestamp.

RFC 5321 Envelope vs RFC 5322 Message Headers

This distinction is the root cause of most email spoofing. RFC 5321 envelope headers: MAIL FROM and RCPT TO — used by SMTP servers for routing, never displayed to users. SPF checks the MAIL FROM domain against the sending IP. RFC 5322 message headers: From:, To:, Subject:, Date:, CC: — what the user sees in their email client. DMARC checks whether the authenticated RFC 5321 domain aligns with the RFC 5322 From: domain.

An attacker sends MAIL FROM: attacker@evil.com (authorized by SPF for evil.com) but writes From: ceo@legitimate.com in the message. The user sees the CEO's address. SPF passes for evil.com. DKIM passes for evil.com. DMARC alignment fails — evil.com does not match legitimate.com. Only DMARC catches this attack. This is why DMARC p=reject is the goal.

Authentication-Results Header

The final receiving server adds Authentication-Results: recording SPF, DKIM, and DMARC pass/fail. In Gmail: open the email → three-dot menu → "Show original" — the full headers are displayed. The Authentication-Results header at the top (prepended last) is the authoritative result from Gmail's infrastructure.

// Chapter 9

Deliverability: Getting Legitimate Email Delivered

High-volume senders (transactional email, marketing) must actively manage deliverability — the probability that a legitimate email reaches the inbox rather than spam folder or being blocked entirely.

IP Warm-Up

New IP addresses have no sending reputation. Major ISPs (Gmail, Outlook) rate-limit mail from unknown IPs. A new server going from 0 to 1 million emails per day will immediately trigger rate limits and spam filtering. Warm-up process: start with 1,000 emails/day to the most engaged users, double every 2–3 days, monitor bounce rates and complaint rates, reach full volume after 4–8 weeks. Services like SendGrid and Mailgun manage warm-up automatically for dedicated IPs.

Bounce and Complaint Management

Hard bounces (5xx — address doesn't exist) must be immediately removed from your list. Sending to known-bad addresses signals poor list hygiene and damages reputation. Soft bounces (4xx — temporary failure) should be retried a few times then removed. Spam complaints from recipients must be processed via Feedback Loop (FBL) subscriptions — most major ISPs provide FBL lists that notify senders when recipients mark their mail as spam. High complaint rates (above 0.1%) trigger filtering.

// Chapter 10

MIME: Multipart Messages and Attachments

MIME (Multipurpose Internet Mail Extensions, RFC 2045–2049) extends SMTP to support non-ASCII content, HTML email, attachments, and multipart messages. The base SMTP protocol supports only 7-bit ASCII text.

A typical HTML email has a multipart/alternative body: two parts — text/plain (plain text fallback) and text/html (HTML version). Email clients display whichever part they prefer. An email with an attachment is multipart/mixed: HTML body + one attachment. Attachments are base64-encoded within the MIME structure.

Key MIME headers: Content-Type: multipart/mixed; boundary="abc123" defines the container and the separator string. Each part begins with --abc123 and ends with --abc123-- for the final part. Content-Transfer-Encoding: base64 signals the content is base64 encoded. Content-Disposition: attachment; filename="file.pdf" signals this part is an attachment with the given filename.

# Send multipart HTML + plain text email (Python)
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

msg = MIMEMultipart('alternative')
msg['From'] = 'sender@example.com'
msg['To']   = 'recipient@example.com'
msg['Subject'] = 'Hello'

plain = MIMEText('Hello, this is plain text.', 'plain', 'utf-8')
html  = MIMEText('<h1>Hello</h1><p>This is HTML.</p>', 'html', 'utf-8')
msg.attach(plain)
msg.attach(html)

with smtplib.SMTP('smtp.example.com', 587) as s:
    s.starttls()
    s.login('user', 'password')
    s.sendmail(msg['From'], msg['To'], msg.as_string())

// Chapter 11

Email Attack Patterns

Business Email Compromise (BEC)

The attacker spoofs or compromises a CEO/CFO/vendor email address and sends a request to an employee to transfer funds, purchase gift cards, or share credentials. BEC generated $2.9B in losses in 2023 (FBI IC3 report). Defenses: DMARC p=reject prevents external spoofing of your domain. MFA prevents account compromise. Financial approval workflows requiring out-of-band verification for large transfers prevent social engineering. Employee training to recognize urgency + financial request patterns is also essential.

Phishing and Spear-Phishing

Mass phishing: bulk campaigns impersonating banks, package carriers, Netflix. Spear-phishing: targeted attacks using personal details (name, company, role) to increase credibility. Email-based malware: malicious attachments (PDF with embedded JavaScript, Office macros, ISO files containing executables). URL-based: links to credential-harvesting sites. Defenses: email gateway scanning, URL rewriting/click tracking, attachment sandboxing, and user training.

SMTP Smuggling

A 2023-discovered attack where different SMTP servers interpret end-of-DATA sequences differently. An attacker embeds a second SMTP transaction within the first message's DATA section using carefully crafted line endings. The receiving server processes two separate messages where the outer server saw only one — the inner message bypasses SPF/DKIM/DMARC checks and potentially bypasses spam filtering. Postfix, Exim, and Sendmail all issued patches to normalize SMTP line ending handling. Ensure your MTA is patched and configured to reject non-standard DATA terminators.

// Chapter 12

Email Forensics: Header Analysis

# Phishing email headers — analyze delivery path
# Read Received: headers BOTTOM-UP for delivery path

Received: from mail.attacker.net (attacker.net [198.51.100.42])
        by mx.victim.com with ESMTP id abc123       ← final hop (top)
        for <ceo@victim.com>; Mon 10:00:00 +0000
Received: from localhost ([127.0.0.1])
        by mail.attacker.net with SMTP id def456    ← attacker's server
        Mon 09:59:55 +0000

From: cfo@trusted-bank.com                          ← spoofed From:
Reply-To: attacker@evil.com                         ← real reply address
To: ceo@victim.com
Subject: Wire Transfer Request

Authentication-Results: mx.victim.com;
  spf=fail smtp.mailfrom=attacker.net               ← SPF FAIL
  dkim=none                                         ← no DKIM
  dmarc=fail action=none header.from=trusted-bank.com

# Key indicators:
# 1. Received: from attacker.net — bad IP
# 2. SPF fail for attacker.net
# 3. No DKIM signature
# 4. DMARC fail — From: domain doesn't align
# 5. Reply-To differs from From: — very suspicious

// Chapter 13

Common Misconceptions

// Chapter 14

IQ Depth Check