HTTP and HTTPS

// Chapter 1

The Protocol That Built the Web

HTTP (HyperText Transfer Protocol) is a stateless, request/response, application-layer protocol. A client sends a request with a method, URL, headers, and optional body. A server returns a response with a status code, headers, and optional body. No connection state persists between requests — each carries all information needed to process it independently.

// Chapter 2

HTTP Methods: The Verbs

HTTP methods describe the intent of a request. They are case-sensitive and conventionally uppercase. Two critical properties define method semantics: safe (no observable side effects — reading only) and idempotent (repeating produces the same state as doing it once).

The Core Methods

• GET: Retrieve a resource. Safe + idempotent. No request body. Responses are cacheable by default.

• POST: Submit data to create a resource or trigger an action. Neither safe nor idempotent. Has a request body. Responses not cacheable by default.

• PUT: Replace a resource entirely at the specified URL. Idempotent — two identical PUT requests produce the same final state. Body contains the complete replacement resource.

• PATCH: Partially update a resource. Not necessarily idempotent (depends on patch semantics — "increment counter" is not idempotent; "set name to X" is). Body contains only the changes.

• DELETE: Remove a resource. Idempotent — deleting an already-deleted resource is a no-op (returns 404, not an error in terms of state).

• HEAD: Same as GET but no response body — headers only. Used to check existence or metadata without downloading content.

• OPTIONS: Returns allowed methods for a resource. Foundation of CORS preflight — browsers use it to ask permission before cross-origin requests.

Idempotency in Production Systems

Idempotency is critical for distributed system reliability. If a POST creates an order and the network drops the response, the client doesn't know if it succeeded. Retrying creates a duplicate order. Solutions: use PUT with a client-generated UUID (PUT /orders/uuid123 is idempotent by definition), or implement idempotency keys — send a unique ID in a header, and the server deduplicates. Stripe's API uses Idempotency-Key: uuid headers on all payment endpoints for exactly this purpose.

// Chapter 3

HTTP Messages: Requests and Responses

HTTP/1.1 messages are human-readable text. A request starts with a request line (method + path + version), followed by header lines (name: value), a blank line, and an optional body. A response starts with a status line (version + code + reason phrase), headers, blank line, and optional body. HTTP/2 and HTTP/3 use binary framing for efficiency — identical semantics, better performance.

Content Negotiation

HTTP supports server-driven content negotiation. The client sends Accept (MIME types), Accept-Language, Accept-Encoding (gzip/br/deflate), listing supported formats with preference weights (q-values, 0.0–1.0). The server selects the best match and responds with the corresponding Content-Type, Content-Language, Content-Encoding. The Vary response header tells caches which request headers must match for a cached response to be reused — Vary: Accept-Encoding means separate cache entries for gzip and non-gzip.

Chunked Transfer Encoding

When the server doesn't know the response size in advance (streaming responses, dynamic content), it uses Transfer-Encoding: chunked. Each chunk is prefixed with its hexadecimal size. A zero-size chunk (0\r\n\r\n) terminates the body. This enables streaming responses without buffering the entire body. HTTP/2 makes this obsolete — DATA frames carry length implicitly in the QUIC/H2 frame header.

// Chapter 4

HTTP Status Codes

Status codes are 3-digit integers organized into five classes by their first digit. Clients can safely treat any 2xx as success and any 5xx as server error, even without recognizing the specific code — forward compatibility is built into the design.

The 401 vs 403 Distinction

401 means "who are you? — authenticate first." It must include WWW-Authenticate describing how. 403 means "I know who you are, but you don't have permission." Confusing them breaks clients: browsers show an auth dialog on 401 but not 403. Security-conscious APIs sometimes return 404 for forbidden resources to avoid confirming their existence (resource enumeration prevention) — a deliberate information hiding trade-off.

307 vs 308: Method-Preserving Redirects

Historical 302/301 redirects allowed browsers to silently change POST to GET on redirect — widespread but non-standard behavior. RFC 7238 added 307 (Temporary Redirect) and 308 (Permanent Redirect) which mandate method preservation. A POST to a 307-redirected URL must POST to the new URL, not GET. This matters for API clients submitting data: always use 307/308 when redirecting POST endpoints.

// Chapter 5

HTTP Headers: The Metadata Layer

HTTP headers are case-insensitive name: value pairs that carry metadata about the request or response. Modern HTTP has 200+ defined headers, though most requests use under 20.

Security Headers

• Strict-Transport-Security (HSTS): max-age=31536000; includeSubDomains; preload — browser refuses HTTP for 1 year. Prevents SSL stripping.

• Content-Security-Policy (CSP): Controls which resources may load. script-src 'nonce-abc123' allows only scripts with the matching nonce — mitigates XSS.

• X-Frame-Options: DENY: Prevents iframe embedding — clickjacking mitigation. Superseded by CSP frame-ancestors 'none'.

• X-Content-Type-Options: nosniff: Browser must trust Content-Type, not guess from content bytes. Prevents MIME sniffing attacks.

• Referrer-Policy: Controls how much of the referring URL is sent in the Referer header. strict-origin-when-cross-origin is the modern recommended default.

• Permissions-Policy: Replaces Feature-Policy. Controls access to powerful browser APIs (camera, microphone, geolocation) per origin.

Caching Headers

• Cache-Control: Primary caching directive — supersedes Pragma and Expires.

• ETag: Opaque resource version token. Enables conditional requests via If-None-Match.

• Last-Modified: Resource modification timestamp. Enables conditional requests via If-Modified-Since.

• Vary: Which request headers create distinct cache entries. Vary: Accept-Encoding creates separate entries for gzip vs non-gzip responses.

// Chapter 6

HTTP Caching

Conditional Requests and Validation

When a cached response has expired (max-age elapsed), the browser can make a conditional request — including the ETag or Last-Modified from the cached response. If the resource hasn't changed, the server returns 304 Not Modified with no body — saving bandwidth while confirming freshness. If changed, it returns 200 OK with the new content. This is the "revalidation" that no-cache triggers on every request regardless of age.

// Chapter 7

HTTPS: HTTP over TLS

HTTPS is HTTP running over a TLS connection. All HTTP semantics are identical — methods, headers, status codes, bodies — TLS adds confidentiality (encrypted), integrity (tamper-evident), and authentication (server identity verified via certificate). HTTPS uses port 443 by default (HTTP uses 80).

The Web's HTTPS Transition

In 2010, fewer than 10% of web page loads were HTTPS. By 2024, over 95% are. Forcing factors: Let's Encrypt (free automated TLS certificates since 2016), Chrome marking HTTP as "Not Secure" (2018), Google search ranking penalizing HTTP sites, browser security features (Service Workers, Push, geolocation) restricted to secure origins. The web transitioned to HTTPS in under a decade — one of the fastest security improvements in internet history.

HSTS: HTTP Strict Transport Security

Once a browser receives Strict-Transport-Security: max-age=31536000; includeSubDomains, it refuses to make HTTP connections to that domain for 1 year — even if the user types "http://". This prevents SSL stripping attacks, where an attacker on the network intercepts the initial HTTP request before the HTTPS redirect can occur. HSTS preloading hardcodes your domain in Chrome/Firefox/Safari's built-in list — HTTPS is enforced on first visit, before any HTTP connection is possible.

# HTTP → HTTPS redirect + HSTS (nginx)
server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    ssl_certificate     /etc/ssl/cert.pem;
    ssl_certificate_key /etc/ssl/key.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    add_header X-Content-Type-Options nosniff always;
    add_header X-Frame-Options DENY always;
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'nonce-{$nonce}'" always;
    add_header Referrer-Policy strict-origin-when-cross-origin always;
}

// Chapter 8

CORS: Cross-Origin Resource Sharing

Browsers enforce the Same-Origin Policy: JavaScript from app.example.com cannot make fetch/XHR calls to api.other.com. Origins are defined by scheme + host + port — all three must match. This prevents malicious scripts from silently making authenticated requests to other sites on behalf of users. Modern web architecture requires cross-origin requests, so CORS provides a controlled relaxation mechanism.

Simple vs Preflighted Requests

Simple requests (GET/HEAD, or POST with only basic headers): browser sends the request with an Origin header. Server responds with Access-Control-Allow-Origin. If they match (or server returns *), browser allows the response. No extra round trip.

Preflighted requests (PUT/PATCH/DELETE, custom headers, JSON Content-Type): browser sends an OPTIONS preflight first. Server responds with CORS permission headers. If approved, browser sends the real request. One additional RTT per non-simple cross-origin request — Access-Control-Max-Age caches the preflight to amortize this cost.

# CORS middleware (Express.js)
app.use((req, res, next) => {
  res.header('Access-Control-Allow-Origin', 'https://app.example.com')
  res.header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS')
  res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization')
  res.header('Access-Control-Max-Age', '86400')   // cache preflight 24h
  if (req.method === 'OPTIONS') return res.sendStatus(204)
  next()
})

# Test CORS preflight:
curl -X OPTIONS https://api.example.com/users \
  -H "Origin: https://app.example.com" \
  -H "Access-Control-Request-Method: DELETE" \
  -H "Access-Control-Request-Headers: Authorization" -v

// Chapter 9

Cookies and Session Management

HTTP is stateless — each request is independent. Cookies are the primary mechanism for maintaining state across requests. The server sets a cookie with Set-Cookie; the browser stores it and sends it automatically with every subsequent request to that origin.

Cookie Security Attributes

• Secure: Cookie sent only over HTTPS. Essential for authentication cookies on any production system.

• HttpOnly: JavaScript cannot read this cookie (document.cookie excluded). Prevents XSS-based session theft — the most important security attribute for session cookies.

• SameSite=Lax: Cookie sent in top-level navigations (GET) but not in cross-site AJAX/fetch POST requests. Prevents CSRF while maintaining SSO compatibility. The browser default since Chrome 80.

• SameSite=Strict: Cookie never sent in any cross-site context. Maximum CSRF protection but breaks most OAuth/SSO flows.

• SameSite=None; Secure: Cookie sent in all contexts including third-party. Required for cross-site cookies (embedded widgets, OAuth redirects). Must be Secure.

Modern secure session cookie: Set-Cookie: session=abc; Secure; HttpOnly; SameSite=Lax; Path=/; Max-Age=86400.

// Chapter 10

HTTP Performance

Content Compression

The client announces compression support with Accept-Encoding: gzip, br. The server compresses the body and responds with Content-Encoding: gzip (or br for Brotli). Compression reduces text body size by 60–90%. Brotli achieves 15–25% better compression than gzip for web content (uses a static dictionary optimized for HTTP). Both achieve decompression speeds of hundreds of MB/s — the CPU cost is negligible compared to the bandwidth savings.

Persistent Connections (Keep-Alive)

HTTP/1.0 opened a new TCP connection for every request — SYN + TLS handshake overhead per request. HTTP/1.1 defaults to persistent connections — the same TCP+TLS connection serves multiple sequential requests. Connection: close signals teardown after the response. HTTP/2 takes this further — multiple concurrent requests over one TCP connection. HTTP/3 over QUIC multiplexes streams with no HoL blocking.

Preloading and Early Hints (103)

HTTP 103 Early Hints allows the server to send response headers before it has finished generating the full response. The browser can start fetching linked CSS/JS resources while the server is still computing the HTML. A 103 response with Link: preload headers fires before the 200 response arrives — shaving 100–300ms from page load times for content-heavy pages.

# Check HTTP version and compression
curl -o /dev/null -s -w "HTTP: %{http_version}\nTime: %{time_total}s\n" https://example.com
curl -H 'Accept-Encoding: br' -v https://example.com 2>&1 | grep -i "content-encoding"

# HTTP/2 server push (nginx — deprecated but still deployed)
location / { http2_push /styles/main.css; }

# Early hints (103) configuration
# Supported in nginx 1.25+ and Cloudflare
add_header Link "</styles/main.css>; rel=preload; as=style" always;

# SecurityHeaders.com audit
curl -I https://example.com | grep -iE "strict-transport|x-frame|csp|x-content"

// Chapter 11

REST APIs and HTTP Semantics

REST (Representational State Transfer), defined by Roy Fielding in his 2000 dissertation, is an architectural style that maps application operations to HTTP's native semantics: resources identified by URLs, operations expressed as HTTP methods, stateless requests, cacheable responses, and a layered system. In practice, "REST API" usually means CRUD operations over JSON using HTTP methods — a subset of the full REST architecture (HATEOAS is almost universally skipped).

REST Method Semantics

• GET /users: list all users (200 + array)

• GET /users/42: get user 42 (200 + object, or 404)

• POST /users: create user (201 + Location header + created object)

• PUT /users/42: replace user 42 entirely (200 or 204)

• PATCH /users/42: partial update (200 + updated fields)

• DELETE /users/42: delete (204 no body)

GraphQL vs REST

REST's practical limitations: over-fetching (getting unneeded fields) and under-fetching (multiple requests to assemble a view). GraphQL addresses both with a query language — the client specifies exactly which fields it needs. Trade-offs: GraphQL is harder to cache (all queries are POST to one URL, defeating HTTP GET caching), requires more tooling, and has a steeper learning curve. REST's strengths: HTTP caching compatibility, universal tooling, simplicity, and debuggability. For most APIs, REST with well-designed endpoints is the pragmatic choice.

// Chapter 12

HTTP Security Attack Patterns

HTTP Request Smuggling

Front-end proxies and back-end servers may disagree on where one HTTP request ends and the next begins — specifically when both Content-Length and Transfer-Encoding: chunked are present in the same request. An attacker crafts a request that the front-end sees as one request but the back-end processes as two, prepending a malicious prefix to the next user's request. PortSwigger's James Kettle documented this class extensively. Mitigations: reject requests with both CL and TE headers, upgrade to HTTP/2 (binary framing eliminates text ambiguity), normalize request parsing at the load balancer.

HTTP/2 Rapid Reset (CVE-2023-44487)

Discovered in 2023: an attacker sends a stream of HEADERS frames immediately followed by RST_STREAM frames, never completing any request. Each pair opens and immediately resets a stream. Since stream IDs increment and the server must track state per stream, this exhausts server concurrency limits and CPU without completing any request. The attack generated record-breaking DDoS floods exceeding 398 million requests per second. Mitigations: limit RST_STREAM rate per connection, implement server-side concurrency limits, and update HTTP/2 implementations.

Cache Poisoning via Web Cache Deception

An attacker tricks a cache into storing a user-specific (authenticated) response under a URL that will be served to all users. Example: a CDN caches the response to /account/profile/style.css (which returns the user's profile page, not CSS). Another user visits the same URL and receives the first user's profile data from cache. Mitigations: never allow caching of authenticated responses without private or no-store directives; configure CDN to strip authentication headers from cached responses.

// Chapter 13

Common Misconceptions

// Chapter 14

IQ Depth Check