HTTP Fundamentals - Web Server Tutorials

Web Servers Speak HTTP

HTTP (Hypertext Transfer Protocol) is the language that web servers and browsers use to communicate. Every web server, regardless of its underlying architecture or implementation language, must understand and produce valid HTTP.

This tutorial focuses on observing HTTP from the server's perspective—seeing exactly what arrives and what gets sent back. For a deep dive into HTTP concepts, methods, status codes, and headers, see the dedicated HTTP section.

HTTP Deep Dive

For comprehensive coverage of HTTP methods, status codes, headers, and the protocol itself, visit the HTTP tutorials:

HTTP Tutorials →

Seeing HTTP with curl

The curl command is your most valuable tool for understanding what servers actually send and receive. Unlike a browser, curl shows you the raw HTTP transaction.

Basic Request

# Simple GET request
curl https://httpbin.org/get

This shows you the response body, but hides the HTTP mechanics. To see everything, use the -v (verbose) flag:

Verbose Mode

# Show the full HTTP conversation
curl -v https://httpbin.org/get

# Output (simplified):
* Connected to httpbin.org (54.243.149.98) port 443
> GET /get HTTP/2
> Host: httpbin.org
> User-Agent: curl/8.1.2
> Accept: */*
>
< HTTP/2 200
< content-type: application/json
< content-length: 256
<
{
  "args": {},
  "headers": { ... },
  "url": "https://httpbin.org/get"
}

The > lines show what curl sent to the server (the request). The < lines show what the server returned (the response).

Reading curl output: Lines starting with * are curl's own messages (connection info). Lines with > are outgoing (request). Lines with < are incoming (response).

Just the Headers

# Show response headers only
curl -I https://example.com

HTTP/2 200
content-type: text/html; charset=UTF-8
content-length: 1256
cache-control: max-age=604800
etag: "3147526947"

The -I flag sends a HEAD request, which asks for headers only without the response body. This is useful for checking caching headers, content types, and server configuration without downloading the full content.

Anatomy of an HTTP Request

When a client connects to your web server, it sends a request that looks like this:

GET /products/widget HTTP/1.1
Host: shop.example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Connection: keep-alive

Breaking this down:

Request line: Method (GET), path (/products/widget), protocol version (HTTP/1.1)
Host header: Which domain the request is for (essential for virtual hosting)
User-Agent: What client is making the request
Accept headers: What content types, languages, and encodings the client understands
Connection: Whether to keep the TCP connection open for more requests

Anatomy of an HTTP Response

Your web server responds with:

HTTP/1.1 200 OK
Server: nginx/1.24.0
Date: Wed, 15 Nov 2024 10:30:00 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 4523
Cache-Control: max-age=3600

<!DOCTYPE html>
<html>
  ...
</html>

The key parts:

Status line: Protocol, status code (200), reason phrase (OK)
Server: Identifies the server software (can be hidden for security)
Content-Type: MIME type of the response body
Content-Length: Size of the body in bytes
Cache-Control: How browsers and proxies should cache this response
Blank line: Separates headers from body
Body: The actual content (HTML, JSON, image data, etc.)

Common curl Commands for Server Testing

Send Custom Headers

# Send a specific header
curl -H "Authorization: Bearer token123" https://api.example.com/me

# Multiple headers
curl -H "Accept: application/json" \
     -H "X-Custom-Header: value" \
     https://api.example.com/data

POST Request with JSON

# Send JSON data
curl -X POST https://api.example.com/users \
     -H "Content-Type: application/json" \
     -d '{"name": "Alice", "email": "alice@example.com"}'

Follow Redirects

# Automatically follow 301/302 redirects
curl -L https://example.com/old-page

# Show redirect chain
curl -v -L https://example.com/old-page 2>&1 | grep -E "^[<>] (HTTP|Location)"

Time the Request

# Show timing breakdown
curl -w "\nDNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTLS: %{time_appconnect}s\nTotal: %{time_total}s\n" \
     -o /dev/null -s https://example.com

# Output:
DNS: 0.023s
Connect: 0.045s
TLS: 0.112s
Total: 0.234s

Try It Yourself

Open your terminal and try curl -v https://example.com. Watch the full HTTP conversation unfold. Notice how many headers are exchanged before any content appears.

Using netcat for Raw HTTP

For the ultimate low-level view, you can speak HTTP manually using netcat (or nc). This shows exactly what a web server receives:

# Connect to a server and type HTTP manually
nc example.com 80
GET / HTTP/1.1
Host: example.com
[press Enter twice]

# The server responds with raw HTTP
HTTP/1.1 200 OK
Content-Type: text/html
...

Why this matters: When debugging server issues, you're often trying to figure out if the problem is in the request, the server config, or the response. Seeing raw HTTP helps you isolate which part is broken.

HTTP/2 and HTTP/3

Modern servers support HTTP/2 and HTTP/3, which use binary framing instead of plain text. curl handles this transparently:

# Force HTTP/2
curl --http2 -v https://example.com

# Force HTTP/1.1 (useful for debugging)
curl --http1.1 -v https://example.com

# Check which HTTP version was used
curl -w "%{http_version}\n" -o /dev/null -s https://example.com
# Output: 2 (meaning HTTP/2)

Even with HTTP/2's binary format, the semantics remain the same: methods, headers, status codes, and bodies all work the same way. The server's configuration for routing, headers, and responses applies regardless of HTTP version.

What's Next

Now that you can observe HTTP conversations, the next tutorial explores how different server architectures handle these connections—and why the choice of architecture affects performance, scalability, and failure modes.