A packet's journey

One page load followed end to end: a laptop on home Wi-Fi opens https://example.com, and each stage links to the sheet that explains it in depth. Read it after Networking fundamentals, or first, as a map of the collection. Protocol details (DNS records, TCP, TLS, HTTP) live in Communication networks.

The trip at a glance

You type example.com and press Enter. Before a single byte of the page arrives, the laptop has joined the network, learned its own address, found the router, looked up the name, and sent a first packet that crosses half a dozen machines run by three or four different organizations. In this story the site sits behind a CDN (content delivery network), as most popular sites do.

Joining the network

The laptop first has to be on the network at all. Over Wi-Fi that means hearing the access point's beacons (it announces its network name about ten times a second), associating with it, and running the WPA2 or WPA3 key exchange so that frames are encrypted over the air. How that works is in Wi-Fi; on a cable, plugging in does the same job.

Now the laptop has a link but no IP address. It gets one from a DHCP server (Dynamic Host Configuration Protocol), which in a home is built into the router. The exchange has four messages, remembered as DORA:

• 192.168.x.x is a private range (RFC 1918): every home reuses it, and it never appears on the internet. The router's NAT, below, makes that work.
• IPv6 usually skips DHCP for the address: the router sends Router Advertisements with the network's /64 prefix, and the laptop builds its own address from it (SLAAC, stateless address autoconfiguration); DNS servers come in the same advertisement or from DHCPv6.

Finding the gateway

The laptop wants to send a packet to the server's address (we'll pretend DNS already said 198.51.100.10; it is next). First a routing decision: is 198.51.100.10 inside 192.168.1.0/24? No, so the packet goes to the default gateway. On a local network, frames are delivered by MAC address (the hardware address burned into every network card), so the laptop needs the router's MAC. ARP, the Address Resolution Protocol, asks for it.

• The answer is cached, typically for tens of seconds to a few minutes, so ARP runs rarely. ip neigh (Linux) or arp -a (macOS, Windows) shows the cache.
• IPv6 does the same with NDP (Neighbor Discovery Protocol): a Neighbor Solicitation sent to a multicast group instead of a broadcast, answered by a Neighbor Advertisement.
• Any device on the LAN can answer ARP falsely (ARP spoofing) and pull traffic through itself. Wi-Fi client isolation, switch protections and TLS everywhere limit the damage.

Looking up the name

Back up one step: before connecting, the browser needs an address for example.com. It asks the operating system's stub resolver, which checks its cache and then asks the DNS server from DHCP (the router), which forwards to a recursive resolver run by the ISP or a public service such as 1.1.1.1 or 8.8.8.8.

• On a cache miss the resolver walks the hierarchy: a root server (anycast, see Autonomous systems & BGP), then the .com servers, then the domain's own authoritative servers.
• For a site on a CDN, the authoritative answer is often chosen for you: an anycast address that every CDN location shares, or the address of a location near your resolver.
• Browsers may skip the router and send DNS over HTTPS (DoH) to a resolver they choose.
• Records, TTLs and dig +trace are covered in Communication networks.

The answer: example.com is at 198.51.100.10. The first packet the browser sends there is a TCP SYN to port 443 (or a QUIC Initial over UDP, for HTTP/3), and that is the packet the rest of this page follows.

Through the home router

A home "router" is several devices in one box: a Wi-Fi access point, a small Ethernet switch, a router (it moves packets between the home network and the ISP), a DHCP server, a DNS forwarder, a firewall and a NAT. The ISP gives the home a single public IPv4 address (203.0.113.7 here), so every device in the house has to share it. NAT (network address translation, strictly NAPT, because ports are translated too) rewrites the private source address and port of each outgoing packet to the public address and a free port, and remembers the mapping so it can reverse the change when the reply comes back.

• The router also decrements the IP TTL (time to live, a hop counter that stops packets looping forever) and fixes the header checksums, as every router does.
• Unsolicited inbound packets match no mapping and are dropped, which is why NAT is often mistaken for a firewall. Hosting a game or server at home needs a port forward: a fixed mapping you add.
• Mappings expire: TCP ones when the connection closes, UDP ones after anything from tens of seconds to a few minutes of silence. That is why apps send keepalives; see Communication networks.
• Many ISPs, especially mobile ones, are out of IPv4 addresses and add a second NAT of their own (CGNAT, carrier-grade NAT): the router's "public" address is then something like 100.64.12.34 from the shared range 100.64.0.0/10, and port forwarding becomes impossible.
• IPv6 has enough addresses for every device, so there is normally no NAT: the laptop's own global address goes out unchanged, and the router's firewall blocks unsolicited inbound traffic instead.

The ISP's network

The packet leaves the house on the access network, the "last mile" that the ISP builds to every customer, the most expensive part of any ISP.

From there the traffic of thousands of customers is merged in aggregation switches and routers and reaches a BNG (broadband network gateway), where each subscriber's session lives: their address, speed plan and filters. Then it enters the ISP's core or backbone: big routers in a few cities connected by long-haul fiber, usually running IS-IS inside and forwarding with MPLS labels or segment routing, which let the ISP steer traffic onto chosen paths and carry business VPNs over the same core. How an ISP is laid out is in Building a network.

Between networks

The ISP's routers know where 198.51.100.10 is because the CDN's AS announces the prefix that contains it, and BGP carried that announcement to every network on the internet. The ISP typically has several routes to choose from:

• a direct link to the CDN: a private interconnect, or a session across an IXP (Internet Exchange Point) where both are members. Cheapest and usually fastest, so it wins on LOCAL_PREF.
• a transit provider that sells access to the whole internet: the fallback, paid per Mbps.
• a CDN cache inside the ISP: big CDNs place servers in ISP buildings, so the packet may never leave the ISP at all.

Each network hands the packet off at the exit nearest to where it entered (hot-potato routing), so the reply may return by a different path. How routes spread, how they are chosen and who pays whom is in Autonomous systems & BGP.

At the website

198.51.100.10 is an anycast address: the CDN announces it from dozens or hundreds of locations, and BGP delivers the packet to a nearby one, often in the same metro area as the user.

1. A router at the CDN location receives it and spreads incoming connections over many servers, usually by hashing the connection's addresses and ports (ECMP, equal-cost multi-path) so all packets of one connection land on the same machine.
2. A load balancer or the edge server itself terminates TCP and TLS: the encrypted session ends here, not at the origin.
3. If the page is in the edge cache, the edge answers straight away. If not (a miss), it fetches the page from the origin, the site's own servers in a data center or cloud region, usually over a connection it keeps open.

Load balancers and caches are in System design; data center networks are in Building a network.

The handshakes

The SYN that crossed all of that is only the start. A page load over HTTPS needs a few round trips before the first byte of content:

So a fresh HTTPS connection costs about three round trips before content flows (two with HTTP/3), and every round trip to a far-away server costs real time. That is the main reason CDNs exist. Details are in Communication networks (TCP), TLS and HTTP/3 and QUIC.

The reply

The response goes back the same way in reverse, with a few differences:

• The return path is chosen by the CDN's routers and every network after them, so it can take a different exchange or provider than the request did (asymmetric routing).
• At the home router, the reply matches the NAT mapping and its destination is rewritten back to 192.168.1.23:51514.
• The router looks up the laptop's MAC (already in its ARP cache) and the access point sends the frame over the air.
• A page is not one packet. Most links carry at most 1,500 bytes per packet (the MTU), so a 100 KB response is about 70 packets, sent in bursts that grow as TCP's congestion control gains confidence, with the laptop acknowledging as they arrive.
• Then the browser parses the HTML and repeats much of this trip for images, scripts and fonts, reusing the connection where it can.

Headers hop by hop

Every link puts its own frame around the packet: a header with the addresses that link uses. A router removes the incoming frame, looks at the IP header, and wraps the packet in a new frame for the next link. The IP header inside is what carries the packet end to end: only NAT and the TTL change it.

• On Wi-Fi the frame has three addresses: the access point that receives it, the laptop that sent it, and the router it is ultimately for. The access point converts it into an Ethernet frame.
• Inside an ISP the packet may also carry MPLS labels, short tags that core routers switch on; they are removed before the packet leaves the ISP.
• How layers and encapsulation work in general is in Networking fundamentals.

Timing budget

Rough figures for a laptop on a good home connection fetching a page from a nearby CDN. They vary widely with the access technology, the distance and congestion; measure your own with the commands below.

A fresh page load to a nearby CDN typically shows its first bytes after 50–150 ms; the same page from an origin on another continent takes several times as long, because every round trip crosses the ocean.

Watch it yourself

Every stage can be observed from the laptop. Linux commands are shown; macOS equivalents are in the comments. More tools are in Communication networks and Sysadmin.

# Your address, lease and gateway
ip -4 addr show dev wlan0      # "dynamic", valid_lft
ip route show default          # macOS: netstat -rn
ipconfig getpacket en0         # macOS: the DHCP reply
 
# The ARP cache: who is the gateway at layer 2?
ip neigh show                  # macOS/Windows: arp -a
 
# Your public address, as the internet sees it
dig +short txt ch whoami.cloudflare @1.1.1.1

If the public address differs from the WAN address your router shows, and the router's starts with 100.64 to 100.127, you are behind CGNAT.

# DNS from the root down, then the answer you get
dig +trace example.com
dig example.com +noall +answer
 
# Every router on the way, with AS numbers and loss
traceroute -A example.com
mtr -rwzbc 50 example.com
 
# Where the time goes in one request (seconds, cumulative)
f='dns %{time_namelookup} tcp %{time_connect}'
f="$f tls %{time_appconnect} ttfb %{time_starttransfer}\n"
curl -so /dev/null -w "$f" https://example.com

Watch the packets themselves with tcpdump (needs root), or the same filters in Wireshark:

sudo tcpdump -ni wlan0 'port 67 or port 68'   # DHCP
sudo tcpdump -ni wlan0 arp                    # ARP
sudo tcpdump -ni any 'port 53'                # DNS
# TCP handshakes to port 443
sudo tcpdump -ni any \
  'tcp port 443 and tcp[tcpflags] & tcp-syn != 0'

References

• RFC 2131: Dynamic Host Configuration Protocol (opens in a new tab): DORA, leases, renewal timers
• RFC 826: An Ethernet Address Resolution Protocol (opens in a new tab): ARP
• RFC 4861: Neighbor Discovery for IPv6 (opens in a new tab): NDP and Router Advertisements
• RFC 4862: IPv6 Stateless Address Autoconfiguration (opens in a new tab): SLAAC
• RFC 1918: Address Allocation for Private Internets (opens in a new tab): the private ranges
• RFC 6598: Shared Address Space for CGN (opens in a new tab): 100.64.0.0/10
• RFC 8446: TLS 1.3 (opens in a new tab): the handshake
• RFC 9000: QUIC (opens in a new tab): transport and handshake for HTTP/3
• Wireshark display filter reference (opens in a new tab): every field you can filter on
• tcpdump and libpcap (opens in a new tab): capture tool and filter syntax
• High Performance Browser Networking (opens in a new tab): latency, TCP, TLS and the browser