CDNs

A content delivery network (CDN) is a shared cache in hundreds of cities, between your users and your servers. It answers repeat requests from a machine a few milliseconds from the user. Only the misses travel to your origin. What makes it work is the part nobody draws. That’s the cache key, how long each response may be kept, how a change reaches the edge, and what happens when the cache is suddenly empty.

The problem it solves

Your servers are in Virginia. A user in Sydney is about 16,000 km away. Light in fiber covers roughly 200 km per millisecond. So one round trip can never be faster than about 160 ms. Real routes are longer, so expect 200 to 230 ms.

A new HTTPS connection costs three round trips before the first byte arrives: the TCP handshake, the TLS 1.3 handshake, and the request itself. So that user waits about 650 ms for the first byte of anything. A page that needs files from a second host pays it again. No amount of server tuning changes this. The only fix is to answer from somewhere closer.

The same request from Sydney, to the origin and to a nearby edge

Scroll or pinch to zoom, drag to pan

• Color key
• Service

Latency is the first reason. Two more arrive with traffic:

1. Offload. A news site peaks at 40,000 requests a second, most of them images, scripts and stylesheets that are identical for every user. One origin file server handles about 5,000 requests a second, so serving everything yourself takes eight of them at the global peak, more with headroom. With 90% of requests answered at the edge the origin sees 4,000 a second. At 99% it sees 400.
2. Egress cost. At 100 KB a response and 15,000 requests a second on average, that site sends about 3.9 PB a month. Cloud internet egress costs about 5 to 9 cents per GB at list price. So that’s roughly $200,000 a month from your own servers. CDN egress at a negotiated $0.01 to $0.02 per GB is $40,000 to $80,000. And on AWS, the transfer from your origin to CloudFront is free.

A CDN also absorbs a DDoS across hundreds of cities instead of on your load balancer.

Think in miss ratio, not hit ratio: going from 90% to 99% hits looks like a 10% gain and cuts origin traffic ten times.

How it works

Step 1: A cache near the users

A CDN runs points of presence (PoPs). These are racks of cache servers in hundreds of cities, connected to local ISPs and internet exchanges. Each holds hot content in memory, and a larger set on SSDs.

Your site’s hostname, say cdn.example.com, points at the CDN instead of at your servers. The CDN handles TLS for that hostname, with your certificate. It looks up the request in its cache. On a miss, it fetches it from your origin: your web servers, or an object storage bucket.

CDN in front of an origin · architecture

Stage 2: A pull CDN

Grow itTrace a request

Color keyServiceDatabaseCacheNew in this stageAsync or occasional

Play

PreviousNext

Stage

12

Requests per second:

40,000

40k peak400k (10×)4M (100×)

Stage 2 of 4

A pull CDN

Why this stage

Most requests are for the same few thousand files, and the far-away users pay the most for them.

What we add

Edge PoPs in hundreds of cities.

DNS or anycast sends each user to a nearby one; on a miss the edge fetches from the origin and keeps the copy for as long as Cache-Control allows.

What it costs

Every edge misses separately, so an unpopular file costs one origin fetch per PoP, and changes stay cached until the TTL ends or a purge lands.

At 40k requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one machine can do

• One origin file server handles about 5,000 requests a second. It does TLS and reads from object storage.
• One shield server handles about 50,000 requests a second from memory and SSD.

What the caches save

• A pull CDN answers 90% of requests at the edge.
• A shield answers 80% of what is left, so only 2% reach the origin.

Other traffic

• From stage 4, 5% of requests are API calls. They pass through the edge without caching.
• Instances scale out to stay near 60% busy, never fewer than 2.

From here on, two things matter most. The edge decides what counts as “the same request” (the cache key, step 4). And your origin’s response headers decide how long it may keep each answer (step 3). Those headers are part of the design, not a detail.

Step 2: Getting each user to a nearby edge

The CDN has to send each user to a PoP that is close and healthy. There are two ways to do it.

DNS steering. cdn.example.com is a CNAME to the CDN’s DNS. That DNS answers each resolver with the IP of a nearby, healthy PoP that isn’t overloaded. It steers on load and on live latency measurements.

Its weakness is that it sees the resolver, not the user. A user in Mumbai whose public resolver queries from Singapore gets a Singapore PoP. EDNS Client Subnet (RFC 7871) fixes this by passing on part of the user’s IP, when the resolver sends it. Changes take effect only as resolvers drop cached answers. So answers carry short TTLs, 20 to 60 seconds.

Anycast. Every PoP announces the same IP prefix over BGP, and routing carries each packet to the PoP closest in routing terms. A failing PoP withdraws its routes and traffic moves within seconds, with no TTL to wait for. A flood aimed at one IP is split across every PoP by the routing itself, which is why DDoS protection is built this way. Its weakness is that BGP’s “closest” is the shortest AS path, not the lowest latency, and shifting load off a busy PoP means changing announcements.

DNS steering answers with a nearby IP, anycast routes one IP to the nearest PoP

Scroll or pinch to zoom, drag to pan

• Color key
• Cache

Akamai is the classic DNS-steered CDN. Cloudflare is the classic anycast one. Most large CDNs mix the two. TCP survives anycast, because routes rarely change in the middle of a connection.

Step 3: Pull on a miss, keep it as long as the headers allow

In a pull CDN, the default everywhere, the edge fetches a file from the origin the first time it is requested at that PoP, and keeps it. You upload nothing to the CDN. In a push CDN, you upload files to the CDN’s storage ahead of time, and the origin is never asked.

How long the edge keeps a response is set by the Cache-Control header the origin sends. The directives that matter:

Directive	Meaning	Typical use
max-age=N	Any cache may treat it as fresh for N seconds	Browser and edge lifetime when they are the same
s-maxage=N	Shared caches (CDNs, proxies) use N instead of max-age	Edge keeps it an hour, browsers a minute
public	Shared caches may store it even if the request carried Authorization	Responses to logged-in requests that are the same
private	Only the user’s own browser may store it	A page with the user’s name on it
no-store	Nobody stores it at all	Account details, payment pages, tokens
no-cache	Store it, but revalidate with the origin before every use	HTML that must always be current but is often 304
immutable	It will never change at this URL, so do not revalidate even on reload	Files with a content hash in the name
stale-while-revalidate=N	For N seconds after expiry, serve the old copy and refresh in the background	Popular objects, so no user waits on a refresh
stale-if-error=N	For N seconds after expiry, serve the old copy if the origin is failing	Everything cacheable, as outage insurance

Go deeperRevalidation and the Age header1 min

A response that expired is not thrown away. If it has an ETag or Last-Modified, the edge sends a conditional request (If-None-Match), and if nothing changed the origin answers 304 Not Modified with no body. That saves the bytes, though not the round trip.

The edge also tells the browser how old its copy is, with an Age header. Say a file has max-age=600, and sat at the edge for 590 seconds. It reaches the browser with Age: 590, and the browser keeps it for only 10 more seconds. Stacked tiers of caches count down the same clock. They do not each start a fresh TTL.

JDirectives.javaPydirectives.pyC#Directives.csGodirectives.goTSdirectives.ts

JavaPythonC#GoTypeScript

JavaPythonC#GoTypeScript

import java.util.List;

class Main {
    public static void main(String[] args) {
        Directives d = Directives.parse(
                "public, max-age=60, s-maxage=300, stale-while-revalidate=30, stale-if-error=86400");
        for (int[] c : List.of(new int[] {120, 1}, new int[] {310, 1}, new int[] {400, 1}, new int[] {400, 0})) {
            boolean healthy = c[1] == 1;
            System.out.printf("age %3d s, origin %s: %s%n", c[0], healthy ? "up" : "down",
                    Freshness.decide(d, c[0], healthy).label);
        }
        System.out.println(Freshness.decide(Directives.parse("private, max-age=60"), 0, true).label);
    }
}

enum Decision {
    BYPASS("bypass, never store"),
    HIT("fresh hit"),
    STALE_WHILE_REVALIDATE("serve stale, refresh in background"),
    STALE_IF_ERROR("serve stale, origin is failing"),
    FETCH("fetch from origin");

    final String label;

    Decision(String label) {
        this.label = label;
    }
}

final class Directives {
    Integer maxAge;
    Integer sMaxage;
    int staleWhileRevalidate;
    int staleIfError;
    boolean isPrivate;
    boolean noStore;

    static Directives parse(String header) {
        Directives d = new Directives();
        for (String part : header.split(",")) {
            String[] kv = part.trim().toLowerCase().split("=", 2);
            int n = kv.length > 1 ? Integer.parseInt(kv[1]) : 0;
            switch (kv[0]) {
                case "max-age" -> d.maxAge = n;
                case "s-maxage" -> d.sMaxage = n;
                case "stale-while-revalidate" -> d.staleWhileRevalidate = n;
                case "stale-if-error" -> d.staleIfError = n;
                case "private" -> d.isPrivate = true;
                case "no-store" -> d.noStore = true;
                default -> { }
            }
        }
        return d;
    }
}

final class Freshness {
    static Decision decide(Directives d, int age, boolean originHealthy) {
        // private means "only the user's own browser may keep this". A shared cache that stores it
        // can hand one user's page to the next user, the worst bug a CDN config can have.
        if (d.isPrivate || d.noStore) return Decision.BYPASS;
        // A shared cache prefers s-maxage, so the edge can keep a file for an hour while browsers,
        // which nobody can purge, keep it for a minute.
        int ttl = d.sMaxage != null ? d.sMaxage : d.maxAge != null ? d.maxAge : 0;
        if (age < ttl) return Decision.HIT;
        // Just expired: answer at once with the old copy and refresh it behind the user's back, so
        // no user ever waits for the origin on a popular object.
        if (age < ttl + d.staleWhileRevalidate) return Decision.STALE_WHILE_REVALIDATE;
        // A real edge tries the origin first and falls back here on a 5xx or a timeout. A stale page
        // beats an error page for anything that is not money or permissions.
        if (!originHealthy && age < ttl + d.staleIfError) return Decision.STALE_IF_ERROR;
        return Decision.FETCH;
    }
}

Common mistake

no-cache means do not cache

no-cache means “store it, but check with the origin before every use”. The response is cached, and each reuse costs a conditional request that usually returns a 304. The directive that forbids storing is no-store. Send no-cache on an account page, and you leave a copy in every cache it passes through. Send no-store on a stylesheet, and you throw away a free cache hit on every page view.

Step 4: The cache key decides the hit ratio

The edge looks up each request by a cache key. On most CDNs the default is the host, the path and the full query string (CloudFront’s recommended policy drops the query string entirely). Anything else that changes the response is declared with the Vary header, which adds that request header to the key. Support differs: Fastly honors Vary, CloudFront keys only on the headers its cache policy names, and Cloudflare ignores most Vary values. Configure the key explicitly. This is where most hit ratios are lost, because every extra input splits one file into many cache objects, each missing separately at each PoP.

• Query strings. A shared link arrives as /img/cat.jpg?fbclid=IwAR3x..., a click ID unique per click, and marketing adds utm_* tags. With the full query string in the key, each is an object nobody else will request. Keep only parameters that change the bytes (w, fmt), sorted.
• Cookies. Analytics cookies are on almost every request. A key with the Cookie header, or Vary: Cookie on a static file, gives every visitor a private copy and a hit ratio near zero. If a response really depends on cookies, it is not shared content and should say private.
• Vary. Vary: Accept-Encoding is fine once normalized to br, gzip or identity. Vary: User-Agent stores each file once per distinct browser string, thousands of times. If the response truly differs by device, normalize to a header with two values and vary on that.

JCacheKeys.javaPyexample.pyC#CacheKeys.csGomain.goTSexample.ts

JavaPythonC#GoTypeScript

JavaPythonC#GoTypeScript

import java.net.URI;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

class Main {
    public static void main(String[] args) {
        String a = CacheKeys.cacheKey(
                "https://CDN.example.com/img/cat.jpg?utm_source=mail&w=400&fmt=webp",
                Map.of("accept-encoding", "gzip, deflate, br", "cookie", "session=abc"));
        String b = CacheKeys.cacheKey(
                "https://cdn.example.com/img/cat.jpg?fmt=webp&w=400&fbclid=XYZ",
                Map.of("accept-encoding", "br, gzip", "cookie", "session=def"));
        System.out.println(a); // cdn.example.com/img/cat.jpg?fmt=webp&w=400|br
        System.out.println(a.equals(b)); // true: two requests, one cache object
    }
}

final class CacheKeys {
    // Query parameters that change the bytes of the response. Everything else (utm_*, fbclid,
    // gclid, cache busters someone added by hand) is dropped, so it cannot split one file into
    // thousands of cache objects. An allowlist is safer than a blocklist: new tracking parameters
    // appear every year, new image parameters only when you add them.
    static final Set<String> ALLOWED_PARAMS = Set.of("w", "h", "fmt", "q");

    // Vary: Accept-Encoding on the raw header gives one copy per browser's exact header string
    // ("gzip, deflate, br", "br, gzip", ...). The edge only stores three real variants.
    static String normalizeEncoding(String acceptEncoding) {
        Set<String> offered = new HashSet<>();
        for (String part : acceptEncoding.split(",")) {
            offered.add(part.split(";")[0].trim().toLowerCase());
        }
        if (offered.contains("br")) return "br";
        if (offered.contains("gzip")) return "gzip";
        return "identity";
    }

    static String cacheKey(String url, Map<String, String> headers) {
        URI u = URI.create(url);
        List<String[]> kept = new ArrayList<>();
        if (u.getRawQuery() != null) {
            for (String pair : u.getRawQuery().split("&")) {
                String[] kv = pair.split("=", 2);
                String k = URLDecoder.decode(kv[0], StandardCharsets.UTF_8);
                String v = kv.length > 1 ? URLDecoder.decode(kv[1], StandardCharsets.UTF_8) : "";
                if (ALLOWED_PARAMS.contains(k)) kept.add(new String[] {k, v});
            }
        }
        // Sorted, so ?w=200&fmt=webp and ?fmt=webp&w=200 are one object.
        kept.sort((a, b) -> a[0].equals(b[0]) ? a[1].compareTo(b[1]) : a[0].compareTo(b[0]));
        List<String> query = new ArrayList<>();
        for (String[] kv : kept) {
            query.add(URLEncoder.encode(kv[0], StandardCharsets.UTF_8) + "="
                    + URLEncoder.encode(kv[1], StandardCharsets.UTF_8));
        }
        String encoding = normalizeEncoding(headers.getOrDefault("accept-encoding", ""));
        // Host is case-insensitive and the scheme does not change the bytes, so neither may split
        // the key. The path is kept exactly: /a.css and /A.css can be different files.
        // No Cookie and no User-Agent: a public file must not depend on who asks for it. If it
        // does, it is not a public file and should say Cache-Control: private.
        return u.getHost().toLowerCase() + u.getPath() + "?" + String.join("&", query) + "|" + encoding;
    }
}

The reverse mistake is worse. If the key leaves out something that does change the response, users get each other’s versions. A page that returns English or French depending on Accept-Language, cached without that header in the key, serves whichever language was fetched first to everybody. That is the same bug that, with a user’s session instead of a language, leaks private data (How it fails).

Step 5: Changing content that is already cached

Once a file sits in 300 PoPs and an unknown number of browsers, changing it is the hard part. Three tools, and they combine:

• Let the TTL run out. Simple and free. The cost is staleness: with a one-hour TTL a fixed typo stays visible for up to an hour, and a different hour in each PoP.
• Purge. Tell the CDN to drop an object (by URL), a group (by tag), or everything. Fastly calls tags surrogate keys, and Cloudflare calls them cache tags. Fastly and Cloudflare advertise global purges in well under a second, up to a few seconds. CloudFront invalidations typically take a minute or more. They cost money beyond the first thousand paths a month. A purge does nothing for browsers, which nobody can purge.
• Versioned URLs. Put a content hash or version in the file name: app.3f9a1c.js, /m/9f3a/v3/640.webp. The file at a given URL never changes, so it can be cached for a year with immutable, in browsers too. A deploy creates new URLs, the HTML points at them, and the old ones age out unread. Nothing is ever stale, and nothing needs purging.

A deploy with purges, and the same deploy with versioned URLs

Scroll or pinch to zoom, drag to pan

• Color key
• Service
• Cache

Order matters in the versioned deploy: upload the new files first, then switch the HTML. The other order serves HTML that names files the origin does not have yet. And the HTML itself cannot be versioned, because its URL is what users type. Give it a short s-maxage (seconds to a minute) with stale-while-revalidate, and purge it on deploy.

TradeoffVersioned file names with year-long TTLs as the default

We gain

The highest possible hit ratio, in the CDN and in browsers, with zero staleness and no purge to forget or to fail.

We give up

A build step that hashes names and rewrites references. Old versions kept at the origin, while old HTML still points at them. And the HTML entry points still need short TTLs and a purge.

Step 6: Tiered caching and an origin shield

A pull CDN with 300 PoPs misses on a file separately in each one. A file watched in 50 cities costs 50 origin fetches. After a purge, every PoP misses at once.

The fix is a hierarchy. Edges miss to a regional tier, or to one origin shield. That’s a large cache close to the origin. Only the shield misses to the origin. CloudFront calls these regional edge caches and Origin Shield. Fastly calls it shielding. Cloudflare calls it tiered cache.

The shield does two jobs. First, it turns N misses for the same object into one, because the first edge to miss fills it for the rest. Second, it does request collapsing. Say 200 edges ask for the same uncached object in the same second. That causes one origin request, and the other 199 wait for its answer. It’s the CDN-sized version of the single-flight fix for cache stampedes.

CDN in front of an origin · architecture

Stage 3: An origin shield

Grow itTrace a request

Color keyServiceDatabaseCacheNew in this stageAsync or occasional

Play

PreviousNext

Stage

123

Requests per second:

40,000

40k peak400k (10×)4M (100×)

Stage 3 of 4

An origin shield

Why this stage

At 400,000 requests a second, the 10% of misses from hundreds of PoPs is more than the origin can serve, and a purge sends all of them at once.

What we add

One large cache tier next to the origin.

Edges miss to the shield, the shield misses to the origin, and concurrent misses for one key collapse into one fetch.

What it costs

A miss pays one more hop, and the shield is a concentrated point of load that needs its own failover.

At 40k requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one machine can do

• One origin file server handles about 5,000 requests a second. It does TLS and reads from object storage.
• One shield server handles about 50,000 requests a second from memory and SSD.

What the caches save

• A pull CDN answers 90% of requests at the edge.
• A shield answers 80% of what is left, so only 2% reach the origin.

Other traffic

• From stage 4, 5% of requests are API calls. They pass through the edge without caching.
• Instances scale out to stay near 60% busy, never fewer than 2.

Below, 2,000 requests a second for 20,000 files run through four edges and a shield. Popularity is realistic: a few files are very hot, and most are rarely asked for. Each cache is a real map of cache key to expiry and content version.

Edge TTL (s-maxage)

10 s1 min1 hour1 day

Cache key

Ignore tracking paramsInclude cookiesVary: User-Agent

Delivery

Origin shieldVersioned file names

Publish an updatePurge everythingPause

Edge hit ratio

-

last 10 s

Origin load

-

requests/s, capacity 400

Stale responses

-

served an old version

Average response

-

user to answer

Origin requests per second, last 90 s. Dashed line: origin capacity.

edge hitshield hitorigin fetchstale hit

2,000 requests a second for 20,000 files, warmed up for two minutes. Change the TTL or the cache key and watch the origin.

Things to try:

• TTL. At 1 minute the edge answers about 70% of requests. At 10 seconds even popular files keep expiring and the origin climbs past capacity. At 1 hour the hit ratio climbs toward its ceiling over the following minutes. Then press Publish an update: 300 popular files change, and about a fifth of all responses are stale until their copies expire.
• Versioned file names. Turn them on, wait for the refill, and publish again. The changed files get new names, each misses once per tier, and nothing is stale. Long TTLs stop costing anything.
• Cache key. Turn off Ignore tracking params and the hit ratio falls by about twenty points. Vary: User-Agent nearly halves it. Include cookies takes it under 10% and puts the origin at four times its capacity. Same files, same users, only the key changed.
• Purge everything, then watch the sparkline: every request misses at once. Do it again with the shield off, and the spike is higher because four edges each fetch the same files. The refill repeats one TTL later, smaller, because everything was fetched in the same few seconds and expires together.

Step 7: Private and dynamic content

Not everything is public. Three tools cover the rest.

Signed URLs and signed cookies. Think of a paid video, a private photo, or a download link. The app checks that the user may see the file. Then it hands out a URL with an expiry, and a signature over the path and the expiry. On most CDNs, the signature is an HMAC with a shared key. On CloudFront, it’s an RSA or ECDSA signature, checked against your uploaded public key. The edge rejects anything expired or altered, without calling the origin.

The signature must not be part of the cache key. Verify it, then look up the path alone. Otherwise every viewer’s link is a separate object. A signed cookie does the same for a whole path prefix. That suits video, where a player fetches hundreds of segment URLs it cannot sign one by one.

JSignedUrls.javaPyexample.pyC#SignedUrls.csGomain.goTSexample.ts

JavaPythonC#GoTypeScript

JavaPythonC#GoTypeScript

import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

class Main {
    public static void main(String[] args) throws Exception {
        String url = SignedUrls.sign("/videos/42/720p/seg-0007.ts", 1_700_000_000L, 300);
        System.out.println(SignedUrls.verify(url, 1_700_000_100L)); // Optional[/videos/42/720p/seg-0007.ts]
        System.out.println(SignedUrls.verify(url.replace("seg-0007", "seg-0008"), 1_700_000_100L)); // empty: tampered
        System.out.println(SignedUrls.verify(url, 1_700_000_400L)); // empty: expired
    }
}

final class SignedUrls {
    // Shared between the app that signs and the edge that checks. Rotate it by accepting two keys
    // for a while; the CDN never calls the origin to ask whether a URL is allowed.
    static final byte[] SECRET = "rotate-me-every-90-days".getBytes(StandardCharsets.UTF_8);

    // Sign the path and the expiry together, so neither can be changed without breaking it.
    static String sig(String path, long expires) throws Exception {
        Mac mac = Mac.getInstance("HmacSHA256");
        mac.init(new SecretKeySpec(SECRET, "HmacSHA256"));
        byte[] digest = mac.doFinal((path + "|" + expires).getBytes(StandardCharsets.UTF_8));
        return Base64.getUrlEncoder().withoutPadding().encodeToString(digest);
    }

    // Short lifetimes (minutes) limit what a leaked link is worth. The app signs after it has
    // checked that this user may see this file.
    static String sign(String path, long now, long ttl) throws Exception {
        long expires = now + ttl;
        return path + "?exp=" + expires + "&sig=" + sig(path, expires);
    }

    // At the edge: the cache key if the URL is valid, empty for a 403.
    static Optional<String> verify(String url, long now) throws Exception {
        URI u = URI.create(url);
        Map<String, String> q = new HashMap<>();
        if (u.getRawQuery() != null) {
            for (String pair : u.getRawQuery().split("&")) {
                String[] kv = pair.split("=", 2);
                q.put(kv[0], kv.length > 1 ? kv[1] : "");
            }
        }
        long expires;
        try {
            expires = Long.parseLong(q.getOrDefault("exp", ""));
        } catch (NumberFormatException e) {
            return Optional.empty();
        }
        if (now > expires) return Optional.empty();
        // Constant-time compare, so response timing does not leak how many characters matched.
        byte[] given = q.getOrDefault("sig", "").getBytes(StandardCharsets.UTF_8);
        byte[] expected = sig(u.getPath(), expires).getBytes(StandardCharsets.UTF_8);
        if (!MessageDigest.isEqual(given, expected)) return Optional.empty();
        // The cache key is the path alone. Keep exp and sig in it and every viewer's link is a
        // separate object: the hit ratio for private files drops to zero.
        return Optional.of(u.getPath());
    }
}

Dynamic content acceleration. An uncached API call still gains. TLS ends at the nearby PoP. Then the edge forwards the request over a connection it keeps open to the origin. So one long round trip replaces three or four. Some CDNs also route it over their own backbone, on measured fastest paths (Cloudflare Argo, Akamai SureRoute). Cache-Control: private, no-store keeps these responses out of every shared cache.

Edge compute. Cloudflare Workers, Fastly Compute and CloudFront Functions run your code in the PoP. They normalize cache keys, check a token, pick an A/B variant, or redirect by country. They start in about a millisecond, under tight CPU limits. (They use V8 isolates, WebAssembly, or a stripped-down JavaScript runtime.) Lambda@Edge is heavier: full Lambda functions in the regional tier, with cold starts. The edge key-value stores they read take seconds to a minute to reach every PoP. So use them for decisions that need no fresh data.

CDN in front of an origin · architecture

The full design, stage by stage

Grow itTrace a requestBreak it

Color keyServiceDatabaseCacheNew in this stageAsync or occasional

Play

PreviousNext

Stage

1234

Requests per second:

40,000

40k peak400k (10×)4M (100×)

Stage 4 of 4

Dynamic and private traffic through the edge

Why this stage

API calls and per-user pages cannot be cached, but they still pay three long round trips to set up TCP and TLS from far away.

What we add

The edge terminates TLS near the user and forwards uncached requests to the API over warm, long-lived connections.

Private files use signed URLs the edge checks.

What it costs

The CDN now sees every request in plaintext and becomes part of the critical path for writes, not just for static files.

At 40k requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one machine can do

• One origin file server handles about 5,000 requests a second. It does TLS and reads from object storage.
• One shield server handles about 50,000 requests a second from memory and SSD.

What the caches save

• A pull CDN answers 90% of requests at the edge.
• A shield answers 80% of what is left, so only 2% reach the origin.

Other traffic

• From stage 4, 5% of requests are API calls. They pass through the edge without caching.
• Instances scale out to stay near 60% busy, never fewer than 2.

CDN in front of an origin · architecture

Trace a request: A miss, through the shield

An image, edge hitA miss, through the shieldAn API call, not cached

Color keyServiceDatabaseCacheNew in this stageAsync or occasionalHop in motionHops doneResponse

Play

PreviousNext

A miss, through the shield: step 1 of 5

Users to Edge PoPs

Same Frankfurt PoP, but nobody there has asked for this file within its TTL.

12 ms of a 300 ms budget

1. Users → Edge PoPs12 ms
2. Edge PoPs → Origin shield90 ms
3. Origin shield → Origin3 ms
4. Origin → Object storage25 ms
5. Response

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

Options and tradeoffs

How users reach an edge

Option	Failover	Wins when	Loses
Anycast	Seconds (BGP withdrawal)	DDoS absorption, fast failover, no dependence on resolvers	“Nearest” is by AS path, not latency; coarse control of load per PoP
DNS steering	DNS TTL (20 to 60 s, longer in practice)	Fine control by load and measured latency per PoP	Sees the resolver, not the user, unless ECS is sent; cached answers
Both (DNS picks an anycast group)	Seconds	Large CDNs that want control and resilience	Two systems to operate; only the CDN can choose this

TradeoffPrefer an anycast CDN when failover speed matters

We gain

A failed PoP drops out in seconds with no DNS caches to wait for, and attack traffic spreads over every PoP by routing alone.

We give up

Less control over which PoP serves whom, and occasional users routed to a PoP that is near in BGP terms but not in milliseconds.

Pull or push

Option	How content arrives	Wins when	Loses
Pull	Edge fetches from the origin on the first miss	Large or unpredictable catalogs; the default for web content and media	First request at each PoP is slow; an origin must stay up
Push	You upload to CDN storage before anyone asks	Big files with known demand: game patches, releases, a scheduled premiere	You manage uploads, deletes and storage cost; no origin to fall back on
Pre-positioned (own boxes)	Your caches inside ISPs, filled overnight by your own logic	Netflix-scale video, where you can predict tomorrow’s popular titles	Hardware, ISP partnerships and a team to run them

Pull plus a shield covers almost every design. Push wins when demand is predictable and huge, as when a game update lands on millions of consoles at 9 a.m. and a cold pull cache would send the first million requests to the origin. Netflix’s Open Connect appliances sit inside ISPs and are filled during off-peak hours with what each region will likely watch the next day.

TradeoffA pull CDN with an origin shield as the default

We gain

Nothing to upload or keep in sync. Whatever gets popular gets cached. And the shield keeps origin load near one fetch per object per TTL, however many PoPs ask.

We give up

The first viewer in each region pays a miss. A cold or purged cache sends a burst to the origin. And the origin must stay up to serve anything not yet cached.

How changes reach the edge

Option	Staleness	Origin load after a change	Wins when	Loses
TTL only	Up to one TTL, per PoP	Smooth	Content where minutes of staleness are fine	Long TTLs mean long staleness; short ones mean misses
Purge by URL	Seconds to minutes at the CDN	A burst for the purged objects	Deletes, takedowns, the HTML entry point	Browsers keep their copy; a lost purge leaves it cached
Purge by tag	Same, for a whole group at once	A burst for the group	One product changed, 400 pages show it	Tags must be set on every response; not on every CDN tier
Versioned URLs	None	One miss per changed file per tier	Static assets, media, anything a build emits	A build step; HTML entry points cannot be versioned

TradeoffVersion what you can, purge what you cannot

We gain

Year-long TTLs for assets and media, with no staleness. Purges only for a handful of HTML and API URLs, so a purge is small, fast and rarely needed.

We give up

Two policies to explain to every team. And a purge path that still has to be reliable, retried until the CDN acknowledges, for deletes and legal takedowns.

Private content

Option	Edge can cache it?	Wins when	Loses
Signed URL	Yes, keyed on path	One file per link: downloads, photos, a paste	A leaked link works until it expires; must strip the signature from the key
Signed cookie	Yes, keyed on path	Many files under one prefix: HLS and DASH segments	Cookie scope and domains to get right; harder to share one file
Token check in edge compute	Yes	Custom rules: per-tenant keys, revocation lists, JWT claims	Your code in the hot path of every request; CPU limits
Bypass the CDN cache	No	Truly per-user responses: account pages, carts	Every request reaches the origin; still gains from edge TLS

TradeoffSigned URLs with minutes of lifetime for private files

We gain

Private files are served from the edge at full hit ratio, and the CDN enforces access without calling your auth service on every request.

We give up

Revoking access takes effect only when the link expires, so lifetimes stay short. And the signing key is a secret that the CDN and the app must share and rotate.

Go deeperOne CDN, several, or your own1 min

Option	Wins when	Loses
One commercial CDN	Almost always: hundreds of PoPs on day one, no hardware	One vendor’s outage is your outage; per-GB price on your biggest bill
Multi-CDN	Availability targets above one vendor’s record; price negotiation	Features to the lowest common denominator; two configs, purges and certs
Own edge or ISP caches	Egress is the dominant cost at hundreds of Tbps (Netflix, YouTube)	Years of work, hardware in other people’s buildings

Start with one commercial CDN. Add a second only when an availability target, or the size of the bill, justifies it. Two CDNs mean two configurations, two purge paths and two sets of certificates.

Go deeperByte hit ratio, cache size and what 'hit' means2 min

Request hit ratio vs byte hit ratio. CDNs report both. Say all the small thumbnails hit, but the large originals mostly miss. The site can show a 95% request hit ratio and a 60% byte hit ratio. The bytes set the egress bill and the origin’s bandwidth. Watch the ratio that matches the cost you care about.

Hit ratio is limited by popularity, not just by TTL. Most sites have a long tail: files requested once a day or less. Each PoP holds only what was recently requested there. A file requested once a day in Lima misses in Lima every time, whatever its TTL. That’s because each cache evicts by recency under memory pressure. A shield raises the ratio the origin sees, because it pools the requests of every PoP.

What counts as a hit. Some CDNs count a revalidation (304 from the origin) as a hit, some as a miss, some separately. A stale-while-revalidate response is a hit for the user and still a request for the origin. Read the vendor’s definition before comparing numbers.

How it fails

CDN in front of an origin · architecture

Break it: what fails, who notices, what users see

Color keyServiceDatabaseCacheNew in this stageAsync or occasional

Break

One edge PoPThe origin shieldThe originThe API serversSteering

Failure: click any box to break it

The origin shield goes down

What breaks

The shield tier in the origin region is down or overloaded.

How we notice

Edge-to-shield errors and timeouts; origin request rate jumps.

What users see

Edges fail over to a secondary shield or straight to the origin.

Every PoP now misses separately, so origin load rises about five times (2% of requests to 10%).

What we do

A secondary shield in another region, and an origin sized to survive its share of edge misses for a while.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

A cold cache overloads the origin. Three things empty the cache at once: a purge of everything, a new CDN configuration that changes the cache key, or a deploy that renames every file. The origin was sized for 2% of traffic, and it gets 50% for the first minutes. It slows down. The edge’s origin requests time out, and the edges retry. The origin falls over, at the moment everyone is watching.

Fixes:

• Version assets, and purge by tag instead of purging everything.
• Keep a shield with request collapsing.
• Rate limit origin fetches at the shield.
• Roll out cache-key changes a few PoPs at a time.

Cache poisoning. The edge keys on host and path. But the origin builds the response from something else as well, an unkeyed input. Here’s a classic case. The origin uses the X-Forwarded-Host header to build absolute URLs. An attacker sends one request with X-Forwarded-Host: evil.example. The response’s script tags now point at the attacker’s host, and the edge caches it under the normal key. Every user who loads that page in the next hour runs the attacker’s script.

Fixes:

• The origin never uses a request header that is not in the cache key, unless it is checked against an allowlist.
• The CDN strips headers the origin does not need.
• Error responses are cached briefly, if at all.

Serving private data from a shared cache. This is the opposite direction. A response that depends on the user is stored as if it were public. It happens three ways:

• The origin forgets private on a personalized page.
• A CDN rule caches by file extension. Then /account/settings/x.css, which the app routes to the settings page, is cached as a stylesheet. This attack is called web cache deception.
• A caching rule is changed under load.

On Christmas Day 2015, Steam changed a caching configuration while absorbing a traffic flood. For about 90 minutes, it served account pages, with personal details, to the wrong users. Valve put the number of affected users at about 34,000.

Defenses: private, no-store by default on authenticated responses. Cache by Cache-Control, not by file extension. And alert on any cached response with Set-Cookie.

Web cache deception, one request from the attacker's link

Scroll or pinch to zoom, drag to pan

• Color key
• Service
• Cache

Stale content that cannot be recalled. Say a bad bundle sits at a stable URL with max-age=86400. It stays broken for a day, in every browser that loaded it. Keep browser lifetimes on stable URLs short (max-age=60, s-maxage=3600). Long ones are for versioned URLs only.

The CDN itself fails. Rarely, and globally. On 8 June 2021, a customer’s valid configuration change triggered a hidden bug at Fastly. 85% of its network returned errors. 95% had recovered within 49 minutes. Defenses: multi-CDN with DNS failover for critical sites, or an origin that can take some traffic directly.

Failure	Blast radius	Detect	Fix
Purge or key change empties cache	Origin, then everything	Origin request rate and shield miss rate jump together	Versioned URLs, tag purges, shield collapsing, staged config rollout
Origin down	Anything not cached	Edge 5xx from origin fetches	stale-if-error, second origin, long TTLs
Cache poisoning	Everyone who requests that key	Cached responses that differ from the origin’s	No unkeyed inputs, strip headers, short TTL on errors
Private page cached	Every user served that copy	Set-Cookie or user data in a response with a hit status	Private by default, cache by header not extension
Hit ratio collapse from the key	Origin load and egress bill	Hit ratio drops after a release or marketing campaign	Allowlist query params, no cookies in the key, normalized Vary
CDN-wide outage	The whole site	External synthetic checks from several networks	Multi-CDN with DNS failover, or origin able to take direct traffic

At scale

Numbers to remember.

What	Typical figure
Light in fiber	About 200 km per ms, so about 1 ms of round trip per 100 km
Round trips, Frankfurt to Virginia / Sydney to Virginia	About 90 ms / 200 to 230 ms
New HTTPS connection	3 round trips to the first byte with TLS 1.3, 4 with TLS 1.2
User to nearest PoP in a big metro	About 5 to 20 ms round trip
Hit ratio, versioned static assets and media	95 to 99%+
Hit ratio, a whole site with short-TTL HTML	Often 80 to 90%
Purge	Under a second to a few seconds (Fastly, Cloudflare); a minute or more (CloudFront)
CDN egress price	About $0.085/GB at list for low volume, a cent or two with commitments
Cloud internet egress	About $0.05 to $0.09/GB at list
Edge DNS answer TTL for steering	20 to 60 s

At 10x (400,000 requests a second), a pull CDN with no shield sends 40,000 misses a second, from hundreds of PoPs. Every point of hit ratio is 4,000 origin requests a second. Add a shield, and audit the cache key. Move assets to versioned URLs, and purges to tags. Negotiate a committed egress rate.

At 100x (4 million requests a second, tens of Tbps), the CDN is the system. Video services add caches inside ISPs, and pre-position tomorrow’s popular content overnight. Multi-CDN appears, sending each country to whichever CDN is faster or cheaper that hour. And cache-key and TTL policies become code, rolled out PoP by PoP like any deploy.

What to monitor:

• Request and byte hit ratio at the edge and at the shield. A drop after a release usually means a new query parameter, cookie or Vary header crept into the key.
• Origin request rate and bandwidth against capacity: the number a purge or key change moves first.
• Edge 5xx split by cause: origin errors, origin timeouts, and the CDN’s own errors.
• Time to first byte from real users, by country, for hits and misses. A country with good hit ratios and slow hits is a steering problem.
• Purge latency and purge failures. A purge that silently fails leaves deleted content online.
• Responses with Set-Cookie or private that were served from cache. Any count above zero is an incident.

Go deeperHTTP/3, connection coalescing and why the edge helps uncached requests2 min

HTTP/3 over QUIC combines the transport and TLS handshakes into one round trip. Repeat visitors resume in zero round trips. (That has replay limits, so it’s only for safe requests.) CDNs adopted it early, because it helps exactly the long, lossy mobile links their users are on. It does not change the origin side, where the edge already holds warm connections.

Connection coalescing. With HTTP/2 and HTTP/3, a browser reuses one connection for several hostnames if they resolve to the same IP and share a certificate. Splitting assets across static1.example.com and static2.example.com (the HTTP/1.1 trick of domain sharding) now costs extra DNS lookups and handshakes for nothing. Serve everything from one CDN hostname.

Why dynamic requests speed up. Take an uncached API call from Sydney. The client does its three handshake round trips with the Sydney PoP, about 15 ms in total. Then the PoP sends the request on an already-open connection to Virginia: one round trip, about 210 ms. Without the edge, the client pays all three round trips to Virginia, about 630 ms. The bytes still cross the Pacific once. The handshakes no longer do.

In the interview

Interviewer asksYour static assets are on a CDN but the hit ratio is 60% and the origin bill keeps rising. How do you find and fix it?

First, the cache key. Hit ratios that low on static files almost always mean the key is split. Common causes are utm_* and fbclid in the query string, a Vary: Cookie or Vary: User-Agent from the framework, or a forwarded cookie. I count distinct keys per path for the top URLs by origin requests. Then I allowlist the query parameters that change the response, and keep cookies and User-Agent out. Next, TTLs: hashed asset names with max-age=31536000, immutable. Last, a shield, because the long tail misses separately in every PoP. Origin load is the miss ratio, so going from 90% to 99% cuts it ten times.

Interviewer asksYou need to push a fix to a JavaScript bundle right now. It is cached for a day. What do you do?

If the bundle has a hashed name: deploy the new file, switch the HTML to it, and purge the HTML. Users get the fix on their next page load. Now say it lives at a stable URL, with a one-day browser TTL. A purge fixes the edge in seconds. But browsers keep their copy for up to a day, and nothing reaches them. The only lever is the HTML: purge it, and point it at a new URL for the bundle. The lasting fix is to version every asset, and keep browser TTLs short on anything unversioned.

Interviewer asksHow do you serve paid videos from a CDN without letting people share the links?

The app checks entitlement. Then it gives the player a signed cookie, or signed URLs, covering that video’s path prefix. It’s valid for a few minutes to a few hours, and signed with a key the CDN knows. The edge checks the signature and the expiry on every segment request. It caches on the path alone, so every subscriber shares one cached copy. A shared link stops working when it expires. For stronger control, the token can include the user’s IP range, or be checked against a revocation list in edge compute. Protecting the stream from paying users who copy it is DRM. That’s a separate layer from the CDN.

Common mistake

A CDN is only for static files

Uncached requests gain from edge TLS and warm origin connections. The edge absorbs DDoS traffic. And short-lived public API responses cache well. Cache a leaderboard or a queue position for one or two seconds, and millions of polls become one origin request per second per PoP. What a CDN must never do is store a response that depends on who is asking.

Mid-level covers

Puts static files and media behind a CDN, explains that it cuts latency by serving from nearby, sets a TTL, and purges when files change.

Senior adds

Explains the latency in round trips, and reasons in miss ratio. Designs the cache key: allowlisted params, no cookies, normalized Vary. Uses s-maxage and stale-while-revalidate. Versions assets instead of purging. Adds a shield for collapsing. Signs URLs, and strips the signature from the key. Names cold-cache overload, poisoning and private-data leaks as the failure modes.

Seen in

• Video streaming: versioned segments cached for a year, and 60 s manifests. Tokens are stripped from the key. A shield plus ISP caches cut the egress bill.
• Pastebin: edge TTLs capped at each paste’s expiry so expiry needs no purge, retried purges on delete, and private pastes kept off the CDN.
• URL shortener: the hottest redirects are answered at the edge with s-maxage=60, taking about 60% of traffic before it reaches any region.
• Instagram: every image and video comes from the CDN, at one versioned URL per rendition, cached for a year. CDN egress is the main cost.
• News feed: media lives in blob storage behind a CDN, and private media uses short-lived signed URLs plus a purge on delete.
• Typeahead: one- and two-character prefixes are cached at the CDN for 5 minutes, which takes a large share of all requests off the servers.
• Ticketmaster: the waiting-room status is one small file, cached for 2 seconds. Millions of polls become about one origin request per PoP every 2 seconds.

Recap

The job. Serve repeat requests from a cache a few milliseconds from the user. That cuts latency, origin load (it sees only the misses) and the egress bill.

Reaching an edge. Anycast fails over in seconds. DNS steering gives finer control, but sees the resolver, not the user. Pull fetches on the first miss, and is the default. Push uploads ahead of huge, predictable demand.

The cache key. Host, path and only the query parameters that change the bytes. No cookies, no tracking parameters, Vary normalized. Leaving out an input that does change the response leaks content between users.

Freshness. s-maxage for the edge. Short max-age for browsers on stable URLs. stale-while-revalidate and stale-if-error on everything cacheable. private, no-store on anything personal. Versioned URLs with immutable are the default for assets. Purges are for HTML and deletes.

Scale. A shield turns many PoP misses into one and collapses concurrent misses. Private files use signed URLs or cookies, with the signature kept out of the key.

Biggest risks. A cold or purged cache overloading the origin. Cache poisoning through unkeyed inputs. Private pages cached as public. Browser copies nobody can purge. And the CDN itself failing.

Further reading

• On this site: caching for TTLs, stampedes and single-flight. Load balancing for anycast, ECMP and GSLB below the CDN. And the video streaming problem, for CDN cost math, shields and ISP caches at full scale.
• RFC 9111, HTTP Caching: the standard for Cache-Control, freshness, Age, validation and Vary.
• RFC 5861, HTTP Cache-Control Extensions for Stale Content: stale-while-revalidate and stale-if-error.
• Practical Web Cache Poisoning, James Kettle, PortSwigger Research: unkeyed inputs and how real caches were poisoned.