Design a video streaming service

Design a video streaming service like YouTube or Netflix. Start with one server and one file, grow it with resumable uploads, a transcoded ladder of segments, a CDN and views off the hot path, then stress it with parallel transcoding, how the player picks quality, an origin shield and caches inside internet providers, with code.

Senior26 min core · 45 min full depth

Read first: Estimation CDNs
Scaling patterns: Hot keys and hot partitions Backpressure and load shedding
Building blocks: CDNs Blob storage Caching Queues and streams Partitioning Consistency models
Related: Instagram Dropbox / Google Drive

“Design YouTube.” Or Netflix. Either way, the prompt sounds like it’s about video files. Most of the answer turns out to be about bytes, caches and money.

A video service has two halves that share almost nothing. The upload side takes a large file from a flaky connection, turns it into a dozen versions in minutes, and must never lose it. The watch side serves those versions at tens of terabits a second. Nearly every decision there is about where a byte comes from, and what it costs.

So this page starts with one server and one file, and lets the numbers grow it. Then it tests the result against long uploads, bad networks and a viral video.

The answer in one minute

If you only have a minute, this is the design. The rest of the page explains each part.

Uploads go straight to storage, in parts. A dropped connection costs one 8 MB part, and our servers never carry the bytes.
Each video is encoded once into a ladder. Six qualities, from 240p to 4K, each cut into 4-second segments that line up exactly. The player can switch quality between any two segments.
The player picks the quality. Before every segment, it looks at its buffer and its download speed. The servers stay simple caches.
Caches carry almost everything. A CDN, then a shield cache per region, then our own cache boxes inside internet providers. Delivery is the biggest bill, so every point of cache hit rate is money.
Views are counted off the playback path. Players send events, a stream job removes repeats, and counts are a minute behind.

Scope it

A few questions decide the shape of the catalog, and so the caching.

Interviewerminute 0

Design a video streaming service like YouTube.

What they are testing

Whether you separate upload from playback. And whether you ask about the catalog shape: a long tail of user uploads, or a curated library. That decides the caching strategy. Three minutes. Asking about codecs before asking about scale reads as trivia, not design.

It’s worth saying out loud: “I’ll treat this as two systems. An upload pipeline measured in minutes, and a delivery system measured in terabits and dollars.”

These four questions matter most:

On demand or live? On demand. Live is a short extension at the end.
How much watching? 200 million daily viewers, an hour each.
How many uploads? 1 million a day, 10 minutes on average.
User uploads or a curated catalog? User uploads: a huge long tail. A Netflix-sized catalog of a few thousand titles could be copied almost whole into every cache.

From these answers we make five promises:

Playback always starts. 99.99% of video starts succeed. Uploads: 99.9%.
Fast start. The first frame within 1 second for most viewers, 2 seconds at worst.
Smooth. Stalls under 0.5% of watch time.
Never lose an upload. Every smaller version can be rebuilt from the original. The original can’t.
Quick to publish. A 10-minute upload plays at 720p within 5 minutes, and has every quality within 30.

We leave out search, recommendations and comments, which are separate systems, and ads and creator payments, which read our view events.

Go deeperMore questions, and what each changes1 min

Question	We assume	If the answer were different
Which devices?	Phones, browsers, TVs, up to 4K	Browsers only means one codec family and a smaller ladder
How fresh are counts?	About a minute behind	Counts that pay creators need an exact batch pipeline beside it
Paid titles?	Some, under DRM	Free only removes the license service
100 times more uploads?	No	Hardware encoders become worth building

The numbers that decide it

Now we work out the numbers. The one that matters most is in dollars.

Assumptions

Daily viewers: 200 million
Minutes watched per viewer a day: 60
Delivered bitrate, Mbps: 3
Peak over average: 2×
Segment length, seconds: 4
Edge hit ratio, by requests: 90%
CDN price, $ per TB: 20
Uploads a day: 1 million
Upload length, minutes: 10
Source bitrate, Mbps: 10
Stored bytes per source byte: 2 to 1
Encode CPU seconds per video second: 8
Years to plan for: 5 years
Storage price, $ per TB-month: 10

Watching8.3Mviewers; 17M at peak

200 million viewers × 60 minutes ÷ 1,440 minutes a day is 8.3 million watching at once, 17 million at the evening peak. Each player fetches one 4-second segment every 4 seconds: 4.2 million requests a second.

So we this is a caching problem: millions of requests a second for files that never change.

Egress50 Tbpsat peak; $160M a month

17 million viewers × 3 Mbps is about 50 terabits a second at peak. At $20 a TB, a big-contract CDN price, that’s $160 million a month, 2.7 cents per hour watched.

So we design around bytes delivered: every point of hit rate and every bit saved is money.

Uploads12/s8.7 GB/s in

1 million a day is only 12 a second. Upload traffic is 0.14% of what goes out.

So we uploads are a reliability problem (resumable, never lost), not a bandwidth one.

Transcoding56,000CPU cores

Encoding the ladder costs about 8 CPU-seconds per second of video.

So we split encoding into many small tasks, on a large fleet of cheap machines.

Storage1.5 PBa day; 2.7 EB in 5 years

Originals plus their versions come to about twice the source size. At $10 per TB-month, that’s about $27 million a month by year five.

So we move old originals to cold storage, but delivery is still about six times the bigger bill.

Video starts are about 28,000 a second at peak, so views can’t be an UPDATE on the video row. Try the calculator: drop the CDN hit rate from 90% to 85%, and the origin passes its limit. That’s what the shield cache is for.

Change an assumption

Daily viewers200 millionMinutes watched per viewer a day60Delivered bitrate, Mbps3Peak over average2×Segment length, seconds4Edge hit ratio, by requests90%CDN price, $ per TB20Uploads a day1 millionUpload length, minutes10Source bitrate, Mbps10Stored bytes per source byte2 to 1Encode CPU seconds per video second8Years to plan for5 yearsStorage price, $ per TB-month10

Concurrent viewers, average: 8.3 million
Concurrent viewers, peak: 17 million
Segment requests, peak: 4,200,000/s; Every one is a GET for an immutable file, so every one is cacheable.
Egress, peak: 6.3 TB/s; Over 8 Tbps: no origin serves this. A CDN is mandatory, and its hit ratio sets the origin size.
Origin requests, peak: 420,000/s; Inside the origin fleet’s 600,000 a second, with little room to spare.
Delivered a month: 8100 PB
CDN bill a month (dollars): 160 million; Big enough that owning caches inside ISPs, with no per-GB fee, beats paying per GB for the popular head.
Delivery cost per hour watched (cents): 2.7; The unit cost to drive down: fewer bits per hour, or cheaper bits.
Uploads, average: 12/s
Upload ingest, average: 8.7 GB/s; A tiny fraction of egress. Uploads are a reliability problem, not a bandwidth one.
Encode cores busy around the clock: 56,000; Hundreds of 64-core machines: a real fleet, but a small bill next to delivery.
Stored a day: 1.5 PB
Stored over the plan: 2700 PB
Storage bill a month, end of plan (dollars): 27 million
Delivery bill over storage bill: 5.9; Delivery, not storage, is the bill: hit ratio and bits per hour are what to optimize.

Build the design

We start with the simplest design that works. Then we let the numbers break it, one at a time. Each step fixes only what broke, adds only the boxes that fix needs, and says what the fix costs. Throughout, follow one scenario: someone uploads a video, and viewers watch it on everything from a phone on 3G to a 4K TV.

Step 1. One server, one file

The simplest version is one app server. It writes each uploaded file to blob storage and a row to a metadata database. On playback, it streams the original file back, and adds one to the video’s view count.

Video streaming service · architecture

Stage 1: One server, one file

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Segment requests per second: 4,200,000

Stage 1 of 6

One server, one file

Why this stage

Nothing yet.

This is the smallest design that lets someone upload a video and someone else watch it.

What we add

An app server that writes the uploaded file to blob storage and a row to a metadata database, and streams the original file back on playback.

What it costs

Every byte of every view passes through one server, the original plays badly on most devices and networks, and every view is an UPDATE on the video row.

At 4.2M segment requests per second, API servers is 210000% busy. Stage 2 is the fix.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

It breaks almost at once. A 10 Mbps 1080p original stalls on a 2 Mbps phone connection, and many devices can’t decode it. One server with a 25 Gbps network card serves about 2,000 segments a second, and the peak is 4.2 million.

Step 2. Uploads that resume

Step 1 takes each upload as one long request through our server. A 2 GB file from a phone takes many minutes, one dropped connection starts it over, and the server is tied up the whole time.

The creator’s app asks our API for an upload. It gets back presigned URLs: links that let it write straight to blob storage, for a limited time. It sends 8 MB parts, several at once. If the connection drops, it asks which parts arrived, and sends only the rest.

A resumable upload survives a dropped connection

Scroll or pinch to zoom, drag to pan

Color key
Database
Service

Uploads now survive bad networks, and their bytes never pass through our servers. But we still stream the original file to every viewer.

Step 3. Transcode once, stream segments

The original file is the wrong thing to stream. It’s 10 Mbps in whatever format the phone chose, and it would stall on a 2 Mbps connection. So each upload is transcoded once, into a ladder of versions:

Rung	Resolution	Bitrate	Who gets it
240p	426×240	300 kbps	Weak mobile, the very start
360p	640×360	700 kbps	A typical phone on cellular
480p	854×480	1.2 Mbps	Small screens on good networks
720p	1280×720	2.5 Mbps	Laptops, good phones
1080p	1920×1080	5 Mbps	TVs, desktops
2160p	3840×2160	16 Mbps (VP9, AV1)	4K TVs, popular videos only

Each version is cut into 4-second segments, listed in a manifest file. The key detail: every version has its keyframes (full pictures the decoder can start from) at exactly the same moments. So segment 17 at 360p and segment 17 at 1080p cover exactly the same 4 seconds. That’s what lets the player switch quality between any two segments without a glitch.

The work is split into many small tasks, so a 10-minute upload is ready in about two minutes instead of ten. The first stress test shows how.

Video streaming service · architecture

Stage 2: Transcode once, stream segments

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 2 of 6

Transcode once, stream segments

Why this stage

A 10 Mbps 1080p original stalls on a 2 Mbps phone connection, many devices cannot decode it, and one server with a 25 Gbps NIC serves about 2,000 segments a second against a 4.2 million peak.

What we add

Resumable uploads straight to blob storage, a job queue, and transcoders that split each upload, encode a ladder of renditions and package 4 second HLS and DASH segments.

API servers only hand out URLs.

What it costs

Every segment request now lands on blob storage, which serves about 600,000 a second.

The peak is 4.2 million.

At 4.2M segment requests per second, Blob storage is 700% busy. Stage 3 is the fix.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

Follow one upload, from the first part to a playable video:

Video streaming service · architecture

Trace a request: Upload one part

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasionalHop in motionHops doneResponse

Upload one part: step 1 of 5

Viewers to API servers

POST /v1/uploads the first time, or GET /v1/uploads/{id} after a dropped connection. The API answers with the part size and presigned URLs for the next parts.

80 ms of a 2000 ms budget

Viewers → API servers80 ms
API servers → Metadata DB5 ms
Viewers → Blob storage1350 ms
Blob storage → Job queueasync
Response

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

Step 4. A CDN in front

Every segment request now lands on blob storage, which serves about 600,000 a second. The peak is 4.2 million. But segments are the perfect thing to cache: written once, never changed, the same for every viewer. A CDN fetches a segment from our origin on its first miss at each location, then serves it from there. Three settings make it work:

Segments are cached for a year. They never change, because a re-encode writes a new version folder, like v_42/v4/ beside v_42/v3/.
Manifests are cached for a minute, because new qualities are added after publishing, and a takedown must take effect.
The cache ignores the viewer’s token. Each URL carries a signed token, checked at the edge. Then the edge caches by path alone, so all viewers share one copy.

At the peak, the CDN answers about 90% of requests, and the origin sees the other 420,000 a second.

Video streaming service · architecture

Stage 3: A CDN in front

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 3 of 6

A CDN in front

Why this stage

At the 4.2 million a second peak, the origin is 700% busy.

Segments never change once written, so any copy anywhere is safe to serve.

What we add

A pull-through CDN.

Players fetch manifests and segments from the nearest edge, and an edge miss reads from blob storage.

What it costs

The long tail misses at every edge separately, and about 10% of requests still reach the origin.

At 4.2M segment requests per second, Metadata DB is 560% busy. Stage 4 is the fix.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

Go deeperWhat decides the CDN hit rate1 min

A CDN location can only serve what it already holds, and each one has limited disk. So the hit rate depends on how much of what people watch fits in that disk.

Video popularity is very uneven. A small share of videos draws most of the views, so a location that keeps the most popular few terabytes answers most requests. The rest is the long tail: millions of videos, each watched rarely. A long-tail segment is often not in the cache, so it misses, and the miss goes to the origin.

Two choices push the hit rate up. The first is segments, not whole files, because people stop watching early. The first minute of a video is requested far more than the last, and a segment cache keeps only the parts people watch. The second is one URL per segment for everyone. That’s why the token is checked at the edge, and left out of the cache key. The ten-times stress test adds a third: a shared cache per region. Then the long tail misses once per region, instead of once per city.

Step 5. Counting views off the path

One write is still on the path of every play. Every video start is a view: 28,000 a second at peak, and a viral video puts thousands a second on one row. So players send events in batches every 30 seconds. The API writes them to Kafka and answers at once. A stream job decides which events are views, removes repeats, and adds one total per video every few seconds. The deep dive at the end of this step shows how.

Video streaming service · architecture

Stage 4: Views off the hot path

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 4 of 6

Views off the hot path

Why this stage

Every video start increments a counter on the video row: 28,000 writes a second at peak, many on the same few viral rows, against a primary that takes about 5,000.

What we add

Players send view events to the API, which writes them to Kafka.

A stream counter deduplicates them per session and writes per-partition running totals to a counts store, summed on read.

What it costs

View counts lag by up to a minute, and the counting rules now live in a pipeline instead of one UPDATE.

At 4.2M segment requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

Follow one video start, from pressing play to the first frame:

Video streaming service · architecture

Trace a request: Start a video

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasionalHop in motionHops doneResponse

Start a video: step 1 of 6

Viewers to API servers

GET /v1/videos/{id}/play from a phone: DNS, a resumed TLS session and one round trip to the nearest region. The API checks the session and the video’s visibility.

90 ms of a 2000 ms budget

Viewers → API servers90 ms
API servers → Metadata DB3 ms
Viewers → CDN edge60 ms
Viewers → CDN edge160 ms
CDN edge → Blob storage120 ms
Response

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

Most of the time goes on round trips. So the play API returns the manifest link and where you left off, in one call. And the first segment comes from a low quality, so it arrives fast.

This is enough for a mid-level answer. Resumable uploads. A ladder of aligned segments. A CDN with the right cache settings. And views counted off the path. The stress tests are what make it a senior answer.

Go deeperThe API, the manifest and the stored files2 min

GET /v1/videos/v_42/play
200 OK
{ "manifests": {
    "hls":  "https://edge.example-cdn.net/v_42/v3/master.m3u8?token=...",
    "dash": "https://edge.example-cdn.net/v_42/v3/manifest.mpd?token=..." },
  "resumeAt": 312.5 }

The token is signed by our API, limited to this video’s folder, and valid for 6 hours. Edges check it without calling us, so a segment request never touches the API.

One set of segments serves both Apple devices (HLS) and everything else (DASH), in a shared format called CMAF. So each version is stored and cached once, not twice. A version’s playlist looks like this:

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-MAP:URI="init.mp4"
#EXTINF:4.000,
000001.m4s
...
#EXT-X-ENDLIST

init.mp4 holds the codec setup, so each segment carries only frames. ENDLIST tells the player the list is complete.

Video and task rows live in sharded relational storage, because the scheduler needs conditional updates. YouTube built Vitess to run its metadata on sharded MySQL. View counts and resume positions are high-volume simple writes, so they live in a wide-column store like Cassandra.

Go deeperHow a view is counted, even for a viral video2 min

A video goes viral: 2 million views in an hour. How does the count stay correct and fast?

A view is a business rule, not an event. For example: “at least 30 seconds of playback, once per session and video, bots removed”.

Events are split across the stream job by session, not by video, so a viral video’s events spread over every partition. Each counter instance remembers which (session, video) pairs it saw in the last day, and keeps a running total per video. Every 5 seconds, after saving its state, it overwrites a row keyed (video, partition) with that total. An overwrite, unlike an increment, makes a repeated write harmless. A count is the sum of at most 100 such rows.

Take a video with 2 million views an hour. Each of 100 partitions gets one write every 5 seconds, instead of 550 writes a second on one row. The watch page caches the rounded total (“1.2M views”) for 30 seconds. A nightly batch recount is what creators are paid on.

Does exactly-once processing count each view once? No. Exactly-once in a stream processor means each event changes the state once, even across restarts. It says nothing about a player that sends its start event twice after a retry, or two tabs playing the same video. Counting each view once is a business rule on (session, video), needed whatever the pipeline promises.

Go deeperOther ways to count views1 min

Approach	How it works	Problem
UPDATE per view	Add one to the video row	One row lock per viral video
Redis INCR per view	Add one to a key in memory	No repeat removal, no rules, one hot key
Stream counter, batched	Kafka, a counter with saved state, one total per video every few seconds	A minute of lag
Nightly recount	Recount from raw events	Exact and audited, but a day late

The saved state matters. Writing only totals that have been saved means a restarted counter never moves a row backwards. The public count uses the stream. Money uses the nightly recount.

Stress it

The design is complete. Now we test it where video is hard. A two-hour upload and a crashed transcoder. A phone on a bad network. Ten times the traffic. And the bill. Only the last two add boxes.

Transcoding in minutes, safely

Interviewerminute 20

A two-hour upload arrives. How do you have it ready in minutes, and what happens when a transcoder dies halfway?

What they are testing

Whether you split the encode into small tasks that are safe to retry, and keep their segments aligned.

Encoding a two-hour file on one machine takes hours, and one crash starts it over. So we split the source at keyframes into 20-second chunks, encode every chunk of every quality as its own task, then join the results. A two-hour film becomes 360 chunks × 5 qualities = 1,800 tasks, running in parallel. A chunk is a whole number of 4-second segments, so chunk edges are segment edges, and joining never cuts a segment in two.

Workers crash, get pre-empted, and freeze. So every task must be safe to run twice:

Predictable task IDs, like v_42/v3/encode/720p/07. A repeated “upload complete” event builds the same tasks, not a second set.
Leases of 2 minutes, renewed every 30 seconds. A dead worker’s task becomes free again.
Attempt numbers as fences. Only the current attempt may complete a task. A worker that wakes from a long pause can’t overwrite its replacement’s result.
One output file per attempt. Completing is a conditional update that records which attempt’s file is the real one. A stale worker’s file is never used, only cleaned up later.

TaskScheduler.javatask_scheduler.pyTaskScheduler.cstask_scheduler.gotask-scheduler.ts

// One upload's transcode work is a DAG: split the source at keyframes, encode every chunk at
// every rendition, then package each rendition into segments and manifests. Production keeps
// each task as a row in the metadata DB and each method below is one conditional UPDATE; a lock
// stands in for that here.
enum TaskState { WAITING, READY, RUNNING, DONE }

final class Task {
    final String id;                                    // deterministic, such as "v_42/v3/encode/720p/07"
    final List<String> dependents = new ArrayList<>();  // tasks that wait on this one
    int waitingOn;                // unfinished dependencies; the task becomes READY at zero
    TaskState state = TaskState.WAITING;
    int attempt;                  // bumped on every claim: the fencing token
    long leaseUntil;              // a worker that goes quiet past this loses the task
    String output;                // blob key of the one accepted output

    Task(String id) { this.id = id; }
}

record Claim(String taskId, int attempt, String outputKey) {}

final class TaskScheduler {
    static final long LEASE_MS = 120_000;  // a 20 s chunk encodes in under a minute

    private final Map<String, Task> tasks = new LinkedHashMap<>();  // oldest first
    private final LongSupplier clock;      // injected so tests control lease expiry

    TaskScheduler(LongSupplier clock) { this.clock = clock; }

    // The "upload complete" event arrives at least once, so the same DAG can be submitted twice.
    // Task ids are deterministic, so the second submission changes nothing.
    synchronized void add(String id, List<String> dependsOn) {
        if (tasks.containsKey(id)) return;
        Task t = new Task(id);
        for (String d : dependsOn) {
            Task dep = tasks.get(d);       // dependencies are always added first
            dep.dependents.add(id);
            if (dep.state != TaskState.DONE) t.waitingOn++;
        }
        if (t.waitingOn == 0) t.state = TaskState.READY;
        tasks.put(id, t);
    }

    synchronized Optional<Claim> claim() {
        long now = clock.getAsLong();
        for (Task t : tasks.values()) {
            // A RUNNING task whose lease ran out belongs to a dead or stuck worker: take it over.
            boolean abandoned = t.state == TaskState.RUNNING && t.leaseUntil < now;
            if (t.state != TaskState.READY && !abandoned) continue;
            t.state = TaskState.RUNNING;
            t.attempt++;
            t.leaseUntil = now + LEASE_MS;
            // Each attempt writes its own key, so a slow earlier attempt can never overwrite the
            // output that was accepted. Keys that were never accepted are garbage collected.
            return Optional.of(new Claim(t.id, t.attempt, outputKey(t.id, t.attempt)));
        }
        return Optional.empty();
    }

    // Workers call this every 30 s, so a long encode keeps its task for as long as it is alive.
    // False means the task was taken over: stop work and throw the output away.
    synchronized boolean heartbeat(String id, int attempt) {
        Task t = tasks.get(id);
        if (t.state != TaskState.RUNNING || t.attempt != attempt) return false;
        t.leaseUntil = clock.getAsLong() + LEASE_MS;
        return true;
    }

    // The output key comes from the attempt, never from the caller, so the commit below is a
    // compare-and-set on the fencing token: what gets recorded is exactly this attempt's file.
    static String outputKey(String id, int attempt) { return id + "/attempt-" + attempt; }

    synchronized boolean complete(String id, int attempt) {
        Task t = tasks.get(id);
        // A repeated report from the accepted attempt (a retried RPC) is fine; any other is not.
        if (t.state == TaskState.DONE) return t.attempt == attempt;
        // Fencing: only the latest attempt may finish the task. A worker whose lease was taken
        // over learns it here, even if it never noticed the lease expiring.
        if (t.state != TaskState.RUNNING || t.attempt != attempt) return false;
        t.state = TaskState.DONE;
        t.output = outputKey(id, attempt);  // the packager reads only this key
        // Dependents are unlocked exactly once, because only one call makes the move to DONE.
        for (String d : t.dependents) {
            Task next = tasks.get(d);
            if (--next.waitingOn == 0) next.state = TaskState.READY;
        }
        return true;
    }
}

A stalled worker comes back after its task was taken over

Scroll or pinch to zoom, drag to pan

Color key
Service
Database

Low qualities and big creators go first, so videos play at 360p and 720p early. A task that fails 5 times is set aside, because a corrupt chunk fails on every machine. And since a pre-empted machine is just another crash, the whole fleet can run on cheap, interruptible machines.

Go deeperOther ways to split the encoding1 min

Approach	How it works	Wins	Loses
One job per video	One machine encodes the whole ladder	Simple; the encoder sees the whole file	Hours for long videos; a crash loses everything
One job per quality	Each quality on its own machine	5 times faster	Still limited by the file’s length
Chunks × qualities	Split at keyframes, encode each piece, join	Minutes for any length; a crash loses one chunk	Chunk edges need care; a scheduler to run

The job queue only carries “wake up, there’s work” messages. The task table is the truth. So losing the queue loses no work: a sweeper finds ready tasks and queues them again.

Go deeperWhere the two minutes go, and better encodes for popular videos2 min

Step	What happens	Time
Probe	Read the format and length; reject broken files	5 s
Split	Cut the source at keyframes into 30 chunks, without re-encoding	15 s
Encode	30 chunks × 5 qualities = 150 tasks in parallel	45 to 75 s
Package	Join chunks into 4-second segments and write manifests	15 s
Publish	Mark versions ready	1 s

A cost of chunks: the encoder sees only 20 seconds at a time, so it can’t spend bits wisely across the whole film. Netflix runs analysis passes first.

Better encodes for popular videos. A fixed ladder wastes bits on simple content, like a lecture, and starves complex content, like sports. Per-title encoding tries several encodes, scores them with a quality metric, and picks the ladder per video. AV1 saves 30 to 50% of bits against H.264, but costs several times the CPU. On a long tail of barely watched uploads, that doesn’t pay. So every upload gets a plain H.264 ladder, and only videos that become popular get the expensive encodes.

How the player picks a quality

The player has six qualities to choose from. How does it decide which one to fetch next?

Choosing the quality is the player’s job, before every segment. The server just publishes the ladder, which is why delivery can be simple caches. The player wants three things that conflict. Never stall. Show the best quality the network allows. And don’t flip back and forth, because viewers notice that more than a steady lower quality.

There are two signals. Download speed reacts fast, but on mobile it’s noisy. Buffer level, how many seconds of video are already downloaded, is stable, but slow at the start, when the buffer is empty. So players combine them: speed while the buffer is thin, buffer level once it’s deep. Netflix’s 2014 study found that choosing by buffer in steady play cut stalls by 10 to 20%, at similar quality.

BitratePicker.javabitrate_picker.pyBitratePicker.csbitrate_picker.gobitrate-picker.ts

// One rung of the encoding ladder, as the master manifest lists it.
record Rendition(String name, int kbps) {}

// Picks the rendition for the next segment. The player calls next() before every segment
// request and onDownloaded() after every response. Throughput decides while the buffer is thin
// or middling; once the buffer is deep it rides out dips, because throughput on a phone swings
// from one second to the next while a 30 second buffer barely moves.
final class BitratePicker {
    static final double LOW_S = 10;     // below this much buffer, avoiding a stall comes first
    static final double HIGH_S = 30;    // above this, the buffer can absorb a throughput dip
    // Hysteresis: step up only to a rung that needs at most 70% of the estimate, step down only
    // when the current rung needs more than 90%. Between the two, hold. One threshold would flap
    // every time the estimate wobbles around it.
    static final double UP_SHARE = 0.7;
    static final double DOWN_SHARE = 0.9;
    static final double ALPHA = 0.3;    // EWMA weight: follows a real change within about 3 segments

    private final List<Rendition> ladder;  // sorted by bitrate, lowest first
    private double kbpsEstimate;           // smoothed throughput; 0 means no sample yet
    private int current;                   // the rung being played

    // Starts from the last session's throughput on this device, so a good connection does not
    // open at 240p. With no history, pass 0 and start at the bottom rung.
    BitratePicker(List<Rendition> ladder, double startKbps) {
        this.ladder = ladder;
        this.kbpsEstimate = startKbps;
        this.current = highestFitting(UP_SHARE);
    }

    void onDownloaded(long bytes, double seconds) {
        double kbps = bytes * 8 / 1000.0 / seconds;
        // Smoothed, so one lucky or unlucky segment does not swing the choice.
        kbpsEstimate = kbpsEstimate == 0 ? kbps : ALPHA * kbps + (1 - ALPHA) * kbpsEstimate;
    }

    Rendition next(double bufferSeconds) {
        int up = highestFitting(UP_SHARE);
        int keep = highestFitting(DOWN_SHARE);
        if (bufferSeconds < LOW_S) {
            // Thin buffer: a stall is worse than any quality drop. Never step up, and drop to
            // a rung with a wide margin.
            current = Math.min(current, up);
        } else if (keep < current && bufferSeconds <= HIGH_S) {
            // The current rung no longer fits and the buffer is not deep enough to wait: down
            // at once, to the highest rung that does fit.
            current = keep;
        } else if (up > current) {
            // Up one rung at a time, so a burst of fast segments cannot cause a jump to 1080p
            // followed by a fall back to 360p. A deep buffer never steps down here.
            current++;
        }
        return ladder.get(current);
    }

    // The highest rung whose bitrate fits the given share of the throughput estimate.
    private int highestFitting(double share) {
        int best = 0;
        for (int i = 0; i < ladder.size(); i++)
            if (ladder.get(i).kbps() <= share * kbpsEstimate) best = i;
        return best;
    }
}

Three details carry most of the quality:

Start low, or where last session’s speed suggests. The first segment’s size is the biggest lever on time to first frame.
Leave a margin. Move up only to a quality that needs at most 70% of the measured speed. Move down only when the current one needs more than 90%. The gap stops a speed that wobbles near one quality from switching every segment.
Step up one level at a time, and down at once. With a deep buffer, don’t step down at all.

Go deeperOther ways to choose a quality1 min

Approach	How it decides	Wins	Loses
By speed	Highest quality under a share of measured speed	Reacts fast; works with no buffer	Mobile speed is noisy, so it flips
By buffer	Low buffer, low quality; full buffer, high	Stable; no stall while it’s full	Slow to climb at the start
Both	Speed while the buffer is thin, buffer once deep	Fast start and steady playback	Thresholds to tune per device type
Look ahead	Scores the next few segments’ choices together	Best measured quality	Needs a good guess of future speed

The open-source dash.js player switches between a speed rule and a buffer rule in a similar way.

Ten times the traffic, with an origin shield

Traffic grows ten times. At ten times, the origin is seven times over its limit. Most of that load is the same long-tail segment, missing separately at dozens of CDN locations. A video watched in 50 cities costs 50 origin reads per segment.

A shield is one big cache per region, between the CDN locations and the origin. Every location in the region misses to the shield, and only the shield misses to the origin. So each segment misses once per region, not once per city. The shield also collapses requests: 200 locations asking for the same uncached segment in the same second cause one origin read. That matters most right after a popular creator publishes, when every cache is cold at once.

Video streaming service · architecture

Stage 5: An origin shield

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 5 of 6

An origin shield

Why this stage

At 10 times the traffic, 4.2 million edge misses a second would need seven times the origin.

Most of them are the same long-tail segment missing at dozens of edges at once.

What we add

A shield tier: one large cache per region between the edges and the origin, with request collapsing, so each region misses on a segment once.

What it costs

One more hop on a miss, and a shield outage sends ten times the misses to the origin.

At 4.2M segment requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

The origin goes from 10% of requests to 1%. Most CDNs sell this as “tiered caching”. The cost is an extra hop of 20 to 40 ms on a full miss, and the origin must survive losing one shield.

Go deeperWhen a shield fails1 min

A shield that goes down sends ten times the misses from its region to the origin. So the origin is sized to survive losing one shield, and CDN locations fall back to a neighbouring region’s shield first. Shields are also where a cold start is felt: a new video from a creator with millions of subscribers is missing everywhere at once. Request collapsing turns those thousands of misses into one origin read per segment, per region.

The extra hop costs 20 to 40 ms only on a full miss, which is rare for anything people actually watch. A hit at the CDN location never reaches the shield at all.

The bill, and caches inside internet providers

The shield fixes the origin, but not the bill. The CDN bill is already $160 million a month.

The bill is bytes × price per byte. So send fewer bytes, or pay less for each. Better codecs for popular videos cut about 30% of their bytes, and popular videos are most of the bytes. But at ten times, delivery would cost $1.6 billion a month. That’s what forces the last step.

We put our own cache boxes inside the largest internet providers, as Netflix Open Connect and Google Global Cache do. The provider gives rack space and power, because the box saves them traffic on their own links. We pay for hardware, and nobody pays per GB. Each box holds tens to hundreds of terabytes.

Video streaming service · architecture

Stage 6: Caches inside ISPs

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 6 of 6

Caches inside ISPs

Why this stage

Cost before load: at 10 times the traffic the CDN bill passes $1.6 billion a month, and the CDN tier would need 525% of its capacity.

What we add

Cache appliances inside the largest ISPs, filled off-peak with the titles predicted to be popular tomorrow, and a steering step in the play API that hands each player a ranked list of caches.

What it costs

Hardware in hundreds of networks to build and run, a popularity predictor, and a steering service that must know every cache’s health.

At 4.2M segment requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

Two things make the boxes useful:

Filling ahead of time. Every night, each box fills with the videos predicted to be popular in its area tomorrow, while links are idle.
Steering. From the viewer’s address, the play API ranks the box in their provider, then a CDN location, then a second CDN. The player moves down the list on errors.

Video streaming service · architecture

Trace a request: Start from an ISP cache

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasionalHop in motionHops doneResponse

Start from an ISP cache: step 1 of 6

Viewers to API servers

The play API now also steers: from the client’s IP prefix it ranks the appliance inside the viewer’s ISP first, then an edge, then a second CDN, filtered by health and load.

90 ms of a 2000 ms budget

Viewers → API servers90 ms
API servers → Metadata DB3 ms
Viewers → ISP caches15 ms
Viewers → ISP caches120 ms
ISP caches → CDN edgeasync
Response

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

With the boxes answering 85% of requests, the CDN carries the long tail and absorbs box failures. Below a few tens of terabits a second, a commercial CDN is cheaper. This is a decision for our size, not for day one.

Go deeperThe long tail, and other levers on the bill2 min

Popularity is very uneven. A study of YouTube in 2007 found about 10% of videos drawing nearly 80% of views. The popular head caches beautifully. The tail is 1.8 billion videos after five years, far more than one location can hold. Three facts make it manageable:

Cache segments, not videos. Viewers give up early. So a segment cache keeps the first minute of millions of videos, and the whole of the popular ones.
Cache qualities, not ladders. 4K segments of an unpopular video never take space.
Shield the tail. Origin load grows with regions, not with locations.

Lever	Effect on the bill	Cost
AV1 or VP9 for popular videos	About 30% fewer bytes on most bytes	Encode CPU, device support
Per-title ladders	10 to 20% more, on the popular head	Trial encodes
Caches inside providers	Price per GB near zero for 85% of bytes	Hardware, operations, partnerships
Several CDNs	Price competition and resilience	Steering and monitoring per CDN

Why steer in the API, not DNS? DNS sees the viewer’s resolver, not the viewer, and caches its answer. It can’t tell which provider a viewer is in. It can’t react within seconds when a box fills up. The play API sees the viewer’s address, and knows which boxes hold this video. It returns several ranked links, so the player can fail over without another lookup.

When things break

Interviewerminute 40

An entire CDN has a bad hour. What do viewers see?

What they are testing

Whether playback slows rather than fails: ranked sources in the player, a second CDN, and an origin sized for the switch.

A viewer already watching should notice nothing worse than a quality drop. Their player holds a ranked list of sources, and several seconds of buffer. When the CDN fails, the player moves to the next source. Steering sends new sessions to the second CDN. That CDN always carries some traffic, so it’s never cold.

Under overload, shed in this order: view counts first, then uploads, then play API calls. Segments never. A viewer mid-video depends only on caches and their own buffer, and the design keeps it that way.

Click any box to break it and see what users would notice:

Video streaming service · architecture

Break it: what fails, who notices, what users see

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Break

Failure: click any box to break it

CDN edge PoP goes down

What breaks

One edge location goes down or degrades.

How we notice

Synthetic probes and per-PoP error and throughput rates from player telemetry.

What users see

Players that were on it rebuffer for a few seconds, then fetch from the next URL in their list.

What we do

The CDN routes around the PoP.

If a whole CDN degrades, steering shifts new sessions to the second CDN.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

The quiet failure is the dangerous one. A crash is loud. A subtly broken encoder release is silent, and spreads green frames or broken audio to every new upload. So encoder releases are tested on a sample of uploads with automatic quality and decode checks. And because encodes live in version folders, rolling back is just pointing the video row at the previous version.

Go deeperEvery failure, and what users see2 min

Part	Failure	What users see	What we do
Box in a provider	Down or full	A short stall, then the CDN	Player moves down its list; steering drops the box
CDN	A location, or all	Stalls for seconds; lower quality	Steering moves new sessions to the second CDN
Shield	Down	Slower starts for long-tail videos	Locations use a neighbouring shield
Blob storage	Region outage	Cold videos fail to start; uploads move	Copies in other regions
Metadata database	Primary down	Uploads pause 30 s; playback fine	Failover; leases expire and are retaken
Transcoder	Crash	A video ready a few minutes later	Fenced retry; set aside after 5 failures
Kafka or counter	Down	Counts freeze; playback fine	Buffer; restart from the saved state

What users can see. The uploader sees their video’s status at once. Others see a new video within seconds. A new quality appears within a minute, when the manifest’s cache expires. A takedown stops new starts at once, and is purged from caches within minutes. A view count can even go down, when fraud review removes bot views hours later.

Go deeperRunning it: alerts, DRM and live2 min

Wake someone up when:

players report more than 0.5% of starts failing, per CDN and provider
time to first frame passes 1.5 seconds for most viewers
stalls pass 0.5% of watch time
the cache hit rate drops 3 points (this predicts both an origin overload and a bill)
the oldest waiting transcode task is more than 5 minutes old

DRM. Paid titles are encrypted once when packaged, in a common format that Widevine, FairPlay and PlayReady can all read. The player gets a license per viewer after an entitlement check. Segments stay cacheable: encrypted once for everyone, with only the license per viewer.

Live streaming. The broadcaster pushes a stream to an ingest server. It encodes the ladder in real time, so there’s nothing to prepare ahead. Manifests change every few seconds. Classic HLS sits 20 to 30 seconds behind live. Low-latency HLS publishes partial segments, and gets to 2 to 5 seconds. Request collapsing becomes essential, because a million viewers ask for the same new segment in the same second.

Wrap up

The whole design

Here is everything we built, in the order we built it. Use the buttons to step through it.

Video streaming service · architecture

The full design, stage by stage

Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage

Segment requests per second: 4,200,000

Stage 6 of 6

Caches inside ISPs

Why this stage

Cost before load: at 10 times the traffic the CDN bill passes $1.6 billion a month, and the CDN tier would need 525% of its capacity.

What we add

Cache appliances inside the largest ISPs, filled off-peak with the titles predicted to be popular tomorrow, and a steering step in the play API that hands each player a ranked list of caches.

What it costs

Hardware in hundreds of networks to build and run, a popularity predictor, and a steering service that must know every cache’s health.

At 4.2M segment requests per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.The whole design. Step through it: one server and one file, transcoded segments, a CDN, views off the hot path, an origin shield, then caches inside internet providers.

The numbers behind the busy bars

What one segment is

A 4-second video segment averages 1.5 MB, about 3 Mbps.

What one machine can do

One server with a 25 Gbps network card serves about 2,000 segments a second.
A shield or ISP cache server serves about 8,000 segments a second (100 Gbps).
One API server handles about 10,000 requests a second.
The metadata primary takes about 5,000 writes a second. Replicas and in-process caches serve about 300,000 reads.

How big the fleet is

The origin fleet in front of blob storage serves about 600,000 segments a second, about 7 Tbps.
The contracted CDN tier carries about 8 million segments a second, about 100 Tbps.

What the caches save

The edge answers 90% of requests, and the shield 90% of the rest.
ISP caches answer 85% of all requests.

The decisions, and what each one costs

Decision	We chose	We gave up	We’d choose differently if
Uploads	Presigned parts, straight to storage	A simple single request	Files were small
Encoding	Once, ahead of time, in parallel chunks	Wasted encodes on unwatched videos	Few uploads were ever watched (encode lazily)
Format	4-second aligned segments, one set for HLS and DASH	Old devices need a separate copy	Only one device family mattered
Quality	The player chooses, by buffer and speed	Control on the server	Never
Delivery	CDN, shields, then boxes inside providers	Hardware and partnerships to run	We were below a few tens of Tbps
Views	Stream counting, overwrites per partition	A minute of lag	Counts paid money in real time

How to spend 45 minutes

0 to 4. Two systems: upload and watch. Ask about the catalog shape.
4 to 9. 17 million watching, 50 Tbps, $160 million a month, 56,000 cores.
9 to 20. Build it: one server, resumable uploads, the ladder and aligned segments, the CDN settings, views off the path. A mid-level answer can stop here.
20 to 30. The stress tests: transcoding under crashes, then the player.
30 to 40. Ten times: the shield, then the bill and caches inside providers.
40 to 45. A CDN failing, then your own questions.

Never skip aligned segments, the player choosing quality, or the bill in dollars.

Follow-up questions

Interviewer asksCan the public view count ever go down?

Yes, when fraud review removes bot views hours later. Keep that rare and batched. But it must never go backwards because one database copy lags another. So the cache only replaces a count with a larger one, unless a correction is flagged.

Interviewer asksA viewer pauses on a laptop and resumes on a TV. How?

The player sends its position with every event batch, every 30 seconds. It’s stored per user and video, last write wins. The TV’s play call returns resumeAt, so playback starts at most 30 seconds early.

Go deeperMore follow-up questions1 min

A creator replaces a video’s audio. How do caches stay correct? The new encode writes a new version folder, like v4, and the video row is pointed at it. Old segments are never overwritten, so a cache can miss but never serve the wrong bytes. The manifest’s one-minute cache means new viewers get the new audio within a minute.

How would this change for a Netflix-style catalog? A few thousand titles, not billions of uploads. The whole popular catalog fits in each cache box, so filling boxes overnight covers almost everything watched, and the hit rate is far higher. Every title is worth expensive per-title and per-scene encoding, because each is watched millions of hours.

How do you stop copyrighted uploads? Fingerprint each upload against reference files from rights holders, as YouTube’s Content ID does. Also match it against known illegal material before publishing. It’s one more task in the transcoding graph, so it runs in parallel with encoding.

Design a video streaming service

The answer in one minute

Scope it

The numbers that decide it

Build the design

Step 1. One server, one file

Stage 1: One server, one file

Why this stage

What we add

What it costs

Step 2. Uploads that resume

Step 3. Transcode once, stream segments

Stage 2: Transcode once, stream segments

Why this stage

What we add

What it costs

Trace a request: Upload one part

Step 4. A CDN in front

Stage 3: A CDN in front

Why this stage

What we add

What it costs

Step 5. Counting views off the path

Stage 4: Views off the hot path

Why this stage

What we add

What it costs

Trace a request: Start a video

Stress it

Transcoding in minutes, safely

How the player picks a quality

Ten times the traffic, with an origin shield

Stage 5: An origin shield

Why this stage

What we add

What it costs

The bill, and caches inside internet providers

Stage 6: Caches inside ISPs

Why this stage

What we add

What it costs

Trace a request: Start from an ISP cache

When things break

Break it: what fails, who notices, what users see

What breaks

How we notice

What users see

What we do

Wrap up

The whole design

The full design, stage by stage

Why this stage

What we add

What it costs

The decisions, and what each one costs

How to spend 45 minutes

Follow-up questions

Further reading

Keep reading

The designbeyondcode digest, every Saturday