Skip to content

System DesignWorked problems

Design a ride hailing service

Design a ride hailing service like Uber or Lyft. Start with one server and one table, grow it with positions in memory in H3 cells, an atomic claim on each driver, sockets to phones and a split by city, then stress it with every race, choosing the driver, and surge and payments, with code.

Senior28 min core · 45 min full depth

“Design Uber.” The rider taps a button, and a car shows up.

It sounds like a “nearest driver” query plus a database row. The trap is treating it as one problem. Two workloads hide in this prompt. A million phones report positions that are worthless 4 seconds later. And a few hundred requests a second must each end with exactly one driver. The design keeps them apart: positions in memory, assignments in a database.

So this page starts with one server and one table, and lets the numbers grow it. Then it tests the result against the races, the choice of driver, and surge pricing.

If you only have a minute, this is the design. The rest of the page explains each part.

  1. Keep driver positions in memory, per city. Only the newest position matters, and the next ping replaces a lost one. So 250,000 pings a second never touch a database.
  2. Claim a driver with a conditional update. “Set this driver to offered, only if they’re still available.” The database refuses the second claim, however many dispatchers race.
  3. Offer to one driver at a time, for 15 seconds. Pick the best road ETA, not the shortest straight line. If the driver says no, or nothing, try the next.
  4. Push over open sockets. Offers reach drivers in tens of milliseconds, and riders watch the car move. Events flow to Kafka for surge pricing and payments.
  5. Split by city, with a standby per city. Every dispatch question is local. When a city’s dispatcher dies, a standby takes over in seconds, and the database still refuses any double claim.

A few questions decide where positions live and how drivers are chosen.

Interviewerminute 0

Design a ride hailing service like Uber or Lyft.

What they are testing

Whether you separate the location stream from matching right away, and pin down how many drivers report how often, because that number decides where positions live. About three minutes. Asking about payments and ratings before drivers and pings reads as missing the hard part.

These four questions matter most:

  • How many drivers online? 1 million at the global peak, 400,000 on average.
  • How often does a driver report? Every 4 seconds while online.
  • How many trips? 20 million a day.
  • How is a driver chosen? The trip is offered to one driver at a time, who has 15 seconds to accept.

From these answers we make five promises:

  • Never double booked. A driver is on at most one trip, and a trip has at most one driver. This holds through retries, timeouts and failovers. It’s the one promise with no “usually”.
  • Always matching. Requesting and matching work 99.99% of the time in every city. A rider who can’t get a car opens a competitor’s app.
  • Fast. A price quote in under 300 ms. The offer leaves our servers within 1 second of the request. A driver’s position reaches the watching rider within 2 seconds.
  • Fresh. Matching only uses positions under 15 seconds old.
  • Charged once. Every finished trip is charged exactly once.

We leave out pooled rides, building the payment system itself (we hand trips to it), driver onboarding, ratings, and the inside of the routing engine.

Go deeperMore questions, and what each changes1 min
Question We assume If the answer were different
Shared or scheduled rides? Not in v1: one rider, one car, now Pooling turns matching into route insertion
How fresh is the rider’s car view? Within a couple of seconds Minute-level freshness would allow polling, and drop the push layer
Is the price fixed at request time? Yes, a quote locked for 2 minutes Metered fares move pricing to the end of the trip
Do we build payments? No: hand finished trips to a payment system See payment system
Do we keep location history? Yes, for receipts, disputes and safety Without it, the location stream needs no durable copy at all

Now we work out the numbers. The most important is how pings compare to requests.

Assumptions
Drivers online at peak
1 million
Seconds between pings
4
Average load as a share of peak
40%
Trips a day
20 million
Peak trips over average
2×
Minutes per trip, pickup included
20
Bytes per ping
100 bytes
Busiest city’s share of drivers
5%
Pings250,000/slocation updates

1 million drivers at peak ÷ 4 s. 100,000 a second on average.

So we positions can't go in a database: one primary takes about 5,000 writes a second, and each position is replaced 4 seconds later anyway.

Requests230/s460/s at peak

20 million trips a day ÷ 86,400 s.

So we dispatch is tiny. It's hard because it must be exact, not because it's busy. About 6 writes per trip is 2,800 a second at peak, which one database primary takes.

Live trips560,000140,000 pushes/s

460 a second × 20 minutes, pickup included. Each watching rider gets a position every 4 seconds.

So we push over open sockets; polling would be mostly empty requests.

Position index200 MB

1 million drivers × about 200 bytes. The busiest city has about 5% of drivers: 13,000 pings a second.

So we memory never forces a split. When we split, it's for write rate and to contain failures.

History860 GBa day

Raw pings at the average rate. Trip rows at 2 KB are about 15 TB a year.

So we stream pings to object storage, and keep 90 days of trips in the trips database.

Try the calculator: double the drivers, or ping every 2 seconds, and pings pass what one index handles, so the split by city comes on day one. Ten times the trips passes one database primary.

Change an assumption
Location pings, peak
250,000/s
Past the 5,000 writes/s one database primary takes: positions live in memory, not in a table.
Busiest city’s pings, peak
13,000/s
One city fits one geo index shard with room to spare.
Ping ingest bandwidth, peak
25 MB/s
Not a design driver. A TLS handshake per ping would be: keep one socket open per phone.
Trip requests, average
230/s
Trip requests, peak
460/s
Low. Dispatch is hard because it must be exact, not because it is busy.
Trip transactions, peak (6 per trip)
2,800/s
One relational primary takes this, with conditional updates for every claim.
Trips in progress, peak
560,000
Drivers busy at peak (percent)
56
About half the fleet carries riders at peak, a plausible rush hour.
Live position pushes to riders, peak
140,000/s
One message per ping per watching rider: push it over open sockets, never by polling.
Geo index memory, all drivers
200 MB
Tiny. Memory never forces sharding here; write rate and failure domains do.
Raw location history a day
860 GB
Append-only and rarely read: stream it to object storage, never into the trips database.
Trip records a year (2 KB each)
15 TB
Too much for one primary within months: keep 90 days hot, archive the rest, partition by city.

We start with the simplest design that is correct. Then we let the numbers break it, one at a time. Each step fixes only what broke, adds only the boxes that fix needs, and says what the fix costs. Throughout, follow one scenario: a driver is online, and a rider taps Request.

The simplest version updates each driver’s row every 4 seconds. A request selects available drivers near the pickup, picks the nearest, and marks them busy.

Ride hailing · architecture

Stage 1: One server, one table

DriversRidersApp serverpositions, matching5000% busyPostgresdrivers, trips5000% busy
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional

Stage 1 of 7

One server, one table

Why this stage

Nothing yet.

This is the smallest design that puts a rider in a car.

What we add

An app server that writes each driver’s latest position to a Postgres row, finds the nearest free driver with a bounding box query, and marks that driver busy.

What it costs

250,000 row updates a second at peak, a check-then-set race on every match, and riders who poll for news.

At 250k location pings per second, Trip service is 5000% busy. Stage 2 is the fix.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.

It works for one small city, with three flaws. At peak it’s 250,000 row updates a second, 50 times what one database takes. Two requests can read the same “available” driver before either marks them busy. And riders poll for news.

A position has three properties that decide where it lives. Only the newest value matters. It’s rewritten every 4 seconds. And if one is lost, the next ping restores it. This is soft state: data that’s fine to lose, because it rebuilds itself.

So a location service keeps an in-memory index per city: a map from driver to position, and a set of drivers per map cell. The cells are H3 cells, Uber’s grid of hexagons. At resolution 9 each hexagon’s edge is about 174 m, and all six neighbors are the same distance away, so “two rings around the pickup” is a fair circle. When a driver crosses into a new cell, they move between two sets, a constant-time update.

GeoIndex.javageo_index.pyGeoIndex.csgeo_index.gogeo-index.ts
// The location path on a city's geo index shard: about 13,000 pings a second in the busiest city.
// Positions are soft state. Nothing here is written to disk, because the next ping, 4 seconds
// later, replaces whatever a crash loses.
record Ping(long driverId, long seq, long atMillis, double lat, double lng, double accuracyM) {}
interface Cells { long cellOf(double lat, double lng); } // H3 resolution 9 in production
interface TripFeed { void publish(long tripId, Ping p); } // the trip's pub/sub channel
interface Trail { void append(Ping p); } // batched to Kafka, never awaited
class GeoIndex {
static final double MAX_SPEED_MPS = 70; // 250 km/h: faster is a GPS jump or a spoofer
static final double MAX_ACCURACY_M = 100; // a worse fix would move the car onto the wrong street
static final long LIVE_MS = 15_000; // no ping for 15 s: not offered trips, kept for a while
static final double M_PER_DEG = 111_320;
static final class Entry {
long seq, atMillis, seenAt;
double lat, lng;
long cell;
boolean available; // a hint from dispatch; the database row is the truth
Long tripId; // set while on a trip, so pings reach the rider
}
private final Map<Long, Entry> drivers = new HashMap<>();
private final Map<Long, Set<Long>> byCell = new HashMap<>();
private final Cells cells;
private final TripFeed feed;
private final Trail trail;
private final LongSupplier clock;
GeoIndex(Cells cells, TripFeed feed, Trail trail, LongSupplier clock) {
this.cells = cells; this.feed = feed; this.trail = trail; this.clock = clock;
}
void onPing(Ping p) {
Long tripId;
// One lock per city shard is enough at 13,000 updates a second: each update is a few
// map operations. Publishing happens outside it, so a slow channel never stalls pings.
synchronized (this) {
Entry e = drivers.get(p.driverId());
// Going online is an explicit, durable action. A ping alone never makes a driver
// dispatchable, so a stale app cannot put a driver back on the map.
if (e == null) return;
// Phones batch and resend pings after a dead zone, so order is not guaranteed.
// The sequence number, not arrival time, decides what is newest.
if (p.seq() <= e.seq) return;
e.seq = p.seq();
e.seenAt = clock.getAsLong(); // server clock: liveness never trusts the phone
if (p.accuracyM() > MAX_ACCURACY_M || impliedSpeed(e, p) > MAX_SPEED_MPS) return; // alive, not moved
long cell = cells.cellOf(p.lat(), p.lng());
if (cell != e.cell) {
// Moving between cell sets is what keeps "drivers near here" a lookup of a few
// cells instead of a scan of the whole city.
Set<Long> old = byCell.get(e.cell);
if (old != null) old.remove(p.driverId());
byCell.computeIfAbsent(cell, k -> new HashSet<>()).add(p.driverId());
e.cell = cell;
}
e.lat = p.lat(); e.lng = p.lng(); e.atMillis = p.atMillis();
tripId = e.tripId;
}
if (tripId != null) feed.publish(tripId, p); // the rider's map moves
trail.append(p); // receipts, disputes, surge supply
}
// What dispatch reads: available, recently seen drivers in the pickup cell and its rings.
synchronized List<Long> nearby(Collection<Long> cellIds) {
long now = clock.getAsLong();
List<Long> out = new ArrayList<>();
for (long cell : cellIds)
for (long id : byCell.getOrDefault(cell, Set.of())) {
Entry e = drivers.get(id);
if (e.available && now - e.seenAt <= LIVE_MS) out.add(id);
}
return out; // a candidate list, not a promise: the claim in the database decides
}
// Pings are 4 s apart, so a flat-earth distance is accurate to well under 1%.
private static double impliedSpeed(Entry e, Ping p) {
if (e.atMillis == 0 || p.atMillis() <= e.atMillis) return 0;
double dy = (p.lat() - e.lat) * M_PER_DEG;
double dx = (p.lng() - e.lng) * M_PER_DEG * Math.cos(Math.toRadians(p.lat()));
return Math.hypot(dx, dy) / ((p.atMillis() - e.atMillis) / 1000.0);
}
}

Three rules in that handler matter more than the data structure:

  • Order comes from the phone’s sequence number. A phone leaving a tunnel resends a batch of old pings, and they must not overwrite newer ones.
  • “Last seen” uses the server’s clock. A phone with a wrong clock can’t make itself look fresh.
  • A ping never makes a driver available. Going online is an explicit, durable call. So an old app or a stale socket can’t put a driver back on the map.

If the index crashes, it reloads the city’s online drivers from the database, and the next round of pings fills in positions 4 seconds later.

Ride hailing · architecture

Stage 2: Locations in memory

DriversRidersLoad balancernewLocation servicevalidate pingsnew21 servers, 60% busyTrip servicetrip state machine4 servers, 57% busyGeo indexdrivers per cellnew63% busyTrips DBtrips only37% busy
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 2 of 7

Locations in memory

Why this stage

At the 250,000 pings/s peak, Postgres would be 50 times past its write capacity, for data that is stale 4 seconds later anyway.

What we add

Stateless location servers that validate pings, and an in-memory geo index of drivers by H3 cell.

Postgres keeps only trips.

What it costs

Positions are soft state.

A lost index forgets every driver until their next ping, 4 seconds later.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.

Positions are now cheap. But two requests can still read the same “available” driver before either marks them busy.

Now the part that must be exact. A dispatcher per city reads about 10 nearby free drivers from the index. It asks a routing service for their road ETAs, because a driver 300 m away across a river can be 9 minutes out. Then it offers the trip to the best one, for 15 seconds.

The offer is a compare-and-set in the database: a write that only succeeds if the current value is what you expect. One transaction makes two conditional updates:

  • the driver goes from available to offered, and
  • the trip goes from requested to offering,

both stamped with a new offer ID and an expiry 15 seconds out, on the database’s clock. If either update changes 0 rows, someone else got there first, and the transaction rolls back. No read comes first. Reading “available” and then writing is exactly the race that broke the one-table version.

DispatchClaims.java_rollback.pyDispatchClaims.csdispatch_claims.godispatch-claims.ts
// Dispatch claims: every change to who holds a driver is one conditional UPDATE in the city's
// trips database. The WHERE clause is the compare, the SET is the swap, and the row lock makes
// the pair atomic. now() is the database clock, so servers whose clocks drift never disagree
// about when an offer ended.
interface Tx {
int execute(String sql, Object... args); // returns the number of rows changed
boolean exists(String sql, Object... args);
}
interface Db {
<T> T inTransaction(Function<Tx, T> work); // commits on return, rolls back on throw
}
enum Offer { OFFERED, DRIVER_BUSY, TRIP_GONE }
enum Accept { ACCEPTED, EXPIRED }
class Rollback extends RuntimeException {
final Object result;
Rollback(Object result) { super(null, null, false, false); this.result = result; }
}
class DispatchClaims {
// Free, or held by an offer whose 15 seconds ran out: a dead dispatcher's offer frees
// itself, with no sweeper on the correctness path.
static final String CLAIM_DRIVER = """
UPDATE drivers SET state = 'offered', trip_id = ?, offer_id = ?,
offer_expires_at = now() + interval '15 seconds'
WHERE driver_id = ?
AND (state = 'available' OR (state = 'offered' AND offer_expires_at < now()))""";
// A trip has at most one live offer. This is what stops two dispatchers (say, an old
// leader that has not noticed its lease is gone) from offering one trip to two drivers.
static final String CLAIM_TRIP = """
UPDATE trips SET state = 'offering', driver_id = ?, offer_id = ?,
offer_expires_at = now() + interval '15 seconds'
WHERE trip_id = ?
AND (state = 'requested' OR (state = 'offering' AND offer_expires_at < now()))""";
// Accept only the exact offer, and only before it expires.
static final String ACCEPT_DRIVER = """
UPDATE drivers SET state = 'on_trip', offer_expires_at = NULL
WHERE driver_id = ? AND state = 'offered' AND offer_id = ? AND offer_expires_at >= now()""";
static final String ACCEPT_TRIP = """
UPDATE trips SET state = 'accepted', accepted_at = now()
WHERE trip_id = ? AND state = 'offering' AND offer_id = ?""";
static final String ALREADY_ACCEPTED = """
SELECT 1 FROM trips WHERE trip_id = ? AND offer_id = ? AND state <> 'offering'
AND state <> 'cancelled'""";
// Timeout and decline: release only if this offer still holds the row. 0 rows means the
// accept won the race, and releasing nothing is the correct outcome.
static final String RELEASE_DRIVER = """
UPDATE drivers SET state = 'available', trip_id = NULL, offer_id = NULL,
offer_expires_at = NULL
WHERE driver_id = ? AND state = 'offered' AND offer_id = ?""";
static final String RELEASE_TRIP = """
UPDATE trips SET state = 'requested', driver_id = NULL, offer_id = NULL
WHERE trip_id = ? AND state = 'offering' AND offer_id = ?""";
private final Db db;
DispatchClaims(Db db) { this.db = db; }
Offer offer(long tripId, long driverId, UUID offerId) {
try {
return db.inTransaction(tx -> {
// No SELECT first: reading "available" and then writing is the race.
if (tx.execute(CLAIM_DRIVER, tripId, offerId, driverId) == 0)
return Offer.DRIVER_BUSY; // the index hint was stale: next driver
if (tx.execute(CLAIM_TRIP, driverId, offerId, tripId) == 0)
throw new Rollback(Offer.TRIP_GONE); // cancelled or offered elsewhere: undo the driver claim
return Offer.OFFERED; // only now send the offer to the phone
});
} catch (Rollback r) {
return (Offer) r.result;
}
}
Accept accept(long tripId, long driverId, UUID offerId) {
try {
return db.inTransaction(tx -> {
if (tx.execute(ACCEPT_DRIVER, driverId, offerId) == 0)
// A retried accept whose first response was lost must not see "expired".
return tx.exists(ALREADY_ACCEPTED, tripId, offerId) ? Accept.ACCEPTED : Accept.EXPIRED;
if (tx.execute(ACCEPT_TRIP, tripId, offerId) == 0)
throw new Rollback(Accept.EXPIRED); // the rider cancelled meanwhile
return Accept.ACCEPTED;
});
} catch (Rollback r) {
return (Accept) r.result;
}
}
// Called when the 15 second timer fires or the driver declines. Safe to call twice.
void release(long tripId, long driverId, UUID offerId) {
db.inTransaction(tx -> {
tx.execute(RELEASE_DRIVER, driverId, offerId);
tx.execute(RELEASE_TRIP, tripId, offerId);
return null;
});
}
}

The driver row is the truth about who is free. The available flag in the memory index is only a hint, copied from the database. So the dispatcher sometimes ranks a driver who was claimed a moment ago. That claim fails, and it tries the next one.

Ride hailing · architecture

Stage 3: Dispatch with atomic claims

DriversRidersLoad balancerLocation servicevalidate pings21 servers, 60% busyDispatchone offer at a timenew3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyGeo indexdrivers per cell63% busyETA + pricingroad graph, surgenew6 servers, 54% busyTrips DBtrips, driver claims55% busy
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 3 of 7

Dispatch with atomic claims

Why this stage

A requirement, not load: two riders must never get the same driver, and a driver who ignores an offer must not strand the rider.

What we add

A dispatch service that ranks nearby drivers by road ETA and offers the trip to one driver at a time for 15 seconds, claiming each with a conditional update in the trips database.

What it costs

A database round trip per offer, and up to 15 seconds lost per driver who does not answer.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.
Go deeperWhy the claim lives in the database, not in Redis1 min

Redis is fast, and a Redis script can also check and set a driver’s state in one atomic step. So why pay for a database transaction on every offer?

Because a claim is hard state: if it is lost, a driver can get two trips. A Redis primary confirms a write before its replica has it. If the primary dies just then, the replica takes over without that write, and the driver looks free again. A database that copies each commit to a second zone before confirming it doesn’t have that gap.

The numbers make this affordable. Positions arrive 250,000 times a second, so they stay in memory. Claims happen about 700 times a second at peak, 1.5 offers for each of about 460 requests. One database primary handles that with room to spare. Each kind of data goes where its rate and its cost of loss say it should.

An offer must reach the driver within a second, and 560,000 riders watch a car move. Polling every second would be 1.5 million mostly empty requests a second. So phones keep a socket open to an edge gateway, and services publish to channels: one per driver for offers, one per trip for the rider. The gateway holding a phone’s socket subscribes to that phone’s channels, so no service needs to know which gateway that is.

Messages are fire and forget, because the truth is always in the database or the next ping. When the app is in the background, offers go by mobile push instead. When the socket is quiet, the app polls its trip every few seconds.

Ride hailing · architecture

Stage 4: Push to phones

DriversRidersPub/subtrip channelsnewEdge gatewaysphone sockets26 servers, 60% busyLocation servicevalidate pings21 servers, 60% busyDispatchone offer at a time3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyGeo indexdrivers per cell63% busyETA + pricingroad graph, surge6 servers, 54% busyTrips DBtrips, driver claims55% busy
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 4 of 7

Push to phones

Why this stage

A requirement: an offer must reach the driver within a second, and the rider watches the car approach.

Polling every second from 1.5 million phones is 1.5 million mostly empty requests a second.

What we add

Edge gateways that hold one socket per phone, and a pub/sub layer with a channel per trip and per driver.

What it costs

Millions of long-lived connections to drain on every deploy, and a fallback to mobile push when a socket is down.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.

Follow a request, then an accept, through the design:

Ride hailing · architecture

Trace a request: Request a ride

DriversRidersPub/subtrip channelsEdge gatewaysphone socketsLocation servicevalidate pingsDispatchone offer at a timeTrip servicetrip state machineGeo indexdrivers per cellETA + pricingroad graph, surgeTrips DBtrips, driver claims
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasionalHop in motionHops doneResponse

Request a ride: step 1 of 10

Riders to Edge gateways

The request arrives on the rider’s open socket with an idempotency key and the quote id. The gateway checks the session token and forwards by city.

2 ms of a 150 ms budget
  1. Riders → Edge gateways2 ms
  2. Edge gateways → Trip service1 ms
  3. Trip service → Trips DB5 ms
  4. Trip service → Dispatch1 ms
  5. Dispatch → Geo index2 ms
  6. Dispatch → ETA + pricing30 ms
  7. Dispatch → Trips DB5 ms
  8. Dispatch → Pub/sub2 ms
  9. Pub/sub → Edge gateways2 ms
  10. Response
Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

This is enough for a mid-level answer. Positions in memory, a conditional claim in the database, one offer at a time, and sockets. Step 5 and the stress tests are what make it a senior answer.

Go deeperOther ways to reach phones1 min
Option Delay Cost
Polling every few seconds Up to the interval Mostly empty requests; drains the battery
WebSocket or gRPC stream Tens of ms Gateways hold sockets; deploys must drain them
Mobile push (APNs, FCM) Seconds, no guarantee Works in the background; rate limited

One socket per phone carries both pings and offers, so there’s no TLS handshake or HTTP headers per ping. The cost is gateways holding millions of long-lived connections. A lost “driver assigned” message leaves a rider on a spinner until the app’s poll catches up, which is why the app polls while its socket is quiet.

Go deeperThe API and the two tables2 min
POST /v1/quotes { "pickup": {...}, "dropoff": {...}, "product": "standard" }
200 OK { "quoteId": "q_8f2c...", "fare": {"amount": 2350, "currency": "USD"},
"surge": 1.4, "pickupEtaSeconds": 240, "expiresAt": "..." }
POST /v1/trips Idempotency-Key: 3b9e7d10-... { "quoteId": "q_8f2c..." }
201 Created { "tripId": 90471123, "state": "requested" }
DRIVER SOCKET up {"type":"ping","seq":5512,"lat":37.7801,"lng":-122.4120,"heading":84}
down {"type":"offer","offerId":"o_51ac...","tripId":90471123,"expiresAt":"..."}
POST /v1/offers/{offerId}/accept 200 {tripId} | 409 {"reason":"expired"}
POST /v1/trips/{tripId}/events {"type":"arrived" | "started" | "completed"}

A quote locks the price for 2 minutes, so surge can’t change between what the rider saw and what they requested. The idempotency key makes a retry after a lost response return the same trip. The accept names the offer, not just the trip, because only the current offer may be accepted.

CREATE TABLE drivers ( -- the truth about who is free
driver_id BIGINT PRIMARY KEY,
city_id INT NOT NULL,
state TEXT NOT NULL, -- offline | available | offered | on_trip
trip_id BIGINT,
offer_id UUID, -- which offer holds the driver right now
offer_expires_at TIMESTAMPTZ
);
CREATE TABLE trips (
trip_id BIGINT PRIMARY KEY,
city_id INT NOT NULL,
rider_id BIGINT NOT NULL,
driver_id BIGINT,
state TEXT NOT NULL, -- requested | offering | accepted | arrived | in_progress | ...
offer_id UUID,
quote_id UUID NOT NULL, -- the locked price
idem_key UUID NOT NULL
);
CREATE UNIQUE INDEX one_active_trip ON trips (rider_id)
WHERE state IN ('requested', 'offering', 'accepted', 'arrived', 'in_progress');

The partial unique index allows one active trip per rider, so a double tap on Request can’t create two trips, even on two app servers.

Go deeperOther places positions and claims could live1 min
Positions in Wins Loses
Postgres with PostGIS Exact queries, one store 50 times past write capacity; index churn on every move
Redis GEOADD Built in, fast, a good start One set per area gets hot; replication can lose writes
In-memory index per city Millions of updates a second We run the service, and a restart starts empty
Claim made by Verdict
The dispatcher’s memory Correct until there are two processes, which failover guarantees one day
A Redis lock per driver A paused holder, or a failover, can give two dispatchers the same driver
A ZooKeeper or etcd lock Safe with fencing, but a consensus write per offer, and it still needs a fence
A conditional update in the trips database The store refuses the second claim, and the claim is durable

Redis is the wrong store for claims even without locks: it copies writes to replicas after answering, so a failover can lose a claim it already confirmed. DynamoDB with conditions, or Spanner, would also work.

Why not a quadtree, since drivers crowd downtown? Every move in a quadtree is a delete and an insert into a tree that may split or merge. That’s a poor fit for 250,000 moves a second. Fixed cells make a move two set operations. Density is handled at search time: downtown, one ring already holds enough drivers, and the search stops early.

The last thing to break is size: ten times the drivers, in ten times the cities.

At ten times the drivers, pings are six times what one index takes, and trip writes five times one database primary. Every dispatch question is local: drivers near this pickup, in this city. So the position index, the dispatcher and the trips database are all split by city, and a request never leaves its city’s shard. A city map says which shard owns each city. Every gateway caches it, and routes a ping by the driver’s city, and a request by its pickup.

Ride hailing · architecture

Stage 5: Shard by city

DriversRidersPub/subtrip channelsEdge gatewaysphone sockets26 servers, 60% busyLocation servicevalidate pings21 servers, 60% busyDispatchone owner a city3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyGeo indexsharded by city3% busyETA + pricingroad graph, surge6 servers, 54% busyTrips DBsharded by city3% busyCity mapcity → shardnew
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 5 of 7

Shard by city

Why this stage

At 10 times the load, 2.5 million pings/s is 6 times what one geo index takes, and trip transactions are 5 times what one primary takes.

What we add

The geo index, the dispatcher and the trips database partitioned by city, and a city map that routes every ping and request to its city’s shard.

What it costs

Trips that start near a city line need a rule, and the biggest city is still one shard.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.
Split by Nearby search Claims Problem
City One shard One shard, one transaction The biggest city is one shard; trips near a city line
H3 cell, hashed 7 to 19 cells, many shards A driver moves between shards Every search asks many shards
Driver ID Every shard One shard Every search asks every shard

The biggest city, at ten times today, is 125,000 pings a second: a third of one shard. Near a city line, like an airport just outside the limits, the pickup decides the owner. Its dispatcher also reads the neighbor’s index for the outer ring of cells. Shards are uneven (São Paulo isn’t Boise), so they’re sized per group of cities, not evenly.

The design is complete. Now we test it where ride hailing is hard: every race that could give a driver two trips, how to choose the driver, and surge pricing with payments. The last one adds a few boxes, because surge and payments need events that leave the dispatch path.

Interviewerminute 20

List every way a driver could end up with two trips, or a trip with two drivers. Show me each one is closed.

What they are testing

Whether you can list the races one by one, including retries and cancels, and show each is closed.

The hardest race is an accept that arrives just as the 15 seconds run out. Both the accept and the timeout are conditional updates on the same offer ID. Whichever commits first wins, and the other changes 0 rows. There’s one outcome, judged by one clock, the database’s.

An accept races the 15 second timeout
DispatcherTrips DBTrip serviceDriver appDispatcherTrips DBTrip serviceDriver apptaps Accept at 18:00:14.9timer fires at 18:00:15.0offer o7, driver available to offered, expires 18:00:15offer o7accept o7release o7 if driver still offered with o7accept o7 if offered with o7 and not expiredrelease applied, driver available0 rows, offer o7 is gone409 offer expiredDispatcherTrips DBTrip serviceDriver appDispatcherTrips DBTrip serviceDriver apptaps Accept at 18:00:14.9timer fires at 18:00:15.0offer o7, driver available to offered, expires 18:00:15offer o7accept o7release o7 if driver still offered with o7accept o7 if offered with o7 and not expiredrelease applied, driver available0 rows, offer o7 is gone409 offer expired
Scroll or pinch to zoom, drag to pan
  • Color key
  • Service
  • Database

Every other race closes the same way:

Race What closes it
Two requests pick the same driver The driver update needs available: one changes 1 row, the other 0 and moves on
Two dispatchers offer one trip The trip allows one live offer; the second transaction rolls back
Accept retried after a lost response The retry finds the trip accepted with its own offer ID, and returns 200
Rider cancels while an offer is out Cancel releases the driver held by that offer; a racing accept needs offering
Rider double taps Request The idempotency key, and the unique index on active trips
Dispatcher dies holding offers Its offers expire after 15 seconds, and anyone can claim those drivers again

The promise, stated exactly. At every committed point, a driver references at most one trip, and a trip at most one driver. This holds whatever leases, clocks, retries and network splits do, because every write that could break it is conditional, in one store, with one clock. We don’t promise an offer never fails on accept, or that matching is never a few seconds late.

The trip itself moves through a small state machine, and every arrow is a compare-and-set:

The trip state machine

rider requests, quote locked

driver claimed

declined or 15 s passed

driver accepts in time

driver at pickup

rider in the car

dropoff, fare fixed

rider cancels

rider cancels

either side cancels

no-show, fee applies

requested

offering

accepted

arrived

in_progress

completed

cancelled

rider requests, quote locked

driver claimed

declined or 15 s passed

driver accepts in time

driver at pickup

rider in the car

dropoff, fare fixed

rider cancels

rider cancels

either side cancels

no-show, fee applies

requested

offering

accepted

arrived

in_progress

completed

cancelled

Scroll or pinch to zoom, drag to pan
  • Color key
  • State
  • Ends well
Go deeperHow exact each answer is1 min
Operation Guarantee What users can see
Driver claim, accept, release Exact per row: conditional updates on one primary Nothing odd; a late accept gets a clear “expired”
Trip state changes Exact per trip, only along the state machine A cancel and an accept never both succeed
Positions in the index Newest wins by sequence number, at most 15 s old A car on the map 4 seconds behind reality
available in the index A hint that catches up with the database Sometimes a claim fails; riders never see it
Surge multiplier About a minute behind Price lags a sudden crowd
Fare charged Exactly once, through the outbox and idempotency key The charge appears minutes after the ride

“The system is eventually consistent”, said of the whole design, is wrong. It invites two riders sharing a car for a few seconds. Positions and surge may lag, with stated limits. Claims and trip states are exact. Charges happen once through idempotency, not through a transaction across systems.

Why not just pick the nearest driver? And why offer to one driver instead of five?

We already rank by road ETA. The open question is how many drivers to ask, and when.

Strategy How it works Wins Loses
Broadcast to several Offer to the 5 best, first accept wins Fast acceptance 4 drivers lose a trip they tapped; they stop trusting offers
Greedy, one at a time Best ETA first, 15 s, then the next Fair to drivers, simple to get right A slow decline costs the rider 15 s
Batched matching Collect 1 to 2 s of requests, solve them together Lower total wait across the city Adds a second or two; more complex

We pick one at a time, with batching for dense cities. In a quiet area, requests rarely compete for drivers, and taking the best driver now is close to the best overall. Downtown at rush hour, it can take the only driver near the next rider. There, solving the last second or two of requests together, as a minimum-cost assignment, cuts the average pickup time. Each chosen pair still goes through the same claim, so the correctness argument doesn’t change.

How do prices go up on a Friday night, and how does the rider get charged?

Three consumers want the same events, and none may slow dispatch. So the location service batches pings into Kafka, split by city. The trip service writes each event to an outbox table in the same transaction as the trip change, and a relay ships them to Kafka. An outbox means the event can’t be lost between the database and Kafka, because it’s committed with the change.

Ride hailing · architecture

Stage 6: Events, surge and payments

DriversRidersPub/subtrip channelsEdge gatewaysphone sockets26 servers, 60% busyLocation servicevalidate pings21 servers, 60% busyDispatchone owner a city3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyKafkapings, trip eventsnewSurge jobper cell, per minutenewGeo indexsharded by city3% busyETA + pricingroad graph, surge6 servers, 54% busyTrips DBsharded by city3% busyPaymentsseparate systemnewCity mapcity → shard
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 6 of 7

Events, surge and payments

Why this stage

A requirement: prices follow supply and demand per area, every fare is charged exactly once, and trip trails are kept for receipts and disputes, none of it on the dispatch path.

What we add

Kafka carrying pings and trip events, a stream job that computes a surge multiplier per cell every minute, and an outbox that hands finished trips to the payment system.

What it costs

Surge lags demand by about a minute, and every consumer must tolerate duplicates.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.
  • Trail. Pings go to object storage, by day and city, for receipts, disputes and ETA training.
  • Surge. Every minute, a stream job compares demand with free drivers in each larger H3 cell (about 5 km²). Demand counts quotes too, which catches riders who looked and left. The ratio maps to a capped multiplier, smoothed across neighboring minutes and cells so it doesn’t flicker.
  • Payments. The fare and an outbox row commit together when a trip ends. The relay hands the trip to the payment system, with the trip ID as the idempotency key, so a retry never charges twice.

A payment outage delays charges; it never stops a ride. The cost is that surge lags demand by about a minute, and every consumer must handle duplicates.

Follow one ping into the event stream:

Ride hailing · architecture

Trace a request: A driver’s ping on a trip

DriversRidersPub/subtrip channelsEdge gatewaysphone socketsLocation servicevalidate pingsDispatchone owner a cityTrip servicetrip state machineKafkapings, trip eventsSurge jobper cell, per minuteGeo indexsharded by cityETA + pricingroad graph, surgeTrips DBsharded by cityPaymentsseparate systemCity mapcity → shard
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasionalHop in motionHops doneResponse

A driver’s ping on a trip: step 1 of 7

Drivers to Edge gateways

A 60 byte frame on the open socket: sequence number, time, position, heading, speed and GPS accuracy. No TLS handshake, no HTTP headers.

1 ms of a 20 ms budget
  1. Drivers → Edge gateways1 ms
  2. Edge gateways → Location service1 ms
  3. Location service → Geo index1 ms
  4. Location service → Pub/sub1 ms
  5. Pub/sub → Edge gateways1 ms
  6. Location service → Kafkaasync
  7. Response
Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
Go deeperETA and price, in more detail1 min

ETA. A routing service holds each region’s road map in memory, with travel times updated from recent trails. With a precomputed shortcut graph (contraction hierarchies), one route takes a few milliseconds, and a one-to-ten table about 30 ms. A model corrects the result, because map ETAs are too hopeful about pickups.

Price. A quote is (base + per_km × km + per_min × minutes) × surge, with a minimum fare. It’s stored and locked for 2 minutes. The trip carries its quote ID, so what the rider saw is what they pay, unless the route changes a lot.

Choice We chose Alternative Why
Surge cell size About 5 km² About 0.1 km² Small cells have too few events to be stable
Surge update Every minute, smoothed Every request Prices that change every second feel random and invite gaming
Where ETA comes from One table call per dispatch A precomputed cell-to-cell table Tables go stale in traffic; the table is the fallback

If the surge job stalls, pricing slowly brings stale multipliers back toward 1.

Interviewerminute 38

The machine running New York’s dispatcher dies at 6 pm on a Friday. What happens, and how do you make sure the replacement never double dispatches?

What they are testing

Whether failover is fast, and whether correctness holds even when two dispatchers act at once.

A city’s dispatcher owns its in-flight matching, so its failure stops matching in that city. The fix has two halves: make the takeover fast, and make it safe even when it isn’t clean.

Fast. Each city runs a hot standby dispatcher. Its own position index reads the same pings from Kafka, at most a second behind. Ownership is a lease, a claim that’s valid for 5 seconds and renewed every second. When renewals stop, the standby takes the lease with a higher number, called an epoch. It reloads driver states from the database, and frees any offer past its expiry. Riders who just tapped Request wait 5 to 10 seconds longer. Matched trips never notice.

Safe. Sometimes two dispatchers will both act for a city: a long garbage-collection pause, a network split, a slow machine. The design makes that harmless, not impossible. Every offer is the same conditional update, so the old dispatcher’s writes fail like anyone else’s that lost a race. The epoch only tells it to step down when it next checks. Correctness never depends on it.

A paused dispatcher wakes up after the standby took over
Trips DBDispatcher B, standbyCity mapDispatcher A, epoch 6Trips DBDispatcher B, standbyCity mapDispatcher A, epoch 6GC pause, lease not renewedwakes, still thinks it owns NYCsteps down, drops its statetake lease for NYC, epoch 7read trips requested or offeringoffer trip 88 to driver 42 if both freeapplied, offer o9offer trip 88 to driver 17 if both free0 rows, trip 88 has a live offercheck lease, epoch 7 is not mineTrips DBDispatcher B, standbyCity mapDispatcher A, epoch 6Trips DBDispatcher B, standbyCity mapDispatcher A, epoch 6GC pause, lease not renewedwakes, still thinks it owns NYCsteps down, drops its statetake lease for NYC, epoch 7read trips requested or offeringoffer trip 88 to driver 42 if both freeapplied, offer o9offer trip 88 to driver 17 if both free0 rows, trip 88 has a live offercheck lease, epoch 7 is not mine
Scroll or pinch to zoom, drag to pan
  • Color key
  • Database
  • Service

Offers the dead dispatcher sent need no cleanup. They expire after 15 seconds. A driver who accepts one in time still gets the trip, because accepts go to the trip service and the database.

These add the last boxes to the drawing: a standby dispatcher and a standby database for each city.

Ride hailing · architecture

Stage 7: Fail over a city

DriversRidersPub/subtrip channelsEdge gatewaysphone sockets26 servers, 60% busyLocation servicevalidate pings21 servers, 60% busyDispatchleader + standby3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyKafkapings, trip eventsSurge jobper cell, per minuteGeo indexprimary + standby3% busyETA + pricingroad graph, surge6 servers, 54% busyTrips DBsharded by city3% busyPaymentsseparate systemCity mapleases, epochs
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 7 of 7

Fail over a city

Why this stage

A city shard is a failure domain: if its dispatcher dies, nobody in that city gets a car.

Failover must never let two dispatchers assign the same driver or the same trip.

What we add

A hot standby per city that tails the same ping stream, ownership leases with a rising epoch, and every claim fenced by the database.

What it costs

Twice the geo index memory, and 5 to 10 seconds without new matches in that city during a failover.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.

Click any box to break it and see what riders and drivers would notice:

Ride hailing · architecture

Break it: what fails, who notices, what users see

DriversRidersPub/subtrip channelsEdge gatewaysphone socketsLocation servicevalidate pingsDispatchleader + standbyTrip servicetrip state machineKafkapings, trip eventsSurge jobper cell, per minuteGeo indexprimary + standbyETA + pricingroad graph, surgeTrips DBsharded by cityPaymentsseparate systemCity mapleases, epochs
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Break

Failure: click any box to break it

A city’s dispatcher goes down

What breaks

New requests in that city are not matched; offers in flight are not followed up.

How we notice

Its lease is not renewed within 5 seconds.

What users see

Riders who just requested wait 5 to 10 seconds longer.

Trips already matched are untouched.

What we do

The standby takes the lease with a higher epoch, reloads driver states and the requested and offering trips from the database, and releases expired offers.

Every claim is a conditional update, so a paused old dispatcher that wakes up has its writes refused.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.

Positions are soft state and claims are hard state, so the failure story splits the same way. Losing a position index costs seconds and heals itself. Losing a claim would be a correctness bug, so claims only live in a database that copies each write to another zone before confirming it.

Go deeperEvery failure, and what users see1 min
Part What users see What we do
Edge gateway 100,000 phones reconnect within seconds Spread-out reconnects; pings deduplicated by sequence
Pub/sub Maps freeze; offers time out Mobile push for offers, polling for trips
Position index A second of slower matching Promote the standby; pings refill any gap in 4 s
Dispatcher New requests wait 5 to 10 s longer Standby takes the lease; claims fenced by the database
Trips database Requests and accepts fail for about 30 s Synchronous replica, safe retries
ETA and pricing Slightly worse matches Straight-line time × a speed factor; a cached ETA table
Kafka Nothing at once; surge and receipts lag The outbox holds trip events; surge drifts toward 1
Whole region Its cities show “no cars” for minutes Promote replicas and leases in the standby region

Under stress, shed in this order: the “cars near you” map for riders who haven’t requested, then quotes (serve cached surge), then position pushes to riders (every 8 seconds). Requests, offers and accepts go last.

A whole region failing is rarer. Each city’s database has a synchronous replica in another zone, and an asynchronous one in a standby region. So promoting the standby region can lose the last second of claims. Matching there resumes after driver apps reconnect and report the trip they’re on, which repairs those rows.

Go deeperRunning it: alerts, abuse and deploys1 min

Wake someone up, per city, when: the share of requests matched within 60 seconds drops below normal; request-to-offer time passes 1 second at p99; the share of online drivers seen in the last 15 seconds drops; the offer acceptance rate falls suddenly (offers aren’t reaching phones); or claims that change 0 rows spike (stale hints, or two active dispatchers).

Abuse and safety. Spoofing apps fake positions to jump airport queues or inflate fares. Reject impossible speeds, cross-check cell tower and Wi-Fi hints, and flag trails that jump. Keep exact trails only as long as disputes and law require.

Deploys. Gateways drain: stop taking sockets, then close old ones a few thousand at a time, so phones don’t all reconnect at once. Dispatchers update the standby first, then hand over the lease. Moving a city to another shard is the same handover.

Here is everything we built, in the order we built it. Use the buttons to step through it.

Ride hailing · architecture

The full design, stage by stage

DriversRidersPub/subtrip channelsEdge gatewaysphone sockets26 servers, 60% busyLocation servicevalidate pings21 servers, 60% busyDispatchleader + standby3 servers, 46% busyTrip servicetrip state machine4 servers, 57% busyKafkapings, trip eventsSurge jobper cell, per minuteGeo indexprimary + standby3% busyETA + pricingroad graph, surge6 servers, 54% busyTrips DBsharded by city3% busyPaymentsseparate systemCity mapleases, epochs
Color keyServiceDatabaseCacheQueue or streamBackground workerNew in this stageAsync or occasional
Stage

Stage 7 of 7

Fail over a city

Why this stage

A city shard is a failure domain: if its dispatcher dies, nobody in that city gets a car.

Failover must never let two dispatchers assign the same driver or the same trip.

What we add

A hot standby per city that tails the same ping stream, ownership leases with a rising epoch, and every claim fenced by the database.

What it costs

Twice the geo index memory, and 5 to 10 seconds without new matches in that city during a failover.

At 250k location pings per second, every part has headroom. Raise the traffic until something turns red.

Pinch or tap the zoom buttons to read small labels.Drag any box to move it. Reset puts every box back.The whole design. Step through it: one server and one table, positions in memory, atomic claims, push to phones, a split by city, events for surge and payments, then a standby per city.
The numbers behind the busy bars

What one machine can do

  • One Postgres primary takes about 5,000 small write transactions a second.
  • One in-memory geo index node takes about 400,000 location updates a second when location servers batch them.
  • One location server handles about 20,000 pings a second.
  • One gateway handles about 25,000 messages a second over about 100,000 sockets.

What one ride request costs

  • Each ride request makes about 6 trip transactions and 1.5 offers.
  • It also makes 7 routing calls: 5 price quotes, 1 candidate matrix and 1 route.
  • Once sharded, there are 20 city shards of each.

Why Kafka has no bar

  • 250,000 batched pings a second is about 25 MB/s. That is a few percent of a 6-broker cluster, which takes several hundred MB/s.
  • It grows by adding brokers and partitions per city.
Decision We chose We gave up We’d choose differently if
Positions In-memory index per city, soft state Durable positions Positions were billed or audited in real time
Spatial index H3 cells with a set of drivers per cell Exact nearest-neighbor structures Points barely moved (see proximity service)
Claims Conditional updates in the trips database A few ms per offer One process per city, with no failover, were fine
Offers One driver at a time, batched when dense Fastest possible acceptance Drivers didn’t mind losing trips they tapped
Split By city Even shard sizes One city outgrew a shard (split it into regions)
Failover Hot standby and leases, fenced by the database Memory for a second index per city Seconds without matching were acceptable
Surge, payments Asynchronous, from Kafka and an outbox Real-time prices, instant charges Prices had to react within seconds
  • 0 to 4. Drivers, pings, trips, offers. State “never double booked” as the one hard promise.
  • 4 to 9. 250,000 pings against 460 requests a second, and a 200 MB index.
  • 9 to 20. Build it: positions in memory, the conditional claim, sockets. A mid-level answer can stop here.
  • 20 to 24. The split by city.
  • 24 to 38. The stress tests: the race table first, always. Then matching, then surge and payments.
  • 38 to 45. Failover, then your own questions.

Never skip the claim, its race with the timeout, or why positions are soft state.

Interviewer asksAdd pooled rides. What changes?

Matching becomes route insertion: for each car already carrying a rider, compute the detour of adding the new pickup and dropoff, and keep it within a limit for both riders. A driver can now be on several trips, so the driver row holds seats left and a version number, and the claim updates it only if the version matches. The race table still applies, with “seats left” in place of “available”.

Interviewer asksThe trip completes, but the payment hand-off times out. Could the rider be charged twice?

No. The fare and an outbox row were committed together. The relay retries with the trip ID as the idempotency key, and the payment system returns the original result for a key it has seen. A trip that’s never acknowledged stays in the outbox, and pages someone.

Go deeperMore follow-up questions1 min

Riders who haven’t requested see little cars moving near them. That’s a million riders refreshing. How? Never search the index per rider. Each city publishes, every few seconds, a rough snapshot per cell: a few approximate positions and headings. Gateways cache it and send each rider their cell’s and its neighbors’. It’s decoration, and it hides exact positions.

What changes at 100 times the scale? The biggest cities split into regions of cells, with dispatch reading across region lines. The trips database moves to a store that splits ranges on its own, such as Spanner or CockroachDB, still keyed by city so a claim stays in one range.

A rider says the driver took a long detour. How do you settle it? From the trail in object storage: the driven route against the route the quote assumed. A large detour triggers an automatic refund. This is why on-trip pings are kept.