API design: REST, gRPC, WebSockets

An API is a contract with programs you do not control, and cannot redeploy. Think of a phone app released two years ago, or a partner who integrated once and moved on. Good API design is mostly about the cases where something changes underneath a call. A response is lost. A row is inserted between two pages. A field arrives that a client has never heard of. Networking for system design covers the HTTP, TCP and TLS underneath this page.

The problem it solves

A shopping app on a train. The user taps Pay. The request reaches the server, the card is charged, and the response is lost in a tunnel. The app sees a timeout. Four things go wrong in APIs that were not designed for this world:

1. Retries duplicate work. The app retries, the server charges again, and the user pays twice. At 10,000 payments a minute, if just 0.01% lose their response after the charge went through, that is 60 double charges an hour.
2. Lists drift. A user scrolls a feed with ?page=2. Three posts arrive at the top while they read page 1, so page 2 starts with three posts they already saw. Deletes do the opposite and rows are skipped silently.
3. Changes break old clients. Someone renames amount to total. Every app version that reads amount now shows an empty price, and those versions stay installed for years.
4. The transport does not fit. A million dashboards poll every 5 seconds, for updates that arrive twice an hour. That’s 200,000 requests a second, nearly all answering “nothing new”.

None of these are exotic. Each has a standard answer: idempotency keys, cursors, additive change with versions, and the right push transport. The rest of the page builds them up.

How it works

Step 1: Model resources, then use HTTP’s verbs and status codes as defined

REST models the system as resources (nouns with stable URLs) and uses HTTP’s methods on them. A collection and its members:

POST   /v1/orders                 create an order, 201 + Location: /v1/orders/ord_88123
GET    /v1/orders?status=open     list, paginated
GET    /v1/orders/ord_88123       read one
PATCH  /v1/orders/ord_88123       change some fields
DELETE /v1/orders/ord_88123       remove, 204
POST   /v1/orders/ord_88123/cancel   a state transition that is not a field write

The last line is fine. Not everything is a field update, and forcing a cancel into PATCH {"status": "cancelled"} hides the rules that cancelling runs (refunds, stock release). Google’s API guidelines call these custom methods and write them POST /v1/orders/ord_88123:cancel. What to avoid is RPC dressed as REST: POST /createOrder, GET /deleteOrder?id=....

The methods carry promises that clients, proxies and caches rely on (RFC 9110):

Method	Safe (no side effects)	Idempotent (repeating = doing once)	Typical success
GET	Yes	Yes	200, or 304 if unchanged
HEAD	Yes	Yes	200
PUT	No	Yes: replaces the whole resource	200 or 201
DELETE	No	Yes: gone is gone	204, or 404 the next time
POST	No	No	201 with Location, 202
PATCH	No	Not guaranteed (+1 is not)	200

Safe means a crawler, a prefetching browser or a retrying proxy may call it freely, so a GET must never change state. Idempotent means the effect of N identical calls equals one call. The response can differ (a second DELETE returns 404), the state cannot.

Status codes are for the client’s code, not for humans. The ones that matter:

Code	Means	Client should
200 / 201 / 204	Done. 201 created (with Location)	Use the result
202 Accepted	Taken, not finished: work continues async	Poll a status URL or wait for an event
304 Not Modified	Your cached copy (by ETag) is still good	Use the cache
400 Bad Request	Malformed or invalid input	Fix the request, do not retry
401 Unauthorized	Not authenticated: missing or expired token	Refresh the token, then retry
403 Forbidden	Authenticated, but not allowed	Stop
404 / 410	Does not exist / existed and is gone for good	Stop
409 Conflict	Clashes with current state (seat taken)	Re-read, decide again
412 Precondition Failed	If-Match ETag no longer matches	Re-read, reapply the change
422 Unprocessable Content	Well formed, but breaks a business rule	Show the error, do not retry
429 Too Many Requests	Over a rate limit	Wait Retry-After, then retry
500 / 502 / 503 / 504	Server or upstream failed, or overloaded	Retry with backoff, if safe to

The split between 4xx (the client must change something) and 5xx (try again later) is what retry logic keys on. An API that returns 200 {"error": "not found"} breaks every generic client, monitor and retry policy at once.

Go deeperETags: cheap revalidation and lost-update protection1 min

ETag with If-None-Match lets a client revalidate a cached resource, and get a body-less 304. ETag with If-Match on a write gives optimistic concurrency. Two editors read version "v7". The first PUT ... If-Match: "v7" wins. The second gets 412, instead of silently overwriting the first edit. It is the HTTP form of a compare-and-set on a version column.

Step 2: Make every write safe to retry

Every remote call ends in success, failure or timeout. A timeout tells the client nothing. The request may never have arrived. Or the work may be done, and only the reply lost. The distributed transactions page makes the same point between services. GET, PUT and DELETE can be retried as they are. A POST cannot, so it gets an idempotency key. That’s a unique value the client generates once per operation, and sends with every attempt.

A lost response, retried with the same key

Scroll or pinch to zoom, drag to pan

• Color key
• Database
• Service

The server side is a small table and four rules:

• Claim the key before doing the work, with a unique index on (caller, key). Two retries that race both try to insert. One wins. The other sees the key in progress, and gets 409 Conflict.
• Write the key’s result in the same transaction as the effect. If the charge row and the key row commit together, “charged but no key saved” cannot happen. Say the work calls another system, such as a card network. Then pass a key derived from yours downstream, because your transaction cannot cover theirs.
• Fingerprint the request. Store a hash of method, path and body with the key. The same key with a different body is a client bug. Replaying the old answer would hide it, so return 422.
• Expire keys. Retries happen within seconds to hours. Stripe keeps keys for at least 24 hours and accepts keys up to 255 characters. After the TTL the key is new again.

Scope keys by caller (API key or account). Then two customers who both send 1 never see each other’s responses. The status codes follow the IETF Idempotency-Key header draft. It’s still an Internet-Draft, not an RFC, but it’s what most APIs converge on:

• 400 when a required key is missing.
• 409 while the first attempt is in flight.
• 422 for the same key with a different payload.

The middleware, with a store that keeps those rules visible:

JIdempotencyStore.javaPyidempotency_store.pyC#IdempotencyStore.csGoidempotency_store.goTSidempotency-store.ts

JavaPythonC#GoTypeScript

JavaPythonC#GoTypeScript

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.HashMap;
import java.util.HexFormat;
import java.util.Map;
import java.util.function.Supplier;

record StoredResponse(int status, String body) {}

// What the store says about a key: run the handler, replay the saved answer, or refuse.
sealed interface Claim {
    record Execute() implements Claim {}
    record Replay(StoredResponse response) implements Claim {}
    record InFlight() implements Claim {} // the first attempt is still running: 409
    record Mismatch() implements Claim {} // same key, different request: 422
}

// In production this is a table with a unique (scope, key) index, written in the SAME
// transaction as the effect, so "charged but key not saved" cannot happen. A map keeps the
// rules visible.
final class IdempotencyStore {
    private static final class Entry {
        final String fingerprint;
        StoredResponse response; // null while the first attempt runs
        long lockedUntil;
        final long expiresAt;

        Entry(String fingerprint, long lockedUntil, long expiresAt) {
            this.fingerprint = fingerprint;
            this.lockedUntil = lockedUntil;
            this.expiresAt = expiresAt;
        }
    }

    private final long ttlMs; // how long a retry is recognised: 24 h is common
    private final long lockMs; // how long a crashed attempt blocks its retries
    private final Map<String, Entry> entries = new HashMap<>();

    IdempotencyStore(long ttlMs, long lockMs) {
        this.ttlMs = ttlMs;
        this.lockMs = lockMs;
    }

    Claim claim(String scope, String key, String fingerprint, long now) {
        // Scope by caller: two customers who both send key "1" must not see each other's answers.
        String id = scope + ":" + key;
        Entry e = entries.get(id);
        if (e == null || e.expiresAt <= now) {
            // First sight (or expired): take the key before doing any work. In SQL this is
            // INSERT ... ON CONFLICT DO NOTHING, so two concurrent first attempts cannot both win.
            entries.put(id, new Entry(fingerprint, now + lockMs, now + ttlMs));
            return new Claim.Execute();
        }
        // Checked before anything else: a different body under the same key is a client bug,
        // and replaying the old answer would hide it.
        if (!e.fingerprint.equals(fingerprint)) return new Claim.Mismatch();
        if (e.response != null) return new Claim.Replay(e.response);
        if (e.lockedUntil > now) return new Claim.InFlight();
        // The first attempt died without finishing. Its transaction rolled back, so it is safe
        // to let this retry run.
        e.lockedUntil = now + lockMs;
        return new Claim.Execute();
    }

    void complete(String scope, String key, StoredResponse response) {
        entries.get(scope + ":" + key).response = response;
    }

    void release(String scope, String key) {
        // The handler threw before committing anything: forget the key so a retry starts fresh.
        entries.remove(scope + ":" + key);
    }
}

final class IdempotencyMiddleware {
    // Every unsafe request with an Idempotency-Key passes through here.
    static StoredResponse handle(IdempotencyStore store, String scope, String key, String method,
                                 String path, String body, long now, Supplier<StoredResponse> handler) {
        if (key == null || key.isEmpty()) return new StoredResponse(400, "Idempotency-Key header required");
        // Fingerprint the whole request, so the same key on another endpoint is a mismatch too.
        String fp = sha256(method + " " + path + "\n" + body);
        Claim c = store.claim(scope, key, fp, now);
        if (c instanceof Claim.Replay r) return r.response();
        if (c instanceof Claim.InFlight) return new StoredResponse(409, "request with this key in progress");
        if (c instanceof Claim.Mismatch) return new StoredResponse(422, "key reused with a different request");
        try {
            StoredResponse res = handler.get();
            // Save every answer the handler gave, declines included: a retry must get the same one.
            store.complete(scope, key, res);
            return res;
        } catch (RuntimeException ex) {
            store.release(scope, key);
            throw ex;
        }
    }

    private static String sha256(String s) {
        try {
            byte[] d = MessageDigest.getInstance("SHA-256").digest(s.getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(d);
        } catch (java.security.NoSuchAlgorithmException e) {
            throw new IllegalStateException(e); // every JVM ships SHA-256
        }
    }
}

class Main {
    static final long HOUR = 3_600_000L;
    static final IdempotencyStore store = new IdempotencyStore(24 * HOUR, 30_000);
    static int charges = 0;

    static StoredResponse charge() {
        charges++;
        return new StoredResponse(201, "{\"id\":\"ch_" + charges + "\"}");
    }

    static StoredResponse pay(String key, String body, long now, Supplier<StoredResponse> h) {
        return IdempotencyMiddleware.handle(store, "acct_42", key, "POST", "/v1/charges", body, now, h);
    }

    public static void main(String[] args) {
        System.out.println(pay("k1", "{\"amount\":6499}", 0, Main::charge)); // 201 ch_1: runs once
        System.out.println(pay("k1", "{\"amount\":6499}", 5_000, Main::charge)); // 201 ch_1 again: replayed
        System.out.println(pay("k1", "{\"amount\":9999}", 6_000, Main::charge).status()); // 422
        // A retry that arrives while the first attempt is still running gets 409, not a second charge.
        pay("k2", "{\"amount\":100}", 10_000, () -> {
            System.out.println(pay("k2", "{\"amount\":100}", 10_500, Main::charge).status()); // 409
            return charge();
        });
        System.out.println(pay("k1", "{\"amount\":6499}", 25 * HOUR, Main::charge)); // 201 ch_3: key expired
        System.out.println(charges); // 3
    }
}

The client makes one key per user intent, when the checkout screen opens. It reuses it on every retry, and every tap of Pay. A key made per HTTP attempt protects nothing. A key made from the order id blocks a legitimate second attempt with a different card, as the payment system shows.

The key makes a retry safe. It does not make retrying cheap. Retry only safe or idempotent calls, or POSTs with a key. Honour Retry-After on 429 and 503. And use the backoff, jitter and retry budget from networking.

Go deeperTwo other ways to make a create idempotent1 min

Client-chosen ids with PUT. The client generates the resource id (a UUID) and calls PUT /v1/orders/{uuid}. A retry writes the same id, and a unique constraint makes the second insert a no-op. It needs no key table, and suits creates whose response is just the resource.

Natural business keys. A unique constraint on (merchant_id, merchant_order_ref) turns a duplicate into a constraint violation the API maps to “already exists, here it is”. It is the cheapest correct option when such a key exists, and the distributed transactions page lists the same techniques for messages between services.

What none of these give you is a replay of the original response, for a call that returns something computed (a charge decline code, a quote). That is what the stored response in an idempotency key table is for.

Step 3: Page through lists that change while you read

Any list that can grow needs pagination. Use a default page size (20 to 50) and a hard maximum (100 to 1,000), so no request returns a million rows.

Offset pagination is the obvious version: ?limit=20&offset=40, which becomes LIMIT 20 OFFSET 40 in SQL. It has two problems.

• It drifts under writes. The offset counts positions, and writes move rows between positions. A row inserted above the reader’s position pushes everything down, so the next page repeats a row. A row deleted above it pulls everything up, so the next page skips one, and nobody notices.
• Deep pages are slow. OFFSET 100000 makes the database find and throw away 100,000 rows before returning 20. The cost grows linearly with the page number, which is how a scraper on page 5,000 becomes a database incident.

Cursor (keyset) pagination remembers where the last page ended, in the sort order, and asks for rows after that: WHERE (created_at, id) < ($last_created_at, $last_id) ORDER BY created_at DESC, id DESC LIMIT 20. With an index on (created_at, id), the database seeks straight to the position, so page 5,000 costs the same as page 1. Inserts and deletes above the position cannot move it. Try both against the same live table:

Next pageNew post arrivesDelete a row already readReset

Auto

No writes1 new post a pageBusy: adds and deletes

Read to the end

The posts table, newest firstoffset got itcursor got itnext page

Offset readerLIMIT 5 OFFSET n

Pages

0

Duplicates

0

Skipped

0

Rows the DB touched

0

Cursor readerWHERE id < cursor LIMIT 5

Pages

0

Duplicates

0

Skipped

0

Rows the DB touched

0

30 posts, 5 a page. Press Next page a couple of times, then add a post or delete one, and page again.

Things to try:

• Next page twice, New post arrives, Next page. The offset reader’s third page repeats a row (red). The cursor reader’s does not.
• Next page twice, Delete a row already read, Next page. The offset reader jumps past a row and lists it under “Never returned”. Nothing in its responses says so.
• Busy, then Read to the end. The offset reader ends with 6 duplicates and 4 gaps in 30 rows. Compare “Rows the DB touched”. The offset reader rescans every earlier row on every page, so the total grows with the square of the page count. The cursor reader touches each row once.

Neither reader shows posts that arrive after it started. That is correct: new items belong to a fresh read from the top, which is what a feed’s “new posts” button does.

Three details make a cursor production-ready:

• A unique tie-breaker. Many rows share a created_at. Sorting by created_at alone loses or repeats rows that tie at a page boundary, so the cursor holds (created_at, id).
• Opaque tokens. Encode the position (base64 JSON is enough). Clients pass it back unchanged and never build one, so the server can change what is inside. Bind it to the query, so a token from ?author=7 replayed on ?author=9 is a 400, not wrong data. Sign it if it holds anything the client must not see.
• Fetch one extra row. Ask for limit + 1. If the extra row exists, return a nextCursor. No COUNT(*), which is itself a slow query on a large table.

JCursors.javaPyinvalid_cursor.pyC#Cursors.csGocursor_data.goTSinvalid-cursor.ts

JavaPythonC#GoTypeScript

JavaPythonC#GoTypeScript

import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Comparator;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

record Post(long createdAt, String id) {}

// What a cursor remembers: where the last page ended, in the sort order, plus which query it
// belongs to. Nothing else, so the server stays stateless. v is a version, so the format can
// change without breaking tokens already handed out; id breaks ties between equal timestamps.
record CursorData(int v, long createdAt, String id, String q) {}

record PageResult(List<Post> items, String next) {}

class InvalidCursor extends RuntimeException {
    InvalidCursor(String message) { super(message); }
}

final class Cursors {
    // Ids and queries in this example are URL-safe ASCII; a real service would use a JSON library.
    private static final Pattern SHAPE =
        Pattern.compile("\\{\"v\":(\\d+),\"createdAt\":(-?\\d+),\"id\":\"([^\"]+)\",\"q\":\"([^\"]*)\"}");

    // Opaque to clients: base64url JSON. Clients must not build or edit it, which leaves the
    // server free to change what is inside. Sign it if it ever holds something the client may
    // not see.
    static String encode(CursorData c) {
        String json = "{\"v\":" + c.v() + ",\"createdAt\":" + c.createdAt()
            + ",\"id\":\"" + c.id() + "\",\"q\":\"" + c.q() + "\"}";
        return Base64.getUrlEncoder().withoutPadding().encodeToString(json.getBytes(StandardCharsets.UTF_8));
    }

    static CursorData decode(String token, String query) {
        String json;
        try {
            json = new String(Base64.getUrlDecoder().decode(token), StandardCharsets.UTF_8);
        } catch (IllegalArgumentException e) {
            throw new InvalidCursor("malformed cursor"); // 400, never a 500
        }
        Matcher m = SHAPE.matcher(json);
        if (!m.matches() || !m.group(1).equals("1")) throw new InvalidCursor("unknown cursor version");
        // A cursor from "?author=7" replayed on "?author=9" would silently return the wrong rows.
        if (!m.group(4).equals(query)) throw new InvalidCursor("cursor belongs to a different query");
        return new CursorData(1, Long.parseLong(m.group(2)), m.group(3), m.group(4));
    }

    // Keyset pagination: "rows after this position", newest first. In SQL:
    //   WHERE (created_at, id) < (?, ?) ORDER BY created_at DESC, id DESC LIMIT ? + 1
    // With an index on (created_at, id) the database seeks straight to the position, so page
    // 10,000 costs the same as page 1, and inserts above the position cannot shift it.
    static PageResult page(List<Post> rows, String token, String query, int limit) {
        CursorData after = token == null ? null : decode(token, query);
        // Fetch one extra row: if it exists there is a next page, with no COUNT(*) query.
        List<Post> hits = new ArrayList<>();
        for (Post p : rows) {
            boolean older = after == null || p.createdAt() < after.createdAt()
                || (p.createdAt() == after.createdAt() && p.id().compareTo(after.id()) < 0);
            if (older && hits.add(p) && hits.size() == limit + 1) break;
        }
        if (hits.size() <= limit) return new PageResult(hits, null);
        List<Post> items = hits.subList(0, limit);
        Post last = items.get(limit - 1);
        return new PageResult(items, encode(new CursorData(1, last.createdAt(), last.id(), query)));
    }
}

class Main {
    static final Comparator<Post> NEWEST_FIRST =
        Comparator.comparingLong(Post::createdAt).thenComparing(Post::id).reversed();

    static List<String> ids(PageResult r) {
        return r.items().stream().map(Post::id).toList();
    }

    public static void main(String[] args) {
        // Two posts share createdAt 1000.
        List<Post> feed = new ArrayList<>(List.of(
            new Post(1000, "p5"), new Post(1000, "p4"), new Post(900, "p3"),
            new Post(800, "p2"), new Post(700, "p1")));
        feed.sort(NEWEST_FIRST);
        String q = "author=7&sort=-createdAt";

        PageResult p1 = Cursors.page(feed, null, q, 2);
        System.out.println(ids(p1) + " " + p1.next()); // [p5, p4] and a token
        // A new post lands at the top between page requests. With OFFSET 2, p4 would come back again.
        feed.add(new Post(1100, "p6"));
        feed.sort(NEWEST_FIRST);
        PageResult p2 = Cursors.page(feed, p1.next(), q, 2);
        System.out.println(ids(p2)); // [p3, p2]: no duplicate, no gap
        PageResult p3 = Cursors.page(feed, p2.next(), q, 2);
        System.out.println(ids(p3) + " " + p3.next()); // [p1] null: the last page
        try {
            Cursors.page(feed, p1.next(), "author=9&sort=-createdAt", 2);
        } catch (InvalidCursor e) {
            System.out.println(e.getMessage()); // cursor belongs to a different query
        }
    }
}

The cost of cursors is what you give up: no “jump to page 37”, and no exact total without a separate count. Offsets remain right for small, slow-changing lists that people navigate by page number, such as an admin table of 2,000 rows.

Step 4: Errors a client can act on

An error body has two readers. The client’s code needs a stable, machine-readable reason. The developer debugging it needs a sentence and a request id. Problem Details (RFC 9457, which replaced RFC 7807) is the standard shape, served as application/problem+json:

HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json

{
  "type": "https://api.example.com/problems/insufficient-funds",
  "title": "Insufficient funds",
  "status": 422,
  "detail": "Card ending 4242 was declined: insufficient funds.",
  "instance": "/v1/charges/req_7Hq2",
  "code": "card_declined",
  "declineCode": "insufficient_funds",
  "requestId": "req_7Hq2"
}

type (a URI) and the code extension are the contract. Client code branches on them, never on detail, whose wording can change. Validation errors list every bad field at once. Never return stack traces or SQL. A 429 or 503 carries Retry-After. The rate limit headers are on the rate limiting page.

Step 5: Change the API without breaking anyone

Most changes can be additive: they add without changing what exists. Clients follow the tolerant reader rule and ignore fields they do not know.

Change	Breaking?
Add an optional request field or response field	No
Add an endpoint, or a new optional query param	No
Add a new value to an enum in a response	Often: strict clients crash on the unknown
Make an optional request field required	Yes
Remove or rename a field, change its type	Yes
Change a default, a unit or the meaning	Yes, and the worst kind: nothing errors
Tighten validation (shorter max length)	Yes, for clients that sent the longer value

When a change must break, it goes into a new version:

Style	Example	Wins when
Major version in path	/v1/orders → /v2/orders	Public APIs: visible in logs, easy to route, easy to reason about
Header or date version	Stripe-Version: 2024-06-20	Many small changes: each account is pinned to the version it started on
Media type	Accept: application/vnd.example.v2+json	Rarely worth the friction
No versions, additive only	Internal protobuf services	You control the clients, and fields are never reused

Stripe pins each account to the API version it first used, and transforms responses back to that shape. GitHub’s REST API takes X-GitHub-Api-Version: 2022-11-28.

Retiring a version is a process, not a date:

1. Announce it with the Deprecation (RFC 9745) and Sunset (RFC 8594) response headers.
2. Measure traffic per version, per API key.
3. Contact the callers who are left.
4. Run short brownouts, where the old version fails for an hour, before the final cut.

Then the last stragglers find out from an error, while someone is watching.

Go deeperCompatibility rules for protobuf and gRPC1 min

Field numbers, not names, are the wire contract. Add new fields with new numbers. Never reuse a deleted field’s number (mark it reserved). Never change a field’s type.

An old reader skips fields it does not know. A missing field reads as its default (0, empty). So “absent” and “zero” look the same, unless the field is declared optional. Enums need a zero value that means “unspecified”, because that is what an old writer’s messages decode to. Renaming a field is safe on the wire. It breaks only JSON mappings and generated code. These rules are why internal gRPC services can often live without version numbers at all.

Step 6: Pick the protocol

Three request-response styles cover nearly every design.

REST with JSON over HTTP is the default for anything public. Every language, browser, proxy, CDN and debugging tool speaks it, and GETs are cacheable by URL. The costs are verbose payloads and no machine-checked schema unless you add OpenAPI.

gRPC sends Protocol Buffers over HTTP/2, with typed clients and servers generated from a .proto file. It wins between services:

• Smaller, faster messages. Binary protobuf is typically several times smaller than the same JSON and much cheaper to parse.
• One connection, many calls. HTTP/2 multiplexes concurrent calls as streams on one TCP connection.
• Streaming. Unary, server streaming, client streaming and bidirectional calls.
• Deadlines. Each call carries a deadline (the grpc-timeout header). A server can pass its incoming context to its own outgoing calls. Then it propagates what remains: if the caller has 300 ms left, every hop below knows it. When the deadline passes, the call is cancelled down the chain, instead of finishing work nobody is waiting for.

Its costs: browsers cannot speak it directly (gRPC-Web needs a proxy such as Envoy). Payloads are unreadable without the schema. And L4 load balancers balance its long-lived connections, not its calls, as the load balancing page explains.

GraphQL exposes one endpoint and a typed schema. Each client asks for exactly the fields it needs, across related objects, in one round trip. It wins when many clients (web, iOS, Android, partners) need different shapes of the same data. Its costs are real:

• N+1 queries. Resolving posts { author { name } } naively runs one author lookup per post. Batching loaders (the DataLoader pattern) collect the ids in a tick and fetch them in one query.
• Caching. Queries usually travel as POST bodies to one URL, so HTTP and CDN caching do not apply. Persisted queries (a hash sent as a GET) bring some of it back.
• Cost control. One query can ask for a million nodes. So servers limit depth, and compute a cost per query. GitHub’s GraphQL API gives each user 5,000 points an hour, instead of a request count.

A common shape: REST or GraphQL at the edge, gRPC inside

Scroll or pinch to zoom, drag to pan

• Color key
• Service

Step 7: Push data to the client

Request-response cannot deliver news the client has not asked for. Five ways to get updates to a client, from simplest to most capable:

• Polling. Ask every N seconds. Trivial and stateless, but latency is N/2 on average and most responses are empty. A million clients every 5 seconds is 200,000 requests a second.
• Long polling. The server holds the request open until something happens, or a timeout passes (20 to 60 s, under proxy and load balancer idle timeouts). Then the client asks again. It gives near-instant delivery over plain HTTP. Carry a cursor in each poll, so an event that lands between two polls is not lost.
• Server-Sent Events (SSE). One long HTTP response that streams data: lines, one event at a time. It’s one direction only (server to client), and text only. The browser’s EventSource reconnects on its own, and sends Last-Event-ID so the server can resume. Over HTTP/1.1, a browser allows only 6 connections per host, which SSE tabs use up. Over HTTP/2, they share one connection.
• WebSockets. An HTTP request upgrades into a full-duplex, message-framed TCP connection, with 2 to 14 bytes of framing per message. Both sides send whenever they like. It’s the right tool when the client sends often too (chat, collaborative editing, games). The cost is stateful servers, plus heartbeats, reconnects and resume logic you build yourself.
• WebTransport. Runs over HTTP/3 (QUIC). It has several independent streams on one connection, so a lost packet stalls only its own stream. It also has unreliable datagrams, for data where late is useless (game state, live cursors). Every major browser has shipped it since Safari 26.4 (March 2026). But servers, proxies and load balancers that handle it are still few. And networks that block UDP need a fallback. Mention it, rarely pick it.

Between servers, push is a webhook. The provider POSTs each event to a URL the customer registered. It signs it (an HMAC over the body and a timestamp), and retries with backoff for hours or days. The receiver dedupes by event id, because delivery is at least once.

Long polling, SSE and a WebSocket delivering the same two events

Scroll or pinch to zoom, drag to pan

• Color key
• Service

A held connection is cheap in memory (about 10 to 20 KB, tuned), and expensive in operations. Think heartbeats, deploys that must drain every connection, and reconnect storms when a box dies. Networking covers those costs. The API-level rule is that push is only a hint. Phones sleep and sockets drop. So the client always resyncs from its last cursor after reconnecting. The chat and file sync problems size these fleets.

Step 8: Authentication at a glance

Mechanism	Identifies	How	Wins when
API key	An application	A long random secret in a header; the server stores only its hash	Server-to-server calls from customers
OAuth 2.0 bearer token	A user, via an app	Short-lived access token (minutes to an hour) in Authorization: Bearer	Apps acting for users, third-party access
Session cookie	A browser user	HttpOnly, Secure, SameSite cookie to your own web app	Your own first-party website
mTLS	A service	Both sides present certificates; a mesh issues and rotates them	Service-to-service inside your network

Credentials go in headers, never in URLs, which end up in logs and browser history.

Go deeperJWT or opaque tokens, and identity inside the network1 min

Access tokens come in two kinds. JWTs can be verified by any service locally, from a signature. But they cannot be revoked before they expire. Opaque tokens need a lookup (cached), but can be revoked at once. Short JWT lifetimes (5 to 15 minutes) plus refresh tokens are the usual compromise.

Inside the network, mTLS proves which service is calling. The end user’s identity travels alongside it, as a token. So each service can check both: the order service may call payments, and this user may pay for this order.

Options and tradeoffs

The protocol

Option	Payload	Strengths	Weaknesses	Wins when
REST + JSON	Text, self-describing	Universal, cacheable GETs, easy to debug with curl	Verbose, over- or under-fetching, schema is optional	Public APIs, browsers, partners, CRUD
gRPC + protobuf	Binary, schema first	Small and fast, typed clients, streaming, deadlines	Not native in browsers, needs L7 balancing, opaque on the wire	Internal service calls, high call rates
GraphQL	JSON, client-shaped	One round trip per screen, typed schema, flexible for many apps	N+1, hard to cache, query cost control, one endpoint to secure	Many clients with different data needs
Raw TCP / custom	Binary, fixed fields	Lowest latency and overhead	Everything (framing, retries, tooling) is yours	Trading, games, a database’s own protocol

TradeoffREST at the edge, gRPC between services

We gain

Outside callers get the most compatible, cacheable, debuggable interface. Inside, typed contracts, binary payloads and propagated deadlines keep calls cheap and failures bounded.

We give up

Two toolchains and a translation at the gateway. Every internal schema change still has to be reflected in the public API, by hand.

Getting updates to the client

Option	Latency	Direction	Server cost per client	Wins when
Polling	Half the interval	Client asks	One request per interval, mostly empty	Updates are rare and minutes of delay are fine
Long polling	Under a second	Client asks	One held request, re-armed per event	Plain HTTP must work everywhere; sync with a cursor
SSE	Under a second	Server to client	One held response	Server pushes, client rarely sends: feeds, prices, AI tokens
WebSocket	Under a second	Both ways	One held connection plus heartbeats	Both sides talk often: chat, editors, multiplayer
WebTransport	Under a second	Both, many streams	One QUIC connection	Head-of-line blocking or unreliable delivery matters

TradeoffSSE for one-way push, WebSocket only when the client talks too

We gain

SSE is plain HTTP: it passes proxies, uses normal auth, and reconnects and resumes by itself. The client’s few writes go as ordinary POSTs with idempotency keys.

We give up

Text only and one direction. When the client sends many small messages a second, a request per message costs far more than a WebSocket frame.

Pagination

Option	Stable under writes	Deep pages	Jump to page N	Wins when
Offset / page number	No: repeats, skips	Linear cost	Yes	Small, slow-changing lists with page links
Cursor (keyset)	Yes	Constant cost	No	Feeds, timelines, any large or busy list
Snapshot (server-side)	Yes, frozen view	Constant, but stateful	Yes	Exports and reports that must be exact

TradeoffOpaque keyset cursors for every list that can grow

We gain

No duplicates or gaps while data changes. The same cost for page 1 and page 10,000. And the freedom to change what the token holds, without breaking anyone.

We give up

No page numbers and no cheap total. Every sort order needs its own index and a unique tie-breaker, so arbitrary user-chosen sorts get expensive.

Where idempotency keys live

Option	Atomic with the effect	Wins when
Table in the same database	Yes: one transaction	The effect is a write to your own database
Redis with SET NX and a TTL	No: a crash between effect and save leaks a duplicate	High-rate, low-stakes dedupe (likes, events)
Unique constraint on a business key	Yes, but no stored response to replay	A natural key exists and the response is the resource

TradeoffKeys in the same transaction as the effect

We gain

“Did the work but forgot the key” is impossible, and a retry gets the exact first response, decline codes included.

We give up

A row per write on the hot path and a cleanup job, and effects in other systems still need their own keys passed downstream.

How it fails

The key exists but did not help. Say the key table lives in Redis, while the charge commits in Postgres. The process dies between the two, and the retry charges again. Or the key expired after 1 hour, while a mobile client retried from an offline queue after 3. Match the TTL to the longest retry path, and keep the key with the effect.

Stale cursors. A cursor points at data that was purged, resharded or re-sorted. Return a specific error that tells the client to start over: 410, or a reset code as the file sync API does. Never return a 500, or an empty page that looks like the end.

Failure	Blast radius	Detect	Fix
Unkeyed or rekeyed retry	Duplicate charges, posts, orders	Identical writes seconds apart	One key per intent, same transaction
Key store separate or TTL short	Rare duplicates, hard to explain	Duplicates despite keys	Key with the effect, TTL past longest retry
Retry storm	The whole service	Requests up, successes down	Jitter, budgets, Retry-After, shedding
Offset drift, deep pages	Wrong lists, slow queries	Duplicate reports, deep offsets in logs	Keyset cursors, cap offsets
Stale cursor	One client’s sync	Cursor decode or lookup failures	Explicit reset error
Breaking change	Every old client	Errors by client version after a deploy	Additive changes, versions, contract tests
Reconnect storm	A realtime fleet	Connection rate and handshake CPU spikes	Jittered reconnect, resume, slow drains

At scale

Numbers to remember.

What	Typical figure
Idle WebSocket or SSE connection	About 10 to 20 KB tuned; 100,000s to millions per box by memory
Polling 1 million clients every 5 s	200,000 requests a second
Long poll hold time	20 to 60 s, under a 60 s load balancer idle timeout
OFFSET 100000 LIMIT 20	Reads and discards 100,000 rows; keyset reads 20
Idempotency key retention	24 hours is common (Stripe: at least 24 h)
Key table size	1,000 writes/s × 86,400 s × about 1 KB = about 86 GB a day of retention
gRPC call on a warm connection, same zone	About 0.5 to 1 ms

At 10x, the key table and connection counts are what grow. Partition the key table by day and drop whole partitions instead of deleting rows. Move realtime connections to a dedicated gateway fleet sized by connections, not requests, with a registry that maps users to gateways.

At 100x, serve GETs from CDN caches where the data allows. Give each partner its own quotas. And expect versioning to become an operations problem. Dozens of versions are alive at once, each with transformations, tests and usage tracking per API key.

What to monitor:

• Rate and latency (p50, p99) per endpoint and status code, with 4xx apart from 5xx. A jump in 4xx after a deploy is usually a contract break.
• Idempotent replays, 409 in-flight conflicts, and 422 key mismatches. (A rising mismatch rate is a client bug reusing keys.) And retries as a share of requests.
• Traffic per API version, and per key on deprecated versions.
• Deepest offsets requested, cursor decode failures, and reconnects per realtime gateway.

In the interview

What the API section holds

The API section is short and fixes the contract the rest of the design serves:

• The 3 to 5 endpoints the core use cases need, with method, path, key fields and the response. Not every CRUD endpoint.
• Idempotency on every non-idempotent write: an Idempotency-Key header, or a client-chosen id.
• Pagination on every list: limit and an opaque cursor, with nextCursor in the response.
• The status codes that carry meaning for this product: 409 seat taken, 202 processing, 429 over the limit.
• The push channel, if anything is real-time, and why that one.
• Auth in one line: bearer token for users, API keys for partners, mTLS inside.

POST /v1/holds    Idempotency-Key: 2b7f...   { "eventId": 81231, "seatIds": ["A12-7-14"] }
  201 { "holdId": "h_f3a9", "expiresAt": "..." }   409 { "error": "SEATS_TAKEN" }
GET  /v1/events?city=london&limit=20&cursor=...   200 { "events": [...], "nextCursor": "..." }
WS   /v1/events/81231/seats                       server pushes seat map changes

Interviewer asksThe app sent POST /payments and timed out. What should it do, and what does the server need?

Retry with the same Idempotency-Key, made when the checkout screen opened. On the server, keep a table with a unique index on caller and key. Claim the key before the work. Store the response in the same transaction as the payment, and keep it at least 24 hours. A retry after success replays the stored response. One while the first is running gets 409. The same key with a different body gets 422. The card network call gets its own key, derived from ours, because our transaction does not cover their system. Retries back off with jitter, and stop after a few.

Interviewer asksHow do you paginate the feed?

With an opaque cursor, not an offset. The feed changes constantly. With offsets, every new post shifts the next page, and the user sees duplicates. And every delete skips one. The cursor encodes the last item’s sort key, plus its id as a tie-breaker. The query is “rows before this position” on an index, so every page costs the same. I fetch limit + 1 to know whether there is a next page. New posts are not part of this pass. The client shows a “new posts” button that reloads from the top. For a ranked feed, the cursor holds a ranking session id and a position. Then the order does not reshuffle while the user scrolls.

Interviewer asksREST or gRPC? And how does the client get live updates?

Both, in different places. The public API is REST with JSON. Every client speaks it, and GETs cache at the CDN. Between services, gRPC: typed contracts, small binary messages on multiplexed HTTP/2, and deadlines that propagate. For live updates, it depends on direction. Use SSE if only the server pushes, since it reconnects and resumes by itself. Use a WebSocket if the client also sends often, as in chat. Either way, the push is a hint, and the client resyncs from a cursor after reconnecting.

Interviewer asksYou need to rename a field in a public API. How?

Not by renaming it. Add the new field alongside the old one and fill both, so old clients see nothing change. Document the old one as deprecated, send Deprecation and Sunset headers, and measure who still reads it, per API key. When usage is near zero, remove it in the next major version, with brownouts before the final date. If the API uses date-pinned versions like Stripe’s, the rename ships as a new version and old accounts keep the old shape.

Common mistake

Making a new idempotency key on every retry

A key generated inside the HTTP client’s retry loop is different on every attempt. So the server sees each retry as a new request, and the duplicate charge happens anyway. The key belongs to the user’s intent. It’s made once, when the operation starts (the checkout screen opens). And it’s reused by every attempt, including a retry after an app restart.

Mid-level covers

Writes clean resource URLs, with the right methods and status codes. Adds an idempotency key on creates, and a cursor on lists. Picks WebSockets or polling for real-time, with a reason.

Senior adds

Explains where the key is made and stored, and what each conflict returns. Explains why offsets drift, and what the cursor must contain. Chooses REST, gRPC or GraphQL by who the callers are. Picks the lightest push transport that fits the direction of traffic. Plans how the API changes over years, without breaking clients.

Seen in

Every problem on this site has an API design section. These use this page’s ideas most clearly:

• Payment system: a key per checkout attempt, 202 for an unknown outcome, 422 for a reused key, and integer minor units for money.
• News feed: an opaque cursor that encodes a position for the chronological feed and a ranking session for the ranked one.
• Ticketmaster: idempotent holds, 409 SEATS_TAKEN as an expected answer, and cursor-paginated search.
• Hotel reservation: a quote that fixes the price, then an idempotent reservation that can answer 409 SOLD_OUT.
• URL shortener: separate APIs for creating and redirecting, keys kept 24 hours, and 410 Gone for expired links.
• Chat: WebSocket chosen over long polling, SSE and MQTT, with client message ids as idempotency keys.
• Collaborative editor: one WebSocket per open document, (clientId, seq) as the key for an edit, and a lossy channel for cursors.
• File sync: long polling with a cursor, content hashes that make uploads idempotent, and a reset error for stale cursors.
• Job scheduler: long-poll leases for workers and 409 when a lease is lost.
• Unique ID generator: batched gRPC calls, and ids sent as strings in JSON.
• Stock exchange: a binary protocol over TCP, with the client order id as the idempotency key and sequence numbers for replay.
• Typeahead: a cacheable GET chosen over a WebSocket, with empty results as 200 so they cache too.

Recap

Resources and verbs. Nouns in URLs. GET is safe, PUT and DELETE are idempotent, and POST is neither. Custom methods for state transitions. 4xx means “change the request”. 5xx means “try later”.

Retries. A timeout means “unknown”. Use a key per user intent. Claim it before the work, store it with the effect in one transaction, and fingerprint it (422 on mismatch, 409 in flight). Keep it 24 hours or more.

Pagination. Offsets repeat and skip rows under writes and slow down with depth. Keyset cursors with a unique tie-breaker, opaque and bound to the query, limit + 1 instead of a count.

Errors and change. Problem Details, with a stable code and a request id. Additive changes and tolerant readers. Versions for breaking changes. Deprecation, Sunset, usage tracking and brownouts to retire them.

Protocols and push. REST at the edge, gRPC inside, and GraphQL when many clients need different shapes. Polling for rare updates. Long polling for plain HTTP. SSE one-way, WebSockets two-way. Resync from a cursor after every reconnect.

Auth. API keys for applications, OAuth bearer tokens for users, mTLS between services.

Further reading

• On this site: networking for system design, for HTTP/2, TLS and connection costs. Rate limiting, for 429 and its headers. Distributed transactions, for idempotency across services. Message queues, for asynchronous APIs and at-least-once delivery. And load balancing, for gRPC and long-lived connections.
• Designing robust and predictable APIs with idempotency, Stripe: why keys exist, and how clients and servers use them.
• AIP-158: Pagination, Google’s API design rules for page tokens and page sizes, part of a full set of guidelines worth browsing.
• RFC 9110: HTTP Semantics, the definitions of methods, safety, idempotency and status codes.