Low Level DesignWorked problems
Design a notification service
Design a notification service step by step for a low level design interview. Grow from a hard-coded email sender to email, SMS and push behind one channel interface, templates written once per channel with checked placeholders, a delivery queue with retries and exponential backoff, and a shared service where duplicate requests send once and two workers never send the same message, with code in Java, Python, C#, Go and TypeScript.
- Read first
- StrategyThread safety basics
- Patterns
- StrategyAdapter and facade
“Design a notification service: other parts of the app ask it to tell a user something, and it sends email, SMS or push.” Then: “the SMS provider is down for ten minutes. What happens to the messages?”
Every product has a notification service. The order service says “order 42 shipped”. The security service says “new sign-in from Lisbon”. Something has to turn that into an email, a text message (SMS) and a phone notification (push), each worded for its channel, and sent to the right address.
That part is a clean exercise in interfaces. What makes it a real design question is failure. Providers (the outside companies that actually deliver email, SMS and push) time out, limit how fast you can send, and go down. Callers retry. And a user who gets the same text message three times notices.
The answer in one minute
Section titled “The answer in one minute”If you only have a minute, this is the design. The rest of the page explains how we get here.
- Every channel is behind one
Channelinterface. Each implementation wraps one provider, so adding WhatsApp is a new class, not an edit. - A template is written once per channel, and every placeholder must be filled, or nothing is sent on any channel.
- Asking for a notification only queues it. Workers send later. A temporary failure is retried with growing waits, up to a limit. A permanent failure isn’t retried.
- A caller’s idempotency key makes a repeated request send once. It’s a name the caller gives each notification, so a retry can be recognised.
- Workers claim a delivery under a lock, and send outside it, so no message is sent twice by two workers, and a slow provider stalls nobody.
We’ll build this in four stages, then share the service between many callers and workers.
Scope it
Section titled “Scope it”Before drawing anything, we ask a few questions. Each answer changes a class. Anything that doesn’t, we assume and move on.
Design a notification service.
What they are testing
Whether you find the scope fast: which channels, who calls it, and what “delivered” means. The red flag is designing an email server, or spending the time on the user’s settings page.
Here are the questions that matter:
- Which channels? Email, SMS and push, with more later.
- Who writes the text? Templates that the service owns.
- Must the caller wait for the send? No: queue it and return.
- What if a provider fails? Retry temporary failures, then give up.
- Can a caller ask twice by mistake? Yes, so repeats must send only once.
These answers give us one promise that matters more than the others: a message is never sent twice, and never sent with a blank placeholder. It must hold when callers retry and workers run in parallel. Two smaller promises come with it. A slow or failing provider never slows the caller. And adding a channel doesn’t change the service.
We leave out user preferences at first, quiet hours, and tracking opens and clicks.
Go deeperThree more questions, and what each changes1 min
| Question | We assume | If the answer were different |
|---|---|---|
| Can users opt out? | Out of scope at first | A preference check per user and category before queuing |
| Quiet hours? | No | Deliveries get a not-before time in the user’s time zone |
| Do we track opens and clicks? | No | Tracking links, and a webhook from the provider per message |
Build it
Section titled “Build it”We’ll build the design in four stages. Each stage adds only the classes that a new requirement needs. In each diagram, the new parts are outlined, and the colors show what each class does (the key is under every diagram). Time is passed in as a number of milliseconds, so the tests control it.
Stage 1: Send an email
Section titled “Stage 1: Send an email”Start simple: when an order ships, email the customer.
What they are testing
Whether you can get something working quickly, and then name what breaks first: a second channel, consistent wording, or a provider failing.
Start with the smallest thing that works. The NotificationService keeps each user’s email
address, and calls an EmailSender.
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
- New in this step
class EmailSender { final List<String> sent = new ArrayList<>();
void send(String to, String subject, String body) { sent.add(to + ": " + subject); }}
class NotificationService { private final Map<String, String> emails = new HashMap<>(); private final EmailSender email;
NotificationService(EmailSender email) { this.email = email; }
void register(String userId, String address) { emails.put(userId, address); }
// notifyUser has three flaws. // flaw 1: email is the only channel, and adding SMS means editing this class. // flaw 2: every caller writes its own text, so the same alert reads differently everywhere. // flaw 3: if the email server is down, the notification is simply lost. void notifyUser(String userId, String subject, String body) { email.send(emails.get(userId), subject, body); }}class EmailSender: def __init__(self) -> None: self.sent: list[str] = []
def send(self, to: str, subject: str, body: str) -> None: self.sent.append(f"{to}: {subject}")
class NotificationService: def __init__(self, email: EmailSender): self._emails: dict[str, str] = {} self._email = email
def register(self, user_id: str, address: str) -> None: self._emails[user_id] = address
# notify_user has three flaws. # flaw 1: email is the only channel, and adding SMS means editing this class. # flaw 2: every caller writes its own text, so the same alert reads differently everywhere. # flaw 3: if the email server is down, the notification is simply lost. def notify_user(self, user_id: str, subject: str, body: str) -> None: self._email.send(self._emails[user_id], subject, body)public sealed class EmailSender{ public List<string> Sent { get; } = new();
public void Send(string to, string subject, string body) => Sent.Add($"{to}: {subject}");}
public sealed class NotificationService{ private readonly Dictionary<string, string> _emails = new(); private readonly EmailSender _email;
public NotificationService(EmailSender email) => _email = email;
public void Register(string userId, string address) => _emails[userId] = address;
// NotifyUser has three flaws. // flaw 1: email is the only channel, and adding SMS means editing this class. // flaw 2: every caller writes its own text, so the same alert reads differently everywhere. // flaw 3: if the email server is down, the notification is simply lost. public void NotifyUser(string userId, string subject, string body) => _email.Send(_emails[userId], subject, body);}type EmailSender struct{ Sent []string }
func (e *EmailSender) Send(to, subject, body string) { e.Sent = append(e.Sent, to+": "+subject) }
type NotificationService struct { emails map[string]string email *EmailSender}
func NewNotificationService(email *EmailSender) *NotificationService { return &NotificationService{emails: map[string]string{}, email: email}}
func (s *NotificationService) Register(userID, address string) { s.emails[userID] = address }
// NotifyUser has three flaws.// flaw 1: email is the only channel, and adding SMS means editing this type.// flaw 2: every caller writes its own text, so the same alert reads differently everywhere.// flaw 3: if the email server is down, the notification is simply lost.func (s *NotificationService) NotifyUser(userID, subject, body string) { s.email.Send(s.emails[userID], subject, body)}class EmailSender { readonly sent: string[] = [];
send(to: string, subject: string, body: string): void { this.sent.push(`${to}: ${subject}`); }}
class NotificationService { private readonly emails = new Map<string, string>();
constructor(private readonly email: EmailSender) {}
register(userId: string, address: string): void { this.emails.set(userId, address); }
// notifyUser has three flaws. // flaw 1: email is the only channel, and adding SMS means editing this class. // flaw 2: every caller writes its own text, so the same alert reads differently everywhere. // flaw 3: if the email server is down, the notification is simply lost. notifyUser(userId: string, subject: string, body: string): void { this.email.send(this.emails.get(userId)!, subject, body); }}It works, but it has three problems, and we leave them in on purpose. Email is the only channel, and adding SMS means editing the service. Every caller writes its own subject and body, so the same event reads differently depending on who sent it. And if the email server is down, the call fails, and the notification is lost.
Stage 2: One interface for every channel
Section titled “Stage 2: One interface for every channel”Now add SMS and push. Next quarter, WhatsApp.
What they are testing
Whether you put the channels behind one interface and let the service pick by type, so a new
channel is a new class. A switch on channel type inside the service is the red flag.
Every channel implements Channel: it says its type, and sends a Message. A message
carries its channel, the address, a subject and a body. SMS ignores the subject.
Each implementation turns one provider’s API into that one method. That’s the
Adapter pattern: an SmsChannel turns a message into a call to the SMS
gateway’s API, and nothing else in the service knows that gateway exists. The service sends the same
way, whatever the channel, which is the Strategy pattern.
A User has an address per channel: an email address, a phone number, a device token (the id a
phone gives for push messages). The service sends on each requested channel the user has an address
for, and skips the rest.
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
- New in this step
// The ways we can reach a user. Adding one is a new value here and a new Channel class.enum ChannelType { EMAIL, SMS, PUSH }
// One message on one channel: where it goes and what it says. SMS ignores the subject.record Message(ChannelType type, String to, String subject, String body) {}
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.interface Channel { ChannelType type(); void send(Message m);}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.record User(String id, Map<ChannelType, String> addresses) {}
class NotificationService { private final Map<ChannelType, Channel> channels = new EnumMap<>(ChannelType.class); private final Map<String, User> users = new HashMap<>();
NotificationService(List<Channel> channels) { for (Channel c : channels) this.channels.put(c.type(), c); }
void addUser(User u) { users.put(u.id(), u); }
// Sends on each requested channel the user has an address for, and returns what was sent. List<Message> notifyUser(String userId, String subject, String body, List<ChannelType> via) { User user = users.get(userId); List<Message> sent = new ArrayList<>(); if (user == null) return sent; for (ChannelType type : via) { String to = user.addresses().get(type); Channel channel = channels.get(type); if (to == null || channel == null) continue; Message m = new Message(type, to, subject, body); channel.send(m); sent.add(m); } return sent; }}# The ways we can reach a user. Adding one is a new value here and a new Channel class.class ChannelType(Enum): EMAIL = auto() SMS = auto() PUSH = auto()
# One message on one channel: where it goes and what it says. SMS ignores the subject.@dataclass(frozen=True)class Message: type: ChannelType to: str subject: str body: str
# Every channel looks the same to the service. Each implementation adapts one provider's# API, such as an email service, an SMS gateway or a push service, to this one method.class Channel(Protocol): type: ChannelType
def send(self, m: Message) -> None: ...
# A user's address on each channel they've given us: an email address, a phone number, a# device token. No address on a channel means nothing is sent there.@dataclass(frozen=True)class User: id: str addresses: dict[ChannelType, str]
class NotificationService: def __init__(self, channels: list[Channel]): self._channels = {c.type: c for c in channels} self._users: dict[str, User] = {}
def add_user(self, u: User) -> None: self._users[u.id] = u
# Sends on each requested channel the user has an address for, and returns what was sent. def notify_user(self, user_id: str, subject: str, body: str, via: list[ChannelType]) -> list[Message]: user = self._users.get(user_id) if user is None: return [] sent = [] for type in via: to = user.addresses.get(type) channel = self._channels.get(type) if to is None or channel is None: continue m = Message(type, to, subject, body) channel.send(m) sent.append(m) return sent// The ways we can reach a user. Adding one is a new value here and a new Channel class.public enum ChannelType { Email, Sms, Push }
// One message on one channel: where it goes and what it says. SMS ignores the subject.public sealed record Message(ChannelType Type, string To, string Subject, string Body);
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.public interface IChannel{ ChannelType Type { get; } void Send(Message m);}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.public sealed record User(string Id, IReadOnlyDictionary<ChannelType, string> Addresses);
public sealed class NotificationService{ private readonly Dictionary<ChannelType, IChannel> _channels; private readonly Dictionary<string, User> _users = new();
public NotificationService(IEnumerable<IChannel> channels) => _channels = channels.ToDictionary(c => c.Type);
public void AddUser(User u) => _users[u.Id] = u;
// Sends on each requested channel the user has an address for, and returns what was sent. public List<Message> NotifyUser(string userId, string subject, string body, params ChannelType[] via) { var sent = new List<Message>(); if (!_users.TryGetValue(userId, out var user)) return sent; foreach (var type in via) { if (!user.Addresses.TryGetValue(type, out var to) || !_channels.TryGetValue(type, out var channel)) continue; var m = new Message(type, to, subject, body); channel.Send(m); sent.Add(m); } return sent; }}// ChannelType is one of the ways we can reach a user. Adding one is a new value here and a new// Channel.type ChannelType int
const ( Email ChannelType = iota SMS Push)
// AllChannels is a fixed order to walk channels in, so results are predictable.var AllChannels = []ChannelType{Email, SMS, Push}
// Message is one message on one channel: where it goes and what it says. SMS ignores the subject.type Message struct { Type ChannelType To string Subject, Body string}
// Channel is what every channel looks like to the service. Each implementation adapts one// provider's API, such as an email service, an SMS gateway or a push service, to this one method.type Channel interface { Type() ChannelType Send(m Message)}
// User holds a user's address on each channel they've given us: an email address, a phone number,// a device token. No address on a channel means nothing is sent there.type User struct { ID string Addresses map[ChannelType]string}
type NotificationService struct { channels map[ChannelType]Channel users map[string]User}
func NewNotificationService(channels ...Channel) *NotificationService { s := &NotificationService{channels: map[ChannelType]Channel{}, users: map[string]User{}} for _, c := range channels { s.channels[c.Type()] = c } return s}
func (s *NotificationService) AddUser(u User) { s.users[u.ID] = u }
// NotifyUser sends on each requested channel the user has an address for, and returns what was sent.func (s *NotificationService) NotifyUser(userID, subject, body string, via ...ChannelType) []Message { user, ok := s.users[userID] if !ok { return nil } var sent []Message for _, t := range via { to, hasAddress := user.Addresses[t] channel, hasChannel := s.channels[t] if !hasAddress || !hasChannel { continue } m := Message{t, to, subject, body} channel.Send(m) sent = append(sent, m) } return sent}// The ways we can reach a user. Adding one is a new value here and a new Channel class.type ChannelType = 'email' | 'sms' | 'push';
// A fixed order to walk channels in, so results are predictable.const ALL_CHANNELS: readonly ChannelType[] = ['email', 'sms', 'push'];
// One message on one channel: where it goes and what it says. SMS ignores the subject.type Message = { type: ChannelType; to: string; subject: string; body: string };
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.interface Channel { readonly type: ChannelType; send(m: Message): void;}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.type User = { id: string; addresses: Partial<Record<ChannelType, string>> };
class NotificationService { private readonly channels = new Map<ChannelType, Channel>(); private readonly users = new Map<string, User>();
constructor(channels: Channel[]) { for (const c of channels) this.channels.set(c.type, c); }
addUser(u: User): void { this.users.set(u.id, u); }
// Sends on each requested channel the user has an address for, and returns what was sent. notifyUser(userId: string, subject: string, body: string, via: ChannelType[]): Message[] { const user = this.users.get(userId); if (!user) return []; const sent: Message[] = []; for (const type of via) { const to = user.addresses[type]; const channel = this.channels.get(type); if (to === undefined || !channel) continue; const m = { type, to, subject, body }; channel.send(m); sent.push(m); } return sent; }}The code shows the interface and the service. The three channel classes are thin wrappers around each provider’s library, and the tests use a fake channel that records what it was given.
Go deeperWhat a channel adapter actually does2 min
Each provider has its own API, and they all look different. An email provider wants a sender, a recipient, a subject and an HTML body, and answers with a message id. An SMS gateway wants a phone number in international format and plain text, usually limited to 160 characters per segment, and answers with a status code. A push service wants a device token and a small JSON payload, and may answer “this token is no longer valid”.
An adapter’s job is to hide all of that behind send(message):
- Translate the message into the provider’s request: format the phone number, wrap the email body in HTML, build the push payload.
- Call the provider, with a time limit, so a hung connection becomes a timeout instead of a stuck worker.
- Translate the answer into our three outcomes. A timeout or “too many requests” becomes RETRY. “Invalid number” or “token not registered” becomes FAILED. Success becomes SENT.
Step 3 is the important one. Only the adapter knows what the provider’s error codes mean, so only the adapter can decide whether a failure is temporary. The rest of the service just reads the outcome. Switching from one SMS provider to another means writing one new adapter, and nothing else changes.
Stage 3: Templates
Section titled “Stage 3: Templates”Product wants every “order shipped” message worded the same, and the SMS must be short. Who writes the text?
What they are testing
Whether the wording moves out of the callers into templates the service owns, one per channel, and whether you think about a template with a value missing. “Hi null” is the red flag.
A Template is one kind of notification, such as shipped, with a subject and one body per
channel. An email can be friendly and long. An SMS is one line. A push notification fits on a lock
screen. A channel with no body in the template isn’t used for that notification: a receipt goes by
email only, even to a user who has a phone number.
Callers now send a template id and values, such as {name: "Ana", order: "42"}. Rendering
replaces every {{name}} in the text with its value. A placeholder with no value is an error.
Here is the shipped template the tests use, rendered for Ana and order 42:
| Channel | Template body | What Ana receives |
|---|---|---|
Hi {{name}}, order {{order}} is on its way. |
Hi Ana, order 42 is on its way. | |
| SMS | Order {{order}} shipped |
Order 42 shipped |
| Push | Order {{order}} shipped |
Nothing: Ana has no push token |
The email’s subject, Order {{order}} shipped, is rendered the same way. One request from the
order service became two messages, each worded for its channel, and nobody outside the service wrote
a word of it.
And the service renders every channel’s message before sending any. So a missing value stops the whole notification, instead of sending the email and then failing on the SMS. Half a notification is worse than none, because the caller can’t tell which half went out.
Scroll or pinch to zoom, drag to pan
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
- New in this step
// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.record Template(String id, String subject, Map<ChannelType, String> bodies) {}
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.class MissingValueException extends RuntimeException { MissingValueException(String name) { super("no value for {{" + name + "}}"); }}
// Every template, by id, and the code that fills them in.class Templates { private static final Pattern PLACEHOLDER = Pattern.compile("\\{\\{(\\w+)}}"); private final Map<String, Template> byId = new HashMap<>();
void add(Template t) { byId.put(t.id(), t); }
Template get(String id) { return byId.get(id); }
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get // "Hi {{name}}, your order {{orderId}} has shipped". static String render(String text, Map<String, String> values) { Matcher m = PLACEHOLDER.matcher(text); StringBuilder out = new StringBuilder(); while (m.find()) { String value = values.get(m.group(1)); if (value == null) throw new MissingValueException(m.group(1)); m.appendReplacement(out, Matcher.quoteReplacement(value)); } return m.appendTail(out).toString(); }}
class NotificationService { private final Map<ChannelType, Channel> channels = new EnumMap<>(ChannelType.class); private final Map<String, User> users = new HashMap<>(); private final Templates templates;
NotificationService(List<Channel> channels, Templates templates) { for (Channel c : channels) this.channels.put(c.type(), c); this.templates = templates; }
void addUser(User u) { users.put(u.id(), u); }
// Renders every message before sending any, so a missing value stops the whole notification // instead of sending the email and then failing on the SMS. List<Message> notifyUser(String userId, String templateId, Map<String, String> values) { List<Message> messages = render(userId, templateId, values); for (Message m : messages) channels.get(m.type()).send(m); return messages; }
List<Message> render(String userId, String templateId, Map<String, String> values) { User user = users.get(userId); Template t = templates.get(templateId); List<Message> out = new ArrayList<>(); if (user == null || t == null) return out; for (ChannelType type : ChannelType.values()) { // a fixed order, so results are predictable String body = t.bodies().get(type); String to = user.addresses().get(type); if (body == null || to == null || !channels.containsKey(type)) continue; out.add(new Message(type, to, Templates.render(t.subject(), values), Templates.render(body, values))); } return out; }}# One kind of notification, such as "order shipped", written once per channel. An SMS is# short; an email can say more. A channel with no text here isn't used for this notification.@dataclass(frozen=True)class Template: id: str subject: str bodies: dict[ChannelType, str]
# Raised when a template names a value the caller didn't give. A bug in the caller, so it# is loud.class MissingValueError(Exception): def __init__(self, name: str): super().__init__(f"no value for {{{{{name}}}}}")
_PLACEHOLDER = re.compile(r"\{\{(\w+)\}\}")
# Fills in every {{name}}. A placeholder with no value raises: a customer should never get# "Hi {{name}}, your order {{order}} has shipped".def render(text: str, values: dict[str, str]) -> str: def value(m: re.Match[str]) -> str: if m.group(1) not in values: raise MissingValueError(m.group(1)) return values[m.group(1)]
return _PLACEHOLDER.sub(value, text)
# Every template, by id, and the code that fills them in.class Templates: def __init__(self) -> None: self._by_id: dict[str, Template] = {}
def add(self, t: Template) -> None: self._by_id[t.id] = t
def get(self, id: str) -> Template | None: return self._by_id.get(id)
class NotificationService: def __init__(self, channels: list[Channel], templates: Templates): self._channels = {c.type: c for c in channels} self._users: dict[str, User] = {} self._templates = templates
def add_user(self, u: User) -> None: self._users[u.id] = u
def notify_user(self, user_id: str, template_id: str, values: dict[str, str]) -> list[Message]: # Renders every message before sending any, so a missing value stops the whole # notification instead of sending the email and then failing on the SMS. messages = self._render(user_id, template_id, values) for m in messages: self._channels[m.type].send(m) return messages
def _render(self, user_id: str, template_id: str, values: dict[str, str]) -> list[Message]: user = self._users.get(user_id) t = self._templates.get(template_id) if user is None or t is None: return [] out = [] for type in ChannelType: # a fixed order, so results are predictable body = t.bodies.get(type) to = user.addresses.get(type) if body is None or to is None or type not in self._channels: continue out.append(Message(type, to, render(t.subject, values), render(body, values))) return out// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.public sealed record Template(string Id, string Subject, IReadOnlyDictionary<ChannelType, string> Bodies);
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.public sealed class MissingValueException : Exception{ public MissingValueException(string name) : base($"no value for {{{{{name}}}}}") { }}
// Every template, by id, and the code that fills them in.public sealed class Templates{ private static readonly Regex Placeholder = new(@"\{\{(\w+)\}\}"); private readonly Dictionary<string, Template> _byId = new();
public void Add(Template t) => _byId[t.Id] = t;
public Template? Get(string id) => _byId.GetValueOrDefault(id);
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get // "Hi {{name}}, your order {{order}} has shipped". public static string Render(string text, IReadOnlyDictionary<string, string> values) => Placeholder.Replace(text, m => values.TryGetValue(m.Groups[1].Value, out var v) ? v : throw new MissingValueException(m.Groups[1].Value));}
public sealed class NotificationService{ private readonly Dictionary<ChannelType, IChannel> _channels; private readonly Dictionary<string, User> _users = new(); private readonly Templates _templates;
public NotificationService(IEnumerable<IChannel> channels, Templates templates) => (_channels, _templates) = (channels.ToDictionary(c => c.Type), templates);
public void AddUser(User u) => _users[u.Id] = u;
// Renders every message before sending any, so a missing value stops the whole notification // instead of sending the email and then failing on the SMS. public List<Message> NotifyUser(string userId, string templateId, IReadOnlyDictionary<string, string> values) { var messages = Render(userId, templateId, values); foreach (var m in messages) _channels[m.Type].Send(m); return messages; }
private List<Message> Render(string userId, string templateId, IReadOnlyDictionary<string, string> values) { var messages = new List<Message>(); if (!_users.TryGetValue(userId, out var user) || _templates.Get(templateId) is not { } t) return messages; foreach (var type in Enum.GetValues<ChannelType>()) // a fixed order, so results are predictable { if (!t.Bodies.TryGetValue(type, out var body) || !user.Addresses.TryGetValue(type, out var to) || !_channels.ContainsKey(type)) continue; messages.Add(new Message(type, to, Templates.Render(t.Subject, values), Templates.Render(body, values))); } return messages; }}// Template is one kind of notification, such as "order shipped", written once per channel. An SMS// is short; an email can say more. A channel with no text here isn't used for this notification.type Template struct { ID string Subject string Bodies map[ChannelType]string}
// MissingValueError means a template names a value the caller didn't give: a bug in the caller.type MissingValueError struct{ Name string }
func (e MissingValueError) Error() string { return "no value for {{" + e.Name + "}}" }
var placeholder = regexp.MustCompile(`\{\{(\w+)\}\}`)
// Render fills in every {{name}}. A placeholder with no value is an error: a customer should never// get "Hi {{name}}, your order {{order}} has shipped".func Render(text string, values map[string]string) (string, error) { var missing error out := placeholder.ReplaceAllStringFunc(text, func(p string) string { name := placeholder.FindStringSubmatch(p)[1] v, ok := values[name] if !ok && missing == nil { missing = MissingValueError{name} } return v }) return out, missing}
// Templates holds every template, by ID, and fills them in.type Templates struct{ byID map[string]Template }
func NewTemplates() *Templates { return &Templates{byID: map[string]Template{}} }
func (t *Templates) Add(tmpl Template) { t.byID[tmpl.ID] = tmpl }
func (t *Templates) Get(id string) (Template, bool) { tmpl, ok := t.byID[id] return tmpl, ok}
type NotificationService struct { channels map[ChannelType]Channel users map[string]User templates *Templates}
func NewNotificationService(templates *Templates, channels ...Channel) *NotificationService { s := &NotificationService{channels: map[ChannelType]Channel{}, users: map[string]User{}, templates: templates} for _, c := range channels { s.channels[c.Type()] = c } return s}
func (s *NotificationService) AddUser(u User) { s.users[u.ID] = u }
// NotifyUser renders every message before sending any, so a missing value stops the whole// notification instead of sending the email and then failing on the SMS.func (s *NotificationService) NotifyUser(userID, templateID string, values map[string]string) ([]Message, error) { messages, err := s.render(userID, templateID, values) if err != nil { return nil, err } for _, m := range messages { s.channels[m.Type].Send(m) } return messages, nil}
func (s *NotificationService) render(userID, templateID string, values map[string]string) ([]Message, error) { user, ok := s.users[userID] tmpl, found := s.templates.Get(templateID) if !ok || !found { return nil, nil } var out []Message for _, t := range AllChannels { body, hasBody := tmpl.Bodies[t] to, hasAddress := user.Addresses[t] if _, hasChannel := s.channels[t]; !hasBody || !hasAddress || !hasChannel { continue } subject, err := Render(tmpl.Subject, values) if err != nil { return nil, err } if body, err = Render(body, values); err != nil { return nil, err } out = append(out, Message{t, to, subject, body}) } return out, nil}// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.type Template = { id: string; subject: string; bodies: Partial<Record<ChannelType, string>> };
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.class MissingValueError extends Error { constructor(readonly name: string) { super(`no value for {{${name}}}`); }}
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get// "Hi {{name}}, your order {{order}} has shipped".function render(text: string, values: Record<string, string>): string { return text.replace(/\{\{(\w+)\}\}/g, (_, name: string) => { if (!(name in values)) throw new MissingValueError(name); return values[name]; });}
// Every template, by id, and the code that fills them in.class Templates { private readonly byId = new Map<string, Template>();
add(t: Template): void { this.byId.set(t.id, t); }
get(id: string): Template | undefined { return this.byId.get(id); }}
class NotificationService { private readonly channels = new Map<ChannelType, Channel>(); private readonly users = new Map<string, User>();
constructor( channels: Channel[], private readonly templates: Templates, ) { for (const c of channels) this.channels.set(c.type, c); }
addUser(u: User): void { this.users.set(u.id, u); }
// Renders every message before sending any, so a missing value stops the whole notification // instead of sending the email and then failing on the SMS. notifyUser(userId: string, templateId: string, values: Record<string, string>): Message[] { const messages = this.render(userId, templateId, values); for (const m of messages) this.channels.get(m.type)!.send(m); return messages; }
private render(userId: string, templateId: string, values: Record<string, string>): Message[] { const user = this.users.get(userId); const t = this.templates.get(templateId); if (!user || !t) return []; const out: Message[] = []; for (const type of ALL_CHANNELS) { const body = t.bodies[type]; const to = user.addresses[type]; if (body === undefined || to === undefined || !this.channels.has(type)) continue; out.push({ type, to, subject: render(t.subject, values), body: render(body, values) }); } return out; }}Stage 4: Queue, retry and back off
Section titled “Stage 4: Queue, retry and back off”The SMS provider times out for ten minutes. What happens to the messages, and to the services calling you?
What they are testing
Whether you separate accepting a notification from sending it, tell temporary failures from permanent ones, and retry with growing waits and a limit. Retrying straight away in a loop is the red flag: it hammers a provider that’s already struggling.
The service now has two halves. notifyUser renders the messages, queues one
Delivery per message, and returns. Callers never wait on a provider. process, run by a
worker in a loop, sends every delivery that’s due.
A channel’s send now returns an Outcome, because failures are not all alike:
- SENT: done.
- RETRY: a temporary problem, such as a timeout, a rate limit, or a “service unavailable” error. Trying later may work.
- FAILED: permanent, such as an invalid phone number or an app that was uninstalled. Trying again will fail again, and sending to a bad number again and again can get the sender marked as spam.
A retried delivery waits before its next attempt, and the wait doubles each time: one second,
then two, then four. This is called exponential backoff. The RetryPolicy holds the limit
and the first wait. Doubling gives a struggling provider room to recover, while retrying at once
from every queue would keep it down. After the last attempt, the delivery is marked failed.
Scroll or pinch to zoom, drag to pan
- Color key
- State
- Fails or stops
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
- New in this step
// What a provider said. RETRY is a temporary problem, such as a timeout or a rate limit, worth// trying again. FAILED is permanent, such as an invalid phone number, and retrying won't help.enum Outcome { SENT, RETRY, FAILED }
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.interface Channel { ChannelType type(); Outcome send(Message m);}
enum Status { PENDING, SENT, FAILED }
// One message's journey: queued, tried one or more times, then sent or given up on.class Delivery { final String id; final Message message; Status status = Status.PENDING; int attempts; // tries so far, for the retry limit and the back-off delay long nextAttemptAt; // not sent again before this time
Delivery(String id, Message message, long now) { this.id = id; this.message = message; this.nextAttemptAt = now; }}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.record RetryPolicy(int maxAttempts, long baseDelay) { long delayAfter(int attempts) { return baseDelay << (attempts - 1); }}
class NotificationService { private final Map<ChannelType, Channel> channels = new EnumMap<>(ChannelType.class); private final Map<String, User> users = new HashMap<>(); private final Templates templates; private final RetryPolicy retry; private final List<Delivery> pending = new ArrayList<>(); private int nextId = 1;
NotificationService(List<Channel> channels, Templates templates, RetryPolicy retry) { for (Channel c : channels) this.channels.put(c.type(), c); this.templates = templates; this.retry = retry; }
void addUser(User u) { users.put(u.id(), u); }
// notifyUser only queues. The caller, say the order service, returns at once instead of // waiting on an email provider, and a failed send can wait and retry without holding anyone up. List<Delivery> notifyUser(String userId, String templateId, Map<String, String> values, long now) { List<Delivery> queued = new ArrayList<>(); for (Message m : render(userId, templateId, values)) queued.add(new Delivery("D" + nextId++, m, now)); pending.addAll(queued); return queued; }
// process tries every delivery that is due, once. A worker calls it in a loop. int process(long now) { List<Delivery> due = pending.stream().filter(d -> d.nextAttemptAt <= now).toList(); for (Delivery d : due) attempt(d, now); pending.removeIf(d -> d.status != Status.PENDING); return due.size(); }
private void attempt(Delivery d, long now) { d.attempts++; Outcome outcome = channels.get(d.message.type()).send(d.message); if (outcome == Outcome.SENT) d.status = Status.SENT; else if (outcome == Outcome.FAILED || d.attempts >= retry.maxAttempts()) d.status = Status.FAILED; else d.nextAttemptAt = now + retry.delayAfter(d.attempts); }
private List<Message> render(String userId, String templateId, Map<String, String> values) { User user = users.get(userId); Template t = templates.get(templateId); List<Message> out = new ArrayList<>(); if (user == null || t == null) return out; for (ChannelType type : ChannelType.values()) { // a fixed order, so results are predictable String body = t.bodies().get(type); String to = user.addresses().get(type); if (body == null || to == null || !channels.containsKey(type)) continue; out.add(new Message(type, to, Templates.render(t.subject(), values), Templates.render(body, values))); } return out; }}# What a provider said. RETRY is a temporary problem, such as a timeout or a rate limit,# worth trying again. FAILED is permanent, such as an invalid phone number.class Outcome(Enum): SENT = auto() RETRY = auto() FAILED = auto()
# Every channel looks the same to the service. Each implementation adapts one provider's API,# such as an email service, an SMS gateway or a push service, to this one method.class Channel(Protocol): type: ChannelType
def send(self, m: Message) -> Outcome: ...
class Status(Enum): PENDING = auto() SENT = auto() FAILED = auto()
# One message's journey: queued, tried one or more times, then sent or given up on.@dataclass(eq=False)class Delivery: id: str message: Message next_attempt_at: int status: Status = Status.PENDING attempts: int = 0
# Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling# gets room to recover instead of a flood of retries from every sender at once.@dataclass(frozen=True)class RetryPolicy: max_attempts: int base_delay: int
def delay_after(self, attempts: int) -> int: return self.base_delay << (attempts - 1)
class NotificationService: def __init__(self, channels: list[Channel], templates: Templates, retry: RetryPolicy): self._channels = {c.type: c for c in channels} self._users: dict[str, User] = {} self._templates = templates self._retry = retry self._pending: list[Delivery] = [] self._next_id = itertools.count(1)
def add_user(self, u: User) -> None: self._users[u.id] = u
def notify_user(self, user_id: str, template_id: str, values: dict[str, str], now: int) -> list[Delivery]: # Only queues. The caller, say the order service, returns at once instead of waiting on # an email provider, and a failed send can wait and retry without holding anyone up. queued = [Delivery(f"D{next(self._next_id)}", m, now) for m in self._render(user_id, template_id, values)] self._pending.extend(queued) return queued
# Tries every delivery that is due, once. A worker calls it in a loop. def process(self, now: int) -> int: due = [d for d in self._pending if d.next_attempt_at <= now] for d in due: self._attempt(d, now) self._pending = [d for d in self._pending if d.status is Status.PENDING] return len(due)
def _attempt(self, d: Delivery, now: int) -> None: d.attempts += 1 outcome = self._channels[d.message.type].send(d.message) if outcome is Outcome.SENT: d.status = Status.SENT elif outcome is Outcome.FAILED or d.attempts >= self._retry.max_attempts: d.status = Status.FAILED else: d.next_attempt_at = now + self._retry.delay_after(d.attempts)
def _render(self, user_id: str, template_id: str, values: dict[str, str]) -> list[Message]: user = self._users.get(user_id) t = self._templates.get(template_id) if user is None or t is None: return [] out = [] for type in ChannelType: body = t.bodies.get(type) to = user.addresses.get(type) if body is None or to is None or type not in self._channels: continue out.append(Message(type, to, render(t.subject, values), render(body, values))) return out// What a provider said. Retry is a temporary problem, such as a timeout or a rate limit, worth// trying again. Failed is permanent, such as an invalid phone number.public enum Outcome { Sent, Retry, Failed }
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.public interface IChannel{ ChannelType Type { get; } Outcome Send(Message m);}
public enum Status { Pending, Sent, Failed }
// One message's journey: queued, tried one or more times, then sent or given up on.public sealed class Delivery{ public Delivery(string id, Message message, long now) => (Id, Message, NextAttemptAt) = (id, message, now);
public string Id { get; } public Message Message { get; } public Status Status { get; set; } = Status.Pending; public int Attempts { get; set; } // tries so far, for the retry limit and the back-off delay public long NextAttemptAt { get; set; } // not sent again before this time}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.public sealed record RetryPolicy(int MaxAttempts, long BaseDelay){ public long DelayAfter(int attempts) => BaseDelay << (attempts - 1);}
public sealed class NotificationService{ private readonly Dictionary<ChannelType, IChannel> _channels; private readonly Dictionary<string, User> _users = new(); private readonly Templates _templates; private readonly RetryPolicy _retry; private readonly List<Delivery> _pending = new(); private int _nextId = 1;
public NotificationService(IEnumerable<IChannel> channels, Templates templates, RetryPolicy retry) => (_channels, _templates, _retry) = (channels.ToDictionary(c => c.Type), templates, retry);
public void AddUser(User u) => _users[u.Id] = u;
// Only queues. The caller, say the order service, returns at once instead of waiting on an // email provider, and a failed send can wait and retry without holding anyone up. public List<Delivery> NotifyUser(string userId, string templateId, IReadOnlyDictionary<string, string> values, long now) { var queued = Render(userId, templateId, values).Select(m => new Delivery($"D{_nextId++}", m, now)).ToList(); _pending.AddRange(queued); return queued; }
// Tries every delivery that is due, once. A worker calls it in a loop. public int Process(long now) { var due = _pending.Where(d => d.NextAttemptAt <= now).ToList(); foreach (var d in due) Attempt(d, now); _pending.RemoveAll(d => d.Status != Status.Pending); return due.Count; }
private void Attempt(Delivery d, long now) { d.Attempts++; var outcome = _channels[d.Message.Type].Send(d.Message); if (outcome == Outcome.Sent) d.Status = Status.Sent; else if (outcome == Outcome.Failed || d.Attempts >= _retry.MaxAttempts) d.Status = Status.Failed; else d.NextAttemptAt = now + _retry.DelayAfter(d.Attempts); }
private List<Message> Render(string userId, string templateId, IReadOnlyDictionary<string, string> values) { var messages = new List<Message>(); if (!_users.TryGetValue(userId, out var user) || _templates.Get(templateId) is not { } t) return messages; foreach (var type in Enum.GetValues<ChannelType>()) { if (!t.Bodies.TryGetValue(type, out var body) || !user.Addresses.TryGetValue(type, out var to) || !_channels.ContainsKey(type)) continue; messages.Add(new Message(type, to, Templates.Render(t.Subject, values), Templates.Render(body, values))); } return messages; }}// Outcome is what a provider said. Retry is a temporary problem, such as a timeout or a rate// limit, worth trying again. Failed is permanent, such as an invalid phone number.type Outcome int
const ( Sent Outcome = iota Retry Failed)
// Channel looks the same to the service for every provider. Each implementation adapts one// provider's API, such as an email service, an SMS gateway or a push service, to this.type Channel interface { Type() ChannelType Send(m Message) Outcome}
type Status int
const ( StatusPending Status = iota StatusSent StatusFailed)
// Delivery is one message's journey: queued, tried one or more times, then sent or given up on.type Delivery struct { ID string Message Message Status Status Attempts int // tries so far, for the retry limit and the back-off delay NextAttemptAt int64 // not sent again before this time}
// RetryPolicy waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's// struggling gets room to recover instead of a flood of retries from every sender at once.type RetryPolicy struct { MaxAttempts int BaseDelay int64}
func (p RetryPolicy) DelayAfter(attempts int) int64 { return p.BaseDelay << (attempts - 1) }
type NotificationService struct { channels map[ChannelType]Channel users map[string]User templates *Templates retry RetryPolicy pending []*Delivery nextID int}
func NewNotificationService(templates *Templates, retry RetryPolicy, channels ...Channel) *NotificationService { s := &NotificationService{channels: map[ChannelType]Channel{}, users: map[string]User{}, templates: templates, retry: retry} for _, c := range channels { s.channels[c.Type()] = c } return s}
func (s *NotificationService) AddUser(u User) { s.users[u.ID] = u }
// NotifyUser only queues. The caller, say the order service, returns at once instead of waiting// on an email provider, and a failed send can wait and retry without holding anyone up.func (s *NotificationService) NotifyUser(userID, templateID string, values map[string]string, now int64) ([]*Delivery, error) { messages, err := s.render(userID, templateID, values) if err != nil { return nil, err } var queued []*Delivery for _, m := range messages { s.nextID++ queued = append(queued, &Delivery{ID: fmt.Sprintf("D%d", s.nextID), Message: m, NextAttemptAt: now}) } s.pending = append(s.pending, queued...) return queued, nil}
// Process tries every delivery that is due, once. A worker calls it in a loop.func (s *NotificationService) Process(now int64) int { tried := 0 for _, d := range s.pending { if d.NextAttemptAt <= now { s.attempt(d, now) tried++ } } s.pending = slices.DeleteFunc(s.pending, func(d *Delivery) bool { return d.Status != StatusPending }) return tried}
func (s *NotificationService) attempt(d *Delivery, now int64) { d.Attempts++ switch outcome := s.channels[d.Message.Type].Send(d.Message); { case outcome == Sent: d.Status = StatusSent case outcome == Failed || d.Attempts >= s.retry.MaxAttempts: d.Status = StatusFailed default: d.NextAttemptAt = now + s.retry.DelayAfter(d.Attempts) }}
func (s *NotificationService) render(userID, templateID string, values map[string]string) ([]Message, error) { user, ok := s.users[userID] tmpl, found := s.templates.Get(templateID) if !ok || !found { return nil, nil } var out []Message for _, t := range AllChannels { body, hasBody := tmpl.Bodies[t] to, hasAddress := user.Addresses[t] if _, hasChannel := s.channels[t]; !hasBody || !hasAddress || !hasChannel { continue } subject, err := Render(tmpl.Subject, values) if err != nil { return nil, err } if body, err = Render(body, values); err != nil { return nil, err } out = append(out, Message{t, to, subject, body}) } return out, nil}// What a provider said. 'retry' is a temporary problem, such as a timeout or a rate limit, worth// trying again. 'failed' is permanent, such as an invalid phone number.type Outcome = 'sent' | 'retry' | 'failed';
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.interface Channel { readonly type: ChannelType; send(m: Message): Outcome;}
type Status = 'pending' | 'sent' | 'failed';
// One message's journey: queued, tried one or more times, then sent or given up on.class Delivery { status: Status = 'pending'; attempts = 0; // tries so far, for the retry limit and the back-off delay
constructor( readonly id: string, readonly message: Message, public nextAttemptAt: number, // not sent again before this time ) {}}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.class RetryPolicy { constructor( readonly maxAttempts: number, readonly baseDelay: number, ) {}
delayAfter(attempts: number): number { return this.baseDelay * 2 ** (attempts - 1); }}
class NotificationService { private readonly channels = new Map<ChannelType, Channel>(); private readonly users = new Map<string, User>(); private pending: Delivery[] = []; private nextId = 1;
constructor( channels: Channel[], private readonly templates: Templates, private readonly retry: RetryPolicy, ) { for (const c of channels) this.channels.set(c.type, c); }
addUser(u: User): void { this.users.set(u.id, u); }
// Only queues. The caller, say the order service, returns at once instead of waiting on an // email provider, and a failed send can wait and retry without holding anyone up. notifyUser(userId: string, templateId: string, values: Record<string, string>, now: number): Delivery[] { const queued = this.render(userId, templateId, values).map((m) => new Delivery(`D${this.nextId++}`, m, now)); this.pending.push(...queued); return queued; }
// Tries every delivery that is due, once. A worker calls it in a loop. process(now: number): number { const due = this.pending.filter((d) => d.nextAttemptAt <= now); for (const d of due) this.attempt(d, now); this.pending = this.pending.filter((d) => d.status === 'pending'); return due.length; }
private attempt(d: Delivery, now: number): void { d.attempts++; const outcome = this.channels.get(d.message.type)!.send(d.message); if (outcome === 'sent') d.status = 'sent'; else if (outcome === 'failed' || d.attempts >= this.retry.maxAttempts) d.status = 'failed'; else d.nextAttemptAt = now + this.retry.delayAfter(d.attempts); }
private render(userId: string, templateId: string, values: Record<string, string>): Message[] { const user = this.users.get(userId); const t = this.templates.get(templateId); if (!user || !t) return []; const out: Message[] = []; for (const type of ALL_CHANNELS) { const body = t.bodies[type]; const to = user.addresses[type]; if (body === undefined || to === undefined || !this.channels.has(type)) continue; out.push({ type, to, subject: render(t.subject, values), body: render(body, values) }); } return out; }}Go deeperA ten-minute SMS outage, minute by minute2 min
Let’s follow one SMS through an outage, with a limit of three attempts and a first wait of one second.
- 12:00:00. The order service calls
notifyUser. The service renders the email, SMS and push messages, queues three deliveries, and returns. The order service is done in milliseconds. - 12:00:00. A worker sends the email (SENT) and the push (SENT). The SMS provider times out: RETRY. The SMS delivery’s next attempt is set to 12:00:01.
- 12:00:01. The worker tries again. Still down: RETRY. Next attempt: 12:00:03, two seconds later.
- 12:00:03. Third attempt, still down. That was the last one, so the delivery is marked failed.
Notice what didn’t happen. The order service never waited. The email and push weren’t held up by the SMS. And the SMS provider got three tries in three seconds, not hundreds.
But a ten-minute outage outlasts three attempts in three seconds. That’s a choice of numbers, not of
design: a policy with a first wait of 30 seconds and eight attempts keeps trying for over an hour
(30 s, 1 min, 2, 4, 8, 16, 32 minutes). For an order update, that’s right. For a sign-in code that
expires in five minutes, retrying for an hour is pointless, so different templates can use different
policies. That’s why RetryPolicy is its own object.
Go deeperHow to wait between retries1 min
| Fixed wait | Exponential backoff (our choice) | Backoff with jitter | |
|---|---|---|---|
| A brief blip | Recovers quickly | Recovers quickly | Recovers quickly |
| A ten-minute outage | Hammers the provider | Backs off, then tries rarely | Backs off, then tries rarely |
| Many messages failing together | Retry together | Still retry together | Spread out, no stampede |
| Simplicity | Simplest | One doubling | One random number more |
In production, add jitter: a random part of each wait, so ten thousand messages that failed in the same second don’t all retry in the same second. It’s left out here so the tests can predict the times.
Go deeperTemporary or permanent: sorting real errors1 min
Getting this sorting right matters more than the retry numbers. Some common cases:
| What the provider says | Outcome | Why |
|---|---|---|
| Timeout, or no answer at all | RETRY | The provider may be slow, or the network blinked |
| “Too many requests” (HTTP 429) | RETRY | We’re sending too fast; waiting is the fix |
| “Service unavailable” (HTTP 503) | RETRY | The provider is down or restarting |
| “Invalid phone number” or “bad address” | FAILED | The same address will fail every time |
| “Token not registered” (app uninstalled) | FAILED | The device will never accept it again |
| “Unauthorised” (our credentials are bad) | FAILED, and alert | Retrying won’t help until a person fixes it |
A wrong sort costs in both directions. Retrying a permanent failure wastes attempts, and can get us marked as spam. Giving up on a temporary failure loses a message that would have gone through a minute later. When unsure, retry: the limit stops it from going on forever.
This is enough for a mid-level answer. Channels behind one interface, templates per channel, a queue, and retries with a limit. The rest of the page is what makes it a senior answer.
The full design
Section titled “The full design”Here is everything together, including the pieces for sharing that we add next. Callers use the service. The service renders templates, queues deliveries, and sends them through channels, each of which wraps one provider.
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
AllInOne.java
AllInOne.java
// The complete design, built from every stage on this page.
import java.util.*;import java.util.concurrent.*;import java.util.regex.*;
// Many services call notifyUser at once, and several workers call process at once. One lock// guards the queue and the delivery records. Sending happens outside it: a provider can take// seconds, and holding the lock that long would stop every other notification.class NotificationService { private final Map<ChannelType, Channel> channels = new EnumMap<>(ChannelType.class); private final Map<String, User> users = new ConcurrentHashMap<>(); private final Templates templates; private final RetryPolicy retry; private final Object lock = new Object(); private final List<Delivery> pending = new ArrayList<>(); // waiting for a worker private final Map<String, List<Delivery>> byKey = new HashMap<>(); // caller's key -> its deliveries private int nextId = 1;
NotificationService(List<Channel> channels, Templates templates, RetryPolicy retry) { for (Channel c : channels) this.channels.put(c.type(), c); this.templates = templates; this.retry = retry; }
void addUser(User u) { users.put(u.id(), u); }
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. Checking and recording the key is one step under the lock, or two retries // racing would both get through. List<Delivery> notifyUser(String key, String userId, String templateId, Map<String, String> values, long now) { List<Message> messages = render(userId, templateId, values); synchronized (lock) { List<Delivery> existing = byKey.get(key); if (existing != null) return existing; List<Delivery> queued = new ArrayList<>(); for (Message m : messages) queued.add(new Delivery("D" + nextId++, m, now)); byKey.put(key, queued); pending.addAll(queued); return queued; } }
// Claim under the lock, send outside it, record under it again. Two workers can never claim the // same delivery, because claiming changes its status before the lock is released. int process(long now) { List<Delivery> claimed = new ArrayList<>(); synchronized (lock) { for (Delivery d : pending) { if (d.status == Status.PENDING && d.nextAttemptAt <= now) { d.status = Status.SENDING; d.attempts++; claimed.add(d); } } } for (Delivery d : claimed) { Outcome outcome = channels.get(d.message.type()).send(d.message); synchronized (lock) { record(d, outcome, now); } } return claimed.size(); }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private void record(Delivery d, Outcome outcome, long now) { if (outcome == Outcome.SENT) d.status = Status.SENT; else if (outcome == Outcome.FAILED || d.attempts >= retry.maxAttempts()) d.status = Status.FAILED; else { d.status = Status.PENDING; d.nextAttemptAt = now + retry.delayAfter(d.attempts); } if (d.status != Status.PENDING) pending.remove(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private List<Message> render(String userId, String templateId, Map<String, String> values) { User user = users.get(userId); Template t = templates.get(templateId); List<Message> out = new ArrayList<>(); if (user == null || t == null) return out; for (ChannelType type : ChannelType.values()) { // a fixed order, so results are predictable String body = t.bodies().get(type); String to = user.addresses().get(type); if (body == null || to == null || !channels.containsKey(type)) continue; out.add(new Message(type, to, Templates.render(t.subject(), values), Templates.render(body, values))); } return out; }}
// The ways we can reach a user. Adding one is a new value here and a new Channel class.enum ChannelType { EMAIL, SMS, PUSH }
// One message on one channel: where it goes and what it says. SMS ignores the subject.record Message(ChannelType type, String to, String subject, String body) {}
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.interface Channel { ChannelType type(); Outcome send(Message m);}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.record User(String id, Map<ChannelType, String> addresses) {}
// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.record Template(String id, String subject, Map<ChannelType, String> bodies) {}
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.class MissingValueException extends RuntimeException { MissingValueException(String name) { super("no value for {{" + name + "}}"); }}
// Every template, by id, and the code that fills them in.class Templates { private static final Pattern PLACEHOLDER = Pattern.compile("\\{\\{(\\w+)}}"); private final Map<String, Template> byId = new HashMap<>();
void add(Template t) { byId.put(t.id(), t); }
Template get(String id) { return byId.get(id); }
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get // "Hi {{name}}, your order {{orderId}} has shipped". static String render(String text, Map<String, String> values) { Matcher m = PLACEHOLDER.matcher(text); StringBuilder out = new StringBuilder(); while (m.find()) { String value = values.get(m.group(1)); if (value == null) throw new MissingValueException(m.group(1)); m.appendReplacement(out, Matcher.quoteReplacement(value)); } return m.appendTail(out).toString(); }}
// What a provider said. RETRY is a temporary problem, such as a timeout or a rate limit, worth// trying again. FAILED is permanent, such as an invalid phone number, and retrying won't help.enum Outcome { SENT, RETRY, FAILED }
// SENDING marks a delivery a worker has claimed, so no other worker sends it too.enum Status { PENDING, SENDING, SENT, FAILED }
// One message's journey: queued, tried one or more times, then sent or given up on.class Delivery { final String id; final Message message; Status status = Status.PENDING; int attempts; // tries so far, for the retry limit and the back-off delay long nextAttemptAt; // not sent again before this time
Delivery(String id, Message message, long now) { this.id = id; this.message = message; this.nextAttemptAt = now; }}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.record RetryPolicy(int maxAttempts, long baseDelay) { long delayAfter(int attempts) { return baseDelay << (attempts - 1); }}Read onlyJava 21Every file builds in CI13 files165 lines
all_in_one.py
all_in_one.py
# The complete design, built from every stage on this page.
from __future__ import annotations
import itertoolsimport reimport threadingfrom dataclasses import dataclassfrom enum import Enum, autofrom typing import Protocol
# Many services call notify_user at once, and several workers call process at once. One# lock guards the queue and the delivery records. Sending happens outside it: a provider can# take seconds, and holding the lock that long would stop every other notification.class NotificationService: def __init__(self, channels: list[Channel], templates: Templates, retry: RetryPolicy): self._channels = {c.type: c for c in channels} self._users: dict[str, User] = {} self._templates = templates self._retry = retry self._lock = threading.Lock() self._pending: list[Delivery] = [] # waiting for a worker self._by_key: dict[str, list[Delivery]] = {} # caller's key -> its deliveries self._next_id = itertools.count(1)
def add_user(self, u: User) -> None: self._users[u.id] = u
def notify_user(self, key: str, user_id: str, template_id: str, values: dict[str, str], now: int) -> list[Delivery]: # The caller passes a key that names this notification, such as "shipped:order-42". A # caller that times out and retries sends the same key, and gets the same deliveries back # instead of a second email. Checking and recording the key is one step under the lock, # or two retries racing would both get through. messages = self._render(user_id, template_id, values) with self._lock: if key in self._by_key: return self._by_key[key] queued = [Delivery(f"D{next(self._next_id)}", m, now) for m in messages] self._by_key[key] = queued self._pending.extend(queued) return queued
def process(self, now: int) -> int: # Claim under the lock, send outside it, record under it again. Two workers can never # claim the same delivery, because claiming changes its status before the lock is # released. with self._lock: claimed = [d for d in self._pending if d.status is Status.PENDING and d.next_attempt_at <= now] for d in claimed: d.status = Status.SENDING d.attempts += 1 for d in claimed: outcome = self._channels[d.message.type].send(d.message) with self._lock: self._record(d, outcome, now) return len(claimed)
# Sent and permanent failures are final. A temporary failure waits longer each time, until # the retry limit makes it final too. def _record(self, d: Delivery, outcome: Outcome, now: int) -> None: if outcome is Outcome.SENT: d.status = Status.SENT elif outcome is Outcome.FAILED or d.attempts >= self._retry.max_attempts: d.status = Status.FAILED else: d.status = Status.PENDING d.next_attempt_at = now + self._retry.delay_after(d.attempts) if d.status is not Status.PENDING: self._pending.remove(d)
# One message per channel where the user has an address and the template has text. An # unknown user or template sends nothing. def _render(self, user_id: str, template_id: str, values: dict[str, str]) -> list[Message]: user = self._users.get(user_id) t = self._templates.get(template_id) if user is None or t is None: return [] out = [] for type in ChannelType: body = t.bodies.get(type) to = user.addresses.get(type) if body is None or to is None or type not in self._channels: continue out.append(Message(type, to, render(t.subject, values), render(body, values))) return out
# The ways we can reach a user. Adding one is a new value here and a new Channel class.class ChannelType(Enum): EMAIL = auto() SMS = auto() PUSH = auto()
# One message on one channel: where it goes and what it says. SMS ignores the subject.@dataclass(frozen=True)class Message: type: ChannelType to: str subject: str body: str
# Every channel looks the same to the service. Each implementation adapts one provider's API,# such as an email service, an SMS gateway or a push service, to this one method.class Channel(Protocol): type: ChannelType
def send(self, m: Message) -> Outcome: ...
# A user's address on each channel they've given us: an email address, a phone number, a# device token. No address on a channel means nothing is sent there.@dataclass(frozen=True)class User: id: str addresses: dict[ChannelType, str]
# One kind of notification, such as "order shipped", written once per channel. An SMS is# short; an email can say more. A channel with no text here isn't used for this notification.@dataclass(frozen=True)class Template: id: str subject: str bodies: dict[ChannelType, str]
# Raised when a template names a value the caller didn't give. A bug in the caller, so it# is loud.class MissingValueError(Exception): def __init__(self, name: str): super().__init__(f"no value for {{{{{name}}}}}")
_PLACEHOLDER = re.compile(r"\{\{(\w+)\}\}")
# Fills in every {{name}}. A placeholder with no value raises: a customer should never get# "Hi {{name}}, your order {{order}} has shipped".def render(text: str, values: dict[str, str]) -> str: def value(m: re.Match[str]) -> str: if m.group(1) not in values: raise MissingValueError(m.group(1)) return values[m.group(1)]
return _PLACEHOLDER.sub(value, text)
# Every template, by id, and the code that fills them in.class Templates: def __init__(self) -> None: self._by_id: dict[str, Template] = {}
def add(self, t: Template) -> None: self._by_id[t.id] = t
def get(self, id: str) -> Template | None: return self._by_id.get(id)
# What a provider said. RETRY is a temporary problem, such as a timeout or a rate limit,# worth trying again. FAILED is permanent, such as an invalid phone number.class Outcome(Enum): SENT = auto() RETRY = auto() FAILED = auto()
class Status(Enum): PENDING = auto() SENDING = auto() # a worker has claimed it, so no other worker sends it too SENT = auto() FAILED = auto()
# One message's journey: queued, tried one or more times, then sent or given up on.@dataclass(eq=False)class Delivery: id: str message: Message next_attempt_at: int # not sent again before this time status: Status = Status.PENDING attempts: int = 0 # tries so far, for the retry limit and the back-off delay
# Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling# gets room to recover instead of a flood of retries from every sender at once.@dataclass(frozen=True)class RetryPolicy: max_attempts: int base_delay: int
def delay_after(self, attempts: int) -> int: return self.base_delay << (attempts - 1)Read onlyPython 3.12Every file builds in CI14 files196 lines
AllInOne.cs
AllInOne.cs
// The complete design, built from every stage on this page.
using System.Collections.Concurrent;using System.Text.RegularExpressions;using System.Threading.Channels;
// Many services call NotifyUser at once, and several workers call Process at once. One lock// guards the queue and the delivery records. Sending happens outside it: a provider can take// seconds, and holding the lock that long would stop every other notification.public sealed class NotificationService{ private readonly Dictionary<ChannelType, IChannel> _channels; private readonly ConcurrentDictionary<string, User> _users = new(); private readonly Templates _templates; private readonly RetryPolicy _retry; private readonly object _lock = new(); private readonly List<Delivery> _pending = new(); // waiting for a worker private readonly Dictionary<string, List<Delivery>> _byKey = new(); // caller's key -> its deliveries private int _nextId = 1;
public NotificationService(IEnumerable<IChannel> channels, Templates templates, RetryPolicy retry) => (_channels, _templates, _retry) = (channels.ToDictionary(c => c.Type), templates, retry);
public void AddUser(User u) => _users[u.Id] = u;
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. Checking and recording the key is one step under the lock, or two retries // racing would both get through. public List<Delivery> NotifyUser(string key, string userId, string templateId, IReadOnlyDictionary<string, string> values, long now) { var messages = Render(userId, templateId, values); lock (_lock) { if (_byKey.TryGetValue(key, out var existing)) return existing; var queued = messages.Select(m => new Delivery($"D{_nextId++}", m, now)).ToList(); _byKey[key] = queued; _pending.AddRange(queued); return queued; } }
// Claim under the lock, send outside it, record under it again. Two workers can never claim // the same delivery, because claiming changes its status before the lock is released. public int Process(long now) { List<Delivery> claimed; lock (_lock) { claimed = _pending.Where(d => d.Status == Status.Pending && d.NextAttemptAt <= now).ToList(); foreach (var d in claimed) { d.Status = Status.Sending; d.Attempts++; } } foreach (var d in claimed) { var outcome = _channels[d.Message.Type].Send(d.Message); lock (_lock) Record(d, outcome, now); } return claimed.Count; }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private void Record(Delivery d, Outcome outcome, long now) { if (outcome == Outcome.Sent) d.Status = Status.Sent; else if (outcome == Outcome.Failed || d.Attempts >= _retry.MaxAttempts) d.Status = Status.Failed; else { d.Status = Status.Pending; d.NextAttemptAt = now + _retry.DelayAfter(d.Attempts); } if (d.Status != Status.Pending) _pending.Remove(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private List<Message> Render(string userId, string templateId, IReadOnlyDictionary<string, string> values) { var messages = new List<Message>(); if (!_users.TryGetValue(userId, out var user) || _templates.Get(templateId) is not { } t) return messages; foreach (var type in Enum.GetValues<ChannelType>()) { if (!t.Bodies.TryGetValue(type, out var body) || !user.Addresses.TryGetValue(type, out var to) || !_channels.ContainsKey(type)) continue; messages.Add(new Message(type, to, Templates.Render(t.Subject, values), Templates.Render(body, values))); } return messages; }}
// The ways we can reach a user. Adding one is a new value here and a new Channel class.public enum ChannelType { Email, Sms, Push }
// One message on one channel: where it goes and what it says. SMS ignores the subject.public sealed record Message(ChannelType Type, string To, string Subject, string Body);
// Every channel looks the same to the service. Each implementation adapts one provider's API,// such as an email service, an SMS gateway or a push service, to this one method.public interface IChannel{ ChannelType Type { get; } Outcome Send(Message m);}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.public sealed record User(string Id, IReadOnlyDictionary<ChannelType, string> Addresses);
// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.public sealed record Template(string Id, string Subject, IReadOnlyDictionary<ChannelType, string> Bodies);
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.public sealed class MissingValueException : Exception{ public MissingValueException(string name) : base($"no value for {{{{{name}}}}}") { }}
// Every template, by id, and the code that fills them in.public sealed class Templates{ private static readonly Regex Placeholder = new(@"\{\{(\w+)\}\}"); private readonly Dictionary<string, Template> _byId = new();
public void Add(Template t) => _byId[t.Id] = t;
public Template? Get(string id) => _byId.GetValueOrDefault(id);
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get // "Hi {{name}}, your order {{order}} has shipped". public static string Render(string text, IReadOnlyDictionary<string, string> values) => Placeholder.Replace(text, m => values.TryGetValue(m.Groups[1].Value, out var v) ? v : throw new MissingValueException(m.Groups[1].Value));}
// What a provider said. Retry is a temporary problem, such as a timeout or a rate limit, worth// trying again. Failed is permanent, such as an invalid phone number.public enum Outcome { Sent, Retry, Failed }
// Sending marks a delivery a worker has claimed, so no other worker sends it too.public enum Status { Pending, Sending, Sent, Failed }
// One message's journey: queued, tried one or more times, then sent or given up on.public sealed class Delivery{ public Delivery(string id, Message message, long now) => (Id, Message, NextAttemptAt) = (id, message, now);
public string Id { get; } public Message Message { get; } public Status Status { get; set; } = Status.Pending; public int Attempts { get; set; } // tries so far, for the retry limit and the back-off delay public long NextAttemptAt { get; set; } // not sent again before this time}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.public sealed record RetryPolicy(int MaxAttempts, long BaseDelay){ public long DelayAfter(int attempts) => BaseDelay << (attempts - 1);}Read onlyC# · .NET 10Every file builds in CI13 files163 lines
all_in_one.go
all_in_one.go
// The complete design, built from every stage on this page.
package notificationservice
import ( "fmt" "regexp" "slices" "sync")
// ChannelType is one of the ways we can reach a user. Adding one is a new value here and a new// Channel.type ChannelType int
const ( Email ChannelType = iota SMS Push)
// AllChannels is a fixed order to walk channels in, so results are predictable.var AllChannels = []ChannelType{Email, SMS, Push}
// Message is one message on one channel: where it goes and what it says. SMS ignores the subject.type Message struct { Type ChannelType To string Subject, Body string}
// User holds a user's address on each channel they've given us: an email address, a phone number,// a device token. No address on a channel means nothing is sent there.type User struct { ID string Addresses map[ChannelType]string}
// Template is one kind of notification, such as "order shipped", written once per channel. An SMS// is short; an email can say more. A channel with no text here isn't used for this notification.type Template struct { ID string Subject string Bodies map[ChannelType]string}
// MissingValueError means a template names a value the caller didn't give: a bug in the caller.type MissingValueError struct{ Name string }
func (e MissingValueError) Error() string { return "no value for {{" + e.Name + "}}" }
var placeholder = regexp.MustCompile(`\{\{(\w+)\}\}`)
// Render fills in every {{name}}. A placeholder with no value is an error: a customer should never// get "Hi {{name}}, your order {{order}} has shipped".func Render(text string, values map[string]string) (string, error) { var missing error out := placeholder.ReplaceAllStringFunc(text, func(p string) string { name := placeholder.FindStringSubmatch(p)[1] v, ok := values[name] if !ok && missing == nil { missing = MissingValueError{name} } return v }) return out, missing}
// Templates holds every template, by ID, and fills them in.type Templates struct{ byID map[string]Template }
func NewTemplates() *Templates { return &Templates{byID: map[string]Template{}} }
func (t *Templates) Add(tmpl Template) { t.byID[tmpl.ID] = tmpl }
func (t *Templates) Get(id string) (Template, bool) { tmpl, ok := t.byID[id] return tmpl, ok}
// Outcome is what a provider said. Retry is a temporary problem, such as a timeout or a rate// limit, worth trying again. Failed is permanent, such as an invalid phone number.type Outcome int
const ( Sent Outcome = iota Retry Failed)
// Channel looks the same to the service for every provider. Each implementation adapts one// provider's API, such as an email service, an SMS gateway or a push service, to this.type Channel interface { Type() ChannelType Send(m Message) Outcome}
// Delivery is one message's journey: queued, tried one or more times, then sent or given up on.type Delivery struct { ID string Message Message Status Status Attempts int // tries so far, for the retry limit and the back-off delay NextAttemptAt int64 // not sent again before this time}
// RetryPolicy waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's// struggling gets room to recover instead of a flood of retries from every sender at once.type RetryPolicy struct { MaxAttempts int BaseDelay int64}
func (p RetryPolicy) DelayAfter(attempts int) int64 { return p.BaseDelay << (attempts - 1) }
type Status int
const ( StatusPending Status = iota StatusSending // a worker has claimed it, so no other worker sends it too StatusSent StatusFailed)
// NotificationService is called by many services at once, and several workers call Process at// once. One mutex guards the queue and the delivery records. Sending happens outside it: a// provider can take seconds, and holding the mutex that long would stop every other notification.type NotificationService struct { channels map[ChannelType]Channel users map[string]User templates *Templates retry RetryPolicy mu sync.Mutex pending []*Delivery // waiting for a worker byKey map[string][]*Delivery // caller's key -> its deliveries nextID int}
func NewNotificationService(templates *Templates, retry RetryPolicy, channels ...Channel) *NotificationService { s := &NotificationService{channels: map[ChannelType]Channel{}, users: map[string]User{}, templates: templates, retry: retry, byKey: map[string][]*Delivery{}} for _, c := range channels { s.channels[c.Type()] = c } return s}
// AddUser is called while setting up, before any notifications are sent.func (s *NotificationService) AddUser(u User) { s.users[u.ID] = u }
// NotifyUser takes a key that names this notification, such as "shipped:order-42". A caller that// times out and retries sends the same key, and gets the same deliveries back instead of a second// email. Checking and recording the key is one step under the mutex, or two retries racing would// both get through.func (s *NotificationService) NotifyUser(key, userID, templateID string, values map[string]string, now int64) ([]*Delivery, error) { messages, err := s.render(userID, templateID, values) if err != nil { return nil, err } s.mu.Lock() defer s.mu.Unlock() if existing, ok := s.byKey[key]; ok { return existing, nil } var queued []*Delivery for _, m := range messages { s.nextID++ queued = append(queued, &Delivery{ID: fmt.Sprintf("D%d", s.nextID), Message: m, NextAttemptAt: now}) } s.byKey[key] = queued s.pending = append(s.pending, queued...) return queued, nil}
// Process claims under the mutex, sends outside it, and records under it again. Two workers can// never claim the same delivery, because claiming changes its status before the mutex is released.func (s *NotificationService) Process(now int64) int { var claimed []*Delivery s.mu.Lock() for _, d := range s.pending { if d.Status == StatusPending && d.NextAttemptAt <= now { d.Status = StatusSending d.Attempts++ claimed = append(claimed, d) } } s.mu.Unlock() for _, d := range claimed { outcome := s.channels[d.Message.Type].Send(d.Message) s.mu.Lock() s.record(d, outcome, now) s.mu.Unlock() } return len(claimed)}
// StatusOf reads a delivery's status under the mutex, since workers change it.func (s *NotificationService) StatusOf(d *Delivery) Status { s.mu.Lock() defer s.mu.Unlock() return d.Status}
// record: sent and permanent failures are final. A temporary failure waits longer each time, until// the retry limit makes it final too.func (s *NotificationService) record(d *Delivery, outcome Outcome, now int64) { switch { case outcome == Sent: d.Status = StatusSent case outcome == Failed || d.Attempts >= s.retry.MaxAttempts: d.Status = StatusFailed default: d.Status = StatusPending d.NextAttemptAt = now + s.retry.DelayAfter(d.Attempts) } if d.Status != StatusPending { s.pending = slices.DeleteFunc(s.pending, func(x *Delivery) bool { return x == d }) }}
// render makes one message per channel where the user has an address and the template has// text. An unknown user or template sends nothing.func (s *NotificationService) render(userID, templateID string, values map[string]string) ([]Message, error) { user, ok := s.users[userID] tmpl, found := s.templates.Get(templateID) if !ok || !found { return nil, nil } var out []Message for _, t := range AllChannels { body, hasBody := tmpl.Bodies[t] to, hasAddress := user.Addresses[t] if _, hasChannel := s.channels[t]; !hasBody || !hasAddress || !hasChannel { continue } subject, err := Render(tmpl.Subject, values) if err != nil { return nil, err } if body, err = Render(body, values); err != nil { return nil, err } out = append(out, Message{t, to, subject, body}) } return out, nil}Read onlyGo 1.24Every file builds in CI13 files245 lines
all-in-one.ts
all-in-one.ts
// The complete design, built from every stage on this page.
// Many services call notifyUser, and several workers run process at the same time. JavaScript// runs one callback at a time, so notifyUser, which has no await, is always one step: checking// and recording the key can't interleave with another call. process is different. It awaits each// send, and while it waits, another worker runs. So a delivery is claimed, marked 'sending',// before the first await, and other workers skip it.class NotificationService { private readonly channels = new Map<ChannelType, Channel>(); private readonly users = new Map<string, User>(); private readonly pending = new Set<Delivery>(); // waiting for a worker private readonly byKey = new Map<string, Delivery[]>(); // caller's key -> its deliveries private nextId = 1;
constructor( channels: Channel[], private readonly templates: Templates, private readonly retry: RetryPolicy, ) { for (const c of channels) this.channels.set(c.type, c); }
addUser(u: User): void { this.users.set(u.id, u); }
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. notifyUser(key: string, userId: string, templateId: string, values: Record<string, string>, now: number): Delivery[] { const existing = this.byKey.get(key); if (existing) return existing; const queued = this.render(userId, templateId, values).map((m) => new Delivery(`D${this.nextId++}`, m, now)); this.byKey.set(key, queued); for (const d of queued) this.pending.add(d); return queued; }
// Claim everything due, then send. The claim happens before the first await, so two workers // can never send the same delivery. async process(now: number): Promise<number> { const claimed = [...this.pending].filter((d) => d.status === 'pending' && d.nextAttemptAt <= now); for (const d of claimed) { d.status = 'sending'; d.attempts++; } for (const d of claimed) this.record(d, await this.channels.get(d.message.type)!.send(d.message), now); return claimed.length; }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private record(d: Delivery, outcome: Outcome, now: number): void { if (outcome === 'sent') d.status = 'sent'; else if (outcome === 'failed' || d.attempts >= this.retry.maxAttempts) d.status = 'failed'; else { d.status = 'pending'; d.nextAttemptAt = now + this.retry.delayAfter(d.attempts); } if (d.status !== 'pending') this.pending.delete(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private render(userId: string, templateId: string, values: Record<string, string>): Message[] { const user = this.users.get(userId); const t = this.templates.get(templateId); if (!user || !t) return []; const out: Message[] = []; for (const type of ALL_CHANNELS) { const body = t.bodies[type]; const to = user.addresses[type]; if (body === undefined || to === undefined || !this.channels.has(type)) continue; out.push({ type, to, subject: render(t.subject, values), body: render(body, values) }); } return out; }}
// The ways we can reach a user. Adding one is a new value here and a new Channel class.type ChannelType = 'email' | 'sms' | 'push';
// A fixed order to walk channels in, so results are predictable.const ALL_CHANNELS: readonly ChannelType[] = ['email', 'sms', 'push'];
// One message on one channel: where it goes and what it says. SMS ignores the subject.type Message = { type: ChannelType; to: string; subject: string; body: string };
// Sending a message goes over the network, so in this stage a channel's send is asynchronous.interface Channel { readonly type: ChannelType; send(m: Message): Promise<Outcome>;}
// A user's address on each channel they've given us: an email address, a phone number, a device// token. No address on a channel means nothing is sent there.type User = { id: string; addresses: Partial<Record<ChannelType, string>> };
// One kind of notification, such as "order shipped", written once per channel. An SMS is short;// an email can say more. A channel with no text here isn't used for this notification.type Template = { id: string; subject: string; bodies: Partial<Record<ChannelType, string>> };
// Thrown when a template names a value the caller didn't give. A bug in the caller, so it is loud.class MissingValueError extends Error { constructor(readonly name: string) { super(`no value for {{${name}}}`); }}
// Fills in every {{name}}. A placeholder with no value throws: a customer should never get// "Hi {{name}}, your order {{order}} has shipped".function render(text: string, values: Record<string, string>): string { return text.replace(/\{\{(\w+)\}\}/g, (_, name: string) => { if (!(name in values)) throw new MissingValueError(name); return values[name]; });}
// Every template, by id, and the code that fills them in.class Templates { private readonly byId = new Map<string, Template>();
add(t: Template): void { this.byId.set(t.id, t); }
get(id: string): Template | undefined { return this.byId.get(id); }}
// What a provider said. 'retry' is a temporary problem, such as a timeout or a rate limit, worth// trying again. 'failed' is permanent, such as an invalid phone number.type Outcome = 'sent' | 'retry' | 'failed';
// 'sending' marks a delivery a worker has claimed, so no other worker sends it too.type Status = 'pending' | 'sending' | 'sent' | 'failed';
// One message's journey: queued, tried one or more times, then sent or given up on.class Delivery { status: Status = 'pending'; attempts = 0; // tries so far, for the retry limit and the back-off delay
constructor( readonly id: string, readonly message: Message, public nextAttemptAt: number, // not sent again before this time ) {}}
// Waits twice as long after each failed attempt: 1s, 2s, 4s. A provider that's struggling gets// room to recover instead of a flood of retries from every sender at once.class RetryPolicy { constructor( readonly maxAttempts: number, readonly baseDelay: number, ) {}
delayAfter(attempts: number): number { return this.baseDelay * 2 ** (attempts - 1); }}Read onlyTypeScript, strictEvery file builds in CI13 files162 lines
Three ideas hold it together:
- Accept, then send. Accepting a notification is fast, and never fails because of a provider. Sending happens later, as often as it takes. Callers see one method, not templates and queues (Facade).
- The service knows no providers. Each channel wraps one (Adapter), behind one interface (Strategy). A new channel or a new provider changes nothing else. How long to wait between retries is a swappable policy too.
- Every delivery has one owner at a time. The queue owns it until a worker claims it, and the worker owns it until it records the outcome.
Stress it
Section titled “Stress it”So far, one caller and one worker. Now many services call at once, and four workers send.
The same text, twice
Section titled “The same text, twice”The order service timed out calling you, and retried. And you now run four workers. Walk me through every way a user could get the same text twice.
What they are testing
Whether you find both duplicates: a caller repeating a request, and two workers sending the same delivery. Then whether the fix for the second keeps the slow send outside the lock, and whether you’re honest about the one duplicate no design here removes.
There are two different duplicates, with two different fixes.
A caller retries. The order service calls notifyUser, the response is lost or slow, and the
order service, reasonably, tries again. Without anything to recognise it, the second call queues a
second set of messages.
The fix is an idempotency key. The caller names each notification, such as shipped:order-42.
A second request with the same key returns the deliveries from the first, and queues nothing.
Checking whether the key was seen and recording it must be one step, under the lock. Otherwise
two retries arriving together both find the key missing, and both queue.
Two workers take the same delivery. Worker A and worker B both read the queue, both see delivery D7 due, and both send it.
The clearly correct fix is to claim. Under the lock, a worker marks each due delivery
SENDING and takes it. Worker B, looking after that, sees SENDING and skips it.
The better fix sends outside the lock. A provider call can take seconds. Holding the lock through
it would stop every caller and every other worker for that time. So process holds the lock only to
claim, and, after each send, to record the outcome.
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Entity
- Value object
- Enum
- Interface
- Implementation
- New in this step
Scroll or pinch to zoom, drag to pan
- Color key
- Facade
- Background worker
| Option | How | Correct? | Cost |
|---|---|---|---|
| 1. Claim under a lock, send after (our choice) | A SENDING status set under a lock |
Yes | One lock for the queue in one process |
| 2. Hold the lock while sending | Simplest | Yes | One slow provider stalls everything |
| 3. One worker only | No sharing at all | Yes | The speed of one sender |
| 4. A queue that hands out messages | A broker gives each message to one consumer | Yes | The usual shape across many machines |
| 5. Check a “sent” flag before send | Read, then send, then set | No | Two workers both read “not sent” |
Go deeperFollowing a retried call and two workers2 min
Let’s put both duplicates through the design.
- The order service calls
notifyUser("shipped:order-42", ...). Under the lock, the service sees the key is new, records it, and queues deliveries D7 (email) and D8 (SMS). The lock is released, and the response heads back. - The response is lost. The order service times out and calls again with the same key. Under the lock, the key is found. The service returns D7 and D8 again, and queues nothing.
- Workers A and B wake up at the same moment. A takes the lock first, finds D7 and D8 due,
marks both
SENDING, and releases the lock. B takes the lock, finds nothing due (both areSENDING), and goes back to waiting. - A sends D7 and D8, with no lock held. The SMS provider is slow and takes two seconds. During those two seconds, other callers queue new notifications, and B claims and sends them. Nobody waits for the slow SMS.
- A takes the lock again to record the outcomes: D7 sent, D8 sent.
Ana gets one email and one text, even though the order service asked twice and two workers were looking at the same queue.
// SENDING marks a delivery a worker has claimed, so no other worker sends it too.enum Status { PENDING, SENDING, SENT, FAILED }
// Many services call notifyUser at once, and several workers call process at once. One lock// guards the queue and the delivery records. Sending happens outside it: a provider can take// seconds, and holding the lock that long would stop every other notification.class NotificationService { private final Map<ChannelType, Channel> channels = new EnumMap<>(ChannelType.class); private final Map<String, User> users = new ConcurrentHashMap<>(); private final Templates templates; private final RetryPolicy retry; private final Object lock = new Object(); private final List<Delivery> pending = new ArrayList<>(); // waiting for a worker private final Map<String, List<Delivery>> byKey = new HashMap<>(); // caller's key -> its deliveries private int nextId = 1;
NotificationService(List<Channel> channels, Templates templates, RetryPolicy retry) { for (Channel c : channels) this.channels.put(c.type(), c); this.templates = templates; this.retry = retry; }
void addUser(User u) { users.put(u.id(), u); }
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. Checking and recording the key is one step under the lock, or two retries // racing would both get through. List<Delivery> notifyUser(String key, String userId, String templateId, Map<String, String> values, long now) { List<Message> messages = render(userId, templateId, values); synchronized (lock) { List<Delivery> existing = byKey.get(key); if (existing != null) return existing; List<Delivery> queued = new ArrayList<>(); for (Message m : messages) queued.add(new Delivery("D" + nextId++, m, now)); byKey.put(key, queued); pending.addAll(queued); return queued; } }
// Claim under the lock, send outside it, record under it again. Two workers can never claim the // same delivery, because claiming changes its status before the lock is released. int process(long now) { List<Delivery> claimed = new ArrayList<>(); synchronized (lock) { for (Delivery d : pending) { if (d.status == Status.PENDING && d.nextAttemptAt <= now) { d.status = Status.SENDING; d.attempts++; claimed.add(d); } } } for (Delivery d : claimed) { Outcome outcome = channels.get(d.message.type()).send(d.message); synchronized (lock) { record(d, outcome, now); } } return claimed.size(); }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private void record(Delivery d, Outcome outcome, long now) { if (outcome == Outcome.SENT) d.status = Status.SENT; else if (outcome == Outcome.FAILED || d.attempts >= retry.maxAttempts()) d.status = Status.FAILED; else { d.status = Status.PENDING; d.nextAttemptAt = now + retry.delayAfter(d.attempts); } if (d.status != Status.PENDING) pending.remove(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private List<Message> render(String userId, String templateId, Map<String, String> values) { User user = users.get(userId); Template t = templates.get(templateId); List<Message> out = new ArrayList<>(); if (user == null || t == null) return out; for (ChannelType type : ChannelType.values()) { // a fixed order, so results are predictable String body = t.bodies().get(type); String to = user.addresses().get(type); if (body == null || to == null || !channels.containsKey(type)) continue; out.add(new Message(type, to, Templates.render(t.subject(), values), Templates.render(body, values))); } return out; }}class Status(Enum): PENDING = auto() SENDING = auto() # a worker has claimed it, so no other worker sends it too SENT = auto() FAILED = auto()
# One message's journey: queued, tried one or more times, then sent or given up on.@dataclass(eq=False)class Delivery: id: str message: Message next_attempt_at: int # not sent again before this time status: Status = Status.PENDING attempts: int = 0 # tries so far, for the retry limit and the back-off delay
# Many services call notify_user at once, and several workers call process at once. One# lock guards the queue and the delivery records. Sending happens outside it: a provider can# take seconds, and holding the lock that long would stop every other notification.class NotificationService: def __init__(self, channels: list[Channel], templates: Templates, retry: RetryPolicy): self._channels = {c.type: c for c in channels} self._users: dict[str, User] = {} self._templates = templates self._retry = retry self._lock = threading.Lock() self._pending: list[Delivery] = [] # waiting for a worker self._by_key: dict[str, list[Delivery]] = {} # caller's key -> its deliveries self._next_id = itertools.count(1)
def add_user(self, u: User) -> None: self._users[u.id] = u
def notify_user(self, key: str, user_id: str, template_id: str, values: dict[str, str], now: int) -> list[Delivery]: # The caller passes a key that names this notification, such as "shipped:order-42". A # caller that times out and retries sends the same key, and gets the same deliveries back # instead of a second email. Checking and recording the key is one step under the lock, # or two retries racing would both get through. messages = self._render(user_id, template_id, values) with self._lock: if key in self._by_key: return self._by_key[key] queued = [Delivery(f"D{next(self._next_id)}", m, now) for m in messages] self._by_key[key] = queued self._pending.extend(queued) return queued
def process(self, now: int) -> int: # Claim under the lock, send outside it, record under it again. Two workers can never # claim the same delivery, because claiming changes its status before the lock is # released. with self._lock: claimed = [d for d in self._pending if d.status is Status.PENDING and d.next_attempt_at <= now] for d in claimed: d.status = Status.SENDING d.attempts += 1 for d in claimed: outcome = self._channels[d.message.type].send(d.message) with self._lock: self._record(d, outcome, now) return len(claimed)
# Sent and permanent failures are final. A temporary failure waits longer each time, until # the retry limit makes it final too. def _record(self, d: Delivery, outcome: Outcome, now: int) -> None: if outcome is Outcome.SENT: d.status = Status.SENT elif outcome is Outcome.FAILED or d.attempts >= self._retry.max_attempts: d.status = Status.FAILED else: d.status = Status.PENDING d.next_attempt_at = now + self._retry.delay_after(d.attempts) if d.status is not Status.PENDING: self._pending.remove(d)
# One message per channel where the user has an address and the template has text. An # unknown user or template sends nothing. def _render(self, user_id: str, template_id: str, values: dict[str, str]) -> list[Message]: user = self._users.get(user_id) t = self._templates.get(template_id) if user is None or t is None: return [] out = [] for type in ChannelType: body = t.bodies.get(type) to = user.addresses.get(type) if body is None or to is None or type not in self._channels: continue out.append(Message(type, to, render(t.subject, values), render(body, values))) return out// Sending marks a delivery a worker has claimed, so no other worker sends it too.public enum Status { Pending, Sending, Sent, Failed }
// Many services call NotifyUser at once, and several workers call Process at once. One lock// guards the queue and the delivery records. Sending happens outside it: a provider can take// seconds, and holding the lock that long would stop every other notification.public sealed class NotificationService{ private readonly Dictionary<ChannelType, IChannel> _channels; private readonly ConcurrentDictionary<string, User> _users = new(); private readonly Templates _templates; private readonly RetryPolicy _retry; private readonly object _lock = new(); private readonly List<Delivery> _pending = new(); // waiting for a worker private readonly Dictionary<string, List<Delivery>> _byKey = new(); // caller's key -> its deliveries private int _nextId = 1;
public NotificationService(IEnumerable<IChannel> channels, Templates templates, RetryPolicy retry) => (_channels, _templates, _retry) = (channels.ToDictionary(c => c.Type), templates, retry);
public void AddUser(User u) => _users[u.Id] = u;
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. Checking and recording the key is one step under the lock, or two retries // racing would both get through. public List<Delivery> NotifyUser(string key, string userId, string templateId, IReadOnlyDictionary<string, string> values, long now) { var messages = Render(userId, templateId, values); lock (_lock) { if (_byKey.TryGetValue(key, out var existing)) return existing; var queued = messages.Select(m => new Delivery($"D{_nextId++}", m, now)).ToList(); _byKey[key] = queued; _pending.AddRange(queued); return queued; } }
// Claim under the lock, send outside it, record under it again. Two workers can never claim // the same delivery, because claiming changes its status before the lock is released. public int Process(long now) { List<Delivery> claimed; lock (_lock) { claimed = _pending.Where(d => d.Status == Status.Pending && d.NextAttemptAt <= now).ToList(); foreach (var d in claimed) { d.Status = Status.Sending; d.Attempts++; } } foreach (var d in claimed) { var outcome = _channels[d.Message.Type].Send(d.Message); lock (_lock) Record(d, outcome, now); } return claimed.Count; }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private void Record(Delivery d, Outcome outcome, long now) { if (outcome == Outcome.Sent) d.Status = Status.Sent; else if (outcome == Outcome.Failed || d.Attempts >= _retry.MaxAttempts) d.Status = Status.Failed; else { d.Status = Status.Pending; d.NextAttemptAt = now + _retry.DelayAfter(d.Attempts); } if (d.Status != Status.Pending) _pending.Remove(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private List<Message> Render(string userId, string templateId, IReadOnlyDictionary<string, string> values) { var messages = new List<Message>(); if (!_users.TryGetValue(userId, out var user) || _templates.Get(templateId) is not { } t) return messages; foreach (var type in Enum.GetValues<ChannelType>()) { if (!t.Bodies.TryGetValue(type, out var body) || !user.Addresses.TryGetValue(type, out var to) || !_channels.ContainsKey(type)) continue; messages.Add(new Message(type, to, Templates.Render(t.Subject, values), Templates.Render(body, values))); } return messages; }}type Status int
const ( StatusPending Status = iota StatusSending // a worker has claimed it, so no other worker sends it too StatusSent StatusFailed)
// NotificationService is called by many services at once, and several workers call Process at// once. One mutex guards the queue and the delivery records. Sending happens outside it: a// provider can take seconds, and holding the mutex that long would stop every other notification.type NotificationService struct { channels map[ChannelType]Channel users map[string]User templates *Templates retry RetryPolicy mu sync.Mutex pending []*Delivery // waiting for a worker byKey map[string][]*Delivery // caller's key -> its deliveries nextID int}
func NewNotificationService(templates *Templates, retry RetryPolicy, channels ...Channel) *NotificationService { s := &NotificationService{channels: map[ChannelType]Channel{}, users: map[string]User{}, templates: templates, retry: retry, byKey: map[string][]*Delivery{}} for _, c := range channels { s.channels[c.Type()] = c } return s}
// AddUser is called while setting up, before any notifications are sent.func (s *NotificationService) AddUser(u User) { s.users[u.ID] = u }
// NotifyUser takes a key that names this notification, such as "shipped:order-42". A caller that// times out and retries sends the same key, and gets the same deliveries back instead of a second// email. Checking and recording the key is one step under the mutex, or two retries racing would// both get through.func (s *NotificationService) NotifyUser(key, userID, templateID string, values map[string]string, now int64) ([]*Delivery, error) { messages, err := s.render(userID, templateID, values) if err != nil { return nil, err } s.mu.Lock() defer s.mu.Unlock() if existing, ok := s.byKey[key]; ok { return existing, nil } var queued []*Delivery for _, m := range messages { s.nextID++ queued = append(queued, &Delivery{ID: fmt.Sprintf("D%d", s.nextID), Message: m, NextAttemptAt: now}) } s.byKey[key] = queued s.pending = append(s.pending, queued...) return queued, nil}
// Process claims under the mutex, sends outside it, and records under it again. Two workers can// never claim the same delivery, because claiming changes its status before the mutex is released.func (s *NotificationService) Process(now int64) int { var claimed []*Delivery s.mu.Lock() for _, d := range s.pending { if d.Status == StatusPending && d.NextAttemptAt <= now { d.Status = StatusSending d.Attempts++ claimed = append(claimed, d) } } s.mu.Unlock() for _, d := range claimed { outcome := s.channels[d.Message.Type].Send(d.Message) s.mu.Lock() s.record(d, outcome, now) s.mu.Unlock() } return len(claimed)}
// StatusOf reads a delivery's status under the mutex, since workers change it.func (s *NotificationService) StatusOf(d *Delivery) Status { s.mu.Lock() defer s.mu.Unlock() return d.Status}
// record: sent and permanent failures are final. A temporary failure waits longer each time, until// the retry limit makes it final too.func (s *NotificationService) record(d *Delivery, outcome Outcome, now int64) { switch { case outcome == Sent: d.Status = StatusSent case outcome == Failed || d.Attempts >= s.retry.MaxAttempts: d.Status = StatusFailed default: d.Status = StatusPending d.NextAttemptAt = now + s.retry.DelayAfter(d.Attempts) } if d.Status != StatusPending { s.pending = slices.DeleteFunc(s.pending, func(x *Delivery) bool { return x == d }) }}
// render makes one message per channel where the user has an address and the template has// text. An unknown user or template sends nothing.func (s *NotificationService) render(userID, templateID string, values map[string]string) ([]Message, error) { user, ok := s.users[userID] tmpl, found := s.templates.Get(templateID) if !ok || !found { return nil, nil } var out []Message for _, t := range AllChannels { body, hasBody := tmpl.Bodies[t] to, hasAddress := user.Addresses[t] if _, hasChannel := s.channels[t]; !hasBody || !hasAddress || !hasChannel { continue } subject, err := Render(tmpl.Subject, values) if err != nil { return nil, err } if body, err = Render(body, values); err != nil { return nil, err } out = append(out, Message{t, to, subject, body}) } return out, nil}// Sending a message goes over the network, so in this stage a channel's send is asynchronous.interface Channel { readonly type: ChannelType; send(m: Message): Promise<Outcome>;}
// 'sending' marks a delivery a worker has claimed, so no other worker sends it too.type Status = 'pending' | 'sending' | 'sent' | 'failed';
// Many services call notifyUser, and several workers run process at the same time. JavaScript// runs one callback at a time, so notifyUser, which has no await, is always one step: checking// and recording the key can't interleave with another call. process is different. It awaits each// send, and while it waits, another worker runs. So a delivery is claimed, marked 'sending',// before the first await, and other workers skip it.class NotificationService { private readonly channels = new Map<ChannelType, Channel>(); private readonly users = new Map<string, User>(); private readonly pending = new Set<Delivery>(); // waiting for a worker private readonly byKey = new Map<string, Delivery[]>(); // caller's key -> its deliveries private nextId = 1;
constructor( channels: Channel[], private readonly templates: Templates, private readonly retry: RetryPolicy, ) { for (const c of channels) this.channels.set(c.type, c); }
addUser(u: User): void { this.users.set(u.id, u); }
// The caller passes a key that names this notification, such as "shipped:order-42". A caller // that times out and retries sends the same key, and gets the same deliveries back instead of // a second email. notifyUser(key: string, userId: string, templateId: string, values: Record<string, string>, now: number): Delivery[] { const existing = this.byKey.get(key); if (existing) return existing; const queued = this.render(userId, templateId, values).map((m) => new Delivery(`D${this.nextId++}`, m, now)); this.byKey.set(key, queued); for (const d of queued) this.pending.add(d); return queued; }
// Claim everything due, then send. The claim happens before the first await, so two workers // can never send the same delivery. async process(now: number): Promise<number> { const claimed = [...this.pending].filter((d) => d.status === 'pending' && d.nextAttemptAt <= now); for (const d of claimed) { d.status = 'sending'; d.attempts++; } for (const d of claimed) this.record(d, await this.channels.get(d.message.type)!.send(d.message), now); return claimed.length; }
// Sent and permanent failures are final. A temporary failure waits longer each time, until // the retry limit makes it final too. private record(d: Delivery, outcome: Outcome, now: number): void { if (outcome === 'sent') d.status = 'sent'; else if (outcome === 'failed' || d.attempts >= this.retry.maxAttempts) d.status = 'failed'; else { d.status = 'pending'; d.nextAttemptAt = now + this.retry.delayAfter(d.attempts); } if (d.status !== 'pending') this.pending.delete(d); }
// One message per channel where the user has an address and the template has text. An // unknown user or template sends nothing. private render(userId: string, templateId: string, values: Record<string, string>): Message[] { const user = this.users.get(userId); const t = this.templates.get(templateId); if (!user || !t) return []; const out: Message[] = []; for (const type of ALL_CHANNELS) { const body = t.bodies[type]; const to = user.addresses[type]; if (body === undefined || to === undefined || !this.channels.has(type)) continue; out.push({ type, to, subject: render(t.subject, values), body: render(body, values) }); } return out; }}Why is this correct? A delivery moves from pending to SENDING inside the lock, so only one
worker can make that move. Every other worker sees SENDING, and skips it. And a key is checked and
recorded in one step, so only one of two matching requests can queue.
What does it cost? A short lock, held only to claim and to record. And there’s one duplicate this can’t remove. If the provider sends the SMS but its response is lost, the worker sees a timeout, retries, and the user gets two. The service can’t know the first one went out. The fix lives at the provider: many accept an idempotency key of their own, and the delivery id is the natural one to pass. That makes the promise “at least once, and usually exactly once”. Say so plainly.
Go deeperHow each language does it1 min
In Java, Python, C# and Go, callers and workers run on their own threads, so the lock is needed.
TypeScript runs one callback at a time: notifyUser has no await, so checking the key can’t
interleave with another call. But process waits on each send, and another worker runs while it
waits, so the claim still matters. TypeScript marks deliveries SENDING before the first await.
How you’d prove it works
Section titled “How you’d prove it works”The tests use fake channels that record what they were given, and answer with a scripted list of
outcomes. They pass now in, so an hour of retries takes a few calls and a millisecond. The tests
that matter:
- A notification goes to each channel the user has an address for, with every placeholder filled.
- A channel the template doesn’t cover isn’t used.
- A missing value sends nothing on any channel.
- An unknown user or template sends nothing.
notifyUseronly queues;processsends.- A temporary failure is retried after one second, not before.
- It gives up after three attempts, having waited one second and then two.
- A permanent failure is not retried.
- The same key twice returns the same deliveries, and sends once.
- Eight callers send the same 50 keys at once, and four workers process the queue with a slow provider. Each of the 50 messages is sent exactly once per channel.
Go deeperThe tests in five languages2 min
The last test fails in Java, Python, Go and TypeScript if workers don’t claim deliveries, and in Java and Go if the key isn’t checked under the lock.
// A channel for tests: it records what it sent, and answers with scripted outcomes, then SENT.static class FakeChannel implements Channel { final ChannelType type; final List<Message> sent = Collections.synchronizedList(new ArrayList<>()); final Deque<Outcome> script = new ArrayDeque<>(); boolean slow; // a real provider takes time, which is when two workers can collide
FakeChannel(ChannelType type, Outcome... outcomes) { this.type = type; script.addAll(List.of(outcomes)); }
public ChannelType type() { return type; }
public Outcome send(Message m) { if (slow) { try { Thread.sleep(1); } catch (InterruptedException e) { throw new RuntimeException(e); } } sent.add(m); synchronized (script) { return script.isEmpty() ? Outcome.SENT : script.poll(); } }}
FakeChannel email = new FakeChannel(ChannelType.EMAIL);FakeChannel sms = new FakeChannel(ChannelType.SMS);FakeChannel push = new FakeChannel(ChannelType.PUSH);Templates templates = new Templates();
{ templates.add(new Template("shipped", "Order {{order}} shipped", Map.of( ChannelType.EMAIL, "Hi {{name}}, order {{order}} is on its way.", ChannelType.SMS, "Order {{order}} shipped", ChannelType.PUSH, "Order {{order}} shipped"))); templates.add(new Template("receipt", "Your receipt", Map.of(ChannelType.EMAIL, "Thanks, {{name}}.")));}
static final Map<String, String> VALUES = Map.of("name", "Ana", "order", "42");
NotificationService service(FakeChannel... channels) { NotificationService s = new NotificationService(List.of(channels), templates, new RetryPolicy(3, 1000)); s.addUser(new User("ana", Map.of(ChannelType.EMAIL, "ana@example.com", ChannelType.SMS, "+15550100"))); return s;}
@Testvoid sendsOnEachChannelTheUserHasAnAddressFor() { NotificationService s = service(email, sms, push); assertEquals(2, s.notifyUser("k1", "ana", "shipped", VALUES, 0).size()); // no push token s.process(0); assertEquals(new Message(ChannelType.EMAIL, "ana@example.com", "Order 42 shipped", "Hi Ana, order 42 is on its way."), email.sent.get(0)); assertEquals("Order 42 shipped", sms.sent.get(0).body()); assertEquals(0, push.sent.size());}
@Testvoid aChannelTheTemplateDoesntCoverIsNotUsed() { NotificationService s = service(email, sms); s.notifyUser("k1", "ana", "receipt", VALUES, 0); s.process(0); assertEquals(1, email.sent.size()); assertEquals(0, sms.sent.size());}
@Testvoid aMissingValueSendsNothing() { NotificationService s = service(email, sms); assertThrows(MissingValueException.class, () -> s.notifyUser("k1", "ana", "shipped", Map.of("name", "Ana"), 0)); assertEquals(0, s.process(0)); assertEquals(0, email.sent.size());}
@Testvoid anUnknownUserOrTemplateSendsNothing() { NotificationService s = service(email); assertEquals(0, s.notifyUser("k1", "bob", "shipped", VALUES, 0).size()); assertEquals(0, s.notifyUser("k2", "ana", "nope", VALUES, 0).size());}
@Testvoid notifyQueuesAndProcessSends() { NotificationService s = service(email); Delivery d = s.notifyUser("k1", "ana", "shipped", VALUES, 0).get(0); assertEquals(Status.PENDING, d.status); assertEquals(0, email.sent.size()); assertEquals(1, s.process(0)); assertEquals(Status.SENT, d.status);}
@Testvoid aTemporaryFailureIsRetriedAfterABackoff() { FakeChannel flaky = new FakeChannel(ChannelType.EMAIL, Outcome.RETRY); NotificationService s = service(flaky); Delivery d = s.notifyUser("k1", "ana", "receipt", VALUES, 0).get(0); s.process(0); assertEquals(0, s.process(999)); // not due yet assertEquals(1, s.process(1000)); assertEquals(Status.SENT, d.status); assertEquals(2, d.attempts);}
@Testvoid itGivesUpAfterTheLastAttempt() { FakeChannel down = new FakeChannel(ChannelType.EMAIL, Outcome.RETRY, Outcome.RETRY, Outcome.RETRY, Outcome.RETRY); NotificationService s = service(down); Delivery d = s.notifyUser("k1", "ana", "receipt", VALUES, 0).get(0); s.process(0); s.process(1000); // waited 1s assertEquals(0, s.process(2999)); s.process(3000); // waited 2s more assertEquals(Status.FAILED, d.status); assertEquals(3, d.attempts); assertEquals(0, s.process(100_000));}
@Testvoid aPermanentFailureIsNotRetried() { FakeChannel bad = new FakeChannel(ChannelType.EMAIL, Outcome.FAILED); NotificationService s = service(bad); Delivery d = s.notifyUser("k1", "ana", "receipt", VALUES, 0).get(0); s.process(0); assertEquals(Status.FAILED, d.status); assertEquals(1, d.attempts);}
@Testvoid theSameKeyNotifiesOnce() { NotificationService s = service(email); List<Delivery> first = s.notifyUser("shipped:42", "ana", "receipt", VALUES, 0); assertSame(first, s.notifyUser("shipped:42", "ana", "receipt", VALUES, 0)); s.process(0); assertEquals(1, email.sent.size());}
@Testvoid concurrentCallersAndWorkersSendEachMessageExactlyOnce() throws Exception { email.slow = sms.slow = true; NotificationService s = service(email, sms); List<Thread> callers = new ArrayList<>(); for (int t = 0; t < 8; t++) { callers.add(Thread.ofPlatform().start(() -> { for (int i = 0; i < 50; i++) s.notifyUser("k" + i, "ana", "shipped", Map.of("name", "Ana", "order", "" + i), 0); })); } for (Thread t : callers) t.join(); List<Thread> workers = new ArrayList<>(); for (int w = 0; w < 4; w++) workers.add(Thread.ofPlatform().start(() -> { while (s.process(0) > 0) { } })); for (Thread t : workers) t.join(); assertEquals(50, email.sent.size()); assertEquals(50, sms.sent.size()); assertEquals(50, new HashSet<>(sms.sent).size());}import sys
# A channel for tests: it records what it sent, and answers with scripted outcomes, then# SENT.class FakeChannel: def __init__(self, type: ChannelType, *outcomes: Outcome, slow: bool = False): self.type = type self.sent: list[Message] = [] self._script = deque(outcomes) self._slow = slow # a real provider takes time, which is when two workers can collide
def send(self, m: Message) -> Outcome: if self._slow: time.sleep(0.001) self.sent.append(m) return self._script.popleft() if self._script else Outcome.SENT
VALUES = {"name": "Ana", "order": "42"}
def service(*channels: FakeChannel) -> NotificationService: templates = Templates() templates.add( Template( "shipped", "Order {{order}} shipped", { ChannelType.EMAIL: "Hi {{name}}, order {{order}} is on its way.", ChannelType.SMS: "Order {{order}} shipped", ChannelType.PUSH: "Order {{order}} shipped", }, ) ) templates.add(Template("receipt", "Your receipt", {ChannelType.EMAIL: "Thanks, {{name}}."})) s = NotificationService(list(channels), templates, RetryPolicy(3, 1000)) s.add_user(User("ana", {ChannelType.EMAIL: "ana@example.com", ChannelType.SMS: "+15550100"})) return s
def test_sends_on_each_channel_the_user_has_an_address_for(): email, sms, push = FakeChannel(ChannelType.EMAIL), FakeChannel(ChannelType.SMS), FakeChannel(ChannelType.PUSH) s = service(email, sms, push) assert len(s.notify_user("k1", "ana", "shipped", VALUES, 0)) == 2 # no push token s.process(0) assert email.sent[0] == Message(ChannelType.EMAIL, "ana@example.com", "Order 42 shipped", "Hi Ana, order 42 is on its way.") assert sms.sent[0].body == "Order 42 shipped" assert push.sent == []
def test_a_channel_the_template_doesnt_cover_is_not_used(): email, sms = FakeChannel(ChannelType.EMAIL), FakeChannel(ChannelType.SMS) s = service(email, sms) s.notify_user("k1", "ana", "receipt", VALUES, 0) s.process(0) assert (len(email.sent), len(sms.sent)) == (1, 0)
def test_a_missing_value_sends_nothing(): email = FakeChannel(ChannelType.EMAIL) s = service(email) try: s.notify_user("k1", "ana", "shipped", {"name": "Ana"}, 0) assert False, "expected MissingValueError" except MissingValueError: pass assert s.process(0) == 0 assert email.sent == []
def test_an_unknown_user_or_template_sends_nothing(): s = service(FakeChannel(ChannelType.EMAIL)) assert s.notify_user("k1", "bob", "shipped", VALUES, 0) == [] assert s.notify_user("k2", "ana", "nope", VALUES, 0) == []
def test_notify_queues_and_process_sends(): email = FakeChannel(ChannelType.EMAIL) s = service(email) d = s.notify_user("k1", "ana", "shipped", VALUES, 0)[0] assert d.status is Status.PENDING and email.sent == [] assert s.process(0) == 1 assert d.status is Status.SENT
def test_a_temporary_failure_is_retried_after_a_backoff(): s = service(FakeChannel(ChannelType.EMAIL, Outcome.RETRY)) d = s.notify_user("k1", "ana", "receipt", VALUES, 0)[0] s.process(0) assert s.process(999) == 0 # not due yet assert s.process(1000) == 1 assert (d.status, d.attempts) == (Status.SENT, 2)
def test_it_gives_up_after_the_last_attempt(): s = service(FakeChannel(ChannelType.EMAIL, *[Outcome.RETRY] * 4)) d = s.notify_user("k1", "ana", "receipt", VALUES, 0)[0] s.process(0) s.process(1000) # waited 1s assert s.process(2999) == 0 s.process(3000) # waited 2s more assert (d.status, d.attempts) == (Status.FAILED, 3) assert s.process(100_000) == 0
def test_a_permanent_failure_is_not_retried(): s = service(FakeChannel(ChannelType.EMAIL, Outcome.FAILED)) d = s.notify_user("k1", "ana", "receipt", VALUES, 0)[0] s.process(0) assert (d.status, d.attempts) == (Status.FAILED, 1)
def test_the_same_key_notifies_once(): email = FakeChannel(ChannelType.EMAIL) s = service(email) first = s.notify_user("shipped:42", "ana", "receipt", VALUES, 0) assert s.notify_user("shipped:42", "ana", "receipt", VALUES, 0) is first s.process(0) assert len(email.sent) == 1
def test_concurrent_callers_and_workers_send_each_message_exactly_once(): old = sys.getswitchinterval() sys.setswitchinterval(1e-6) # switch threads often, so a missing lock shows up try: email, sms = FakeChannel(ChannelType.EMAIL, slow=True), FakeChannel(ChannelType.SMS, slow=True) s = service(email, sms)
def caller() -> None: for i in range(50): s.notify_user(f"k{i}", "ana", "shipped", {"name": "Ana", "order": str(i)}, 0)
def worker() -> None: while s.process(0) > 0: pass
with ThreadPoolExecutor(8) as pool: for f in [pool.submit(caller) for _ in range(8)]: f.result() with ThreadPoolExecutor(4) as pool: for f in [pool.submit(worker) for _ in range(4)]: f.result() assert (len(email.sent), len(sms.sent)) == (50, 50) assert len(set(sms.sent)) == 50 finally: sys.setswitchinterval(old)// A channel for tests: it records what it sent, and answers with scripted outcomes, then Sent.private sealed class FakeChannel : IChannel{ private readonly Queue<Outcome> _script; private readonly bool _slow; // a real provider takes time, which is when two workers can collide
public FakeChannel(ChannelType type, bool slow = false, params Outcome[] outcomes) => (Type, _slow, _script) = (type, slow, new Queue<Outcome>(outcomes));
public ChannelType Type { get; } public ConcurrentQueue<Message> Sent { get; } = new();
public Outcome Send(Message m) { if (_slow) Thread.Sleep(1); Sent.Enqueue(m); lock (_script) return _script.Count == 0 ? Outcome.Sent : _script.Dequeue(); }}
private static readonly Dictionary<string, string> Values = new() { ["name"] = "Ana", ["order"] = "42" };
private static NotificationService Service(params IChannel[] channels){ var templates = new Templates(); templates.Add(new Template("shipped", "Order {{order}} shipped", new Dictionary<ChannelType, string> { [ChannelType.Email] = "Hi {{name}}, order {{order}} is on its way.", [ChannelType.Sms] = "Order {{order}} shipped", [ChannelType.Push] = "Order {{order}} shipped", })); templates.Add(new Template("receipt", "Your receipt", new Dictionary<ChannelType, string> { [ChannelType.Email] = "Thanks, {{name}}." })); var s = new NotificationService(channels, templates, new RetryPolicy(3, 1000)); s.AddUser(new User("ana", new Dictionary<ChannelType, string> { [ChannelType.Email] = "ana@example.com", [ChannelType.Sms] = "+15550100" })); return s;}
[Fact]public void SendsOnEachChannelTheUserHasAnAddressFor(){ FakeChannel email = new(ChannelType.Email), sms = new(ChannelType.Sms), push = new(ChannelType.Push); var s = Service(email, sms, push); Assert.Equal(2, s.NotifyUser("k1", "ana", "shipped", Values, 0).Count); // no push token s.Process(0); Assert.Equal(new Message(ChannelType.Email, "ana@example.com", "Order 42 shipped", "Hi Ana, order 42 is on its way."), email.Sent.First()); Assert.Equal("Order 42 shipped", sms.Sent.First().Body); Assert.Empty(push.Sent);}
[Fact]public void AChannelTheTemplateDoesntCoverIsNotUsed(){ FakeChannel email = new(ChannelType.Email), sms = new(ChannelType.Sms); var s = Service(email, sms); s.NotifyUser("k1", "ana", "receipt", Values, 0); s.Process(0); Assert.Single(email.Sent); Assert.Empty(sms.Sent);}
[Fact]public void AMissingValueSendsNothing(){ var email = new FakeChannel(ChannelType.Email); var s = Service(email); Assert.Throws<MissingValueException>(() => s.NotifyUser("k1", "ana", "shipped", new Dictionary<string, string> { ["name"] = "Ana" }, 0)); Assert.Equal(0, s.Process(0)); Assert.Empty(email.Sent);}
[Fact]public void AnUnknownUserOrTemplateSendsNothing(){ var s = Service(new FakeChannel(ChannelType.Email)); Assert.Empty(s.NotifyUser("k1", "bob", "shipped", Values, 0)); Assert.Empty(s.NotifyUser("k2", "ana", "nope", Values, 0));}
[Fact]public void NotifyQueuesAndProcessSends(){ var email = new FakeChannel(ChannelType.Email); var s = Service(email); var d = s.NotifyUser("k1", "ana", "shipped", Values, 0)[0]; Assert.Equal(Status.Pending, d.Status); Assert.Empty(email.Sent); Assert.Equal(1, s.Process(0)); Assert.Equal(Status.Sent, d.Status);}
[Fact]public void ATemporaryFailureIsRetriedAfterABackoff(){ var s = Service(new FakeChannel(ChannelType.Email, false, Outcome.Retry)); var d = s.NotifyUser("k1", "ana", "receipt", Values, 0)[0]; s.Process(0); Assert.Equal(0, s.Process(999)); // not due yet Assert.Equal(1, s.Process(1000)); Assert.Equal(Status.Sent, d.Status); Assert.Equal(2, d.Attempts);}
[Fact]public void ItGivesUpAfterTheLastAttempt(){ var s = Service(new FakeChannel(ChannelType.Email, false, Outcome.Retry, Outcome.Retry, Outcome.Retry, Outcome.Retry)); var d = s.NotifyUser("k1", "ana", "receipt", Values, 0)[0]; s.Process(0); s.Process(1000); // waited 1s Assert.Equal(0, s.Process(2999)); s.Process(3000); // waited 2s more Assert.Equal(Status.Failed, d.Status); Assert.Equal(3, d.Attempts); Assert.Equal(0, s.Process(100_000));}
[Fact]public void APermanentFailureIsNotRetried(){ var s = Service(new FakeChannel(ChannelType.Email, false, Outcome.Failed)); var d = s.NotifyUser("k1", "ana", "receipt", Values, 0)[0]; s.Process(0); Assert.Equal(Status.Failed, d.Status); Assert.Equal(1, d.Attempts);}
[Fact]public void TheSameKeyNotifiesOnce(){ var email = new FakeChannel(ChannelType.Email); var s = Service(email); var first = s.NotifyUser("shipped:42", "ana", "receipt", Values, 0); Assert.Same(first, s.NotifyUser("shipped:42", "ana", "receipt", Values, 0)); s.Process(0); Assert.Single(email.Sent);}
[Fact]public void ConcurrentCallersAndWorkersSendEachMessageExactlyOnce(){ FakeChannel email = new(ChannelType.Email, true), sms = new(ChannelType.Sms, true); var s = Service(email, sms); Parallel.For(0, 8, _ => { for (var i = 0; i < 50; i++) s.NotifyUser($"k{i}", "ana", "shipped", new Dictionary<string, string> { ["name"] = "Ana", ["order"] = $"{i}" }, 0); }); Parallel.For(0, 4, _ => { while (s.Process(0) > 0) { } }); Assert.Equal(50, email.Sent.Count); Assert.Equal(50, sms.Sent.Count); Assert.Equal(50, sms.Sent.Distinct().Count());}// FakeChannel records what it sent, and answers with scripted outcomes, then Sent.type FakeChannel struct { kind ChannelType slow bool // a real provider takes time, which is when two workers can collide mu sync.Mutex sent []Message script []Outcome}
func (c *FakeChannel) Type() ChannelType { return c.kind }
func (c *FakeChannel) Send(m Message) Outcome { if c.slow { time.Sleep(time.Millisecond) } c.mu.Lock() defer c.mu.Unlock() c.sent = append(c.sent, m) if len(c.script) == 0 { return Sent } out := c.script[0] c.script = c.script[1:] return out}
func (c *FakeChannel) Messages() []Message { c.mu.Lock() defer c.mu.Unlock() return slices.Clone(c.sent)}
var values = map[string]string{"name": "Ana", "order": "42"}
func newService(channels ...Channel) *NotificationService { templates := NewTemplates() templates.Add(Template{"shipped", "Order {{order}} shipped", map[ChannelType]string{ Email: "Hi {{name}}, order {{order}} is on its way.", SMS: "Order {{order}} shipped", Push: "Order {{order}} shipped", }}) templates.Add(Template{"receipt", "Your receipt", map[ChannelType]string{Email: "Thanks, {{name}}."}}) s := NewNotificationService(templates, RetryPolicy{3, 1000}, channels...) s.AddUser(User{"ana", map[ChannelType]string{Email: "ana@example.com", SMS: "+15550100"}}) return s}
func expect[T comparable](t *testing.T, want, got T) { t.Helper() if want != got { t.Fatalf("want %v, got %v", want, got) }}
func notify(t *testing.T, s *NotificationService, key, templateID string) []*Delivery { t.Helper() ds, err := s.NotifyUser(key, "ana", templateID, values, 0) expect(t, nil, err) return ds}
func TestSendsOnEachChannelTheUserHasAnAddressFor(t *testing.T) { email, sms, push := &FakeChannel{kind: Email}, &FakeChannel{kind: SMS}, &FakeChannel{kind: Push} s := newService(email, sms, push) expect(t, 2, len(notify(t, s, "k1", "shipped"))) // no push token s.Process(0) expect(t, Message{Email, "ana@example.com", "Order 42 shipped", "Hi Ana, order 42 is on its way."}, email.Messages()[0]) expect(t, "Order 42 shipped", sms.Messages()[0].Body) expect(t, 0, len(push.Messages()))}
func TestAChannelTheTemplateDoesntCoverIsNotUsed(t *testing.T) { email, sms := &FakeChannel{kind: Email}, &FakeChannel{kind: SMS} s := newService(email, sms) notify(t, s, "k1", "receipt") s.Process(0) expect(t, 1, len(email.Messages())) expect(t, 0, len(sms.Messages()))}
func TestAMissingValueSendsNothing(t *testing.T) { email := &FakeChannel{kind: Email} s := newService(email) _, err := s.NotifyUser("k1", "ana", "shipped", map[string]string{"name": "Ana"}, 0) expect(t, error(MissingValueError{"order"}), err) expect(t, 0, s.Process(0)) expect(t, 0, len(email.Messages()))}
func TestAnUnknownUserOrTemplateSendsNothing(t *testing.T) { s := newService(&FakeChannel{kind: Email}) ds, _ := s.NotifyUser("k1", "bob", "shipped", values, 0) expect(t, 0, len(ds)) expect(t, 0, len(notify(t, s, "k2", "nope")))}
func TestNotifyQueuesAndProcessSends(t *testing.T) { email := &FakeChannel{kind: Email} s := newService(email) d := notify(t, s, "k1", "shipped")[0] expect(t, StatusPending, s.StatusOf(d)) expect(t, 0, len(email.Messages())) expect(t, 1, s.Process(0)) expect(t, StatusSent, s.StatusOf(d))}
func TestATemporaryFailureIsRetriedAfterABackoff(t *testing.T) { s := newService(&FakeChannel{kind: Email, script: []Outcome{Retry}}) d := notify(t, s, "k1", "receipt")[0] s.Process(0) expect(t, 0, s.Process(999)) // not due yet expect(t, 1, s.Process(1000)) expect(t, StatusSent, s.StatusOf(d)) expect(t, 2, d.Attempts)}
func TestItGivesUpAfterTheLastAttempt(t *testing.T) { s := newService(&FakeChannel{kind: Email, script: []Outcome{Retry, Retry, Retry, Retry}}) d := notify(t, s, "k1", "receipt")[0] s.Process(0) s.Process(1000) // waited 1s expect(t, 0, s.Process(2999)) s.Process(3000) // waited 2s more expect(t, StatusFailed, s.StatusOf(d)) expect(t, 3, d.Attempts) expect(t, 0, s.Process(100_000))}
func TestAPermanentFailureIsNotRetried(t *testing.T) { s := newService(&FakeChannel{kind: Email, script: []Outcome{Failed}}) d := notify(t, s, "k1", "receipt")[0] s.Process(0) expect(t, StatusFailed, s.StatusOf(d)) expect(t, 1, d.Attempts)}
func TestTheSameKeyNotifiesOnce(t *testing.T) { email := &FakeChannel{kind: Email} s := newService(email) first := notify(t, s, "shipped:42", "receipt") expect(t, first[0], notify(t, s, "shipped:42", "receipt")[0]) s.Process(0) expect(t, 1, len(email.Messages()))}
func TestConcurrentCallersAndWorkersSendEachMessageExactlyOnce(t *testing.T) { email, sms := &FakeChannel{kind: Email, slow: true}, &FakeChannel{kind: SMS, slow: true} s := newService(email, sms) var wg sync.WaitGroup for range 8 { wg.Add(1) go func() { defer wg.Done() for i := range 50 { s.NotifyUser(fmt.Sprintf("k%d", i), "ana", "shipped", map[string]string{"name": "Ana", "order": fmt.Sprint(i)}, 0) } }() } wg.Wait() for range 4 { wg.Add(1) go func() { defer wg.Done() for s.Process(0) > 0 { } }() } wg.Wait() expect(t, 50, len(email.Messages())) expect(t, 50, len(sms.Messages())) seen := map[Message]bool{} for _, m := range sms.Messages() { seen[m] = true } expect(t, 50, len(seen))}import { test } from 'node:test';import assert from 'node:assert/strict';
// A channel for tests: it records what it sent, and answers with scripted outcomes, then 'sent'.class FakeChannel implements Channel { readonly sent: Message[] = []; private readonly script: Outcome[];
constructor( readonly type: ChannelType, outcomes: Outcome[] = [], private readonly slow = false, // a real provider takes time, which is when two workers can collide ) { this.script = [...outcomes]; }
async send(m: Message): Promise<Outcome> { if (this.slow) await new Promise((r) => setTimeout(r, 1)); this.sent.push(m); return this.script.shift() ?? 'sent'; }}
const VALUES = { name: 'Ana', order: '42' };
function service(...channels: FakeChannel[]): NotificationService { const templates = new Templates(); templates.add({ id: 'shipped', subject: 'Order {{order}} shipped', bodies: { email: 'Hi {{name}}, order {{order}} is on its way.', sms: 'Order {{order}} shipped', push: 'Order {{order}} shipped' }, }); templates.add({ id: 'receipt', subject: 'Your receipt', bodies: { email: 'Thanks, {{name}}.' } }); const s = new NotificationService(channels, templates, new RetryPolicy(3, 1000)); s.addUser({ id: 'ana', addresses: { email: 'ana@example.com', sms: '+15550100' } }); return s;}
test('sends on each channel the user has an address for', async () => { const [email, sms, push] = [new FakeChannel('email'), new FakeChannel('sms'), new FakeChannel('push')]; const s = service(email, sms, push); assert.equal(s.notifyUser('k1', 'ana', 'shipped', VALUES, 0).length, 2); // no push token await s.process(0); assert.deepEqual(email.sent[0], { type: 'email', to: 'ana@example.com', subject: 'Order 42 shipped', body: 'Hi Ana, order 42 is on its way.' }); assert.equal(sms.sent[0].body, 'Order 42 shipped'); assert.equal(push.sent.length, 0);});
test("a channel the template doesn't cover is not used", async () => { const [email, sms] = [new FakeChannel('email'), new FakeChannel('sms')]; const s = service(email, sms); s.notifyUser('k1', 'ana', 'receipt', VALUES, 0); await s.process(0); assert.deepEqual([email.sent.length, sms.sent.length], [1, 0]);});
test('a missing value sends nothing', async () => { const email = new FakeChannel('email'); const s = service(email); assert.throws(() => s.notifyUser('k1', 'ana', 'shipped', { name: 'Ana' }, 0), MissingValueError); assert.equal(await s.process(0), 0); assert.equal(email.sent.length, 0);});
test('an unknown user or template sends nothing', () => { const s = service(new FakeChannel('email')); assert.deepEqual(s.notifyUser('k1', 'bob', 'shipped', VALUES, 0), []); assert.deepEqual(s.notifyUser('k2', 'ana', 'nope', VALUES, 0), []);});
test('notify queues and process sends', async () => { const email = new FakeChannel('email'); const s = service(email); const [d] = s.notifyUser('k1', 'ana', 'shipped', VALUES, 0); assert.equal(d.status, 'pending'); assert.equal(email.sent.length, 0); assert.equal(await s.process(0), 1); assert.equal(d.status, 'sent');});
test('a temporary failure is retried after a backoff', async () => { const s = service(new FakeChannel('email', ['retry'])); const [d] = s.notifyUser('k1', 'ana', 'receipt', VALUES, 0); await s.process(0); assert.equal(await s.process(999), 0); // not due yet assert.equal(await s.process(1000), 1); assert.deepEqual([d.status, d.attempts], ['sent', 2]);});
test('it gives up after the last attempt', async () => { const s = service(new FakeChannel('email', ['retry', 'retry', 'retry', 'retry'])); const [d] = s.notifyUser('k1', 'ana', 'receipt', VALUES, 0); await s.process(0); await s.process(1000); // waited 1s assert.equal(await s.process(2999), 0); await s.process(3000); // waited 2s more assert.deepEqual([d.status, d.attempts], ['failed', 3]); assert.equal(await s.process(100_000), 0);});
test('a permanent failure is not retried', async () => { const s = service(new FakeChannel('email', ['failed'])); const [d] = s.notifyUser('k1', 'ana', 'receipt', VALUES, 0); await s.process(0); assert.deepEqual([d.status, d.attempts], ['failed', 1]);});
test('the same key notifies once', async () => { const email = new FakeChannel('email'); const s = service(email); const first = s.notifyUser('shipped:42', 'ana', 'receipt', VALUES, 0); assert.equal(s.notifyUser('shipped:42', 'ana', 'receipt', VALUES, 0), first); await s.process(0); assert.equal(email.sent.length, 1);});
test('concurrent workers send each message exactly once', async () => { const [email, sms] = [new FakeChannel('email', [], true), new FakeChannel('sms', [], true)]; const s = service(email, sms); for (let caller = 0; caller < 8; caller++) { for (let i = 0; i < 50; i++) s.notifyUser(`k${i}`, 'ana', 'shipped', { name: 'Ana', order: String(i) }, 0); } const worker = async () => { while ((await s.process(0)) > 0) {} }; await Promise.all([worker(), worker(), worker(), worker()]); assert.deepEqual([email.sent.length, sms.sent.length], [50, 50]); assert.equal(new Set(sms.sent.map((m) => m.body)).size, 50);});Go deeperEvery edge case, and where it is handled2 min
| Case | What happens | Where |
|---|---|---|
| Unknown user or template | Nothing queued | render |
| User has no address on a channel | That channel is skipped | render |
| Template has no body for a channel | That channel is skipped | render |
| A placeholder with no value | An error; nothing queued on any channel | Templates.render |
| Provider times out | Retried after 1 s, 2 s, 4 s | RetryPolicy |
| Invalid phone number | Failed at once, not retried | Outcome.FAILED |
| Still failing after the last attempt | Marked failed | process |
| Caller sends the same key twice | Same deliveries returned, nothing new queued | byKey |
| Two workers at once | Each delivery claimed by one | SENDING |
| Provider sent it, but the response was lost | Retried; may duplicate unless the provider dedupes | the provider’s key |
Wrap up
Section titled “Wrap up”The decisions, and what each one costs
Section titled “The decisions, and what each one costs”| Decision | We chose | We gave up |
|---|---|---|
| Channels | One interface; each class wraps one provider | Calling providers directly |
| Wording | Templates per channel; every value required | Callers writing their own text |
| Sending | Queue now, send later | Knowing at once whether it was delivered |
| Failures | Retry temporary ones with doubling waits, then give up | Guaranteed delivery |
| Repeated calls | An idempotency key, checked under the lock | Nothing: callers must send a key |
| Several workers | Claim under the lock, send outside it | A broker’s scale, until we need it |
Go deeperGenuinely different designs, and when to pick them2 min
A message broker between the halves. At scale, notifyUser publishes to a queue such as Kafka,
SQS or RabbitMQ, one topic per channel, and workers on many machines consume. The broker does the
claiming: it hands each message to one consumer, and delivers it again if that consumer dies. That’s
option 4 from the table, and it’s what real notification systems use.
A third-party notification platform. Services like Twilio, SendGrid or a hosted notification platform already provide templates, preferences, retries and analytics. For a small team, adopting one is often the right call, and the design above becomes a thin adapter to it.
Event-driven notifications. Instead of services calling notifyUser, the notification service
listens for events, such as “order shipped”, and decides itself who to tell. Callers no longer need
to know about notifications at all; see the pub-sub page.
How to spend 45 minutes
Section titled “How to spend 45 minutes”- 0 to 3. Channels; who writes the text; does the caller wait; what failure means.
- 3 to 5. The promises. Name “never twice, never half-filled” as the hard one.
- 5 to 22. One email sender and its problems, the channel interface, templates per channel, then
the queue with retries and backoff. Code
process. - 22 to 24. Walk through the full diagram, and why accepting and sending are separate.
- 24 to 29. The two duplicates, the key, the claim, and the duplicate nobody removes.
- 29 to 45. Preferences and fallback, then your own questions.
If time is short, describe templates in a sentence, and spend the time on retries and duplicates. Those decide this question for most interviewers.
Follow-up questions
Section titled “Follow-up questions”Users want to turn off marketing messages but keep security alerts. And if SMS fails completely, fall back to email. Where do those go?
What they are testing
Whether preferences become a check before queuing, by category, and whether a fallback is a new delivery created when one fails, not a special case inside a channel.
Most extensions fit into one place:
| Extension | What changes | Touches sending? |
|---|---|---|
| Opt-outs by category | A check in render |
No |
| Fallback to another channel | A new delivery when one fails | No |
| Quiet hours | nextAttemptAt starts at the end of the user’s quiet hours |
No |
| Rate limit per user | A counter checked when queuing | No |
| A new channel, such as WhatsApp | A new Channel class |
No |
| Two SMS providers, with failover | A Channel that tries one provider, then the other |
No |
| Scheduled sends | notifyUser takes a send-at time as now |
No |
| A queue that survives restarts | The pending list moves into a database or message broker | Storage only |
Go deeperAdding opt-outs and fallback, step by step2 min
Opt-outs.
- Give each template a category, such as
security,ordersormarketing. - Give each user a set of opted-out categories, per channel. Ana might turn off marketing by SMS, but keep it by email.
- Check in
render. Before building a channel’s message, skip it if the user opted out of that category on that channel. Nothing is queued, so nothing is sent. - Some categories can’t be turned off. A security alert must always reach the user. That’s a property of the category, not of the user, so the check simply ignores opt-outs for it.
Fallback.
- Give the service a fallback order, for example SMS, then email.
- When a delivery is finally marked failed, look up the next channel in the order.
- If the template has a body for it, and the user has an address, render it, and queue a new delivery. It gets its own retries, and its own id.
- The channels never know about each other. The SMS channel just reports FAILED. The decision to try email lives in one place, in the service.
Both changes sit before the queue or after a delivery ends. The worker loop, the claim and the retry policy don’t change.
Go deeperMaking the queue survive a restart, step by step2 min
- Store deliveries in a database table, with the id, the message, the status, the attempt
count, and the next attempt time.
notifyUserinserts rows instead of adding to a list. - Store keys in the same table, with a unique index. A second insert with the same key fails, and the service returns the existing rows. The database’s index replaces our lock for this check, and works across many servers.
- Claim with one update. A worker runs something like
UPDATE deliveries SET status = 'SENDING', claimed_at = now() WHERE id IN (SELECT id ... WHERE status = 'PENDING' AND next_attempt_at <= now() LIMIT 20 FOR UPDATE SKIP LOCKED).SKIP LOCKEDlets several workers claim different rows at the same time, without waiting on each other. - Send, then record, in a separate short update, exactly as now.
- Sweep stuck deliveries. A worker that crashes mid-send leaves rows in
SENDING. A job moves rows that have beenSENDINGfor too long back toPENDING. This is where the lost-response duplicate becomes more likely, so passing the delivery id to the provider as its key matters even more.
The shape of the design doesn’t change: accept, claim, send outside any lock, record. Only where the state lives changes.
Go deeperFour more follow-up questions2 min
Interviewer asksWhat happens if the service restarts with messages in the queue?
In this design, they’re lost, because the queue is in memory. A real service keeps deliveries in a
database table or a broker that saves to disk. A worker that dies mid-send leaves its delivery in
SENDING, so a cleanup job returns deliveries stuck in SENDING for too long to PENDING.
That’s another reason the provider’s idempotency key matters.
Interviewer asksHow do you send one announcement to ten million users?
Not as ten million calls from the caller. A bulk job takes the template and an audience, and queues deliveries in batches, at a rate the providers allow. Providers have send limits, so the workers limit their rate per provider, and the announcement takes as long as it takes.
Interviewer asksWhere do you check that a push token is still valid?
The push provider tells you: sending to an uninstalled app returns an error saying the token is
dead. The push channel reports that as FAILED, and a listener removes the token from the user’s
addresses, so the next notification skips it.
Interviewer asksWhy not let each service send its own emails?
Then every service needs every provider’s passwords, its own retry logic, its own templates and its own duplicate handling. Users get messages that don’t match, and no single place to opt out. Notifications are a shared concern, so they get one owner.
Further reading
Section titled “Further reading”- Adapter pattern, how each channel wraps one provider
- Strategy pattern, choosing a channel and a retry policy
- Design a pub/sub system, notifications driven by events
- Design a food delivery app, whose push listener sends through a service like this one
The designbeyondcode digest, every Saturday
New deep dives, interview tips and site news in one short email. Free. Leave with one click.
We never sell or share your email. We only use it to send this newsletter.