Your country

Tools that support it use your country for local currency, number formats, units and paper size. Your choice is saved only in this browser.

Type a name or a two-letter code. Use the up and down arrow keys to move through the countries, Enter to choose one and Escape to close.

System Design (High-Level Design)  Module 24 – Case studies: messaging, real time and collaboration

Design a chat app like WhatsApp: delivery

Design one-to-one chat delivery end to end: estimates, gateways, per-conversation sequence numbers, receipts, offline sync and push notifications.

  • Intermediate
  • 35 minutes
  • Examples run with Python 3.14.8, Pyodide 314.0.7, Node.js 24.21.0 and quickjs 0.32.0
  • By MySmartCoPilot
Free: 35 minutes

What you will learn

  • Estimate the message rate, connections, bandwidth and storage of a chat service from stated assumptions
  • Design one-to-one delivery with sent, delivered and read states kept as two numbers per member
  • Order messages within a conversation and make retried sends harmless with client ids
  • Update offline devices from a cursor per conversation and wake phones with push notifications

Before you start

What you’ll learn · Before you start

What you will learn

  • Estimate the message rate, connections, bandwidth and storage of a chat service from stated assumptions
  • Design one-to-one delivery with sent, delivered and read states kept as two numbers per member
  • Order messages within a conversation and make retried sends harmless with client ids
  • Update offline devices from a cursor per conversation and wake phones with push notifications

Before you start

On this page

A chat app has one job: put every message on the other person’s phone and laptop within a second when they are online, and the moment they come back when they are not, without losing, repeating or reordering anything. For 200 million people a day that is 10 billion messages, and the design that carries them rests on one rule: store first, then push.

The sending device tags each message with idempotency key: A unique id that a client chooses once for an operation and sends with every attempt of it. The server stores the result under the key, so a repeat gets the stored result instead of a second charge or write. More in “How distributed systems fail”. A message service gives the message the next The next whole number that a conversation's owner gives each message it stores. Every device shows the conversation in that order, and a missing number tells a device that a message exists that it has not received. of its conversation, stores it, and only then tells the sender “sent”. A server that holds many long-lived connections, such as WebSockets, one per device, and pushes messages down them. The rest of the service reaches a device through the gateway that holds its connection. push it to the recipient’s devices that are online, and a phone without a connection gets a notification that wakes the app. Every device keeps a The highest sequence number a device holds for a conversation with nothing missing below it. On reconnecting the device fetches only the messages after it, and it ignores any message at or below it as a repeat. per conversation and fetches whatever lies past it, so a lost push costs a second, never a message. This case study builds that path with numbers and code, and answers the deep dives of a design round.

Not affiliated

Based on public information about WhatsApp. MySmartCoPilot is not affiliated with WhatsApp, and WhatsApp’s real architecture may differ from what this lesson describes.

Six questions fix the scope, and the server keeps no history

Ask before you draw. Each question below comes with the answer this lesson assumes:

  • Who talks to whom? Two people in a conversation. Groups, photos and videos, and end-to-end encryption change the design enough to have their own lesson.
  • How many people? 200 million people a day, each sending 50 messages.
  • On which devices? A phone each, and a laptop or a browser for every second person, all signed in at once.
  • What does the sender see? Sent: the service has the message. Delivered: one of the recipient’s devices has it. Read: the recipient has opened the conversation past it.
  • Does the server keep history? No. It keeps a message until every device in the conversation has it, and for 30 days at most; history lives on the devices. WhatsApp’s privacy policy describes the same contract: delivered messages are deleted from its servers, and an undelivered one is kept, encrypted, for up to 30 days while delivery is retried (WhatsApp).
  • How fast? A message reaches a device that is online within a second for 99 % of messages.

The functional requirements follow. People send text messages, see each one move from sent to delivered to read, receive messages on every device they are signed in on, get a notification when their phone has no open connection, and see the same messages in the same order everywhere, including after days offline. Calls, contacts, search, spam and backups are out of scope. The non-functional requirements are what the design is judged by:

Requirement Target
Acknowledged messages lost none, ever
A message shown twice never
Order one order per conversation, on every device
Sends that succeed 99.95 % in a month
A message to a device that is online within 1 second for 99 % of messages
A device that was away caught up within a few seconds of coming back

Checkpoint

Check: What the service must do, and keep

  1. One question Ravi's phone has been switched off for 12 days while his laptop stays online. Which of his messages does the server still hold?

    Choose one answer.

    Show the answer: One question

    Answer: Every message his phone has not stored yet, because the server keeps a message until all devices have it, for 30 days at most

    The server keeps a message until every device in the conversation has stored it, and 30 days at most. The laptop has the messages, the phone does not, so they stay on the server until the phone syncs or the 30 days pass.

10 billion messages a day need 858 gateways, and retention sets the storage

The sheet below works the numbers out from those assumptions. Every constant has a name, so you can put in your own and run it again:

The chat service's estimate sheet Python · estimate/chat_storage.py
# The estimate sheet of a one-to-one chat service (lesson system-design-hld/design-chat-delivery). Every constant is an
# assumption with a name, so you can put your own numbers in and run it again. Sizes are decimal (1 TB = 10**12 bytes).
DAY = 86_400

DAILY_USERS = 200_000_000
MESSAGES_EACH = 50  # messages a user sends in a day
PEAK = 3  # the busiest minutes of the evening against the daily average
DEVICES_EACH = 1.5  # a phone each, and a laptop or a browser for every second user
ONLINE_AT_PEAK = 0.20  # share of all devices with an open connection at the peak
PER_GATEWAY = 70_000  # connections per gateway: 100,000 it can hold, run at 70 %
MESSAGE_BYTES = 250  # about 100 bytes of text, 150 of ids, sequence number, times and flags
REPLICAS = 3
BUCKET = 1_000  # messages per storage partition

# How long a message waits on the server until every device in its conversation has it: (share, seconds).
WAITS = [(0.95, 30), (0.04, 8 * 3600), (0.01, 10 * DAY)]
OFFLINE_SHARE = 0.5  # messages whose recipient's phone has no open connection, so it needs a push notification
BURST = 4  # messages in a typical burst from one sender, merged into one notification
ANDROID_SHARE = 0.75  # notifications that go to Android phones through Firebase Cloud Messaging
FCM_DEFAULT_PER_MINUTE = 600_000  # the default quota of Firebase Cloud Messaging's HTTP v1 API, per project


def size(n):
    for unit, step in (("PB", 10**15), ("TB", 10**12), ("GB", 10**9), ("MB", 10**6), ("KB", 10**3)):
        if n >= step:
            return f"{n / step:,.1f} {unit}"
    return f"{n:,.0f} B"


per_day = DAILY_USERS * MESSAGES_EACH
average = per_day / DAY
peak = average * PEAK
# Every message goes to the recipient's devices and to the sender's other devices: 1.5 + 0.5 = 2 on average.
deliveries = peak * (DEVICES_EACH + (DEVICES_EACH - 1))
devices = DAILY_USERS * DEVICES_EACH
sockets = devices * ONLINE_AT_PEAK
gateways = -(-sockets // PER_GATEWAY)

print("Chat delivery: the estimate sheet")
print(f"  Messages      {per_day / 10**9:.0f} billion a day: {average:,.0f}/s on average, {peak:,.0f}/s at the peak")
print(f"  Deliveries    {deliveries:,.0f}/s at the peak, to {DEVICES_EACH + DEVICES_EACH - 1:.1f} devices per message")
print(f"  Connections   {sockets / 10**6:.0f} million open at the peak, on {gateways:,.0f} gateways of {PER_GATEWAY:,}")
print(f"  Bandwidth     {peak * MESSAGE_BYTES * 8 / 10**9:.2f} Gbit/s in, {deliveries * MESSAGE_BYTES * 8 / 10**9:.2f} Gbit/s out (text only)")

print()
print("Storage, two ways")
year = per_day * MESSAGE_BYTES * 365
print(f"  Keep every message      {size(per_day * MESSAGE_BYTES)} a day, {size(year)} a year, {size(year * REPLICAS)} with {REPLICAS} replicas")
wait = sum(share * seconds for share, seconds in WAITS)
resident = average * wait  # Little's law: messages on the server = arrival rate x average time there
print(f"  Keep until delivered    an average wait of {wait:,.0f} s, so {resident / 10**9:.2f} billion messages on the server at once")
print(f"                          {size(resident * MESSAGE_BYTES)}, {size(resident * MESSAGE_BYTES * REPLICAS)} with {REPLICAS} replicas")
for share, seconds in WAITS:
    part = share * seconds / wait
    print(f"    {share:5.0%} of messages wait {seconds:>9,} s and make {part:5.1%} of what is stored")
print(f"  One partition           {BUCKET:,} messages x {MESSAGE_BYTES} B = {size(BUCKET * MESSAGE_BYTES)} at most")

print()
print("Push notifications at the peak")
pushes = peak * OFFLINE_SHARE / BURST
fcm = pushes * ANDROID_SHARE * 60
print(f"  {peak * OFFLINE_SHARE:,.0f} messages/s find the recipient's phone without a connection")
print(f"  {pushes:,.0f} notifications/s when a burst of {BURST} messages makes one notification")
print(f"  {fcm / 10**6:.2f} million a minute to Android, {fcm / FCM_DEFAULT_PER_MINUTE:.1f} x the default quota of {FCM_DEFAULT_PER_MINUTE:,}")

Output

Chat delivery: the estimate sheet
  Messages      10 billion a day: 115,741/s on average, 347,222/s at the peak
  Deliveries    694,444/s at the peak, to 2.0 devices per message
  Connections   60 million open at the peak, on 858 gateways of 70,000
  Bandwidth     0.69 Gbit/s in, 1.39 Gbit/s out (text only)

Storage, two ways
  Keep every message      2.5 TB a day, 912.5 TB a year, 2.7 PB with 3 replicas
  Keep until delivered    an average wait of 9,820 s, so 1.14 billion messages on the server at once
                          284.2 GB, 852.5 GB with 3 replicas
      95% of messages wait        30 s and make  0.3% of what is stored
       4% of messages wait    28,800 s and make 11.7% of what is stored
       1% of messages wait   864,000 s and make 88.0% of what is stored
  One partition           1,000 messages x 250 B = 250.0 KB at most

Push notifications at the peak
  173,611 messages/s find the recipient's phone without a connection
  43,403 notifications/s when a burst of 4 messages makes one notification
  1.95 million a minute to Android, 3.3 x the default quota of 600,000

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 chat_storage.py

Quantity Assumption Result
Messages 200 million people × 50 messages a day 10 billion a day: 115,741 a second, about 347,000 at a peak of 3 ×
Deliveries each message goes to the recipient’s 1.5 devices and to the sender’s other 0.5 about 694,000 a second at the peak
Connections 20 % of 300 million devices online at the peak 60 million sockets on 858 gateways of 70,000 each
Bandwidth 250 bytes a message, text only 0.69 Gbit/s in and 1.39 Gbit/s out at the peak
Storage, keeping everything 2.5 TB a day 912.5 TB a year, 2.7 PB with three replicas
Storage, until delivered an average wait of 9,820 seconds 1.14 billion messages at once: 284 GB, 853 GB with three replicas
Notifications half the messages find the recipient’s phone without a connection, four to a burst 43,403 a second at the peak

Three results in the table deserve a second look. The deliveries are twice the messages, because people use more than one device. The storage depends far more on the retention rule than on the traffic: keeping messages only until they are delivered needs 284 GB, against 912.5 TB for a year of keeping them all. That figure comes from Little's law: In a stable system, the average number of requests inside equals the arrival rate times the average time each one spends inside, L = λW. At 100 requests a second and 50 ms each, about 5 are in progress. More in “Little's law, utilisation and server counts”, which says the number of messages on the server at any moment is the arrival rate times the average time each one stays. And the 1 % of messages that wait days for a device that is away make 88 % of what is stored, which is why the 30-day limit matters more than any compression.

The notifications are the surprise. Even with a burst of four messages merged into one, 43,403 notifications a second at the peak means 1.95 million a minute to Android phones. Firebase Cloud Messaging’s default quota is 600,000 messages a minute per project, answered with HTTP 429 above it, and a larger quota for an event needs 15 days of notice (Firebase documentation). Capacity planning for a chat service includes its push providers.

Data Storage Converter Convert the sheet's decimal terabytes and petabytes to the binary units a disk vendor or an operating system shows.

Key idea

The retention rule, not the traffic, sets the storage: keeping each message only until every device has it needs 284 GB, against 912.5 TB for a year of keeping everything.

Checkpoint

Check the estimate

  1. Question 1 of 2 200 million people each send 50 messages a day. How many messages a second is that on average?

    Type a number.

    A hint

    A day has 86,400 seconds.

    Show the answer: Question 1 of 2

    Answer: 115741 messages a second (anything from 104167 to 127315 counts)

    200,000,000 × 50 = 10 billion messages a day, and 10,000,000,000 ÷ 86,400 ≈ 115,741 a second. Size the service for the peak, not the average: at three times the average that is about 347,000.

  2. Question 2 of 2 At the peak, 20 % of 300 million devices hold a WebSocket open. A gateway holds 100,000 connections and runs at 70 % of that. How many gateways do you need?

    Type a number.

    A hint

    First count the open connections, then divide by what one gateway carries at 70 %.

    Show the answer: Question 2 of 2

    Answer: 858 gateways (anything from 772 to 944 counts)

    300,000,000 × 0.20 = 60 million connections, and each gateway takes 70,000 of them: 60,000,000 ÷ 70,000 ≈ 857.1, rounded up to 858. Add spares for a deploy and for losing a zone.

A few frames and seven stores, all keyed by conversation

Devices talk to the service over one WebSocket: A protocol, RFC 6455, that upgrades one HTTP request into a long-lived connection that carries messages in both directions. More in “Real-time delivery: polling, SSE and WebSockets” each, the two-way connection that RFC 6455 defines, so that the server can push without being asked (RFC 6455). A few frames carry everything:

  • send {conversation, client_id, text} from a device, answered by ack {client_id, seq, time}.
  • message {conversation, seq, sender, client_id, text} pushed to a device.
  • stored {conversation, up_to} and read {conversation, up_to} from a device, which the service passes on to the other member as receipt {conversation, user, delivered_up_to, read_up_to}.

Catching up uses plain requests, which also work when a socket cannot be opened:

  • GET /v1/sync?since=<token> returns the conversations whose last sequence number has moved, and a new token.
  • GET /v1/conversations/{id}/messages?after=<seq>&limit=100 returns one page of messages after a cursor.
  • PUT /v1/devices/{id}/push-token registers the token that FCM or APNs gave the app.
Store Key Holds Kind of store
Conversations conversation id the two members and the last sequence number a key-value store with conditional writes
Messages (conversation id, bucket), then seq sender, client id, text, server time a wide-column store, three replicas
Client ids (conversation, sender, client id) the sequence number the send got the same store, kept for 7 days
Receipts (conversation, user) delivered up to, read up to a key-value store
Device positions (conversation, device) how far the device has stored a key-value store
Session registry device id gateway and connection id in memory, with gateway leases
Push tokens device id the FCM or APNs token a key-value store

The sequence number is the order and the gap detector

The sequence number is the heart of the model. Phones’ clocks disagree with each other and with the servers, and two messages can carry the same millisecond. Discord’s engineers made the same point when they stopped sorting messages by a creation time and used time-ordered Snowflake ids instead: “two messages can have the same creation time” (Discord). A number per conversation goes further. It gives every device one total order, and it makes a missing message visible: a device holding 6 and 8 knows that 7 exists, which no timestamp can tell it.

Messages are partitioned by conversation and bucket, with the bucket computed from the sequence number: seq ÷ 1,000, rounded down. A partition then holds at most 1,000 messages, 250 KB, however busy the conversation, and a device with cursor 4,120 knows to start reading in bucket 4. Discord partitions its messages the same way, by channel and a bucket of a static time window. Its engineers chose about 10 days per bucket to keep partitions under 100 MB, and required that a bucket be computable from the message id or a timestamp (Discord). Their later post describes that layout on 177 Cassandra nodes holding trillions of messages, each partition on three replicas and read and written at quorum, before its move to 72 ScyllaDB nodes (Discord).

One owner per conversation, with an insert that never repeats a number

Who gives out the numbers? Each conversation has one owner at a time: the message service routes every send by a hash of the conversation id, so one instance numbers each conversation. The owner keeps the last number in memory and inserts each message with a condition, “only if this sequence number is still unused”, a A write that succeeds only if the key, here a conversation and a sequence number, holds nothing yet, as one atomic step. Of two writers that race for the same key, exactly one succeeds.. If an owner fails, the new one reads the last number from the store and carries on. If two owners overlap for a moment, the conditional insert lets only one of them have each number, and the loser reloads before it tries again. Step through such a moment:

Interactive · step by step Two owners for a moment, and still one message per number Step through a failover in which the old owner of conversation c7 has not yet noticed its replacement.
Two owners for a moment, and still one message per number, step 6 of 6. Showing Owner A (old) (last seq 42), Message store (c7 up to 43), Owner B (new) (last seq 43).Owner A (old)last seq 42Message storec7 up to 43Owner B (new)last seq 43

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text
  1. One owner. Owner A numbers conversation c7. It keeps the last number it gave out, 41, in memory, and the store holds every message up to 41.
  2. A failover. A failover hands c7 to owner B. Before it takes any send, owner B reads the conversation's last number from the store, which is one read.
  3. The old owner keeps going. Owner A has not noticed yet. A send reaches it, and it inserts the message as seq 42, only if 42 is still unused. It is, so the message is stored.
  4. Refused. Owner B takes a send too and tries seq 42, only if unused. The store refuses, because 42 is taken, and owner B's sender has no ack yet.
  5. Reload, then 43. Owner B reloads the last number, 42, and stores its message as 43. Its sender's ack carries seq 43.
  6. One message per number. However long the overlap lasts, the store accepts each number once, so no two messages of c7 share a seq, and a loser always reloads before it tries again.

Checkpoint

Check: Frames, stores and sequence numbers

  1. Question 1 of 2 Partitions hold 1,000 messages each, and a message's bucket is its seq ÷ 1,000, rounded down. A device's cursor is 4,120. In which bucket does its sync start reading?

    Type a number.

    A hint

    The first message the device lacks is the one after its cursor.

    Show the answer: Question 1 of 2

    Answer: 4 bucket

    The device lacks 4,121 onwards, and 4,121 ÷ 1,000 rounded down is 4. Because the bucket comes from the number, the device needs no index to find where to start, and no partition grows past 1,000 messages, about 250 KB.

  2. Question 2 of 2 Why does the design number messages per conversation instead of sorting them by the time on the sender's phone?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: Clocks disagree and two messages can share a millisecond, while a number gives one order and shows when a message is missing

    A sender's clock can be minutes off and two messages can carry the same millisecond, so time can show different orders on different devices. The owner's next number defines one order, and a missing number tells a device to fetch.

Store first, then push: the send, delivery and sync paths

Three paths cross these services:

Chat design: devices, gateways, a message service that numbers and stores messages, push notifications and a sync API.Sender's devicesConnection gatewaysWebSockets, heartbeatsMessage servicenumber, store, routeMessage store(conversation, bucket), seqSession registrydevice to gatewayNotification serviceone per burstFCM and APNsSync APIchanges after a cursorRecipient's devices1. send with a client id2. forward3. next seq, then store4. ack: the seq5. find devices6. push to connected devices7. deliver6b. no connectionnotifywake the app8. catch up after each cursorread9. delivered and read up to N

The send path, the delivery path and the sync path of one-to-one chat

Text description of the diagram

The sender's devices hold a WebSocket connection to the connection gateways, which keep the sockets open and check them with heartbeats.

  1. A device sends a message with a client id it chose itself.
  2. The gateway forwards it to the message service.
  3. The message service takes the conversation's next sequence number and stores the message in the message store, whose partitions are keyed by conversation and bucket.
  4. Only then does it acknowledge the send to the sender, with the sequence number.
  5. It looks up the recipient's devices in the session registry, which maps each connected device to its gateway.
  6. It pushes the message through the gateways to every device that has a connection, and (6b) asks the notification service to notify phones that have none. The notification service sends one notification per burst through FCM and APNs, which wake the app.
  7. The gateway delivers the message to the recipient's device.
  8. Any device that was away catches up through the sync API, which reads the message store after each of the device's cursors.
  9. Devices report how far they have stored and read, as "delivered up to N" and "read up to N", through their gateway.

The send path. The app shows the new message at once, marked as waiting, and sends it with its client id. The gateway forwards it to the conversation’s owner, which takes the next sequence number, writes the message to a quorum (N, R and W): The number of a key's N replicas that must take part: W for a write to succeed, R for a read. When R + W is more than N, every set of replicas a read asks shares at least one replica with every set a write reached. More in “Leaderless replication and quorums: N, R and W” of replicas and answers with an ack. Only now does the app show “sent”. The order is the guarantee: nothing is pushed, counted or acknowledged before the message is stored.

The delivery path. The owner looks up the devices of both members in the A map from each connected device to the gateway and connection that hold its socket, so that a service can push to that device. An entry goes when its connection does., which records the gateway that holds each connected device. It pushes the message through those gateways to every device except the one that sent it, so the sender’s own laptop gets a copy too. For each phone without a connection it asks the notification service for a A short message sent through the phone platform's service, Firebase Cloud Messaging on Android or APNs on Apple devices, that wakes an app with no open connection. It is best effort: it can be delayed, merged or dropped. instead. A push is a fast path and may be lost: the store is the truth.

The sync path. A device that connects, or that sees a gap, asks the sync API which conversations moved past its cursors, then fetches each one’s messages after the cursor, a page of 100 at a time. It reports how far it has stored and read, and the service turns those reports into receipts for the other member. Follow one message along the first two paths, from Asha’s phone to Ravi’s and back:

Interactive · message by message One message from send to read Step through Asha's "hi" from her phone to Ravi's and back, until her phone shows it as read.
One message from send to read, step 8 of 8. Messages so far: Asha's phone to Message service: send a1 "hi"; Message service to Asha's phone: ack: a1 is seq 41; Message service to Ravi's phone: push seq 41; Ravi's phone to Message service: delivered up to 41; Message service to Asha's phone: receipt: delivered 41; Ravi's phone to Message service: read up to 41; Message service to Asha's phone: receipt: read 41.Asha's phonea1: readMessage serviceowner and storeRavi's phonestored up to 41send a1 "hi"ack: a1 is seq 41push seq 41delivered up to 41receipt: delivered 41read up to 41receipt: read 41

The drawing shows the whole scene; its 8 steps are listed below.

Every step as text
  1. Shown at once, as waiting. Asha types "hi". Her app shows it straight away, marked as waiting, and sends it with the client id a1 that the app made itself.
  2. Numbered, then stored. The conversation's owner takes the next number, 41, and inserts the message only if 41 is still unused. It waits until a quorum of replicas has stored it.
  3. Only now, sent. Only after the store does the owner acknowledge. The ack says a1 is seq 41, and Asha's phone shows one tick, sent.
  4. Pushed, maybe. The owner pushes seq 41 through the gateway that holds Ravi's connection. A push is a fast path that may be lost; the store already has the message.
  5. Stored on the phone. Ravi's phone writes the message to its own storage and only then reports that it holds everything up to 41.
  6. Two ticks. The service passes the report on as a receipt, and Asha's phone shows the message as delivered.
  7. Read up to 41. Ravi opens the conversation. His phone reports read up to 41, a number that covers every earlier message as well.
  8. Read. The receipt reaches Asha's phone, and the message shows as read. Two numbers moved on the server; no event was stored per message.

The owner’s numbering and one device’s cursor take about 40 lines, shown here in 14 languages:

One program · 14 languages Store first, then push: one chat and one device one lookup a send; a gap costs one read of the store

Python 3.14 · main.py

class Chat:
    def __init__(self) -> None:
        self.log: list[str] = []
        self.seq_of: dict[str, int] = {}

    def send(self, client_id: str, text: str) -> int:
        if client_id in self.seq_of:  # a retry: the same answer, no second copy
            return self.seq_of[client_id]
        self.log.append(text)  # stored before anything is pushed
        self.seq_of[client_id] = len(self.log)
        return len(self.log)


class Device:
    def __init__(self) -> None:
        self.cursor = 0  # the highest seq shown, nothing missing below it
        self.shown: list[str] = []

    def on_push(self, store: Chat, seq: int, text: str) -> str:
        if seq <= self.cursor:
            return "a repeat, ignored"
        if seq == self.cursor + 1:
            self.shown.append(text)
            self.cursor = seq
            return "shown"
        first = self.cursor + 1  # a gap: fetch the rest from the store
        self.shown += store.log[self.cursor :]
        self.cursor = len(store.log)
        return f"gap, fetched {first} to {self.cursor}"


# a2's ack is lost, and so is the push of seq 2
SENDS = [("a1", "hi"), ("a2", "lunch?"), ("a2", "lunch?"), ("a3", "at 1")]
PUSHES = [(1, "hi"), (3, "at 1"), (3, "at 1")]

chat = Chat()
for client_id, text in SENDS:
    seq = chat.send(client_id, text)
    print(f"send {client_id}: seq {seq}, {len(chat.log)} stored")
laptop = Device()
for seq, text in PUSHES:
    result = laptop.on_push(chat, seq, text)
    print(f"push seq {seq}: {result}, cursor {laptop.cursor}")
print(f"laptop shows: {', '.join(laptop.shown)}")

In Python: store.log[self.cursor :] is the slice after the cursor, exactly what a gap leaves missing, because seq n sits at position n - 1 of the log; the space before the colon is how ruff formats a slice bound that is an expression.

Recorded with CPython 3.14.8 on macOS 26 arm64. It runs in your browser with Pyodide 314.0.7 (CPython 3.14.2). To run it yourself: python3 main.py

JavaScript Node 24 · main.mjs

class Chat {
  log = [];
  #seqOf = new Map();

  send(clientId, text) {
    if (this.#seqOf.has(clientId)) {
      return this.#seqOf.get(clientId); // a retry: the same answer, no second copy
    }
    this.log.push(text); // stored before anything is pushed
    this.#seqOf.set(clientId, this.log.length);
    return this.log.length;
  }
}

class Device {
  cursor = 0; // the highest seq shown, nothing missing below it
  shown = [];

  onPush(store, seq, text) {
    if (seq <= this.cursor) {
      return "a repeat, ignored";
    }
    if (seq === this.cursor + 1) {
      this.shown.push(text);
      this.cursor = seq;
      return "shown";
    }
    const first = this.cursor + 1; // a gap: fetch the rest from the store
    this.shown.push(...store.log.slice(this.cursor));
    this.cursor = store.log.length;
    return `gap, fetched ${first} to ${this.cursor}`;
  }
}

// a2's ack is lost, and so is the push of seq 2
const sends = [
  ["a1", "hi"],
  ["a2", "lunch?"],
  ["a2", "lunch?"],
  ["a3", "at 1"],
];
const pushes = [
  [1, "hi"],
  [3, "at 1"],
  [3, "at 1"],
];

const chat = new Chat();
for (const [clientId, text] of sends) {
  const seq = chat.send(clientId, text);
  console.log(`send ${clientId}: seq ${seq}, ${chat.log.length} stored`);
}
const laptop = new Device();
for (const [seq, text] of pushes) {
  const result = laptop.onPush(chat, seq, text);
  console.log(`push seq ${seq}: ${result}, cursor ${laptop.cursor}`);
}
console.log(`laptop shows: ${laptop.shown.join(", ")}`);

In JavaScript: #seqOf is a private field, so only Chat reads the client ids, and push(...store.log.slice(this.cursor)) appends the fetched messages one by one.

Recorded with Node.js 24.21.0 LTS (V8 13.6) on macOS 26 arm64. It runs in your browser with QuickJS (quickjs-emscripten 0.32.0). To run it yourself: node main.mjs

Java 25 LTS · Main.java

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class Main {
  static final class Chat {
    final List<String> log = new ArrayList<>();
    private final Map<String, Integer> seqOf = new HashMap<>();

    int send(String clientId, String text) {
      Integer known = seqOf.get(clientId);
      if (known != null) {
        return known; // a retry: the same answer, no second copy
      }
      log.add(text); // stored before anything is pushed
      seqOf.put(clientId, log.size());
      return log.size();
    }
  }

  static final class Device {
    int cursor = 0; // the highest seq shown, nothing missing below it
    final List<String> shown = new ArrayList<>();

    String onPush(Chat store, int seq, String text) {
      if (seq <= cursor) {
        return "a repeat, ignored";
      }
      if (seq == cursor + 1) {
        shown.add(text);
        cursor = seq;
        return "shown";
      }
      int first = cursor + 1; // a gap: fetch the rest from the store
      shown.addAll(store.log.subList(cursor, store.log.size()));
      cursor = store.log.size();
      return "gap, fetched " + first + " to " + cursor;
    }
  }

  record Push(int seq, String text) {}

  public static void main(String[] args) {
    // a2's ack is lost, and so is the push of seq 2
    String[][] sends = {{"a1", "hi"}, {"a2", "lunch?"}, {"a2", "lunch?"}, {"a3", "at 1"}};
    List<Push> pushes = List.of(new Push(1, "hi"), new Push(3, "at 1"), new Push(3, "at 1"));
    var chat = new Chat();
    for (String[] send : sends) {
      int seq = chat.send(send[0], send[1]);
      System.out.println("send " + send[0] + ": seq " + seq + ", " + chat.log.size() + " stored");
    }
    var laptop = new Device();
    for (Push push : pushes) {
      String result = laptop.onPush(chat, push.seq(), push.text());
      System.out.println("push seq " + push.seq() + ": " + result + ", cursor " + laptop.cursor);
    }
    System.out.println("laptop shows: " + String.join(", ", laptop.shown));
  }
}

In Java: subList(cursor, size) is a view of the log after the cursor and addAll copies it; a Push is a record, Java's class for a small value.

Recorded with OpenJDK 25.0.4.1 LTS (Eclipse Temurin) on macOS 26 arm64. It runs in your browser with a TeaVM 0.16.0 build in QuickJS (quickjs-emscripten 0.32.0). To run it yourself: java Main.java

C++23 · main.cpp

#include <format>
#include <map>
#include <print>
#include <string>
#include <utility>
#include <vector>

class Chat {
  std::map<std::string, int> seqOf;

public:
  std::vector<std::string> log;

  int send(const std::string &clientId, const std::string &text) {
    if (const auto known = seqOf.find(clientId); known != seqOf.end()) {
      return known->second; // a retry: the same answer, no second copy
    }
    log.push_back(text); // stored before anything is pushed
    return seqOf[clientId] = static_cast<int>(log.size());
  }
};

struct Device {
  int cursor = 0; // the highest seq shown, nothing missing below it
  std::vector<std::string> shown;

  std::string onPush(const Chat &store, int seq, std::string text) {
    if (seq <= cursor) {
      return "a repeat, ignored";
    }
    if (seq == cursor + 1) {
      shown.push_back(std::move(text));
      cursor = seq;
      return "shown";
    }
    const int first = cursor + 1; // a gap: fetch the rest from the store
    shown.insert(shown.end(), store.log.begin() + cursor, store.log.end());
    cursor = static_cast<int>(store.log.size());
    return std::format("gap, fetched {} to {}", first, cursor);
  }
};

int main() {
  // a2's ack is lost, and so is the push of seq 2
  const std::vector<std::pair<std::string, std::string>> sends = {
      {"a1", "hi"}, {"a2", "lunch?"}, {"a2", "lunch?"}, {"a3", "at 1"}};
  const std::vector<std::pair<int, std::string>> pushes = {
      {1, "hi"}, {3, "at 1"}, {3, "at 1"}};
  Chat chat;
  for (const auto &[clientId, text] : sends) {
    const int seq = chat.send(clientId, text);
    std::println("send {}: seq {}, {} stored", clientId, seq, chat.log.size());
  }
  Device laptop;
  for (const auto &[seq, text] : pushes) {
    const std::string result = laptop.onPush(chat, seq, text);
    std::println("push seq {}: {}, cursor {}", seq, result, laptop.cursor);
  }
  std::println("laptop shows: {:n:}", laptop.shown);
}

In C++: find in the if statement's initializer looks a retry up once, and the {:n:} format of C++23 prints a vector of strings without brackets or quotes: hi, lunch?, at 1.

Recorded with Apple clang 21.0.0 (C++23) and GCC 16.2.0 on macOS 26 arm64. It runs in your browser with Clang in WebAssembly (YoWASP) 22.0.0-git20542-10. To run it yourself: clang++ -std=c++23 -O2 -Wall -Wextra -Werror -pedantic -fsanitize=address,undefined main.cpp -o main && ./main

C# 14 · .NET 10 · app.cs

using System.Globalization;

CultureInfo.CurrentCulture = CultureInfo.InvariantCulture;

// a2's ack is lost, and so is the push of seq 2
(string ClientId, string Text)[] sends =
    [("a1", "hi"), ("a2", "lunch?"), ("a2", "lunch?"), ("a3", "at 1")];
(int Seq, string Text)[] pushes = [(1, "hi"), (3, "at 1"), (3, "at 1")];

var chat = new Chat();
foreach (var (clientId, text) in sends)
{
    int seq = chat.Send(clientId, text);
    Console.WriteLine($"send {clientId}: seq {seq}, {chat.Log.Count} stored");
}
var laptop = new Device();
foreach (var (seq, text) in pushes)
{
    string result = laptop.OnPush(chat, seq, text);
    Console.WriteLine($"push seq {seq}: {result}, cursor {laptop.Cursor}");
}
Console.WriteLine($"laptop shows: {string.Join(", ", laptop.Shown)}");

sealed class Chat
{
    public List<string> Log { get; } = [];
    private readonly Dictionary<string, int> seqOf = new();

    public int Send(string clientId, string text)
    {
        if (seqOf.TryGetValue(clientId, out int known))
        {
            return known; // a retry: the same answer, no second copy
        }
        Log.Add(text); // stored before anything is pushed
        seqOf[clientId] = Log.Count;
        return Log.Count;
    }
}

sealed class Device
{
    public int Cursor { get; private set; } // the highest seq shown, nothing missing below it
    public List<string> Shown { get; } = [];

    public string OnPush(Chat store, int seq, string text)
    {
        if (seq <= Cursor)
        {
            return "a repeat, ignored";
        }
        if (seq == Cursor + 1)
        {
            Shown.Add(text);
            Cursor = seq;
            return "shown";
        }
        int first = Cursor + 1; // a gap: fetch the rest from the store
        Shown.AddRange(store.Log.Skip(Cursor));
        Cursor = store.Log.Count;
        return $"gap, fetched {first} to {Cursor}";
    }
}

In C#: TryGetValue looks a retry up once, Skip(Cursor) is the LINQ slice after the cursor, and the sends and pushes are arrays of named tuples.

Recorded with C# 14 on .NET 10.0.12 (SDK 10.0.401) on macOS 26 arm64. It runs in your browser with the .NET runtime for WebAssembly 10.0.12. To run it yourself: dotnet run app.cs

C23 · main.c

#include <stdio.h>
#include <string.h>

enum { MAX_MESSAGES = 8 };

typedef struct {
  const char *log[MAX_MESSAGES];
  const char *ids[MAX_MESSAGES];
  int count;
} Chat;

static int chat_send(Chat *chat, const char *id, const char *text) {
  for (int i = 0; i < chat->count; i++) {
    if (strcmp(chat->ids[i], id) == 0) {
      return i + 1; // a retry: the same answer, no second copy
    }
  }
  chat->log[chat->count] = text; // stored before anything is pushed
  chat->ids[chat->count] = id;
  return ++chat->count;
}

typedef struct {
  int cursor; // the highest seq shown, nothing missing below it
  const char *shown[MAX_MESSAGES];
} Device;

static const char *on_push(Device *device, const Chat *store, int seq,
                           const char *text, char gap[static 48]) {
  if (seq <= device->cursor) {
    return "a repeat, ignored";
  }
  if (seq == device->cursor + 1) {
    device->shown[device->cursor++] = text;
    return "shown";
  }
  const int first = device->cursor + 1; // a gap: fetch the rest from the store
  for (; device->cursor < store->count; device->cursor++) {
    device->shown[device->cursor] = store->log[device->cursor];
  }
  snprintf(gap, 48, "gap, fetched %d to %d", first, device->cursor);
  return gap;
}

int main(void) {
  // a2's ack is lost, and so is the push of seq 2
  static const char *const sends[][2] = {
      {"a1", "hi"}, {"a2", "lunch?"}, {"a2", "lunch?"}, {"a3", "at 1"}};
  static const int seqs[] = {1, 3, 3};
  static const char *const texts[] = {"hi", "at 1", "at 1"};
  Chat chat = {0};
  for (size_t i = 0; i < sizeof sends / sizeof sends[0]; i++) {
    const int seq = chat_send(&chat, sends[i][0], sends[i][1]);
    printf("send %s: seq %d, %d stored\n", sends[i][0], seq, chat.count);
  }
  Device laptop = {0};
  for (size_t i = 0; i < sizeof seqs / sizeof seqs[0]; i++) {
    char gap[48];
    const char *result = on_push(&laptop, &chat, seqs[i], texts[i], gap);
    printf("push seq %d: %s, cursor %d\n", seqs[i], result, laptop.cursor);
  }
  printf("laptop shows:");
  for (int i = 0; i < laptop.cursor; i++)
    printf("%s %s", i ? "," : "", laptop.shown[i]);
  printf("\n");
  return 0;
}

In C: C has no map, so the client id of seq n is kept at ids[n - 1] and a retry is found by a scan with strcmp; the device's cursor doubles as the count of what it shows.

Recorded with Apple clang 21.0.0 (C23) and GCC 16.2.0 on macOS 26 arm64. It runs in your browser with Clang in WebAssembly (YoWASP) 22.0.0-git20542-10. To run it yourself: clang -std=c23 -O2 -Wall -Wextra -Werror -pedantic -fsanitize=address,undefined main.c -o main && ./main

PHP 8.5 · main.php

<?php

declare(strict_types=1);

final class Chat
{
    /** @var list<string> */
    public array $log = [];
    /** @var array<string, int> */
    private array $seqOf = [];

    public function send(string $clientId, string $text): int
    {
        if (isset($this->seqOf[$clientId])) {
            return $this->seqOf[$clientId]; // a retry: the same answer, no second copy
        }
        $this->log[] = $text; // stored before anything is pushed
        $this->seqOf[$clientId] = count($this->log);
        return count($this->log);
    }
}

final class Device
{
    public int $cursor = 0; // the highest seq shown, nothing missing below it
    /** @var list<string> */
    public array $shown = [];

    public function onPush(Chat $store, int $seq, string $text): string
    {
        if ($seq <= $this->cursor) {
            return 'a repeat, ignored';
        }
        if ($seq === $this->cursor + 1) {
            $this->shown[] = $text;
            $this->cursor = $seq;
            return 'shown';
        }
        $first = $this->cursor + 1; // a gap: fetch the rest from the store
        array_push($this->shown, ...array_slice($store->log, $this->cursor));
        $this->cursor = count($store->log);
        return "gap, fetched {$first} to {$this->cursor}";
    }
}

// a2's ack is lost, and so is the push of seq 2
$sends = [['a1', 'hi'], ['a2', 'lunch?'], ['a2', 'lunch?'], ['a3', 'at 1']];
$pushes = [[1, 'hi'], [3, 'at 1'], [3, 'at 1']];

$chat = new Chat();
foreach ($sends as [$clientId, $text]) {
    $seq = $chat->send($clientId, $text);
    echo "send {$clientId}: seq {$seq}, " . count($chat->log) . " stored\n";
}
$laptop = new Device();
foreach ($pushes as [$seq, $text]) {
    $result = $laptop->onPush($chat, $seq, $text);
    echo "push seq {$seq}: {$result}, cursor {$laptop->cursor}\n";
}
echo 'laptop shows: ' . implode(', ', $laptop->shown) . "\n";

In PHP: array_slice($store->log, $this->cursor) is the slice after the cursor, the spread operator appends it in one call, and isset finds a retry without a warning for a missing key.

Recorded with PHP 8.5.11 (CLI) on macOS 26 arm64. It runs in your browser with php-wasm 0.2.0 (PHP 8.5.2). To run it yourself: php main.php

Rust 1.99 · main.rs

use std::collections::HashMap;

#[derive(Default)]
struct Chat {
    log: Vec<String>,
    seq_of: HashMap<String, usize>,
}

impl Chat {
    fn send(&mut self, client_id: &str, text: &str) -> usize {
        if let Some(&seq) = self.seq_of.get(client_id) {
            return seq; // a retry: the same answer, no second copy
        }
        self.log.push(text.to_string()); // stored before anything is pushed
        self.seq_of.insert(client_id.to_string(), self.log.len());
        self.log.len()
    }
}

#[derive(Default)]
struct Device {
    cursor: usize, // the highest seq shown, nothing missing below it
    shown: Vec<String>,
}

impl Device {
    fn on_push(&mut self, store: &Chat, seq: usize, text: &str) -> String {
        if seq <= self.cursor {
            return "a repeat, ignored".to_string();
        }
        if seq == self.cursor + 1 {
            self.shown.push(text.to_string());
            self.cursor = seq;
            return "shown".to_string();
        }
        let first = self.cursor + 1; // a gap: fetch the rest from the store
        self.shown.extend_from_slice(&store.log[self.cursor..]);
        self.cursor = store.log.len();
        format!("gap, fetched {first} to {}", self.cursor)
    }
}

fn main() {
    // a2's ack is lost, and so is the push of seq 2
    let sends = [
        ("a1", "hi"),
        ("a2", "lunch?"),
        ("a2", "lunch?"),
        ("a3", "at 1"),
    ];
    let pushes = [(1, "hi"), (3, "at 1"), (3, "at 1")];
    let mut chat = Chat::default();
    for (client_id, text) in sends {
        let seq = chat.send(client_id, text);
        println!("send {client_id}: seq {seq}, {} stored", chat.log.len());
    }
    let mut laptop = Device::default();
    for (seq, text) in pushes {
        let result = laptop.on_push(&chat, seq, text);
        println!("push seq {seq}: {result}, cursor {}", laptop.cursor);
    }
    println!("laptop shows: {}", laptop.shown.join(", "));
}

In Rust: on_push borrows the chat (&Chat) only for the call, so the store and the device are never borrowed mutably at once; extend_from_slice copies the slice after the cursor.

Recorded with rustc 1.99.0 (edition 2024) on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: rustc --edition 2024 -O main.rs && ./main

TypeScript 6.0 · main.ts

class Chat {
  readonly log: string[] = [];
  readonly #seqOf = new Map<string, number>();

  send(clientId: string, text: string): number {
    const known = this.#seqOf.get(clientId);
    if (known !== undefined) {
      return known; // a retry: the same answer, no second copy
    }
    this.log.push(text); // stored before anything is pushed
    this.#seqOf.set(clientId, this.log.length);
    return this.log.length;
  }
}

class Device {
  cursor = 0; // the highest seq shown, nothing missing below it
  readonly shown: string[] = [];

  onPush(store: Chat, seq: number, text: string): string {
    if (seq <= this.cursor) {
      return "a repeat, ignored";
    }
    if (seq === this.cursor + 1) {
      this.shown.push(text);
      this.cursor = seq;
      return "shown";
    }
    const first = this.cursor + 1; // a gap: fetch the rest from the store
    this.shown.push(...store.log.slice(this.cursor));
    this.cursor = store.log.length;
    return `gap, fetched ${first} to ${this.cursor}`;
  }
}

// a2's ack is lost, and so is the push of seq 2
const sends: [string, string][] = [
  ["a1", "hi"],
  ["a2", "lunch?"],
  ["a2", "lunch?"],
  ["a3", "at 1"],
];
const pushes: [number, string][] = [
  [1, "hi"],
  [3, "at 1"],
  [3, "at 1"],
];

const chat = new Chat();
for (const [clientId, text] of sends) {
  const seq = chat.send(clientId, text);
  console.log(`send ${clientId}: seq ${seq}, ${chat.log.length} stored`);
}
const laptop = new Device();
for (const [seq, text] of pushes) {
  const result = laptop.onPush(chat, seq, text);
  console.log(`push seq ${seq}: ${result}, cursor ${laptop.cursor}`);
}
console.log(`laptop shows: ${laptop.shown.join(", ")}`);

In TypeScript: #seqOf is a private field of standard JavaScript, so it survives when the types are stripped, and Map.get returns undefined for a new client id, which send checks before it returns.

Recorded with TypeScript 6.0.3 (strict), run by Node.js 24.21.0 on macOS 26 arm64. It runs in your browser with QuickJS (quickjs-emscripten 0.32.0) after Sucrase 3.35.1. To run it yourself: npx -p typescript@6.0.3 tsc --noEmit --strict main.ts && node main.ts

Go 1.27 · main.go

package main

import (
	"fmt"
	"strings"
)

type Chat struct {
	log   []string
	seqOf map[string]int
}

func (c *Chat) send(clientID, text string) int {
	if seq, ok := c.seqOf[clientID]; ok {
		return seq // a retry: the same answer, no second copy
	}
	c.log = append(c.log, text) // stored before anything is pushed
	c.seqOf[clientID] = len(c.log)
	return len(c.log)
}

type Device struct {
	cursor int // the highest seq shown, nothing missing below it
	shown  []string
}

func (d *Device) onPush(store *Chat, seq int, text string) string {
	switch {
	case seq <= d.cursor:
		return "a repeat, ignored"
	case seq == d.cursor+1:
		d.shown = append(d.shown, text)
		d.cursor = seq
		return "shown"
	}
	first := d.cursor + 1 // a gap: fetch the rest from the store
	d.shown = append(d.shown, store.log[d.cursor:]...)
	d.cursor = len(store.log)
	return fmt.Sprintf("gap, fetched %d to %d", first, d.cursor)
}

func main() {
	// a2's ack is lost, and so is the push of seq 2
	sends := [][2]string{{"a1", "hi"}, {"a2", "lunch?"}, {"a2", "lunch?"}, {"a3", "at 1"}}
	pushes := []struct {
		seq  int
		text string
	}{{1, "hi"}, {3, "at 1"}, {3, "at 1"}}
	chat := &Chat{seqOf: map[string]int{}}
	for _, send := range sends {
		seq := chat.send(send[0], send[1])
		fmt.Printf("send %s: seq %d, %d stored\n", send[0], seq, len(chat.log))
	}
	laptop := &Device{}
	for _, push := range pushes {
		result := laptop.onPush(chat, push.seq, push.text)
		fmt.Printf("push seq %d: %s, cursor %d\n", push.seq, result, laptop.cursor)
	}
	fmt.Printf("laptop shows: %s\n", strings.Join(laptop.shown, ", "))
}

In Go: A switch with no condition replaces the if chain, append(d.shown, store.log[d.cursor:]...) adds the slice after the cursor, and the map of client ids is made with the chat.

Recorded with Go 1.27.1 on macOS 26 arm64. It runs in your browser with the Go compiler in WebAssembly 1.27.1. To run it yourself: go run main.go

Swift 6.4 · main.swift

struct Chat {
  private(set) var log: [String] = []
  private var seqOf: [String: Int] = [:]

  mutating func send(clientID: String, text: String) -> Int {
    if let known = seqOf[clientID] {
      return known  // a retry: the same answer, no second copy
    }
    log.append(text)  // stored before anything is pushed
    seqOf[clientID] = log.count
    return log.count
  }
}

struct Device {
  private(set) var cursor = 0  // the highest seq shown, nothing missing below it
  private(set) var shown: [String] = []

  mutating func onPush(from store: Chat, seq: Int, text: String) -> String {
    if seq <= cursor {
      return "a repeat, ignored"
    }
    if seq == cursor + 1 {
      shown.append(text)
      cursor = seq
      return "shown"
    }
    let first = cursor + 1  // a gap: fetch the rest from the store
    shown += store.log[cursor...]
    cursor = store.log.count
    return "gap, fetched \(first) to \(cursor)"
  }
}

// a2's ack is lost, and so is the push of seq 2
let sends = [("a1", "hi"), ("a2", "lunch?"), ("a2", "lunch?"), ("a3", "at 1")]
let pushes = [(1, "hi"), (3, "at 1"), (3, "at 1")]

var chat = Chat()
for (clientID, text) in sends {
  let seq = chat.send(clientID: clientID, text: text)
  print("send \(clientID): seq \(seq), \(chat.log.count) stored")
}
var laptop = Device()
for (seq, text) in pushes {
  let result = laptop.onPush(from: chat, seq: seq, text: text)
  print("push seq \(seq): \(result), cursor \(laptop.cursor)")
}
print("laptop shows: \(laptop.shown.joined(separator: ", "))")

In Swift: Chat and Device are structs, so onPush(from:) receives a copy of the store and send is a mutating method; store.log[cursor...] is the slice after the cursor.

Recorded with Apple Swift 6.4 on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: swiftc -O main.swift -o main && ./main

Ruby 4.0 · main.rb

# frozen_string_literal: true

# Numbers each message, stores it and answers retries.
class Chat
  attr_reader :log

  def initialize
    @log = []
    @seq_of = {}
  end

  def send_message(client_id, text)
    return @seq_of[client_id] if @seq_of.key?(client_id) # a retry: the same answer, no second copy

    @log << text # stored before anything is pushed
    @seq_of[client_id] = @log.size
  end
end

# Shows messages in seq order, each once.
class Device
  attr_reader :cursor, :shown

  def initialize
    @cursor = 0 # the highest seq shown, nothing missing below it
    @shown = []
  end

  def on_push(store, seq, text)
    return 'a repeat, ignored' if seq <= @cursor

    if seq == @cursor + 1
      @shown << text
      @cursor = seq
      return 'shown'
    end
    first = @cursor + 1 # a gap: fetch the rest from the store
    @shown.concat(store.log[@cursor..])
    @cursor = store.log.size
    "gap, fetched #{first} to #{@cursor}"
  end
end

# a2's ack is lost, and so is the push of seq 2
SENDS = [%w[a1 hi], %w[a2 lunch?], %w[a2 lunch?], ['a3', 'at 1']].freeze
PUSHES = [[1, 'hi'], [3, 'at 1'], [3, 'at 1']].freeze

chat = Chat.new
SENDS.each do |client_id, text|
  seq = chat.send_message(client_id, text)
  puts "send #{client_id}: seq #{seq}, #{chat.log.size} stored"
end
laptop = Device.new
PUSHES.each do |seq, text|
  result = laptop.on_push(chat, seq, text)
  puts "push seq #{seq}: #{result}, cursor #{laptop.cursor}"
end
puts "laptop shows: #{laptop.shown.join(', ')}"

In Ruby: send would replace Object#send, which every Ruby object has, so the method is send_message, and it ends with an assignment, whose value is the new seq. RuboCop asks for a comment above each class.

Recorded with CRuby 4.0.7 on macOS 26 arm64. It runs in your browser with ruby.wasm 2.10.1 (Ruby 4.0.0). To run it yourself: ruby main.rb

Kotlin 2.4 · Main.kt

class Chat {
    val log = mutableListOf<String>()
    private val seqOf = mutableMapOf<String, Int>()

    fun send(
        clientId: String,
        text: String,
    ): Int {
        seqOf[clientId]?.let { return it } // a retry: the same answer, no second copy
        log.add(text) // stored before anything is pushed
        seqOf[clientId] = log.size
        return log.size
    }
}

class Device {
    var cursor = 0 // the highest seq shown, nothing missing below it
        private set
    val shown = mutableListOf<String>()

    fun onPush(
        store: Chat,
        seq: Int,
        text: String,
    ): String {
        if (seq <= cursor) return "a repeat, ignored"
        if (seq == cursor + 1) {
            shown.add(text)
            cursor = seq
            return "shown"
        }
        val first = cursor + 1 // a gap: fetch the rest from the store
        shown.addAll(store.log.drop(cursor))
        cursor = store.log.size
        return "gap, fetched $first to $cursor"
    }
}

fun main() {
    // a2's ack is lost, and so is the push of seq 2
    val sends = listOf("a1" to "hi", "a2" to "lunch?", "a2" to "lunch?", "a3" to "at 1")
    val pushes = listOf(1 to "hi", 3 to "at 1", 3 to "at 1")
    val chat = Chat()
    for ((clientId, text) in sends) {
        val seq = chat.send(clientId, text)
        println("send $clientId: seq $seq, ${chat.log.size} stored")
    }
    val laptop = Device()
    for ((seq, text) in pushes) {
        val result = laptop.onPush(chat, seq, text)
        println("push seq $seq: $result, cursor ${laptop.cursor}")
    }
    println("laptop shows: ${laptop.shown.joinToString(", ")}")
}

In Kotlin: seqOf[clientId]?.let { return it } answers a retry in one line, drop(cursor) gives the messages after the cursor, and cursor has a private setter.

Recorded with Kotlin 2.4.20 on Java 25.0.4.1 on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: kotlinc Main.kt -include-runtime -d main.jar && java -jar main.jar

Dart 3.13 · main.dart

class Chat {
  final List<String> log = [];
  final Map<String, int> _seqOf = {};

  int send(String clientId, String text) {
    final known = _seqOf[clientId];
    if (known != null) {
      return known; // a retry: the same answer, no second copy
    }
    log.add(text); // stored before anything is pushed
    _seqOf[clientId] = log.length;
    return log.length;
  }
}

class Device {
  int cursor = 0; // the highest seq shown, nothing missing below it
  final List<String> shown = [];

  String onPush(Chat store, int seq, String text) {
    if (seq <= cursor) {
      return 'a repeat, ignored';
    }
    if (seq == cursor + 1) {
      shown.add(text);
      cursor = seq;
      return 'shown';
    }
    final first = cursor + 1; // a gap: fetch the rest from the store
    shown.addAll(store.log.sublist(cursor));
    cursor = store.log.length;
    return 'gap, fetched $first to $cursor';
  }
}

void main() {
  // a2's ack is lost, and so is the push of seq 2
  final sends = [
    ('a1', 'hi'),
    ('a2', 'lunch?'),
    ('a2', 'lunch?'),
    ('a3', 'at 1'),
  ];
  final pushes = [(1, 'hi'), (3, 'at 1'), (3, 'at 1')];
  final chat = Chat();
  for (final (clientId, text) in sends) {
    final seq = chat.send(clientId, text);
    print('send $clientId: seq $seq, ${chat.log.length} stored');
  }
  final laptop = Device();
  for (final (seq, text) in pushes) {
    final result = laptop.onPush(chat, seq, text);
    print('push seq $seq: $result, cursor ${laptop.cursor}');
  }
  print('laptop shows: ${laptop.shown.join(', ')}');
}

In Dart: sublist(cursor) is the slice after the cursor, and the sends and pushes are records, taken apart by the for-in pattern (clientId, text).

Recorded with Dart 3.13.5 on macOS 26 arm64. It runs in your browser with a WebAssembly build by dart2wasm 3.13.5. To run it yourself: dart run main.dart

What every language printed

send a1: seq 1, 1 stored
send a2: seq 2, 2 stored
send a2: seq 2, 2 stored
send a3: seq 3, 3 stored
push seq 1: shown, cursor 1
push seq 3: gap, fetched 2 to 3, cursor 3
push seq 3: a repeat, ignored, cursor 3
laptop shows: hi, lunch?, at 1

The retried send of a2 got seq 2 again, and the store still held two messages. The push of seq 2 was lost, so the push of seq 3 found a gap and the laptop fetched 2 and 3 from the store; the second push of 3 was a repeat and changed nothing. The laptop ends with all three messages, in order, each once.

Lose a different push

Press Edit in the Python tab, change PUSHES to [(2, "lunch?"), (1, "hi"), (3, "at 1")] and run it: the push of 2 now finds the gap, the laptop fetches 1 to 3 at once, and the pushes of 1 and 3 are both repeats.

The full model, in six scenarios

The model below runs the three paths on in-memory parts. Asha has a phone, and Ravi has a phone and a laptop. Partitions hold four messages, so trimming shows within a dozen. Pushes wait in an outbox until the service flushes it, which lets two sends cross in flight, as they do on real networks.

Sequence numbers, client ids, gaps, receipts, notifications and trimming Python · model/chat_core.py
# A one-to-one chat service over in-memory parts (lesson system-design-hld/design-chat-delivery). The message service
# numbers each conversation's messages and stores them before it pushes anything. Pushes wait in an outbox until the
# service flushes it, so two sends can cross in flight, and a push can be lost, as one on a real socket can. A phone
# without a connection gets one notification per burst. Devices keep a cursor per conversation, show messages strictly
# in sequence order and fetch any gap from the store. Receipts are two numbers per member: delivered up to, read up to.

BUCKET = 4  # messages per storage partition (tiny here, so that trimming shows)


def say(text, depth=1):
    print("  " * depth + text)


class Store:
    """Conversation logs in partitions keyed by (conversation, seq // BUCKET), and the numbers kept beside them."""

    def __init__(self):
        self.parts = {}  # (conversation, bucket) -> {seq: message}
        self.last_seq = {}  # conversation -> the highest seq given out
        self.by_client_id = {}  # (conversation, sender, client id) -> seq: makes a retried send harmless
        self.at = {}  # (conversation, device) -> the highest seq that device has stored
        self.delivered = {}  # (conversation, user) -> the highest seq one of the user's devices has stored
        self.read = {}  # (conversation, user) -> the highest seq the user has read

    def insert(self, conv, msg):
        part = self.parts.setdefault((conv, msg["seq"] // BUCKET), {})
        if msg["seq"] in part:  # a conditional insert: a second writer of the same seq is refused
            raise ValueError(f"seq {msg['seq']} of {conv} is taken")
        part[msg["seq"]] = msg

    def after(self, conv, seq):
        """The sync read: every stored message of conv after seq, oldest first."""
        out = []
        for b in range((seq + 1) // BUCKET, self.last_seq.get(conv, 0) // BUCKET + 1):
            out += [m for s, m in sorted(self.parts.get((conv, b), {}).items()) if s > seq]
        return out

    def count(self, conv, client_id):
        return sum(1 for (c, _), p in self.parts.items() if c == conv for m in p.values() if m["client_id"] == client_id)


class Device:
    def __init__(self, name, user, kind, service):
        self.name, self.user, self.kind, self.svc = name, user, kind, service
        self.cursor = {}  # conversation -> highest seq shown, with nothing missing below it
        self.early = {}  # conversation -> {seq: message} that arrived above a gap
        self.shown = {}  # conversation -> messages in seq order
        self.pending = {}  # (conversation, client id) -> text shown as waiting until the service acknowledges it

    def accept(self, conv, msgs):
        """Keep what is new and show it in seq order; True if something still waits above a gap."""
        early = self.early.setdefault(conv, {})
        cur = self.cursor.get(conv, 0)
        for m in msgs:
            if m["seq"] > cur:  # duplicates and old messages change nothing
                early[m["seq"]] = m
        while cur + 1 in early:
            cur += 1
            self.shown.setdefault(conv, []).append(early.pop(cur))
        self.cursor[conv] = cur
        return bool(early)

    def on_push(self, conv, msg):
        if msg["seq"] <= self.cursor.get(conv, 0):
            say(f"{self.name} got seq {msg['seq']} again: a duplicate, ignored", 2)
        elif self.accept(conv, [msg]):
            say(f"{self.name} got seq {msg['seq']} but has only up to {self.cursor[conv]}: it syncs", 2)
            self.sync(conv)
        self.svc.stored(conv, self)

    def on_ack(self, conv, client_id, seq, text):
        del self.pending[(conv, client_id)]
        if self.accept(conv, [{"seq": seq, "sender": self.user, "client_id": client_id, "text": text}]):
            say(f"{self.name}: its own message is seq {seq}, but seq {self.cursor[conv] + 1} is missing: it syncs", 2)
            self.sync(conv)
        self.svc.stored(conv, self)

    def sync(self, conv):
        got = self.svc.store.after(conv, self.cursor.get(conv, 0))
        self.accept(conv, got)
        say(f"{self.name} fetched seq {', '.join(str(m['seq']) for m in got)} of {conv} from the store", 2)

    def view(self, conv):
        return "  ".join(f"{m['seq']}:{m['text']}" for m in self.shown.get(conv, []))


class ChatService:
    def __init__(self):
        self.store = Store()
        self.members = {}  # conversation -> its two users
        self.devices = {}  # user -> devices
        self.online = {}  # device name -> gateway: the session registry
        self.outbox = []  # (device, conversation, message): pushes in flight
        self.lose = set()  # (device name, seq): pushes the network will lose
        self.notices = {}  # (conversation, device name) -> messages merged into the notification on screen
        self.sent_notices = 0

    def add(self, name, user, kind, gateway=None):
        d = Device(name, user, kind, self)
        self.devices.setdefault(user, []).append(d)
        if gateway:
            self.online[name] = gateway
        return d

    def send(self, device, conv, client_id, text, lose_ack=False, flush=True):
        st = self.store
        device.pending[(conv, client_id)] = text
        key = (conv, device.user, client_id)
        if key in st.by_client_id:  # a retry of a send that was stored: the same answer, and no second copy
            seq = st.by_client_id[key]
            say(f"{device.name} retries {client_id}: it is stored already as seq {seq}, so the answer is seq {seq} again")
            device.on_ack(conv, client_id, seq, text)
            return seq
        seq = st.last_seq.get(conv, 0) + 1
        msg = {"seq": seq, "sender": device.user, "client_id": client_id, "text": text}
        st.insert(conv, msg)  # stored (on a quorum of replicas, in a real store) before anything else happens
        st.last_seq[conv] = seq
        st.by_client_id[key] = seq
        say(f"{device.name} sends {client_id} {text!r}: stored as seq {seq} of {conv}")
        if lose_ack:
            say("the acknowledgement is lost on the way back: the app still shows the message as waiting", 2)
        else:
            device.on_ack(conv, client_id, seq, text)
        self.outbox += [(d, conv, msg) for u in self.members[conv] for d in self.devices[u] if d is not device]
        if flush:
            self.flush()
        return seq

    def flush(self):
        out, self.outbox = self.outbox, []
        for d, conv, msg in out:
            if d.name not in self.online:
                self.notify(conv, d)
            elif (d.name, msg["seq"]) in self.lose:
                say(f"the push of seq {msg['seq']} to {d.name} on {self.online[d.name]} is lost", 2)
            else:
                d.on_push(conv, msg)

    def notify(self, conv, d):
        if d.kind != "phone":
            return  # laptops and browsers catch up when they next connect
        key = (conv, d.name)
        if key in self.notices:
            self.notices[key] += 1
            say(f"{d.name} is offline: merged into its notification ({self.notices[key]} new in {conv})", 2)
        else:
            self.notices[key] = 1
            self.sent_notices += 1
            say(f"{d.name} is offline: one push notification for {conv}", 2)

    def stored(self, conv, d):
        """A device reports how far it has stored: one cumulative receipt, never one per message."""
        st, seq = self.store, d.cursor.get(conv, 0)
        if seq > st.at.get((conv, d.name), 0):
            st.at[(conv, d.name)] = seq
            st.delivered[(conv, d.user)] = max(st.delivered.get((conv, d.user), 0), seq)

    def mark_read(self, conv, d):
        st = self.store
        st.read[(conv, d.user)] = max(st.read.get((conv, d.user), 0), d.cursor.get(conv, 0))
        say(f"{d.user} reads {conv} on {d.name}: read up to seq {st.read[(conv, d.user)]}")

    def connect(self, d, gateway):
        self.online[d.name] = gateway
        say(f"{d.name} connects to {gateway} and asks which conversations moved past its cursors")
        for conv, users in self.members.items():
            if d.user in users and self.store.last_seq.get(conv, 0) > d.cursor.get(conv, 0):
                d.sync(conv)
                self.stored(conv, d)
                self.notices.pop((conv, d.name), None)

    def ticks(self, conv, d):
        """What the sender's device shows for each of its own messages."""
        st = self.store
        other = next(u for u in self.members[conv] if u != d.user)
        out = []
        for m in d.shown.get(conv, []):
            if m["sender"] == d.user:
                s = m["seq"]
                state = "read" if s <= st.read.get((conv, other), 0) else "delivered" if s <= st.delivered.get((conv, other), 0) else "sent"
                out.append(f"{s} {state}")
        return ", ".join(out + [f"{cid} waiting" for (c, cid) in d.pending if c == conv])

    def trim(self, conv):
        """Drop whole partitions that every device of the conversation has stored."""
        st = self.store
        low = min(st.at.get((conv, d.name), 0) for u in self.members[conv] for d in self.devices[u])
        gone = sorted(b for (c, b) in st.parts if c == conv and b * BUCKET + BUCKET - 1 <= low)
        for b in gone:
            del st.parts[(conv, b)]
        return low, gone


svc = ChatService()
asha = svc.add("asha-phone", "asha", "phone", "g1")
ravi_phone = svc.add("ravi-phone", "ravi", "phone", "g1")
ravi_laptop = svc.add("ravi-laptop", "ravi", "laptop", "g2")
svc.members["c1"] = ["asha", "ravi"]

print("1. Three messages while both of Ravi's devices are online")
for i, text in enumerate(["hi", "lunch?", "at 1"], start=1):
    svc.send(asha, "c1", f"a{i}", text)
say(f"asha-phone shows: {svc.ticks('c1', asha)}")
svc.mark_read("c1", ravi_laptop)
say(f"asha-phone shows: {svc.ticks('c1', asha)}")

print("2. The acknowledgement is lost, and the app retries with the same client id")
svc.send(asha, "c1", "a4", "ok?", lose_ack=True)
say(f"asha-phone shows: {svc.ticks('c1', asha)}")
svc.send(asha, "c1", "a4", "ok?")
copies = svc.store.count("c1", "a4")
say(f"copies of a4 in the store: {copies}; ravi-phone shows: {ravi_phone.view('c1')}")

print("3. Both send at once: Ravi's push is still in flight when Asha sends")
svc.send(ravi_laptop, "c1", "r1", "yes", flush=False)
svc.send(asha, "c1", "a5", "see you", flush=False)
svc.flush()
say(f"asha-phone shows:  {asha.view('c1')}")
say(f"ravi-laptop shows: {ravi_laptop.view('c1')}")
svc.mark_read("c1", asha)

print("4. A push is lost; the next one reveals the gap")
svc.lose.add(("ravi-laptop", 7))
for i, text in enumerate(["menu", "link", "thx"], start=6):
    svc.send(asha, "c1", f"a{i}", text)

print("5. Ravi's phone goes offline; Asha sends a burst")
del svc.online["ravi-phone"]
for i, text in enumerate(["call me", "now?", "pls"], start=9):
    svc.send(asha, "c1", f"a{i}", text)
svc.connect(ravi_phone, "g3")
svc.mark_read("c1", ravi_phone)
say(f"asha-phone shows: {svc.ticks('c1', asha)}")

print("6. Trimming what every device has")
low, gone = svc.trim("c1")
say(f"every device has stored up to seq {low}: partitions {gone} deleted")
say(f"the store still holds seq {', '.join(str(m['seq']) for m in svc.store.after('c1', 0))}")

print()
seqs = {d.name: [m["seq"] for m in d.shown["c1"]] for d in (asha, ravi_phone, ravi_laptop)}
same = len({tuple(s) for s in seqs.values()}) == 1 and seqs["asha-phone"] == list(range(1, 13))
checks = [
    ("a retried send is stored once and shown once", copies == 1 and [m["client_id"] for m in ravi_phone.shown["c1"]].count("a4") == 1),
    ("every device shows seq 1 to 12 in the same order", same),
    ("a lost push is repaired from the store", 7 in seqs["ravi-laptop"]),
    ("three messages to an offline phone made one notification", svc.sent_notices == 1),
    ("receipts are four numbers for two members, whatever the number of messages", len(svc.store.delivered) + len(svc.store.read) == 4),
    ("the store keeps only partitions that some device may still need", all(b * BUCKET + BUCKET - 1 > low for (c, b) in svc.store.parts)),
]
for name, ok in checks:
    print(f"{'ok  ' if ok else 'FAIL'} {name}")

Output

1. Three messages while both of Ravi's devices are online
  asha-phone sends a1 'hi': stored as seq 1 of c1
  asha-phone sends a2 'lunch?': stored as seq 2 of c1
  asha-phone sends a3 'at 1': stored as seq 3 of c1
  asha-phone shows: 1 delivered, 2 delivered, 3 delivered
  ravi reads c1 on ravi-laptop: read up to seq 3
  asha-phone shows: 1 read, 2 read, 3 read
2. The acknowledgement is lost, and the app retries with the same client id
  asha-phone sends a4 'ok?': stored as seq 4 of c1
    the acknowledgement is lost on the way back: the app still shows the message as waiting
  asha-phone shows: 1 read, 2 read, 3 read, a4 waiting
  asha-phone retries a4: it is stored already as seq 4, so the answer is seq 4 again
  copies of a4 in the store: 1; ravi-phone shows: 1:hi  2:lunch?  3:at 1  4:ok?
3. Both send at once: Ravi's push is still in flight when Asha sends
  ravi-laptop sends r1 'yes': stored as seq 5 of c1
  asha-phone sends a5 'see you': stored as seq 6 of c1
    asha-phone: its own message is seq 6, but seq 5 is missing: it syncs
    asha-phone fetched seq 5, 6 of c1 from the store
    asha-phone got seq 5 again: a duplicate, ignored
  asha-phone shows:  1:hi  2:lunch?  3:at 1  4:ok?  5:yes  6:see you
  ravi-laptop shows: 1:hi  2:lunch?  3:at 1  4:ok?  5:yes  6:see you
  asha reads c1 on asha-phone: read up to seq 6
4. A push is lost; the next one reveals the gap
  asha-phone sends a6 'menu': stored as seq 7 of c1
    the push of seq 7 to ravi-laptop on g2 is lost
  asha-phone sends a7 'link': stored as seq 8 of c1
    ravi-laptop got seq 8 but has only up to 6: it syncs
    ravi-laptop fetched seq 7, 8 of c1 from the store
  asha-phone sends a8 'thx': stored as seq 9 of c1
5. Ravi's phone goes offline; Asha sends a burst
  asha-phone sends a9 'call me': stored as seq 10 of c1
    ravi-phone is offline: one push notification for c1
  asha-phone sends a10 'now?': stored as seq 11 of c1
    ravi-phone is offline: merged into its notification (2 new in c1)
  asha-phone sends a11 'pls': stored as seq 12 of c1
    ravi-phone is offline: merged into its notification (3 new in c1)
  ravi-phone connects to g3 and asks which conversations moved past its cursors
    ravi-phone fetched seq 10, 11, 12 of c1 from the store
  ravi reads c1 on ravi-phone: read up to seq 12
  asha-phone shows: 1 read, 2 read, 3 read, 4 read, 6 read, 7 read, 8 read, 9 read, 10 read, 11 read, 12 read
6. Trimming what every device has
  every device has stored up to seq 12: partitions [0, 1, 2] deleted
  the store still holds seq 12

ok   a retried send is stored once and shown once
ok   every device shows seq 1 to 12 in the same order
ok   a lost push is repaired from the store
ok   three messages to an offline phone made one notification
ok   receipts are four numbers for two members, whatever the number of messages
ok   the store keeps only partitions that some device may still need

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 chat_core.py

Read the output by scenario. In the first, the three messages go from sent to delivered as Ravi’s devices store them, and to read when he opens the conversation on his laptop. One receipt, “read up to 3”, marks all three as read. In the second, the acknowledgement of a4 is lost and the app sends it again with the same client id; the store holds one copy. The third is the race that confuses people: Ravi and Asha send before either has seen the other’s message, the service numbers them 5 and 6 in the order they reach it, and Asha’s phone fetches the 5 it is missing.

In the fourth, the push of seq 7 to Ravi’s laptop is lost, and nothing notices until seq 8 reveals the gap. In the fifth, Ravi’s phone has no connection while Asha sends three messages; it gets one notification, with the next two merged into it, and on reconnecting it fetches 10 to 12. The last scenario trims the store: every device has stored up to 12, so the three partitions that end at or below 12 are deleted. The sections below take these rules one at a time.

Checkpoint

Check: Store first, then push

  1. Question 1 of 2 Which of these happen before the sender's app may show a message as sent?

    Choose every answer that is right.

    Show the answer: Question 1 of 2

    Answer:

    • The message is stored on a quorum of replicas
    • The conversation's owner gives the message its sequence number

    Sent means the service has the message: it is numbered and stored, and the ack carries its sequence number. Pushes and notifications come after the store, and the recipient's report is what moves the message on to delivered.

  2. Question 2 of 2 The push of a message to Ravi's laptop is lost on the network. What brings the message to the laptop?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: The laptop sees a gap when the next message arrives, or syncs when it reconnects, and fetches the message from the store

    Every message is stored before it is pushed. A push that is lost leaves a gap that the next message reveals, or a cursor behind the store that the next sync closes. In the block, the push of seq 2 was lost and the push of 3 made the laptop fetch 2 and 3.

Two growing numbers per member drive every tick

A receipt per message would multiply the traffic. Each message would cause two more events, delivered and read, and each event would be a row to write and a push to send. The design stores two numbers per member of a conversation instead, both only ever growing:

  • Delivered up to N: the highest sequence number that at least one of the member’s devices has stored. A device reports it after it has written messages to its own storage, never when a push merely arrives, so “delivered” means the message survives the app being killed.
  • Read up to N: the highest sequence number the member has seen with the conversation open, on any device.

The sender’s app compares each of its own messages with the other member’s two numbers. With Ravi at delivered up to 10 and read up to 7, Asha’s messages 5 to 12 show like this:

Asha’s message Against delivered up to 10 and read up to 7 Her phone shows
seq 5 to 7 at or below read up to 7 read
seq 8 to 10 above 7, at or below 10 delivered
seq 11 and 12 above both sent

One A device's report that it has stored, or shown with the conversation open, every message of a conversation up to a sequence number. One receipt covers any number of messages. covers any number of messages, so a person who opens a conversation with 40 unread messages sends one receipt, not 40. The read number also goes to the reader’s other devices, which clear their unread counts and remove the notification for that conversation. If a person turns read receipts off, their own devices still share “read up to” with each other, and the service stops passing it to the other member.

Checkpoint

Check: Delivered and read as two numbers

  1. One question Asha sent messages 5 to 12 of a conversation. Ravi's receipts say delivered up to 10 and read up to 7. How many of her messages show as delivered but not yet read?

    Type a number.

    A hint

    A message at or below read up to is read; above it but at or below delivered up to, it is delivered.

    Show the answer: One question

    Answer: 3 messages

    5, 6 and 7 are at or below read up to 7, so they are read. 8, 9 and 10 are above 7 but at or below 10: three messages delivered and not read. 11 and 12 are above both numbers, so they are sent.

A client id makes a retried send harmless

Networks fail between a write and its acknowledgement, so the app will resend messages the service already has. The cure is a key that belongs to the message, not to the attempt. The app creates a client id before the first try, for example a UUID version 7, whose most significant 48 bits are a Unix time in milliseconds and whose other bits are mostly random (RFC 9562), and sends the same id on every retry. The service keeps an index from (conversation, sender, client id) to the sequence number for 7 days, longer than any app keeps retrying, and answers a repeat with the original number. Step through the model’s second scenario:

Interactive · message by message A lost acknowledgement, and a retry that stores nothing twice Step through Asha's message a4 when its acknowledgement never reaches her phone.
A lost acknowledgement, and a retry that stores nothing twice, step 7 of 7. Messages so far: Asha's phone to Message service: send a4 "ok?"; Message service to Ravi's phone: push seq 4; Message service to Asha's phone: ack: a4 is seq 4; Asha's phone to Message service: send a4 "ok?" again; Message service to Asha's phone: ack: a4 is seq 4; Message service to Ravi's phone: push seq 4 again.Asha's phonea4: sent, seq 4Message servicea4 is seq 4Ravi's phonecursor 4send a4 "ok?"push seq 4ack: a4 is seq 4send a4 "ok?" againack: a4 is seq 4push seq 4 againa repeat, ignored

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text
  1. The first try. Asha sends "ok?" with the client id a4, which her app made once for this message.
  2. Stored as 4. The owner stores the message as seq 4 and records a4 → 4 in its client-id index, which it keeps for 7 days.
  3. Pushed to Ravi. The message is stored, so the owner pushes seq 4 to Ravi's phone, which shows it and moves its cursor to 4.
  4. The ack is lost. The acknowledgement leaves the service but never reaches Asha's phone, so the app still shows the message as waiting.
  5. The same id again. The app sends the message again with the same client id, a4. A new id for the same message would make a second copy.
  6. The same answer. The index already holds a4, so the owner answers seq 4 again, stores nothing new and pushes nothing. The phone now shows sent.
  7. One copy on screen. Ravi's phone holds seq 4 once. If a reconnection pushes it again, the phone ignores it, because 4 is not above its cursor.

UUID Generator Make version 7 UUIDs and see the time inside them: the shape of a client id that also sorts by when it was made.

Delivery to devices is at least once. Pushes can repeat after a reconnection, and a sync can return a message that a push also brought. The device makes that harmless by ignoring any sequence number at or below its cursor, so “at least once” on the network becomes “exactly once” on the screen.

Checkpoint

Check: Client ids and repeats

  1. One question The service already holds client id a4 as seq 4 when the same send arrives again. What does it answer?

    Choose one answer.

    Show the answer: One question

    Answer: Seq 4 again, without storing a second copy

    A client id belongs to the message, not to the attempt. The service keeps (conversation, sender, client id) → seq for 7 days and answers every repeat with the same number, so the store holds one copy.

The owner’s order of arrival decides, and a gap is fetched after a second

Order is kept per conversation and nowhere else. A total order across all conversations would need one sequencer for 10 billion messages a day, and nobody can see it: Asha cannot tell whether her message to Ravi was numbered before or after a stranger’s message to someone else. The list of conversations is sorted by the server time of each one’s last message, which needs no global order either.

Inside a conversation, the order of arrival at the owner decides. A device shows its own unacknowledged messages below everything else, and moves each one to its place when the ack brings its sequence number. When an ack or a push reveals a gap, the device first waits about a second for the push of the missing message, and then fetches it. Waiting briefly avoids a request for every push that arrives out of order; fetching afterwards bounds the time a message can stay missing. The model fetches at once, which keeps its output short. Step through its third scenario:

Interactive · message by message Two senders at once, and one order on both screens Step through two messages that cross in flight, until both screens show seq 1 to 6 in the same order.
Two senders at once, and one order on both screens, step 7 of 7. Messages so far: Ravi's laptop to Conversation owner: send r1 "yes"; Asha's phone to Conversation owner: send a5 "see you"; Conversation owner to Asha's phone: ack: a5 is seq 6; Asha's phone to Conversation owner: fetch after seq 4; Conversation owner to Asha's phone: seq 5 and seq 6; Conversation owner to Ravi's laptop: push seq 6.Asha's phoneshows up to 6Conversation ownerRavi's laptopshows up to 6send r1 "yes"send a5 "see you"ack: a5 is seq 6fetch after seq 4seq 5 and seq 6push seq 6

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text
  1. Both up to 4. Both screens show the conversation up to seq 4. Ravi types "yes" on his laptop while Asha types "see you"; neither has seen the other's message.
  2. Ravi's arrives first. Ravi's send reaches the owner first, so it becomes seq 5.
  3. Asha's comes next. Asha's send arrives a moment later and becomes seq 6. The order of arrival at the owner decides; neither device does.
  4. A gap at 5. Asha's ack says seq 6, but her phone shows only up to 4, so it knows that a message 5 exists that it has not got.
  5. Wait, then fetch. The phone waits about a second for the push of 5. When it does not come, the phone fetches whatever after 4 is still missing from the store.
  6. In order. The fetch returns 5 and 6, and Asha's phone shows 1 to 6 in sequence order.
  7. One order everywhere. Ravi's laptop gets 6 by push, and a late push of 5 to Asha's phone changes nothing. Both screens show the same order, decided by the owner.

Checkpoint

Check: Order and gaps within a conversation

  1. One question A device shows a conversation up to seq 6, and a push of seq 8 arrives. What should it do?

    Choose one answer.

    Show the answer: One question

    Answer: Hold 8, wait about a second for 7, then fetch whatever after 6 is still missing from the store

    Showing messages strictly in sequence order is what keeps every device in one order, and the store has every message the service acknowledged. A brief wait avoids a fetch for pushes that are merely out of order; the fetch bounds how long a lost push can keep a message missing.

A notification is a wake-up call, never the delivery

Push providers are best-effort, and their documented limits shape the design:

Provider Documented limit What the design does about it
APNs may reorder notifications to one device token; keeps one notification per app, usually the latest, for an offline device, up to 30 days by the expiry the sender sets; 4 KB of payload treats a notification as a hint and syncs on wake (Apple)
Firebase Cloud Messaging at most 100 non-collapsible messages kept for an offline Android device; above that all are discarded and the app is told when it reconnects runs a full sync on that signal (Firebase)
Firebase Cloud Messaging 4,096 bytes of payload a message; at most 240 a minute and 5,000 an hour to a single Android device, and a quota per project merges bursts and plans quota (Firebase, quotas)

So a notification is a wake-up call with a preview, never the delivery itself. The notification service skips devices that have an open connection. It sends at most one notification per conversation and device for a burst, merging later messages into it until the device connects, and it removes a conversation’s notification when the person reads it elsewhere. When the app wakes, it syncs: the store is still the truth.

WebSocket & SSE Tester Open a WebSocket to an echo server and watch the frames go both ways: the connection a gateway keeps for every device.

Checkpoint

Check: Push notifications as wake-ups

  1. One question Ravi's phone has an open WebSocket when Asha's message is stored. What does the notification service do for that phone?

    Choose one answer.

    Show the answer: One question

    Answer: Nothing, because the push over its connection delivers the message and a notification would only add load and quota

    Notifications are for phones that cannot be reached over a connection. The service skips devices that have one, merges a burst into one notification per conversation and device, and removes it when the person reads elsewhere.

The store forgets on purpose: the slowest device decides what stays

The message store is a buffer, not an archive. The service records how far each device has stored, and a background job deletes every partition that all the devices of its conversation have passed: the lowest device position of the conversation decides. Deleting whole partitions is one operation per 1,000 messages instead of one per message. A second rule removes messages older than 30 days whatever the positions say, because a device that never comes back would otherwise hold its conversations’ messages forever. A sender whose message expires that way keeps seeing “sent”, which is the truth.

The sheet’s three groups of messages show why the slowest device decides:

Messages Wait on the server Share of what is stored
95 %, every device online 30 seconds 0.3 %
4 %, a device away for hours 28,800 seconds (8 hours) 11.7 %
1 %, a device away for days 864,000 seconds (10 days) 88.0 %

A conversation whose devices are all online keeps a few seconds of messages. One with a phone switched off for a fortnight keeps a fortnight of them, and those few conversations are where the storage sits.

Checkpoint

Check: What the store keeps, and for how long

  1. One question In the model, partitions hold seq 0 to 3, 4 to 7, 8 to 11 and 12 to 15. The conversation's three devices have stored up to 12, 12 and 9. Which partitions can be deleted?

    Choose one answer.

    Show the answer: One question

    Answer: 0 to 3 and 4 to 7

    The background job deletes every partition that all the devices of the conversation have passed. With positions 12, 12 and 9, the lowest is 9, so the partitions ending at 3 and 7 go and the one holding 8 to 11 stays.

A dead gateway or owner costs seconds, never a message

Next, an interviewer pushes on the design, and this section and the next two answer.

What fails What is lost What brings it back
A gateway with 70,000 connections its sockets, and pushes in flight to it devices reconnect elsewhere after a random delay and sync from their cursors
The owner of a conversation numbering, for one read the new owner reads the last number from the store; apps retry unacknowledged sends

A gateway dies with 70,000 connections

Its devices lose their sockets and reconnect to other gateways after a random delay, then sync from their cursors. Spread over 30 seconds, that is about 2,300 reconnections a second for the sign-in and sync APIs to absorb. Pushes sent to the dead gateway in the meantime are lost, and the sync brings them back, because every one was stored before it was pushed. The registry entries of the dead gateway are removed only if they still name its connections, so that a device which has already reconnected elsewhere keeps its new entry; the real-time connections lesson reproduces that race.

The owner of a conversation fails

Ownership moves to another instance, which reads the conversation’s last sequence number from the store, one read, before it takes sends. Sends that reached the old owner and were stored are safe; sends that were not stored fail or time out, and the apps retry them with the same client ids. The conditional insert is what keeps an old owner that has not yet noticed its replacement from writing a second message with the same number, as the two owners in the data-model section showed.

Checkpoint

Check: A dead gateway or owner

  1. One question A gateway with 70,000 connections dies, and its devices reconnect after random delays spread over 30 seconds. About how many reconnections a second do the other gateways and the sync API absorb?

    Type a number.

    A hint

    Spread evenly, the 70,000 reconnections arrive over 30 seconds.

    Show the answer: One question

    Answer: 2333 reconnections a second (anything from 2100 to 2566 counts)

    70,000 ÷ 30 ≈ 2,333 reconnections a second, each followed by a sync from the device's cursors. Without the random delay all 70,000 would arrive in the same second.

Long absences, new laptops and a second region change only the catch-up

A phone was away for two weeks

On reconnection, GET /v1/sync returns the conversations that moved and the device’s position in each. The app fetches the newest page of each conversation first, so the screen is useful within a second, then fills in older pages in the background, 100 messages at a time. It reports “stored up to” only for the messages it has actually written, so the senders’ ticks move as the backlog lands. Firebase may already have discarded its own queued messages for this device; its special message saying that the limit was reached is one more reason to sync, which the app was going to do anyway. Walk through the weeks:

Timeline · 6 events in order A phone away for weeks, phase by phase Walk through what the service, the push provider and the app do while Ravi's phone is away, and when it returns.
  1. Day 0

    The phone goes quiet

    Ravi's phone loses its connection. The server keeps every message of his conversations until all his devices have stored it, for 30 days at most.

  2. 100 queued messages

    The push provider gives up provider limit

    Firebase Cloud Messaging keeps at most 100 non-collapsible messages for an offline Android device. Above that it discards them all, and tells the app when it reconnects.

  3. Day 14

    Back online

    GET /v1/sync returns the conversations that moved past the phone's cursors. The app fetches the newest page of each first, so the screen is useful within a second, then older pages of 100.

  4. Pages land

    Ticks move as the backlog lands

    The app reports stored up to N only for messages it has written to its own storage, so the senders see delivered move page by page.

  5. Day 30

    The oldest messages expire retention

    Messages older than 30 days are deleted whatever the device positions say, so one device that never returns cannot keep a conversation's messages forever.

  6. After 30 days

    A note instead of a silent gap

    When a conversation's oldest stored message is above the cursor plus one, messages were lost. The sync says so and the app shows a note; their senders still see sent.

The timeline has 6 events, from Day 0 to After 30 days.

A device was away for more than 30 days

Its oldest messages are gone from the server. The sync answer says so: a conversation whose oldest stored message is above the device’s cursor plus one has lost messages, and the app shows a note in that conversation instead of pretending nothing is missing. The senders’ messages stay at “sent”, which is accurate.

A person signs in on a new laptop

The laptop starts with each conversation’s current sequence number as its cursor and receives everything from then on. Older history is not on the server to give. Copying it from the person’s phone, encrypted from device to device, belongs to the groups and encryption lesson.

Two regions

Give every conversation a home region: In an active-active design, the one region that takes all the writes of a given user or tenant, so that each of their items has one place to be written. More in “Multi-region: active-passive vs active-active” whose owner numbers its messages, and keep gateways close to people. When Asha in Mumbai writes to Ravi in São Paulo, her gateway forwards the send to the conversation’s home, and pushes go out from the home to Ravi’s region. Distance alone sets a floor: about 13,800 km at about 200,000 km a second in fibre is a round trip of at least 138 milliseconds, and real routes are longer (latency numbers). Choose the home where the conversation began, and move it only when both members have lived elsewhere for weeks, because a move needs a short pause in numbering.

Checkpoint

Check: Long absences, new laptops and regions

  1. One question Mumbai to São Paulo is about 13,800 km, and light in fibre covers about 200,000 km a second. What is the least possible round trip, in milliseconds?

    Type a number.

    A hint

    A round trip covers the distance twice.

    Show the answer: One question

    Answer: 138 ms (anything from 124 to 152 counts)

    2 × 13,800 km ÷ 200,000 km a second = 0.138 s, about 140 ms, before any routing or queueing. Real routes are longer, so a send that crosses to the conversation's home region and back costs at least that.

At ten times the load, the owners and the gateways give out first

Each row multiplies one load:

Load What gives out first What to change
10 × the messages: 3.5 million a second at the peak The owners’ write rate and the store’s partitions More owner instances and store nodes; batch writes per conversation within a few milliseconds
10 × the people online Gateways: 600 million sockets About 8,600 gateways, more regions, and a registry sharded by device id
100 × the notifications Push provider quotas and per-device limits Quota plans agreed with the providers long before launch, stronger merging of bursts, and no notifications to devices that are online

What each level is expected to cover

Each level builds on the one above:

Level Covers
Fresher and early career WebSockets and gateways, storing before pushing, sent, delivered and read, and push notifications for offline phones
Mid level (2 to 5 years) The estimates, per-conversation sequence numbers, client ids for retries, cursors and sync, and the data model
Senior (5 to 10 years) Receipts as numbers, gaps and the race between senders, conversation owners with conditional inserts, retention by device positions, and push provider limits
Staff (above 10 years) Two regions, failure of owners and gateways at scale, capacity at 10 × and 100 ×, the cost of the retention rule, and planning quotas with outside providers

A 45-minute self-mock

Rehearse the round against a clock, out loud. Give the opening 5 minutes to clarifying questions, the next 5 to the estimate sheet, 10 to the frames, the stores and the three paths, 15 to the three hardest deep dives, 5 to what fails at ten times the load, and the closing 5 to the trade-offs you would defend. Afterwards, mark every requirement your answer never returned to.

Checkpoint

Check: Ten and a hundred times the load

  1. One question Ten times as many people are online at once. What gives out first, and what do you change?

    Choose one answer.

    Show the answer: One question

    Answer: The gateways, at 600 million sockets; about 8,600 gateways, more regions and a session registry sharded by device id

    At 70,000 connections a gateway, 600 million sockets need about 8,600 gateways, spread over more regions, and the session registry must be sharded by device id. Ten times the messages is a different problem: the owners' write rate and the store's partitions.

Practise: show pushed messages in order

The exercise is the receiving device’s rule: take a batch of pushed messages in any order, show what can be shown in sequence order, hold what sits above a gap and list the numbers to fetch. The sample tests run in the page:

Exercise · Medium · Python

Show pushed messages in order and find the gaps

Write receive(cursor, early, batch) in receive.py: what a device does with one batch of messages pushed to it for one conversation.

  • cursor is the highest sequence number the device shows with nothing missing below it.
  • early is a dict {seq: text} of messages the device holds but cannot show yet, because a lower one is missing.
  • batch is a list of (seq, text) pairs that just arrived, in any order. It may repeat a message, and it may hold messages at or below cursor, which the device has already shown.

Return a tuple (cursor, early, shown, missing):

  • cursor: the new highest sequence number shown with nothing missing below it;
  • early: the messages still held above a gap, as a dict;
  • shown: the (seq, text) pairs shown now, in sequence order, each once;
  • missing: the sorted list of sequence numbers above the new cursor and below the highest one held, which the device must fetch from the store.

Starter code · receive.py

def receive(cursor, early, batch):
    """Apply one pushed batch to a conversation: (cursor, early, shown, missing)."""
    # Replace these lines with your code.
    return cursor, early, [], []
The sample tests · test_receive.py
from receive import receive


def test_in_order():
    """messages that arrive in order are shown at once"""
    assert receive(0, {}, [(1, "hi"), (2, "lunch?")]) == (2, {}, [(1, "hi"), (2, "lunch?")], [])


def test_out_of_order_batch():
    """a batch in the wrong order is shown in sequence order"""
    assert receive(0, {}, [(3, "c"), (1, "a"), (2, "b")]) == (3, {}, [(1, "a"), (2, "b"), (3, "c")], [])


def test_duplicates_and_old_messages():
    """repeats and messages already shown change nothing"""
    assert receive(2, {}, [(2, "b"), (1, "a"), (3, "c"), (3, "c")]) == (3, {}, [(3, "c")], [])


def test_gap_holds_and_reports():
    """messages above a gap wait, and the gap is reported"""
    assert receive(2, {}, [(5, "e"), (4, "d")]) == (2, {4: "d", 5: "e"}, [], [3])


def test_gap_filled_later():
    """the missing message releases everything held above it"""
    assert receive(2, {4: "d", 5: "e"}, [(3, "c")]) == (5, {}, [(3, "c"), (4, "d"), (5, "e")], [])


def test_two_gaps():
    """every missing number below the highest one held is reported"""
    assert receive(1, {}, [(3, "c"), (6, "f")]) == (1, {3: "c", 6: "f"}, [], [2, 4, 5])


def test_empty_batch():
    """nothing arrives, nothing changes"""
    assert receive(4, {}, []) == (4, {}, [], [])
A hint

Copy early into a new dict and add every pair of the batch whose seq is greater than cursor. Then, while cursor + 1 is in the dict, move that message to shown and add one to cursor. The missing numbers are those between cursor + 1 and the largest key still in the dict that the dict does not have.

The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.

Then test yourself on the twelve ideas above, one question at a time; each session opens on the idea your answers show is weakest:

Practice that adapts to you

Practise this lesson

15 questions about 12 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

  1. Question 1 of 15 In this design, what does "delivered" mean on the sender's screen?

    Choose one answer.

    Show the answer: Question 1 of 15

    Answer: At least one of the recipient's devices has written the message to its own storage

    Sent: the service has it. Delivered: one of the recipient's devices has stored it. Read: the recipient has had the conversation open past it on some device.

  2. Question 2 of 15 If the server kept every message, at 10 billion messages a day and 250 bytes each, how many terabytes would one year take, before replication?

    Type a number.

    A hint

    First the bytes a day, then a year of them.

    Show the answer: Question 2 of 15

    Answer: 912.5 TB (anything from 821.5 to 1003.5 counts)

    10,000,000,000 × 250 bytes = 2.5 TB a day, and 2.5 TB × 365 = 912.5 TB a year, about 2.7 PB with three replicas. Keeping messages only until every device has them needs 284 GB, about 1/3,200 of that.

  3. Question 3 of 15 Messages arrive at 115,741 a second and wait on the server 9,820 seconds on average until every device has them. How many billion messages are on the server at any moment?

    Type a number.

    A hint

    Little's law multiplies the arrival rate by the average time spent inside.

    Show the answer: Question 3 of 15

    Answer: 1.14 billion messages (anything from 1.03 to 1.25 counts)

    By Little's law the number in the system is the arrival rate times the average time spent there: 115,741 × 9,820 ≈ 1.137 billion messages, about 284 GB at 250 bytes each, or 853 GB with three replicas.

  4. Question 4 of 15 During a failover, the old owner of a conversation and the new one both try to store a message as seq 42. What happens?

    Choose one answer.

    Show the answer: Question 4 of 15

    Answer: The conditional insert accepts the first and refuses the second, whose owner reloads the last number and stores its message as 43

    Ownership can briefly overlap, so the owner never trusts its memory alone: it inserts "only if this number is unused". The loser of a race reloads the last number before it tries again, which is what the two-owner scene shows.

  5. Question 5 of 15 In the model's third scenario, Asha's ack says seq 6 while her phone shows messages up to 4. What does her phone do?

    Choose one answer.

    Show the answer: Question 5 of 15

    Answer: It sees that seq 5 is missing, fetches it from the store, and later ignores the push of 5 as a duplicate

    The model printed that Asha's phone saw seq 5 missing, fetched 5 and 6 from the store, and ignored the push of 5 that arrived afterwards. Both screens then showed 1 to 6 in the same order.

  6. Question 6 of 15 Ravi opens a conversation with 40 unread messages. How many read receipts does his phone send?

    Choose one answer.

    Show the answer: Question 6 of 15

    Answer: One, read up to the last of them

    Receipts are two growing numbers per member. Opening the conversation moves read up to to the last message shown, one receipt, and the sender's app marks every message at or below it as read.

  7. Question 7 of 15 The client-id index is kept for 7 days. What goes wrong if it is kept for only one minute?

    Choose one answer.

    Show the answer: Question 7 of 15

    Answer: A phone that retries after a long time without a network gets a new number, and the message is stored and shown twice

    A retry is harmless only while the service still remembers the client id. Kept for 7 days, the index outlasts any app's retries; kept for a minute, a late retry looks like a new message.

  8. Question 8 of 15 After a reconnection the gateway pushes seq 9 again to a device whose cursor is 12. What does the device do?

    Choose one answer.

    Show the answer: Question 8 of 15

    Answer: Ignores it, because 9 is at or below its cursor

    Delivery to devices is at least once: pushes can repeat after a reconnection and a sync can return what a push brought. Ignoring every number at or below the cursor makes each message appear once.

  9. Question 9 of 15 Why does the design keep order per conversation only, and not across all conversations?

    Choose one answer.

    Show the answer: Question 9 of 15

    Answer: A total order would need one sequencer for 10 billion messages a day, and nobody can see the order of two different conversations

    Asha cannot tell whether her message to Ravi was numbered before or after a stranger's message, so a global order would buy nothing for one sequencer's worth of cost.

  10. Question 10 of 15 An Android phone stays offline while 150 non-collapsible messages are sent to it through Firebase Cloud Messaging. What happens?

    Choose one answer.

    Show the answer: Question 10 of 15

    Answer: Firebase discards all of them and tells the app when it reconnects, so the app runs a full sync from the store

    Firebase keeps at most 100 non-collapsible messages for an offline Android device. Above that it discards them and sends the app a special message on reconnection, which should lead to a full sync, as this design does anyway.

  11. Question 11 of 15 The sheet says 1 % of messages, those waiting for a device away for days, make 88 % of what is stored. Which change cuts the storage most?

    Choose one answer.

    Show the answer: Question 11 of 15

    Answer: A shorter limit for messages that wait, such as 14 days instead of 30

    By Little's law the stored messages are the arrival rate times the average wait, and the average wait is set by the few messages that wait days. Changing how long those may wait changes the total most.

  12. Question 12 of 15 Pushes sent to a gateway in the second before it died are lost. Why does no message go missing?

    Choose one answer.

    Show the answer: Question 12 of 15

    Answer: Every message was stored before it was pushed, so the devices fetch it when they sync after reconnecting

    The pushes were copies of messages already stored on a quorum. The devices reconnect to other gateways, sync from their cursors and get every message they lacked.

  13. Question 13 of 15 A device returns after 40 days. A conversation's oldest stored message is above the device's cursor plus one. What does the app do?

    Choose one answer.

    Show the answer: Question 13 of 15

    Answer: It shows a note in that conversation that some messages were lost, instead of a silent gap

    Messages older than 30 days are deleted whatever the device positions say. A gap between the cursor and the oldest stored message means messages were lost, and the app shows a note there.

  14. Question 14 of 15 A person signs in on a new laptop. What does the laptop show of their existing conversations?

    Choose one answer.

    Show the answer: Question 14 of 15

    Answer: Everything from now on, starting from each conversation's current number; older history can only come from another of their devices

    History lives on the devices. The new laptop starts its cursors at each conversation's current sequence number, and copying older history from the phone, encrypted between devices, is a separate feature.

  15. Question 15 of 15 Notifications are expected to grow a hundredfold for a festival peak. What do you do before it?

    Choose one answer.

    Show the answer: Question 15 of 15

    Answer: Agree quota plans with the push providers long in advance, merge bursts more strongly and send nothing to devices that are online

    At the estimate's peak, notifications already need 1.95 million a minute to Android against a default quota of 600,000. A hundredfold growth is planned with the providers ahead of time, and the service sends fewer notifications by merging bursts and skipping online devices.

Interview questions

Why does a chat app number messages per conversation instead of ordering them by timestamp? Because clocks disagree and numbers do not. A phone’s clock can be minutes off, servers’ clocks drift apart, and two messages can carry the same millisecond, so sorting by time can show different orders on different devices. A sequence number given out by the conversation’s owner, the next integer each time, defines one order that every device shows. It also makes gaps visible: a device holding messages 6 and 8 knows that 7 exists and fetches it, which no timestamp can tell it. The order is kept per conversation only, because nobody can see the order of messages in two different conversations.

What do the sent, delivered and read states mean, and how are they stored? Sent means the service has stored the message and answered with its sequence number. Delivered means at least one of the recipient’s devices has written it to its own storage. Read means the recipient has had the conversation open past it on some device. They are stored as two numbers per member of a conversation, “delivered up to” and “read up to”, which only grow. The sender’s app compares each of its messages with them, so one receipt covers any number of messages, and opening a conversation with 40 unread messages sends one receipt, not 40.

An interviewer who hears these answers tends to probe two corners: two messages that tie on the clock, and a person with several devices. Choose your reply to each:

Checkpoint

Interview round: two follow-up questions

  1. Question 1 of 2 Interview follow-up: two people send at the same millisecond by the server's clock. Which message comes first in the conversation?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: Whichever reaches the conversation's owner first; its sequence number decides, and every device shows that order

    Time never decides the order inside a conversation. The owner gives the next number to whichever send it handles first, so two sends in one millisecond still get 5 and 6, and every device shows 5 before 6.

  2. Question 2 of 2 Interview follow-up: Ravi's laptop has stored a message, but his phone, which he is looking at, has not. What does Asha see, and is it right?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: Delivered, because one of Ravi's devices has stored it; the definition is per person, not per device

    This design keeps two numbers per member. Delivered moves when any of the member's devices has written the message to its own storage, so the laptop's report is enough; the phone will catch up when it syncs.

Key takeaways

  • Store first, then push: a message is numbered and stored on a quorum before the sender sees “sent” and before any device is told.
  • Number messages per conversation, from one owner at a time with conditional inserts; never use timestamps for order.
  • Make retries harmless with client ids kept by the service, and make repeated deliveries harmless with cursors kept by the devices.
  • Keep receipts as two growing numbers per member, and let devices fetch any gap from the store.
  • Treat push notifications as best-effort wake-ups, merged per burst and planned against the providers’ quotas.
  • Delete what every device has stored and anything older than 30 days; most of the storage belongs to devices that are away for days.

That was a complete case study. 67 more case studies have in-depth parts that open with any pass.

References

Related tools

Report a problem with this lesson

Quick answers and tool search

Type to search tools or to get a quick answer, for example 18% of 2500. Use the up and down arrow keys to move through the results, Enter to choose, and Escape to close.