When messaging apps first appeared, a sent message was permanent. SMS had no concept of editing or unsending — once a text left your phone, it existed forever on every recipient’s device, typos and all. Early internet chat (IRC, early instant messengers) inherited the same assumption: messages were an append-only stream, and the only way to “fix” a mistake was to send a follow-up message. This mental model — messages as immutable, append-only events — is actually foundational to how modern systems still work under the hood, even though the user experience has evolved dramatically.

1.1 A Brief History of Editable Messaging

The shift began as messaging platforms started treating conversations less like a transcript and more like a shared, mutable document that multiple parties observe simultaneously. WhatsApp introduced “Delete for Everyone” in 2017, giving users a short window to retract a message from every recipient’s device, not just their own. Telegram went further, allowing edits and deletions with no time limit at all, and iMessage later added the ability to edit a sent message within a 15-minute window, with the recipient seeing both the edit and a visual history of prior versions. Slack and Discord, built for longer-lived, searchable conversations rather than ephemeral chat, allow indefinite editing since messages there often function as a persistent knowledge record.

What all of these products have in common, underneath very different UX decisions, is a hard distributed systems problem: a message, once sent, may already be in flight to — or already displayed on — devices you have no live connection to. An edit or delete is not just a database update; it is a correction that has to be redelivered, tracked, and eventually applied everywhere that message ever existed, including devices that are currently powered off, in airplane mode, or simply have not opened the app in a week. This is the central engineering problem this document works through end to end.

1.2 Why Event Sourcing Is the Right Foundation

There is a deeper reason event sourcing is the right foundation here, beyond convenience: it converts a distributed-consistency problem into a much more tractable replay problem. In a naive in-place-mutation design, “did device A see the same thing as device B” depends on a tangled combination of network timing, which server handled which request, and whatever partial state happened to be visible at each read. In an event-sourced design, that question reduces to “have device A and device B applied the same prefix of the same ordered event stream” — a question you can answer precisely, test deterministically, and reason about with the same tools used to verify replication correctness in databases generally. This reframing is why nearly every serious messaging platform, regardless of their specific product decisions about edit windows or delete semantics, converges on some variant of this architecture under the hood.

1.3 What Makes This Problem Hard

Supporting edits and deletes reliably requires re-framing how you think about “sending a message” in the first place. A message is not just content — it is an entity with a lifecycle, a version history, and a delivery status that must be tracked per recipient, per device.

2.1 Component Responsibilities

2.2 Why a Separate Materialized State Store Instead of Reading the Log Directly

It is tempting to ask: if the event log has everything, why not just read directly from it every time a client needs to render a conversation? The answer is a classic read-versus-write optimization trade-off. The event log is optimized for durable, ordered appends and sequential replay — exactly what sync needs. But rendering “open this conversation right now” needs fast, random-access reads of current state, ideally without replaying potentially thousands of historical events per message just to determine what to show. The materialized state store exists purely to serve that read pattern cheaply, kept eventually consistent with the log through the same fan-out mechanism that delivers events to clients. This is the same read / write-path separation that CQRS (Command Query Responsibility Segregation) formalizes in general distributed-systems design, applied here to a very concrete, very high-traffic use case.

2.3 Why the Cursor Store Deserves Its Own Component

It would be simpler, on paper, to derive “what has this device seen” implicitly from delivery acknowledgments scattered across logs. In practice, that makes correctness nearly impossible to verify or reason about at scale — you would need to reconstruct a device’s exact sync position from scattered evidence every time it reconnects, which is slow and error-prone. Making the cursor an explicit, first-class, directly-queryable piece of state turns “what does this device need next” into a single, cheap lookup, and makes the entire sync protocol auditable: you can inspect any device’s cursor at any time and know exactly what it has and has not seen, which is invaluable both for debugging production issues and for building product features like accurate “delivered” and “seen” indicators.

This is the heart of the design. Let’s walk through exactly what happens, mechanically, when a user edits or deletes a message.

3.1 Messages as an Append-Only Event Stream

The critical design decision that makes this whole problem tractable is: never mutate a message in place at the storage layer. Instead, every action — original send, edit, delete — is recorded as a new, immutable event referencing the original message’s ID. A message’s “current state” is always a derived value, computed by replaying its event history: original content, followed by the latest edit (if any), followed by a delete marker (if any). This is the same event-sourcing principle used in financial ledgers, and for good reason: it gives you a complete, replayable audit trail, and — crucially for this problem — it gives every device, no matter how far behind, a well-defined way to catch up by simply replaying events it has not seen yet, in order.

3.2 Ordering Within a Conversation

Every mutation needs a well-defined position in a total order, scoped per conversation, so that a device applying events out of arrival order (which will happen — network delivery is not guaranteed to preserve order) can still reconstruct the correct final state. The standard approach is a monotonically increasing sequence number assigned by the Message Service at write time, backed by a single ordering authority per conversation shard (not a wall-clock timestamp, which is unreliable across distributed servers due to clock skew). An edit or delete event carries both its own new sequence number and a reference to the original message’s sequence number, so any device can determine “this edit applies to message #482” regardless of when the edit event physically arrives.

3.3 Applying an Edit

1. Sender submits an edit request referencing the original message ID and new content.
2. Message Service validates the request (message exists, sender owns it, still within any platform-defined edit window) and writes a new MessageEdited event to the log with a fresh sequence number.
3. The materialized state store is updated: the message’s current content pointer now points to the new version; the original content is retained in history, not discarded.
4. Fan-Out Service reads the event and determines the recipient set (all active conversation members, all their registered devices).
5. For each recipient device: if online, the Real-Time Gateway pushes the edit event immediately over the open connection. If offline, the event waits in that device’s durable queue and a lightweight “silent” push notification may be sent to encourage reconnection, but delivery of the actual edit content happens through the Sync Service on next connect — not through the push payload itself, since push payloads are size-limited and not delivery-guaranteed.

3.4 Applying a Delete

Delete follows the same event-sourced pattern but the result is a tombstone rather than new content: a MessageDeleted event is written, referencing the original message ID. The materialized state store replaces the message’s renderable content with a deletion marker (e.g., “This message was deleted”), while the event log — and often a separate, access-restricted audit log — retains the original content for compliance, moderation, and abuse-investigation purposes, even though normal clients will never render it again. This distinction between “gone from the log” and “gone from every UI” is important and frequently misunderstood: a true “delete from all storage everywhere” is a much stronger, harder guarantee (and one with real privacy and compliance implications) than “no longer shown to users,” and most platforms deliberately implement the latter while retaining the former internally for a bounded retention period.

3.5 Snapshotting — Bounding the Cost of Deep History

Pure event replay is elegant but does not scale forever: a device offline for a year in a conversation with millions of historical events would face an enormous, slow catch-up. The standard fix is snapshotting — periodically (e.g., after every N events, or on a time-based schedule) the Message Service materializes a full “current state as of sequence number K” snapshot and persists it. A device reconnecting after a long absence fetches the most recent snapshot at or before its cursor, then replays only the small tail of events between the snapshot point and the present — turning an unbounded replay into a bounded one. Snapshots do not replace the event log (which remains the durable source of truth and audit trail); they are a pure performance optimization layered on top, and can even be regenerated from the log if ever lost or corrupted.

3.6 Multi-Device Complications

Modern messaging accounts routinely span multiple devices — phone, tablet, desktop client, web client — all logged into the same account and expected to show an identical, converged view. This means the recipient set for fan-out is not “one device per user,” it is “every registered device per user,” and the sync / cursor machinery described above applies not just across different users in a conversation but across a single user’s own device fleet. A subtlety worth naming: if a user edits a message from their laptop while their phone is offline, the phone must eventually see that edit through exactly the same sync mechanism as any other recipient — there is no privileged “the sender already knows” shortcut, because from the system’s perspective, the laptop and phone are just two more devices that need to converge.

Let’s trace an edit end to end — from sender request through durable append, real-time push to online devices, and eventual catch-up for an offline device that only reconnects days later.

4.1 Lifecycle Stages

Every design here is really a decision about where you spend complexity budget: storage cost vs. replayability, in-memory speed vs. audit trail, delete strength vs. compliance flexibility. Making those explicit turns the design into a defensible engineering choice.

5.1 Key Trade-offs

Assume a platform with 500 million daily active users, averaging 40 messages sent per user per day, with roughly 5% of sent messages later edited and 2% deleted.

6.1 Fan-Out Amplification

A single mutation event does not cost one unit of work — it costs one unit of work per recipient device. A message edited in a 200-person group chat, where the average user has 1.5 active devices, generates roughly 300 individual delivery jobs from a single edit event. This fan-out multiplier is the dominant scaling factor for the Fan-Out Service, and group conversations (not 1:1 chats) are almost always the long tail that drives peak load, since a single popular group can have thousands of members.

6.2 Where the Bottlenecks Actually Are

6.3 Scaling Techniques

6.4 CAP Theorem and the Event Log

During a network partition, the event log’s per-conversation ordering authority faces the same choice as any distributed system: preserve availability by allowing writes on multiple isolated sides of the partition (risking a merge conflict on healing), or preserve strict ordering by refusing writes on the minority side until the partition heals. Because the entire correctness model here depends on a single, unambiguous total order per conversation, this is one of the places in the system where leaning CP (consistent over available) for the ordering authority specifically is usually the right call — a brief write unavailability for one specific conversation shard during a partition is a far more tolerable failure mode than two divergent, unmergeable histories for the same conversation that could later show different users irreconcilably different pasts. Contrast this with something like the cursor store, discussed later, which can safely lean AP.

6.5 Cost Considerations at Scale

Storage cost is driven primarily by three things: raw event volume (sends plus edits plus deletes), how long full event history is retained before being eligible for compaction into snapshots, and how many devices are actively maintaining cursors. A pragmatic cost lever many platforms use is tiered retention: keep full granular event history “hot” for a relatively short window (weeks), then compact older history into periodic snapshots plus a much smaller retained tail, trading a small amount of audit-trail granularity for a large reduction in long-term storage cost — while keeping the compliance-relevant retention window (for legal hold or abuse investigation) explicitly separate and typically longer-lived in a more access-restricted store.

The core reliability requirement here is unusually strict: every recipient must eventually converge on the identical final state, with no permanently divergent views, even across arbitrary network partitions, device outages, and server failures.

7.1 Reliability Techniques

7.2 Failure Mode Table

7.3 Disaster Recovery for the Event Log

Because the event log is the irreplaceable source of truth — unlike the materialized state store or snapshots, which can always be regenerated by replaying it — it deserves the platform’s strongest durability guarantees: cross-region replication, regular backup verification (not just backup creation), and a tested, rehearsed regional failover procedure. A useful design discipline is to treat every other data store in this system (materialized state, cursor store, snapshots) as recoverable-by-construction from the event log, and to size your operational rigor accordingly: maximal care for the log itself, and comparatively lighter-weight recovery procedures for everything derived from it, since “rebuild it by replaying the log” is always available as a fallback.

7.4 Defining an Honest SLA

Similar to any distributed delivery system, it is worth being explicit about what is actually being promised. A defensible SLA here reads something like: “100% of accepted mutations are durably logged and will eventually be delivered to every recipient device once that device is online and syncs, with real-time delivery to already-connected devices within a low single-digit number of seconds under normal operating conditions.” This is a strictly stronger guarantee than the push-notification case discussed in related literature — because delivery here does not depend on an external, uncontrolled gateway’s best-effort final hop, the system can and should commit to eventual, guaranteed delivery, with only the timing of that delivery being variable based on the recipient device’s connectivity.

Message mutations touch content that is intrinsically sensitive, so authorization, retention control, and tamper-evidence all have to be first-class concerns, not afterthoughts.

8.1 Threat Modeling the Sync Path Specifically

Beyond authorization on individual mutations, it is worth threat-modeling the Sync API itself, since it is a high-value target: an attacker who could forge or manipulate cursor values might attempt to trigger excessive replay (a denial-of-service vector against the Sync Service) or, worse, attempt to fetch events for conversations they were never a member of by guessing or brute-forcing conversation identifiers. Defending against this means the Sync API must independently verify conversation membership for the requesting user on every call — never assuming that possessing a valid cursor implies valid access — and should rate-limit sync requests per device to blunt any attempt to use repeated full-history replay as an amplification or resource-exhaustion attack.

8.2 Privacy-by-Design for Retained Content

Because deleted content that is internally retained for abuse investigation is, functionally, a liability if mishandled, it is worth designing its access path with the same rigor as any other sensitive data category: encryption at rest with keys scoped separately from normal message content, mandatory access logging so every read of retained deleted content is itself auditable, and an automatic, enforced expiry so retained content does not silently accumulate indefinitely beyond whatever policy justified keeping it in the first place. Treating this store as an afterthought bolted onto the main event log is a common early-stage mistake that becomes expensive to unwind once compliance or security review catches up with a growing platform.

Convergence bugs are silent by nature — two devices quietly disagreeing about what a message says is exactly the class of failure users report late, if at all. Instrumentation has to be designed to surface that class of problem long before a support ticket arrives.

9.1 Key Metrics

9.2 Logging & Tracing

Every mutation event should carry a trace ID propagated from the original client request through log append, fan-out, and every downstream delivery attempt, so a support escalation like “my friend still sees my old message, I edited it an hour ago” can be root-caused precisely — was the edit delivered to their device, did their client fail to apply it, or is their cursor stuck?

The stateless and stateful tiers here have very different operational profiles, and treating them uniformly is the fastest way to break sync correctness during a routine deploy.

Different pieces of state in this system have wildly different durability and read-pattern requirements, and matching each to the right store is what keeps costs bounded and behavior predictable.

11.1 Event Log Storage

The event log favors an append-optimized, horizontally partitioned store — a distributed log system (Kafka-like) or a database explicitly designed for ordered, immutable event streams, partitioned by conversation ID for both write scalability and to keep a conversation’s full ordered history colocated for fast replay.

11.2 Materialized State Store

The current-state store is read-optimized, typically a key-value or document store keyed by message ID (or conversation ID + message ID), rebuilt incrementally as events are applied — this is what powers the fast “open a conversation and see current content instantly” experience without replaying history on every read.

11.3 Cursor Store

A high-write-throughput key-value store keyed by (device ID, conversation ID), storing just the last-applied sequence number. Given this is written extremely frequently (every device, every event, potentially batched), it benefits from a store optimized for high write QPS with relaxed durability-per-write requirements — losing the very latest cursor advance in a rare crash just means a small amount of redundant replay next reconnect, not a correctness problem.

11.4 Caching

11.5 Load Balancing

Layer-7 balancing for stateless API traffic; consistent-hashing-based routing for the stateful Real-Time Gateway tier, ensuring even connection distribution while preserving the sticky affinity each connection needs for its lifetime.

The public surface is deliberately narrow: mutations in, sync out, and a small set of specialized internal channels for presence, moderation, and audit.

These services communicate primarily through the event log itself, which doubles as both the durability mechanism and the primary internal message bus — a design that reduces the number of distinct integration points other services need to reason about.

The whole system is essentially a careful stack of well-understood patterns — and a matching set of anti-patterns it deliberately refuses to use.

13.1 Patterns That Work

13.2 Anti-patterns to Avoid

The disciplines below are the difference between a design that works in the demo and a design that keeps working at billion-message scale, three years and many client versions later.

14.1 Testing Convergence, Not Just Delivery

Standard integration tests tend to check “was the message delivered,” which is necessary but not sufficient here — the harder, more valuable test is “do N independently-simulated devices, each following a different, randomized pattern of online / offline transitions and reconnect timing, all converge on byte-identical final state after a sequence of interleaved sends, edits, and deletes.” Building this kind of property-based, randomized convergence test early — effectively a chaos test for the sync protocol itself — catches an entire class of subtle ordering and idempotency bugs that conventional request / response testing simply cannot reach, since those bugs only manifest from specific, unusual interleavings of concurrent mutations and device connectivity states.

Very different products, very similar underlying architecture — the mutation-consistency problem is the same regardless of whether the surface UX is ephemeral chat or a permanent team knowledge base.

15.1 A Cross-Platform Pattern, Not a Single Vendor’s Trick

What is instructive across these five examples is that none of them arrived at event-sourced, cursor-based sync because it was fashionable — each was independently pushed toward it by the same underlying constraint: users expect a conversation to look identical no matter which device or how much time has passed since they last opened the app. That constraint does not change based on whether the product optimizes for ephemeral personal chat (WhatsApp), permanent searchable records (Slack), or large public communities (Discord) — only the product-level policy choices layered on top (edit windows, retention periods, who can moderate) differ. This is a useful lens for an interview: when asked to design “a messaging feature,” look past the product-specific UI decisions to the underlying convergence guarantee being asked for, since that is almost always the harder and more transferable engineering problem.

The questions that come up most often on this design — and the crisp answers that show a clear understanding of where the real correctness guarantees live.

A compact recap of the whole design, and the mental model worth carrying into any future system where many independent, sometimes-offline observers must converge on a shared, mutable truth.

The recurring theme across this entire design is that “consistency for offline users” is not a special feature bolted onto a messaging system — it is what falls out naturally once you commit to treating every message mutation as a durable, ordered, replayable event rather than a database row you overwrite. Once that commitment is made, online delivery, offline catch-up, multi-device sync, and even audit / compliance requirements all become variations of the same underlying mechanism, rather than separate problems each requiring their own bespoke solution.

For an interview setting specifically, the strongest signal you can give is walking through why each piece exists in terms of the failure mode it prevents — sequence numbers exist because clocks skew, cursors exist because “since when” needs a precise answer, snapshots exist because unbounded replay does not scale, and idempotent application exists because at-least-once delivery is the only realistic guarantee a distributed system can make. A candidate who can trace each component back to the specific correctness or scalability problem it solves demonstrates a genuinely deeper understanding than one who can only describe the boxes and arrows in isolation.