Two of the smallest-looking features in any messaging app — the green dot next to a contact’s name, and the “typing…” bubble that appears while someone composes a reply — are, underneath the surface, among the hardest real-time distributed systems problems a messaging platform has to solve. They look trivial because the UI is trivial: a dot, a label, three animated dots. But building them correctly for hundreds of millions of users, each holding open a persistent connection that can drop at any moment, on flaky mobile networks, across multiple devices per person, is a genuinely difficult engineering problem that touches almost every hard topic in distributed systems: connection management at extreme scale, fan-out, eventual consistency, heartbeats, and the CAP theorem in a very concrete, user-visible way.

In the early days of instant messaging — think desktop clients like ICQ, MSN Messenger, and early Yahoo Messenger in the late 1990s and 2000s — presence was relatively simple because scale was smaller and clients were mostly desktop applications with stable, long-lived TCP connections. A single presence server could track a few million users without much trouble. As messaging moved to mobile — with users constantly switching between Wi-Fi and cellular, backgrounding apps, losing signal in elevators and tunnels — presence became dramatically harder. A “connection” is no longer a reliable signal of “the user is present”; a mobile app can hold a socket open while backgrounded, or can silently drop a connection without either side immediately knowing. Group chats and large channels made the fan-out problem — telling everyone in a 500-person group who else is currently typing — a genuine scaling challenge, not just a broadcast to a handful of friends.

Modern messaging platforms (WhatsApp, Messenger, Slack, Discord, Telegram) serve hundreds of millions to billions of users, many of whom are connected simultaneously at any given moment, and they treat presence and typing indicators as first-class, heavily-optimized subsystems — not afterthoughts bolted onto the chat backend. This tutorial designs such a system from first principles: how to track “online,” “away,” and “last seen,” how to propagate an ephemeral “typing…” signal to the right people within tens of milliseconds, and how to do all of this without melting your database under the write load of hundreds of millions of users whose online status changes constantly.

It’s worth spending a moment on why this specific pair of features gets so much dedicated engineering attention at large messaging companies rather than being treated as a minor UI flourish. Presence and typing indicators are, in aggregate, by far the highest-frequency events flowing through a messaging platform’s real-time infrastructure — a typical user sends only a handful of actual messages per session, but their client sends dozens of heartbeats and potentially dozens of typing events in the same window. If this traffic were routed through the same systems built to guarantee reliable message delivery, it would dominate capacity planning and cost for a feature that, unlike message delivery, has no correctness requirement at all — nobody has ever filed a support ticket because a typing indicator was one second late. That asymmetry — extremely high volume, extremely low value-per-event, zero durability requirement — is exactly what makes this such a distinctive and interesting system design problem, because it rewards a completely different set of engineering instincts than the “make every write durable and consistent” instincts that dominate most backend systems.

1.1 Why this system matters for an architect

Presence and typing indicators sit at the intersection of five hard sub-problems that architects are expected to reason about together:

• Persistent-connection management at extreme scale — tens of millions of concurrent WebSockets are a different capacity-planning problem than any request-per-second system.
• Failure detection over unreliable networks — a “silent” connection and a dead one look identical unless you probe.
• Massive fan-out with skewed audience sizes — most audiences are tiny, but a few are enormous, and the same code path must serve both without pricing itself out of business.
• Ephemeral, high-volume, low-value-per-event traffic — the cost-per-event math is unforgiving, so architectural choices matter far more than instance-type tuning.
• Privacy and reciprocity constraints — who is allowed to see whose activity is itself a product decision, not just a technical filter.

Before designing anything, it’s worth being precise about what “presence” and “typing indicator” actually mean as distinct problems, because they have very different consistency, durability, and latency requirements — a distinction that is easy to miss and that fundamentally shapes the architecture.

2.1 Presence vs. typing indicators: two different problems

This distinction matters because it means the two features, despite looking similar in the UI, are best served by architecturally different data paths: presence benefits from being cached, batched, and eventually-consistent, while typing indicators are pure ephemeral pub/sub events that should never touch a durable database at all in the common case.

2.2 The core hard problem: distinguishing “disconnected” from “silent”

The fundamental difficulty in presence systems is that a dropped TCP/WebSocket connection and a client that is simply not sending anything right now look identical from the server’s point of view unless you actively probe for the difference. A mobile client can lose connectivity in a tunnel without either side receiving an explicit “goodbye” — the server has to infer offline status from the absence of expected signals (heartbeats), which is inherently a timing/threshold problem, not a binary fact. This is fundamentally different from most distributed-systems failure detection problems, where you can often afford to wait a long time before declaring a node dead; here, the whole point of the feature is to reflect reality quickly, so the grace period has to be short enough to feel responsive while still being long enough to tolerate ordinary transient network hiccups (a brief cellular handoff between towers, a few seconds of Wi-Fi-to-cellular transition) without generating false “offline” flickers.

3.1 Functional requirements

• Track each user’s online/offline/away status in real time, visible to their contacts or group members.
• Support “last seen at [time]” for users who are currently offline.
• Propagate a “typing…” indicator to other participants in a specific conversation within a low, near-instant latency, and automatically clear it after a short timeout of inactivity.
• Support multiple simultaneous device sessions per user (phone, desktop, web) with correct aggregate presence (user is “online” if any device is connected).
• Support both 1:1 conversations and large group conversations/channels (up to thousands of members) for both features.
• Allow users to control presence visibility (privacy settings — e.g., “last seen” hidden from non-contacts).

3.2 Non-functional requirements

• Scale: Hundreds of millions of registered users, tens to hundreds of millions of concurrently connected sessions at peak, billions of presence state changes and typing events per day.
• Latency: Typing indicator delivery under ~200 ms end-to-end for a snappy feel; presence updates can tolerate a few seconds of propagation delay.
• Consistency: Eventual consistency is fully acceptable for both features — this is one of the rare subsystems in a messaging platform where strict consistency is actively the wrong goal, since it would add unnecessary latency and cost for a UI signal users already intuitively understand as “approximate.”
• Availability: The core messaging path (sending/receiving actual messages) must never depend on presence/typing subsystems — these are strictly best-effort, additive features, and their failure must never degrade core message delivery.
• Cost efficiency: Because presence/typing events are extremely high volume but low value-per-event, the system must be designed for very low marginal cost per event — this is not a subsystem where you can afford to write every state change durably to a relational database.

The architecture centers on persistent WebSocket (or similar bidirectional) connections held open between clients and a fleet of connection-gateway servers, an in-memory presence store, and a lightweight pub/sub fan-out layer — deliberately kept separate from the durable message-storage path used for actual chat messages.

4.1 Component-by-component breakdown

5.1 Why gateway servers are stateful (and why that’s unusual)

Most modern backend services are deliberately designed to be stateless, so that any request can be routed to any instance and instances can be freely added, removed, or restarted without coordination. The connection gateway fleet in this system is a deliberate exception, and it’s worth understanding why that exception is necessary rather than a design flaw. A WebSocket connection is, by its nature, a long-lived, stateful binding between a specific client and a specific server process holding the open socket — there is no way to “load balance” an already-established connection’s traffic across multiple servers the way you can with independent HTTP requests. This means the gateway fleet has to be treated with different operational discipline than the rest of the platform: capacity planning is driven by concurrent connection count rather than requests-per-second, deploys require connection draining rather than simple instance replacement, and routing any message to a specific user requires first consulting the Connection Registry to find which specific node currently holds their socket.

5.2 How a client becomes “online”

1. The client establishes a persistent WebSocket connection to a gateway node (chosen via a load balancer, often with some geographic/latency-aware routing).
2. On successful connection and authentication, the gateway registers the (user ID, device ID) → gateway node mapping in the Connection Registry, and writes an “online” entry with a short TTL into the Presence Store.
3. The client begins sending periodic heartbeat pings (commonly every 15–30 seconds) over the same connection; each heartbeat refreshes the TTL on the presence entry, effectively acting as a lease renewal.
4. The Presence Manager computes aggregate status: if any of a user’s registered devices has a live “online” entry, the user’s externally-visible status is “online”; if all devices’ entries have expired or been explicitly removed, the user transitions to “offline” and a last-seen timestamp is recorded.

5.3 How “offline” is detected

There are two paths to offline, and a robust design needs both:

• Graceful disconnect: The client sends an explicit close frame (app backgrounded cleanly, user logged out), and the gateway immediately clears the presence entry and notifies the fan-out layer — this is the fast, clean path.
• Ungraceful disconnect (the hard case): The connection simply drops — network loss, app killed by the OS, device battery died — with no close frame. Here, the system relies entirely on the TTL expiring: since every presence entry has a TTL refreshed only by heartbeats, the absence of a heartbeat within the grace window causes the entry to expire automatically, and this expiry itself (using Redis keyspace notifications or a similar expiry-event mechanism) triggers the offline transition and fan-out, without any polling loop needed to “notice” the absence.

public void onHeartbeat(String userId, String deviceId) {
    String key = presenceKey(userId, deviceId);
    // Idempotent lease renewal - no read needed, just extend the TTL.
    presenceStore.setWithTtl(key, ONLINE_MARKER, HEARTBEAT_TTL);
}

public void onGracefulDisconnect(String userId, String deviceId) {
    presenceStore.delete(presenceKey(userId, deviceId));
    fanOut.publish(new PresenceChanged(userId, computeAggregate(userId)));
}

// Redis keyspace notification handler - fires when any device key expires.
public void onKeyExpired(String expiredKey) {
    UserDevice ud = parsePresenceKey(expiredKey);
    fanOut.publish(new PresenceChanged(ud.userId, computeAggregate(ud.userId)));
    lastSeenBuffer.enqueue(ud.userId, Instant.now()); // batched flush
}

5.4 How typing indicators flow

1. As the user types, the client debounces keystrokes locally and sends a lightweight “typing” event to its gateway connection at most once every couple of seconds (never on every keystroke, to control volume) while actively typing, and a “stopped typing” event after a short pause (commonly 3–5 seconds of inactivity) or on message send.
2. The gateway forwards this event to the Typing Event Router, tagged with the conversation ID.
3. The router looks up which users are currently active participants viewing that conversation (not necessarily all group members — many clients only fan out typing indicators to members who have the conversation currently open) and publishes the event to the pub/sub layer keyed by conversation ID.
4. The pub/sub layer delivers the event to the gateway nodes holding connections for each relevant recipient, which push it down each recipient’s WebSocket.
5. On the receiving client, a local timer automatically clears the “typing…” UI after a few seconds even without an explicit “stopped typing” event — a crucial client-side safety net for the ungraceful-disconnect case, where the typer’s device drops without ever sending “stopped typing.”

6.1 Presence lifecycle across a session

• Connect: Presence entry created with short TTL; contacts/subscribers notified of “now online” via the fan-out layer (often with a small debounce to avoid flooding contacts during flaky reconnect loops).
• Active session: TTL continuously refreshed by heartbeats; no further fan-out needed unless status changes (e.g., online → away after a period of client-detected inactivity).
• Multi-device transitions: If a second device connects while the first is still active, aggregate presence remains “online” with no visible change; only when the last remaining device disconnects does the aggregate flip to “offline.”
• Disconnect / TTL expiry: Presence entry removed; last-seen timestamp asynchronously persisted; “now offline” fan-out to subscribers.
• Reconnect storms: After events like a regional network outage or app update rollout, millions of clients can reconnect within seconds — the system must handle this “thundering herd” of simultaneous connection and presence-write requests without falling over, discussed further in the scalability section.

7.1 CAP theorem applied to presence

Presence is one of the cleanest real-world illustrations of choosing availability and partition tolerance over strict consistency (AP over CP), and it’s worth being explicit about why. During a network partition between regions, a strictly consistent design would have to either block presence reads/writes until the partition healed, or risk serving contradictory answers — neither is acceptable for a feature users expect to always render something instantly. The AP choice means that during a partition, each region simply serves its own local view of presence (which may briefly disagree with another region’s view for cross-region contacts), and the two views reconcile naturally within one heartbeat cycle after the partition heals, with no manual conflict resolution required because the lease model has no conflicting writes to reconcile in the first place — only the most recent heartbeat matters, and stale entries expire on their own.

7.2 TTL-based lease expiry as the core primitive

The single most important algorithmic idea in this entire system is representing “online” as a lease with a TTL rather than as a persisted boolean flag that something has to remember to flip. This turns “detect that a user silently went offline” from an active polling/scanning problem into a passive expiry problem that the key-value store’s own engine handles efficiently — Redis, for example, uses a combination of lazy expiry (checked on access) and an active background sweep (a small random sample of keys checked periodically) to reclaim expired keys without a linear scan of the entire keyspace.

7.3 Consistent hashing for connection routing

The Connection Registry and Presence Store are sharded using consistent hashing over user ID, so that a given user’s presence data and connection mapping always land on a predictable, small set of nodes. This matters enormously at rebalancing time: when a node is added or removed from the fleet (routine scaling or a node failure), consistent hashing ensures only a small fraction of keys need to move, rather than a full reshuffle that would momentarily disrupt presence tracking for a huge fraction of users. A common refinement is to use virtual nodes (many logical shard positions per physical node) so that load distributes evenly even with a modest number of physical nodes, avoiding hotspots that plain consistent hashing can otherwise produce.

7.4 Debouncing and throttling for typing events

Naively sending a typing event on every keystroke would multiply write volume by the average word length, so clients apply local debouncing — collapsing a burst of keystrokes into at most one “typing” event per short interval (e.g., one event per 2 seconds while continuously typing), plus a single “stopped” event after a pause. This is a classic debounce/throttle pattern from event-driven systems, applied at the source to protect every downstream component from unnecessary load. Server-side, a secondary rate limiter acts as a backstop against clients that don’t debounce correctly (buggy clients, malicious clients, or older app versions), ensuring the fan-out layer’s load is bounded regardless of client behavior.

const TYPING_MIN_GAP_MS = 2000;
const STOPPED_AFTER_IDLE_MS = 4000;
let lastTypingSentAt = 0;
let idleTimer: number | null = null;

function onKeystroke(conversationId: string) {
  const now = Date.now();
  if (now - lastTypingSentAt > TYPING_MIN_GAP_MS) {
    ws.send({ type: 'typing', conversationId, state: 'start' });
    lastTypingSentAt = now;
  }
  if (idleTimer) clearTimeout(idleTimer);
  idleTimer = window.setTimeout(() => {
    ws.send({ type: 'typing', conversationId, state: 'stop' });
    lastTypingSentAt = 0;
  }, STOPPED_AFTER_IDLE_MS);
}

7.5 Fan-out: push vs. pull, and the “celebrity problem”

For 1:1 chats and small groups, a push-based fan-out (actively delivering the event to every subscriber’s gateway connection) is simple and cheap. For extremely large groups or public/celebrity accounts with millions of followers whose presence contacts want to see, naive push fan-out becomes a “thundering herd” write amplification problem — one status change fanning out to millions of connections instantly. The mitigation is a hybrid approach: for very large audiences, presence is served pull-style (a client queries current status when it renders the relevant UI, rather than being pushed every change), while push fan-out is reserved for small, bounded audiences like direct contacts and typing indicators in an actively-open conversation. A useful mental model borrowed from social-graph systems is the same one used for post fan-out on very large accounts: fan-out-on-write works well until audience size crosses a threshold, past which fan-out-on-read (pull) becomes cheaper, and a well-designed presence system dynamically picks the strategy per account based on observed audience size.

7.6 Bitmap/bitset representation for multi-device aggregation

For users with several registered devices, aggregate presence can be efficiently represented as a small bitset (one bit per active device session), where the aggregate “online” status is simply a bitwise OR across all bits — an extremely cheap operation that avoids needing to enumerate and check every device record on every presence query.

7.7 Approximate counting for aggregate metrics

Product and infrastructure dashboards often need aggregate figures like “concurrent online users right now” across the whole platform. Computing this exactly across every regional shard on every request is unnecessarily expensive for a number that only needs to be roughly right; approximate structures like HyperLogLog, or simply periodic sampled aggregation from each shard rolled up on a fixed interval, provide a good-enough answer at a tiny fraction of the cost of an exact global count.

8.1 Why not just use the primary relational database?

This is one of the most common design mistakes for engineers new to this problem: routing presence/typing writes through the same relational database used for user profiles and message metadata. At hundreds of millions of users with connect/disconnect/heartbeat events happening constantly, this write volume would overwhelm a relational database’s transactional write path almost immediately, and none of that data actually needs relational guarantees (foreign keys, joins, ACID transactions) — it needs raw throughput and automatic expiry, which is exactly what an in-memory store with TTL support is built for.

8.2 Caching strategy

Because presence status is read far more often than it changes (many clients render a contact list showing dozens of people’s status on every app open), a read-through cache in front of the presence store — or simply relying on the presence store itself being in-memory — keeps read latency low. For the pull-based “celebrity” presence path, an additional short-TTL cache (a few seconds) absorbs read bursts when a popular account’s profile is viewed by many users simultaneously, trading a few seconds of staleness for a massive reduction in backend query volume.

8.3 Batched writes for the last-seen path

Even the last-seen durable store, which is the only component that touches disk, does not receive individual writes per offline event. Instead, offline events are buffered in memory per node and flushed as a batched write every few seconds, dramatically reducing write amplification at the durable store while keeping the observable staleness of “last seen” timestamps well within the level users perceive as accurate.

• Connection/Gateway Protocol (WebSocket): Not a traditional REST API — a persistent bidirectional protocol carrying connection setup, heartbeats, typing events, and presence subscription/unsubscription messages as lightweight framed messages.
• Presence Query API (used for the pull-based large-audience path): GET /v1/presence/{user_id} — returns current status and last-seen timestamp, served from cache wherever possible.
• Typing Event API (sent over the existing WebSocket connection, not a separate REST call): a small framed message type, e.g. {type: "typing", conversation_id, state: "start"|"stop"}.
• Presence Subscription API: Allows a client to subscribe to presence updates for a bounded set of contacts (its visible contact list), so the fan-out service only pushes updates to clients that have actually expressed interest, rather than broadcasting every user’s status globally.
• Privacy Settings API: CRUD endpoints for a user’s presence-visibility preferences (e.g., “hide last seen from everyone except contacts”), consulted by the Presence Manager before including a user’s status in any fan-out or query response.

9.1 Sample API contracts

GET /v1/presence/u_9f8a11
Response: {
  "userId": "u_9f8a11",
  "status": "online",           // "online" | "away" | "offline"
  "lastSeenAt": null,           // populated only when status = "offline"
  "visibilityScope": "contacts" // caller-scoped, respecting privacy settings
}

// Client -> Server (typing)
{ "type": "typing", "conversationId": "c_88a...", "state": "start" }
{ "type": "typing", "conversationId": "c_88a...", "state": "stop"  }

// Client -> Server (heartbeat)
{ "type": "heartbeat", "ts": 1723371840 }

// Server -> Client (presence change fanned out to a subscriber)
{ "type": "presence", "userId": "u_9f8a11", "status": "offline",
  "lastSeenAt": "2026-08-11T18:22:14Z" }

The Connection Gateway fleet, Presence Manager, and Typing Event Router are deliberately kept as separate, independently-scalable services from the core message-storage and message-delivery services, so that a spike in typing/presence traffic (which is naturally much higher volume than actual message send volume) never contends for capacity with the guaranteed-delivery message path.

10.1 Patterns used

10.2 Anti-patterns to avoid

At hundreds of millions of users with tens to hundreds of millions of concurrent connections, three dimensions dominate the scaling conversation: the number of concurrently held connections per gateway node, the write throughput of the presence store under constant heartbeat traffic, and fan-out amplification for popular conversations and accounts.

• Connection density per node: Modern event-driven network stacks (using epoll/io_uring-style asynchronous I/O rather than one OS thread per connection) allow a single gateway node to hold hundreds of thousands of concurrent idle WebSocket connections, since most connections are idle most of the time between heartbeats and events.
• Horizontal gateway scaling: The gateway fleet scales horizontally by simply adding more nodes behind the load balancer; because connections are long-lived, scaling events (adding capacity) do not disrupt existing connections, but scaling down requires a careful, gradual connection-draining process to avoid mass-disconnecting users.
• Sharding the presence store: Consistent-hash sharding across many in-memory store nodes spreads both the heartbeat write load and the read load for presence queries, with shard count sized to keep per-shard throughput well within the store’s comfortable operating range.
• Heartbeat interval tuning: The heartbeat interval is a direct scalability lever — a longer interval (e.g., 30 s instead of 10 s) cuts presence-store write volume by 3x at the cost of a slower offline-detection window; this is a tunable trade-off platforms adjust based on observed load and acceptable staleness.
• Regional deployment: Gateway fleets and presence stores are deployed regionally close to users to minimize connection latency, with cross-region replication only for the data that genuinely needs global visibility (e.g., a user’s aggregate online status, if they have contacts across regions), keeping the vast majority of heartbeat and typing traffic entirely regional. This regional-first approach also naturally aligns with data-residency requirements in some markets, since presence data for a region’s users never needs to leave that region except for the small aggregate status fact required by cross-region contacts.

11.1 Capacity math walkthrough

Concrete numbers help calibrate intuition. Suppose the platform sustains $100$ million concurrent connections with a heartbeat every $20$ seconds. That is a global heartbeat rate of $100{,}000{,}000 / 20 = 5{,}000{,}000$ writes/second against the presence store — small per-shard if sharded across, say, $500$ nodes ($10{,}000$ writes/s per shard, well within a comfortable Redis operating range). Doubling the heartbeat interval to $40$ seconds cuts that to $2.5$ million/second globally, or $5{,}000$ per shard — a direct, linear cost lever the operator controls.

• Gateway fleet redundancy: Many gateway nodes behind a load balancer with health checks; a node failure only affects the connections it was directly holding, which reconnect automatically (with jittered backoff) to a healthy node.
• Presence store replication: Each shard is replicated (e.g., primary plus replicas) so a single node failure does not wipe out presence data for the users on that shard; given the short TTL nature of the data, even a brief data-loss window on failover is self-healing within one heartbeat interval as clients re-register.
• Self-healing by design: This is one of the rare subsystems where the data model itself provides a natural recovery mechanism — because every presence fact is really a short-lived lease continuously refreshed by the client, the system recovers correct state automatically within one heartbeat cycle after almost any failure, without needing complex reconciliation logic.
• Decoupling from core messaging reliability: As emphasized throughout, the architecture deliberately ensures that a full outage of the presence/typing subsystem degrades only cosmetic features, never message delivery — this is a conscious reliability-budget allocation decision, not an accident.
• Circuit breaking on fan-out: If the pub/sub layer becomes overloaded, typing-event delivery is the first thing shed (clients simply stop seeing typing indicators temporarily) while presence updates, being lower frequency, continue to be served.
• Replica promotion on shard failure: When a presence-store primary shard fails, a replica is promoted automatically (via the store’s built-in failover mechanism, such as Redis Sentinel or a cluster-mode equivalent), and because the data is short-lived and constantly refreshed, even a brief gap during failover is invisible to users — the next heartbeat from each affected client simply re-populates the new primary.
• Load shedding under extreme spike: During an unusually large reconnect storm (e.g., following a major client app update pushed to hundreds of millions of devices simultaneously), the gateway layer can temporarily shed non-essential presence fan-out entirely while still accepting and authenticating connections, ensuring users can at least send and receive messages even if their online status momentarily lags.

• Connection authentication: Every WebSocket connection is authenticated at handshake time (session token/JWT validated by the gateway) before any presence or typing data is accepted or served.
• Authorization for presence visibility: The Presence Manager enforces each user’s privacy settings before including their status in any query response or fan-out — a user who has hidden “last seen” from non-contacts must never leak that data through a presence query.
• Rate limiting on typing/presence events: Per-connection rate limits prevent a malicious or buggy client from flooding the system with excessive typing events, protecting the fan-out layer from abuse-driven overload.
• Encrypted transport: All WebSocket connections use TLS; no presence or typing metadata is ever transmitted in the clear, since even “user X is typing” can be sensitive metadata (e.g., revealing someone is awake and active at an unusual hour).
• Metadata minimization: Typing and presence events carry only the minimum necessary identifiers (user ID, conversation ID) and are never logged with full message content, keeping the security surface of this subsystem deliberately narrow.
• Protection against presence enumeration attacks: Without safeguards, a malicious actor could probe the presence query API against a large list of phone numbers or user IDs to build a profile of who is currently active (a real privacy concern that has affected several messaging platforms historically). Mitigations include strict authorization checks (only contacts can query presence), aggressive rate limiting per querying account, and anomaly detection on query patterns that resemble bulk enumeration rather than normal contact-list rendering.
• Denial-of-service resilience at the connection layer: Because the gateway fleet accepts a very large number of long-lived connections, it is a natural target for connection-exhaustion attacks; protections include per-IP connection limits, requiring authentication before a connection consumes significant server-side resources, and SYN-flood-resistant load balancing in front of the gateway tier.

13.1 Privacy controls as a first-class design concern

Presence is unusually privacy-sensitive compared to most metadata a messaging platform handles, because “is this person currently active” is itself a meaningful piece of information about someone’s life — whether they are awake, at work, avoiding a specific contact, or currently available. This is why mature platforms treat presence-visibility settings (who can see my online status, who can see my last-seen time, whether typing indicators are shown at all) as first-class, per-relationship privacy controls enforced consistently at the Presence Manager layer, rather than as an afterthought bolted onto the UI. A common design detail worth calling out: many platforms tie last-seen visibility reciprocally — if you hide your last-seen from others, you also lose the ability to see theirs — which is a product policy decision enforced at exactly the same authorization checkpoint as the technical privacy filtering.

14.1 Key metrics to track

14.2 Observability practices

• Synthetic canary connections: Continuously-running test clients that measure real end-to-end typing-indicator latency from a known sender to a known receiver, catching regional or fleet-wide degradation before it’s reported by real users.
• Per-region dashboards: Since connections and presence are largely regional, dashboards broken out by region make it easy to spot a single-region degradation rather than only seeing a diluted global average.
• Heartbeat interval vs. actual offline-detection-time tracking: Monitoring the real observed delay between a client actually disconnecting and the system marking it offline, to validate that the TTL/grace-period tuning is behaving as designed under real-world network conditions.
• Connection churn correlation with client releases: Tracking connection drop rates segmented by client app version and platform (iOS/Android/web) helps catch a buggy client release that reconnects too aggressively or fails to send heartbeats correctly, before it degrades the whole fleet’s load profile.
• Shard hot-key detection: Monitoring for individual presence-store shards receiving disproportionate load (often caused by a small number of extremely popular accounts landing on the same shard) so that hot shards can be split or given dedicated capacity before they become a bottleneck.

14.3 Alerting philosophy

Alerts are tiered by user impact, not just raw metric threshold:

• P1 (page immediately): mass connection drop rate above a fleet-wide threshold, presence-store shard down without replica promotion, cross-region sync completely stalled.
• P2 (urgent): single-region typing latency regression above SLA, single shard’s hot-key ratio outside expected band.
• P3 (informational): minor auto-scaling events, small latency drift within SLA.

• Regional, multi-cluster deployment: Gateway fleets and presence-store shards are deployed per region close to users, minimizing connection round-trip latency and keeping the vast majority of traffic local.
• Rolling, connection-draining deployments: Because gateway nodes hold long-lived stateful connections, deploys use a graceful drain process — new connections stop being routed to a node marked for replacement, existing connections are asked to reconnect gracefully (often with a brief delay to spread the reconnect load), and the old node is only terminated once drained.
• Auto-scaling on connection count and CPU: Gateway fleets auto-scale primarily on live connection count and CPU utilization from the async I/O event loop, rather than traditional request-rate metrics, since a gateway node’s load is dominated by held-open connections, not discrete requests.
• Cost optimization: Because this subsystem is extremely high volume but individually cheap per event, cost efficiency comes primarily from tuning heartbeat intervals, debounce windows, and choosing in-memory stores over durable ones wherever durability isn’t actually needed — architectural choices matter far more here than instance-type tuning.
• Blast-radius containment: Deployment topology deliberately limits how much of the user base a single bad deploy or configuration change can affect at once — canary rollouts to a small percentage of gateway nodes in a single region first, with automated rollback triggers tied to connection-error-rate and latency regressions, before a change is promoted fleet-wide.

15.1 Infrastructure as code

The full topology — gateway pools, presence-store shards, pub/sub topics, IAM policies, and load balancer configuration — is declared in version-controlled infrastructure-as-code so any region can be re-created reproducibly. This matters both for disaster recovery and for expanding into a new region without hand-configuring dozens of moving parts under time pressure.

17.1 Best practices

• Model presence as a lease/TTL, never as a manually-managed boolean flag that something has to remember to flip on disconnect.
• Keep the typing-indicator path entirely in-memory and database-free — durability adds cost and latency for zero user-visible benefit on a signal with a multi-second useful lifespan.
• Debounce and throttle at the client, the earliest possible point, to protect every downstream system from unnecessary load.
• Always implement a client-side safety-net timeout that clears a stale typing indicator even if the “stopped” event never arrives — never rely solely on the sender’s cooperation.
• Architecturally isolate presence/typing infrastructure from the durable message-delivery path so a failure in one never threatens the other.
• Use a hybrid push/pull fan-out strategy so a small number of very-large-audience accounts do not dictate the cost profile of the entire system.
• Choose heartbeat and debounce intervals deliberately as explicit product/cost trade-offs, and revisit them as the user base grows rather than treating early defaults as permanent.
• Prefer regional autonomy over global synchronous coordination — let each region serve its own local presence view and reconcile asynchronously, rather than paying cross-region latency on every heartbeat.

17.2 Common mistakes

• Writing every heartbeat or typing keystroke to a relational database, which cannot sustain the write volume at hundreds of millions of users.
• Assuming a TCP/WebSocket connection being open is sufficient proof of “online,” without accounting for silently dead connections on mobile networks.
• Broadcasting presence changes globally instead of only to interested subscribers, wasting enormous bandwidth at scale.
• Coupling gateway deploy/restart cycles to abrupt connection termination instead of graceful draining, causing visible disruption (mass “offline” flickers) during routine deployments.
• Ignoring multi-device aggregation logic, leading to a user’s status flickering online/offline as they switch between phone and desktop rather than showing a stable aggregate status.
• Skipping hysteresis on offline transitions, causing brief network hiccups to produce visible “offline…online…offline” flicker to every contact.

20.1 Key takeaways to carry into an interview

• Presence and typing indicators look like trivial UI features but are, underneath, hard distributed-systems problems involving massive persistent-connection fan-out, eventual consistency, and failure detection over unreliable networks.
• The single most important design idea is treating presence as a TTL-backed lease rather than a durably-persisted flag — this makes the system self-healing by construction and avoids an entire category of “who resets this on failure” bugs.
• Typing indicators should be treated as pure, ephemeral, in-memory pub/sub events with zero durability requirement, protected by both server-side debouncing and client-side safety-net timeouts.
• Fan-out strategy must adapt to audience size: push for small, bounded audiences (contacts, active conversation participants) and pull-with-caching for large, unbounded audiences (celebrity/business accounts) to avoid the “thundering herd” cost of naive global broadcast.
• This subsystem should be architecturally isolated from the durable, guaranteed-delivery message-storage path, so failures here degrade only a cosmetic feature and never threaten core messaging reliability.
• At hundreds of millions of users, the scaling levers that matter most are connection density per gateway node, heartbeat interval tuning, and consistent-hash sharding of presence state — not exotic new algorithms, but disciplined application of well-understood distributed-systems fundamentals to an extremely high-volume, low-value-per-event workload.

20.2 The one idea to remember

If you take one architectural lesson from this guide into your next system design interview or your own production system, let it be this: separate the parts of your system that must be perfectly correct from the parts that only need to be roughly right. Messages must be delivered exactly once, in order, durably; presence and typing only need to feel accurate and disappear cleanly when wrong. Building both on the same durable, consistent substrate is the single most expensive mistake in this space — and building them on genuinely different substrates, with genuinely different reliability budgets, is what lets a single platform serve billions of users without the ephemeral traffic eating the entire cost of the durable one.