Almost every social platform you have ever used — Facebook, LinkedIn, Instagram, Twitter/X, Snapchat — has a small box somewhere in the interface that says something like “People You May Know,” “Suggested for You,” or “People to Follow.” It shows a handful of faces with a name and a small reason underneath, such as “5 mutual friends” or “Works at the same company.” This feature looks tiny, but it is one of the most important growth engines a social network owns.

The idea traces back to the early 2000s. As soon as social graphs became large enough that people could no longer manually find everyone they knew, platforms needed a way to help users rebuild their real-world social circle inside the app. Facebook is widely credited with popularizing “People You May Know” (PYMK) around 2008, using mutual-friend counts as the first, simplest signal. LinkedIn built something similar for professional connections, and over the years both companies (along with Twitter, Instagram, Snapchat, TikTok, and others) evolved these systems from simple graph-counting scripts into massive, multi-stage machine learning pipelines that process billions of graph edges every single day.

Today, a PYMK system is not one algorithm — it is an entire distributed system: graph storage, offline batch jobs, streaming pipelines, machine learning models, caching layers, ranking services, and experimentation frameworks, all working together to answer one deceptively simple question: “Out of the billions of people on this platform, which handful should we show this one user right now?”

A Short Timeline of PYMK

1.1 Why This Is a Great System Design Interview Topic

PYMK shows up constantly in system design interviews for a good reason: it touches almost every major theme that interviewers care about in one relatively self-contained problem. It requires graph thinking (how do you represent and traverse a massive relationship graph efficiently), distributed systems thinking (how do you shard, replicate, and keep data fresh across thousands of machines), machine learning systems thinking (how do you go from raw signals to a trained ranking model, and keep training and serving consistent), and product/privacy thinking (how do you balance showing genuinely useful suggestions against very real privacy risks). Very few interview topics let you demonstrate breadth across all of these areas in a single 45-minute conversation, which is exactly why it remains a favorite prompt at companies building any kind of social or networked product.

1.2 Roadmap for This Tutorial

We’ll build up the system in the same order most strong interview answers do: first pin down the problem precisely (Chapter 2), then sketch the high-level architecture (Chapter 3), then zoom into how each piece actually works internally (Chapters 4 through 6), review the trade-offs (Chapter 7), and finally harden the design against the realities of running at scale — performance, reliability, security, observability, deployment, and storage (Chapters 8 through 14) — before closing with patterns, common pitfalls, industry examples, and a summary you can use as a quick-reference cheat sheet (Chapters 15 through 19).

Let’s define the problem precisely before designing anything. What is PYMK supposed to accomplish, why do naive approaches fail, and what must the system guarantee?

2.1 The Business Problem

A social network’s value comes from its network effect — the platform becomes more useful as more of a user’s real-world connections join and stay active. New users who fail to connect with people they know within the first few sessions are far more likely to churn (stop using the app). PYMK exists to solve exactly this: quickly and accurately rebuild a user’s real-world social graph inside the product, which increases engagement, retention, and ultimately the size and health of the whole network.

2.2 The Technical Problem

Formally, for every user u in a graph with hundreds of millions or billions of nodes, we want to produce a ranked list of the top K (say, 10–50) other users who are not already connected to u, ordered by the probability that u will want to connect with them if shown. This must happen:

• At the scale of the entire user base (hundreds of millions to billions of nodes, tens of billions of edges).
• Fast enough to render in a mobile app feed — typically under 100–200 milliseconds for the “read path.”
• Fresh enough to reflect graph changes from the last few minutes to few hours (a user just added 5 new friends; recommendations should adapt).
• Personalized — no two users should generally see the same list.
• Respectful of privacy settings, blocks, and platform policy (never suggest someone who blocked you, or violate a “do not suggest me” preference).

2.3 Why Naive Approaches Fail

The obvious first idea is: “recommend friends-of-friends, ranked by number of mutual connections.” This is a reasonable starting signal, but it breaks down at scale for several reasons, which we’ll unpack throughout this tutorial:

The real solution, which we will build up piece by piece, separates the problem into an offline/near-real-time candidate generation stage (find a manageable set of plausible people, maybe a few hundred, out of billions) and an online ranking stage (score and order that small set using a machine learning model, in milliseconds).

2.4 Functional Requirements

Before drawing any boxes and arrows, it helps to write down exactly what the system must do:

• Given a user, return a ranked list of up to K suggested people, each with a human-readable “reason” (mutual friends, same workplace, and so on).
• Never suggest someone who is already connected to the user, someone who has blocked or been blocked by the user, or someone the user has explicitly asked not to be shown.
• Allow the suggestion list to change over time as the underlying graph and user behavior change — a person you just connected with should disappear from the list quickly, and a brand new mutual connection should be able to surface a new suggestion.
• Support feedback actions (dismiss, “not interested,” connect) that influence future suggestions for that user.
• Work uniformly whether the requesting user has 5 connections (a brand-new user) or 5,000 connections (a power user), which is a very different traversal problem in each case.

2.5 Non-Functional Requirements

A production PYMK system is generally organized into three broad layers: offline/batch (heavy graph computation done periodically), near-real-time/streaming (keeps things fresh between batch runs), and online serving (answers the actual app request in milliseconds).

How to read this diagram: Data enters from the left (raw graph edges, contact uploads, profile fields, activity events). The offline layer periodically (say, every 6–24 hours) crunches the entire graph to generate a broad candidate list per user — this is the expensive part done in bulk, using distributed processing frameworks like Apache Spark or Flink. The streaming layer patches those candidate lists in near-real-time so that very recent actions (like accepting a new friend request) are reflected quickly without waiting for the next full batch run. Finally, the online serving layer is what actually answers the mobile app’s request: it reads precomputed candidates from a fast key-value store, re-ranks them using a lightweight ML model, applies privacy/policy filters, and returns the final list — all within a strict latency budget.

3.1 Core Components Explained

3.2 Graph Partitioning Strategies

How the graph is split across machines has a large effect on how efficient the offline batch computation is, since graph algorithms are fundamentally about following edges between nodes — and following an edge that crosses machine boundaries is far more expensive (it requires network communication) than following an edge that stays within the same machine.

3.3 Separation of Concerns Between Layers

A useful mental model for this architecture is that each layer optimizes for a different variable. The offline batch layer optimizes for thoroughness and cost-efficiency, since it has hours to work with and can use the cheapest available compute. The streaming layer optimizes for freshness, accepting a simpler, less thorough computation in exchange for reacting within minutes. The online serving layer optimizes purely for latency, deliberately pushing as much work as possible into the other two layers so that the request-time computation is as close to “just a few fast lookups plus one small model inference call” as possible. Keeping this separation clean — resisting the temptation to do “just one more computation” inline in the serving path — is one of the most important architectural disciplines in a system like this.

The two-stage funnel in detail: how a bounded set of plausible candidates is generated, and how a machine-learned ranker turns that set into an ordered, personalized list.

4.1 Candidate Generation Strategies

No single method finds every good suggestion, so real systems combine several candidate-generation strategies and merge the results (this is often called a “multi-source” or “multi-channel” candidate generation approach).

4.2 The Friends-of-Friends Graph Traversal, in Detail

This is the foundational signal, so it deserves a close look. Consider a small example: Alice is connected to Bob and Carol. Bob is connected to Dave. Carol is also connected to Dave. Dave is not connected to Alice. From Alice’s perspective, Dave is a strong candidate because there are two independent paths (through Bob and through Carol) connecting Alice to Dave — this is the “2 mutual friends” signal.

Computing this for every user across the whole graph naively (a full breadth-first search from every node) is far too slow — this is a classic example of why we cannot run graph algorithms live, per-request, in a system with billions of edges. Instead, this is computed as a distributed batch job, often using a “triangle counting” or “two-hop neighborhood” style computation across a cluster, where the graph is partitioned across many machines and each machine works on its slice in parallel.

Beginner Example

Think of it as: “for every pair of people who share at least one mutual friend, and who are not already friends, count how many mutual friends they share, then keep the top pairs.” On a whiteboard with 10 people, you could do this by hand in a minute. At 500 million people, you need a distributed system that shards the graph across thousands of machines and runs this counting job in parallel, usually finishing in a few hours.

Production Example

Companies operating at this scale typically run this kind of computation as a nightly or multiple-times-per-day Spark/Flink job over a partitioned copy of the graph stored in a distributed file system (like HDFS or cloud object storage), writing the resulting candidate lists into a fast key-value store that the online service can read with single-digit-millisecond latency.

4.3 Sample Code — Bounded 2-Hop Traversal (Java)

The following simplified Java snippet shows the core idea of a bounded, degree-capped 2-hop traversal, similar to what a batch job would compute per user (in reality this runs distributed across a cluster, not on a single machine, but the logic is the same).

public class MutualFriendCandidateFinder {

    // adjacency: userId -> set of friend userIds
    private final Map<Long, Set<Long>> adjacency;
    private static final int MAX_DEGREE_TO_EXPAND = 2000; // cap high-degree hubs

    public MutualFriendCandidateFinder(Map<Long, Set<Long>> adjacency) {
        this.adjacency = adjacency;
    }

    // returns candidateId -> mutualFriendCount for the given user
    public Map<Long, Integer> findCandidates(long userId) {
        Set<Long> directFriends = adjacency.getOrDefault(userId, Collections.emptySet());
        Map<Long, Integer> mutualCounts = new HashMap<>();

        for (Long friendId : directFriends) {
            Set<Long> friendOfFriend = adjacency.getOrDefault(friendId, Collections.emptySet());

            // cap traversal fan-out for extremely high-degree "hub" nodes
            if (friendOfFriend.size() > MAX_DEGREE_TO_EXPAND) {
                continue;
            }

            for (Long candidate : friendOfFriend) {
                if (candidate.equals(userId) || directFriends.contains(candidate)) {
                    continue; // skip self and existing friends
                }
                mutualCounts.merge(candidate, 1, Integer::sum);
            }
        }
        return mutualCounts;
    }
}

Notice the MAX_DEGREE_TO_EXPAND cap — this is important in real systems. Some accounts (celebrities, brand pages) have millions of connections; expanding through them would be extremely expensive and would produce low-quality, generic candidates. Capping fan-out through high-degree “hub” nodes is a common, practical optimization.

4.4 Ranking: Turning Candidates into a Final Ordered List

Once we have a bounded candidate set (typically a few hundred people per user), the online ranking stage scores each one using a machine learning model trained to predict the probability that the user will send a connection request (or accept one) if shown that suggestion. Typical input features include:

• Number of mutual connections, and their “quality” (mutual friends who are themselves close friends count more than distant acquaintances)
• Profile similarity: same city, same employer, same school, overlapping interests
• Interaction history: has the user viewed this candidate’s profile recently, exchanged messages, or been suggested before and ignored them
• Embedding similarity score from the graph neural network
• Recency signals: the candidate joined recently, or the mutual connection was made recently

These features feed into a model — historically gradient-boosted decision trees (like XGBoost/LightGBM) were very common for this because they are fast, interpretable, and work well on structured/tabular features; increasingly, deep learning ranking models (including two-tower neural networks) are used, especially when combined with learned embeddings.

4.5 Feature Engineering in Detail

The quality of the ranking model depends heavily on the features it is given, arguably even more than on the specific model architecture chosen. Features generally fall into a few buckets:

A subtlety worth understanding: features must be computed consistently between training time (using historical data) and serving time (using live data), otherwise the model will behave unpredictably in production even though it looked accurate during training — this mismatch is known as “training-serving skew,” and is one of the most common real-world bugs in recommendation systems. This is precisely why a dedicated feature store (Chapter 3.1) is used: it acts as a single source of truth for how each feature is computed, shared by both the offline training pipeline and the online serving path.

4.6 How the Ranking Model Is Trained

The model is trained as a supervised learning problem. Historical logs of past suggestions — which candidates were shown to which users, and whether the user subsequently sent or accepted a connection request — become the labeled training data (a suggestion the user acted on is a positive example; one they ignored is a negative example). Because the number of “ignored” suggestions vastly outnumbers “acted on” suggestions, techniques like negative sampling or class-weighting are used so the model doesn’t simply learn to predict “no” for everything.

The trained model is validated offline against a held-out slice of historical data (checking metrics like AUC — area under the ROC curve — which measures how well the model ranks positive examples above negative ones), and only promoted to production after passing these offline checks and, subsequently, a live online experiment (see Chapter 12 on canary/shadow deployment).

4.7 Sample Code — Simplified Feature Vector Assembly (Java)

public class CandidateFeatureBuilder {

    private final FeatureStoreClient featureStore;

    public CandidateFeatureBuilder(FeatureStoreClient featureStore) {
        this.featureStore = featureStore;
    }

    public FeatureVector build(long userId, long candidateId) {
        int     mutualCount              = featureStore.getMutualConnectionCount(userId, candidateId);
        double  embeddingSimilarity      = featureStore.getEmbeddingCosineSimilarity(userId, candidateId);
        boolean sameEmployer             = featureStore.isSameEmployer(userId, candidateId);
        boolean sameSchool               = featureStore.isSameSchool(userId, candidateId);
        long    daysSinceCandidateJoined = featureStore.getDaysSinceJoined(candidateId);
        int     candidateDegree          = featureStore.getConnectionCount(candidateId);

        return FeatureVector.builder()
                .mutualConnectionCount(mutualCount)
                .embeddingCosineSimilarity(embeddingSimilarity)
                .sameEmployer(sameEmployer)
                .sameSchool(sameSchool)
                .daysSinceCandidateJoined(daysSinceCandidateJoined)
                .candidateDegree(candidateDegree)
                .build();
    }
}

4.8 Diversity and the “Filter Bubble” Problem

Pure score-maximization in the ranking stage has a subtle failure mode: if you simply sort every candidate by predicted score and take the top K, you can end up showing a user ten near-duplicate suggestions — for instance, ten different coworkers from the same small team, when the user might actually be more interested in seeing a broader mix of coworkers, former classmates, and friends-of-friends. To address this, production ranking pipelines usually apply a diversity or re-ranking pass after the raw model scores are computed, using techniques such as capping how many top results can share the same “source strategy” or the same underlying mutual connection, so the final list feels varied and useful rather than narrowly repetitive.

4.9 Generating Human-Readable Reasons

The short “reason” text shown under each suggestion (like “5 mutual connections” or “Works at the same company”) is not just a cosmetic detail — research and product experience across social platforms consistently show that a suggestion accompanied by a clear, truthful reason is far more likely to be acted on than one shown with no context at all, since it gives the user an immediate, low-effort way to judge relevance. Generating this reason is typically done by picking the single strongest contributing feature for that specific candidate (whichever feature had the largest positive influence on the model’s score) and mapping it to a pre-written, human-friendly template, rather than trying to explain the full, more complex combination of signals the ranking model actually used internally.

Let’s trace a request end-to-end, followed by how the underlying data refreshes over time.

5.1 Lifecycle of a Suggestion

5.2 What the Streaming Layer Actually Patches

It’s worth being concrete about what “patching” means in practice, since the streaming layer does not attempt to redo the full candidate-generation computation — that stays the job of the batch layer. Instead, it handles a small set of high-value, cheap-to-apply updates:

• Removing stale candidates: if user A and user B just became connected, B should be removed from A’s candidate list (and vice versa) immediately, rather than waiting for the next batch run — showing someone a suggestion to connect with someone they’re already connected to looks obviously broken.
• Applying new exclusions: if user A blocks user B, that exclusion must propagate to the candidate store within seconds, not hours, given the privacy sensitivity involved.
• Light-touch promotion of very strong new signals: if two users who weren’t previously connected suddenly share a brand new, strong mutual connection (for example, a mutual friend they just both connected with), a lightweight incremental score bump can be applied without a full recomputation.

Anything requiring a broader recomputation — like fully re-deriving someone’s entire candidate pool from scratch — is deliberately left to the next batch cycle, since attempting to do that work incrementally, for every single event, would essentially mean re-implementing the batch pipeline’s logic in a much more complex, per-event streaming form, for relatively little added benefit.

The specific algorithms and data structures that make the funnel work at scale — plus a back-of-the-envelope walkthrough of the sizing numbers that justify every downstream design choice.

6.1 Graph Representation

At this scale, the graph cannot live in memory on a single machine. It is partitioned (“sharded”) across many machines, typically by user ID ranges or using a graph-partitioning algorithm that tries to keep highly-connected clusters of users together on the same shard to minimize cross-machine traffic during traversal. Each shard stores an adjacency list: for each user, the list of user IDs they are connected to.

6.2 Approximate Nearest Neighbor (ANN) Search

Once users are represented as dense vectors (embeddings) via a graph neural network or similar technique, finding “similar” users becomes a nearest-neighbor search problem in high-dimensional space. Doing this exactly (comparing a user’s vector against every other user’s vector) is far too slow at hundreds of millions of vectors. Instead, systems use approximate nearest neighbor structures such as HNSW (Hierarchical Navigable Small World graphs) or IVF (Inverted File Index) that trade a small amount of accuracy for enormous speed gains — typically finding “good enough” nearest neighbors in milliseconds instead of seconds.

6.3 Union-Find / Connected Components for Community Detection

Some candidate-generation strategies benefit from knowing which “community” or cluster a user belongs to (e.g., a friend group, a school cohort). Union-Find (Disjoint Set Union) is a classic data structure used in distributed graph processing to efficiently group nodes into connected components, which can be used as an additional candidate-generation signal (“suggest other members of this densely-connected friend cluster”).

6.4 Bloom Filters for Fast “Already Connected” Checks

A very practical optimization: before doing expensive candidate scoring, systems often use a Bloom filter (a compact, probabilistic data structure) to quickly check “is this candidate already a connection, or already dismissed by the user?” A Bloom filter can answer this in constant time with very low memory usage, at the cost of a small false-positive rate (which is acceptable here, since worst case we just skip a few valid candidates).

6.5 Consistent Hashing for Sharding the Cache and Store

To distribute candidate lists and cache entries evenly across many machines — and to avoid a massive reshuffle whenever a machine is added or removed — consistent hashing is used, just as in most large-scale distributed caching systems.

6.6 Consensus for Metadata, Not for Every Edge

It’s a common misconception that a system like this needs a strong consensus protocol (like Raft or Paxos) guarding every single graph edge write. In practice, consensus algorithms are reserved for a much smaller, more critical set of metadata — for example, agreeing on shard ownership assignments, coordinating leader election within a storage cluster, or managing configuration changes to the partitioning scheme. The actual high-volume graph edge writes and candidate list updates use the simpler, higher-throughput eventual-consistency replication model described in Chapter 8, because requiring full consensus for every one of billions of daily graph mutations would be prohibitively slow and would not meaningfully improve the user-facing product, given that eventual consistency is already an acceptable trade-off here.

6.7 How Graph Embeddings Are Actually Learned

It’s worth spending a moment on how a user “becomes” a vector, since this is one of the more conceptually tricky parts of the system for newcomers. The general idea, shared by techniques like node2vec and modern Graph Neural Networks (GNNs), is: a user’s vector should be learned such that users who are structurally close in the graph end up close together in vector space, and users who are structurally distant end up far apart.

One classical approach, node2vec, works by simulating many random walks starting from each node in the graph (a “random walk” means repeatedly hopping to a randomly chosen neighbor). This turns the graph into something that looks like a collection of “sentences,” where each “word” is a user ID and each “sentence” is a random walk path. These synthetic sentences are then fed into a word-embedding-style model (conceptually similar to Word2Vec, originally built for natural language) which learns vectors such that users appearing near each other in these random-walk sentences end up with similar vectors.

Modern systems increasingly use Graph Neural Networks instead, which improve on this by also incorporating each user’s own features (not just graph position) — for example, profile attributes, activity level, and the features of their neighbors — through a process called “message passing,” where each node repeatedly aggregates information from its neighbors across several rounds (or “layers”) to build a richer representation. This lets the model learn things like “users are similar not only because they are graph-close, but because their neighborhoods share similar characteristics.”

6.8 Capacity Estimation Walkthrough

Interviewers often want to see you reason quantitatively about scale, not just describe components. Here is a representative walkthrough:

None of these numbers need to be exact in an interview — the point is to demonstrate that you can translate a vague scale (“social-network scale”) into concrete numbers that justify specific design decisions, like why a single-machine graph database is not viable and why sharding and caching are necessary rather than optional.

Every choice this design makes gives up something in exchange for something else. Reasoning through these trade-offs explicitly is often what separates a strong system design answer from an average one.

7.1 Thinking in Trade-offs, Not Absolutes

Every card above is a reminder that system design is rarely about finding a single “correct” answer — it’s about understanding which properties matter most for the specific problem at hand and consciously giving up less important properties to get them. For PYMK specifically, the recurring theme is that availability, latency, and cost efficiency are prioritized over perfect accuracy or perfect consistency, because the cost of a slightly imperfect or slightly stale suggestion is low (the user simply ignores it), whereas the cost of a slow, unavailable, or extremely expensive system is high. Recognizing and articulating this priority ordering explicitly, rather than treating every requirement as equally important, is often what separates a strong system design answer from an average one.

A strict latency budget for the read path, horizontally scalable stateless services, and specific mitigations for the “celebrity problem” that any real social graph produces.

8.1 Sharding the Graph

The social graph is partitioned across many storage nodes. A common approach is to shard by user ID using a hashing scheme, though smarter graph-aware partitioning (keeping tightly-connected communities on the same shard) reduces expensive cross-shard lookups during traversal jobs.

8.2 Batch Processing at Scale

The offline candidate-generation jobs run on distributed processing frameworks (Spark, Flink, or similar) across large clusters, processing the graph in parallel across thousands of machine-hours. Techniques like the Pregel-style “bulk synchronous parallel” model (used in graph processing frameworks) let each node compute locally and only exchange messages with neighbors at defined synchronization points, which scales well for graph algorithms like 2-hop traversal.

8.3 Read Path Latency Budget

8.4 Horizontal Scaling of the Serving Layer

The online PYMK service is stateless (all state lives in the cache/store layers), so it scales horizontally behind a load balancer — simply add more service instances as request volume grows. The ranking model inference service is typically scaled separately, often on hardware optimized for the specific model type, and can use request batching to improve throughput.

8.5 Handling Hot Spots (“Celebrity Problem”)

In any social graph, connection counts follow a highly skewed, long-tailed distribution — most users have a modest number of connections, while a small number of accounts (celebrities, public figures, popular brands) have connection counts orders of magnitude larger than average. This creates “hot spots”: specific shards or specific keys in the graph store that receive disproportionately high read and write traffic, and specific nodes whose adjacency lists are so large that any traversal touching them becomes slow.

Common mitigations include: replicating hot user data across multiple shard replicas so reads can be load-balanced rather than hammering a single shard; capping and sampling traversal through very high-degree nodes (as shown in the Java example in Chapter 4.3); and caching hot users’ precomputed candidate lists more aggressively, sometimes with a longer TTL, since their popularity also makes them useful candidates to reuse across other users’ friends-of-friends computations.

8.6 Write Path Considerations

Although this tutorial focuses mostly on the read-heavy PYMK request path, the write path — updating the graph when a user forms a new connection, edits their profile, or blocks someone — matters too. These writes typically go through the primary graph storage system first (ensuring durability and correctness), and are asynchronously published as events onto a message queue like Kafka, which is what the streaming/freshening layer (Chapter 5) consumes to keep the candidate store reasonably up to date without requiring the write path itself to be slowed down by recommendation-related side effects.

8.7 Replication Strategy

Both the graph storage and the candidate store are typically replicated using a leader-follower (primary-replica) model within each shard: writes go to a primary replica and are asynchronously propagated to follower replicas, which serve the bulk of read traffic. This favors read scalability and availability over strict write consistency — an acceptable trade-off here, since a few seconds of replication lag rarely matters for a feature that already tolerates minutes of overall staleness.

Redundancy, graceful degradation, circuit breakers, validated batch pipelines, disaster recovery, idempotency, and deliberate chaos engineering — the disciplines that keep a system like this actually available, not just theoretically available.

9.1 Redundancy

Every layer — graph storage, candidate store, cache, ranking service — is deployed across multiple availability zones or data centers with replicated data, so the loss of a single machine or even a whole zone does not take down the feature.

9.2 Graceful Degradation

PYMK is a good example of a feature that should degrade gracefully rather than fail hard. If the ranking model service is unavailable, the system can fall back to a simpler ranking (e.g., raw mutual-friend count) rather than showing an error or an empty box. If the streaming freshness layer is delayed, the system can serve slightly stale (but still reasonably good) candidates from the last successful batch run.

9.3 Circuit Breakers and Timeouts

Calls between the PYMK service, the ranking service, and the feature store are protected with circuit breakers and strict timeouts, so a slow or failing downstream dependency doesn’t cascade into a broader outage.

9.4 Batch Job Reliability

Offline candidate-generation jobs are checkpointed, so a failure partway through a multi-hour Spark job doesn’t require restarting from scratch. Job outputs are validated against sanity checks (e.g., “did the number of candidates generated fall within an expected range?”) before being promoted to serve live traffic — this catches upstream data corruption before it reaches users.

9.5 Disaster Recovery & Backups

Beyond routine redundancy, the system needs a plan for larger-scale failures — a full region outage, or data corruption discovered after the fact. Typical practices include: taking regular, versioned snapshots of the core graph storage so it can be restored to a known-good point in time; retaining the last several successful outputs of the offline candidate-generation pipeline (not just the latest one) so a bad run can be rolled back quickly by simply pointing the serving layer at the previous good snapshot; and running periodic disaster-recovery drills where a region is deliberately failed over to verify that traffic can be redirected without manual, error-prone intervention under pressure.

9.6 Idempotency and Exactly-Once-Feeling Processing

Because the streaming freshening layer processes events from a message queue, and message queues generally provide at-least-once delivery (a message might be delivered more than once during retries or rebalances), the event processing logic must be idempotent — applying the same “user X added connection Y” event twice should leave the system in the same state as applying it once. This is usually achieved by making updates to the candidate store naturally idempotent (e.g., “add Y to X’s exclusion set” rather than “increment a counter”), which sidesteps the need for complex deduplication logic.

9.7 Proactively Testing for Failure

Mature teams operating a system at this scale don’t just wait for real failures to test their redundancy assumptions — they deliberately inject failure in controlled ways (a practice often called chaos engineering) to verify the system actually behaves as designed under stress. For a PYMK system, this might include deliberately killing a fraction of ranking-service instances during low-traffic hours to confirm the fallback heuristic ranking kicks in correctly, deliberately delaying the streaming freshening pipeline to confirm the system still serves reasonable (if slightly stale) results rather than failing, or deliberately overloading the candidate cache to confirm request coalescing and timeouts behave as expected rather than allowing a cascading overload of the backing store. Finding these gaps in a controlled experiment during business hours is far preferable to discovering them for the first time during a real incident.

PYMK sits directly on top of extremely sensitive data — who knows whom — so privacy engineering is not optional, it’s central to the design.

10.1 Regulatory Considerations

Because a PYMK system processes relationship and contact data, it typically falls under data protection regulations such as the GDPR in Europe or similar regional laws elsewhere, which usually require: a lawful basis for processing personal data (often explicit consent for something like contact-book uploads), the ability for a user to request deletion of their data (including any derived data like embeddings or cached candidate lists), and data minimization — only collecting and retaining what’s actually needed for the feature to work, for only as long as needed. This “delete on request” requirement has a real architectural implication worth calling out: because a user’s data flows through so many derived artifacts in this pipeline (raw graph edges, computed features, trained embeddings, cached candidate lists, and logged historical impressions used for model training), a deletion request needs to be propagated through all of these downstream systems, not just the primary graph store, which is why many teams maintain an explicit data lineage map showing everywhere a given piece of personal data ends up so that deletion requests can be fulfilled completely and verifiably rather than partially.

10.2 Threat Modeling

Beyond privacy compliance, it helps to think through who might try to abuse this system and how:

• Graph scraping: An attacker could try to repeatedly query the PYMK API (or the underlying profile/connections APIs) to reconstruct large portions of the private social graph. Mitigations include per-user and per-IP rate limiting, anomaly detection on request patterns, and CAPTCHA-style friction for suspicious traffic.
• Fake account farms: Networks of fake accounts might try to manipulate suggestions (for example, to get suggested to real users for spam or scam purposes). This is usually addressed by a separate trust-and-safety/anti-abuse system that feeds signals (like “is this account likely fake”) into the candidate filter as an additional exclusion criterion.
• Inference attacks: Even without directly scraping the graph, an attacker might try to infer whether two specific people are connected by observing whether they appear in each other’s mutual-friend counts through indirect means. Being conservative about what numeric detail is exposed in API responses (e.g., exact mutual counts vs. bucketed ranges) can reduce this risk.

Two very different metric families — system health and product quality — watched simultaneously, tied together by distributed tracing and disciplined A/B experimentation.

11.1 System Health Metrics

11.2 Product / ML Quality Metrics

11.3 Tracing & Debugging

Distributed tracing (tagging every request with a correlation ID that flows through the gateway, PYMK service, ranking service, and filter service) is essential for debugging latency spikes or unexpected empty results, since a single user-facing request touches many independent systems.

11.4 Experimentation

Because PYMK directly affects growth metrics, changes to ranking models or candidate-generation strategies are almost always rolled out through A/B testing frameworks, comparing key metrics (like connection acceptance rate) between a control group and a treatment group before a full rollout.

A well-run experiment for a change like this typically defines its success metric and minimum detectable effect in advance (to avoid the temptation of cherry-picking a metric that happened to move favorably after the fact), runs long enough to cover natural weekly usage cycles (since social app usage patterns often differ meaningfully between weekdays and weekends), and checks for both the primary metric (connection acceptance rate) and guardrail metrics that should not regress (like overall session length or complaint/report rates), since a change that boosts acceptance rate by making suggestions overly aggressive or spammy would be a net loss even if the headline number looks good.

11.5 Alerting Philosophy

Not every metric deviation should page an on-call engineer at 3 a.m. A healthy alerting setup distinguishes between symptom-based alerts (things directly affecting users right now, like elevated p99 latency or a spike in 5xx errors on the PYMK API — these should page immediately) and cause-based signals (like a slightly slower-than-usual batch job — these can wait for business hours unless they threaten to breach a freshness SLA). Dashboards should make it easy to move from a symptom alert down to the likely root cause quickly, which is exactly what the distributed tracing and per-component metrics described above are for.

Containerized microservices for the online path, elastic (often interruptible) compute for the batch path, GPU-accelerated pipelines for model training, and canary/shadow rollout for every model change.

The online serving components (PYMK API, ranking inference service, filter service) are typically packaged as containerized microservices, deployed on an orchestration platform (such as Kubernetes) so they can be scaled horizontally and rolled out/rolled back safely with standard deployment strategies (rolling updates, canary releases).

The offline batch layer runs on a separate, elastic compute cluster (often cloud-managed, spinning up large numbers of worker nodes for the duration of the job and releasing them afterward to control cost). Because batch jobs are not latency-sensitive in the same way as the serving path, they are good candidates for cheaper, interruptible compute resources (such as spot/preemptible instances), as long as the processing framework can tolerate and recover from node interruptions.

Model training for the ranking model and the graph embedding model typically runs on GPU-accelerated infrastructure, on a recurring schedule (e.g., daily or weekly retraining), with trained models versioned and validated against held-out data before being promoted to serve live traffic — mirroring standard MLOps practice. Every trained model artifact is tagged with the exact dataset snapshot and feature-store version used to produce it, so that if a quality regression is discovered after deployment, engineers can reliably reproduce the training run, diagnose whether the issue came from the data, the features, or the model itself, and roll back to a specific known-good prior version rather than guessing.

12.1 Cost Optimization

At the scale this system operates, infrastructure cost becomes a real design constraint, not an afterthought. A few practical levers commonly used:

No single database technology is well-suited to every kind of data this system produces, which is exactly why the architecture uses several purpose-built storage systems rather than one general-purpose database for everything.

13.1 Storage Choices

13.2 Caching Strategy

The read-heavy nature of PYMK (many more reads — app opens — than writes — new connections formed) makes it a great fit for aggressive caching. A common pattern is cache-aside: the service checks a Redis-style in-memory cache first; on a miss, it reads from the underlying candidate store and populates the cache with a time-to-live (TTL), so entries naturally expire and get refreshed rather than growing stale indefinitely.

13.3 Load Balancing

Requests are distributed across many stateless service instances using a standard load balancer (round-robin or least-connections), sitting behind the API gateway. For the ranking inference service specifically, load balancing often also considers current GPU/CPU utilization, since inference cost per request can vary.

13.4 Example Candidate Store Schema

A simplified schema for the precomputed candidate store might look like this, stored as a wide-row or document keyed by user ID:

{
  "userId": "123",
  "generatedAt": "2026-07-27T02:00:00Z",
  "candidates": [
    {
      "candidateId": "987",
      "mutualCount": 5,
      "sourceStrategy": "FRIENDS_OF_FRIENDS",
      "embeddingSimilarity": 0.71
    },
    {
      "candidateId": "552",
      "mutualCount": 0,
      "sourceStrategy": "CONTACT_BOOK_MATCH",
      "embeddingSimilarity": 0.44
    }
  ]
}

Storing the sourceStrategy alongside each candidate is useful in practice: it lets the ranking model use “how was this candidate found” as a feature, and it makes debugging much easier when a particular suggestion looks wrong — you can immediately tell which part of the candidate-generation pipeline produced it.

13.5 Consistent Hashing in Practice

When the candidate store or cache layer is sharded across many nodes, consistent hashing maps each user ID to a position on a hash ring, and each node owns a contiguous arc of that ring. The key benefit becomes clear when the cluster resizes: adding or removing a node only requires reassigning the small arc of keys immediately adjacent to it, rather than reshuffling the entire keyspace — which is essential for a system that needs to scale its cluster size up and down over time without triggering a massive, latency-impacting data migration every time.

13.6 Indexing Strategy for the Feature Store

The feature store needs to answer two very different access patterns efficiently: fast point lookups at serving time (“give me the features for this specific user-candidate pair, right now, in a few milliseconds”) and large-scale scans at training time (“give me millions of historical user-candidate feature rows to train the next model version”). These two patterns are usually served by two different physical storage systems under one logical feature-store interface: an online store (a low-latency key-value store, indexed by user/candidate ID for fast point lookups) and an offline store (a columnar data warehouse table, optimized for large sequential scans and joins during training data preparation). Keeping the feature definitions identical between the two, while letting the physical storage differ based on access pattern, is the essence of the feature store pattern discussed in Chapter 15.

A tiny public API surface backed by a handful of internal microservices, each independently scaled and deployed — gRPC on the inside for speed, REST on the outside for compatibility.

A minimal PYMK API contract might look like this:

Response 200:
{
  "userId": "123",
  "suggestions": [
    {
      "candidateId": "987",
      "score":       0.83,
      "reasons": ["5 mutual connections", "Same employer"]
    },
    {
      "candidateId": "654",
      "score":       0.77,
      "reasons": ["2 mutual connections"]
    }
  ]
}

Internally, this is composed of several microservices communicating over lightweight protocols (often gRPC internally for lower latency and strongly typed contracts, REST/JSON at the public-facing edge):

Splitting these into separate services (rather than one large service) allows each to be scaled, deployed, and owned independently — for example, the ranking inference service might need GPU machines and a completely different deployment cadence than the lightweight filter service.

14.1 Why gRPC Internally and REST Externally

Internal service-to-service calls (orchestration service to candidate retrieval, to feature service, to ranking service) typically favor gRPC over plain REST/JSON because it uses a compact binary format (Protocol Buffers) that is faster to serialize and deserialize, enforces a strongly typed contract between services (reducing a whole category of integration bugs caused by mismatched field names or types), and supports efficient streaming, which can matter when passing large candidate batches between services. The public-facing edge, on the other hand, usually still exposes a simpler REST/JSON API, since external client compatibility, debuggability, and broad tooling support matter more there than raw performance.

14.2 Service Ownership and Independent Scaling

Because each microservice in this pipeline has very different resource characteristics — the ranking inference service is compute-heavy and benefits from specialized hardware, while the filter service is lightweight and mostly CPU-bound rule evaluation — decomposing the system this way allows each team that owns a service to make independent decisions about scaling policy, hardware choice, and deployment cadence, without needing to coordinate every change across the entire pipeline. This is one of the core arguments for a microservices architecture over a single monolithic service in a system with such heterogeneous components.

The specific patterns that keep this system tractable at scale, and the anti-patterns that quietly break it if not actively avoided.

15.1 Useful Patterns

15.2 Anti-Patterns to Avoid

15.3 Pattern Deep Dive — The Funnel in More Detail

It is worth walking through why the funnel pattern generalizes so well beyond PYMK. Any recommendation problem at scale shares the same underlying tension: the universe of possible items (or people) to recommend is enormous, but the amount of compute you can spend per request is tightly bounded by a latency budget. The funnel resolves this tension by spending compute unevenly — cheap, approximate methods are applied to the entire universe to cut it down to a manageable size, and expensive, precise methods are reserved for that much smaller shortlist.

This same shape appears in web search (a fast inverted index narrows billions of documents down to thousands, then a more expensive relevance model ranks those thousands), in advertising systems (a fast eligibility and targeting pass narrows the ad inventory before an expensive auction-and-prediction model runs), and in video/content feeds (fast retrieval from an index, followed by a heavier ranking model). Recognizing this shared shape is valuable in interviews, since it shows you understand the underlying principle rather than having memorized one specific system.

It’s also worth noting that some very large-scale systems extend this into a three-stage funnel rather than two: a first-pass retrieval stage that is extremely cheap but fairly imprecise, a second-pass “pre-ranking” or “lightweight scoring” stage that narrows things further using a cheaper, smaller model, and a final, most expensive full-ranking stage applied only to the last few dozen candidates. Whether a PYMK system needs this extra middle stage generally depends on how large the initial candidate pool coming out of generation tends to be — if candidate generation already reliably produces a pool in the low hundreds, a two-stage funnel is usually sufficient, and adding a third stage would mostly add operational complexity without a meaningful quality gain.

15.4 Pattern Deep Dive — Feature Store as a Contract

The feature store pattern deserves a second look because it solves a problem that is easy to underestimate until you’ve been burned by it: if the code that computes a feature during model training is even slightly different from the code that computes the same feature at serving time — different rounding, a different time window, a slightly different definition of “mutual connection” — the model’s real-world performance can silently degrade even though nothing looks obviously broken. Treating the feature store as a shared contract, with a single, versioned definition for each feature used by both training and serving pipelines, removes an entire category of hard-to-diagnose bugs.

Habits worth building into the team’s default workflow, and failure modes worth actively guarding against.

16.1 On Balancing Engineering Effort Against Product Impact

One mistake that even experienced engineers make when building a system like this is over-investing in the most technically interesting component (often the graph embedding model or the approximate nearest-neighbor infrastructure) while under-investing in the less glamorous but equally important pieces — the filter/policy service, the monitoring and alerting setup, or simply making sure the “reason” text shown to users is clear and accurate. In practice, product impact and engineering interest are only loosely correlated: a well-tuned diversity re-ranking pass or a better-written suggestion reason can sometimes move engagement metrics as much as a more sophisticated embedding model, at a fraction of the implementation cost. Regularly revisiting where effort is actually going, relative to where measured impact is coming from, helps keep a team’s roadmap grounded in outcomes rather than in what happens to be the most interesting problem to work on.

16.2 A Practical Design Review Checklist

When reviewing a proposed change to a system like this — a new candidate-generation strategy, a new model version, or a new infrastructure component — the following questions tend to surface the most important issues early:

• Does this change increase the candidate set size, and if so, has the added ranking/inference cost been measured?
• Is the block/privacy filter still guaranteed to run as the final, mandatory step after this change?
• What happens to this component’s output if an upstream dependency is completely unavailable — does the system degrade gracefully or fail hard?
• Is there a rollback path if this change causes a regression in production (a previous model version, a previous candidate snapshot)?
• Does this change introduce any new personal data collection or retention, and if so, has it been reviewed against data protection requirements?
• Will this be validated through a controlled experiment (A/B test) before a full rollout, with clear success metrics defined in advance?

Treating this as a standard checklist — rather than relying on any single engineer to remember every consideration — is itself a best practice, since it turns tribal knowledge into a repeatable process that scales as the team and system grow.

16.3 How This Compares to Building It From Scratch at a Small Startup

It’s worth explicitly contrasting the full architecture in this tutorial against what a two-person engineering team at an early-stage social app should realistically build. At small scale, the entire “candidate generation plus ranking” pipeline can often run as a single nightly script: compute mutual-friend counts with a straightforward graph query, order by count plus a couple of profile-similarity checks, and write the result into a simple database table that the app reads directly, with no dedicated cache layer, no embedding model, and no streaming freshness layer at all. This is not a lesser version of the “real” design — it is the correct design for that scale. The architecture in this tutorial should be read as a description of where the system naturally grows toward as user count, request volume, and graph density increase past the point where simpler approaches start to show measurable strain, not as a mandatory starting point for every team regardless of scale.

Publicly discussed approaches from major social platforms, the evolutionary path that most large-scale PYMK systems trace, and what a smaller platform should actually build today.

While the exact internals of any specific company’s PYMK system are proprietary, the broad architecture described in this tutorial mirrors publicly discussed approaches from major social platforms:

Across all of these, the common thread is the same: a funnel architecture separating broad, cheap candidate generation from precise, personalized ranking, running on top of a massively sharded graph storage layer, refreshed through a mix of batch and streaming computation.

17.1 Lessons From Industry Evolution

It is useful to understand how these systems generally evolved over time, since the same evolutionary path tends to repeat at any growing company building this kind of feature:

Most engineering teams building a system like this today can, and should, start closer to Stage 2 or 3 rather than reinventing every stage from scratch — the point of studying this evolution is to understand why each layer exists, so you know which pieces are essential at your current scale and which are premature optimization.

17.2 Scaling Down: What a Smaller Platform Should Actually Build

Not every platform needs the full five-stage architecture described above on day one, and it’s worth explicitly saying so — a common mistake among engineers newer to system design is assuming “social-network scale” always means building every component described in this tutorial immediately. A platform with a few hundred thousand users can get very good results from Stage 2 or Stage 3 alone: a nightly batch job computing mutual-friend-based candidates, ranked with a modest heuristic or a simple logistic regression model, served from a straightforward key-value cache. The more advanced pieces — graph embeddings, real-time streaming freshness, elaborate approximate nearest-neighbor infrastructure — earn their complexity cost only once the graph and traffic are large enough that the simpler approach starts to show real cracks, whether in suggestion quality, computation time, or infrastructure cost. Recognizing this — matching architectural complexity to actual current scale rather than hypothetical future scale — is itself one of the most valuable system design instincts to develop, and interviewers frequently reward candidates who explicitly call out which pieces of a design are necessary now versus which could reasonably be deferred.

The questions interviewers reach for when probing whether a candidate has genuinely internalized the shape of the design, or merely memorized its parts.

The narrative worth being able to walk through cleanly, start to finish, if asked to design this system live.

Putting It All Together

If you had to explain this entire system in one breath, it would sound something like this: a massively sharded graph store holds the raw relationship data; a scheduled, distributed batch pipeline periodically mines that graph — using multiple complementary strategies like bounded friends-of-friends traversal, contact matching, and learned embedding similarity — to produce a bounded, precomputed candidate pool for every user; a lightweight streaming layer keeps that pool reasonably fresh between batch runs by patching in the highest-value recent changes; and a fast, horizontally scaled, stateless online service reads those precomputed candidates from cache, scores them with a compact machine learning model trained on historical engagement data, filters out anything that violates privacy or policy rules, and returns a small, personalized, diverse list — all within a latency budget measured in tens of milliseconds. Every individual design decision throughout this tutorial, from degree-capping high-connectivity nodes to choosing eventual consistency over strong consistency, traces back to the same underlying constraint: an enormous search space and a strict, unforgiving latency budget, reconciled by doing as much expensive work as possible ahead of time and keeping the request-time path as thin as it can possibly be.