Imagine you have 2,000 followers on a social app. Most of them are acquaintances — people from school, coworkers, strangers who followed you back. But you have twelve people you actually trust: your best friend, your sister, your two closest college roommates. Sometimes you want to post something just for them — a candid photo, a rant about your day, something you would never want your boss or your old classmate to see. That is the exact problem that a “close friends” feature solves.

In plain terms, a close friends selective sharing system is a feature that lets a user (we will call them the poster) choose a private, hand-picked subset of their followers (we will call this list the close friends list) and publish content — a post, a photo, a story, a status update — that is visible only to people on that list. Everyone else who follows the poster, even people who are very close to them in real life but were not added to the list, simply does not see that content at all. It does not show up in their feed. It does not show up if they visit the poster’s profile. It is invisible to them, as if it does not exist.

1.1 A short history

The idea did not appear overnight. Social platforms have long wrestled with the tension between “sharing everything with everyone” and “sharing selectively with the right people.” Early social networks like Google+ experimented with Circles in 2011, letting users organize their contacts into named groups (Family, Coworkers, Acquaintances) and choose which circle saw which post. It was a powerful idea but the user experience was heavy — most people never bothered to organize their circles carefully, and Google+ eventually shut down in 2019.

Facebook offered a similar capability through Custom Friend Lists and post-level audience selectors (“Friends except…”, “Specific friends”), but again, the friction of managing lists meant adoption stayed low among casual users.

The breakthrough in mainstream adoption came from Instagram in November 2018, when it launched the Close Friends feature for Stories. The genius of the design was simplicity: one single, unnamed list per user, accessible with one tap, marked with a distinctive green ring around the poster’s profile picture so close friends instantly recognized this content was exclusive. There was no complex circle management — just “the people I trust,” a single toggle when posting, and a green badge. This reduced the cognitive overhead to almost zero, and it became one of the most beloved features on the platform, later extended to feed posts and reels in various forms.

Since then, the pattern has been copied and adapted across the industry: Snapchat’s “Private Story” (custom-named, shareable friend groups), BeReal’s flexible audience controls, and various niche social apps built entire products around the idea of small, curated audiences instead of broadcast-to-everyone feeds — a reaction to “context collapse,” the anxiety of posting something knowing your boss, your parents, and your childhood friends all see the exact same content.

1.2 Timeline of the idea

1.3 Why platforms keep reinventing this idea

It’s worth pausing on why so many independent product teams, at different companies, in different eras, converged on nearly the same feature. The underlying human behavior is consistent: people do not actually want to share everything with everyone. Psychologists who study online self-presentation describe a phenomenon called context collapse — the flattening of many different real-world audiences (family, coworkers, close friends, strangers) into one single, undifferentiated online audience. In real life, you naturally adjust what you say depending on who is in the room. A public, broadcast-only social feed removes that adjustment entirely, and it makes people self-censor, post less, or post only the safest, most polished version of their lives.

A close friends feature is, in effect, an engineering solution to a social and psychological problem: it recreates “the room” — a bounded, known, trusted audience — inside a platform that is otherwise built around broadcasting to the largest possible audience. This is precisely why the feature tends to drive disproportionately high engagement relative to its footprint: users post more often, more candidly, and more frequently to a close friends audience than to their public feed, because the perceived social risk of oversharing drops sharply once the audience is small and trusted.

From a systems perspective, this human motivation translates into a concrete engineering requirement: the feature must feel instantaneous and effortless, because any friction (a slow list-management screen, a confusing multi-step audience picker, or content that visibly takes longer to publish than a normal post) undermines the very spontaneity that makes close-friends sharing valuable in the first place. This is one of the reasons the architecture in this document leans so heavily on asynchronous, event-driven fan-out — the user should never feel like the “restricted” version of posting is a slower, second-class experience.

Let’s define the problem precisely, the way you would in a system design interview or a real design document.

2.1 Functional requirements

• Manage a close friends list: a user can add or remove followers from their personal close friends list at any time. The list is private — nobody can see who is on someone else’s close friends list, not even the people on it.
• Create restricted content: when creating a post, story, or status update, the user can mark it as “Close Friends only.” The system must then guarantee that only people on that specific list (at the time of viewing, or at time of posting — a design decision we will revisit) can see the content.
• Feed & profile visibility: the content must appear in the feed and profile view of close friends, and must be completely invisible — not just “blurred” or “locked,” but absent — to everyone else, including the platform’s own search and recommendation surfaces.
• Visual distinction: close-friends-only content should be visually marked (e.g., a colored ring or badge) for the audience who can see it, so they immediately understand its exclusivity.
• Interaction propagation: likes, comments, and replies on close-friends content must themselves respect the same visibility boundary — a comment thread on a restricted post cannot leak to non-members.

2.2 Non-functional requirements

• Low read latency: feed and profile reads that include an authorization check must remain in the tens of milliseconds, even though we’ve added an extra access-control layer.
• Strong privacy guarantee: a false negative (a close friend not seeing something they should) is annoying; a false positive (a random follower seeing something they should not) is a serious trust and safety failure. The system must be heavily biased toward correctness on the “who can see this” question.
• High write throughput: popular accounts can have millions of followers; a design that naively “fans out” a close-friends post the same way as a public post (writing a copy into every follower’s feed) would be wasteful, since only a tiny fraction of followers are eligible.
• List scalability: some users may have very large close-friends lists (creators sometimes use it as a “paid community” surrogate), so the list storage and lookup pattern must scale from lists of 5 people to lists of tens of thousands.
• Consistency of list changes: if a user removes someone from their close friends list, that person should lose access to future (and typically also past) restricted content promptly — not after an unpredictable caching delay.

2.3 What makes this hard

Most feed systems are optimized for the common case: “this post is public, show it to everyone who follows this account.” The close friends feature inverts a core assumption — visibility is no longer a function of the follow relationship alone, but of a second, much smaller, user-controlled relationship layered on top of it. Every read path that touches this content now needs an authorization check, and every write path needs to know, cheaply, who currently qualifies.

Before we draw any architecture diagrams, let’s get comfortable with the vocabulary and mental models this system relies on.

3.1 The close friends list as an “audience”

What: An audience is simply a named set of user IDs that a piece of content is restricted to. In our system there is exactly one implicit audience per user: their close friends list.

Why: Instead of thinking about “who can see post X” directly, it’s cleaner to think of each restricted post as pointing to an audience snapshot — a specific version of the poster’s close friends list at a specific moment.

Beginner example: Priya has 5 people on her close friends list: Ana, Bo, Cy, Dee, and Emi. She posts a photo restricted to close friends. Only those 5 people can see it.

Software example: The audience is represented as a foreign key relationship or an embedded snapshot — a close_friends_list_id plus a version number, or a materialized array of member IDs frozen at post-creation time.

Production example: Instagram’s Close Friends stores a single mutable list per user; when you post to Close Friends, the system does not necessarily freeze a snapshot — instead it typically evaluates membership dynamically, so if you add someone to your list an hour after posting, design choices vary on whether they retroactively gain access. We’ll explore both the “live” and “frozen” approaches later, because this single decision changes a lot of the architecture.

3.2 Push (fan-out-on-write) vs. Pull (fan-out-on-read)

What: These are the two classic strategies for delivering content to followers in any feed system.

• Fan-out-on-write (push): when a post is created, the system immediately writes a reference to that post into every eligible viewer’s personal feed/inbox (often stored in a fast key-value store like Redis or Cassandra). Reading the feed later is then just “read my own inbox” — extremely fast.
• Fan-out-on-read (pull): when a post is created, nothing is pushed anywhere. Instead, when a viewer opens their feed, the system queries “which of the accounts I follow posted something I’m allowed to see recently” and assembles the feed on the fly.

Why it matters here: Because a close friends list is usually a tiny fraction of a user’s total follower count (a creator might have 500,000 followers but only 40 close friends), fan-out-on-write becomes very cheap for this specific content type — we only write to 40 inboxes, not 500,000. This is one of the few places in a large feed system where push is a clear, cheap win.

3.3 Access Control List (ACL) evaluation

What: An ACL is a general computer science concept: a list attached to a resource that specifies who is allowed to do what with it. Here, our “resource” is a post, and the “permission” is “view.”

Why: Every read of a restricted post — whether from the feed, the profile grid, a direct link, search, or a notification — needs to pass through the same ACL check, otherwise you get an inconsistent, exploitable privacy boundary.

Practical example: Think of an office building where your badge (identity) is checked against a per-room access list (ACL) before a door unlocks — not just at the front entrance, but at every single door inside, including the fire escape and the freight elevator. If any door skips the check, that’s a security hole.

3.4 Eventual vs. strong consistency for list membership

What: When a user adds or removes someone from their close friends list, how quickly must that change propagate everywhere the list is used?

Why: Removing someone should feel immediate and permanent from a trust and privacy standpoint (strong consistency is preferred for removals), while adding someone can tolerate a small propagation delay (eventual consistency is acceptable for additions) without much user-facing harm.

Let’s now zoom out and look at the full system as a set of services. We’ll design this as a set of independently deployable microservices, communicating over gRPC/REST internally and via an event bus for asynchronous work.

4.1 Component responsibilities

4.2 Alternative architecture considered: embedding authorization inside the Post Service

A reasonable first instinct is to skip the separate Authorization Service entirely and simply have the Post Service itself decide, on every read, whether the requesting viewer is allowed to see a given post — after all, the Post Service already owns the visibility flag. This works fine at small scale, but it breaks down as more services need to make the same decision. Search needs to filter restricted posts out of its index. Notifications need to know who to notify. Comments need to inherit the parent post’s visibility. Profile views need to filter a user’s post grid. If each of these services calls into the Post Service just to ask “can X see post Y,” the Post Service becomes an availability bottleneck for the entire platform’s read path, coupling unrelated features to its uptime.

Extracting a dedicated, narrowly-scoped Authorization Service avoids this coupling: it owns exactly one responsibility (answering visibility questions quickly and correctly), can be scaled and cached independently of the heavier Post Service, and gives every other team in the organization one obvious, well-documented place to integrate rather than reinventing the check themselves. This is a specific instance of the general single-responsibility principle applied at the service-boundary level rather than the class level.

4.3 Why an event bus instead of direct service-to-service calls for fan-out

An equally reasonable alternative would have the Post Service call the Fan-out Service directly and synchronously as part of handling a post-creation request. This was deliberately avoided for two reasons. First, it would make the user-facing “create post” request latency dependent on fan-out completion time, which grows with audience size — a bad trade for a feature whose whole appeal is spontaneous, low-friction sharing. Second, it would tightly couple the two services’ availability: if the Fan-out Service were degraded, post creation itself would start failing, even though creating a post and delivering it to viewers are conceptually separable concerns. Introducing the event bus as a buffer decouples these lifecycles — the Post Service’s job ends the moment the event is durably published, and the Fan-out Service can catch up at its own pace, even under load, without ever blocking a user’s “Post” button.

5.1 List management internals

The close friends list is stored as a simple adjacency structure: for every user, a set of user IDs representing who they’ve added. Because followers can number in the millions but a close friends list rarely exceeds a few thousand entries, this is a small, hot dataset per user — perfect for storing in a fast key-value or wide-column store, indexed by owner ID, with the member IDs as either a native “set” data type (Redis Set) or a column-family row (Cassandra) for large scale.

public class CloseFriendsListService {

    private final CloseFriendsRepository repository;
    private final EventPublisher eventPublisher;

    public void addToCloseFriends(String ownerId, String memberId) {
        // Guard: you can only add someone who follows you
        if (!followGraphClient.isFollower(ownerId, memberId)) {
            throw new IllegalArgumentException(
                "Cannot add a non-follower to close friends list");
        }

        repository.addMember(ownerId, memberId);

        // Emit an event so downstream consumers (ACL cache, feed service)
        // can invalidate stale caches promptly.
        eventPublisher.publish(
            new CloseFriendsListChangedEvent(ownerId, memberId, ChangeType.ADDED)
        );
    }

    public void removeFromCloseFriends(String ownerId, String memberId) {
        repository.removeMember(ownerId, memberId);

        // Removals are latency-sensitive from a trust standpoint --
        // publish with high priority and force cache invalidation.
        eventPublisher.publishHighPriority(
            new CloseFriendsListChangedEvent(ownerId, memberId, ChangeType.REMOVED)
        );
    }
}

Notice the guard clause: you can only add followers to your close friends list, not arbitrary users. This keeps the feature scoped correctly and reuses the existing follow-graph data.

5.2 Post creation and the fan-out decision

When a post is created with visibility = CLOSE_FRIENDS, the Post Service does not try to resolve the audience itself. It simply persists the post with its visibility flag and publishes a PostCreated event onto the event bus. This keeps the write path for post creation fast and decoupled from the (potentially larger) fan-out work.

public class PostService {

    private final PostRepository postRepository;
    private final EventPublisher eventPublisher;

    public Post createPost(CreatePostRequest request) {
        Post post = Post.builder()
            .id(idGenerator.next())
            .authorId(request.getAuthorId())
            .content(request.getContent())
            .mediaUrl(request.getMediaUrl())
            .visibility(request.getVisibility()) // PUBLIC or CLOSE_FRIENDS
            .createdAt(Instant.now())
            .build();

        postRepository.save(post);

        eventPublisher.publish(new PostCreatedEvent(
            post.getId(),
            post.getAuthorId(),
            post.getVisibility()
        ));

        return post;
    }
}

Downstream, the Feed/Fan-out Service consumes this event. If the visibility is CLOSE_FRIENDS, it asks the Authorization Service to resolve the current close friends list for the author, and writes a lightweight feed entry (just the post ID and a timestamp, not the full content) into each eligible member’s personal feed store.

@KafkaListener(topics = "post-created")
public void onPostCreated(PostCreatedEvent event) {

    if (event.getVisibility() == Visibility.CLOSE_FRIENDS) {
        Set<String> eligibleViewers =
            authorizationClient.resolveCloseFriends(event.getAuthorId());

        // Small audience -> cheap to fan out on write
        for (String viewerId : eligibleViewers) {
            feedStore.pushEntry(
                viewerId,
                new FeedEntry(event.getPostId(), event.getAuthorId(), Instant.now())
            );
        }
    } else {
        // Public posts use a different, hybrid fan-out strategy
        // (fan-out-on-write for regular followers, fan-out-on-read
        // for celebrity accounts with huge follower counts).
        publicFanOutStrategy.fanOut(event);
    }
}

5.3 Reading a feed: defense in depth

Even though we already filtered the audience at write time, the read path performs a second, cheap authorization check before returning content. This “defense in depth” approach protects against stale feed entries (e.g., someone was removed from close friends after the fan-out already happened).

5.4 Content hydration: turning IDs into a real feed

Once the Feed Service has a filtered, ordered list of post IDs the viewer is allowed to see, it still needs to turn those bare IDs into something the client can render — captions, media URLs, like counts, author profile info. This step is called hydration, and it is deliberately kept separate from both the authorization check and the ID-lookup step, so each concern can be optimized and scaled independently.

public class FeedHydrationService {

    private final PostContentCache contentCache;
    private final PostRepository postRepository;

    public List<FeedItem> hydrate(List<String> approvedPostIds) {
        // Batch cache lookup first, to avoid N individual round trips
        Map<String, PostContent> cached = contentCache.multiGet(approvedPostIds);

        List<String> missingIds = approvedPostIds.stream()
            .filter(id -> !cached.containsKey(id))
            .collect(Collectors.toList());

        if (!missingIds.isEmpty()) {
            List<PostContent> fetched = postRepository.findAllByIds(missingIds);
            fetched.forEach(p -> contentCache.put(p.getId(), p));
            fetched.forEach(p -> cached.put(p.getId(), p));
        }

        // Preserve the original, already-authorized ordering
        return approvedPostIds.stream()
            .map(id -> FeedItem.from(cached.get(id)))
            .filter(Objects::nonNull) // handle rare race: post deleted mid-hydration
            .collect(Collectors.toList());
    }
}

Two details matter here. First, hydration happens strictly after authorization, never before — the content cache is a pure convenience layer for already-approved IDs, not a gate of any kind. Second, the batch multiGet call avoids the classic “N+1 query” anti-pattern, where fetching 50 feed items would otherwise trigger 50 separate database round trips instead of one.

5.5 Expiring story-type content

Many close-friends implementations mirror the ephemeral “story” format — content that disappears after 24 hours. Rather than running a constantly-polling cleanup job, this is best implemented using native TTL (time-to-live) support in the underlying storage engine, so expiry is handled by the database itself rather than by application code that could fail to run.

INSERT INTO posts (post_id, author_id, caption, media_url, visibility, created_at)
VALUES (?, ?, ?, ?, 'CLOSE_FRIENDS', toTimestamp(now()))
USING TTL 86400; -- 24 hours, expressed in seconds

-- Feed store entries referencing story content use a matching TTL
-- so a stale reference never outlives its underlying content.

Using native TTL avoids an entire class of bugs where a scheduled cleanup job falls behind, crashes silently, or double-processes records — the storage engine guarantees the row disappears on schedule regardless of application-layer health.

Let’s trace the complete lifecycle of a single close-friends post, end to end, the way you’d narrate it in a design review.

1. Compose: The user writes a caption, attaches media, and toggles “Share with Close Friends” in the client app.
2. Upload: Media is uploaded directly to object storage (e.g., S3-compatible storage) via a pre-signed URL, bypassing the application servers for the heavy bytes.
3. Create: The client calls POST /posts with the caption, media reference, and visibility=CLOSE_FRIENDS. The Post Service persists the row and returns a post ID immediately — the user sees “Posted!” right away.
4. Emit event: The Post Service asynchronously emits a PostCreated event to Kafka. The user-facing request does not wait for fan-out to complete.
5. Resolve audience: The Fan-out Consumer reads the event, calls the Authorization Service to fetch the author’s current close friends list.
6. Fan-out write: A lightweight feed entry (post ID + timestamp) is pushed into each eligible viewer’s feed store, typically completing within a second or two even for lists of thousands.
7. Notify: The Notification Service, listening to the same event, sends a push notification only to the resolved audience.
8. Read: When a close friend opens the app, their feed reads their own pre-populated feed store — an O(1) lookup — and each entry passes through the read-time re-verification described earlier.
9. Interact: Likes and comments on the post are themselves tagged with the post’s visibility; the Comment Service enforces the same ACL check before showing a comment thread.
10. List change (removal): If the author removes a viewer from their close friends list, a CloseFriendsListChangedEvent fires. The ACL cache entry for that (viewer, author) pair is proactively invalidated so the very next read reflects the new state.
11. Expiry (for story-type content): If the content is a 24-hour story, a scheduled job or TTL-based storage expiry removes both the underlying content and any feed entries referencing it.

6.1 Latency targets at a glance

Choosing the right storage engine for each piece of data is one of the most consequential decisions in this system. Let’s go table by table.

7.1 Close Friends List

Access pattern: “given a user, get their full list” (for management UI) and “given a user and a candidate viewer, is the viewer on the list” (for ACL checks) — both extremely frequent, low-latency reads with occasional writes. A wide-column store like Cassandra or a Redis Set is ideal.

TABLE close_friends (
    owner_id      UUID,
    member_id     UUID,
    added_at      TIMESTAMP,
    PRIMARY KEY (owner_id, member_id)
)
-- Partitioned by owner_id: all of one user's close friends
-- live on the same partition, making "get my list" a single
-- fast read. "is member_id in owner_id's list" is also a
-- single-partition point lookup.

7.2 Posts

TABLE posts (
    post_id       UUID PRIMARY KEY,
    author_id     UUID NOT NULL,
    caption       TEXT,
    media_url     TEXT,
    visibility    ENUM('PUBLIC','CLOSE_FRIENDS') NOT NULL,
    created_at    TIMESTAMP NOT NULL,
    expires_at    TIMESTAMP NULL,   -- for story-type content
    INDEX idx_author_created (author_id, created_at)
)

7.3 Feed store (per-viewer inbox)

KEY: feed:{viewer_id}
TYPE: Sorted Set
MEMBER: post_id
SCORE: created_at (epoch millis)

-- ZADD feed:viewer123 1732550400000 post_abc
-- ZREVRANGE feed:viewer123 0 49  → most recent 50 entries

Using a sorted set keyed by timestamp gives us O(log N) inserts and cheap “give me the latest 50” range queries — exactly the access pattern a feed needs.

7.4 Entity relationships at a glance

7.5 Choosing between SQL and NoSQL for each table

This is a case where a single database technology for the whole feature would be a mistake. The Posts table benefits from a relational database’s strong consistency guarantees and mature indexing for author-scoped time-range queries — a user’s own profile grid needs “all my posts, newest first,” which a B-tree index on (author_id, created_at) handles well. The Close Friends List and Feed Entry tables, by contrast, are accessed almost exclusively through a single partition key (owner_id or viewer_id respectively) with very high read throughput and simple point lookups — exactly the profile a wide-column or key-value store is built for, and exactly where a relational database’s stronger consistency guarantees would mostly go unused while its lower raw throughput would become a bottleneck.

8.1 Key trade-off: frozen snapshot vs. live evaluation

Most production systems land on a hybrid: cache membership with a short TTL (a few seconds to a couple of minutes) so reads stay fast, but propagate removals via an active cache-invalidation event so the “you’ve been removed” case doesn’t have to wait out the TTL.

8.2 Key trade-off: strong consistency vs. availability during a partition

The CAP theorem tells us that during a network partition, a distributed system must choose between consistency (every read sees the latest write) and availability (every request gets a response, even if it might be slightly stale). This system does not make one single global choice — it makes different choices for different data, based on the cost of being wrong in each case. For close-friends list removals, we lean toward consistency: it is better to briefly return an error or a conservative “deny” than to risk showing content to someone who was just removed. For public post visibility and general feed content, we lean toward availability: it is better to show a slightly stale feed than to show an error page, since the cost of staleness there is low and purely cosmetic. Recognizing that CAP trade-offs can and should be made independently per data type, rather than applied uniformly across an entire system, is one of the more mature judgments a system designer can bring to a design like this.

8.3 Key trade-off: simplicity of a single list vs. flexibility of multiple named groups

As the Real-World Examples section will explore in more depth, some platforms (Snapchat) chose to support multiple, independently named friend groups instead of Instagram’s single unnamed list. This is a genuine product and architecture trade-off, not just a UI difference: multiple named lists mean a post’s audience resolution becomes “union of members across N selected lists” rather than a single lookup, list-management UI becomes more complex, and the underlying storage schema needs an additional layer (a list-membership table keyed by list ID, plus a lists table per owner) rather than the simple owner-to-member adjacency structure used throughout this document. The single-list approach was chosen here deliberately because it maps to the dominant real-world use case (one trusted inner circle) with the least engineering and cognitive overhead, while remaining straightforward to extend into a multi-list model later if product requirements demand it — the owner-to-member table shown earlier could be extended with a list_id column with relatively little disruption if that day comes.

At small scale, none of this matters — a single database and a naive loop would work fine. The interesting engineering begins once you imagine a platform with hundreds of millions of users, some of whom are creators with tens of millions of followers.

9.1 Fan-out cost analysis

Fan-out cost is proportional to close friends list size, not follower count — this is the single most important scalability property of this feature. A celebrity with 50 million followers but 200 close friends triggers only 200 feed-store writes per post, not 50 million. Compare this to public post fan-out, where naive push-based fan-out to every follower would be catastrophically expensive for such accounts (this is why public post fan-out for huge accounts typically uses a hybrid pull-based strategy instead).

9.2 Little’s Law and capacity planning

To size the fan-out worker pool, we can apply Little’s Law: L = λ × W, where L is the average number of in-flight fan-out jobs, λ is the arrival rate of close-friends posts per second, and W is the average time to complete one fan-out job. If we expect λ = 500 posts/second platform-wide with restricted visibility, and each fan-out job (average list size ~150 members) takes W = 40ms to complete, then L = 500 × 0.04 = 20 concurrent jobs on average — a very manageable worker pool size, with headroom built in for spikes.

9.3 Horizontal scaling strategies

• Shard the Close Friends List store by owner_id so list reads and writes distribute evenly across nodes, avoiding hot partitions from a handful of viral accounts.
• Shard the Feed store by viewer_id for the same reason — each viewer’s feed lives on a predictable, evenly distributed partition.
• Scale the Fan-out Consumer horizontally as a Kafka consumer group — adding more consumer instances increases parallel processing of PostCreated events linearly, up to the number of partitions in the topic.
• Cap or paginate very large close-friends lists at the application layer (some platforms cap this list at a few thousand) to bound worst-case fan-out latency for pathological cases.

9.4 Handling the “mega close-friends list” edge case

A small but important edge case: what happens when a creator treats their close friends list less like a handful of real friends and more like a community — adding thousands, or even tens of thousands, of followers? This is a real pattern observed on several platforms, where creators use the “close friends” mechanic informally as a lightweight subscriber tier. The naive fan-out design (push a feed entry to every list member synchronously) starts to strain under this load: a single post could suddenly require 40,000 feed-store writes instead of 40, and if many such creators post around the same time (for example, a coordinated content drop), the Fan-out Consumer pool could see a sharp, correlated spike in load.

The standard mitigation is a size-based routing decision at the moment a PostCreated event is processed: if the resolved audience size is below a configured threshold (say, 5,000 members), fan-out proceeds as pure push, exactly as described earlier. If the audience exceeds that threshold, the system instead falls back to a hybrid strategy — writing a single lightweight marker into a shared “large-list post” index rather than fanning out individually, and letting affected viewers’ feed reads perform a cheap, targeted pull query against that index at read time. This keeps the worst-case write amplification bounded regardless of how large any individual list grows, at the cost of slightly higher read complexity for that one specific case.

9.5 Read-path scalability: pagination and cold caches

Feed reads must also scale gracefully for the pagination case — a user scrolling back through days of history rather than just checking their most recent updates. Because the feed store is a sorted set keyed by timestamp, pagination is a natural fit: each page request simply asks for the next timestamp range below the last item seen (a cursor-based approach), which avoids the classic “offset pagination” performance trap where requesting page 500 of a feed would otherwise require scanning and discarding the first 499 pages’ worth of entries.

Cold caches deserve explicit handling too. A user who has not opened the app in weeks will have an empty or stale feed store entry; rather than synchronously rebuilding their entire feed history on that first request (which could be slow and put unpredictable load on the Posts table), the system can serve a smaller, fast “recent activity” slice immediately while asynchronously warming the rest of the feed store in the background for subsequent requests — trading a small amount of initial completeness for a consistently fast perceived load time.

The system must stay correct and available even when individual components fail. Let’s cover the main failure modes.

10.1 Replication

Both the Close Friends List store and the Feed store should be replicated across at least three nodes (or across availability zones in the cloud), using leader-follower or leaderless replication depending on the datastore. Cassandra-style stores commonly use tunable, leaderless replication (e.g., quorum reads/writes with replication factor 3) so that the loss of one node does not interrupt reads or writes.

10.2 Graceful degradation of the ACL Service

If the Authorization Service becomes slow or partially unavailable, the system must fail closed, not open — meaning if we cannot confidently verify that a viewer is allowed to see a piece of restricted content, we hide it rather than show it. This is the opposite default from most availability-first systems (which typically fail open to preserve uptime), because here a privacy leak is a worse outcome than a temporarily incomplete feed.

10.3 Circuit breakers and bulkheads

The Feed Service should wrap calls to the Authorization Service in a circuit breaker. If the ACL Service starts timing out repeatedly, the breaker trips and the Feed Service falls back to a conservative default (omit close-friends content from the feed entirely, but keep serving public content) rather than letting the failure cascade and take down feed reads altogether. This is a direct application of the bulkhead pattern — isolating the blast radius of one dependency’s failure.

10.4 Disaster recovery

• Backups: The Close Friends List store and Posts table are backed up continuously (e.g., point-in-time recovery windows of 7–30 days), since this data represents real user trust relationships that cannot be recreated automatically.
• Feed store is treated as a rebuildable cache: because feed entries are derived data (they can be reconstructed by replaying recent PostCreated events against current list membership), the feed store does not need the same backup rigor — it can be repopulated from source-of-truth data after a catastrophic loss.
• Multi-region failover: for global platforms, the Posts and Close Friends List data is often replicated across regions with a defined RPO (recovery point objective) and RTO (recovery time objective), commonly targeting RPO under a minute and RTO under a few minutes for tier-1 social data.

10.5 Handling event bus failures

The Kafka-based event pipeline connecting post creation to fan-out and notifications is itself a potential single point of delay if not designed carefully. If the event bus becomes temporarily unavailable, the Post Service should still accept and persist new posts successfully — the user’s “Posted!” confirmation must not depend on the health of a downstream, asynchronous system. Events that fail to publish immediately are queued locally (using a durable outbox pattern backed by the same database transaction as the post write) and retried once the event bus recovers, guaranteeing that no post silently fails to fan out simply because of a transient messaging outage.

10.6 The transactional outbox pattern

A subtle but important reliability detail: writing a post to the database and publishing a PostCreated event are two separate operations against two separate systems (the Posts database and Kafka). If the service crashes between these two steps, you can end up with a post that exists but was never fanned out — a silent, hard-to-detect bug. The transactional outbox pattern solves this by writing the event, as a row in an “outbox” table, within the very same database transaction that creates the post. A separate, simple relay process then reads unpublished outbox rows and forwards them to Kafka, retrying until successful. Because the post write and the event write are now atomic with respect to each other, there is no window in which one can succeed without the other.

10.7 Testing reliability: chaos engineering

Beyond passive monitoring, mature teams deliberately inject failure into staging (and sometimes production, carefully) environments to validate that the system behaves as designed under stress — a practice known as chaos engineering. For this feature specifically, valuable chaos experiments include: killing an Authorization Service instance mid-request to confirm the circuit breaker trips and the system fails closed as intended; introducing artificial network latency between the Fan-out Consumer and the Feed store to confirm consumer lag alerts fire correctly; and simulating a full close-friends-list-store partition outage to confirm reads degrade gracefully (hiding restricted content) rather than crashing the entire feed-read path for all users, including those only viewing public content.

This feature exists entirely because of a privacy promise, so security is not a bolt-on concern here — it is the core value proposition.

11.1 The list itself is private

Nobody, including the people on someone’s close friends list, should be able to see the full list or even confirm who else is on it. The API must never expose a “list all members” endpoint to anyone except the owner, and audit logging should flag any anomalous access pattern (e.g., an internal tool querying many users’ lists in bulk).

11.2 Enforcing the ACL at the data layer, not the presentation layer

As discussed earlier, filtering must happen server-side, ideally as close to the data source as possible, never trusting the client to simply “not render” restricted content. A malicious or modified client, a leaked API response, or a misconfigured cache could otherwise expose data that was only meant to be hidden by the UI.

public class AuthorizationService {

    private final CloseFriendsRepository closeFriendsRepository;
    private final Cache<String, Boolean> membershipCache;

    public boolean canView(String viewerId, Post post) {
        if (post.getVisibility() == Visibility.PUBLIC) {
            return !blockService.isBlocked(post.getAuthorId(), viewerId);
        }

        if (post.getVisibility() == Visibility.CLOSE_FRIENDS) {
            if (viewerId.equals(post.getAuthorId())) {
                return true; // authors can always see their own posts
            }
            String cacheKey = post.getAuthorId() + ":" + viewerId;
            Boolean cached = membershipCache.getIfPresent(cacheKey);
            if (cached != null) return cached;

            boolean isMember = closeFriendsRepository
                .isMember(post.getAuthorId(), viewerId);
            membershipCache.put(cacheKey, isMember); // short TTL, e.g. 30s
            return isMember;
        }
        return false; // fail closed for unknown visibility types
    }
}

11.3 Preventing enumeration and side-channel leaks

• Consistent error responses: requesting a restricted post you cannot see should return the same “not found” response as requesting a post that genuinely does not exist — never a distinguishable “forbidden” response that would let an attacker confirm the post’s existence.
• Search and recommendation isolation: restricted posts must be excluded from any index that a non-eligible user could query, including full-text search and “explore” style recommendation feeds.
• Notification content minimization: a push notification about a close-friends post should avoid echoing the caption or preview image on a locked screen where a stranger might glance at it.
• Rate limiting on list management endpoints to prevent automated scraping attempts that try to infer list membership through timing or side channels.

11.4 Encryption and access hygiene

• All data in transit uses TLS; sensitive list data at rest is encrypted using standard disk/volume-level encryption at minimum, with field-level encryption considered for extremely sensitive deployments.
• Internal service-to-service calls (Feed Service → Authorization Service) use mutual TLS and short-lived service credentials, following the principle of least privilege — the Feed Service can ask “can X see Y” but cannot bulk-export list data.
• Every authorization decision is logged (without leaking content) for audit purposes, so a security review can reconstruct who accessed what and when if an incident is suspected.

11.5 Regulatory and compliance considerations

Because a close friends list is, by definition, sensitive relationship data about real people, it typically falls under the same regulatory umbrella as other personal data — for example, GDPR in the European Union or comparable data-protection regimes elsewhere. Two obligations are especially relevant to this feature’s design. First, the right to erasure: when a user deletes their account, both their close friends list and any lists they belong to as a member (i.e., other users’ lists referencing them) must be cleaned up, which argues for storing the relationship bidirectionally-indexable so a deletion job can efficiently find and remove all references to a given user ID rather than scanning every other user’s list. Second, the right to data portability/access: a user requesting an export of their own data should be able to receive their own close friends list, but the system must be careful that fulfilling this request never inadvertently discloses another user’s private list membership in the process.

These regulatory requirements are a good illustration of why the earlier decision to partition the Close Friends List table by owner_id is not purely a performance optimization — it also happens to align well with “find and delete everything this user owns” compliance workflows. Good data-modeling decisions frequently pay off in more than one dimension at once.

Because a silent privacy leak is far more dangerous than a visible outage, observability for this feature has to go beyond typical uptime and latency dashboards — it needs privacy-specific signals too.

12.1 Key metrics to track

12.2 Logging practices

• Structured logs for every ACL decision, including the outcome (allow/deny) and the reason, but never the content itself — logs should be safe to inspect broadly without becoming a secondary leak vector.
• Correlation IDs threaded through the post-creation → fan-out → notification pipeline so an on-call engineer can trace a single post’s journey across services during an incident.
• Separate, tightly access-controlled audit logs for any administrative or support-tooling access to list data, since these tools are a common blind spot for privacy incidents.

12.3 Alerting

Two categories of alerts deserve paging-level urgency: (1) any canary test that detects unauthorized access to restricted content, treated as a security incident, not a routine bug; and (2) fan-out consumer lag exceeding a defined threshold (e.g., more than 30 seconds of backlog sustained for 5 minutes), since delayed fan-out directly harms the feature’s core promise of timely, trusted sharing.

12.4 Building an operational dashboard

A well-designed on-call dashboard for this feature groups metrics into three tiers, so an engineer responding to an alert at 3 a.m. can triage quickly without hunting across a dozen unrelated panels. The top tier shows privacy-critical signals — canary test pass rate and any anomalous ACL “allow” spikes — always visible, always the first thing checked, since these represent the worst possible category of incident. The middle tier shows pipeline health — fan-out consumer lag, dead-letter queue depth, event-bus publish success rate — representing degraded-but-not-unsafe states. The bottom tier shows standard service health — request latency percentiles, error rates, and instance CPU/memory for each microservice — the same category of metric you’d track for any service, privacy-sensitive or not. Structuring the dashboard by severity tier, rather than by service name alone, helps responders instinctively understand not just “what broke” but “how bad is this,” which materially shortens incident response time during a real production issue.

Each service in this architecture is deployed independently as a containerized microservice (e.g., Docker images orchestrated by Kubernetes), which allows the Fan-out Service, prone to bursty load around viral moments, to scale independently from the more steady-state Close Friends List Service.

13.1 Deployment strategy: canary releases

Given that a regression in the Authorization Service could cause a privacy incident, changes to it are rolled out via canary deployment: a new version receives a small percentage of production traffic (e.g., 1–5%) while the previous stable version continues serving the rest. Automated canary analysis compares error rates, latency, and — critically — the privacy canary test results between the old and new versions before progressively increasing traffic to the new version. Blue-green deployment is used as an alternative for services where an instant, full cutover with a fast rollback path is preferred over gradual traffic shifting.

13.2 Infrastructure as Code

All infrastructure (Kubernetes manifests, Kafka topic configuration, database provisioning, IAM roles) is defined declaratively (e.g., Terraform, Helm charts) and version-controlled, so environments are reproducible and every infrastructure change goes through the same review process as application code.

13.3 Multi-region considerations

• Feed reads are served from the region closest to the viewer for latency, with the underlying data asynchronously replicated across regions.
• List-change events (particularly removals) are given priority replication to minimize the window where a removed viewer in a different region might still see cached content.
• Data residency requirements (e.g., regional privacy regulations) may require pinning certain users’ list and post data to specific regions rather than freely replicating globally.

13.4 Cost optimization

Because feed-store data is derived and rebuildable, it can live on cheaper, ephemeral-friendly infrastructure with aggressive TTL-based expiry for story-type content, while the durable Posts and Close Friends List data justify the higher cost of fully backed-up, multi-AZ storage. Right-sizing the Fan-out Consumer pool with autoscaling (rather than provisioning for peak year-round) further reduces idle compute cost during non-peak hours.

It is worth explicitly separating two categories of infrastructure cost when planning a budget for this feature: the cost that scales with total user base (list storage, which grows slowly and predictably as accounts are created) and the cost that scales with engagement volume (fan-out compute and feed-store writes, which spike unpredictably around product launches, viral moments, or regional time-zone peaks). Autoscaling policies should be tuned differently for each — list storage capacity planning can use simple, gradual forecasting, while fan-out compute needs aggressive, fast-reacting autoscaling rules (scaling on a short window of consumer lag rather than a long window of average CPU) so the system can absorb a sudden burst of posting activity without falling behind and accumulating a backlog that then takes hours to work through even after the traffic spike subsides.

14.1 What gets cached, and for how long

14.2 Cache invalidation pattern

We use an event-driven, write-through invalidation pattern rather than relying purely on TTL expiry: when a CloseFriendsListChangedEvent (specifically a removal) is published, the Authorization Service immediately deletes the corresponding cache entries for that (owner, member) pair across all cache nodes, rather than waiting for the TTL to lapse naturally. This bounds the “stale access window” to the propagation time of the event (typically well under a second) instead of the full cache TTL.

14.3 Load balancing

• API Gateway / Load Balancer: distributes incoming client traffic across stateless service instances using round-robin or least-connections algorithms, with health checks removing unhealthy instances automatically.
• Consistent hashing for the feed store: viewer feed data is distributed across cache/database nodes using consistent hashing, so adding or removing nodes only reshuffles a small fraction of keys instead of the entire dataset.
• Read replicas: the Close Friends List store uses read replicas to absorb the high read volume from ACL checks, while writes (adding/removing members) go to the primary and replicate asynchronously with monitored replication lag.

14.4 Guarding against cache stampedes

A cache stampede happens when a popular cache entry expires and a large burst of concurrent requests all miss the cache at the same moment, hammering the underlying database simultaneously. This is a realistic risk here: imagine a creator with a large close-friends list posts something, and thousands of eligible viewers open the app within the same few seconds — if all of their ACL cache entries happened to expire together, the Close Friends List database could see a sudden, correlated spike. The standard mitigation is to add small random jitter to cache TTLs (so entries expire at slightly staggered times rather than all at once) combined with a “single-flight” pattern at the Authorization Service layer, where only the first request for a given missing cache key actually queries the database, while concurrent requests for that same key wait for and reuse that single in-flight result instead of each issuing their own redundant query.

14.5 Hot key mitigation

Because cache and feed-store keys are partitioned by owner or viewer ID, an extremely popular account can create a “hot key” — a single partition receiving disproportionate read traffic relative to the rest of the cluster. For the Close Friends List store, this is less severe than it would be for follower-graph data, since close-friends lookups happen at a much smaller scale than follower-scale operations. But for viral posts, the mitigation strategies remain useful to know: request coalescing (as described above), local in-process caching at each service instance as a first layer before hitting the shared cache, and, in extreme cases, deliberately replicating a single hot key’s data across multiple physical shards with client-side load distribution across the replicas.

15.1 Representative REST API surface

15.2 Why microservices, and where the boundaries are drawn

The service boundaries follow the classic guideline of splitting along independent scaling and change patterns. The Fan-out Service scales with post volume and audience size; the Close Friends List Service scales with list read/write volume; the Authorization Service is a low-latency, extremely hot dependency called by nearly everything else, so it’s kept minimal and deliberately simple to keep its P99 latency low and predictable. Splitting these apart lets each team iterate and scale independently without a change to, say, notification logic requiring a redeploy of the core authorization path.

15.3 Inter-service communication

• Synchronous (gRPC): used for latency-sensitive request/response calls like “Feed Service asks Authorization Service to verify a batch of post IDs” — gRPC’s binary protocol and HTTP/2 multiplexing keep this fast.
• Asynchronous (Kafka events): used for anything that can tolerate a small delay and benefits from decoupling — post creation triggering fan-out, list changes triggering cache invalidation, and posts triggering notifications.

15.4 API design details worth calling out

A few smaller API design choices carry outsized importance for this feature. The GET /close-friends endpoint should never accept an arbitrary user ID parameter — it must always resolve implicitly to the authenticated caller’s own ID, extracted from their session token, never trusted from client-supplied input. This closes off an entire class of “insecure direct object reference” vulnerabilities where an attacker could otherwise simply change a user ID in the request URL to view someone else’s private list.

The POST /posts endpoint’s response should be idempotent under retry — mobile clients on flaky networks frequently retry requests that appear to have failed but actually succeeded server-side. Including a client-generated idempotency key in the request lets the Post Service recognize and safely ignore a duplicate submission rather than creating two identical posts and triggering two separate fan-out events for the same content.

Batch-oriented endpoints matter too: rather than the Feed Service calling the Authorization Service once per post ID (which would mean dozens of round trips to render a single feed page), the gRPC contract accepts a batched list of post IDs and returns a batched list of allow/deny decisions in a single call, dramatically cutting network overhead for the single hottest read path in the whole system.

15.5 Service ownership and organizational alignment

Beyond the purely technical reasoning, service boundaries in a mature organization tend to also reflect team ownership boundaries — a pattern often summarized as Conway’s Law, the observation that system architecture tends to mirror the communication structure of the organization that builds it. In practice, the Close Friends List Service and Authorization Service are often owned by a “trust and privacy” or “core identity” team responsible for access-control correctness across the entire platform, not just this one feature, while the Feed and Fan-out services are owned by a separate “feed and ranking” team focused on relevance, latency, and engagement. Structuring the architecture this way lets each team move independently while still sharing a single, authoritative source of truth for authorization decisions.

16.1 Patterns applied

16.2 Anti-patterns to avoid

17.1 Best practices

• Centralize authorization logic in a single, well-tested service or library — treat it as security-critical infrastructure, with its own on-call rotation and stricter review requirements.
• Write automated, continuously running “privacy regression tests” that specifically try (and expect to fail) to access restricted content as a non-member, run against every environment including production.
• Design cache keys around the (viewer, resource) pair, never the resource alone, whenever the resource has non-public visibility.
• Prioritize propagation speed for removals over additions — the cost of a slightly delayed addition is low, but a slow removal directly undermines user trust.
• Log authorization decisions with enough context to audit an incident, but never log the protected content itself.

17.2 Common mistakes

• Treating this as “just a filter”: underestimating how many surfaces (search, notifications, deep links, shared previews, export tools) need to respect the same boundary.
• Over-caching for performance without an invalidation strategy: chasing low latency by caching ACL decisions for minutes or hours, forgetting that a removal needs near-immediate effect.
• Ignoring the “who can add whom” constraint: allowing arbitrary users (not just existing followers) to be added to a close friends list opens the door to confusing, exploitable states.
• Forgetting comments and reactions: restricting the post itself but leaving its comment thread or reaction counts visible through a separate, unguarded API.
• Not load-testing large lists: assuming all lists are small (a handful of friends) and being surprised when a creator with a 20,000-person “close friends” list causes a fan-out latency spike.

17.3 A checklist for reviewing a design like this

When reviewing an architecture or a pull request that touches this feature, it helps to work through a short, deliberate checklist rather than relying purely on general code-review instinct, since privacy bugs are easy to miss in an otherwise well-written change:

• Does every new read path that could return this content call through the central Authorization Service, rather than re-implementing its own visibility check?
• Does any new cache introduced by this change key on the (viewer, resource) pair, not the resource alone?
• If the Authorization Service or its cache is unavailable, does this new code path fail closed (hide content) rather than fail open (show content)?
• Does a close-friends-list removal propagate to this new code path within the expected latency window, or does it rely on a long TTL that could leave a stale access window open?
• Is there a canary or automated test covering this specific new surface, so a future regression here would be caught automatically rather than relying on manual QA or a user report?

Teams that bake a checklist like this into their standard review process for privacy-sensitive features tend to catch far more issues before launch than teams relying solely on ad hoc review judgment, precisely because it turns tacit domain knowledge (the kind built up over the course of a document like this one) into an explicit, repeatable practice that does not depend on any single reviewer remembering every subtle failure mode.

Across these examples, a consistent lesson emerges: the products that succeeded kept the user-facing mental model extremely simple (one list, one toggle) while the underlying system handled the real complexity — audience resolution, fan-out efficiency, and airtight access control — invisibly. Complexity was pushed into the architecture, not onto the user.

18.1 What Instagram’s launch teaches about rollout strategy

When Instagram shipped Close Friends in 2018, it did not launch to every user simultaneously. Like most features at that scale, it rolled out progressively — first to a small percentage of accounts, then expanded region by region while engineering teams watched fan-out latency, error rates, and (critically) any signal of a visibility leak. This mirrors the canary deployment strategy described earlier in this document: a feature whose entire value proposition rests on a privacy guarantee cannot be safely validated with load testing alone — it needs a careful, observable, reversible rollout against real production traffic at increasing scale, with an easy kill-switch if anything looks wrong.

18.2 Lessons from Google+ Circles’ decline

It’s worth studying the failure case as closely as the successes. Google+ Circles was, in some ways, technically more powerful than what Instagram eventually built — it allowed unlimited named groups, cross-posting to multiple circles at once, and fine-grained per-circle settings. But usage data showed most people never organized more than one or two circles, and the feature’s complexity became a barrier rather than a benefit. The system design lesson generalizes well beyond this specific feature: a technically superior access-control model that increases user cognitive load will often lose to a simpler model that covers 90% of the real use case with a fraction of the setup friction. When designing the audience model for a system like this, it is worth deliberately resisting the temptation to add configurability that most users will never touch, since every added dimension of configuration is also an added dimension of potential misconfiguration and privacy risk.

A “close friends” selective sharing system looks like a small feature from the outside — a list and a toggle — but it is a genuinely rich distributed systems problem once you look underneath. It touches access control, fan-out strategy, caching invalidation, consistency trade-offs, and defense-in-depth security design, all under the constraint that mistakes here are privacy incidents, not just bugs.

If you take one architectural principle away from this entire design, let it be this: visibility rules belong as close to the data as possible, and must be re-verified at the moment of access, not assumed from an earlier decision. Every other design choice in this system — from schema shape to caching strategy to deployment rollout gates — exists in service of that single guarantee.

It is also worth remembering that none of the individual techniques covered here — event-driven fan-out, centralized authorization, defense in depth, circuit breakers, TTL-based expiry, canary deployments — are unique to a close-friends feature. They are general-purpose distributed systems tools that show up again and again across very different products: a subscriber-only content tier, a private group-chat visibility model, an enterprise document-sharing permission system, or a healthcare application’s patient-data access controls all rest on essentially the same foundation described in this document. Once you understand how to reason about audience resolution, fan-out cost, and fail-closed authorization for something as small in scope as a close friends list, you have effectively learned the pattern for a much larger family of access-controlled, at-scale systems — which is exactly why this seemingly modest feature makes such a rich subject for a full system design study.