Amazon SNS, Deconstructed
A ground-up, advanced-level tour of how Amazon Simple Notification Service actually routes, retries, secures, and scales trillions of messages a year — for engineers who already know what a topic and a subscription are and want to go deeper.
Amazon Simple Notification Service (SNS) is one of the oldest services in AWS, and also one of the most quietly misunderstood. Most engineers meet it as “the thing that sends an email or triggers a Lambda when something happens.” That surface description is true, but it hides a genuinely sophisticated distributed pub/sub system: one that has to accept a publish call in milliseconds, fan that single message out to potentially thousands of independent subscribers, retry each of those deliveries independently, respect per-subscriber filtering, and do all of this across three or more Availability Zones without ever losing a message or blocking the publisher. This guide skips the “what is a topic” basics and goes straight into the advanced mechanics: how SNS is built internally, how messages actually flow through the system, where the durability guarantees come from, how the security model really works at the policy-evaluation level, and the patterns and anti-patterns that separate a production-grade event-driven architecture from a fragile one.
AAdvanced Core Concepts
We assume you already know that SNS is a publish/subscribe service. This chapter covers the concepts that matter once you’re building real systems on top of it: message structure at the wire level, filter policy evaluation, FIFO ordering guarantees, and the subtle difference between a topic’s logical identity and its physical implementation.
The topic as a routing contract, not a queue
A common mental error is treating an SNS topic like a buffer that holds messages. It does not. A topic is a routing contract: a named rule that says “when a message arrives here, evaluate it against every current subscription and hand a copy to each one that matches.” There is no persistent message store behind a standard topic. If zero subscriptions exist at publish time, the message is accepted, routed against an empty subscription set, and discarded. This is fundamentally different from Amazon SQS, where a queue retains messages until a consumer deletes them.
Think of a standard SNS topic as a live radio broadcast tower, not a mailbox. The tower doesn’t store the broadcast — it transmits it the instant it’s received, to every radio that is tuned in at that exact moment. A radio that was switched off during the broadcast never hears it, no matter how it’s switched on later. That’s why durable consumption of SNS fanout usually pairs the topic with an SQS queue “radio” that does store the message.
Message attributes and the filter policy evaluation model
Every SNS message can carry up to 10 message attributes — key/value metadata separate from the message body (String, Number, or Binary typed). Subscribers attach a filter policy, a JSON document that describes which attribute values they care about. At publish time, SNS evaluates the filter policy against the message attributes (or, in newer filter-policy-scope configurations, against the message body itself) using an AND-across-keys, OR-within-values logic: every key in the policy must match, but a key can match any one of several listed values, numeric ranges, prefixes, or existence checks.
This filtering happens before delivery, inside the SNS service itself — the subscriber’s endpoint never even sees a message it filtered out. That distinction matters at advanced scale: filtering is a free, server-side operation that reduces cross-service traffic, cost, and unnecessary invocations, rather than a client-side responsibility.
Best-effort ordering
Extremely high throughput, at-least-once delivery, no ordering guarantee across messages, occasional duplicates possible.
Strict ordering per group
Exactly-once publish semantics, strict FIFO delivery within a message group, must be paired with FIFO SQS subscribers for end-to-end ordering.
The ordering unit
In FIFO topics, ordering is guaranteed only within a message group — parallelism across groups is what keeps FIFO topics scalable.
Exactly-once publish
Either supplied explicitly or derived via content-based deduplication (a SHA-256 hash of the body) within a 5-minute deduplication interval.
Delivery protocols are not interchangeable citizens
SNS supports HTTP/S, Amazon SQS, AWS Lambda, mobile push (APNs, FCM), SMS, email/email-JSON, and Kinesis Data Firehose as subscriber protocols. Advanced designs treat these very differently: SQS and Lambda subscriptions get SNS’s full retry and dead-letter-queue machinery; HTTP/S endpoints get an exponential backoff retry policy you configure yourself; SMS and mobile push have no DLQ concept at all because they’re inherently best-effort, human-facing channels. Choosing the wrong protocol for a durability-sensitive workflow is one of the most common design mistakes in production SNS usage.
BInternal Working
AWS doesn’t publish SNS’s internals in full, but its documented behavior, published limits, and observable failure modes let us reconstruct the architecture with a high degree of confidence.
graph LR
P[Publisher] -->|Publish API call| LB[Regional API Endpoint / Load Balancer]
LB --> AC[Accept & Persist Layer]
AC --> RT[Routing Engine]
RT -->|Filter policy match| SQ1[Subscriber: SQS Queue]
RT -->|Filter policy match| LM1[Subscriber: Lambda Function]
RT -->|Filter policy match| HT1[Subscriber: HTTPS Endpoint]
RT -.->|No match| DR[Discarded - no delivery]
SQ1 --> RD1{Delivery Success?}
LM1 --> RD2{Delivery Success?}
HT1 --> RD3{Delivery Success?}
RD1 -->|No, retries exhausted| DLQ1[Dead-Letter Queue]
RD2 -->|No, retries exhausted| DLQ2[Dead-Letter Queue]
RD3 -->|No, retries exhausted| DLQ3[Dead-Letter Queue]
Fig 2.1 — Publish, route, fan-out, and per-subscriber retry paths inside SNS
When a publisher calls the Publish API, the request lands on a regional, horizontally-scaled fleet of stateless API hosts. That layer authenticates the caller against IAM, validates the payload against size limits (256 KB per message, including attributes), and hands the message to an internal durable storage and routing tier. AWS has described SNS as being built on top of a distributed, redundantly-replicated storage layer that persists the message across multiple Availability Zones before the publish call returns success to the caller — this is what makes the publish operation itself durable, even though the topic doesn’t retain the message afterward for late subscribers.
The routing engine then evaluates every active subscription against the message’s attributes in parallel. Because subscriptions can number in the thousands per topic, this fan-out step is designed to scale horizontally and independently of the publish path — a topic with 10 subscribers and a topic with 10,000 subscribers should see comparable publish latency, because the expensive fan-out work happens asynchronously after the durability guarantee is already satisfied.
“Why does SNS publish latency stay flat as subscriber count grows?” — because the publish call only needs to guarantee durable, replicated acceptance of the message; the fan-out to N subscribers is decoupled, asynchronous, and independently retried per subscriber, so it never sits on the publisher’s critical path.
Per-subscriber delivery workers
Each subscription is served by its own logical delivery pipeline with its own retry state. This is why one subscriber having an outage (say, a Lambda function that’s throttled) never slows down or blocks delivery to the other 999 subscribers on the same topic — the failures are isolated per subscription, not per topic.
CData Flow & Lifecycle
Tracing a single message from the moment a producer calls Publish to the moment it either lands successfully or dies in a dead-letter queue reveals the full lifecycle SNS manages on your behalf.
Publish & validate
Caller’s IAM identity is checked against the topic policy; payload size, attribute count, and (for FIFO) group/dedup IDs are validated.
Durable acceptance
The message is redundantly stored across multiple AZs. The Publish API returns a MessageId only after this succeeds — this is the durability guarantee’s origin point.
Filter evaluation
Every active subscription’s filter policy is evaluated against the message. Non-matching subscriptions are skipped entirely — no attempt, no charge, no log entry on their side.
Per-protocol delivery attempt
SNS formats the payload for the target protocol (raw JSON envelope by default, or raw message body if “raw message delivery” is enabled for SQS/HTTP subscriptions) and attempts delivery.
Retry with backoff
On failure, SNS retries according to a protocol-specific default policy (or a custom delivery policy you define): immediate retries, then a backoff schedule, for a bounded retry window — commonly up to 23 further attempts over roughly 20 hours for HTTP/S by default, though this is configurable.
Dead-letter routing or drop
If a redrive policy with a dead-letter queue is configured and retries are exhausted, the message is placed on that DLQ for later inspection. Without a DLQ configured, an exhausted message is simply dropped — silently, unless you’re watching CloudWatch metrics.
Context
A team assumes SNS “retries forever” and skips configuring a redrive policy on a critical HTTPS webhook subscription.
Consequence
When the receiving endpoint has an extended outage beyond the retry window, every message published during that outage is silently and permanently lost — with no DLQ, there’s no record they ever existed.
Resolution
Always attach a redrive policy pointing at an SQS dead-letter queue to every subscription that carries business-critical events, regardless of protocol.
DAdvantages, Disadvantages & Trade-offs
Advantages
- True one-to-many fan-out with no code to manage multiple delivery targets
- Fully managed, serverless — no brokers, partitions, or clusters to size
- Native filter policies push routing logic out of application code
- Deep native integration with Lambda, SQS, Kinesis Firehose, and mobile push providers
- FIFO topics bring strict ordering to a pub/sub model when paired with FIFO queues
Disadvantages / Trade-offs
- No built-in message retention for slow or offline subscribers on standard topics
- 256 KB message size ceiling requires an offloading pattern for larger payloads
- No consumer-side polling or replay — a subscriber must exist and be healthy at publish time, or pair with SQS
- Cross-subscriber ordering across message groups is not guaranteed even in FIFO topics
- Cost can grow non-obviously with very high fan-out ratios (per-delivery pricing, not per-publish)
Production example — Netflix
Netflix has publicly discussed using SNS/SQS fan-out patterns as part of its event-driven microservices backbone, where a single upstream event (like a playback state change) needs to reach many independent downstream services — recommendation updates, billing checks, analytics pipelines — without the publisher needing to know who’s listening.
EPerformance & Scalability
SNS’s scaling model is worth understanding at the mechanism level, not just as a marketing claim of “unlimited scale.”
Standard topics scale essentially horizontally and elastically: AWS does not publish a hard topic-level publish TPS ceiling for standard topics because the service is designed to absorb bursty, unpredictable traffic across a shared multi-tenant fleet. FIFO topics, by contrast, have documented throughput ceilings per topic (measured in messages per second, higher with batching), because strict per-group ordering requires more coordinated, less parallelizable processing than best-effort delivery.
Standard topics behave like a general highway with as many lanes as traffic demands — congestion is smoothed by adding capacity behind the scenes. A FIFO topic is more like a single-file toll bridge for each message group: individually reliable and perfectly ordered, but the bridge for any one group can only carry so many cars per second, which is why sharding work across many message groups is the standard scaling technique for FIFO topics.
Batching as a scalability lever
The PublishBatch API lets a producer submit up to 10 messages in a single call. At advanced scale, this is not just a convenience — it materially reduces per-message API overhead and is often the difference between a producer service being CPU/network-bound on outbound calls versus comfortably absorbing traffic spikes.
FHigh Availability & Reliability
SNS is a regional service that replicates data across multiple Availability Zones within that region automatically — there is no “Multi-AZ mode” to opt into, unlike, say, RDS. This means a single AZ failure does not take the topic down, and no customer action is needed to achieve that baseline resilience.
“SNS guarantees delivery.” It does not guarantee delivery to a subscriber that is permanently unreachable — it guarantees at-least-once delivery attempts with retries, and durable handling of the message up to the point of accepted publish. True end-to-end reliability requires you to pair SNS with SQS and a dead-letter queue for anything that must never be silently lost.
Cross-region reliability
SNS topics are region-scoped; there is no native cross-region replication of a topic. Multi-region architectures typically publish independently to a topic in each region, or replicate messages downstream via SQS/Lambda/Firehose subscriptions that write cross-region, since SNS itself does not fail over a topic between regions automatically.
Production example — Amazon.com order pipeline
Amazon’s own retail order processing systems have referenced SNS/SQS decoupling patterns where an order-placed event fans out to inventory, fraud detection, and shipping systems independently, so a slowdown in one downstream system (say, fraud scoring under load) never blocks or delays the others.
GSecurity
SNS security operates on two layers that advanced practitioners must keep distinct: identity-based IAM policies attached to users/roles, and resource-based topic policies attached to the topic itself.
Topic policies and cross-account access
A topic policy is evaluated independently of, and in addition to, any IAM identity policy. This is what allows cross-account publishing — Account B can be granted sns:Publish on a topic owned by Account A purely through the topic’s resource policy, without Account A needing to touch Account B’s IAM at all. Both the identity policy (if one applies) and the resource policy must allow the action; an explicit Deny in either overrides any Allow.
graph TD
IAM[IAM Identity Policy on Caller] -->|Must Allow, if scoped| EVAL{Combined Evaluation}
RP[Topic Resource Policy] -->|Must Allow| EVAL
EVAL -->|Any explicit Deny| DENY[Request Denied]
EVAL -->|Both Allow, no Deny| ALLOW[Publish/Subscribe Permitted]
Fig 7.1 — Dual-layer policy evaluation for SNS actions
Encryption
SNS supports server-side encryption (SSE) using AWS KMS customer managed keys or the AWS managed key, encrypting the message body at rest within the service. In-transit protection is provided via HTTPS/TLS endpoints; publishing over plaintext HTTP is possible for HTTP subscriber endpoints but strongly discouraged for anything carrying sensitive data.
Combine SSE-KMS on the topic, a least-privilege topic policy scoped to specific principals and source ARNs, and VPC endpoints (interface endpoints via AWS PrivateLink) so publishers inside a VPC never need to traverse the public internet to reach SNS.
HMonitoring, Logging & Metrics
SNS emits detailed CloudWatch metrics per topic: NumberOfMessagesPublished, NumberOfNotificationsDelivered, NumberOfNotificationsFailed, and protocol-specific metrics like SMSSuccessRate. Advanced observability setups alarm specifically on NumberOfNotificationsFailed and DLQ queue depth — publish success alone tells you nothing about whether subscribers actually received their messages.
| Metric | What it reveals | Alarm on |
|---|---|---|
| NumberOfNotificationsFailed | Delivery attempts exhausted for a subscription | Any sustained non-zero value |
| NumberOfNotificationsFilteredOut | Messages skipped by filter policies | Unexpected spikes (policy misconfiguration) |
| PublishSize | Payload size trends | Approaching the 256 KB ceiling |
| SMSMonthToDateSpentUSD | SMS spend guardrail | Approaching your account SMS spend limit |
SNS Delivery Status logging can be enabled per subscription to push successful and failed delivery attempts into CloudWatch Logs, including HTTP response codes for HTTP/S endpoints — indispensable when debugging a webhook subscriber that claims it “never gets anything.”
IDeployment & Cloud Integration
In practice, SNS rarely sits alone — it’s the fan-out layer in a chain. The canonical “fan-out” pattern pairs one SNS topic with multiple SQS queues as subscribers, so each downstream service gets its own durable, independently-consumable queue while the publisher still only makes a single Publish call.
graph LR
EV[Event Source] --> T[SNS Topic]
T --> Q1[SQS: Billing Queue]
T --> Q2[SQS: Analytics Queue]
T --> Q3[SQS: Notification Queue]
Q1 --> S1[Billing Service]
Q2 --> S2[Analytics Service]
Q3 --> S3[Notification Service]
Fig 9.1 — Fan-out pattern: one publish, three independently scaling, durable consumers
Infrastructure as Code tools (CloudFormation, CDK, Terraform) treat topics, subscriptions, and their filter/redrive policies as first-class resources, which matters because a topic’s filter policies are effectively business logic and belong under version control, not click-ops configuration.
JDesign Patterns & Anti-patterns
Fan-out to SQS
Pair SNS with SQS per subscriber so every consumer gets durability, retry, and independent scaling that raw HTTP subscriptions can’t offer.
Content-based routing via filter policies
Publish a single, generically-shaped event and let each subscriber declare its own filter policy, keeping the publisher decoupled from downstream logic.
Claim-check for large payloads
Store the large object in S3, publish only a reference (bucket/key) through SNS, staying comfortably under the 256 KB limit.
Direct HTTP subscribers with no DLQ
Treats a best-effort webhook call as if it were guaranteed delivery — an extended outage on the receiver silently loses events.
KBest Practices & Common Mistakes
Best practices
- Always attach a redrive policy / DLQ to every non-trivial subscription
- Push routing logic into filter policies instead of subscriber-side if/else code
- Use resource policies for cross-account access instead of sharing credentials
- Enable delivery status logging while debugging webhook integrations
- Use FIFO topics with well-chosen message group IDs, not a single global group
Common mistakes
- Assuming a standard topic retains messages for later subscribers
- Publishing payloads near or over 256 KB instead of using a claim-check pattern
- Putting every message in a single FIFO message group, serializing all throughput
- Ignoring
NumberOfNotificationsFailedbecause publish calls “look successful” - Over-broad topic policies (`Principal: *`) exposing publish/subscribe to the internet
LReal-World & Industry Examples
Uber — trip lifecycle events
Ride-hailing platforms with similar architectures use pub/sub fan-out so a single “trip status changed” event can simultaneously update rider notifications, driver payout calculations, and fraud/anomaly detection pipelines, each evolving and scaling independently of the others.
Airbnb-style booking confirmations
A booking-confirmed event fanning out to email confirmation, host notification, and calendar-sync services is a textbook SNS fan-out use case: one publish, multiple independently-owned downstream teams, no publisher awareness of who’s listening.
Financial services alerting
Fraud detection systems commonly use SNS to fan a single suspicious-transaction event out to SMS alerting, an internal case-management queue, and an audit-logging Firehose stream simultaneously, each with different latency and durability requirements.
MFrequently Asked Questions
NSummary & Key Takeaways
Key Takeaways
- SNS is a routing contract, not a message store — durability for late or slow consumers comes from pairing it with SQS.
- Filter policies push routing decisions server-side, reducing both cost and coupling between publisher and subscribers.
- FIFO topics trade some throughput and parallelism for strict per-group ordering and exactly-once publish semantics.
- Publish latency stays flat as subscriber count grows because durability and fan-out are decoupled internally.
- Security is dual-layered: IAM identity policies and topic resource policies are evaluated together, with any explicit Deny winning.
- Every business-critical subscription should have a dead-letter queue — SNS retries generously, but not forever.
- The most robust production pattern remains fan-out to SQS: one publish, many independently durable, independently scaling consumers.