Before diving into details, we need a clear picture of the problem messaging solves and what Pub/Sub delivers.

The problem of tightly connected systems

When one program needs to tell another program something, the simplest approach is a direct call: program A contacts program B and waits for an answer. This works for small systems, but it becomes fragile as systems grow. If program B is slow, program A is slow. If program B is down, program A fails. If ten other programs also want to know the same news, program A must contact all ten, and adding an eleventh means changing program A. Everything becomes tangled together, and a problem in one place spreads to many.

Messaging breaks these chains. Instead of calling each other directly, programs hand messages to an intermediary. The sender does not know or care who reads the message, and readers do not need the sender to be available at the moment they read. This idea is called decoupling, and it is one of the most important ideas in building large, reliable systems.

What the service gives you

Pub/Sub is a fully managed, global messaging and event streaming service on Google Cloud. Publishers send messages to a named channel called a topic. Subscribers receive those messages through subscriptions attached to the topic. The service stores messages durably, delivers them at least once by default, scales automatically to very high volumes, and works across regions without you running any servers or brokers. It is used for event-driven architectures, streaming data pipelines, asynchronous work queues, log and telemetry collection, and integration between systems.

Every messaging system has its own words. These are the ones you will hear in nearly every Pub/Sub conversation.

How the words fit together

A publisher sends a message to a topic. The topic delivers a copy of that message to every subscription attached to it. Each subscription keeps its own record of which messages have been delivered and acknowledged. Subscribers read from a subscription, process each message, and acknowledge it. If several subscribers share one subscription, they split the messages among them, which spreads the work. If several different subscriptions are attached to the same topic, each gets every message, which spreads the news.

Now let us follow a message from the moment it is published to the moment it is acknowledged.

The shape of the system

Pub/Sub is a global service with a shared control plane and regional data handling. Publishers connect to a nearby endpoint, and the service stores each message durably in more than one zone before confirming to the publisher that the message was accepted. That confirmation is important: once a publisher receives it, the message is safe even if a data center fails a moment later. You do not manage brokers, partitions, or capacity. The service scales on its own as publish and subscribe traffic grows.

flowchart LR
    P1["Publisher A"] --> T["Topic"]
    P2["Publisher B"] --> T
    T --> S1["Subscription 1"]
    T --> S2["Subscription 2"]
    T --> S3["Subscription 3"]
    S1 --> R1["Order service workers"]
    S2 --> R2["Analytics pipeline"]
    S3 --> R3["Audit archive"]
        

The life of a message

sequenceDiagram
    participant P as Publisher
    participant T as Topic
    participant S as Subscription
    participant W as Subscriber
    P->>T: Publish message
    T-->>P: Confirm stored
    T->>S: Make available
    S->>W: Deliver message
    W->>W: Process
    W-->>S: Acknowledge
    S->>S: Mark complete
        

The type of subscription decides how messages reach your code and who is in control of the pace.

Pull subscriptions

With a pull subscription, your subscriber asks the service for messages when it is ready. The subscriber controls the pace: it can take a few messages, process them, and ask for more. This makes pull ideal for work that varies in speed, for batch processing, and for any case where you want to apply backpressure, meaning slowing down when the consumer is overwhelmed. The popular way to pull is a streaming connection that keeps messages flowing with low delay, while a simpler request-based pull is available when you want direct control.

Push subscriptions

With a push subscription, the service sends each message to an HTTPS address you provide, as a web request. Your endpoint processes the message and replies with a success code to acknowledge it. Push is convenient because your code needs no special client library and can live behind a normal web address, which suits serverless services and webhooks. The trade-off is that the service sets the pace, so your endpoint must handle the incoming rate, and it responds to failures with retries and backoff.

Direct-to-storage subscriptions

Some subscriptions deliver messages straight into a destination without any code of your own. A BigQuery subscription writes each message as a row in a table in the data warehouse. A Cloud Storage subscription writes messages into files in a bucket. A Bigtable subscription writes into a wide-column table. These remove the need to build and run a pipeline just to land data, and they are excellent for ingestion and archiving.

Choosing between them

flowchart TD
    T["Topic"] --> A["Pull subscription"]
    T --> B["Push subscription"]
    T --> C["BigQuery subscription"]
    T --> D["Cloud Storage subscription"]
    A --> A1["Worker asks for messages"]
    B --> B1["Service posts to HTTPS endpoint"]
    C --> C1["Rows in warehouse table"]
    D --> D1["Files in bucket"]
        

The most important behavior to understand is what the service promises about delivery, and what it deliberately does not.

At-least-once delivery

By default, Pub/Sub guarantees that every message is delivered at least once. That means a message will not be silently lost, but it may be delivered more than once. Duplicates can happen for several reasons: an acknowledgment was lost on the network, a subscriber took too long and the deadline expired, or the service retried to be safe. This is a deliberate trade-off. It is much easier to make a system that never loses messages and sometimes repeats them than one that never repeats and sometimes loses.

The acknowledgment deadline

When a message is delivered, the service starts a timer called the acknowledgment deadline. If the subscriber acknowledges before the timer ends, the message is done. If not, the service assumes the delivery failed and sends the message again, possibly to a different subscriber. For slow work, client libraries automatically extend the deadline while the message is being processed, which is called lease management. For very long tasks, you may need to tune this or move the heavy work out of the message handler.

Retry policy

When a message is redelivered, the subscription can use a retry policy that waits between attempts. With exponential backoff, the delay grows with each failed attempt, between a minimum and a maximum you set. This prevents a struggling downstream service from being hammered with immediate retries. Without a retry policy, the service redelivers as soon as it can, which can make an overload worse.

Idempotency: your safety net

Since duplicates are possible, the subscriber should produce the same final result whether it processes a message once or several times. This property is called idempotency. A handler that adds a row, charges a card, or sends an email is not naturally idempotent, and must be made so. The usual approach is to use a unique identifier carried by the message, or a business key, and record in a durable store that it was already handled. If the same identifier appears again, the handler skips the work and acknowledges.

flowchart TD
    M["Message arrives"] --> C{"Already processed this id?"}
    C -->|"Yes"| A1["Acknowledge and skip"]
    C -->|"No"| W["Do the work"]
    W --> OK{"Succeeded?"}
    OK -->|"Yes"| R["Record id and acknowledge"]
    OK -->|"Temporary failure"| N["Negative acknowledge for retry"]
    OK -->|"Permanent failure"| D["Let it reach dead letter handling"]
        

Some messages can never succeed. A good design decides in advance where such messages go.

The poison message problem

A poison message is one that always fails when processed, for example because its data is malformed or because it triggers a bug. Without protection, the service keeps redelivering it, the subscriber keeps failing, and this wastes capacity and can block other messages, especially when ordering is used. Worse, the failure can hide among normal noise, so nobody notices real problems.

Dead letter topics

A dead letter topic is a separate topic where the subscription forwards messages that have failed too many times. You set the maximum number of delivery attempts. After that number is reached, the message is moved to the dead letter topic and removed from the main flow. A person or a repair process can then inspect it, fix the cause, and resubmit it if appropriate. The dead letter topic needs its own subscription so the forwarded messages are actually retained, and the service needs permission to publish to it.

flowchart LR
    TP["Main topic"] --> SUB["Main subscription"]
    SUB --> WK["Subscriber"]
    WK -->|"Failure"| SUB
    SUB -->|"Attempts exceed limit"| DLT["Dead letter topic"]
    DLT --> DLS["Dead letter subscription"]
    DLS --> REV["Review and repair process"]
    REV -->|"Fixed message"| TP
        

Designing the failure path

Two optional features give stronger guarantees. Each comes with trade-offs you should understand before enabling it.

Message ordering

By default, messages may arrive in a different order from the one in which they were published, because the service spreads work across many machines for scale. When order matters, such as a sequence of updates to the same customer record, you can attach an ordering key to each message. Messages that share an ordering key are delivered in the order they were published, while messages with different keys are processed independently and in parallel. Ordering must also be enabled on the subscription, and messages must be acknowledged in order.

flowchart LR
    P["Publisher"] -->|"key A: update 1"| T["Topic with ordering"]
    P -->|"key A: update 2"| T
    P -->|"key B: update 1"| T
    T --> S["Subscription with ordering enabled"]
    S --> W1["Worker handling key A in order"]
    S --> W2["Worker handling key B in order"]
        

Costs of ordering

Exactly-once delivery

For workloads where duplicates cause real harm, a subscription can enable exactly-once delivery. When enabled, the service gives stronger guarantees: once a message has been successfully acknowledged, it will not be redelivered, and acknowledgment results are reported accurately to the client so the subscriber knows whether the acknowledgment truly succeeded. It relies on cooperation with the client libraries and has constraints, such as applying to pull style subscriptions and to messages handled within a region, and it can slightly raise latency. It also does not remove the need to think about what your own handler does with side effects outside the service.

As more teams share topics, agreeing on the shape of messages and routing the right ones to the right readers becomes essential.

Schemas

A schema describes the structure a message must follow, such as which fields exist and what types they have. Attaching a schema to a topic makes the service reject messages that do not match, which stops malformed data at the door instead of letting it poison every subscriber. Schemas also act as a contract between publishers and subscribers, so teams can evolve independently while still understanding each other. Plan evolution from the beginning: prefer optional fields, avoid breaking changes, and version schemas deliberately.

Message filtering

A subscription can carry a filter that looks at message attributes and delivers only matching messages. Messages that do not match are simply not delivered to that subscription and are treated as acknowledged for it. This lets one topic serve many different consumers, each interested in a slice, without creating a topic for every combination. Because the filter works on attributes, publishers should set meaningful attributes such as event type or region, so consumers can select precisely what they need.

Single message transforms

The service also supports lightweight transformations applied to individual messages as they are published or delivered, such as reshaping or masking fields, so simple adjustments do not require a separate processing service. These are suited to small, stateless changes. Larger transformations, enrichment from other data sources, or aggregation across messages are better handled by a processing pipeline.

Messages do not have to vanish the instant they are processed. You can keep them, replay them, and even bring in streams from elsewhere.

Message retention

An unacknowledged message is retained by a subscription for a limited period, so a subscriber that is down for a while can catch up when it returns. You can configure how long messages are kept, within allowed limits. A topic can also retain messages itself, independent of subscriptions, which lets new subscriptions be created later and still read recent history. Longer retention improves resilience and flexibility but adds storage cost.

Replay

Subscriptions can be configured to keep acknowledged messages for a time as well, and you can then rewind the subscription to an earlier point in time or to a saved marker called a snapshot, so that messages are delivered again. This is invaluable when a bug in your subscriber corrupted data, or when you need to rebuild a downstream system from scratch. Replay turns a message stream from a one-shot delivery into something closer to a recoverable log.

Importing streams from other platforms

The service offers import topics that pull data from certain external streaming platforms into Pub/Sub, including streams from other clouds and from popular streaming systems. This helps teams consolidate event data in one place or migrate gradually without rewriting every producer at once. The availability of specific sources and their setup requirements evolve, so confirm the current list.

Pub/Sub scales on its own, but your publishers and subscribers must be designed to keep up and to protect themselves.

The service scales, your code must too

The service absorbs very high publish rates and spreads delivery across many machines without any capacity planning from you. The limits you feel are usually in your own code: how fast publishers can batch and send, how many messages your subscribers can process at once, and how quickly the systems behind your subscribers can respond. Understanding where the true bottleneck is saves a lot of wasted effort.

Publishing efficiently

Publisher client libraries batch many small messages into fewer requests, which greatly improves throughput and reduces cost. You can tune batch size and wait time to trade a little latency for much higher efficiency. Publishing each message in a separate blocking call is one of the slowest possible patterns. Keep messages reasonably small and place large payloads in object storage, sending only a reference in the message.

Subscribing efficiently

flowchart TD
    B["Backlog grows on subscription"] --> C{"Is processing slow or the downstream limited?"}
    C -->|"Subscribers too few"| S["Add subscriber instances"]
    C -->|"Handler too slow"| H["Optimize handler or move heavy work out"]
    C -->|"Downstream limited"| D["Reduce concurrency and protect downstream"]
    S --> R["Backlog drains"]
    H --> R
    D --> R
        

Push delivery and rate

For push subscriptions, the service decides how fast to send requests, adapting to how quickly your endpoint responds. A slow or failing endpoint causes the service to slow down and back off, but a fast endpoint may receive a heavy stream. Make sure your endpoint can scale, and set limits on concurrency where available, so it is not overwhelmed during bursts.

Messages often carry business data. Securing who can publish, who can subscribe, and how data is protected is essential.

Access control

Access is governed by Google Cloud identity and access management. You grant roles on a topic or subscription, separating the ability to publish from the ability to subscribe and from administrative rights. Give each application its own service account with only the roles it needs. A service that only publishes order events should not be able to read them, and a service that only consumes should not be able to publish.

Securing push endpoints

A push endpoint is a web address that accepts messages, so it must verify that requests really come from Pub/Sub. With authentication enabled, each request carries a signed token tied to a service account and an intended audience. Your endpoint validates the token before trusting the message. Without verification, anyone who discovers the address could send forged messages. Also keep the endpoint private to the extent possible, using ingress restrictions on the receiving service.

Protecting sensitive data in messages

Treat message contents as data that may be stored, retained, replayed, and copied to several subscriptions. Avoid placing secrets such as passwords or private keys in messages. For personal or regulated data, minimize what is included, consider encrypting sensitive fields at the application level, and remember that retention and replay mean data lives longer than a single delivery. Choose topic locations carefully, because they affect where data is stored, which matters for residency rules.

An asynchronous system hides problems unless you watch the right signals. A few metrics tell you almost everything.

The signals that matter

Diagnosing a growing backlog

Alerts worth setting

Understanding what drives the bill and where the service has limits keeps designs honest.

What you pay for

Pricing is based mainly on the volume of data published and delivered, with a monthly free allowance for small workloads. Additional charges can apply for features such as retaining messages beyond defaults, storing messages on topics, snapshots, replay, and data transfer across regions. Direct-to-storage subscriptions may have their own charges in addition to the costs of the destination service. Because every subscription receives its own copy of each message, delivery volume grows with the number of subscriptions, so fan-out increases cost. Exact rates change, so use the official pricing page and calculator.

Advantages and trade-offs

Where Pub/Sub sits among options

Good patterns make event-driven systems feel calm. Anti-patterns make them feel haunted.

Patterns worth adopting

Anti-patterns to avoid

A checklist of habits that experienced teams build into every Pub/Sub deployment, and the traps that catch newcomers.

Best practices

Common mistakes

A short pre-launch review

These illustrative scenarios, drawn from common industry patterns, show how the ideas combine in practice.

What these scenarios share

Each team published events once and let many consumers subscribe independently. Each kept messages small and meaningful, made handlers safe to repeat, attached dead letter handling, secured identities narrowly, and watched backlog age. None treated the topic as a synchronous call or assumed messages arrive exactly once without proof.

Short, direct answers to questions that come up again and again.

A topic is the channel publishers send messages to. A subscription is an attachment to a topic that represents one stream of messages for a group of readers. Subscribers read from subscriptions.

Use pull when you want control over pace, high throughput, or ordered processing. Use push for simple serverless handlers or webhooks at low to moderate volume. Direct-to-storage subscriptions need no code at all.

Yes. The default guarantee is at-least-once delivery. Design handlers to be idempotent, and consider exactly-once delivery for workloads where duplicates are especially harmful.

It is the time the service waits for a subscriber to acknowledge a message before sending it again. Client libraries can extend it while work continues.

Configure a dead letter topic with a maximum number of delivery attempts, give it its own subscription so failures are retained, set alerts on its size, and decide who repairs and replays those messages.

Publish with an ordering key, enable ordering on the subscription, and acknowledge in order. Use many distinct keys so different entities still process in parallel.

Yes. Create one subscription per service on the same topic, and each receives its own copy. Multiple subscribers on one subscription instead share the messages.

Yes, with a BigQuery subscription that writes messages as rows, and similar subscriptions exist for Cloud Storage and Bigtable. These remove the need to write pipeline code for simple ingestion.

Unacknowledged messages are retained for a configurable period, and you can also retain acknowledged messages or store messages on the topic. Check current limits and costs in the official documentation.

Yes, if you retain acknowledged messages. You can seek a subscription to a time or a snapshot so messages are delivered again, but make sure your handlers can safely process repeats.

Enable authentication so each request carries a signed token tied to a service account, verify the token and audience in your endpoint, and restrict who can reach the endpoint.

The age of the oldest unacknowledged message and the backlog size on each subscription, plus the size of any dead letter topic and the rate of expired acknowledgments.

Usually not. It is designed for asynchronous messaging and events. For immediate answers, use direct calls or a clearly designed asynchronous reply flow.

Pub/Sub is Google Cloud’s fully managed, global messaging service for decoupling the parts of a system. Publishers send messages to topics, subscriptions attached to the topic each receive their own copy, and subscribers process and acknowledge messages. Subscriptions can deliver by pull for controlled pace, by push to an HTTPS endpoint, or directly into BigQuery, Cloud Storage, or Bigtable with no code. The default guarantee is at-least-once delivery, so duplicates are possible, and idempotent handlers are the foundation of correctness. Acknowledgment deadlines, retry policies with backoff, and dead letter topics handle slow and failing consumers, while ordering keys and exactly-once delivery add stronger guarantees at a cost. Schemas, attributes, filters, and transforms keep shared topics orderly. Retention and replay turn the stream into something recoverable, and import topics bring in data from other platforms. Scaling depends on batching publishers, flow-controlled and scalable subscribers, and protected downstream systems. Security rests on narrow identities per application, authenticated push, encryption, and careful handling of sensitive data that may be copied widely. Monitoring the age of the oldest message, the backlog, expired acknowledgments, and dead letter size reveals trouble early. Teams that publish small events, tolerate duplicates, plan the failure path, and watch their queues build systems that stay calm under pressure.