A pipeline is one assembly line for one kind of telemetry. Multiple pipelines simply means running several of these lines inside the same Collector.

OpenTelemetry is an open-source project that gives software a standard way to produce telemetry, the stream of information a program sends out about itself: what it did, how long it took, what went wrong. The OpenTelemetry Collector is a separate program that sits between your applications and your monitoring tools. It receives telemetry, changes it if needed, and forwards it onward. You can think of it as a sorting and packing centre for monitoring data.

Inside the Collector, the work is organised into pipelines. A pipeline is a fixed path with three kinds of stops. Data enters through a receiver, passes through zero or more processors that inspect or change it, and leaves through one or more exporters that send it to a destination. Every pipeline handles exactly one signal, which is a category of telemetry: traces, metrics or logs.

Multiple pipelines means that your Collector configuration defines more than one of these lines. They may handle different signals, such as one pipeline for traces and another for metrics. Or they may handle the same signal in different ways, such as one traces pipeline that keeps everything for an archive and a second traces pipeline that samples and removes private details before sending data to a vendor. The pipelines run at the same time, share the same program, and can even share parts.

What a pipeline is not

The word pipeline is used in many places in technology, so let us fence it in. A Collector pipeline is not a build or deployment pipeline for software. It is not the span processing chain inside an application’s SDK, although that has a similar flavour. It is a named path inside the Collector’s configuration file, listed under a section called service. Everything in this guide is about those Collector paths.

Why a single pipeline is not enough

A single pipeline works well for a small system with one destination. But real systems quickly collect extra needs. Different teams want different tools. Some data is sensitive. Some data is huge and must be sampled. Some destinations are slow or unreliable. When all of this is forced through one line, every rule applies to every item of data, and one problem can slow everything. Multiple pipelines give each need its own lane.

A simple mental model: three questions

When you meet any Collector configuration, three questions unlock it. First, where does the data come in? Look at the receivers. Second, what happens to it on the way? Look at the processors of each pipeline, in order. Third, where does it go out? Look at the exporters. Then repeat for each pipeline. With multiple pipelines, you simply ask the three questions several times and notice which parts repeat.

This method turns a long, intimidating file into a handful of short stories. Each pipeline is one story with a beginning, a middle and an end. The only extra thing to watch for is a connector, which is a place where one story ends and another begins. Once you see a connector, you follow the data into the next pipeline and carry on reading.

Every extra pipeline exists because one line could not meet two different needs at the same time.

Need 1: send the same data to more than one place

A company is moving from one monitoring tool to another. For three months it wants the data to reach both, so that engineers can compare the screens before they switch. Or a security team wants a full copy of every log in cheap storage, while the engineering team wants only the useful logs in an expensive search tool. Both cases require the same data to be delivered to several destinations.

Need 2: treat each destination differently

The archive wants everything exactly as it arrived. The vendor tool charges by volume, so it should get a sampled set. A partner should receive only data without customer details. If all of these shared a single line of processors, you could not give each one its own treatment, because whatever you did for one would also happen to the rest.

Need 3: derive new data from existing data

You may want request counts and durations calculated from your traces and stored as metrics. This turns one kind of data into another. It needs a path that begins as a traces pipeline and ends as a metrics pipeline, and the Collector provides this with a special part called a connector.

Need 4: keep one problem from spreading

If a destination becomes slow or goes down, data waiting for it can pile up. When many destinations share one line, a single sick destination can hold back healthy ones. Separate pipelines, each with its own buffers, help contain the damage.

flowchart LR
subgraph One["One pipeline: one treatment for everyone"]
A1["Receiver"] --> P1["Same processors"]
P1 --> E1["Archive"]
P1 --> E2["Vendor tool"]
P1 --> E3["Partner"]
end
subgraph Many["Several pipelines: each tuned"]
A2["Receiver"] --> Q1["Pipeline 1: keep all"]
A2 --> Q2["Pipeline 2: sample"]
A2 --> Q3["Pipeline 3: redact"]
Q1 --> F1["Archive"]
Q2 --> F2["Vendor tool"]
Q3 --> F3["Partner"]
end

Five kinds of building blocks make up every Collector. Knowing their jobs makes everything else easy.

Signals

A signal is a category of telemetry. Traces follow one request through many services. Metrics are numbers measured over time, like requests per second. Logs are time-stamped records of events. A pipeline handles one signal only, so a traces pipeline never carries metrics. If you want all three, you define at least three pipelines.

The component types

The configuration file

The Collector is configured with a text file, usually written in a format called YAML. The file has sections. Separate sections define receivers, processors, exporters, connectors and extensions, giving each one a name and its settings. A final section called service then assembles them: it lists which extensions to run, and it declares the pipelines, each one naming which of the defined components it uses and in what order.

This split between “defining” and “using” is the key to multiple pipelines. A component defined once can be used in several pipelines, and a component that is defined but never used in any pipeline simply does nothing. We will build on this idea in every chapter that follows.

Where the Collector lives

The Collector can run in two common shapes. As an agent, it runs close to the applications, for example one copy on every server or every Kubernetes node. As a gateway, it runs as a central service that many agents and applications send to. Both shapes use the same pipelines idea. Large systems often use both, with an agent doing light work near the source and a gateway doing heavier shared work.

flowchart LR
App["Applications with OpenTelemetry SDKs"] --> Agent["Collector as agent"]
Agent --> Gw["Collector as gateway"]
Gw --> B1["Backend A"]
Gw --> B2["Backend B"]

Components come from distributions

You do not write receivers, processors and exporters yourself in normal use. They are shipped as ready-made components inside a Collector distribution, a packaged build of the Collector. A small core distribution contains the most common components. A larger contrib distribution adds hundreds of community-maintained ones, such as receivers for many data formats, exporters for many vendors, and the more specialised connectors. If you need only a few components, you can build a custom Collector that contains exactly those, which makes the program smaller and reduces what an attacker could reach.

Each component also has a stability level, such as alpha, beta or stable, and a list of the signals it supports. Both facts appear in its documentation. Reading them before you design your pipelines saves surprises, because a component that supports traces and metrics may not support logs, and an alpha component may change its settings in the next release.

Before multiplying, we must understand one. A pipeline has a signal, a start, a middle and an end.

The three stages

Data enters a pipeline at one or more receivers. A receiver speaks a protocol. The most common is OTLP, the OpenTelemetry Protocol, which is the standard language of OpenTelemetry. Others read data in the formats of older tools, or collect data by scraping, which means visiting a target on a timer and reading its numbers. A pipeline must have at least one receiver.

Data then flows through processors, in the exact order in which they are listed. A processor might drop data you do not want, add or remove labels, limit memory use, group small items into larger batches, or make decisions about sampling. A pipeline may have no processors at all, although production setups almost always have a few.

Finally data reaches exporters. An exporter converts the data to the format of its destination and sends it, with settings for retries and queues. A pipeline must have at least one exporter. Data that reaches the exporter stage is copied to every exporter listed, so one pipeline can already deliver to several destinations.

flowchart LR
R["Receiver: OTLP"] --> M["Processor: memory limiter"]
M --> A["Processor: add attributes"]
A --> B["Processor: batch"]
B --> X["Exporter: backend"]

Why order matters

Because processors run one after another, the order changes the result. If a filter that drops health-check traces sits before a step that adds labels, the labels are never wasted on data that gets dropped. If a batching step sits before a sampling step, the sampler sees data in lumps rather than individually. A widely used guideline is to place the memory limiter first, so it can protect the Collector before anything else uses memory, and to place batching near the end, so that the data is grouped just before it is sent.

What a pipeline is made of at runtime

When the Collector starts, it reads the configuration and builds each pipeline as a chain of live objects. The configuration is the plan, and the running objects are the building. A subtle point, which we meet properly in later chapters, is that the building is not built exactly as the plan suggests: some components are created once and shared, and others are created separately for each pipeline.

Naming is the secret. A pipeline’s name has two parts, and the second part is what allows many of the same kind.

The type and the optional name

Each pipeline is identified by its signal type, optionally followed by a slash and a name of your choice. The signal type must be one of the known ones, such as traces, metrics or logs. If you need a second traces pipeline, you cannot repeat the same identifier, so you add a name: traces/archive and traces/vendor are two different pipelines that both carry traces. The part after the slash is only a label, so make it describe the purpose.

The same naming idea applies to components. You may define two OTLP exporters, one named otlp/archive and one named otlp/vendor, each with its own address and settings. The first part names the kind of component, and the second part tells your instances apart.

A planning table

Before you write any configuration, it helps to plan the pipelines as a simple table. The table below describes an imaginary setup that keeps a full copy of traces for an archive, and a sampled and cleaned copy for a vendor.

Notice what the table shows. The same otlp receiver is listed in four pipelines, so it is defined once but used four times. The otlp/vendor exporter is used by three different pipelines. Only the processors and the archive exporter differ. This is the heart of multiple pipelines: shared ends, different middles.

Rules to remember

flowchart LR
R["otlp receiver"] --> T1["traces/archive"]
R --> T2["traces/vendor"]
R --> M1["metrics"]
R --> L1["logs"]
T1 --> EA["otlp/archive exporter"]
T2 --> EV["otlp/vendor exporter"]
M1 --> EV
L1 --> EV

Reading a multi-pipeline file, section by section

A configuration with several pipelines is easier to follow if you read it from the bottom up. Start at the service section. It tells you which pipelines exist, and for each one it lists the receivers, processors and exporters by name. Make a quick note of every name you see. Then go upward to the definition sections and find each of those names, reading its settings. You now know what each part is and how it is set.

Pay particular attention to three clues. The first is the second half of a name, after the slash. Names like otlp/archive and otlp/vendor tell you that the same kind of component was defined twice with different settings, usually different destinations. The second is a name used in more than one pipeline, which reveals what is shared. The third is a connector appearing in two pipelines, once among the exporters and once among the receivers, which reveals a link between pipelines.

When you change a configuration, work in the same order. Edit the definitions first, then update the service section, then validate the file, and finally test with sample data. Most mistakes come from changing one half and forgetting the other.

When several pipelines list the same receiver, every piece of data that arrives is handed to all of them.

What fan-out means

Fan-out is the spreading of one stream into several. Imagine one hose with a splitter that feeds three garden beds. The Collector does exactly this when a receiver is listed in more than one pipeline of the same signal. The receiver is created once, listens on one port, and for every batch of data it receives, it passes the data to the start of each pipeline that uses it.

This is a powerful saving. Your applications send their data to one address only, and they do not need to know how many pipelines are behind it. You can add a new pipeline, say for an audit copy, without touching a single application. The change happens in the Collector alone.

flowchart LR
S["Applications"] --> R["One OTLP receiver"]
R --> P1["Pipeline A: archive everything"]
R --> P2["Pipeline B: sample and send to vendor"]
R --> P3["Pipeline C: filter errors for alerts"]

Do the pipelines share the same data?

A natural worry is that one pipeline may change the data and surprise the others. If pipeline B removes a label, does pipeline A lose it too? The Collector is built to prevent this. Components declare whether they mutate, which means change, the data they handle. When a stream fans out to several pipelines and at least one of them would change the data, the Collector makes copies so that each pipeline works on its own version. When no pipeline would change the data, the Collector can pass the same read-only data to all of them, saving memory and time.

The result is a simple rule you can rely on: pipelines do not affect each other’s data. A redaction in one pipeline never removes information from another. The cost is that copying uses memory and processor time, so very large fan-outs of heavy data should be planned with care.

Each pipeline gets its own processors

Another important fact is that processors are not shared between pipelines. If you list a batch processor named batch in three pipelines, the Collector creates three separate batch processors. Each one keeps its own pile of waiting data and its own timer. This matters most for processors that keep state, such as batchers, samplers and anything that counts. Two pipelines using the same processor name do not pool their data; they each work alone.

A rough cost picture, with illustrative numbers

Let us work through an imaginary example to feel the cost of fan-out. Suppose a gateway receives 5,000 spans per second, and each span is about one kilobyte when held in memory. That is roughly five megabytes of fresh data every second. If three pipelines each need to change the data, the Collector makes copies, and for a short time up to three times that amount is in flight. Add the data waiting in each pipeline’s batch processor and each exporter’s queue, and the memory in use can be many times the size of the incoming stream.

These numbers are only an example, and real values depend on the shape of your data and your settings. But the lesson is general. Memory use grows with the number of pipelines that change data, and with the size of every buffer. Before adding a pipeline to a busy gateway, estimate its extra memory, set the memory limiter below the real limit of the machine, and watch the effect after the change.

Fan-in is the mirror image. Several streams are merged into one outgoing connection.

Two kinds of fan-in

The first kind happens at the start of a pipeline. A pipeline may list several receivers, and data from all of them enters the same line. For example, one traces pipeline might accept data from an OTLP receiver and also from a receiver that understands an older tracing format. After that point, the data is treated identically no matter which door it used.

The second kind happens at the end. When several pipelines list the same exporter, the data from all of them flows into that one exporter. The exporter is created once, holds one connection and one queue, and sends everything it receives to its destination. This is efficient, because you do not open many connections to the same backend. It also means that the pipelines are mixed together at that point.

flowchart LR
A["Pipeline: traces from services"] --> X["One exporter to the vendor"]
B["Pipeline: traces from batch jobs"] --> X
C["Pipeline: traces derived from logs"] --> X
X --> V["Vendor backend"]

Why use separate pipelines before a shared exporter?

You might ask why anyone would build three pipelines that end in the same place. The reason is that the middles can differ. Traces from services may need careful sampling. Traces from batch jobs may need a different attribute added. Traces derived from logs may need filtering. Each pipeline applies its own rules, and then all the results leave through the same door.

The shared queue and its consequences

Many exporters have a sending queue, a waiting area that holds data while it is being sent, and retry settings that decide how to handle failures. Because the exporter is shared, all pipelines that use it share the queue and the retry behaviour. If the destination becomes slow, the queue fills with data from all of those pipelines, and when it is full, new data may be refused. A noisy pipeline can therefore crowd out a quiet but important one.

If this matters to you, give important data its own exporter. Define two exporters of the same kind with different names, such as otlp/critical and otlp/bulk, each pointing at the same destination, and give each its own queue size. Now the bulk data can fill its own queue without blocking the critical data.

This is the most common design question. Both choices send data to two places, but they behave very differently.

The simple choice: two exporters in one pipeline

The easiest way to deliver the same data to two destinations is to list two exporters in one pipeline. Every item that reaches the end of the line is copied to both exporters. There is one set of processors, so both destinations receive data that was treated exactly the same way. This is perfect when both destinations should see the same data, such as when you send the same sampled traces to a primary and a backup tool.

The flexible choice: two pipelines

When the destinations need different treatment, you use two pipelines. Each has its own processors, and so each can sample, filter or redact on its own terms. The price is that you write and maintain more configuration, and some work, like batching, happens twice.

flowchart TD
subgraph OneP["Design A: one pipeline, two exporters"]
RA["Receiver"] --> PA["Shared processors"]
PA --> EA1["Exporter 1"]
PA --> EA2["Exporter 2"]
end
subgraph TwoP["Design B: two pipelines"]
RB["Receiver"] --> PB1["Processors for destination 1"]
RB --> PB2["Processors for destination 2"]
PB1 --> EB1["Exporter 1"]
PB2 --> EB2["Exporter 2"]
end

A short decision guide

Traces, metrics and logs each need their own pipeline, and each signal has its own favourite tools.

One signal, one pipeline

A pipeline carries a single signal, so a full setup has at least one pipeline per signal. The three can share a receiver, since OTLP carries all three on the same port. They can also share an exporter, since an OTLP exporter can send all three to a backend that accepts them. What differs is the processors, because some processors only make sense for one kind of data.

Why not one pipeline for everything?

The Collector does not allow it, and for good reason. A trace and a metric have completely different shapes. A processor that samples traces by trace identity has nothing to say about a metric. Keeping signals in separate pipelines keeps every component’s job clear and lets the Collector check at startup that you did not connect a traces-only component to a logs pipeline.

Several pipelines of the same signal

Within one signal, you may want more than one pipeline for the reasons we have discussed. A typical logs setup might have a pipeline for application logs, a pipeline for audit logs that must never be dropped, and a pipeline for debug logs that are heavily filtered. Each has its own filters and its own exporter, even though all three carry the same signal.

Designing each signal on purpose

A good habit is to design the three signals one at a time, asking the same questions of each. For traces, ask how much you can afford to keep, whether any spans are noise, and whether a full copy must be archived. For metrics, ask which labels create too many different series, and whether you need derived metrics from traces or logs. For logs, ask where private data may hide, which severity levels are worth paying for, and whether any logs, such as audit records, must never be dropped.

Writing down these answers before you write any configuration turns a vague wish into a clear plan. It also shows you quickly where two signals need the same treatment, such as the same memory limiter and batching, and where they need different treatment, such as sampling for traces and parsing for logs.

So far, pipelines have lived side by side without talking to each other. Connectors let one pipeline hand data to another.

What a connector is

A connector is a component with two faces. In one pipeline it acts as an exporter, and in another pipeline it acts as a receiver. Data that leaves the first pipeline through the connector enters the second pipeline through it. In between, the connector may simply pass the data along, or it may change it, or it may even create new data of a different signal.

The signals matter. A connector declares which signals it accepts on its exporter side and which it produces on its receiver side. A connector that turns traces into metrics is listed as an exporter in a traces pipeline and as a receiver in a metrics pipeline. This is the only way that data can cross from one signal to another inside the Collector.

flowchart LR
R["OTLP receiver"] --> T["Traces pipeline"]
T --> E["Vendor exporter for traces"]
T --> C["Connector: spanmetrics"]
C --> M["Metrics pipeline"]
M --> E2["Vendor exporter for metrics"]

Common connectors

The list of available connectors grows over time, and their stability levels differ. Some are mature, and others are still marked alpha or beta. Check the documentation for the version of the Collector you use before you depend on one.

The forward connector as glue

The forward connector deserves special mention, because it makes complex designs tidy. Suppose three different pipelines all need the same shared cleaning step before they split into different treatments. You can send all three into the forward connector, which feeds a single pipeline that does the cleaning, and then fan out again. This avoids repeating the same processors in each pipeline.

Rules for connectors

The failover connector in practice

The failover connector solves a different problem from routing. Instead of choosing a pipeline by the content of the data, it chooses by the health of the destination. You give it a ranked list of pipelines. All the data goes to the first pipeline while it works. If the first pipeline starts to fail, because its exporter cannot reach the destination, the connector switches the data to the second pipeline, and later it can switch back when the first recovers.

A typical use is a primary and a backup destination. The primary pipeline sends to the main vendor. The backup pipeline sends to a second location, such as a cheaper store or another region. During an outage at the primary, no data is lost, and the team can read it from the backup later. Failover is different from sending to both at once. It uses only one path at a time, so it costs less, but it relies on detecting failure quickly.

flowchart LR
In["Traces pipeline"] --> FC{"Failover connector"}
FC -- "priority 1" --> P1["Primary pipeline"]
FC -- "priority 2" --> P2["Backup pipeline"]
P1 --> E1["Main vendor"]
P2 --> E2["Backup store"]

Sometimes the question is not “send it everywhere” but “send it to the right place”. Routing answers that.

The routing connector

The routing connector looks at each item of data and decides which pipeline should receive it. You give it a list of rules. Each rule has a condition, written in a small language for working with telemetry, and a target pipeline. Data that matches a condition goes to that pipeline. Data that matches no rule goes to a default pipeline, so nothing is silently lost.

An older component called the routing processor did a similar job, but it has been replaced by the connector, and new designs should use the connector. Because a connector sits between pipelines, it can feed full pipelines with their own processors and exporters, which is more flexible than a processor could be.

flowchart LR
In["Incoming traces pipeline"] --> RC{"Routing connector"}
RC -- "tenant is team-a" --> A["Pipeline for team A"]
RC -- "tenant is team-b" --> B["Pipeline for team B"]
RC -- "no match" --> D["Default pipeline"]
A --> EA["Exporter for team A backend"]
B --> EB["Exporter for team B backend"]
D --> ED["Exporter for shared backend"]

Typical routing needs

Routing and tail sampling: a trap

Tail sampling, described in the signals chapter, needs to see all the spans of a trace in one place before it can decide. If spans from the same trace are scattered over several Collector instances, no instance sees the whole trace, and the decisions are wrong. Routing inside one Collector is fine, but when you run many gateway copies behind a load balancer, you must make sure that all spans of one trace reach the same copy. A special exporter, called the load balancing exporter, does this by sending data to a chosen copy based on the trace identity. A two-layer design, with an outer layer that routes by trace and an inner layer that does tail sampling, is the standard solution.

The configuration file looks like lists, but the Collector turns it into a graph of connected parts. Understanding that graph explains almost every behaviour you will see.

From text to a graph

When the Collector starts, it reads the configuration and checks it. It confirms that every component named in a pipeline is defined, that every component supports the signal of the pipeline it is in, and that connectors appear on both sides. It then builds a graph: a map of nodes, which are components, and arrows, which show where data flows. Receivers and connectors that are used in several pipelines become single nodes with several outgoing arrows. Exporters used in several pipelines become single nodes with several incoming arrows. Processors, as we learned, are created separately for each pipeline.

Because it is a true graph, the Collector can also check it for problems. A connector loop, where data could go round and round, is detected and rejected. A component defined but never used in any pipeline is simply left out and never started, which is why unused definitions are harmless but also easy to forget.

Start order and stop order

The order in which the parts start matters. The Collector starts the parts at the end of the line first: exporters, then processors, then receivers. This means that by the time a receiver begins accepting data, everything downstream is ready to handle it. When the Collector shuts down, it reverses the order. It stops the receivers first, so no new data arrives, then lets the processors finish what they hold, and finally stops the exporters after they have sent their remaining data. This careful order helps prevent data loss during restarts, although a sudden crash can still lose data that was waiting in memory.

Data moves by direct hand-off

Inside the Collector, each component passes data to the next by calling it directly. A receiver takes in a batch of data and calls the first processor. That processor does its work and calls the next, and so on until the exporter is called. The consequence is that, unless a queue sits in between, the receiver’s call does not finish until the data has been handled further down. A slow component therefore slows the receiver, and an error deep in the chain can be reported back to the sender. This direct hand-off is what gives the Collector its back-pressure behaviour, which we meet in the scaling chapter.

flowchart TD
Cfg["Configuration file"] --> Check["Validate components, signals and connectors"]
Check --> Graph["Build the pipeline graph"]
Graph --> S1["Start exporters"]
S1 --> S2["Start processors"]
S2 --> S3["Start receivers"]
S3 --> Run["Run: data flows through the graph"]
Run --> Stop["Stop: receivers first, exporters last"]

The life of one batch of data

Reading the startup output

The first seconds of the Collector’s own output tell you whether your graph was built as you intended. Look for lines that name each pipeline and its components as they start. If a pipeline you wrote does not appear, it was probably not listed in the service section. If a component fails, the message normally says which one and why, for example that it does not support the signal of the pipeline you placed it in. If there are no errors, the Collector will say that it is ready to process data.

It is also worth checking what the Collector says about the components it was built with. A Collector distribution contains only certain components. If the configuration mentions a component that is not in your build, the Collector will refuse to start and tell you the name is unknown. The fix is to use a distribution that includes it, or to build a custom Collector with exactly the components you need.

More pipelines mean more work. This chapter shows how to keep a multi-pipeline Collector fast, steady and safe from data loss.

Protect memory first

Every copy of data in a fan-out takes memory, and every batcher and queue holds data waiting for its turn. A Collector that runs out of memory is restarted by its host, and data in memory is lost. The memory limiter processor guards against this. It watches the memory use of the Collector, and when use passes a soft limit it starts refusing new data, and when it passes a hard limit it forces cleaning. Because refusing data pushes the problem back to the sender, the senders must be prepared to retry. Put the memory limiter first in every pipeline.

Batching and queues

Sending data one item at a time would be wasteful. The batch processor groups items, so the exporter sends fewer, larger requests. Remember that each pipeline has its own batch processor, so ten pipelines mean ten batchers, each holding a little data. Exporters add a sending queue that absorbs short slowdowns of the destination and retry rules that handle brief failures. For important data, a persistent queue, backed by a storage extension that writes to disk, lets data survive a restart of the Collector.

Back-pressure and isolation

Back-pressure is what happens when downstream parts cannot keep up and the slowness travels back toward the source. With separate pipelines and separate exporters, each with its own queue, the slowness of one destination fills only its own queue. With a shared exporter, the queue is shared, and everyone feels it. Isolation is therefore a design choice: give important and bulky data their own exporters, and decide what should happen when each queue is full.

Scaling out with agents and a gateway

A single Collector has limits. When load grows, you run several copies behind a load balancer. Most pipelines scale easily in this way, because each batch of data is independent. Two kinds of work are different. Tail sampling needs all spans of a trace in one place, and metrics derived from traces, such as those produced by the spanmetrics connector, are computed per copy, so each copy produces only a partial count. For both, route by trace identity or add a label that identifies the producing copy, so that the partial results can be combined later.

flowchart LR
Apps["Applications"] --> Ag["Agent Collectors: light pipelines"]
Ag --> LB["Load balancing exporter"]
LB --> G1["Gateway copy 1: tail sampling and routing"]
LB --> G2["Gateway copy 2: tail sampling and routing"]
G1 --> Back["Backends"]
G2 --> Back

Reliability patterns

Multiple pipelines let you treat data differently for safety, but they also multiply what you must watch and pay for.

Security: one pipeline per trust level

Telemetry often holds private details, such as email addresses in logs, account numbers in span labels or secrets in URLs. Not every destination should receive them. A good design uses a separate pipeline for each trust level. The internal archive, which sits in your own protected storage, may keep the full data. The pipeline for an outside vendor first removes or masks sensitive fields, using processors that delete labels, replace patterns or apply a redaction list. A partner pipeline might allow only a short, approved list of fields.

Monitoring the Collector itself

A Collector that fails quietly is dangerous, so watch it. The Collector reports its own metrics about what each component does. They include counts of items accepted by receivers, items sent by exporters, items refused or dropped, items that failed to send, and the current size of exporter queues. Exact names change between versions, but the idea is stable: compare what came in to what went out, per pipeline. If the numbers do not match and you did not intend to drop data, something is wrong.

The Collector can also offer a health check extension that load balancers use to see whether a copy is alive, and a diagnostic page extension that shows live details of its pipelines. For new setups, a debug exporter prints a summary of data to the Collector’s own output, which is invaluable for confirming that a pipeline receives what you expect.

Cost: copies cost money

Sending the same data to two vendors can double your data transfer charges, and each vendor may bill you for the volume it ingests. Archive storage adds its own fee. Extra pipelines also use more memory and processor time in the Collector. The best defence is to be deliberate: sample before sending to expensive tools, drop data nobody uses, remove labels that create endless variety, and compare the cost of each pipeline against the value of the destination it feeds.

A pre-release checklist

Before you release a new multi-pipeline configuration to production, walk through a short checklist. Each item takes a minute and prevents a common kind of incident.

• Validation: the file passes the Collector’s own validation without errors.
• Coverage: every intended pipeline is listed in the service section, and every defined component is used by at least one pipeline.
• Privacy: every pipeline that sends data outside your control includes the redaction steps, and a sample run has been inspected by a person.
• Limits: the memory limiter is first, queue sizes are bounded and retry rules are set on purpose.
• Routing: each routing connector has a default route.
• Monitoring: a dashboard and an alert exist for items received versus items sent in each pipeline.
• Rollback: the previous working configuration is saved and can be restored quickly.

Several pipelines give you control. Control has a price, and an honest look at both sides prevents over-building.

Trade-offs you will face

Teams that run many pipelines well share a few habits, and teams that struggle repeat a few mistakes.

Patterns that work

Anti-patterns to avoid

Best practices at a glance

When not to add a pipeline

Knowing when to stop is as valuable as knowing how to build. Do not add a pipeline when a single processor could do the job inside an existing one. If two destinations need identical data, two exporters in one pipeline are enough. Do not add a pipeline only because a different team asked for “their own”, if they will receive exactly the same treatment. And do not add a pipeline to hide a design problem, such as a destination that cannot keep up. Fix the destination, or the sampling, rather than building more lines around it.

A useful test is to try to explain the new pipeline in one sentence that names a difference. “This pipeline sends the vendor a copy with customer emails removed” passes the test. “This pipeline is another way to send traces” does not. If you cannot name the difference, you probably do not need the pipeline.

Common mistakes

Let us design a multi-pipeline Collector for an imaginary company, then see where the same patterns appear in practice.

ShopFast: one Collector, four goals

ShopFast is an imaginary online store. Its services send traces, metrics and logs to a gateway Collector. The company has four goals. The compliance team wants a full copy of all logs and traces in cheap storage for a year. The engineering team wants traces in a vendor tool, but sampled, because full volume is too costly. The vendor must never receive customer emails or card fragments. And the on-call team wants request-rate and error-rate metrics even for traces that were sampled away.

flowchart LR
In["OTLP receiver"] --> TA["traces/archive"]
In --> TV["traces/vendor"]
In --> TM["traces/metrics"]
In --> LA["logs/archive"]
In --> LV["logs/vendor"]
TA --> Arch["Archive exporter"]
LA --> Arch
TV --> Vend["Vendor exporter"]
LV --> Vend
TM --> SM["spanmetrics connector"]
SM --> MP["metrics pipeline"]
MP --> Vend

The result is that each goal is met by a pipeline built for it. No application changed. When ShopFast later adds a second vendor for a trial, it adds one exporter and one more pipeline, and removes them when the trial ends.

Where the same patterns appear in industry

A second illustrative story: BookNest and regional rules

BookNest is another imaginary company, which sells in several regions. Its legal team requires that customer-related logs from one region never leave that region. The platform team runs one gateway Collector per region, but a central team also wants a small set of safe, anonymised metrics from every region in one global tool.

The design satisfies the legal rule by construction. A reviewer does not have to inspect individual data items, because the structure of the pipelines shows where each kind of data can go. That ability to reason about data flow from the shape of the configuration is one of the quiet benefits of multiple pipelines.

Short answers to the questions people ask most often.

If you remember only a handful of ideas from this guide, make it these.