Before meeting the tool, it helps to feel the problem it solves.

Telemetry is the general name for the information a running system sends out about itself so that people can understand how it behaves. OpenTelemetry organises telemetry into signals. Traces follow a single request as it travels through many services. Metrics are numbers measured over time, such as requests per second. Logs are timestamped records of events. Each of these is useful, but each can also be produced in enormous quantities.

More data is not always better data. Every item has to be created, sent over the network, processed, stored and queried, and most monitoring services charge by volume. A large share of the telemetry in a typical system is noise: it repeats, it describes routine and harmless events, or it simply belongs to something nobody is investigating. That noise costs money, slows down searches and makes the useful signals harder to find.

Where the noise usually comes from

It is a Collector component that removes telemetry which matches rules you write.

The OpenTelemetry Collector is a standalone program that receives telemetry, optionally changes it, and sends it onward to one or more destinations. Inside it, data flows through pipelines. A pipeline is made of receivers that bring data in, processors that work on it in the middle, and exporters that send it out. The Filter Processor is one of the processors. Its job is simple: look at each item passing through, check it against your rules, and drop the items that match.

It can examine traces (down to individual spans and span events), metrics (down to individual metrics and data points) and logs (individual log records). It can also look at the resource and scope information attached to them, which describe the service and the library that produced the data. Because it can look so closely, it can remove very precise slices of telemetry while leaving everything else alone.

flowchart LR
  A["Receiver: data comes in"] --> B["Filter processor: matching items are dropped"]
  B --> C["Other processors"]
  C --> D["Exporter: data goes out"]
  B -.-> X["Dropped items"]
        

What it does and what it does not do

To use a processor, you must understand how it is defined and switched on.

The Collector is controlled by a configuration file written in YAML, a plain-text format that is easy for people to read. The file has top-level sections for receivers, processors, exporters, connectors and extensions, and a final section called service. The first group is where components are defined. The service section is where components are enabled and joined into pipelines.

This matters for the Filter Processor in two ways. First, you describe its rules in the processors section. Second, nothing happens until you list the processor inside the processors list of one or more pipelines in the service section. A filter that is defined but never listed in a pipeline does nothing at all.

One definition, several pipelines

You can use the same filter definition in several pipelines. However, a filter defined for traces cannot be applied to a metrics pipeline in any meaningful way. Each signal has its own set of conditions. A single filter definition can hold rules for traces, metrics and logs side by side, and each pipeline will use the part that matches its own signal.

The conditions in the filter processor are written in a small language designed for exactly this purpose.

The OpenTelemetry Transformation Language, usually shortened to OTTL, is a compact language for describing conditions and changes on telemetry. You can think of an OTTL condition as a yes-or-no question about one item: is this span’s name equal to a certain health-check route? Does this log’s severity fall below a chosen level? Does this metric’s name start with a certain prefix? If the answer is yes, the condition is said to be true or met.

The building blocks of a condition

Why OTTL is a good fit

Because OTTL is shared by several Collector components, the same way of describing data works in the filter processor, the transform processor and others. Learning it once pays off many times. It is also expressive enough to handle both simple needs, such as matching an exact name, and complex ones, such as combining several attributes in one test.

This single rule explains how almost every filter behaves, and how almost every filter goes wrong.

In the Filter Processor, you list one or more conditions. If any of those conditions is met for an item, the item is dropped. If none is met, the item continues down the pipeline. The conditions are combined with a logical OR: one yes is enough.

flowchart TD
  A["Item arrives at the filter"] --> B{"Is any condition true?"}
  B -- Yes --> C["Drop the item"]
  B -- No --> D["Pass the item on"]
        

Thinking in reverse

Because conditions describe what to remove, a rule meant to keep only certain data has to be written as the opposite. If you want to keep only the traces from one service, you cannot simply write “the service is checkout”. That would drop the checkout data and keep everything else. Instead, you write a condition that is true for everything except checkout, using a not-equals comparison.

Combining conditions

There are two ways to combine ideas, and they behave differently. Several separate conditions in the list are joined by OR: any one of them drops the item. Inside a single condition, the word and joins ideas so that all of them must be true together for that condition to be met.

Telemetry is layered like a set of nesting boxes, and the filter can look at any layer.

Telemetry is organised in layers. At the outside is the resource, which describes the thing that produced the data, such as a service or a host. Inside it is the scope, which describes the library or component that created the data. Inside that are the actual items: spans, metrics or log records. For some signals there are even smaller pieces inside those, such as events within a span or data points within a metric. Each of these layers is called a context, and a condition is evaluated in the context that matches the data it looks at.

flowchart TD
  R["Resource: the service or host"] --> S["Scope: the library that produced the data"]
  S --> T["Span / Metric / Log record"]
  T --> F["Span event / Data point"]
        

Outer decisions come first

The filter evaluates the outer layers before the inner ones. If a condition at a higher level is met, the whole thing at that level is dropped, and the inner levels are never examined. For example, if a span is dropped, the filter does not bother to check the events inside that span, because they disappear along with it.

Letting the processor work out the context

You rarely need to name the context yourself. The processor looks at the path your condition uses and works out which layer it belongs to. A condition that looks at a span’s attributes is understood to be a span-level condition, and one that looks at a log’s severity is understood to be a log-level condition. Naming the context explicitly is possible, but it is seldom necessary.

Traces are the signal where filtering needs the most care, because spans belong to families.

A trace is the story of one request. It is made of spans, each representing one step, such as a database call or a call to another service. Spans are arranged in a family tree: a parent span can have children, and together they show the full path of the request. A span can also carry span events, which are small timestamped notes recorded during the span, such as an exception being thrown.

Common reasons to filter spans

The orphan problem

Dropping a span that has children creates orphaned spans: children whose parent no longer exists in the stored trace. Viewing tools then show a trace with a gap in the middle, or children that float without a visible parent. This can make traces harder to understand, and it can confuse tools that calculate things such as the critical path.

flowchart TD
  A["Parent span: dropped by the filter"] -.-> B["Child span: kept but orphaned"]
  A -.-> C["Child span: kept but orphaned"]
        

Filtering span events

Span events are finer-grained notes inside a span. You can drop specific events while keeping the span itself. This is useful when a library records many repetitive events that add bulk without adding insight. Remember that if the span itself is dropped, its events go with it, so event conditions only run on spans that survived.

Metrics can be filtered by name, by type, or by the individual data points inside them.

A metric is a measurement tracked over time, such as the number of requests handled. A single metric usually contains many data points, each representing the value for one particular combination of attributes at one moment. For example, one request-count metric may hold separate data points for each route and status code.

Two levels, two kinds of decisions

Common reasons to filter metrics

Logs are usually the biggest source of volume, which makes them the most rewarding place to filter.

A log record is a single event written down by an application, with a timestamp, a message called the body, a severity that says how important it is, and extra attributes. Severity is the key to the most common log filter. Typical levels run from trace and debug, which are very detailed, through info, warning and error, up to fatal.

Common reasons to filter logs

The processor offers a simple style for everyday needs and a more detailed style for special cases.

The Filter Processor’s configuration has a section for each signal, and inside each section it groups conditions by the layer they apply to. There are two styles of writing these, and you can choose the one that suits the situation.

What should the filter do when a condition cannot be evaluated?

Sometimes a condition cannot be evaluated for a particular item. It might refer to an attribute that is missing, or try to compare values of different kinds. The processor needs a policy for this situation, and that policy is controlled by a setting called error mode. It has three possible values.

Processors run in the order they are listed, so position is part of the design.

The Collector applies processors one after another, in exactly the order they appear in the pipeline. The output of one becomes the input of the next. For the Filter Processor, the best position is usually early, because dropping unwanted data first means that later processors do less work.

flowchart LR
  A["Receiver"] --> B["Memory limiter"]
  B --> C["Filter processor"]
  C --> D["Transform or enrich"]
  D --> E["Batch"]
  E --> F["Exporter"]
        

When filtering should come later

Sometimes a filter needs information that another processor adds. If your condition depends on an attribute that a resource-detection or transform step creates, the filter must come after that step. The rule of thumb is that a filter should come as early as possible, but never earlier than the processor that produces the data it depends on.

Several tools reduce or reshape telemetry. Picking the right one avoids a lot of trouble.

The Filter Processor is not the only way to reduce data. It helps to know how it differs from related tools so you can choose the right one for each job.

Five everyday situations show how filtering is used in practice.

flowchart TD
  A["Is the data clearly unwanted?"] -- Yes --> B["Can a rule name it precisely?"]
  A -- No --> G["Consider sampling or leave it"]
  B -- Yes --> C["Filter it, test, then roll out"]
  B -- No --> D["Use sampling, transform or routing"]
        

Filtering is powerful, and like every powerful tool it needs respect.

These mistakes appear again and again, and every one of them is avoidable.

Because dropped data is gone for good, a careful rollout is part of the design.

The safest way to introduce a filter is to treat it like any change that can remove data: test it, measure it, and increase its reach gradually. The documentation itself advises testing configurations thoroughly against your real data patterns, because the exact shape of real telemetry is often different from what people imagine.

Filtering is often introduced to save money, but it also plays a part in privacy and governance.

Controlling cost

Most monitoring services charge by the volume of data ingested, the number of active metric series, or both. A filter that removes health-check traces, debug logs and unused metrics before they leave your network cuts those charges directly. It also lowers network traffic and the load on the rest of your pipeline.

Protecting privacy

Telemetry sometimes contains information that should not travel far: names, email addresses, tokens or other personal details accidentally written into log messages or attributes. A filter can remove whole items known to contain such details, which is a simple and robust form of protection. When you want to keep the item but remove only the sensitive field, a transform or attributes processor is usually a better fit.

Meeting data-handling rules

Some organisations must keep certain categories of data inside a region or away from third parties. A gateway Collector with filters can enforce this by dropping the restricted data before it reaches an external exporter. Because this is a policy decision, record it clearly and have the responsible team approve the rules.

Keep this list beside you whenever you write or review a filter.

Quick answers to the questions beginners ask most often.

It removes telemetry that matches conditions you define. Items that match are dropped, and everything else continues down the pipeline.

Traces, metrics and logs, and in newer versions profiles as well, though the support for profiles has been described as less mature. Check the documentation for your version for the current status of each signal.

What to drop. If any condition in the list is met for an item, that item is removed. To keep only certain data, describe everything outside that set.

Separate conditions in the list are combined with OR, so any one match drops the item. Inside a single condition, the word and requires all parts to be true together.

The OpenTelemetry Transformation Language, known as OTTL, which is shared by several Collector components.

They are dropped with it. Higher-level matches prevent lower-level checks, so events are only evaluated for spans that survived.

The metric itself is dropped too, because a metric with no data points carries no information.

If you drop a parent span, its children become orphans in the stored trace, which can make traces look broken and complicate analysis. Prefer dropping leaf spans, or whole branches together.

Ignore logs the problem and continues, silent continues without logging, and propagate returns the error and drops the affected payload. Set the mode explicitly rather than relying on a default.

Usually early: after the memory limiter, before enrichment and transformation, and well before batching. The exception is when your condition needs an attribute that another processor creates.

No. Filtering removes items that match content rules. Sampling keeps a fraction of traces, based on probability or policy, to reduce volume while staying representative.

The most common cause is that the processor was defined but never added to a pipeline in the service section. Other causes are a condition that never matches, or a typo in an attribute name.

No, not from the Collector. Once an item is dropped, it is gone, which is why testing and gradual rollout matter so much.

The Filter Processor is the guard at the door of an OpenTelemetry Collector pipeline. It checks each item of telemetry against conditions written in OTTL and drops the items that match, whether those items are spans, span events, metrics, data points or log records. Its golden rule is easy to state and easy to forget: a condition describes what to remove, not what to keep, and any single true condition is enough. Because it can look at several layers, from the whole resource down to a single data point, it can remove very precisely targeted slices of data. Used carefully, early in the pipeline, with tested rules and a safe rollout, it cuts cost, reduces noise and helps protect sensitive data, while the cautions about orphaned spans, emptied metrics and fragile text matching keep you from deleting something you will later need.