Before we meet the backend, let us feel why it is needed.

Software produces a river of signals

Every running application can describe itself through telemetry: traces that follow a request across services, metrics that count and measure, and logs that record events. In a modern system with hundreds of services, this adds up to millions or billions of records a day.

A signal that is not kept cannot be searched

If telemetry is produced and then thrown away, nobody can look at it later. When a customer complains about a slow checkout that happened an hour ago, the evidence must still exist. Someone has to receive the data, store it, organise it so it can be found quickly, and show it in a form humans can understand.

Raw data is not understanding

Even stored telemetry is not enough. A billion log lines in a pile tell you nothing. People need ways to ask questions, to see trends as pictures, to follow a single request from start to finish, and to be told automatically when something goes wrong. All of that is the job of the backend.

The backend is the last stop of a longer journey.

OpenTelemetry, often shortened to OTel, is an open-source project that defines a common way to create, process and send telemetry. It is made of several parts that work together as a chain.

flowchart LR
  APP["Applications with OpenTelemetry"] -->|"OTLP"| COL["Collector"]
  COL -->|"OTLP or native format"| BE["Observability Backend"]
  BE --> ST["Storage"]
  BE --> QU["Query"]
  BE --> VI["Dashboards"]
  BE --> AL["Alerts"]
  VI --> HU["People"]
  AL --> HU
        

Why OpenTelemetry has no backend of its own

This is a deliberate choice, not an omission. OpenTelemetry aims to be vendor-neutral. It standardises how telemetry is made and moved, and leaves the choice of where to keep it to you. That means you can change your backend without re-instrumenting your applications. The producer side stays the same, and only the destination changes.

A plain definition first, then the reasons behind each word.

An observability backend is a system that receives telemetry data, stores it, makes it searchable, and gives people tools to explore it, draw it as charts and be alerted about it. It is the destination of your telemetry pipeline and the place where engineers actually look when they investigate a problem.

A backend is really a bundle of several jobs working together.

Traces, metrics and logs have very different shapes, which affects the backend you choose.

Each telemetry signal has its own character, and storing each one well requires different techniques.

Two common approaches

Neither approach is simply better. Small teams often prefer a unified product for convenience. Large organisations sometimes prefer separate best-of-breed systems for control and cost. What matters is that the signals can be connected, which we will explore soon.

There is a wide landscape, and it helps to know its main regions.

There are a few roads from your application to the backend.

Road 1: Native OTLP

Many modern backends can accept the OpenTelemetry Protocol directly. In that case, the Collector or the application’s exporter sends OTLP, and the backend understands it without any translation. This is the simplest road and the most future-proof, because it uses the standard format.

Road 2: Through a translating exporter

Some backends speak their own format. The Collector offers exporters that convert OpenTelemetry data into that format and send it. This lets you keep a standard pipeline while still feeding a backend that does not yet understand OTLP.

Road 3: Pull instead of push

Some systems collect by asking: the backend visits a web address on a schedule and reads the current numbers. The Collector can expose metrics in that style, so the backend can pull from it. Push and pull are two ways of moving the same data.

flowchart TD
  APP["Applications"] --> COL["Collector"]
  COL -->|"Road 1: push OTLP"| B1["Backend that speaks OTLP"]
  COL -->|"Road 2: translating exporter"| B2["Backend with its own format"]
  B3["Backend that pulls"] -->|"Road 3: scrapes metrics"| COL
        

Why the Collector in the middle helps

Placing a Collector between applications and the backend gives you a single, central place to change the destination, add a second destination, remove sensitive details, sample data and handle retries. Applications never need to know which backend is in use.

Storage decides how fast you can search and how much you will pay.

Telemetry is written continuously and read in bursts, usually when something is wrong. Backends therefore design their storage around two goals: absorb heavy writing cheaply, and answer common questions quickly. Different signals favour different designs.

Hot, warm and cold

Not all data deserves equal treatment. Recent data is looked at constantly and should be on fast, more expensive storage. Older data is rarely read and can move to slower, cheaper storage. Many backends support tiers that move data automatically as it ages.

Storing data is only half the job. Finding the answer is the other half.

Each backend offers its own way to ask questions, and there is no single universal query language. What matters is understanding the kinds of questions you will ask, and checking that the backend answers them quickly.

The power of consistent attribute names

Queries become far easier when all services label their data in the same way. OpenTelemetry publishes semantic conventions, which are agreed names for common attributes such as service name, request method or database operation. When every service follows them, one query works across the whole system, and ready-made dashboards fit your data without custom work.

People understand pictures faster than tables.

A backend turns raw records into pictures that tell a story at a glance. The common ones are worth knowing.

Good dashboards answer questions

A dashboard crowded with fifty charts helps nobody. A good one starts with the questions “is it healthy?” and “if not, where is the problem?”, and leads the viewer from a high-level summary down to detail. Many teams build dashboards around the user-facing symptoms first: how fast, how often it fails, and how busy, and keep deeper technical charts one click away.

Nobody can watch dashboards twenty-four hours a day.

An alert is a rule that the backend checks continuously. When the rule becomes true, someone is notified, by message, email, phone call or a link to an incident tool. Good alerting is the difference between learning about a problem from your monitoring tool and learning about it from angry customers.

Alert on symptoms, not on every cause

Alerts that fire for every small blip train people to ignore them. It is usually better to alert on what users feel, such as slow responses or failed purchases, and use dashboards and traces to find the cause. This keeps the number of alerts small and each one meaningful.

The real power of a backend appears when signals are connected.

Imagine a chart showing a spike in slow responses. The natural next questions are: which requests were slow, and what happened inside them? The best backends let you click from the spike to example traces, and from a trace to the log lines written during it. OpenTelemetry makes this possible because it carries shared identifiers and labels through every signal.

flowchart LR
  M["Metric chart shows a spike"] -->|"exemplar link"| T["Example slow trace"]
  T -->|"trace and span identifiers"| L["Log lines from that request"]
  L -->|"resource labels"| S["Other signals from the same service"]
        

Telemetry is cheap to create and expensive to keep.

Adding instrumentation takes an afternoon. Storing and searching its output for a year can cost more than the servers running the applications. Understanding what drives cost lets you control it from the start.

Levers you can pull

Telemetry often contains more sensitive information than people expect.

Logs and trace attributes can accidentally capture email addresses, account numbers, tokens or health details. Once in the backend, that data may be readable by many people and stored for a long time. Treating the backend as a store of sensitive data is a healthy default.

Run it yourself, or let someone else run it?

One of the first choices is whether to operate the backend yourself or to pay a provider to do it. Both are legitimate, and the right answer depends on your team and your needs.

The biggest gift of OpenTelemetry is freedom to move.

In the past, each monitoring vendor supplied its own agents and libraries. Choosing a vendor meant embedding their code in every application, and changing vendors meant rewriting instrumentation across the company. That cost kept many organisations stuck with tools they had outgrown.

With OpenTelemetry, applications emit standard telemetry that any compatible backend can read. Moving to a new backend becomes mostly a matter of changing the destination in the Collector, rather than changing application code.

Match the backend to your real situation, not to the loudest advertisement.

Run a small trial first

Habits that keep a backend useful, affordable and trusted.

Good habits

Mistakes to avoid

Picture how different teams use a backend every day.

It is the system where your telemetry ends up. It receives traces, metrics and logs, stores them, lets you search them, draws dashboards and sends alerts, so people can understand and fix problems in their software.

No. OpenTelemetry standardises how telemetry is created and moved, but deliberately does not store or display it. You choose a backend separately, which is what keeps the project vendor-neutral.

Not strictly, because applications can often export directly to a backend that accepts OTLP. But a Collector gives you a central place to filter, redact, sample, add labels and change destinations, so most production systems use one.

Yes. The Collector can send the same data to several destinations at once, which is useful for trials, migrations, or sending different kinds of data to different places.

Both work. A unified platform is easier to use and correlate, while separate specialist systems give flexibility and cost control. The key requirement is that you can move easily between metrics, traces and logs during an investigation.

Not always. Self-hosting can be cheaper at large scale in raw infrastructure terms, but you must also count the people and time needed to run it reliably. For small teams, managed services are often cheaper overall.

Mainly data volume, metric label cardinality, retention length, indexing depth and query load. Sampling, filtering, label control and tiered retention are the main ways to reduce cost.

It removes lock-in at the instrumentation level, which is the hardest part to change. But dashboards, alert rules, saved queries and team habits are specific to each backend and would need rebuilding if you moved.

Through shared identifiers and labels. Exemplars on metrics can point to example traces, and trace and span identifiers in log records tie each line to its trace. Choose a backend that makes these jumps easy.

Only with care. Telemetry can contain personal or secret data by accident. Redact sensitive values in the Collector, control access by role, encrypt data, and set retention that matches your rules.

It depends on value and obligations. Detailed traces and logs are often kept for days or weeks, aggregated metrics for months or longer, and audit-related data as long as rules require. Tiered storage helps balance cost and history.

List the questions your team asks most during incidents, shortlist two or three candidates, send the same real data to each through the Collector, replay a real incident, estimate cost at full scale including people time, and prefer options with native OTLP support and easy export.

An observability backend is the destination of an OpenTelemetry pipeline: the system that receives telemetry, stores it, indexes it, answers queries, draws dashboards and trace views, and raises alerts. OpenTelemetry deliberately does not include one, so that you can choose freely and change later without touching application code. Backends differ in how they store traces, metrics and logs, in whether they are open source, managed or provided by a cloud vendor, and in how well they connect the signals through shared labels, trace identifiers and exemplars. Their cost is driven by volume, cardinality, retention, indexing and query load, so sampling, filtering in the Collector and tiered retention matter from day one. Treat the backend as a store of sensitive data, restrict access, redact early and keep it independent from the systems it watches. Choose by running a small trial with real data and a replayed incident, favour native OTLP support, and remember that dashboards and alert rules remain yours to rebuild if you ever move.