Before we define a metric, let us feel the question it answers.

Software is invisible

You cannot look at a running program and see whether it is happy. There is no smoke, no noise and no warning light. A web service might be handling ten requests a second or ten thousand. It might be using a little memory or almost all of it. It might be answering in twenty milliseconds or in twenty seconds. Unless the program tells you, you simply do not know.

Why detailed records are not always the answer

One way to understand a system is to record every single thing it does. Logs do that: they write a line for each event. Traces do something similar: they follow each request in detail. These are wonderful for investigating one particular problem. But if your shop handles a million requests an hour, reading a million log lines to answer “are we healthy?” is slow, expensive and tiring.

What you often want is a short summary: “how many requests per second?”, “how many failed?”, “what is the typical waiting time?”, “how full is the memory?”. You want a handful of numbers that update over time and can be drawn as a line on a chart. That is the job of metrics.

A plain definition first, then the reasons behind each word.

A metric is a measurement of something about your system, captured at regular intervals and summarised so that it can be tracked over time. It is usually a number, such as a count of requests, a measure of memory used, or the time a task took, together with a few labels describing what the number refers to.

That definition has three important ideas inside it.

Metrics are one of three sisters, and each sister answers a different question.

OpenTelemetry is an open-source project that provides a common way to create and send telemetry, which is data that software produces about itself. It handles three main kinds of signals, and they are designed to work together.

Where metrics fit in a real investigation

A typical story goes like this. A metric shows that the error rate has doubled. That tells you that something is wrong and when it started. You then open traces from that time window to see where in the system requests are failing. Finally, you read logs from the failing step to learn why. Metrics are usually the first alarm bell.

Every metric is built from the same few parts.

When OpenTelemetry sends a metric, it packages it in a standard shape. Learning these parts helps you read any metric from any system.

One metric, many series

Imagine a metric that counts requests. If you attach an attribute for the route, you do not get one number. You get one number for each route: one for the home page, one for checkout, one for search. Each unique combination of attribute values forms a separate time series, which is simply one line on a chart. One metric name can therefore produce many lines.

In OpenTelemetry, you do not write numbers straight onto a metric. You use an instrument.

An instrument is the object that your code uses to record a measurement. You pick the instrument that matches the kind of thing you want to measure. OpenTelemetry offers a small set, and choosing the right one is the most important metric decision you will make.

Synchronous and asynchronous instruments

Each instrument comes in two styles. A synchronous instrument is used inline, at the moment something happens. When a request arrives, your code immediately adds one to a counter. An asynchronous instrument, sometimes called an observable instrument, works differently. You register a small function, and the system calls it later, at each collection time, to read the current value. This suits things that you cannot easily update every time they change, such as the memory currently used by the process, or the temperature of a processor.

A short set of questions will lead you to the correct choice nearly every time.

Histograms are the most powerful instrument and the one beginners find hardest.

Suppose ten customers each wait for an answer. Nine of them wait one second, and one waits ninety-one seconds. The average wait is ten seconds. That number describes nobody. Nine people were happy and one was furious. An average hid the real story. A histogram keeps the shape of the data.

How a histogram works

A histogram sorts measurements into ranges called buckets. Each bucket counts how many measurements fell inside its range. For response times, one bucket might hold answers under a tenth of a second, the next might hold answers between a tenth and half a second, and so on. A histogram also usually records the total number of measurements and the sum of all of them, so that an average can still be calculated.

Percentiles in simple words

From a histogram, tools can estimate percentiles. The 50th percentile, often called the median, is the value that half of all measurements are below. The 95th percentile is the value that 95 out of every 100 measurements are below. The 99th percentile focuses on the unlucky slowest one in a hundred. Teams often watch the 95th or 99th percentile because it represents what the unhappiest real users experience.

A short tour of the machinery that turns instrument calls into data on a dashboard.

You do not have to master these parts, but knowing their roles makes everything else click. Inside an application, OpenTelemetry sets up a small pipeline.

flowchart LR
  C["Your code records a measurement"] --> I["Instrument"]
  I --> A["Aggregation combines measurements"]
  A --> V["Views adjust names and labels"]
  V --> R["Metric Reader collects on a schedule"]
  R --> E["Exporter sends data"]
  E --> CO["OpenTelemetry Collector"]
  CO --> B["Metrics Backend and Dashboard"]
        

Aggregation happens early

Here is a key idea. Your application does not send every individual measurement. As measurements arrive, the SDK combines them right away into running summaries. A counter keeps a running total. A histogram keeps bucket counts. A gauge keeps the latest value. Only these small summaries are sent at each collection. This is why metrics stay cheap even when traffic is huge.

Collection happens on a rhythm

The metric reader typically gathers the current summaries at regular intervals, often about once a minute by default, though the interval can be changed. Between collections, the application simply keeps updating its in-memory summaries. A shorter interval gives fresher charts but more data to send.

The same counter can be reported in two different ways, and both are valid.

Imagine a shop that counts customers entering. By midday the total is 300, and by evening it is 700. There are two ways to report this to head office. You can say “700 so far today”, a running total that always grows. Or you can say “400 since the last report”, the amount added in the latest period. The first is called cumulative and the second is called delta. The word for this choice is temporality.

Labels make metrics useful, and too many labels make them dangerous.

Attributes let you break a metric into slices. A request counter with a route attribute can show which pages are busiest. With a status attribute, it can show how many requests succeeded or failed. These slices are what turn a flat number into real insight.

What is cardinality?

Cardinality is the number of different values something can take. An attribute for HTTP method has very low cardinality, since there are only a handful of methods. An attribute for the customer’s user identifier has very high cardinality, since there might be millions of different users.

Because every unique combination of attribute values creates its own time series, the total number of lines multiplies quickly. Three routes, five status codes and four regions already make sixty series. Add a user identifier with a million values, and you could create sixty million series. Storage fills, queries slow down and bills explode.

Two extra pieces of context explain where a metric came from.

Besides its own attributes, every metric carries information about its producer. This comes in two layers.

These layers mean that metric data points do not need to repeat “service name equals checkout” on every line. The information is attached once, at a higher level, and applies to everything underneath. When you query for all metrics from one service, the resource is what makes that possible.

Agreeing on names is what makes metrics from different teams comparable.

If one team calls its metric “req_count”, another calls it “requests_total”, and a third calls it “http_hits”, then building a shared dashboard is painful. OpenTelemetry solves this with semantic conventions, a published set of agreed names, units and attributes for common situations such as web servers, databases, messaging systems and runtime behaviour.

When your instrumentation follows these conventions, any tool that understands them can recognise, for example, that a metric describes the duration of web requests, and can draw a ready-made chart. Many automatic instrumentation libraries already follow these conventions, so you get consistent metrics without extra work.

There are two classic ways for measurements to reach a monitoring tool.

Push

In the push style, your application sends metrics out on a schedule using the OpenTelemetry Protocol, often shortened to OTLP. It is like a reporter who phones the head office every minute with the latest figures. This is the native OpenTelemetry approach.

Pull

In the pull style, a monitoring tool visits your application on a schedule and reads the current values from a web page the application exposes. It is like an inspector who walks from shop to shop, reading each meter. Many open-source systems work this way, and OpenTelemetry can expose its metrics in that format.

The Collector in the middle

The OpenTelemetry Collector is a separate program that receives telemetry, processes it, and forwards it. For metrics, it can accept both pushed OTLP data and pulled data from many sources, convert between cumulative and delta styles, add labels, remove unwanted ones, batch data for efficiency, and send it to one or several destinations at once.

flowchart LR
  A1["App pushing OTLP metrics"] --> COL["Collector"]
  A2["App exposing a metrics page"] --> COL
  H["Host and container metrics"] --> COL
  COL --> P1["Add labels and remove noisy ones"]
  P1 --> P2["Convert temporality and batch"]
  P2 --> B1["Open-source metrics store"]
  P2 --> B2["Commercial monitoring platform"]
        

Each signal is strongest when it can point to the others.

OpenTelemetry is designed so that metrics, traces and logs can reference each other. This is called correlation, and it turns three separate piles of data into one connected investigation.

flowchart LR
  M["Metric alert: error rate jumped"] --> X["Open an exemplar trace"]
  X --> T["Find the failing span"]
  T --> L["Read the linked logs"]
  L --> F["Understand and fix the cause"]
        

Good metrics answer the questions people really ask at three in the morning.

Beginners often ask, “What should I measure?” There are a few well-known frameworks that turn that vague question into a short checklist.

The RED method for services

The USE method for resources

Business metrics

Technical health is only part of the picture. Counting orders placed, payments completed or sign-ups finished tells you whether the business is working, not just the machines. A sudden drop in orders with perfectly healthy servers can reveal problems that technical metrics never show.

Metrics look harmless, but they can still reveal more than you expect.

Because metrics are summaries, they feel safer than logs. But attributes can still carry sensitive values, and a metrics endpoint can reveal how your system is built. Treat metrics with the same respect as any other operational data.

Habits that keep metrics useful, affordable and trustworthy.

Good habits

Mistakes to avoid

Metrics are the cheapest and fastest signal, but they have limits.

Metrics, traces and logs compared

Picture how different teams put metrics to work every day.

You do not need to learn everything at once.

It is a number that describes something about your software, measured again and again over time, and usually summarised so it stays small. Examples include requests per second, memory in use and response time distribution.

A metric is a summary number over time, a trace follows one request across services, and a log is a text record of one event. Metrics tell you that something changed, traces show where, and logs often explain why.

An instrument is the object your code uses to record a measurement. Counters, up-down counters, histograms and gauges are the main kinds, and each comes in a synchronous and an asynchronous style.

Use a counter for totals that only increase, such as requests or errors, and let your tool compute the rate. Use a gauge for a value that you simply read at a moment in time, such as current temperature or a configured limit.

Because averages hide slow outliers. A histogram keeps the shape of the data, so you can see the median and high percentiles, which reflect what your unluckiest users actually experience.

Cumulative reports the running total since the start, while delta reports only the change since the previous report. Different backends prefer different styles, and the Collector can convert between them.

Cardinality is the number of different values an attribute can take. Every unique combination of attribute values makes a new time series, so high-cardinality labels such as user identifiers can create huge numbers of series, raising cost and slowing queries.

Typically on a regular rhythm, often around once a minute by default, though you can change the interval. Shorter intervals give fresher charts but produce more data. In pull setups, the monitoring tool decides how often to visit.

No, applications can send metrics straight to a compatible backend. A Collector is recommended as systems grow, because it centralises label rules, temporality conversion, batching and destination changes.

An exemplar is a small sample attached to a metric data point that points to a specific trace. It lets you jump from a spike on a chart to a real example request that contributed to it.

For each service, start with request rate, error count and request duration. For key resources, add utilisation, saturation and errors. Then add one or two business metrics, such as orders completed.

They can if you put personal or secret values into attributes. Keep attributes small, bounded and free of personal data, protect metrics endpoints, encrypt data in transit, and consider a Collector rule that removes risky labels as a safety net.

A metric in OpenTelemetry is a summarised measurement of your software, tracked over time and described by a name, a unit, a type and a set of labelled data points. Your code records measurements with instruments: counters for totals, up-down counters and gauges for levels, and histograms for distributions. The SDK combines measurements into compact summaries, a reader collects them on a schedule, and an exporter sends them to a Collector and then to a backend, where they become lines on dashboards and triggers for alerts. Choose the right instrument, set clear units, keep attributes small and bounded, follow semantic conventions, and watch percentiles rather than averages. Used well, metrics are your earliest and cheapest warning system, and they point you towards the traces and logs that explain why.