What Is Prometheus in OpenTelemetry?

What Is Prometheus in OpenTelemetry?

A friendly, ground-up guide to how two famous monitoring projects fit together: what Prometheus is, what OpenTelemetry is, the different ways they connect, how their data is translated, and how to avoid the common traps.

Imagine two friends who both love tracking things. One friend, Prometheus, keeps a neat notebook of numbers: how many visitors came, how hot the oven is, how many orders failed. Every few seconds, Prometheus walks around the house with a clipboard and writes down each reading. The other friend, OpenTelemetry, has a different habit. OpenTelemetry gives every room a small messenger who walks over and delivers readings on their own, in a standard format that any notebook can understand.

These two friends do not compete. In most modern systems, they work together. OpenTelemetry is very good at collecting telemetry in a standard way, and Prometheus is very good at storing and querying numbers over time. In this guide, we will learn what Prometheus is, how it connects with OpenTelemetry, and how to choose the right connection, using simple words and everyday examples.

Metrics
The one signal Prometheus focuses on, out of traces, metrics and logs
3
Main ways data travels: scrape, remote write and native OTLP
2018
Year Prometheus graduated in the CNCF, the home of OpenTelemetry too

1The Problem: Two Great Tools, One Confusing Question

Before learning the details, let us see why people ask this question at all.

Why the confusion happens

Many teams already run Prometheus. It has been the favourite metrics tool for years, especially in Kubernetes. Then they hear about OpenTelemetry, which also deals with metrics. They wonder: “Does OpenTelemetry replace Prometheus? Do I need both? Do I have to rewrite everything?” The honest answer is that they overlap in some areas, but they are built for different jobs, and they were designed to work together.

The core difference in one idea

OpenTelemetry is a standard for producing and moving telemetry, which is information that software sends out about itself. It covers traces, metrics and logs. It does not store or show the data. Prometheus is a system for storing and querying metrics. It does not cover traces or logs. One is mainly the delivery service. The other is mainly the warehouse and the search desk.

Everyday Analogy

Think of a library. OpenTelemetry is like the standard book format and the delivery van that brings books from many publishers. Prometheus is like the shelf and the catalogue where numeric books are stored and searched. The same library can use the standard van to fill the shelf. They are different parts of one system.

2So, What Is Prometheus?

First the tool by itself, with no OpenTelemetry in the picture yet.

Prometheus is an open-source monitoring system and a time-series database. It began at SoundCloud in the early 2010s and later became a graduated project of the Cloud Native Computing Foundation, the same neutral foundation that looks after Kubernetes and OpenTelemetry. It has become a standard choice for numeric monitoring and alerting.

The key words, one by one

Data Shape

Time Series

A list of values over time for one thing, such as “requests handled by the checkout service”. Each value has a timestamp.

Identity

Metric Name and Labels

A series is identified by a name plus labels, which are key and value tags such as method equals GET, or region equals eu-west.

Collection

Scraping

Prometheus pulls data. At regular times, it visits a web address on each target, usually named metrics, and reads the numbers there.

Format

Exposition Format

A simple text format that a program uses to list its current numbers on that web address. A related standard is called OpenMetrics.

Search

PromQL

Prometheus’s query language, used to ask questions like “what was the error rate over the last five minutes?”

Alerts

Alert Rules and Alertmanager

Rules that watch for trouble, and a companion program that groups, silences and sends notifications.

Everyday Analogy

Think of a school caretaker doing a daily round. Every ten minutes, the caretaker walks past each classroom and reads the thermometer on the wall, then writes the reading in a big book with the room name and time. That walk is scraping, the thermometers are the targets, and the book is the time-series database.

What Prometheus is not

  • It is not a tracing system. It cannot show the journey of one request through many services.
  • It is not a log store. It does not keep text messages for searching.
  • It is not built for unlimited history on one machine. A single Prometheus server keeps data locally for a limited time, and long-term storage usually needs helper systems.
  • It does not like unlimited label values. Too many unique label combinations make it slow and hungry for memory.

3Core Concepts: Pull versus Push

The biggest practical difference between the two worlds is how data moves.

Prometheus is famous for the pull model. It reaches out and collects data. OpenTelemetry is mainly built around the push model. Applications and Collectors send data out using the OpenTelemetry Protocol, called OTLP. Neither is right or wrong. They are two ways of moving the same numbers.

sequenceDiagram
    participant P as Prometheus Server
    participant A as Application with metrics endpoint
    participant S as Application with OTel SDK
    participant C as OTel Collector
    P->>A: Scrape request every interval
    A-->>P: Current metric values
    S->>C: Push metrics using OTLP
    C->>P: Remote write or OTLP push
    
Figure 1: Prometheus pulls from endpoints, while OpenTelemetry pushes through a Collector
QuestionPull (Prometheus Style)Push (OpenTelemetry Style)
Who starts the conversation?The monitoring serverThe application or Collector
How do you know a target is alive?If the scrape fails, the target is probably down, and Prometheus records itYou notice silence, which needs extra alerts, since no data may just mean no message
Good forLong-running services with a stable addressShort-lived jobs, serverless functions and apps behind firewalls
Control of rateThe server decides how often to collectThe sender decides how often to send
Network directionServer must be able to reach targetsSenders must be able to reach the receiver
i
Tip

OpenTelemetry can also work in a pull style, and Prometheus can also receive pushed data. In practice, you can mix and match. The model is a choice you make for each part of your system, not a wall between two products.

4The Big Picture: How They Connect

There are several bridges between OpenTelemetry and Prometheus. Here they are on one map.

flowchart LR
    subgraph SRC["Sources of metrics"]
        APP1["App with OpenTelemetry SDK"]
        APP2["Legacy app with Prometheus endpoint"]
        INF["Infrastructure exporters"]
    end
    APP1 -->|"OTLP push"| COL["OpenTelemetry Collector"]
    APP2 -->|"Scraped by Prometheus receiver"| COL
    INF -->|"Scraped by Prometheus receiver"| COL
    COL -->|"Remote write"| LTS["Prometheus-compatible long term storage"]
    COL -->|"OTLP push"| PROM["Prometheus server with OTLP receiver"]
    COL -->|"Prometheus exporter endpoint"| PROM2["Prometheus scraping the Collector"]
    PROM --> GR["Dashboards and alerts"]
    LTS --> GR
    PROM2 --> GR
    
Figure 2: Several bridges connect OpenTelemetry sources to Prometheus storage

The four main bridges

Bridge 1

Prometheus Receiver

A Collector component that scrapes Prometheus-style endpoints, so the Collector can collect from everything that Prometheus normally would.

Bridge 2

Prometheus Exporter

A Collector component that exposes its own metrics endpoint, so that a Prometheus server can scrape the Collector.

Bridge 3

Remote Write Exporter

A Collector component that pushes metrics to Prometheus-compatible systems using the remote write protocol.

Bridge 4

Native OTLP Ingestion

Recent Prometheus versions can accept OTLP metrics directly when this feature is turned on, with no translation layer needed in front.

There is also a fifth, smaller bridge on the application side: some OpenTelemetry SDKs offer a Prometheus exporter that exposes a metrics endpoint inside the application, so that a Prometheus server can scrape the app directly. The status of this exporter differs between languages, so check your language’s documentation.

Everyday Analogy

Think of several roads between two towns. One is a bus that picks passengers up from many villages (the receiver). One is a bus stop where the other town’s bus comes and collects people (the exporter endpoint). One is a direct delivery truck (remote write). And, recently, a new direct road has opened (native OTLP). All lead to the same town, but each suits a different traveller.

5Bridge 1: The Prometheus Receiver

Bringing the old Prometheus world into the OpenTelemetry Collector.

Years of work have gone into small programs called exporters that expose metrics in the Prometheus format. There are exporters for databases, hardware, message queues and operating systems. Thousands of applications also expose a metrics endpoint. Rewriting all of these would be impossible. The Prometheus receiver solves this by letting the Collector behave like a Prometheus scraper. It even understands the same scrape configuration style, including how to discover targets, so teams can often move their existing setup over with little change.

1

Discover Targets

The receiver uses the configured discovery method, such as Kubernetes pod discovery, to find which endpoints to visit.

2

Scrape on a Schedule

At every interval, it calls each target’s metrics address and reads the text output.

3

Convert to the OpenTelemetry Model

The Prometheus series are turned into OpenTelemetry metrics, with types, labels turned into attributes, and scrape health recorded.

4

Flow Through the Pipeline

The data passes through Collector processors, such as filtering and batching, like any other metric.

5

Export Anywhere

Exporters then send the data to Prometheus-compatible storage or any other backend.

Scaling the receiver with the Target Allocator

A single Collector scraping thousands of targets can become overloaded. If you run several Collector copies, you must make sure each target is scraped by exactly one of them. Otherwise, you get duplicates or gaps. The Target Allocator, part of the OpenTelemetry Operator for Kubernetes, solves this. It discovers targets, splits them fairly between Collector replicas, and hands each replica its own list. It can also read the same Kubernetes resources, called ServiceMonitor and PodMonitor, that many Prometheus users already have.

flowchart TD
    K["Kubernetes API with ServiceMonitors and PodMonitors"] --> TA["Target Allocator"]
    TA -->|"Assigns targets 1 to 100"| C1["Collector replica 1"]
    TA -->|"Assigns targets 101 to 200"| C2["Collector replica 2"]
    TA -->|"Assigns targets 201 to 300"| C3["Collector replica 3"]
    C1 --> T1["Scrape assigned pods"]
    C2 --> T2["Scrape assigned pods"]
    C3 --> T3["Scrape assigned pods"]
    C1 --> OUT["Exporters to storage"]
    C2 --> OUT
    C3 --> OUT
    
Figure 3: The Target Allocator shares scrape targets fairly among Collector copies
!
Watch Out

Do not run several Collectors with the same scrape settings and no sharing of targets. Every target will be scraped many times, which wastes resources, creates duplicates and can overload your applications.

6Bridges 2 and 3: Exporting to Prometheus

Two ways to hand OpenTelemetry metrics to a Prometheus world.

The Prometheus exporter: wait to be scraped

With the Prometheus exporter, the Collector holds the latest metric values in memory and shows them on a web address. A Prometheus server then scrapes that address like any other target. This is simple and fits teams who already use a Prometheus server and want it to stay in charge of collecting.

  • Good: very simple, and works with any standard Prometheus setup.
  • Watch: only the latest value is held, so the scrape interval decides the resolution. Data can go stale if nothing new arrives.
  • Watch: a single Collector copy usually serves this endpoint, which can become a bottleneck for very large volumes.

The remote write exporter: push to storage

Remote write is a protocol that Prometheus created for sending its data to other systems. Many long-term storage and managed services accept it. The remote write exporter in the Collector converts OpenTelemetry metrics into this format and pushes them. This is the usual path for sending data to large, scalable backends.

Self-Hosted

Thanos, Mimir and Cortex

Open-source systems that provide long-term, scalable storage for Prometheus-style data, often receiving data through remote write.

Managed

Cloud Prometheus Services

Services such as Amazon Managed Service for Prometheus, Google Cloud Managed Service for Prometheus and Grafana Cloud accept this style of data, so you do not run storage yourself.

QuestionPrometheus ExporterRemote Write Exporter
DirectionPrometheus scrapes the CollectorCollector pushes to the destination
Best forSmall and medium setups with an existing Prometheus serverLarge setups, long-term storage and managed services
Extra featuresNone needed beyond a scrape jobRetries, queues and authentication to protect delivery
Main riskStale or lost values if scraping stopsDropped data if the destination is slow and queues overflow

7Bridge 4: Native OTLP Ingestion in Prometheus

The newest and simplest road, if your version supports it.

Recent versions of Prometheus can act as a receiver of OTLP metrics. When this feature is switched on, an application or Collector can push OTLP metrics straight to a Prometheus web address, with no remote write conversion needed in between. This removes a step and makes the path much shorter.

Benefits

  • Fewer moving parts and one less conversion step
  • Natural fit for applications that already speak OTLP
  • Good for short-lived jobs that cannot be scraped

Cautions

  • The feature must be enabled and may be treated as newer than other paths
  • Name and label translation rules must be chosen with care
  • Pushing at large scale into one server needs capacity planning
  • Only metrics are accepted, so traces and logs need other destinations
i
Tip

Features in this area have been changing quickly between Prometheus releases. Check the documentation for the exact version you run before building a design that depends on native OTLP ingestion.

8Internal Working: Translating Between Two Data Models

The two projects describe metrics in slightly different ways. Translation is where most surprises hide.

Metric types

OpenTelemetry TypeBecomes in PrometheusPlain Meaning
Counter (monotonic sum)CounterA value that only goes up, such as total orders
UpDownCounter (non-monotonic sum)GaugeA value that goes up and down, such as items in a queue
GaugeGaugeThe latest value at a moment, such as temperature
Histogram with fixed bucketsHistogram, shown as bucket, sum and count seriesA spread of values grouped into ranges
Exponential histogramNative histogram, in versions that support itA more compact and precise spread of values
SummarySummaryPre-computed percentiles, mostly seen in older setups

Temporality: the quiet trap

Temporality describes whether a reported number is a running total since the start, called cumulative, or just the change since the last report, called delta. OpenTelemetry supports both. Prometheus works with cumulative values. If an application sends delta values and nobody converts them, the numbers in Prometheus will make no sense. The Collector can help by converting delta to cumulative before export.

Everyday Analogy

Think of a car. The odometer shows the total distance since it was built: that is cumulative. The trip meter, reset each morning, shows only today’s distance: that is delta. Prometheus expects odometer readings so it can work out speed by comparing two readings. If you hand it trip meter readings that keep resetting, its calculations will go wrong.

Names and units

OpenTelemetry metric names often use dots, such as http.server.request.duration, and carry a separate unit field. Prometheus traditionally allows only letters, digits, underscores and colons in names, and it has a convention of putting the unit and type into the name itself. So the translation usually changes dots to underscores, adds a unit suffix such as seconds or bytes, and adds a total suffix for counters.

OpenTelemetry MetricTypical Prometheus Name
Counter orders.placedorders_placed_total
Histogram http.server.request.duration with unit secondshttp_server_request_duration_seconds with bucket, sum and count series
Gauge memory.usage with unit bytesmemory_usage_bytes

Newer Prometheus versions support a wider range of characters in names, so dots can sometimes be kept, and translation strategies can be chosen. This is helpful, but it means that two teams can end up with different names for the same metric if they choose different strategies. Agree on one approach for the whole company.

Labels, resources and scope

In OpenTelemetry, a measurement has attributes, which become Prometheus labels. The program that produced it is described by resource attributes, such as the service name and instance ID. The library that created the metric is described by the instrumentation scope. Prometheus has no separate place for these, so the translation follows a few conventions.

flowchart TD
    M["OpenTelemetry metric point"] --> A["Attributes"]
    M --> R["Resource attributes"]
    M --> S["Instrumentation scope"]
    A --> L1["Become Prometheus labels"]
    R --> L2["Service name and namespace form the job label"]
    R --> L3["Service instance ID forms the instance label"]
    R --> L4["Other resource attributes go into a target_info metric"]
    S --> L5["Scope name and version become otel_scope labels"]
    
Figure 4: How the parts of an OpenTelemetry metric land in Prometheus
  • The service name, with the namespace if present, becomes the job label.
  • The service instance ID becomes the instance label.
  • The remaining resource attributes are stored in a separate metric called target_info, which you can join with your other metrics in queries. Many setups also promote a few chosen resource attributes, such as the environment or cluster, directly into labels so queries are easier.
  • The instrumentation scope often appears as labels such as otel_scope_name.
!
Watch Out

Promoting too many resource attributes to labels increases the number of unique series, which Prometheus handles badly. Promote only a short, stable list such as environment, cluster and region.

9Data Flow and Lifecycle of One Metric

Follow one number from your code to a dashboard.

1

Code Records a Measurement

Your application uses the OpenTelemetry API to add one to a counter, such as “orders placed”.

2

The SDK Aggregates

The SDK adds up the measurements for each combination of attributes during the reporting interval.

3

The Exporter Sends OTLP

On a schedule, the SDK sends the summary to the Collector using OTLP, along with resource information.

4

The Collector Processes

Processors can convert delta to cumulative, filter noise, add labels, and batch the data.

5

Translation Happens

The exporter converts the names, types, labels and resource information into Prometheus style.

6

Prometheus-Compatible Storage Keeps It

The time series are stored with timestamps, ready to query.

7

PromQL, Dashboards and Alerts

Engineers query the data, draw graphs, and alert rules watch for trouble.

Exemplars: a link from numbers to traces

Prometheus can store exemplars, which are sample values with a trace ID attached. OpenTelemetry metrics can carry exemplars too. When this link is kept through the pipeline, an engineer who sees a spike on a graph can click a dot and jump straight to an example slow trace. This ties together Prometheus for metrics and a tracing system for traces, and shows why the two work well together.

Production Example: From Graph Spike to Slow Trace

A latency graph for the checkout service shows a sudden jump. Hovering over the spike reveals an exemplar with a trace ID. One click opens that exact trace, which shows that a payment call was slow because of a database lock. What would have been an hour of guessing took two minutes.

10Choosing the Right Path

Which bridge should you use? It depends on where you are starting.

flowchart TD
    Q["Where are you starting?"] --> A{"Do you already have Prometheus exporters and endpoints?"}
    A -->|Yes| B["Use the Prometheus receiver in the Collector, with Target Allocator on Kubernetes"]
    A -->|No| C{"Are your apps instrumented with OpenTelemetry SDKs?"}
    C -->|Yes| D{"Do you need long term storage or a managed service?"}
    D -->|Yes| E["Collector with remote write exporter"]
    D -->|No| F{"Does your Prometheus version support OTLP ingestion?"}
    F -->|Yes| G["Push OTLP directly to Prometheus"]
    F -->|No| H["Use the Prometheus exporter and let Prometheus scrape the Collector"]
    C -->|No| I["Start with auto-instrumentation and then pick a path above"]
    
Figure 5: A simple guide to choosing an integration path
SituationSuggested PathWhy
Many existing Prometheus exportersPrometheus receiver in the CollectorKeep what works and gain Collector processing
New services using OpenTelemetry SDKsOTLP to a Collector, then remote write or OTLP to storageStandard, flexible, vendor neutral
Short-lived jobs and serverlessPush using OTLPThey cannot wait to be scraped
Large scale and long retentionRemote write to Thanos, Mimir, Cortex or a managed serviceScalable storage beyond one server
Small team with one Prometheus serverPrometheus exporter on the Collector, or native OTLPSimplest setup
Gradual migrationRun both and compare, then move services one by oneLower risk

11Advantages, Disadvantages and Trade-offs

An honest look at combining the two.

Advantages of Using Them Together

  • Best of both: OpenTelemetry for standard collection, Prometheus for storage and query
  • Keep existing investment: old exporters, dashboards and alerts continue to work
  • Vendor freedom: change storage without changing instrumentation
  • One pipeline: traces, metrics and logs can share the Collector
  • Strong ecosystem: Grafana, Alertmanager and many tools understand Prometheus data
  • Smooth migration: move gradually, not all at once

Disadvantages and Challenges

  • Translation complexity: names, types and labels do not map perfectly
  • Temporality mismatch: delta data needs conversion
  • More moving parts: Collector, Target Allocator and storage all need care
  • Cardinality risk: rich attributes can overwhelm Prometheus
  • Evolving features: native OTLP and name handling keep changing
  • Overlap confusion: teams may duplicate collection without noticing

Key trade-offs

DecisionOption AOption B
Collection stylePull with Prometheus, simple and familiarPush with OTLP, flexible for short-lived and remote workloads
Where to scrapeA Prometheus server, as beforeThe Collector, to centralise processing
Metric namesTranslate to classic Prometheus names for old dashboardsKeep OpenTelemetry style names with newer UTF-8 support
Resource attributesPromote a few to labels for easy queriesKeep them in target_info for fewer series
StorageOne Prometheus server, simple but limitedRemote write to scalable storage, more power with more parts

12Security Considerations

Metrics endpoints and ingestion paths are doors into your systems. Treat them that way.

Exposure

Metrics Endpoints Reveal a Lot

A public metrics page can show internal service names, versions, error counts and traffic patterns. Keep these endpoints on private networks.

Access

Authenticate and Encrypt

Use encrypted connections and authentication for scraping, remote write and OTLP, especially across networks you do not fully control.

Ingestion

Protect Write Endpoints

Anyone who can push data into Prometheus can fill it with junk or hide real signals. Restrict who may write.

Labels

No Personal Data in Labels

Never use emails, names or tokens as label values. They become permanent parts of series names and are hard to delete.

Kubernetes

Least Privilege for Discovery

The Collector and Target Allocator need permission to list pods and services. Grant only what is needed.

Limits

Guard Against Cardinality Floods

A bug or attacker can create endless new label values. Use limits on labels and series to protect memory.

!
Watch Out

Exemplars carry trace IDs. Trace IDs are not secrets, but they connect metrics to traces, so make sure that whoever can see the metrics is allowed to follow those links into trace data.

13Monitoring, Logging and Troubleshooting

The pipeline that carries your metrics should itself be watched.

  • Target health: the scrape health of every target. Prometheus-style scraping records whether a target answered, which is the first thing to check when data goes missing.
  • Scrape duration and failures: slow or failing scrapes may mean overloaded targets or a busy Collector.
  • Collector send failures: watch the counts of failed and dropped metric points from the exporters, and the size of their queues.
  • Remote write lag: if data arrives in storage minutes late, queues may be full or the destination may be slow.
  • Series count and growth: a sudden jump in the number of series is the classic warning of a cardinality problem.
  • Target Allocator balance: check that targets are spread fairly and that none are left unassigned.
  • Duplicate data: look for the same series arriving from two sources, which suggests double scraping.
  • Collector resources: memory and CPU, especially when scraping many targets.

Common symptoms and likely causes

SymptomLikely CauseWhat to Try
Metric names differ from the old dashboardsName translation adds suffixes or changes charactersChoose one translation approach and update dashboards or add recording rules
Counters look wrong or flatDelta values sent to Prometheus without conversionConvert delta to cumulative in the Collector, or send cumulative from the SDK
Doubled numbersThe same target is scraped twiceUse the Target Allocator or split scrape jobs clearly
Memory keeps growing in PrometheusHigh-cardinality labels or too many promoted attributesRemove unbounded labels and promote fewer resource attributes
Gaps in graphsFailed scrapes, dropped pushes or stale valuesCheck target health, exporter failures and scrape intervals
Cannot find resource details in queriesThey are in target_info and not on each seriesJoin with target_info or promote selected attributes

14Design Patterns and Anti-Patterns

Habits worth copying and traps worth avoiding.

Good design patterns

Pattern

Collector in the Middle

Put a Collector between applications and storage, so filtering, naming and routing live in one place.

Pattern

Strangler Migration

Keep old Prometheus endpoints, scrape them with the Collector, and move services to OpenTelemetry SDKs one at a time.

Pattern

Agent plus Gateway

Light Collectors near the workloads scrape and batch, and a central gateway handles remote write to storage.

Pattern

Sharded Scraping

Spread scrape targets across Collector replicas with the Target Allocator, to scale horizontally.

Pattern

Label Budget

Agree a short approved list of labels per metric, and review any new label for cardinality before it ships.

Pattern

Naming Standard

Choose one name translation strategy and unit conventions for the whole company.

Anti-patterns

ANTI-PATTERN 01Avoid
The Mistake

Using user IDs, order IDs or request IDs as metric attributes that become Prometheus labels.

Why It Hurts

Each value creates a separate series. Memory, disk and query time explode, and Prometheus may fail.

Better Approach

Keep labels to small, known sets. Put detailed identifiers on traces or logs, and use exemplars to link them.

ANTI-PATTERN 02Avoid
The Mistake

Sending delta temporality straight to Prometheus without conversion.

Why It Hurts

Prometheus expects running totals. Rates, increases and graphs will be wrong or confusing.

Better Approach

Send cumulative values, or convert delta to cumulative in the Collector before export.

ANTI-PATTERN 03Avoid
The Mistake

Collecting the same metrics twice, once with a Prometheus server and once with a Collector, into the same storage.

Why It Hurts

You pay double for storage and see duplicated or conflicting data.

Better Approach

Decide who owns each target. If you migrate, run both only for a short comparison period.

ANTI-PATTERN 04Avoid
The Mistake

Promoting every resource attribute into a label.

Why It Hurts

The number of series multiplies, even though most of these attributes rarely change or are not used in queries.

Better Approach

Promote a short list, such as environment, cluster and region, and leave the rest in target_info.

ANTI-PATTERN 05Avoid
The Mistake

Expecting Prometheus to replace tracing and logging because OpenTelemetry data goes into it.

Why It Hurts

Prometheus handles metrics only. Traces and logs sent there are lost or unusable.

Better Approach

Route each signal to a suitable backend through separate Collector pipelines, and connect them with trace IDs and exemplars.

15Best Practices and Common Mistakes

A practical checklist for a healthy setup.

Best practices

  • Understand the translation. Learn how names, types, labels and resources are converted, before building dashboards.
  • Prefer cumulative temporality for Prometheus. Convert delta values where needed.
  • Keep label values bounded. Review every new label for how many distinct values it can take.
  • Set a clear service name and instance ID. They become the job and instance labels that every query relies on.
  • Use the Target Allocator when scaling scraping. Avoid duplicates and gaps.
  • Use batching, queues and retries on remote write. Protect data during short outages.
  • Secure every endpoint. Encrypt, authenticate and restrict networks for scraping and ingestion.
  • Monitor the pipeline. Watch scrape health, send failures, queue sizes and series growth.
  • Migrate gradually. Run old and new paths side by side and compare before cutting over.
  • Check version notes. Native OTLP ingestion, name handling and histograms have been changing, so confirm behaviour for your versions.

Common mistakes beginners make

  • Thinking OpenTelemetry replaces Prometheus, or the reverse. They do different jobs and often cooperate.
  • Expecting metric names to look the same on both sides.
  • Forgetting that Prometheus handles only metrics, not traces or logs.
  • Ignoring temporality and getting strange counters.
  • Scraping the same targets from several places.
  • Letting high-cardinality attributes become labels.
  • Leaving metrics endpoints open to the public internet.

16Real-World and Industry Examples

How the two projects appear together in tools and platforms you may already know.

Kubernetes Platforms

Kubernetes itself and many add-ons expose metrics in the Prometheus format. Platform teams use the Collector’s Prometheus receiver, often with the Target Allocator, to gather these cluster metrics alongside application metrics from OpenTelemetry SDKs, all in one pipeline.

Amazon Web Services

The AWS Distro for OpenTelemetry includes a remote write exporter that can send metrics to Amazon Managed Service for Prometheus with the required AWS authentication. This lets teams use OpenTelemetry for collection and still query with PromQL in a managed service.

Grafana Stack

Grafana’s metrics storage products, such as Mimir, and Grafana Cloud accept Prometheus remote write and OTLP. Teams often send OpenTelemetry data this way and view it in Grafana dashboards, with exemplars linking to their trace storage.

Google Cloud

Google Cloud Managed Service for Prometheus lets teams keep Prometheus-style collection and queries without running the storage. Teams can also feed OpenTelemetry data into Google’s monitoring services, giving them a choice of paths depending on their existing tools.

Thanos and Cortex Users

Organisations that already run long-term storage for Prometheus data, such as with Thanos or Cortex, add a Collector with a remote write exporter in front. They keep their existing storage and dashboards and gain a vendor-neutral way to ingest new services.

A typical journey for a growing company

1

Start With Prometheus

A single server scrapes services and infrastructure, with Grafana dashboards and alert rules.

2

Add a Collector

Introduce the Collector with the Prometheus receiver, so existing endpoints flow through it unchanged.

3

Instrument New Services With OpenTelemetry

New services send OTLP, with traces and logs joining the same Collector.

4

Scale the Storage

Move to remote write and long-term, scalable storage as volume and retention needs grow.

5

Standardise and Govern

Agree on naming, label budgets and temporality, and monitor the whole pipeline.

17Frequently Asked Questions

Quick answers to questions beginners ask most.

Q1What is Prometheus in OpenTelemetry, in one sentence?
Prometheus is a popular metrics storage and query system, and OpenTelemetry connects to it through the Collector’s receiver and exporters, remote write, and, in newer versions, direct OTLP ingestion.
Q2Does OpenTelemetry replace Prometheus?
Not exactly. OpenTelemetry standardises how telemetry is produced and moved. Prometheus stores and queries metrics. Many teams use both together.
Q3Can the Collector scrape Prometheus endpoints?
Yes. The Prometheus receiver does this, using scrape settings similar to Prometheus’s own, so existing exporters keep working.
Q4How do I send OpenTelemetry metrics to Prometheus?
Common ways are the Collector’s Prometheus exporter, which Prometheus scrapes, the remote write exporter, which pushes to compatible storage, or direct OTLP push where your Prometheus version supports it.
Q5Why do my metric names look different in Prometheus?
Translation usually changes dots to underscores and adds unit and total suffixes. Newer versions can keep more characters, depending on the strategy you choose.
Q6What is the difference between delta and cumulative temporality?
Cumulative values are running totals since the start. Delta values are only the change since the last report. Prometheus expects cumulative, so delta must be converted.
Q7What happens to resource attributes like the service name?
The service name and namespace become the job label, and the instance ID becomes the instance label. Other resource attributes go into a target_info metric, and some can be promoted to labels.
Q8Can Prometheus store traces or logs from OpenTelemetry?
No. Prometheus handles metrics only. Use separate backends for traces and logs, and connect them with trace IDs and exemplars.
Q9What is the Target Allocator?
It is part of the OpenTelemetry Operator for Kubernetes. It discovers scrape targets and shares them fairly among Collector replicas so each target is scraped once.
Q10What is remote write?
It is a protocol created in the Prometheus world for sending metrics to other storage systems. The Collector’s remote write exporter uses it to push data to long-term or managed storage.
Q11Why is Prometheus using so much memory after I added OpenTelemetry data?
Most often because of high-cardinality attributes or too many promoted resource attributes creating huge numbers of series. Remove unbounded labels and promote fewer attributes.
Q12Is native OTLP ingestion in Prometheus ready for everything?
It is useful, but it has been evolving between versions. Check the documentation for your version, plan capacity for push volume, and test name translation before depending on it.

18Summary and Key Takeaways

If you remember only a few things, remember these.

Key Takeaways

  • They are partners, not rivals. OpenTelemetry standardises how telemetry is produced and moved. Prometheus stores and queries metrics. Together they cover collection and storage.
  • Prometheus handles metrics only. Traces and logs need other backends, linked to metrics through trace IDs and exemplars.
  • There are several bridges. The Collector’s Prometheus receiver scrapes existing endpoints, the Prometheus exporter and remote write exporter send data out, and newer Prometheus versions can accept OTLP directly.
  • Pull and push both have a place. Pull suits long-running services, and push suits short-lived jobs and remote workloads. You can mix them.
  • Translation matters. Names, types, labels and resource information are converted, and delta values must become cumulative before Prometheus can use them correctly.
  • Guard cardinality. Keep labels bounded and promote only a short list of resource attributes, or Prometheus will struggle.
  • Operate it like a product. Secure endpoints, use the Target Allocator for scale, monitor scrape health and send failures, migrate gradually, and check version notes because this area keeps evolving.