OpenTelemetry as a Vendor Migration Strategy: Switch Backends Without Starting Over
Changing your observability vendor usually means rewriting instrumentation, rebuilding dashboards and re-training every engineer. OpenTelemetry changes the economics of that move. This guide explains, from zero, how teams use it to escape lock-in, run old and new vendors side by side, and cut over with confidence.
Picture a company that has used the same monitoring vendor for six years. Every service has the vendor’s agent installed. Every dashboard, alert and on-call runbook refers to the vendor’s screens. Then the contract comes up for renewal with a price increase of forty percent, and someone asks the obvious question: “Can we move to a cheaper vendor?” The answer from the engineering team is a long, tired sigh. Moving would take a year, and nobody wants to be the person who loses visibility during that year.
That sigh is what we call vendor lock-in, and it is exactly the problem OpenTelemetry was designed to dissolve. OpenTelemetry does not replace your monitoring vendor. It replaces the private plumbing between your code and that vendor with a shared, open standard. Once the plumbing is standard, the vendor at the other end becomes a choice you can revisit, not a marriage. This article shows how teams turn that idea into a real, step-by-step migration strategy, including the parts that are still hard.
1The Lock-in Problem and Why Migrations Hurt
To understand why OpenTelemetry helps, we first need to understand why switching vendors has always been so painful.
What observability vendors do
Observability is the ability to understand what a software system is doing by looking at the data it produces. That data comes as traces (the journey of one request through many services), metrics (numbers over time, such as requests per second) and logs (timestamped text records). An observability vendor is a company that collects this data, stores it, and gives you screens, searches and alerts on top of it. Well-known examples include Datadog, New Relic, Dynatrace, Splunk, Honeycomb and Grafana Labs, and cloud providers offer their own services too.
The old deal
In the old deal, each vendor gave you its own agent, a piece of software that you installed next to or inside your application. The agent watched your code, packaged what it saw in the vendor’s private format, and sent it to the vendor’s servers. It was convenient: install the agent and, within minutes, data appeared on screens. The price of that convenience was that your application, your servers and your pipeline were all wired to one vendor’s way of doing things.
Why moving is so costly
When a team decides to change vendors, it faces several linked jobs at once. Every service needs its old agent removed and a new one installed, which means code changes, build changes, new deployments and risk of outages. The data from the new vendor looks slightly different, with different names for the same things. Dashboards and alerts built over years do not copy across, because they are written in each vendor’s own query language. People must learn a new tool while still being on call. And during the whole transition, the company needs to keep seeing its systems, so it often pays two vendors at once.
Imagine every appliance in your house has a plug shaped for one electricity company’s sockets. If you want to change electricity company, you must replace every plug and every wall socket. Now imagine the whole world agrees on one plug shape. Changing company means changing a contract, not touching the wiring. OpenTelemetry is the agreed plug shape for telemetry.
Keep one sentence in mind through the whole article: OpenTelemetry makes the sending side portable. It does not magically make the screens portable. Most of the real effort in a migration is on the screens side, and we will be honest about that.
2The Building Blocks in Plain English
Six small pieces make up the OpenTelemetry story. Learn these and the rest of the article reads easily.
OpenTelemetry itself
OpenTelemetry, shortened to OTel, is an open-source project under the Cloud Native Computing Foundation. It was formed in 2019 by merging two earlier projects, OpenTracing and OpenCensus, which were competing ways of doing the same job. It provides a standard way to create, process and export telemetry, in all the popular programming languages, independent of any vendor.
The API and the SDK
OTel splits its language libraries into two layers. The API is the small set of calls your code uses to say “a task started” or “record this number”. The SDK is the machinery behind the API that actually collects, batches and sends the data. This split matters for migration because your application code and the libraries you depend on only touch the API. The choice of where data goes is made in the SDK’s configuration, which you can change without touching business code.
Instrumentation
Instrumentation is the act of making code produce telemetry. Automatic instrumentation attaches to well-known frameworks and libraries (web servers, database clients, message queues) and produces telemetry without code changes, often just by adding an agent or startup flag. Manual instrumentation means you add calls yourself to describe your own business operations, such as “process payment”.
OTLP
OTLP, the OpenTelemetry Protocol, is the standard language used to ship telemetry from one place to another. It defines exactly what a span, a metric point and a log record look like on the wire. Because most major vendors can now accept OTLP directly, one stream of OTLP data can feed many different backends.
The Collector
The OpenTelemetry Collector is a separate program that receives telemetry, optionally changes it, and sends it onward. It has three kinds of parts: receivers that accept data in various formats, processors that filter, enrich, sample or reshape it, and exporters that send it to destinations. A Collector is the single most important tool in a migration, because it can receive from old sources, send to old and new destinations at the same time, and adapt data in between.
Semantic conventions
Semantic conventions are the agreed names for common things. For example, there is one standard name for the HTTP method of a request, one for the database system in use, one for the cloud region. When everyone uses the same names, the same query works on data from any language or tool. They are the dictionary that makes OTel data portable in meaning, not only in format.
| Piece | What it is | Role in a migration |
|---|---|---|
| API | Calls your code makes to describe work | Stays unchanged when the vendor changes |
| SDK | Engine that batches and exports data | Only its destination setting changes |
| Auto-instrumentation | Attaches to common libraries for free telemetry | Replaces vendor agents with little code change |
| OTLP | Standard wire format | Lets many vendors accept the same data |
| Collector | Receive, process, export in one program | The control point for dual-writing and cutover |
| Semantic conventions | Standard names for attributes | Make dashboards and queries portable |
3What Exactly Is Locked In
Lock-in is not one thing. It is a stack of seven layers, and OpenTelemetry loosens some much more than others. Knowing which is which keeps expectations honest.
Instrumentation
Vendor agents and vendor SDK calls embedded in your code. OTel loosens this layer almost completely.
Transport and format
Private wire formats and proprietary forwarders. OTLP and the Collector replace these.
Data naming
Vendor-specific attribute names and tags. Semantic conventions help, but old names must be mapped.
Dashboards and queries
Written in each vendor’s query language. OTel does not help directly, so this layer needs rebuilding.
Alerts, SLOs and runbooks
Built on the vendor’s rules engine and notification routing. Must be re-created and re-tested.
Integrations and features
Special features such as session replay, profilers or AI assistants. Some have no equivalent elsewhere.
People and habits
Muscle memory and tribal knowledge. Only training and time loosen this one.
The honest scorecard
OpenTelemetry fully attacks layers one and two, helps layer three, supports layers four and five only indirectly through consistent data, and does not touch layers six and seven. If you hear “OpenTelemetry makes vendor switching trivial”, the speaker is thinking about layers one and two only. If you hear “OpenTelemetry does not help because dashboards still need rebuilding”, the speaker is ignoring that layers one and two are usually the largest and riskiest part of the work, since they touch every deployed service.
Some vendors offer their own “distribution” of OpenTelemetry with proprietary additions. These are convenient, but if you depend on the proprietary parts, you have rebuilt layer one lock-in with an OpenTelemetry label on it. Prefer the plain upstream components and standard OTLP.
4The Core Idea: Separate What You Collect From Where You Send It
Every OpenTelemetry migration strategy rests on one simple architectural move.
Before: instrumentation welded to a vendor
In the old arrangement, the thing that produces the data and the place that receives it are glued together. A vendor’s agent knows how to watch your code and how to talk to only that vendor. Changing one means changing the other. The data source and the data destination are a single inseparable unit.
After: a clean seam in the middle
With OpenTelemetry, there is a deliberate seam. On one side are the applications, which produce standard OTel telemetry. On the other side are the backends, which consume standard OTLP. In between sits the Collector, which can be reconfigured at any time. To change vendor, you change the Collector’s destination, not the applications. The applications are only touched once, when moving to OTel itself. After that, the destination is a dial you can turn.
Think of a water tap and a water company. Your kitchen tap does not care which company supplies the water. A single pipe junction outside your house decides that. OpenTelemetry installs that junction for telemetry: the applications are the taps, the Collector is the junction, and the vendors are the water companies.
Two moves, not one
A migration therefore has two separate moves, and it helps enormously to keep them separate. Move one is going from vendor-specific instrumentation to OpenTelemetry. This is the heavy lift that touches every service, done once. Move two is going from the old vendor to the new vendor. With OTel in place, this is light: add a destination, compare, switch, remove the old one. Many teams do move one under their existing vendor, so that nothing visible changes and risk stays low. Only then do they do move two.
| Move one: adopt OpenTelemetry | Move two: change vendor | |
|---|---|---|
| Touches applications | Yes, every service | No |
| Risk | Moderate, spread over time | Low, controlled in the Collector |
| Visible to engineers | Should be invisible | New screens appear |
| Reversible | Yes, per service | Yes, by flipping routing |
| Repeatable | Done once | Can be repeated whenever needed |
5Architecture and Components of a Migration-Ready Pipeline
Here is the target shape every phase of the migration steers toward.
flowchart LR
SVC1["Service A with OTel SDK"] -->|"OTLP"| AGENT["Agent Collector on each node"]
SVC2["Service B with OTel auto instrumentation"] -->|"OTLP"| AGENT
LEGACY["Legacy service with old format"] -->|"Zipkin or Prometheus or StatsD"| AGENT
AGENT -->|"OTLP"| GW["Gateway Collector pool"]
GW -->|"Exporter 1"| OLD["Old vendor backend"]
GW -->|"Exporter 2"| NEW["New vendor backend"]
GW -->|"Optional"| STORE["Archive in object storage"]
The components, one by one
The applications carry the OTel SDK or an auto-instrumentation agent and know nothing about vendors. Their only configuration is “send OTLP to the Collector at this address.” The agent Collectors run close to the applications, perhaps as a daemon on each server or a sidecar next to each pod. They accept data quickly, add information about the host or container, and forward it. The gateway Collectors are a shared pool, often behind a load balancer. This is the control room: it filters out sensitive data, samples, converts formats, and fans the data out to one or many destinations.
Notice the legacy service at the left of the diagram. Not everything will be re-instrumented at once, so the Collector accepts older formats too. Collectors ship with receivers for several common open formats, such as Zipkin traces, Prometheus metrics and StatsD metrics, and the community builds include more, with varying maturity. This is how old and new coexist in one pipeline during the long middle phase of a migration.
Many vendors now accept OTLP directly. When you have the choice, use the plain OTLP exporter rather than a vendor-specific exporter. A vendor-specific exporter quietly re-introduces the lock-in you are trying to escape.
Production example: Jaeger rebuilt on the Collector
The Jaeger tracing project, originally created at Uber, retired its own client libraries in favour of OpenTelemetry SDKs, and its newer major version is built on top of the OpenTelemetry Collector’s core. When a flagship open-source tracing backend adopts the Collector as its foundation, it shows how central this component has become for portability.
6How a Collector Pipeline Works Internally
The Collector is the star of the migration, so it is worth understanding how it handles data step by step.
flowchart LR
R1["Receiver OTLP"] --> P1["Processor memory limiter"]
R2["Receiver Zipkin"] --> P1
R3["Receiver Prometheus"] --> P1
P1 --> P2["Processor attributes mapping"]
P2 --> P3["Processor sampling"]
P3 --> P4["Processor batching"]
P4 --> E1["Exporter to old vendor"]
P4 --> E2["Exporter to new vendor"]
Receivers
A receiver opens a door for incoming data. It can listen on a network port for OTLP, or reach out and collect, such as scraping a Prometheus endpoint or reading log files. Each receiver converts what it hears into the Collector’s internal standard form, so everything downstream looks the same regardless of the original format. This conversion step is the quiet hero of migration: old formats and new formats become one thing as soon as they enter.
Processors
Processors work on the data in order, like stations on an assembly line. A memory limiter protects the Collector from overload. An attributes processor can rename, add or delete labels, which is how you map old vendor names to standard names. A sampling processor keeps only a fraction of traces. A batch processor groups items for efficient sending. Order matters: put the memory limiter first and the batcher last.
Exporters and fan-out
An exporter sends data to a destination. Crucially, one pipeline can have several exporters, and each receives its own copy of the data. This is fan-out, and it is the mechanism behind running two vendors side by side. You can also define separate pipelines, for example sending the full stream to the new vendor and a reduced, filtered stream to the old one to control cost.
Connectors
A connector is a part that acts as an exporter at the end of one pipeline and a receiver at the start of another. A useful example is the span-metrics connector, which creates request-rate, error-rate and duration metrics from traces. This matters in migrations because many old vendors computed such “service health” numbers automatically on their side. With a connector, you can produce comparable numbers yourself, independent of any vendor.
A Collector is a sorting office. Letters arrive in different envelope styles at different doors (receivers). Clerks fix addresses, remove junk and bundle letters (processors). Then copies go out on several delivery trucks, one for each recipient (exporters). Adding a new recipient means adding one more truck, not rebuilding the post office.
You can build a slimmed-down custom Collector containing only the components you need, using the project’s builder tool. Smaller builds mean fewer security patches to track and less memory use. Many mature teams do this once their pipeline shape is stable.
7Data Flow and the Lifecycle of a Migration
Now zoom out. A full migration moves through phases, and each phase has a clear goal and a clear exit test.
flowchart TD
P0["Phase 0 Inventory and goals"] --> P1["Phase 1 Put Collectors in front of old agents"]
P1 --> P2["Phase 2 Re-instrument with OpenTelemetry"]
P2 --> P3["Phase 3 Dual write to old and new vendor"]
P3 --> P4["Phase 4 Rebuild dashboards alerts and SLOs"]
P4 --> P5["Phase 5 Cut over and decommission old vendor"]
P5 --> DONE["Vendor is now a swappable choice"]
Inventory and goals
List every service, agent, dashboard and alert. Decide why you are migrating: cost, features, consolidation, or risk reduction.
Collectors in front
Place Collectors between existing agents and the current vendor so you gain a control point without changing any data.
Re-instrument with OTel
Replace vendor agents and SDK calls with OpenTelemetry, service by service, still shipping to the old vendor.
Dual write
Send the same data to both vendors. Compare the numbers, and fix differences in naming and settings.
Rebuild the screens
Re-create dashboards, alerts and service level objectives in the new tool and verify them against the old ones.
Cut over and retire
Move on-call to the new tool, stop sending to the old vendor, and finally end the contract.
The data’s life during dual write
Follow one span through the middle of this journey. Your code finishes handling a request. The OTel SDK creates a span, batches it, and sends it as OTLP to the local agent Collector. The agent adds node details and forwards to the gateway. The gateway scrubs private data, applies sampling, then copies the span to two exporters. One copy goes to the old vendor, where the existing dashboards still work. The other goes to the new vendor, where engineers start building and testing new screens. Both vendors see the same facts, so any disagreement between them points at a configuration difference rather than a real change in the system.
Each phase is a safe stopping point. If priorities change after phase two, you still hold a more portable, more standard system than you started with. That is a rare and valuable property in a large migration.
8Phase 0: Inventory, Goals and a Realistic Scope
The cheapest mistakes in a migration are the ones you catch before touching anything. This phase is mostly spreadsheets and conversations.
Why are you migrating?
Write down the real reason. The answer shapes every later choice. If the reason is cost, you will care about data volume, sampling and how the new vendor charges. If it is features, you will care about which capabilities the new tool offers and whether OTel data can feed them. If it is consolidation, you will care about merging several tools into one. If it is risk reduction, such as avoiding dependence on a single supplier, the real prize may simply be adopting OpenTelemetry and keeping the option to move, with no actual vendor change at all. That last case is common and perfectly valid.
The inventory
List everything the current vendor touches. Collect services and their languages and frameworks, how each is instrumented today, which agents run on which hosts, and which third-party integrations pull data in, such as cloud provider metrics or database monitors. Then list the consumers: dashboards, alerts, reports, service level objectives, on-call routing rules, and any automation or scripts that call the vendor’s API. The consumers list is the one people forget, and it is usually bigger than expected.
| Inventory item | Question to answer | Why it matters |
|---|---|---|
| Services and languages | Does OTel have mature support for each language? | Maturity differs, especially for logs and some auto-instrumentation |
| Current agents | Are they running on hosts, containers, or serverless? | Decides the replacement approach |
| Custom instrumentation | How many calls use the vendor’s own SDK? | This is the bulk of the code effort |
| Dashboards and alerts | How many, and which are truly used? | Rebuilding unused ones is wasted effort |
| API-based automation | What scripts depend on the vendor’s API? | These break silently at cutover |
| Contracts and dates | When does the old contract end? | Sets the real deadline |
Pruning before you migrate
Almost every long-lived observability setup contains dead weight: dashboards nobody opens, alerts that always fire and are always ignored, metrics nobody queries. A migration is the perfect excuse to delete them. Ask the vendor for usage reports, and for each dashboard or alert ask a human who owns it. Anything with no owner and no recent use goes onto a “do not migrate” list. Teams often find that thirty to sixty percent of their old content can be dropped, which shrinks phase four dramatically.
Define success in numbers
Write exit criteria before you begin: for example, “ninety-nine percent of services emitting OTLP”, “new vendor’s request counts within two percent of the old vendor’s for two weeks”, “all production alerts re-created and tested”, “on-call has used only the new tool for one full rotation”. Numeric targets prevent endless arguments about whether the migration is done.
Moving house starts with deciding what you actually own. Nobody wants to pay movers to carry broken furniture across the city. The inventory is your declutter weekend before the truck arrives.
9Choosing the Target: A Vendor Scorecard
OpenTelemetry makes moving easier, but you still want to pick a destination that handles OTel data well. A written scorecard turns sales promises into comparable facts.
Questions about OpenTelemetry support
Ask each candidate vendor whether it accepts OTLP directly, and over which transports (the gRPC and HTTP flavours). Ask which of the three signals, traces, metrics and logs, are supported through OTLP, and whether any signal needs a vendor-specific component instead. Ask how the vendor treats OTel resource attributes and semantic conventions: are they shown as first-class fields, or flattened into generic tags that your queries cannot easily reach? Ask whether features such as service maps, anomaly detection and service level objectives work with plain OTLP input or only with the vendor’s own agent.
Questions about metrics behaviour
Ask which temporality the vendor prefers, whether exponential histograms are accepted, how metric series are counted for billing, and what limits exist on label counts and name lengths. Ask how it handles metric names and units from OTel, since conversions between naming styles can surprise you.
Questions about limits and operations
Ask about maximum request sizes, rate limits, ingestion delay and retry guidance. Ask about data retention options and prices, because long retention can cost more than ingestion. Ask where data is stored, how deletion works, and what happens to your data when you leave. Ask what export options exist, since the day you leave this vendor you will be glad you asked.
Questions about price
Pricing models differ enormously: by data volume, by number of hosts, by number of users, by number of metric series, by number of events, or by combinations. A model that looks cheap for your old data shape may be expensive for OTel’s shape. For example, a vendor that charges per host may punish a Kubernetes estate with many small nodes, while a vendor that charges per metric series may punish a metrics-heavy estate. Run your real numbers, from the Collector’s own counters, through each vendor’s calculator.
| Criterion | What to ask | Score from 1 to 5 |
|---|---|---|
| OTLP ingestion | Direct, all three signals, both transports? | Fill in |
| Semantic conventions | Treated as first-class fields? | Fill in |
| Feature parity | Which features need proprietary agents? | Fill in |
| Metrics behaviour | Temporality, histograms, cardinality rules | Fill in |
| Price model fit | Cost estimated from your real data shape | Fill in |
| Data location and retention | Meets regulations and needs? | Fill in |
| Exit terms | Export format, deletion guarantees, notice periods | Fill in |
| Query and alert tooling | As code, import and export options? | Fill in |
Run a proof of concept with real traffic from one pilot service via your gateway, not a vendor demo with sample data. It exposes surprises about attribute handling and costs within days.
10Phase 1: Put Collectors in Front of the Old World
The safest first step changes nothing visible. You simply add a control point.
The idea
Before re-instrumenting anything, deploy Collectors so that existing data flows through them on its way to the current vendor. Wherever the old agents can export in an open format, such as Zipkin, Prometheus or StatsD, point them at a Collector instead of the vendor. If your old agent can only speak a private format, leave it alone for now. Many vendor agents can already forward to a Collector or accept configuration for it, and some vendors offer OTLP ingestion, which makes the interim hop straightforward.
What you gain immediately
- A single place to apply policy. Scrubbing private data, dropping noisy low-value telemetry and tagging by team all happen in the gateway, not in hundreds of services.
- Cost control. You can see exactly how much data each service sends and trim wasteful sources before the new vendor ever sees them.
- Practice. Your platform team learns to run, scale and monitor Collectors in a low-risk setting.
- A fan-out point. The mechanism needed for dual writing is already in place.
Where to run the Collectors
Most teams adopt the agent-plus-gateway pattern from the previous chapters. On Kubernetes, the agent often runs as a daemon on every node, and the gateway as a regular scalable deployment. On virtual machines, the agent runs as a service on each host. On AWS, teams may run the AWS Distro for OpenTelemetry, which is a supported build of the Collector with AWS-specific components. For serverless functions, there are packaged layers that include a small Collector so the function can export without a network hop to a shared pool.
A Collector introduces a new thing that can fail. Give it the same care as any production service: set memory limits, run several replicas, monitor its queues, and test what happens when it restarts. A badly run Collector can turn a migration project into an outage.
Production example: Splunk and the Smart Agent
Splunk’s observability platform retired its older SignalFx Smart Agent and pointed customers to its distribution of the OpenTelemetry Collector. That change shows a pattern repeating across the industry: vendors themselves are replacing private agents with the open Collector, which makes the Collector the natural landing zone for any migration.
Exit test for phase 1
You can declare phase one finished when all telemetry that can pass through Collectors does, nothing visible has changed for engineers, and your platform team can answer “how much data does each team send?” from the Collector’s own numbers.
11Phase 2: Re-Instrument With OpenTelemetry
This is the heavy phase. It touches every service once, and then, ideally, never again.
Start with automatic instrumentation
For most popular languages, OpenTelemetry offers automatic instrumentation. In Java, a single agent file added to the startup command can produce traces and metrics for common frameworks with no code changes. In Python, Node.js and .NET there are similar approaches that attach at startup. On Kubernetes, the OpenTelemetry Operator can inject this instrumentation into pods through annotations, so developers do not even edit their deployment files by hand. This makes the first pass quick: swap the vendor agent for the OTel one, point it at the Collector, and verify that data arrives.
Then convert custom instrumentation
Wherever the code calls the old vendor’s own SDK to create custom spans, record custom metrics or add business tags, you must translate those calls to the OTel API. This is the labour-intensive part. The translation is usually mechanical: “start a custom operation named checkout” becomes “start a span named checkout”, and “add a tag customer tier” becomes “set an attribute customer tier”. Some teams write a thin internal wrapper library first, so that application teams change one import and the wrapper calls OTel underneath. A wrapper also gives you a place to enforce naming rules.
Service by service, not big bang
Do not flip the whole company in one weekend. Pick a small, low-risk service as a pilot, then a few friendly teams, then expand in waves. For each service, run a short checklist: confirm traces appear with the right service name, confirm key metrics match, confirm logs correlate to traces, and confirm performance overhead is acceptable. Keep the old agent available so a service can roll back within minutes.
| Approach | Effort | Best for |
|---|---|---|
| Automatic instrumentation only | Low | Standard web services using popular frameworks |
| Automatic plus a few manual spans | Medium | Services with important business operations to track |
| Wrapper library for custom calls | Medium, paid once | Large estates with many teams |
| Full manual rewrite | High | Rare, only where no automatic option exists |
The performance question
Any instrumentation adds some cost to your application, in CPU, memory and latency. Measure it. Run a load test with the old agent and again with the OTel agent, and compare. Typical overhead is modest, but it varies by language, by how many libraries are instrumented, and by sampling settings. Tune batching and sampling rather than assuming. In the rare case of a latency-critical path, you can limit instrumentation to selected libraries.
Trace context across the mixed estate
While some services still use old agents and others use OTel, a request may pass through both kinds. For the trace to stay connected end to end, all services must agree on how to pass the trace identity from one to the next. The chapter on keeping traces connected, a little further on, covers exactly that.
While moving to OTel, keep sending to the old vendor. Engineers see familiar screens, risk stays low, and any difference you find can be blamed on the instrumentation change alone, not on a vendor change at the same time.
12Edge Cases: Serverless, Batch Jobs, Legacy and Third-Party Software
Real estates are not tidy fleets of long-running web services. These awkward corners cause the most surprises during phase two.
Serverless functions
A serverless function may live for only a few hundred milliseconds. Telemetry must be flushed before the function ends, or it is lost. Vendors often shipped special layers with their own agents, so replacing them means adopting the OTel equivalent, usually a packaged layer that includes a lightweight Collector or exporter. Expect slightly higher start-up time and test cold starts carefully. Many teams send function telemetry to a nearby regional Collector so the function does not wait on a distant vendor.
Batch and scheduled jobs
Short jobs that run and exit have the same flush problem. Make sure the SDK shuts down cleanly at the end of the job, which gives it time to send the final batch. Jobs that run for hours should send periodic metrics, not only a final summary, so you can see progress and failures while they run.
Legacy applications that cannot be changed
Some systems are old, unowned or too risky to touch. For these, keep whatever monitoring they have and bridge it through a Collector: scrape their Prometheus-style endpoint, read their logs, or accept their Zipkin-format traces. You will not get full OTel richness, but you will still get the data into the common pipeline. Mark these services clearly in the inventory as “bridged, not instrumented”.
Third-party and commercial software
Databases, message brokers, caches and commercial products usually expose their own metrics rather than OTel traces. The Collector has receivers for many of them, and cloud providers publish metrics through their own integrations. List each one, find its replacement path, and verify the metrics you actually use still appear after the switch. These integrations tend to be configured once years ago and forgotten, so they are the most likely source of cutover-day surprises.
Frontend and mobile
Browser and mobile monitoring, often called real-user monitoring, is a special case. OTel has browser and mobile libraries, but their maturity and feature set are behind server-side tracing, and vendors often provide richer proprietary agents for session replay and page performance. Treat this as a separate, later workstream. It is reasonable to leave frontend monitoring on the old vendor longer than the backend, or to split vendors by layer for a time.
Lost final batch
Short-lived code exits before sending. Flush on shutdown.
Cold start slowdown
Extra libraries add time. Measure before rolling out widely.
Orphaned integrations
Old vendor-side integrations stop silently. Inventory them early.
Frontend gaps
Real-user features may lack equivalents. Plan a separate track.
13Keeping Traces Connected Across a Mixed Estate
A trace is only useful if all its pieces link together. During a migration, different services may speak different dialects of “trace identity”.
What context propagation is
When service A calls service B, A must tell B “this call belongs to trace number 123, and the call that triggered it was span number 45.” That information travels in small headers attached to the request. This hand-off is called context propagation. If B does not understand A’s headers, B starts a brand-new trace, and the story breaks into disconnected fragments.
Many dialects
Over the years, different tools invented different header formats. The open standard today is W3C Trace Context, which is the default in OpenTelemetry. Older formats include B3 from Zipkin, the Jaeger header, the AWS X-Ray header, and the private headers of various commercial agents. Some older systems also use shorter 64-bit trace identifiers, while the standard uses 128-bit ones, which can complicate how identifiers are matched across tools.
flowchart LR
A["Service A on old agent"] -->|"Vendor header"| B["Service B on OTel reads both formats"]
B -->|"Vendor header and W3C header"| C["Service C on old agent"]
B -->|"W3C header"| D["Service D on OTel"]
The migration trick: composite propagators
OpenTelemetry lets you configure several propagators at once. A service can read incoming context in any of several formats and write outgoing context in all of them. During the mixed period you configure OTel services to understand both W3C and whatever your old agents use, so traces survive every hop. When the last old agent is gone, you remove the old format and keep only the standard. This single setting prevents the most common source of “our traces are all broken” panic during migrations.
AWS-specific note
If you use AWS X-Ray today, there is an X-Ray propagator and an ID generator in the OTel ecosystem. X-Ray’s trace identifiers embed a timestamp, so OTel can be configured to generate identifiers in that style when you still need to feed X-Ray. AWS recommends OpenTelemetry-based instrumentation through its distribution for new work, so moving from older X-Ray libraries to OTel is a well-trodden path.
Gateways, proxies and message queues sometimes strip unknown headers. If traces break at a particular hop, check that the intermediary forwards every trace header you rely on, not just the standard one.
14Phase 3: Dual Write and Parallel Running
This is the phase that makes migrations safe. For a while, both vendors see everything.
How dual write works
Add a second exporter to the gateway Collector’s pipeline, pointed at the new vendor. Every span, metric and log now goes to both. No application changes are needed. The old vendor continues to power on-call and dashboards. The new vendor quietly fills with identical data, and engineers can begin exploring it without any pressure.
What to compare
Counts
Total spans, metric points and log lines per service per hour. Gaps reveal dropped or filtered data.
Error and request rates
Compare request counts and error percentages for key services. They should match within a small tolerance.
Latency percentiles
Compare median and slow-tail latency. Different calculation methods can shift numbers slightly.
Trace completeness
Check that traces show all expected hops and are not broken into fragments.
Ingestion lag
Measure how long data takes to appear. Alerts depend on it.
Series counts
Count distinct metric series. Surprising growth predicts a large bill.
Expect differences, then explain each one
The two vendors will not match perfectly, and that is fine. Your job is to explain every material difference. Some differences come from sampling: if one vendor keeps different traces than the other, counts will diverge. Some come from calculation methods: vendors compute percentiles from different data structures. Some come from naming: the same fact is stored under different names. Some come from vendor-added data: an old agent may have computed extra numbers that your OTel pipeline does not yet create. Keep a log of differences, each with an owner and an outcome of “explained”, “fixed” or “accepted”.
The cost of running two
Dual write means paying for ingestion at both vendors. Plan for it. A simple approach is to send the new vendor everything from a small set of pilot services first, then widen. Another is to send the new vendor a sampled subset for most services, since you are comparing shapes and rates, not storing every byte. Many vendors will give a trial or migration discount for exactly this reason, so ask before you sign.
| Dual-write strategy | Extra cost | Confidence gained |
|---|---|---|
| All services, all data | Highest | Highest |
| Pilot services, all data | Low | High for pilots, unknown for rest |
| All services, sampled subset | Medium | Good for rates and shapes |
| Rolling waves of services | Medium, spread over time | Builds confidence step by step |
Dual write is like running a new engine alongside the old one on a test bench, connected to the same fuel line. You watch whether the new engine produces the same power and sounds the same before you take the old one out of the car.
15Phase 4: Rebuilding Dashboards, Alerts and SLOs
This is the part OpenTelemetry cannot do for you, and it is where many migrations quietly overrun their schedule.
Why this is hard
Dashboards and alerts are written in the vendor’s own query language and rely on that vendor’s data model. A graph that says “average response time by service, last hour” looks simple, yet it encodes assumptions: which metric name, which aggregation, which filters, which time buckets. None of this transfers automatically. Some vendors provide import tools for open formats like Prometheus-style alert rules or Grafana dashboards, which helps if your old dashboards were built in those, but for most commercial tools you rebuild by hand.
A sensible order
- Start with what pages people. Rebuild and test alerts that wake on-call engineers first, because those carry real risk if missing.
- Then service level objectives. Recreate each target and compare the calculated numbers across vendors for two weeks.
- Then the daily dashboards. Focus on screens that engineers open every day, usually per-service overviews and a few business dashboards.
- Then everything else, or let it lapse. If nobody asks for a dashboard after cutover, it was not important.
Parallel alerting
For critical alerts, run old and new versions side by side for a while. Every time one fires and the other does not, investigate. This is tedious and invaluable. It catches subtle differences, such as how each tool treats missing data, how it handles a service that stops reporting entirely, and what time windows it uses.
Make dashboards portable next time
Take the chance to lower future lock-in. Where possible, define alerts and dashboards as code in files kept in version control, using formats that more than one tool understands. Use standard semantic-convention names in queries so they do not depend on one vendor’s private attribute names. Keep a short list of the “golden signals” for each service (traffic, errors, latency, saturation) in a vendor-neutral document, so the next migration begins from a specification rather than from screenshots.
Generate request, error and duration metrics from traces with a Collector connector, rather than relying on a vendor’s hidden calculation. Then both vendors, and any future one, can chart exactly the same numbers.
Production example: treating alerts as code
Large engineering organisations commonly keep alert rules and dashboards in version-controlled repositories and deploy them through pipelines, the same way they ship software. When they later change vendors, the content of those files is the specification they translate, which turns months of screen-by-screen guesswork into a reviewable change.
16Phase 5: Cutover and Decommissioning
The finish line is less dramatic than people fear, because the earlier phases did the hard work.
Choosing the moment
Cut over when your exit criteria are met, not when the calendar says so. Avoid peak business periods, such as a retail company’s holiday season, and avoid weeks when major releases are planned. Announce the date widely, with a clear explanation of what will change for each team.
A staged cutover
Switch on-call to the new tool
Alert routing now sends pages from the new vendor. Keep the old alerts active in silent mode as a safety net.
Watch one full on-call rotation
Collect feedback, fix gaps, and note any incident where engineers reached for the old tool out of habit.
Stop sending to the old vendor
Remove its exporter from the gateway. This is a configuration change, easily reversed in minutes if needed.
Export what you must keep
Retain historic data required for compliance or trend analysis, either as exports or in cheap archive storage.
Remove leftovers
Uninstall remaining old agents, delete API keys, remove old integrations, and update runbooks and documentation.
End the contract
Only after a quiet period, and with data safely exported, close the account.
Historic data
The new vendor starts with no history. Decide in advance what you need from the past. Incident reviews usually need recent weeks. Capacity planning may need a year of metrics. Compliance may require longer retention for logs. Options include asking the old vendor for exports, keeping a read-only account for a short while, or storing raw OTLP data in object storage from the dual-write period onward, which is cheap and portable.
Search the codebase and your automation for the old vendor’s API keys and endpoints before cutover. Scripts, scheduled jobs and deployment pipelines often call vendor APIs to mark deployments or fetch status, and they fail quietly when the account closes.
17A Worked Scenario: Migrating a Fictional Online Shop
Let us walk through the phases with an imaginary company, so the steps feel concrete. Every detail here is invented for teaching.
The starting point
“CartWave” is an online shop with 120 services written in Java, Python and Node.js, running on Kubernetes on AWS. It uses a commercial vendor’s agent in every container, has 340 dashboards and 210 alerts, and pays 55,000 dollars a month. The renewal arrives with a 35 percent increase. The leadership team asks engineering whether a move is possible within nine months.
Month 1: inventory and pruning
The platform team, five engineers, pulls usage reports. Of the 340 dashboards, only 110 were opened in the last 90 days. Of the 210 alerts, 60 have never paged anyone and 25 fire constantly and are muted. After owner reviews, the team decides to migrate 100 dashboards and 90 alerts. They also learn that eleven scripts call the vendor’s API. They set exit criteria: all services emitting OTLP, parity within three percent for two weeks, all paging alerts rebuilt and tested, one full on-call rotation on the new tool.
Months 2 and 3: Collectors and pilot
They deploy agent Collectors as a daemon on every Kubernetes node and a gateway pool of six replicas. Existing agents are still in place, so they route what they can through the gateway and learn which services send the most data. A noisy health-check span from one framework turns out to be forty percent of all spans, and a gateway filter removes it, cutting volume sharply. They pilot OTel on three low-risk services using the Kubernetes operator’s automatic injection, still sending to the old vendor, and fix two naming mismatches in service names.
Months 3 to 6: re-instrument in rings
Ring one is the pilot, ring two is five friendly teams, ring three covers the main body of 90 services, and ring four is the payment and checkout path, done last with extra care. A small wrapper library replaces the vendor’s custom-span calls in the Java and Node.js services. Composite propagators keep traces connected while old and new coexist. By month six, 95 percent of services emit OTLP, and the vendor agents are removed from them.
Months 5 to 7: dual write and rebuilds
With a shortlist of two candidate vendors, CartWave runs a proof of concept using a sampled quarter of traffic from ring two. The winner is chosen on a scorecard. The gateway then fans out the full stream for pilot services and a sampled stream for the rest. Weekly parity reports show request counts within two percent, but latency percentiles differ by up to six percent on one service because of histogram handling. The team documents the cause and sets the alerting thresholds accordingly. Engineers rebuild the 90 alerts, running old and new in parallel for four weeks. They find three alerts that behaved differently when a service stopped reporting entirely, and fix them.
Months 8 and 9: cutover
On-call moves to the new tool in week one of month eight. The old tool remains in silent mode as a safety net. After one rotation without needing it, the team removes the old exporter from the gateway, exports a year of key metrics to archive storage, updates the eleven scripts, and ends the old contract at renewal.
flowchart LR
M1["Month 1 Inventory and pruning"] --> M3["Months 2 to 3 Collectors and pilot"]
M3 --> M6["Months 3 to 6 Re-instrument in rings"]
M6 --> M7["Months 5 to 7 Dual write and rebuild"]
M7 --> M9["Months 8 to 9 Cutover and retire"]
What the numbers looked like
CartWave’s new vendor cost about 38,000 dollars a month after filtering noise, compared with 74,250 dollars a month at the renewal increase, since 55,000 times 1.35 is 74,250. The saving is 36,250 dollars a month, which is 435,000 dollars a year. Engineering effort was roughly 120 engineer-months. This is an illustrative story, but it shows the typical shape: a large share of the savings comes from noise filtering the Collector made possible, not just from the new vendor’s price.
18Semantic Conventions and Attribute Mapping
Portable data needs portable names. This chapter explains why naming is a quiet but major part of every migration.
The same fact, many names
Suppose a web request used the method GET. One vendor’s agent might store that as http.method, another as request.method, a third as verb. A dashboard filtered on one name finds nothing in data stored under another. Across hundreds of attributes, this naming drift is why switching vendors has traditionally broken queries even when the underlying facts were identical.
What the standard names give you
OpenTelemetry’s semantic conventions publish agreed names for common concepts: web requests, database calls, messaging, cloud resources, containers, hosts and more. When every service uses the same names, a single query works across all languages and all vendors. The project has also been stabilising these over time. For example, the older web request attribute names were renamed in the stable HTTP conventions to forms such as http.request.method and http.response.status_code. Different SDK versions can therefore emit different names for the same idea.
Handling the rename period
To avoid breaking existing dashboards while the ecosystem moves, OpenTelemetry provides an opt-in setting that lets instrumentation emit both the old and new attribute names for a while. You turn on the “emit both” mode, update dashboards to the new names, then switch off the old ones. This is the same dual-run idea as vendor migration, applied to naming.
Mapping old vendor tags to standard names
For data coming from legacy sources, the Collector’s attribute and transform processors can rename, copy and delete attributes in flight. Create a mapping table: old name, standard name, who owns the change, and whether to keep both for a transition period. Apply it at the gateway so applications stay untouched.
| Concept | Typical old names | Standard direction |
|---|---|---|
| Service identity | Application name, app, service tag | One service name resource attribute |
| Deployment environment | env, stage, tier | One deployment environment attribute |
| Version | release, build, app version | One service version attribute |
| HTTP method and status | Varied per agent | Stable HTTP request and response attributes |
| Cloud location | region, zone, dc | Cloud region and availability zone attributes |
| Team ownership | owner, squad, team | A single custom attribute defined by your company |
Resource attributes deserve extra care
Resource attributes describe where telemetry came from: service name, version, environment, host, cluster. Backends use these to group and filter. If service names differ between old agents and OTel (for instance, “checkout-service” versus “CheckoutService”), your dashboards will show two services where there is one. Agree on naming rules and set them consistently in the SDK configuration or through the Operator.
Define a short company-wide attribute standard, maybe a single page, before phase two begins. It is far cheaper to agree on names up front than to rename them across hundreds of services later.
19Metrics: The Trickiest Signal to Move
Traces migrate fairly smoothly. Metrics hide more traps, because vendors disagree about how numbers over time should be represented.
Cumulative versus delta
Imagine counting cars passing a bridge. You can report the running total since the counter started, which is called cumulative temporality. Or you can report how many passed in the last minute, which is called delta. Both describe the same traffic. Prometheus-style systems expect cumulative. Several commercial vendors prefer delta. OTel supports both, and the exporter setting for temporality tells the SDK which to send. If you send the wrong kind to a backend, graphs show nonsense: flat lines, giant spikes, or numbers that reset strangely.
Converting between them
The Collector includes processors that convert between cumulative and delta. This is valuable during dual write when your two vendors have opposite preferences: the pipeline can send cumulative metrics to one and delta to the other. Note that conversion needs the Collector to remember previous values, so it requires consistent routing of each series to one Collector instance.
Histograms
A histogram records how many measurements fell into each range, for example how many responses took under 10 milliseconds, 10 to 50, and so on. OTel supports classic histograms with fixed ranges and newer exponential histograms with automatically adjusting ranges that are more accurate and compact. Not every backend accepts exponential histograms yet, and percentile numbers computed from histograms can differ between tools because each tool interpolates between ranges its own way. Do not be alarmed if the ninety-ninth percentile differs by a few percent between vendors. Do investigate if it differs by a lot.
Cardinality and cost
Cardinality means the number of distinct combinations of label values on a metric. A request counter labelled by service and status code has low cardinality. If someone adds a user identifier as a label, it explodes into millions of series, which can crash systems and generate huge bills. Vendors price metrics differently, so a metric that was cheap at the old vendor can be costly at the new one. During dual write, count series per metric on both sides and prune high-cardinality labels at the gateway.
Infrastructure and cloud metrics
Not all metrics come from your application code. Host, container and cloud service metrics come from other sources: the Collector’s host metrics receivers, Kubernetes receivers, and cloud provider integrations. Check each integration your old vendor provided, and decide whether the Collector, the new vendor’s native integration or a cloud-native pipeline will replace it. Gaps here are a frequent surprise at cutover, because these integrations tend to be configured once and forgotten.
Temporality
Confirm each backend’s preference and set it per exporter.
Histogram support
Verify that the new vendor accepts the histogram type you send.
Series counts
Compare distinct series between vendors and trim risky labels.
Infrastructure integrations
List every cloud and host integration and decide its replacement.
Metrics from custom counters tend to be the last thing people verify and the first thing that looks wrong after cutover. Test them early with real traffic.
20Logs and Traces: Correlation and Maturity
Logs and traces are most powerful together. Migration is a chance to link them properly, and a moment to be realistic about maturity.
Logs in OpenTelemetry
The OTel log signal arrived after traces and metrics, and support varies by language. Two common approaches exist. In the first, applications keep using their existing logging library and the OTel instrumentation attaches trace and span identifiers to each log line, while a Collector reads the log files and ships them. In the second, an OTel logging bridge sends logs through the SDK as OTLP. For migration, the first approach is usually easier because applications do not change their logging code at all. A file-reading receiver in the Collector picks up the log files and forwards them.
Correlation
Once logs carry the trace identifier, an engineer looking at a slow trace can jump straight to the log lines produced during that request, and an engineer reading an error log can jump to the whole trace. Many old vendor agents injected such identifiers in their own format. When moving to OTel, make sure the standard identifiers are injected into logs, and update log parsing rules and saved searches that looked for the old format.
Being honest about signal maturity
Not every language and signal is equally mature. Traces are the most mature across most languages. Metrics are mature in many. Logs are catching up. Before committing, check the status of each signal in each language you use, and plan a fallback where needed, such as keeping a language’s existing log pipeline unchanged during the first stages. The project documents the stability of each signal per language, and this is worth reading for every language in your inventory.
Sampling and trace completeness
If you sample traces, decide where. Head sampling decides when a trace starts, cheaply and simply, but it cannot know whether the trace will turn out to be interesting. Tail sampling waits until the trace is finished and keeps the slow or erroneous ones, which is smarter but needs all pieces of a trace to reach one Collector. During dual write, feed both vendors from the same sampling decision, otherwise their numbers diverge for reasons that have nothing to do with the vendors.
flowchart TD
IN["Incoming spans"] --> S1{"Sampling decision made once"}
S1 -->|"Keep"| FAN["Fan out to both exporters"]
S1 -->|"Drop"| GONE["Dropped for everyone"]
FAN --> OLD["Old vendor receives same traces"]
FAN --> NEW["New vendor receives same traces"]
Production example: Honeycomb and its beelines
Honeycomb, an observability vendor, steered its customers from its own proprietary instrumentation libraries, called beelines, towards OpenTelemetry. A vendor retiring its private libraries in favour of the open standard is a strong sign that instrumentation has become a shared commodity, with vendors competing on what they do with the data instead.
21Cost Modelling for a Migration
Migrations are often justified by savings, yet they have their own price. A rough model keeps everyone honest. The numbers below are invented for illustration, so replace them with yours.
The cost buckets
- Dual-run vendor fees: paying two vendors during the parallel period.
- Engineering time: platform engineers, plus application teams converting custom instrumentation.
- Collector infrastructure: servers and containers running agents and gateways, plus network traffic.
- Rebuilding screens: time spent re-creating dashboards, alerts and SLOs.
- Training and productivity dip: engineers slower during the transition.
- Exit costs: data export fees, early contract termination, retention of historic data.
A worked illustration
Suppose a company pays its current vendor 60,000 dollars a month, and the new vendor quotes 45,000 dollars a month for similar volume. The yearly saving after migration would be 15,000 times 12, which is 180,000 dollars. Now estimate the one-time costs. Suppose dual-running takes two months. If the new vendor receives a sampled quarter of the data during that time, the extra vendor bill is about a quarter of 45,000 dollars per month, which is 11,250 dollars, for two months, 22,500 dollars. Suppose six engineers spend about three months on average at a loaded cost of 15,000 dollars per engineer-month, which is 6 times 3 times 15,000, or 270,000 dollars. Add 20,000 dollars for Collector infrastructure and 30,000 dollars for training and productivity dip. The total one-time cost is about 342,500 dollars.
Dividing the one-time cost by the yearly saving gives about 1.9 years to break even on cost alone. That might still be a fine decision if features, risk and negotiating power are the real motives. It might be a poor decision if savings are the only reason. The point is that the calculation takes ten minutes and prevents a very expensive surprise.
| Item | Illustrative amount | How it is derived |
|---|---|---|
| Yearly saving after migration | 180,000 dollars | (60,000 minus 45,000) times 12 |
| Dual-run bill | 22,500 dollars | A quarter of 45,000 for two months |
| Engineering time | 270,000 dollars | 6 engineers times 3 months times 15,000 |
| Collector infrastructure | 20,000 dollars | Assumed |
| Training and dip | 30,000 dollars | Assumed |
| Total one-time cost | 342,500 dollars | Sum of the above |
| Break-even | about 1.9 years | 342,500 divided by 180,000 |
How OpenTelemetry changes the equation
The engineering line is where OTel helps. After the first migration to OTel, the next vendor change costs far less, because the biggest bucket, touching every service, is already paid. The value of OTel is therefore partly an option: it makes future switching, or credible negotiation, cheap. Teams often find that the ability to say “we can move in a quarter” improves the renewal price with the current vendor, even if they never move.
Reduce data volume before dual-running. Filtering noisy health-check spans, trimming huge attributes and sampling low-value traffic at the gateway can cut the bill at the new vendor, sometimes by enough to change the whole business case.
22Advantages, Disadvantages and Trade-offs
A balanced view helps you decide whether this strategy is right for your situation.
Advantages
- Instrumentation is written once and kept
- Backends become swappable with configuration
- Dual-write makes comparison safe and measurable
- One central place for filtering, privacy and sampling
- Stronger negotiating position with vendors
- Large community, wide language coverage, wide vendor acceptance
Disadvantages
- Up-front migration effort across every service
- Dashboards and alerts still need rebuilding
- Collectors are new infrastructure to operate
- Maturity varies by language and signal
- Vendor-unique features may not be reachable through OTel data
- Fast-moving project means regular upgrades and renamed attributes
Features you may lose
Vendors differentiate with features built on their own data collection: deep code-level profilers, session replay for web users, automatic discovery maps, anomaly detection tuned on their own agents’ data. Some of these work with OTel data, some partially, and some only with the vendor’s own agent. Before committing, list the features your teams truly use and ask the target vendor, in writing, which of them work with plain OTLP input.
Trade-off: vendor distribution versus upstream
| Choice | Benefit | Cost |
|---|---|---|
| Upstream OpenTelemetry components | Maximum portability and community support | You operate and upgrade them yourself |
| Vendor-supported distribution | Support contract, tested defaults, easy setup | Risk of proprietary add-ons creeping in |
| Cloud provider distribution (such as AWS Distro for OpenTelemetry) | Tested with that cloud’s services | Ties some configuration to one cloud |
23Security, Privacy and Governance in a Multi-Destination World
Sending data to two vendors doubles the places sensitive information can leak, so governance matters more during migration, not less.
Scrub once, at the gateway
Telemetry can accidentally contain passwords, tokens, email addresses, card numbers or health information, often inside database statements, URLs or error messages. Apply redaction rules at the gateway Collector so that data is cleaned before it reaches any destination. This is one of the strongest arguments for the gateway design, since one rule protects both vendors and any future ones.
Credentials per destination
Each exporter needs its own credentials for its vendor. Store them in a secret manager, inject them into the Collector at startup, and rotate them on a schedule. Never place vendor keys in application configuration, because that spreads secrets across hundreds of services. In the OTel design, only the gateway holds vendor credentials, and applications hold none, which is both safer and simpler for cutover.
Encryption and access
Use TLS on every hop, and mutual TLS between internal tiers when practical. Limit who can edit Collector configuration, because a single change there can drop or redirect all telemetry. Keep the configuration in version control with peer review, and record changes as a normal change process would.
Data residency and contracts
Check where each vendor stores data and whether that satisfies your regulations. Check the contracts for retention periods, deletion obligations after termination, and who can access data for support. During the dual-run, two vendors hold copies of your telemetry, so confirm the new vendor’s data handling terms before sending real production data. At the end, require written confirmation that the old vendor deleted your data after the retention period.
Do not skip redaction because “it is only telemetry”. Telemetry is among the most common accidental leak paths for personal data, and a migration copies it to a second company.
24Monitoring the Migration Itself
A migration is a project with measurable progress. Treat it with the same discipline as a production system.
Progress metrics
Percent of services on OTel
Track weekly, by team. It shows pace and highlights teams needing help.
Difference between vendors
Percentage gap in request counts and error rates for key services.
Dashboards and alerts rebuilt
Count rebuilt against the pruned target list, and how many are verified.
Collector health
Queue depth, refused items, memory use and export failures per destination.
A parity report
Produce a weekly parity report during dual write. For each service in scope, list request count at each vendor, error count at each vendor, difference percentage, status (explained, under investigation, accepted) and owner. A shared report keeps the project honest and gives leaders an easy view of risk without reading dashboards.
Keep an eye on the pipeline
The Collector publishes its own metrics: how many items it received, how many each exporter sent, how many failed, and how full the queues are. Alert on exporters failing for either vendor. A silent failure in the new vendor’s exporter during dual write would otherwise look like missing data on the new side and waste days of investigation.
Send the Collector’s own health metrics to a destination that is independent of the migration, so you can still see problems if one of the two vendors stops accepting data.
25Running the Collector Fleet Over Time
After the migration the Collectors remain, and they are production infrastructure. A few habits keep them healthy for years.
Version and upgrade policy
The OpenTelemetry project releases new Collector versions frequently, and components move between stability levels. Pick a regular cadence, such as monthly or quarterly, to upgrade. Read the release notes for breaking changes, especially renamed components, changed defaults and renamed attributes. Test each upgrade on a staging Collector fed with replayed or sampled production traffic before rolling to production.
Configuration as code
Keep every Collector configuration in version control. Require review for changes. Generate environment-specific files from shared templates so that staging and production differ only where they must. Because a single typo in the gateway can silently drop all telemetry, add automated checks: validate the configuration file with the Collector’s own validation command, and run a smoke test that sends a test span and confirms it arrives.
Canary the config
Roll configuration changes to one replica first, watch the refusal, failure and queue metrics for a few minutes, and only then continue. For fleets of agents, a remote management protocol exists in the OpenTelemetry ecosystem, called OpAMP, which lets a central server push configuration and report health for many Collectors. Adoption is growing and maturity varies, so evaluate it for large fleets and treat it as optional for small ones.
Capacity and resilience
Run at least three gateway replicas across availability zones, with memory limits and a memory limiter processor set below the hard container limit. Watch queue use and refused items to decide when to scale. If you perform tail sampling, keep that tier separate and route by trace identifier, because every span of a trace must reach the same instance.
Keep the exit door open
Once a year, rehearse a vendor switch on paper or in staging: add a second exporter to a test pipeline, send a sample of data and check that the backend ingests it correctly. This keeps the portability you paid for real, instead of letting it decay as people forget how the pipeline works.
Write a one-page runbook for the gateway: how to scale it, how to roll back a configuration, and who to call. When the pipeline is down, nobody can see anything else, so this page earns its keep.
26Design Patterns and Anti-Patterns
Teams that have done this before keep repeating the same good ideas and the same mistakes.
Pattern: the gateway as the single control point
Applications talk only to Collectors. All vendor knowledge, including credentials, destinations, sampling and redaction, lives in the gateway. To change vendor, change the gateway.
Pattern: standard first, vendor extras second
Use semantic conventions and plain OTLP as the baseline for every service. Where a vendor feature needs something extra, add it as an optional enrichment in the gateway, not in application code, so removing it later is a one-file change.
Pattern: ring-based rollout
Move services in rings: a pilot ring of two or three low-risk services, then friendly teams, then the main body, then the most critical systems last. Learn from each ring before starting the next.
The mistake
Changing instrumentation and vendor on the same day for the same service.
Why it hurts
If numbers look different afterwards, you cannot tell whether the new instrumentation or the new vendor is responsible.
The better way
Change one thing at a time: adopt OTel while still sending to the old vendor, then change destination later.
The mistake
Using a vendor-specific SDK exporter inside applications instead of sending OTLP to a Collector.
Why it hurts
Vendor knowledge is again embedded in every service, and fan-out to two vendors needs code changes.
The better way
Applications export OTLP to a local Collector, and only the gateway knows about vendors.
The mistake
Treating the dashboard rebuild as an afterthought and scheduling cutover before it is verified.
Why it hurts
On-call engineers meet missing or misleading alerts during their first real incident on the new platform.
The better way
Rebuild and parallel-test paging alerts first, and make their verification an explicit gate for cutover.
The mistake
Adopting a vendor’s proprietary extensions to OpenTelemetry for convenience and then depending on them.
Why it hurts
The lock-in you removed returns under a friendlier name, and the next migration is as hard as this one.
The better way
Keep a short, reviewed list of any proprietary dependencies and an exit plan for each.
27Myths and Misconceptions
Half-remembered claims about OpenTelemetry travel far. Here are the most common, corrected.
The claim
Adopting OpenTelemetry means you can switch vendors in a day.
The reality
You can switch the destination of the data in a day. Dashboards, alerts, training and vendor-specific features still take months to move.
The claim
OpenTelemetry is a monitoring tool.
The reality
It is a set of standards and software for producing and moving telemetry. It has no storage or user interface of its own and always needs a backend.
The claim
“OpenTelemetry compatible” means the vendor works equally well with OTel data.
The reality
Compatibility varies widely. A vendor may accept OTLP but treat the data as second-class, or require proprietary parts for its best features. Verify with a proof of concept.
The claim
You must re-instrument everything before getting any value.
The reality
A Collector delivers value from day one, through filtering, cost control and centralised policy, even while most services still use old agents.
The claim
OpenTelemetry will always be cheaper than a vendor agent.
The reality
The software is free, but running Collectors, upgrading them and training people costs money. Savings come from control over volume and freedom to negotiate, not from the licence.
The claim
Open standards never change, so the migration is one and done.
The reality
The project evolves, with renamed attributes and new component versions. A regular upgrade routine is part of owning the pipeline.
28Best Practices and Common Mistakes
A short checklist distilled from the previous chapters.
Do these things
- Separate the two moves. Adopt OpenTelemetry first while still sending to the current vendor. Change vendor second.
- Put a Collector in the middle early. It is your control point for filtering, sampling, redaction and fan-out.
- Prefer plain OTLP exporters over vendor-specific ones wherever the target accepts them.
- Prune before you migrate. Delete unused dashboards, alerts and noisy telemetry first.
- Agree attribute names early, using semantic conventions as the foundation.
- Configure composite propagators so traces stay connected across mixed old and new services.
- Make one sampling decision before fan-out so both vendors hold comparable data.
- Roll out in rings, with a rollback path for every service.
- Test paging alerts in parallel before any cutover.
- Write down exit criteria in numbers before you start.
- Keep dashboards and alerts as code to make the next migration cheaper.
Common mistakes
| Mistake | What happens | Fix |
|---|---|---|
| Big-bang switch of all services | Outages and impossible debugging | Ring-based rollout |
| Skipping the inventory | Forgotten scripts and integrations break at cutover | Search code and automation for vendor endpoints |
| Ignoring propagation formats | Traces break into fragments | Composite propagators during the mixed period |
| Mixing sampling decisions | Vendors disagree for no real reason | Sample once, before fan-out |
| Letting metric cardinality grow | Surprise bill at new vendor | Count series and prune labels at the gateway |
| Rebuilding every dashboard | Months wasted on unused content | Prune with owner confirmation |
| No Collector monitoring | Silent data loss | Alert on exporter failures and full queues |
| Closing the old account too early | History lost, automation broken | Quiet period and exported archives |
| Skipping redaction | Personal data copied to a second vendor | Redact in the gateway |
29Real-World Patterns and Industry Signals
Few companies publish every detail of a migration, but the broad shapes are visible across the industry.
The merger that created the standard
In 2019, two competing open-source tracing projects, OpenTracing and OpenCensus, merged into OpenTelemetry. Many companies that had adopted one of them faced their own small migration, which is why the project works hard on compatibility bridges and clear upgrade guidance. That history is a useful reminder that standards also evolve, and why a Collector layer that absorbs change is so valuable.
Vendors adopting the open Collector and OTLP
Major commercial vendors such as Datadog, New Relic, Dynatrace, Splunk, Elastic and Honeycomb, and cloud providers including AWS, Google Cloud and Microsoft Azure, now accept OpenTelemetry data in some form, and several have retired private agents or libraries in favour of OTel components. When competitors all accept the same input, the input becomes a neutral meeting point, which is exactly what makes switching feasible.
Open-source backends built on OTel
Jaeger’s newer major version is built on the OpenTelemetry Collector, and Grafana’s open-source stack, including Tempo for traces and Loki for logs, ingests OTLP. Teams moving from a commercial vendor to a self-hosted stack, or the reverse, use the same OTel pipeline for both. This is one of the strongest uses of the strategy: the same instrumentation serves commercial and open-source destinations.
Cloud-native migrations on AWS
Teams moving from older AWS X-Ray libraries or from third-party agents often adopt the AWS Distro for OpenTelemetry, send OTLP to Collectors in each cluster, and then choose destinations: AWS-native services, a commercial vendor or a self-hosted stack. Because the distribution is a supported build of the same open Collector, the pipeline remains portable to other clouds and vendors later.
Mergers and acquisitions
When two companies merge, each often brings a different observability vendor. Teams that already use OpenTelemetry can send both estates through a common gateway, route to whichever backend the merged company chooses, and standardise attribute names, without asking every service team to reinstrument twice.
Vendor support for OTLP and for specific signals changes quickly. Always check the target vendor’s current documentation and ask for written confirmation of which features work with plain OTLP data.
30People, Process and Ownership
Technology is half the job. Migrations fail more often from unclear ownership than from missing features.
A small platform team leads
Form a core team that owns the Collectors, the standards and the migration plan. Their job is not to convert every service themselves but to make conversion easy: write guides, build the wrapper library, run the pilot, maintain the parity report. Application teams then do the small amount of work that only they can do, such as verifying their own services and rebuilding their own dashboards.
Champions in every team
Ask each application team to name a champion, someone who learns the new tooling first, answers neighbours’ questions and reports problems. Champions reduce the load on the platform team and make the change feel like something done with teams, not to them.
Communication rhythm
Publish a simple plan with phases and dates, update it every week or two, and share the parity report openly. Explain in plain language what changes for engineers on cutover day: where to log in, where alerts will come from, who to call. Surprise is the enemy of on-call confidence.
Training
The new vendor’s tool will feel unfamiliar. Run hands-on sessions using real incidents from the past quarter: “here is last month’s outage, find the cause in the new tool.” This is far more effective than slides and also validates that the new tool can answer the questions your teams actually ask.
Roles at a glance
| Role | Responsibility |
|---|---|
| Executive sponsor | Owns the business case, removes blockers, signs contract changes |
| Platform team | Runs Collectors, writes standards, supports teams |
| Team champions | Lead conversion within their team and give feedback |
| On-call engineers | Validate alerts and report gaps during parallel running |
| Security and compliance | Review redaction rules and vendor data handling |
| Finance or procurement | Manage dual-run costs and contract timing |
31A Troubleshooting Playbook
When something looks wrong in the new vendor, work from the symptom to the likely cause.
| Symptom | Likely cause | First check |
|---|---|---|
| Traces broken into fragments | Propagation formats do not match | Propagator settings on both sides of the broken hop |
| Two copies of the same service | Inconsistent service names | Resource attributes in old and new instrumentation |
| Request counts lower at new vendor | Sampling, dropped data, filtering rule | Collector processors and drop counters |
| Metrics look flat or spiky | Wrong temporality for that backend | Exporter temporality preference |
| Percentiles differ markedly | Different histogram handling | Histogram type and bucket choices |
| Logs not linked to traces | Trace identifiers missing from log lines | Log injection setting and parsing rules |
| No data from one service | Protocol or port mismatch, blocked network | Endpoint and protocol settings in the SDK |
| Sudden cost spike | Cardinality explosion or noisy spans | Series count per metric and spans per service |
| Alerts fire in one tool only | Different handling of missing data or windows | Alert query details and evaluation settings |
| Data arrives late | Queues backing up or vendor ingestion lag | Collector queue depth and exporter latency |
A reliable order of investigation
Does the application emit it?
Use a console exporter in the SDK temporarily to confirm the data exists at the source.
Does the agent Collector receive it?
Check receiver counters and, if needed, enable a debug exporter to print samples.
Does the gateway pass it on?
Compare items received against items sent for each exporter, and look for processors that drop data.
Does the vendor accept it?
Look at ingestion errors, quotas and rejected data counts in the vendor’s console.
Is the screen asking the right question?
Only now examine the query, time range and filters. Many apparent data problems are query problems.
Change one thing at a time while investigating. Several simultaneous changes make it impossible to know which one fixed or broke the system.
32Should You Do This? A Decision Guide
OpenTelemetry is a strong strategy, but not every team needs a full migration right now. Use these questions to find your path.
flowchart TD
Q1{"Do you plan to change vendor within two years?"} -->|"Yes"| FULL["Do the full strategy with dual write"]
Q1 -->|"No or unsure"| Q2{"Is lock-in or price a worry?"}
Q2 -->|"Yes"| ADOPT["Adopt OTel and Collectors now, keep current vendor"]
Q2 -->|"No"| Q3{"Are you building new services?"}
Q3 -->|"Yes"| NEW["Use OTel for all new services from day one"]
Q3 -->|"No"| WAIT["Defer and revisit at the next renewal"]
When the strategy fits well
- A renewal is approaching and you want real negotiating leverage.
- You run several vendors or expect a merger and want a common pipeline.
- You are building many new services and want to avoid adding more vendor-specific code.
- Data volume and cost are growing fast and you need central control over filtering and sampling.
When to be cautious
- You depend heavily on a vendor feature that does not work with plain OTLP input.
- Your key languages have immature OTel support for the signals you need.
- You have no platform team capacity to run Collectors reliably.
- The current vendor is satisfactory in cost and function and there is no change on the horizon. Adopting OTel for new services still makes sense, but a big conversion programme may not.
33Frequently Asked Questions
34Summary and Key Takeaways
What to remember
- OpenTelemetry makes the sending side portable. It replaces vendor-specific agents and formats with an open standard, so the backend becomes a choice you can change with configuration.
- Separate two moves. First adopt OTel while keeping your current vendor. Then change vendor. Doing them apart keeps risk low and makes problems easy to trace.
- The Collector is your control point. It accepts old and new formats, applies redaction and sampling once, and fans out to as many vendors as you like.
- Dual write makes the switch safe. Run both vendors in parallel, compare counts, errors and latency, and explain every difference before you cut over.
- The screens are still your work. Dashboards, alerts and SLOs must be rebuilt. Prune first, rebuild paging alerts in parallel, and keep them as code for next time.
- Watch the traps. Connect traces across mixed estates with composite propagators, agree attribute names early, match metric temporality, and control cardinality and cost.
- Be honest about fit. Adopt OTel for new work almost everywhere, and migrate vendors when you have a clear reason, a funded team and a plan with measurable exit criteria.