Tracing only works if every step of a request knows which request it belongs to.

Years ago, most applications were a single program. When something went wrong, you looked at that one program’s logs and found the cause. Today, applications are usually built from many small services that call each other over the network. A single user action can touch ten or twenty services, each with its own logs, its own timing, and its own failures. When a customer says “checkout is slow”, the cause might be in any one of them.

Distributed tracing was invented to solve this. A trace is the complete story of one request, made up of spans, each span being one measured step of work. A good trace shows you the whole journey on one timeline, so you can see at a glance that the payment step took four seconds while everything else was fast. But there is a hidden difficulty. Each service creates its own spans independently, on its own machine, with no shared memory. How does the payment service know that its work belongs to the same story as the work done by the web server a moment earlier?

The answer to all four questions is context propagation. It is the mechanism that carries a small package of identifying information along with the request, from the first service to the last.

The thing that actually travels is a small, fixed package called the span context.

Before we talk about moving context around, we need to know what is inside it. In OpenTelemetry, the most important part of the context that travels between services is the span context. It is a compact set of facts that identifies a particular span and the trace it belongs to. It is small on purpose, because it must be attached to every request that crosses a service boundary.

flowchart TD
  T["Trace: one whole request"] --> A["Span A in service 1"]
  A --> B["Span B in service 2"]
  B --> C["Span C in service 3"]
  A --> D["Span D in service 1"]
        

Why Two Identifiers?

The trace identifier answers “which request?”, and it never changes as the request moves along. The span identifier answers “which step?”, and it changes at every step. When service one calls service two, it sends its own span identifier. Service two creates a new span, gives it a new span identifier, and records service one’s span as its parent. That single link is what lets a tool draw the tree of calls.

Context propagation is the act of passing the context from one place to the next so a request keeps its identity.

Context propagation is the mechanism by which OpenTelemetry carries context, mainly the span context and optionally extra key-value information called baggage, along the path of a request. It makes sure that every piece of work done on behalf of a request can discover which trace it belongs to and which step triggered it.

There are really two separate problems hidden inside this one idea, and it helps to keep them apart. The first is passing context inside a single program, from one function to another, even across threads. The second is passing context between different programs, over the network, from one service to another. OpenTelemetry solves each with a different technique, and both are needed for complete traces.

The Good News: It Is Mostly Automatic

For most applications, you do not write propagation code. Automatic instrumentation libraries for popular web frameworks, HTTP clients, database drivers, and messaging clients already handle it. When your service receives a request, the library reads the incoming context. When your service makes an outgoing call, the library attaches the context. You only need to understand the idea so that you can recognize when something goes wrong, and so that you can handle the unusual places where automation cannot reach.

Before context can cross the network, it has to travel correctly inside a single program.

Inside a single program, many functions run one after another, calling each other. When your code handles a request and starts a span, the program needs a way for deeper functions to discover that span without passing it as an argument through every single call. If every function needed an extra parameter just for tracing, code would become cluttered and fragile.

OpenTelemetry solves this with a context object. You can think of it as a small, read-only notebook that holds a few entries, such as the currently active span. Whenever new information is added, a new notebook is created with the extra entry, while the old one is left untouched. The program keeps track of which notebook is current for the code that is running right now, and any code can ask what the current context is.

How Child Spans Find Their Parents

This is how parent-child relationships form automatically. When your code starts a new span, the library looks at the current context, finds the span that is active, and records it as the new span’s parent. Then the new span becomes active, so any deeper work becomes its child. When the new span ends, the previous span becomes active again. You never name the parent yourself; the context does the bookkeeping.

Crossing the network needs a way to write context into a message and read it back out.

When one service calls another over the network, the context must be written into the message so it can survive the trip, then read out again on the other side. OpenTelemetry describes this with a small set of simple ideas.

sequenceDiagram
  participant A as Service A
  participant N as Network
  participant B as Service B
  A->>A: Start span, make it current
  A->>A: Inject context into outgoing headers
  A->>N: Send request with headers
  N->>B: Deliver request with headers
  B->>B: Extract context from incoming headers
  B->>B: Start child span under the extracted parent
  B-->>A: Send response
        

Step by Step

A shared standard lets services written in different languages and by different vendors understand each other.

For propagation to work across a whole company, or across companies, everyone needs to agree on how the context is written. In the past, each tracing system invented its own header names and formats, which made mixing tools painful. To end this, the World Wide Web Consortium published a standard called W3C Trace Context. It is now the default propagation format in OpenTelemetry and is widely supported across the industry.

The Two Headers

The standard defines two headers. The first, called traceparent, carries the essential information. The second, called tracestate, carries optional extra notes for individual vendors. Most of the time, only the first matters to you.

The four parts are written as a single compact line of text, separated by dashes. Humans do not need to read it, but it is worth knowing that the whole thing is short, usually well under a hundred characters, which keeps the cost of attaching it to every request very small.

The Tracestate Header

The second header is optional. It is a list of named entries that individual tracing vendors can use to carry their own information alongside the standard one. For example, a vendor might record how it made a sampling decision. Services that do not understand a vendor’s entries simply pass them along unchanged. This allows different tools to coexist without breaking each other.

Sometimes you need to send your own information along with the request, not just identifiers.

The span context carries identity: which trace and which step. But sometimes a downstream service needs a business fact that only an earlier service knows. For example, the web server may know the customer’s plan tier from the login, while a service four steps later wants to treat premium customers differently. Passing that fact as an argument through every intermediate service would be clumsy. Baggage solves this.

Baggage is a set of named values that travels along with the request, separate from the span context. Any service along the path can read the baggage and add to it. It is also defined by a W3C standard and travels in its own header, so any compatible tool can carry it.

An Important Point About Baggage

Beginners often assume that baggage values automatically show up as attributes on spans. They do not. Baggage is simply carried along; it is not recorded anywhere unless you take extra steps. If you want a baggage value to be searchable on spans, you must deliberately copy it onto the spans, usually with a small piece of configuration or a processor that reads the baggage and sets matching attributes.

Size and Cost

Baggage is sent with every outgoing request, so the more you put in it, the more every call costs in bandwidth and processing. The standards also limit how many entries and how many bytes baggage may contain, and libraries may drop entries beyond those limits. Keep baggage small and put only genuinely useful values in it, such as short, stable categories.

Older systems use other formats, and OpenTelemetry can speak several at once.

W3C Trace Context is the modern default, but it is not the only format in the world. Before it existed, several tracing systems defined their own headers, and many of those systems are still running. If part of your infrastructure uses an older format, the services must still be able to understand each other, or your traces will break at the boundary.

Composite Propagators

A composite propagator combines several formats at once. When sending, it writes context in all the chosen formats. When receiving, it tries each format in turn until one succeeds. This is extremely helpful during migrations. A service in the middle of an upgrade can accept both the old and the new format and emit both, so neighbors on either side still work. Once everything has moved to the standard, you can remove the extras.

Choosing Which Propagators to Use

Propagators are normally chosen through simple configuration, often a single setting that lists the formats by name, without any code changes. The key rule is consistency: every service that takes part in a trace must understand at least one format that its neighbors emit. A mixture of services each using a different single format, with no overlap, will produce broken traces.

Propagation also makes sure every service agrees on whether a trace is being recorded.

Recording every trace in a busy system is often too expensive, so systems typically record only a fraction. This is called sampling. But consider what would happen if each service made its own random choice. The first service might record its part, the second might skip it, and the third might record again. The resulting trace would be full of holes, and nobody could use it.

Propagation solves this by carrying the decision. The first service in a request, called the root, makes the sampling decision, and records it in the trace flags that travel with the context. Every later service reads that flag. A common setting, called parent-based sampling, tells each service to follow the decision of its parent: if the parent was sampled, record; if not, skip. As a result, a trace is either complete or entirely absent, never half there.

flowchart LR
  A["Service A: makes the decision"] -->|"flag: sampled"| B["Service B: follows the flag"]
  B -->|"flag: sampled"| C["Service C: follows the flag"]
  A2["Service A: decides not to record"] -->|"flag: not sampled"| B2["Service B: skips"]
  B2 -->|"flag: not sampled"| C2["Service C: skips"]
        

Not Sampled Does Not Mean Not Propagated

Even when a trace is not being recorded, the context must still be propagated. The identifiers and the not-sampled flag still need to travel, so downstream services know to skip as well, and so that logs written along the way still carry consistent identifiers. A service that stops propagating when a trace is unsampled would break the chain for everyone after it.

The idea is the same everywhere; only the carrier changes.

Requests do not only travel as web calls. Systems also talk through remote procedure calls, message queues, scheduled jobs, and file-based handoffs. The principle is always the same: write the context into whatever the carrier is on the sending side, and read it back out on the receiving side. What changes is where the context is stored.

Messaging: Parent or Link?

Message queues add an interesting question. When a producer sends a message and a consumer processes it later, should the consumer’s span be a child of the producer’s span? Often yes, for a single message handled right away. But consider a consumer that picks up a batch of fifty messages and processes them together. That one piece of work relates to fifty different traces, and a span can have only one parent. For this case OpenTelemetry offers links: references from a span to other spans that are related but are not the parent. The batch span links back to the context of each message it handled.

Background Jobs and Long Delays

Sometimes a request creates work that will run much later, such as an email scheduled for tomorrow. The context can be saved along with the job, in a field in the database or the queue entry, and restored when the job runs. Whether the later work should be a child in the same trace or a new trace linked to the original is a design choice. Long delays often make a link more sensible than a child, because a trace that spans a whole day is awkward to view.

Even within one program, context can slip away when work moves between threads or callbacks.

We said earlier that the current context is tracked for the code that is running right now. But modern programs often move work around: they hand tasks to a pool of worker threads, schedule callbacks to run later, or use asynchronous styles where a function pauses and resumes. At these boundaries, the program may resume the work in a place where the current context is no longer set, and the active span silently disappears.

What Happens When It Is Lost

If the context is lost, the next span created has no parent to find. It starts a brand new trace, becoming an orphan root. Calls it makes to other services carry the new trace’s identifiers rather than the original ones. On a trace viewer, the original trace appears to stop, and a separate unrelated trace appears, with no visible link between them.

Most broken traces come from a handful of predictable causes.

When a trace shows up in pieces, the culprit is almost always one of a short list of causes. Knowing the list saves hours of confusion. Because a broken link does not produce an error message, you only find out when you look at a trace and see it stop where it should have continued.

Gateways, Proxies, and Meshes

Infrastructure between services deserves special attention. API gateways, reverse proxies, and load balancers sometimes remove unknown headers or rebuild requests from scratch. A service mesh sidecar may create its own spans but still needs the application to forward the incoming headers to its outgoing calls, because the mesh cannot guess which outgoing call belongs to which incoming request. Whenever traces break at an infrastructure boundary, check its header handling first.

Let us follow one request through a whole system and see propagation at every step.

To pull everything together, imagine a customer placing an order on a shopping website. The request passes through a gateway, an order service, an inventory service, a payment service, and finally a message queue for shipping. Let us trace how context moves at each step.

flowchart LR
  U["Customer browser"] --> G["Gateway"]
  G --> O["Order service"]
  O --> I["Inventory service"]
  O --> P["Payment service"]
  O --> Q["Message queue"]
  Q --> S["Shipping worker"]
        

A simple routine finds the broken link in most cases.

When a trace is split in two, you need to find the exact place where the context was lost. The broken link is always between the last span that appears in the first piece and the first span that appears in the second. A methodical routine will find it quickly.

Useful Clues

A few clues help. Many orphan root spans appearing in a service that should only ever be called by others suggests extraction is failing. A very high number of separate short traces, instead of fewer long ones, is a typical symptom of context loss. If one specific route is always broken but others work, the cause is probably on that route’s path, such as a proxy rule or a custom handoff in that code.

Headers that cross boundaries can be forged or can leak information.

Propagation passes information across boundaries, and every boundary raises security questions. The trace identifiers themselves are random and reveal nothing about your users. But the way propagation works creates a few risks that are worth understanding, especially where your system meets the outside world.

Trust Boundaries

The most important idea is the trust boundary. Inside your own network, services trust each other’s context. At the edge, where requests come from the public internet, you should decide deliberately whether to trust incoming headers. A common approach is to ignore incoming trace context at the public edge and start a fresh trace, optionally linking to the incoming identifiers rather than continuing them. The same care applies to outgoing calls to third parties: consider turning off propagation, especially baggage, for calls leaving your organization.

Propagation happens in your applications; the Collector benefits from it.

A common point of confusion is whether the OpenTelemetry Collector takes part in context propagation. The answer is that propagation happens inside your applications and between them, in the headers of real requests. The Collector sits to the side, receiving the finished spans that applications export. It does not forward your customers’ requests, so it does not inject or extract context into them.

But the Collector depends heavily on the results of good propagation. Every span it receives carries a trace identifier and a parent identifier that were set by propagation. If propagation was broken, the Collector will faithfully receive fragments that cannot be joined. And features such as tail sampling rely on grouping all spans with the same trace identifier together so a decision can be made about the whole trace. Good propagation is therefore the foundation on which the Collector’s smarter features stand.

Seeing propagation in familiar situations makes the ideas concrete.

Context propagation is essential for distributed tracing, but it has costs and responsibilities.

Is It Worth the Trouble?

For any system with more than one service, the answer is yes. Without propagation, distributed tracing is impossible, and you are left correlating logs from different services by hand, guessing from timestamps. The overhead of a few short headers is tiny compared with the time saved during incidents. The trade-offs are best handled by good defaults, consistent configuration across services, and a habit of checking that new components forward context properly.

A Simple Adoption Path

You do not need to get everything right on day one. Start by enabling automatic instrumentation in your main services, using the default propagators. Then verify that traces cross your most important boundaries, especially gateways and queues. Next, fix any breaks by checking headers and async handoffs. After that, decide your trust-boundary policy for the public edge and third parties. Finally, add baggage only if you have a clear use for it, with strict rules about what may go inside. Each step delivers value on its own.

A handful of habits prevent most problems.

Habits Worth Building

Mistakes to Avoid

Quick answers to the questions beginners ask most often.

It is the mechanism that carries a request’s identity, mainly the trace and span identifiers and optionally baggage, from one place to the next so that all the work for one request joins into a single trace.

Usually not. Automatic instrumentation for popular frameworks, clients, and messaging libraries already injects and extracts context. You only need manual work for custom transports and custom handoffs between threads or queues.

Span context identifies the trace and the current step and has a fixed format. Baggage is a set of your own named values that travel with the request and are not recorded on spans unless you copy them.

A component that knows how to write context into a carrier, such as request headers, and read it back out, following a specific format such as W3C Trace Context.

W3C Trace Context for trace identity, together with W3C Baggage. Other formats such as B3, Jaeger, and X-Ray are available for compatibility with older systems.

Context was lost at some boundary. Common causes are an uninstrumented service, a proxy that drops headers, mismatched formats, or a lost context at a thread or callback boundary. Check the sender, the path, and the receiver in turn.

No. Propagation happens in applications and between them. The Collector receives exported spans that already carry identifiers, and it depends on good propagation for features such as tail sampling.

No. Baggage is only carried along. To make a baggage value searchable on spans, you must copy it onto them explicitly.

The producer injects context into message headers or properties, and the consumer extracts it. For batches that handle many messages at once, links are used instead of a single parent.

Usually not. Many organizations start a fresh trace at the public edge and link to the incoming context instead of continuing it, and they avoid sending baggage to external partners.

Context propagation is the thread that connects the pieces of a distributed request. Inside a single program, a context object tracks the active span so child spans find their parents. Between programs, a propagator injects the context into a carrier such as request headers and extracts it on the other side, following a shared standard, W3C Trace Context, so that services written anywhere can understand each other. The same mechanism carries the sampling decision so traces are complete or absent, carries optional baggage for business context, and lets logs and metrics link back to traces. Most of the work is automatic, but broken links produce silent fragments, so learn the usual causes: uninstrumented services, headers dropped by infrastructure, mismatched formats, and lost context at asynchronous boundaries. Protect your edge, keep baggage small and harmless, and your traces will tell a complete, trustworthy story.