Before we can appreciate a standard, we need to feel the pain it removes.

Years ago, most programs were built as one large block of code running on one machine. When something went wrong, a developer opened one log file and read it from top to bottom. Today, many programs are built as microservices, which means the application is split into many small programs that talk to each other over the network. This design makes it easier for many teams to work at the same time and lets each part grow on its own. The price is that a single user action now touches many programs, each writing its own separate logs.

Think about what happens when a customer presses “Place order”. The request may reach an edge gateway, then an order service, then an inventory service, then a payment service, then a notification service. If the customer waits eight seconds, the total time is the sum of many small waits spread over many machines. Looking at each service’s logs alone will not tell you which wait was the long one, because each service sees only its own slice of the story.

The solution is called distributed tracing. A trace is the complete story of one request as it moves through the system. It is made of smaller pieces called spans. A span describes one unit of work, such as “the payment service charged the card”, and records when it started, how long it took, and whether it succeeded. Spans are connected in a parent and child tree, so you can see that the checkout span contained the payment span, which contained the database span.

Consider how much harder life is without a trace. An engineer sees a spike of slow requests on a dashboard, and then must guess. Perhaps the database is slow. Perhaps one service is overloaded. Perhaps a recent release changed behavior. Each guess means opening several tools, comparing timestamps by eye, and asking colleagues from other teams. With a trace, the same engineer clicks on one slow request and sees a picture: the request spent two seconds in the gateway, forty milliseconds in the order service, and seven seconds waiting for a card network. The guess becomes a fact in a minute.

For this to work, every service must learn two things when a request arrives: which trace this request belongs to, and which span is its parent. That information is called context. Passing it from one service to the next is called context propagation. And the question this whole article answers is: in what exact format should that context travel, so that a service written in Go, a gateway made by a cloud vendor, and a library written in Java can all understand each other without any private arrangement? That format is W3C Trace Context.

A short glossary keeps the rest of the article smooth.

A few more words will come up later. A propagator is the piece of OpenTelemetry that knows how to write context into a carrier and read it back out. Inject means writing the context into an outgoing carrier. Extract means reading the context out of an incoming carrier. A sampler is a small decision maker that chooses which traces are worth keeping, since keeping every trace is usually too expensive. A vendor is a company that sells a monitoring or tracing product. And a W3C, short for World Wide Web Consortium, is the international group that publishes the standards that keep the web working across different companies.

One point is worth fixing in your mind now. The trace ID is like the number on a parcel’s label and stays the same from start to finish. The span ID is like the name of the current worker handling the parcel and changes at every hop. Much of what follows is just this simple idea, applied carefully.

A short history, and a short definition.

Before the standard existed, every tracing product invented its own way to pass context. One popular open-source tracer used a set of several headers with names starting with “X-B3”. Another used a single header called “uber-trace-id”. A large cloud provider used a header called “X-Amzn-Trace-Id”. Each header had its own layout, its own number sizes, and its own rules. A request travelling through a service that used one style and then a service that used another would lose its identity at the border, and the trace would break into disconnected fragments.

This was a painful problem for companies that mixed products, and even more painful for the growing number of cloud services that sat in the middle of requests, such as load balancers, message brokers, and serverless platforms. If a cloud service did not understand your tracer’s private header, it would usually drop it, and your trace would end there. The tracing community, together with major monitoring vendors and cloud providers, joined forces under the W3C to write one shared format. The result, called Trace Context, became an official W3C Recommendation in early 2020.

The standard defines exactly two HTTP headers.

• traceparent carries the essential, universal facts: which trace, which parent span, and whether the trace is being recorded. Every compliant system reads and writes it in the same way.
• tracestate carries optional, vendor-specific extras, so that different monitoring products can attach their own private details without breaking each other.

The standard also describes the exact steps a system must follow when it receives these headers: what to do when they look valid, what to do when they look broken, and how to modify them before passing them on. These rules are what make different products interoperate predictably. Without the rules, two systems could both claim to support the format and still disagree about edge cases.

Where OpenTelemetry Comes In

OpenTelemetry, often shortened to OTel, is an open-source project that provides one vendor-neutral way to create traces, metrics, and logs. It supplies instrumentation libraries for most popular programming languages, and a Collector program that receives and forwards telemetry. When OpenTelemetry needs to move trace context between services, it uses W3C Trace Context as its default format. In other words, if you install OpenTelemetry in two services and change no settings, they will already speak W3C Trace Context to each other. This default is a deliberate choice, because a standard that all tools share removes the need for private agreements.

Who Built It and Why It Was Accepted

A standard only matters if people adopt it. Trace Context succeeded because the people who write tracing tools, the people who run large cloud platforms, and the people who build web servers all had the same pain and sat at the same table. Each group had reasons to agree. Tool makers wanted their traces to continue across cloud services. Cloud operators wanted a single header to forward instead of a dozen private ones. Application teams wanted freedom to change tools without rewriting every service. The result was a compact design that every group could implement in a few hours of work.

It also helped that the standard is small. A short specification with clear examples and a public set of test cases is easy to implement correctly. Anyone can check their own implementation against these tests and see where it differs. This kind of shared test suite is a quiet but important reason the format behaves the same in so many different products.

What the Standard Does Not Do

It helps to know the limits. Trace Context does not decide which traces to keep, it only carries the flag. It does not describe what a span looks like or how spans are exported, which is the job of the wider OpenTelemetry specification. It does not carry your business data, which is the job of baggage and span attributes. And it does not guarantee delivery: if a service refuses to forward the headers, the standard cannot force it to. Think of it as a precise language for one specific message, and not a complete tracing system by itself.

Standard Versus Implementation

It helps to keep two layers separate. The W3C standard describes the wire format: what text appears in the headers and what each part means. OpenTelemetry describes the machinery: how code inside your program stores the current context, how it turns that context into header text, and how it turns header text back into context. The first is like the rules of a language. The second is like the person who speaks it. You can speak the same language with many different people, and that is why a Java service using OpenTelemetry can talk to a Python service using OpenTelemetry, or to a vendor’s agent that implements the standard on its own.

One line of text, four small parts. Learn them once and you can read any trace header you meet.

The traceparent header is a single line made of four fields joined by dashes. The four fields are the version, the trace ID, the parent ID, and the trace flags. Each field is written in hexadecimal, a way of writing numbers using the digits zero to nine and the letters a to f. Hexadecimal is popular for identifiers because it packs a lot of information into few characters and is easy for computers to read. The standard requires the letters to be lowercase.

Put together with dashes between the fields, the example values above form one complete header value, 55 characters long. A fixed length is a gift to software, because it can be checked quickly and cheaply. Every service on the path reads exactly the same positions.

Version

The first field tells the reader which layout to expect. Right now there is just one real version, written as zero zero. The field exists so that the format can grow in the future without breaking old software. A value of “ff” is declared invalid on purpose, which gives software a reliable way to reject garbage. If a receiver sees a newer version that it does not know, the standard tells it to be forgiving: read the first four fields in the layout it knows, and ignore anything extra, rather than crashing or dropping the trace.

Trace ID

The trace ID is 16 bytes, which is 128 bits, written as 32 hexadecimal characters. The number of possible values is so large that if two services each pick one at random, the chance of picking the same one is, for all practical purposes, zero. This is why no central office is needed to hand out numbers. Each service can invent a trace ID on its own and trust that it will not collide with anyone else’s. The standard forbids a trace ID made entirely of zeros, since that value is reserved to mean “invalid”.

Parent ID

The parent ID is 8 bytes, written as 16 characters. Despite the name, it holds the span ID of the span that is making the call, which becomes the parent of the next span on the receiving side. The name makes sense from the receiver’s point of view: “this is the ID of my parent”. Like the trace ID, an all-zero value is invalid. A new value appears at every hop, because each hop creates new work with a new span.

Trace Flags

The last field is a byte of eight on-off switches. Only one switch is defined in the first version of the standard: the sampled flag, the lowest bit. When it is on, the caller is saying “I am recording this trace, and I would like you to keep your part too”. When it is off, the caller is saying “I am not keeping this trace”. So the value 01 means sampled and 00 means not sampled. A newer revision of the standard, known as Level 2, adds a second switch that tells readers the trace ID was generated randomly, which helps systems that choose samples based on the trace ID. Check the current status of that revision before you depend on it.

Why These Particular Sizes?

The sizes in the header were chosen with care. A 128-bit trace ID is large enough that random choice is safe at the scale of the whole internet, and it matches what most tracing systems were already moving toward. A 64-bit span ID is enough, because spans only need to be unique inside one trace, which holds a limited number of spans. Fixed sizes also make parsing fast: a program can read characters at known positions without scanning for separators or guessing lengths. When a header passes through thousands of requests per second, that speed matters.

Why Lowercase Hexadecimal?

Insisting on lowercase may seem fussy, but it removes a whole family of bugs. If one system wrote uppercase letters and another compared text exactly, the same trace ID would look like two different IDs, and the trace would split. By allowing only one spelling, the standard makes text comparison, storage, and searching reliable across every tool.

Reading a Header by Eye

With practice, you can read a real header the way a mechanic reads a dashboard light. The leading two characters are almost always the version. The next long run is the trace ID, and it is the string you copy and paste into your tracing tool’s search box. The shorter run after it is the span that called you. The last two characters tell you in a glance whether this request is part of a recorded trace. When support engineers ask you for “the trace ID”, this is the part they mean, and knowing exactly where to find it saves time during an incident.

The second header is optional, but it is the reason the standard works so well in a world of many vendors.

The traceparent header is deliberately small and strict. It carries only what every system needs. But real monitoring products often need to carry a little extra data of their own. One product might need to remember an internal sampling priority. Another might need a private identifier for its own database. If each vendor stuffed this data into custom headers, the old problem of many incompatible formats would return. The tracestate header is the shared place to put such extras.

Tracestate is a list of entries separated by commas. Each entry is a key and a value joined by an equals sign. The key names the vendor or system that owns the entry, and the value is whatever private text that owner wants to keep. The standard also supports keys with a tenant name and a system name joined by an at sign, so that a shared platform can hold entries for many customers without clashes.

Rules That Keep the List Healthy

• Order matters. The most recently updated entry goes at the left end of the list. When a system changes its own entry, it moves that entry to the front.
• Everyone edits only their own entry. A vendor may add, update, or remove its own key. It must pass every other vendor’s entries on unchanged.
• The list has a size limit. There is a maximum of 32 entries. If a list grows too long, systems drop entries from the right end, which are the oldest ones. The standard also asks systems to carry at least 512 characters of tracestate, so that a modest amount of vendor data survives normal hops.
• Keep values short and plain. Values may not contain commas or equals signs or unusual characters, and should be short, since headers travel on every single request.
• Tracestate depends on traceparent. If the traceparent is invalid, the tracestate is ignored.

How OpenTelemetry Uses It

In OpenTelemetry, every span context carries a trace state object that follows the W3C rules. Your code normally does not touch it. It simply rides along: OpenTelemetry reads the incoming entries, keeps them, and writes them out again on outgoing calls. Tools can add their own keys when they need to. OpenTelemetry itself reserves a key for its own use in some sampling features, which carry extra facts that help different parts of a system make consistent sampling decisions. As a user, the important lesson is that you should not strip or rewrite tracestate yourself, because other products on the path may be counting on their entries.

The standard describes text. OpenTelemetry describes the machinery that makes and reads that text.

Inside an OpenTelemetry program, the idea of “current context” is a first-class object. A Context is a small, unchangeable container that holds values such as the active span. When your code starts a span, the new span is stored in a new Context, which becomes the current one for the work that follows. Each language has its own way to keep the current Context, such as thread-local storage, async-aware variables, or an explicit object passed around. The details differ, but the goal is the same: wherever your code is running, it can ask “which span am I inside?”

The span context is the portion of a span that must cross boundaries. It holds exactly the facts the W3C standard defines: the trace ID, the span ID, the trace flags, and the trace state. It also carries one extra fact, a flag called remote, which says whether the span context came from another process. A span context that was extracted from an incoming request is marked remote, so the SDK knows its parent lives elsewhere.

The piece that converts between the span context and header text is the propagator. OpenTelemetry defines a small standard interface for propagators with two actions. Inject takes the current context and writes it into a carrier. Extract takes a carrier and returns a new context that contains the remote span context. Because the interface is small and shared, adding support for another format is just writing another propagator, and swapping between formats is a configuration change instead of a code change. The default propagator in the specification and in most SDKs is a composite of two parts: one for W3C Trace Context and one for W3C Baggage.

flowchart LR
  A["Incoming HTTP Request with traceparent and tracestate"] --> B["Server Instrumentation Library"]
  B --> C["Propagator Extract"]
  C --> D["Context with Remote Span Context"]
  D --> E["Tracer Starts New Server Span as Child"]
  E --> F["Your Business Code"]
  F --> G["Client Instrumentation Library"]
  G --> H["Propagator Inject"]
  H --> I["Outgoing HTTP Request with new traceparent"]
  E --> J["Span Processor and Exporter"]
  J --> K["Collector or Tracing Backend"]
    

Let us walk through the figure. A request arrives carrying the two headers. The server-side instrumentation library, which is code that wraps your web framework, hands the headers to the propagator. The propagator extracts a remote span context and places it into a Context. The tracer then starts a new span for this service’s work, using the remote span context as the parent. Your business code runs inside that span. When it makes an outgoing call, the client-side instrumentation library asks the propagator to inject the current context into the new request’s headers. Meanwhile, the span is finished and sent to the exporter, then on to a Collector or a tracing backend.

Notice that you, the application developer, wrote no code about headers. That is the real gift of using instrumentation libraries: they do the extracting and injecting automatically for popular web frameworks, HTTP clients, database drivers, and messaging libraries. Your code only decides which extra spans are worth creating.

Why Context Is Unchangeable

OpenTelemetry treats a Context as an object that is never edited in place. When you want to add a value, you create a new Context that contains the old values plus the new one. This may sound wasteful, but it prevents a nasty class of bugs. If many tasks shared one editable Context, one task could change the active span underneath another task, and spans would end up attached to the wrong parents. With unchangeable contexts, each task sees exactly the context that was handed to it, no matter what the other tasks do at the same moment.

The Role of the Tracer Provider

Another piece of the machinery is the tracer provider, which creates tracers. A tracer is the object that starts spans. When it starts a span, it checks the current Context for a parent. If a parent exists, the new span copies its trace ID and trace flags and takes it as parent. If no parent exists, the tracer creates a new random trace ID, which makes the new span the root of a new trace. This is the only place where new trace IDs are born, which is why a clean extract failure leads naturally to a clean new trace.

No-Op Behavior Without an SDK

OpenTelemetry splits its code into an API and an SDK. A library author can add tracing calls using only the API. If the application never installs the SDK, those calls do almost nothing, but the span context still flows through them, so the trace headers continue to be passed along. This design means a small library in the middle of a request does not break the trace, even in an application that has chosen not to collect any telemetry.

Choosing Propagators

OpenTelemetry lets you choose propagators through configuration. A common setting is an environment variable that lists the propagators to use, with a default of W3C Trace Context plus W3C Baggage. If you also need to talk to older systems, you can add other formats to the list. The composite propagator then writes all of them on the way out, and tries each one on the way in. We return to this when we discuss mixed fleets.

What exactly happens in the instant before your service calls another service.

When your code is about to make an outgoing HTTP call, the HTTP client instrumentation does several steps in a fixed order. First, it creates a new client span to describe the call itself, such as “GET to the payment service”. This new span becomes a child of whichever span is current. Second, it makes this new client span the current one in the Context. Third, it asks the propagator to inject the current context into the request’s headers.

The propagator reads the span context from the current span and builds the header text. It writes the version, which is zero zero. It writes the trace ID of the current span, which is the same as the trace ID of every span so far. It writes the span ID of the client span just created into the parent ID field. It writes the trace flags, including whether the span is sampled. And if the span context carries trace state entries, it writes the tracestate header too. The request then goes out on the network with its new label.

Why the Parent ID Changes at Every Hop

A beginner often asks why the parent ID changes if the trace ID stays the same. The reason is that the parent ID is how the tree of spans is built. Every span records its own ID and the ID of its parent. When service A calls service B, B must know exactly which span in A made the call, so that the tracing backend can draw B’s work underneath that one. If A kept sending its original ID forever, every downstream span would point to the same single parent, and the picture of the request would be flat and wrong.

What Happens If There Is No Valid Span

If there is no active span, or the span context is invalid, the propagator simply writes nothing. This is intentional. Writing an invalid or empty header would only confuse the receiver. An outgoing call with no headers is treated by the receiver as the start of a brand new trace, which is the correct behavior when no trace existed.

Carriers and Setters

The propagator does not know what a carrier looks like. A carrier might be an HTTP header map, a gRPC metadata object, a Kafka record’s headers, or a plain dictionary. To stay flexible, the propagator is given a small helper, called a setter, which knows how to write one named value into that kind of carrier. When reading, it is given a getter that knows how to read one named value back out. This small trick is why one propagator works for every protocol. Supporting a new protocol means writing a tiny getter and setter, not a new propagator.

The receiving side must be both polite and careful. It reads what is offered and refuses what is broken.

When a request arrives, the server instrumentation reads the headers through a getter and calls the propagator’s extract action. The propagator looks for the traceparent header and follows a strict checklist taken from the W3C rules. The checklist exists so that bad data can never poison your traces.

flowchart TD
  S["Request arrives"] --> Q1{"Is there exactly one traceparent header?"}
  Q1 -->|"No, missing or repeated"| N["Start a brand new trace"]
  Q1 -->|"Yes"| Q2{"Is the version not ff and the format correct?"}
  Q2 -->|"No"| N
  Q2 -->|"Yes"| Q3{"Are trace ID and parent ID not all zeros?"}
  Q3 -->|"No"| N
  Q3 -->|"Yes"| Q4{"Is the version newer than known?"}
  Q4 -->|"Yes"| P1["Read the first four fields and ignore the rest"]
  Q4 -->|"No"| P2["Read all four fields"]
  P1 --> T["Read tracestate if present and valid"]
  P2 --> T
  T --> R["Create remote span context and continue the trace"]
    

Read the figure from top to bottom. First, the receiver checks that exactly one traceparent header is present. If there are none, or several, it cannot tell which to trust, so it starts fresh. Second, it checks the format: the right number of characters, valid hexadecimal, and a version that is not the forbidden “ff”. Third, it checks that the trace ID and the parent ID are not made of all zeros. If any check fails, the whole header is ignored and the receiver begins a new trace with a new random trace ID. Only when all checks pass does the receiver continue the caller’s trace.

This “fail clean” rule is very important. A half-valid header could lead to traces stitched together by mistake, or to spans that point to impossible parents. By discarding anything suspicious, the standard guarantees that every trace in your backend is internally consistent, even when some clients send garbage.

After a Successful Extract

The propagator builds a span context holding the caller’s trace ID, the caller’s span ID, the trace flags, and the trace state, and marks it as remote. It stores this in a new Context. Next, the server instrumentation starts a server span as a child of that remote span context. The server span receives a fresh span ID of its own, but keeps the same trace ID. From this moment on, every span created inside this request, whether by your code or by libraries, will share that trace ID, and will be children or descendants of the server span.

sequenceDiagram
  participant C as Caller Service A
  participant P as Propagator in Service B
  participant T as Tracer in Service B
  participant B as Business Code in Service B
  C->>P: HTTP request with traceparent and tracestate
  P->>P: Validate and extract remote span context
  P->>T: Context containing remote parent
  T->>T: Start server span with new span ID
  T->>B: Run request inside the span
  B-->>T: Work finished
  T-->>C: Response returned and span ended
    

Why Some Spans Look Orphaned

Sometimes a tracing backend shows a span whose parent ID points to a span that is nowhere to be found. This is called an orphan. It usually means the parent span was never exported, perhaps because the parent service was not sampled differently, crashed before exporting, or has not yet delivered its data. Because spans are exported independently and arrive at different times, the backend often needs to wait before assembling a complete tree. A missing parent is therefore not always a bug in propagation. Look at timing and sampling first.

The Role of the Remote Flag

When a span context is extracted, it is marked as remote. This small flag helps tools decide what kind of span to create. A server span whose parent is remote is the entry point of this service’s part of the trace. Backends use this to draw service maps and to calculate how much time was spent inside each service. Without the remote flag, tools would have difficulty telling the difference between work that happens inside a process and work handed over from another process.

Case Sensitivity and Duplicates

HTTP header names are not case sensitive, so “traceparent”, “Traceparent”, and “TRACEPARENT” all mean the same header. The values inside, however, must use lowercase hexadecimal. Some HTTP libraries combine repeated headers into one comma-joined string, which would make a duplicated traceparent look like a single corrupt value. Either way, the receiver restarts the trace, which is the safe outcome.

Let us follow one imaginary request through four services and watch the headers change.

Suppose a customer taps “Buy” in a mobile app. The request passes through an API gateway, then an order service, then a payment service. We will call the customer’s action the root of the trace.

Two things stand out in this story. First, the services never talked to the tracing backend about each other. They never needed a shared database or a central coordinator. Each one simply read and wrote headers, and exported its own spans. The backend did the stitching later using the two kinds of IDs. Second, the trace survived because every hop followed the same small rules. If any service on the path had dropped the header, the tree would have broken at that point, and the downstream spans would have appeared as a separate trace.

The Response Direction

The W3C standard defines request headers. It does not require anything on the response. Some systems add a response header that tells the client which trace ID was used, so that a support team can ask a customer for that number when they report a problem. A separate standard proposal exists for this, but it is not part of the Trace Context recommendation, so you should not assume every system supports it.

Keeping every trace is usually too expensive, so a decision must be made. The sampled flag carries that decision across services.

A busy system may handle many thousands of requests every second. Recording every span of every request would create a flood of data, cost a great deal to store, and slow the application. So most systems sample: they keep only a portion of traces. The key question is how to make sure that when a trace is kept, it is kept in full, from the first service to the last. A half-recorded trace, where some services kept their spans and others threw them away, is nearly useless.

The sampled flag in traceparent is the answer. The first service decides whether to sample, writes the decision into the flag, and every later service follows it. In OpenTelemetry, this behavior is built into a standard sampler called the parent-based sampler. It says: if there is a parent span context, do what the parent decided. Only when there is no parent, which means this service is the start of the trace, does it make its own choice using a root rule such as “keep everything” or “keep one in ten”. The default behavior in OpenTelemetry SDKs follows the parent and keeps traces that have no parent.

Head Sampling and Tail Sampling

The sampled flag supports head sampling, where the decision is made at the very start of the trace, before anyone knows how it will turn out. This is cheap and simple, but it cannot know whether the trace will be slow or contain an error. A different approach is tail sampling, where all spans are collected first and a decision is made after the trace is complete, usually in the Collector. Tail sampling can keep every error and every slow request, but it needs more memory and a design that routes all spans of a trace to the same place. The two approaches can be combined, and in both cases the trace context headers still do the same job: they keep spans tied to one trace ID.

Choosing a Sampling Rate

Picking how many traces to keep is partly arithmetic and partly judgment. If your service handles ten thousand requests per second and you keep one in a hundred, you retain one hundred traces per second, which is usually plenty to see typical behavior. But rare problems, such as an error that affects one request in ten thousand, will appear only occasionally. This is why many teams combine a modest head sampling rate with tail sampling rules that always keep errors and unusually slow requests. The trace headers do the same job in every case: they make sure all the spans that do get kept belong to the right trace.

Sampling and Consistency Across Services

Consistency matters most when a trace crosses many services. Imagine four services in a chain, each deciding independently to keep one request in ten. The chance that all four keep the same request is one in ten thousand, so nearly every kept trace would be missing three of its four parts. Parent-based sampling solves this by making the first decision count for everyone. It is one of the most effective single settings for improving the quality of your traces.

An Important Subtlety

The sampled flag is a request, not an order. The standard says it represents the caller’s decision, but a receiver is allowed to make its own choice, for example to protect itself from too much load. When a service decides differently from its caller, it should record its own decision in the flag it passes onward. That is why a well-behaved system may sometimes show a flag change from one hop to the next, even though the trace ID stays the same.

The standard names HTTP headers, but the idea works anywhere data can carry a few extra labels.

Not every hop is a direct web call. A service often drops a message onto a queue, and another service picks it up seconds or hours later. A user may upload a file that is processed by a background job. In all these cases, the trace should continue. The solution is to treat the message as the carrier. When the producer sends a message, it injects the same two pieces of text, traceparent and tracestate, into the message’s headers or attributes. When the consumer reads the message, it extracts them and continues the trace.

OpenTelemetry’s messaging instrumentation does this for many common systems. Because the propagator only needs a getter and a setter, adding a new messaging system is easy. The messages still carry the same W3C text, so a trace can start in a web request, pass through a queue, and continue inside a background worker, all under one trace ID.

Parent-Child Versus Links

When one message produces one piece of work, making the consumer’s span a child of the producer’s span is natural. But sometimes a worker takes a batch of fifty messages and processes them together. That single batch span cannot sensibly be the child of fifty different parents. For this situation OpenTelemetry offers span links. A link is a pointer from one span to the span context of another, without claiming a parent-child relationship. The batch span links to the producer span context of every message it handles. The trace context from each message is still extracted in the standard way, and then used as a link instead of a parent.

Retries, Timeouts, and Fan-Out

Real systems retry failed calls, give up after timeouts, and call several services at once. Trace context handles all three well, but you should know how it looks. When a client retries a call, each attempt usually gets its own client span with its own span ID, all as children of the same parent. This makes a retry storm visible as a row of sibling spans. When a service fans out to five other services in parallel, each call carries the same trace ID but a different parent ID, and the backend draws them side by side. And when a call times out, the client span ends with an error while the server span, if it finished later, still appears under the same trace, showing exactly how long the server really took.

Context Lost in Threads and Async Code

Inside a single program, the current context can also be lost. If your code hands work to a thread pool, a timer, or an asynchronous callback, the new thread may not know about the current Context, and any outgoing call it makes would start a new unconnected trace. Most OpenTelemetry libraries provide wrappers that carry the Context into such tasks. When you write your own handoff, you may need to capture the context before the handoff and restore it afterwards. This is one of the most common reasons for broken traces within a single service.

Browsers and Mobile Apps

Traces can begin in a web browser or a mobile app. A browser instrumentation library can create a root span for a page action and inject traceparent into the requests it sends. A detail that surprises many teams is that browsers treat extra request headers on calls to a different website as needing special permission. Because traceparent is not on the browser’s short list of always-allowed headers, a call to another origin first triggers a preflight check, and the server must explicitly allow those headers. Browser instrumentation therefore usually propagates only to your own origins unless you list other allowed addresses. If you forget the server-side permission, the browser blocks the call, and that mistake looks like a mysterious network failure.

Few companies can switch every service on the same day. Here is how to live with more than one format.

Before the standard, popular tracers invented their own headers. Some of these are still in use inside older services. The most common ones are the B3 format used by Zipkin, which has a multi-header and a single-header form, the Jaeger format with its “uber-trace-id” header, and the AWS X-Ray header. OpenTelemetry provides propagators for these as well. During a gradual migration, you can configure a composite propagator that writes both the new and the old headers on the way out, and accepts either on the way in. That way, a service that has been upgraded and a service that has not can still share one trace.

One detail deserves attention: size. Some older systems used 64-bit trace IDs, while W3C Trace Context uses 128 bits. When a shorter ID is converted into the longer format, it is padded with zeros on the left. Going the other way, the longer ID may need to be cut, which can lose information. Mixed fleets should therefore plan for 128-bit IDs everywhere as soon as possible.

Planning a Migration

A calm migration to W3C Trace Context follows a simple order. First, take stock of every service and note which header format it reads and writes. Second, upgrade the shared entry points, such as gateways and the most central services, to accept and write both formats. Third, upgrade the remaining services one group at a time, always keeping the composite propagator in place. Fourth, watch the ratio of requests that start new traces, because a rise means a service has stopped understanding a format that its callers still send. Fifth, once every service reads W3C, stop writing the old format and remove the extra propagators. This gradual path avoids a risky all-at-once change and keeps traces intact every day of the migration.

Baggage Is a Different Thing

Many people confuse Trace Context with Baggage. They are separate W3C specifications with separate purposes. Trace Context carries the identity of the trace: who the request is and who its parent is. Baggage carries your own application data, such as a customer tier or an experiment name, as a list of key and value pairs in a header named “baggage”, so that any service on the path can read it. Baggage is not automatically attached to spans. If you want a baggage value to appear on spans, you must copy it onto them yourself or use a processor that does so.

No standard is perfect. Here is an honest look at what it gives and what it costs.

Service Meshes, Gateways, and Proxies

Many production systems place proxies between services. A service mesh, an API gateway, or a load balancer can sit on every call. These components can help or harm tracing. If they forward the two headers, the trace continues. If they also create their own spans, you gain visibility into network time and queuing at each hop. But if they strip or rewrite headers, the trace breaks, and because the break happens outside application code, it is easy to miss. Whenever you add a new proxy layer, send a test request with a known trace ID through it and check that the same ID appears on the other side.

Cost of Carrying the Headers

The two headers add roughly a hundred bytes to a request, and more if tracestate is long. On a single call this is nothing, but a system making millions of calls per minute carries many gigabytes of extra header data per day. This cost is almost always worth paying, but it explains why the standard keeps the header small, caps tracestate at a fixed number of entries, and why teams should resist putting large values in baggage.

The Key Trade-offs

The first trade-off is completeness versus cost. Passing context on every call is cheap, but exporting a span for every call is not, which is why sampling exists. The second is trust versus visibility. Accepting trace headers from outside lets you see a customer’s request as one continuous trace, but it also lets strangers send you arbitrary IDs and flags. The third is standardization versus flexibility. A strict, small standard is predictable for everyone, but vendors who want rich private data must squeeze it into the tracestate or send it by other means.

Headers cross networks and company borders, so treat them as untrusted input and as possible data leaks.

Trace Context headers look harmless, but they deserve a short security review. The first issue is trust at the border. Anyone on the internet can send a traceparent header to your public service. A malicious or buggy client could send a flag that says “sampled” on every request, forcing your system to record far more than you planned and raising your storage bill. A client could also reuse the same trace ID again and again, mixing unrelated requests into one giant trace. For these reasons, many teams configure their public edge, such as an API gateway or load balancer, to ignore incoming trace headers from untrusted callers and to start a fresh trace of their own. If the original caller’s trace ID is still useful, it can be stored as an attribute or a link instead of being adopted as the parent.

The second issue is data leakage in tracestate and baggage. The tracestate header is passed to every service on the path, including third parties. Never put secrets, personal data, or internal-only details into it. The same rule applies even more strongly to baggage. When your service calls an outside vendor, baggage and tracestate go along unless you stop them. Review what is attached to outgoing calls to third parties, and strip or limit it.

The third issue is predictability. The standard asks for trace IDs and span IDs to be generated randomly. This is not only about avoiding clashes. Sequential or guessable IDs could let an observer infer how many requests your system handles or link together requests that should look unrelated. Use the random ID generators in OpenTelemetry’s SDKs rather than inventing your own scheme.

The fourth issue is size and abuse. A client could send huge or malformed headers. Compliant parsers reject headers that break the format rules, and web servers limit total header size. Keep these limits in place, and watch for requests that trigger many validation failures, since they can signal probing.

When traces look wrong, a calm, step-by-step approach finds the cause quickly.

Connecting Logs to Traces

One of the most valuable uses of the trace ID is inside your logs. OpenTelemetry can add the trace ID and span ID to every log line written while a span is active. Then, when you see a worrying trace, you can search all logs for that one ID and see every log line from every service that touched the request. In the other direction, when a log line shows an error, the ID in it leads you straight to the full trace. This link between logs and traces is often the main reason teams adopt tracing in the first place.

Common Symptoms and Causes

A Methodical Debugging Order

Work from the outside in. First, confirm that the very first service really creates or accepts a trace ID. Second, look at the headers on a single outgoing call from that service, using a debug log, a proxy log, or a network inspection tool. You should see a traceparent with the expected trace ID. Third, look at the same request on arrival at the next service. If the header is there, the problem is in the next service’s extraction. If the header is missing, something between the two, perhaps a proxy, a gateway, or a custom client, removed it. Repeat this check hop by hop until you find the break.

Testing Propagation Before Production

Propagation is easy to test, and testing it early saves painful surprises. A simple approach is an automated test that starts two small services, makes a call from the first to the second, and then checks that the exported spans share one trace ID and that the second span’s parent is the first span’s client span. Run this test whenever you upgrade a library, change a proxy, or add a new kind of call, such as a message queue. A five-line check in your build pipeline can prevent weeks of mysterious missing data.

Watching the Tracing Pipeline Itself

Tracing is a system in its own right and deserves monitoring. Keep an eye on the number of spans exported, the number dropped by queues, the ratio of traces that have a single span only, and the number of requests that started a new trace instead of continuing one. A sudden rise in new traces at a mid-tier service often means a recent change broke header passing. Alerting on this ratio catches silent breakage long before an engineer notices missing data.

Habits that keep traces whole, and habits that quietly break them.

Patterns That Work

• Use the default. Keep W3C Trace Context and Baggage as your default propagators unless you have a specific reason to change them.
• Use instrumentation libraries. Let tested libraries handle extracting and injecting for web frameworks, HTTP clients, and messaging clients, instead of writing header code yourself.
• Pass context through every handoff. When work moves to another thread, task, or process, carry the context along deliberately.
• Make gateways and proxies transparent. Configure them to forward the two headers and, where supported, to take part in tracing themselves.
• Decide trust at the edge. Choose clearly whether the public entry point continues or restarts the caller’s trace.
• Log the trace ID. Include trace and span IDs in logs so that logs and traces can be joined.

Anti-Patterns to Avoid

Teaching Trace Context to a Whole Team

Technology alone does not keep traces healthy. People do. A short team habit goes a long way: whenever someone adds a new kind of call, such as a new queue, a new vendor integration, or a new proxy, they paste a sample trace into the pull request showing that the trace continued across the new hop. Over time, this builds a shared instinct. Engineers begin to ask “will the context survive this handoff?” the same way they ask “will this code handle errors?” The standard gives the team a common language for the answer, and the habit keeps the language alive.

It also pays to write down, in one page, how your company handles the public edge, which propagators are enabled, which headers proxies must forward, and who owns the tracing pipeline. New team members then learn the rules on their first day, and incidents are shorter because nobody has to rediscover how the system behaves.

A Short Checklist

• Are the default propagators in place on every service?
• Do proxies, gateways, and service meshes forward traceparent and tracestate?
• Is context carried through thread pools, timers, and async callbacks?
• Do messages carry the headers, and are batch consumers using links?
• Is the public edge configured with a clear trust rule?
• Do logs contain trace and span IDs?
• Is there an alert on the ratio of requests that start new traces?
• Have browser calls been tested with the required cross-origin permissions?

Common situations that show the standard at work. These are illustrative patterns frequently seen in production systems.

Short answers to questions that appear again and again.

If you remember only a few things, remember these.

The W3C Trace Context standard is the shared tracking label for requests in distributed systems. It defines two small HTTP headers, traceparent and tracestate, and the exact rules for reading, writing, and rejecting them. OpenTelemetry implements it as its default propagation format, so that services in different languages, owned by different teams, and watched by different products can all contribute to one story about one request.