The short answer first, then the words that make it clear.

ParentBased sampling is a rule inside OpenTelemetry that decides whether a piece of a trace should be kept or thrown away by looking at what the parent of that piece decided. If the parent was kept, the child is kept. If the parent was thrown away, the child is thrown away. Only when there is no parent at all, because this is the very first step of a request, does the rule fall back on a different rule that you choose yourself.

Let us unpack the vocabulary. OpenTelemetry, often shortened to OTel, is a free, open source framework, hosted by the Cloud Native Computing Foundation, that gives developers one standard way to produce telemetry: the stream of traces, metrics, and logs that describes how software behaves. A trace follows one request through all the services it touches. A trace is built from spans, and a span is one timed step of the work, such as “look up the cart” or “charge the card.” Spans form a family tree: a span that starts another span is its parent, and the new one is its child. The very first span of a trace has no parent and is called the root span.

Sampling means keeping only a portion of the traces instead of all of them. A sampler is the small piece of code that makes the keep-or-drop choice every time a new span is about to start. ParentBased is not a sampler that makes a decision from scratch. It is a wrapper: a sampler that holds other samplers inside it and chooses which one of them to ask, depending on what kind of parent the new span has.

What ParentBased answers

• Who decides? The first service in the chain decides, and everyone downstream follows.
• What if there is no earlier service? Then the “root” rule you configured decides, such as “keep ten percent.”
• What if the parent lives in another service? The decision arrives inside a small header, and ParentBased reads it.
• What if the parent lives in the same program? ParentBased reads the decision straight from memory.
• Can I override a decision? Yes, each of the five situations has its own slot, so you can change the behaviour of any one of them.

What ParentBased is not

It is not a percentage by itself. People often say “we use ParentBased sampling at ten percent,” but the ten percent actually belongs to the rule used at the root. ParentBased is the part that makes everyone else agree with that root. It is also not tail sampling, which decides after a trace has finished and can therefore keep only the slow or broken ones. ParentBased decides at the beginning, before anyone knows how the request will turn out. We will compare the two carefully in a later chapter.

A Closer Look at the Word “Parent”

The word “parent” may sound heavy, but it is simply the span that was running when the new span was created. When a web server receives a request and starts a span called “handle request,” and that code then calls the database, the database call becomes a child of “handle request.” Children can have children of their own, and in this way a whole request becomes a tree. The tree can cross the boundary between programs, because a service that calls another service can pass along the identity of its current span in a message header.

Because every span has at most one parent, the family tree always leads back to a single root. This is why one decision at the root can be enough for the entire tree: the answer can be handed down from parent to child, one step at a time, like a message passed along a line of people.

Recording everything sounds ideal until the bill arrives.

Every span costs a little: a little time to create, a little memory to hold, a little network traffic to send, and a little money to store. For a small program, these costs are invisible. For a busy shop that handles millions of requests a day, with a dozen steps per request, they become very visible. Teams therefore decide to keep only a slice of their traces. The sliced portion is usually enough to find patterns, measure speed, and investigate problems, as long as the slice is chosen carefully.

Here is a simple calculation. Imagine one million requests per day. Each request creates a trace with twelve spans, and each span takes about five hundred bytes when stored. Multiply: 1,000,000 × 12 × 500 bytes = 6,000,000,000 bytes, which is six gigabytes every day, or about 180 gigabytes in a thirty-day month. If the team keeps ten percent of the traces, the daily amount drops to 600 megabytes, and the monthly amount to about 18 gigabytes. The saving is large, and the picture of how the system behaves is still sharp.

The problem that parents solve

Now comes the subtle part. Suppose a request passes through five services, and each service decides on its own, by rolling a ten-sided die, whether to keep its piece. The chance that one service keeps its piece is one in ten. The chance that all five keep their pieces is one in ten, five times in a row: 0.1 × 0.1 × 0.1 × 0.1 × 0.1 = 0.00001, which is a thousandth of one percent. Almost every stored trace would be missing most of its steps. The few spans you did keep would be orphans, pointing at parents that were never saved. Engineers looking at such a trace would see holes where the story should be.

With ParentBased sampling, the decision is made once, at the root, and every service follows it. The chance that all five services keep their pieces is the same as the chance that the root keeps its piece: one in ten. Each stored trace is complete, and each dropped trace is dropped completely. You pay the same ten percent cost, but every trace you pay for is whole.

A short history

What Could Go Wrong Without Any Sampling Rule

It is tempting to think that cost is the only reason to sample. In fact, volume can also harm the very system you are trying to observe. A program that creates, stores, and sends millions of spans competes with the business logic for processor time, memory, and network bandwidth. A tracing pipeline that cannot keep up may start refusing data, and then the spans you lose are chosen by accident, not by design. Deliberate sampling replaces accidental loss with a rule you understand and control.

There is also a human reason. Engineers who open a tracing tool and find hundreds of thousands of near-identical traces cannot see the unusual ones. A well-chosen sample reveals the typical shape of the system, and a small number of carefully chosen extra traces reveal the exceptions. Less data, chosen well, often leads to faster understanding than more data chosen carelessly.

Eight small ideas hold up the whole topic. Learn these and the rest will feel natural.

A closer look at the three possible decisions

The OpenTelemetry standard names three outcomes. Drop means the span is not even recorded: no timing, no attributes, nothing is stored, and the cost is almost zero. Record only means the span is recorded inside the program, so that local processors can look at it, but it is not marked as sampled and is normally not exported. Record and sample means the span is recorded, marked as sampled, and sent to the backend. For everyday use you can picture a simple switch with two positions: kept or dropped.

Non-recording spans still carry identity

A dropped span is not invisible. Even when the decision is “do not record,” the program still creates a very light placeholder span that carries the trace identity and the unsampled flag. This placeholder costs almost nothing, but it allows the identity and the decision to continue to the next service. Without it, a downstream service would think it was the start of a brand-new trace and would make a brand-new decision, and the agreement would be lost.

How These Ideas Fit Together

It helps to see the eight ideas as a single short story. A request arrives, and the library creates a span. Before the span starts, the sampler is asked for a decision. ParentBased looks at the parent, and if there is none, the root sampler answers. The decision becomes the sampled flag, which is stored in the span’s identity. When the program calls another program, context propagation writes that flag into a header. The next program reads the header, treats the caller as a remote parent, and follows the flag. At the end, only spans marked as sampled are exported.

Every idea in the chapter appears somewhere in that story, and each one has a single, small job. This is the real strength of the design: no part has to be clever, and the whole behaves intelligently because the small parts agree with each other.

ParentBased lives alongside three simple relatives. Meeting them first makes ParentBased easy to understand.

OpenTelemetry ships with a small set of samplers that every language implementation provides. Three of them make a decision on their own. The fourth, ParentBased, makes no decision of its own, but chooses among the others. This design is a classic example of building complicated behaviour from simple parts.

AlwaysOn and AlwaysOff

These are the simplest samplers, and they are useful as building blocks. AlwaysOn says “keep it” every time. AlwaysOff says “drop it” every time. On their own they are blunt tools, but they appear inside ParentBased as the defaults for four of its five slots, as we will see in the next chapter.

TraceIdRatioBased

This sampler keeps a fixed fraction, such as one in ten. It does not use a random number generator at the moment of the decision. Instead it looks at the trace identity, a long random number that every trace receives when it is born, and keeps the trace when that number falls inside a chosen portion of the possible range. Because the decision depends only on the trace identity, two services that use the same ratio will reach the same decision for the same trace. That is a nice property, but it only works when every service uses the same ratio, and it breaks down as soon as the ratios differ.

ParentBased

The ParentBased sampler holds several other samplers inside it, like a switchboard. When a new span asks, “should I be kept?”, ParentBased first looks at the span’s parent, works out which of five situations applies, and hands the question to the matching inner sampler. The inner samplers can be any of the simple ones, and you can combine them freely.

Choosing Among the Family

For a tiny project on one laptop, AlwaysOn is perfectly fine, because the volume is small and every trace is precious. For a switched-off environment, AlwaysOff silences tracing without removing any instrumentation. For a single service that starts every trace itself and has no neighbours, a plain ratio sampler is enough. As soon as two or more services share a request, ParentBased wrapped around a ratio sampler becomes the sensible default, because it keeps the whole chain in agreement while still limiting volume.

ParentBased is really five small decisions bundled into one. Each slot handles one situation.

When a new span is about to start, ParentBased asks two questions about its parent. First, does a parent exist at all? Second, if it does, was the parent kept, and did it come from another program or from the same program? The answers lead to exactly one of five slots.

The defaults make perfect sense for most systems, which is why ParentBased works well even when you only configure the root slot. The four parent-aware slots simply copy the parent’s verdict, and only the root slot involves a real choice.

Why separate “remote” from “local”?

At first glance, the difference between a remote and a local parent seems unimportant, since in both cases we just follow. The reason for the split is trust. A local parent was created by your own code, inside your own program, so you can trust it completely. A remote parent arrived over the network, perhaps from a service owned by another team, or even from a stranger’s browser. You might want to treat it with care. For example, you might choose to accept a remote “kept” mark only some of the time, or to ignore remote marks entirely at your public entrance. Having separate slots lets you do this without disturbing the behaviour of the inside of your program.

Reading the table like a story

Trace a request through three services: a gateway, a checkout service, and a payment service. The request arrives at the gateway with no earlier trace, so the gateway uses its root slot, perhaps a ten percent ratio, and decides to keep this one. The gateway calls the checkout service and passes the decision along in a header. Checkout sees a remote parent, sampled, applies AlwaysOn, and keeps its span. Inside checkout, a child span is created for a database call. Its parent is in the same program and was kept, so it uses local parent, sampled, which also keeps. Checkout then calls payment, which again sees remote parent, sampled. All the spans are kept, and the trace is whole.

Run the story again with the opposite result at the start. The gateway’s root slot decides to drop. Every other service sees remote parent, not sampled, applies AlwaysOff, and drops its spans too. The request still flows normally, and the customer notices nothing, but no span is stored. One decision at the start has silently shaped the behaviour of the entire chain.

The sticky note on the request is a real, tiny piece of text. Seeing what it contains makes the whole mechanism feel concrete.

When one service calls another, the caller adds a few extra lines to the request, called headers. Headers are like the writing on the outside of an envelope: they are not the letter itself, but they tell the postal workers how to handle it. OpenTelemetry uses a standard header named traceparent, defined by the World Wide Web Consortium, usually called the W3C. Because it is a public standard, services written in different languages, and tools from different vendors, all understand it in the same way.

The traceparent header contains four parts separated by dashes. The first part is a version number. The second is the trace identity, a long random number that all spans of one trace share. The third is the identity of the parent span, the step that made this call. The fourth is a small set of flags, and the lowest flag is the one we care about: the sampled flag.

When the receiving service starts its own span, its OpenTelemetry library reads this header, creates a record of the parent called a remote span context, and marks it as remote, because it came from outside. ParentBased then looks at the flags. If the last digit says one, it picks the remote parent, sampled slot. If it says zero, it picks the remote parent, not sampled slot.

sequenceDiagram
    participant U as Customer
    participant G as Gateway
    participant C as Checkout
    participant P as Payment
    U->>G: Clicks the buy button, no trace exists yet
    Note over G: Root slot checks the ratio and says keep
    G->>C: Request with traceparent flag 01
    Note over C: Remote parent sampled, so keep
    C->>P: Request with the same traceparent flag 01
    Note over P: Remote parent sampled, so keep
    P-->>C: Reply
    C-->>G: Reply
    G-->>U: Confirmation page

Explaining Figure 1

Read the diagram from top to bottom. A customer clicks the buy button, which reaches the gateway. The gateway sees no earlier trace, so it uses its root slot, which applies the ratio, and the answer is “keep.” The gateway then calls checkout and writes the sampled flag into the header. Checkout reads the flag, follows it, and passes the same flag to payment. Replies flow back along the same path. The decision was made once, written once, and read twice.

The companion header: tracestate

Next to traceparent, the standard defines a second header, tracestate, which lets vendors and tools attach extra notes. Newer sampling work uses it to carry a threshold value, so that services can agree not only on whether a trace is kept, but also on the probability with which it was chosen. This is useful for statistics, because it lets backends scale counts back up correctly. For everyday use you can think of tracestate as an optional margin note beside the main sticker.

What happens when no header arrives

If a request arrives without a traceparent header, the receiving service has no remote parent. It treats the new span as a root and uses its root slot. This happens for requests from browsers, mobile apps, scheduled jobs, and older services that do not yet pass headers. It is not an error, but it explains many surprises in practice: a service you expected to be in the middle of a trace appears as the start of a new one.

Why the Flag Is So Small

The designers of the standard kept the sampled flag to a single bit on purpose. Headers travel with every network call, and a busy system may make billions of calls a day. Anything large in a header would add up to a heavy burden on bandwidth, and anything complicated would be hard for every language and tool to implement the same way. A single bit is easy to write, easy to read, and impossible to misunderstand.

The trade-off is that one bit can only say “kept” or “not kept.” It cannot say why, or with what probability. That is exactly why the optional tracestate header exists: it lets a more advanced system add the extra detail without burdening the simple one.

Every time a span is about to begin, the same short sequence of questions runs. It takes a tiny fraction of a millisecond.

The sampler is called at one precise moment: just before a span is created. At that moment the library hands the sampler a few facts: the context that holds the parent, the trace identity, the name of the span, the kind of span (for example, a server handling a request), some starting attributes, and any links to other traces. ParentBased uses only the first of these, the parent context, to choose the slot. The chosen inner sampler may use the others.

flowchart TD
    A["A new span is about to start"] --> B{"Does the span have a valid parent?"}
    B -- "No" --> R["Root slot: ask the root sampler"]
    B -- "Yes" --> C{"Did the parent come from another program?"}
    C -- "Yes, remote" --> D{"Was the remote parent sampled?"}
    C -- "No, local" --> E{"Was the local parent sampled?"}
    D -- "Yes" --> F["Remote sampled slot"]
    D -- "No" --> G["Remote not sampled slot"]
    E -- "Yes" --> H["Local sampled slot"]
    E -- "No" --> I["Local not sampled slot"]
    R --> Z["Decision: drop, record only, or record and sample"]
    F --> Z
    G --> Z
    H --> Z
    I --> Z

Explaining Figure 2

The diagram shows a funnel with two questions and five exits. The first question asks whether a parent exists. If not, the root slot handles the span. If a parent does exist, the second question asks whether it was remote or local, and the third asks whether it was sampled. Each of the four answers leads to a slot, and each slot passes the question to its inner sampler. All five paths join at the bottom, where the decision is handed back to the library.

What happens after the decision

Why the cost is so small

Dropping early is the whole point of head sampling. When a span is dropped at its birth, the library does not allocate space for attributes, does not measure the details of timing, and never queues the span for export. For a busy service, this means that ninety percent of the tracing work vanishes when the ratio is ten percent. The remaining ten percent pays the full price, and that is the portion you actually wanted.

Most teams set it once, through two environment settings, and never think about it again.

An environment variable is a named setting that lives outside your program, in the place where the program is started, such as a deployment file. OpenTelemetry defines two standard variables for sampling, and every language implementation understands them. This is convenient, because the platform team can set sampling for all services in the same way, without changing any application code.

The first variable, which selects the sampler, is called OTEL_TRACES_SAMPLER. The second, called OTEL_TRACES_SAMPLER_ARG, supplies a number to the sampler, such as the ratio. A ratio of 0.1 means ten percent, 0.5 means half, and 1 means all. If you set neither variable, the standard default is parentbased_always_on: follow any parent, and keep every trace that has no parent. This default is a sensible starting point for a small system, but it keeps everything that starts in your service, which becomes costly as traffic grows.

Choosing the ratio

Pick the ratio from three numbers: how many requests you receive, how large an average trace is, and how much storage and money you are willing to spend. Returning to our example of one million daily requests and 6,000 bytes per trace, a ratio of 0.1 yields about 600 megabytes per day. If your budget allows only 60 megabytes, the ratio must be 0.01, which keeps ten thousand traces per day. Ten thousand traces are usually still plenty to see the shape of ordinary behaviour, though too few to catch rare problems.

Setting the other four slots

The environment variables expose only the common combinations. If you need to give the remote or local slots a different inner sampler, you configure the ParentBased sampler directly in your language’s OpenTelemetry settings, where each of the five slots is a separate option. The slot names are the same everywhere, so what you learn in one language carries over to the others.

Where to set it

• In the deployment template. One shared template means every new service begins with the same sampling choice.
• In the platform’s defaults. The platform team can provide the values, and individual teams override them only with a reason.
• Not scattered in code. Hard-coded ratios are forgotten, and are hard to change during an incident.

A Sample Planning Exercise

Suppose a team receives 4,000,000 requests per day, with 10 spans per trace and 500 bytes per span. Keeping everything gives 4,000,000 × 10 × 500 = 20,000,000,000 bytes, or 20 gigabytes per day. Their storage budget allows about 1 gigabyte per day. The ratio they need is 1 divided by 20, which is 0.05, or five percent. They set the sampler to parentbased_traceidratio and the argument to 0.05, and the plan is complete. They then verify after one day that about 200,000 traces were kept, because five percent of 4,000,000 is 200,000.

The exercise shows how simple the arithmetic is. Three numbers, one division, and two settings. The harder work lies in checking that the plan matches reality, which is the purpose of the monitoring chapter.

ParentBased decides early. Another family decides late. Many teams use both, and they must understand the difference.

Head sampling decides at the beginning of a trace, before anyone knows what will happen. It is cheap and simple, and ParentBased is its most popular form. Its weakness is that it is blind. It cannot know that a particular request will turn out slow or broken. If it drops that trace, the interesting evidence is gone.

Tail sampling decides at the end, after all the spans of a trace have arrived. It can then keep every trace that contained an error, every trace slower than a threshold, and only a small portion of the ordinary ones. It is powerful, but it costs more, because something must collect and hold every span of every trace for a short time while waiting for the trace to finish. That something is usually the OpenTelemetry Collector, a standalone program that receives telemetry, processes it, and forwards it.

flowchart LR
    A["Application with ParentBased head sampling"] -->|"Only kept traces are sent"| B["Collector: load balancing by trace identity"]
    B --> C["Collector: tail sampling policies"]
    C -->|"Errors and slow traces"| D["Backend storage"]
    C -->|"Small portion of ordinary traces"| D
    C -.->|"The rest are dropped"| E["Discarded"]

Explaining Figure 3

The diagram shows a two-stage filter. In the first stage, applications use ParentBased with a ratio to cut the volume sharply. The traces that survive travel to a Collector. A load-balancing step makes sure that all spans with the same trace identity reach the same Collector instance, because a tail sampler can only judge a trace if it sees all of it. In the second stage, rules inspect each complete trace and keep the errors and the slow ones, plus a small share of the ordinary ones. Everything else is discarded.

Choosing between them

Use ParentBased head sampling alone when traffic is moderate, errors are common enough to appear in a sample, and simplicity matters. Add a tail sampler when you must reliably capture rare failures and unusual slowness, and you can afford to run the extra Collector layer. Whichever you choose, keep the rule simple enough that a new engineer can explain it in a minute.

Why All Spans of a Trace Must Meet

A tail sampler works like a judge who must hear the whole case before giving a verdict. If half of the spans of a trace arrive at one Collector and the other half at another, neither Collector can see the whole trace, and each may reach a different verdict. That would recreate the broken-trace problem at a new layer. To avoid this, a load-balancing step sends all spans with the same trace identity to the same Collector instance, usually by using the trace identity as a routing key.

The tail sampler also needs patience. It waits for a short time after the first span arrives, to let the rest of the spans catch up, and only then applies its rules. Setting this wait too short produces decisions on unfinished traces, and setting it too long increases memory use. Choose a wait that slightly exceeds the longest ordinary request in your system.

Sampled traces are a gift for debugging and a trap for counting.

Many tools can compute handy numbers from spans: how many requests arrived, how many failed, how long they took. These are often called RED metrics, for rate, errors, and duration. The trap is simple. If the numbers are computed from the spans that survived sampling, they describe only the survivors. With a ten percent ratio, a service that really handled 1,000,000 requests will appear to have handled about 100,000. Dashboards that show request rate will be ten times too low, and alerts that depend on them will be wrong.

Three safe ways to count

• Use metrics that do not depend on tracing. Many OpenTelemetry instrumentation libraries record request counts and durations as real metrics, which are updated for every request, whether or not the trace is kept.
• Compute span metrics before sampling. If you derive metrics from spans in a Collector, place that step before any sampler, so it sees every span. This requires that applications send all spans to the Collector, which costs more network and compute.
• Scale the counts back up. If every trace was kept with the same known probability, a backend can multiply counts by the inverse of that probability. This works only when the probability is known and uniform, which is why tracestate-based probability information is becoming popular.

Why changing ratios makes it worse

If different services use different ratios, or if the ratio changes during the day, scaling counts back up becomes unreliable. A dashboard that adjusts for ten percent will be wrong when the ratio is changed to five percent during an incident. This is another reason to keep counting separate from sampling: counters should be exact, and traces should be a representative sample.

A Worked Illustration of the Trap

Imagine an alert that fires when failed requests exceed 200 per hour. In reality, the service sees 600 failures in one bad hour, which is well above the line. But the alert is computed from sampled spans at five percent, so it sees only about 30 failures, and the alert stays silent. The incident goes unnoticed for hours. The fix is not to raise the ratio, but to move the alert to a real error counter that sees every request.

The reverse mistake also happens. Teams sometimes try to compensate by multiplying every sampled count by twenty. That works while the ratio is exactly five percent and every service agrees, but it quietly breaks the day someone changes the ratio. Exact counters have no such weakness.

ParentBased trusts the parent. At your public front door, that trust deserves a second thought.

Inside your own network, all services are yours, and believing the parent is perfectly reasonable. At the public edge, where requests come from browsers, mobile apps, and partner systems, the picture changes. Anyone who can send a request can also send a traceparent header, and anyone can write a one into the sampled flag. A service that blindly follows a remote parent will then record and export every such request.

Consider the numbers. Normally your gateway keeps ten percent of 1,000,000 daily requests, which is 100,000 traces. Now suppose an outsider, by mistake or by design, sends 500,000 extra requests with the sampled flag set to one. The gateway obeys the flag, so it keeps all of them: 100,000 + 500,000 = 600,000 traces, six times the planned volume. Your storage bill, your network use, and your Collector load all rise by the same factor, and the cause is invisible unless you look for it.

flowchart TD
    X["Request from the public internet"] --> Q{"Does it carry a sampled flag?"}
    Q -- "Trusted internal caller" --> T["Follow the parent as usual"]
    Q -- "Untrusted outside caller" --> I["Ignore the incoming flag and treat the span as a new root"]
    I --> RS["Root sampler applies your own ratio"]
    T --> K["Normal ParentBased behaviour"]
    RS --> K

Explaining Figure 4

The diagram shows a checkpoint at the public door. A request from a trusted internal caller is handled as usual, and its flag is followed. A request from an untrusted outsider has its flag ignored, and the gateway starts a fresh trace under its own root rule. The result is that only your own ratio decides how many traces are kept, and outsiders cannot raise or lower it.

A balanced approach

You do not need to ignore outside flags everywhere. A reasonable pattern is to ignore them at the single public entrance, where every request is unknown, and follow them everywhere inside, where every caller is yours. Some teams also allow a trusted mobile app or a partner to continue its own trace by recognising an agreed marker, and apply the ratio to the rest.

Protecting Without Breaking Collaboration

Ignoring outside flags has a small cost: a customer or partner who wants to follow one request across both companies cannot have their trace joined to yours automatically. If this matters, agree on a controlled arrangement, such as accepting flags from a short list of known partners, or recording the partner’s trace identity as an ordinary attribute so that engineers can search for it later, even though the two traces stay separate.

Another safe option is a rate limit. The gateway can accept outside flags, but only up to a fixed number of forced traces per minute. Anything above the limit is handled by your own ratio. This keeps cooperation possible and still caps the worst-case cost, which makes it a favourite compromise in larger organizations.

Real companies are never uniform. Old services, new services, and borrowed services all meet in one request.

In theory, every service uses ParentBased, every service passes headers, and every trace is whole. In practice, a company grows over years, and its services differ in age, language, and care. A few common mismatches cause most of the broken traces that engineers complain about.

flowchart LR
    A["Service A: ParentBased, keeps the trace"] -->|"Header passed"| B["Service B: ParentBased, keeps the span"]
    B -->|"Header lost by an old proxy"| C["Service C: sees no parent and starts a new root"]
    C --> D["Service D: follows service C"]
    A -.->|"Result"| R["Two partial traces instead of one whole trace"]

Explaining Figure 5

Services A and B behave perfectly, and the first two spans of the request are kept together. Between B and C, an old proxy drops the header. Service C sees a request without a parent and starts a brand-new trace with its own decision. Service D faithfully follows C. The backend now holds two separate, partial traces for a single customer click, and nothing tells the engineer they belong together. If C’s fresh decision is “drop,” the second half of the story is missing entirely.

Queues and background work

Messages sent through queues need special care. A message placed on a queue by one service may be picked up by another service seconds or hours later. Unless the trace context is stored with the message, the receiving service starts a new root. A common solution is to put the context into the message’s attributes. When that is not possible, or when one message triggers many consumers, a span link can connect the new trace to the old one without making it a child.

Prevention Is Cheaper Than Detection

Most mixed-fleet problems appear during growth, when a new service or a new library version enters the system. A short checklist for every new service prevents the majority of them. Confirm that the service uses the standard propagation format. Confirm that it uses ParentBased. Confirm that every proxy, load balancer, and queue between it and its neighbours copies the trace headers. Confirm that its first test request produces one whole trace.

When these four checks are part of the standard launch process, the broken-trace complaints that engineers once raised every month shrink to a few each year, and each one is usually traceable to a single missing header on a single hop.

Sampling fails quietly. Nothing crashes when a ratio is wrong; the pictures simply become less trustworthy.

If a service starts dropping everything, the application keeps working and customers notice nothing. The only symptom is that traces stop appearing, and someone, some day, cannot find what they need during an incident. Because the failure is silent, you must look for it on purpose. A short routine of checks can keep sampling healthy.

The most useful number: trace completeness

If you track only one number, make it trace completeness: the share of stored traces that have no missing parents. In a healthy ParentBased setup, completeness should be very high, close to one hundred percent. A sudden fall means that somewhere a header is being lost or a service has changed its sampler. Because completeness is a ratio, it stays meaningful even when the sampling rate changes.

Check after every change

Every design has a price. An honest view helps you decide when ParentBased is the right tool.

Common trade-offs in plain language

When ParentBased Is Not Enough

ParentBased is an excellent baseline, but some goals need more. If your business must investigate every failed payment, a fixed ratio will never be enough, because failures are rare and the ratio treats them like everything else. If you operate a large batch system where a few giant traces matter more than millions of small ones, a uniform ratio wastes budget on the unimportant. In both cases, keep ParentBased as the foundation and add rules on top, such as tail sampling for errors or special handling for important endpoints.

Patterns are tried-and-true recipes. Habits are the small routines that keep sampling clean, cheap, and honest.

Habits that pay off

• Use ParentBased everywhere. Make exceptions rare, written down, and reviewed.
• Configure from outside the code. Environment variables keep sampling visible and easy to change.
• Pass headers through every hop. Proxies, queues, and background jobs must copy the trace context.
• Protect the front door. Decide deliberately how to treat flags that arrive from outsiders.
• Keep counting separate. Never base request counts on sampled spans.
• Measure completeness. Treat a fall in complete traces as an alert, not a curiosity.
• Review the ratio each quarter. Traffic and budgets change, and the ratio should follow.
• Document the standard. A one-page guide with a diagram makes the right way the easy way.

A gentle path to adopting ParentBased

Keeping the Standard Alive

Writing a standard is the easy part. Keeping it alive through years of change is harder. Teams reorganize, languages are added, and libraries are upgraded, and every change is a chance for someone to invent a private sampling rule. Give the standard an owner, publish it where engineers actually look, and include a worked example. When the standard changes, update the shared template first, so that new releases pick up the change in the ordinary course of work instead of through a long manual campaign.

Most sampling problems come from the same few mistakes. Knowing them in advance saves a lot of confusion.

Myths worth clearing up

Let us follow an imaginary online shop from a costly, confusing setup to a calm, affordable one, using simple numbers.

The shop, called Bazaar, receives 2,000,000 requests per day. Each request creates a trace with 15 spans of about 400 bytes. Four services take part: a gateway, a checkout service, a payment service, and an inventory service. At first, every service uses AlwaysOn, so everything is kept.

Step 1: count the cost of keeping everything

Multiply the numbers: 2,000,000 requests × 15 spans × 400 bytes = 12,000,000,000 bytes, which is 12 gigabytes per day, or 360 gigabytes in a thirty-day month. The team decides that this is too much and sets a target of about five percent.

Step 2: a first mistake

One team changes only its own service to a five percent ratio and ignores the parent. The gateway still keeps everything. Now the gateway sends the flag 01 to checkout, but checkout drops 95 percent of the time anyway. The result is a flood of orphan fragments: gateway spans without matching checkout spans. The team sees storage fall a little, and traces fill with holes.

Step 3: the fix

The same setting, ParentBased with a five percent root ratio, is placed in the shared template for all four. Only the gateway’s remote slots differ, because only the gateway faces the public. After the change, 100,000 traces per day are kept, each one complete from the gateway to inventory, and the volume is about 600 megabytes per day.

Step 4: the forced-sampling surprise

A partner’s old script sends 1,000,000 extra requests per day, each with the sampled flag set. Had the gateway followed outside flags, the day would hold 100,000 + 1,000,000 = 1,100,000 traces. At 15 spans of 400 bytes, that is 6,600,000,000 bytes, or 6.6 gigabytes, eleven times the plan. Because the gateway ignores outside flags and applies its own five percent, the partner’s traffic adds only 5 percent of 1,000,000, which is 50,000 traces. The bill grows a little, not eleven times.

Step 5: the errors problem

About 0.5 percent of requests fail, which is 10,000 failures per day. With five percent head sampling, only about 500 of those failures are kept. That is enough to see that something is wrong, but not enough to investigate a rare cause. The team adds a Collector with a tail sampler that keeps every trace containing an error. To make it work, the head ratio is raised to twenty-five percent, which is 500,000 traces per day, and the tail sampler thins the ordinary ones back down. After tail sampling, all 10,000 failures are kept plus a small share of ordinary traces, and the final stored volume stays near the original budget.

What the Bazaar Team Learned Afterwards

Looking back, the team noticed that their most valuable decisions were also their simplest. They used one standard setting for every service. They gave the gateway a special rule because it faced the public. They kept counting separate from tracing, so their dashboards stayed exact. And they added tail sampling only after they could name a concrete gap: the shortage of error traces.

They also learned to treat the first day after any change as a checking day. They compared the number of kept traces with the plan, looked at trace completeness, and sent one flagged test request and one unflagged test request through the full path. Over several months, these small habits turned what had once been a source of surprise bills and half-empty traces into a quiet, predictable part of the platform.

Three short stories show how the rule behaves in everyday situations.

ParentBased sampling is a small idea with a large effect: one decision, made once, followed by everyone.

We began with a school trip where every child decided alone whether to take a photo, and ended up with an album full of holes. We saw how a single decision made by the first service and carried in a tiny header keeps traces whole. We looked at the five slots, how the flag travels, how to configure it, how it relates to tail sampling and metrics, where it must be guarded, and how to monitor it. The cure for nearly every sampling problem is the same: restore agreement, protect the front door, count with real metrics, and check completeness regularly.