What Is Span Status in OpenTelemetry?
A plain-English guide to the small field on every span that says whether the step went wrong. You will learn the three status codes, who is allowed to set them, how HTTP and other protocols map onto them, and how a tiny field ends up powering error rates, alerts, and sampling.
Think of a parcel moving through a delivery network. At each stop, a worker scans it and the system records what happened. Most scans say nothing special, and the parcel just moves on. Sometimes a scan says “damaged”, and a person is called. There is also an occasional scan that says “delivered and signed for”, which closes the matter for good.
Every step in a traced request works the same way. A span, which is a record of one timed step, carries a small field called its status. The status answers one simple question about that step: did it go wrong? The answer has only three possible values, and the way those values are chosen decides whether your dashboards tell the truth.
Span status looks tiny, but it quietly drives some of the most important things you do with tracing. Error rates on dashboards usually count spans with an error status. Alerts that wake people at night often trigger on them. Smart sampling rules that keep the failing traces look at them. If the field is set badly, every one of these is wrong in ways that are hard to notice. This guide assumes you know nothing, and every new word is explained when it first appears, with an everyday comparison.
1The Short Answer: One Page Before the Long Story
If you only have two minutes, this chapter gives you the whole idea. The remaining chapters explain why each rule exists and how to apply it.
Span status is a field on every span that says whether the operation the span describes succeeded or failed. It has a code with three possible values: Unset, Ok, and Error. It also has an optional description, a short human-readable message, which is allowed only when the code is Error.
Unset is the default. It means “nobody said anything special”, and in practice it usually means the step finished normally. Error means the step failed. Ok means that someone deliberately confirmed success and wants it left alone. The standard guidance is that instrumentation libraries mark failures as Error and leave successes as Unset, while Ok is kept for application developers who want to state success explicitly.
| Code | Meaning | Who normally sets it | Has a message? |
|---|---|---|---|
| Unset | Nothing special reported, usually means finished normally | Nobody. It is the default | No |
| Error | The operation failed | Instrumentation libraries or application code | Yes, optional and short |
| Ok | Success was explicitly confirmed and is final | Application developers, on purpose | No |
Think of a hospital wristband system. Most patients wear a plain band, which is Unset and means nothing is wrong. A red band means a problem needs attention, which is Error. A special “cleared by the doctor” band, Ok, is used rarely and cannot be changed by a nurse who walks past. The rarity of the special band is what makes it meaningful.
Remember the shorthand: leave it alone when things are fine, set Error when they are not, and use Ok only when you truly mean “this is final and good”.
2Why Span Status Exists: The Problem Being Solved
A trace is a long list of steps. Before status existed, the only way to know whether a step failed was to read every detail by hand.
A trace is the story of one request moving through your system, made of spans. Each span records a name, a start time, an end time, and many small notes called attributes. A single request can produce dozens of spans, and a busy system can produce millions of traces every hour.
Now imagine you want to answer a simple question: “How many of my requests failed in the last hour?” Without a common marker, each tool would have to guess. One team might record a failure as an attribute named failed, another as error, another as result with the word “bad”. Each backend, dashboard, and alert rule would need to learn every convention. That is the problem span status solves.
One shared way to say “this went wrong”
The status field gives every span the same small vocabulary. A backend does not need to know whether the span came from a web server, a database call, or a message queue. It can look at one field and count the failures. This is the reason that trace viewers show red marks on failing spans, and the reason that many systems can compute an error rate without knowing anything about your code.
A summary that is cheap to read
Attributes carry the detail, such as the exact HTTP status code or the database error number. The status is a summary that sits above that detail. A person scanning a trace with fifty spans sees the red ones first, then reads the attributes of those spans for the reason. A machine counting failures reads only the status. Both are served by one field.
A school report card has a final pass or fail line at the bottom, above all the detailed marks. A parent glancing at the card sees the summary first, and only checks the details if the summary is worrying. The status field is the pass or fail line for each step of a request.
The same idea appears in other signals. Logs have a severity level, and metrics often carry an error type label. Span status is the tracing counterpart, and the three share the goal of making failures easy to find.
3Core Vocabulary You Need
A short list of words appears in every conversation about span status. Learn these once, and the rest of the guide reads easily.
| Term | Plain meaning | Everyday picture |
|---|---|---|
| Span | A record of one timed step | One line on a receipt |
| Trace | All spans of one request | The whole receipt |
| Status | The span’s summary of success or failure | The pass or fail line |
| Status code | One of Unset, Ok, or Error | The color of a wristband |
| Description | A short message explaining an Error | A note on the red wristband |
| Attribute | A key-value note attached to a span | A sticker on the parcel |
| Event | A timestamped note inside a span | A line in a diary |
| Exception | An error object raised by code | A dropped plate |
| Instrumentation library | Ready-made code that creates spans for you | A pre-built sensor |
| Semantic conventions | Agreed names and rules for attributes and behavior | A shared dictionary |
| Backend | The system that stores and shows traces | The archive and reading room |
Two ideas hiding in the table
First, the status is about the operation the span describes, not about the whole request. A request may involve a database call that failed and was retried successfully. The database span is an Error, and the request span may be fine. Each span reports on its own step.
Second, status and exceptions are different things. An exception is an event that your code raised. Status is the span’s overall verdict. Code can raise an exception without the span being marked as failed, and a span can be marked as failed without any exception, such as when a server returns a 503 response. They often go together, but they are separate, and mixing them up is a frequent source of confusion.
In a restaurant kitchen, a dropped plate is an event, which is the exception. The chef’s decision to mark the dish as “failed, remake” is the status. A dropped plate might still be caught in mid-air and the dish served, so the two are linked but not the same.
When reading any documentation, ask whether a sentence is about the status code, the description, an exception event, or an attribute. They are four different places that all talk about failure.
4Where Status Lives Inside a Span
To place the status correctly in your mind, it helps to see everything else a span carries.
flowchart TB
SPAN["Span"] --> ID["Trace ID, span ID, parent span ID"]
SPAN --> NAME["Name such as GET orders"]
SPAN --> KIND["Kind: server, client, internal, producer, consumer"]
SPAN --> TIME["Start time and end time"]
SPAN --> ATTR["Attributes: key-value details"]
SPAN --> EVT["Events: timestamped notes, including exceptions"]
SPAN --> LNK["Links to other spans"]
SPAN --> STAT["Status: code and optional description"]
STAT --> C1["Unset"]
STAT --> C2["Ok"]
STAT --> C3["Error with message"]
Reading the picture
A span holds identifiers that tie it to its trace and to its parent, a name that says what it did, a kind that says its role, and two timestamps. It then holds three flexible containers. Attributes hold details as key-value pairs, such as the URL or the database name. Events hold timestamped notes, such as an exception that occurred halfway through. Links point to other spans, which is useful for batch work. Finally, the status holds the verdict.
Notice what the picture tells you about the status. It is not an attribute, and it is not an event. It is its own dedicated field, which means every tool knows exactly where to look. This is deliberate. Because attributes are free-form, no tool can be sure what a given attribute means. Status is standardized, so every backend, library, and dashboard agrees on it.
The status is set during the span, read after it
Code can set the status at any moment while the span is still open. Once the span ends, it becomes read-only, and attempts to change the status are ignored. Exporters send the final status along with everything else. This is why status must be decided before the span ends, a point that matters in several later chapters.
A shipping label has a box for the destination, a box for weight, a box for handling notes, and one reserved stamp box for the final verdict, such as “delivered”. Every worker knows where the stamp goes, so anyone can check the stamp without reading the rest of the label.
Because status is a dedicated field, it is also very cheap. It adds only a tiny code and, for errors, one short message to each span.
5The Three Status Codes at a Glance
There are only three values, but each carries a specific promise. Understanding the promises prevents most mistakes.
Unset
No verdict has been given. A span that ends with Unset normally describes a step that completed without a reported problem.
Error
The operation failed. This code may carry a short message, and tools typically highlight such spans in red.
Ok
Someone with knowledge of the application explicitly declared success. It is final, and later attempts to change it are ignored.
How they appear on the wire
When spans travel in the OTLP format, the status code is sent as a number, and many tools display it as a name. Zero means Unset, one means Ok, and two means Error. You will see names such as STATUS_CODE_UNSET, STATUS_CODE_OK, and STATUS_CODE_ERROR in backends and in metrics labels. Knowing this helps you read raw data and configuration rules.
| Code | Number on the wire | Name seen in tools | Typical color in viewers |
|---|---|---|---|
| Unset | 0 | STATUS_CODE_UNSET | Normal, no highlight |
| Ok | 1 | STATUS_CODE_OK | Normal or a subtle success mark |
| Error | 2 | STATUS_CODE_ERROR | Red or another warning color |
Why there is no “Warning” code
Newcomers often ask why there is no middle value such as warning. The design keeps the status small and clear: either something failed or it did not. Shades of concern belong elsewhere, in attributes or events, or in a different signal such as logs, which have severity levels. A small set of codes is easier for every tool to interpret in the same way.
A traffic light has three colors, and each has a clear meaning, with no shade of orange-ish in between. Drivers across the world respond to the same signals. If every city invented its own shades, nobody could drive safely in a new place.
If you want to record something that is bad but not a failure, such as a slow cache or a retry that succeeded, use an attribute or an event, and keep the status for real failures.
6Unset: The Default That Does Not Mean “Unknown”
The first code is the most misunderstood. Its name sounds like “no information”, but in practice it carries a useful meaning.
Every span starts with the status Unset. If nothing in your code or in a library changes it, the span ends with Unset. For most healthy spans, that is exactly right: the step ran, nothing went wrong, and nobody needed to say anything more.
Why not simply default to Ok?
If every span started as Ok, then the Ok value would mean nothing. It would be on every span, including those where nobody checked anything. By starting at Unset, the system keeps Ok as a meaningful signal that someone looked and confirmed the outcome. Unset means “no complaint”, and Ok means “I vouch for this”.
How tools treat Unset
Almost all tools count Unset spans as non-errors. When a dashboard computes the error rate, it counts spans with Error as failures, and treats Unset and Ok as non-failures. So in everyday use, you can read Unset as “not an error”. The difference between Unset and Ok matters mostly to people who want to explicitly protect a span from later being marked as an error, which the Ok chapter covers.
The subtle trap
Because Unset means “no complaint”, a span whose failure nobody noticed also ends as Unset. A swallowed exception, a failed call that code ignored, or an instrumentation library that does not detect a particular failure all leave the span as Unset. The trace looks healthy while the user suffered. This is why status quality is a matter of coverage: failures only show up if some code sets Error.
A smoke detector that is silent means no smoke was detected, not that the building is guaranteed safe. If the detector has no battery, it is silent too. Unset is silence, and silence is only as trustworthy as the sensor behind it.
Do not read a trace full of Unset spans as proof that everything succeeded. It proves only that nothing was reported. Check that your instrumentation actually sets Error for the failures you care about.
7Error and Its Description
Error is the code that does the most work in practice. It is the one that colors spans red and counts toward error rates.
A span is marked Error when the operation it describes failed. Examples include a server that could not complete a request because of its own fault, a database call that was rejected, a timeout, and a code path that raised an unhandled exception. The code that notices the failure sets the status to Error.
The description message
When setting Error, you may add a description, a short text that explains what went wrong, for example “connection refused” or “payment gateway timed out”. This message is intended for developers who read traces. It is optional, and it is allowed only with the Error code. If you try to attach a description to Unset or Ok, it is ignored.
What makes a good description
Good descriptions
- Short, one line, plain words
- State the failure, such as “upstream timeout”
- Stable text that repeats across many failures, so they group together
- Free of secrets and personal data
Poor descriptions
- Long stack traces pasted in
- Text with user names, emails, or card numbers
- Unique text with request IDs, so no two errors look alike
- Vague words such as “failed” or “error” that explain nothing
Where the detail should go instead
Detailed information belongs in attributes and events. The standard way to record an exception, with its type, message, and stack trace, is an event on the span, covered in a later chapter. The status description is the headline, and the event is the full article. Keeping them separate prevents the status message from becoming a dumping ground.
On a repair ticket, the title line says “boiler not heating”, and the notes below explain the details. A good title is short and the same for similar jobs, so the repair shop can sort tickets. A title that includes the customer’s whole story is hard to sort and might reveal private details.
Write descriptions as if they will be grouped and counted. A few stable phrases are far more useful than thousands of unique ones.
8Ok: The Rare, Final Success Mark
Ok looks like the natural opposite of Error, but it has a different and more careful role. It is used far less often than newcomers expect.
Setting the status to Ok means “the application developer has confirmed that this operation succeeded, and nothing should change that”. The specification makes Ok final: once a span is Ok, later attempts to set it to Error or anything else are meant to be ignored.
Why anyone would want that
Sometimes a library marks a span as Error for something that your application knows is fine. For example, an HTTP client library might treat a 404 response as an error, while your code deliberately checks whether a record exists and is happy to hear “not found”. Your code can set the span to Ok to state that this outcome is expected, and later automatic marking cannot override it.
Who should set it
The guidance for instrumentation libraries is that they should not set Ok. They should leave successful spans as Unset, because a library cannot know whether the application considers the result good. Ok is for people who understand the business meaning. If everyone set Ok automatically, the code would lose its meaning, just as a “verified” badge on every product would verify nothing.
| Situation | Suggested status | Reason |
|---|---|---|
| A library finishes a request normally | Leave Unset | Libraries should not claim Ok |
| Your code checks for a record and “not found” is expected | Ok, if a library might mark it as Error | You know the outcome is acceptable and want it protected |
| A job completes all steps and your code confirms it | Ok, if you want explicit confirmation | You have domain knowledge that the library lacks |
| Nothing special happens | Leave Unset | No need to add anything |
A caution about Ok
Because Ok is final, setting it too early can hide a later failure. If you set Ok at the start of a task and then a step fails, the failure will not appear as an Error on that span. Set Ok at the end, after you are sure about the result, and only when you have a reason to.
A building inspector’s “approved” stamp is final for that inspection. If an assistant stamped every form on arrival, before looking at anything, the stamp would protect bad buildings. The stamp is valuable because it comes at the end and from someone who knows.
Do not set Ok on every span as a habit. It hides failures that later code or libraries would have reported, and it removes the difference between “confirmed” and “not checked”.
9The Rules of Changing Status
Several pieces of code can touch the same span’s status. The specification gives simple rules for who wins when they disagree.
stateDiagram-v2
[*] --> Unset
Unset --> Error: set Error
Unset --> Ok: set Ok
Error --> Ok: set Ok
Ok --> Ok: any later change is ignored
Error --> Error: set Error again updates message
Ok --> [*]: span ends
Error --> [*]: span ends
Unset --> [*]: span ends
Reading the picture
Every span starts at Unset. From there, code may set it to Error or to Ok. If the span is already Error, code may update the Error with a new message, or may raise it to Ok. Once the span is Ok, nothing changes it. And when the span ends, whatever state it is in becomes the final status that is exported.
The precedence in words
The simple way to remember the order is that Ok beats Error, and Error beats Unset. A later request to set Unset is ignored, because Unset is only the starting value and is not a verdict that can be restored. A later request to set Error on a span that is Ok is ignored, because Ok is final. A request to set Error on a span that is Unset or already Error is accepted.
| Current status | Request | Result |
|---|---|---|
| Unset | Set Error | Becomes Error |
| Unset | Set Ok | Becomes Ok |
| Error | Set Unset | Ignored, stays Error |
| Error | Set Ok | Becomes Ok |
| Ok | Set Error | Ignored, stays Ok |
| Any | Set after the span has ended | Ignored |
These rules are in the specification, and implementations in each language follow them, though small differences of behavior can exist. If your outcome seems surprising, check your SDK’s documentation.
On a form, a pencil mark can be rewritten, but an ink signature at the bottom cannot be changed by the next person who handles the form. Ok is the ink signature. Error is a pencil note, and Unset is the blank space.
Because the rules favor Ok, use it with care and late. If a later layer might discover a real failure, leave the decision to that layer.
10Who Sets the Status: Libraries Versus Your Code
Most spans in a real system are created not by your own code but by ready-made instrumentation. Knowing who is responsible for the status prevents gaps and double work.
Instrumentation libraries
An instrumentation library is code that wraps a common component, such as a web framework, an HTTP client, or a database driver, and creates spans for you. These libraries follow the rules of the semantic conventions, a shared set of agreements about names, attributes, and also status. A web framework library, for example, sets Error on a server span when it sends a server failure response. A database driver sets Error when a query raises an error. You get meaningful statuses without writing any code.
Your own code
Spans you create yourself, around business steps such as “calculate shipping” or “apply discount”, get no automatic status. They end as Unset unless you set something. If a failure in that step should count, your code must say so, usually when it catches an exception or detects a bad condition. Many tracing APIs also offer a convenience that marks a span as Error automatically when an unhandled exception leaves the span’s scope, though the behavior varies by language.
Application developers on top
The people who know the business meaning can refine statuses. They can mark an expected condition as Ok, or mark a business failure as Error when a library does not see it. This layered approach lets libraries cover the technical failures and lets the application cover the domain ones.
Library defaults
Technical failures such as server errors, failed queries, and connection problems.
Your spans
Business steps you wrap yourself. You decide what failure means here.
Overrides
Application knowledge that corrects or confirms library behavior, such as Ok for an expected “not found”.
In a factory, machine sensors flag mechanical faults automatically. A quality inspector who understands the product catches defects that sensors cannot see. A good factory uses both, and neither replaces the other.
Auto-instrumentation agents set status for the libraries they cover, but they know nothing about your own business steps. That is where hand-written spans and status decisions come in.
11HTTP Servers: When Is a Response an Error?
HTTP is the most common protocol you will trace, and its rules have a twist that surprises many people. A “client error” response is usually not an error on the server span.
An HTTP status code is a three-digit number a server returns. Codes starting with 2 mean success, 3 mean redirection, 4 mean the client made a bad request, and 5 mean the server failed. Span status and HTTP status code are different things, but the semantic conventions say how one maps to the other.
The rule for server spans
For a span of kind server, which describes the work of handling an incoming request, the convention is: a response in the 5xx range sets the status to Error, and responses in the 4xx range leave the status Unset. The reasoning is that a 4xx response means the caller sent something wrong, and the server did its job properly by rejecting it. Counting those as server errors would make the server’s error rate rise whenever clients misbehave, which is not the server’s fault.
| HTTP response on a server span | Span status | Why |
|---|---|---|
| 200, 201, 204 | Unset | Normal success |
| 301, 302, 304 | Unset | Redirects and cache hits are not failures |
| 400, 401, 403, 404, 409, 422 | Unset | The caller made the mistake, the server behaved correctly |
| 429 Too Many Requests | Unset | Deliberate rate limiting by the server is not a server fault |
| 500, 502, 503, 504 | Error | The server or something it depends on failed |
| Unhandled exception, connection dropped | Error | The server failed to produce a proper response |
The details still live in an attribute
Even when the status is Unset for a 404, the exact response code is still recorded in an attribute, usually named http.response.status_code in the current conventions. So nothing is lost. You can still chart 404s, but they do not inflate the error rate.
A librarian who tells a visitor “that book is not in our collection” has done the job correctly. The librarian’s performance score should not drop because visitors asked for missing books. But a librarian who loses the catalog is failing, and that should count.
Older instrumentation or custom code may mark every 4xx as Error on server spans. This makes error dashboards noisy, and teams stop trusting them. Check what your libraries do, and note that conventions evolve between versions.
12HTTP Clients: The Opposite Viewpoint
The same HTTP response is judged differently when you are the one making the call. Perspective changes the verdict.
For a span of kind client, which describes an outgoing call to another service, the convention is stricter: both 4xx and 5xx responses set the status to Error. The reasoning is that from the caller’s point of view, a 404 or a 400 usually means that the call did not achieve what it set out to do. The caller wanted data and did not get it.
| Response | Server span status (the callee) | Client span status (the caller) |
|---|---|---|
| 200 | Unset | Unset |
| 404 | Unset | Error |
| 429 | Unset | Error |
| 503 | Error | Error |
| Connection refused or timeout | Not applicable, the server never saw it | Error |
flowchart LR
CALLER["Caller service"] -- "request" --> CLIENTSPAN["Client span"]
CLIENTSPAN --> SERVERSPAN["Server span in the other service"]
SERVERSPAN -- "404 response" --> SERVERSPAN
SERVERSPAN -. "Unset: the server did its job" .-> NOTE1["Server view"]
CLIENTSPAN -. "Error: the caller did not get what it needed" .-> NOTE2["Client view"]
What this means when reading a trace
You may see a red client span and a plain server span in the same call. That is not a contradiction. The two spans describe the same exchange from two sides, and each is judged by its own rules. The red client span tells you that the caller’s step did not succeed. The plain server span tells you that the callee did not fail.
When the caller expects a 404
Sometimes the caller expects a “not found” and handles it as a normal outcome, such as checking whether a username is free. The convention would mark the client span as Error anyway. The application can override this by setting Ok on the span, as the earlier chapter explained, or can reduce the noise in the Collector. Which one you choose depends on how often the pattern occurs and who owns the instrumentation.
When you ask a shop for an item that is out of stock, the shop did nothing wrong, but your shopping trip did not succeed. The shop’s record says “request handled”. Your record says “item not obtained”. Each record is true from its own side.
When calculating a service’s own error rate, use server spans. When calculating how healthy that service’s dependencies look from its side, use client spans.
13gRPC, Databases, and Messaging: Other Protocols
HTTP is only one of the protocols in a real system. Each has its own idea of failure, and the conventions map those ideas onto the same three codes.
gRPC
gRPC is a popular protocol for service-to-service calls. It has its own status codes with names such as Unavailable, DeadlineExceeded, and NotFound. The conventions treat these differently for servers and clients, in a way parallel to HTTP. On the client side, any status other than the success code is generally an Error. On the server side, only codes that signal a fault of the server, such as internal failures, unavailability, deadline problems, and data loss, are an Error. Codes that signal the caller’s mistake, such as invalid arguments or not found, leave the status Unset. The exact list is in the conventions, and it can change with versions, so check the current version.
Databases
For a database client span, a failure to execute a query, such as a syntax error, a lost connection, or a timeout, makes the span an Error. A query that runs correctly and returns no rows is not an error. It is a successful query with an empty result. Libraries usually record an error type attribute describing the kind of failure.
Messaging
For messaging systems, spans describe sending a message, receiving it, and processing it. A send that is rejected by the broker is an Error. A processing step that raises an exception is an Error on the processing span. An important detail is that sending a message successfully says nothing about whether the receiver later succeeded, since processing happens later in a different span, often in a different trace linked by a link.
| Protocol | Marks Error when | Leaves Unset when |
|---|---|---|
| HTTP server | 5xx or unhandled failure | 2xx, 3xx, 4xx |
| HTTP client | 4xx, 5xx, or connection failure | 2xx, 3xx |
| gRPC server | Server-fault codes | Success and caller-fault codes |
| gRPC client | Any non-success code | Success |
| Database client | Query or connection failure | Query ran, even with no rows |
| Message producer | Broker rejected or send failed | Send accepted |
| Message consumer or processor | Processing raised a failure | Processing completed |
Different postal services have their own words for problems, such as “address unknown” or “refused at door”. A good international system maps all of them to a small set of common outcomes: delivered, returned, or lost. The status codes play the role of that common set.
The details of these mappings are maintained in the semantic conventions, which are updated over time. Treat the tables here as the general shape, and consult the current conventions for exact lists.
14Status Versus Exceptions: Two Different Records
Both record trouble, and they are often confused. Each has a separate job, and the best setups use both.
An exception is a problem object raised by code, with a type, a message, and a call history called a stack trace. OpenTelemetry records an exception on a span as an event named exception, carrying attributes for the type, message, and stack trace. The status is the span’s overall verdict.
sequenceDiagram
participant C as Your code
participant S as Span
participant X as Exporter
C->>S: Start span
C->>C: Work raises an exception
C->>S: Record exception as an event
C->>S: Set status to Error with a short message
C->>S: End span
S->>X: Export span with event and status
The two records answer different questions
The status answers “Did this step fail?” in a way that every tool understands. The exception event answers “What exactly happened?” with the type, message, and stack trace. A dashboard counting failures reads the status. An engineer investigating one failure reads the event.
They do not always come together
In some language SDKs, helpers that run a span around a block of code will, by default, record any exception that escapes and set the status to Error. In others, recording the exception and setting the status are separate steps you must do yourself. Recording an exception without setting the status leaves the span looking healthy while carrying a hidden exception event. Setting the status without recording the exception leaves a red span with little explanation. Check which of the two your SDK does for you.
| Combination | What a viewer sees | Verdict |
|---|---|---|
| Event and Error status | Red span with a full explanation | Best for real failures |
| Event only, status Unset | Normal span with a hidden exception note | Fine for handled exceptions that are not real failures, confusing otherwise |
| Error status only | Red span with little detail | Acceptable when the failure has no exception, such as a 503 response |
| Neither | Normal span | Failure invisible if there was one |
When to record an exception without failing the span
Sometimes code catches an exception and recovers, such as a cache miss that raised an error before the code fell back to the database. Recording the exception as an event keeps a useful clue. Leaving the status unchanged tells viewers the step still succeeded. This is a legitimate choice, as long as it is intentional.
A mechanic’s work order has a line for the final result, such as “fixed” or “not fixed”, and a separate notes area describing what they found. Some jobs have notes about a broken part that was replaced during the visit, yet the final result still reads “fixed”.
For real failures, do both: record the exception for detail, and set Error for visibility. For handled problems, record the event if useful and leave the status alone.
15Status Versus Attributes: Summary Versus Detail
Spans carry several fields that talk about failure. Knowing which carries what stops you from searching in the wrong place.
| Field | What it holds | Who reads it |
|---|---|---|
| Status code | Unset, Ok, or Error | Every tool, for counting and highlighting |
| Status description | Short message with Error only | Developers scanning traces |
| HTTP response status code attribute | The exact number such as 503 | Anyone studying HTTP behavior |
| Error type attribute | A short, stable name for the failure class, such as a timeout or a specific exception class | Grouping and charts by kind of failure |
| Exception event | Type, message, and stack trace | Engineers debugging one failure |
The error type attribute
The current semantic conventions define an attribute usually named error.type. It holds a low-variety label describing the kind of failure, for example the exception class name, a protocol-specific code, or a word such as “timeout”. Because it has few distinct values, it works well for grouping failures on charts, and it appears on metrics as well as spans, which lets spans and metrics speak the same language about failures.
How they fit together
Think of layers of detail. At the top, the status code says that the step failed. Next, the error type says what kind of failure it was. Then, the protocol attribute gives the exact code. Finally, the exception event gives the stack trace. A good trace lets you start at the top and drill down only as far as you need.
Status code
Is this step a failure at all? Red or not.
Error type
What kind of failure? Timeout, refused, invalid input.
Protocol attribute
Which exact code? HTTP 503 or a database error number.
Exception event
Where in the code? Type, message, and stack trace.
A doctor’s file starts with a diagnosis category on the cover, then a specific condition inside, then test numbers, and finally detailed lab reports. You read only as deep as your question requires.
Attribute names in conventions have been renamed over time, for example HTTP attributes were updated and stabilized. If you see older names in existing data, the meaning is usually the same.
16Technical Failures Versus Business Outcomes
The hardest status decisions are not about exceptions. They are about outcomes that are bad for the user, yet perfectly normal for the system.
Consider a payment service that declines a card because the balance is too low. The system worked perfectly. The code ran, the bank answered, and the decline was a correct result. Yet the customer did not get what they wanted. Should the payment span be an Error?
There is no single answer, but there is a useful way to think. The status is mostly about whether the operation failed to do its job. If the job is “process this payment request correctly”, a decline is a successful outcome of that job. If the job is “charge this card”, a decline is a failure. The span’s name and purpose decide.
flowchart TB
START["Something did not go the user's way"] --> Q1{"Did our system or a dependency malfunction?"}
Q1 -- "yes" --> ERR["Set Error"]
Q1 -- "no" --> Q2{"Is the outcome a normal, expected result of the operation?"}
Q2 -- "yes" --> UNSET["Leave Unset and add an attribute for the outcome"]
Q2 -- "no" --> Q3{"Would an engineer want this counted as a failure?"}
Q3 -- "yes" --> ERR
Q3 -- "no" --> UNSET
Explaining the path
First ask whether something broke: a crash, a timeout, a dependency failure. If so, set Error. If nothing broke, ask whether the outcome is a normal result of the operation, such as a declined card, an invalid password, or an out-of-stock item. If so, leave the status Unset and record the outcome in an attribute, such as payment.result with a value of “declined”. If it is not normal, or an engineer would want it counted as a failure, set Error.
Why this matters for dashboards
If every declined card marks an Error, then the error rate of the payment service tracks customer behavior instead of system health. Alerts would fire when many customers run out of money. If none of them marks Error, you may miss a real problem hidden among declines. A separate business metric, such as a decline rate, is the better place to watch customer outcomes.
A vending machine that correctly says “insufficient coins” is working. A vending machine that jams is broken. The first outcome belongs on a sales report, and the second belongs on a repair ticket. Mixing them confuses both teams.
Agree on a short written rule in your team about which business outcomes count as Error, so that services behave consistently.
17Retries and Handled Errors
Real systems retry failed calls and recover from many problems. Status should describe each step honestly without confusing the whole request.
Suppose a service calls another service, gets a timeout, waits briefly, and tries again. The second try succeeds. The user sees a normal response. What should the trace show?
One span per attempt
The usual recommendation is that each attempt is its own span. The first attempt’s client span is marked Error, because that call did fail. The second attempt’s span is Unset, because it succeeded. The parent span that wraps the whole operation, including the retry, ends as Unset, because the operation as a whole succeeded. The trace shows the truth: one failed attempt, then a good one.
flowchart TB
PARENT["Parent span: fetch profile, status Unset"] --> A1["Attempt 1: client span, status Error, timeout"]
PARENT --> WAIT["Wait before retry"]
PARENT --> A2["Attempt 2: client span, status Unset, success"]
Why this is useful
If the error rate counts the red attempt, you see how often dependencies misbehave, even when retries hide it from users. That is valuable early warning. If you counted only the parent, you would see nothing until retries start failing too. At the same time, the parent’s clean status tells you that users were not hurt.
Fallbacks and graceful degradation
The same idea applies when code falls back to a simpler result. A recommendation service that cannot reach its model might show popular items instead. The call to the model is an Error, and the recommendation operation might be Unset, or you might add an attribute noting that a fallback was used. Choose the status that matches what you want to count, and record the fallback in an attribute so that the story is clear.
If a first taxi does not arrive, you call a second and reach the airport on time. Your trip record says “arrived on time”, and the taxi company’s record notes one missed pickup. Both are true, and each is useful to a different reader.
If a library marks the parent span as Error just because an inner call failed, even though your code recovered, your error rate will overstate user-visible failures. Check how your code and libraries handle caught exceptions.
18Parent and Child Spans: Errors Do Not Travel Upward
A common question is whether an error in a child span automatically makes its parent an error. It does not, and the reasons are worth understanding.
Each span has its own status, set independently. If a child span ends as Error, the parent span does not change automatically. The parent is marked as Error only if the code that owns the parent decides to do so, for example because the exception propagated up through it, or because the operation as a whole failed.
flowchart TB
ROOT["Root span: POST checkout, status Error"] --> A["Validate cart, status Unset"]
ROOT --> B["Charge payment, status Error"]
B --> B1["Call bank API, status Error, timeout"]
B --> B2["Write audit row, status Unset"]
ROOT --> C["Send receipt, not reached"]
What happens in the figure
The call to the bank timed out, so that span is an Error. The span that charges the payment also failed, because the exception left it. The root span for the checkout request failed, since the whole request returned a failure. Meanwhile, validating the cart succeeded, and writing the audit row succeeded, so both are Unset. The receipt step was never reached. Status is a property of each span, and the pattern of reds shows the path the failure took.
Backends and the idea of an “error trace”
Many backends flag a whole trace as containing an error if any span in it is an Error. This is a convenience for searching, such as “show me traces with errors”. It does not mean that every span or the root is an Error. A trace can contain red spans deep inside while the root is clean, as in the retry example. Be careful when reading “trace has error” and ask which span carries it.
In a relay race, one runner may stumble and recover, while the team still finishes well. A stumble is recorded for that runner, and the team result is separate. Only if the stumble loses the race does the team’s result change.
Because status does not propagate automatically, the person or library that owns each span must decide. This is why good instrumentation handles status at every level where a failure could matter.
19Timeouts, Cancellations, and Partial Results
Some outcomes sit awkwardly between success and failure. How you label them decides whether your charts reflect reality.
Timeouts
A timeout happens when an operation takes longer than allowed and is abandoned. From the caller’s viewpoint, the step did not achieve its goal, so it is normally an Error. The span should carry a short description such as “timeout”, and an error.type attribute with a stable value, so timeouts can be counted separately from other failures. On the callee’s side, the server may not even know the caller gave up, and it may finish its work normally.
Cancellations
A cancellation happens when the caller deliberately stops waiting, such as a user closing a page, or a request that was no longer needed. Conventions in different areas treat this in different ways. Many teams choose to leave cancelled spans Unset on the side that was cancelled, because nothing malfunctioned, and mark an attribute to say that the request was cancelled. Others treat it as an Error on the client. Pick a rule, write it down, and use it consistently, since there is no single universal answer.
Partial results
A request that returns a partial result, such as a search that could query only two of three data sources, is a gray area. If your product treats this as a success with reduced quality, leave Unset and add an attribute such as result.partial. If your product treats it as a failure, set Error. What matters is that the same situation is handled the same way every time.
| Situation | Common choice | Extra detail to record |
|---|---|---|
| Client timeout waiting for a response | Error on the client span | error.type as timeout |
| Server finished after the caller gave up | Usually Unset on the server span | Attribute noting late completion if known |
| User cancelled a request | Often Unset | Attribute noting cancellation |
| Partial result returned | Depends on the product, often Unset | Attribute marking the result as partial |
| Fallback value used | Unset on the operation, Error on the failed inner call | Attribute noting the fallback |
A customer who hangs up after waiting twenty minutes on hold is different from a call that was dropped by the phone company. Both end the call, but one is a customer choice and the other is a system failure. A good call center records them differently.
For gray areas, favor an attribute that records the nuance, and keep the status for clear failures. Attributes can be queried and charted without distorting error rates.
20How Status Travels: OTLP and Backends
Once a span ends, its status leaves your program and is stored, searched, and drawn. Following that path shows what each tool does with it.
flowchart LR
SPAN["Ended span with status"] --> EXP["Exporter"]
EXP -- "OTLP message with status code and message" --> COL["Collector"]
COL --> BE["Tracing backend"]
COL --> SM["Span metrics generator"]
BE --> UI["Trace viewer shows red spans"]
SM --> DASH["Error rate dashboards and alerts"]
On the wire
The exporter packs the span, including its status, into an OTLP message. The status appears as a small object with a code and a message. The Collector can read this status and use it in rules, such as keeping all spans with an Error status. The backend stores it alongside the span and offers it for search and display.
In the backend
Tracing backends use status in several ways. They color failing spans red in waterfall views. They offer filters such as “show traces with errors”. Query languages let you select spans by status, for example by asking for spans where the status equals error. Some backends convert the status into a tag or label of their own for historical reasons, so you may see a field named error set to true in older tools, which stands for a status of Error.
In derived metrics
A Collector component can turn spans into metrics, including call counts and durations, labeled by service, operation, and status code. This produces numbers like “requests per second” and “errors per second” for every service without extra instrumentation. Because the labels include the status, the quality of these metrics depends directly on how well the status was set.
A hospital’s admission form travels to many departments. The first department fills in a priority tag. Later, billing, the records office, and the planning team all read the same tag. If it was wrong at the start, every department makes the wrong decision.
The exact field names and filter syntax differ between backends. The concept is universal: select spans by status, and count them by status.
21Error Rates: The Number Everyone Watches
Most teams reduce service health to three numbers: how many requests, how many failed, and how slow. Span status supplies the middle one.
The popular shorthand for these three numbers is RED, which stands for rate, errors, and duration. The error rate is the share of requests that failed. With spans, it is computed as the number of server spans with status Error divided by the total number of server spans, over a time window.
A worked example with made-up numbers
An imaginary service, “Quill Search”, handles 1,200 requests per minute. Over one minute, the server spans show 1,150 Unset, 40 Error, and 10 spans with a 404 response that were left Unset by the convention. The error rate is 40 divided by 1,200, which is about 3.3 percent. If the 404s had been counted as errors, the rate would have been 50 divided by 1,200, about 4.2 percent, which is almost a third higher, and the extra would reflect user typing mistakes rather than a failing service.
| Count (made-up) | Spans | Counted as error? |
|---|---|---|
| Successful responses | 1,150 | No |
| Server failures, 5xx | 40 | Yes |
| Not found, 404 | 10 | No, Unset by the convention |
| Total | 1,200 | Error rate about 3.3 percent |
Accuracy depends on sampling
If you compute the error rate from stored spans after sampling, the ratio is usually still close to the truth when sampling is fair and uniform, because both the failed and the total counts shrink by the same fraction. But if the sampling favors errors, for example by keeping all error traces and few healthy ones, the stored spans overstate the error rate badly. That is why error rates are best computed from metrics generated before sampling, as the sampling chapter explains.
A bakery’s complaint rate is the number of complaints divided by the number of customers served. If staff also counted people who asked for a flavor the bakery never sold, the rate would rise without any drop in quality.
A tiny change in which responses count as Error can move your error rate noticeably. When you upgrade instrumentation libraries, compare the error rate before and after, since a change in convention can look like a change in system health.
22Alerting on Status
Alerts are the moment status stops being a display detail and starts waking people up. Mistakes here have a human cost.
What to alert on
Alert on a rising error rate, computed from spans or from metrics generated from spans, for services whose failures users feel. Add a waiting period, so a brief blip does not page someone. Link each alert to a dashboard and a short written guide that tells the responder where to start.
Why status quality becomes an operational issue
If status marks too many harmless things as Error, the alert fires on nonsense, and the team learns to ignore it, a state called alert fatigue. If status misses real failures, the alert stays silent during an outage, and users find the problem before you do. Both are failures of status quality, not of the alerting tool.
Alert on server spans
Use the service’s own server spans for its health alerts, since conventions ensure caller mistakes are excluded.
Use a ratio, not a count
Errors per total requests keeps the alert meaningful as traffic rises and falls.
Group by error type
Breakdowns by error type show whether one failure class dominates.
Alerting on every client Error
Client spans for expected 404s or retries create noise. Alert on user-facing symptoms.
A quiet system is not always a healthy one
Also consider alerting on the absence of data. If a service stops sending spans, the error rate may drop to zero because there is nothing to count, and the alert will stay silent. A separate alert on request volume falling too low catches this case.
A smoke alarm that rings for every piece of toast is turned off by the household. A smoke alarm with a dead battery is silent during a fire. A good alarm system is quiet when safe, loud when not, and checks its own battery.
Before adding an alert on status, look at a week of data for the alert’s condition and count how often it would have fired, and whether each firing would have needed action.
23Status and Sampling: Why Errors Are Hard to Keep
Teams that sample traces to save cost soon discover a puzzle: the traces they most want, the failing ones, are the hardest to keep.
Head sampling, such as the ratio-based sampler, decides at the start of a trace. At that moment the status is still Unset, because nothing has happened yet. So a head sampler cannot choose to keep failing traces. It treats failing and healthy requests the same way, and with a low ratio, it keeps only a small share of the failures.
Tail sampling decides after the trace is complete, so it can read the final statuses. A common rule is “keep every trace that contains a span with status Error, and a small share of the rest”. This is where status becomes a sampling input, and it is why the quality of status matters even more once you adopt tail sampling: wrongly marked spans decide which traces you keep.
flowchart TB
START["Trace begins"] --> HEAD{"Head sampler: status is still Unset"}
HEAD -- "dropped" --> LOST["Failures in these traces are never seen"]
HEAD -- "kept" --> RUN["Trace runs and spans end with final statuses"]
RUN --> TAIL{"Tail sampler reads final statuses"}
TAIL -- "any span has Error" --> KEEP["Keep the trace"]
TAIL -- "all healthy" --> SMALL["Keep a small share, drop the rest"]
Practical consequences
- If you rely only on head sampling at a low ratio, many failures will have no stored trace, even though the status was set correctly.
- If you use tail sampling by status, keep your status rules precise, or you may keep far too many traces because of noisy Error marks.
- Compute error rates from metrics created before sampling, since the stored traces are biased toward errors.
A news editor who decides at dawn which events to cover cannot know which will turn into big stories. An editor who reviews all the wire reports at night and picks the biggest ones can keep the best, but needs a large desk and the night to do it.
The sampling guides in this series go deeper on head and tail sampling. Here the point is only that status is the main signal tail sampling uses to find the interesting traces.
24Long-Running, Asynchronous, and Streaming Work
Not every span lasts a few milliseconds. Background jobs, queues, and streams need a little extra thought about when and how status is set.
Background jobs
A background job may run for minutes and has one span for the whole job, with children for steps. The job’s span ends as Error if the job fails and is Unset or Ok if it completes. If a job processes a thousand items and fails on ten, the job span’s status depends on your definition of success. Many teams mark the job as failed only if it could not complete its purpose, and record the number of failed items in an attribute.
Queues and asynchronous steps
With messaging, a producer span ends as soon as the message is accepted. The consumer processes it later, in its own span, which is linked to the producer. A failure in processing marks the consumer span as Error but cannot change the producer’s span, since it already ended. When you analyze a “whole business flow”, you must follow the links across spans and traces, because the status of the flow is spread over several spans.
Streams and long connections
A streaming call, such as a long-lived connection, may have one span that lasts a long time. Status is set when the stream ends. If it ends cleanly, leave Unset. If it ends with a failure, set Error. For very long streams, teams often create shorter spans for each message or time window, so failures are visible sooner and the status represents a smaller piece of work.
| Kind of work | When status is final | Common practice |
|---|---|---|
| Short request | At the end of the request | Libraries set it |
| Background job | At job end | Error if the job fails its purpose, count failed items in an attribute |
| Message processing | At the end of processing one message | Error on the consumer span, link back to the producer |
| Long stream | At the end of the stream | Use shorter spans per message or window |
A letter that is posted and later opened has two stories: the posting and the reading. The post office can say “accepted for delivery” at the first step, but only the reader’s reaction tells you what happened. They are linked, and each has its own result.
If a status would remain unknown for hours, add smaller spans or events along the way, so that an unfinished job still shows progress and partial failures.
25Writing Good Descriptions: Examples
The description is only a few words, but good words make a difference when a tired engineer reads a trace at midnight.
| Poor description | Better description | Why better |
|---|---|---|
| Error | Upstream timeout | Says what kind of failure |
| Failed to process order 8472913 for user anna@example.com | Order processing failed | Removes IDs and personal data, and stays stable for grouping |
| java.lang.NullPointerException at line 118 … (long stack) | Missing required field | Keep the stack trace in the exception event |
| Something went wrong, please try again later | Payment gateway unavailable | Written for the developer, not the end user |
| Connection to 10.0.4.17:5432 refused at 12:03:55 | Database connection refused | Address and time belong in attributes |
Guidelines in plain words
- Keep it short. A phrase, not a paragraph.
- Keep it stable. The same problem should produce the same words, so counting and grouping work.
- Keep it private. No personal data, no secrets, no tokens.
- Write it for developers. The reader knows the system and needs a clue, not an apology.
- Put details elsewhere. Specific IDs, addresses, and stack traces go in attributes and events.
Why stable text matters
Many tools group failures by their description. If every description contains a unique request number, every failure appears unique, and you cannot see that the same problem happened ten thousand times. Stable descriptions turn a haystack into a handful of piles.
A library files returned books by category on the cart, such as “water damage” or “missing cover”, not by a long story about each borrower. Short stable labels let the staff handle the whole cart quickly.
Skim twenty real Error descriptions from your system. If more than a few look unique or very long, fix the instrumentation that creates them.
26Security and Privacy Considerations
Error information is among the most revealing data in a system. Handle it with the care you would give to log lines.
Error messages leak
Messages from databases, libraries, and frameworks often contain the very details you would not want to share: a table name, a query fragment with user input, a file path, a hostname, or a credential in a connection string. If the status description or the exception event copies such messages, every person with access to traces can read them. Treat error text as sensitive until proven otherwise.
Protective habits
Use fixed descriptions
Choose short, fixed phrases for the status description, rather than copying raw exception messages.
Scrub events
Remove or mask sensitive values in exception messages and stack traces, in code or in a Collector processor.
Limit access
Give trace viewers the same permissions you would give to production logs.
Review samples
Read a handful of real failure spans each quarter and look for leaks you did not expect.
Status as an information channel
Status codes themselves are not secret, but error patterns can reveal weaknesses. A flood of errors from one endpoint, visible to a wide audience, may tell an attacker where your system is fragile. Keep dashboards and alerts access-controlled in the same way you control other operational data.
A hospital does not post the reason for each patient’s visit on the waiting-room wall. Staff see what they need, and visitors see a general status. Error details in traces deserve the same discretion.
Automatic instrumentation sometimes records the full raw exception message as the status description or as an event. Check what your libraries record, and add scrubbing if needed.
27Checking the Quality of Your Status Data
Status looks tiny, and bad status is invisible. A few simple checks reveal whether your data tells the truth.
Compare with an independent source
Compare the span-based error count with your load balancer’s or web server’s own 5xx count. A large gap means missing or extra status marks.
Look for Error spans that look healthy
Search for Error spans with a 4xx server response, or with an Ok-looking attribute such as a normal outcome. They suggest overly eager marking.
Look for healthy spans that failed
Search for spans that carry an exception event or a 5xx attribute but have status Unset. They suggest missed marking.
Check the share of Ok
If a large share of spans are Ok, someone may be setting it as a habit. It should be rare.
Check description variety
Count distinct description values. A huge number means unique text, so fix grouping.
Where to run these checks
Most backends let you query spans by status and attributes. Run these checks once when you onboard a new service, and again after upgrading instrumentation libraries or changing frameworks. A small table of results, kept in a shared page, shows whether data quality is improving or slipping.
A shop that counts its cash register totals against its bank deposits once a week soon notices when something is off. The register may be accurate, but only an independent comparison proves it.
The best independent source is the edge: the load balancer or gateway sees every request and its response code, regardless of what any internal service reports about itself.
28What Goes Wrong: Failure Scenarios
Most problems with span status are quiet. Nothing crashes, and the charts keep drawing, but they are wrong. Knowing the common shapes of wrongness helps you spot them.
| Failure | What you observe | Likely cause |
|---|---|---|
| Error rate always zero | Dashboards look perfect during incidents | Failures never set Error, exceptions swallowed, libraries not covering the failing path |
| Error rate permanently high | Constant noise, alerts ignored | 4xx marked as Error on server spans, expected not-found treated as failure |
| Red spans with no explanation | Error status but no event or description | Status set without recording the exception or a message |
| Exceptions visible but spans look healthy | Events present, status Unset | Exception recorded but status not set |
| Failures hidden by Ok | Known failures appear healthy | Ok set early or as a habit, and later errors ignored |
| Parent clean though user saw a failure | Child red, root Unset | Code caught the exception and returned a failure response without marking the span |
| Different services disagree | Same failure red in one and clean in another | Inconsistent rules about business outcomes and cancellations |
| Numbers jump after an upgrade | Error rate changes with no real change in health | Convention changed in a newer instrumentation library |
The silent failure is the worst kind
None of these produces an error message in your own system. The only sign is that the data does not match reality. Users complain, but dashboards say all is well, or alerts fire constantly at things nobody cares about. This is why the quality checks of the previous chapter are worth the time.
A car dashboard with a warning lamp whose bulb has burned out looks reassuring, and the engine could still be failing. A warning lamp that is always lit is ignored. Status marks that are always zero or always high are like these two kinds of broken lamp.
An error rate of exactly zero for a service that handles real traffic is more often a sign of missing instrumentation than of perfection. Treat it with suspicion.
29Good Patterns
Teams with trustworthy error data tend to follow the same few habits.
Follow the conventions
Let instrumentation libraries apply the standard HTTP, gRPC, and database rules, and avoid reinventing them.
Unset by default, Error for failures
Successful spans stay Unset. Ok is rare and deliberate.
Event plus status
Real failures record the exception as an event and set Error with a short stable description.
Outcome attributes for business results
Declined, out of stock, and partial results go in attributes, and the status stays for real faults.
One span per attempt
Retries get their own spans, so failed attempts are visible while the parent reports the final result.
A written status policy
A short team page that states how to treat cancellations, validation failures, and expected not-found results.
Why the written policy pays off
Without a policy, each developer decides separately, and the result is a patchwork. One service marks validation failures as Error, another leaves them Unset, and a third marks them Ok. A short written policy, a page long at most, aligns the choices. New teammates read it once and apply it forever. The policy is also the place to record decisions that caused arguments, so that nobody has to have the same argument twice.
A sports league publishes the rulebook so that every referee calls the same foul the same way. Without it, each game is judged differently, and the results cannot be compared across teams.
Store the status policy with your instrumentation guidelines and link to it from code review templates, so it is read at the right moment.
30Anti-patterns
These habits look harmless and slowly poison the data.
Marking every 4xx as Error on server spans. The error rate then tracks client mistakes, and alerts fire for reasons the service owner cannot fix.
Setting Ok on every span. This removes the difference between confirmed and unchecked, and it hides later failures because Ok is final.
Swallowing exceptions silently. A catch block that returns a default value and records nothing leaves a failure invisible.
Pasting stack traces or raw messages into the description. It leaks data, wastes space, and makes every failure look unique.
Treating business declines as technical errors. Customer behavior then drives the error rate and the alerts.
Using the status description for variable data. IDs and addresses belong in attributes, so descriptions can be grouped.
Computing error rates from sampled traces that favor errors. The rate is inflated, and nobody remembers why.
Anti-patterns are like skipping the label on a jar. It saves a few seconds today, and next month nobody can tell the salt from the sugar.
31Best Practices and Common Mistakes
This is the checklist to keep next to your instrumentation guide.
Best practices
- Leave successful spans Unset, and reserve Ok for deliberate, final confirmation.
- Set Error when the operation failed to do its job, and let libraries do it for technical failures.
- On server spans, treat 5xx as Error and 4xx as Unset, following the conventions.
- Record exceptions as events, and keep the status description short, stable, and private.
- Use an error type attribute with a small set of values for grouping.
- Use separate spans for retries, and let the parent reflect the final result.
- Keep business outcomes in attributes unless they truly are failures of the system.
- Write down a team policy for gray cases such as cancellations and partial results.
- Check status quality against independent sources such as the load balancer.
- Compute error rates and alerts from metrics before sampling.
Common mistakes and their cures
| Mistake | Symptom | Cure |
|---|---|---|
| Setting Ok early in a task | Later failures hidden | Set Ok at the end, or leave Unset |
| Setting status after the span has ended | Status silently unchanged | Set status before ending the span |
| Recording an exception only | Healthy-looking span with a hidden exception | Also set Error for real failures |
| Setting Error only | Red span without explanation | Add the exception event and a short description |
| Description with personal data | Privacy risk across all traces | Use fixed phrases and scrub events |
| Assuming status travels to the parent | Root looks healthy though a child failed | Decide status at each level that matters |
| Alerting on client spans for expected 404s | Alert noise | Alert on server spans and user-facing symptoms |
Pilots read checklists not because they forget how to fly, but because even experts skip small steps under pressure. This list does the same for status.
When reviewing code that creates spans, look at every catch block and ask what status the span will have afterward. That one question finds most mistakes.
32Real-World Style Examples
These stories are invented to illustrate common situations. They are not reports about real companies.
Example one: the dashboard that was always green
A team’s order service has an error rate of zero for months, while support receives complaints about failed checkouts. A review finds that a catch block returns a polite failure message to the user but never marks the span. The spans look perfectly healthy. After the team sets Error and records the exception in that block, the dashboard shows the true failure rate of about two percent, and a long-standing bug is found within a day.
Example two: the alert nobody trusted
A search service pages the on-call engineer several times a week for high error rates. Each time, the engineer finds nothing wrong. The cause is that an older instrumentation library marks every 404 response as an Error on server spans, and users often follow broken links from old emails. After the team upgrades the library and applies the standard rule, the false alerts stop, and the next real alert gets prompt attention.
Example three: the retry that told the truth
A profile service calls a user database. During a minor database slowdown, about one call in twenty times out and succeeds on retry. Users notice nothing. Because each attempt is its own span, the failed attempts show up red in the error metrics for the database client, while the profile operation stays clean. The team sees the slowdown a day before it would have affected users and adds capacity.
Example four: the secret in the message
During a security review, an engineer searches trace descriptions and finds that a database error message copied into the status description contains parts of a connection string. The team replaces raw messages with fixed phrases such as “database connection refused”, scrubs exception events in the Collector, and rotates the exposed credentials.
Each story is like a lesson from a driving instructor: one rule, one mistake, one fix. The same few lessons repeat for many drivers on many roads.
All four examples are fictional and simplified. Real incidents usually mix several causes, so confirm your diagnosis with measured data before changing instrumentation.
33Troubleshooting: My Spans Show the Wrong Status
When status looks wrong, work through the path from the code that sets it to the tool that shows it.
flowchart TB
START["Status looks wrong in the trace viewer"] --> A{"Is the status wrong at the source span?"}
A -- "no, correct in the app but wrong in the viewer" --> A1["Check Collector rules and backend mapping"]
A -- "yes" --> B{"Is the span from a library or from my code?"}
B -- "library" --> B1["Check library version and convention then override if needed"]
B -- "my code" --> C{"Is status set before the span ends?"}
C -- "no" --> C1["Move the status call before the end"]
C -- "yes" --> D{"Was Ok set earlier?"}
D -- "yes" --> D1["Ok is final so remove the early Ok"]
D -- "no" --> E{"Was the exception caught and swallowed?"}
E -- "yes" --> E1["Record the exception and set Error where it is handled"]
E -- "no" --> E2["Check whether the SDK sets status automatically on exceptions"]
Explaining the checklist
First, find out whether the status was already wrong when it left the application, or whether something between the application and the viewer changed it. A Collector rule or a backend mapping can rewrite or reinterpret statuses, so compare the data at both ends. If the source span is wrong, find out who created it. For a library span, check the library version against the conventions you expect, and decide whether to upgrade or override.
For your own spans, check the order of operations. Status must be set before the span ends, and an early Ok blocks later Error. Then check for swallowed exceptions, the most common reason a failing span looks healthy. Finally, check what your SDK does by default when an exception leaves a span, because some set the status automatically and some do not.
When a package’s tracking page shows the wrong status, you ask whether the scanner at the depot made the error, or whether the website misread the scan. You check at the source first, and then at each step toward the screen.
Temporarily print the span status to a console exporter in a test environment. Seeing the raw status at the source removes half the guesswork.
34How It Looks in Different Languages
The three codes and the rules are the same everywhere. The way you reach them in code looks a little different in each language.
The shared shape
Every OpenTelemetry API offers a way to set the status on the current span. You provide the code, which is Unset, Ok, or Error, and for Error you can add a description. Every API also offers a way to record an exception on a span, and many offer an option on the helper that wraps a block of code in a span, so that unhandled exceptions are recorded and the status is set to Error automatically.
Where languages differ
| Topic | What varies |
|---|---|
| Names | Some languages call the codes Unset, Ok, and Error in each style, such as StatusCode.ERROR or SpanStatus.error, and the setter method has slightly different names |
| Automatic handling of exceptions | Some span helpers record the exception and set Error by default, and others leave both to you |
| Description support | The description is accepted only with Error in all of them, and others ignore it for the other codes |
| Handling of the final Ok rule | Follows the specification, though edge cases differ in older versions |
| Auto-instrumentation | Agents and libraries differ in how closely they follow the newest conventions |
How to check your own language
Find the page in your language’s documentation about tracing and spans, and look for the section on status and on recording exceptions. Then write a tiny test that creates a span, raises an exception inside it, and prints the final status using a console exporter. In a few minutes, you will know what your SDK does by default, and you can adjust your code accordingly.
Road signs differ in shape and language from one country to the next, yet a red octagon means stop almost everywhere. The meaning is shared, and only the packaging differs.
Do not assume that an example from one language behaves the same way in another, especially regarding automatic status on exceptions. Always test in your own environment.
35Frequently Asked Questions
It is a field on every span that summarizes whether the operation succeeded. It has a code, which is Unset, Ok, or Error, and an optional short description allowed only with Error.
Unset is the default and means nothing special was reported, which usually means the step completed normally. Ok is an explicit, final confirmation of success set by the application developer. Libraries should not set Ok.
Usually not. Leave it Unset. Use Ok only when you want to state success explicitly and protect it from later being marked as an error.
On a server span, the conventions leave 404 as Unset, because the caller made the mistake. On a client span, a 404 is typically an Error from the caller’s point of view.
Not necessarily. Recording an exception adds an event. Setting the status is a separate step, and some SDK helpers do both by default. Check your language.
No, not automatically. Each span carries its own status. The parent becomes an Error only if the code that owns it marks it, for example because the exception propagated or the operation as a whole failed.
No. Once a span has ended, it is read-only, and attempts to change the status are ignored.
A short, stable, developer-facing phrase such as “upstream timeout”, without personal data, identifiers, or stack traces. Put details in attributes and exception events.
Usually not, since the system worked correctly. Leave Unset and record the outcome in an attribute. Mark Error only when the system or a dependency malfunctioned.
The error rate is typically the number of server spans with status Error divided by all server spans. The quality of the number depends on how well the status is set.
Head sampling decides before the status is known, so it cannot favor failing traces. Tail sampling can read final statuses and keep the traces with errors. Compute error rates from metrics created before sampling.
No. The three codes are Unset, Ok, and Error. Record nuances such as degraded results in attributes or events.
36Summary and Key Takeaways
Span status is a small, standardized field that tells every tool whether the step a span describes went wrong. It has three codes. Unset is the default and usually means normal. Error means failure and may carry a short description. Ok is a rare, final confirmation set by people who understand the business meaning. Because the field is shared, backends, dashboards, error-rate metrics, alerts, and tail sampling all rely on it, which makes its quality far more important than its size suggests. Good status comes from following the conventions, recording exceptions as events beside the status, keeping business outcomes in attributes, and checking the data against independent sources.
Key Takeaways
- Span status has three codes: Unset by default, Error for failure, and Ok as a rare, final, deliberate confirmation. Only Error may carry a short description.
- Ok beats Error, and Error beats Unset. Once a span is Ok, later changes are ignored, and nothing changes after the span ends.
- On server spans, 5xx responses are Errors and 4xx responses stay Unset. On client spans, both 4xx and 5xx are Errors.
- Record exceptions as events and set the status separately. Real failures need both, handled problems may need only the event.
- Status does not travel from child to parent. Decide it at every level that matters, and use one span per retry attempt.
- Keep business outcomes in attributes, keep descriptions short, stable, and private, and compute error rates from metrics created before sampling.
- Check status quality against the load balancer or edge, and write down a team policy for gray cases.