Before learning the tool, let us feel the problem it solves.

A span tells you how long, but not what happened

Suppose your payment service has a span called “Charge card” that took six seconds. That is slow, and you can see it clearly in a trace. But why was it slow? Maybe the first attempt failed and the code waited and retried. Maybe a cache lookup missed and a slower database call was needed. Maybe an exception was caught and handled in the middle. The span’s start and end time show the total, but they hide the story inside.

You could solve this by creating many tiny child spans for every little step. But that quickly becomes heavy and noisy. Many of these small happenings are not “work that takes time.” They are simply moments: “retry started,” “cache missed,” “payment approved.” For moments like these, we need something lighter than a span.

What we really need

We need a way to say, “Pin a small note on this span at this exact time.” The note should have a name, and it may carry a few details. It should cost very little. And it should stay attached to the span, so that anyone who views the trace sees the note in the right place on the timeline. This is exactly what a span event is.

The short answer first, and then the pieces.

A span event is a timestamped message attached to a span. It records that something meaningful happened at one specific moment during the span’s life. Unlike a span, an event has no start and end. It has just one time. It is a dot on the timeline, not a bar.

The three parts of an event

flowchart LR
    A["Span starts: Charge card"] --> B["Event: cache.miss"]
    B --> C["Event: retry.started"]
    C --> D["Event: exception"]
    D --> E["Event: payment.approved"]
    E --> F["Span ends"]
    

In the diagram, the span runs from left to right. Along the way, four different moments were noted. If a viewer opens this trace in a tracing tool, the events usually show up as small marks on the span’s bar, and clicking one shows its name, time and attributes.

Knowing the origin explains the name and some of the habits.

Before OpenTelemetry, older tracing systems already had a way to attach time-stamped notes to a span. In Zipkin, they were called annotations. In OpenTracing, they were called logs, meaning small structured messages stored inside a span. When OpenTelemetry was formed from OpenTracing and OpenCensus, the community kept the idea but chose the name events. One reason was to avoid confusing these in-span notes with the separate, independent log records that the logging signal handles.

Some tracing tools still show span events under the old word “logs.” If you ever look at a trace viewer and see a “logs” section on a span, it may be showing span events.

The biggest beginner question is “which tool should I use for this?” Let us answer it clearly.

flowchart TD
    Q["Something worth recording"] --> A{"Does it describe the whole span?"}
    A -->|Yes| ATT["Use a span attribute"]
    A -->|No| B{"Does it take meaningful time?"}
    B -->|Yes| CS["Use a child span"]
    B -->|No| C{"Is it a notable moment inside this span?"}
    C -->|Yes| EV["Use a span event"]
    C -->|No| D{"Is it a count or total across many requests?"}
    D -->|Yes| MET["Use a metric"]
    D -->|No| LOG["Use a log record"]
    

How to remember it

Ask three questions. First, “Is this true for the whole span?” If yes, use an attribute. Second, “Does this take real time and deserve its own bar?” If yes, use a child span. Third, “Is this a moment worth marking?” If yes, use an event. This small habit prevents most mistakes.

An important idea: an event is not a separate piece of data. It lives inside its span.

When your code adds an event, the SDK does not send it away right then. Instead, it stores the event inside the span’s own memory, in a list. Only when the span ends is the whole span, including all its events, passed to the processors and exporters. Then the span and its events are sent together to the backend.

sequenceDiagram
    participant App as Application Code
    participant Span as Active Span in SDK
    participant Proc as Span Processor
    participant Exp as Exporter
    participant Back as Tracing Backend
    App->>Span: Add event cache.miss
    Span->>Span: Store event in span memory
    App->>Span: Add event retry.started
    Span->>Span: Store event in span memory
    App->>Span: End span
    Span->>Proc: Hand over finished span with events
    Proc->>Exp: Batch of spans
    Exp->>Back: Send spans and events together
    

Four things this design means for you

Non-recording spans ignore events

When a span is not recording, for instance because the trace was sampled out or because no SDK is installed, calls to add events do nothing. This is a feature. It means your code can add events freely, and you pay almost no cost when the data would not be kept anyway. For very expensive event details, many APIs let you check whether the span is recording before you prepare them.

Limits and what happens when you exceed them

Most SDKs protect your memory with limits. There is usually a maximum number of events per span, a maximum number of attributes per event, and sometimes a maximum length for attribute values. A common default for the number of events is 128, but defaults can differ, so check your SDK. When the limit is reached, extra events are dropped, and the span records how many were dropped. A trace viewer or an exporter may then show a “dropped events” count, which is a sign you are adding too many.

When something goes wrong, the standard way to note it on a span is with an exception event.

OpenTelemetry has a special, standard event for errors. Most APIs offer a helper called something like “record exception.” When you call it, the SDK adds an event with the well-known name exception. It carries standard attributes that describe what went wrong.

Recording an exception is not the same as marking the span as failed

This is one of the most common beginner surprises. Adding an exception event does not automatically change the span’s status to error. They are two separate steps. The event says “an exception happened at this moment.” The status says “the overall operation failed.” A span can contain an exception that was caught and handled, such as a failed first attempt followed by a successful retry, and still end as a success.

flowchart TD
    S["Span starts"] --> W["Work is done"]
    W --> X{"Did an error occur?"}
    X -->|No| OK["Leave status as unset or ok"]
    X -->|Yes| H{"Was the error handled and the operation still succeeded?"}
    H -->|Yes| EV["Record exception event only"]
    H -->|No| ST["Record exception event and set status to error"]
    OK --> END["End the span"]
    EV --> END
    ST --> END
    

A warning about duplicate recording

If every layer of your code catches an exception, records it on the span, and throws it again, the same error may appear many times on the same span or on several parent spans. Pick one clear place to record it, usually where it is finally handled, so each failure is recorded once.

Exceptions are only the beginning. Here are the kinds of moments that deserve an event.

A good name makes events searchable. A bad name makes them useless.

An event’s name is a label, like a category. It should describe the kind of thing that happened, not the specific details. Details belong in attributes. If you build the name from changing values, such as an order number, every event gets a unique name, and you can no longer group or count them.

Practical naming habits

• Use lowercase words joined by dots or underscores, and use the same style across your whole company.
• Group related events with a shared first word, such as “cache” or “retry”.
• Put changing values, like attempt numbers and cache names, into attributes.
• Reuse the standard names from OpenTelemetry’s conventions when one exists, such as the exception event. Tools know how to display them.
• Keep a short shared list of the events your teams use, so that everyone speaks the same language.

Follow one event from the moment it is created to the moment you read it.

Timestamps and a small tip

Most of the time, you let the SDK stamp the current time. Sometimes you may want to record an event that happened a little earlier, for example when processing a log of earlier activity. You can supply your own time, but keep it inside the span’s own start and end window. Events that fall outside the span’s lifetime can confuse viewers and make timelines look broken.

These two are close cousins, and many people mix them up.

A useful rule of thumb: if the information only makes sense inside a traced operation and you want to see it on the timeline, an event is a good fit. If it must be kept even when traces are sampled out, or if it needs a severity level and long-term search, a log record is a better fit. Many teams use both, with logs carrying trace and span IDs so that the two can be connected.

Every tool has strengths and limits. Let us see them honestly.

Key trade-offs

Events often carry error messages, and error messages often carry surprises.

Events are data, so they need watching too.

• Dropped events: if spans report dropped events, you are adding too many for the limit. Find the code that adds them in loops.
• Spans with huge event counts: sort spans by event count to find the worst ones. These often reveal a loop that adds an event on each iteration.
• Exception types: group exception events by type to see which errors are most common, including handled ones that never raise an alarm.
• Handled versus failed: compare spans that have exception events and an error status with those that have exception events and a normal status. A large second group may show hidden weaknesses.
• Missing expected events: if a key event, such as “payment.authorised”, stops appearing, a code path may have changed.
• Memory and span size: large spans can slow exporters and raise costs. Track average span size and the share of oversized spans.
• Counts that matter: if you need a reliable count or rate, such as cache misses per minute, use a metric rather than counting events, since events vanish with unsampled traces.

Habits worth copying and traps worth avoiding.

Good design patterns

Anti-patterns

A practical checklist for events that stay useful.

Best practices

• Use events for moments, not durations. If it takes time, make it a span.
• Keep names stable and general. Put the details in attributes.
• Use the standard exception event. It is understood by most tools and gives consistent error views.
• Set status deliberately. Record exceptions as events, and mark the span as failed only when the whole operation failed.
• Record each failure once. Choose one clear place to record exceptions.
• Be sparing. A few well-chosen events beat hundreds of noisy ones.
• Add events while the span is open. Events added after the span ends are ignored.
• Review error messages for secrets. Clean them in code or in the Collector.
• Use metrics for counts. Do not depend on events for rates and totals.
• Document your event names. A shared list helps every team name things the same way.

Common mistakes beginners make

• Thinking an event has a duration. It has only one time.
• Assuming that recording an exception marks the span as failed.
• Expecting events to appear in the backend even when the trace was not sampled.
• Adding events to a span after it has ended.
• Confusing span events with log records. They are stored and sampled differently.
• Putting personal data or full error details into event attributes without checking.
• Forgetting that events are held in memory until the span ends.

How events appear in tools and platforms you may already know.

A typical journey for a growing team

Quick answers to questions beginners ask most.

If you remember only a few things, remember these.