Beyond basic setup, these are the SDK-level configuration concepts that shape what data actually gets produced and how efficiently.

A Resource represents the entity producing telemetry — such as the service name, version, and the host or container it’s running on — and is attached to all telemetry emitted by that SDK instance, providing essential context for every trace, metric, and log.

Resource detectors automatically discover environment information (like cloud provider, Kubernetes pod name, or hostname) and populate it into the Resource, saving you from manually specifying infrastructure details.

A Simple Span Processor exports each span immediately as it finishes, which is easy to debug but inefficient, while a Batch Span Processor groups multiple spans together before exporting, which is far more efficient and the standard choice for production.

A sampler decides which traces are actually recorded and exported, versus dropped, based on a configured strategy — controlling this is essential for managing data volume in high-traffic systems.

TraceIdRatioBased sampling keeps a fixed percentage of traces (like 10%) based on a deterministic calculation from the trace ID, ensuring a consistent, statistically representative sample without coordinating across services.

ParentBased sampling makes a child span’s sampling decision follow whatever decision was already made for its parent span, ensuring a whole trace is either fully sampled or not, rather than having inconsistent gaps within it.

A metric view lets you customize how a specific instrument’s data is aggregated or renamed before export, such as changing histogram bucket boundaries or dropping unwanted attributes, without modifying the application code that created the metric.

Beyond a basic span, these concepts add the precision needed to interpret traces correctly in complex systems.

Span Kind categorizes what role a span plays — Client, Server, Producer, Consumer, or Internal — which helps observability tools correctly visualize the relationship between services, such as distinguishing a request sender from its receiver.

Span status indicates whether an operation completed successfully (Ok), failed (Error), or wasn’t explicitly set (Unset), giving a quick, standardized way to identify failed operations across a trace.

A span link connects a span to another span that isn’t a direct parent or child — for example, linking a batch-processing span back to each of the individual requests that were bundled into that batch.

Trace flags are part of the trace context and include information like whether the trace was sampled, allowing downstream services to know the sampling decision without recalculating it themselves.

Trace state carries additional vendor-specific information alongside the trace context, allowing different tracing systems to pass along their own metadata without interfering with the standard trace identifiers.

Semantic conventions are standardized names and formats for common attributes (like http.method or db.system), ensuring telemetry data is consistent and interoperable across different services, languages, and vendors.

Metrics have more nuance than “counter versus gauge” once you’re building instrumentation deliberately.

Synchronous instruments are updated directly within your application code at the moment something happens (like incrementing a counter), while asynchronous instruments use a callback function that’s invoked periodically to report a current value, useful for things you can only measure on demand.

An Observable Gauge is an asynchronous instrument that reports a current value through a callback — commonly used for things like current memory usage, which isn’t naturally tied to a specific code event.

Aggregation temporality defines whether a metric’s reported value represents the total since the start (cumulative) or just the change since the last report (delta) — different backends expect different temporality, and this affects how data must be exported.

An exemplar is a specific trace reference attached to a metric data point, letting you jump from an aggregated metric (like a latency spike in a histogram) directly to an actual example trace that contributed to it.

Cardinality refers to the number of unique attribute combinations a metric can have — high cardinality (like including a unique user ID as an attribute) can overwhelm a metrics backend, so it’s an important design constraint when choosing what attributes to attach.

Understanding exactly how trace context crosses process boundaries is essential for debugging broken traces.

W3C Trace Context is the standardized HTTP header format (traceparent and tracestate) for passing trace identifiers between services, ensuring different tools and languages can interoperate correctly.

B3 is an older propagation format originally created for Zipkin, still supported by OpenTelemetry for compatibility with existing systems that haven’t migrated to the W3C standard.

A composite propagator combines multiple propagation formats (like W3C and Baggage) so an application can correctly read and write several types of context information at once, especially useful during a migration between formats.

Injection is the act of writing trace context into outgoing request headers, while extraction is reading that context back out of incoming request headers — together, these are what actually connect spans across a network call.

Baggage travels alongside the trace context in request headers, meaning any custom key-value data you attach in one service becomes readable by every downstream service in the same request chain, unless explicitly stripped.

flowchart LR
    A["Service A"] -->|"Inject: traceparent header"| B["HTTP Request"]
    B -->|"Extract: traceparent header"| C["Service B"]
    C -->|"Continues same trace"| D["New Child Span"]
    

Configuring the Collector well is a core intermediate skill — this chapter covers the pieces of a real Collector config.

A Collector config defines receivers, processors, and exporters as named components, then wires them together into one or more pipelines under a service section, specifying exactly how data should flow through the Collector.

The memory limiter processor protects the Collector from running out of memory under heavy load by dropping data or refusing new data once memory usage crosses a configured threshold.

The attributes processor can add, update, delete, or rename attributes on telemetry data as it passes through the Collector, useful for enriching or cleaning up data centrally instead of in every application.

A Collector config can define separate pipelines for traces, metrics, and logs (or even multiple pipelines for the same signal type), each with its own combination of receivers, processors, and exporters.

Head-based sampling makes the sampling decision early, typically at the start of a trace, before knowing how the trace will ultimately turn out — simple and low-overhead, but can’t specifically prioritize interesting traces like errors.

Tail-based sampling waits until an entire trace has completed before deciding whether to keep it, allowing rules like “always keep traces containing an error,” at the cost of needing to temporarily buffer complete traces in the Collector.

The filter processor drops telemetry data matching specific conditions (like excluding health-check endpoint traces), reducing noise and volume before data reaches your backend.

Auto-instrumentation looks simple on the surface, but understanding how it works helps you troubleshoot and customize it.

An instrumentation agent (like the OpenTelemetry Java agent) attaches to a running application at startup and automatically modifies its behavior to generate telemetry, without requiring any changes to the application’s own source code.

Zero-code instrumentation is the broader term for adding observability to an application entirely through external configuration or agents, without touching the application’s source code at all.

Instrumentation scope identifies which specific library or instrumentation package generated a given piece of telemetry data, which is useful for filtering or disabling telemetry from a particular source.

Most auto-instrumentation agents allow specific instrumentation libraries to be disabled through configuration, useful when a particular library’s automatic tracing is too noisy or not needed for your use case.

The real power of OpenTelemetry comes from linking traces, metrics, and logs together — here’s how that actually happens.

When a metric data point is recorded (like a slow request duration in a histogram), the SDK can attach an exemplar containing the trace ID of the specific request, so a dashboard can let you click straight from a spike in the graph to the actual trace.

Log-trace correlation works by reading the active trace context at the moment a log is written and automatically injecting the trace ID and span ID into that log record, which most OpenTelemetry logging integrations handle automatically.

A unified observability pipeline routes traces, metrics, and logs through the same Collector infrastructure, applying consistent processing (like adding the same resource attributes) across all three signal types.

A spanmetrics connector generates metrics (like request rate and latency) directly from trace span data inside the Collector, letting you derive standard service-level metrics without instrumenting them separately in application code.

These concepts matter once you’re running OpenTelemetry-instrumented applications reliably in real environments.

These are the top-level SDK objects responsible for creating tracers, meters, and loggers respectively, each configured with its own processors, exporters, and resource information for that signal type.

A global provider is registered once and used implicitly throughout an application, simplifying instrumentation code, while a local provider is explicitly passed around, giving more control in complex applications or libraries that shouldn’t assume global state.

Since telemetry data is often batched before export, an application must explicitly flush and shut down its providers before exiting, or else recently generated data sitting in the batch buffer can be lost entirely.

OpenTelemetry defines a standard set of environment variables (like OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_SERVICE_NAME) that let you configure SDK behavior externally, without changing application code, which is especially useful across different deployment environments.

Head-based sampling is simpler and sufficient for many systems, but tail-based sampling is worth the added complexity when you specifically need to guarantee that error traces and slow requests are always kept, regardless of overall sample rate.

This is often a context propagation issue — check that trace context headers are being correctly injected on the outgoing call and extracted on the receiving service, since a missing propagator on either side breaks the trace chain.

Usually the SDK defaults align with common backend expectations, but it’s worth confirming — some backends specifically expect cumulative rather than delta temporality, and mismatches can produce confusing or incorrect metric values.

Keep unique, high-variability values (like user IDs or full URLs with query parameters) out of metric attributes entirely — reserve those for span attributes or logs instead, where high cardinality is far less costly.

Yes — they’re designed to work together. Auto-instrumentation typically creates a base tracer provider that manual instrumentation code can also use to add custom spans on top of the automatically generated ones.