An exporter looks small from the outside. Inside, it carries a surprising amount of responsibility.

What telemetry is, in plain words

Telemetry is the data that software produces about itself: what it did, how long it took, and what went wrong. OpenTelemetry names three main kinds of it, called signals. Traces follow a single request across many services. Metrics are numbers measured over time, like requests per second. Logs are time-stamped text records of events.

Producing this data is only half the story. It is useless until it reaches somewhere that stores it, searches it, and draws charts from it. That somewhere is called a backend. It might be a commercial monitoring product, an open-source database, a cloud service, a message queue, or even a plain file on a disk.

What an exporter does

An exporter takes telemetry that OpenTelemetry has already collected and sends it out of the process toward a backend. It has four jobs. It converts the data into whatever format the destination understands. It delivers the data over a network or to storage. It copes with failure, because networks drop packets and servers go down. And it behaves politely, which means it must never crash or freeze the application it serves.

Why one bad exporter hurts everyone

Exporters run next to your business code. If an exporter is slow, the application can slow down. If it leaks memory, the application can run out of memory. If it throws an uncaught error, the application can crash. If it silently drops data, nobody notices until the day an outage needs the missing traces. Because the failure mode is so quiet, exporters deserve more care than their small size suggests.

An exporter never works alone. To build one, you need a clear picture of the components that feed it and the rules they follow.

The journey of one span

Follow a single piece of data from birth to delivery. Application code starts a span through the API. The SDK applies a sampler (the component that decides whether to keep the data). When the span ends, the SDK hands it to a span processor. The processor collects spans into a batch, and when the batch is ready or a timer fires, it hands the batch to the exporter. The exporter converts it, sends it, and reports success or failure back to the processor.

flowchart LR
    A["Application code using the API"] --> B["SDK creates spans"]
    B --> C["Sampler keeps or drops"]
    C --> D["Batch span processor queues spans"]
    D --> E["Exporter converts and sends"]
    E --> F["Backend stores and displays"]

Two directions: push and pull

Exporters deliver data in one of two styles. A push exporter sends data to the backend on its own schedule, like a delivery driver who rings your bell. A pull exporter waits quietly and exposes data on a web address so that the backend can come and fetch it, like a mailbox that the postman never visits but you check yourself. Most trace and log exporters push. Some metric systems, with Prometheus the best-known example, pull.

Why the three signals need different exporters

Each signal has its own interface because the data behaves differently. Spans are individual events that finish at unpredictable times, so they flow continuously in batches. Metrics are periodic snapshots, and the exporter must understand whether each number is a running total or a change since the last snapshot. Logs are high-volume text records that carry severity and optional links to traces. A custom exporter that supports all three has three separate paths, and many real exporters begin with just one.

The first design decision is the most important: write the exporter inside the application’s SDK, or inside the Collector.

Option one: an SDK exporter

An SDK exporter is a library, written in the same language as your application, that plugs into the SDK. A Java service needs a Java exporter, a Python service needs a Python one, and a Go service needs a Go one. It runs inside your application process and shares its memory and CPU. The application owner chooses it through configuration or code.

Option two: a Collector exporter

A Collector exporter is a component inside the OpenTelemetry Collector, written in Go. Applications send standard OTLP data to the Collector, and the exporter inside the Collector converts it for the backend. All your services, in any language, benefit from one implementation. The Collector also provides ready-made helpers for queuing, retrying, and timeouts, so you write far less reliability code.

flowchart TB
    subgraph APP["Inside the application process"]
        A1["Application code"] --> A2["SDK with batch processor"]
        A2 --> A3["SDK exporter option"]
    end
    subgraph COL["Inside the Collector process"]
        C1["Receiver accepts OTLP"] --> C2["Processors: memory limiter, batch"]
        C2 --> C3["Collector exporter with queue and retry"]
    end
    A2 -->|"OTLP over network"| C1
    A3 -->|"direct delivery"| B["Backend"]
    C3 --> B

Comparing the two homes

A middle path: the SDK exporter that talks to a Collector

In practice, most applications use the standard OTLP exporter inside the SDK and let a nearby Collector do the heavy lifting. This is not a custom exporter at all, and that is the point. The application stays simple and uses a stable, well-tested component, while the unusual work happens in one place you control. When people say “we need a custom exporter”, this middle path is often what they actually need.

The cheapest exporter to maintain is the one you never write. Check these alternatives before you start.

The questions to ask first

Writing an exporter creates a long-term commitment. It must be tested, secured, updated when OpenTelemetry changes, and supported when it breaks at three in the morning. Before committing, work through a short list of questions.

flowchart TD
    Q1{"Does the backend already accept OTLP?"}
    Q1 -- "Yes" --> R1["Use the standard OTLP exporter"]
    Q1 -- "No" --> Q2{"Does a ready-made exporter exist for it?"}
    Q2 -- "Yes" --> R2["Use the existing exporter from the Collector distributions"]
    Q2 -- "No" --> Q3{"Can a small gateway translate OTLP for the backend?"}
    Q3 -- "Yes" --> R3["Run a gateway service that receives OTLP and forwards"]
    Q3 -- "No" --> Q4{"Can a Collector be deployed?"}
    Q4 -- "Yes" --> R4["Build a custom Collector exporter"]
    Q4 -- "No" --> R5["Build a custom SDK exporter"]

Common alternatives

Good reasons to build your own

• Your backend has a proprietary or internal protocol that nobody else uses.
• You must apply special rules during export, such as splitting data by tenant into separate destinations, or encrypting fields in a way only your company defines.
• You need guaranteed delivery behavior that standard exporters do not give, such as writing to a company-specific durable queue.
• You need to feed a legacy system whose format cannot be changed.
• You are a vendor or open-source project that wants to offer first-class support for your product.

Weak reasons that cause regret

• “The existing exporter is missing one small feature.” Often a processor or a small contribution to the existing exporter is cheaper.
• “We want to learn.” That is a fine reason for a side project but a poor one for production.
• “We do not trust open-source components.” A custom exporter has a larger risk surface, because you must handle every edge case yourself.

The true cost of owning an exporter

The first version of an exporter is the cheap part. The cost that follows is the one that surprises teams. Someone must answer questions from other teams, fix bugs found in production, rebuild the component for each Collector release, review security notices for every library it uses, and keep the knowledge alive when the original author changes jobs. A rough rule is to expect the yearly upkeep to equal a meaningful share of the first build, and to plan that time on a named person’s calendar. If no one can promise that time, reconsider the gateway or file-based alternatives, even if they feel less elegant.

It also helps to write a one-page proposal before building. State the problem, the alternatives you rejected and why, the owner, the expected data volume, and the exit plan if the destination is retired. Teams that write this page often discover that a simpler option exists. Teams that still decide to build have a shared reference that keeps the project focused.

Every exporter, in every language, promises the same small set of behaviors. Learning them is the foundation of everything that follows.

The three promises

An exporter is defined by three operations. Export receives a batch of telemetry and tries to deliver it, then reports whether it worked. ForceFlush means “send whatever you are holding right now”, which matters when a program is about to freeze or exit. Shutdown means “finish your work, release your resources, and stop for good”. Metric exporters have a slightly different shape, because a reader gathers numbers and hands them over, but the same three ideas apply.

The rules the specification sets

The OpenTelemetry specification, the written rulebook that all languages follow, sets several expectations that a custom exporter must respect.

• Never block forever. Export must finish, or give up, within a reasonable time limit. A hung network call must not freeze the application.
• One call at a time. For SDK exporters, the same exporter object will not receive two export calls at once, so the exporter need not be built for parallel exports unless it chooses to be.
• Shut down once. Shutdown is called at most once. After it, any new export request should fail fast and politely, not crash.
• Report honestly. Success must mean the data was accepted. Failure must be reported, so that callers can log it or retry.
• Do not throw into application code. Errors inside an exporter must be caught and handled internally. Telemetry problems must never turn into business problems.

The lifecycle of an exporter

stateDiagram-v2
    [*] --> Created
    Created --> Started: configuration validated and connections opened
    Started --> Exporting: batch arrives
    Exporting --> Started: delivery succeeded
    Exporting --> Retrying: delivery failed but may succeed later
    Retrying --> Exporting: wait finished, try again
    Retrying --> Started: gave up and dropped the batch
    Started --> ShuttingDown: shutdown requested
    Exporting --> ShuttingDown: shutdown requested
    ShuttingDown --> Stopped: queue drained and resources released
    Stopped --> [*]

Read this diagram as a story. The exporter is created from a configuration, which is checked for mistakes before anything starts. It starts, which usually means opening connections. It waits for batches. Each batch moves it into an exporting state. If the delivery fails in a way that might fix itself, it enters a retrying state and waits before trying again, and if it runs out of patience, it gives up on that batch and goes back to waiting. At any time a shutdown may arrive, and the exporter must finish or abandon its work in an orderly way before it stops.

You cannot convert what you do not understand. This chapter walks through what each signal looks like when it reaches your exporter.

The wrapping: resource, scope, and records

Telemetry arrives in layers, like nested boxes. The outer box is the resource, which describes the producer: service name, version, host, container, cloud account. Inside it sit one or more scope boxes, one per instrumentation library. Inside each scope sit the actual records: spans, metric data points, or log records. Your exporter must decide where to place the resource and scope information in the target format, because dropping them loses the answer to “which service sent this?”.

Traces: spans and their parts

Metrics: five shapes and two time styles

Metrics have more variety. A gauge is a value at a moment, like current memory use. A sum adds up over time, like total requests served, and may be monotonic (only ever rising) or not. A histogram counts how many measurements fell into each range of values, such as response times grouped into buckets. An exponential histogram is a more compact histogram whose buckets grow by a constant factor. A summary is an older form that stores precomputed percentiles.

Sums and histograms also come with a time style called temporality. With cumulative temporality, each report shows the total since the program started. With delta temporality, each report shows only the change since the previous report. Different backends prefer different styles, and an exporter for a delta-based backend that receives cumulative data must either request the style it wants (SDK metric exporters can state a preference) or convert it, which requires remembering previous values.

Logs: records with optional trace links

A log record has a time, a severity (such as debug, info, warn, error), a body (often text, but it can be structured), attributes, and optional trace and span IDs that tie the message to a request. Logs arrive in huge volumes, so exporters for logs are the most sensitive to speed and memory use.

What counts as a successful export

Success deserves a precise definition, because the word is easy to abuse. For a network exporter, success should mean the destination acknowledged the data, not merely that the bytes left the machine. For a file exporter, success means the data was written and, if durability matters, flushed to disk. For a queue exporter, success means the broker confirmed receipt at the level of safety you configured. Write this definition into your documentation. Two people who disagree on what success means will argue for hours about “lost” data that was never really delivered.

Also decide what failure means for the caller. In an SDK, the batch processor only needs a success or failure result, and it logs or counts failures. In a Collector, a returned error flows back toward the receiver, which can pass it to the sender. A permanent error should be marked as permanent so the pipeline does not waste effort on retries elsewhere.

Conversion is the heart of a custom exporter. It is also where most of the subtle bugs live.

The mapping exercise

Start on paper, before writing code. Draw two columns. On the left, list every field in the OpenTelemetry data model. On the right, list every field the destination accepts. Draw a line from each left item to its right-hand partner. Then mark the items with no partner. Those are your design decisions: you must choose whether to drop the item, store it in a generic “extra” field, or ask the destination’s owner for a new field.

flowchart LR
    subgraph IN["OTLP input"]
        R["Resource attributes"]
        S["Scope name and version"]
        P["Span fields: IDs, times, kind, status"]
        A["Span attributes, events, links"]
    end
    subgraph MAP["Exporter mapping layer"]
        M1["Rename and retype fields"]
        M2["Convert times and IDs"]
        M3["Handle missing and oversized values"]
    end
    subgraph OUT["Target record"]
        T1["Native fields"]
        T2["Tags or labels"]
        T3["Extra or overflow field"]
    end
    R --> M1
    S --> M1
    P --> M2
    A --> M3
    M1 --> T2
    M2 --> T1
    M3 --> T3

The usual mismatches

Keep the mapping pure

The best design makes the mapping a pure function: given the same input, it always returns the same output, with no network calls, no clocks, and no hidden state. Pure mapping code is fast, easy to test with hundreds of sample inputs, and safe to run on many threads at once. Keep the network code in a separate layer that only receives already-converted records.

Lossy conversion is a feature to document

Every mapping loses something. The right response is to be honest about it. Write a short mapping document that lists what is preserved, what is changed, and what is dropped, and publish a counter in the exporter’s own metrics for each kind of loss. A reader who sees “attributes truncated: 4,200 in the last hour” can act. A reader who sees nothing assumes the data is perfect.

After conversion comes delivery. How you package and move the bytes decides your speed, your cost, and your failure behavior.

Serialization: what the bytes look like

Serialization means turning structured data into a stream of bytes. JSON is readable by humans and easy to debug, but it is large and slow to produce. Protocol Buffers are compact and fast and carry an enforced structure. Avro and Parquet are column-friendly formats that suit data lakes, where files are stored for later analysis. Pick the one your destination expects, and when you have a choice, prefer the compact binary form for high-volume paths and the readable form for low-volume or debugging paths.

Transport: how the bytes travel

Compression, security, and connection care

Compression such as gzip or zstd can shrink telemetry by a large factor, because spans and logs are repetitive text. It saves network cost and transfer time at the price of CPU. Measure both. For security, always support encrypted connections (TLS), allow mutual authentication where the backend requires it, and accept secrets such as API keys through configuration that supports environment variables or secret files rather than plain text in a config file.

Reuse connections. Opening a new connection for every batch wastes time on handshakes. A connection pool, a limit on simultaneous requests, and sensible timeouts for connecting, sending, and waiting for a reply will prevent most production surprises. Remember that a load balancer in front of the backend may close idle connections, so the exporter must handle a reused connection that was closed behind its back.

Choosing between HTTP and gRPC

Many teams ask which to choose. HTTP is the safer default when you control the client but not the network: it passes through nearly every proxy, is simple to debug with common tools, and keeps working in locked-down environments. gRPC suits traffic between your own services, where its efficient encoding, built-in deadlines, and ability to carry many calls over one connection pay off. It can be harder where load balancers or proxies do not fully support its protocol. If your destination offers both, begin with the one your operators already know how to monitor and troubleshoot, because the best protocol is the one your team can debug at midnight.

Sending one item at a time is slow and wasteful. Holding too many items in memory is dangerous. Good exporters live between those extremes.

Why batching exists

Every network call has a fixed cost: connection setup, headers, waiting for a reply. Sending one span per call would make that cost dominate. Batching groups many items into one call, which divides the fixed cost across all of them. The SDK’s batch span processor does this for you, keeping a bounded queue, sending when a timer fires or the batch is full, and dropping the oldest data if the queue overflows. Its documented defaults are a queue of 2,048 items, a batch size of up to 512, a five-second delay, and a thirty-second export timeout.

Queues are shock absorbers

A queue is a waiting line between the producer and the exporter. It absorbs short bursts and slow moments. But a queue must be bounded, meaning it has a fixed maximum size. An unbounded queue turns a slow backend into a memory leak, and the process eventually dies. When the queue is full, something has to give, and the exporter must choose what: drop the newest items, drop the oldest items, or push back on the producer.

Persistent queues

An in-memory queue loses its contents when the process restarts. For critical data, the Collector offers a persistent queue that stores waiting batches on disk through a storage extension. This survives restarts and short outages, at the cost of disk space, slower throughput, and the need to manage that disk. It is worth the trade-off only when losing a few minutes of data would be costly.

Backpressure

Backpressure is the way a slow component tells the components before it to slow down. In a Collector, if an exporter’s queue is full, the exporter returns an error to the pipeline. The receiver may then tell the sender to retry later. This works well when senders can wait. It works badly when senders cannot, so many designs put a memory limiter at the front of the Collector to refuse data politely before memory runs out.

Batch size and timing in practice

Two settings shape how batches behave: the largest batch allowed, and the longest time an item may wait before being sent. A large batch limit with a short wait gives low delay at quiet times and efficient batches at busy times. A small batch limit with a long wait gives the opposite, with items sitting around for no reason. Start with limits that match the destination’s documented maximum request size, keep the wait to a few seconds so traces stay fresh, and adjust after you have seen real traffic.

Remember also that batch size is measured in two ways, by count and by bytes. A batch of one thousand tiny spans and a batch of one thousand spans with huge attributes have very different sizes. If the destination limits request size in bytes, measure bytes, and split a batch before sending when it grows too big. Finding out about a size limit from a rejection is a poor way to learn it.

Networks fail all the time. An exporter is judged by how it behaves in the minutes after something goes wrong.

Retryable and permanent errors

Not every failure deserves a second attempt. A retryable error is temporary: the server was busy, a connection dropped, a request timed out. Trying again later may succeed. A permanent error will never succeed no matter how often you repeat it: the credentials are wrong, the request is malformed, the destination refuses the data. Retrying a permanent error wastes resources and delays everything behind it.

Backoff and jitter

Exponential backoff means that each failed attempt waits longer than the last, for example five seconds, then seven and a half, then eleven, up to a ceiling. This gives a struggling server room to recover. Jitter adds a small random amount to each wait. Without jitter, a thousand exporters that failed at the same moment would all retry at the same moment, and hit the recovering server like a stampede. The Collector’s standard retry helper uses defaults of roughly a five-second first wait, growing toward thirty seconds, and gives up after about five minutes, though you should check the settings of the version you use.

sequenceDiagram
    participant Q as Export Queue
    participant X as Custom Exporter
    participant B as Backend
    Q->>X: Batch 1
    X->>B: Send batch 1
    B-->>X: 503 service unavailable, retry after 10 seconds
    Note over X: Classify as retryable, wait with backoff and jitter
    X->>B: Send batch 1 again
    B-->>X: 200 accepted
    X-->>Q: Success, remove batch 1
    Q->>X: Batch 2
    X->>B: Send batch 2
    B-->>X: 400 invalid payload
    Note over X: Classify as permanent, do not retry
    X-->>Q: Failure, drop batch 2 and count it

Partial success

Sometimes a backend accepts most of a batch and rejects a few records. The OTLP protocol includes a field for this, called partial success, which reports how many items were rejected and why. An exporter that treats this as total failure and retries the whole batch will resend items that were already accepted, creating duplicates. The right behavior is to treat the batch as delivered, count and log the rejected items, and never retry what the server already refused.

Duplicates and ordering

Retries can cause duplicates. If a request reached the server but the reply was lost, the exporter will send the same data again. This is the classic trade-off between at-least-once delivery (nothing is lost, duplicates are possible) and at-most-once delivery (no duplicates, losses are possible). Telemetry usually chooses at-least-once, and relies on identifiers such as span IDs so the backend can ignore repeats. If your destination cannot de-duplicate, design for that, perhaps by attaching a batch identifier that lets the receiver recognize a resend.

Also, do not promise order. Batches may be retried, sent in parallel, or arrive late. Backends that need time order must sort by the timestamps inside the data.

The poison batch

One malformed record can make an entire batch fail permanently, again and again. A careful exporter limits the damage: validate during conversion, skip and count records that cannot be converted, and if the server says the whole batch is invalid, optionally split it in half and retry each half so that the single bad record is isolated and the rest is saved.

Exporters sit on a hot path where thousands of items move every second. Small mistakes in sharing data between threads become large production incidents.

Who calls you, and how often

Inside an SDK, the batch processor usually calls the exporter from a single background worker, one export at a time. Inside the Collector, however, the pipeline may call your exporter from many threads at once, because many receivers are feeding it. A Collector exporter must therefore be safe for concurrent use, meaning that two calls running at the same time must not corrupt shared state, such as a connection pool, a counter, or a cache.

Do not change what you were given

In a Collector, one batch of data can be delivered to several exporters at the same time, for example one that sends to a vendor and another that writes to a file. They all look at the same underlying data. If your exporter edits that data in place, such as removing an attribute or rewriting a name, it silently changes what the other exporters see. The rule is simple: treat input as read-only. If you must change something, make a copy first, or declare to the Collector that your component modifies data so it can protect the others.

Memory and speed

At a million items per second, tiny waste adds up. Creating a new string for every attribute, a new buffer for every batch, or a new connection for every call will make the garbage collector (the part of the runtime that reclaims unused memory) work hard and cause pauses. Reuse buffers carefully, preallocate collections to the expected size, and avoid converting data more than once. Measure with a profiler instead of guessing, because the actual hot spot is often a surprise, such as a logging call inside a loop.

Cancellation and shutdown

Every long operation should respect a cancellation signal or deadline passed from above. When shutdown arrives, background workers must stop, in-flight requests must be cancelled or allowed to finish within a time limit, and queued data must be flushed or counted as lost. A common bug is a worker that never stops, so the process refuses to exit, or a worker that outlives shutdown and writes into a closed connection.

The steps are the same in every language, even though the names of the interfaces differ. This chapter describes them in order, without tying you to one language.

Metric exporters have two extra decisions

A metric exporter must tell the SDK which temporality it prefers (cumulative or delta) for each kind of instrument, and may choose how histograms are aggregated. It is handed data by a reader, usually a periodic one that collects numbers on a timer (the specification default is every sixty seconds). A pull-style exporter, such as one that serves a scrape endpoint, is built as a reader that holds the latest numbers and answers requests from the backend.

The Collector route needs more scaffolding at the beginning and far less reliability code afterwards.

The three parts of every Collector component

Let the helper do the hard work

The Collector provides an exporter helper library. You give it a single function that sends one batch, and it wraps that function with a configurable timeout, a retry policy with backoff, an optional sending queue (memory or persistent), and standard metrics about how many items were sent or failed. This replaces hundreds of lines of careful code, and it gives users the same familiar settings they know from every other exporter. Reusing the standard sub-settings for HTTP clients, gRPC clients, encryption, and secret values also gives your exporter the same options users already understand.

Steps to build

How the helper wraps your send function

It helps to picture the layers. When data arrives, the helper first places it in the sending queue, if you enabled one. A worker then takes a batch from the queue and applies the timeout, so a stuck call cannot last forever. If your send function returns a retryable error, the retry layer waits and calls it again. If it returns a permanent error, the batch is dropped and counted. Only success removes the batch for good. Each layer also updates the standard metrics, so users see familiar numbers without you writing a line of counting code.

This layering means your send function should be small and honest. It should convert, send once, and return a clear result. It should not contain its own retry loop, its own queue, or its own timer, because those would fight with the helper’s versions and produce confusing behavior, such as retries inside retries that multiply the wait time.

Configuration is the user interface of an exporter. People will forgive a missing feature faster than a confusing setting that causes an outage.

Principles of good configuration

Changing configuration safely

Once people depend on a setting, renaming or removing it breaks their deployments. Treat configuration like a public interface: add new settings freely, deprecate old ones with a warning and a long notice period, and remove them only in a major version. Include a changelog that lists every configuration change in plain words.

Settings that deserve special attention

A handful of settings cause most production trouble, so give them extra thought. The timeout must be shorter than any timeout upstream, or callers will give up before the exporter does, and work will be wasted. The retry limit must fit inside the queue’s capacity, because long retries on many batches fill the queue quickly. The queue size must match the memory the process really has. The concurrency limit must stay within what the destination can accept. And the endpoint must be validated, because a typo that points to the wrong place is one of the quietest, costliest mistakes. Document how these settings relate to each other, since changing one in isolation often breaks another.

A short table of recommended starting values for small, medium, and large deployments gives new users a safe place to begin and sparks fewer support questions than a long list of unexplained knobs.

An exporter moves your monitoring data, so who monitors the exporter? It must report on itself, or failures stay invisible.

The numbers every exporter should publish

If you use the Collector’s exporter helper, several of these appear automatically, with names that start with the exporter’s identity. Add your own counters for conversion losses and for anything unique to your destination. Choose labels with care: reasons and signal types are fine, but customer names or trace IDs would create too many distinct series.

Logs and errors

Write logs for events people must act on: first connection failure, credentials rejected, entering a retry loop, giving up on a batch, and recovering. Do not log every successful export, since that creates volume without value. Do not print telemetry contents or secrets in log lines. And rate-limit repeated messages, so that a backend outage does not fill the disk with a million identical complaints.

Health and readiness

Expose a way to ask “can you currently reach the destination?”. In the Collector, health checks can report on pipeline status. For critical paths, add a canary: a synthetic piece of telemetry sent every minute with a known name, and an alert if it does not appear at the destination within a few minutes. This tests the entire chain end to end, including parts your own metrics cannot see.

Exporters fail in ways that ordinary tests never touch. A good test plan deliberately breaks things.

Test the sad paths first

Happy-path tests are easy, and they pass on the first day. The bugs that hurt arrive on the sad paths: the backend slows to a crawl, then returns errors for ten minutes, then recovers with a flood. Write that scenario as a test. Check that memory stays bounded, that the queue drops data in the rule you documented, that retries stop, and that the exporter returns to normal speed after recovery.

Test shutdown as a feature

Shutdown bugs are common and expensive. Test shutdown with an empty queue, a full queue, a delivery in progress, a hung backend, and a second shutdown call. Confirm that each case finishes within the time limit and leaves no leftover workers behind.

Contract tests and a compatibility matrix

Your exporter has two partners: the OpenTelemetry side that gives it data, and the destination that receives it. Each partner can change independently. A contract test captures what you expect from a partner and checks it automatically. On the destination side, keep a small suite that sends known records to a staging copy of the destination and verifies that they appear with the right fields. On the OpenTelemetry side, keep sample inputs captured from real traffic and confirm the exporter still handles them after each library upgrade.

Publish a compatibility matrix, a simple table that lists which versions of the SDK or Collector the exporter has been tested against, and which destination versions it supports. Users should never have to guess whether an upgrade is safe. A short table with test dates answers the question and shows that the project is alive.

Telemetry often carries addresses, user identifiers, and error messages. An exporter is a doorway through which that data leaves your network, so it must be built like one.

Protect the connection

Encrypt every connection with TLS, verify the server’s certificate by default, and offer mutual authentication when the destination requires it. Never ship a configuration option that disables certificate checking without a loud warning, and never make it the default. Support rotating credentials without a restart where possible, because long-lived secrets are a common source of breaches.

Protect the secrets

Credentials should be read from environment variables or files and stored in types that hide their value when printed. Review every log line and error message to confirm that headers, tokens, and URLs with embedded passwords cannot leak. A frequent mistake is logging the full failing request for debugging, which includes the authorization header.

Protect the data

The exporter may be the last chance to remove sensitive content, but it should not be the only one. A better design redacts early, with a processor placed before the exporter, so every destination benefits. Inside the exporter, avoid keeping copies of data longer than needed, never write telemetry to temporary files unless the user asked for it, and if you use a persistent queue, remember that the data on that disk deserves the same protection as a database.

Protect the destination and the neighbors

A buggy exporter can overwhelm a backend with too many connections or huge payloads. Limit concurrency, cap batch size, and honor wait hints. Validate the endpoint setting, so a typo cannot send data to an unintended address. In shared systems, include tenant identity in each request in a way the backend can verify, so one team’s data never lands in another team’s account.

Compliance and data residency

Some data may not legally leave a country or region, and some data must be kept for a minimum time. An exporter is where these rules meet reality. If your destination lives in another region, check whether the attributes you send include personal information that regulations restrict. Provide settings that let operators choose a regional endpoint, and make sure the exporter never silently falls back to a different region when the preferred one fails. Where rules differ by tenant, route each tenant’s data to its own destination and keep an audit trail of what was sent where.

It is also wise to document, in plain language, exactly which fields your exporter sends. When a security reviewer asks “what leaves our network?”, a one-page answer saves weeks.

A custom exporter that works at one hundred spans per second may fall over at one hundred thousand. Plan for scale from the start.

Do the arithmetic

Suppose a Collector fleet must export 500,000 spans per second. If each span is about 500 bytes, that is 250 megabytes per second, or 20 terabytes per day before compression. If your exporter’s conversion step costs ten microseconds per span, that is five full CPU cores for conversion alone, before any network work. If each outgoing request carries 1,000 spans, the exporter makes 500 requests per second, and with an average of 200 milliseconds per request, about 100 requests are in flight at once. These numbers tell you how many connections, threads, and machines you need.

The levers you can pull

Find the bottleneck before changing anything

Measure where time goes: conversion, compression, waiting for the network, or waiting for the destination’s reply. If the destination is slow, speeding up your code changes nothing, and adding more senders may make things worse. If the CPU is saturated, more senders also do nothing. A simple profile run during a load test usually points to a single dominant cost, and fixing that one cost beats a dozen guesses.

Sending the same data to several destinations

A Collector pipeline can list more than one exporter, and the data is then delivered to each. This is called fan-out. It is handy for migrations, for example sending to an old and a new backend in parallel, but it has a cost: the slowest exporter can hold back the pipeline if queues are not isolated. Give each exporter its own queue and retry settings, so a failing destination only hurts itself. Also watch memory: with three exporters, each holding its own queue, the memory budget triples.

If destinations need different data, use separate pipelines with their own processors, instead of one pipeline with exporters that each filter in their own way. That keeps each path simple to read and each rule in a single, visible place.

An exporter is finished only when other people can install it, trust it, and upgrade it.

Shipping an SDK exporter

Publish it as a normal library in your language’s package registry, with a clear version number, a short guide, and a list of the SDK versions it supports. Keep its dependencies small, because every extra library is a possible conflict inside someone else’s application. Follow semantic versioning: a breaking change means a new major version.

Shipping a Collector exporter

Collector components are compiled into the Collector program. A build pipeline reads a manifest listing the components, compiles a custom Collector, runs tests, and produces a container image. Teams then deploy that image like any other service.

flowchart LR
    A["Source repository with exporter code"] --> B["CI pipeline runs unit and integration tests"]
    B --> C["Collector Builder reads manifest and compiles custom Collector"]
    C --> D["Container image with version tag"]
    D --> E["Image registry"]
    E --> F["Staging environment with canary traffic"]
    F --> G["Production rollout in stages"]
    G --> H["Metrics and canary checks"]
    H -->|"healthy"| I["Full rollout"]
    H -->|"unhealthy"| J["Automatic rollback"]

Staying current

OpenTelemetry moves fast. The protocol, the data model, the naming rules for attributes (called semantic conventions), and the Collector’s own interfaces all change over time. Plan a recurring task to rebuild your exporter against each new Collector release, run the full test suite, and read the notes for renamed fields. A custom exporter that is not updated for a year will become a barrier that stops the whole Collector fleet from upgrading.

Stability levels and honesty

Collector components declare a stability level, ranging from development through alpha and beta to stable. Choose honestly. Marking an unproven exporter as stable invites users to depend on behavior that you may need to change. Start low, move up as tests and production use build confidence, and say plainly in the documentation which data and features are fully supported.

Contribute or keep private?

If your destination is a public product, consider contributing the exporter to the community repository for Collector components. You gain review from experienced maintainers and shared upkeep, and you give up some control over timing. If the destination is internal to your company, keep it private, but still follow the same conventions, so that a future colleague can understand it.

Documentation people actually need

A custom exporter lives or dies by its documentation. Write five short pages. The first is a quick start that gets a working example running in ten minutes. The second lists every setting, with its default, its allowed range, and an example. The third is the mapping document, which says how each field is converted and what is lost. The fourth is a troubleshooting guide organized by symptom, such as “no data appears” or “queue stays full”, with the metrics and log lines to check. The fifth is an operations page with sizing advice, upgrade steps, and the rollback procedure.

Keep examples realistic. A sample configuration copied by a hundred engineers will become the configuration of a hundred production systems, so show safe settings: encryption on, bounded queues, sensible timeouts, and secrets read from the environment.

Teams that have built many exporters keep arriving at the same good habits, and the same costly mistakes.

Patterns that work

Anti-patterns to avoid

Principles stick when you see them applied. Here is a complete design, from the first question to the rollout, for an imaginary company.

The situation

A logistics company, which we will call Northwind, runs an internal audit store built years ago. It accepts newline-separated JSON records over a secure web connection. It uses its own field names, expects times as whole seconds, accepts only text values in its tag fields, refuses any request larger than one megabyte, and needs a tenant identifier in a request header. No product supports it, and it will not be replaced for years. Northwind wants selected spans from its shipment services delivered to this store.

The decisions, one by one

What the exporter looks like when done

The Collector configuration lists one pipeline: a receiver that accepts OTLP, a memory limiter, a filter that keeps shipment spans, a batch step, and then the Northwind exporter. The exporter’s own settings are an endpoint, a secret token read from an environment variable, a timeout, standard retry and queue settings, and a switch for how to handle spans with missing tenant information. It publishes counters for spans sent, spans dropped by reason, and requests split because of size. A canary span with a fixed name is generated every minute, and an alert fires if it does not appear in the store.

The rollout

The team first tests against a fake store that can be told to fail in various ways. Then it runs a staging Collector with copy traffic. After a week without surprises, one production Collector instance sends real data while the old path stays in place, and the team compares record counts between the two. Only then does the old path switch off. At each step, a rollback means changing one line in a configuration file.

The Collector’s public repositories hold dozens of exporters. Reading them is the best free education on exporter design.

Every production exporter eventually meets these situations. Decide how yours will respond before they happen.

Scenario 1: The backend is down for an hour

Batches fail, retries run out, and the queue fills. With a memory queue, the oldest or newest data is dropped according to your rule, and counters show the loss. With a persistent queue, data waits on disk until the backend returns, if the disk is large enough. Recovery: when the backend returns, send the backlog gradually rather than all at once, or the recovering system will fall over again. Alert on queue depth and on the drop counter, and write down the exact window of lost data for anyone who needs it.

Scenario 2: The backend is slow, not down

This is more dangerous than a clean outage. Requests take twenty seconds instead of two hundred milliseconds, workers pile up, queues fill, and memory grows. Recovery: use strict timeouts, cap simultaneous requests, and treat a timeout as a retryable failure with backoff. Slowness should shrink throughput, not consume the process.

Scenario 3: Credentials expire or are revoked

Every request now returns an authorization error. Retrying changes nothing, and the queue fills with data that cannot be delivered. Recovery: detect the error class, raise a high-priority alert, and stop retrying aggressively. Where possible, support reloading credentials without a restart so that a fix takes effect quickly.

Scenario 4: The destination changes its format

The backend team renames a field or tightens validation, and suddenly part of your data is rejected as invalid. The exporter logs rejections and drops those records, which is correct, but the effect is a quiet gap. Recovery: watch the rejected-record counter closely, and keep contract tests that run against a staging copy of the backend, so a change is noticed before production.

Scenario 5: A certificate expires

Encrypted connections begin to fail on a date nobody remembered. Recovery: monitor certificate expiry dates in advance, support reloading certificates, and make the error message say clearly that the certificate is the problem.

Scenario 6: Clock differences

A host with a wrong clock produces spans dated hours in the past or in the future. Some backends reject timestamps outside a window. Recovery: count records rejected for time reasons, keep host clocks synchronized, and decide whether the exporter should clamp, drop, or pass such records unchanged.

Scenario 7: An upgrade breaks the build

A new Collector release changes an internal interface, and your component no longer compiles, which blocks security fixes for the whole fleet. Recovery: track upcoming releases, build against release candidates in a side pipeline, and keep the exporter small so adapting it takes hours rather than weeks.

An OpenTelemetry exporter is the courier that carries telemetry out of the pipeline and into the world. Building one is less about clever code and more about disciplined behavior: convert data faithfully, deliver it politely, fail safely, and report honestly about what happened. Begin by asking whether you need one at all, choose the Collector as its home unless something forces otherwise, separate mapping from transport, bound every queue, classify every error, test the sad paths, and ship through a canary. If you follow these habits, your exporter will be the quiet, trustworthy kind that nobody has to think about.