To see why dead letter queues exist, start with what goes wrong without one.

Kafka is patient, and that is the problem

A Kafka topic is a long notebook of records. A consumer reads it one line at a time, in order, and remembers its place using a number called an offset. If the consumer cannot understand line 5,000, it cannot simply jump to line 5,001 without making a decision. If it retries line 5,000 forever, it never moves on. Everything behind that line waits.

A record that a consumer can never process is often called a poison pill. It is poisonous because it does not only fail itself. It blocks all the healthy records queued behind it.

What makes a record bad?

Records fail for many ordinary reasons. Nobody has to do anything wrong on purpose.

Three ways to react

A short list of words makes the rest of this article easy.

Kafka Connect

Kafka Connect is a tool that comes with Apache Kafka. Its job is to move data between Kafka and other systems, without you writing custom programs. You describe what you want in a small configuration, and Connect does the moving, the retrying, and the tracking of progress.

Source and sink

A source connector brings data into Kafka from somewhere else, for example from a database. A sink connector pushes data out of Kafka into somewhere else, for example into cloud storage or a search engine. This difference matters a lot in this article, because the dead letter queue feature is built for sinks.

Worker, connector, and task

• A worker is a running program (a process) that hosts connectors. Several workers together form a Connect cluster.
• A connector is the named job you configure, such as “copy orders into cloud storage”.
• A task is one unit of work inside a connector. A connector can split its job into several tasks that run in parallel, often one per group of partitions.

Converter

Kafka stores records as raw bytes. A converter turns those bytes into a structured object that the connector understands, and the other way around. Common converters read JSON, Avro, or Protobuf. There is one converter for the record key, one for the value, and one for headers.

Single message transform

A single message transform, or SMT, is a small step that changes each record on its way through, such as renaming a field, dropping a field, or routing the record to a different topic. SMTs can be chained one after another.

Offset and commit

An offset is a record’s position number in its partition. Connect commits offsets, meaning it records “I have finished up to here”, so a restart continues from the right place.

Here is the whole idea in one picture and a few sentences.

Without a dead letter queue, a sink connector reads records from a topic and writes them to a destination. If a record fails, the connector has only two options: stop, or skip. With a dead letter queue, there is a third path. The failed record is copied into a separate topic, and the connector goes on to the next record.

flowchart LR
    T["Source topic: orders"] --> C["Sink connector task"]
    C -->|"good records"| D[("Destination: cloud storage or database")]
    C -->|"bad records"| Q["Dead letter queue topic: dlq-orders-sink"]
    Q --> H["Human or repair job investigates"]
    

The picture shows a source topic feeding a sink connector task. Healthy records go through to the destination. Records that fail are diverted into the dead letter queue topic. From there, a person or a repair job can look at them, fix the cause, and decide what to do next.

Three key facts to remember

• It is opt-in. Nothing goes to a DLQ unless you name a topic in the configuration.
• It only works when errors are tolerated. You must also tell Connect to keep going after errors, otherwise the task still fails on the first bad record.
• It is just a topic. You can read it with any consumer, set retention on it, secure it, and monitor it like any other topic.

A record passes through several stations. A problem at any one of them can trigger error handling.

The stations of a sink connector

flowchart LR
    K["Kafka consumer reads raw bytes"] --> HC["Header converter"]
    HC --> KC["Key converter"]
    KC --> VC["Value converter"]
    VC --> SMT["Transformations (SMTs)"]
    SMT --> PUT["Connector writes to destination"]
    

A sink task first reads raw bytes with a normal Kafka consumer. Next come the converters for headers, key, and value. Then any transformations run in order. Finally the connector’s own code writes the record to the destination. Errors can occur at any of these stations.

Source connectors have stations too

A source connector reads from an outside system, builds a record, runs transformations, and then uses converters to turn the record into bytes for Kafka. Errors in the transformation and converter stations can be tolerated, but there is no built-in dead letter queue for sources. Later we look at the practical alternatives.

Before dead letter queues, let us understand what Connect does out of the box.

The setting called errors.tolerance

Connect has a setting named errors.tolerance. It has two values.

• none is the default. The first error that Connect cannot fix makes the task fail.
• all tells Connect to tolerate errors: skip the bad record and carry on.

What “fail” looks like

When fail-fast is the right choice

Stopping is not always bad. It is the safest option when every record matters absolutely and you want a human to look at the very first problem. Examples include early testing, brand-new pipelines, and financial feeds where a skipped record would be a serious issue. Fail-fast also protects you from a wider mistake, such as a wrong topic name, where every record is bad and continuing would just fill a DLQ with junk.

Setting tolerance to “all” keeps the pipeline moving, but the details are important.

What changes

With errors.tolerance set to all, a record that fails in a converter or transformation is no longer a reason to stop. Connect marks it as skipped and moves on to the next record. The task stays healthy, and lag stays low.

The silent data loss trap

Here is the dangerous part. If you set tolerance to “all” and do nothing else, the bad records are simply dropped. Nothing tells you. The task state says RUNNING. The dashboard looks green. Yet some percentage of your data never reached the destination.

Log the errors at least

Two settings help you see what is happening:

• errors.log.enable writes each error into the Connect worker’s log. The default is false.
• errors.log.include.messages adds details of the failing record to that log, such as its topic, partition, and offset. The default is false, because record contents can be sensitive.

Logging is better than nothing, but it has a weakness: a log line is not the record. You know something failed, but the original bytes are not preserved in a form that you can replay. That is exactly the gap that a dead letter queue fills.

Some errors are temporary. Connect can wait and try again before it decides a record is truly bad.

Why retry helps

Not every failure means the record is broken. Sometimes the record is fine and the world around it is having a bad moment. A converter may need to ask a schema registry for a schema, and the registry may be restarting. A destination may be briefly overloaded. If Connect gave up at once, it would send perfectly good records to the DLQ. Retrying gives temporary problems time to pass.

The two retry settings

This pattern of waiting longer after each failure is called backoff. It avoids hammering a struggling service with a flood of requests.

How retries and the DLQ work together

flowchart TD
    A["Record fails at a station"] --> B{"Retry time left?"}
    B -->|"yes"| C["Wait with backoff then try again"]
    C --> D{"Worked this time?"}
    D -->|"yes"| E["Continue normally"]
    D -->|"no"| B
    B -->|"no"| F{"Tolerance = all?"}
    F -->|"no"| G["Task fails"]
    F -->|"yes"| H{"DLQ configured?"}
    H -->|"yes"| I["Send record to DLQ and continue"]
    H -->|"no"| J["Skip record and continue"]
    

Follow the arrows. A failure first asks whether there is retry time left. If so, Connect waits and tries again. When time runs out, the next question is whether errors are tolerated. If not, the task fails. If yes, a configured DLQ receives the record; otherwise the record is skipped.

Now we can finally look closely at the tray.

What it is

The dead letter queue is a plain Kafka topic. When a sink connector cannot process a record and errors are tolerated, Connect uses a Kafka producer to write a copy of the original record into that topic. The key and value are saved as the raw bytes that were originally read, so nothing has been changed or lost by the failed conversion.

The settings that control it

A minimal mental picture of the configuration

A sink connector with a working dead letter queue needs just three ideas in its configuration: tolerate errors, name a DLQ topic, and add context headers. Optionally, you add logging and a small retry window. That is all. There is no extra service to install, and no separate product to buy.

A failed record with no explanation is a puzzle. Headers turn it into a clue.

What headers are

Every Kafka record can carry small labelled pieces of extra information called headers. They travel beside the key and value. When you turn on the context headers setting, Connect adds a set of headers to each record it writes into the DLQ. The names begin with __connect.errors, so they are easy to recognize.

What the headers tell you

Using the headers well

• Group by reason. Count how many DLQ records share the same exception message. Often one cause explains ninety percent of them.
• Group by origin. Count by original topic and partition to see whether only one producer or one partition is misbehaving.
• Trace back. Use the original topic, partition, and offset to look at the record in its source, along with the records around it.
• Watch the stage. A flood of converter errors usually means a format or schema problem. A flood of transformation errors points to a bug or an unexpected null in your own settings.

Let us see every part involved when a bad record is caught.

flowchart TD
    subgraph Kafka["Kafka Cluster"]
        SRC["Source topic"]
        DLQ["Dead letter topic"]
        CFG["Connect internal topics: configs, offsets, status"]
    end
    subgraph Worker["Connect Worker Process"]
        CON["Consumer"]
        CNV["Converters"]
        SMT["Transformations"]
        SNK["Sink task logic"]
        EH["Error handling: retry and tolerance"]
        DP["DLQ producer"]
    end
    EXT[("External system")]
    SRC --> CON
    CON --> CNV
    CNV --> SMT
    SMT --> SNK
    SNK --> EXT
    CNV -. "error" .-> EH
    SMT -. "error" .-> EH
    SNK -. "reported error" .-> EH
    EH --> DP
    DP --> DLQ
    

The diagram has two big boxes. The Kafka cluster holds the source topic, the dead letter topic, and Connect’s own internal topics. The worker process holds a consumer, the converters, the transformations, and the sink task logic. When an error appears at any station, it flows into the error handling layer. That layer decides whether to retry, fail, skip, or hand the record to a DLQ producer, which writes it into the dead letter topic.

Connect’s own internal topics

In distributed mode, Connect stores its own state in Kafka topics: one for connector configuration, one for source offsets, and one for status. These are separate from your DLQ. The DLQ is a data topic that you own, name, and manage, while the internal topics are managed by Connect.

Follow one broken record from arrival to rest in the DLQ.

Batches and the write step

Sink tasks usually process records in batches. Converters and transformations run on each record one at a time, so a bad record is removed individually while the rest of the batch is delivered. The connector’s write step then receives only the healthy records.

What happens to ordering?

When a record is diverted, the records after it still flow in their normal order. The bad record simply misses its turn. If you later repair and replay it, it arrives after newer records for the same key. For destinations that care about the newest value per key, replay carefully, and check whether a newer version has already been written.

Some of the most common problems happen at the last step, when the connector talks to the destination.

The original gap

When the dead letter queue feature first arrived, it covered errors in converters and transformations. But suppose your sink connector writes to a database and the database rejects one row because a text value is too long. That error happens inside the connector’s own code, after converters and transformations have succeeded. The basic error handling could not send such a record to the DLQ, because the connector did not tell Connect which record was at fault.

The fix: reporting bad records from the connector

Later versions of Kafka Connect added a feature for sink connector developers, known as the errant record reporter. It lets a connector say, “I could not process this particular record, and here is the reason.” Connect then applies the usual rules: retry, tolerate, log, and send the record to the DLQ. The connector can keep going with the rest of its batch.

flowchart LR
    B["Batch of 100 records"] --> S["Sink connector code"]
    S -->|"97 accepted"| DB[("Destination")]
    S -->|"3 rejected, reported one by one"| R["Errant record reporter"]
    R --> EH["Connect error handling"]
    EH --> DLQ["Dead letter topic"]
    

In the picture, a batch of one hundred records goes to the sink connector. Ninety-seven are accepted by the destination. Three are rejected. The connector reports these three one by one through the reporter, and Connect sends them to the dead letter topic. The other ninety-seven are not held up.

Connectors with their own error settings

Some connectors also provide their own settings for problems at the destination. For example, a search-engine sink may have a setting for how to treat documents the search engine refuses. These settings work alongside the general Connect error handling. Check which mode each connector supports, and prefer the ones that route bad records into your DLQ over the ones that quietly ignore them.

The DLQ setting exists only for sink connectors, so sources need other plans.

Why there is no source DLQ

A dead letter queue puts the failed Kafka record into another Kafka topic. In a sink, the failed item is already a Kafka record, so saving a copy is natural. In a source, the failed item comes from outside Kafka, such as a database row or a file line. There is no Kafka record yet, and Connect has no general way to store the outside item as one.

For a source connector, the error handling settings still apply to converters and transformations, but the choices are limited to failing the task or skipping the item, with optional logging.

Practical alternatives

flowchart LR
    X[("Outside system")] --> SC["Source connector (light parsing)"]
    SC --> RAW["Raw topic"]
    RAW --> ST["Stream job validates and parses"]
    ST -->|"good"| CLEAN["Clean topic"]
    ST -->|"bad"| ERR["Error topic for this stage"]
    

The diagram shows a common pattern. The source connector only lifts data from the outside system with minimal parsing, so it rarely fails. A downstream stream processing job does the strict checking. Good records go to a clean topic, and bad ones go to an error topic, which plays the same role as a DLQ.

A record in the DLQ has a life story, and a good team knows every chapter.

flowchart LR
    A["Record fails in Connect"] --> B["Stored in DLQ topic"]
    B --> C["Alert fires: DLQ volume above normal"]
    C --> D["Engineer triages using headers"]
    D --> E{"Cause understood?"}
    E -->|"data problem"| F["Repair the record"]
    E -->|"config or code problem"| G["Fix the connector, converter, or schema"]
    E -->|"truly junk"| H["Discard and record the reason"]
    F --> I["Replay to the source topic or destination"]
    G --> I
    I --> J["DLQ record expires by retention"]
    H --> J
    

The path begins when a record fails and lands in the DLQ. An alert tells the team that something arrived. The engineer reads the headers to understand the cause. There are three outcomes. The data may need repair. The pipeline itself may need a fix. Or the record may be truly junk, which is discarded with a note. Repaired or unblocked records are replayed. Eventually, retention removes old DLQ records.

Triage in five questions

Retention: the tray has a time limit

Like any topic, a DLQ deletes old records after its retention time. If your team takes a week to react and the retention is three days, the evidence disappears. Choose a retention long enough to cover holidays, weekends, and slow investigations. Many teams choose a couple of weeks for DLQ topics.

Saving bad records is only half the job. The other half is putting the fixed ones back.

Connect does not replay DLQ records for you. You choose a replay strategy that fits the cause of failure.

Common replay strategies

flowchart LR
    DLQ["DLQ topic"] --> RJ["Repair job"]
    RJ -->|"fixed record"| SRC["Original source topic"]
    SRC --> CON["Sink connector (now fixed)"]
    CON --> DEST[("Destination")]
    RJ -->|"cannot repair"| PARK["Parking topic for manual review"]
    

The repair job reads records from the DLQ, tries to fix each one, and sends good results back to the source topic where the fixed connector picks them up. Records that cannot be repaired go to a parking topic for a person to review. This keeps the DLQ from turning into a permanent pile.

Be careful with duplicates and order

• Duplicates. If the record partly reached the destination before failing, a replay could create a duplicate. Prefer destinations and settings where writes with the same key replace the earlier version.
• Order. A replayed record arrives late. If newer data for the same key has been written, the old value could overwrite the newer one. Compare timestamps or version fields before replaying.
• Loops. If you replay a record before fixing the cause, it will fail again and return to the DLQ. Add a counter or a header to stop endless loops.

Most DLQ records come from a short list of causes. Learn the list and triage becomes quick.

The special case of tombstones

A tombstone is a record with a key but an empty value. In Kafka, it means “delete this key”, especially in compacted topics. Many transformations and some connectors do not expect an empty value and fail. Records like these are perfectly valid, but they will keep appearing in your DLQ until you handle them, for instance by making the transformation skip empty values or by configuring the connector to treat them as deletes.

Bulk failures versus trickles

The dead letter idea is older than Kafka. Comparing it helps you place Connect’s version correctly.

Classic message queues such as Amazon SQS or RabbitMQ have a dead letter feature built into the broker. After a message fails delivery several times, the broker itself moves it to a special queue. Kafka works differently. Kafka brokers do not track whether a consumer succeeded. They only store records and serve them by offset. So any dead letter behavior in Kafka must be built in the client or framework that reads the records.

The “retry topic” cousin

Some teams use a related design with several topics, for example a “retry in one minute” topic, a “retry in ten minutes” topic, and finally the dead letter topic. A failed record hops from one to the next, giving temporary problems time to clear without blocking the main flow. Kafka Connect does not build this ladder for you. It offers in-place retries with backoff and a single DLQ.

Walk through these questions when setting up a new sink connector.

Matching strategy to situation

A DLQ buys you continuity. It costs you attention.

Trade-offs at a glance

The deepest trade-off is latency of attention. Fail-fast makes the problem loud right now. A DLQ makes it quiet now, and important later. If your team is good at watching dashboards and responding within hours, a DLQ is a great fit. If nobody looks at anything for days, a loud failure may protect you better.

A dead letter queue is a copy of your problem data. Treat it as seriously as the original.

Personal data in the DLQ

If the source topic contains personal data, so does its DLQ. The DLQ can even hold more risk, because records in it often live longer while people investigate, and because headers can include text from error messages and stack traces. Apply the same retention, deletion, and privacy rules to the DLQ as to the source. If a customer asks for deletion, remember that the DLQ may hold a copy.

Authorization for the Connect worker

The Connect worker identity needs permission to write to the DLQ topic, and to create it if you let Connect create topics. In tightly controlled clusters, create the DLQ topic yourself in advance, with the exact permissions you want, and give the worker only write access. This also avoids surprises in topic settings.

Cleaning up sensitive stack traces

Stack traces help developers but can expose internal class names and library versions. If DLQ topics are shared with a wider audience, such as support staff, consider giving them a cleaned view: a small job that reads the DLQ, removes stack traces, and writes a summary to another topic for wider access.

The DLQ only helps if somebody notices when it fills.

Metrics that Connect gives you

Each task reports error metrics. The names differ slightly between tools, but the ideas are the same.

flowchart LR
    W["Connect workers metrics"] --> COL["Metrics collector"]
    DT["DLQ topic size and rate"] --> COL
    LG["Worker logs"] --> LP["Log platform"]
    COL --> DB["Dashboards"]
    COL --> AL["Alert rules"]
    LP --> DB
    AL --> ON["On-call team"]
    

Worker metrics and DLQ topic statistics flow into a collector. Worker logs flow to a log platform. Dashboards show trends, and alert rules notify the on-call team when numbers cross a line.

Alerts worth setting

• Any DLQ arrivals for critical connectors. Even one record may deserve a look.
• A sudden jump in the arrival rate. This points to a flood, not a trickle.
• DLQ produce failures above zero. This means Connect could not save a bad record, and that record may be lost.
• Old records sitting in the DLQ. Alert when the oldest record is approaching the retention limit.
• Task in the FAILED state. The classic sign of fail-fast or an unhandled error.
• Growing consumer lag. Slow processing may hide behind a busy retry loop.

Reading the DLQ itself

Because the DLQ is an ordinary topic, you can build a small dashboard on it: counts by connector, by stage, by exception message, and by original topic. Many teams build one small consumer that reads the DLQ, extracts these header fields, and writes counts to their metrics system.

A well-behaved pipeline rarely feels the DLQ. A misbehaving one can feel it a lot.

The normal case: almost free

When bad records are rare, the cost of the DLQ is tiny. The DLQ producer sits idle most of the time. A few extra writes per hour do not matter to a cluster that handles thousands of records per second.

The storm case: errors at scale

The picture changes when a large share of records fail. Imagine a wrong converter setting: every record fails. The connector now does extra work for each record: it creates an error context, retries (if configured), logs (if enabled), and produces the record to the DLQ. That means the pipeline can slow down and the DLQ topic can grow as fast as the source topic.

Keeping storms small

• Alert on the rate and pause the connector quickly during a flood.
• Turn off record details in logs so a storm does not also flood log storage.
• Size the DLQ generously, with enough partitions and disk to absorb an unexpected burst.
• Avoid long retry windows for permanent errors. A retry that cannot succeed only slows the pipeline. Keep retries for external services.
• Test with bad data before launch to learn how the pipeline behaves.

Scaling the DLQ with the pipeline

If a connector runs many tasks, each task writes to the same DLQ topic. Choose a partition count that supports the combined worst-case rate. Because DLQ records are usually read by a low-volume triage process, you may not need many partitions for consumers, but keep enough to spread the write load.

The tray must not have holes. If it fails, bad records vanish.

The DLQ is a Kafka topic, so use Kafka’s protections

What if writing to the DLQ fails?

Writing to the DLQ is a normal Kafka produce call, and it can fail, for example when the topic does not exist and cannot be created, when permissions are missing, or when the brokers are unreachable. In such cases Connect logs the problem and counts it in the “DLQ produce failures” metric. The record that could not be saved may be lost, since it has already been skipped. This is why the produce failure metric deserves an alert of its own, and why the topic and permissions should be prepared in advance.

Worker and task failures

Connect itself is built for reliability. If a worker dies, its tasks move to other workers, and they resume from the last committed offsets. Records that were already sent to the DLQ stay there safely in Kafka. Records that were in flight may be processed again after the move, so a bad record might be written to the DLQ more than once. Design your triage and replay steps to tolerate duplicates.

flowchart LR
    W1["Worker A runs task 1"] --> DLQ["DLQ topic replicated on three brokers"]
    W2["Worker B runs task 2"] --> DLQ
    W1 -. "worker fails" .-> X["Task 1 moves"]
    X --> W3["Worker C resumes task 1 from committed offset"]
    W3 --> DLQ
    

The picture shows two workers writing to a replicated DLQ topic. When worker A fails, its task moves to worker C, which resumes from the last committed offset. The DLQ keeps every record already written, and the resumed task may re-send a small number of records, which is why duplicates are possible.

A few simple habits make DLQs predictable across environments.

Treat error settings as code

Keep connector configurations in source control, not only in a running system. The error handling settings belong right beside the rest of the connector’s settings, with comments that explain the choices. Reviews then catch dangerous combinations, such as tolerance “all” without a DLQ.

Naming conventions

Creating the topic ahead of time

Connect can create a missing DLQ topic if permissions allow, but production teams normally create it in advance. That way they choose the partition count, retention, replication, minimum in-sync copies, and permissions, instead of accepting defaults.

Managed and cloud services

Cloud teams have several ways to run Connect. Amazon MSK Connect and managed Connect offerings from Kafka vendors run standard Kafka Connect underneath, so the same error settings apply. Some managed connectors expose dead letter queue options in their own screens or configuration forms. Details differ between providers and connectors, so check the current documentation for the service you use.

The same good ideas and the same bad ideas show up in every team.

Patterns that work

Anti-patterns to avoid

Small habits that prevent most dead letter surprises.

Habits that pay off

Common mistakes

Let us design the error handling for one real pipeline from start to finish.

Brightcart, an online store, publishes an orders topic. A sink connector copies orders into a cloud storage bucket every few minutes, where analysts query them. The connector uses an Avro converter with a schema registry, and one transformation that adds a “received time” field.

flowchart LR
    APP["Mobile and web apps"] --> ORD["orders topic"]
    ORD --> SNK["Storage sink connector: tolerance all, retry 5 minutes"]
    SNK -->|"good records"| S3[("Cloud storage bucket")]
    SNK -->|"bad records with headers"| DLQ["dlq-orders-storage-sink"]
    DLQ --> AL["Alert on arrivals"]
    DLQ --> RJ["Repair job"]
    RJ -->|"fixed"| ORD
    RJ -->|"unfixable"| PK["parking-orders topic"]
    

A week in production

On Monday, a new mobile release accidentally sends order values as text instead of numbers for a small share of users. Fifty orders reach the DLQ. The alert fires, and an engineer groups the DLQ records by exception message. One message dominates: a type mismatch. The original topic and partition point to a single app version.

On Tuesday, the mobile team ships a fix. The data team writes a repair job that converts the text values to numbers and sends the fixed orders back to the orders topic. They replay ten records first, check the results in storage, and then replay the rest. Analysts see complete data, and nobody had to stop the pipeline.

On Thursday, the registry is restarted for an upgrade. For four minutes, the converter cannot fetch schemas. Because retries are set for five minutes, nothing reaches the DLQ. Records are simply delayed, then processed.

Wrong ideas about dead letter queues travel from team to team. Here are the corrections.

These situations appear again and again wherever teams move data with Kafka Connect.

Knowing the everyday controls turns a stressful incident into a routine job.

Kafka Connect in distributed mode comes with a small web interface, often called the REST interface, for managing connectors. You use it to look at what is happening and to change it. During a dead letter incident, you will use it in four situations.

Changing settings safely

When you change a connector’s configuration, such as adding a DLQ or fixing a converter, Connect stores the new configuration and restarts the connector’s tasks so that they pick it up. During that moment, tasks stop and restart, which is a small pause in processing. Plan changes for a quiet time when you can, and check the status afterwards to confirm that every task returned to the running state.

Where the connector’s position is stored

A sink connector’s position in its source topic is stored in the normal Kafka consumer offsets for a group that Connect creates for that connector. This has a useful consequence. When you fix a problem and resume, the connector continues where it left off, and no records are skipped or repeated, apart from the small number that may be repeated after an unclean stop. Newer versions of Kafka Connect also allow a stopped connector’s offsets to be viewed and changed through the same interface, which can help in rare cases where you need to move the connector’s position deliberately. Use this carefully, because moving the position forward skips records and moving it backward repeats them.

Reading the worker log

The worker log shows what the connector was doing when an error happened. With error logging enabled, each failure produces a line naming the stage, the exception, and the location of the record. Combined with the DLQ headers, it lets you rebuild the story: which record, which step, which reason, and at what time. Keep logs long enough to cover your investigation window, and remember that log lines with record details can hold sensitive data.

A dead letter queue in Kafka Connect is a practical answer to a simple problem: one bad record should not stop a whole pipeline. By setting aside records that a sink connector cannot process, Connect keeps healthy data flowing and keeps the failed records safe for study and repair. The idea is easy, but it works well only when the surrounding habits are in place: tolerance, headers, retries, alerts, owners, and a plan for replay.