What Is kafka-topics.sh?
A beginner-friendly guide to the command-line tool that creates, lists, inspects, changes, and deletes Kafka topics: what each action does, what it really changes inside the cluster, and the habits that keep it safe.
Imagine a large warehouse with hundreds of labelled storage bays. Each bay holds one kind of goods and has its own rules: how many shelves it has, how many spare copies of the goods are kept in other buildings, and how long the goods may stay. Somebody must be able to open a new bay, look at the list of bays, check how a bay is set up, adjust it, or close it. In the warehouse, that somebody uses a control counter with a small set of clearly labelled buttons. In Apache Kafka, the bays are topics, and the control counter is a small command-line tool called kafka-topics.sh. This guide explains what it is, what each of its buttons does, and how to use it wisely, in simple words.
1The Background: Topics and Why We Manage Them
To understand a tool that manages topics, we first need to remember what a topic is.
Topics, partitions, and copies
Apache Kafka stores streams of events, such as orders, payments, clicks, or sensor readings. Events are grouped into named categories called topics. Each topic is split into slices called partitions, and each partition is copied to several Kafka servers, called brokers, so that the data survives failures. The number of partitions and the number of copies are chosen when the topic is created.
Topics need care
A topic is not just a name. It carries important decisions that shape speed, safety, and cost: how many partitions it has, how many copies exist, how long events are kept, and whether older events with the same key are compacted. Creating a topic with the wrong settings can cause problems that are hard to repair later, such as too few partitions to scale readers, or too few copies to survive a broker failure.
Who does the managing?
Someone or something must create topics, look at them, and sometimes change or remove them. This work is called administration. Kafka offers several ways to do it, and one of the oldest, simplest, and most widely used is the command-line tool named kafka-topics.sh.
A topic is like a shop unit in a mall: it has a name, a size, a lease length, and a backup arrangement. The mall office keeps a simple form for opening a new unit, checking its details, extending its size, or closing it. kafka-topics.sh is that office form for Kafka.
The tool does not store or move events. It only manages the containers that will hold them. Producers and consumers do the actual reading and writing.
2What Is kafka-topics.sh?
It is a small program, shipped with Kafka, that lets you manage topics from a terminal.
The simple definition
kafka-topics.sh is a command-line script included in the bin folder of every Kafka download. When you run it, it connects to a Kafka cluster, asks the cluster to perform one topic-related action, and prints the result. It can create a topic, list all topics, describe the details of a topic, change a topic, and delete a topic.
Where it comes from
The tool is part of a family of small scripts that ship with Kafka for administration and testing. Each script has a name beginning with “kafka-” and does one job. On Windows, the same tools appear as batch files with a different ending. In some packaged distributions, such as Confluent’s, the script is provided without the “.sh” ending, but the behaviour is the same.
What happens when you run it
The script is a thin wrapper. It starts a small Java program that uses Kafka’s administrative client library. That client connects to the brokers you name, sends a request such as “create this topic”, and waits for the answer. The real work of creating and placing partitions is done by the cluster itself, in particular by the controller, the part of the cluster that makes decisions about topics and partitions.
flowchart LR
U["You in a terminal"] --> S["kafka-topics.sh script"]
S --> A["Kafka admin client library"]
A --> B["Bootstrap broker"]
B --> C["Cluster controller"]
C --> T["Topics and partitions on brokers"]
Ordering a parcel at a shop counter: you fill in a form, the clerk passes it to the warehouse manager, and the manager makes it happen. You never enter the warehouse yourself. The script is the form and the clerk, and the controller is the manager.
3How the Tool Finds Your Cluster
Before doing anything, the tool must know where the cluster lives.
The bootstrap server option
Every use of the tool needs the address of at least one broker, given through the bootstrap server option. It is like the first phone number in a contact book. The tool contacts that broker, which tells it about the rest of the cluster, so you do not need to list every broker. Giving two or three addresses is wise, so the tool still works if one broker is down.
A note about the past
In older versions, the tool could talk directly to ZooKeeper, a separate coordination service that Kafka used to depend on, through a different option. Modern Kafka no longer requires ZooKeeper, and the tool talks to the brokers instead. If you find an old guide that uses the ZooKeeper option, remember that it belongs to the past and may not work with current versions.
Talking to a secured cluster
Production clusters usually require encrypted connections and proof of identity. The tool supports this through an option that points to a small properties file holding the connection and login settings. Without it, the tool will be refused by a secured cluster.
| Situation | What You Need to Give the Tool |
|---|---|
| Local practice cluster with no security | Just a bootstrap server address |
| Production cluster with encryption and login | A bootstrap server address and a properties file with security settings |
| Old cluster from a past era | Check the version; old guides may use options that no longer exist |
Entering an office building, you first need the address. If it is a secure building, you also need a badge. The bootstrap address is the street address, and the properties file is the badge.
Copying a command from an old tutorial that uses the ZooKeeper option and being puzzled by errors. Always check which Kafka version a guide was written for.
4The Five Main Actions
The tool has a small set of actions. Each run performs exactly one of them.
Create
Makes a new topic with a chosen number of partitions, a replication factor, and optional settings.
List
Shows the names of the topics that exist in the cluster.
Describe
Shows the details of one or more topics: partitions, leaders, replicas, and settings.
Alter
Changes an existing topic, most commonly by increasing its number of partitions.
Delete
Removes a topic and its data from the cluster.
The supporting options
| Option | What It Does | Used With |
|---|---|---|
| bootstrap server | Tells the tool which broker to contact | Every action |
| command config | Points to a file with security settings | Every action on a secured cluster |
| topic | Names the topic to act on, or a pattern for several | Create, describe, alter, delete, list |
| partitions | Sets the number of partitions | Create, alter |
| replication factor | Sets the number of copies of each partition | Create |
| config | Sets a topic setting such as retention, as a name and value pair | Create |
| if not exists | Makes create succeed quietly when the topic is already there | Create |
| if exists | Makes alter or delete succeed quietly when the topic is missing | Alter, delete |
| exclude internal | Hides Kafka’s own internal topics | List, describe |
A remote control has a few main buttons, such as power, volume, and channel, and a few modifiers. Nobody needs to memorise every feature to use it. The tool’s five actions are its main buttons, and the options are the fine adjustments.
Running the tool with its help option prints the complete, current list of options for your version. It is the most reliable reference, since options are added and retired over time.
5Creating a Topic
Creation is the most common use, and the choices made here echo for years.
The decisions you must make
The name
Choose a clear, consistent name. Allowed characters are letters, digits, dots, underscores, and hyphens, and the name may be at most 249 characters long. Avoid mixing dots and underscores in the same naming scheme, because Kafka’s metrics treat them as similar and names can collide.
The number of partitions
This sets the ceiling on parallel reading in a consumer group and spreads the load over brokers. It can be increased later but never decreased, so plan with headroom.
The replication factor
This is the number of copies of each partition. Three is the usual choice for important data. It cannot be higher than the number of brokers, because copies must live on different brokers.
Optional settings
Settings such as the retention period, the cleanup policy, the minimum in-sync copies, and the maximum message size can be given at creation, so the topic is right from its first day.
What the cluster does
The tool sends the request
It asks a broker to create the topic with the chosen numbers and settings.
The controller checks the rules
It confirms that the name is valid, the topic does not already exist, and enough brokers are available for the requested copies.
Partitions are placed
The controller spreads the partitions and their copies across brokers, trying to keep leaders and copies balanced, and to use different racks where labelled.
Brokers create the files
Each chosen broker creates the storage for its partitions and followers begin copying from leaders.
The tool reports success
It prints a confirmation, and the topic is ready for producers and consumers.
flowchart TD
R["Create request: name, partitions, replication factor, settings"] --> V{"Valid name and enough brokers?"}
V -->|"No"| E["Error message returned"]
V -->|"Yes"| X{"Topic already exists?"}
X -->|"Yes"| E2["Error, or quiet success if the if-not-exists option is used"]
X -->|"No"| P["Controller places partitions and copies on brokers"]
P --> OK["Topic is ready"]
Opening a new shop unit means choosing its name, its size, how many backup stockrooms it has, and how long goods may stay. The mall manager checks the plan, assigns physical space in different wings, and hands you the keys.
Creating topics with a replication factor of one to “keep things simple”. A single copy means any broker failure can make the topic unavailable or lose its data. Use at least three copies for anything that matters.
6Listing Topics
Listing is the simplest action: it answers “what topics exist?”
What you see
The list action prints the name of every topic the tool is allowed to see, one per line. On a new cluster, the list may be empty or show only Kafka’s own internal topics. On a busy cluster, it can run to hundreds or thousands of names.
Internal topics
Kafka keeps some topics for its own use. One stores the bookmarks of consumer groups, and another stores information about transactions. Their names begin with two underscores. The exclude-internal option hides them so you see only business topics.
Practical uses
Checking that a topic exists
Before starting an application, confirm that its topics have been created.
Finding a topic by pattern
Combined with ordinary text-search tools, the list helps find topics whose names contain a word, such as the name of a team or product.
Auditing what is in the cluster
Compare the list with your documentation to spot forgotten or unexpected topics, including accidentally auto-created ones.
Listing topics is like reading the directory board at the entrance of a mall: it shows every shop name, and some special back-room units that ordinary visitors normally do not need to see.
The list only shows topics that your identity is allowed to see. On a secured cluster, two different users may see two different lists.
7Describing a Topic: Reading the Details
Describe is the most useful action for understanding what is really happening to a topic.
What the description shows
The describe action prints a summary line for the topic and then one line for each partition. The summary shows the topic name, its unique identifier, the number of partitions, the replication factor, and any settings that differ from the defaults. Each partition line shows which broker is the leader, which brokers hold copies, and which of those copies are in sync.
Partition number
The slice of the topic, starting from zero.
Leader
The broker currently handling reads and writes for that partition.
Replicas
All brokers that hold a copy of that partition, whether up to date or not.
In-sync replicas
The subset of replicas that are fully caught up with the leader. Only these can safely take over.
Configs
Topic-level settings that override the cluster defaults, such as retention or cleanup policy.
Reading health from the details
| What You See | What It Means | Action |
|---|---|---|
| Replicas and in-sync replicas are the same list | All copies are caught up; the partition is healthy | Nothing to do |
| In-sync replicas are fewer than replicas | Some copy is behind or its broker is down | Check the broker and the network |
| Leader is shown as missing or none | No broker is currently leading the partition, so it is unavailable | Urgent: investigate broker health at once |
| Leaders concentrated on one broker | Uneven load | Consider rebalancing leadership |
Focused views for troubleshooting
The tool can also filter the description to show only trouble spots: partitions with fewer in-sync copies than required, partitions below the minimum in-sync count, partitions with no leader, and partitions where the number of in-sync copies is exactly at the minimum. These filters are handy first checks when something feels wrong.
flowchart LR
D["Describe a topic"] --> S["Summary: id, partitions, replication factor, configs"]
D --> P0["Partition 0: leader Broker 1, replicas 1,2,3, in-sync 1,2,3"]
D --> P1["Partition 1: leader Broker 2, replicas 2,3,1, in-sync 2,3"]
D --> P2["Partition 2: leader Broker 3, replicas 3,1,2, in-sync 3,1,2"]
P1 --> W["In-sync list is shorter than replicas: Broker 1 is behind or down"]
Describing a topic is like reading the staff board of a shop: it shows who is the manager on duty, who the trained backups are, and which backups are actually present and up to date today. A missing name is an early warning.
When a topic behaves oddly, describe it first. Most producer and consumer problems become obvious once you see leaders and in-sync copies.
8Altering a Topic: What You Can and Cannot Change
Alter sounds all-powerful, but Kafka only allows a few safe changes.
Increasing the number of partitions
The main use of alter is to raise the partition count of a topic. New, empty partitions are added, and the cluster places them on brokers. Existing events stay where they are. You can only increase the count, never reduce it.
The side effect for keyed topics
Producers choose a partition for keyed events by calculating from the key and the partition count. When the count changes, the same key may map to a different partition. New events for a key may go to a new partition while older events for it stay in the old one. For a while, order per key can look broken, and consumers that keep state per key may be surprised. This is why the partition count should be planned generously at creation.
What alter does not do
Reduce partitions
Impossible. To shrink, create a new topic with fewer partitions and move data and applications across.
Change the replication factor
Not through this tool. It needs a separate reassignment tool that moves copies between brokers carefully.
Change most settings
Modern practice is to change topic settings, such as retention, with the separate configuration tool, kafka-configs.sh, rather than through alter.
Rename a topic
There is no rename. The usual method is to create a new topic and migrate.
| You Want To | Possible? | How |
|---|---|---|
| Add partitions | Yes | The alter action with a larger partition count |
| Remove partitions | No | Create a new topic and migrate |
| Change replication factor | Yes, but not here | Use the partition reassignment tool |
| Change retention or other settings | Yes, but not here | Use kafka-configs.sh |
| Rename a topic | No | Create a new topic and migrate |
A mall lets a shop take over an extra empty unit next door, but it will not shrink a shop or move walls that have already been built. Renaming a shop means opening a new one with the new sign and moving the goods.
Adding partitions to a busy keyed topic during peak hours. Key-to-partition mapping shifts at once, and consumers relying on strict order per key may see surprises. Plan changes for quiet periods, and test first.
9Deleting a Topic
Delete is the most dangerous button on the panel, because it cannot be undone.
What deletion does
Deleting a topic removes its partitions, all its stored events, and all copies on every broker. The topic’s name becomes free to use again. Consumers that were reading it will start to see errors, and producers writing to it will fail or, on clusters that auto-create topics, may quietly create a fresh empty topic with default settings, which is a nasty surprise.
Deletion is not instant
The tool marks the topic for deletion, and the cluster then removes the data in the background. For a large topic this can take some time. During this period, the topic may still appear in listings, or reappear briefly if a client recreates it. Some clusters have the deletion feature switched off by an administrator setting, in which case the request is refused or has no effect.
You request the deletion
The tool asks the cluster to delete the named topic.
The topic is marked
The controller records that the topic is to be deleted and stops serving it.
Brokers remove the files
Each broker deletes the partition files it holds for the topic.
Metadata is cleaned
The topic vanishes from the cluster’s records, and its consumer group bookmarks for it eventually become meaningless.
Closing a shop unit means clearing out all the stock from every stockroom, removing the sign, and handing back the keys. Once the shelves are emptied and the goods disposed of, there is no bringing them back.
Deleting the wrong topic because of a typo, a similar name, or a terminal pointing at production instead of a test cluster. Always double-check the cluster address and the name, and consider restricting who may delete topics.
Before deleting, confirm that no application still reads or writes the topic, and copy out any data that someone might still need. A safe habit is to first block access, wait a few days, and only then delete.
10The Danger of Auto-Created Topics
The tool is only needed because topics can be created deliberately. But Kafka can also create them by accident.
What auto-creation is
A broker can be set to create a topic automatically the first time anyone writes to it, reads from it, or asks for its information. This feature is convenient for quick experiments. The new topic gets the cluster’s default number of partitions and default replication factor, which are often small, and default settings.
Why it causes trouble in production
Typos become topics
A producer with a misspelled topic name quietly creates a new, wrongly named topic, and the events flow into a place nobody reads.
Wrong settings by default
Auto-created topics use defaults that may not suit the data, for example a single partition or a short retention, and nobody reviewed them.
Zombie topics after deletion
After you delete a topic, a forgotten client still trying to use it recreates it, and the topic reappears with default settings.
Clutter and cost
Test and mistaken topics pile up, using partitions, disk, and management effort.
The safer habit
Many organisations switch auto-creation off in production and create every topic deliberately, using kafka-topics.sh or an automated system, with reviewed settings. Then a mistyped name produces a clear error instead of a silent new topic.
Imagine a mall where any delivery driver who knocks on an empty unit is handed the keys, and a shop appears with default fittings. A misdelivered parcel would create a new shop nobody asked for. Better to require a signed request at the office.
Leaving auto-creation on in production because “it has always worked”. It hides mistakes, and every hidden mistake eventually costs an investigation.
11The Tool’s Family of Siblings
kafka-topics.sh does one job. Other small tools handle the neighbouring jobs.
| Tool | Job | When You Use It |
|---|---|---|
| kafka-topics.sh | Create, list, describe, alter, delete topics | Managing the containers |
| kafka-configs.sh | View and change settings of topics, brokers, and clients | Changing retention, cleanup policy, or limits |
| kafka-console-producer.sh | Send a few test events from the terminal | Quick tests and demonstrations |
| kafka-console-consumer.sh | Read and print events from a topic | Checking what is inside a topic |
| kafka-consumer-groups.sh | View groups, lag, and reset positions | Investigating slow consumers |
| kafka-reassign-partitions.sh | Move partitions and copies between brokers | Rebalancing load or changing the replication factor |
| kafka-acls.sh | Manage who may do what | Setting access rules on secured clusters |
flowchart TD
T["kafka-topics.sh: manage topics"] --- C["kafka-configs.sh: change settings"]
T --- R["kafka-reassign-partitions.sh: move partitions"]
T --- G["kafka-consumer-groups.sh: watch consumers"]
T --- P["Console producer and consumer: test data"]
T --- A["kafka-acls.sh: control access"]
A workshop has a toolbox with a hammer, a screwdriver, a spanner, and a tape measure. Each tool is simple and does one thing well. You combine them to complete bigger jobs. Kafka’s command-line tools follow the same idea.
12Security and Permissions
Because the tool can delete data, who is allowed to run it matters greatly.
The tool is only a messenger
The script itself has no special powers. What it can do depends entirely on the identity it presents to the cluster and the permissions that identity holds. On a secured cluster, creating topics, deleting topics, altering topics, and even describing them are separate permissions that administrators grant or withhold.
Sensible arrangements
Separate identities for people and applications
Applications should normally only read and write. Topic management should belong to a small number of trusted administrators or an automated pipeline.
Restrict delete rights
Give the permission to delete topics to very few identities, and require a review step for production.
Protect credentials
Keep the properties file with passwords or certificates outside version control and away from shared machines.
Keep an audit trail
Record who created, changed, or deleted which topics and when, through cluster logs or your automation system.
A master key is dangerous in the wrong hands. A mall office gives the key to closing shops only to senior managers and records every use in a logbook. The tool is the door, and the permissions are the keys.
Being able to describe or list topics is also a permission. If the list looks empty on a busy cluster, your identity may simply lack the right to see them.
13Beyond the Command Line: When to Automate
The command-line tool is perfect for learning and small tasks. Bigger organisations usually add automation on top.
Why not type commands by hand forever?
Typing commands is fine for a practice cluster. On real systems, it invites mistakes: a wrong server, a typo in a setting, a forgotten step. It also leaves no record of why a topic exists or who asked for it, and topics on different environments drift apart over time.
Common ways to automate
Scripts in version control
A list of topics and their settings is kept in a file that is reviewed like code. A script reads it and uses the tool, with the option that ignores topics already created, so it can be run again safely.
The Admin client in your own program
Applications or deployment tools can call the same administrative library directly, without going through the script.
Infrastructure-as-code tools
Declarative tools describe the desired topics, and the tool makes the cluster match, showing a preview of changes before applying them.
Operators on container platforms
On platforms such as Kubernetes, special controllers manage topics from declared resources, keeping cluster and declaration in step.
Self-service portals
Large companies build internal pages where teams request topics, and the portal enforces naming rules, sensible defaults, and approvals.
A small cafe may write orders on paper, but a large restaurant uses a ticket system that records every order and avoids mistakes. The paper works for a few tables, and the system is needed as the numbers grow.
Letting each developer create topics by hand with personal preferences. Topics in test and production then differ, and problems appear only after release. Keep one reviewed source of truth.
14Advantages and Trade-offs of the Tool
Like every tool, it has strengths and limits.
Advantages
- Ships with Kafka, so it is available everywhere from day one
- Simple actions that are easy to learn
- Works against any cluster, local or remote
- Describe output gives instant insight into leaders and in-sync copies
- Easy to wrap in scripts and pipelines
- No extra software to install or license
Disadvantages / Trade-offs
- Manual use is error-prone and leaves no built-in history
- Text output is designed for humans, so scripts must parse it carefully
- Limited alter abilities, with other tools needed for settings and replica moves
- Deleting is powerful and irreversible
- Options change between versions, so old tutorials may mislead
- Needs a Java runtime and network access to the brokers
15Anti-Patterns and Best Practices
Most problems with this tool repeat across teams.
Problem
Relying on auto-created topics in production instead of creating topics deliberately.
Why It’s Harmful
Typos create stray topics, defaults may be wrong, and deleted topics reappear when forgotten clients keep using them.
Correct Approach
Turn auto-creation off in production, and create every topic with reviewed settings through the tool or an automated system.
Problem
Creating topics with one partition and one copy because it works on a laptop.
Why It’s Harmful
Reading cannot be spread over several consumers, and a single broker failure can stop or destroy the topic.
Correct Approach
Choose partitions with headroom for growth and use three copies with a sensible minimum in-sync count for important topics.
Problem
Running delete commands from a shared terminal without checking which cluster it points to.
Why It’s Harmful
A topic on production can be destroyed by mistake, taking its data and stopping applications.
Correct Approach
Restrict delete permissions, use clearly named environments, require review for production changes, and back up or archive important data first.
Problem
Adding partitions to a keyed, busy topic without planning for the effect on ordering.
Why It’s Harmful
Keys start mapping to new partitions, splitting each key’s history and disturbing consumers that keep state.
Correct Approach
Plan the count generously at creation, change it in quiet periods, and inform consumers, or migrate to a new topic.
Problem
Following an old tutorial blindly, using options that belong to an older Kafka version.
Why It’s Harmful
Commands fail, or worse, behave differently from what the guide promises.
Correct Approach
Check the tool’s built-in help for your version and read the release notes before relying on remembered commands.
Best practices at a glance
Create deliberately
Every topic is created by a person or pipeline that chose its settings.
Name consistently
Use a clear scheme, avoid mixing dots and underscores, and stay well under the length limit.
Plan partitions and copies
Leave room to grow, and use three copies for important data.
Set settings at creation
Retention, cleanup policy, and minimum in-sync copies belong in the first version of the topic.
Describe before you worry
Check leaders and in-sync copies as the first step in any investigation.
Automate and review
Keep topic definitions in version control and apply them through scripts or tools.
Guard deletion
Limit who can delete, double-check names, and confirm nobody still uses the topic.
Use the right sibling tool
Use the configuration and reassignment tools for jobs the topics tool does not do.
16A Real-World Walk-Through
Let us follow a platform team preparing the topics for a new online-shop feature.
Plan the topics on paper
The team decides it needs an orders topic, a payments topic, and a dead letter topic. It agrees on names following the company’s naming scheme, and chooses twelve partitions for orders and payments, with three copies each.
Check what already exists
An engineer uses the list action on the test cluster to make sure the names are free and no similar stray topics exist.
Create on the test cluster first
Using the create action with the agreed partition count, replication factor, retention of fourteen days for orders and payments and thirty for the dead letter topic, and a minimum of two in-sync copies.
Describe and verify
The engineer describes each topic and checks that leaders are spread evenly across the brokers and that every partition shows all three copies in sync.
Move the definitions into version control
The same settings are written in a reviewed file. A pipeline applies it to staging and then production, using the option that ignores topics that already exist.
Grow later, carefully
Months later, traffic doubles. The team increases the orders topic to twenty-four partitions during a quiet night, after warning the teams whose consumers keep per-customer state.
Retire a test topic safely
An old experiment topic is blocked from writes, watched for a week to confirm nobody complains, archived, and finally deleted through the tool by a permitted administrator.
flowchart LR
P["Plan names, partitions, copies, settings"] --> L["List: are names free?"]
L --> C["Create on test cluster"]
C --> D["Describe: check leaders and in-sync copies"]
D --> V["Definitions stored in version control"]
V --> PR["Pipeline applies to staging and production"]
PR --> G["Grow or retire later with care"]
17Frequently Asked Questions
It is the command-line tool shipped with Kafka for creating, listing, describing, altering, and deleting topics on a cluster.
In the bin folder of the Kafka download. On Windows, the equivalent is a batch file, and in some distributions the script has no “.sh” ending.
No. It only manages topics. Events are written by producers and read by consumers.
At least one broker address through the bootstrap server option, and on secured clusters a properties file with connection and login settings.
No. Partitions can only be increased. To reduce them, create a new topic with fewer partitions and migrate the data and applications.
Not directly. The replication factor is changed through the separate partition reassignment tool, which moves copies between brokers.
No. Deletion removes the topic’s events from every broker. Restore is possible only from copies you made yourself, such as an archive or a mirrored cluster.
A client that still tried to write, read, or ask about it may have caused the cluster to auto-create it again with default settings. Turn off auto-creation and stop the stray clients.
It shows which copies of a partition are fully caught up with the leader. Only those copies can safely become the new leader if the current leader fails.
You can, especially with the option that ignores existing topics, but many teams prefer the admin client library, infrastructure-as-code tools, or operators, which are easier to review and less error-prone than parsing text output.
18Summary and Key Takeaways
We began with a warehouse whose storage bays each have their own size, backup arrangement, and keep-time, and a control counter with a few clearly labelled buttons. kafka-topics.sh is that counter for Kafka. It ships with Kafka, connects to the cluster through a bootstrap broker, and performs one of five actions: create, list, describe, alter, or delete. Creation fixes the choices that echo for years: the name, the number of partitions, the replication factor, and the early settings. Describing shows leaders and in-sync copies, which is the quickest way to judge a topic’s health. Alteration is limited: partitions can only grow, with side effects on keyed data, while settings and replica moves belong to sibling tools. Deletion is powerful and irreversible, and permissions and habits must guard it. Auto-created topics undermine deliberate design, so production clusters usually turn them off. As organisations grow, the same actions are wrapped in reviewed scripts, admin libraries, infrastructure-as-code tools, or operators so that topics are consistent, recorded, and safe. Learn the tool well, use it deliberately, and automate what you repeat.
Key Takeaways
- It manages topics, not events — the tool creates, lists, describes, alters, and deletes topics, while producers and consumers handle the events themselves.
- It talks to the cluster through a bootstrap server — secured clusters also need a properties file, and old ZooKeeper-based guides no longer apply to modern versions.
- Creation sets lasting choices — name, partitions, replication factor, and early settings should be planned, with three copies for important data and headroom for partitions.
- Describe reveals health — leaders, replicas, and in-sync replicas show at a glance whether a partition is safe or in trouble.
- Alter is limited — partitions can only be increased, which changes key placement, while settings and replication changes use other tools.
- Delete cannot be undone — restrict permission, double-check names and clusters, and confirm no application still needs the topic.
- Create on purpose and automate — turn off auto-creation in production, keep topic definitions in version control, and use scripts or platform tools for repeatable, reviewed changes.