Secret Types (Opaque, TLS, Docker Registry) in Kubernetes
Learn how Kubernetes stores passwords, certificates, and registry logins, what each built-in secret type is for, and how to handle them safely from the first day.
Every house has a few things that should not be left on the kitchen table: the spare key, the passport, the bank card. Sensible families keep these in a drawer with a lock, and they think carefully about who gets a copy of the key. They also label the drawers, so that the passport is never mixed up with the receipts, and so that a visitor who needs to find the spare key knows exactly where to look.
A Kubernetes cluster has the same kind of valuables. Applications need database passwords, security certificates, and logins for private image stores. If these are pasted into ordinary settings files, anyone who can read the files can read the valuables. Kubernetes gives us a special object for this job, called a Secret, and it comes in several labeled kinds, called secret types. In this guide, we will build the whole idea from zero, with a special focus on three types you will meet again and again: Opaque, TLS, and Docker registry. You do not need any earlier knowledge. Every new word is explained the first time it appears, with a simple comparison from everyday life.
1What Is a Kubernetes Secret?
A Secret is a small Kubernetes object designed to hold sensitive information, so that it can be kept apart from application code and ordinary settings.
The Simple Definition
In Kubernetes, a Secret is an object that stores a small amount of sensitive data, such as a password, a token, or a private key. Instead of writing the password inside the application’s image or inside a plain settings file, you place it in a Secret. Then you tell Kubernetes which applications may use it, and Kubernetes delivers it to those applications when they start. The application reads the value at run time, and the value never has to be baked into the software itself.
This separation brings several benefits. The same application package can be used in testing and production, with different Secrets holding different passwords. Passwords can be changed without rebuilding the application. And the people who write code do not need to know the production passwords, because those live in a separate object that is guarded separately.
Think of a hotel safe deposit box. The hotel staff, the guests, and the cleaners all use the same building, but the valuables are kept in a small locked box that only certain people may open. A Secret is that small box: it keeps the valuables out of the open rooms of the cluster and lets only chosen visitors reach them.
What a Secret Is Not
It is important to be honest from the start about what a Secret is not. A Secret is not a magic vault. By default, Kubernetes does not encrypt Secrets while they sit in its storage, and it only hides their values from casual view. A Secret is a better place for sensitive data than a plain settings file, but it becomes truly safe only when you add proper protections around it. We will explore those protections in detail in later chapters.
Many beginners believe that simply using a Secret makes their data safe. It does not. A Secret is a tool that gives you the chance to protect data well. Real safety comes from controlling who can read it, encrypting it in storage, and handling it carefully in your applications.
2Why Sensitive Data Needs Special Handling
Leaking a single password can undo years of careful engineering, so sensitive data deserves habits that ordinary settings do not.
The Cost of a Leak
A leaked password is not like a leaked recipe for pancakes. It can open the door to customer data, allow attackers to send fake messages, or let strangers run expensive machines on your account. The damage can arrive within minutes of the leak, and the cleanup can take weeks. Companies have lost customers and paid large fines because a key was accidentally left in a public place.
Worse, secrets tend to leak in boring, everyday ways. A developer pastes a password into a chat to help a colleague. A settings file with a real key is committed to a shared code repository. A log line prints a whole request, including the password inside it. None of these involve a clever attacker. They are simple slips, and good systems are designed to make such slips less likely and less harmful.
The Human Side of Secrets
Technology is only half of the story. Most secret leaks begin with people who are tired, rushed, or trying to be helpful. A support engineer pastes a real password into a ticket so a colleague can reproduce a bug. A new hire receives a spreadsheet of keys by email because nobody explained the proper route. A contractor keeps their access long after the project ends. Good habits and clear routines protect against these ordinary situations far better than any single tool.
Teams that handle secrets well make the safe path the easy path. They give people a clear, quick way to request access, a standard place to store values, and a routine for removing access when someone changes roles. When the right way is also the fast way, people follow it without needing to be nagged.
The Idea of Separation
The central idea behind Secrets is separation. Sensitive values are kept in one clearly labeled place, with their own access rules, their own protections, and their own history of who changed them. Ordinary settings, such as the color of a button or the number of items per page, are kept elsewhere. This separation lets you be strict about the small set of things that truly need strictness, without making every settings change a slow ceremony.
A restaurant keeps its menu on the wall, where everyone can read it, but its cash and its safe combination are kept in the back office. Nobody thinks it strange that the menu and the safe are treated differently, because they hold very different risks. Secrets and ordinary settings deserve the same distinction.
The rule of thumb is simple: if leaking a value would make you nervous, it belongs in a Secret, or in an even stronger system behind it. If leaking it would be merely embarrassing, an ordinary settings object is usually enough.
3Kubernetes Building Blocks You Need First
A short list of basic terms will make everything else in this guide easy to follow.
Kubernetes is a system that runs applications packed inside containers, which are lightweight boxes holding an application together with everything it needs. Kubernetes decides where these boxes run, restarts them when they fail, and connects them together. Think of Kubernetes as the manager of a large warehouse who keeps every box in the right place. These are the words we will use most.
Pod
The smallest unit that Kubernetes runs. A pod wraps one or more containers that live and die together, like one apartment for close roommates.
Namespace
A named section of a cluster used to separate teams or projects, like different floors of the same office building. Secrets belong to exactly one namespace.
ConfigMap
A sibling of the Secret, used for ordinary, non-sensitive settings such as a website title or a feature switch.
Volume
A folder made available inside a container. A Secret can be shown to an application as files in such a folder.
Environment Variable
A named value handed to a program when it starts. A Secret can also be delivered this way.
Service Account
An identity given to pods so they can talk to the cluster. Think of it as a staff badge worn by an application.
etcd
The database at the heart of the cluster, where every object, including every Secret, is stored.
RBAC
The rulebook that decides which people and programs may do which actions on which objects. It is the lock and key system of the cluster.
Two Helpers Worth Knowing
Two more helpers appear in the story of Secrets. The API server is the front desk of the cluster: every request to read or change an object goes through it, and it checks permission first. The kubelet is a program on every machine that starts the pods assigned to that machine. When a pod needs a Secret, the kubelet asks the API server for it and then hands it to the pod. Keep these two in mind, because they explain how a Secret travels from storage to the application.
Picture a large office. The records room is etcd, the receptionist is the API server, the badge rules are RBAC, and each floor manager is a kubelet. When an employee needs a confidential file, the floor manager asks the receptionist, who checks the rules and fetches the file from the records room, and only then hands it over.
4The Anatomy of a Secret
Every Secret is made of the same few parts, and understanding them makes the different types easy to grasp.
The Parts
A Secret has a name, which must be unique within its namespace. It has a type, which tells Kubernetes and other tools what kind of information it holds. And it has data: a set of labeled values. Each label is called a key and each value is the sensitive content. For example, a Secret for a database login might hold two entries, one labeled username and one labeled password. The labels are chosen by you for ordinary Secrets, but for some special types they are fixed by Kubernetes, as we will see.
Name and Namespace
Identify the Secret. A pod can only use Secrets that live in its own namespace, which is an important safety boundary.
Type
A label that says what kind of Secret this is. It can trigger checks that the right keys are present, and it cannot be changed after creation.
Data
The actual entries, stored in an encoded form. Each entry has a key and a value.
String Data
A convenience for writing entries in plain text when creating a Secret. Kubernetes encodes them for you and never shows this field again.
Immutable Flag
An optional switch that locks the Secret’s data so it can never be edited, only replaced.
Size Limit
A single Secret may hold at most about one megabyte, so it is meant for small values and not for large files.
Data Versus String Data
When Kubernetes stores a Secret, the values are kept in a text-friendly encoded form. If you write the encoded form yourself, you must convert every value first, which is tedious and error-prone. To make life easier, Kubernetes accepts a second field, in which you can write the values in ordinary readable text. When the Secret is created, Kubernetes converts these values and stores them in the data field. Afterwards, the readable field is not shown again. This is a convenience for writing only, and it does not add any extra protection.
Imagine handing a letter to a courier who seals it in a standard envelope before delivery. You write the letter in plain handwriting, and the courier takes care of the envelope. The string data field is your plain handwriting, and the data field is the sealed standard envelope that everyone in the system actually handles.
Names of keys inside a Secret may contain letters, numbers, dashes, underscores, and dots. When a Secret is shown to an application as files, each key becomes a file name, so choose key names that make sense as file names.
5Base64 Is Not Encryption
The single most common misunderstanding about Secrets is thinking that their strange-looking values are protected.
What the Encoding Really Is
If you look at a stored Secret, the values look like meaningless jumbles of letters and numbers. This is because they are written in an encoding called base64. The purpose of base64 is purely practical: it turns any kind of data, including binary files with unusual characters, into plain text that can be safely written into a text-based object. Anyone can turn it back into the original value in a moment, with no key and no password. It is like writing a word in capital letters. It changes the appearance, but it does not hide anything.
Base64 is like writing your message in a different alphabet that anyone can look up in a public chart. It looks secret to a child, but it fools nobody who knows the chart exists. True encryption, by contrast, is like a lock that needs a private key nobody else has.
Why This Matters in Practice
Because base64 is not protection, anyone who can read a Secret through the cluster’s interface can read its real value. So the protection of Secrets never comes from the encoding. It comes from three other things: strict rules about who may read Secrets, encryption of Secrets in the cluster’s storage, and careful handling in the applications that use them. These three form the core of the safety story in this guide.
Encoding hides nothing
Base64 only makes the value text-safe. Anyone with read access can decode it instantly.
Access rules guard the door
Only people and programs that truly need a Secret should be allowed to read it.
Encryption guards the storage
Encrypting Secrets at rest protects them if storage or backups fall into the wrong hands.
Careful handling guards the use
Applications must avoid printing, logging, or sharing the values they receive.
Never commit a Secret file to a shared code repository and assume the encoded values are safe. Anyone who can see the file can decode it in seconds, and the history of the repository will keep it forever.
6Secret Types: Why They Exist
The type of a Secret is a label that tells the cluster and its tools what kind of sensitive data to expect inside.
A Label With Consequences
All Secrets store values under keys, so at first it seems unnecessary to have different types. But the type does real work. Some parts of Kubernetes expect a Secret to have a particular shape. When a web gateway wants a certificate, it looks for a Secret of the TLS type with two specific keys. When the machine that starts pods needs to log in to a private image store, it looks for a Secret of the Docker registry type. If the type and the shape do not match, the cluster can refuse the Secret when it is created, catching mistakes early instead of letting them cause confusing failures later.
For your own custom data, where no special shape is expected, the general-purpose type called Opaque is used. It tells Kubernetes only one thing: the content is arbitrary, and the cluster should not try to understand it.
| Type | Purpose | Required Keys |
|---|---|---|
| Opaque | Any custom sensitive data you define | None, you choose the keys |
| TLS | A security certificate and its private key | tls.crt and tls.key |
| Docker config JSON | Login details for a private image registry | .dockerconfigjson |
| Basic authentication | A username and password pair | username and password |
| SSH authentication | A private key for secure shell logins | ssh-privatekey |
| Service account token | A legacy way to hold an identity token | Set by the cluster |
| Bootstrap token | A short-lived token used when machines join a cluster | Set by the cluster tooling |
Think of different kinds of keys in a key cabinet: house keys, car keys, and mailbox keys. They are all keys, but each has its own shape and its own tag, so that nobody tries to start a car with a mailbox key. The secret type is the tag on each key.
The Type Cannot Be Changed Later
A Secret’s type is fixed at the moment it is created. If you picked the wrong one, you cannot edit it. You must create a new Secret with the correct type and remove the old one. This rule protects the rest of the cluster from surprises, since other objects may depend on the Secret being of a certain kind.
Always use the most specific type that fits. A TLS certificate stored as a plain Opaque Secret works for your own programs, but the gateway and other tools will not recognize it, and you lose the automatic checks that catch mistakes early.
7Opaque Secrets: The Everyday Type
Opaque is the default type and the right choice for most application passwords, tokens, and keys.
What Opaque Means
The word opaque means that you cannot see through something. In this context, it means that Kubernetes does not look inside the Secret or expect anything about its content. You choose the keys and the values. This makes it perfect for the wide variety of sensitive values that applications need: a database password, a key for an email service, a token for a payment provider, a signing secret for web sessions, or a set of several related values bundled together.
If you create a Secret and do not state a type, Kubernetes assigns Opaque automatically. Because it has no required keys, the cluster performs no checking of its shape. This freedom is convenient, but it also means that typing mistakes in key names will not be caught until the application fails to find the value.
Database credentials
A Secret with entries for the username, the password, and perhaps the host name, so the application can connect without any of these being written in its image.
Third-party service keys
API keys for email, messaging, maps, or payments. Each key can be stored under a clear name that describes the provider.
Signing and encryption keys
Values that an application uses to sign login sessions or protect its own stored data. These must stay private and stable.
Small secret files
A short license file, a key file, or a settings file that contains passwords and must be shown to the application as a file.
Naming Keys Well
The names you give the entries inside an Opaque Secret become part of your daily vocabulary. Applications refer to them, deployment descriptions mention them, and teammates search for them during incidents. Choose names that say what the value is, such as database-password or payment-api-key, instead of vague ones like value1 or secret. If an application expects a value as a file, remember that the key name becomes the file name, so pick something a file system will accept and a human can recognize.
Consistency across the whole cluster pays off. If every team calls the same thing by a different name, tools and reviewers cannot spot mistakes at a glance. A short naming guide, agreed once and shared widely, prevents an entire category of confusing failures where an application looks for a key that exists under a slightly different spelling.
Designing a Good Opaque Secret
A little thought at the start saves trouble later. Group values that are always used together, and separate values that belong to different applications or that change at different times. A database password that rotates monthly should not share a Secret with an unrelated signing key that never changes, because updating one would force a needless touch of the other. Use key names that are clear and consistent, and avoid abbreviations that only their author understands.
An Opaque Secret is like a plain, unmarked envelope. You can put anything inside, and the post office does not care what it is. That freedom is useful, but it means the label on the outside, the name of the Secret and the names of its keys, is entirely your responsibility.
Storing a certificate and key as Opaque, and then wondering why an ingress or gateway ignores it. Those components look for the TLS type by design. Use Opaque only when no more specific type fits.
8TLS Secrets: Certificates and Private Keys
A TLS Secret holds the pair of files that lets a server prove its identity and encrypt its conversations.
What TLS Is
TLS is the technology that puts the small padlock in the browser’s address bar. It does two jobs. First, it lets a website prove that it really is who it claims to be, using a certificate issued by a trusted authority. Second, it encrypts everything sent between the visitor and the site, so that eavesdroppers on the network see only scrambled data. To do this, a server needs two things: the certificate, which is public and can be shown to anyone, and the private key, which must be kept strictly secret. The two are mathematically linked.
The certificate is like a passport issued by a trusted government: anyone can look at it to see who you claim to be. The private key is like the secret ability that only the real owner has, such as the exact signature that matches the passport. Showing the passport without being able to sign would prove nothing.
The Shape of a TLS Secret
A TLS Secret has exactly two required entries. One is labeled tls.crt and holds the certificate. The other is labeled tls.key and holds the private key. Kubernetes checks that both are present when the Secret is created, and rejects a TLS Secret that is missing either one. It does not, however, check for you that the key really matches the certificate, or that the certificate has not expired, so those remain your responsibility.
tls.crt
The certificate, and any intermediate certificates that link it to a trusted authority, with the server’s own certificate placed first.
tls.key
The private key that pairs with the certificate. This is the truly sensitive part and must never be shared or logged.
Public Authorities and Private Ones
Certificates for public websites are usually issued by authorities that every browser already trusts. Inside a company, teams often run their own private authority to issue certificates for internal services. Those certificates are trusted only by machines that have been told to trust the private authority. A TLS Secret can hold either kind, but the clients must know which authority to believe, or they will reject the connection as untrusted.
Some teams create self-signed certificates for quick tests. These are certificates that vouch for themselves, and no outside authority stands behind them. They are fine for a lab but unsuitable for anything customers touch, because browsers will show warnings. When in doubt, use a properly issued certificate for everything visible to the public, and a private authority for internal traffic.
Who Uses a TLS Secret
Most often, the consumer is the cluster’s front gate, the ingress or gateway, which needs the certificate to serve secure web pages for your applications. The Secret must live in the same namespace as the ingress that uses it. Applications can also mount the TLS Secret as files, to serve secure traffic themselves or to identify themselves to other services. In every case, the cluster hands the certificate and key to the consumer, which then does all the secure communication.
graph LR
V["Visitor's browser"] -->|"Secure connection"| G["Ingress or gateway"]
S["TLS Secret with certificate and key"] -.->|"Supplies identity"| G
G -->|"Forwards request"| A["Application pods"]
Certificates expire, often after a few months to a year. An expired certificate makes browsers show frightening warnings and can make automated clients refuse to connect at all. Expiry is one of the most common causes of sudden, embarrassing outages, so plan for renewal from the start.
9Docker Registry Secrets: Pulling Private Images
A Docker registry Secret holds the login that lets the cluster download application images from a private store.
What an Image Registry Is
Applications in Kubernetes are packaged as images, which are frozen bundles of the application and everything it needs. These images are stored in a registry, which is like a warehouse of ready-to-ship packages. Public registries let anyone download popular images. But most companies keep their own images in private registries, because the images contain proprietary code. To download from a private registry, you must log in, exactly as you would log in to any private website.
Who Does the Logging In
The one that actually downloads the image is not your application but the kubelet, the helper on the machine where the pod will run. Before the pod can start, the kubelet must fetch the image. If the registry is private, the kubelet needs credentials. Kubernetes lets you supply these through a Docker registry Secret, which is referenced from the pod’s description as an image pull secret. The kubelet reads it, logs in to the registry, downloads the image, and then starts the pod.
The pod is assigned to a machine
The scheduler decides where the pod will run, and the kubelet on that machine takes over.
The kubelet reads the pull secret
It finds the registry login named in the pod description and reads its contents.
The image is downloaded
Using the login, the kubelet downloads the image from the private registry.
The container starts
With the image in place, the container begins running. The application itself never sees the registry login.
Passwords Versus Access Tokens
A registry login can be a plain password, but many registries prefer access tokens. A token is a long, random string that stands in for a password and can be limited in what it may do and how long it lasts. Tokens have a big advantage: if one leaks, you can cancel just that token without changing the password of the whole account. For automated systems like clusters, a narrowly limited token, allowed only to download images, is almost always better than a personal password.
Whichever you choose, treat the stored login exactly like any other important secret. Limit who can read it, rotate it on a schedule, and remove it as soon as the cluster or the team no longer needs it. A forgotten pull login for a long-retired project is a small unlocked door that nobody remembers to check.
The Shape of the Secret
The modern type for this purpose is called Docker config JSON. It requires one entry, labeled .dockerconfigjson, whose content is a small structured document listing one or more registries and the login for each. For each registry it records the address, the username, the password, and often an email, along with an encoded combination of the username and password. Kubernetes checks that this entry exists and is properly structured. An older type, called Docker config, exists for legacy reasons, but new work should use the modern one.
Imagine a delivery driver who must collect packages from a private warehouse. The driver, not the customer, carries the warehouse pass. A Docker registry Secret is that pass, handed to the driver, the kubelet, so it can collect the image and bring it to the machine.
The registry Secret must live in the same namespace as the pods that use it. A pod cannot borrow a pull secret from another namespace, which means each namespace needs its own copy of the login.
10The Other Built-In Secret Types
Besides the three main types, Kubernetes offers a few more, each for a narrow, well-defined purpose.
Basic Authentication
Holds a username and a password under fixed key names. Useful when a tool expects exactly this shape and wants the cluster to check that both entries exist.
SSH Authentication
Holds a private key used to log in to remote machines or code repositories over the secure shell protocol.
Service Account Token
A legacy way to hold the identity token of a service account. Modern clusters prefer short-lived tokens delivered directly to pods, and no longer create these automatically.
Bootstrap Token
A short-lived token used when new machines join a cluster. It lives in a system namespace and is managed by cluster tooling.
Docker Config (Legacy)
The older form of registry login. It still works, but the newer Docker config JSON type is preferred for anything new.
Why Service Account Secrets Faded Away
In older clusters, every service account automatically received a Secret holding a long-lived token. This was convenient but risky: the token never expired, and anyone who copied it could use it forever. Newer clusters replace this with tokens that are issued on demand, tied to a specific pod, and valid only for a limited time. The identity token is delivered to the pod through a special kind of volume rather than a stored Secret. This is a good example of how the design of Secrets evolves toward shorter lifetimes and narrower access.
A permanent staff badge that never expires is dangerous if lost. A visitor pass that works only for one day, and only for one building, is far safer. The move from stored, permanent tokens to short-lived tokens is the move from permanent badges to day passes.
If you find old long-lived service account token Secrets in a cluster, treat them as a risk. Review who could have read them, and replace them with short-lived tokens where possible.
11Getting Secrets Into Pods
A Secret is useful only when an application can read it, and Kubernetes offers two main delivery routes, each with its own trade-offs.
Route One: Environment Variables
In the first route, the value of a Secret entry is handed to the program as an environment variable when the container starts. The application simply reads the variable, which is the simplest possible integration and works with almost every programming language. It is popular for small values like passwords and tokens.
The drawbacks are real, however. Environment variables can be accidentally printed by debugging tools, included in crash reports, or inherited by other programs started by the application. They are also fixed at the moment the container starts, so if the Secret changes, the running application keeps the old value until it is restarted.
Route Two: Mounted Files
In the second route, the Secret is shown to the container as a folder of files, where each key becomes a file and the value is its content. The application opens the file and reads it. This route suits values that are naturally files, such as certificates and keys, and it has a useful extra property: when the Secret is updated, the files eventually change on their own, so a well-written application can pick up the new value without restarting. The files are kept in the memory of the machine and are not written to its disk, which reduces the chance of leftovers.
| Question | Environment Variables | Mounted Files |
|---|---|---|
| Ease of use | Very easy, works everywhere | Needs the application to read files |
| Updates after the Secret changes | Not seen until the container restarts | Changes appear after a short delay |
| Risk of accidental exposure | Higher, easily printed or inherited | Lower, but files must have sensible permissions |
| Best suited for | Short values like passwords and tokens | Certificates, keys, and multi-line values |
| Stored on the machine’s disk | Not applicable | No, kept in memory |
graph TD
S["Secret in the cluster"] --> E["Delivered as environment variable"]
S --> F["Delivered as mounted file"]
E --> A1["Read once at startup"]
F --> A2["Read from file, may refresh later"]
An environment variable is like a note handed to an employee at the start of the day: quick and easy, but if the boss changes the instructions at noon, the employee is still working from the morning note. A mounted file is like a notice board in the office, which the employee can check again whenever needed, and which the boss can update during the day.
Prefer mounted files for anything sensitive or likely to change, and always make sure your application never prints the values it receives, whichever route you choose.
12How a Secret Travels From Storage to Container
Following one Secret on its whole journey shows exactly where it is exposed and where it is protected.
The Journey Step by Step
A person or a tool creates the Secret by sending it to the API server. The API server checks that the sender is allowed to do this, checks the shape of the Secret against its type, and then writes it into etcd, the cluster’s database. Later, when a pod that needs the Secret is scheduled onto a machine, the kubelet on that machine asks the API server for the Secret. The API server checks that this kubelet has a legitimate reason to see it, meaning that a pod on that machine really refers to it. Only then does the Secret leave the database and travel to the machine.
On the machine, the kubelet places the Secret in memory-backed storage, presents it to the container as files or variables, and starts the container. When the pod is removed, the kubelet cleans up the files. An important design choice appears here: Secrets are sent only to machines that actually run pods needing them, not to every machine in the cluster. This limits how many places a Secret can be exposed.
sequenceDiagram
participant U as Person or tool
participant A as API server
participant E as etcd storage
participant K as Kubelet on the machine
participant P as Pod container
U->>A: Create the Secret
A->>A: Check permission and shape
A->>E: Store the Secret
K->>A: Pod needs this Secret
A->>A: Check that the kubelet has a valid reason
A-->>K: Send the Secret
K->>K: Place in memory-backed storage
K->>P: Start container with the Secret
What Happens When the Pod Ends
Cleaning up matters as much as delivering. When a pod is removed, the kubelet deletes the files it created for that pod’s Secrets from the machine’s memory-backed storage. Because these files never touched the machine’s disk, there are no leftovers for someone to recover later from a discarded drive. This tidy behavior is one of the quiet benefits of using the built-in Secret object instead of copying values by hand onto machines.
The cleanup does not reach into places the application itself wrote. If an application copies a password into its own log file, a cache, or a temporary folder that persists, the kubelet knows nothing about it. That is why careful application behavior remains an essential final link in the chain.
The Places of Exposure
Every stop on this journey is a place where a Secret can be seen, and each stop needs its own protection. In the database, the Secret must be encrypted and the backups guarded. At the API server, strict rules must decide who may read. On the machine, the memory-backed storage and the file permissions matter. Inside the application, the code must handle the value carefully. A chain is only as strong as its weakest link, and beginners often protect one link while forgetting the others.
In Transit to the Cluster
Connections to the API server are encrypted, so the Secret cannot be read on the network while being created.
In the Database
By default stored without encryption. Turn on encryption at rest and protect backups.
In the API
Access rules decide who may read the Secret. Overly broad rules are the most common weakness.
On the Machine
Kept in memory, but anyone with deep access to the machine could still reach it. Protect your machines too.
In the Application
The code must not print, log, or forward the value. This is the last link and often the weakest.
Sending valuable jewelry involves the shop’s safe, the delivery truck, the receiving building’s front desk, and the customer’s own drawer. A thief only needs the weakest one. Protecting the truck but leaving the customer’s drawer open would waste the effort.
Anyone who can create a pod in a namespace can usually mount any Secret in that namespace into their pod and read it. Permission to run pods is therefore a powerful permission, and should be granted as carefully as permission to read Secrets directly.
13Immutable Secrets and Update Timing
Sometimes the safest Secret is one that cannot be edited, and knowing exactly when an update reaches an application avoids nasty surprises.
The Immutable Switch
Every Secret can carry a switch that marks it as immutable. Once this switch is on, the data inside can never be edited. If the value needs to change, you must create a new Secret and point your applications at it. This sounds inconvenient, but it brings two real benefits. First, it protects against accidental or malicious edits: a stray command cannot silently change a password that hundreds of pods rely on. Second, it reduces work for the cluster, because the machines no longer need to keep watching an immutable Secret for changes.
The trade-off is that the switch cannot be turned back off. To make a change, the Secret must be deleted and recreated, and the pods that use it must be restarted to notice. Many teams pair this with a naming habit in which each version of a Secret gets its own name, such as a base name followed by a version number. The application description then points to a specific version, and changing the version in the description triggers a controlled restart of the application.
Think of a printed and laminated notice on a wall. Nobody can scribble on it by accident, and everyone can trust that it says the same thing all day. If the rule changes, you print a new notice and replace the old one, and everyone knows exactly which version they are reading.
When an Update Reaches a Running Application
When a Secret that is not immutable is edited, the news does not reach applications instantly. The delay depends on how the Secret was delivered. Files mounted from the Secret are refreshed by the kubelet on its regular schedule, and there is a short cache in between, so the change typically shows up after a minute or so. Environment variables are never refreshed while the container runs, because the operating system fixes them at startup. And files mounted using a special sub-path option are not refreshed either, which surprises many people.
| Delivery Method | Sees Updates While Running? | What To Do After Changing the Secret |
|---|---|---|
| Environment variable | No | Restart the pods |
| Mounted folder of files | Yes, after a short delay | Ensure the application rereads the files |
| Single file mounted with sub-path | No | Restart the pods, or avoid the sub-path option |
| Immutable Secret | Cannot be edited | Create a new Secret and update the pod description |
A team rotates a password by editing the Secret and assumes all applications now use the new value. Those using environment variables continue with the old password until restarted, and when the old password is finally disabled, they suddenly fail. Always plan a restart or a reread when rotating.
14Encryption at Rest and the Cluster Database
Turning on encryption in storage is the single most important step for making Secrets more than a labeled drawer.
What At Rest Means
Data at rest means data that is sitting in storage, as opposed to data traveling across a network. All Kubernetes objects, including Secrets, are stored in the cluster’s database. By default, the database holds Secrets in a form that anyone with access to the database files can read. That includes people who can read backups, people who can access the disks of the database machines, and anyone who obtains a copy of a database snapshot. Encryption at rest scrambles Secrets before they are written, so that stolen files reveal nothing without the key.
How It Is Set Up
Cluster administrators configure the API server with a list of encryption providers and which kinds of objects to protect. The list is ordered: the first provider is used to encrypt new writes, and the others are tried when reading older data. Common choices include providers using well-known encryption methods with keys stored in a configuration file, and providers that hand the key management to an external key management service, so that the key never sits next to the data. The external approach is stronger, because an attacker must break into two separate systems.
No Encryption
The default in a plain cluster. Secrets are stored readable by anyone with access to the storage files.
Local Key Encryption
The API server encrypts Secrets using a key held in its own configuration. Better than nothing, but the key sits near the data.
External Key Service
Encryption keys live in a separate key management service, so stealing the database alone is not enough to read Secrets.
Managed Cluster Defaults
Many managed cluster services offer built-in encryption options. Check what is on by default and what you must turn on yourself.
Two Details That Catch People Out
First, turning on encryption only affects data written afterwards. Secrets that were stored before remain readable until they are rewritten, so administrators run a step that rewrites every Secret to apply the new protection. Second, keys must themselves be rotated and backed up. If the key is lost, the encrypted Secrets are lost with it. Encryption at rest is a strong shield, but it needs the same care as any other important system.
Encrypting the database is like putting a lock on the filing cabinet in the records room. Old files that were already in the cabinet must be taken out and put back before the lock protects them. And you must keep the key somewhere safe, but not taped to the cabinet.
Protect backups of the cluster database as strictly as the live database. A backup contains every Secret, and it often has weaker guarding than production systems.
15Controlling Access With RBAC
The rules that decide who may read Secrets are the front line of defense, and they are often set far too loosely.
The Actions That Matter
RBAC rules describe which actions are allowed on which kinds of objects. For Secrets, three actions matter most. The get action reads one Secret by name. The list action reads all Secrets in a namespace at once. The watch action streams changes to Secrets as they happen. A surprise for many beginners is that list and watch reveal the full contents, not just the names. Granting someone permission to list Secrets is effectively granting permission to read every Secret in scope.
Get
Reads a single named Secret. It can be limited to specific names, which is the tightest and safest form of access.
List
Reads every Secret in scope, including the values. Grant it very rarely.
Watch
Streams changes and contents as they happen. Equivalent in power to list, so treat it the same way.
Create and Update
Lets someone add or change Secrets, which can also let them redirect applications to values they control.
Separating People From Programs
People and programs have very different needs, and their access should be designed separately. A running application usually needs no direct permission to read Secrets through the cluster’s interface, because the cluster already places what its pods need in their containers. Giving applications broad read access to Secrets only widens the damage if one of them is compromised.
People, on the other hand, sometimes need to inspect or replace a Secret. Rather than granting permanent access, many teams give temporary access that expires after a few hours and leaves a record of who asked and why. This approach keeps the everyday state of the cluster locked down, while still letting people fix real problems quickly.
Least Privilege in Practice
The guiding principle is least privilege: give each person and each program only the access needed for their job. In practice, this means avoiding broad rules that cover all Secrets in the whole cluster, avoiding wildcard rules, and preferring rules limited to a namespace. Applications generally do not need permission to read Secrets through the cluster’s interface at all, because the cluster already delivers what their pods need. Humans rarely need day-to-day access to production Secrets either, and when they do, the access should be temporary and recorded.
In an office, the master key that opens every cabinet is held by very few people, while most employees carry a key for only their own drawer. Giving everyone the master key because it is easier would defeat the whole point of having locks.
The Hidden Power of Creating Pods
As we saw earlier, anyone who can create a pod in a namespace can usually mount any Secret in that namespace and read it. This means that permission to run pods is nearly as powerful as permission to read Secrets. Teams that lock down Secrets carefully but hand out permission to run pods freely have left a side door wide open. Combine tight Secret rules with careful control over who can create pods, and use namespaces to keep sensitive Secrets apart from areas where many people can run workloads.
Problem
A single broad role that allows reading all Secrets across all namespaces is assigned to a large group of developers, for convenience during troubleshooting.
Why It’s Harmful
Every member of the group, and every stolen account among them, can read every production password, certificate, and registry login in the cluster.
Correct Approach
Grant narrow, namespace-level access, use named-Secret rules where possible, and provide temporary, recorded access for real emergencies.
16Keeping Secrets Out of Code Repositories
Modern teams describe their whole cluster in files kept in version control, which creates a dangerous temptation for Secrets.
The GitOps Dilemma
Many teams practice a style called GitOps, in which the desired state of the cluster is written in files stored in a code repository, and automatic tools make the cluster match those files. This is wonderful for ordinary objects, giving history, review, and easy rollback. But a Secret file contains sensitive values in easily decoded form. Putting it in the repository means everyone with read access to the repository can read the Secret, and the repository’s history keeps every old version forever, even after the file is deleted.
Leaks through repositories are among the most common real-world incidents. Automated scanners run by attackers watch public repositories and can find a freshly committed key within minutes. Even private repositories are risky, because access tends to widen over time as more people, tools, and contractors join.
A repository is like a filing cabinet that keeps every version of every page ever written. If someone once wrote the vault combination on a page and later erased it, the earlier version is still in the cabinet. Only never writing it there in the first place is truly safe.
Approaches That Work
Encrypt before committing
Tools exist that encrypt Secret values with a key before they are written to the repository. Only the cluster, or people with the key, can decrypt them. The repository holds harmless scrambled text.
Keep values in an external manager
The repository holds only a reference, such as the name of an entry in a secret manager. The real value is fetched at run time and never appears in the repository.
Create Secrets out of band
A trusted person or automated process creates the Secret directly in the cluster, separately from the repository, and the repository never mentions the value.
Scan for accidents
Automatic scanning of commits catches keys and passwords before they are published, giving a chance to fix the mistake and rotate the value.
If a real Secret was committed, deleting the file is not enough. Assume the value is compromised, replace it with a new one, and rotate anything that depended on it. History cannot be trusted to forget.
17External Secret Managers
Dedicated secret management systems offer stronger protection and richer features than the built-in Secret object alone.
Why Go Beyond the Built-In Object
The Secret object is a basic tool. It has no built-in expiry, no automatic rotation, no detailed history of who read what, and limited ways to hand out short-lived values. Dedicated secret managers, offered by cloud providers and by independent products, fill these gaps. They store values in specialized, heavily protected systems, record every access, support automatic rotation, and can generate short-lived credentials on demand. Organizations with strict requirements often make such a manager the source of truth, and treat the cluster as merely a place where values are delivered.
Two Common Ways to Connect Them
The first way is a synchronizing helper that runs in the cluster. It reads values from the external manager and creates or updates ordinary Secrets in the cluster. Applications continue to use Secrets exactly as before, and the helper keeps them fresh. The second way is a driver that mounts values from the external manager directly into pods as files, without ever creating a Secret object in the cluster. This reduces the number of places the value is stored, at the cost of a little more setup.
graph LR
M["External secret manager"] -->|"Helper reads values"| H["Sync helper in cluster"]
H -->|"Creates or updates"| S["Kubernetes Secret"]
S --> P1["Pods use it as usual"]
M -->|"Driver mounts directly"| D["Storage driver"]
D --> P2["Pods read files"]
| Question | Sync Helper | Direct Mount Driver |
|---|---|---|
| Creates a Secret object in the cluster | Yes | Usually not, unless asked to |
| Works with existing applications unchanged | Yes, they use ordinary Secrets | Needs the application to read mounted files |
| Places where the value is stored | External manager and the cluster database | External manager and pod memory |
| Extra parts to operate | A helper program | A driver on each machine |
A bank vault holds valuables with logs of every visit. Some companies send a courier who copies a needed item into the office safe, and others let staff view it briefly in a private room at the bank. Both keep the main vault as the source of truth, but they differ in how much of the valuable leaves the vault.
An external manager adds a dependency. If it becomes unreachable, new pods may not start. Plan for this with sensible caching, retries, and knowing exactly how your applications behave if the manager is briefly unavailable.
18The Life of a TLS Certificate
A certificate is born, works for a while, and expires, so a healthy system must handle its whole life automatically.
Issuing and Renewing
A certificate is issued by a certificate authority, an organization that browsers and operating systems trust. The owner proves control of the domain name, and the authority signs a certificate. Some authorities issue certificates that last only ninety days, which keeps mistakes short-lived but demands frequent renewal. Doing this by hand across dozens of applications is a recipe for a missed date and a sudden outage. A common answer is a helper program in the cluster that requests, receives, and renews certificates automatically, and stores each one in a TLS Secret.
A request is made
The helper program asks the authority for a certificate for a named domain.
Control of the domain is proven
The authority sets a small challenge, and the helper answers it to show the domain really belongs to the requester.
The certificate is stored
The signed certificate and its private key are written into a TLS Secret in the right namespace.
The gateway picks it up
The ingress or gateway notices the updated Secret and begins serving the new certificate.
Renewal begins early
Well before expiry, the helper repeats the process, so the site never runs out of valid certificate.
Certificates Between Internal Services
Not all certificates face the public. Modern systems often encrypt traffic between their own internal services, and sometimes require both sides to present certificates, so that each service can prove its identity to the other. This is called mutual authentication, and it turns every internal call into a checked conversation instead of an open one. The certificates involved are usually short-lived and issued automatically by a private authority, and stored in TLS Secrets or delivered by a helper directly to the pods.
The lifecycle rules are the same as for public certificates, but the number of certificates can be huge, one per service or even per pod. This makes automation not just helpful but essential. Nobody can renew thousands of short-lived certificates by hand, and a single missed renewal can silently break one link in a chain of services.
Common Certificate Pitfalls
Several errors recur. The certificate file must list the server’s own certificate first, followed by any intermediate certificates, because some clients reject an incomplete chain. The key must match the certificate, or the gateway will fail to start serving it. A single certificate may cover many names, or use a wildcard to cover a whole family of names, which is convenient but means a leak has a wider effect. And certificates used between internal services need the same lifecycle care as public ones, even though nobody sees a padlock.
A certificate is like a passport with an expiry date. A wise traveler starts the renewal months before the trip, not on the airport curb. Automatic renewal is a personal assistant who keeps every passport current without anyone remembering.
Automation can fail silently. Monitor the days left before each certificate expires, and alert well in advance, so a broken renewal becomes a calm ticket and not a midnight emergency.
19Registry Credentials at Scale
One registry login for one pod is easy, but a cluster with many namespaces and many teams needs a smarter approach.
Attaching to Service Accounts
Naming a pull secret in every single pod description is repetitive and easy to forget. A neater method is to attach the pull secret to the service account that the pods use. Any pod running under that service account then receives the registry login automatically, without naming it. Teams often set this up once per namespace, so that developers never have to think about registry logins when writing their application descriptions.
Copies in Every Namespace
Because a pod can only use pull secrets from its own namespace, a cluster with fifty namespaces needs fifty copies of the login. Doing this by hand invites drift: some copies expire while others do not. Many teams use a small automated helper that copies a master pull secret into each new namespace and keeps the copies up to date. This centralizes the change: when a password rotates, one edit spreads everywhere.
Short-Lived Cloud Registry Tokens
Registries run by cloud providers often issue login tokens that last only a few hours. A static Secret holding such a token works at first and then fails a few hours later, with pods suddenly unable to pull images. Solutions include a helper that refreshes the Secret regularly, or a machine-level plug-in that lets the kubelet obtain fresh credentials itself, using the identity of the machine. The second approach avoids storing registry passwords in Secrets at all, which is both simpler to maintain and safer.
Pull Secret in Each Pod
Simple for one application, but repetitive and easy to forget across many.
Attach to the Service Account
Pods using that identity get the login automatically, so descriptions stay short and consistent.
Automated Copies Across Namespaces
A helper spreads and refreshes one master login into every namespace that needs it.
Machine-Level Credential Plug-In
The kubelet obtains short-lived registry credentials itself, so no long-lived password is stored in the cluster.
Giving every delivery driver a personal copy of the warehouse pass is tiring to manage, and the passes go missing. Giving each driver’s company a pass that is checked at the gate, and renewing it centrally, is much easier to keep tidy.
Use registry accounts that can only pull images, not push or delete them. If a pull login leaks, an attacker can read images, but cannot replace them with malicious versions.
20Monitoring, Auditing, and Detection
Protection is not complete without the ability to notice when something unusual happens to a Secret.
Audit Logs
The API server can record who did what, and when. When audit logging is turned on for Secrets, every read, creation, change, and deletion leaves a record. This turns mysteries into answers: who read the production database password last Tuesday? Which account changed the certificate? Because Secret contents are sensitive, audit settings are usually chosen so that the record notes that a Secret was accessed without copying its value into the log.
Unusual Reads
A person or program reading many Secrets at once, or reading Secrets it never used before, is a warning sign worth alerting on.
Changes Outside Process
Edits made directly, outside the normal automated pipeline, should be visible and questioned.
Days Until Certificate Expiry
Track the remaining lifetime of every certificate and alert long before the date arrives.
Image Pull Failures
A sudden rise in failed image downloads often means an expired or wrong registry login.
When Something Looks Wrong
Detection is only useful if someone knows what to do next. Decide in advance the steps for a suspected exposure: identify which Secret is involved, replace its value at the source, restart or refresh the applications that use it, and check the audit records to see whether the old value was read by anyone who should not have. Practice these steps in calm moments, so that during a real incident the team follows a checklist instead of inventing a plan under pressure.
Speed matters, and so does calm. The first hour after a suspected leak is when the most damage can be prevented, and also when the most mistakes are made. A rehearsed routine turns a stressful event into a set of familiar tasks.
Scanning and Hygiene
Regular scans can find Secrets that should not exist: old service account tokens, unused Secrets, values committed by mistake, and Secrets shared across too many applications. Housekeeping matters because an unused Secret still carries risk, and nobody remembers to protect what they have forgotten. A simple periodic review, listing every Secret with its owner and its last change, keeps the collection tidy and reveals surprises early.
A building with a visitor logbook and cameras at the vault door does not stop every intruder, but it makes theft harder and investigations possible. Audit logs and alerts play the role of the logbook and cameras for your Secrets.
Keep a short inventory listing each Secret, what it is for, who owns it, and when it should be rotated. During an incident, this inventory turns panic into a checklist.
21Troubleshooting Secret Problems
A short list of common symptoms and their usual causes will resolve most puzzling situations quickly.
Most Secret problems fall into a few families. A Secret is missing or in the wrong place. A key inside it is misspelled. The type does not match what a consumer expects. Or the value has expired or changed without the application noticing. Start with the simplest checks: does the Secret exist in the right namespace, and does it have the expected keys?
| Symptom | Likely Cause | What To Check |
|---|---|---|
| Pod stuck waiting to start, mentions a missing Secret | The Secret does not exist in the pod’s namespace, or the name is misspelled | Namespace and exact name of the Secret |
| Container starts but the value is missing or empty | The key name does not match what the pod description asks for | Exact key names inside the Secret |
| TLS Secret rejected at creation | Missing tls.crt or tls.key | Both entries present and properly formed |
| Gateway shows the wrong or default certificate | Secret is in a different namespace, or has the wrong type | Namespace, type, and the name referenced by the ingress |
| Browsers report a certificate problem | Expired certificate, missing intermediate, or name mismatch | Expiry date, chain order, and covered names |
| Image cannot be pulled, access denied | Pull secret missing, in the wrong namespace, or holding wrong or expired login | Pod’s pull secret name, namespace, and registry address in the login |
| Image pulls worked yesterday and fail today | Short-lived registry token expired | Token lifetime and refresh mechanism |
| Application still uses old password after rotation | Value delivered as environment variable, or file not reread | Delivery method and whether the application rereads |
| Cannot edit an existing Secret | It was marked immutable | Create a new Secret and update the pod description |
A Calm Order of Investigation
First confirm the Secret exists in the correct namespace and has the expected type and keys. Second confirm that the consumer refers to it by the exact name and key. Third confirm that the delivery method supports the behavior you expect, especially around updates. Fourth, confirm the value itself: is it valid, current, and the right one for this environment? Careful investigators view the structure of a Secret without printing its values into shared terminals or chat channels.
Never paste the full contents of a Secret into a chat, ticket, or shared document while troubleshooting. Describe the shape and the error, and check the values privately. Debugging is one of the most common moments when Secrets leak.
22Choosing the Right Type and the Right Object
A simple set of questions will lead you to the correct type, and to the decision of whether you need a Secret at all.
Secret or ConfigMap?
The first question is whether the value is sensitive. If leaking it would hurt, use a Secret. If it is only a setting, a ConfigMap is the right object. Secrets carry extra handling, such as being kept in memory on machines and being separately guarded, so using them for ordinary settings adds friction without benefit. Equally, using a ConfigMap for a password gives up protections you would want.
| Feature | Secret | ConfigMap |
|---|---|---|
| Intended content | Sensitive values | Ordinary settings |
| Stored on machines | In memory only | May be written to disk |
| Separate access rules are common | Yes, and strongly recommended | Usually looser |
| Supports types with required keys | Yes | No |
| Size limit | About one megabyte | About one megabyte |
Then Choose the Type
flowchart TD
A["You have a sensitive value"] --> B{"Is it a certificate and private key pair?"}
B -->|"Yes"| C["TLS type"]
B -->|"No"| D{"Is it a login to pull images from a registry?"}
D -->|"Yes"| E["Docker config JSON type"]
D -->|"No"| F{"Is it a username and password, or an SSH key expected in a fixed shape?"}
F -->|"Yes"| G["Basic authentication or SSH authentication type"]
F -->|"No"| H["Opaque type"]
Choosing a type is like choosing the right container at a shop. A wine bottle goes in a wine rack, a passport goes in a document wallet, and everything else goes in a general-purpose box. The specific container lets others, and the tools, know at a glance what is inside.
If a specific type exists and a tool expects it, use it. Fall back to Opaque only when your data has no fixed shape. The extra checking done by specific types costs nothing and catches mistakes early.
23Advantages and Trade-offs
Every tool has a price, and honest teams look at both sides before trusting it fully.
Advantages
- Keeps sensitive values out of application images and code
- Lets the same application run in many environments with different values
- Typed Secrets catch missing keys early and integrate with gateways and registries
- Delivered only to machines that run pods needing them
- Kept in memory on machines, not written to disk
- Native to Kubernetes, with no extra products needed to start
Disadvantages / Trade-offs
- Not encrypted by default, only encoded
- Broad access rules can expose every Secret in a namespace
- Permission to run pods effectively grants permission to read Secrets
- No built-in rotation, expiry, or rich access history
- Updates reach applications unevenly, depending on how they were delivered
- Easy to leak through repositories, logs, and debugging
When the Built-In Object Is Enough
For small clusters, low-risk values, and teams that are just starting, the built-in Secret, combined with encryption at rest and tight access rules, is a sensible and honest solution. Do not let the pursuit of a perfect system stop you from doing the basics well. Turn on encryption, restrict access, avoid committing values, and rotate what you can.
When You Need More
As requirements grow, such as strict audit demands, automatic rotation, short-lived credentials, or values shared across many clusters, the gaps of the built-in object become painful. That is the moment to bring in an external secret manager. The two approaches are not rivals but stages, and the concepts you learn from the built-in object carry over directly.
A drawer with a lock is perfect for a family home. A bank needs a vault, guards, and cameras. Both are valid, and the right choice depends on what you are protecting and how much rides on losing it.
Secrets are neither a weakness nor a complete solution. They are a solid building block that rewards care and punishes carelessness. What surrounds them matters more than the object itself.
24Best Practices and Common Mistakes
Teams that handle Secrets well follow the same habits, and teams that suffer breaches repeat the same mistakes.
Habits That Work
Turn on encryption at rest
Do this early, rewrite existing Secrets so they are protected, and guard the keys and the backups with equal care.
Give each Secret one clear purpose
Separate values that belong to different applications or rotate at different times, so a change to one never disturbs another.
Prefer mounted files and short lifetimes
Files are less likely to leak than environment variables and can refresh after updates. Prefer short-lived credentials wherever they are available.
Rotate on a schedule
Change passwords and keys regularly and after any suspected exposure, and practice the rotation so it is routine and not frightening.
Keep an inventory with owners
Know which Secrets exist, what they protect, and who is responsible. Unowned Secrets are the ones that outlive their purpose and become risks.
Mistakes to Avoid
Problem
A Secret file with real production values is committed to the team’s code repository, because the encoded values look unreadable.
Why It’s Harmful
Encoded values are decoded in seconds. Everyone with repository access, now and in the future, and any leaked copy, exposes the values. History keeps them even after deletion.
Correct Approach
Encrypt Secrets before committing, or keep values in an external manager and store only references in the repository. Rotate any value that was ever committed.
Problem
One large Secret holds every password for every application, and every application is allowed to mount it.
Why It’s Harmful
A compromise of any one application exposes every other application’s credentials. Rotating one value forces the whole Secret to change and every application to restart.
Correct Approach
Create small Secrets, one per application and purpose, and let each application mount only the Secrets it truly needs.
Problem
Applications print their whole configuration, including passwords received from Secrets, into their logs at startup to help with debugging.
Why It’s Harmful
Logs are copied to many systems, kept for long periods, and read by many people. A password in a log has effectively been published to everyone with log access.
Correct Approach
Never log secret values. Log only that a value was loaded, and mask sensitive fields in any diagnostic output.
Problem
A certificate is issued once by hand, stored in a TLS Secret, and left unmonitored until a customer reports that the site shows a security warning.
Why It’s Harmful
Expired certificates break trust and can bring down services that depend on secure connections. The outage was entirely predictable.
Correct Approach
Automate issuing and renewal, and monitor the days left before expiry, with alerts long before the deadline.
25Real-World Examples and an Adoption Roadmap
The same few patterns appear again and again in real organizations, and a staged path helps you reach a mature setup safely.
Online shops and payment providers
Shops keep payment provider keys and database passwords in Opaque Secrets, serve their storefront with TLS Secrets held by the gateway, and pull private application images with a registry Secret attached to each namespace’s service account.
Regulated industries
Banks and health providers add encryption at rest with an external key service, strict audit logging of Secret access, and an external secret manager as the source of truth, so that every access can be explained to an auditor.
Software companies with private images
Companies that keep proprietary images in private registries rely on pull secrets, often refreshed automatically because their cloud registries issue short-lived tokens.
Platform teams serving many product teams
Platform teams offer a standard path: certificates issued and renewed automatically, registry logins copied into each namespace, and clear rules on how to request and rotate application Secrets.
A Roadmap for Growing Safely
Find what you already have
List existing Secrets, look for passwords hidden in plain settings and images, and note who can read what.
Fix the basics
Move sensitive values into Secrets with the right types, remove them from code and repositories, and rotate anything that was exposed.
Encrypt and restrict
Turn on encryption at rest, tighten access rules, and limit who can create pods in sensitive namespaces.
Automate lifecycles
Automate certificate renewal and registry login refresh, and add alerts for approaching expiry.
Add audit and detection
Turn on audit logging for Secrets, watch for unusual reads, and keep an inventory with owners.
Consider an external manager
When needs outgrow the built-in object, introduce a dedicated secret manager for rotation, short-lived credentials, and richer history.
26Frequently Asked Questions
Not by default. The values are only encoded, which anyone can reverse. To have real protection, you must enable encryption of Secrets in the cluster’s storage, control who may read them, and handle them carefully in applications.
Both hold key and value data. A ConfigMap is for ordinary settings, while a Secret is for sensitive values. Secrets have extra handling, such as being kept in memory on machines and being easier to guard with separate access rules and encryption.
Use Opaque for custom sensitive values with no special shape, such as passwords, tokens, and application keys. If a more specific type fits, such as TLS or Docker config JSON, use that instead.
Two: tls.crt for the certificate, and tls.key for the private key. Kubernetes checks that both exist but does not check that they match or that the certificate is still valid.
Common causes are that the Secret is in a different namespace from the pod, that the pod does not name it as an image pull secret, that the login is wrong or expired, or that the registry address in the Secret does not match the image’s registry.
If the Secret is mounted as files, the files update after a short delay, and an application that rereads them will see the change. If it was delivered as environment variables, the application keeps the old value until it restarts.
No. The type is fixed at creation. Create a new Secret with the right type and remove the old one.
Roughly one megabyte. Secrets are meant for small values. Larger files belong in proper storage, with only a small key or reference kept in a Secret.
Not as plain Secret files. The encoded values are easily decoded, and repository history keeps them forever. Use tools that encrypt Secrets before they reach the repository, or keep the values in an external secret manager.
Anyone allowed to read Secrets through the cluster’s rules, and anyone who can create pods in the same namespace and mount them. Also anyone with deep access to the database or its backups. Keep all of these groups small.
27Summary and Key Takeaways
Kubernetes Secrets give applications a dedicated place to receive sensitive values, separate from code and ordinary settings. The type of a Secret tells the cluster and its tools what to expect inside. Opaque is the flexible general-purpose type for your own passwords and tokens. TLS holds a certificate and private key pair for secure web traffic. Docker config JSON holds the login that lets machines download images from private registries. A few other types serve narrow, specialized purposes.
The most important lesson is that a Secret is a container for sensitive data, not a guarantee of safety. Base64 encoding hides nothing, so real protection must come from strict access rules, encryption in storage, careful delivery to pods, and disciplined handling in applications. Teams that understand this build systems in which a single mistake does not become a disaster.
Key Takeaways
- Secrets separate sensitive data — they keep passwords, keys, and logins out of code and ordinary settings.
- Base64 is not encryption — anyone who can read a Secret can decode it instantly, so protection must come from elsewhere.
- Type gives meaning — Opaque is for custom values, TLS holds a certificate and key, and Docker config JSON holds registry logins.
- Choose the most specific type — specific types let the cluster catch mistakes early, and the type cannot be changed later.
- Prefer mounted files for sensitive values — files can refresh after updates and are less likely to leak than environment variables.
- Guard every stop on the journey — storage, access rules, machines, and application code each need their own protection.
- Plan for change — certificates expire and passwords rotate, so build renewal and rotation in from the start.