Supply Chain Security in Kubernetes: SBOM, Image Signing and Admission Verification

Supply Chain Security in Kubernetes: SBOM, Image Signing and Admission Verification

A plain-English guide to knowing what is inside your containers, proving who built them, and refusing to run anything that cannot show its papers.

Imagine you run a restaurant. You can have the cleanest kitchen in the city, the best chefs and the strictest hygiene rules. But if the flour you buy has been tampered with before it reaches your door, none of that matters. Your customers still get sick. The danger did not come from your kitchen. It came from the chain of suppliers behind it.

Software works the same way. A modern application is not written from scratch. It is assembled from hundreds of open-source libraries, a base operating system image, build tools, and automated pipelines, many of them made by people you have never met. Each piece is a supplier. If an attacker poisons just one of them, the poison flows straight into your Kubernetes cluster, wrapped in a container that looks perfectly normal.

This guide teaches the three big defenses that the industry has settled on. First, the SBOM, a complete ingredient list for your software. Second, image signing, a tamper-proof seal that proves who made a container and that nobody changed it. Third, admission verification, a security guard at the cluster door who checks the seal and the ingredient list before letting anything in. By the end, you will understand how the pieces fit together, how they work inside, and how to roll them out without breaking your teams.

1Why the Software Supply Chain Is Under Attack

Attackers follow the path of least resistance. Increasingly, that path runs through the tools and ingredients that developers trust, not through the front door of the finished application.

What is a software supply chain?

A software supply chain is everything that touches your code on its way from a developer’s keyboard to a running program in production. That includes the source code repository, the third-party libraries your code pulls in, the build servers, the scripts that package everything, the registry where finished images are stored, and the cluster that finally runs them. Each step is a link in a chain. A chain is only as strong as its weakest link, and attackers love to hunt for that link.

Everyday Analogy

Think of a packaged medicine. A trustworthy bottle shows the ingredient list, a batch number, a tamper-proof seal over the cap, and a pharmacy that checks the seal before handing it over. Software containers need the same four things: an ingredient list (SBOM), a batch record (provenance), a seal (signature) and a pharmacist who checks (admission control).

Attacks that changed the industry

These real events explain why every large company now talks about supply chain security.

  • SolarWinds (2020). Attackers broke into the build system of a popular network management product and slipped malicious code into a legitimate software update. Thousands of customers installed the update, trusting that it was genuine. The code was signed by the real vendor, because it was built by the real vendor’s pipeline.
  • Codecov (2021). A tool that many teams ran inside their build pipelines was altered, so that it quietly sent secrets from those pipelines to the attacker.
  • Log4Shell (2021). A serious flaw was found in Log4j, a logging library buried inside countless Java applications. The big problem was not only the flaw. It was that most organizations could not answer a simple question: “Which of our systems contain Log4j?”
  • The xz backdoor (2024). A hidden backdoor was planted into a widely used compression library by an attacker who spent years earning the trust of its maintainers. It was caught by luck, before reaching most production systems.
  • Dependency confusion (2021). A researcher showed that build tools could be tricked into downloading a public package instead of a company’s private one that had the same name, and ran code inside many large companies.
Hundreds
of third-party packages inside a typical application
1 weak link
is enough to compromise the whole chain
Seconds
it takes a bad image to start once it is accepted

Why Kubernetes makes this more urgent

Kubernetes is built to run containers quickly and at scale. Its job is to take an image name and start it. By default, it does not ask who built the image, what is inside it, or whether it was changed along the way. If a pod asks for an image, and the registry hands it over, the cluster runs it. That convenience is wonderful for speed and dangerous for safety.

A single compromised image, deployed through an automated pipeline, can reach hundreds of nodes within minutes. The scale that makes Kubernetes valuable is the same scale that makes a supply chain attack expensive.

!
Watch Out

Scanning your own application code for bugs is not enough. Most of the code in a container was never written by your team. Supply chain security is about the trustworthiness of everything you did not write.

The three questions every cluster should be able to answer

  • What is inside this image? This is answered by the SBOM.
  • Who built it, and was it changed afterward? This is answered by signatures and provenance.
  • Is it allowed to run here? This is answered by admission policy.

The rest of this guide is a tour of how to answer those three questions automatically, for every image, every time.

2The Path From Code to Cluster

Before adding defenses, we need a map. Seeing the whole path shows exactly where each defense belongs.

The five stages

1

Source

Developers write code and keep it in a repository. The risks here include stolen developer accounts, malicious commits and unreviewed changes.

2

Dependencies

The code pulls in open-source libraries and base images. The risks include poisoned packages, look-alike names and abandoned projects taken over by strangers.

3

Build

A build system compiles the code and packages it into an image. The risks include a hacked build server, leaked secrets and tampered build scripts.

4

Registry

The finished image is stored in a registry. The risks include changed tags, stolen registry credentials and swapped images.

5

Cluster

Kubernetes pulls the image and runs it. The risk is running something that was never checked.

Where each defense fits

At the build

Create the SBOM

While the image is being built, a tool lists every package inside it and writes down the result. The list travels with the image.

At the build

Sign and attest

The pipeline signs the image and records signed statements about how it was built. These are proofs that someone can check later.

At the registry

Store everything together

The image, its signature, its SBOM and its build record are stored side by side, so they can be found by anyone who has the image.

At the cluster

Verify at admission

When a pod is created, a policy engine checks the signature and the attached records. If anything is missing or wrong, the pod is refused.

flowchart LR
    DEV["Developer commits code"] --> REPO["Source repository"]
    REPO --> CI["Build pipeline"]
    DEPS["Open source dependencies"] --> CI
    CI -->|"build image"| IMG["Container image"]
    CI -->|"generate"| SBOM["SBOM"]
    CI -->|"sign"| SIG["Signature and provenance"]
    IMG --> REG["Container registry"]
    SBOM --> REG
    SIG --> REG
    REG -->|"pull"| ADM["Admission policy check"]
    ADM -->|"allowed"| K8S["Kubernetes cluster runs the pod"]
    ADM -->|"denied"| REJ["Pod rejected"]
Figure 1 – The path from source to cluster, with the SBOM and signature created at build time and checked at admission time.

The big idea: create evidence early, check it late

Notice the pattern in the figure. All the evidence is created at the left side, where the build happens and the facts are known. All the checking happens at the right side, at the door of the cluster, where the decision to run must be made. The registry in the middle is just storage, and it is not trusted to tell the truth by itself. This design matters because it keeps the trust anchored in cryptography, not in the location of a file.

i
Good to Know

Security people call this “shifting trust from places to proofs”. Instead of saying “this image came from our registry, so it must be fine”, you say “this image carries proof that our pipeline built it, so it must be fine.”

3Container Images: Tags, Digests and Why Names Lie

To sign or verify an image, you must be able to point at exactly one image. That sounds easy, but image names are surprisingly slippery.

What is inside an image?

A container image is not a single file. It is a small package made of three kinds of parts. The layers hold the actual files, stacked on top of each other like transparent sheets. The config describes how to run the image, such as the starting command. The manifest is a short list that names the config and every layer. Registries store each part separately, and the manifest ties them together.

Tags are labels, not identities

Most people refer to images by a tag, such as a version label that looks like “1.4” or the word “latest”. A tag is just a sticky note attached to an image. Anyone with permission to write to the registry can peel that note off and stick it on a different image. The name stays the same, but the contents change.

This is a huge problem for security. Today the tag “1.4” might point at a safe image. Tomorrow, after an attacker pushes a replacement, the same tag might point at a malicious one. Every deployment that uses the tag will quietly pick up the change.

Digests are fingerprints

A digest is a long string of letters and numbers, produced by running the content of the image’s manifest through a mathematical function called a hash. A hash function turns any input into a fixed-size fingerprint. Change even one tiny bit of the input and the fingerprint changes completely. It is also practically impossible to build a different input that gives the same fingerprint.

Because the manifest names every layer by its own fingerprint, the digest of the manifest covers the whole image. If anyone changes a single byte in any layer, the digest changes. This makes the digest a true identity: one digest means exactly one image, forever.

Everyday Analogy

A tag is like a person’s job title, such as “the mail clerk”. The person holding that title can change next week. A digest is like their fingerprint. No matter what title they hold, the fingerprint tells you exactly who they are, and nobody else shares it.

PropertyTagDigest
Readable by humansYes, short and friendlyNo, long and unfriendly
Can point to different images over timeYesNo, it is fixed
Proves the content did not changeNoYes
Safe to sign and verifyNo, unless resolved to a digestYes
Typical useFinding the newest versionRunning exactly what was approved

The time-of-check problem

Here is a subtle trap. Suppose a policy checks an image by its tag at the moment a pod is created. The tag points at a good, signed image, so the pod is admitted. A moment later, an attacker moves the tag to a bad image. When the node actually pulls the image to start the container, it resolves the tag again and gets the bad one. The check passed on one image and the node ran another.

The cure is to resolve every tag into its digest at admission time and to make the pod use the digest from then on. Good verification tools do this automatically, and we will see how in later chapters.

!
Watch Out

Signing a tag is meaningless. Signing a digest is meaningful. Whenever a tool says it “signs an image”, it is really signing the digest that the image name currently points to.

4SBOM: The Ingredient List for Software

An SBOM answers the question that caught the whole industry off guard during Log4Shell: “What exactly is inside our software?”

What is an SBOM?

SBOM stands for Software Bill of Materials. It is a structured, machine-readable list of every component that goes into a piece of software, including each library, its version, where it came from, and often its license. It is the software version of the ingredient label on a food package.

The idea comes from manufacturing. A car maker keeps a bill of materials for every vehicle. If a supplier announces that a batch of brake parts was faulty, the maker can look up exactly which cars received those parts. An SBOM gives software teams the same power.

Everyday Analogy

An SBOM is the ingredient list on a cereal box. It does not say the cereal is healthy or unhealthy. It only tells you what is in it. When a farm recalls peanuts, you can check the box and instantly know whether your cereal is affected, without opening it and tasting.

What an SBOM contains

Field

Component name and version

The exact library or package, such as a logging library at a specific release. Without the version, you cannot tell if it is a vulnerable one.

Field

Supplier and origin

Who made it and where it was downloaded from. This helps spot packages that came from unexpected places.

Field

Unique identifiers

Standard names such as package URLs, plus fingerprints of the files, so tools can match components against vulnerability databases without guessing.

Field

Relationships

Which component depends on which. A flaw deep in a chain of dependencies still matters, and the relationships show how it got there.

Field

Licenses

The legal terms attached to each component. Some licenses place conditions on how you may distribute your product.

Field

Author and timestamp

Who generated the SBOM and when. An old SBOM may no longer describe what is actually deployed.

Two main formats

Two open formats dominate. SPDX came from the Linux Foundation community and is an international standard, with a strong history in license compliance. CycloneDX came from the security community, is now standardized through Ecma International, and has strong support for vulnerability information and many component types. Both are widely supported by tools. Many organizations produce one primary format and convert when partners ask for the other.

Why the world suddenly cares

In 2021, a United States executive order on cybersecurity put SBOMs in the spotlight by requiring them from software sold to the federal government. Other regulators and large customers followed, and new laws such as the European Cyber Resilience Act push vendors to know and document the components in their products. Even without regulation, the practical value is clear: when the next Log4Shell arrives, teams with SBOMs answer in minutes, and teams without them spend weeks searching.

How an SBOM is made

Tools called SBOM generators read an image or a source folder and detect the components inside. They look at package manager records, such as those kept by the operating system, and at files left by language ecosystems, such as lock files. Popular open-source generators include Syft and Trivy, and container build tools can also produce SBOMs as part of the build. The best time to generate an SBOM is during the build, because the build knows precisely what it just assembled. Scanning an image after the fact works, but it can miss things that were compiled in or copied without a package record.

flowchart TB
    SRC["Source code and lock files"] --> BLD["Image build"]
    BASE["Base image"] --> BLD
    BLD --> IMG["Container image"]
    IMG --> GEN["SBOM generator tool"]
    GEN --> SB["SBOM document - SPDX or CycloneDX"]
    SB --> ST["Stored next to the image in the registry"]
    SB --> VS["Vulnerability matching"]
    SB --> LC["License review"]
    SB --> IR["Incident response lookup"]
Figure 2 – An SBOM is generated at build time and then reused for several jobs.
i
Good to Know

An SBOM is a list of facts, not a verdict. It does not say whether a component is vulnerable. It only lets other tools ask that question reliably.

5Putting SBOMs to Work

An SBOM sitting in a folder helps nobody. Its value appears when other systems read it and act.

Use 1: Vulnerability matching

A vulnerability scanner takes the component list from the SBOM and compares it with public databases of known flaws, such as the National Vulnerability Database and vendor advisories. Each known flaw has an identifier called a CVE, short for Common Vulnerabilities and Exposures. The scanner reports which components match which CVEs. Because the SBOM already exists, the scanner can run again tomorrow, against new CVEs, without rebuilding or even touching the image.

This second point is very powerful. New flaws are discovered every day in software that was shipped months ago. An image that was clean at build time can become vulnerable overnight. By keeping the SBOM and re-scanning it, you learn about the new risk the same day it is published.

Use 2: Incident response

When a famous flaw hits the news, the first question is always “Are we affected, and where?” With a central store of SBOMs, the answer is a search query. A team can ask which images contain a given library at a given version, then match those images against what is running in each cluster. What took weeks during Log4Shell can take minutes.

Use 3: License and policy checks

Legal teams care about licenses. A component with a restrictive license might be fine for internal use but a problem for a product you sell. An SBOM lets policy tools block images that include unwanted licenses before they ever ship.

Use 4: Evidence for customers and auditors

Customers in regulated industries increasingly ask for SBOMs as part of buying decisions. An SBOM produced and signed by your pipeline is solid evidence that you know what you ship.

Vulnerabilities that do not matter: VEX

A big frustration is that scanners often report flaws in components that your software never actually uses in a dangerous way. Teams drown in alerts. To help, the industry created VEX, the Vulnerability Exploitability eXchange. A VEX statement is a short note, published by the maker of software, that says something like “This flaw is in a library we include, but it cannot be reached in our product, so it is not exploitable here.” Scanners can read these notes and drop the noise, so humans can focus on the real risks.

i
Good to Know

The cleanest way to keep SBOMs useful is to store them in the registry right next to the image they describe, and to sign them. That way, anyone who has the image can find its SBOM, and nobody can quietly swap in a different one.

Limits to be honest about

  • Accuracy varies. Generators can miss statically compiled code or files copied in by hand. Different tools may produce slightly different lists for the same image.
  • It can go stale. An SBOM describes an image at one moment. If you rebuild, you need a new one.
  • It does not prove safety. A complete list can still contain bad components. The SBOM is the input to decisions, not the decision itself.
  • It can be forged. An unsigned SBOM can be edited. That is why we sign it and why the next chapters matter.

6Image Signing: A Tamper-Proof Seal

An SBOM tells you what is inside. A signature tells you who vouches for it and that it has not changed since.

The cryptography you need, in plain words

Digital signatures rely on a pair of linked keys. The private key is a secret that only the signer holds. The public key is shared with everyone. The two keys are mathematically linked in a special way: something locked with the private key can be checked with the public key, but the public key cannot be used to forge the lock.

To sign an image, the signer first computes the digest, the fingerprint we met earlier. Then they use the private key to produce a signature over that digest. Anyone who later receives the image, the signature and the public key can run a check. The check recomputes the digest and confirms that the signature matches. If even one byte of the image changed, the digest changes and the check fails.

Everyday Analogy

Picture a king’s wax seal on a letter. Only the king owns the signet ring, which is the private key. Everyone knows what the genuine seal looks like, which is the public key. If a messenger opens the letter and reseals it, the seal breaks. If a forger tries to draw a copy of the seal, experts can tell it is fake.

What a signature proves, and what it does not

It proves

  • The image has not been modified since it was signed.
  • The signature was made by the holder of a particular key or identity.
  • The signer made a deliberate choice to vouch for this exact digest.

It does not prove

  • That the image is free of bugs or malware.
  • That the signer’s build system was not itself compromised.
  • That the code inside is good, only that the signer approved it.

This is a crucial point. The SolarWinds update carried a genuine signature. A signature answers “who”, not “is it safe”. That is why signing alone is never enough, and why we combine it with provenance, SBOMs and scanning.

Where do signatures live?

Modern signing tools store the signature in the same registry as the image, as a small extra object attached to the image’s digest. This means you do not need a separate server to hold signatures. Anyone who can pull the image can also fetch its signature. Because the signature refers to the digest and not to a tag, moving a tag does not break it, and signing one image does not affect another.

The old problem: managing keys

Traditional signing asks teams to create a long-lived key pair, protect the private key forever, distribute the public key everywhere, and rotate the pair when staff leave or secrets leak. In practice, many private keys end up in build system settings, on laptops, or in chat messages. A stolen private key lets an attacker sign anything as you, and you may not notice for months.

This pain led to two different directions in the community: managing keys more carefully in hardware or cloud key services, or avoiding long-lived keys altogether. The second path is what Sigstore pioneered.

7Sigstore and Keyless Signing

Sigstore is an open-source project, now hosted by the Open Source Security Foundation, that makes signing as easy as logging in. It is the most widely used approach for container images today.

The parts of Sigstore

Tool

Cosign

The command-line tool and library that signs and verifies container images and other files stored in registries. Many admission tools use it under the hood.

Service

Fulcio

A certificate authority that hands out very short-lived certificates. Each certificate ties a one-time public key to a verified identity, such as an email address or a build workflow.

Service

Rekor

A public transparency log. Every signing event is recorded in an append-only ledger that anyone can inspect. It cannot be quietly edited after the fact.

Idea

OIDC identity

OpenID Connect is the standard login system behind “sign in with your account”. Sigstore uses it to learn who the signer is, with no new password or key to manage.

How keyless signing works

The word “keyless” is a little misleading. There are still keys, but they are created fresh for every signing and thrown away within moments. Nobody has to store, protect or rotate them. Here is the flow, step by step.

1

Make a one-time key pair

The signing tool creates a brand-new key pair in memory, just for this one signing.

2

Prove who you are

The signer logs in with an identity provider. In a build pipeline, the pipeline itself receives a signed identity token that says which repository and workflow is running.

3

Get a short-lived certificate

The tool sends the token and the new public key to Fulcio. Fulcio checks the token and issues a certificate that lasts only a few minutes and names the identity.

4

Sign the digest

The tool signs the image digest with the one-time private key and then discards that key.

5

Record it publicly

The signature and certificate are logged in Rekor, which proves the signing happened while the certificate was still valid.

6

Store the signature

The signature and certificate are pushed to the registry next to the image.

sequenceDiagram
    participant P as Build pipeline
    participant I as Identity provider
    participant F as Fulcio CA
    participant R as Rekor log
    participant G as Registry
    P->>P: Create one-time key pair
    P->>I: Ask for identity token
    I-->>P: Signed token for this workflow
    P->>F: Send token and public key
    F-->>P: Short-lived certificate
    P->>P: Sign image digest and discard key
    P->>R: Record signature and certificate
    R-->>P: Log entry proof
    P->>G: Push signature next to image
Figure 3 – Keyless signing. No long-lived secret ever has to be stored.

How verification works later

Verifying is the reverse. The checker fetches the signature and certificate from the registry. It confirms that the signature matches the digest, that the certificate was issued by Fulcio, that the certificate was valid at the time of signing as proven by the Rekor entry, and, most importantly, that the identity inside the certificate matches what the policy expects. A good policy says something like “only accept images signed by the release workflow of our own repository”. An identity from anywhere else, even if the signature itself is perfectly valid, is rejected.

!
Watch Out

A policy that only checks “is there a valid signature?” is almost worthless. Anyone in the world can sign anything with Sigstore. The power of the policy is in checking who signed. Always pin the exact identity and issuer you trust.

Key-based signing is still valid

Keyless is not the only choice. Many organizations prefer classic key pairs held in a cloud key management service or a hardware security module, so the private key never leaves protected hardware. This is a good fit when systems cannot reach the public Sigstore services, such as air-gapped environments, or when internal rules demand it. Another ecosystem, the Notary Project with its Notation tool, takes the traditional certificate approach and is popular with several cloud providers. The admission-side tools we will meet support more than one of these, so you can choose the signing model that fits your company.

ApproachKey handlingGood fit
Keyless with SigstoreOne-time keys, short-lived certificates, public logOpen-source and cloud-native pipelines with internet access
Cosign with managed keysPrivate key in a cloud KMS or HSMTeams wanting simple, controlled keys and offline verification
Notation with certificatesCompany certificate authority and trust storesEnterprises with existing certificate infrastructure

8Provenance, Attestations and SLSA

A signature says who vouches. Provenance says how the thing was made. Together they let you trust the build itself, not just the person pressing the button.

What is provenance?

Provenance is a record of an image’s origin story. It states which source code was used, from which repository and which exact commit, which build system ran, which steps and settings it used, and what image it produced. Think of the pedigree papers for a purebred dog or the chain of custody for evidence in a court case.

What is an attestation?

An attestation is a signed statement about a piece of software. The statement has a standard shape: it names the subject, which is the image digest, states a claim, such as “this image was built from this commit”, and carries a signature from whoever is making the claim. The community framework for these statements is called in-toto. A provenance record, an SBOM, a vulnerability scan result and a test report can all be wrapped as attestations. This is what lets policies say “require a signed SBOM” or “require a signed build record” in a uniform way.

Everyday Analogy

A signature is a notary stamping a document. An attestation is the notary stamping a specific sentence on the document, such as “I saw this person sign this form on this date”. Different sentences, same trusted stamp.

SLSA: a ladder of build trust

SLSA, pronounced “salsa”, stands for Supply-chain Levels for Software Artifacts. It is a framework that describes how strongly a build can be trusted, using a simple ladder of levels. In its current version, the build track has three levels above zero.

L1

Provenance exists

The build produces a provenance record describing how the artifact was made. It may be easy to forge, but the practice has begun.

L2

Hosted and signed

The build runs on a hosted build service, and that service signs the provenance, so it cannot be faked by the developer who triggered the build.

L3

Hardened builds

The build platform isolates builds from each other and protects the signing material, so even a malicious build step cannot forge provenance or influence other builds.

The levels give teams a shared vocabulary and a path to climb. A policy can ask for a minimum level, for example “only run images whose provenance proves a hosted, isolated build”.

Why provenance beats signatures alone

Imagine a developer’s laptop is infected. The attacker builds a malicious image there and signs it with the developer’s real key. The signature is valid. But if your policy demands provenance that says “built by the official build service from the main branch of the official repository”, the laptop-built image cannot produce it. The provenance lets you reject a validly signed but wrongly built image.

flowchart TB
    IMG["Image digest"] --> A1["Attestation: build provenance"]
    IMG --> A2["Attestation: SBOM"]
    IMG --> A3["Attestation: vulnerability scan result"]
    IMG --> A4["Attestation: test results"]
    A1 --> SG["Signed by the build service identity"]
    A2 --> SG
    A3 --> SG
    A4 --> SG
    SG --> POL["Admission policy reads and checks them"]
Figure 4 – One image can carry many signed claims. The admission policy decides which ones it requires.
i
Good to Know

Many popular build platforms can now generate signed provenance automatically. Often it needs just a setting in the pipeline, which makes it one of the cheapest big wins in this whole guide.

9Admission Control: The Guard at the Cluster Door

Signatures and SBOMs are only useful if somebody checks them. In Kubernetes, the place to check is called admission control.

The life of an API request

Everything in Kubernetes happens through the API server. When someone creates a pod, they send a request to it. The API server does not simply save the request. It runs it through a series of gates, in a fixed order.

1

Authentication

Who is making this request? The server checks certificates, tokens or other credentials.

2

Authorization

Is this person allowed to do this? Role-based access control rules are checked here.

3

Mutating admission

Hooks may change the request, for example adding default settings or replacing an image tag with its digest.

4

Validation of the object

The server checks that the object is well formed according to its schema.

5

Validating admission

Final hooks may approve or reject the request. This is where “no valid signature, no entry” is enforced.

6

Persist

If everything passes, the object is saved to the cluster’s data store, and the scheduler and kubelet take over.

Everyday Analogy

Admission control is the airport security checkpoint. Showing your passport proves who you are (authentication). Your boarding pass shows you may fly to this destination (authorization). Then the scanner and the officer inspect your bag, and may even ask you to pull out a prohibited item (admission). Only then do you reach the gate.

Webhooks: admission by outside experts

Kubernetes lets you plug in your own admission logic using admission webhooks. A webhook is a small web service that the API server calls whenever a matching request arrives. The server sends the object, and the webhook answers “allowed”, “denied with a reason”, or, for mutating hooks, “allowed, with these changes”. Policy engines such as Kyverno and the Sigstore policy controller run inside the cluster as webhooks. This is the mechanism that lets them verify signatures: they can reach out to a registry, fetch the signature, check it and then answer.

Built-in policies with CEL

Kubernetes also has a built-in way to write rules without running any webhook. ValidatingAdmissionPolicy became generally available in version 1.30. It uses the Common Expression Language, or CEL, to evaluate simple conditions inside the API server itself. This is fast and has no extra service to maintain. It is excellent for rules like “images must come from our registry” or “images must use a digest”. However, it cannot call outside services, so it cannot download and cryptographically check a signature. For that job you still need a webhook-based tool.

sequenceDiagram
    participant U as User or CI system
    participant A as API Server
    participant W as Policy webhook
    participant R as Registry
    participant E as etcd store
    U->>A: Create pod
    A->>A: Authenticate and authorize
    A->>W: Admission review request
    W->>R: Resolve tag to digest and fetch signature
    R-->>W: Digest, signature, attestations
    W->>W: Verify signature identity and policy
    W-->>A: Allow with digest pinned, or deny
    A->>E: Save pod if allowed
    A-->>U: Success or rejection message
Figure 5 – A policy webhook checks the image during pod creation.

What admission can and cannot see

  • It sees every pod creation. This includes pods created directly and pods created by deployments, jobs and similar controllers, because controllers ultimately create pods through the same API.
  • It applies to new requests. Pods that were already running before the policy existed are not re-judged automatically. Good tools offer background scans that report on existing workloads.
  • It does not cover the kubelet’s own static pods. Static pods are defined as files on a node, and the node starts them directly. Protect the nodes themselves to cover this gap.
  • It must cover every place an image can appear. Regular containers, init containers and ephemeral debugging containers all carry images, and a policy that forgets one leaves a back door open.
!
Watch Out

Anyone who can delete or edit the webhook configuration can switch the guard off. Treat the permissions that change admission webhooks and policies as some of the most sensitive in the entire cluster.

10Admission Verification Tools

Several mature open-source tools do the actual checking. They overlap, but each has a different personality.

Policy engine

Kyverno

A Kubernetes-native policy engine whose rules are written as Kubernetes objects, with no new programming language. It has a dedicated image verification rule type that checks signatures and attestations, and it can rewrite tags into digests.

Sigstore project

Policy Controller

A webhook from the Sigstore community focused purely on verifying signatures and attestations with cosign-style policies. It is a tight fit for teams that live in the Sigstore world.

Policy engine

OPA Gatekeeper with Ratify

Gatekeeper is the Kubernetes front end of the Open Policy Agent. Paired with a verification service called Ratify, which understands both cosign and Notation, it can enforce signature and SBOM rules.

Focused tool

Connaisseur

A lightweight admission controller built just for image signature verification, supporting several signing styles. It suits small, simple setups.

Cloud service

Managed admission

Cloud providers offer built-in options, such as Binary Authorization on Google Kubernetes Engine, where signed attestations from trusted attestors are required before deployment.

Built in

ValidatingAdmissionPolicy

Native CEL rules for cheap checks like approved registries and digest usage. It complements a signature tool instead of replacing it.

CapabilityKyvernoPolicy ControllerGatekeeper plus RatifyValidatingAdmissionPolicy
Verify cosign signaturesYesYesYes, through RatifyNo
Verify Notation signaturesYesLimitedYes, through RatifyNo
Check attestations such as SBOM or provenanceYesYesYes, depending on setupNo
Rewrite tag to digestYesYesLimitedNo, validation only
Covers general policies beyond imagesYes, broadNarrowYes, broadYes, for simple rules
Runs inside the API server, no extra serviceNoNoNoYes
Learning curveGentleGentleSteeperModerate, requires CEL

Features move quickly in this area, so treat the table as a general shape of the landscape and check each project’s current documentation before choosing.

How to choose

  • If you want one tool for many kinds of Kubernetes policy as well as image signing, and prefer writing rules without learning a new language, Kyverno is often the friendliest start.
  • If your world is entirely cosign and Sigstore and you want the narrowest, simplest component, the Policy Controller is a good match.
  • If your organization already uses Open Policy Agent widely, or uses Notation-style signing in a cloud like Azure, Gatekeeper with Ratify is a natural path.
  • Whatever you pick, add a few ValidatingAdmissionPolicy rules for the cheap checks. They cost almost nothing at runtime.

Verification at the node level too

Admission is not the only possible checkpoint. Some container runtimes, such as CRI-O, can be configured with their own signature policy, so that the node itself refuses to pull unsigned images, even if the request somehow bypassed admission. This adds a second layer, and is a fine choice for high-security clusters, at the cost of more configuration on every node.

11The Complete Pipeline, End to End

Now let us connect everything into one working story, from a developer’s commit to a running pod.

1

Commit and review

A developer opens a change. Branch protection requires a review by another person, so no single account can push code straight to the main branch.

2

Build in a clean environment

The pipeline builds the image in a fresh, isolated environment, pinning its base image by digest and its dependencies by lock file.

3

Generate the SBOM

An SBOM generator inspects the finished image and writes the component list.

4

Scan

A scanner reads the SBOM and image, reports known vulnerabilities, and fails the build if something above the agreed severity is found.

5

Push, sign and attest

The image goes to the registry. The pipeline signs its digest keylessly and attaches signed attestations for the SBOM, the scan result and the build provenance.

6

Deploy by digest

The deployment configuration refers to the new image by digest, or the deployment tool resolves it that way.

7

Admission verification

The policy engine checks that the signature comes from the expected pipeline identity, that the right attestations are present, and that the registry is approved.

8

Run and watch

The pod starts. Continuous scanning keeps comparing the stored SBOM against new vulnerability data, and alerts if something new appears.

flowchart TB
    subgraph BUILD["Build and release - trusted pipeline"]
        B1["Clean isolated build"] --> B2["Generate SBOM"]
        B2 --> B3["Vulnerability scan"]
        B3 --> B4["Sign image digest"]
        B4 --> B5["Attach signed attestations"]
    end
    B5 --> REG["Container registry"]
    subgraph CLUSTER["Kubernetes cluster"]
        D1["Deployment with image by digest"] --> API["API Server"]
        API --> WH["Policy webhook - Kyverno or similar"]
        WH -->|"fetch evidence"| REG
        WH -->|"decision"| API
        API --> POD["Running pods"]
    end
    REG -->|"pull verified image"| POD
    POD --> MON["Continuous rescan and alerts"]
    REG --> MON
Figure 6 – The full supply chain security architecture. Evidence is created on the left, and enforced at the cluster door on the right.

Two ideas that hold it together

  • The cluster trusts identities, not locations. Policies name the exact pipeline that is allowed to produce production images. Any other signer, however legitimate, is refused.
  • Evidence is checked every time. The cluster does not remember that an image was fine yesterday. Each admission request is judged fresh against current policy.
i
Good to Know

Teams often add one more gate in the pipeline itself: before promoting an image from a testing registry to a production registry, a promotion step re-checks the signature and attestations. Production registry rules can then say “only the promotion job may write here”.

12What Rules Should the Cluster Enforce?

Tools are only as good as the rules you give them. Here is a sensible menu of rules, ordered roughly from easiest to hardest.

Rule 1

Approved registries only

Pods may only pull from registries your company controls. This blocks surprise images from public sources and look-alike names.

Rule 2

No mutable tags

Require digests, or have the policy engine convert tags to digests on the way in. This removes the time-of-check problem.

Rule 3

Must be signed by our pipeline

Require a valid signature from the exact identity of your official release workflow. This is the heart of admission verification.

Rule 4

Must carry an SBOM

Require a signed SBOM attestation so that every running image has an ingredient list on record.

Rule 5

Provenance at a minimum level

Require a signed build record showing the image came from the main branch of your own repository on a hosted builder.

Rule 6

Scan results are recent and clean

Require a signed scan attestation that is newer than a set age and shows no findings above your severity limit.

Rule 7

Approved base images

Allow only a short list of hardened base images, so that teams do not build on random starting points.

Rule 8

Cover every container type

Apply the rules to init containers and ephemeral containers as well, not only the main ones.

Audit first, then enforce

Most policy engines support two modes. Audit mode allows everything but records which pods would have been rejected. Enforce mode actually blocks them. Starting directly in enforce mode almost always breaks something important on day one, such as a monitoring agent from a public registry that nobody remembered. The safe path is to run in audit mode for a couple of weeks, read the reports, fix or exempt the findings, and only then switch on enforcement, one namespace at a time.

Handling exceptions on purpose

Some system components will not be signed by your pipeline. Vendor tools, cluster add-ons and the cluster’s own system images come from outside. You have three honest options: verify them against the vendor’s own signing identity if they publish one, mirror them into your own registry and sign them yourself after review, or add a narrow, documented exception for a specific namespace and image. What you must avoid is a broad exemption such as “everything in the system namespace is allowed”, because attackers love to deploy into exempt places.

!
Watch Out

Every exception is a hole in the wall. Give each one an owner, a reason and an expiry date, and review them on a schedule.

13Threats, and Which Control Stops Which

No single control stops every attack. A clear threat map shows what each layer covers and where the gaps remain.

ThreatWhat it looks likeControls that help
Tag swapping in the registryA tag is moved to a malicious image after approvalDigest pinning, signatures over digests
Stolen registry credentialsAn attacker pushes a fake image to your registrySignature identity checks at admission, restricted push rights
Look-alike or typo-squatted imagesA pod pulls an image with a name that is one letter offApproved registry rules, signature checks
Dependency confusionA public package replaces an internal one during the buildLocked dependencies, private registries, SBOM review
Known vulnerable componentAn old library with a published flaw is inside the imageSBOM plus continuous scanning, scan attestations
Compromised developer laptopA build is made and signed from an infected machineProvenance requiring hosted builds, branch protection
Compromised build systemMalicious code is injected during the build, as in SolarWindsHardened and isolated builders, SLSA level 3, reproducible builds, monitoring
Stolen signing keyAn attacker signs anything as your companyKeyless signing, short-lived certificates, KMS or HSM keys, transparency log monitoring
Malicious upstream packageA trusted open-source project is poisoned at the sourceLimited help. SBOM and scanning speed up response, but prevention is hard
Admission bypassSomeone edits or disables the policy webhookTight access control, alerts on policy changes, node-level verification

Be honest about the gaps

The last rows reveal something important. Supply chain controls are excellent at detecting tampering and unauthorized builds, and at speeding up your response. They are weaker against a malicious change introduced by a trusted insider or by a trusted upstream maintainer, because in those cases the signature is legitimate and the provenance is clean. Defending against that needs human practices as well: code review, careful dependency selection, slow adoption of new dependency versions and quick access to the SBOM when a warning arrives.

Everyday Analogy

Airport security stops weapons in luggage, but it cannot stop a pilot who has been properly vetted and later decides to misbehave. Different threats need different protections, such as background checks, cockpit procedures and monitoring. Supply chain security is the same: strong checkpoints, plus other layers behind them.

The runtime still matters

Even a perfectly signed, perfectly scanned image can contain an unknown flaw. Supply chain controls reduce the chance that bad software gets in. Runtime defenses, such as least-privilege pod settings, network policies and sandboxed runtimes, limit the damage if something slips through. Think of them as partners, not competitors.

14Monitoring, Logging and Alerting

A guard who never writes anything down cannot help after an incident. Good supply chain security produces a trail you can use.

What to record

Log

Admission decisions

Every allow and deny decision, with the image digest, the namespace, the user and the reason. Denials are a precious early warning.

Log

Kubernetes audit events

The API server’s audit log shows who created or changed workloads, policies and webhook settings. It is your evidence if something is tampered with.

Log

Transparency log entries

With keyless signing, every signature appears in a public log. You can watch it for signatures made under your identity that your pipeline did not create.

Report

Policy reports

Audit-mode and background-scan findings, such as running pods that violate the rules, give you a living list of work to do.

Store

SBOM inventory

A searchable collection of all SBOMs, mapped to what is running where. This is the tool you reach for when the next big flaw hits the news.

Metric

Policy engine health

Request counts, latency, error rates and timeouts of the webhook. A slow or failing guard can slow down or block deployments.

Alerts worth setting up

  • A spike in denied pods, which might mean either a broken pipeline or an attempted attack.
  • Any change to admission webhook configurations or policy objects, especially outside normal deployment windows.
  • A signature appearing in the transparency log under your pipeline identity that does not match a known build.
  • A newly published CVE that matches a component in a running image’s SBOM, ranked by severity and by whether the workload faces the internet.
  • Pods running with images that no longer have a passing signature, found by background scans.
  • Admission webhook errors or timeouts, since these may leave the cluster either unprotected or unable to deploy, depending on how failure is configured.
i
Good to Know

Track a simple health number for your program: the percentage of running workloads that are signed, have an SBOM and meet the provenance rule. Watching that number rise over the months is the clearest sign of progress.

15Operating It: Failure Modes, Keys and Scale

The hardest part of any security control is keeping it running quietly for years. These are the operational questions that decide whether your rollout survives contact with reality.

Fail open or fail closed?

What should the cluster do if the policy webhook is down or slow? There are two choices. Fail closed means the API server rejects requests when it cannot reach the webhook. This is safest, but an outage of the policy service can stop deployments and even prevent pods from restarting. Fail open means the API server allows requests when the webhook is unavailable. This keeps the cluster running but opens a window where unverified images can enter, and attackers can create such windows on purpose.

A sensible approach is to fail closed for production namespaces holding sensitive workloads, run the webhook with several replicas spread across nodes, exclude the webhook’s own namespace and critical system namespaces so that the guard can always restart itself, and rehearse an outage before it happens.

Latency and cost

Verifying a signature means reaching out to a registry and sometimes to a transparency log, which takes time. Because the API server waits for the webhook, a slow check delays every deployment. Good tools cache verification results for a given digest, so the same image is checked once and reused. Keep your registry close to the cluster, set sensible timeouts and measure the added delay during a big rollout.

Key and trust rotation

  • Managed keys. If you use keys in a KMS, plan rotation. Policies should accept both the old and new public keys for a transition period, and then retire the old one.
  • Keyless trust roots. Sigstore’s public trust material is updated from time to time using a secure update framework. Your verification tools must keep their copy fresh, or valid signatures may start to fail.
  • Identity changes. If you rename a repository or restructure a pipeline, its signing identity changes. Policies referring to the old identity will start rejecting good images. Update the policy first.

Air-gapped and private environments

Clusters with no internet access cannot reach the public Sigstore services. They can still verify signatures by running a private copy of the Sigstore components, or by using managed keys and distributing the public keys inside the network. Private registries mirror approved images along with their signatures and attestations, and the verifier works entirely against the mirror.

Multi-cluster and multi-team scale

In a large organization, define policies centrally and distribute them to every cluster using the same delivery tooling that manages other cluster configuration. Allow teams to make narrower rules for their own namespaces but not to loosen the central ones. A small platform team usually owns the policy engine, the signing identities and the exception process, while application teams only need to know “my pipeline template signs and attests automatically”.

flowchart LR
    REQ["Pod creation request"] --> API["API Server"]
    API --> HC{"Webhook reachable?"}
    HC -->|"yes"| VER["Verify signature and attestations"]
    HC -->|"no"| FP{"Failure policy"}
    FP -->|"fail closed"| DENY["Request rejected"]
    FP -->|"fail open"| ALLOW["Request allowed - unverified"]
    VER -->|"valid"| OK["Pod admitted"]
    VER -->|"invalid"| DENY
Figure 7 – The decision tree of an admission check, including what happens when the checker itself is unavailable.

16Design Patterns and Anti-Patterns

Teams that succeed with supply chain security repeat the same few good habits. Teams that struggle repeat the same few mistakes.

Patterns that work

Pattern

Golden pipeline

Offer one well-maintained pipeline template that builds, scans, signs and attests automatically. Teams get security for free by using it.

Pattern

Trusted base image program

Maintain a small set of hardened, regularly rebuilt base images that everyone builds on, so fixes flow to all teams at once.

Pattern

Promotion between registries

Images move from a build registry to a production registry only after passing checks, and only a promotion job can write to production.

Pattern

Mirror and re-sign third-party images

Pull outside images into your registry, review them, scan them and sign them with your own identity. The cluster then needs to trust only you.

Pattern

Policy as code

Keep admission rules in version control, reviewed like any code, and deliver them to clusters through automation.

Pattern

Continuous re-evaluation

Re-scan stored SBOMs every day against fresh vulnerability data, and flag running workloads that become risky after deployment.

Anti-patterns to avoid

ANTI-PATTERN 01Avoid
What it looks like

The policy checks only that “some valid signature exists” and does not check the identity of the signer or the issuer.

Why it hurts

Anyone can obtain a valid signature from a public signing service. An attacker simply signs their malicious image and passes the check.

Better approach

Pin the exact signing identity and issuer, such as a specific repository workflow, or pin your own managed public key.

ANTI-PATTERN 02Avoid
What it looks like

SBOMs are generated and then dropped into a storage bucket that nobody reads. No signing, no scanning, no searching.

Why it hurts

The team can say it has SBOMs, but cannot answer questions during an incident. The effort produces compliance paperwork instead of security.

Better approach

Attach SBOMs to images as signed attestations, ingest them into a searchable inventory and wire them to daily vulnerability matching.

ANTI-PATTERN 03Avoid
What it looks like

Enforcement is switched on across the whole cluster on a Friday afternoon, with broad exemptions added in a hurry when things break.

Why it hurts

Outages erode trust in the program, and the emergency exemptions quietly become permanent holes that attackers can use.

Better approach

Run in audit mode first, roll out namespace by namespace, and make every exception specific, owned and time-limited.

ANTI-PATTERN 04Avoid
What it looks like

A single long-lived signing key is shared by every team and stored as a secret in the build system.

Why it hurts

Leaking the key anywhere lets an attacker sign for the whole company. Rotating it is painful, so it is rarely done.

Better approach

Use keyless signing or keys protected in a KMS or HSM, with separate identities per team or environment.

17Best Practices and a Rollout Roadmap

You do not need to do everything at once. A staged plan delivers real protection early and avoids overwhelming your teams.

A practical roadmap

1

Get visibility

Generate SBOMs for your images, scan them, and put the results in one place. Learn what you actually run.

2

Clean up the basics

Move to approved registries, pin images by digest, remove unused images and standardize base images.

3

Sign everything

Add signing to the golden pipeline. Sign images, SBOMs and provenance. Do not enforce yet.

4

Audit at admission

Install the policy engine in audit mode. Read the reports and fix the exceptions.

5

Enforce gradually

Turn on enforcement for one low-risk namespace, then more, ending with production and sensitive workloads.

6

Raise the bar

Require higher provenance levels, fresh scan attestations and node-level verification for the most sensitive clusters.

Best practices at a glance

  • Sign digests, not tags, and make deployments run exactly the digest that was verified.
  • Pin identities in policy. Name the repository, workflow and issuer you trust.
  • Generate SBOMs at build time, sign them and store them next to the image.
  • Prefer short-lived credentials and keyless signing over long-lived secrets.
  • Protect the build. Use isolated, ephemeral builders, restricted permissions and reviewed pipeline definitions.
  • Use two layers of image rules: cheap native rules for registries and digests, and a webhook tool for signatures and attestations.
  • Cover every container type in pods, including init and ephemeral containers.
  • Run the guard reliably: several replicas, sensible timeouts, a known failure policy and tested recovery.
  • Watch the watchers. Alert on any change to policies, webhooks and signing identities.
  • Practice an incident. Pretend a famous library is vulnerable and see how fast you can list affected workloads.

Common mistakes

  • Believing that a signature means an image is safe, when it only says who approved it.
  • Verifying tags rather than digests, which leaves the time-of-check gap open.
  • Skipping audit mode and breaking production on the first day.
  • Forgetting that existing workloads are not re-checked unless background scanning is enabled.
  • Exempting whole namespaces instead of specific images.
  • Letting too many people edit admission policies or webhook configuration.
  • Ignoring the registry: weak push permissions can undermine even a good verification setup.
  • Collecting SBOMs but never searching them until the day an emergency arrives.

18Real-World Usage

These practices are no longer experimental. They are in daily use across the industry.

Google: Binary Authorization and SLSA

Google has long described an internal approach where only code built by trusted systems, from reviewed source, is allowed to run in production. It turned that thinking into a product, Binary Authorization, for its managed Kubernetes service, where deployments require signed attestations from trusted attestors. Google was also a key force behind the SLSA framework.

The Kubernetes project itself

The Kubernetes project signs its release artifacts, including container images, using Sigstore tooling, and publishes SBOMs for releases. This lets users verify that the images they pull were produced by the project’s release process.

Open-source ecosystems

Large package ecosystems have started to support build provenance for published packages, so that consumers can see which repository and workflow produced a given release. This brings the same ideas used for containers to libraries, closing the earlier link of the chain.

Cloud registries and build services

Major cloud and source hosting platforms now offer built-in features for generating provenance, storing signatures and attestations beside images, and integrating signing with their identity systems. Some providers promote the Notation approach and some the cosign approach, and common verification tools support both, which keeps customers from being locked in.

Hardened image vendors

A growing number of vendors publish minimal base images that ship with signatures, SBOMs and provenance by default and are rebuilt frequently, so that customers begin from a clean, verifiable foundation instead of an unknown one.

19Frequently Asked Questions

Short answers to the questions teams ask most.

Q1Is an SBOM the same as a vulnerability scan?
No. An SBOM is a list of ingredients. A scan compares that list, or the image itself, against databases of known flaws. The SBOM makes scanning repeatable and cheap, because you can re-check it any day without rebuilding.
Q2Which SBOM format should I choose, SPDX or CycloneDX?
Either works, and most tools support both. Choose the one your scanners, customers and regulators prefer, and convert if needed. Consistency inside your company matters more than the choice.
Q3Does signing an image make it secure?
No. Signing proves who approved the image and that it has not changed. It does not prove the image is free of flaws. Combine signing with provenance, scanning and runtime hardening.
Q4What does “keyless” really mean?
It means you do not manage long-lived private keys. A fresh key is created for each signing, tied to your identity by a short-lived certificate, and then discarded. The signing event is recorded in a public log.
Q5Can Kubernetes verify signatures by itself?
Not through its built-in admission policies, because they cannot contact registries or check signatures. You need a webhook-based tool such as Kyverno, the Sigstore policy controller or Gatekeeper with Ratify. Some container runtimes can also verify signatures at pull time.
Q6What happens to pods that were running before I turned on the policy?
They keep running. Admission only judges new or updated requests. Use the background scan features of your policy tool to report on existing workloads, and plan a rollout to bring them into compliance.
Q7Should the webhook fail open or fail closed?
For sensitive production environments, failing closed is safer, as long as you run the webhook with high availability and exclude its own namespace. For less sensitive environments, failing open may be an acceptable trade for availability. Decide deliberately and test it.
Q8How do I handle third-party images I do not build?
Either verify them against the vendor’s published signing identity, or mirror them into your registry, review and scan them and sign them yourself. Never rely on an unreviewed public image directly in production.
Q9Do I need all of this for a small team?
Start small. Pin by digest, use trusted base images, generate SBOMs and turn on keyless signing in your pipeline. Add admission verification when your cluster serves real users or sensitive data. Many of these steps cost very little with modern tooling.
Q10Will verification slow down deployments?
Slightly, for the first time an image is seen. Results can be cached per digest, so repeated checks are quick. Keep the registry close to the cluster and monitor the webhook’s latency.
Q11What is the difference between provenance and an SBOM?
An SBOM describes what is inside the software. Provenance describes how and where it was built. You want both: one tells you the ingredients, the other tells you the kitchen and the chef.
Q12Can attackers bypass admission verification?
If they can edit or delete the webhook or policies, or run workloads outside the API path, such as static pods on a compromised node, yes. That is why access to policy objects must be tightly controlled, changes must be alerted on, and nodes must be protected.

20Summary and Key Takeaways

What to remember

  • Your risk is mostly code you did not write. Libraries, base images, build tools and pipelines are suppliers, and each one can be a point of attack.
  • An SBOM is your ingredient list. Generate it at build time, sign it, store it with the image and search it the day the next big flaw is announced.
  • Digests are identities, tags are labels. Sign, verify and run by digest to close the gap between checking and running.
  • Signing proves who, not whether it is safe. Keyless signing with Sigstore removes the burden of long-lived keys, and policies must pin the exact identity they trust.
  • Provenance proves how it was built. Signed attestations and SLSA levels let you reject images that were validly signed but built in the wrong place.
  • Admission control is the enforcement point. Use a webhook tool such as Kyverno, the Sigstore policy controller or Gatekeeper with Ratify for signatures, and native policies for simple registry and digest rules.
  • Roll out carefully and operate reliably. Audit before enforcing, make exceptions narrow and temporary, run the guard with high availability and watch for any change to the policies themselves.