Almost every team that adopts Kubernetes goes through the same surprise. The platform works well, and then the cloud invoice arrives and nobody can explain it. Let us start by understanding why.

What you actually pay for

A cloud provider does not charge you for the applications you run. It charges you for the machines you reserve, for every hour they exist, whether or not they are busy. Kubernetes sits on top of those machines. It decides which application runs on which machine, but it does not change the fact that the machines are rented by the hour. So the real question of cost optimization is simple: how many machines do we rent, how big are they, how much do each cost, and how much useful work happens on them?

Why the bill drifts upward

The two levers in this post

Cost work has many parts, and later chapters cover several. But two levers stand out. The first is density: getting more useful work out of each machine through bin packing. The second is price: paying less per machine through spot capacity and commitments. Density reduces the number of machines. Price reduces the cost of each one. They multiply: if you need half as many machines, and each costs a third less, the bill falls to a third of what it was.

Before cutting costs, you must know what you are cutting. A Kubernetes bill has several parts, and they do not all shrink with the same tools.

The main cost buckets

Compute is the big one

In most clusters, compute dominates. Compute cost depends on three numbers: how many nodes, how large each is, and the hourly price of each. The first number is driven by how well work is packed. The second is a design choice. The third depends on how you buy the machines, which is the subject of the spot and commitment chapters.

The hidden costs that surprise people

To understand bin packing, you must understand how Kubernetes decides where to put an application. Three ideas are needed: requests, limits, and the scheduler.

Pods and nodes

A pod is the smallest unit Kubernetes runs. It holds one or more containers, which are the running copies of your applications. A node is a machine, virtual or physical, that runs pods. A cluster is a group of nodes managed together. The scheduler is the part of Kubernetes that looks at each new pod and chooses which node it should run on.

Requests: what you reserve

Every container can declare a request for CPU and memory. A request is a promise from the cluster: “this container is guaranteed at least this much.” The scheduler uses requests, and only requests, to decide whether a pod fits on a node. If a node has four CPU cores and pods already on it have requested three, then a pod requesting two will not fit, even if the three existing pods are actually using only a tenth of a core.

Limits: the ceiling

A limit is the maximum a container may use. If a container tries to use more CPU than its limit, it is slowed down. If it tries to use more memory than its limit, it is killed and restarted. Limits protect neighbors from a greedy container. They do not affect where the pod is placed.

Why the scheduler looks at requests, not usage

This choice is deliberate. Real usage changes second by second, so if the scheduler packed pods by current usage, a quiet moment could lead to a crowded node that then collapses when everyone gets busy. Requests are stable promises, so they give predictable placement. The price of this safety is that requests, not real usage, decide how many nodes you need. If requests are inflated, you pay for capacity that is reserved but empty.

flowchart LR
  P["New pod with CPU and memory requests"] --> S["Scheduler"]
  S --> F{"Filter: which nodes have enough unreserved room?"}
  F -->|"none"| PEND["Pod stays pending, autoscaler may add a node"]
  F -->|"some"| SC["Score the remaining nodes"]
  SC --> BEST["Choose the best-scoring node"]
  BEST --> RUN["Pod runs on that node"]
    

Allocatable capacity is smaller than the machine

A node’s advertised size is not all available for your pods. The operating system, the node agent called the kubelet, the container runtime, and system pods reserve a share. What remains is called allocatable capacity. On small nodes, the reserved share is a larger fraction, which is one reason very small nodes waste more. Chapter 12 returns to this point.

The most common source of waste is a simple gap between what applications ask for and what they use. We call this gap slack.

How the gap appears

When a team deploys a new service, nobody knows how much CPU and memory it will need. Asking for too little risks slow responses or crashes in front of customers. Asking for too much risks nothing that anyone sees. So teams pick generous numbers, often copied from another service, and then never revisit them. Months later, the service may be using a fifth of what it reserves.

flowchart TD
  NODE["Node: 8 CPU cores allocatable"] --> R["Requested by pods: 6 cores"]
  NODE --> U["Unreserved: 2 cores, can still schedule"]
  R --> USED["Actually used: 1.5 cores"]
  R --> SLACK["Slack: 4.5 cores reserved but idle"]
    

Typical sizes of the gap

Surveys of real clusters by cost tools and cloud vendors have repeatedly found that average CPU usage is a small fraction of what is requested, often well under half, and memory usage is higher but still well below requests. Exact numbers vary widely from company to company, so measure your own. The key lesson is that the gap is usually large enough that fixing it can shrink the node count noticeably, before any other technique is applied.

Why the gap persists

Bin packing works on the numbers you give it. If the numbers are inflated, even perfect packing wastes money. So the first job is to make requests honest.

The right-sizing loop

CPU and memory behave differently

CPU is a compressible resource. If a container wants more than it has, it simply runs slower. Memory is incompressible. If a container needs more memory than its limit, the system kills it. This means you can be bolder when trimming CPU requests, and you must be more careful with memory. Many teams set memory requests equal to memory limits, for predictability, and treat CPU limits with caution, since very tight CPU limits can slow down applications even when the node has spare capacity.

Tools that help

Automatic versus manual changes

Now we reach the first main idea. Bin packing is an old problem from computer science, and it describes exactly what a cluster must do all day long.

The classic problem

You have items of different sizes, and bins of a fixed size. You want to fit all the items using as few bins as possible. In Kubernetes, the items are pods, sized by their requests. The bins are nodes, sized by their allocatable capacity. Fewer bins means fewer nodes, which means a smaller bill.

Two dimensions at once

Real packing in Kubernetes is harder than packing a single number, because every pod has at least two sizes: CPU and memory. A node can be full on one and empty on the other. A memory-hungry pod and a CPU-hungry pod can share a node well, because they need different things. Two memory-hungry pods may fill the memory while leaving CPU unused. This mismatch is called stranded capacity, and we meet it again in the next chapters.

No perfect answer, only good ones

Finding the absolute best packing is a famously hard problem. Real systems use fast, simple rules that give good results most of the time. A common rule is to place larger items first and put each into the fullest bin that still has room. The Kubernetes scheduler and the node autoscalers use rules of this kind, plus extra constraints such as zones, affinities, and spreading requirements.

flowchart LR
  subgraph A["Poor packing: 4 nodes"]
    A1["Node 1: 40 percent full"]
    A2["Node 2: 35 percent full"]
    A3["Node 3: 30 percent full"]
    A4["Node 4: 45 percent full"]
  end
  subgraph B["Good packing: 2 nodes"]
    B1["Node 1: 90 percent full"]
    B2["Node 2: 60 percent full"]
  end
  A -->|"consolidate"| B
    

Why tight packing is not always the goal

Packing nodes to one hundred percent leaves no room for traffic spikes, node failures, or rolling updates. A cluster with no spare room will leave new pods pending whenever anything unusual happens. So the aim is tight but not brittle: high utilization with deliberate headroom. Many teams aim for requested utilization well above half and below the top, and tune from there according to how quickly new nodes can be added and how risky an outage is.

The scheduler is the daily packer of the cluster. Its default behavior is tuned for safety, not for thrift, and understanding that default is the key to packing better.

Filter, then score

As Figure 1 showed, the scheduler works in two steps. First it filters: it removes nodes that cannot host the pod, because of insufficient requested room, taints, required labels, or other rules. Then it scores the remaining nodes with several plug-in rules and picks the highest total. Each rule looks at a different concern, such as whether the image is already downloaded on the node, whether the pod’s affinity wishes are met, or how balanced the node’s resources are.

The default favors spreading

By default, the scheduler tends to prefer nodes with more free room. This spreads pods across many nodes. Spreading is good for resilience, because no single node holds everything, and good for performance, because each pod has less competition. But it is bad for cost, because it keeps many nodes partly filled instead of filling a few completely, which blocks the autoscaler from removing the emptier nodes.

Choosing a packing strategy

Where the setting lives

The scheduler can be configured through scheduler profiles, which let operators choose which scoring rules are active and how strongly each counts. On self-managed clusters you can change the default profile, or define a second scheduler profile and select it by name in specific pods. On many managed services, the default scheduler cannot be freely changed, and the cloud provider may offer a packing-friendly option or a different node provisioning tool instead. Always check what your provider allows, since this varies by service and by version.

Packing at placement time versus at removal time

There are two moments to pack well. At placement time, the scheduler chooses a good node for a new pod. At removal time, a consolidation tool moves pods off lightly used nodes and deletes those nodes. The first creates a good layout from the start. The second repairs the layout as pods come and go. Strong cost control uses both. The next chapters look at the second.

Even with a good scheduler, nodes develop small unusable gaps. Understanding these gaps explains why clusters seem full while still having plenty of unused capacity.

What fragmentation is

Fragmentation means that free capacity exists in total, but it is spread across nodes in pieces too small for any new pod. Imagine five nodes, each with one free CPU core. In total you have five free cores, yet a pod that needs four cores cannot be placed anywhere. The pod stays pending, and the autoscaler adds a new node, even though five cores of room were sitting idle.

Stranded resources

A related problem is stranding. Suppose a node has plenty of free CPU but no free memory, because the pods on it ask for a lot of memory. The spare CPU cannot be used by pods that also need memory, so it is stranded. Stranding happens when the mix of pod shapes does not match the shape of the nodes.

flowchart TD
  N1["Node A: CPU free, memory full"] --> S1["Spare CPU is stranded"]
  N2["Node B: memory free, CPU full"] --> S2["Spare memory is stranded"]
  S1 --> FIX["Fix: mix pod shapes, choose node shapes that match your workloads"]
  S2 --> FIX
    

Ways to reduce fragmentation

Packing pods well is only half the story. Someone must also add nodes when pods do not fit and remove nodes when they are no longer needed. That is the job of node autoscalers.

The basic loop

A node autoscaler watches for pods that are pending because no node has room. It then adds a node that can fit them. It also watches for nodes that are empty or lightly used, and removes them when their pods can be safely moved elsewhere. The cluster therefore grows and shrinks with the work, and the bill follows.

sequenceDiagram
  participant D as Deployment
  participant S as Scheduler
  participant A as Node autoscaler
  participant C as Cloud API
  D->>S: Create new pods
  S-->>D: No node has room, pods pending
  A->>S: Sees pending pods
  A->>C: Request a new node
  C-->>A: Node joins the cluster
  S->>S: Place pending pods on the new node
  Note over A,S: Later, load falls
  A->>S: Finds a lightly used node
  A->>C: Drain pods and delete the node
    

Two families of autoscaler

Why scale-down is the hard half

Adding a node is easy. Removing one is careful work. Before deleting a node, the autoscaler must check that every pod on it can be moved: that other nodes have room, that no pod has a rule that forbids moving, and that moving it will not break a service. Many clusters keep nodes alive for a long time because of a single stubborn pod. Common blockers are listed below.

Pods come and go all day. Over time, a once tidy cluster becomes patchy, with many half-empty nodes. Consolidation is the practice of tidying it again while it keeps running.

Why clusters get messy

Imagine a cluster at the end of a busy day. Traffic peaked at noon, so the autoscaler added many nodes. By evening, most pods were removed, but each node still holds a few survivors. Nothing is fully empty, so no node can be deleted, yet the total load would fit on far fewer machines. This is the natural drift toward waste.

How consolidation works

Tools that consolidate

The descheduler

A separate tool called the descheduler evicts pods that are badly placed so that the scheduler can place them again. It can, for instance, evict pods from lightly loaded nodes so that those nodes empty out, or fix placements that no longer follow the rules. It does not add or remove nodes. It only moves pods, and then a node autoscaler removes the emptied nodes. Used together, they form a repacking pipeline.

flowchart LR
  A["Patchy cluster: many half-empty nodes"] --> B["Descheduler or consolidation evicts pods from light nodes"]
  B --> C["Scheduler repacks pods onto fuller nodes"]
  C --> D["Emptied nodes are deleted by the node autoscaler"]
  D --> E["Tighter cluster and lower bill"]
    

The size of your nodes changes how well pods pack. This is one of the quietest, most powerful design choices in a cluster.

The trade-off in plain words

Large nodes pack well, because big gaps are rare and fixed overhead is shared by more pods. But a large node holds many pods, so losing one hurts more. Small nodes limit the damage of a failure and add capacity in small steps, but they waste more on overhead and leave awkward gaps. Neither extreme is right for most clusters.

Hidden limits to check

Some cloud networking models limit how many pods a node can hold, based on how many network addresses the machine type supports. A large node may therefore run out of pod slots before it runs out of CPU, wasting capacity again. Check the pod-per-node limit for the instance types you plan to use, and the options your provider gives to raise it. These limits differ by provider and by version.

Mixed fleets win

The best answer is often not one size but a range. Direct provisioning autoscalers can choose from many instance types according to what is pending, which fits big pods onto big nodes and small leftovers onto small ones. Even with group-based autoscalers, a small number of groups with different sizes can help. Mixed fleets also help for a second reason that later chapters explain: they make spot capacity far more reliable.

flowchart TD
  Q1{"Are your pods large, with a few needing many cores?"} -->|"yes"| BIG["Include large node types"]
  Q1 -->|"no"| Q2{"Do you hit pod-per-node limits before CPU?"}
  Q2 -->|"yes"| MID["Prefer medium nodes or raise the limit"]
  Q2 -->|"no"| Q3{"Is a single node failure very costly?"}
  Q3 -->|"yes"| SMALL["Prefer more, smaller nodes"]
  Q3 -->|"no"| MIX["Use a mixed fleet and let the autoscaler choose"]
    

Matching the CPU-to-memory ratio

Cloud machine families come in shapes: some have much memory per core, some have little. If your workloads use about four gigabytes per core, a family with that ratio avoids stranding. If a few workloads are unusual, such as large caches or compute-heavy jobs, put them on separate node pools labeled for them. Check your actual ratio from your own usage data before deciding.

Newer processor families

Many clouds offer processors based on different designs, often priced lower for similar performance on suitable workloads. If your applications run on more than one processor architecture, a mixed fleet can lower costs further. This requires images built for each architecture and testing, so treat it as a later optimization once the basics are done.

We now turn to the second lever: price. Spot capacity is the largest single discount most teams can access, and also the one that demands the most care.

What spot capacity is

Cloud providers keep large amounts of spare machines so they can meet demand. When some of those machines are idle, the provider sells them at a steep discount. This is called spot capacity on some clouds, and goes by similar names such as preemptible or low-priority machines on others. The discount is commonly large, often well over half off the standard price, though the exact figure changes by time, region, and machine type. The catch is that the provider can take the machine back when it needs the capacity, usually with a short warning.

The three ways to buy compute

Why spot fits Kubernetes so well

Kubernetes was built on the assumption that machines fail. It restarts pods elsewhere, replaces unhealthy nodes, and keeps the desired number of copies running. A spot interruption is like a machine failure that comes with advance notice, which is actually easier to handle. So if your applications already tolerate a node disappearing, spot is a natural fit. The remaining work is making that tolerance real instead of assumed.

The central skill of running on spot is handling the moment a node is taken away. Let us walk through what happens, and what makes the difference between a calm event and an outage.

The life of an interruption

sequenceDiagram
  participant Cloud as Cloud provider
  participant H as Termination handler
  participant K as Kubernetes
  participant P as Pods on the node
  participant A as Autoscaler
  Cloud->>H: Interruption notice with short countdown
  H->>K: Mark node unschedulable (cordon)
  H->>K: Drain node, evict pods gently
  K->>P: Send termination signal, start grace period
  P-->>K: Finish requests and shut down cleanly
  K->>K: Recreate pods on other nodes
  A->>Cloud: Request replacement capacity if needed
  Cloud->>Cloud: Take the machine back
    

The pieces that make it safe

The two-minute reality

Warnings are short. Some clouds give around two minutes, others give less, and some give very little for certain machine types. That means a drain must be quick, and your pod’s shutdown grace period should fit inside the warning window. A pod that takes five minutes to shut down will simply be killed halfway. Test your actual shutdown time before trusting it.

Pod disruption budgets in practice

A pod disruption budget states the smallest number, or share, of replicas that must stay available during voluntary disruptions such as drains. It protects against well-behaved removals, which include consolidation and graceful spot handling. It cannot stop a hard failure or an abrupt reclaim. Set budgets that match reality: a budget that allows no disruption at all blocks consolidation and draining, while one that is too loose may allow too many replicas to leave together.

Spot interruptions are not random in one way: machines of the same type in the same zone are often reclaimed together. The way to stay safe is to avoid putting all your eggs in one pool.

What a capacity pool is

A capacity pool is the set of spare machines of one type in one zone. Prices and interruption chances are set per pool. When demand rises in a pool, spot machines there may be reclaimed together, and new ones may be unavailable for a while. If your whole fleet is one instance type in one zone, a single pool event can remove everything at once.

How to diversify

flowchart TD
  W["Workload needs capacity"] --> SP{"Is spot capacity available in any allowed pool?"}
  SP -->|"yes"| PICK["Pick a pool with low interruption chance and good price"]
  SP -->|"no"| OD["Fall back to on-demand capacity"]
  PICK --> RUN["Run pods on spot nodes"]
  OD --> RUN2["Run pods on on-demand nodes"]
  RUN -.->|"interruption"| REPL["Replace in another pool"]
    

The cost of a narrow list

When teams choose only one or two instance types for spot, they see frequent interruptions and long periods with no capacity, and conclude that spot is unreliable. Often the real cause is the narrow list. Widening it usually improves both price and stability. It does require your workloads to run acceptably on any of the allowed types, so avoid requiring a very specific machine unless truly necessary.

Not every workload should run on spot. The skill is sorting your applications into the ones that welcome interruption and the ones that cannot take it.

A simple sorting guide

The mixed-floor pattern

For customer-facing services, a popular pattern keeps a floor of on-demand capacity that can serve the minimum safe traffic by itself, and runs the rest of the replicas on spot. If every spot node vanishes, the service continues at a reduced but safe level while replacements arrive. This pattern gives most of the discount with a modest risk.

flowchart LR
  subgraph FLOOR["On-demand floor: always on"]
    F1["Replica 1"]
    F2["Replica 2"]
  end
  subgraph SPOTS["Spot layer: cheap, can disappear"]
    S1["Replica 3"]
    S2["Replica 4"]
    S3["Replica 5"]
    S4["Replica 6"]
  end
  LB["Traffic"] --> FLOOR
  LB --> SPOTS
    

How to steer workloads

Kubernetes lets you steer pods using node labels, taints and tolerations, and affinity rules. Spot nodes are usually labeled and sometimes tainted so that only workloads that explicitly accept them land there. A pod can say it prefers spot but accepts on-demand, or that it must avoid spot entirely. Preferences, rather than hard requirements, keep pods schedulable when spot is scarce.

Cost savings come from many scaling mechanisms working together, each at a different layer. Knowing which layer does what stops them from fighting each other.

Four layers, four questions

flowchart TD
  T["Traffic rises"] --> H["Horizontal autoscaler adds pods"]
  H --> PEND["New pods may be pending"]
  PEND --> N["Node autoscaler adds nodes"]
  N --> BUY["Allocation policy chooses spot or on-demand"]
  V["Vertical recommendations tune pod sizes over time"] -.-> H
  T2["Traffic falls"] --> H2["Horizontal autoscaler removes pods"]
  H2 --> C["Consolidation removes empty nodes"]
    

The horizontal autoscaler

The Horizontal Pod Autoscaler changes the number of replicas according to a measured signal, most often average CPU use, and in newer setups custom metrics such as requests per second. Because it measures usage as a share of the request, an inflated request makes it scale too late, since utilization appears low. Another reason right-sizing comes first.

Event-driven scaling

Event-driven autoscalers, with KEDA being a widely used example, scale workloads according to external signals such as queue length, and can scale to zero when there is no work. This is useful for background workers, which often idle for long stretches. Scaling to zero removes their cost entirely during those stretches, and lets nodes be released.

The pitfalls of stacking

Headroom: paying a little for speed

When a spike arrives, adding a node takes time, commonly from tens of seconds to several minutes. Some teams run a small amount of placeholder pods at low priority that reserve space. When real pods arrive, the placeholders are evicted instantly, the real pods start at once, and the autoscaler quietly adds a node to restore the placeholders. This costs a little but protects user experience during bursts.

Spot is for flexible work. For the steady core of your load, there is a different discount: promise the cloud provider a certain amount of usage in return for a lower price.

How commitments work

Cloud providers offer discounts, commonly called reserved instances, savings plans, or committed use discounts, in exchange for committing to a level of spending or capacity for one to three years. The discount can be substantial, often between a quarter and well over a third or more depending on term and flexibility. The risk is that you pay for the commitment even if you use less than promised.

Layering the three purchase types

flowchart TD
  L["Total compute demand over time"] --> B["Steady baseline: covered by commitments"]
  L --> M["Predictable variable load: on-demand"]
  L --> S["Flexible, interruption-tolerant load: spot"]
  B --> C1["Lowest guaranteed price"]
  M --> C2["Full price but no commitment"]
  S --> C3["Cheapest per hour but can be reclaimed"]
    

The right order of operations

After compute, the next surprises are usually disks and data transfer. They are less visible, and often easier to reduce than people expect.

Storage costs

Pods that need durable data request persistent volumes, which are disks from the cloud. These disks are billed whether or not anything reads them. Several patterns inflate the bill: disks left behind after a workload was deleted, volumes sized far above real need, expensive high-performance disk types used for data that is rarely touched, and snapshots kept forever.

Network costs

Cloud providers usually charge for data that crosses zone boundaries, and more for data that leaves to the internet. Inside a cluster that spans several zones, services that talk to each other a lot can generate large cross-zone transfer without anyone noticing. A chatty service pair in different zones, exchanging a steady flow of data, can cost more in transfer than the machines themselves.

Ways to reduce transfer costs

The cheapest machine is one that is not running. Many environments are used only during working hours, yet run all day and night.

The simple arithmetic

A week has one hundred sixty-eight hours. A development environment used from nine to six on weekdays is needed for only about forty-five of them, which is under thirty percent. Running it around the clock means paying for more than seventy percent of the time it sits idle. Switching it off outside working hours cuts its compute cost by well over half.

Ways to turn things off

The cold start question

Switching things off has a price: when someone arrives in the morning, the environment must start. If starting takes ten minutes, people complain. Good designs start environments a little before working hours, keep images cached, and keep startup times short. For on-demand scale-to-zero, the first request may be slow, which is acceptable for internal tools but not for customer-facing paths.

Technical levers only work if the organization notices cost. People optimize what they can see. This chapter covers how to make spend visible and owned.

Allocation: who spent what

Cloud invoices list machines, not teams. In a shared cluster, a single machine may host pods from ten teams. To divide the bill, tools measure each pod’s share of the node, based on its requests and sometimes its usage, and add up the shares per namespace, label, or team. Open-source and commercial tools, such as OpenCost and Kubecost, do this by combining cluster metrics with cloud price data.

Showback and chargeback

Guardrails inside the cluster

Labels make everything possible

Consistent labels for team, service, environment, and cost center are the foundation of allocation. Without them, a large share of spend stays “unallocated” and nobody feels responsible. Make labels required by policy at deployment, rather than asking people to remember.

The FinOps habit

The discipline of managing cloud cost as a shared engineering and finance activity is often called FinOps. Its core cycle is to inform teams with clear data, to optimize with the techniques in this post, and to operate with regular reviews and ownership. The practice matters as much as any single tool, because savings that are not reviewed tend to slip away.

Let us walk through a story, using plain numbers and no code. The company and figures are invented to teach the order of steps. Real results depend on your workloads, prices, and starting waste.

The starting point

A mid-sized online shop runs its services on one cluster with sixty identical on-demand nodes, running day and night. A review shows that nodes are about thirty percent utilized by real usage, while the scheduler sees them as around seventy percent full by requests. Development and test environments sit in the same cluster. The monthly compute bill is the baseline we call one hundred units.

The five steps

An illustrative result

flowchart LR
  S0["Start: index 100"] --> S1["Cleanup: 95"]
  S1 --> S2["Right-size: 72"]
  S2 --> S3["Consolidate and mix: 58"]
  S3 --> S4["Spot for flexible work: 40"]
  S4 --> S5["Commit the floor: 34"]
    

What the team measured along the way

At each step, the team watched user-facing latency, error rates, pod restarts, and pending pods. One step, right-sizing memory, caused a handful of restarts that were fixed by raising two requests. Without these measurements, the team would not have known whether a saving was healthy or a hidden risk.

The following story is a fictional composite of events that teams commonly describe. It shows how a good design absorbs a spot shortage, and how a weak one does not.

The setting

A ticketing company runs its booking front end on spot nodes, to save money. On a busy afternoon, with a popular event on sale, a large number of spot nodes in one zone receive interruption notices within minutes of each other.

Version one: the weak design

Version two: the redesigned system

After the incident, the team made five changes. They widened the allowed instance list to many families and sizes, spread nodes over three zones, kept a small on-demand floor able to serve minimum traffic, set pod disruption budgets, and rehearsed interruptions monthly by terminating spot nodes in a staging cluster.

sequenceDiagram
  participant Cloud as Cloud provider
  participant K as Cluster
  participant F as On-demand floor
  participant S as Spot layer
  participant U as Users
  Cloud->>S: Interruption notices in one pool
  S->>K: Nodes cordoned and drained
  K->>F: Floor keeps serving traffic
  K->>Cloud: Autoscaler requests other pools and zones
  Cloud-->>K: Capacity found in other pools
  K->>S: New spot nodes join, pods rescheduled
  U->>F: Requests continue, slightly slower
    

What the team learned

Every technique in this post trades something for money. Good engineering does not hide these trades. It names them and chooses them on purpose.

The triangle

Think of three goals pulling against each other: low cost, high reliability, and fast response to change. You can improve any two easily, but the third suffers. A cluster packed tightly and fed on spot is cheap, and may be less reliable and slower to absorb spikes. A cluster with generous headroom and on-demand machines is reliable and quick, and expensive. The art is choosing the right mix for each workload.

flowchart TD
  C["Low cost"] --- R["High reliability"]
  R --- F["Fast response to spikes"]
  F --- C
  C -.- NOTE1["Pushing cost down: tighter packing, more spot"]
  R -.- NOTE2["Pushing reliability up: spread, headroom, on-demand"]
  F -.- NOTE3["Pushing speed up: spare capacity, warm nodes"]
    

Technique by technique

Different services deserve different settings

Not every workload needs the same balance. A payment service deserves generous headroom and on-demand machines. A nightly report generator deserves the cheapest spot nodes and tight packing. Create a small number of service tiers, each with a clear recipe for requests, spread, purchase type, and disruption rules. Teams then choose a tier instead of arguing about every setting.

Here are the habits that teams with lean, stable clusters tend to share. Think of them as a checklist to return to.

A starter sequence for a new team

An anti-pattern is a habit that looks sensible but leads to trouble. Each one below is written as a short decision record so you can see why people choose it and what to do instead.

The ideas in this post are the same on every cloud, but the product names, defaults, and limits differ. This chapter gives a map, so you know what to look up for your own platform.

The same ideas under different names

Two billing models

Most clusters bill by the node, so you pay for the machine whether or not pods fill it. This is why packing matters. Some managed modes bill by the pod request instead, so you pay for what your pods ask for, and the provider handles packing. In those modes, right-sizing requests becomes the single most important lever, because requests are directly your bill. Packing is still important, but it is the provider’s problem and not yours.

With so many levers, where should you begin? This guide helps you pick the next step based on what you see in your own cluster.

flowchart TD
  START["Cluster bill is too high"] --> Q1{"Are there unused or forgotten resources?"}
  Q1 -->|"yes"| A1["Delete them and schedule non-production shutdowns"]
  Q1 -->|"no"| Q2{"Is requested capacity far above real usage?"}
  Q2 -->|"yes"| A2["Right-size requests with measured data"]
  Q2 -->|"no"| Q3{"Are many nodes half full?"}
  Q3 -->|"yes"| A3["Improve packing: consolidation, wider instance list, fewer pod sizes"]
  Q3 -->|"no"| Q4{"Is most work tolerant of interruption?"}
  Q4 -->|"yes"| A4["Move flexible work to diversified spot with a floor"]
  Q4 -->|"no"| Q5{"Is a stable always-on floor measured?"}
  Q5 -->|"yes"| A5["Commit to a conservative share of the floor"]
  Q5 -->|"no"| A6["Improve visibility with labels and allocation reports"]
    

Comparing the main levers

These are the questions people ask most often when they begin to work on Kubernetes cost.

We began on a moving day with a hundred boxes and a fleet of rented trucks. Kubernetes cost work is the same game: fewer trucks, cheaper trucks, and a plan for the ones taken back.