Before learning any tool, it helps to feel the problem it solves.

Every application needs computers to run on. In Kubernetes, those computers are called nodes. If you buy too few nodes, your application slows down or stops working when many users arrive. If you buy too many, you pay for computers that do nothing. Cloud providers bill you for every running virtual machine, usually by the second, so an idle machine is simply money leaving your account.

Traffic is rarely flat. A food-delivery app is busy at lunch and dinner. A ticket-booking site explodes when a popular concert goes on sale. A payroll system works hard on the last day of the month. A fixed-size cluster has to be sized for the busiest moment, which means it is wasteful for the other twenty-three hours of the day. The smarter plan is a cluster that breathes: it grows when work arrives and shrinks when work leaves.

Three different things can scale

Kubernetes has more than one autoscaler, and beginners often mix them up. Each one answers a different question.

• How many copies of my app should run? The Horizontal Pod Autoscaler (HPA) answers this by adding or removing pods.
• How big should each copy be? The Vertical Pod Autoscaler (VPA) answers this by adjusting the CPU and memory each pod asks for.
• How many servers do I need to hold all those copies? The Cluster Autoscaler answers this by adding or removing nodes.

flowchart TB
  A["Traffic grows"] --> B["HPA creates more pods"]
  B --> C["Scheduler tries to place the pods"]
  C --> D{"Is there room on existing nodes?"}
  D -->|"Yes"| E["Pods start running"]
  D -->|"No"| F["Pods wait in Pending state"]
  F --> G["Cluster Autoscaler adds nodes"]
  G --> E
    

A handful of words carry almost the whole story. Learn them once and everything else becomes easy.

Requests are the currency of autoscaling

This is the single most important idea in the whole article. The scheduler does not look at how busy a node really is. It looks at the requests written in pod definitions and adds them up. If a node has 4 CPUs available and four pods each request 1 CPU, the node is full in the eyes of the scheduler, even if those pods are actually sitting idle and using almost nothing.

The Cluster Autoscaler uses the very same arithmetic. It makes its decisions from requests, not from live usage. That is why a cluster can look half empty on a monitoring dashboard and still be unable to take one more pod. It is also why pods without any requests confuse the autoscaler badly, a problem we return to later.

Rules that narrow the choices

Pods can carry extra rules. A node selector says “only run me on nodes with this label”. A taint on a node says “keep out unless you carry my matching toleration”. Affinity and topology spread rules say things like “keep my replicas in different zones”. These rules matter a lot, because the autoscaler can only add a node that would truly accept the waiting pod. A node of the wrong kind, however large, does not help.

A short history, a clear definition, and a list of common misunderstandings.

The Cluster Autoscaler, often shortened to CA, is an open-source program that lives in the Kubernetes autoscaler repository and is maintained by the Kubernetes SIG Autoscaling community. It has been part of the ecosystem since the mid-2010s, which makes it one of the most battle-tested pieces of the platform. It runs as an ordinary pod inside your cluster, talks to the Kubernetes API on one side and to your cloud provider on the other, and changes the number of nodes in your node groups.

Its job can be said in two sentences. If some pods cannot be scheduled because no node has enough free room, and a new node from one of your groups would fix that, add the node. If some nodes have been mostly unused for a while and their pods could happily live elsewhere, remove those nodes.

How it differs from its neighbours

Three popular misunderstandings

• “It scales my pods.” No. It never changes how many pods a Deployment has. That is the HPA’s job.
• “It reacts to CPU spikes.” No. It reacts to pods that cannot be scheduled, which is a scheduling event, not a usage event.
• “It makes my nodes bigger.” No. It only adds or removes whole nodes using templates you already defined in node groups.

One small program, several clear responsibilities.

The autoscaler is a single binary running one main loop. It does not need a database. It reads the world from the Kubernetes API, makes decisions in memory, and writes its orders to the cloud provider. Here is the big picture.

flowchart LR
  U["Users and CI systems"] --> API["Kubernetes API server"]
  API --> SCH["kube-scheduler"]
  SCH -->|"cannot place pod"| PEND["Pending pods"]
  CA["Cluster Autoscaler"] -->|"reads pods and nodes"| API
  PEND --> CA
  CA -->|"asks to change size"| CP["Cloud provider API"]
  CP --> NG1["Node group A"]
  CP --> NG2["Node group B"]
  NG1 --> N1["New nodes"]
  N1 -->|"register and become Ready"| API
  CA -->|"writes health and status"| CM["Status ConfigMap in kube-system"]
    

The moving parts inside

Where it runs

On a self-managed cluster or on Amazon EKS, the autoscaler normally runs as a Deployment in the kube-system namespace on one of your worker nodes. You may run more than one replica, but only one is active at a time because they elect a leader. On Google Kubernetes Engine, the autoscaler is part of the managed control plane, so you configure it but never see its pod.

From a pod stuck in Pending to a fresh node, step by step.

Scale-up begins when the scheduler gives up on a pod. The scheduler writes a condition on the pod saying it is unschedulable, and that is the signal the autoscaler waits for. On its next scan it collects all such pods. Pods that are brand new may be ignored for a moment so the scheduler gets a fair chance first.

flowchart TD
  A["Scan every 10 seconds"] --> B{"Any unschedulable pods?"}
  B -->|"No"| C["Check scale-down instead"]
  B -->|"Yes"| D["Build a template node for each group"]
  D --> E["Simulate: would the pending pods fit?"]
  E --> F{"Does any group work?"}
  F -->|"No"| G["Record a NotTriggerScaleUp event"]
  F -->|"Yes"| H["Expander picks one group"]
  H --> I["Increase the group size"]
  I --> J["Wait for nodes to register"]
  J --> K["Scheduler places the pending pods"]
    

Template nodes, in simple words

The autoscaler cannot ask a node that does not exist yet how big it will be, so it guesses from a template. If the group already has a running node, that node is copied as the model. If the group is empty, the autoscaler asks the cloud provider or reads tags on the group. System pods that run on every node, called DaemonSets, are subtracted from the template, because they will take their share of each new machine before your pods get any.

When the cloud says no

Sometimes a group is asked to grow but nodes never appear: the cloud has no capacity of that machine type, a quota is exhausted, or the machine boots but never joins the cluster. The autoscaler waits up to a limit, fifteen minutes by default, then gives up on that attempt and places the group in a temporary backoff. During backoff it tries other groups instead. Repeated failures lengthen the backoff, so a broken group is not hammered over and over.

Removing a server is much more delicate than adding one, so the autoscaler is deliberately cautious.

Adding a node is almost risk-free. Removing a node can hurt running users if done carelessly. For that reason scale-down has many safety checks and long waiting times. A node is removed only when every check says it is safe.

flowchart TD
  A["Compute utilization of every node"] --> B{"Below the threshold, 50 percent by default?"}
  B -->|"No"| Z["Keep the node"]
  B -->|"Yes"| C["Mark the node as unneeded"]
  C --> D{"Unneeded for 10 minutes?"}
  D -->|"No"| C
  D -->|"Yes"| E{"Can all its pods fit elsewhere?"}
  E -->|"No"| Z
  E -->|"Yes"| F["Taint the node and evict its pods"]
  F --> G["Delete the machine from the node group"]
    

How utilization is measured

For each node the autoscaler adds up the requests of all pods on it and divides by the node’s allocatable amount, once for CPU and once for memory. The larger of the two ratios is the node’s utilization. Take a node with 4 CPUs allocatable where pods request 1.2 CPUs in total: that is 30 percent. If memory is at 45 percent, the node’s utilization is 45 percent. Both are under the default threshold of 50 percent, so the node becomes a candidate.

Safety checks before removal

Being under the threshold is only the first gate. The autoscaler then simulates moving every pod to other nodes. The node is spared if any of these is true:

• A pod is not owned by a controller such as a Deployment, so nothing would recreate it after eviction.
• A pod uses local storage that would be lost, unless it is marked as safe to evict.
• A pod carries the annotation cluster-autoscaler.kubernetes.io/safe-to-evict set to false.
• Evicting a pod would break a PodDisruptionBudget, a promise that a minimum number of replicas stay up.
• A pod has strict placement rules and no other node in the cluster would accept it.
• The node would drop the group below its minimum size, or the node is annotated to disable scale-down.

The graceful goodbye

When a node passes all checks, the autoscaler first adds a taint so that no new pods land on it. Then it asks Kubernetes to evict each pod politely through the Eviction API, which respects disruption budgets and the pod’s shutdown grace period. Only when the node is empty does it tell the cloud provider to delete the machine and shrink the group.

Follow one burst of traffic from the first new pod to the last deleted machine.

Now let us put the pieces together. Here is the life of a node, told as a conversation between the components.

sequenceDiagram
  participant D as Deployment
  participant S as Scheduler
  participant C as Cluster Autoscaler
  participant P as Cloud Provider
  participant K as New Node
  D->>S: Create 8 new pods
  S->>S: Only 2 pods fit on existing nodes
  S-->>C: 6 pods marked Unschedulable
  C->>C: Simulate and choose a node group
  C->>P: Raise desired size by 2
  P->>K: Boot two virtual machines
  K->>S: Nodes join and become Ready
  S->>K: Place the pending pods
  Note over C,K: Later, when load drops, scale-down reverses this
    

A worked example with real numbers

Suppose a cluster has three nodes, each with 4 CPUs available to pods, so 12 CPUs in total. Existing pods request 10 CPUs, leaving 2 CPUs free. Each pod of a web service requests 1 CPU. A flash sale begins, and the HPA raises the web service by 8 pods.

• Two of the new pods fit into the free 2 CPUs. Six pods stay Pending.
• Each new node can hold four such pods, so six pods need two nodes (6 divided by 4 is 1.5, rounded up to 2).
• The autoscaler asks the node group to grow by two.

The exact times depend on your cloud, instance type and image size, so treat them as rough guides. The important lesson is that a new node is never instant. From the first Pending pod to a running pod often takes several minutes, and that delay shapes many design choices later in this guide.

Two sets of dials control almost all of the autoscaler’s personality.

Expanders: choosing which group to grow

When more than one node group could host the waiting pods, the expander breaks the tie. You pick the strategy with a command-line flag. You can even chain several, for example priority first and least-waste as a tie-breaker.

The settings that matter most

The defaults below come from the open-source project. Managed services may change them, so always check your provider’s documentation.

Each node group also has its own minimum and maximum size. These two numbers are your safety rails: the minimum guarantees a baseline of capacity, and the maximum caps the bill. You can also set cluster-wide limits on total cores, total memory and total node count.

The autoscalers are teammates. Used together they are strong, used carelessly they fight.

HPA and Cluster Autoscaler

This is the classic pair. The HPA reacts to load by creating pods. When those pods cannot be placed, the Cluster Autoscaler provides nodes. Both are needed: HPA alone runs out of room, and Cluster Autoscaler alone has nothing to react to. The HPA relies on a metrics source such as the Kubernetes Metrics Server to read usage. Event-driven tools like KEDA can feed the HPA from queues and other signals, which suits background workers.

VPA and Cluster Autoscaler

The VPA changes the requests of pods, usually by restarting them. Better-sized requests give the autoscaler a more truthful picture, which is a real benefit. The warning is to avoid letting VPA and HPA act on the same CPU or memory signal for the same workload, because they will pull in opposite directions.

The waiting problem and the headroom trick

Remember that a new node takes minutes to arrive. During a sudden spike, users may wait that long. A popular fix is overprovisioning, also called headroom. You run a small Deployment of dummy “pause” pods that do nothing but reserve CPU and memory. These pods get a very low priority, lower than every real workload.

flowchart LR
  PP["Low priority pause pods hold spare room"] --> NEW["A real pod arrives"]
  NEW -->|"preempts a pause pod"| RUN["Real pod starts in seconds"]
  NEW --> EV["Evicted pause pod becomes Pending"]
  EV --> CA["Cluster Autoscaler adds a node in the background"]
  CA --> PP
    

When a real pod arrives and the cluster is otherwise full, Kubernetes evicts a pause pod instantly to make space. The real pod starts in seconds. The evicted pause pod becomes Pending, which triggers the autoscaler to add a node, quietly rebuilding the spare room. You pay for a little unused capacity and receive fast reaction in return.

The idea is the same everywhere. The plumbing underneath differs.

Amazon Web Services details

On AWS, the autoscaler can discover groups automatically by looking for two tags on each Auto Scaling group: one marking it as enabled and one carrying your cluster name. Alternatively you list groups by name with their minimum and maximum sizes. Several lessons come up again and again.

• Do not fight it. Do not attach the Auto Scaling group’s own scaling policies, and do not change desired capacity by hand, while the Cluster Autoscaler is in charge. Two controllers managing one number leads to chaos.
• Match the version. Use an autoscaler release whose minor version matches your Kubernetes minor version, because it embeds scheduler logic from that release.
• Similar instance types. If one group mixes instance types, choose types with the same CPU and memory. The autoscaler builds its template from one type, so a wildly different type makes its predictions wrong.
• Zones and storage. Disks such as EBS volumes live in a single zone. If pods use them, create one node group per zone and enable balancing between similar groups, so a pod needing a zone can always get a node there.
• Scaling from zero. A group may rest at zero nodes. To do this, the autoscaler needs to know what a node would look like, so you add tags for labels and taints on the group.

Google Cloud and Azure in short

On GKE you simply switch autoscaling on for a node pool and set its limits. The service hides the controller and offers profiles that trade speed against cost: one favours availability, the other removes underused nodes more eagerly. GKE Autopilot goes one step further and removes node management from you altogether. On AKS you enable the autoscaler per node pool with minimum and maximum counts, and a profile object holds the fine-tuning settings.

No tool is perfect. Knowing the costs helps you choose wisely.

Cluster Autoscaler or Karpenter?

Karpenter is a newer approach, reached its 1.0 release in 2024, and is now developed within the Kubernetes SIG Autoscaling community with a mature AWS implementation. Instead of growing fixed node groups, it looks at the waiting pods and launches machines of whatever suitable type fits them best, then regularly consolidates workloads onto fewer nodes. Neither is simply better.

The autoscaler can create and destroy servers. That is real power, so treat it carefully.

If an attacker or a bug controlled the autoscaler, they could launch hundreds of machines and run up a large bill, or delete nodes and cause an outage. Security here is mostly about giving it only the permissions it needs and keeping it healthy.

Cloud permissions

The autoscaler needs permission to read group information and to change group sizes or terminate instances inside those groups. Grant only those actions. Where the cloud supports it, restrict the write actions to groups carrying your cluster’s tag, so that a mistake in one cluster cannot touch another. Use short-lived workload identity, such as IAM roles for service accounts on AWS or workload identity on Google Cloud, rather than long-lived keys stored in a secret.

Kubernetes permissions

Inside the cluster, the autoscaler uses a service account with a role that allows it to read pods, nodes and related objects, evict pods, and update its own status ConfigMap and leader-election lease. Review the standard manifest and remove anything you do not use. Run it in kube-system with a high priority class so it is never the first pod to be pushed out under pressure.

Protecting the bill and the fleet

• Always set maximums. A node group maximum and a cluster-wide node limit are your cost firewall.
• Keep it off nodes it manages. Run the autoscaler on a small, stable node group it does not shrink, so it cannot delete the machine it lives on.
• Harden nodes the usual way. Patched images, restricted network rules and minimal permissions on the node role still apply to every machine it creates.
• Watch for abuse. A user who can create unlimited large pods can cause large scale-ups. Use namespace resource quotas and limit ranges to put a ceiling on what any team can request.

You cannot trust what you cannot see. The autoscaler gives you three windows into its mind.

Window one: events

Kubernetes events are short notes attached to pods and nodes. The autoscaler writes a note named TriggeredScaleUp on pods that caused a scale-up, and NotTriggerScaleUp on pods it could not help, with a reason such as “no node group fits” or “max size reached”. It also records scale-down events on nodes. Reading the events of a Pending pod is usually the fastest way to learn why it is stuck.

Window two: logs and the status ConfigMap

The autoscaler logs each loop’s key decisions. Raising log verbosity shows the simulation details and which blockers kept a node alive, but it is noisy, so use it for investigations rather than all the time. The status ConfigMap in kube-system shows cluster health, per-group sizes, and whether a group is in backoff.

Window three: metrics

The autoscaler exposes Prometheus metrics, by default on port 8085. These are the ones worth putting on a dashboard.

What to alert on

• Pending pods stay above zero for longer than your tolerance, for example fifteen minutes.
• Failed scale-ups keep climbing, which hints at quota or capacity trouble.
• The last-activity time stops advancing, meaning the loop may be hung or unhealthy.
• Node count nears the cluster maximum, so you are warned before the ceiling blocks growth.
• Average requested-to-allocatable ratio stays very low for days, hinting at inflated requests or a wasteful minimum size.

Habits that make autoscaling smooth, and habits that quietly break it.

Patterns that work

Anti-patterns to avoid

A checklist you can follow, then a troubleshooting path for the day something goes wrong.

Best practices

• Write honest requests. Everything else depends on them.
• Set sensible minimums and maximums. A minimum for a baseline, a maximum for cost safety.
• Match versions. Upgrade the autoscaler together with Kubernetes, one minor version at a time.
• Run it highly available. Two replicas with leader election and a critical priority class.
• Use graceful shutdown. Give pods realistic grace periods and a preStop hook so drains do not drop requests.
• Test scaling before you need it. Create a controlled load and watch pods, nodes and timings end to end.
• Review quotas and limits quarterly. Cloud quotas, subnet IP space and node limits all cap how far you can grow.

Common mistakes

• Expecting instant reaction and skipping headroom for traffic spikes.
• Leaving bare pods, or pods using local disk, on nodes you hoped would shrink.
• Forgetting that DaemonSets and system pods can block scale-down on small clusters.
• Setting a minimum size too high and then wondering why the bill never drops.
• Ignoring the events on Pending pods, which usually explain the problem outright.

Troubleshooting: pods are Pending and no node appears

Troubleshooting: idle nodes never go away

Work in the opposite direction. Check the node’s utilization against the threshold, look for blocking pods (bare pods, local storage, strict budgets, the do-not-evict annotation), confirm the group is above its minimum, and make sure no recent scale-up is still inside its cooling-off delay. The autoscaler’s logs usually say which pod blocked the node.

How large teams think about node autoscaling, plus a few typical scenarios.

Every autoscaling problem eventually leads back to one humble field in the pod definition.

We have said many times that the autoscaler works from requests. This chapter explains how to choose good ones, because a team that gets this right will see the rest of the system behave almost by itself. A team that gets it wrong will fight strange symptoms for months.

Too low, too high, and just right

If a request is too low, the scheduler believes the pod is tiny. It places many such pods on one node. When they all become busy at once, the node runs out of real CPU or memory. CPU shortage makes everything slow. Memory shortage is worse, because the kernel starts killing containers. The autoscaler never saw a problem, since on paper the node was never full.

If a request is too high, the opposite happens. The cluster looks full while the machines are actually quiet. The autoscaler buys more nodes to satisfy the paper demand, and the bill grows with no benefit to users. This waste is invisible unless you compare requested amounts with measured usage.

A healthy request sits a little above the typical busy usage of the pod, with room for normal spikes. Many teams aim for the request to cover the usual high point of the day, not the rarest peak of the year.

Quality of Service classes

Kubernetes sorts pods into three classes based on how requests and limits are set. These classes decide who is protected and who is sacrificed when a node runs short of memory.

For autoscaling, the key point is that BestEffort pods look free. They do not make a node look fuller, so they never contribute to scale-up and they quietly crowd nodes that the autoscaler believes are empty.

Guardrails you can put in place

• LimitRange. A namespace rule that gives default requests and limits to pods that forget to set them, and can also reject absurd values.
• ResourceQuota. A namespace rule that caps the total CPU, memory and pod count a team may request, protecting both fairness and the cloud bill.
• Admission policies. Cluster-wide checks that block deployments without requests at all.
• VPA in recommendation mode. The Vertical Pod Autoscaler can watch real usage and publish suggested requests without changing anything. Engineers then review and apply the suggestions on a schedule.

A worked example

A team runs 40 copies of a checkout service. Each pod requests 2 CPUs, yet measured usage shows a busy-hour average of 0.7 CPUs and a peak of 1.1. The team lowers the request to 1.25 CPUs, which still gives comfortable room above the peak. The total reserved CPU falls from 80 to 50. On nodes with 8 usable CPUs each, that is the difference between ten nodes and about six or seven. The autoscaler does the rest by removing the nodes that are no longer needed, and the saving arrives without a single change to the application code.

Cheap machines that the cloud may take back at short notice, and how to use them safely.

Cloud providers sell spare machines at a deep discount, often a large fraction below the regular price. On Amazon these are Spot Instances, and on other clouds they have similar names. The catch is that the provider can reclaim them when it needs the capacity back. On AWS the warning is usually about two minutes. For stateless, replicated workloads this trade is excellent, and the Cluster Autoscaler is a natural partner for it.

Which workloads suit spot machines

• Good fit: stateless web replicas with several copies, queue workers, batch processing that can retry, build and test jobs, and development environments.
• Poor fit: a single-replica database, long jobs that cannot save progress, and anything where a two-minute warning is not enough to finish cleanly.

The recommended shape

Most teams build two kinds of node groups: a spot group for flexible work and an on-demand group as the dependable base. The priority expander is then told to try the spot group first and fall back to on-demand when spot capacity is not available. Pods that must be safe carry a node selector or affinity that points only to on-demand nodes.

flowchart TD
  A["Pending pods need nodes"] --> B["Priority expander reads its ranked list"]
  B --> C{"Is the spot group able to grow?"}
  C -->|"Yes"| D["Add spot nodes"]
  C -->|"No or in backoff"| E["Add on-demand nodes"]
  D --> F["Pods run at lower cost"]
  E --> G["Pods run at normal cost"]
  F --> H{"Provider reclaims a spot node?"}
  H -->|"Yes"| I["Node drains and pods return to Pending"]
  I --> A
    

Making spot capacity dependable

Spot reliability depends mostly on variety. If a group uses only one machine type, a shortage of that type hurts you completely. If it can use many types of similar size, the provider usually finds some capacity somewhere. On AWS, a mixed instances policy with several equal-sized types and a capacity-aware allocation strategy lowers the chance of interruption. Remember the earlier rule: the types in one group should have matching CPU and memory so the autoscaler’s template stays truthful.

It also helps to run a small helper that listens for the provider’s interruption notice and starts a graceful drain at once, rather than waiting until the machine disappears. On AWS this job is done by the Node Termination Handler, and similar tools exist elsewhere.

Protect the service itself

• Keep at least two replicas of every service and spread them across nodes and zones, so one interruption cannot take everything down.
• Use PodDisruptionBudgets so that a burst of reclaimed nodes cannot remove too many replicas at once.
• Make shutdown fast and tidy, so the pod finishes its work inside the warning window.
• Keep a floor of on-demand capacity big enough to carry the minimum acceptable traffic.

Spreading across data centres makes a service sturdier, and also makes autoscaling a little more interesting.

A cloud region is split into several availability zones, which are separate buildings or groups of buildings with their own power and networking. Running copies of your application in more than one zone means a failure in one building does not stop the service. Kubernetes lets you ask for this with topology spread constraints. They say things such as “keep the number of replicas in each zone roughly equal”.

How spreading affects scale-up

Imagine a service that must keep an even number of pods in each of three zones, and the zone called A has no room left. Adding capacity in zone B does not satisfy the rule. The pod still waits, because the autoscaler must create a node in the right zone. This is why one node group per zone is so popular. The autoscaler can see exactly which group serves which zone and can grow the correct one.

When you do run one group per zone, turn on balancing between similar groups. The autoscaler then treats groups with matching machine shapes and labels as a family and grows them evenly, instead of filling one zone while the others stay small. A few labels naturally differ between zones, such as the zone name itself, so there is a setting that tells the autoscaler which labels to ignore when it decides that two groups are similar.

Storage is tied to a zone

Block disks on most clouds live inside one zone. A pod that uses such a disk can only run in that zone. If the pod is evicted and no node has room in that zone, it will stay Pending even though other zones have plenty of space. The autoscaler must add a node in the right zone, and this works well only when you have separate groups per zone.

A second helpful setting is the volume binding mode called wait for first consumer. With it, Kubernetes delays creating the disk until the pod has been placed, and then creates the disk in the same zone as the chosen node. This avoids the awkward situation where a disk is created in a zone first and the pod cannot find a node there.

Strict rules have a cost

The more rules you add, the fewer places a pod can go, and the harder it is for the autoscaler to find a node that helps. Treat strict anti-affinity and strict spread rules as precious tools. For many services, a softer preference expresses the same wish without risking a stuck pod. A common compromise is to make zone spreading strict for the services that truly need it, and preferred for everything else.

Launching autoscaling is the easy day. Keeping it healthy for years is the real work.

Engineers sometimes call the first deployment day one, and everything after it day two. Day two is where small habits decide whether the system is boring and reliable or a constant source of surprises.

Upgrading Kubernetes with the autoscaler in place

A cluster upgrade usually follows a clear order. The control plane moves first, then the worker nodes, then the add-ons. The Cluster Autoscaler is an add-on that must be moved to the matching version, because it embeds the scheduler’s logic for that Kubernetes release. A mismatch can cause strange mistakes in its simulations, such as believing a pod fits when the real scheduler disagrees.

• Read the release notes of both Kubernetes and the autoscaler for removed or changed settings.
• Test the new combination in a staging cluster that has similar node groups and similar workloads.
• Upgrade one minor version at a time, never skipping.
• Pause or watch scale-down during the rollout, so nodes are not removed while the cluster is mid-change.

Rotating node images

Nodes should be replaced regularly with fresh, patched machine images. When a node group gets a new image, old nodes are drained and replaced. During this process the autoscaler sees pods being evicted and Pending for a short time, and it may add temporary nodes. This is normal. Good disruption budgets and enough spare capacity make the rotation smooth. Some teams set a maximum node lifetime so that no machine lives forever and surprises never pile up.

A simple runbook

Changing settings safely

Treat the autoscaler’s configuration like application code. Keep it in version control, review changes, roll them out in staging first, and record why each value was chosen. When behaviour changes after an edit, you will be glad to know exactly what was edited and when. Tune one thing at a time and give each change a few days of normal traffic before judging it.

Startup delays and node readiness

A node is not useful the moment it appears. It needs to start the kubelet, join the cluster, run its networking and storage helpers, and often wait for system add-ons. Some teams apply a startup taint that keeps application pods away until every helper reports ready. The autoscaler has an option to ignore such known startup taints when it builds its template node, so it does not wrongly decide that the new node can never accept pods. Without this, you can see the autoscaler add more nodes than needed because it keeps thinking the earlier ones are unusable.

Autoscaling saves money only if the pieces around it are chosen with cost in mind.

The Cluster Autoscaler keeps your node count close to what your pods need. It does not, by itself, decide whether those pods are efficient or whether the machines are the best value. Cost comes from several layers, and each layer has its own lever.

The layers of cost

Large nodes or small nodes?

This is one of the most debated design choices. Both extremes have a price.

Many teams settle on a middle size and keep one or two special groups for unusually large or small needs. A useful guide is to pick a node size that can hold at least several of your biggest typical pods, so that rounding errors do not leave much space unusable.

Fragmentation, the hidden waste

Imagine nodes with 8 CPUs where most pods ask for 3 CPUs. Two pods fit per node and use 6 CPUs, leaving 2 CPUs that nothing can use. A quarter of every node is wasted, and the autoscaler cannot fix it, because it simply obeys the requests. Changing the request to 2.5 CPUs would allow three pods per node and use 7.5 CPUs. Small changes in pod size can alter the packing a lot, so it pays to look at the shape of your pods against the shape of your nodes.

Stable base, flexible top

Cloud providers offer discounts for committing to a steady amount of usage over a year or longer. The autoscaler makes this planning easier. Look at the lowest level your cluster reaches in a normal week. That floor is a good amount to cover with commitments. Everything above it moves up and down with traffic, and is best served by on-demand or spot capacity that you pay for only while it exists.

Pod density limits

Kubernetes allows a default of about one hundred and ten pods per node, and some clouds set a lower limit based on how many network addresses a machine can carry. A node may be nearly empty in CPU terms yet full by pod count. When pods are small and many, check this limit, since the autoscaler will add nodes for pod count too, and the machine size you picked may be wasteful in that case.

What changes when clusters grow big, and the quirks that surprise even experienced engineers.

Scale and speed

Kubernetes itself is designed for clusters of up to a few thousand nodes. In big clusters the autoscaler’s main loop has more to examine each time, and the simulation work grows with the number of pods and node groups. Its function-duration metric shows if loops are taking too long. A loop that takes longer than the scan interval means decisions arrive late. Remedies include fewer and larger node groups, avoiding needlessly complex scheduling rules, and giving the autoscaler enough CPU and memory of its own. Many organisations also prefer several medium clusters over one huge one, which limits the blast radius of any problem.

Parallel work

Removing empty nodes can happen in bulk, up to a configurable number at once, which keeps large clean-ups from being slow. Removing nodes that still hold pods is more careful and usually proceeds more gently. Recent releases have also improved how quickly the autoscaler can handle many node groups growing at the same time. Check the release notes of the version you run, since this area improves steadily.

Edge cases worth knowing

When things look healthy but are not

One of the trickiest situations is when the autoscaler reports no problem while users complain. This can happen when pods are scheduled fine but are starved of real resources because their requests are too low, or when a service reaches a limit that autoscaling does not see, such as a database connection limit, a network quota or a rate limit on a third-party service. The autoscaler solves the compute problem only. Always trace a slow user request along its full path before blaming capacity.

The human side

Autoscaling is not a replacement for load testing or capacity thinking. It is a safety net and a cost saver. The best-run teams combine it with regular tests, clear ownership of settings, and short written notes explaining the reasons behind each unusual value in their configuration. When a person new to the team reads those notes a year later, the system still makes sense.

If you remember only a handful of things, remember these.