Before looking at the machinery, we need to feel the problem it solves. Demand for software is never flat, and fixed capacity is always wrong.

The two ways fixed capacity fails

If you run a fixed number of servers, you must pick that number in advance. Pick too few, and a traffic spike overwhelms them. Pages load slowly, requests time out and customers leave. Pick too many, and you pay for idle machines around the clock. Most real traffic has peaks and valleys, such as a morning rush, a lunchtime lull, a sale event or a viral post. A single fixed number cannot be right for all of them.

Teams used to solve this by guessing the peak and paying for it all day. That works, but it is expensive. Cloud bills are driven by how much capacity you reserve, not by how much you use. Autoscaling aims to make the reserved amount follow the used amount as closely as possible.

Three different things can scale

Beginners often think “autoscaling” means one thing. In Kubernetes it means three different kinds of change, each handled by a different tool.

Those three defaults already tell a story. The system reacts every few seconds, but it is deliberately slow and cautious about shrinking. We will see why later in the guide.

Why this topic is harder than it looks

On a diagram, autoscaling looks simple: if busy, add more. In practice, every step has a delay. Measuring takes time. Computing takes time. Starting a new pod takes time. Finding or creating a new machine takes much more time. Meanwhile the traffic keeps changing. An autoscaler that reacts too fast thrashes up and down like a nervous driver pumping the brakes. One that reacts too slowly lets users suffer. The craft of autoscaling is the craft of tuning these delays.

Let us see all the players on one page, and how they cooperate like layers of a cake.

The layers

Autoscaling in Kubernetes works in layers. The layers depend on each other, and the bottom layers react to the top ones.

flowchart TB
    APP["Running pods"] -->|"usage numbers"| MET["Metrics pipeline"]
    MET --> HPA["HPA - how many pods"]
    MET --> VPA["VPA - how big each pod"]
    MET --> KEDA["KEDA - event driven scaling"]
    HPA --> DEP["Deployment replica count changes"]
    KEDA --> DEP
    VPA --> RES["Pod resource requests change"]
    DEP --> SCH["Scheduler places new pods"]
    RES --> SCH
    SCH -->|"no room - Pending pods"| CA["Cluster Autoscaler or Karpenter"]
    CA -->|"add or remove machines"| NODES["Cluster nodes"]
    NODES --> SCH

A chain reaction in plain words

Imagine traffic doubles. Pods become busy, so the metrics layer reports high CPU. The HPA asks for more pods. The scheduler tries to place them, but the existing machines are already full, so the new pods stay Pending. The Cluster Autoscaler sees Pending pods, asks the cloud for a new machine, and waits for it to join the cluster. The scheduler then places the pods on the new machine. Finally, the pods start, pass their health checks and begin serving traffic.

Count the steps and you can already see why this takes minutes, not seconds. Each handoff adds delay. A big goal of this guide is to help you shorten or hide these delays.

Every autoscaler in Kubernetes is built on one simple idea called the control loop. Understanding it makes all the later chapters feel natural.

Desired state versus actual state

Kubernetes works by comparing two things. The desired state is what you asked for, such as “run five copies of this application”. The actual state is what is really running right now. A small program called a controller watches both. When they differ, it takes an action to close the gap. Then it looks again. This repeating cycle of observe, compare and act is called a control loop or a reconciliation loop.

Where the autoscaler fits

The Deployment controller already keeps the number of pods equal to the replica count you wrote. An autoscaler adds a second, higher loop on top. It watches load, decides what the replica count should be, and writes that number into the Deployment. Then the Deployment controller does the work of creating or deleting pods. In this way, autoscalers never start containers themselves. They only change a number, and other controllers do the rest.

This works because Kubernetes defines a standard scale subresource, a tiny interface that says “here is the replica count, and you may read or change it”. Any object that supports it can be autoscaled, including Deployments, StatefulSets and ReplicaSets. Even custom resources can opt in.

Why loops are periodic, not instant

Control loops run on a timer. The HPA checks roughly every 15 seconds. This is a deliberate choice. Reacting to every tiny change would waste effort and cause constant flapping. A short sleep between checks smooths the behavior, much like how a thermostat has a small dead zone before it switches. It also protects the API server from being flooded with requests.

flowchart LR
    A["Read current metrics"] --> B["Compare with the target"]
    B --> C{"Outside the tolerance band?"}
    C -->|"no"| D["Do nothing"]
    C -->|"yes"| E["Compute desired replicas"]
    E --> F["Apply limits and stabilization"]
    F --> G["Write new replica count"]
    G --> H["Deployment controller adds or removes pods"]
    D --> I["Wait for next cycle"]
    H --> I
    I --> A

An autoscaler is only as smart as its data. Let us follow a single CPU number from the moment a container does some work to the moment an autoscaler sees it.

First, what is being measured?

CPU in Kubernetes is measured in cores or in thousandths of a core called millicores. A value of 500m means half of one CPU core. Memory is measured in bytes, usually written as mebibytes or gibibytes. The number Kubernetes uses for memory is the working set, which is the memory the container is actively using and that cannot easily be taken back by the system.

Two settings on every container matter a great deal. The request is how much the container is promised, and it is what the scheduler uses to find room. The limit is the most it is allowed to use. When we later talk about “CPU utilization”, the percentage is measured against the request, not against the limit and not against the machine. This single fact explains a huge number of autoscaling mysteries.

The measuring chain for CPU and memory

flowchart LR
    K["Linux kernel counters"] --> C["cAdvisor inside kubelet"]
    C --> KL["Kubelet resource metrics endpoint"]
    KL -->|"periodic scrape"| MS["Metrics Server - in memory"]
    MS --> AG["Kubernetes API aggregation layer"]
    AG --> HPA["HPA controller"]
    AG --> TOP["kubectl top command"]

From counters to rates

Remember that the kernel gives cumulative totals, such as “this container has used 3,000 seconds of CPU since it started”. An autoscaler wants a rate, such as “how much CPU is it using right now”. To get a rate, the system takes two readings a short time apart and divides the difference by the time between them. If the container used 6 more CPU seconds over a 12 second window, it was using half a core. That is why a freshly started container has no usable CPU number for a little while: there is only one reading, so no rate can be computed yet.

Metrics Server is a small program with a narrow job. Knowing exactly what it does, and what it refuses to do, saves a lot of confusion.

What it does

Metrics Server periodically contacts the kubelet on every node, reads the current CPU and memory numbers for every pod and node, and keeps only the most recent values in its own memory. It does not write to a disk. It does not keep history. It exists for one reason: to feed autoscalers and simple tools like the kubectl top command with fresh, cheap numbers.

In recent versions it collects data roughly every 15 seconds by default, which is why it is described as built for fast autoscaling. The exact interval is a setting, and managed Kubernetes services may configure it differently, so check your own cluster.

How it plugs into Kubernetes: the aggregation layer

Metrics Server is not part of the core API server. It is a separate program, yet you can ask the main API server for metrics as if it were built in. This works through the API aggregation layer. An object called an APIService tells the API server: “any request for the metrics group should be forwarded to this other service”. The API server acts as a front door and passes the request along, after checking who is asking and whether they have permission.

The API group used for resource metrics is named metrics.k8s.io. When the HPA wants CPU numbers, it sends a request to this group, and Metrics Server answers.

What it deliberately does not do

Common problems

• It is not installed. Some clusters do not ship with it. Without it, CPU and memory based autoscaling cannot work at all.
• It cannot reach the kubelets. Network rules or certificate settings may block it, and the HPA then reports that it cannot fetch metrics.
• A single replica becomes a single point of failure. If it crashes, autoscaling for resource metrics stops until it returns. Production clusters often run more than one copy.
• Brand-new pods show no data. For a short while after start, there is no complete reading, so the HPA treats those pods carefully, as we will see.

CPU is a poor signal for many applications. To scale on requests per second, queue length or response time, you need a richer pipeline.

Three metric APIs

Kubernetes defines three standard places where an autoscaler can ask for numbers. They are all served through the aggregation layer, but by different programs.

The Prometheus path

Most teams that need custom metrics already run Prometheus, a popular monitoring system. Prometheus works by scraping, which means it visits a list of targets on a schedule, such as every 15 or 30 seconds, and records the numbers they publish. Applications publish their own numbers, such as total requests handled, through a small web page of metrics. Other helper programs publish numbers about Kubernetes objects. One example is kube-state-metrics, which reports things like how many replicas a Deployment wants and has.

Prometheus stores the scraped data in a time series database. But the HPA does not know how to speak to Prometheus. It only speaks the Kubernetes metrics APIs. So a translator is placed in between. The Prometheus Adapter reads rules that you write, runs queries against Prometheus, and presents the answers through the custom or external metrics API. The HPA asks the Kubernetes API, the API forwards the question to the adapter, the adapter asks Prometheus, and the answer travels back.

flowchart LR
    APP["Application pods expose metrics"] -->|"scrape every 15-30 s"| PROM["Prometheus"]
    KSM["kube-state-metrics"] -->|"scrape"| PROM
    PROM -->|"query"| ADP["Prometheus Adapter - translation rules"]
    ADP --> AG["API aggregation layer"]
    AG -->|"custom.metrics.k8s.io"| HPA["HPA controller"]
    HPA --> DEP["Deployment replicas"]

The event-driven path with KEDA

KEDA, short for Kubernetes Event-driven Autoscaling, takes a different route. It ships with dozens of small connectors, called scalers, that know how to read numbers directly from systems such as message queues, streaming platforms, databases and cloud services. KEDA turns these into external metrics for an HPA, and also handles the special case of moving from zero pods to one and back, which the plain HPA does not do by default. We will study KEDA in a later chapter.

Why every extra hop matters

Count the stages in the Prometheus path. The application updates a counter. Prometheus scrapes it at the next interval. The adapter queries a rate over a window of time, often a couple of minutes. The HPA reads the answer on its next cycle. Each of these has its own timer. A spike that starts right after a scrape may not be visible for one full interval. A rate calculated over a long window smooths the spike and hides how sudden it was. So custom metric pipelines are usually slower than the Metrics Server path, and this is a price paid for better signals.

The pod autoscaler follows a clear recipe. Once you know it, you can predict exactly what it will do and why.

The core formula

Every cycle, the HPA calculates how many replicas it believes are needed using one simple proportional rule: take the current number of replicas, multiply by the ratio of the current metric value to the target value, and round up to the next whole number. In words, if the metric is twice the target, double the pods. If it is half the target, halve the pods.

A worked example

Suppose a web application runs 5 pods. Each pod requests 500m of CPU. You set a target of 60 percent average CPU utilization. During a busy period, each pod is actually using about 750m. That equals 150 percent of its request. The ratio of current to target is 150 divided by 60, which is 2.5. Multiply the 5 current pods by 2.5 and you get 12.5. Rounding up gives 13 pods. The HPA asks for 13 replicas.

After the new pods start, the same total work is spread across 13 pods instead of 5. Each now uses about 288m, which is roughly 58 percent of its request, very close to the 60 percent target. The system has settled.

The tolerance band

To avoid constant tiny adjustments, the HPA skips any change if the ratio is close to one. By default, if the current value is within 10 percent of the target, nothing happens. In our example, a ratio between 0.9 and 1.1 means “close enough”. This dead zone is like the small gap in a thermostat that stops it clicking on and off every few seconds.

What counts as a pod?

The HPA is careful about which pods it includes in the math.

• Pods being deleted are ignored.
• Pods that are not ready yet, or only just started, are treated cautiously. For CPU, a new pod may use unusual amounts while starting, so the HPA waits through a short initialization period before trusting its readings.
• Pods with missing metrics are handled with conservative guesses. When the HPA wants to scale up, it assumes a pod with missing data was using nothing, which makes scale-up smaller. When it wants to scale down, it assumes such a pod was fully loaded, which makes scale-down smaller. In both cases, the effect is to hold back when it is not sure.

Several metrics at once

An HPA can look at more than one metric, for example CPU and requests per second. In each cycle it computes a desired replica count for every metric, then chooses the largest answer. This means the HPA will scale up if any signal is high, and will scale down only when all signals agree that fewer pods are enough. This is a safe choice because it favors availability.

Types of metrics the HPA understands

The raw formula reacts instantly to every change. Real systems need brakes, so the HPA has a set of controls that decide how fast it may move.

The problem of flapping

Imagine traffic that bounces up and down every minute. A naive autoscaler would add ten pods, then remove ten pods, then add ten again. Each change costs time and resources: new pods must start, warm up and register with load balancers, and removing pods drops their open connections. This pattern is called flapping or thrashing, and it hurts both stability and cost.

Stabilization windows

The HPA remembers its recent decisions. A stabilization window is a period of time during which the HPA looks back at all the recommendations it made, and picks the safest one. For scaling down, it picks the highest recommendation seen during the window. This means the HPA will not shrink the deployment until the lower number has been consistently recommended for the whole window.

• The default scale-down window is 5 minutes. This is why Kubernetes feels slow to shrink: it wants five calm minutes before removing capacity.
• The default scale-up window is zero. The HPA is allowed to grow immediately, because under-capacity hurts users right now.

Rate limiting policies

Beyond the window, the HPA’s behavior settings let you limit how many pods may be added or removed in a given period. A policy can be written as a fixed number of pods, such as “no more than 4 pods per minute”, or as a percentage, such as “no more than 10 percent per minute”. You can define several policies for each direction, along with a rule called the select policy which tells the HPA whether to follow the policy that allows the biggest change, the smallest change, or to disable scaling in that direction entirely.

By default, scale-up is quite bold. The defaults allow the number of pods to double, or to grow by four pods, whichever is larger, every 15 seconds. Scale-down by default is allowed to remove everything it wants but only after the long stabilization window.

flowchart TB
    R["Raw recommendation from the formula"] --> W["Stabilization window - pick the safest recent value"]
    W --> P["Behavior policies - limit pods or percent per period"]
    P --> M["Clamp between minReplicas and maxReplicas"]
    M --> O["Final desired replica count"]

The minimum and maximum

Two plain numbers act as the outer guard rails. minReplicas is the floor, the fewest pods you will ever run. maxReplicas is the ceiling, the most the autoscaler may ever create. The ceiling is your emergency brake against runaway cost, a bug that makes metrics explode, or an attack that tries to drive your bill up. Never leave it unreasonably high.

The most important autoscaling decision is not a setting. It is the choice of which number to follow. A great algorithm following a bad signal still makes bad decisions.

What makes a good scaling signal?

A good signal has three qualities. It should rise when more capacity is needed and fall when it is not. It should be proportional, so that doubling the load roughly doubles the number. And it should respond to scaling: when you add pods, the number should go down per pod. If adding pods does not change the metric, the autoscaler will keep adding pods forever.

Why CPU is not always enough

Think of a service that mostly waits for a database. Each request spends 95 percent of its time waiting and only 5 percent computing. The pods can be overloaded with waiting requests while their CPU stays low. The HPA sees low CPU and does nothing, while users stare at slow pages. For such services, scaling on concurrent requests or requests per second is much better.

A simple sizing rule from queueing theory

A famous result called Little’s Law says that the average number of requests being handled at once equals the arrival rate multiplied by the average time each request takes. This gives a handy way to estimate capacity. If your service receives 500 requests each second and each takes 0.2 seconds, then on average 100 requests are in flight at any moment. If one pod can comfortably handle 10 at once, you need around 10 pods, plus headroom for bursts and for pods that are starting up.

This reasoning is why scaling on in-flight requests per pod works well. You set the target to the number a pod can handle, and the HPA does the division.

Rules of thumb

• Choose a signal that measures demand or saturation, not one that only reflects cause or symptoms inside the pod.
• Set the target below the true limit, leaving a safety margin such as 60 to 70 percent, so there is room to absorb bursts while new pods start.
• Prefer averages over single readings, but not so heavily averaged that a real spike disappears.
• Test the signal by load testing. Add load and check that the metric rises, then add pods and check that it falls.

The HPA adds more pods. The VPA makes each pod the right size. Together they answer different questions.

The problem it solves

Most teams guess the CPU and memory requests for their pods, and the guesses are often wrong. Requests that are too high waste money, because the scheduler reserves room that is never used. Requests that are too low lead to crowded nodes, throttling and out-of-memory kills. The Vertical Pod Autoscaler watches real usage over time and recommends, or applies, better numbers.

Three components, three jobs

Update modes

• Off. Only produces recommendations that humans can read. This is the safest way to start, and a useful tool for right-sizing.
• Initial. Sets requests only when a pod is first created. Running pods are never disturbed.
• Recreate. Evicts pods when their requests drift too far from the recommendation, so they restart with new values.
• Auto. Applies recommendations using the best method available. In many versions this is the same as Recreate. Newer Kubernetes versions support resizing some pods in place without restarting, and VPA is gradually adopting that capability, so check what your version offers.

flowchart TB
    M["Metrics history"] --> REC["VPA Recommender"]
    REC --> OBJ["Recommendation: lower bound, target, upper bound"]
    OBJ --> UPD["VPA Updater"]
    OBJ --> ADM["VPA Admission Controller"]
    UPD -->|"evict a drifted pod"| POD["Old pod"]
    POD --> NEW["Replacement pod is created"]
    NEW --> ADM
    ADM -->|"rewrites CPU and memory requests"| NEW

What the recommendation looks like

The VPA does not give a single number. It publishes a lower bound, a target and an upper bound, along with an uncapped target that ignores any limits you configured. The target is what it believes the container should request. The bounds express its confidence. Early in a pod’s life, with little data, the bounds are wide. As history accumulates, they narrow.

Good fits and poor fits

• Good fit: workloads that cannot scale out easily, such as a single database primary, a cache with a fixed shard count or a machine learning job whose memory needs vary.
• Good fit: teams that want data-driven right-sizing across hundreds of services, using the Off mode to produce reports.
• Poor fit: services with sudden, short bursts, since the VPA works from history and cannot predict the unexpected.

It is tempting to turn on both autoscalers everywhere. That can work, but only if you understand how they interact.

The conflict

Remember that the HPA calculates CPU utilization as usage divided by the request. Now suppose the VPA changes the request. The percentage changes even though the real workload did not. If the VPA increases the request, utilization falls, and the HPA removes pods. Then each remaining pod is busier, usage rises, the VPA raises requests again, and the cycle can continue. The two controllers are chasing each other because they share the same metric.

Safe combinations

A practical workflow

Pods need machines to live on. When pods multiply, someone must add machines, and later remove the ones nobody needs. That is the Cluster Autoscaler’s job.

The trigger: Pending pods

The Cluster Autoscaler does not watch CPU or memory of nodes. It watches for one specific condition: pods that the scheduler marked as unschedulable because no node has enough room. Those Pending pods are a clear sign that the cluster is out of capacity. This makes its logic wonderfully simple and also explains why it reacts only after pods are already waiting.

How scale-up works

How scale-down works

The autoscaler also looks for nodes that are underused. By default, a node whose requested resources are below about half of its capacity becomes a candidate. If it stays a candidate for around ten minutes, and if all its pods could be moved elsewhere, the autoscaler drains the node, which means safely evicting its pods, and then asks the cloud to delete it.

Many things prevent a node from being removed. Pods that are not managed by a controller, pods with local storage, pods blocked by a disruption budget, pods marked as not safe to evict, and pods with rules that tie them to a particular node all make the node sticky. This is a major reason clusters stay larger than expected.

flowchart TB
    S["Scan every few seconds"] --> Q{"Unschedulable pods?"}
    Q -->|"yes"| SIM["Simulate adding a node in each node group"]
    SIM --> EXP["Expander picks the best group"]
    EXP --> CLOUD["Ask cloud to add nodes"]
    CLOUD --> READY["Node boots and joins"]
    READY --> PLACE["Scheduler places Pending pods"]
    Q -->|"no"| U{"Underused nodes for long enough?"}
    U -->|"yes"| CHK{"Can pods move safely elsewhere?"}
    CHK -->|"yes"| DRAIN["Drain and delete node"]
    CHK -->|"no"| KEEP["Keep node"]
    U -->|"no"| KEEP

Things that surprise people

• It reacts to requests, not usage. A node full of idle pods with large requests looks full and will not be removed. A node with tiny requests but heavy actual load looks empty.
• It needs fixed node shapes per group. Because it works with node groups, a pod that needs a very different machine shape may need a new group created in advance.
• Cloud limits matter. Quotas, lack of capacity in a zone and exhausted IP address ranges can all block scale-up even though the autoscaler is working perfectly.
• Machines are slow. Even a quick cloud machine can take a minute or several minutes to be ready, plus the time to pull container images.

Node-group-based scaling is a bit like ordering from a fixed menu. Karpenter takes a different approach: it looks at what you need and builds exactly that.

The big idea

Karpenter is a node provisioning project that began at Amazon Web Services and now lives under the Kubernetes autoscaling community. Instead of resizing predefined groups, it watches Pending pods directly, looks at their combined requirements, such as CPU, memory, zone, architecture and special hardware, and then asks the cloud for machines of the best fitting type. It can choose from a wide range of instance types, including cheaper spot capacity, without you maintaining dozens of groups.

How it works

• Batching. Karpenter gathers Pending pods for a brief moment so that it can plan machines for many pods together.
• Bin packing. It computes the cheapest set of machines that can hold the pods, respecting rules about zones, hardware and spreading.
• Direct launch. It calls the cloud’s compute service directly to create the machines, which tends to be faster than going through scaling groups.
• Consolidation. It continuously looks for ways to reduce cost by deleting empty or underused nodes, and by replacing several nodes with fewer or cheaper ones, moving pods safely in between.
• Expiry and drift. It can retire nodes after a set age or when their configuration drifts from the desired template, which helps with security patching.

Choosing between them

If your cloud or distribution has a well-supported Karpenter provider and you run varied workloads, Karpenter can reduce both waiting time and cost. If you run on a cloud where it is not available, or you need the predictability of fixed groups, the Cluster Autoscaler remains a solid and proven choice. Managed offerings from cloud providers also bundle their own node auto-provisioning features, which follow similar ideas.

Some workloads should sleep entirely when there is nothing to do. They wake only when work arrives. This is where event-driven autoscaling shines.

Why the plain HPA struggles at zero

The standard HPA works from the metrics of running pods. If there are zero pods, there is no pod to measure, and nothing to wake the system when work arrives. Also, the HPA’s normal minimum is one replica. Support for scaling to zero exists only as an optional feature in some versions, and it needs metrics that come from outside the pods. So tools were built to close this gap.

KEDA in detail

KEDA is a project that started as a joint effort between Microsoft and Red Hat, and graduated within the Cloud Native Computing Foundation. It runs as a small operator in your cluster and adds new resource types. The main one, the ScaledObject, links a workload to a trigger, such as the length of a queue, the lag of a streaming topic, a database query result, or a schedule.

flowchart LR
    SRC["Event source - queue, stream, database"] --> SC["KEDA scaler reads the value"]
    SC --> OP["KEDA operator"]
    OP -->|"0 to 1 and 1 to 0"| DEP["Deployment"]
    SC --> AD["KEDA metrics adapter"]
    AD -->|"external.metrics.k8s.io"| HPA["HPA managed by KEDA"]
    HPA -->|"1 to N"| DEP
    DEP --> PODS["Worker pods consume events"]
    PODS -->|"drain the source"| SRC

Cooldown and cold start

Scaling to zero saves money, but it introduces the cold start problem. When work arrives and no pod is running, someone has to start a pod, pull the image and wait for the application to become ready. The first request or message waits for all that. For queue workers, a short delay is usually fine. For user-facing web requests, it can be unacceptable, so such services often keep a minimum of one or two pods, or use request-buffering layers that hold the request until a pod is ready.

KEDA also has a cooldown setting that decides how long to wait after the last activity before scaling down to zero, preventing sleep and wake cycles from happening too frequently.

Request-driven scale to zero

Projects such as Knative Serving take a different path for web traffic. They put a small proxy in front of the application that counts requests in flight and holds requests while pods start. Knative’s own autoscaler is based on concurrency, meaning how many requests each pod handles at once. This matches Little’s Law from earlier and suits serverless-style services on Kubernetes.

Everyone expects autoscaling to react in seconds. In practice it takes minutes. Adding up each delay shows why, and where you can win time back.

Follow one spike through the system

Ways to win time back

Autoscaling is powerful but never free. A clear view of the trade lets you decide how much to rely on it.

The main tensions

There is no universal best setting. A latency-sensitive payment service and an overnight reporting job want opposite choices on almost every row of this table. The right answer comes from knowing the cost of being late for your particular workload.

Autoscalers are part of your production safety net, so they must be treated as production systems that can themselves fail or be abused.

What happens when the metrics pipeline breaks

Designing for failure

• Set a sensible floor. The minimum replica count should cover your normal load, so that a metrics outage does not leave you under-provisioned.
• Set a sensible ceiling. The maximum replica count limits the damage from a metric gone wrong, and should respect what your databases and downstream services can take.
• Run the pipeline with redundancy. Multiple copies of Metrics Server and the adapter, with proper resource requests of their own.
• Use disruption budgets. They protect your service while nodes are drained for scale-down.
• Spread pods across zones. Topology spread rules ensure one zone failure does not take out the whole service, and that scale-up lands in several zones.
• Protect downstream systems. If 50 new pods all open database connections at once, the database may collapse. Use connection pooling and limit scale-up speed with behavior policies.

Security considerations

When scaling misbehaves, you need to find out which layer is at fault. A methodical approach saves hours.

What to watch

A troubleshooting walk-through

The same few good habits show up in teams that scale well. The same few traps show up in teams that struggle.

Patterns that work

Anti-patterns to avoid

Here is a checklist you can carry into your next design review.

Best practices

• Always set requests. Autoscaling on utilization depends on them, and the scheduler and node autoscalers rely on them too.
• Pick the signal with care. Use CPU only if the service is truly CPU-bound. Otherwise follow concurrency, request rate or queue depth.
• Leave headroom. Target around 60 to 70 percent so the system has time to react before it hits the wall.
• Use readiness and startup probes properly. They keep half-started pods out of both traffic and the scaling math.
• Tune behavior deliberately. Choose stabilization windows and rate limits based on how your traffic behaves, not defaults alone.
• Protect dependencies. Limit how fast pods can be added, use connection pools, and test that your database can handle the maximum.
• Add disruption budgets and zone spreading. They keep the service healthy while nodes are added and removed.
• Keep images small and startup fast. Faster start means smaller headroom is needed.
• Load test the whole chain. Simulate a spike and measure the time from load rise to extra capacity serving traffic. Do this before the real event does it for you.
• Watch cost and capacity. Alert on replica or node counts nearing their ceilings and on unusual spending.
• Version everything. Keep autoscaler settings in code, reviewed like any other change.

Common mistakes

• Believing that autoscaling will rescue a poorly performing application.
• Scaling on memory for programs that never release memory, so scale-down never happens.
• Forgetting that new pods need time, and expecting capacity in seconds.
• Letting sidecars or startup bursts distort CPU measurements.
• Ignoring the node layer, and discovering Pending pods only during an incident.
• Using the same settings for every service, even though their traffic patterns differ.
• Making the stabilization window so short that the system flaps, or so long that cost never falls.
• Running Metrics Server with no redundancy, then losing autoscaling when it restarts.
• Not testing what happens when the metrics pipeline itself is down.

These ideas run in production across the industry. Here are common patterns, along with the projects behind them.

Short answers to questions that come up again and again.