Traffic is never constant, and a fixed number of pods is always wrong some of the time.

Real applications face changing demand. A shop’s website is quiet at night and crowded during a sale. A news site can be overwhelmed within minutes by a breaking story. A payroll system is idle most of the month and heavily used at the end. If you choose a fixed number of pods, you face an uncomfortable choice. Pick enough for the busiest moment, and you pay for idle capacity almost all the time. Pick enough for the average, and users suffer whenever demand rises above it.

Adjusting the number by hand does not work well either. People cannot watch dashboards around the clock, and they react slowly. By the time someone notices that the application is struggling, users have already been affected. Automation solves this by turning a human reaction into a rule that the cluster follows continuously, in seconds instead of minutes.

What automatic scaling offers

Automatic scaling gives you three benefits at once. Users get consistent performance because capacity follows demand. You save money because idle capacity is released. And operations become calmer because nobody needs to adjust replica counts at unsociable hours. The trade-off is that the autoscaler is another moving part, with settings you must understand and tune.

A control loop that changes the number of pod copies based on observed metrics.

The Horizontal Pod Autoscaler, usually shortened to HPA, is a built-in Kubernetes feature. You describe which workload it should manage, what to measure, what target to aim for, and the smallest and largest number of pods allowed. The HPA then runs a control loop that regularly reads the metrics, compares them with the target, and changes the replica count of the workload to bring the measurement back toward the target.

The word horizontal is important. Scaling horizontally means adding or removing copies of the same pod. The alternative, scaling vertically, means making each pod bigger or smaller, by giving it more or less CPU and memory. Both are useful, but they answer different problems. Horizontal scaling is ideal for stateless services that can run many identical copies side by side.

Observe, compare, decide, act, and repeat.

The autoscaler is a controller that runs inside the cluster’s control plane. About every fifteen seconds by default, it goes around a loop. It reads the current metric values for the pods of each workload it manages. It compares those values with the targets you set. It computes how many pods would be needed to meet the target. Then it adjusts the workload’s desired replica count, subject to safety limits, and the normal Kubernetes machinery creates or removes pods.

sequenceDiagram
    participant H as HPA controller
    participant M as Metrics API
    participant T as Target workload
    participant K as Kubernetes controllers
    loop Every sync period
        H->>M: Read current metrics for the pods
        M-->>H: Values
        H->>H: Compare with target and compute desired replicas
        H->>H: Apply limits, tolerance, and stabilisation
        H->>T: Update desired replica count
        T->>K: Create or remove pods
    end
        

Only workloads that expose a scale interface can be managed.

The autoscaler works with any workload that supports a scale interface, which is a standard way to read and change a replica count. The most common targets are Deployments, which manage stateless pods, and StatefulSets, which manage stateful ones. ReplicaSets also work, and some custom resources from add-ons do too. You name the target in the autoscaler’s definition, and it takes care of the rest.

flowchart TD
    HPA[HPA definition] -->|names the target| SR[Scale interface of the target]
    SR --> D[Deployment]
    SR --> S[StatefulSet]
    SR --> R[ReplicaSet]
    SR --> C[Custom resource with scale support]
    D --> P1[Pods created or removed]
    S --> P2[Pods created or removed in order]
        

The autoscaler asks different interfaces for different kinds of numbers.

The autoscaler does not measure anything itself. It reads metrics from standard interfaces of the Kubernetes API, and it uses different interfaces for different kinds of data. The first covers basic resource use and is provided by the Metrics Server. The second and third cover richer data and must be provided by an adapter that you install, connecting the cluster to a monitoring system or another source.

flowchart TD
    HPA[Horizontal Pod Autoscaler] --> A1[Resource metrics interface]
    HPA --> A2[Custom metrics interface]
    HPA --> A3[External metrics interface]
    A1 --> MS[Metrics Server
reads kubelets]
    A2 --> AD1[Metrics adapter
reads monitoring system]
    A3 --> AD2[Event or external adapter
reads queues and services]
    MS --> K[Kubelets on nodes]
    AD1 --> PR[Monitoring system]
    AD2 --> EX[Queues, cloud services]
        

Five kinds of metric, each fitting a different situation.

The autoscaler definition lets you list one or more metrics, and each metric has a type. The type tells the autoscaler where the number comes from and how to interpret it. Choosing the right type is central to good autoscaling, because a metric that does not reflect real load will scale the wrong way at the wrong time.

One proportional rule sits at the heart of every scaling decision.

The autoscaler uses a simple proportional formula. The desired number of replicas equals the current number of replicas multiplied by the ratio of the current metric value to the desired metric value, and the result is rounded up to the next whole number. In words: if the pods are twice as busy as the target, double the number of pods. If they are half as busy, halve it.

flowchart LR
    A[Current replicas] --> F[Multiply by ratio]
    B[Current metric value] --> R[Ratio of current to target]
    C[Target metric value] --> R
    R --> F
    F --> U[Round up to a whole number]
    U --> L[Apply min, max, and rate limits]
    L --> D[Desired replicas]
        

Percent utilisation is measured against what each pod asked for, not against the machine.

When you scale on CPU utilisation, the number the autoscaler uses is a percentage. That percentage is the pod’s actual usage divided by the amount of CPU the pod requested. A pod that requested half a core and is using a quarter of a core is at fifty percent. This has an important consequence: the autoscaler needs pods to declare requests, and the quality of scaling depends on how realistic those requests are.

flowchart TD
    U[Actual CPU use of the pod] --> P[Divide by the pod CPU request]
    R[CPU request declared by the pod] --> P
    P --> V[Utilisation percentage]
    V --> C[Compare with the target percentage]
    C --> D[Decide to scale up, down, or hold]
        

Pods with several containers

For the standard resource metric, the autoscaler adds up the usage and the requests of all containers in the pod. A pod with a small application container and a large sidecar, such as a logging or proxy container, can therefore show misleading utilisation. Where this matters, the container resource type lets you target a single container, so scaling follows the container that actually carries the load.

Three boundaries stop the autoscaler from overreacting or running away.

flowchart TD
    A[Computed desired replicas] --> B{Ratio inside the tolerance band?}
    B -->|Yes| C[Keep current replicas]
    B -->|No| D{Below the minimum?}
    D -->|Yes| E[Use the minimum]
    D -->|No| F{Above the maximum?}
    F -->|Yes| G[Use the maximum]
    F -->|No| H[Use the computed value]
        

Choosing the minimum

The minimum should cover your baseline load and your availability needs. A service that must survive the loss of a node or a zone needs at least two or three replicas, spread across failure domains, regardless of how quiet it is. Setting the minimum to one saves money but leaves a single point of failure and a slow response to sudden traffic.

Choosing the maximum

The maximum should reflect the largest load you can serve and afford. It must also fit within your cluster’s capacity and your namespace quotas. If the maximum is higher than the cluster can actually run, extra pods will sit pending, and the autoscaler will believe it has scaled while users still suffer.

When you list more than one metric, the largest answer wins.

An autoscaler can be given several metrics at the same time, for example CPU utilisation and requests per second. For every metric, it independently computes how many replicas would be needed. It then takes the largest of those numbers. The reasoning is straightforward: if any one measure says the application is under pressure, the application needs more capacity, and it should not be starved just because another measure looks comfortable.

flowchart TD
    M1[Metric 1: CPU utilisation] --> C1[Replicas needed: 6]
    M2[Metric 2: requests per pod] --> C2[Replicas needed: 9]
    M3[Metric 3: queue length] --> C3[Replicas needed: 7]
    C1 --> MAX[Take the maximum]
    C2 --> MAX
    C3 --> MAX
    MAX --> R[Desired replicas: 9]
        

Controls that decide how fast the autoscaler may act and how long it waits before shrinking.

A purely proportional rule would react to every wobble in the metrics. Real load is noisy, and reacting to noise causes flapping, where the number of pods rises and falls repeatedly. The behaviour settings let you tame this. They are configured separately for scaling up and for scaling down, since the two directions have different risks. Scaling up too slowly hurts users. Scaling down too quickly causes a shortage moments later.

sequenceDiagram
    participant H as HPA
    participant W as Recommendation history
    Note over H: Load falls, HPA computes fewer pods
    H->>W: Record the recommendation
    Note over H,W: Stabilisation window looks at recent recommendations
    H->>H: Use the highest recommendation in the window when scaling down
    Note over H: Scale down only if all recent recommendations agree
    H->>H: Apply rate limit from policies
    H->>H: Change the replica count in small steps
        

New and unhealthy pods can distort the numbers, so the autoscaler treats them carefully.

When a pod has just started, its early CPU use is often unusually high, because it is loading code and warming caches, and unrepresentative of steady work. When a pod is not ready, it is not receiving traffic, so its usage says little about load. And sometimes metrics for a pod are simply missing. If the autoscaler took all of these numbers at face value, it could make poor decisions, for example adding pods because new pods look busy while starting.

sequenceDiagram
    participant H as HPA
    participant P as New pod
    participant M as Metrics
    H->>P: New pod created because of scale-up
    P->>P: Starting, loading, not ready
    H->>M: Read metrics
    M-->>H: Pod shows high start-up CPU
    Note over H: Ignore early readings and treat the pod as not yet contributing
    P->>P: Becomes ready and settles
    H->>M: Read metrics again
    M-->>H: Steady values
    Note over H: Pod now counted normally
        

Going beyond CPU requires a bridge between your monitoring data and the cluster.

CPU and memory are only rough indicators of load. Many services are better measured by how many requests they handle, how many connections are open, or how many jobs wait in a queue. To use such metrics, the cluster needs a component that reads them from wherever they live and presents them through the custom or external metrics interface. This component is called an adapter.

flowchart LR
    APP[Application exposes metrics] --> MON[Monitoring system collects them]
    MON --> AD[Metrics adapter]
    AD --> API[Custom or external metrics interface]
    API --> HPA[Horizontal Pod Autoscaler]
    HPA --> DEP[Deployment replica count]
    DEP --> APP
        

The autoscaler tells you what it thinks and why, if you know where to look.

Every autoscaler reports its own state. A quick listing shows the current metric value against its target, the minimum, maximum, and current replica counts. A detailed view adds conditions and recent events that explain what the autoscaler did and why it did not do something. Learning to read these is the fastest route to understanding any autoscaling behaviour.

Two owners of the replica count cause a fight.

A Deployment normally has its own replica count, written in its definition. When an autoscaler manages the Deployment, it takes over that number. If your deployment manifests still contain a fixed replica count and you apply them regularly, each application of the manifest resets the count to the fixed value, undoing the autoscaler’s work until it corrects again. The result is a sudden drop in pods every time you deploy, a common and painful surprise.

sequenceDiagram
    participant H as HPA
    participant D as Deployment
    participant P as Pipeline applying manifests
    H->>D: Set replicas to 12 for current load
    P->>D: Apply manifest containing replicas 3
    D->>D: Scale down to 3 pods immediately
    Note over D: Users suffer a sudden capacity drop
    H->>D: Sets replicas back to 12 after next check
        

Rolling updates while autoscaling

Autoscaling and rolling updates can happen at the same time. During a rollout, the Deployment holds two ReplicaSets, and the autoscaler’s change in replicas is distributed between them in proportion. This works well in practice. But it does mean a rollout during peak traffic is riskier, because capacity is being both replaced and adjusted, so avoid deploying at moments of extreme load when you can.

Several scaling mechanisms coexist, and each covers a different level.

The Horizontal Pod Autoscaler changes the number of pods. But pods need nodes to run on. If the cluster has no free room, the new pods stay pending, and the scale-up achieves nothing. A second mechanism, the cluster autoscaler, adds nodes when pods cannot be scheduled and removes idle nodes. The two work as a chain: the pod autoscaler asks for more pods, and the cluster autoscaler provides the machines for them.

sequenceDiagram
    participant L as Load
    participant H as HPA
    participant S as Scheduler
    participant C as Cluster autoscaler
    participant N as New node
    L->>H: Metrics rise
    H->>S: Desired pods increased
    S->>S: No room, some pods pending
    S->>C: Pending pods cannot be scheduled
    C->>N: Add a node
    N-->>S: Node ready
    S->>N: Schedule pending pods
    Note over L,N: Capacity is finally available
        

The best metric is the one that rises before users suffer and falls when capacity is no longer needed.

Picking a metric is the most important design choice for an autoscaler. A good metric reflects real load on the application, moves quickly enough to warn you in time, and falls again when the load falls. A bad metric can be flat while users suffer, or stay high after load has gone, making the autoscaler either useless or wasteful.

flowchart TD
    Q{What limits this service?} -->|Processor time| CPU[CPU utilisation]
    Q -->|Number of requests| RPS[Requests per second per pod]
    Q -->|Backlog of work| QL[Queue length per worker]
    Q -->|Open connections| CON[Active connections per pod]
    Q -->|Memory that is released after load| MEM[Memory utilisation with care]
    CPU --> CHK[Check the metric rises before users suffer]
    RPS --> CHK
    QL --> CHK
    CON --> CHK
    MEM --> CHK
        

Autoscaling is never instant, so targets must leave room for the wait.

Between a rise in load and the arrival of new capacity, there are several delays. The metrics are collected on an interval. The autoscaler checks on its own interval. New pods must be scheduled, their images pulled, their containers started, and their readiness checks passed. If nodes are needed, that adds minutes. During all this time, the existing pods carry the extra load. The target you choose determines how much spare capacity exists to absorb it.

The autoscaler itself is light, but its dependencies and side effects are not.

The HPA controller does modest work: it reads metrics and updates a number. Even hundreds of autoscalers add little load to the control plane. The real costs lie in what it depends on and what it triggers. The metrics source must answer quickly for every autoscaler every fifteen seconds. Each scale-up creates pods, which places work on the scheduler, the nodes, the image registry, and any shared services that the new pods connect to.

Whoever controls the autoscaler’s inputs and limits controls your capacity and your bill.

Autoscaling looks like a purely technical feature, but it has security and cost implications. The autoscaler decides how much of the cluster a workload may use. Anyone who can change the autoscaler’s maximum can raise your costs and crowd out other workloads. Anyone who can influence the metrics it reads can trigger scaling, deliberately or not. And the metrics adapters themselves hold credentials to your monitoring systems.

Most autoscaling problems have plain, visible causes.

Reliable arrangements used in production clusters.

flowchart TD
    A[Measure real usage] --> B[Set realistic requests]
    B --> C[Choose metrics that reflect load]
    C --> D[Set minimum, maximum, and behaviour]
    D --> E[Load test the whole chain]
    E --> F[Monitor status and user impact]
    F --> G[Review and adjust regularly]
    G --> A
        

Habits that make autoscaling ineffective or dangerous.

A short checklist for dependable autoscaling.

How teams use the autoscaler in daily operations.

It automatically changes the number of pods of a workload, up or down, by comparing observed metrics with a target and applying a proportional formula, within a minimum and maximum.

A working Metrics Server, realistic CPU requests on the pods, and an autoscaler definition naming the target workload, the target utilisation, and the minimum and maximum replicas.

Multiply the current replica count by the ratio of the current metric to the target metric, and round up. If the ratio is within the tolerance band, nothing changes.

Usually because the pods have no resource requests, or the metrics source is unavailable. Check requests, the Metrics Server or adapter, and the condition messages on the autoscaler.

Not by default. The standard autoscaler keeps at least one replica. Event-driven add-ons, or an experimental feature, can scale to zero for suitable workloads.

A stabilisation window, by default five minutes, makes it use the highest recent recommendation. This prevents flapping and shortages when load returns quickly.

Yes. It computes a desired count for each metric and uses the largest one, so scale-down happens only when all metrics agree that fewer pods are enough.

No. A fixed count in the manifest resets capacity on every apply. Leave it out and let the autoscaler own the replica count.

Perhaps the pods are pending for lack of capacity, are slow to become ready, or the true bottleneck is elsewhere, such as a database. Check pod status, readiness, and shared dependencies.

Not on the same CPU or memory metric for the same workload, since they interfere. Use the vertical one for recommendations, or apply them to different signals.

The one that rises before users suffer and falls when capacity is no longer needed. CPU is a good start for compute-bound services, and request rate or queue length suits many others.

It is never instant. Metric collection, autoscaler checks, pod scheduling, start-up, and readiness all add delay, so leave headroom in your target and consider warming capacity before known peaks.

The Horizontal Pod Autoscaler is the Kubernetes control loop that matches the number of pods to the load on an application. On a regular cycle, it reads metrics, from the Metrics Server for CPU and memory or from adapters for custom and external signals, compares them with your targets, and applies a proportional formula to compute the desired number of replicas. Tolerance, minimum and maximum limits, stabilisation windows, and scaling policies keep it from overreacting.

Good autoscaling depends on the surroundings as much as on the autoscaler. Pods need realistic resource requests, fast start-up, and accurate readiness checks. The metric must reflect what truly limits the service. The cluster needs room for new pods, or a cluster autoscaler to provide it, and shared dependencies must survive the growth. Manifests must not fight the autoscaler over the replica count. And the whole chain deserves monitoring, alerts, and load tests. Built that way, autoscaling gives you consistent performance and lower costs without anyone watching the dashboards at night.