Before learning the tool, we need to understand the problem it solves and why the default answer is often wrong.

What is autoscaling?

Autoscaling means that a system changes its own size automatically to match the work it has to do. When there is a lot of work, it adds more workers. When the work disappears, it removes the extra workers so you stop paying for them. In Kubernetes, the “workers” are usually pods, and a pod is the smallest unit you can run: one or more containers that live and die together.

Autoscaling matters for two reasons. The first is reliability: if traffic doubles and you do nothing, your service slows down or crashes. The second is cost: if you always run enough pods for your busiest day, you waste money on the other 364 days. Good autoscaling gives you both safety and savings.

The default: scaling on CPU

The original Kubernetes autoscaler, the Horizontal Pod Autoscaler (HPA), could only look at CPU usage in its earliest version. If your pods averaged more than, say, 70 percent of the CPU they asked for, the HPA added pods. If usage fell, it removed some. For many simple web applications that do a lot of calculation per request, this is a reasonable rule.

The word horizontal matters. Scaling horizontally means adding more copies of the same thing. Scaling vertically means making one copy bigger, for example giving a pod more CPU. Kubernetes can do both, but horizontal scaling is the more common tool because it also improves availability: more copies means one failure hurts less.

Where CPU lies to you

CPU is a proxy, meaning a stand-in for the thing you really care about. What you really care about is whether users are getting fast, correct answers. CPU only tells you about that when the work is CPU-bound. Many real systems are not.

What custom metrics change

A metric is simply a number that is measured over time, like a thermometer reading. A custom metric is any metric that is not CPU or memory, usually one your own application publishes. Examples include “requests handled per second”, “jobs waiting in the queue”, “active user sessions” or “frames waiting to be rendered”.

When Kubernetes can scale on those numbers, your scaling rule becomes close to a sentence a human would say: “Keep about 100 requests per second on each pod” or “Never let more than 50 jobs wait for each worker”. That rule is easier to reason about, easier to test, and far closer to what users experience.

Kubernetes has several autoscalers that solve different problems. Knowing which is which prevents a lot of confusion later.

People often say “Kubernetes autoscaling” as if it were one thing. In reality there are at least four tools, and a healthy production setup usually uses more than one of them at the same time. Each one answers a different question.

How the layers fit together

The easiest way to remember the relationship is to think of a restaurant again. The HPA decides how many waiters are on the floor. The node autoscaler decides how big the building is. Waiters need a place to stand, so when the HPA wants more waiters than the building can hold, the building must grow.

In practice this is exactly what happens. The HPA creates new pods. If no machine has room for them, they sit in a Pending state, which is Kubernetes saying “I have no place to put this yet”. The Cluster Autoscaler, or a newer tool called Karpenter, sees the pending pods and starts new machines. Once the machines are ready, the pods land on them and begin serving traffic. Custom metrics only influence the first step of this chain, but the whole chain determines how quickly you really get capacity.

A short history

The HPA first appeared in the early days of Kubernetes and could only use CPU. Later versions added memory and then the ability to read custom metrics, first in beta versions of the API and eventually in the stable autoscaling/v2 API that reached general availability in Kubernetes 1.23. Around version 1.18, a behavior section was added so engineers could control how fast scaling happens in each direction, which solved many “flapping” problems we will discuss later.

In 2019, Microsoft and Red Hat created KEDA (Kubernetes Event-driven Autoscaling) to make it easy to scale on external event sources. It joined the Cloud Native Computing Foundation (CNCF) and reached the top “graduated” maturity level in 2023, which signals that it is widely used and well governed.

At its heart the HPA is a small piece of arithmetic repeated forever. Once you understand the arithmetic, every behavior makes sense.

The control loop

The HPA is not a separate program you install. It is a controller that lives inside the Kubernetes control plane, in a component called the controller manager. A controller is a loop that repeatedly compares the world as it is with the world as you asked for, and then takes action to close the gap. The HPA loop wakes up about every 15 seconds. Each time, it reads the current metric values, does the arithmetic, and decides whether to change the number of replicas.

You describe what you want in an object called a HorizontalPodAutoscaler. It contains four main ideas: which workload to scale (the target), the smallest and largest number of replicas allowed, which metrics to watch, and what value of each metric counts as “just right”.

The core formula

The formula is short enough to remember:

desired replicas = ceiling of (current replicas × current metric value ÷ target metric value)

The “ceiling” means round up to the next whole number, because you cannot run 5.3 pods. Let us walk through two examples with plain numbers.

Tolerance: ignoring tiny differences

If the HPA reacted to every tiny wiggle, the replica count would jitter constantly. To prevent that, there is a tolerance, which is 10 percent by default. If the ratio of current to target is within 10 percent of 1.0, the HPA does nothing. A reading of 105 against a target of 100 gives a ratio of 1.05, which is inside the tolerance, so nothing changes.

More than one metric

An HPA can watch several metrics at once, such as CPU and requests per second. It performs the calculation separately for each metric and then picks the largest answer. This is a safety-first rule: whichever signal says “I need the most help” wins. The result is then clamped between your configured minimum and maximum replica counts.

Missing data is treated carefully

Sometimes a pod has no metric reading, perhaps because it just started or its metrics endpoint failed. The HPA does not guess optimistically. When deciding to scale up, it pretends the missing pods are using none of the target, which avoids overreacting. When deciding to scale down, it pretends the missing pods are using exactly the full target, which avoids removing capacity based on incomplete information. Pods that are still starting up are also treated specially for a short period so that their startup spike does not fool the autoscaler.

The HPA does not talk to your database of metrics directly. It talks to the Kubernetes API, and special add-ons fill that API with data.

Five kinds of metric in an HPA

When you write an HPA you choose, for each metric, one of five types. Each type tells the HPA where the number comes from and how to interpret it.

Three metric APIs

Kubernetes has a feature called the aggregation layer. It lets add-on programs plug extra endpoints into the main Kubernetes API, as though those endpoints were built in. The HPA uses this to ask three different questions through three different API groups.

Here is a detail that surprises many newcomers. Kubernetes allows only one provider per API group at a time. That means you cannot run two different programs that both serve custom.metrics.k8s.io. You can, however, run one provider for the custom API and a different provider for the external API, which is common: a Prometheus-based adapter for custom metrics next to KEDA for external ones.

Understanding the units

Metric values are written as Kubernetes quantities. A small letter “m” means “milli”, or one thousandth. So a value of 500m means 0.5, and 100 means one hundred. You will see this in the HPA status when a rate such as 0.3 requests per second appears as 300m. Beginners often stare at it, thinking something is broken, when it is only a unit.

Value versus average value

When you set a target you can often choose between a total value and an average value per pod. Use the average per pod for anything that grows as you add pods, such as requests per pod. Use a total value only when the number describes a single shared thing, for example one Ingress. Mixing these up is a classic mistake: if you target a total queue length of 100 with a value-type target, the HPA compares the whole number of jobs, not the per-worker share, and your scaling math will not match your intention.

A custom metrics pipeline has several moving parts. Seeing them on one map makes the rest of this guide easier to follow.

Let us follow a number from the moment it is born inside your application until it causes a new pod to appear. The diagram below shows a very common setup based on Prometheus, which is a popular open source system that collects and stores metrics.

flowchart LR
    U["Users / Clients"] --> LB["Load Balancer / Ingress"]
    LB --> APP["Application Pods"]
    APP -->|"expose /metrics"| PROM["Prometheus"]
    PROM -->|"PromQL query"| ADP["Metrics Adapter"]
    ADP -->|"registered APIService"| AGG["Kubernetes API Server + Aggregation Layer"]
    HPA["HPA Controller in Controller Manager"] -->|"asks for custom metric"| AGG
    HPA -->|"updates replicas"| DEP["Deployment"]
    DEP --> APP
    APP -.->|"pending pods"| CA["Cluster Autoscaler / Karpenter"]
    CA -.->|"adds nodes"| NODES["Worker Nodes"]
    

Walking through the diagram

Start on the left. Users send traffic through a load balancer or an Ingress (the Kubernetes front door) to your application pods. Each pod keeps counters of what it is doing, such as the total number of requests handled, and publishes them on a web page, conventionally at the path /metrics.

Prometheus visits each pod on a schedule, called the scrape interval, copies the numbers, and stores them with timestamps. Prometheus alone, however, is not part of Kubernetes. The HPA has no idea how to talk to it. That gap is filled by the metrics adapter, a small server that translates between two languages. It asks Prometheus questions in Prometheus’s query language and answers the HPA in the format of the Kubernetes custom metrics API.

The adapter registers itself with the Kubernetes API server using an object called an APIService. From then on, when anyone asks the main API server for a custom metric, the server forwards that question to the adapter. The HPA controller is one such asker. It receives the numbers, applies the formula from the previous chapter, and writes a new replica count to the Deployment.

The cast of components

What the HPA actually scales

The HPA does not edit pods directly. It writes to something called the scale subresource of its target. Any Kubernetes object that exposes this small standard interface can be scaled, which includes Deployments, StatefulSets and ReplicaSets, and also custom resources defined by operators. This design is why the HPA is so general: it only needs to know “how many replicas are desired”, and it leaves the details of creating them to the owner of the target.

A scaling event is not instant. It is a relay race with several handoffs, and each handoff adds delay.

Suppose a marketing email goes out and your traffic doubles within a minute. Let us trace what happens, step by step, and see where time is spent. This is called the data flow and lifecycle of an autoscaling event.

sequenceDiagram
    participant App as Application Pods
    participant Prom as Prometheus
    participant Adp as Metrics Adapter
    participant API as API Server
    participant HPA as HPA Controller
    participant Dep as Deployment
    participant Sch as Scheduler and Kubelet
    App->>Prom: Metrics scraped every 15 to 30 seconds
    HPA->>API: Every 15 seconds, ask for metric
    API->>Adp: Forward the request
    Adp->>Prom: Run the configured query
    Prom-->>Adp: Current rate per pod
    Adp-->>API: Metric values
    API-->>HPA: Metric values
    HPA->>HPA: Apply formula and limits
    HPA->>Dep: Set new replica count
    Dep->>Sch: Create new pods
    Sch->>App: Start containers and pass readiness checks
    

Adding up the delay

If you add the pieces, a typical well-tuned system needs about one to two minutes to react end to end, and longer if new machines are required. That is acceptable for gradual traffic growth, but it can be too slow for a sudden spike. Understanding this chain tells you where to invest: shorter scrape intervals and lighter images reduce delay, while a small buffer of spare capacity covers the part you cannot shorten.

The most common way to feed custom metrics to Kubernetes is the Prometheus Adapter. Here is how it thinks.

What it is and why it exists

The Prometheus Adapter is a community project maintained under the Kubernetes special interest groups. Its job is simple to say: take numbers stored in Prometheus and serve them through the Kubernetes custom metrics API and, if you wish, the external metrics API. Many teams already run Prometheus for dashboards and alerts, so reusing it for autoscaling means no new data collection system is needed.

The rulebook: four ideas

The adapter is configured with a list of rules. Every rule answers four questions, and understanding them removes most of the mystery.

Counters, rates and why they matter

Most application metrics are counters, numbers that only ever go up, such as “total requests served since this pod started”. A counter by itself is useless for autoscaling: a pod that has been up for a month has a huge number even if it is idle right now. What you need is the rate, meaning how fast the counter is growing per second. Prometheus can compute this over a recent window, for example the last two minutes. The adapter rule wraps the counter in that calculation, so the HPA sees “requests per second” rather than a meaningless total.

The window length is a trade-off. A short window reacts quickly but is noisy. A long window is smooth but slow to notice a real change. A window of around two to five times your scrape interval is a common starting point.

Checking that it works

A very helpful habit is to query the custom metrics API directly and confirm your metric appears for the expected pods before you create the HPA. If the metric is missing there, the HPA will never find it either. Most “my HPA shows unknown” problems are solved by this one check, because they come from a rule that selects nothing or a label that does not line up with Kubernetes object names.

KEDA takes a different path. Instead of translating one metrics store, it speaks directly to dozens of event sources.

The problem KEDA solves

Suppose your worker reads messages from Amazon SQS, Apache Kafka, RabbitMQ, or an Azure Service Bus queue. To scale it with the Prometheus Adapter you would first have to get those queue lengths into Prometheus through an exporter, then write adapter rules. KEDA removes those steps. It ships with a large library of scalers, each of which knows how to ask one kind of system “how much work is waiting?” Cloud queues, streaming platforms, databases, Prometheus itself, cron schedules and many more are covered.

How KEDA is built

flowchart TB
    SRC["Event Sources: queues, streams, databases, Prometheus, cron"] --> SCL["KEDA Scalers"]
    SO["ScaledObject - your scaling rules"] --> OP["KEDA Operator"]
    OP -->|"creates and manages"| HPA2["HPA"]
    OP -->|"activates 0 to 1 and deactivates 1 to 0"| TGT["Target Deployment"]
    SCL --> MS["KEDA Metrics Server - external.metrics.k8s.io"]
    MS --> HPA2
    HPA2 -->|"scales 1 to N"| TGT
    TA["TriggerAuthentication / secrets / identity"] --> SCL
    

KEDA has a few parts. You describe your wishes in a custom object called a ScaledObject, which names the target workload, the minimum and maximum replicas, and one or more triggers (for example, “scale on the length of this queue”). The KEDA operator watches ScaledObjects and creates a regular HPA behind the scenes. A metrics server inside KEDA serves the numbers through the external metrics API, so the HPA reads them the usual way.

This is an important point for beginners: KEDA does not replace the HPA. It generates and feeds it. Everything you learned about the formula, tolerance and behavior settings still applies. KEDA’s additional skill is that it can also take a workload all the way down to zero replicas and bring it back, which the plain HPA does not do by default.

Scale to zero

Scaling to zero means running no pods at all while there is no work. This saves money for rarely used workloads, such as a nightly report builder or a development environment. The standard HPA can only do this with an alpha, off-by-default feature gate, so most teams do not rely on it. KEDA handles the 0-to-1 and 1-to-0 steps itself, and hands over to the HPA for 1-to-N scaling. Two settings control the rhythm: a polling interval (how often KEDA checks the source, 30 seconds by default) and a cooldown period (how long to wait after the last activity before going to zero, 300 seconds by default).

Jobs instead of long-running workers

For batch work, KEDA offers a ScaledJob. Instead of scaling a set of long-lived workers, it launches a separate Kubernetes Job for each batch of messages, and each Job finishes and disappears. This is useful when every task is long or heavy and you do not want one worker to be interrupted halfway by a scale-down.

Authentication

Event sources usually need credentials. KEDA keeps these out of your scaling rules by using a separate object, called a TriggerAuthentication, that points to Kubernetes secrets or to cloud identity systems. Using cloud workload identity is much safer than storing long-lived keys, because there is nothing secret to leak.

Where it is available

KEDA is open source and installs into any cluster. Several platforms also offer it as a managed feature. Azure Kubernetes Service has a KEDA add-on, and Red Hat’s OpenShift ships a Custom Metrics Autoscaler Operator that is based on KEDA.

The tooling is the easy half. The hard half is deciding which number deserves to control your fleet.

A good scaling metric has three qualities. It must be causal: more load must raise it. It must be responsive: adding pods must lower it. And it must be timely: it should move before users feel pain, not after. Let us look at the common candidates through that lens.

Find the capacity number first

To pick a target such as “100 requests per second per pod”, you need to know what one pod can really do. The honest way is to run a load test. Increase traffic on a single pod until latency starts to climb, note the throughput just before that point, and then set your target at about 60 to 70 percent of it. That headroom absorbs the delay of scaling. Guessing here is the origin of many outages and many wasted dollars.

Beware of metrics that do not respond

Consider latency caused by a slow database. Your pods are healthy, but every request waits on a shared database that is saturated. If you scale on latency, the HPA adds more pods. More pods send more queries, the database gets worse, latency rises, and the HPA adds even more pods. This is a runaway feedback loop, and it can turn a small problem into a full outage. The cure is to choose a signal that reflects your own capacity, and to set a sensible maximum replica count as a safety fence.

Combining signals

Because the HPA takes the largest result across metrics, you can combine a primary signal with a safety net. For example, scale mainly on requests per second, but also on CPU, so that a surprising change in per-request cost still triggers growth. Keep the list short, as every metric you add is another thing that can go missing and confuse the controller.

Reacting fast and staying calm pull in opposite directions. The behavior settings let you choose where to stand.

The flapping problem

Imagine the load wobbles around the target. The HPA adds pods, the load per pod drops below target, the HPA removes pods, the load per pod rises, and the cycle repeats. This constant yo-yo is called flapping. It wastes resources, restarts connections and can cause errors every time a pod terminates while handling requests.

Stabilization windows

The stabilization window tells the HPA to look back at the recommendations from the last N seconds and pick the safest one. For scale-down, the default window is 300 seconds (five minutes): the HPA will only shrink to the highest replica count it recommended over the past five minutes. For scale-up, the default window is zero, meaning it reacts immediately. This asymmetry is deliberate: it is safer to add capacity quickly and release it slowly.

Scaling policies

Policies put a speed limit on each step. You can say “add at most 4 pods every 60 seconds” or “remove at most 10 percent of pods every 2 minutes”. When several policies apply, a select policy decides which wins: the one allowing the largest change, the smallest change, or none at all (which disables scaling in that direction). By default, scale-up may double the pods or add 4 pods every 15 seconds, whichever is more permissive, and scale-down may remove everything the formula suggests, once the stabilization window agrees.

Graceful shutdown

Scaling down is only safe if pods leave politely. When Kubernetes removes a pod, it sends a termination signal and waits a grace period before forcing it off. A well-behaved application stops accepting new work, finishes what is in progress, and exits. A readiness change plus a short pre-stop pause lets the load balancer stop sending traffic before the pod closes its doors. Without this, every scale-down creates a handful of failed requests, and users notice.

Make new pods ready quickly

The faster a new pod is ready, the less headroom you need. Small container images, warm caches loaded lazily, and a readiness probe that truly reflects readiness all shorten the delay. A startup probe helps slow-starting applications avoid being killed while they initialize.

Autoscaling gives numbers the power to spend money and move servers. That power needs protecting.

Why security matters here

If an attacker or a bug can change a metric, they can change your infrastructure. A forged queue length could cause the cluster to scale up to its maximum, running up a cloud bill. A forged low value could cause the cluster to scale down during peak traffic, causing an outage. So the metric path deserves the same care as any other production dependency.

You watch your application. You must also watch the thing that resizes it.

What the HPA tells you about itself

Every HPA records its own status. It shows the current and desired replica counts, the latest metric values it observed, and a list of conditions explaining its state. Three conditions are especially useful:

The HPA also emits events with clear reasons, such as failures to get a pods metric or an external metric. Describing the HPA with the standard Kubernetes command line tool shows these events, which makes it the first stop for any investigation.

Metrics about the autoscaler

The cluster add-on called kube-state-metrics turns the HPA’s status into Prometheus metrics, such as the maximum replicas, current replicas and desired replicas of each autoscaler. Graph those alongside your traffic and you can literally see the control loop at work: load goes up, desired replicas follows, current replicas catches up a minute later.

flowchart LR
    APP2["Application Pods"] --> PM["Prometheus"]
    KSM["kube-state-metrics"] --> PM
    ADP2["Metrics Adapter"] --> PM
    PM --> GR["Dashboards"]
    PM --> AL["Alert Manager"]
    AL --> ONC["On-call Engineer"]
    

Alerts worth having

• Pinned at maximum: an HPA has been at its maximum replica count for more than fifteen minutes.
• Metrics unavailable: the ScalingActive condition has been false for several minutes.
• Adapter health: the custom or external metrics API returns errors or high latency.
• Pending pods: pods remain Pending for longer than a few minutes, which means node capacity is the bottleneck.
• Flapping: the replica count changes direction many times in an hour.

Logging

Keep the logs of the adapter or KEDA components in your central logging system. When a metric disappears at 3 a.m., these logs show whether a query failed, a credential expired or a source timed out. Also log deployments and configuration changes to HPAs so you can connect a change in behavior to the change in rules that caused it.

Teams that run autoscaling for years end up using the same few patterns, and avoiding the same few traps.

Patterns that work

Anti-patterns to avoid

When autoscaling misbehaves, the cause is almost always one of a short list. Work through it in order.

The classic symptom: unknown

The most frequent complaint is an HPA that shows unknown instead of a number next to its target. This simply means the HPA could not get a value. Use the list below to walk from the outside in.

Mistakes that look like bugs

Here is how well-known organizations and platforms approach custom metrics scaling, plus a few realistic scenarios.

Realistic scenarios

The following are illustrations of how the ideas combine. They are teaching examples and do not describe the internal systems of the companies they mention.

You do not need advanced mathematics to size an autoscaled service. A few small formulas will carry you a long way.

Little’s Law in plain words

There is a beautiful rule from queueing theory called Little’s Law. It says that the average number of things inside a system equals the rate at which they arrive multiplied by the average time each one spends inside. In a web service, the “things” are requests that are currently being handled, the arrival rate is requests per second, and the time inside is the response time.

For example, if 2,400 requests arrive every second and each takes an average of 0.1 seconds to finish, then about 240 requests are being processed at any moment. This number is called concurrency. It is one of the most useful numbers in capacity planning, because it tells you how many requests your pods must juggle at the same time.

From concurrency to pod count

Once you know how many requests a single pod can handle at the same time while staying fast, the pod count is just a division. If one pod can comfortably hold 5 requests in flight, then 240 concurrent requests need 48 pods. Add a margin for safety and you have your target. This is the same answer you would get from the HPA formula, but now you understand where the number comes from instead of treating it as magic.

Three numbers every service owner should know

With those three numbers, the minimum and maximum replica counts almost choose themselves. The maximum is peak demand divided by the target per pod, plus a margin, but never more than your database and other dependencies can safely serve. The minimum is the larger of two things: the number of pods you need for availability (usually two or three across zones) and the number needed to handle the quietest time plus a sudden jump.

The reaction-time budget

Another useful calculation is how much extra load can arrive while you wait for new pods. If load can grow by 30 percent per minute and it takes two minutes to get new pods ready, you must carry about 60 percent spare capacity or accept slower service for a short time. If that is too expensive, work on reducing the delay or smoothing the load with queues, rather than keeping huge idle capacity all day.

Let us put everything together in one realistic design, step by step, using plain numbers.

Imagine an online store with an order service. It receives order requests from the website and the mobile app, saves them, and puts a message on a queue for a separate fulfillment worker. We will design the autoscaling for both parts.

flowchart LR
    CL["Web and Mobile Clients"] --> GW["API Gateway / Load Balancer"]
    GW --> ORD["Order Service Pods"]
    ORD --> DB["Orders Database"]
    ORD --> Q["Message Queue"]
    Q --> WRK["Fulfillment Worker Pods"]
    WRK --> SHIP["Shipping and Payment Partners"]
    PRM["Prometheus"] -->|"requests per pod"| ORD
    PRM -.-> HPA3["HPA with Prometheus Adapter"]
    KD["KEDA"] -->|"queue length per worker"| WRK
    

Walking through a spike

At 12:00 a promotion goes live and traffic jumps from 300 to 1,200 requests per second in three minutes. The six running pods suddenly see 200 requests per second each, four times the target. The HPA computes 6 times 4, which is 24 pods, and begins adding them within the allowed step sizes. Over the next two minutes the pod count climbs through 9, 14 and 21 to 24. Meanwhile a queue of orders builds slowly behind the API. KEDA sees the queue grow past 10 per worker and brings the workers up. After the promotion ends, the system waits ten minutes, then gradually removes pods so that no sudden drop in capacity occurs.

Autoscaling should make your service more reliable, not less. A few extra settings make sure of that.

Spread pods across zones

A zone is a separate data center area inside a cloud region. If every pod lands in the same zone and that zone has trouble, your whole service goes down. Kubernetes lets you declare topology spread constraints, a rule that says “keep the pods roughly evenly distributed across zones”. When the HPA adds pods, the scheduler follows this rule, so the extra capacity also improves resilience.

Protect pods during maintenance

A PodDisruptionBudget tells Kubernetes the minimum number of pods that must stay available when machines are being drained for maintenance or removed by the node autoscaler. Without one, a scale-down of nodes could remove several pods of the same service at once. With one, Kubernetes removes them in a safe order. Make sure the budget is compatible with your minimum replicas, or draining nodes can get stuck.

Health checks that tell the truth

Kubernetes uses three kinds of probes. A liveness probe asks “is this pod stuck and in need of a restart?”. A readiness probe asks “can this pod serve traffic right now?”. A startup probe gives slow starters time to initialize. Autoscaling depends heavily on readiness: until a new pod is ready, it takes no traffic, so a readiness probe that passes too early makes new pods fail users, and one that passes too late wastes the scale-up.

Failure scenarios and recovery

Autoscaling during a rollout

When you release a new version, a Deployment replaces old pods with new ones gradually. The HPA keeps working during this time, but metrics from new pods may look different for a while as caches warm up. Use readiness probes and a small surge allowance so that capacity does not dip during the rollout, and avoid pushing releases at the exact moment of your daily peak.

Saving money is one of the main reasons to autoscale. It is also an easy place to make false savings.

Where the money goes

You pay for machines, not for pods. A pod only costs money because it occupies space on a machine. So the real saving from scaling pods down comes when the node autoscaler can also remove an emptied machine. If pods are scattered thinly across many nodes, no node ever becomes empty, and the savings never reach your bill. Settings that help the scheduler pack pods tightly, and a node autoscaler that consolidates underused machines, turn pod-level savings into real money.

Practical cost levers

Once the basics are stable, these ideas help you handle bigger, trickier systems.

Predictive and scheduled scaling

The HPA reacts. It does not predict. For traffic with a clear daily or weekly rhythm, you can add a prediction by hand: use a cron-style trigger to raise the minimum ahead of the busy period. Some cloud providers offer predictive scaling for virtual machine groups that learns from history, but Kubernetes itself has no built-in predictive HPA, so in most clusters scheduled floors are the practical tool.

Autoscaling several tiers together

Real systems are chains of services. If the front end scales up but the service behind it does not, the bottleneck simply moves. Think about scaling as a pipeline: when the API doubles, the services it calls will need to grow too, and the database must be able to absorb it. Often the right answer is to scale the front end on request rate, scale the back-end on its own queue or concurrency, and protect the database with connection pooling and caching rather than hoping it will scale with them.

Multiple clusters

An HPA works inside one cluster. Companies that run many clusters, for example one per region, either let each cluster scale itself and balance traffic between them, or use multi-cluster tools. Some projects, such as Karmada, provide a federated version of the HPA for this purpose. For most teams, scaling each cluster independently behind a global load balancer is simpler and more robust.

Service level objectives

A service level objective (SLO) is a promise, such as “99 percent of requests finish within 300 milliseconds”. Autoscaling is one tool that helps you keep that promise, but SLO measures like latency are usually better used for alerts and capacity reviews than as the direct scaling input. The scaling metric should be a leading indicator, such as concurrency or backlog, that you have shown by testing to protect the SLO.

A testing strategy that works

Short answers to questions that come up again and again.

No. Prometheus is popular, but any system that can serve the Kubernetes custom or external metrics API will do. Cloud monitoring products, Datadog and KEDA’s direct scalers all work without Prometheus.

Custom metrics describe something attached to a Kubernetes object, such as requests per pod. External metrics describe something outside the cluster, such as the length of a cloud queue. The API groups and the HPA metric types are different for each.

Yes. The HPA calculates a replica count for each metric and uses the largest, so the most demanding signal wins.

Not by default. It has long been an alpha feature behind a feature gate, so check your Kubernetes version before relying on it. KEDA supports scale to zero out of the box and is the usual choice.

Yes, as long as they do not serve the same API group. A common setup uses the Prometheus Adapter for the custom metrics API and KEDA for the external metrics API, with each workload controlled by only one of them.

Expect roughly one to two minutes from a change in load to ready pods if nodes are available, and longer if new machines must start. The scrape interval, HPA cycle, image size and readiness time all add up.

The letter m means milli, or one thousandth. So 500m is 0.5. It is just a unit, not an error.

Rarely. Many applications keep memory they have used, so it does not fall after load drops, and it does not predict load well. It is better as a protection than as a main signal.

Yes, as long as the workload exposes the scale interface. Be careful, though: stateful systems often need data to be rebalanced when replicas change, so test scaling behavior and use conservative scale-down rules.

Load test one pod to find its comfortable throughput, then set the target to about 60 to 70 percent of that. Review it after real traffic teaches you more.

The HPA cannot compute a new count and will keep the current replicas in most cases. Because of that, alert on metric availability and keep a reasonable minimum so a monitoring outage does not leave you under-provisioned.

For workloads that cannot be split across copies, or that waste resources because their requests are guessed wrong, the Vertical Pod Autoscaler helps. Use it for right-sizing and use the HPA for handling load changes, without both reacting to the same metric.

If you remember only a handful of things from this guide, make them these.

Custom metrics autoscaling is the practice of letting the numbers that describe real work, not just processor usage, decide how many copies of your application run. The machinery is a chain: your application publishes a number, a collector stores it, an adapter serves it through a Kubernetes API, the HPA applies a simple proportional formula, and the Deployment creates or removes pods. Each link adds some delay, and each link can fail, so a good design measures and protects every one.