Before we learn the settings, let us understand the problem they were invented to solve.

A node is one machine in a Kubernetes cluster. It has a fixed amount of processing power, called CPU, and a fixed amount of working memory, called memory or RAM. Kubernetes places many small programs, packed in containers, onto each node so that the machine is used well. This is efficient, but it creates a sharing problem.

Suppose a node has four CPUs and sixteen gigabytes of memory. If ten containers each try to use as much as they can, they will fight. Some programs will run slowly. Others may be killed by the operating system when memory runs out. The worst part is that the fight is unpredictable: a program that worked yesterday might fail today because of a noisy neighbor.

The noisy neighbor problem

The technical name for this problem is the noisy neighbor problem. One container on a node consumes so much of a shared resource that other containers suffer. It is common in every shared system, from apartment buildings to cloud data centers. Kubernetes cannot remove the sharing, but it can make sharing fair and predictable.

A request is a promise Kubernetes makes to your container. A limit is a ceiling your container must respect.

Every container in a pod can be given a request and a limit for each resource. The two settings sound similar, but they answer two very different questions.

What it is, why it exists, where it is used

What it is: two numbers per resource, per container, that describe the minimum needed and the maximum allowed.

Why it exists: so the scheduler can place pods wisely, the node can enforce fairness, and the cluster can survive greedy programs.

Where it is used: in every production cluster. It influences scheduling, autoscaling, eviction, cost reports, and quota systems.

A few basic words are enough to follow the rest of this guide.

Kubernetes is a system that runs your containers across many machines and keeps them healthy. You say what you want, and Kubernetes makes it true and keeps it true. Here are the parts that matter for resources.

Two kinds of resources

Kubernetes treats resources in two families, and the difference is the key to almost every surprise later in this guide.

A compressible resource can be squeezed without breaking anything. CPU is compressible. If a program wants more CPU than it can get, it simply runs slower. It waits its turn.

An incompressible resource cannot be squeezed. Memory is incompressible. If a program needs memory that is not available, nothing can be slowed down to make up for it. Something must be stopped. Remember this pair of words, because they explain why CPU and memory behave so differently when a limit is crossed.

Several parts of Kubernetes cooperate to turn two numbers in a file into real behavior on a real machine.

The diagram below follows a pod from the moment you submit it until it runs inside its boundaries. Notice that different parts read different numbers. The scheduler cares about requests. The node cares about limits. Autoscalers care about actual usage compared with requests.

flowchart LR
    Dev["Developer submits a pod with requests and limits"] --> API["API Server"]
    API --> Adm["Admission checks with LimitRange and ResourceQuota"]
    Adm --> Sched["Scheduler"]
    Sched -->|picks a node with enough room| Kubelet
    subgraph Node["Worker Node"]
        Kubelet["kubelet"] --> Runtime["Container runtime"]
        Runtime --> CG["Linux cgroups"]
        CG --> Cont["Container"]
    end
    Metrics["Metrics server"] -.->|reads usage| Kubelet
    Auto["Autoscalers"] -.->|reads usage| Metrics
    

Meet the components

CPU is measured in a simple unit, and limiting it works very differently from limiting memory.

How CPU is counted

Kubernetes measures CPU in cores. One CPU means one full core, whether it is a physical core or a virtual one in the cloud. You can ask for fractions. Because decimals are awkward, Kubernetes also lets you write in millicores, where a thousand millicores make one core.

The smallest amount you can ask for is one millicore. Half a core does not mean the program only runs half the time on a slow chip. It means the program is entitled to the equivalent of half a core’s worth of time.

What a CPU request does

The CPU request is turned into a weight for sharing. When the node has spare CPU, a container may use much more than it requested, and nobody minds. When the node is busy and many containers want CPU at once, the operating system divides the processor in proportion to their requests. A container that requested twice as much gets twice the share. This is how a request becomes a real, guaranteed minimum under pressure.

What a CPU limit does

A CPU limit is a hard ceiling, enforced by time slicing. The operating system works in short windows of one hundred milliseconds. A limit of half a core means the container may run for fifty milliseconds of CPU time in each window, counted across all its threads together. When the fifty milliseconds are used, the container is paused until the next window begins. This pausing is called throttling.

Memory is counted in bytes, and going over the limit has a much sharper consequence than going over a CPU limit.

How memory is counted

Memory is written in bytes, with helpful suffixes. Two families exist, and confusing them is a classic beginner mistake. The binary family uses suffixes like Ki, Mi, and Gi, which are powers of 1,024. The decimal family uses K, M, and G, which are powers of 1,000. They differ by a few percent. Most people prefer Mi and Gi, because they match how computers actually count.

What a memory request does

The memory request is mainly used for placement. The scheduler makes sure that the node has that much memory left unreserved. The request is not a fence at runtime. A container may use more than it requested if memory is free. But the request has another job: when the node runs short of memory, containers that use more than they requested are the first candidates to be removed.

What a memory limit does

A memory limit is a hard wall. If a container tries to use more memory than its limit, the operating system’s memory guard steps in and kills the offending process. Kubernetes reports this as OOMKilled, where OOM stands for out of memory. The container is then restarted according to the pod’s restart rules. Its exit code is usually 137, a number worth remembering because it appears in many incident reports.

flowchart TD
    U["Container uses more than its request"] --> Q1{"Which resource"}
    Q1 -->|CPU| C1{"Above its limit"}
    C1 -->|Yes| T["Throttled and slowed but keeps running"]
    C1 -->|No| OK1["Fine if CPU is free and shares fairly when busy"]
    Q1 -->|Memory| M1{"Above its limit"}
    M1 -->|Yes| K["OOMKilled so the container is killed and restarted"]
    M1 -->|No| M2{"Node under memory pressure"}
    M2 -->|Yes| E["Pod may be evicted and over-request pods go first"]
    M2 -->|No| OK2["Keeps running"]
    

CPU and memory get all the attention, but a container can also fill a disk or need special hardware.

Ephemeral storage

Every container writes some files while it runs: temporary files, logs, caches, and scratch space. This space is called ephemeral storage because it disappears when the pod goes away. It lives on the node’s disk, which is shared by everyone. A single container that fills the disk with logs can make the whole node unhealthy.

You can give ephemeral storage a request and a limit, just like CPU and memory. The request helps the scheduler avoid nodes with little free disk. The limit protects the node: if a pod writes more than its limit, Kubernetes removes the pod. Unlike CPU, there is no gentle slowdown. It is closer to the memory case.

Extended resources such as GPUs

Some nodes have special hardware, most famously graphics processors used for machine learning. Kubernetes handles these as extended resources. A device plug-in announces the hardware to the cluster, and pods ask for it by name and count. There are some special rules: devices are counted in whole numbers, they cannot be shared in fractions by default, and the amount is normally written as a limit, which then also serves as the request.

The scheduler is a bookkeeper. It counts reservations, not real usage.

When a new pod arrives, the scheduler looks at each node and asks a simple question: after adding this pod’s requests to the requests of the pods already there, do we still stay within the node’s allocatable capacity? If yes, the node is a candidate. If no, the node is skipped. Among the candidates, the scheduler picks the best one using other rules like spreading and preferences.

flowchart TD
    N1["Node A has 4 CPU allocatable and 3.5 CPU already requested"] --> R1{"New pod requests 1 CPU. Does it fit"}
    R1 -->|No because only 0.5 CPU is unreserved| X1["Node A is skipped"]
    N2["Node B has 4 CPU allocatable and 2 CPU already requested"] --> R2{"New pod requests 1 CPU. Does it fit"}
    R2 -->|Yes because 2 CPU is unreserved| OK["Pod can be placed on Node B"]
    

Reserved is not the same as used

This creates two opposite problems. If you request far more than you use, the node looks full while its processors sit idle. You pay for machines that do nothing. If you request far less than you use, the scheduler crams too many pods on one node, and when they all become busy, they compete and suffer. Right-sizing your requests is therefore the heart of good resource management.

When nothing fits

If no node can satisfy a pod’s requests, the pod stays in the Pending state, and its event log says something like “insufficient cpu” or “insufficient memory”. This is not an error in the usual sense. It is Kubernetes telling you honestly that there is no room. The cure is one of three things: ask for less, free up space by removing other pods, or add a bigger or additional node.

A node never offers all its resources to your pods. Some are kept for the machine itself.

A machine with sixteen gigabytes of memory does not have sixteen gigabytes to give your applications. The operating system needs some. Kubernetes’ own helper programs need some. And the node keeps a safety buffer so that it can react before memory runs out completely. What remains for pods is called allocatable.

On small machines this hidden reservation is a noticeable share of the total. A tiny node might offer only around three quarters of its memory to pods. On very large machines the share is smaller. When you plan how many pods fit, always start from allocatable, not from the number on the machine’s price sheet.

The numbers you write become settings in the Linux kernel. Following that path removes all the mystery.

sequenceDiagram
    participant D as Developer
    participant A as API Server
    participant S as Scheduler
    participant K as kubelet
    participant R as Runtime and cgroups
    D->>A: Create pod with requests and limits
    A->>A: Fill in defaults and check quota
    S->>A: Pick a node with enough unreserved room
    K->>A: See the pod assigned to this node
    K->>R: Create the container with CPU weight, CPU time limit, memory limit
    R-->>K: Container running inside its boundaries
    K->>A: Report status and usage
    

Kubernetes sorts every pod into one of three classes, based purely on how you set requests and limits.

You never choose the class directly. Kubernetes works it out from your numbers. The class matters most when the node is in trouble, because it decides who is protected and who is removed first.

flowchart TD
    S["Look at every container in the pod"] --> Q1{"Is any request or limit set anywhere"}
    Q1 -->|None at all| BE["BestEffort"]
    Q1 -->|Yes| Q2{"Does every container have CPU and memory limits equal to requests"}
    Q2 -->|Yes| G["Guaranteed"]
    Q2 -->|No| B["Burstable"]
    

What happens during memory trouble

When a node begins to run out of memory, the kubelet steps in before the kernel has to. It starts evicting pods to free space. It first picks pods whose usage exceeds their requests, then considers pod priority, and among those it takes the ones with the largest overuse first. A BestEffort pod has no request, so any usage counts as overuse, which is why these pods vanish first. A Guaranteed pod never uses more than its request, so it sits at the back of the queue.

If memory runs out so fast that the kubelet cannot react, the kernel’s memory guard takes over and picks victims by that score we saw earlier. The same ordering applies: BestEffort first, Burstable in between, Guaranteed last.

Asking every developer to remember to set sensible numbers does not scale. Namespace-level rules do the remembering for you.

A namespace is a named section of a cluster, often given to one team or one project. Kubernetes lets an administrator attach two kinds of rules to a namespace. They sound alike but do very different jobs.

How they work together

If a namespace has a quota on CPU or memory, then every new pod there must state its requests and limits for those resources, or the API server refuses it. A LimitRange with defaults solves this neatly: developers who forget still get pods with sensible numbers, and the quota can count them. The two rules form a pair.

A pod is more than one container, and its total size is not always a simple sum.

Many pods hold a main application and one or more helpers, called sidecars. A sidecar might collect logs, handle encryption, or route network traffic. Each container has its own requests and limits, and the pod’s total is built from them.

The basic rule: add them up

For containers that run side by side, the pod’s request is the sum of the containers’ requests. A pod with a main container asking for half a core and a helper asking for a quarter of a core needs three quarters of a core. The scheduler uses this total. Small helpers therefore matter: ten pods each carrying an oversized sidecar can quietly consume a whole node.

Init containers: a special case

An init container is a short-lived helper that runs to completion before the main containers start. It might prepare data or wait for a database to be ready. Because it runs first and then exits, it does not add to the total in the same way. The pod’s effective request is the larger of two numbers: the sum of all main containers, or the biggest single init container. This is like a moving crew that arrives, unpacks, and leaves before the residents move in. You only need room for whichever is bigger, the crew or the residents.

Extra costs hiding around a pod

Your request numbers quietly steer every autoscaler in the cluster.

Kubernetes can grow and shrink automatically. There are three main kinds of automatic scaling, and all three depend on the numbers you set.

The percentage trap

When you tell the horizontal autoscaler, “keep average CPU at seventy percent”, seventy percent of what? The answer is: seventy percent of each container’s CPU request. If you request a tiny amount, the pod looks overloaded at once and the autoscaler creates far too many copies. If you request a huge amount, the pod never looks busy, and the autoscaler never adds a copy even when users complain. The request number is therefore also a dial on your scaling behavior.

How nodes are added

When pods stay pending because no node has room for their requests, a cluster autoscaler notices and asks the cloud for another machine. Once the machine joins, the pending pods are placed. So oversized requests do not just waste room on existing nodes. They also make the cluster buy more machines than needed.

Every crossing of a boundary has a different symptom. Learning them is like learning to read a car’s dashboard lights.

Why memory kills but CPU only slows

Remember the two families from the refresher. CPU time is renewed every instant, so a shortage can be handled by making programs wait. Memory is a stock that is filled and held. If a program holds memory and the limit is reached, there is nothing to wait for. The only way forward is to take something away, so the guard kills a process.

There is no magic formula. There is a method: measure, choose, watch, adjust.

Beginners often copy numbers from a tutorial, and those numbers fit the tutorial’s toy program, not your real one. A sound approach follows four steps.

Different programs, different shapes

The great debate about CPU limits

Experienced teams disagree about whether to set CPU limits at all. Both sides have good reasons, and a beginner should understand each.

A common middle path is this: always set CPU requests, set a generous CPU limit for untrusted or shared environments, and monitor throttling either way. For memory the advice is simpler: always set a limit, and for important programs, make it equal to the request so that the pod is well protected and behaves predictably.

Resource settings are also performance settings. A wrong number can make a fast program slow.

Threads and throttling

A program with many threads can use up a small CPU limit very quickly. Imagine a limit of one core and a program with eight busy threads. All eight run at once and consume the hundred-millisecond budget in about twelve milliseconds. Then the program stalls for the remaining eighty-eight. Average CPU usage looks low, yet latency is terrible. Increasing the limit, or reducing the number of threads, often fixes a mystery slowdown.

Language runtimes need to know the limit

Some programming platforms decide how many threads to run, or how much memory to use, by looking at the machine they think they are on. If they see the whole node instead of their container’s slice, they overreach. Modern versions of popular runtimes read container limits correctly, but older versions may not. A program that sizes its memory pool as a fraction of the machine and ignores its container limit will be OOMKilled regularly. Always check how your runtime reads its limits, and leave headroom between the program’s own memory ceiling and the container’s limit for non-heap memory.

Pinning CPUs for special workloads

For latency-sensitive programs, Kubernetes has an optional feature called the static CPU manager policy. When a Guaranteed pod asks for whole cores, for example exactly two, it is given exclusive use of those cores. No other pod runs on them. This avoids the small delays caused by sharing and switching, at the price of less flexible packing.

Packing and overcommit

Requests decide how tightly pods are packed. Limits may add up to more than the node has. This deliberate gap is called overcommit. It works because pods rarely all peak at once. Overcommit on CPU is fairly safe since the worst outcome is slowness. Overcommit on memory is riskier since the worst outcome is kills and evictions. Many teams therefore overcommit CPU freely and memory cautiously.

Good resource settings are a quiet form of insurance against outages.

Priority and preemption

Not all pods are equally important. Kubernetes lets you assign a priority class to a pod. When the cluster is full, and an important pod is pending, the scheduler may remove lower-priority pods to make room. This is called preemption. It is separate from quality of service classes, but they work together: priority decides who gets a seat, and the service class decides who is removed first under memory pressure.

Headroom for failure

If you pack nodes to the very brim, then losing one node means its pods have nowhere to go. They sit pending until a new node is added. A sturdy cluster keeps spare room, often enough to absorb the loss of a node or a whole zone. This spare room is not waste. It is the cost of surviving failures without a wait.

Resource limits are also a defense. A program with no ceiling can be turned into a weapon against its neighbors.

In a cluster shared by many teams, or running code from customers, resource control is part of security. An attacker who takes over a container can try to use every bit of CPU and memory to starve other workloads. This is a form of denial of service. Limits contain the blast.

You can only right-size what you can see. A few key graphs answer most questions.

Alerts worth setting up

• Any container restarted because of OOMKilled.
• A container throttled for more than a small share of its time over an hour.
• A container whose memory stays above ninety percent of its limit.
• Pods pending for longer than a few minutes because of insufficient resources.
• A namespace using more than eighty percent of its quota.
• Nodes reporting pressure conditions, or pods being evicted.

Resource settings bring order. They also bring decisions you cannot avoid.

The main trade-offs

Safety versus cost. Large requests keep applications safe and cost more. Small requests save money and raise the risk of noisy neighbors. There is no free choice; you pick a point on the line.

Predictability versus efficiency. Setting limits equal to requests makes behavior predictable but stops pods from borrowing spare capacity. Letting pods burst uses machines better but makes performance vary with the neighbors.

Strictness versus developer speed. Requiring numbers on every pod protects the cluster, and it also adds a step for developers who just want to try something. Namespace defaults soften this.

Some habits keep clusters healthy. Others create silent trouble.

Patterns that work

Anti-patterns to avoid

A short list of habits prevents most resource problems.

Best practices

Common mistakes

The same ideas appear at every scale, from a small startup to a very large platform.

Match the symptom to its cause, then apply the fix.

Quick answers to the questions beginners ask most often.