Kubernetes Requests, Limits, and Autoscaling: A Practical Capacity Model

Requests answer a scheduler question

A container's CPU and memory requests tell Kubernetes how much resource to reserve when placing a pod. The scheduler uses requests to decide whether a node has room. For a CPU-based Horizontal Pod Autoscaler (HPA), utilization is measured relative to the requested CPU, so an incorrect request can make scaling behavior misleading even when the workload itself has not changed.

Requests are not a promise that the process will use exactly that amount. They are a planning input. If every workload requests far less than its normal peak, nodes can become crowded and latency becomes noisy. If requests are inflated without evidence, the scheduler may leave usable capacity stranded and pods can remain pending unnecessarily.

Limits have different failure modes

CPU is compressible: when a container reaches its CPU limit, it can be throttled. The process usually remains alive, but request latency and queueing can rise. Memory is different. If a container exceeds its memory limit, the kernel may kill it, producing an OOMKilled termination and a restart loop if the workload repeats the same allocation pattern.

This difference is why a CPU limit that looks harmless in a manifest can create tail-latency incidents. Measure throttling and application latency together before deciding whether a CPU limit is appropriate for the workload and cluster policy.

HPA is not a capacity guarantee

An HPA adjusts replica count based on metrics; it does not provision nodes by itself, ensure that new pods can be scheduled, or add capacity to a saturated database. If the cluster autoscaler is slow, resource requests are too large for available nodes, or startup takes minutes, scaling may react after users already feel the load.

Set a realistic minimum replica count for predictable traffic, allow enough headroom for startup and rollout overlap, and test scale-up under production-like load. Watch pending pods, node pressure, CPU throttling, memory working set, and downstream saturation together. A rising desired replica count paired with no schedulable capacity is a signal to fix cluster capacity, not simply raise the HPA maximum.

A useful tuning loop

Start with representative load tests. Record steady-state and peak CPU, memory, latency percentiles, throttling, and restart behavior. Choose requests that reflect normal sustained demand with room for expected bursts, then apply limits only where the operational trade-off is understood. Revisit values after major runtime, workload, or traffic changes.

Resource settings are part of the application's performance contract. They influence scheduling, autoscaling, and failure behavior, so treat them as measured production configuration rather than boilerplate YAML.