Kubernetes Autoscaling: HPA, VPA, and Cluster Autoscaler

Let me start with a failure scenario I have seen countless times: a team launched an API and statically provisioned 20 Pods based on "peak traffic × 3." They ran well during the day, but all 20 Pods kept burning money after traffic disappeared at night. The team later heard about HPA and configured CPU-based scaling. During a major promotion, however, CPU usage barely increased because the bottleneck was the number of DB connections. HPA never added any Pods, and the service collapsed.

These two extremes are exactly what Kubernetes autoscaling is meant to address. Yet many people still think "HPA = add Pods based on CPU" without realizing that Kubernetes actually has three layers of scaling, each addressing a different dimension of the problem:

• HPA (horizontal scaling): adds or removes Pods. The primary mechanism for handling traffic.
• VPA (vertical scaling): adjusts resources for each Pod (requests/limits). It answers the question of how much to allocate.
• Cluster Autoscaler (cluster scaling): adds or removes nodes. When Pods have nowhere to run, it requests more machines.

These are not mutually exclusive choices. They work together. This article examines each mechanism, then covers the cold-start problem that costs teams the most, and finally provides a practical configuration you can use directly: HPA, custom metrics, and a PDB.

1. Why Three Scaling Layers Are Necessary: Three Dimensions, Three Problems

Start with the right mental model. Autoscaling fundamentally answers one question: "What should change when the load changes?"

┌───────────────────────────────────────────────────────────────┐
│                 Load arrives / load subsides                  │
└──────────────────────────────┬────────────────────────────────┘
                               │
           ┌───────────────────┼───────────────────┐
           ▼                   ▼                   ▼
    ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
    │     HPA     │     │     VPA     │     │      CA     │
    │ Horizontal  │     │  Vertical   │     │    Nodes    │
    ├─────────────┤     ├─────────────┤     ├─────────────┤
    │ Add/remove  │     │ Adjust Pod  │     │ Add/remove  │
    │ Pod replicas│     │ resources   │     │ Nodes       │
    └─────────────┘     └─────────────┘     └─────────────┘
 "More people working"  "A larger desk each"  "Rent another floor"

These three layers form a bottom-up trigger chain:

1. HPA detects high Pod CPU usage and increases the replica count from 3 to 10.
2. The cluster's nodes cannot fit all 10 Pods, so the new Pods enter the Pending state because no node can schedule them.
3. Cluster Autoscaler detects the Pending Pods and requests new nodes from the cloud provider.
4. The new nodes become ready, the Pending Pods are scheduled, and traffic is distributed among them.

VPA sits outside this chain because it solves a different problem: the Pod requested the wrong amount of resources from the start. Suppose a service actually uses only 100m of CPU, but its request is set to 1000m. The scheduler reserves node capacity based on 1000m, so 10 such Pods waste 9 CPU cores. By observing actual usage, VPA tells you the correct value or applies it automatically.

Key insight: HPA and VPA cannot operate on the same resource dimension at the same time, such as both using CPU. Adding replicas through HPA reduces CPU usage per Pod. VPA then sees the lower usage and reduces the request, creating a feedback loop in which the controllers work against each other. We will cover the correct way to combine them later.

2. HPA: Horizontal Scaling, the Primary Tool for Handling Traffic

HPA (HorizontalPodAutoscaler) is the first scaling mechanism most teams adopt and the one they use most often. Its control loop is straightforward: periodically query metrics → calculate the desired replica count → update the Deployment's replicas.

1. The Core Algorithm: One Formula Determines Everything

HPA makes scaling decisions with this formula:

Desired replicas = ceil(current replicas × (current metric value / target metric value))

For example, suppose there are currently 3 replicas, the HPA target CPU utilization is 50%, and actual CPU utilization is 90%:

Desired replicas = ceil(3 × (90 / 50)) = ceil(5.4) = 6

HPA therefore scales from 3 replicas to 6. If you configure multiple metrics, such as CPU and custom QPS, HPA calculates the desired replica count for each metric separately and uses the largest value. It is a conservative policy: whichever metric calls for help gets priority.

One detail you must remember is tolerance. By default, fluctuations within ±10% trigger no action. With a 50% target, HPA ignores actual values between 45% and 55%. This prevents small fluctuations from repeatedly changing the replica count, also known as flapping. If HPA seems "unresponsive," this 10% deadband may be the reason.

2. CPU Metrics Are Percentages Relative to Requests

This is the first major trap for beginners. CPU utilization in HPA is calculated relative to the Pod's request, not as an absolute value.

resources:
  requests:
    cpu: 200m     # 0.2 cores

Setting an HPA CPU target of 50% means each Pod reaches the target when it uses 100m, which is 50% of 200m. If you change the request to 2000m while keeping the same 50% target, each Pod must use 1000m before scaling is triggered. You have silently raised the scaling threshold by a factor of 10.

This is why an incorrect request completely distorts HPA behavior. When teams say "HPA is not working," the root cause is often an absurd request value, such as assigning every service 1 CPU core and 2 GB of memory for convenience. The CPU percentage remains low, so HPA never scales out.

3. The Most Basic HPA: CPU and Memory

Here is a common configuration that covers most web services:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3          # Safety floor: never go below 3, even when idle
  maxReplicas: 50         # Upper bound: prevent uncontrolled scaling from exhausting the cluster
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60   # Target CPU utilization: 60%
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 70   # Target memory utilization: 70%

Keep these points in mind:

• Do not set minReplicas to 1. Keep at least 2 or 3 production replicas as a safety floor. If you scale down to one and that Pod fails, the service immediately becomes unavailable.
• maxReplicas is your budget guardrail. Set a limit you can afford. Otherwise, one bad metric can drive the replica count into the hundreds and exhaust all node resources.
• autoscaling/v2 is the current stable version. v2beta2 is deprecated. It supports multiple metrics and behavior configuration.

4. Behavior Tuning: scaleDown Is the Hard Part

Many teams scale out smoothly but fail during scale-in. As soon as traffic falls, HPA removes Pods immediately. When the next traffic wave arrives, replacements cannot start in time, and the service collapses again. The behavior field addresses this problem:

spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0    # Scale out immediately without waiting
      policies:
      - type: Percent
        value: 100                     # At most double each time
        periodSeconds: 60
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300  # Observe for 5 minutes before scaling in
      policies:
      - type: Pods
        value: 2                       # Remove at most 2 Pods every 60s
        periodSeconds: 60
      - type: Percent
        value: 25                      # Or remove at most 25%
        periodSeconds: 60
      selectPolicy: Min                # Use the more conservative policy

Practical lessons from this configuration:

• Scale out quickly and scale in slowly. Set the scaleUp stabilization window to 0 for immediate action, and give scaleDown at least a 5- to 10-minute stabilization window. Traffic is often spiky. Scaling out too late causes incidents, while scaling in too quickly causes the same problem during the next wave.
• Use selectPolicy: Min for scale-in and configure both an absolute Pods policy and a percentage policy. Selecting the smaller change prevents too many Pods from being removed at once.
• Use selectPolicy: Max for scale-out so the number of Pods catches up with traffic as quickly as possible.

5. Custom Metrics: Where HPA Becomes Truly Valuable

CPU-based scaling makes one fatal assumption: CPU is the bottleneck. In practice, many services are constrained by queue depth, the number of DB connections, active connections, or QPS. CPU may remain low in all these cases, making a CPU-only HPA effectively deaf. This is when you need custom metrics.

HPA supports four metric types. Custom metrics use the last three:

To use custom metrics, first deploy a Metrics API aggregator. The most common options are Prometheus Adapter and KEDA. They expose Prometheus metrics through the Kubernetes Metrics API so HPA can query them.

Here is an HPA based on Ingress QPS:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  # Key point: scale by per-replica QPS, which reflects real load
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second   # Provided by Prometheus Adapter
      target:
        type: AverageValue
        averageValue: "100"              # Scale when each Pod handles 100 QPS

A Prometheus Adapter rule that translates PromQL into a metric HPA can query looks like this:

# adapter config (rules)
- seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
  resources:
    overrides:
      namespace: {resource: "namespace"}
      pod: {resource: "pod"}
  name:
    matches: "^(.*)_total"
    as: "${1}_per_second"
  metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'

My strong recommendation: do not scale production services only by CPU. Configure at least one business-level metric, such as QPS, queue depth, or a latency percentile. CPU is an effect; business metrics are the cause. Scaling by the cause always gives you a head start over scaling by the effect. If Prometheus Adapter feels cumbersome, use KEDA directly. As discussed in the cold-start section, it represents scaling against Prometheus, SQS, Kafka, and other data sources as straightforward ScaledObject configurations.

3. VPA: Vertical Scaling to Adjust Each Pod's "Desk Size"

VPA (VerticalPodAutoscaler) addresses a different question: how many resources should this Pod request? It does not change the replica count. Instead, it modifies the Pod's resources.requests.

1. The Real Problem VPA Solves

Every team has done this: a new service is about to launch, nobody knows how many resources it needs, so someone guesses cpu: 500m, memory: 512Mi. Two weeks later, monitoring shows that it actually uses only 30m of CPU and 80Mi of memory. You have reserved 16 times more capacity than necessary. The reverse is worse: if the request is too low, the Pod is throttled by CPU or OOMKilled, while you investigate why the service is so slow.

VPA continuously observes actual Pod usage → uses a sliding window, by default 8 days of history plus real-time data, to calculate P50/P95/P99 percentiles → produces recommendations (target, lowerBound, and upperBound) → applies them or not, depending on its mode.

View the recommendations with:

kubectl describe vpa api-server-vpa
# The output includes:
# Recommendation:
#   Target:  cpu 78m  memory 312Mi
#   ...

2. Three Modes: Off / Initial / Auto

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api-server-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: api-server
  updatePolicy:
    updateMode: "Auto"   # Key field
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 50m
        memory: 64Mi
      maxAllowed:
        cpu: 1
        memory: 1Gi

The three updateMode values are:

• Off: observes usage and provides recommendations, but changes nothing. I strongly recommend running this way for the first week so you can understand the recommendations.
• Initial: applies recommendations only when a Pod is created and does not touch running Pods. This is the safest semi-automatic mode. The recommendation takes effect naturally during the next rolling update without restarting existing Pods.
• Auto: actively evicts and recreates Pods to apply new requests. This is the most aggressive mode.

Treat minAllowed and maxAllowed as guardrails. They prevent VPA from setting an unreasonable request because of a temporary fluctuation. Always configure them in production.

3. VPA's Fundamental Limitation: Changing Requests Requires Restarting Pods

This is the main reason VPA has not seen broader adoption. In Kubernetes, the resource requests of a running Pod are immutable, unless the InPlacePodVerticalScaling feature is enabled. That feature graduated in 1.33 and is currently available only in newer versions. Therefore, when VPA in Auto mode needs to change a request, its only option is to terminate and recreate the Pod.

This introduces several side effects:

• The replica goes offline while being recreated. Without a PDB, or with too few replicas, availability may suffer.
• VPA does not terminate every Pod at once. It follows eviction rules and respects PDBs, but frequent resource adjustments still mean frequent restarts.
• JVM and other warm-up-heavy services must warm up again after every restart, which can be expensive.

Practical recommendations:

• Use Initial mode in production for the safest behavior. Observe the recommendations and let them take effect naturally during rolling releases, with no additional restarts.
• Before enabling automatic updates, confirm that the service can restart gracefully and frequently, and configure a PDB.
• Do not overlook the value of Off mode itself. Use VPA as a resource-tuning advisor and manually apply its recommendations to the Deployment. That is ten times more accurate than guessing.

4. How HPA and VPA Can Coexist

Remember one hard rule: HPA and VPA must never operate on the same resource dimension.

• ❌ HPA scales by CPU while VPA adjusts CPU requests: VPA increases the request → CPU utilization per Pod falls → HPA decides fewer replicas are necessary and scales in → the two controllers fight each other, causing replica counts and requests to fluctuate repeatedly.
• ✅ HPA scales by a custom metric such as QPS while VPA adjusts CPU and memory requests: each controller owns a separate concern. QPS determines the replica count, while CPU and memory determine the desk size for each replica, so they do not interfere with each other.

The community even recommends this sequence: run VPA in Off mode for two weeks to understand the real resource profile → set requests accurately → enable HPA afterward. Build the foundation before attempting horizontal scaling.

4. Cluster Autoscaler: Scaling at the Node Level

The first two layers operate around Pods, but Pods ultimately need Nodes on which to run. If HPA creates 10 new Pods when the cluster is already full, those Pods remain Pending. This is where Cluster Autoscaler (CA) comes in.

1. How CA Works: Watch Pending Pods and Idle Nodes

┌──────────────────────────────────────────────────────────────┐
│ 1. HPA scales out → new Pods become Pending because no node  │
│    can schedule them                                         │
└────────────────────────────┬─────────────────────────────────┘
                             ▼
┌──────────────────────────────────────────────────────────────┐
│ 2. CA detects Pending Pods and simulates scheduling:         │
│    "Could these Pods run if an m5.xlarge were added?"       │
└────────────────────────────┬─────────────────────────────────┘
                             ▼
┌──────────────────────────────────────────────────────────────┐
│ 3. Call the cloud provider API to create a node              │
│    (AWS/GCP/Azure)                                           │
│    Node becomes ready and registers → Pods are scheduled     │
└──────────────────────────────────────────────────────────────┘
                             ▲
                             │
┌────────────────────────────┴─────────────────────────────────┐
│ 4. Scale in: node utilization stays below the threshold      │
│    (50% by default) for 10 minutes                           │
│    → CA moves its Pods elsewhere → node shuts down           │
└──────────────────────────────────────────────────────────────┘

CA's scale-out logic is relatively direct: when it sees Pending Pods, it starts the most suitable node type. Scale-in is more subtle. CA identifies nodes whose utilization remains below the threshold, 50% by default, moves their Pods elsewhere, and then shuts them down. Some Pods cannot be moved, including those with local storage, non-evictable Pods, and DaemonSet Pods.

2. CA Is Structurally Slow

From the moment a Pod becomes Pending until a cloud node is ready usually takes 1 to 5 minutes, including boot time, kubelet registration, and image pulls. This delay is a major source of the cold-start problem discussed later.

CA usually runs as a Deployment in the cluster. Its key arguments include:

# Cluster Autoscaler startup arguments (excerpt)
- --scale-down-unneeded-time=10m        # How long a node must be idle before scale-in
- --scale-down-utilization-threshold=0.5 # Utilization below this value counts as "empty"
- --max-node-provision-time=15m         # Maximum wait for a new node
- --balance-similar-node-groups=true    # Balance across availability zones
- --expander=least-waste                # Node type selection strategy

expander determines which node type to create. Common strategies include random, most-pods for the type that fits the most Pending Pods, least-waste for the least wasted capacity and the recommended default, and priority for a predefined priority table suitable for mixed Spot and On-Demand capacity.

3. CA vs. Karpenter: This Space Is Changing

Traditional Cluster Autoscaler is driven by node groups (Node Group / ASG / MIG). You must define several node pools with fixed instance types in advance, and CA adds or removes machines within those pools. The problem is that the instance types are predetermined, while Pod shapes vary widely. Pods often do not fit, leaving fragmented capacity.

AWS's Karpenter, which was later donated to the community as cross-cloud support advances, takes another approach: it has no node groups and chooses the most suitable instance type directly from current Pod requirements. It translates the requirements of Pending Pods, including CPU, memory, GPU, topology, and tolerations, into a real-time decision about the most cost-effective machine to purchase, then starts that node within seconds. On AWS, Karpenter has nearly become the default choice for new clusters.

Selection guidance:

• On AWS, with diverse instance types and a desire to save money using Spot → choose Karpenter. The experience is dramatically better.
• On GCP or Azure, or when strict node-group governance is required → choose Cluster Autoscaler, which is mature and stable.
• Both share one prerequisite: Pod resource requests must be accurate. CA and Karpenter both schedule from requests. Inflated requests mean unnecessary nodes and wasted money.

5. Cold Starts: The Hard Part Is Never Calculating Scale, but Scaling in Time

At this point, a common weakness becomes clear: all scaling mechanisms are reactive. Traffic has already arrived → metrics rise → HPA calculates a scale-out → new Pods start → new Pods become ready → only then can they receive traffic. The entire chain often takes 2 to 5 minutes.

A traffic spike can overwhelm existing Pods in just 30 seconds. This is the cold-start problem: autoscaling is always one step behind.

1. Break Down the Delay and Optimize Each Segment

Start with the complete scale-out timeline:

[Traffic arrives]                        t=0
[Metric collection window, usually 30-60s] +30-60s  ← Segment 1: metric delay
[HPA calculates and updates replicas]       +seconds ← Segment 2: control loop
[New Pod scheduling + image pull + startup] +20-90s  ← Segment 3: Pod startup
[Readiness probe succeeds]                  +5-30s   ← Segment 4: first ready
[If nodes are full, CA starts a node]        +60-300s ← Segment 5: node cold start
────────────────────────────────────────────────────
Worst case: capacity arrives 5 minutes later

Optimize each segment:

• Segment 1, metric delay: shorten the collection window, such as changing PromQL from [2m] to [1m], but an excessively short window causes instability.
• Segment 2, control loop: HPA runs every 15 seconds by default through --horizontal-pod-autoscaler-sync-period. You can reduce it to 10 seconds, but the marginal benefit is small.
• Segment 3, Pod startup: keep images small by using distroless/alpine and removing debugging tools. Speed up pulls by pre-pulling images onto nodes or using lazydocker/soci to accelerate layer pulls for large images. JVM applications start slowly, so consider GraalVM native images or CRaC.
• Segment 4, readiness probe: do not set an excessive readinessProbe.initialDelaySeconds. Many people enter 30 seconds for convenience and waste all 30 seconds. Use a startupProbe for slow startup instead.
• Segment 5, node cold start: this is the hardest segment. Use the following two techniques.

2. Overprovisioning (Pod Overprovisioning / Pause Pods): Trade Money for Time

The simplest and most effective technique is to keep placeholder Pods running so the nodes are already provisioned and new Pods can be scheduled immediately.

Use a low-priority Pause Pod to reserve node resources. When a real workload Pod arrives, its higher priority allows the scheduler to evict the Pause Pod immediately and place the workload Pod in its slot. No new node is necessary, eliminating the minutes spent in Segment 5.

# 1. Define a low-priority PriorityClass
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: overprovisioning
value: -1                    # Negative, lower than every workload Pod
globalDefault: false
description: "Pause Pods reserved for overprovisioning"
---
# 2. Placeholder Pods that do nothing except reserve resources
apiVersion: apps/v1
kind: Deployment
metadata:
  name: overprovisioning
spec:
  replicas: 3                # Reserve 3 "virtual slots"
  selector:
    matchLabels:
      app: overprovisioning
  template:
    metadata:
      labels:
        app: overprovisioning
    spec:
      priorityClassName: overprovisioning
      containers:
      - name: reserve
        image: registry.k8s.io/pause:3.9
        resources:
          requests:
            cpu: 1
            memory: 2Gi      # Slot size = resources used by one workload Pod

This is equivalent to paying for 3 Pods in advance to reserve scaling slots that are always available. When traffic arrives, workload Pods take the slots within seconds and the Pause Pods are evicted. When traffic falls, the Pause Pods return and reserve the capacity again. It is simple, blunt, and extremely effective. Nearly every large company with latency-sensitive workloads uses some form of this technique.

3. Predictive Scaling: Use Schedules to Beat Reactive Delays

If your load follows a pattern, such as daily peaks, weekly peaks, or major promotions, do not wait for metrics to rise. Scale in advance.

Technique one: CronHPA / scheduled scaling. The Kubernetes community offers CronHPA, contributed by Huawei, or you can use KEDA's cron scaler. For example, scale to 20 replicas at 10:00 every day and return to 5 at 22:00:

# KEDA cron trigger
triggers:
- type: cron
  metadata:
    timezone: Asia/Shanghai
    start: "0 9 * * *"      # Finish scaling before 9:00 AM
    end: "0 23 * * *"        # Scale in after 11:00 PM
    desiredReplicas: "20"

Technique two: forecasting from historical metrics. A more advanced approach uses metrics from the same time over the previous 7 days to forecast today's load curve and scale N minutes early. There is currently no official first-class solution. Common approaches include:

• Build a script or Operator that queries Prometheus history and raises the HPA's minReplicas.
• Use predictive scaling from a cloud provider, such as AWS Predictive Scaling for EKS nodes.
• Use a managed offering from a commercial platform, such as Datadog Adaptive Scaling.

My assessment: 90% of teams will gain far more from implementing overprovisioning plus a business-metric HPA than from experimenting with forecasting algorithms. Predictive scaling is intended for highly regular workloads or extremely latency-sensitive scenarios. Do not start there.

6. Practical Configuration: A Combination You Can Use Directly

The following production-ready configuration combines the preceding techniques. The scenario is an HTTP API service that should scale by QPS and CPU, scale in conservatively, and avoid losing every replica during disruptions.

1. Deployment: Accurate Resource Settings Are a Prerequisite

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
  namespace: prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-server
  template:
    metadata:
      labels:
        app: api-server
    spec:
      containers:
      - name: api
        image: registry.example.com/api-server:v1.2.3
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: 250m       # Set this accurately! It is the denominator for HPA CPU%
            memory: 256Mi
          limits:
            memory: 512Mi   # No CPU limit, avoiding misleading throttling behavior
        readinessProbe:     # Readiness probe: can this Pod receive traffic?
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 2
          periodSeconds: 5
        startupProbe:        # Startup probe: handles slow starts without blocking readiness
          httpGet:
            path: /healthz
            port: 8080
          failureThreshold: 30
          periodSeconds: 5   # Allow up to 150s for startup
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 10"]  # Graceful exit: wait for connections to drain
      terminationGracePeriodSeconds: 60

Several key decisions:

• Do not set a CPU limit, only a request. This prevents CPU throttling from increasing latency and keeps the utilization HPA observes more linear. With a limit, cgroup throttling begins near the ceiling while flattening the utilization curve, which can mislead HPA.
• Configure a separate startupProbe. Give slow-starting services, such as JVM applications and model-loading services, sufficient startup time through startupProbe. After it succeeds, readinessProbe takes over, allowing initialDelaySeconds to remain small.
• Use a preStop sleep. Give kube-proxy time to remove the Pod from the Service endpoints so traffic does not reach a Pod that is shutting down.

2. HPA: Two Metrics and Conservative Scale-In

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 30
  metrics:
  # Metric 1: CPU as a fallback
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 65
  # Metric 2: per-replica QPS, the real business signal from Prometheus Adapter
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "150"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 60
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 2
        periodSeconds: 60
      selectPolicy: Min

3. PDB: A Lifeline During Scale-In and Eviction

Many people forget to configure a PDB (PodDisruptionBudget), allowing one node scale-in or maintenance operation to remove every Pod at once. A PDB limits how many Pods can be removed simultaneously by voluntary disruptions such as node scale-in and kubectl drain:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-server-pdb
  namespace: prod
spec:
  minAvailable: 2          # Always retain at least 2 Pods
  # Alternatively, use maxUnavailable: 1
  selector:
    matchLabels:
      app: api-server

A PDB protects only against voluntary disruptions, such as CA scale-in, node drain, and VPA eviction. It does not protect against involuntary disruptions such as Pod crashes. Replica count must cover those failures. Calculate minReplicas together with PDB.minAvailable so enough replicas remain available in the worst case.

4. Validate the Complete Setup

After deployment, use these commands to verify that autoscaling is healthy:

# View the HPA's current decision and metrics
kubectl get hpa api-server-hpa -n prod
# Check the TARGETS column: if actual usage far exceeds the target without scaling, inspect the metric source

# View HPA decision events: how often it scaled and why
kubectl describe hpa api-server-hpa -n prod | grep -A20 Events

# View CA decisions: did Pending Pods trigger node scale-out?
kubectl logs -n kube-system deploy/cluster-autoscaler | grep "scale up"

# Simulate traffic with a load test and observe the scale-out path
kubectl get pods -n prod -w   # -w continuously watches replica changes

A common troubleshooting case is a TARGETS value of <unknown> in kubectl get hpa, which means the metric cannot be queried. An unknown CPU metric indicates that metrics-server is not installed or working correctly. An unknown custom metric means the Prometheus Adapter rule did not match. Fix metric collection before attempting any tuning.

Summary

• Each of the three scaling layers has a separate responsibility: HPA adds or removes Pod replicas to handle traffic, VPA adjusts a Pod's requests/limits to allocate resources, and Cluster Autoscaler adds or removes Nodes to provide capacity for Pods. They form a bottom-up trigger chain rather than mutually exclusive choices, and production systems usually combine them.
• CPU-based HPA is the starting point; business-metric HPA is where the real value lies. CPU is an effect, while QPS, queue depth, and latency are causes. Remember that HPA CPU utilization is a percentage relative to the request. If the request is wrong, HPA behavior will also be wrong. Scale out quickly and scale in slowly through the behavior field, with a five-minute stabilization window before scale-in.
• VPA is most valuable in Off mode as a resource advisor. Auto mode restarts Pods, so use it cautiously in production; Initial mode is the safest. HPA and VPA must never use the same resource dimension, or they will work against each other.
• Cluster Autoscaler is reactive and slow, operating on a scale of minutes. On AWS, consider Karpenter, which starts nodes within seconds and selects instance types on demand. In either case, accurate requests are essential because both schedule from requests.
• Cold starts are the hardest part of scaling. Break the delay into metric collection, control loop, Pod startup, readiness, and node provisioning, then optimize each segment. The two most effective techniques are overprovisioning with placeholder Pause Pods to trade money for time and scheduled or predictive scaling to use prior knowledge against reactive delays. Do not forget the PDB, or one scale-in operation may remove everything at once.

Key takeaway: Kubernetes autoscaling fundamentally trades reactive delay for resource efficiency. HPA, VPA, and CA all act after metrics rise, so they are always one step behind. Whether you can survive a spike depends less on the precision of HPA's calculation than on whether you are willing to pay for a little excess capacity and reserve a few placeholder Pods during those minutes of delay. Autoscaling optimization ultimately means finding the line between saving money and sleeping soundly.