There are two utilisation numbers in a Kubernetes cluster and conflating them wastes a great deal of money. The first is allocation efficiency: what workloads request divided by what they use. The second is packing efficiency: what workloads request divided by what the cluster provisions. Right-sizing improves the first. It does nothing whatsoever for the second.

A cluster where every pod requests exactly what it consumes, running on nodes that are 45 percent allocated, is wasting 55 percent of its hardware — and no amount of further right-sizing will recover a cent of it. The scheduler's default behaviour is not trying to fix this, either: LeastAllocated scoring, which spreads pods across nodes to balance load, is the opposite of packing.

Measure the gap before touching anything

# allocated CPU as a fraction of allocatable, per node
sum by (node) (
  kube_pod_container_resource_requests{resource="cpu", node!=""}
)
/ on(node) group_left
sum by (node) (
  kube_node_status_allocatable{resource="cpu"}
)

Run it for CPU and memory. Two things to look at: the fleet mean, and the shape of the distribution.

A mean around 45 to 55 percent is the typical unoptimised cluster. Above 75 percent is well-packed. But the distribution matters more — a cluster averaging 60 percent because half the nodes are at 90 and half are at 30 has a different problem from one where every node sits at 60. The first is a fragmentation problem you can fix by consolidating; the second is a node-shape problem.

Also compute the dominant resource ratio: for each node, the ratio of CPU allocation to memory allocation. If your nodes are consistently at 85 percent CPU and 40 percent memory, you are running compute-optimised workloads on general-purpose instances, and the memory you are paying for is structurally unusable. That is a shape mismatch and no scheduler configuration fixes it.

Node shape is the biggest single lever

The scheduler can only pack into the shapes you give it. Get those wrong and everything downstream is compensation.

Match the ratio to the workload. AWS general-purpose (m) instances are 4 GB per vCPU, compute-optimised (c) are 2 GB, memory-optimised (r) are 8 GB. Compute your fleet's aggregate request ratio — total memory requested divided by total CPU requested — and pick the family that matches. A fleet at 2.2 GB per core running on m instances is stranding roughly 45 percent of its memory permanently.

Bigger nodes pack better, up to a point. Two effects. Fragmentation waste is roughly proportional to the number of nodes: with an average leftover of half a pod's worth per node, forty small nodes waste ten times what four large ones do. And DaemonSet overhead is fixed per node — 900m CPU and 1.5 GiB of agents is 22 percent of a 4-core node and 2.8 percent of a 32-core one.

The counterweights: larger nodes mean a bigger blast radius per failure, slower drain during upgrades, and coarser autoscaling granularity — adding a 64-core node to serve three pending pods is its own kind of waste. The sweet spot for general-purpose workloads is usually 16 to 32 vCPU. Below 8 the DaemonSet tax dominates; above 48 the granularity and blast-radius costs start to bite.

Know your largest pod. The biggest single pod that must schedule sets a floor on node size, and it should not exceed roughly 40 percent of a node's allocatable capacity. A 10-core pod on a 16-core node leaves 5 cores after DaemonSets, which will pack poorly with anything. Either use larger nodes for that workload or give it its own pool.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
      - key: karpenter.k8s.aws/instance-cpu
        operator: In
        values: ["16", "32"]        # floor kills DaemonSet tax
      - key: karpenter.k8s.aws/instance-memory
        operator: Gt
        values: ["32768"]
      - key: karpenter.k8s.aws/instance-family
        operator: In
        values: ["m6i","m6a","m7i","m7a","c6i","c7i","r6i"]
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 5m

Tell the scheduler to pack

The default NodeResourcesFit plugin scores with LeastAllocated, which prefers emptier nodes. That is a reasonable default for a fixed-size cluster where spreading improves resilience. It is the wrong default for an autoscaled cluster, where spreading means more nodes for the same work.

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
- schedulerName: default-scheduler
  pluginConfig:
  - name: NodeResourcesFit
    args:
      scoringStrategy:
        type: MostAllocated
        resources:
        - name: cpu
          weight: 1
        - name: memory
          weight: 1

On a managed control plane you cannot edit the scheduler config, so the options are running a second scheduler for workloads that opt in, or relying on the autoscaler's consolidation to clean up after the fact. Karpenter's consolidation does a reasonable job of this — it will actively delete underutilised nodes and repack their pods — which is one of the substantive advantages in the comparison against cluster-autoscaler.

An alternative worth knowing: RequestedToCapacityRatio with a custom shape lets you target a specific utilisation band rather than maximising monotonically. Setting the peak score at 85 percent rather than 100 leaves deliberate headroom on each node, which matters if your workloads burst above their requests.

Constraints are where capacity goes to die

Every scheduling constraint reduces the set of valid placements, and reducing the placement set is the definition of a packing problem getting harder. Most clusters accumulate constraints that nobody has revisited since they were added.

Hard topology spread. maxSkew: 1 with DoNotSchedule across hostname means a 12-replica Deployment requires 12 distinct nodes, regardless of whether those pods would comfortably fit on four. For a service that genuinely needs host-level anti-affinity, that is the price. For a service where zone-level spread would suffice, it is pure fragmentation.

Audit these. The common pattern is a hostname-level hard constraint copied from a template into services that only ever needed zone-level protection:

topologySpreadConstraints:
- maxSkew: 1
  topologyKey: topology.kubernetes.io/zone
  whenUnsatisfiable: DoNotSchedule      # zone: worth being strict
  labelSelector:
    matchLabels: { app: api }
- maxSkew: 2
  topologyKey: kubernetes.io/hostname
  whenUnsatisfiable: ScheduleAnyway     # host: a preference, not a law
  labelSelector:
    matchLabels: { app: api }

Taints and dedicated pools. Each dedicated node pool is a separate bin, and separate bins cannot share leftovers. Five pools each averaging one node of waste is five nodes of waste. Consolidate pools wherever the isolation is preference rather than requirement, and where a pool genuinely must exist, make its idle capacity visible to the team that requested it — that is usually what causes the pool to eventually get re-examined.

Pod anti-affinity. requiredDuringSchedulingIgnoredDuringExecution is expensive both in packing terms and in scheduler CPU — it is O(pods × nodes) to evaluate and is a known source of scheduling latency in large clusters. Topology spread constraints express most of the same intent more cheaply.

Repacking what is already placed

The scheduler places a pod once and never reconsiders. Over days of scale-ups, scale-downs, and deploys, a cluster drifts into a fragmented state — pods scattered across nodes that could be consolidated onto half as many. Karpenter's consolidation handles a good portion of this. For clusters on cluster-autoscaler, the descheduler is the tool:

apiVersion: descheduler/v1alpha2
kind: DeschedulerPolicy
profiles:
- name: consolidate
  pluginConfig:
  - name: RemovePodsViolatingNodeTaints
  - name: LowNodeUtilization
    args:
      thresholds:            { cpu: 30, memory: 30, pods: 30 }
      targetThresholds:      { cpu: 70, memory: 70, pods: 70 }
      evictableNamespaces:
        exclude: [kube-system, istio-system]
  plugins:
    balance:
      enabled: [LowNodeUtilization]

Run it on a schedule, not continuously, and always with PodDisruptionBudgets in place — the descheduler respects them, and without one it will evict past your availability floor. Every eviction is a pod restart, so the cost is real: on a cluster with slow-starting services, aggressive descheduling can cost more in latency than it saves in nodes.

What good looks like

Realistic targets for a well-packed general-purpose cluster: 70 to 80 percent CPU allocation, 65 to 75 percent memory allocation, with the dominant-resource ratio within about 15 percent of the node family's ratio. Batch-heavy clusters can push higher; latency-critical clusters should sit lower and deliberately, because packing tightly removes the burst headroom that queueing behaviour requires.

Chasing 95 percent is a mistake. At very high allocation, every scale-up requires a new node (there is no room anywhere for a pending pod), scheduling latency rises as the scheduler works harder to find valid placements, and a single node failure has nowhere to drain to. The last 15 percent of packing efficiency costs more in provisioning latency and operational fragility than it returns.

The ordering that works: fix node shapes first, since that is one configuration change with fleet-wide effect. Audit constraints second — usually a week of work that removes constraints nobody could justify. Change scheduler scoring third. Add consolidation last, once the structural problems are gone, because otherwise the descheduler spends its time churning pods around a fragmentation problem it cannot solve.

Two special cases worth flagging. GPU nodes need entirely different treatment — the accelerator is the only resource that matters and CPU/memory packing around it is secondary, which is a different problem with different mechanisms. And none of this substitutes for accurate requests: packing efficiency measured against inflated requests is a well-packed cluster full of air, so get the requests honest first and then pack.