Your Kubernetes Cluster Is Wasting 50-70% of What You Pay For
Here is something that will either validate your suspicions or ruin your morning: the average Kubernetes cluster runs at 20-35% resource utilization. That means for every dollar you spend on K8s compute, $0.50 to $0.70 is paying for CPU and memory that sits idle.
This is not because your team made bad decisions. It is because Kubernetes creates a gap between what you request and what you actually use, and the entire ecosystem is designed to encourage over-provisioning.
Engineers set resource requests high because they do not want their pods evicted. The cluster autoscaler adds nodes based on pending requests, not actual utilization. And nobody downsizes because the risk of a production outage outweighs the cost savings nobody can see.
The result is a cluster that looks healthy from an operations perspective but is hemorrhaging money from a financial perspective.
This guide will show you exactly where your Kubernetes costs hide, which optimizations deliver the biggest savings with the least risk, and the step-by-step playbook to cut your K8s bill by 40-60%.
Where Kubernetes Money Actually Goes
Before optimizing, you need to understand the cost structure. Kubernetes spending breaks down into five categories, and most teams only think about one of them.
1. Compute (Worker Nodes): 60-75% of Total K8s Cost
This is the obvious one. Your worker nodes (EC2 instances on EKS, VMs on GKE or AKS) are your biggest expense. But the real cost driver is not the instances themselves. It is the gap between what your pods request and what they use.
Here is what this looks like in practice:
| Resource | Requested (Total Across Pods) | Actually Used (Peak) | Actually Used (Average) | Waste |
|---|---|---|---|---|
| CPU | 48 vCPUs | 28 vCPUs (peak) | 14 vCPUs (avg) | 71% average waste |
| Memory | 192 GB | 110 GB (peak) | 72 GB (avg) | 63% average waste |
Those 48 vCPUs of requests force the cluster autoscaler to provision enough nodes to satisfy them, even though average usage is only 14 vCPUs. You are paying for 48 vCPUs worth of instances to serve 14 vCPUs of actual work.
2. Control Plane: 3-8% of Total K8s Cost
The EKS control plane costs $73/month. GKE charges $73/month for standard clusters (Autopilot pricing varies). AKS control plane is free, but you pay more for certain features.
This seems small, but multiply it by dev, staging, QA, and production clusters, and you are looking at $292-584/month just for control planes. Teams running separate clusters per team or per application can have 10-20 clusters, costing $730-1,460/month in control plane fees alone.
3. Networking: 5-15% of Total K8s Cost
Kubernetes networking costs are sneaky. They include:
- Load balancers: Each Kubernetes Service of type LoadBalancer creates a cloud load balancer. On AWS, each ALB costs $16+/month plus data processing. Ten services = $160+/month in load balancer base costs.
- NAT gateway: Pods in private subnets route through NAT gateways at $0.045/GB processed. A busy cluster pushing 500GB/month through NAT pays $22.50 in processing fees plus $32.40 for the gateway.
- Cross-AZ traffic: Pods communicating across availability zones pay $0.01/GB each way. Service mesh sidecars and inter-service communication can generate terabytes of cross-AZ traffic monthly.
4. Storage: 5-10% of Total K8s Cost
Persistent volumes, especially EBS gp3 on AWS, charge per GB provisioned, not per GB used. A 100GB PVC that uses 15GB still costs $8/month for the full 100GB. Multiply by dozens of stateful pods and backups, and storage adds up.
5. Observability Tax: 5-15% of Total K8s Cost
This is the cost nobody budgets for but everyone pays. Kubernetes generates massive telemetry: pod metrics, node metrics, container logs, service mesh traces, event streams. Sending all of this to Datadog ($15/host/month for infrastructure + $0.10/GB for logs) or New Relic can easily exceed your compute costs.
A 20-node cluster with verbose logging and full trace coverage can generate $2,000-5,000/month in observability costs alone.
The Resource Request Problem (And Why It Costs You Thousands)
This is the single biggest cost issue in Kubernetes, and it deserves its own section because it is so widely misunderstood.
How Resource Requests Actually Work
When you set a pod's resource request to 500m CPU and 512Mi memory, you are telling the Kubernetes scheduler: "this pod needs at least 500m CPU and 512Mi memory reserved on a node." The scheduler will not place the pod on a node that does not have that capacity available.
The critical word is "reserved." Those resources are unavailable to other pods even if your pod uses only 50m CPU and 128Mi memory. The cluster autoscaler sees the reservation, not the actual usage, and provisions nodes accordingly.
The Vicious Cycle
- Engineers set requests high to avoid OOMKills and CPU throttling
- High requests mean nodes fill up quickly (by reservation, not by usage)
- Cluster autoscaler adds more nodes to handle pending pods
- Actual utilization on those nodes is 15-30%
- Nobody lowers the requests because "it works" and there is no visible incentive
- Costs climb linearly with every new service deployed
The Fix: Data-Driven Resource Requests
Use the Vertical Pod Autoscaler (VPA) in recommendation mode to see what your pods actually use. Do not enable the "Auto" mode in production initially. Just collect recommendations for 2-4 weeks.
Here is what VPA recommendations typically reveal:
| Pod | Current Request (CPU/Memory) | VPA Recommendation | Savings |
|---|---|---|---|
| API server | 1000m / 2Gi | 250m / 512Mi | 75% / 75% |
| Worker | 500m / 1Gi | 150m / 384Mi | 70% / 62% |
| Redis cache | 500m / 2Gi | 100m / 1.2Gi | 80% / 40% |
| Celery workers | 1000m / 1Gi | 400m / 768Mi | 60% / 23% |
Apply the recommendations with a 20-30% buffer above the VPA suggestion. This gives you headroom for traffic spikes while still saving 40-60% compared to the original requests.
Tools that help: Kubecost (free tier shows right-sizing recommendations), Goldilocks (open-source VPA dashboard), and CAST AI (automated optimization).
The 10-Step Kubernetes Cost Optimization Playbook
Ordered by impact and ease of implementation. Start from the top.
Step 1: Install Cost Visibility (Day 1)
You cannot optimize what you cannot see. Install Kubecost (free tier covers one cluster) or OpenCost (fully open-source). Both show cost per namespace, deployment, pod, and label.
Within an hour, you will know exactly which workloads cost the most and which have the worst utilization ratios. This information alone changes engineering behavior.
Step 2: Right-Size Resource Requests (Week 1)
Deploy VPA in recommendation mode. After collecting data for at least 7 days (ideally 14), adjust resource requests based on actual usage plus a 20-30% buffer.
Critical rule: Never set requests equal to limits for CPU. CPU throttling (from hitting limits) is far more damaging to performance than slightly over-requesting. Set CPU requests based on average usage and limits at 2-3x the request. Memory limits should be closer to the request (1.2-1.5x) because memory overuse causes OOMKills.
Expected savings: 30-50% on compute costs.
Step 3: Schedule Non-Production Clusters (Week 1)
If your dev, staging, and QA clusters run 24/7, you are wasting 65-70% of their cost. Most teams only use non-production clusters during business hours (roughly 10 hours per day, 5 days per week = 50 out of 168 hours).
Options:
- Scale node groups to zero after hours using scheduled scaling or cron-based scripts
- Use Karpenter with consolidated scheduling to automatically remove nodes when pods are not running
- For even more savings, adopt ephemeral preview environments that spin up per pull request and destroy after merge
Expected savings: 60-70% on non-production compute.
Step 4: Use Spot Instances for Worker Nodes (Week 2)
Spot instances save 60-80% compared to on-demand pricing. The interruption risk scares teams away, but with the right setup, it is manageable.
The safe approach:
- Run system-critical pods (databases, stateful services) on on-demand nodes using node affinity or taints
- Run stateless application pods on spot instances
- Use pod disruption budgets to ensure at least N replicas survive a spot interruption
- Diversify across 5-10 instance types and 3 availability zones to reduce interruption risk
With Karpenter on EKS, this is almost entirely automated. Karpenter selects the cheapest available instance type that meets your pod requirements, prefers spot, and handles interruptions gracefully.
Expected savings: 50-70% on eligible (stateless) workloads, which are typically 60-80% of cluster compute.
Step 5: Switch to ARM-Based Nodes (Week 2-3)
AWS Graviton (m7g, c7g, r7g), GKE ARM nodes, and Azure Cobalt offer 20-30% better price-performance than equivalent x86 instances.
Most containerized applications run on ARM without modification. The exceptions are workloads with x86-specific compiled binaries or native dependencies. Multi-architecture Docker builds (docker buildx) solve this at the CI/CD level.
Test strategy: Start by migrating your staging cluster to ARM nodes. Run for two weeks. If nothing breaks, migrate production node groups one at a time.
Expected savings: 20-30% on compute costs.
Step 6: Consolidate Namespaces and Clusters (Week 3)
Every additional cluster adds control plane costs, duplicate monitoring infrastructure, and operational overhead. Every namespace with its own ingress controller, cert-manager, and monitoring stack adds resource overhead.
Ask these questions:
- Do you have separate clusters for dev, staging, and QA? Could they share one cluster with namespace isolation?
- Do you have one cluster per team? Could teams share a cluster with RBAC and resource quotas?
- Do you have separate ingress controllers per namespace? Could one shared ingress handle all traffic?
Expected savings: $73-150/month per eliminated cluster (control plane), plus $200-500/month in reduced per-cluster overhead (monitoring, ingress, cert-manager).
Step 7: Optimize Persistent Volume Usage (Week 3-4)
Audit your PVCs:
- Delete PVCs for pods that no longer exist (orphaned PVCs still cost money)
- Right-size PVCs based on actual usage (a 100GB PVC using 15GB should be 20-30GB)
- Use gp3 instead of gp2 on AWS (gp3 is 20% cheaper with better baseline performance)
- Move cold data off persistent volumes to object storage (S3, GCS, Azure Blob)
Expected savings: 20-40% on storage costs.
Step 8: Tame the Observability Tax (Month 2)
Kubernetes observability costs grow linearly with cluster size and logarithmically with your willingness to question defaults.
Quick wins:
- Set log retention to 14-30 days instead of "forever" (the default on CloudWatch Logs)
- Sample traces at 5-10% instead of 100% (you do not need every trace to debug issues)
- Use Prometheus + Grafana (self-hosted, open-source) instead of per-host SaaS monitoring for basic metrics
- Filter noisy logs at the source (do not send debug-level logs to your paid log aggregator)
Bigger moves:
- Adopt OpenTelemetry as your telemetry pipeline. It decouples collection from backends, letting you switch providers or self-host without re-instrumenting.
- Use VictoriaMetrics as a Prometheus-compatible, long-term storage backend. It uses 7x less RAM than Prometheus for the same data and is free for single-node deployments.
Expected savings: 40-70% on observability costs.
Step 9: Implement Pod Priority and Preemption (Month 2)
Not all pods are equal. Your production API server matters more than a batch job. Yet by default, Kubernetes treats them equally.
Set Pod Priority Classes:
- Critical (priority 1000): Production APIs, databases, core services. Never preempted.
- Standard (priority 100): Background workers, async processors. Can be preempted by critical pods.
- Low (priority 0): Batch jobs, data processing, CI runners. First to go when nodes are under pressure.
This lets you pack clusters more tightly. The autoscaler does not need to provision extra nodes "just in case" for every workload, because low-priority pods will yield resources when critical pods need them.
Expected savings: 10-20% from tighter bin-packing.
Step 10: Implement FinOps Governance (Ongoing)
Technical optimizations decay without process. Set up:
- Weekly cost reviews: 15 minutes in engineering standup reviewing Kubecost dashboards. Track cost per namespace and cost per deployment.
- Resource quotas per namespace: Prevent any single team or service from consuming unbounded resources. Use LimitRanges to set default requests and limits for pods that do not specify them.
- Cost allocation tags: Label every deployment with team, application, and environment. Feed this into your FinOps practice for chargeback or showback.
- Deployment cost gates: Use Infracost or Kubecost cost prediction in CI/CD to flag deployments that would increase costs above a threshold.
Expected savings: 15-25% from behavioral change and continuous governance.
Karpenter vs. Cluster Autoscaler: Which Saves More?
If you are running EKS, this is one of the most impactful decisions you can make.
| Feature | Cluster Autoscaler | Karpenter |
|---|---|---|
| Node provisioning speed | 2-5 minutes | 15-60 seconds |
| Instance type selection | Fixed per node group | Dynamic, picks cheapest option |
| Spot integration | Manual node group setup | Automatic, multi-instance diversification |
| Bin-packing efficiency | Moderate (fixed instance types) | High (right-sizes nodes to pod requirements) |
| Consolidation | None (does not replace underutilized nodes) | Active (replaces nodes to improve packing) |
| Cost savings vs. manual | 10-20% | 30-50% |
Karpenter is not just faster. It fundamentally changes how nodes are provisioned. Instead of fitting pods into pre-defined node groups, Karpenter provisions nodes that exactly match pending pod requirements. A pod that needs 2 vCPUs and 8GB RAM gets a node sized for that, not a 16-vCPU node with 90% waste.
The consolidation feature is the real money saver. Karpenter continuously evaluates whether existing nodes can be replaced with cheaper or smaller ones. If three nodes are running at 30% utilization, Karpenter will consolidate those pods onto two nodes and terminate the third.
Savings: Teams switching from Cluster Autoscaler to Karpenter typically see 20-35% additional compute cost reduction.
The "Do I Even Need Kubernetes?" Question
This might be the most valuable section in this guide. Because the cheapest Kubernetes cluster is the one you do not run.
You probably do not need Kubernetes if:
- You run fewer than 8 services
- Your team has fewer than 10 engineers
- You do not need custom scheduling, service mesh, or multi-cloud portability
- You are spending more time managing Kubernetes than building your product
Alternatives that cost 40-60% less:
- ECS Fargate (AWS): No cluster management, pay per pod, auto-scales natively
- Cloud Run (GCP): Scale to zero, per-request pricing, no infrastructure to manage
- Azure Container Apps: Managed containers with KEDA-based auto-scaling
These alternatives eliminate control plane costs, node management overhead, and the engineering time to operate Kubernetes. For many workloads, especially web applications, APIs, and microservices under moderate scale, they deliver the same functionality at a fraction of the cost.
Read more about the hidden Kubernetes tax that teams rarely account for.
Frequently Asked Questions
How much does Kubernetes really cost per month?
The minimum cost for a production-grade EKS cluster is roughly $350-500/month: $73 for the control plane, $150-300 for a two-node worker group (t3.medium), plus $50-100 for load balancers and networking. A typical mid-size production cluster (10-20 nodes) costs $2,000-8,000/month. Enterprise clusters with GPU workloads can reach $50,000-200,000/month.
What is the fastest way to reduce Kubernetes costs?
Right-sizing resource requests. It requires no architectural changes, carries minimal risk, and typically saves 30-50% on compute. Install Kubecost or Goldilocks today, collect data for a week, then adjust requests based on actual usage plus a 20-30% buffer.
Should I use Karpenter or Cluster Autoscaler?
If you are on EKS, use Karpenter. It is faster, smarter about instance selection, and actively consolidates underutilized nodes. Teams typically save 20-35% more with Karpenter compared to Cluster Autoscaler. For GKE, use GKE Autopilot which handles node management similarly. For AKS, Cluster Autoscaler is still the primary option, though AKS Node Auto-Provisioning is the equivalent approach.
Is it safe to run production workloads on spot instances?
Yes, with proper setup. Use pod disruption budgets to maintain minimum replica counts, diversify across 5+ instance types and 3 availability zones, and keep stateful workloads (databases, persistent queues) on on-demand instances. The interruption rate for diversified spot fleets in less-popular instance families is typically below 5%.
How do I reduce Kubernetes observability costs?
Switch from per-host SaaS monitoring to a self-hosted Prometheus + Grafana + OpenTelemetry stack for basic metrics and traces. Set log retention to 14-30 days. Sample traces at 5-10% instead of 100%. These changes typically save 40-70% on observability without meaningfully impacting debugging capability.
Can I optimize Kubernetes costs without dedicated DevOps expertise?
For basic optimizations (right-sizing, scheduling non-prod, spot instances), yes. Kubecost and VPA provide clear recommendations that any developer can implement. For advanced optimizations (Karpenter, pod priority, custom autoscaling), some Kubernetes operational experience is needed. If your team lacks this expertise, a cloud cost optimization partner can implement these changes in a few weeks.
How often should I review Kubernetes costs?
Weekly for the first month during optimization, then monthly. Use Kubecost or OpenCost dashboards in your engineering standup. Track cost per namespace, cost per deployment, and overall utilization percentage. Set budget alerts for anomalies so you catch regressions between reviews.
Start Cutting Your K8s Bill This Week
You do not need a month-long project to see results. Here is your first-week plan:
Day 1: Install Kubecost or OpenCost. See where your money goes. Day 2: Deploy VPA in recommendation mode on your largest namespace. Day 3: Schedule non-production clusters to stop after hours. Day 4: Create a Karpenter provisioner (EKS) or enable GKE Autopilot to replace your static node groups. Day 5: Review VPA recommendations and start right-sizing your top 5 most expensive deployments.
Those five days will typically recover 30-50% of your Kubernetes spend. From there, work through the rest of the playbook at your own pace.
For help optimizing your Kubernetes infrastructure, reach out to our FinOps team. We help teams at every stage, from first-time K8s operators to large-scale multi-cluster environments. And for ongoing Kubernetes management, explore our cloud operations services.
Because Kubernetes is supposed to make your infrastructure more efficient, not more expensive.
