AWS Cloud Migration & Cost

Kubernetes in Production: The CTO's Field Guide

ILMTEC
ILMTEC Team
ILMTEC Engineering
Aug 30, 2025
6 min read
Kubernetes in Production: The CTO's Field Guide
The short answer

Run Kubernetes in production by treating it as a platform to operate, not a tool to install: start on a managed control plane (EKS/GKE/AKS), set resource limits and health probes everywhere, deploy via GitOps, spread across availability zones, and wire in observability, autoscaling guardrails, and backups before real traffic arrives.

How do you run Kubernetes in production without it running you?

You run Kubernetes in production successfully by treating it as a platform to operate, not a tool to install. In practice that means starting on a managed control plane, setting resource limits and health checks on every workload, shipping changes through GitOps, and wiring in observability and disaster recovery before your first real traffic โ€” not after your first outage.

Kubernetes is now the default way to run containers at scale, and for good reason: self-healing deployments, horizontal autoscaling, and one consistent API across clouds. But it is also a distributed system in its own right, with its own failure modes. The teams that struggle are almost always the ones who adopted it for the rรฉsumรฉ line rather than a concrete need. This guide is written for the founder or CTO deciding whether โ€” and how โ€” to make that bet.

Do you actually need Kubernetes?

Most early-stage products do not. If you run a handful of services, a managed platform-as-a-service or a serverless container runtime will get you to product-market fit faster and cheaper, with a fraction of the operational surface area. Kubernetes earns its complexity only when specific triggers appear.

Reach for it when these are true, not when they might one day be true:

  • You run many services. Dozens of deployments that need consistent rollout, scaling, and networking benefit from a shared orchestrator.
  • You need portability. Multi-cloud, hybrid, or on-prem-plus-cloud strategies are far easier on a common Kubernetes API than on proprietary primitives.
  • You have someone to own it. At least one engineer who can live in cluster operations. Below that headcount, the cluster owns you.
  • Your workloads are heterogeneous and bursty. Batch jobs, GPU inference, and web services with different scaling profiles all share capacity well under Kubernetes.

If none of these apply yet, the honest answer is to wait. Adopting Kubernetes too early is one of the most expensive premature optimisations a small engineering team can make.

Should you use managed Kubernetes or self-manage the control plane?

Use managed. Running your own control plane โ€” etcd, the API server, the scheduler, certificate rotation, version upgrades โ€” is a full-time job that adds zero product value. On AWS that means Amazon EKS; the direct equivalents are AKS on Azure and GKE on Google Cloud, and the differences matter more than most teams expect when you weigh them in an AWS vs Azure vs Google Cloud comparison.

The real decision is not which cloud, but how much of the stack you want to operate at all:

ApproachWhat you operateOps burdenBest for
Self-managed KubernetesControl plane, etcd, nodes, upgrades, everythingVery highStrict data-sovereignty or air-gapped estates with a dedicated platform team
Managed Kubernetes (EKS / GKE / AKS)Worker nodes, workloads, add-onsModerateTeams that want Kubernetes portability without babysitting the control plane
Serverless containers (Fargate / Cloud Run)Just your containersLowSmaller estates and teams without a full-time ops owner

A pragmatic middle path is managed Kubernetes with a serverless data plane โ€” EKS on Fargate, for example โ€” so you keep the Kubernetes API but stop patching individual nodes. Start there and pull node management back in only when cost or control demands it.

What actually breaks Kubernetes in production?

Clusters rarely fall over because of exotic bugs. They fall over because of a short list of predictable mistakes:

  • No resource requests or limits. Without them, one memory leak evicts its neighbours and the scheduler makes bad packing decisions. This is the single most common cause of mystery restarts.
  • Missing readiness and liveness probes. Kubernetes routes traffic to pods that are running but not ready, and never restarts pods that are wedged. Both show up as intermittent errors that are miserable to debug.
  • Single-AZ deployments. Running every replica in one availability zone turns a routine AWS AZ blip into a full outage. Production workloads must spread across at least two.
  • Unbounded autoscaling. A misconfigured Horizontal Pod Autoscaler plus an aggressive cluster autoscaler can quietly scale you into a five-figure surprise bill overnight.
  • Secrets in plaintext. Base64 is not encryption. Secrets belong in a managed store (AWS Secrets Manager, external-secrets), not in a Git-committed manifest.
  • Neglected upgrades. Kubernetes ships a new minor version roughly every four months and support windows are short. Skip upgrades and you eventually face a forced, high-risk jump across several versions at once.

None of these are hard to prevent. They are simply the things teams skip when they treat Kubernetes as fire-and-forget rather than something to operate.

How much does Kubernetes cost to run?

The bill has three layers, and only one of them is obvious. The control plane is a flat fee โ€” EKS charges about $0.10 per hour per cluster, roughly $73 a month, before you run a single workload. The compute is where the money actually goes, and where most teams overspend by carrying idle capacity. The third layer โ€” data transfer, load balancers, and storage โ€” is the one that creeps up unnoticed.

The biggest lever is utilisation. Right-sized requests, the cluster autoscaler, and Spot instances for stateless and batch work routinely cut node cost by half or more, and moving x86 workloads to Graviton (Arm) nodes reduces equivalent compute cost again. These are the same levers that make a broader cloud move pay off; if you are budgeting a migration around a new cluster, our breakdown of AWS migration cost in 2026 shows where Kubernetes spend fits into the total. The mistake to avoid is running dozens of tiny clusters "for isolation" โ€” each one carries that control-plane fee and its own operational overhead. Prefer fewer clusters with strong namespace and network-policy boundaries.

What does a production-ready Kubernetes setup look like?

Before you put customer traffic on a cluster, work down this checklist. It maps directly onto the reliability and operational-excellence pillars covered in the AWS Well-Architected Framework:

  1. Managed control plane, multi-AZ node groups. No single point of failure at the zone level.
  2. GitOps deployments. Argo CD or Flux, so the cluster's desired state lives in version control and every change is auditable and reversible.
  3. Resource requests, limits, and probes on every workload. Non-negotiable defaults enforced by policy.
  4. Autoscaling with guardrails. HPA plus cluster autoscaler, with maximum node counts and budget alarms so scaling cannot run away.
  5. Observability from day one. Metrics, logs, and traces (Prometheus, OpenTelemetry) wired in before launch, not bolted on during an incident.
  6. Secrets, RBAC, and network policies. Least-privilege access and explicit pod-to-pod rules; default-open networking is a liability.
  7. Backups and a tested restore. etcd and persistent-volume backups mean nothing until you have actually rehearsed recovering from them.
  8. A standing upgrade cadence. Schedule minor-version upgrades so you are never more than one behind.

Tick every box and Kubernetes becomes boring in the best way โ€” predictable, self-healing, and quiet. That is the goal.

How ILMTEC helps

ILMTEC designs and operates production Kubernetes on AWS as part of our AWS cloud migration service, delivered in fixed six-week cycles. A bounded scope is deliberate: it forces the hard decisions โ€” managed versus self-managed, cluster count, autoscaling guardrails, cost model โ€” to be made up front instead of discovered in production. Our engineers in Pune, Dubai, and Berlin stand up EKS with GitOps, multi-AZ resilience, observability, and right-sized Spot-and-Graviton node pools, then hand you a platform your team can actually run. If you are weighing whether Kubernetes is the right bet โ€” or want an existing cluster reviewed against the checklist above โ€” that first cycle is the fastest way to a straight answer.

ILMTEC Service
AWS Cloud Migration
Zero-downtime AWS migrations โ€” free with an ILMTEC build.

Frequently Asked Questions

Do I actually need Kubernetes for my startup?

Most early-stage products do not. If you run only a handful of services, a managed platform-as-a-service or a serverless container runtime (Fargate, Cloud Run) is faster and cheaper with far less operational surface. Kubernetes earns its complexity once you run many services, need multi-cloud portability, have bursty heterogeneous workloads, and have at least one engineer to own cluster operations.

Should I use managed Kubernetes or self-manage the control plane?

Use managed โ€” EKS on AWS, GKE on Google Cloud, or AKS on Azure. Self-managing etcd, the API server, certificate rotation, and version upgrades is a full-time job that adds no product value. For smaller teams, managed Kubernetes with a serverless data plane like EKS on Fargate removes node patching while keeping the Kubernetes API.

What most commonly breaks Kubernetes in production?

The usual causes are predictable, not exotic: missing resource requests and limits (leading to noisy-neighbour evictions), absent readiness and liveness probes, single-availability-zone deployments, unbounded autoscaling that produces surprise bills, plaintext secrets, and neglected version upgrades. All are cheap to prevent and expensive to debug after the fact.

How much does it cost to run Kubernetes on AWS?

EKS charges a flat control-plane fee of about $0.10 per hour per cluster (roughly $73 a month) before any workloads run. Compute is the largest cost and where most overspend happens. Right-sized requests, the cluster autoscaler, Spot instances for stateless work, and Graviton (Arm) nodes typically cut node cost by half or more.

How many Kubernetes clusters should I run?

Prefer fewer clusters with strong namespace and network-policy isolation over many small ones. Every cluster carries its own control-plane fee and operational overhead, so running dozens 'for isolation' multiplies both cost and toil. Use separate clusters mainly for hard boundaries such as production versus non-production or distinct compliance domains.

What belongs on a production-ready Kubernetes checklist?

A managed control plane with multi-AZ node groups, GitOps deployments (Argo CD or Flux), resource limits and health probes on every workload, autoscaling with maximum-node and budget guardrails, observability wired in before launch, secrets in a managed store, least-privilege RBAC and network policies, tested backups, and a standing upgrade cadence so you are never more than one version behind.

Topics
Kubernetes
DevOps
Cloud Native
AWS EKS
Container Orchestration

Found this useful? Share it

AWS Cloud Migration

Ready to put this into production?

ILMTEC delivers in 6-week cycles. Book a free consultation or explore the service.

Explore AWS Cloud Migration
Chat on WhatsApp