Skip to content

Commit c6d3b17

Browse files
ricmattjyunjianlugeoffcline
authored
Add auto mode recommendations (#640)
* add auto mode recommendations * Add FAQ section to AutoMode page * Adding auto mode session on recommendation vpc-cni (#1) * Adding auto mode session on recommendation vpc-cni * Adding auto mode into loadbalancing section. * minor cleanup of auto mode documentation in networking docs * add auto-mode tips to the upgrades section * Add advice on leaning on AWS support for troubleshooting auto mode components * chore: style updates from Codium AI agent * fixup for docs platform --------- Co-authored-by: Yunjian Lu <[email protected]> Co-authored-by: Geoffrey Cline <[email protected]>
1 parent 86a0892 commit c6d3b17

File tree

10 files changed

+140
-2
lines changed

10 files changed

+140
-2
lines changed

Diff for: .prettierignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
**

Diff for: content/index.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ We currently have published guides for the following topics:
1818

1919
* [Best Practices for Security](security/docs/)
2020
* [Best Practices for Reliability](reliability/docs/)
21-
* Best Practices for Cluster Autoscaling: [karpenter](karpenter/), [cluster-autoscaler](cluster-autoscaling/)
21+
* Best Practices for Cluster Autoscaling: [karpenter](karpenter/), [cluster-autoscaler](cluster-autoscaling/), [EKS Auto Mode](auto-mode/)
2222
* [Best Practices for Networking](networking/index/)
2323
* [Best Practices for Scalability](scalability/docs/)
2424
* [Best Practices for Cluster Upgrades](upgrades/)

Diff for: latest/bpg/autoscaling/auto-mode.adoc

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
[."topic"]
2+
[[automode,automode.title]]
3+
= EKS Auto Mode
4+
:info_doctype: section
5+
:info_title: EKS Auto Mode
6+
:info_abstract: EKS Auto Mode
7+
:info_titleabbrev: EKS Auto Mode
8+
:imagesdir: images/autoscaling
9+
10+
11+
Amazon EKS Auto Mode represents a significant evolution in Kubernetes infrastructure management, combining secure and scalable cluster infrastructure with integrated Kubernetes capabilities managed by AWS . The service provides fully-managed worker node operations, eliminating the need for customers to set up Managed Node Groups or AutoScaling groups .
12+
13+
The key architectural difference is that EKS Auto Mode uses a Karpenter-based system that automatically provisions EC2 instances in response to pod requests . These instances run on Bottlerocket AMIs with pre-installed add-ons like EBS CSI drivers, making the infrastructure truly managed by AWS . In contrast to traditional scaling methods:
14+
15+
* Traditional Cluster Autoscaler (CAS) requires manual node group management and can only create nodes with a single instance type per node group
16+
* Self-managed Karpenter offers more flexibility by working with EC2 Fleet API and can provision different instance types, but requires customer management
17+
* EKS Auto Mode handles all scaling operations automatically through managed NodePools and NodeClasses
18+
19+
20+
The new system introduces several operational improvements:
21+
22+
* Automatic pod-driven scaling without manual node group configuration
23+
* Built-in managed load balancer controllers that automatically create ALB/NLB based on Ingress resources
24+
* Integrated security features with pre-configured Pod identity
25+
* Maximum node runtime of 21 days with automatic replacement
26+
27+
28+
From a cost perspective, EKS Auto Mode maintains standard EC2 pricing while adding a management fee only for Auto Mode-managed nodes. Importantly, customers can still mix Auto Mode managed nodes with self-managed nodes in the same cluster .
29+
30+
While AWS handles most operational aspects, customers retain responsibility for https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html[cluster version management] and can perform controlled upgrades that trigger rolling updates of worker nodes .
31+
32+
== Reasons to use Auto Mode
33+
34+
Auto Mode is geared towards users that want the benefits of Kubernetes and EKS but need to minimize operational burden around Kubernetes like upgrades and installation/maintenance of critical platform pieces like auto-scaling, load balancing, and storage. Auto Mode takes EKS a step further in the minimization of the undifferentiated heavy lifting that goes along with Kubernetes maintenance
35+
36+
37+
== FAQ
38+
39+
=== What is the difference between EKS Auto Mode and Open Source Karpenter?
40+
41+
EKS Auto Mode is a large suite of features that make running production-grade
42+
Kubernetes simple. One of these features is the auto-scaling benefits of Karpenter,
43+
fully managed. From an operations standpoint, the only difference is in EKS
44+
Auto Mode you do not need to manage the deployment, scaling, and upgrade of
45+
the Karpenter pods themselves. All other operations, like managed NodeClasses
46+
and NodePools works the same as with open source Karpenter.
47+
48+
=== Can I run managed node groups alongside Auto Mode-managed nodes?
49+
50+
Yes, you may run static nodes via a managed node groups alongside your autoscaling nodes provided with Auto Mode
51+
52+
=== Can I migrate a cluster from standard EKS to EKS Auto Mode?
53+
54+
Yes, instructions to enable EKS Auto Mode on an existing cluster can be found
55+
in the official https://docs.aws.amazon.com/eks/latest/userguide/auto-enable-existing.html[AWS Documentation]
56+
57+
Things to note:
58+
1. After enabling Auto Mode, you'll want to uninstall any components you had installed that are now managed by Auto Mode, like Karpenter or the AWS Load Balancer Controller
59+
2. You need to make sure your installed add-ons are up-to-date. See documentation.
60+
61+
=== How do I configure NodePools in EKS Auto Mode?
62+
63+
A new cluster will come pre-configured with two NodePools
64+
65+
==== general-purpose
66+
67+
image:gp_nodepool.png[General Purpose NodePool]
68+
69+
This NodePool instructs Karpenter to launch nodes with the following characteristics:
70+
71+
1. Capacity Type of “On Demand”
72+
2. Instance Types of C, M, or R
73+
3. Instance Generation of 4
74+
4. AMD architecture
75+
5. Linux OS
76+
77+
78+
It also defines what the scale down logic is by declaring that only 10% of all nodes may be in a disrupted state at any given time and that consolidation should only occur when nodes are empty or underutilized.
79+
80+
81+
==== system
82+
83+
image:system_nodepool.png[System NodePool]
84+
85+
86+
This NodePool is similar to “general-purpose” except for the following differences:
87+
88+
1. It allows for nodes with the ARM architecture as well as AMD architecture
89+
2. It taints these nodes with a NoSchedule unless there’s a toleration for “CriticalAddonsOnly”. This is for internal use by EKS add-ons
90+
91+
==== custom
92+
93+
You may create your own custom NodePools depending on your needs. To learn more about
94+
NodePools please consult the https://karpenter.sh/docs/concepts/nodepools/[Karpenter Documentation].
95+
96+
=== Can I customize the AMI used by Auto Mode when new nodes are launched?
97+
98+
No, currently the only supported AMIs are for Amazon-provided Bottlerocket
99+
100+
=== How can I install custom tooling or agents on my Kubernetes hosts? ===
101+
102+
Because AMI customization is not supported, if you have a need for host-level software for things like security scanning you should
103+
be deploying the workload as a Kubernetes https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSet].
104+
105+
=== What components are running in my cluster data plane when I provision a new EKS Auto Mode cluster?
106+
107+
By default, the only pods running in an EKS Auto Mode cluster are Kubernetes Metrics Server pods. The other components of EKS Auto Mode
108+
like Karpenter, the AWS Load Balancer Controller, and the EBS CSI Driver are all running and managed off-cluster.
109+
110+
=== What managed components are running to support my new EKS Auto Mode cluster?
111+
112+
EKS Auto Mode completely automates the deployment most of the pieces of a data plane needed for production-grade Kubernetes. This includes:
113+
114+
* Karpenter, for auto-scaling the compute of your cluster
115+
* AWS Load Balancer Controller to allow you to easily expose Kubernetes services via automated Elastic Load Balancer integration
116+
* EBS CSI
117+
* VPC CNI
118+
* EKS Pod Identity Agent
119+
120+
=== How do I troubleshoot the components of Auto Mode that used to run as pods in my cluster? ===
121+
122+
With EKS Auto Mode, many of the components like the AWS Load Balancer Controller and Karpenter are managed for you outside of your cluster, therefore you won't have the same visibility into the logs that you are used to when self-managing. If you are in a situation where you need to troubleshoot the functionality of a piece of Auto Mode functionality create an AWS Support Ticket.

Diff for: latest/bpg/autoscaling/index.adoc

+2
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,5 @@ include::karpenter.adoc[leveloffset=+1]
2626

2727
include::cluster-autoscaler.adoc[leveloffset=+1]
2828

29+
include::auto-mode.adoc[leveloffset=+1]
30+

Diff for: latest/bpg/images/autoscaling/gp_nodepool.png

116 KB
Loading

Diff for: latest/bpg/images/autoscaling/system_nodepool.png

125 KB
Loading

Diff for: latest/bpg/index.adoc

+1
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ We currently have published guides for the following topics:
3333
* xref:reliability[Best Practices for Reliability]
3434
* xref:karpenter[Best Practices for Cluster Autoscaling: Karpenter]
3535
* xref:cas[Best Practices for Cluster Autoscaling: cluster-autoscaler]
36+
* xref:automode[Best Practices for Cluster Autoscaling: EKS Auto Mode]
3637
* xref:networking[Best Practices for Networking]
3738
* xref:scalability[Best Practices for Scalability]
3839
* xref:cluster-upgrades[Best Practices for Cluster Upgrades]

Diff for: latest/bpg/networking/loadbalancing.adoc

+3
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,9 @@ The AWS Load Balancer Controller (LBC) has to be installed in the EKS
102102
clusters and provisions AWS load balancers that point to cluster Service
103103
or Ingress resources.
104104

105+
If you are utilizing link: https://docs.aws.amazon.com/eks/latest/userguide/automode.html[EKS Auto Mode]
106+
the AWS Load Balancer is provided for you automatically; no installation necessary.
107+
105108
In order for the LBC to manage the reconciliation of Kubernetes Service
106109
resources of type LoadBalancer, you need to offload the reconciliation
107110
from the in-tree controller to the LBC, explicitly. With

Diff for: latest/bpg/networking/vpc-cni.adoc

+4
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,10 @@ image::cni_image-5.png[illustration of multiple ENIs attached to a node]
102102

103103
== Recommendations
104104

105+
=== Deploy EKS cluster with Auto Mode
106+
107+
When you use EKS Auto Mode to create a cluster, AWS manages the VPC Container Network Interface (CNI) configuration for your cluster. With Amazon EKS Auto Mode, you don’t need to install or upgrade networking add-ons. However, ensure your workloads are compatible with the managed VPC CNI configuration.
108+
105109
=== Deploy VPC CNI Managed Add-On
106110

107111
When you provision a cluster, Amazon EKS installs VPC CNI automatically. Amazon EKS nevertheless supports managed add-ons that enable the cluster to interact with underlying AWS resources such as computing, storage, and networking. We highly recommend that you deploy clusters with managed add-ons including VPC CNI.

Diff for: latest/bpg/upgrades/index.adoc

+6-1
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,10 @@ controllers, as required.
181181
kubectl.]
182182
. https://docs.aws.amazon.com/eks/latest/userguide/update-managed-node-group.html[Upgrade
183183
the cluster data plane.] Upgrade your nodes to the same Kubernetes minor
184-
version as your upgraded cluster.
184+
version as your upgraded cluster.
185+
186+
TIP: If your cluster was created using EKS Auto Mode you do not need to upgrade your cluster data plane. After upgrading your control plane, EKS Auto Mode will begin incrementally updating managed nodes while respecting all pod disruption budgets. Ensure to monitor these updates to verify compliance with your operational requirements.
187+
185188

186189
[[usedocs, usedocs.title]]
187190
== Use the EKS Documentation to create an upgrade checklist
@@ -275,6 +278,8 @@ Kubernetes minor version.
275278
* *Karpenter:* For installation and upgrade information, see the
276279
https://karpenter.sh/docs/upgrading/[Karpenter documentation.]
277280

281+
TIP: You do not have to manually upgrade any of the capabilities of Amazon EKS Auto Mode, including the compute autoscaling, block storage, and load balancing capabilities.
282+
278283
== Verify basic EKS requirements before upgrading
279284

280285
AWS requires certain resources in your account to complete the upgrade

0 commit comments

Comments
 (0)