Skip to content

Commit c082d0e

Browse files
bryantbiggsbryantbiggs
authored
feat: Update Karpenter examples to v1 (#2017)
1 parent 5255ef6 commit c082d0e

File tree

13 files changed

+313
-279
lines changed

13 files changed

+313
-279
lines changed

‎patterns/karpenter-mng/README.md

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -11,24 +11,30 @@ This pattern demonstrates how to provision Karpenter on an EKS managed node grou
1111

1212
## Code
1313

14-
The areas of significance related to this pattern are highlighted in the code provided below:
14+
The areas of significance related to this pattern are highlighted in the code provided below.
1515

16-
```terraform hl_lines="20-28 31 49-62 67-70 89-92 102-126"
16+
### Cluster
17+
18+
```terraform hl_lines="20-28 47-50 52-60 64-69"
1719
{% include "../../patterns/karpenter-mng/eks.tf" %}
1820
```
1921

20-
### VPC
22+
### Karpenter Resources
2123

22-
```terraform hl_lines="21-22"
23-
{% include "../../patterns/karpenter-mng/vpc.tf" %}
24+
```terraform hl_lines="2 14-15 17-20 42-55"
25+
{% include "../../patterns/karpenter-mng/karpenter.tf" %}
2426
```
2527

26-
### EC2NodeClass and NodePool
27-
28-
```yaml hl_lines="8-16"
28+
```yaml hl_lines="9-17 28-29"
2929
{% include "../../patterns/karpenter-mng/karpenter.yaml" %}
3030
```
3131

32+
### VPC
33+
34+
```terraform hl_lines="21-22"
35+
{% include "../../patterns/karpenter-mng/vpc.tf" %}
36+
```
37+
3238
## Deploy
3339

3440
See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and steps to deploy this pattern.
@@ -40,9 +46,9 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
4046
```sh
4147
kubectl get nodes
4248

43-
NAME STATUS ROLES AGE VERSION
44-
ip-10-0-38-5.us-west-2.compute.internal Ready <none> 2m40s v1.29.3-eks-ae9a62a
45-
ip-10-0-9-38.us-west-2.compute.internal Ready <none> 2m35s v1.29.3-eks-ae9a62a
49+
NAME STATUS ROLES AGE VERSION
50+
ip-10-0-23-32.us-west-2.compute.internal Ready <none> 10m v1.30.4-eks-a737599
51+
ip-10-0-6-222.us-west-2.compute.internal Ready <none> 10m v1.30.4-eks-a737599
4652
```
4753

4854
2. Provision the Karpenter `EC2NodeClass` and `NodePool` resources which provide Karpenter the necessary configurations to provision EC2 resources:
@@ -69,9 +75,9 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
6975
kubectl get nodes
7076
7177
NAME STATUS ROLES AGE VERSION
72-
ip-10-0-38-109.us-west-2.compute.internal Ready <none> 11s v1.29.3-eks-ae9a62a # <== EC2 created by Karpenter
73-
ip-10-0-38-5.us-west-2.compute.internal Ready <none> 3m54s v1.29.3-eks-ae9a62a
74-
ip-10-0-9-38.us-west-2.compute.internal Ready <none> 3m49s v1.29.3-eks-ae9a62a
78+
ip-10-0-23-32.us-west-2.compute.internal Ready <none> 10m v1.30.4-eks-a737599
79+
ip-10-0-46-239.us-west-2.compute.internal Ready <none> 20s v1.30.1-eks-e564799 # <== EC2 created by Karpenter
80+
ip-10-0-6-222.us-west-2.compute.internal Ready <none> 10m v1.30.4-eks-a737599
7581
```
7682
7783
## Destroy
@@ -82,6 +88,12 @@ Scale down the deployment to de-provision Karpenter created resources first:
8288
kubectl delete -f example.yaml
8389
```
8490
91+
Remove the Karpenter Helm chart:
92+
93+
```sh
94+
terraform destroy -target=helm_release.karpenter --auto-approve
95+
```
96+
8597
{%
8698
include-markdown "../../docs/_partials/destroy.md"
8799
%}

‎patterns/karpenter-mng/eks.tf

Lines changed: 2 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
module "eks" {
66
source = "terraform-aws-modules/eks/aws"
7-
version = "~> 20.11"
7+
version = "~> 20.24"
88

99
cluster_name = local.name
1010
cluster_version = "1.30"
@@ -38,6 +38,7 @@ module "eks" {
3838

3939
eks_managed_node_groups = {
4040
karpenter = {
41+
ami_type = "BOTTLEROCKET_x86_64"
4142
instance_types = ["m5.large"]
4243

4344
min_size = 2
@@ -61,12 +62,6 @@ module "eks" {
6162
}
6263
}
6364

64-
# cluster_tags = merge(local.tags, {
65-
# NOTE - only use this option if you are using "attach_cluster_primary_security_group"
66-
# and you know what you're doing. In this case, you can remove the "node_security_group_tags" below.
67-
# "karpenter.sh/discovery" = local.name
68-
# })
69-
7065
node_security_group_tags = merge(local.tags, {
7166
# NOTE - if creating multiple security groups with this module, only tag the
7267
# security group that Karpenter should utilize with the following tag
@@ -81,59 +76,3 @@ output "configure_kubectl" {
8176
description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
8277
value = "aws eks --region ${local.region} update-kubeconfig --name ${module.eks.cluster_name}"
8378
}
84-
85-
################################################################################
86-
# Controller & Node IAM roles, SQS Queue, Eventbridge Rules
87-
################################################################################
88-
89-
module "karpenter" {
90-
source = "terraform-aws-modules/eks/aws//modules/karpenter"
91-
version = "~> 20.11"
92-
93-
cluster_name = module.eks.cluster_name
94-
95-
# Name needs to match role name passed to the EC2NodeClass
96-
node_iam_role_use_name_prefix = false
97-
node_iam_role_name = local.name
98-
create_pod_identity_association = true
99-
100-
tags = local.tags
101-
}
102-
103-
################################################################################
104-
# Helm charts
105-
################################################################################
106-
107-
resource "helm_release" "karpenter" {
108-
namespace = "kube-system"
109-
name = "karpenter"
110-
repository = "oci://public.ecr.aws/karpenter"
111-
repository_username = data.aws_ecrpublic_authorization_token.token.user_name
112-
repository_password = data.aws_ecrpublic_authorization_token.token.password
113-
chart = "karpenter"
114-
version = "0.36.2"
115-
wait = false
116-
117-
values = [
118-
<<-EOT
119-
nodeSelector:
120-
karpenter.sh/controller: 'true'
121-
tolerations:
122-
- key: CriticalAddonsOnly
123-
operator: Exists
124-
- key: karpenter.sh/controller
125-
operator: Exists
126-
effect: NoSchedule
127-
settings:
128-
clusterName: ${module.eks.cluster_name}
129-
clusterEndpoint: ${module.eks.cluster_endpoint}
130-
interruptionQueue: ${module.karpenter.queue_name}
131-
EOT
132-
]
133-
134-
lifecycle {
135-
ignore_changes = [
136-
repository_password
137-
]
138-
}
139-
}

‎patterns/karpenter-mng/karpenter.tf

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
locals {
2+
namespace = "karpenter"
3+
}
4+
5+
################################################################################
6+
# Controller & Node IAM roles, SQS Queue, Eventbridge Rules
7+
################################################################################
8+
9+
module "karpenter" {
10+
source = "terraform-aws-modules/eks/aws//modules/karpenter"
11+
version = "~> 20.24"
12+
13+
cluster_name = module.eks.cluster_name
14+
enable_v1_permissions = true
15+
namespace = local.namespace
16+
17+
# Name needs to match role name passed to the EC2NodeClass
18+
node_iam_role_use_name_prefix = false
19+
node_iam_role_name = local.name
20+
create_pod_identity_association = true
21+
22+
tags = local.tags
23+
}
24+
25+
################################################################################
26+
# Helm charts
27+
################################################################################
28+
29+
resource "helm_release" "karpenter" {
30+
name = "karpenter"
31+
namespace = local.namespace
32+
create_namespace = true
33+
repository = "oci://public.ecr.aws/karpenter"
34+
repository_username = data.aws_ecrpublic_authorization_token.token.user_name
35+
repository_password = data.aws_ecrpublic_authorization_token.token.password
36+
chart = "karpenter"
37+
version = "1.0.2"
38+
wait = false
39+
40+
values = [
41+
<<-EOT
42+
nodeSelector:
43+
karpenter.sh/controller: 'true'
44+
settings:
45+
clusterName: ${module.eks.cluster_name}
46+
clusterEndpoint: ${module.eks.cluster_endpoint}
47+
interruptionQueue: ${module.karpenter.queue_name}
48+
tolerations:
49+
- key: CriticalAddonsOnly
50+
operator: Exists
51+
- key: karpenter.sh/controller
52+
operator: Exists
53+
effect: NoSchedule
54+
webhook:
55+
enabled: false
56+
EOT
57+
]
58+
59+
lifecycle {
60+
ignore_changes = [
61+
repository_password
62+
]
63+
}
64+
}

‎patterns/karpenter-mng/karpenter.yaml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
---
2-
apiVersion: karpenter.k8s.aws/v1beta1
2+
apiVersion: karpenter.k8s.aws/v1
33
kind: EC2NodeClass
44
metadata:
55
name: default
66
spec:
7-
amiFamily: AL2
7+
amiSelectorTerms:
8+
- alias: bottlerocket@latest
89
role: ex-karpenter-mng
910
subnetSelectorTerms:
1011
- tags:
@@ -15,14 +16,16 @@ spec:
1516
tags:
1617
karpenter.sh/discovery: ex-karpenter-mng
1718
---
18-
apiVersion: karpenter.sh/v1beta1
19+
apiVersion: karpenter.sh/v1
1920
kind: NodePool
2021
metadata:
2122
name: default
2223
spec:
2324
template:
2425
spec:
2526
nodeClassRef:
27+
group: karpenter.k8s.aws
28+
kind: EC2NodeClass
2629
name: default
2730
requirements:
2831
- key: "karpenter.k8s.aws/instance-category"

‎patterns/karpenter/README.md

Lines changed: 40 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,22 +2,46 @@
22

33
This pattern demonstrates how to provision Karpenter on a serverless cluster (serverless data plane) using Fargate Profiles.
44

5+
## Code
6+
7+
The areas of significance related to this pattern are highlighted in the code provided below.
8+
9+
### Cluster
10+
11+
```terraform hl_lines="18-19 28-31 34-38 42-45"
12+
{% include "../../patterns/karpenter/eks.tf" %}
13+
```
14+
15+
### Karpenter Resources
16+
17+
```terraform hl_lines="2 14-15 17-19 21-24 46-55"
18+
{% include "../../patterns/karpenter/karpenter.tf" %}
19+
```
20+
21+
```yaml hl_lines="9-17 28-29"
22+
{% include "../../patterns/karpenter/karpenter.yaml" %}
23+
```
24+
25+
### VPC
26+
27+
```terraform hl_lines="21-22"
28+
{% include "../../patterns/karpenter/vpc.tf" %}
29+
```
30+
531
## Deploy
632

733
See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and steps to deploy this pattern.
834

935
## Validate
1036

11-
1. Test by listing the nodes in the cluster. You should see four Fargate nodes in the cluster:
37+
1. Test by listing the nodes in the cluster. You should see two Fargate nodes in the cluster:
1238

1339
```sh
1440
kubectl get nodes
1541

16-
NAME STATUS ROLES AGE VERSION
17-
fargate-ip-10-0-11-195.us-west-2.compute.internal Ready <none> 5m20s v1.28.2-eks-f8587cb
18-
fargate-ip-10-0-27-183.us-west-2.compute.internal Ready <none> 5m2s v1.28.2-eks-f8587cb
19-
fargate-ip-10-0-4-169.us-west-2.compute.internal Ready <none> 5m3s v1.28.2-eks-f8587cb
20-
fargate-ip-10-0-44-106.us-west-2.compute.internal Ready <none> 5m12s v1.28.2-eks-f8587cb
42+
NAME STATUS ROLES AGE VERSION
43+
fargate-ip-10-0-16-92.us-west-2.compute.internal Ready <none> 2m3s v1.30.0-eks-404b9c6
44+
fargate-ip-10-0-8-95.us-west-2.compute.internal Ready <none> 2m3s v1.30.0-eks-404b9c6
2145
```
2246

2347
2. Provision the Karpenter `EC2NodeClass` and `NodePool` resources which provide Karpenter the necessary configurations to provision EC2 resources:
@@ -43,12 +67,10 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
4367
```sh
4468
kubectl get nodes
4569
46-
NAME STATUS ROLES AGE VERSION
47-
fargate-ip-10-0-11-195.us-west-2.compute.internal Ready <none> 13m v1.28.2-eks-f8587cb
48-
fargate-ip-10-0-27-183.us-west-2.compute.internal Ready <none> 12m v1.28.2-eks-f8587cb
49-
fargate-ip-10-0-4-169.us-west-2.compute.internal Ready <none> 12m v1.28.2-eks-f8587cb
50-
fargate-ip-10-0-44-106.us-west-2.compute.internal Ready <none> 13m v1.28.2-eks-f8587cb
51-
ip-10-0-32-199.us-west-2.compute.internal Ready <none> 29s v1.28.2-eks-a5df82a # <== EC2 created by Karpenter
70+
NAME STATUS ROLES AGE VERSION
71+
fargate-ip-10-0-16-92.us-west-2.compute.internal Ready <none> 2m3s v1.30.0-eks-404b9c6
72+
fargate-ip-10-0-8-95.us-west-2.compute.internal Ready <none> 2m3s v1.30.0-eks-404b9c6
73+
ip-10-0-21-175.us-west-2.compute.internal Ready <none> 88s v1.30.1-eks-e564799 # <== EC2 created by Karpenter
5274
```
5375
5476
## Destroy
@@ -59,6 +81,12 @@ Scale down the deployment to de-provision Karpenter created resources first:
5981
kubectl delete -f example.yaml
6082
```
6183
84+
Remove the Karpenter Helm chart:
85+
86+
```sh
87+
terraform destroy -target=helm_release.karpenter --auto-approve
88+
```
89+
6290
{%
6391
include-markdown "../../docs/_partials/destroy.md"
6492
%}

0 commit comments

Comments
 (0)