Skip to content

Commit cdf1869

Browse files
author
bcm820
authored
e2e: Replace Kuttl-in-Kuttl (#131)
1 parent 43af4d4 commit cdf1869

9 files changed

+111
-163
lines changed

‎e2e/Makefile

-22
Original file line numberDiff line numberDiff line change
@@ -14,28 +14,6 @@ _runThisTest:
1414
cp -r $(PWD) $$D/suit ;\
1515
ROOT_DIR=$(ROOT_DIR) KUBECONFIG="$(ROOT_DIR)/kubeconfig" kubectl-kuttl test --timeout 300 --skip-delete "$$D/suit"
1616

17-
runTestSuit:
18-
@T="$$(KUBECONFIG="$$ROOT_DIR/kubeconfig" kubectl-kuttl test --timeout 300 --skip-delete --namespace "$$NAMESPACE" "$$TS" 2>&1)" ;\
19-
echo "$$T" |\
20-
grep -v harness.go |\
21-
grep -v ^=== |\
22-
grep -v "running without a 'kuttl-test.yaml" |\
23-
grep -v "creation of user-supplied namespace" |\
24-
grep -v "not match file name regexp" |\
25-
grep -v "kutt-test config testdirs is overridden" ;\
26-
echo "$$T" | \
27-
grep '^PASS' || \
28-
(echo "$$T" | grep harness.go && find $$TS -name *.yaml -exec cat {} \; && exit 1)
29-
30-
renderTestCase:
31-
@D="$$(mktemp -d)" ;\
32-
mkdir -p "$$D/case" ;\
33-
envsubst -i "$$TPL" -o "$$D/case/00-case.yaml" ;\
34-
printf "$$D\n"
35-
36-
renderManifest:
37-
@echo $(shell make --no-print-directory renderTestCase)/case/00-case.yaml
38-
3917
getKubeUid:
4018
@kubectl get -o jsonpath='{.metadata.uid}' -n "$$NAMESPACE" "$$OBJ"
4119

‎e2e/README.MD

+21-51
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,38 @@
11
# E2E Framework
22

3-
The E2E framework uses Kuttl under the hood. Kuttl is a simple and most importantly static tool.
4-
That means there is no way to dynamically generate manifests.
5-
The solution is KIK (Kuttl-In-Kuttl)!
3+
The E2E framework uses Kuttl under the hood. Kuttl is a simple and most importantly static tool that handles applying manifests for running assertions in a namespace it generates for each test.
64

7-
## Kuttl-In-Kuttl
5+
In order to dynamically generate manifests, Kuttl supports declaring a `TestStep` manifest which includes an option for running scripts. This allows us apply manifests with dynamic values from stdin.
86

9-
## How does it work
7+
Additionally, helper functions can be invoked in the scripts by creating a Makefile in each test folder that references `../../Makefile` (i.e. `e2e/Makefile`).
108

11-
- Instead of creating a static Kuttl manifest, you can create template. Template extension must be `.yml` otherwise Kuttl starts executing it.
12-
- Template is rendered via `envsubst`, so you can use Bash variables for dynamic data injection.
13-
- Variables always present
14-
- `${KUBECONFIG}`
15-
- `${NAMESPACE}`
16-
- `${ROOT_DIR}`
9+
## Example TestStep
1710

18-
Example template
19-
```yaml
20-
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
21-
kind: LinodeMachine
22-
metadata:
23-
ownerReferences:
24-
- apiVersion: cluster.x-k8s.io/v1beta1
25-
kind: Machine
26-
name: machine-sample
27-
uid: ${MACHINE_UID}
28-
name: linodemachine-sample
29-
spec:
30-
region: us-sea
31-
type: g5-nanode-1
32-
```
33-
34-
- Helper tools are available, please create a Makefile in your test folder. In this way you can use built in helpers without including any script in the test, plus this file is a good place for test specific helpers.
35-
36-
Example Makefile
37-
```makefile
38-
include ../../Makefile
39-
```
40-
41-
- Template is nothing without a runner, please use helpers provided in Makefile to render and run template.
42-
43-
Example template runner
4411
```yaml
4512
apiVersion: kuttl.dev/v1beta1
4613
kind: TestStep
4714
commands:
4815
- script: |-
49-
MACHINE_UID="$(OBJ=machines/machine-sample make getKubeUid)" \
50-
TS="$(TPL="$PWD/02-create-linodemachine.tpl.yml" make renderTestCase)" make runTestSuit
16+
cat <<EOF | kubectl apply -n $NAMESPACE -f -
17+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
18+
kind: LinodeMachine
19+
metadata:
20+
ownerReferences:
21+
- apiVersion: cluster.x-k8s.io/v1beta1
22+
kind: Machine
23+
name: machine-sample
24+
uid: $(OBJ=machines/machine-sample make getKubeUid)
25+
name: linodemachine-sample
26+
spec:
27+
region: us-sea
28+
type: g5-nanode-1
29+
EOF
5130
```
5231
53-
- You can run testsuits (which can contain template rendering).
54-
55-
Example testsuit runner:
56-
```yaml
57-
apiVersion: kuttl.dev/v1beta1
58-
kind: TestStep
59-
commands:
60-
- script: TS="$PWD/testsuit" make runTestSuit
61-
```
32+
## Test Execution
6233
63-
- Test execution order is not guaranteed. If a test depends on another resource it is strongly suggested to create `assert` files to wait for an event. Almost every test depends on CAPI providers, so there is a good chance, you have to create a `00-assert.yaml` in your test folder.
34+
Note that test execution order is not guaranteed. If a test depends on another resource it is strongly suggested to create `assert` files to wait for an event. Almost every test depends on CAPI providers, so there is a good chance you have to create a `00-assert.yaml` in your test folder that asserts the following:
6435

65-
Example `00-assert.yaml`:
6636
```yaml
6737
apiVersion: apps/v1
6838
kind: Deployment
@@ -81,7 +51,7 @@ status:
8151
availableReplicas: 1
8252
```
8353

84-
## Executing individual test suit
54+
## Executing individual tests
8555

8656
```bash
8757
make _e2etest-infra # Only once per cluster!

‎e2e/linodemachine-controller/byovpc/01-create-cluster.tpl.yml

-48
This file was deleted.

‎e2e/linodemachine-controller/byovpc/01-create-cluster.yaml

+62-4
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,67 @@ kind: TestStep
33
commands:
44
- command: make createVPC
55
- script: |-
6-
VPC_ID=$(make fetchVPCID) \
7-
TS="$(TPL="$PWD/01-create-vpc.tpl.yml" make renderTestCase)" make runTestSuit
6+
cat <<EOF | kubectl apply -n $NAMESPACE -f -
7+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
8+
kind: LinodeVPC
9+
metadata:
10+
annotations:
11+
cluster.x-k8s.io/paused: "true"
12+
name: linodevpc-sample
13+
spec:
14+
label: capli-e2e-byovpc-for-all-tests
15+
region: us-sea
16+
vpcID: $(make fetchVPCID)
17+
EOF
818
- command: make enableVPC
919
- script: |-
10-
VPC_UID="$(OBJ=linodevpcs/linodevpc-sample make getKubeUid)" \
11-
TS="$(TPL="$PWD/01-create-cluster.tpl.yml" make renderTestCase)" make runTestSuit
20+
cat <<EOF | kubectl apply -n $NAMESPACE -f -
21+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
22+
kind: LinodeCluster
23+
metadata:
24+
annotations:
25+
cluster.x-k8s.io/paused: "true"
26+
name: linodecluster-sample
27+
spec:
28+
region: us-sea
29+
vpcRef:
30+
kind: LinodeVPC
31+
name: linodevpc-sample
32+
namespace: $NAMESPACE
33+
uid: $(OBJ=linodevpcs/linodevpc-sample make getKubeUid)
34+
---
35+
apiVersion: cluster.x-k8s.io/v1beta1
36+
kind: Cluster
37+
metadata:
38+
annotations:
39+
cluster.x-k8s.io/paused: "true"
40+
name: cluster-sample
41+
spec:
42+
paused: true
43+
infrastructureRef:
44+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
45+
kind: LinodeCluster
46+
name: linodecluster-sample
47+
---
48+
apiVersion: cluster.x-k8s.io/v1beta1
49+
kind: Machine
50+
metadata:
51+
annotations:
52+
cluster.x-k8s.io/paused: "true"
53+
name: machine-sample
54+
spec:
55+
clusterName: cluster-sample
56+
bootstrap:
57+
configRef:
58+
apiVersion: v1
59+
kind: "ConfigMap"
60+
name: "boostrap-sample"
61+
dataSecretName: bootstrap-data-sample
62+
---
63+
apiVersion: v1
64+
kind: Secret
65+
metadata:
66+
name: bootstrap-data-sample
67+
data:
68+
value: dG91Y2ggL29rCg==
69+
EOF

‎e2e/linodemachine-controller/byovpc/01-create-vpc.tpl.yml

-10
This file was deleted.

‎e2e/linodemachine-controller/byovpc/02-create-linodemachine.tpl.yml

-12
This file was deleted.

‎e2e/linodemachine-controller/byovpc/02-create-linodemachine.yaml

+14-2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,17 @@ apiVersion: kuttl.dev/v1beta1
22
kind: TestStep
33
commands:
44
- script: |-
5-
MACHINE_UID="$(OBJ=machines/machine-sample make getKubeUid)" \
6-
TS="$(TPL="$PWD/02-create-linodemachine.tpl.yml" make renderTestCase)" make runTestSuit
5+
cat <<EOF | kubectl apply -n $NAMESPACE -f -
6+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
7+
kind: LinodeMachine
8+
metadata:
9+
ownerReferences:
10+
- apiVersion: cluster.x-k8s.io/v1beta1
11+
kind: Machine
12+
name: machine-sample
13+
uid: $(OBJ=machines/machine-sample make getKubeUid)
14+
name: linodemachine-sample
15+
spec:
16+
region: us-sea
17+
type: g5-nanode-1
18+
EOF

‎e2e/linodemachine-controller/minimal/02-create-linodemachine.tpl.yml

-12
This file was deleted.

‎e2e/linodemachine-controller/minimal/02-create-linodemachine.yaml

+14-2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,17 @@ apiVersion: kuttl.dev/v1beta1
22
kind: TestStep
33
commands:
44
- script: |-
5-
MACHINE_UID="$(OBJ=machines/machine-sample make getKubeUid)" \
6-
TS="$(TPL="$PWD/02-create-linodemachine.tpl.yml" make renderTestCase)" make runTestSuit
5+
cat <<EOF | kubectl apply -n $NAMESPACE -f -
6+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
7+
kind: LinodeMachine
8+
metadata:
9+
ownerReferences:
10+
- apiVersion: cluster.x-k8s.io/v1beta1
11+
kind: Machine
12+
name: machine-sample
13+
uid: $(OBJ=machines/machine-sample make getKubeUid)
14+
name: linodemachine-sample
15+
spec:
16+
region: us-sea
17+
type: g5-nanode-1
18+
EOF

0 commit comments

Comments
 (0)