How to rotate karmada certificate if it is expired

[chaosi-zju]

Please provide an in-depth description of the question you have:

In many installation methods, the expiration time of the Karmada certificates is 365 days, when the time is up, most components will crash, just like:

$ kubectl get pod
NAME                                               READY   STATUS             RESTARTS            AGE
etcd-0                                             1/1     Running            22688 (38s ago)     569d
karmada-apiserver-6dd844fdfd-stzzg                 0/1     CrashLoopBackOff   62176 (4m54s ago)   569d
karmada-controller-manager-7dbf7c6578-kmztg        0/1     CrashLoopBackOff   57450 (4m9s ago)    326d
karmada-kube-controller-manager-656cdc675f-cj6vw   0/1     CrashLoopBackOff   57286 (3m42s ago)   569d
karmada-scheduler-764fbdcd6d-6jhrd                 1/1     Running            1                   569d
karmada-webhook-6489787db4-wc9pp                   1/1     Running            0                   569d

$ kubectl logs -f karmada-apiserver-6dd844fdfd-stzzg
W0328 02:20:08.682680       1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://etcd-client.karmada-system.svc.cluster.local:2379/  <nil> 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid: current time 2024-03-28T02:20:08Z is after 2023-09-05T06:51:02Z". Reconnecting...

$ kubectl logs -f karmada-controller-manager-7dbf7c6578-kmztg
E0328 02:26:10.815013       1 controllermanager.go:78] failed to build controller manager: Get "https://karmada-apiserver.karmada-system.svc.cluster.local:5443/api?timeout=32s": dial tcp 10.254.52.163:5443: connect: connection refused
Get "https://karmada-apiserver.karmada-system.svc.cluster.local:5443/api?timeout=32s": dial tcp 10.254.52.163:5443: connect: connection refused

$ kubectl logs -f karmada-kube-controller-manager-656cdc675f-cj6vw
I0328 02:26:42.065358       1 dynamic_cafile_content.go:129] Loaded a new CA Bundle and Verifier for "client-ca-bundle::/etc/karmada/pki/server-ca.crt"
unable to load configmap based request-header-client-ca-file: Get "https://karmada-apiserver.karmada-system.svc.cluster.local:5443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication": dial tcp 10.254.52.163:5443: connect: connection refused

so, how to rotate karmada certificate if it is expired?

How to reproduce it?:

Step 1. modify hack/util.sh:

karmada/hack/util.sh

Lines 216 to 217 in c81649a

	${OPENSSL_BIN} req -x509 -sha256 -new -nodes -days 3650 -newkey rsa:2048 -keyout "${dest_dir}/${id}.key" -out "${dest_dir}/${id}.crt" -subj "/CN=${cn}/"
	echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment",${purpose}]}}}' > "${dest_dir}/${id}-config.json"

modify -days 365 to -days 1 in Line 216, so that default CA cert expiration time change to 1 day,
modify "expiry":"43800h" to "expiry":"1h" in Line 217, so that default karmada-apiserver cert expiration time change to 1 hour,
just like:

  ${OPENSSL_BIN} req -x509 -sha256 -new -nodes -days 1 -newkey rsa:2048 -keyout "${dest_dir}/${id}.key" -out "${dest_dir}/${id}.crt" -subj "/CN=${cn}/"
  echo '{"signing":{"default":{"expiry":"1h","usages":["signing","key encipherment",${purpose}]}}}' > "${dest_dir}/${id}-config.json"

Step 2. re-install karmada by hack/local-up-karmada.sh

Step 3. 1 hour later, the most component would crash down.

Environment:

• Karmada version:
• Kubernetes version:
• Others:

[pptfz]

Is there a solution to this problem?

[RainbowMango]

Hope we can get an answer by karmada-io/community#69.