Skip to content

Commit 2749046

Browse files
somtochiamahiddeco
somtochiama
and
hiddeco
committed
Support Azure Workload Identity
With an update to github.com/fluxcd/pkg/oci v0.22.0. This includes a pin of `github.com/docker/docker` to `v20.10.x`, to prevent Oras from complaining. Co-authored-by: Hidde Beydals <[email protected]> Signed-off-by: Somtochi Onyekwere <[email protected]>
1 parent fba4310 commit 2749046

File tree

7 files changed

+283
-78
lines changed

7 files changed

+283
-78
lines changed

‎docs/spec/v1beta2/buckets.md

Lines changed: 89 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,7 @@ Without a [Secret reference](#secret-reference), authentication using a chain
280280
with:
281281

282282
- [Environment credentials](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#EnvironmentCredential)
283+
- [Workload Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/[email protected]#WorkloadIdentityCredential)
283284
- [Managed Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ManagedIdentityCredential)
284285
with the `AZURE_CLIENT_ID`
285286
- Managed Identity with a system-assigned identity
@@ -436,16 +437,97 @@ data:
436437
accountKey: <BASE64>
437438
```
438439

439-
#### Managed Identity with AAD Pod Identity
440+
##### Workload Identity
440441

441-
If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs), you can create an identity that has access to Azure Storage.
442+
If you have [Workload Identity mutating webhook](https://azure.github.io/azure-workload-identity/docs/installation/managed-clusters.html)
443+
installed on your cluster. You would need to create an Azure Identity and
444+
give it access to Azure Blob Storage.
445+
446+
```shell
447+
export IDENTITY_NAME="blob-access"
448+
449+
az role assignment create --role "Storage Blob Data Reader" \
450+
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
451+
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
452+
```
453+
454+
Establish federated identity between the identity and the source-controller
455+
ServiceAccount.
456+
457+
```shell
458+
export SERVICE_ACCOUNT_ISSUER="$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER-NAME> --query "oidcIssuerProfile.issuerUrl" -otsv)"
459+
460+
az identity federated-credential create \
461+
--name "kubernetes-federated-credential" \
462+
--identity-name "${IDENTITY_NAME}" \
463+
--resource-group "${RESOURCE_GROUP}" \
464+
--issuer "${SERVICE_ACCOUNT_ISSUER}" \
465+
--subject "system:serviceaccount:flux-system:source-controller"
466+
```
467+
468+
Add a patch to label and annotate the source-controller Pods and ServiceAccount
469+
correctly so that it can match an identity binding:
470+
471+
```yaml
472+
apiVersion: kustomize.config.k8s.io/v1beta1
473+
kind: Kustomization
474+
resources:
475+
- gotk-components.yaml
476+
- gotk-sync.yaml
477+
patches:
478+
- patch: |-
479+
apiVersion: v1
480+
kind: ServiceAccount
481+
metadata:
482+
name: source-controller
483+
namespace: flux-system
484+
annotations:
485+
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
486+
labels:
487+
azure.workload.identity/use: "true"
488+
- patch: |-
489+
apiVersion: apps/v1
490+
kind: Deployment
491+
metadata:
492+
name: source-controller
493+
namespace: flux-system
494+
labels:
495+
azure.workload.identity/use: "true"
496+
spec:
497+
template:
498+
metadata:
499+
labels:
500+
azure.workload.identity/use: "true"
501+
```
502+
503+
If you have set Workload Identity up correctly and labeled the source-controller
504+
Pod and ServiceAccount, then you don't need to reference a Secret. For more information,
505+
please see [documentation](https://azure.github.io/azure-workload-identity/docs/quick-start.html).
506+
507+
```yaml
508+
apiVersion: source.toolkit.fluxcd.io/v1beta2
509+
kind: Bucket
510+
metadata:
511+
name: azure-bucket
512+
namespace: flux-system
513+
spec:
514+
interval: 5m0s
515+
provider: azure
516+
bucketName: testsas
517+
endpoint: https://testfluxsas.blob.core.windows.net
518+
```
519+
520+
##### Managed Identity with AAD Pod Identity
521+
522+
If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs),
523+
you can create an identity that has access to Azure Storage.
442524

443525
```sh
444526
export IDENTITY_NAME="blob-access"
445527
446-
az role assignment create --role "Storage Blob Data Contributor" \
447-
--assignee-object-id "$(az identity show -n blob-access -o tsv --query principalId -g $RESOURCE_GROUP)" \
448-
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/aks-somto/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
528+
az role assignment create --role "Storage Blob Data Reader" \
529+
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
530+
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
449531
450532
export IDENTITY_CLIENT_ID="$(az identity show -n ${IDENTITY_NAME} -g ${RESOURCE_GROUP} -otsv --query clientId)"
451533
export IDENTITY_RESOURCE_ID="$(az identity show -n ${IDENTITY_NAME} -otsv --query id)"
@@ -493,7 +575,8 @@ spec:
493575
aadpodidbinding: ${IDENTITY_NAME} # match the AzureIdentity name
494576
```
495577

496-
If you have set aad-pod-identity up correctly and labeled the source-controller pod, then you don't need to reference a secret.
578+
If you have set aad-pod-identity up correctly and labeled the source-controller
579+
pod, then you don't need to reference a secret.
497580

498581
```yaml
499582
apiVersion: source.toolkit.fluxcd.io/v1beta2

‎docs/spec/v1beta2/helmrepositories.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,9 +190,13 @@ The `aws` provider can be used to authenticate automatically using the EKS worke
190190
node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access
191191
to ECR.
192192

193+
##### EKS Worker Node IAM Role
194+
193195
When the worker node IAM role has access to ECR, source-controller running on it
194196
will also have access to ECR.
195197

198+
##### IAM Role for Service Accounts (IRSA)
199+
196200
When using IRSA to enable access to ECR, add the following patch to your bootstrap
197201
repository, in the `flux-system/kustomization.yaml` file:
198202

@@ -224,9 +228,56 @@ The `azure` provider can be used to authenticate automatically using kubelet man
224228
identity or Azure Active Directory pod-managed identity (aad-pod-identity), and
225229
by extension gain access to ACR.
226230

231+
##### Kubelet Managed Identity
232+
227233
When the kubelet managed identity has access to ACR, source-controller running on
228234
it will also have access to ACR.
229235

236+
##### Workload Identity
237+
238+
When using Workload Identity to enable access to ACR, add the following patch to
239+
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
240+
241+
```yaml
242+
apiVersion: kustomize.config.k8s.io/v1beta1
243+
kind: Kustomization
244+
resources:
245+
- gotk-components.yaml
246+
- gotk-sync.yaml
247+
patches:
248+
- patch: |-
249+
apiVersion: v1
250+
kind: ServiceAccount
251+
metadata:
252+
name: source-controller
253+
namespace: flux-system
254+
annotations:
255+
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
256+
labels:
257+
azure.workload.identity/use: "true"
258+
- patch: |-
259+
apiVersion: apps/v1
260+
kind: Deployment
261+
metadata:
262+
name: source-controller
263+
namespace: flux-system
264+
labels:
265+
azure.workload.identity/use: "true"
266+
spec:
267+
template:
268+
metadata:
269+
labels:
270+
azure.workload.identity/use: "true"
271+
```
272+
273+
To use Workload Identity, you have to install the Workload Identity
274+
mutating webhook and create an identity that has access to ACR. Next, establish
275+
a federated identity between the source-controller Service Account and the
276+
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
277+
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
278+
279+
##### AAD Pod Identity
280+
230281
When using aad-pod-identity to enable access to ACR, add the following patch to
231282
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
232283

@@ -261,9 +312,13 @@ if you want to use AKS pod-managed identities add-on that is in preview.
261312
The `gcp` provider can be used to authenticate automatically using OAuth scopes or
262313
Workload Identity, and by extension gain access to GCR or Artifact Registry.
263314

315+
#### Access Scopes
316+
264317
When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry,
265318
source-controller running on it will also have access to them.
266319

320+
#### Workload Identity
321+
267322
When using Workload Identity to enable access to GCR or Artifact Registry, add the
268323
following patch to your bootstrap repository, in the `flux-system/kustomization.yaml`
269324
file:

‎docs/spec/v1beta2/ocirepositories.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,9 +161,55 @@ The `azure` provider can be used to authenticate automatically using kubelet
161161
managed identity or Azure Active Directory pod-managed identity (aad-pod-identity),
162162
and by extension gain access to ACR.
163163

164+
##### Kubelet Managed Identity
165+
164166
When the kubelet managed identity has access to ACR, source-controller running
165167
on it will also have access to ACR.
166168

169+
##### Workload Identity
170+
171+
When using Workload Identity to enable access to ACR, add the following patch to
172+
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
173+
174+
```yaml
175+
apiVersion: kustomize.config.k8s.io/v1beta1
176+
kind: Kustomization
177+
resources:
178+
- gotk-components.yaml
179+
- gotk-sync.yaml
180+
patches:
181+
- patch: |-
182+
apiVersion: v1
183+
kind: ServiceAccount
184+
metadata:
185+
name: source-controller
186+
namespace: flux-system
187+
annotations:
188+
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
189+
labels:
190+
azure.workload.identity/use: "true"
191+
- patch: |-
192+
apiVersion: apps/v1
193+
kind: Deployment
194+
metadata:
195+
name: source-controller
196+
namespace: flux-system
197+
labels:
198+
azure.workload.identity/use: "true"
199+
spec:
200+
template:
201+
metadata:
202+
labels:
203+
azure.workload.identity/use: "true"
204+
```
205+
206+
To use Workload Identity, you have to install the Workload Identity
207+
mutating webhook and create an identity that has access to ACR. Next, establish
208+
a federated identity between the source-controller Service Account and the
209+
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
210+
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
211+
212+
##### AAD Pod Identity
167213
When using aad-pod-identity to enable access to ACR, add the following patch to
168214
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
169215

‎go.mod

Lines changed: 28 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,18 @@ replace github.com/fluxcd/source-controller/api => ./api
88
// xref: https://github.com/opencontainers/go-digest/pull/66
99
replace github.com/opencontainers/go-digest => github.com/opencontainers/go-digest v1.0.1-0.20220411205349-bde1400a84be
1010

11+
// Required to keep oras.land/oras-go happy, as it will otherwise fail with
12+
// "assignment mismatch: 3 variables but registry.PingV2Registry returns 2 values"
13+
//
14+
// Check again when oras.land/oras-go is updated to >=v2.0.0, which is a
15+
// dependency of Helm.
16+
replace github.com/docker/docker => github.com/docker/docker v20.10.23+incompatible
17+
1118
require (
1219
cloud.google.com/go/storage v1.30.1
1320
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230106234847-43070de90fa1
14-
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.4.0
15-
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.1
21+
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.5.0-beta.1
22+
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.0-beta.4
1623
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0
1724
github.com/Masterminds/semver/v3 v3.2.0
1825
github.com/cyphar/filepath-securejoin v0.2.3
@@ -28,7 +35,7 @@ require (
2835
github.com/fluxcd/pkg/helmtestserver v0.12.0
2936
github.com/fluxcd/pkg/lockedfile v0.1.0
3037
github.com/fluxcd/pkg/masktoken v0.2.0
31-
github.com/fluxcd/pkg/oci v0.21.1
38+
github.com/fluxcd/pkg/oci v0.22.0
3239
github.com/fluxcd/pkg/runtime v0.35.0
3340
github.com/fluxcd/pkg/sourceignore v0.3.3
3441
github.com/fluxcd/pkg/ssh v0.7.3
@@ -38,7 +45,7 @@ require (
3845
github.com/fluxcd/source-controller/api v0.36.1
3946
github.com/go-git/go-billy/v5 v5.4.1
4047
github.com/go-logr/logr v1.2.3
41-
github.com/google/go-containerregistry v0.13.0
48+
github.com/google/go-containerregistry v0.14.0
4249
github.com/google/go-containerregistry/pkg/authn/k8schain v0.0.0-20230307034325-57f010d26af8
4350
github.com/google/uuid v1.3.0
4451
github.com/minio/minio-go/v7 v7.0.50
@@ -85,7 +92,7 @@ require (
8592
github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
8693
github.com/Azure/go-autorest/logger v0.2.1 // indirect
8794
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
88-
github.com/AzureAD/microsoft-authentication-library-for-go v0.8.1 // indirect
95+
github.com/AzureAD/microsoft-authentication-library-for-go v0.9.0 // indirect
8996
github.com/BurntSushi/toml v1.2.1 // indirect
9097
github.com/MakeNowJust/heredoc v1.0.0 // indirect
9198
github.com/Masterminds/goutils v1.1.1 // indirect
@@ -109,19 +116,19 @@ require (
109116
github.com/alibabacloud-go/tea-xml v1.1.2 // indirect
110117
github.com/aliyun/credentials-go v1.2.3 // indirect
111118
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
112-
github.com/aws/aws-sdk-go-v2 v1.17.5 // indirect
113-
github.com/aws/aws-sdk-go-v2/config v1.18.15 // indirect
114-
github.com/aws/aws-sdk-go-v2/credentials v1.13.15 // indirect
115-
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23 // indirect
116-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.29 // indirect
117-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.23 // indirect
118-
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.30 // indirect
119-
github.com/aws/aws-sdk-go-v2/service/ecr v1.18.5 // indirect
119+
github.com/aws/aws-sdk-go-v2 v1.17.7 // indirect
120+
github.com/aws/aws-sdk-go-v2/config v1.18.19 // indirect
121+
github.com/aws/aws-sdk-go-v2/credentials v1.13.18 // indirect
122+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.1 // indirect
123+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.31 // indirect
124+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.25 // indirect
125+
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.32 // indirect
126+
github.com/aws/aws-sdk-go-v2/service/ecr v1.18.7 // indirect
120127
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.13.17 // indirect
121-
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.23 // indirect
122-
github.com/aws/aws-sdk-go-v2/service/sso v1.12.4 // indirect
123-
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4 // indirect
124-
github.com/aws/aws-sdk-go-v2/service/sts v1.18.5 // indirect
128+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.25 // indirect
129+
github.com/aws/aws-sdk-go-v2/service/sso v1.12.6 // indirect
130+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.6 // indirect
131+
github.com/aws/aws-sdk-go-v2/service/sts v1.18.7 // indirect
125132
github.com/aws/smithy-go v1.13.5 // indirect
126133
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20221004211355-a250ad2ca1e3 // indirect
127134
github.com/benbjohnson/clock v1.1.0 // indirect
@@ -144,7 +151,7 @@ require (
144151
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be // indirect
145152
github.com/containerd/containerd v1.6.18 // indirect
146153
github.com/containerd/continuity v0.3.0 // indirect
147-
github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
154+
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
148155
github.com/coreos/go-oidc/v3 v3.5.0 // indirect
149156
github.com/coreos/go-semver v0.3.0 // indirect
150157
github.com/coreos/go-systemd/v22 v22.3.2 // indirect
@@ -153,7 +160,7 @@ require (
153160
github.com/davecgh/go-spew v1.1.1 // indirect
154161
github.com/dimchansky/utfbom v1.1.1 // indirect
155162
github.com/docker/distribution v2.8.1+incompatible // indirect
156-
github.com/docker/docker v20.10.21+incompatible // indirect
163+
github.com/docker/docker v23.0.1+incompatible // indirect
157164
github.com/docker/docker-credential-helpers v0.7.0 // indirect
158165
github.com/docker/go-connections v0.4.0 // indirect
159166
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
@@ -198,7 +205,7 @@ require (
198205
github.com/gofrs/uuid v4.2.0+incompatible // indirect
199206
github.com/gogo/protobuf v1.3.2 // indirect
200207
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
201-
github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
208+
github.com/golang-jwt/jwt/v4 v4.4.3 // indirect
202209
github.com/golang/glog v1.0.0 // indirect
203210
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
204211
github.com/golang/mock v1.6.0 // indirect
@@ -233,7 +240,7 @@ require (
233240
github.com/huandu/xstrings v1.4.0 // indirect
234241
github.com/imdario/mergo v0.3.13 // indirect
235242
github.com/in-toto/in-toto-golang v0.3.4-0.20220709202702-fa494aaa0add // indirect
236-
github.com/inconshreveable/mousetrap v1.0.1 // indirect
243+
github.com/inconshreveable/mousetrap v1.1.0 // indirect
237244
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
238245
github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
239246
github.com/jhump/protoreflect v1.14.0 // indirect

0 commit comments

Comments
 (0)