diff --git a/CHANGELOG/CHANGELOG-v1.2.0-b3.md b/CHANGELOG/CHANGELOG-v1.2.0-b3.md index 94375dd5fe..f1ffe6370c 100644 --- a/CHANGELOG/CHANGELOG-v1.2.0-b3.md +++ b/CHANGELOG/CHANGELOG-v1.2.0-b3.md @@ -1,3 +1,3 @@ # Flyte v1.2.0-b3 Changelog -Allow adding sidecar containers to flyteadmin +Allow adding uploader containers to flyteadmin diff --git a/CHANGELOG/CHANGELOG-v1.3.0-b5.md b/CHANGELOG/CHANGELOG-v1.3.0-b5.md index 81ae2e1233..7a47eaa89b 100644 --- a/CHANGELOG/CHANGELOG-v1.3.0-b5.md +++ b/CHANGELOG/CHANGELOG-v1.3.0-b5.md @@ -1,20 +1,23 @@ - # Flyte v1.3.0-b5 Changelog This pulls in Databricks support. Please see the [GH issue](https://github.com/flyteorg/flyte/issues/3173) for a listing of the relevant PRs. There are other changes included in this beta release as well. ## Try it out locally + You can try out these changes "locally", as they've been included in the `flytectl demo` image for this beta release, but since the demo cluster is meant to be an isolated, local-only cluster, you'll have to make some changes to get it to talk to a live databricks account. You'll also need to configure access to a real S3 bucket (as opposed to Minio, which is what the demo local cluster typically relies on). ### S3 Setup + Follow the [AWS instructions](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html) for generating access and secret keys that can be used to hit your S3 bucket of choice. ### Flyte Demo Cluster + #### Starting the cluster + Run flytectl demo start with the image argument ```bash @@ -22,87 +25,95 @@ flytectl demo start --image ghcr.io/flyteorg/flyte-sandbox-bundled:sha-e240038be ``` #### Configure the Demo Cluster + 1. Update the Flyte configmap - ```bash - kubectl -n flyte edit cm sandbox-flyte-binary-config - ``` - 1. Update the `003-storage.yaml` section - Make the storage section look like the following. You should update the propeller `rawoutput-prefix` setting as well. + ```bash + kubectl -n flyte edit cm sandbox-flyte-binary-config + ``` +1. Update the `003-storage.yaml` section + Make the storage section look like the following. You should update the propeller `rawoutput-prefix` setting as well. + ``` + storage: + type: s3 + container: "your-bucket" + stow: + kind: s3 + config: + access_key_id: AKIAYOURKEY + auth_type: accesskey + secret_key: YOUR+SECRET + disable_ssl: true + region: us-east-2 + ``` +1. Update the `010-inline-config.yaml` section + + 1. Under the existing `plugins` section, as a sibling to `k8s`, add + ``` + databricks: + databricksInstance: dbc-abc-123.cloud.databricks.com + entrypointFile: dbfs:///FileStore/tables/entrypoint.py + ``` + 2. In the `k8s` section, update the `default-env-vars` section + + ``` + - FLYTE_AWS_ACCESS_KEY_ID: AKIAYOURKEY + - AWS_DEFAULT_REGION: us-east-2 + - FLYTE_AWS_SECRET_ACCESS_KEY: YOUR+SECRET + ``` + + These are the same values as in the storage section above. + + 3. Add in an section for data proxy ``` - storage: - type: s3 - container: "your-bucket" - stow: - kind: s3 - config: - access_key_id: AKIAYOURKEY - auth_type: accesskey - secret_key: YOUR+SECRET - disable_ssl: true - region: us-east-2 + remoteData: + region: us-east-2 + scheme: aws + signedUrls: + durationMinutes: 3 + ``` + 4. Enable databricks plugin + + ```shell + task-plugins: + default-for-task-types: + container: container + container_array: k8s-array + uploader: uploader + ray: ray + spark: databricks + enabled-plugins: + - container + - databricks + - ray + - uploader + - k8s-array + ``` - 1. Update the `010-inline-config.yaml` section - 1. Under the existing `plugins` section, as a sibling to `k8s`, add - ``` - databricks: - databricksInstance: dbc-abc-123.cloud.databricks.com - entrypointFile: dbfs:///FileStore/tables/entrypoint.py - ``` - 2. In the `k8s` section, update the `default-env-vars` section - ``` - - FLYTE_AWS_ACCESS_KEY_ID: AKIAYOURKEY - - AWS_DEFAULT_REGION: us-east-2 - - FLYTE_AWS_SECRET_ACCESS_KEY: YOUR+SECRET - ``` - These are the same values as in the storage section above. - - 3. Add in an section for data proxy - ``` - remoteData: - region: us-east-2 - scheme: aws - signedUrls: - durationMinutes: 3 - ``` - 4. Enable databricks plugin - ```shell - task-plugins: - default-for-task-types: - container: container - container_array: k8s-array - sidecar: sidecar - ray: ray - spark: databricks - enabled-plugins: - - container - - databricks - - ray - - sidecar - - k8s-array - - ``` 1. Update the Flyte deployment + ``` kubectl -n flyte edit deploy sandbox-flyte-binary ``` - + Add an environment variable for your databricks token to the flyte pod + ``` - name: FLYTE_SECRET_FLYTE_DATABRICKS_API_TOKEN value: dapixyzxyzxyz - ``` - + ``` + 1. Restart the deployment ``` kubectl -n flyte rollout restart deploy sandbox-flyte-binary ``` ### Databricks Code -You'll need to upload an [entrypoint](https://gist.github.com/pingsutw/482e7f0134414dac437500344bac5134) file to your dbfs (or S3). This is the referenced gist from the primary [Databricks plugin documentation](https://github.com/flyteorg/flyte/blob/master/docs/deployment/plugin_setup/webapi/databricks.rst) as well, which currently only covers the `flyte-core` Helm chart installation. +You'll need to upload an [entrypoint](https://gist.github.com/pingsutw/482e7f0134414dac437500344bac5134) file to your dbfs (or S3). This is the referenced gist from the primary [Databricks plugin documentation](https://github.com/flyteorg/flyte/blob/master/docs/deployment/plugin_setup/webapi/databricks.rst) as well, which currently only covers the `flyte-core` Helm chart installation. ### User Code + 1. a sample py file that has a simple spark task. ```python @@ -194,7 +205,9 @@ if __name__ == "__main__": ) ``` + 2. Build a custom image for spark clusters + ```dockerfile FROM databricksruntime/standard:11.3-LTS ENV PATH $PATH:/databricks/python3/bin @@ -205,13 +218,15 @@ RUN /databricks/python3/bin/pip install awscli flytekitplugins-spark==v1.3.0b5 # Copy the actual code COPY ./ /databricks/driver ``` + 3. image building command if necessary. + ```shell docker build -t pingsutw/databricks:test -f Dockerfile . ``` + 4. pyflyte command to register the flyte workflow and task. ```shell pyflyte --config ~/.flyte/config-sandbox.yaml register --destination-dir . --image pingsutw/databricks:test databricks.py ``` - diff --git a/charts/flyte-binary/README.md b/charts/flyte-binary/README.md index 9ab746d48d..7d90951a50 100644 --- a/charts/flyte-binary/README.md +++ b/charts/flyte-binary/README.md @@ -6,164 +6,163 @@ Chart for basic single Flyte executable deployment ## Requirements -| Repository | Name | Version | -|------------|------|---------| +| Repository | Name | Version | +| -------------------- | ---------------------- | ------- | | file://../flyteagent | flyteagent(flyteagent) | v0.1.10 | ## Values -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| clusterResourceTemplates.annotations | object | `{}` | | -| clusterResourceTemplates.externalConfigMap | string | `""` | | -| clusterResourceTemplates.inline | object | `{}` | | -| clusterResourceTemplates.inlineConfigMap | string | `""` | | -| clusterResourceTemplates.labels | object | `{}` | | -| commonAnnotations | object | `{}` | | -| commonLabels | object | `{}` | | -| configuration.agentService.defaultAgent.defaultTimeout | string | `"10s"` | | -| configuration.agentService.defaultAgent.endpoint | string | `"dns:///flyteagent.flyte.svc.cluster.local:8000"` | | -| configuration.agentService.defaultAgent.insecure | bool | `true` | | -| configuration.agentService.defaultAgent.timeouts.GetTask | string | `"10s"` | | -| configuration.annotations | object | `{}` | | -| configuration.auth.authorizedUris | list | `[]` | | -| configuration.auth.clientSecretsExternalSecretRef | string | `""` | | -| configuration.auth.enableAuthServer | bool | `true` | | -| configuration.auth.enabled | bool | `false` | | -| configuration.auth.flyteClient.audience | string | `""` | | -| configuration.auth.flyteClient.clientId | string | `"flytectl"` | | -| configuration.auth.flyteClient.redirectUri | string | `"http://localhost:53593/callback"` | | -| configuration.auth.flyteClient.scopes[0] | string | `"all"` | | -| configuration.auth.internal.clientId | string | `"flytepropeller"` | | -| configuration.auth.internal.clientSecret | string | `""` | | -| configuration.auth.internal.clientSecretHash | string | `""` | | -| configuration.auth.oidc.baseUrl | string | `""` | | -| configuration.auth.oidc.clientId | string | `""` | | -| configuration.auth.oidc.clientSecret | string | `""` | | -| configuration.co-pilot.image.repository | string | `"cr.flyte.org/flyteorg/flytecopilot"` | | -| configuration.co-pilot.image.tag | string | `"v1.13.2"` | | -| configuration.database.dbname | string | `"flyte"` | | -| configuration.database.host | string | `"127.0.0.1"` | | -| configuration.database.options | string | `"sslmode=disable"` | | -| configuration.database.password | string | `""` | | -| configuration.database.passwordPath | string | `""` | | -| configuration.database.port | int | `5432` | | -| configuration.database.username | string | `"postgres"` | | -| configuration.externalConfigMap | string | `""` | | -| configuration.externalSecretRef | string | `""` | | -| configuration.inline | object | `{}` | | -| configuration.inlineConfigMap | string | `""` | | -| configuration.inlineSecretRef | string | `""` | | -| configuration.labels | object | `{}` | | -| configuration.logging.level | int | `1` | | -| configuration.logging.plugins.cloudwatch.enabled | bool | `false` | | -| configuration.logging.plugins.cloudwatch.templateUri | string | `""` | | -| configuration.logging.plugins.custom | list | `[]` | | -| configuration.logging.plugins.kubernetes.enabled | bool | `false` | | -| configuration.logging.plugins.kubernetes.templateUri | string | `""` | | -| configuration.logging.plugins.stackdriver.enabled | bool | `false` | | -| configuration.logging.plugins.stackdriver.templateUri | string | `""` | | -| configuration.propeller.createCRDs | bool | `true` | | -| configuration.storage.metadataContainer | string | `"my-organization-flyte-container"` | | -| configuration.storage.provider | string | `"s3"` | | -| configuration.storage.providerConfig.azure.account | string | `"storage-account-name"` | | -| configuration.storage.providerConfig.azure.configDomainSuffix | string | `""` | | -| configuration.storage.providerConfig.azure.configUploadConcurrency | int | `4` | | -| configuration.storage.providerConfig.azure.key | string | `""` | | -| configuration.storage.providerConfig.gcs.project | string | `"my-organization-gcp-project"` | | -| configuration.storage.providerConfig.s3.accessKey | string | `""` | | -| configuration.storage.providerConfig.s3.authType | string | `"iam"` | | -| configuration.storage.providerConfig.s3.disableSSL | bool | `false` | | -| configuration.storage.providerConfig.s3.endpoint | string | `""` | | -| configuration.storage.providerConfig.s3.region | string | `"us-east-1"` | | -| configuration.storage.providerConfig.s3.secretKey | string | `""` | | -| configuration.storage.providerConfig.s3.v2Signing | bool | `false` | | -| configuration.storage.userDataContainer | string | `"my-organization-flyte-container"` | | -| deployment.annotations | object | `{}` | | -| deployment.args | list | `[]` | | -| deployment.command | list | `[]` | | -| deployment.extraEnvVars | list | `[]` | | -| deployment.extraEnvVarsConfigMap | string | `""` | | -| deployment.extraEnvVarsSecret | string | `""` | | -| deployment.extraPodSpec | object | `{}` | | -| deployment.extraVolumeMounts | list | `[]` | | -| deployment.extraVolumes | list | `[]` | | -| deployment.genAdminAuthSecret.args | list | `[]` | | -| deployment.genAdminAuthSecret.command | list | `[]` | | -| deployment.image.pullPolicy | string | `"IfNotPresent"` | | -| deployment.image.repository | string | `"cr.flyte.org/flyteorg/flyte-binary"` | | -| deployment.image.tag | string | `"latest"` | | -| deployment.initContainers | list | `[]` | | -| deployment.labels | object | `{}` | | -| deployment.lifecycleHooks | object | `{}` | | -| deployment.livenessProbe | object | `{}` | | -| deployment.podAnnotations | object | `{}` | | -| deployment.podLabels | object | `{}` | | -| deployment.podSecurityContext.enabled | bool | `false` | | -| deployment.podSecurityContext.fsGroup | int | `65534` | | -| deployment.podSecurityContext.runAsGroup | int | `65534` | | -| deployment.podSecurityContext.runAsUser | int | `65534` | | -| deployment.readinessProbe | object | `{}` | | -| deployment.sidecars | list | `[]` | | -| deployment.startupProbe | object | `{}` | | -| deployment.waitForDB.args | list | `[]` | | -| deployment.waitForDB.command | list | `[]` | | -| deployment.waitForDB.image.pullPolicy | string | `"IfNotPresent"` | | -| deployment.waitForDB.image.repository | string | `"postgres"` | | -| deployment.waitForDB.image.tag | string | `"15-alpine"` | | -| enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | -| enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | -| enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins | -| flyte-core-components.admin.disableClusterResourceManager | bool | `false` | | -| flyte-core-components.admin.disableScheduler | bool | `false` | | -| flyte-core-components.admin.disabled | bool | `false` | | -| flyte-core-components.admin.seedProjectsWithDetails[0].description | string | `"Default project setup."` | | -| flyte-core-components.admin.seedProjectsWithDetails[0].name | string | `"flytesnacks"` | | -| flyte-core-components.admin.seedProjects[0] | string | `"flytesnacks"` | | -| flyte-core-components.dataCatalog.disabled | bool | `false` | | -| flyte-core-components.propeller.disableWebhook | bool | `false` | | -| flyte-core-components.propeller.disabled | bool | `false` | | -| flyteagent.enabled | bool | `false` | | -| fullnameOverride | string | `""` | | -| ingress.commonAnnotations | object | `{}` | | -| ingress.create | bool | `false` | | -| ingress.grpcAnnotations | object | `{}` | | -| ingress.grpcExtraPaths.append | list | `[]` | | -| ingress.grpcExtraPaths.prepend | list | `[]` | | -| ingress.grpcIngressClassName | string | `""` | | -| ingress.grpcTls | list | `[]` | | -| ingress.host | string | `""` | | -| ingress.httpAnnotations | object | `{}` | | -| ingress.httpExtraPaths.append | list | `[]` | | -| ingress.httpExtraPaths.prepend | list | `[]` | | -| ingress.httpIngressClassName | string | `""` | | -| ingress.httpTls | list | `[]` | | -| ingress.ingressClassName | string | `""` | | -| ingress.labels | object | `{}` | | -| ingress.separateGrpcIngress | bool | `true` | | -| ingress.tls | list | `[]` | | -| nameOverride | string | `""` | | -| rbac.annotations | object | `{}` | | -| rbac.create | bool | `true` | | -| rbac.extraRules | list | `[]` | | -| rbac.labels | object | `{}` | | -| service.clusterIP | string | `""` | | -| service.commonAnnotations | object | `{}` | | -| service.externalTrafficPolicy | string | `"Cluster"` | | -| service.extraPorts | list | `[]` | | -| service.grpcAnnotations | object | `{}` | | -| service.httpAnnotations | object | `{}` | | -| service.labels | object | `{}` | | -| service.loadBalancerIP | string | `""` | | -| service.loadBalancerSourceRanges | list | `[]` | | -| service.nodePorts.grpc | string | `""` | | -| service.nodePorts.http | string | `""` | | -| service.ports.grpc | string | `""` | | -| service.ports.http | string | `""` | | -| service.type | string | `"ClusterIP"` | | -| serviceAccount.annotations | object | `{}` | | -| serviceAccount.create | bool | `true` | | -| serviceAccount.imagePullSecrets | list | `[]` | | -| serviceAccount.labels | object | `{}` | | -| serviceAccount.name | string | `""` | | - +| Key | Type | Default | Description | +| ------------------------------------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | +| clusterResourceTemplates.annotations | object | `{}` | | +| clusterResourceTemplates.externalConfigMap | string | `""` | | +| clusterResourceTemplates.inline | object | `{}` | | +| clusterResourceTemplates.inlineConfigMap | string | `""` | | +| clusterResourceTemplates.labels | object | `{}` | | +| commonAnnotations | object | `{}` | | +| commonLabels | object | `{}` | | +| configuration.agentService.defaultAgent.defaultTimeout | string | `"10s"` | | +| configuration.agentService.defaultAgent.endpoint | string | `"dns:///flyteagent.flyte.svc.cluster.local:8000"` | | +| configuration.agentService.defaultAgent.insecure | bool | `true` | | +| configuration.agentService.defaultAgent.timeouts.GetTask | string | `"10s"` | | +| configuration.annotations | object | `{}` | | +| configuration.auth.authorizedUris | list | `[]` | | +| configuration.auth.clientSecretsExternalSecretRef | string | `""` | | +| configuration.auth.enableAuthServer | bool | `true` | | +| configuration.auth.enabled | bool | `false` | | +| configuration.auth.flyteClient.audience | string | `""` | | +| configuration.auth.flyteClient.clientId | string | `"flytectl"` | | +| configuration.auth.flyteClient.redirectUri | string | `"http://localhost:53593/callback"` | | +| configuration.auth.flyteClient.scopes[0] | string | `"all"` | | +| configuration.auth.internal.clientId | string | `"flytepropeller"` | | +| configuration.auth.internal.clientSecret | string | `""` | | +| configuration.auth.internal.clientSecretHash | string | `""` | | +| configuration.auth.oidc.baseUrl | string | `""` | | +| configuration.auth.oidc.clientId | string | `""` | | +| configuration.auth.oidc.clientSecret | string | `""` | | +| configuration.co-pilot.image.repository | string | `"cr.flyte.org/flyteorg/flytecopilot"` | | +| configuration.co-pilot.image.tag | string | `"v1.13.2"` | | +| configuration.database.dbname | string | `"flyte"` | | +| configuration.database.host | string | `"127.0.0.1"` | | +| configuration.database.options | string | `"sslmode=disable"` | | +| configuration.database.password | string | `""` | | +| configuration.database.passwordPath | string | `""` | | +| configuration.database.port | int | `5432` | | +| configuration.database.username | string | `"postgres"` | | +| configuration.externalConfigMap | string | `""` | | +| configuration.externalSecretRef | string | `""` | | +| configuration.inline | object | `{}` | | +| configuration.inlineConfigMap | string | `""` | | +| configuration.inlineSecretRef | string | `""` | | +| configuration.labels | object | `{}` | | +| configuration.logging.level | int | `1` | | +| configuration.logging.plugins.cloudwatch.enabled | bool | `false` | | +| configuration.logging.plugins.cloudwatch.templateUri | string | `""` | | +| configuration.logging.plugins.custom | list | `[]` | | +| configuration.logging.plugins.kubernetes.enabled | bool | `false` | | +| configuration.logging.plugins.kubernetes.templateUri | string | `""` | | +| configuration.logging.plugins.stackdriver.enabled | bool | `false` | | +| configuration.logging.plugins.stackdriver.templateUri | string | `""` | | +| configuration.propeller.createCRDs | bool | `true` | | +| configuration.storage.metadataContainer | string | `"my-organization-flyte-container"` | | +| configuration.storage.provider | string | `"s3"` | | +| configuration.storage.providerConfig.azure.account | string | `"storage-account-name"` | | +| configuration.storage.providerConfig.azure.configDomainSuffix | string | `""` | | +| configuration.storage.providerConfig.azure.configUploadConcurrency | int | `4` | | +| configuration.storage.providerConfig.azure.key | string | `""` | | +| configuration.storage.providerConfig.gcs.project | string | `"my-organization-gcp-project"` | | +| configuration.storage.providerConfig.s3.accessKey | string | `""` | | +| configuration.storage.providerConfig.s3.authType | string | `"iam"` | | +| configuration.storage.providerConfig.s3.disableSSL | bool | `false` | | +| configuration.storage.providerConfig.s3.endpoint | string | `""` | | +| configuration.storage.providerConfig.s3.region | string | `"us-east-1"` | | +| configuration.storage.providerConfig.s3.secretKey | string | `""` | | +| configuration.storage.providerConfig.s3.v2Signing | bool | `false` | | +| configuration.storage.userDataContainer | string | `"my-organization-flyte-container"` | | +| deployment.annotations | object | `{}` | | +| deployment.args | list | `[]` | | +| deployment.command | list | `[]` | | +| deployment.extraEnvVars | list | `[]` | | +| deployment.extraEnvVarsConfigMap | string | `""` | | +| deployment.extraEnvVarsSecret | string | `""` | | +| deployment.extraPodSpec | object | `{}` | | +| deployment.extraVolumeMounts | list | `[]` | | +| deployment.extraVolumes | list | `[]` | | +| deployment.genAdminAuthSecret.args | list | `[]` | | +| deployment.genAdminAuthSecret.command | list | `[]` | | +| deployment.image.pullPolicy | string | `"IfNotPresent"` | | +| deployment.image.repository | string | `"cr.flyte.org/flyteorg/flyte-binary"` | | +| deployment.image.tag | string | `"latest"` | | +| deployment.initContainers | list | `[]` | | +| deployment.labels | object | `{}` | | +| deployment.lifecycleHooks | object | `{}` | | +| deployment.livenessProbe | object | `{}` | | +| deployment.podAnnotations | object | `{}` | | +| deployment.podLabels | object | `{}` | | +| deployment.podSecurityContext.enabled | bool | `false` | | +| deployment.podSecurityContext.fsGroup | int | `65534` | | +| deployment.podSecurityContext.runAsGroup | int | `65534` | | +| deployment.podSecurityContext.runAsUser | int | `65534` | | +| deployment.readinessProbe | object | `{}` | | +| deployment.uploaders | list | `[]` | | +| deployment.startupProbe | object | `{}` | | +| deployment.waitForDB.args | list | `[]` | | +| deployment.waitForDB.command | list | `[]` | | +| deployment.waitForDB.image.pullPolicy | string | `"IfNotPresent"` | | +| deployment.waitForDB.image.repository | string | `"postgres"` | | +| deployment.waitForDB.image.tag | string | `"15-alpine"` | | +| enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | +| enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | +| enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","uploader","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker\*, athena if you install the backend plugins | +| flyte-core-components.admin.disableClusterResourceManager | bool | `false` | | +| flyte-core-components.admin.disableScheduler | bool | `false` | | +| flyte-core-components.admin.disabled | bool | `false` | | +| flyte-core-components.admin.seedProjectsWithDetails[0].description | string | `"Default project setup."` | | +| flyte-core-components.admin.seedProjectsWithDetails[0].name | string | `"flytesnacks"` | | +| flyte-core-components.admin.seedProjects[0] | string | `"flytesnacks"` | | +| flyte-core-components.dataCatalog.disabled | bool | `false` | | +| flyte-core-components.propeller.disableWebhook | bool | `false` | | +| flyte-core-components.propeller.disabled | bool | `false` | | +| flyteagent.enabled | bool | `false` | | +| fullnameOverride | string | `""` | | +| ingress.commonAnnotations | object | `{}` | | +| ingress.create | bool | `false` | | +| ingress.grpcAnnotations | object | `{}` | | +| ingress.grpcExtraPaths.append | list | `[]` | | +| ingress.grpcExtraPaths.prepend | list | `[]` | | +| ingress.grpcIngressClassName | string | `""` | | +| ingress.grpcTls | list | `[]` | | +| ingress.host | string | `""` | | +| ingress.httpAnnotations | object | `{}` | | +| ingress.httpExtraPaths.append | list | `[]` | | +| ingress.httpExtraPaths.prepend | list | `[]` | | +| ingress.httpIngressClassName | string | `""` | | +| ingress.httpTls | list | `[]` | | +| ingress.ingressClassName | string | `""` | | +| ingress.labels | object | `{}` | | +| ingress.separateGrpcIngress | bool | `true` | | +| ingress.tls | list | `[]` | | +| nameOverride | string | `""` | | +| rbac.annotations | object | `{}` | | +| rbac.create | bool | `true` | | +| rbac.extraRules | list | `[]` | | +| rbac.labels | object | `{}` | | +| service.clusterIP | string | `""` | | +| service.commonAnnotations | object | `{}` | | +| service.externalTrafficPolicy | string | `"Cluster"` | | +| service.extraPorts | list | `[]` | | +| service.grpcAnnotations | object | `{}` | | +| service.httpAnnotations | object | `{}` | | +| service.labels | object | `{}` | | +| service.loadBalancerIP | string | `""` | | +| service.loadBalancerSourceRanges | list | `[]` | | +| service.nodePorts.grpc | string | `""` | | +| service.nodePorts.http | string | `""` | | +| service.ports.grpc | string | `""` | | +| service.ports.http | string | `""` | | +| service.type | string | `"ClusterIP"` | | +| serviceAccount.annotations | object | `{}` | | +| serviceAccount.create | bool | `true` | | +| serviceAccount.imagePullSecrets | list | `[]` | | +| serviceAccount.labels | object | `{}` | | +| serviceAccount.name | string | `""` | | diff --git a/charts/flyte-binary/eks-production.yaml b/charts/flyte-binary/eks-production.yaml index 11cd55f6cd..f9ef8b49a0 100644 --- a/charts/flyte-binary/eks-production.yaml +++ b/charts/flyte-binary/eks-production.yaml @@ -13,7 +13,7 @@ configuration: s3: region: "" authType: "iam" - #For logging to work, you need to setup an agent. + #For logging to work, you need to setup an agent. # Learn more: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-logs.html logging: level: 5 @@ -33,21 +33,21 @@ configuration: clientSecret: clientSecretHash: authorizedUris: - - https://flyte.company.com #change to your authorized URI + - https://flyte.company.com #change to your authorized URI inline: #This section automates the IAM Role annotation for the default KSA on each project namespace to enable IRSA #Learn more: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html cluster_resources: customData: - - production: - - defaultIamRole: - value: - - staging: - - defaultIamRole: - value: - - development: - - defaultIamRole: - value: + - production: + - defaultIamRole: + value: + - staging: + - defaultIamRole: + value: + - development: + - defaultIamRole: + value: flyteadmin: roleNameKey: "iam.amazonaws.com/role" plugins: @@ -57,7 +57,7 @@ configuration: - AWS_METADATA_SERVICE_TIMEOUT: 5 - AWS_METADATA_SERVICE_NUM_ATTEMPTS: 20 # Configuration for the Datacatalog engine, used when caching is enabled - # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html + # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html storage: cache: max_size_mbs: 10 @@ -66,7 +66,7 @@ configuration: task-plugins: enabled-plugins: - container - - sidecar + - uploader - K8S-ARRAY #used for MapTasks - agent-service - echo @@ -103,22 +103,22 @@ ingress: # nginx.ingress.kubernetes.io/backend-protocol: GRPC #host: # change for the URL you'll use to connect to Flyte ## --- - + #This section assumes you are using the ALB Ingress controller. ingressClassName: alb commonAnnotations: - alb.ingress.kubernetes.io/certificate-arn: 'arn:aws:acm:::certificate/' + alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:::certificate/" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing - alb.ingress.kubernetes.io/ssl-redirect: '443' + alb.ingress.kubernetes.io/ssl-redirect: "443" alb.ingress.kubernetes.io/target-type: ip httpAnnotations: alb.ingress.kubernetes.io/actions.app-root: '{"Type": "redirect", "RedirectConfig": {"Path": "/console", "StatusCode": "HTTP_302"}}' grpcAnnotations: - alb.ingress.kubernetes.io/backend-protocol-version: GRPC + alb.ingress.kubernetes.io/backend-protocol-version: GRPC host: flyte.mydomain.com #replace with your fully-qualified domain name serviceAccount: create: true annotations: - eks.amazonaws.com/role-arn: "" \ No newline at end of file + eks.amazonaws.com/role-arn: "" diff --git a/charts/flyte-binary/eks-starter.yaml b/charts/flyte-binary/eks-starter.yaml index 215cf5f2ba..3933690a55 100644 --- a/charts/flyte-binary/eks-starter.yaml +++ b/charts/flyte-binary/eks-starter.yaml @@ -13,7 +13,7 @@ configuration: s3: region: "" authType: "iam" - #For logging to work, you need to setup an agent. + #For logging to work, you need to setup an agent. # Learn more: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-logs.html logging: level: 5 @@ -33,21 +33,21 @@ configuration: clientSecret: clientSecretHash: authorizedUris: - - https://flyte.company.com #change to your authorized URI + - https://flyte.company.com #change to your authorized URI inline: #This section automates the IAM Role annotation for the default KSA on each project namespace to enable IRSA #Learn more: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html cluster_resources: customData: - - production: - - defaultIamRole: - value: - - staging: - - defaultIamRole: - value: - - development: - - defaultIamRole: - value: + - production: + - defaultIamRole: + value: + - staging: + - defaultIamRole: + value: + - development: + - defaultIamRole: + value: flyteadmin: roleNameKey: "iam.amazonaws.com/role" plugins: @@ -57,7 +57,7 @@ configuration: - AWS_METADATA_SERVICE_TIMEOUT: 5 - AWS_METADATA_SERVICE_NUM_ATTEMPTS: 20 # Configuration for the Datacatalog engine, used when caching is enabled - # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html + # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html storage: cache: max_size_mbs: 10 @@ -66,7 +66,7 @@ configuration: task-plugins: enabled-plugins: - container - - sidecar + - uploader - K8S-ARRAY #used for MapTasks - agent-service - echo @@ -103,22 +103,22 @@ ingress: # nginx.ingress.kubernetes.io/backend-protocol: GRPC #host: # change for the URL you'll use to connect to Flyte ## --- - + #This section assumes you are using the ALB Ingress controller. ingressClassName: alb commonAnnotations: - alb.ingress.kubernetes.io/certificate-arn: 'arn:aws:acm:::certificate/' + alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:::certificate/" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing - alb.ingress.kubernetes.io/ssl-redirect: '443' + alb.ingress.kubernetes.io/ssl-redirect: "443" alb.ingress.kubernetes.io/target-type: ip httpAnnotations: alb.ingress.kubernetes.io/actions.app-root: '{"Type": "redirect", "RedirectConfig": {"Path": "/console", "StatusCode": "HTTP_302"}}' grpcAnnotations: - alb.ingress.kubernetes.io/backend-protocol-version: GRPC + alb.ingress.kubernetes.io/backend-protocol-version: GRPC host: flyte.mydomain.com #replace with your fully-qualified domain name serviceAccount: create: true annotations: - eks.amazonaws.com/role-arn: "" \ No newline at end of file + eks.amazonaws.com/role-arn: "" diff --git a/charts/flyte-binary/gke-starter.yaml b/charts/flyte-binary/gke-starter.yaml index 75bd00c513..c24a4aed05 100644 --- a/charts/flyte-binary/gke-starter.yaml +++ b/charts/flyte-binary/gke-starter.yaml @@ -62,20 +62,20 @@ configuration: authorizedUris: [] # inline Specify additional configuration or overrides for Flyte, to be merged with the base configuration - inline: + inline: #This section automates the IAM Role annotation for the default KSA on each project namespace to enable IRSA #Learn more: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html cluster_resources: customData: - - production: - - defaultIamServiceAccount: - value: - - staging: - - defaultIamServiceAccount: - value: - - development: - - defaultIamServiceAccount: - value: + - production: + - defaultIamServiceAccount: + value: + - staging: + - defaultIamServiceAccount: + value: + - development: + - defaultIamServiceAccount: + value: plugins: k8s: inject-finalizer: true @@ -83,12 +83,12 @@ configuration: gpu-partition-size-node-label: cloud.google.com/gke-gpu-partition-size resource-tolerations: - nvidia.com/gpu: - - key: "nvidia.com/gpu" - operator: "Equal" - value: "present" - effect: "NoSchedule" + - key: "nvidia.com/gpu" + operator: "Equal" + value: "present" + effect: "NoSchedule" # Configuration for the Datacatalog engine, used when caching is enabled - # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html + # Learn more: https://docs.flyte.org/en/latest/deployment/configuration/generated/datacatalog_config.html storage: cache: max_size_mbs: 10 @@ -97,7 +97,7 @@ configuration: task-plugins: enabled-plugins: - container - - sidecar + - uploader - K8S-ARRAY #used for MapTasks - agent-service - echo @@ -108,7 +108,7 @@ configuration: # clusterResourceTemplates Specify templates for Kubernetes resources that should be created for new Flyte projects clusterResourceTemplates: # inline Specify additional cluster resource templates, to be merged with the base configuration - inline: + inline: #This section automates the creation of the project-domain namespaces 001_namespace.yaml: | apiVersion: v1 @@ -123,7 +123,7 @@ clusterResourceTemplates: name: default namespace: '{{ namespace }}' annotations: - iam.gke.io/gcp-service-account: '{{ defaultIamServiceAccount }}' + iam.gke.io/gcp-service-account: '{{ defaultIamServiceAccount }}' # serviceAccount Configure Flyte ServiceAccount serviceAccount: @@ -131,18 +131,18 @@ serviceAccount: create: true #Automates annotation of default flyte-binary KSA. Make sure to bind the KSA to the GSA: https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#authenticating_to annotations: - iam.gke.io/gcp-service-account: + iam.gke.io/gcp-service-account: # rbac Configure Kubernetes RBAC for Flyte rbac: # create Create ClusterRole and ClusterRoleBinding resources create: true # extraRules Add additional rules to the ClusterRole - extraRules: - - apiGroups: - - "" - resources: - - serviceaccounts - verbs: - - create - - get - - patch + extraRules: + - apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - create + - get + - patch diff --git a/charts/flyte-binary/templates/deployment.yaml b/charts/flyte-binary/templates/deployment.yaml index d1be13d166..4e7db447a3 100644 --- a/charts/flyte-binary/templates/deployment.yaml +++ b/charts/flyte-binary/templates/deployment.yaml @@ -224,8 +224,8 @@ spec: {{- if .Values.deployment.extraVolumeMounts }} {{- tpl ( .Values.deployment.extraVolumeMounts | toYaml ) . | nindent 12 }} {{- end }} - {{- if .Values.deployment.sidecars }} - {{- tpl ( .Values.deployment.sidecars | toYaml ) . | nindent 8 }} + {{- if .Values.deployment.uploaders }} + {{- tpl ( .Values.deployment.uploaders | toYaml ) . | nindent 8 }} {{- end }} volumes: {{- if .Values.configuration.auth.enabled }} diff --git a/charts/flyte-binary/values.yaml b/charts/flyte-binary/values.yaml index 97fa1fe8d3..51a911cdff 100644 --- a/charts/flyte-binary/values.yaml +++ b/charts/flyte-binary/values.yaml @@ -162,12 +162,12 @@ configuration: clientSecretsExternalSecretRef: "" # co-pilot Configuration for Flyte CoPilot co-pilot: - # image Configure image to use for CoPilot sidecar + # image Configure image to use for CoPilot uploader image: - # repository CoPilot sidecar image repository - repository: cr.flyte.org/flyteorg/flytecopilot # FLYTECOPILOT_IMAGE - # tag CoPilot sidecar image tag - tag: v1.13.2 # FLYTECOPILOT_TAG + # repository CoPilot uploader image repository + repository: cr.flyte.org/flyteorg/flytecopilot # FLYTECOPILOT_IMAGE + # tag CoPilot uploader image tag + tag: v1.13.2 # FLYTECOPILOT_TAG # agentService Flyte Agent configuration agentService: defaultAgent: @@ -219,9 +219,9 @@ deployment: # image Configure image to use for Flyte image: # repository Flyte image repository - repository: cr.flyte.org/flyteorg/flyte-binary # FLYTE_IMAGE + repository: cr.flyte.org/flyteorg/flyte-binary # FLYTE_IMAGE # tag Flyte image tag - tag: latest # FLYTE_TAG + tag: latest # FLYTE_TAG # pullPolicy Flyte image pull policy pullPolicy: IfNotPresent # extraEnvVars Array with extra environment variables to add to Flyte @@ -290,8 +290,8 @@ deployment: extraVolumeMounts: [] # extraVolume Specify additional volumes for Flyte pod extraVolumes: [] - # sidecars Specify additional containers for Flyte pod - sidecars: [] + # uploaders Specify additional containers for Flyte pod + uploaders: [] # initContainers Specify additional init containers for Flyte pod initContainers: [] # extraPodSpec Specify additional configuration for Flyte pod @@ -406,13 +406,13 @@ enabled_plugins: # Enable sagemaker*, athena if you install the backend plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array # -- Uncomment to enable task type that uses Flyte Agent # bigquery_query_job_task: agent-service diff --git a/charts/flyte-core/README.md b/charts/flyte-core/README.md index 6aed892810..a5048a4842 100644 --- a/charts/flyte-core/README.md +++ b/charts/flyte-core/README.md @@ -6,11 +6,12 @@ A Helm chart for Flyte core ## Requirements -| Repository | Name | Version | -|------------|------|---------| +| Repository | Name | Version | +| -------------------- | ---------------------- | ------- | | file://../flyteagent | flyteagent(flyteagent) | v0.1.10 | ### Flyte INSTALLATION: + - [Install helm 3](https://helm.sh/docs/intro/install/) - Fetch chart dependencies `` - Install Flyte: @@ -29,17 +30,20 @@ helm diff upgrade -f values-eks.yaml flyte flyte/flyte-core ``` Then apply your changes: + ```bash helm upgrade -f values-eks.yaml flyte flyte/flyte-core ``` Install ingress controller (By default Flyte helm chart have contour ingress resource) + ```bash helm repo add bitnami https://charts.bitnami.com/bitnami helm install gateway bitnami/contour -n flyte ``` #### Alternative: Generate raw kubernetes yaml with helm template + - `helm template --name-template=flyte-eks . -n flyte -f values-eks.yaml > flyte_generated_eks.yaml` - Deploy the manifest `kubectl apply -f flyte_generated_eks.yaml` @@ -47,302 +51,303 @@ helm install gateway bitnami/contour -n flyte - Get flyte host `minikube service contour -n heptio-contour --url`. And then visit `http:///console` ### CONFIGURATION NOTES: + - The docker images, their tags and other default parameters are configured in `values.yaml` file. - Each Flyte installation type should have separate `values-*.yaml` file: for sandbox, EKS and etc. The configuration in `values.yaml` and the chosen config `values-*.yaml` are merged when generating the deployment manifest. - The configuration in `values-sandbox.yaml` is ready for installation in minikube. But `values-eks.yaml` should be edited before installation: s3 bucket, RDS hosts, iam roles, secrets and etc need to be modified. ## Values -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| cloud_events.aws | object | `{"region":"us-east-2"}` | Configuration for sending cloud events to AWS SNS | -| cloud_events.enable | bool | `false` | | -| cloud_events.eventsPublisher.eventTypes[0] | string | `"all"` | | -| cloud_events.eventsPublisher.topicName | string | `"arn:aws:sns:us-east-2:123456:123-my-topic"` | | -| cloud_events.gcp | object | `{"region":"us-east1"}` | Configuration for sending cloud events to GCP Pub Sub | -| cloud_events.kafka | object | `{"brokers":["mybroker:443"],"saslConfig":{"enabled":false,"handshake":true,"mechanism":"PLAIN","password":"","passwordPath":"","user":"kafka"},"tlsConfig":{"certPath":"/etc/ssl/certs/kafka-client.crt","enabled":false,"keyPath":"/etc/ssl/certs/kafka-client.key"},"version":"3.7.0"}` | Configuration for sending cloud events to Kafka | -| cloud_events.kafka.brokers | list | `["mybroker:443"]` | The kafka brokers to talk to | -| cloud_events.kafka.saslConfig | object | `{"enabled":false,"handshake":true,"mechanism":"PLAIN","password":"","passwordPath":"","user":"kafka"}` | SASL based authentication | -| cloud_events.kafka.saslConfig.enabled | bool | `false` | Whether to use SASL authentication | -| cloud_events.kafka.saslConfig.handshake | bool | `true` | Whether the send the SASL handsahke first | -| cloud_events.kafka.saslConfig.mechanism | string | `"PLAIN"` | Which SASL mechanism to use. Defaults to PLAIN | -| cloud_events.kafka.saslConfig.password | string | `""` | The password for the kafka user | -| cloud_events.kafka.saslConfig.passwordPath | string | `""` | Optional mount path of file containing the kafka password. | -| cloud_events.kafka.saslConfig.user | string | `"kafka"` | The kafka user | -| cloud_events.kafka.tlsConfig | object | `{"certPath":"/etc/ssl/certs/kafka-client.crt","enabled":false,"keyPath":"/etc/ssl/certs/kafka-client.key"}` | Certificate based authentication | -| cloud_events.kafka.tlsConfig.certPath | string | `"/etc/ssl/certs/kafka-client.crt"` | Path to the client certificate | -| cloud_events.kafka.tlsConfig.enabled | bool | `false` | Whether to use certificate based authentication or TLS | -| cloud_events.kafka.tlsConfig.keyPath | string | `"/etc/ssl/certs/kafka-client.key"` | Path to the client private key | -| cloud_events.kafka.version | string | `"3.7.0"` | The version of Kafka | -| cloud_events.type | string | `"aws"` | | -| cluster_resource_manager | object | `{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"nodeSelector":{},"podAnnotations":{},"podEnv":{},"podLabels":{},"prometheus":{"enabled":false,"path":"/metrics","port":10254},"resources":{},"service_account_name":"flyteadmin","standaloneDeployment":false,"templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]}` | Configuration for the Cluster resource manager component. This is an optional component, that enables automatic cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain | -| cluster_resource_manager.config | object | `{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}}` | Configmap for ClusterResource parameters | -| cluster_resource_manager.config.cluster_resources | object | `{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}` | ClusterResource parameters Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. | -| cluster_resource_manager.config.cluster_resources.refreshInterval | string | `"5m"` | How frequently to run the sync process | -| cluster_resource_manager.config.cluster_resources.standaloneDeployment | bool | `false` | Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints | -| cluster_resource_manager.enabled | bool | `true` | Enables the Cluster resource manager component | -| cluster_resource_manager.nodeSelector | object | `{}` | nodeSelector for ClusterResource deployment | -| cluster_resource_manager.podAnnotations | object | `{}` | Annotations for ClusterResource pods | -| cluster_resource_manager.podEnv | object | `{}` | Additional ClusterResource container environment variables | -| cluster_resource_manager.podLabels | object | `{}` | Labels for ClusterResource pods | -| cluster_resource_manager.resources | object | `{}` | Resources for ClusterResource deployment | -| cluster_resource_manager.service_account_name | string | `"flyteadmin"` | Service account name to run with | -| cluster_resource_manager.templates | list | `[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]` | Resource templates that should be applied | -| cluster_resource_manager.templates[0] | object | `{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"}` | Template for namespaces resources | -| common | object | `{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console","nginx.ingress.kubernetes.io/service-upstream":"true"},"enabled":true,"ingressClassName":null,"separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":false}}` | ---------------------------------------------- COMMON SETTINGS | -| common.databaseSecret.name | string | `""` | Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret | -| common.databaseSecret.secretManifest | object | `{}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets | -| common.flyteNamespaceTemplate.enabled | bool | `false` | - Enable or disable creating Flyte namespace in template. Enable when using helm as template-engine only. Disable when using `helm install ...`. | -| common.ingress.albSSLRedirect | bool | `false` | - albSSLRedirect adds a special route for ssl redirect. Only useful in combination with the AWS LoadBalancer Controller. | -| common.ingress.annotations | object | `{"nginx.ingress.kubernetes.io/app-root":"/console","nginx.ingress.kubernetes.io/service-upstream":"true"}` | - Ingress annotations applied to both HTTP and GRPC ingresses. | -| common.ingress.enabled | bool | `true` | - Enable or disable creating Ingress for Flyte. Relevant to disable when using e.g. Istio as ingress controller. | -| common.ingress.ingressClassName | string | `nil` | - Sets the ingressClassName | -| common.ingress.separateGrpcIngress | bool | `false` | - separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx. | -| common.ingress.separateGrpcIngressAnnotations | object | `{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"}` | - Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled. | -| common.ingress.tls | object | `{"enabled":false}` | - Ingress hostname host: | -| common.ingress.webpackHMR | bool | `false` | - Enable or disable HMR route to flyteconsole. This is useful only for frontend development. | -| configmap.admin | object | `{"admin":{"clientId":"{{ .Values.secrets.adminOauthClientCredentials.clientId }}","clientSecretLocation":"/etc/secrets/client_secret","endpoint":"flyteadmin:81","insecure":true},"event":{"capacity":1000,"rate":500,"type":"admin"}}` | Admin Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/subworkflow/launchplan#AdminConfig) | -| configmap.adminServer | object | `{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}}` | FlyteAdmin server configuration | -| configmap.adminServer.auth | object | `{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}}` | Authentication configuration | -| configmap.adminServer.server.security.secure | bool | `false` | Controls whether to serve requests over SSL/TLS. | -| configmap.adminServer.server.security.useAuth | bool | `false` | Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. | -| configmap.catalog | object | `{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}}` | Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) | -| configmap.clusters.clusterConfigs | list | `[]` | | -| configmap.clusters.labelClusterMap | object | `{}` | | -| configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI | -| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration | -| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) | -| configmap.core | object | `{"manager":{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"},"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration | -| configmap.core.manager | object | `{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config). | -| configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). | -| configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"heartbeat-grace-period-multiplier":3,"max-reservation-heartbeat":"30s","metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config | -| configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. | -| configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | -| configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | -| configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins | -| configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}}}` | Kubernetes specific Flyte configuration | -| configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) | -| configmap.remoteData.remoteData.region | string | `"us-east-1"` | | -| configmap.remoteData.remoteData.scheme | string | `"local"` | | -| configmap.remoteData.remoteData.signedUrls.durationMinutes | int | `3` | | -| configmap.resource_manager | object | `{"propeller":{"resourcemanager":{"type":"noop"}}}` | Resource manager configuration | -| configmap.resource_manager.propeller | object | `{"resourcemanager":{"type":"noop"}}` | resource manager configuration | -| configmap.schedulerConfig.scheduler.metricsScope | string | `"flyte:"` | | -| configmap.schedulerConfig.scheduler.profilerPort | int | `10254` | | -| configmap.task_logs | object | `{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":false}}}` | Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various logging engines | -| configmap.task_logs.plugins.logs.cloudwatch-enabled | bool | `false` | One option is to enable cloudwatch logging for EKS, update the region and log group accordingly | -| configmap.task_resource_defaults | object | `{"task_resources":{"defaults":{"cpu":"100m","memory":"500Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi"}}}` | Task default resources configuration Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). | -| configmap.task_resource_defaults.task_resources | object | `{"defaults":{"cpu":"100m","memory":"500Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi"}}` | Task default resources parameters | -| daskoperator | object | `{"enabled":false}` | Optional: Dask Plugin using the Dask Operator | -| daskoperator.enabled | bool | `false` | - enable or disable the dask operator deployment installation | -| databricks | object | `{"enabled":false,"plugin_config":{"plugins":{"databricks":{"databricksInstance":"dbc-a53b7a3c-614c","entrypointFile":"dbfs:///FileStore/tables/entrypoint.py"}}}}` | Optional: Databricks Plugin allows us to run the spark job on the Databricks platform. | -| datacatalog.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | -| datacatalog.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | -| datacatalog.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | -| datacatalog.affinity | object | `{}` | affinity for Datacatalog deployment | -| datacatalog.configPath | string | `"/etc/datacatalog/config/*.yaml"` | Default regex string for searching configuration files | -| datacatalog.enabled | bool | `true` | | -| datacatalog.extraArgs | object | `{}` | Appends extra command line arguments to the main command | -| datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment | -| datacatalog.image.tag | string | `"v1.13.2"` | Docker image tag | -| datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment | -| datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods | -| datacatalog.podEnv | object | `{}` | Additional Datacatalog container environment variables | -| datacatalog.podLabels | object | `{}` | Labels for Datacatalog pods | -| datacatalog.priorityClassName | string | `""` | Sets priorityClassName for datacatalog pod(s). | -| datacatalog.replicaCount | int | `1` | Replicas count for Datacatalog deployment | -| datacatalog.resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Datacatalog deployment | -| datacatalog.securityContext | object | `{"fsGroup":1001,"fsGroupChangePolicy":"OnRootMismatch","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for datacatalog pod(s). | -| datacatalog.service | object | `{"additionalPorts":[],"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"}` | Service settings for Datacatalog | -| datacatalog.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | -| datacatalog.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Datacatalog | -| datacatalog.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Datacatalog pods | -| datacatalog.serviceAccount.create | bool | `true` | Should a service account be created for Datacatalog | -| datacatalog.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| datacatalog.tolerations | list | `[]` | tolerations for Datacatalog deployment | -| db.admin.database.dbname | string | `"flyteadmin"` | | -| db.admin.database.host | string | `"postgres"` | | -| db.admin.database.port | int | `5432` | | -| db.admin.database.username | string | `"postgres"` | | -| db.datacatalog.database.dbname | string | `"datacatalog"` | | -| db.datacatalog.database.host | string | `"postgres"` | | -| db.datacatalog.database.port | int | `5432` | | -| db.datacatalog.database.username | string | `"postgres"` | | -| deployRedoc | bool | `false` | | -| external_events | object | `{"aws":{"region":"us-east-2"},"enable":false,"eventsPublisher":{"eventTypes":["all"],"topicName":"arn:aws:sns:us-east-2:123456:123-my-topic"},"type":"aws"}` | **Optional Component** External events are used to send events (unprocessed, as Admin see them) to an SNS topic (or gcp equivalent) The config is here as an example only - if not enabled, it won't be used. | -| flyteadmin.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | -| flyteadmin.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | -| flyteadmin.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | -| flyteadmin.affinity | object | `{}` | affinity for Flyteadmin deployment | -| flyteadmin.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flyteadmin.enabled | bool | `true` | | -| flyteadmin.env | list | `[]` | Additional flyteadmin container environment variables e.g. SendGrid's API key - name: SENDGRID_API_KEY value: "" e.g. secret environment variable (you can combine it with .additionalVolumes): - name: SENDGRID_API_KEY valueFrom: secretKeyRef: name: sendgrid-secret key: api_key | -| flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command | -| flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | | -| flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment | -| flyteadmin.image.tag | string | `"v1.13.2"` | | -| flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create | -| flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment | -| flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods | -| flyteadmin.podLabels | object | `{}` | Labels for Flyteadmin pods | -| flyteadmin.priorityClassName | string | `""` | Sets priorityClassName for flyteadmin pod(s). | -| flyteadmin.replicaCount | int | `1` | Replicas count for Flyteadmin deployment | -| flyteadmin.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flyteadmin deployment | -| flyteadmin.secrets | object | `{}` | | -| flyteadmin.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flyteadmin pod(s). | -| flyteadmin.service | object | `{"additionalPorts":[],"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"}` | Service settings for Flyteadmin | -| flyteadmin.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | -| flyteadmin.serviceAccount | object | `{"alwaysCreate":false,"annotations":{},"clusterRole":{"apiGroups":["","flyte.lyft.com","rbac.authorization.k8s.io"],"resources":["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"],"verbs":["*"]},"create":true,"createClusterRole":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlyteAdmin | -| flyteadmin.serviceAccount.alwaysCreate | bool | `false` | Should a service account always be created for flyteadmin even without an actual flyteadmin deployment running (e.g. for multi-cluster setups) | -| flyteadmin.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flyteadmin pods | -| flyteadmin.serviceAccount.clusterRole | object | `{"apiGroups":["","flyte.lyft.com","rbac.authorization.k8s.io"],"resources":["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"],"verbs":["*"]}` | Configuration for ClusterRole created for Flyteadmin | -| flyteadmin.serviceAccount.clusterRole.apiGroups | list | `["","flyte.lyft.com","rbac.authorization.k8s.io"]` | Specifies the API groups that this ClusterRole can access | -| flyteadmin.serviceAccount.clusterRole.resources | list | `["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"]` | Specifies the resources that this ClusterRole can access | -| flyteadmin.serviceAccount.clusterRole.verbs | list | `["*"]` | Specifies the verbs (actions) that this ClusterRole can perform on the specified resources | -| flyteadmin.serviceAccount.create | bool | `true` | Should a service account be created for flyteadmin | -| flyteadmin.serviceAccount.createClusterRole | bool | `true` | Should a ClusterRole be created for Flyteadmin | -| flyteadmin.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyteadmin.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flyteadmin service monitor | -| flyteadmin.serviceMonitor.enabled | bool | `false` | If enabled create the flyteadmin service monitor | -| flyteadmin.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | -| flyteadmin.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | -| flyteadmin.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | -| flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment | -| flyteagent.enabled | bool | `false` | | -| flyteagent.plugin_config.plugins.agent-service | object | `{"defaultAgent":{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true},"supportedTaskTypes":[]}` | Agent service configuration for propeller. | -| flyteagent.plugin_config.plugins.agent-service.defaultAgent | object | `{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true}` | The default agent service to use for plugin tasks. | -| flyteagent.plugin_config.plugins.agent-service.defaultAgent.endpoint | string | `"dns:///flyteagent.flyte.svc.cluster.local:8000"` | The agent service endpoint propeller should connect to. | -| flyteagent.plugin_config.plugins.agent-service.defaultAgent.insecure | bool | `true` | Whether the connection from propeller to the agent service should use TLS. | -| flyteagent.plugin_config.plugins.agent-service.supportedTaskTypes | list | `[]` | The task types supported by the default agent. As of #5460 these are discovered automatically and don't need to be configured. | -| flyteagent.podLabels | object | `{}` | Labels for flyteagent pods | -| flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment | -| flyteconsole.enabled | bool | `true` | | -| flyteconsole.ga.enabled | bool | `false` | | -| flyteconsole.ga.tracking_id | string | `"G-0QW4DJWJ20"` | | -| flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | | -| flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment | -| flyteconsole.image.tag | string | `"v1.17.1"` | | -| flyteconsole.imagePullSecrets | list | `[]` | ImagePullSecrets to assign to the Flyteconsole deployment | -| flyteconsole.livenessProbe | object | `{}` | | -| flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment | -| flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods | -| flyteconsole.podEnv | object | `{}` | Additional Flyteconsole container environment variables | -| flyteconsole.podLabels | object | `{}` | Labels for Flyteconsole pods | -| flyteconsole.priorityClassName | string | `""` | Sets priorityClassName for flyte console pod(s). | -| flyteconsole.readinessProbe | object | `{}` | | -| flyteconsole.replicaCount | int | `1` | Replicas count for Flyteconsole deployment | -| flyteconsole.resources | object | `{"limits":{"cpu":"500m","memory":"250Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Flyteconsole deployment | -| flyteconsole.securityContext | object | `{"fsGroupChangePolicy":"OnRootMismatch","runAsNonRoot":true,"runAsUser":1000,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flyteconsole pod(s). | -| flyteconsole.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Flyteconsole | -| flyteconsole.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flyteconsole service monitor | -| flyteconsole.serviceMonitor.enabled | bool | `false` | If enabled create the flyteconsole service monitor | -| flyteconsole.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | -| flyteconsole.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | -| flyteconsole.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | -| flyteconsole.tolerations | list | `[]` | tolerations for Flyteconsole deployment | -| flytepropeller.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | -| flytepropeller.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | -| flytepropeller.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | -| flytepropeller.affinity | object | `{}` | affinity for Flytepropeller deployment | -| flytepropeller.clusterName | string | `""` | Defines the cluster name used in events sent to Admin | -| flytepropeller.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flytepropeller.createCRDs | bool | `true` | Whether to install the flyteworkflows CRD with helm | -| flytepropeller.enabled | bool | `true` | | -| flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command | -| flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | | -| flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment | -| flytepropeller.image.tag | string | `"v1.13.2"` | | -| flytepropeller.manager | bool | `false` | | -| flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment | -| flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods | -| flytepropeller.podEnv | object | `{}` | Additional Flytepropeller container environment variables | -| flytepropeller.podLabels | object | `{}` | Labels for Flytepropeller pods | -| flytepropeller.priorityClassName | string | `""` | Sets priorityClassName for propeller pod(s). | -| flytepropeller.prometheus.enabled | bool | `false` | | -| flytepropeller.replicaCount | int | `1` | Replicas count for Flytepropeller deployment | -| flytepropeller.resources | object | `{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"100Mi"}}` | Default resources requests and limits for Flytepropeller deployment | -| flytepropeller.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsUser":1001}` | Sets securityContext for flytepropeller pod(s). | -| flytepropeller.service | object | `{"additionalPorts":[],"enabled":false}` | Settings for flytepropeller service | -| flytepropeller.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | -| flytepropeller.service.enabled | bool | `false` | If enabled create the flytepropeller service | -| flytepropeller.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlytePropeller | -| flytepropeller.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to FlytePropeller pods | -| flytepropeller.serviceAccount.create | bool | `true` | Should a service account be created for FlytePropeller | -| flytepropeller.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flytepropeller.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flytepropeller service monitor | -| flytepropeller.serviceMonitor.enabled | bool | `false` | If enabled create the flyetepropeller service monitor | -| flytepropeller.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | -| flytepropeller.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | -| flytepropeller.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | -| flytepropeller.terminationMessagePolicy | string | `"FallbackToLogsOnError"` | Error reporting | -| flytepropeller.tolerations | list | `[]` | tolerations for Flytepropeller deployment | -| flytescheduler.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | -| flytescheduler.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | -| flytescheduler.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | -| flytescheduler.affinity | object | `{}` | affinity for Flytescheduler deployment | -| flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment | -| flytescheduler.image.tag | string | `"v1.13.2"` | Docker image tag | -| flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment | -| flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods | -| flytescheduler.podEnv | object | `{}` | Additional Flytescheduler container environment variables | -| flytescheduler.podLabels | object | `{}` | Labels for Flytescheduler pods | -| flytescheduler.priorityClassName | string | `""` | Sets priorityClassName for flyte scheduler pod(s). | -| flytescheduler.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytescheduler deployment | -| flytescheduler.runPrecheck | bool | `true` | Whether to inject an init container which waits on flyteadmin | -| flytescheduler.secrets | object | `{}` | | -| flytescheduler.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flytescheduler pod(s). | -| flytescheduler.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Flytescheduler | -| flytescheduler.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flytescheduler pods | -| flytescheduler.serviceAccount.create | bool | `true` | Should a service account be created for Flytescheduler | -| flytescheduler.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flytescheduler.tolerations | list | `[]` | tolerations for Flytescheduler deployment | -| secrets.adminOauthClientCredentials.clientId | string | `"flytepropeller"` | | -| secrets.adminOauthClientCredentials.clientSecret | string | `"foobar"` | | -| secrets.adminOauthClientCredentials.enabled | bool | `true` | | -| sparkoperator | object | `{"enabled":false,"plugin_config":{"plugins":{"spark":{"spark-config-default":[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]}}}}` | Optional: Spark Plugin using the Spark Operator | -| sparkoperator.enabled | bool | `false` | - enable or disable Sparkoperator deployment installation | -| sparkoperator.plugin_config | object | `{"plugins":{"spark":{"spark-config-default":[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]}}}` | Spark plugin configuration | -| sparkoperator.plugin_config.plugins.spark.spark-config-default | list | `[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]` | Spark default configuration | -| storage | object | `{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"enableMultiContainer":false,"gcs":null,"limits":{"maxDownloadMBs":10},"s3":{"accessKey":"","authType":"iam","region":"us-east-1","secretKey":""},"type":"sandbox"}` | ---------------------------------------------------- STORAGE SETTINGS | -| storage.bucketName | string | `"my-s3-bucket"` | bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. | -| storage.custom | object | `{}` | Settings for storage type custom. See https://github.com/graymeta/stow for supported storage providers/settings. | -| storage.enableMultiContainer | bool | `false` | toggles multi-container storage config | -| storage.gcs | string | `nil` | settings for storage type gcs | -| storage.limits | object | `{"maxDownloadMBs":10}` | default limits being applied to storage config | -| storage.s3 | object | `{"accessKey":"","authType":"iam","region":"us-east-1","secretKey":""}` | settings for storage type s3 | -| storage.s3.accessKey | string | `""` | AWS IAM user access key ID to use for S3 bucket auth, only used if authType is set to accesskey | -| storage.s3.authType | string | `"iam"` | type of authentication to use for S3 buckets, can either be iam or accesskey | -| storage.s3.secretKey | string | `""` | AWS IAM user secret access key to use for S3 bucket auth, only used if authType is set to accesskey | -| storage.type | string | `"sandbox"` | Sets the storage type. Supported values are sandbox, s3, gcs and custom. | -| webhook.autoscaling.enabled | bool | `false` | | -| webhook.autoscaling.maxReplicas | int | `10` | | -| webhook.autoscaling.metrics[0].resource.name | string | `"cpu"` | | -| webhook.autoscaling.metrics[0].resource.target.averageUtilization | int | `80` | | -| webhook.autoscaling.metrics[0].resource.target.type | string | `"Utilization"` | | -| webhook.autoscaling.metrics[0].type | string | `"Resource"` | | -| webhook.autoscaling.metrics[1].resource.name | string | `"memory"` | | -| webhook.autoscaling.metrics[1].resource.target.averageUtilization | int | `80` | | -| webhook.autoscaling.metrics[1].resource.target.type | string | `"Utilization"` | | -| webhook.autoscaling.metrics[1].type | string | `"Resource"` | | -| webhook.autoscaling.minReplicas | int | `1` | | -| webhook.enabled | bool | `true` | enable or disable secrets webhook | -| webhook.nodeSelector | object | `{}` | nodeSelector for webhook deployment | -| webhook.podAnnotations | object | `{}` | Annotations for webhook pods | -| webhook.podEnv | object | `{}` | Additional webhook container environment variables | -| webhook.podLabels | object | `{}` | Labels for webhook pods | -| webhook.priorityClassName | string | `""` | Sets priorityClassName for webhook pod | -| webhook.prometheus.enabled | bool | `false` | | -| webhook.resources.requests.cpu | string | `"200m"` | | -| webhook.resources.requests.ephemeral-storage | string | `"500Mi"` | | -| webhook.resources.requests.memory | string | `"500Mi"` | | -| webhook.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for webhook pod(s). | -| webhook.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for the webhook | -| webhook.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for the webhook | -| webhook.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to the webhook | -| webhook.serviceAccount.create | bool | `true` | Should a service account be created for the webhook | -| webhook.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| workflow_notifications | object | `{"config":{},"enabled":false}` | **Optional Component** Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of various events in their workflows | -| workflow_scheduler | object | `{"config":{},"enabled":false,"type":""}` | **Optional Component** Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, this module, you will not have scheduled launchplans / workflows. Docs: https://docs.flyte.org/en/latest/howto/enable_and_use_schedules.html#setting-up-scheduled-workflows | +| Key | Type | Default | Description | +| ---------------------------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| cloud_events.aws | object | `{"region":"us-east-2"}` | Configuration for sending cloud events to AWS SNS | +| cloud_events.enable | bool | `false` | | +| cloud_events.eventsPublisher.eventTypes[0] | string | `"all"` | | +| cloud_events.eventsPublisher.topicName | string | `"arn:aws:sns:us-east-2:123456:123-my-topic"` | | +| cloud_events.gcp | object | `{"region":"us-east1"}` | Configuration for sending cloud events to GCP Pub Sub | +| cloud_events.kafka | object | `{"brokers":["mybroker:443"],"saslConfig":{"enabled":false,"handshake":true,"mechanism":"PLAIN","password":"","passwordPath":"","user":"kafka"},"tlsConfig":{"certPath":"/etc/ssl/certs/kafka-client.crt","enabled":false,"keyPath":"/etc/ssl/certs/kafka-client.key"},"version":"3.7.0"}` | Configuration for sending cloud events to Kafka | +| cloud_events.kafka.brokers | list | `["mybroker:443"]` | The kafka brokers to talk to | +| cloud_events.kafka.saslConfig | object | `{"enabled":false,"handshake":true,"mechanism":"PLAIN","password":"","passwordPath":"","user":"kafka"}` | SASL based authentication | +| cloud_events.kafka.saslConfig.enabled | bool | `false` | Whether to use SASL authentication | +| cloud_events.kafka.saslConfig.handshake | bool | `true` | Whether the send the SASL handsahke first | +| cloud_events.kafka.saslConfig.mechanism | string | `"PLAIN"` | Which SASL mechanism to use. Defaults to PLAIN | +| cloud_events.kafka.saslConfig.password | string | `""` | The password for the kafka user | +| cloud_events.kafka.saslConfig.passwordPath | string | `""` | Optional mount path of file containing the kafka password. | +| cloud_events.kafka.saslConfig.user | string | `"kafka"` | The kafka user | +| cloud_events.kafka.tlsConfig | object | `{"certPath":"/etc/ssl/certs/kafka-client.crt","enabled":false,"keyPath":"/etc/ssl/certs/kafka-client.key"}` | Certificate based authentication | +| cloud_events.kafka.tlsConfig.certPath | string | `"/etc/ssl/certs/kafka-client.crt"` | Path to the client certificate | +| cloud_events.kafka.tlsConfig.enabled | bool | `false` | Whether to use certificate based authentication or TLS | +| cloud_events.kafka.tlsConfig.keyPath | string | `"/etc/ssl/certs/kafka-client.key"` | Path to the client private key | +| cloud_events.kafka.version | string | `"3.7.0"` | The version of Kafka | +| cloud_events.type | string | `"aws"` | | +| cluster_resource_manager | object | `{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"nodeSelector":{},"podAnnotations":{},"podEnv":{},"podLabels":{},"prometheus":{"enabled":false,"path":"/metrics","port":10254},"resources":{},"service_account_name":"flyteadmin","standaloneDeployment":false,"templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]}` | Configuration for the Cluster resource manager component. This is an optional component, that enables automatic cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain | +| cluster_resource_manager.config | object | `{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}}` | Configmap for ClusterResource parameters | +| cluster_resource_manager.config.cluster_resources | object | `{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}` | ClusterResource parameters Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. | +| cluster_resource_manager.config.cluster_resources.refreshInterval | string | `"5m"` | How frequently to run the sync process | +| cluster_resource_manager.config.cluster_resources.standaloneDeployment | bool | `false` | Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints | +| cluster_resource_manager.enabled | bool | `true` | Enables the Cluster resource manager component | +| cluster_resource_manager.nodeSelector | object | `{}` | nodeSelector for ClusterResource deployment | +| cluster_resource_manager.podAnnotations | object | `{}` | Annotations for ClusterResource pods | +| cluster_resource_manager.podEnv | object | `{}` | Additional ClusterResource container environment variables | +| cluster_resource_manager.podLabels | object | `{}` | Labels for ClusterResource pods | +| cluster_resource_manager.resources | object | `{}` | Resources for ClusterResource deployment | +| cluster_resource_manager.service_account_name | string | `"flyteadmin"` | Service account name to run with | +| cluster_resource_manager.templates | list | `[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]` | Resource templates that should be applied | +| cluster_resource_manager.templates[0] | object | `{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"}` | Template for namespaces resources | +| common | object | `{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console","nginx.ingress.kubernetes.io/service-upstream":"true"},"enabled":true,"ingressClassName":null,"separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":false}}` | ---------------------------------------------- COMMON SETTINGS | +| common.databaseSecret.name | string | `""` | Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret | +| common.databaseSecret.secretManifest | object | `{}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets | +| common.flyteNamespaceTemplate.enabled | bool | `false` | - Enable or disable creating Flyte namespace in template. Enable when using helm as template-engine only. Disable when using `helm install ...`. | +| common.ingress.albSSLRedirect | bool | `false` | - albSSLRedirect adds a special route for ssl redirect. Only useful in combination with the AWS LoadBalancer Controller. | +| common.ingress.annotations | object | `{"nginx.ingress.kubernetes.io/app-root":"/console","nginx.ingress.kubernetes.io/service-upstream":"true"}` | - Ingress annotations applied to both HTTP and GRPC ingresses. | +| common.ingress.enabled | bool | `true` | - Enable or disable creating Ingress for Flyte. Relevant to disable when using e.g. Istio as ingress controller. | +| common.ingress.ingressClassName | string | `nil` | - Sets the ingressClassName | +| common.ingress.separateGrpcIngress | bool | `false` | - separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx. | +| common.ingress.separateGrpcIngressAnnotations | object | `{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"}` | - Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled. | +| common.ingress.tls | object | `{"enabled":false}` | - Ingress hostname host: | +| common.ingress.webpackHMR | bool | `false` | - Enable or disable HMR route to flyteconsole. This is useful only for frontend development. | +| configmap.admin | object | `{"admin":{"clientId":"{{ .Values.secrets.adminOauthClientCredentials.clientId }}","clientSecretLocation":"/etc/secrets/client_secret","endpoint":"flyteadmin:81","insecure":true},"event":{"capacity":1000,"rate":500,"type":"admin"}}` | Admin Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/subworkflow/launchplan#AdminConfig) | +| configmap.adminServer | object | `{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}}` | FlyteAdmin server configuration | +| configmap.adminServer.auth | object | `{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}}` | Authentication configuration | +| configmap.adminServer.server.security.secure | bool | `false` | Controls whether to serve requests over SSL/TLS. | +| configmap.adminServer.server.security.useAuth | bool | `false` | Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. | +| configmap.catalog | object | `{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}}` | Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) | +| configmap.clusters.clusterConfigs | list | `[]` | | +| configmap.clusters.labelClusterMap | object | `{}` | | +| configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI | +| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration | +| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) | +| configmap.core | object | `{"manager":{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"},"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration | +| configmap.core.manager | object | `{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config). | +| configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). | +| configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"heartbeat-grace-period-multiplier":3,"max-reservation-heartbeat":"30s","metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config | +| configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. | +| configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | +| configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | +| configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","uploader","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker\*, athena if you install the backend plugins | +| configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}}}` | Kubernetes specific Flyte configuration | +| configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) | +| configmap.remoteData.remoteData.region | string | `"us-east-1"` | | +| configmap.remoteData.remoteData.scheme | string | `"local"` | | +| configmap.remoteData.remoteData.signedUrls.durationMinutes | int | `3` | | +| configmap.resource_manager | object | `{"propeller":{"resourcemanager":{"type":"noop"}}}` | Resource manager configuration | +| configmap.resource_manager.propeller | object | `{"resourcemanager":{"type":"noop"}}` | resource manager configuration | +| configmap.schedulerConfig.scheduler.metricsScope | string | `"flyte:"` | | +| configmap.schedulerConfig.scheduler.profilerPort | int | `10254` | | +| configmap.task_logs | object | `{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":false}}}` | Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various logging engines | +| configmap.task_logs.plugins.logs.cloudwatch-enabled | bool | `false` | One option is to enable cloudwatch logging for EKS, update the region and log group accordingly | +| configmap.task_resource_defaults | object | `{"task_resources":{"defaults":{"cpu":"100m","memory":"500Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi"}}}` | Task default resources configuration Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). | +| configmap.task_resource_defaults.task_resources | object | `{"defaults":{"cpu":"100m","memory":"500Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi"}}` | Task default resources parameters | +| daskoperator | object | `{"enabled":false}` | Optional: Dask Plugin using the Dask Operator | +| daskoperator.enabled | bool | `false` | - enable or disable the dask operator deployment installation | +| databricks | object | `{"enabled":false,"plugin_config":{"plugins":{"databricks":{"databricksInstance":"dbc-a53b7a3c-614c","entrypointFile":"dbfs:///FileStore/tables/entrypoint.py"}}}}` | Optional: Databricks Plugin allows us to run the spark job on the Databricks platform. | +| datacatalog.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | +| datacatalog.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | +| datacatalog.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | +| datacatalog.affinity | object | `{}` | affinity for Datacatalog deployment | +| datacatalog.configPath | string | `"/etc/datacatalog/config/*.yaml"` | Default regex string for searching configuration files | +| datacatalog.enabled | bool | `true` | | +| datacatalog.extraArgs | object | `{}` | Appends extra command line arguments to the main command | +| datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment | +| datacatalog.image.tag | string | `"v1.13.2"` | Docker image tag | +| datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment | +| datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods | +| datacatalog.podEnv | object | `{}` | Additional Datacatalog container environment variables | +| datacatalog.podLabels | object | `{}` | Labels for Datacatalog pods | +| datacatalog.priorityClassName | string | `""` | Sets priorityClassName for datacatalog pod(s). | +| datacatalog.replicaCount | int | `1` | Replicas count for Datacatalog deployment | +| datacatalog.resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Datacatalog deployment | +| datacatalog.securityContext | object | `{"fsGroup":1001,"fsGroupChangePolicy":"OnRootMismatch","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for datacatalog pod(s). | +| datacatalog.service | object | `{"additionalPorts":[],"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"}` | Service settings for Datacatalog | +| datacatalog.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | +| datacatalog.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Datacatalog | +| datacatalog.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Datacatalog pods | +| datacatalog.serviceAccount.create | bool | `true` | Should a service account be created for Datacatalog | +| datacatalog.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| datacatalog.tolerations | list | `[]` | tolerations for Datacatalog deployment | +| db.admin.database.dbname | string | `"flyteadmin"` | | +| db.admin.database.host | string | `"postgres"` | | +| db.admin.database.port | int | `5432` | | +| db.admin.database.username | string | `"postgres"` | | +| db.datacatalog.database.dbname | string | `"datacatalog"` | | +| db.datacatalog.database.host | string | `"postgres"` | | +| db.datacatalog.database.port | int | `5432` | | +| db.datacatalog.database.username | string | `"postgres"` | | +| deployRedoc | bool | `false` | | +| external_events | object | `{"aws":{"region":"us-east-2"},"enable":false,"eventsPublisher":{"eventTypes":["all"],"topicName":"arn:aws:sns:us-east-2:123456:123-my-topic"},"type":"aws"}` | **Optional Component** External events are used to send events (unprocessed, as Admin see them) to an SNS topic (or gcp equivalent) The config is here as an example only - if not enabled, it won't be used. | +| flyteadmin.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | +| flyteadmin.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | +| flyteadmin.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | +| flyteadmin.affinity | object | `{}` | affinity for Flyteadmin deployment | +| flyteadmin.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flyteadmin.enabled | bool | `true` | | +| flyteadmin.env | list | `[]` | Additional flyteadmin container environment variables e.g. SendGrid's API key - name: SENDGRID_API_KEY value: "" e.g. secret environment variable (you can combine it with .additionalVolumes): - name: SENDGRID_API_KEY valueFrom: secretKeyRef: name: sendgrid-secret key: api_key | +| flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command | +| flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | | +| flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment | +| flyteadmin.image.tag | string | `"v1.13.2"` | | +| flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create | +| flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment | +| flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods | +| flyteadmin.podLabels | object | `{}` | Labels for Flyteadmin pods | +| flyteadmin.priorityClassName | string | `""` | Sets priorityClassName for flyteadmin pod(s). | +| flyteadmin.replicaCount | int | `1` | Replicas count for Flyteadmin deployment | +| flyteadmin.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flyteadmin deployment | +| flyteadmin.secrets | object | `{}` | | +| flyteadmin.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flyteadmin pod(s). | +| flyteadmin.service | object | `{"additionalPorts":[],"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"}` | Service settings for Flyteadmin | +| flyteadmin.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | +| flyteadmin.serviceAccount | object | `{"alwaysCreate":false,"annotations":{},"clusterRole":{"apiGroups":["","flyte.lyft.com","rbac.authorization.k8s.io"],"resources":["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"],"verbs":["*"]},"create":true,"createClusterRole":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlyteAdmin | +| flyteadmin.serviceAccount.alwaysCreate | bool | `false` | Should a service account always be created for flyteadmin even without an actual flyteadmin deployment running (e.g. for multi-cluster setups) | +| flyteadmin.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flyteadmin pods | +| flyteadmin.serviceAccount.clusterRole | object | `{"apiGroups":["","flyte.lyft.com","rbac.authorization.k8s.io"],"resources":["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"],"verbs":["*"]}` | Configuration for ClusterRole created for Flyteadmin | +| flyteadmin.serviceAccount.clusterRole.apiGroups | list | `["","flyte.lyft.com","rbac.authorization.k8s.io"]` | Specifies the API groups that this ClusterRole can access | +| flyteadmin.serviceAccount.clusterRole.resources | list | `["configmaps","flyteworkflows","namespaces","pods","resourcequotas","roles","rolebindings","secrets","services","serviceaccounts","spark-role","limitranges"]` | Specifies the resources that this ClusterRole can access | +| flyteadmin.serviceAccount.clusterRole.verbs | list | `["*"]` | Specifies the verbs (actions) that this ClusterRole can perform on the specified resources | +| flyteadmin.serviceAccount.create | bool | `true` | Should a service account be created for flyteadmin | +| flyteadmin.serviceAccount.createClusterRole | bool | `true` | Should a ClusterRole be created for Flyteadmin | +| flyteadmin.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyteadmin.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flyteadmin service monitor | +| flyteadmin.serviceMonitor.enabled | bool | `false` | If enabled create the flyteadmin service monitor | +| flyteadmin.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | +| flyteadmin.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | +| flyteadmin.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | +| flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment | +| flyteagent.enabled | bool | `false` | | +| flyteagent.plugin_config.plugins.agent-service | object | `{"defaultAgent":{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true},"supportedTaskTypes":[]}` | Agent service configuration for propeller. | +| flyteagent.plugin_config.plugins.agent-service.defaultAgent | object | `{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true}` | The default agent service to use for plugin tasks. | +| flyteagent.plugin_config.plugins.agent-service.defaultAgent.endpoint | string | `"dns:///flyteagent.flyte.svc.cluster.local:8000"` | The agent service endpoint propeller should connect to. | +| flyteagent.plugin_config.plugins.agent-service.defaultAgent.insecure | bool | `true` | Whether the connection from propeller to the agent service should use TLS. | +| flyteagent.plugin_config.plugins.agent-service.supportedTaskTypes | list | `[]` | The task types supported by the default agent. As of #5460 these are discovered automatically and don't need to be configured. | +| flyteagent.podLabels | object | `{}` | Labels for flyteagent pods | +| flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment | +| flyteconsole.enabled | bool | `true` | | +| flyteconsole.ga.enabled | bool | `false` | | +| flyteconsole.ga.tracking_id | string | `"G-0QW4DJWJ20"` | | +| flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | | +| flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment | +| flyteconsole.image.tag | string | `"v1.17.1"` | | +| flyteconsole.imagePullSecrets | list | `[]` | ImagePullSecrets to assign to the Flyteconsole deployment | +| flyteconsole.livenessProbe | object | `{}` | | +| flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment | +| flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods | +| flyteconsole.podEnv | object | `{}` | Additional Flyteconsole container environment variables | +| flyteconsole.podLabels | object | `{}` | Labels for Flyteconsole pods | +| flyteconsole.priorityClassName | string | `""` | Sets priorityClassName for flyte console pod(s). | +| flyteconsole.readinessProbe | object | `{}` | | +| flyteconsole.replicaCount | int | `1` | Replicas count for Flyteconsole deployment | +| flyteconsole.resources | object | `{"limits":{"cpu":"500m","memory":"250Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Flyteconsole deployment | +| flyteconsole.securityContext | object | `{"fsGroupChangePolicy":"OnRootMismatch","runAsNonRoot":true,"runAsUser":1000,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flyteconsole pod(s). | +| flyteconsole.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Flyteconsole | +| flyteconsole.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flyteconsole service monitor | +| flyteconsole.serviceMonitor.enabled | bool | `false` | If enabled create the flyteconsole service monitor | +| flyteconsole.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | +| flyteconsole.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | +| flyteconsole.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | +| flyteconsole.tolerations | list | `[]` | tolerations for Flyteconsole deployment | +| flytepropeller.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | +| flytepropeller.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | +| flytepropeller.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | +| flytepropeller.affinity | object | `{}` | affinity for Flytepropeller deployment | +| flytepropeller.clusterName | string | `""` | Defines the cluster name used in events sent to Admin | +| flytepropeller.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flytepropeller.createCRDs | bool | `true` | Whether to install the flyteworkflows CRD with helm | +| flytepropeller.enabled | bool | `true` | | +| flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command | +| flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | | +| flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment | +| flytepropeller.image.tag | string | `"v1.13.2"` | | +| flytepropeller.manager | bool | `false` | | +| flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment | +| flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods | +| flytepropeller.podEnv | object | `{}` | Additional Flytepropeller container environment variables | +| flytepropeller.podLabels | object | `{}` | Labels for Flytepropeller pods | +| flytepropeller.priorityClassName | string | `""` | Sets priorityClassName for propeller pod(s). | +| flytepropeller.prometheus.enabled | bool | `false` | | +| flytepropeller.replicaCount | int | `1` | Replicas count for Flytepropeller deployment | +| flytepropeller.resources | object | `{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"100Mi"}}` | Default resources requests and limits for Flytepropeller deployment | +| flytepropeller.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsUser":1001}` | Sets securityContext for flytepropeller pod(s). | +| flytepropeller.service | object | `{"additionalPorts":[],"enabled":false}` | Settings for flytepropeller service | +| flytepropeller.service.additionalPorts | list | `[]` | Appends additional ports to the service spec. | +| flytepropeller.service.enabled | bool | `false` | If enabled create the flytepropeller service | +| flytepropeller.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlytePropeller | +| flytepropeller.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to FlytePropeller pods | +| flytepropeller.serviceAccount.create | bool | `true` | Should a service account be created for FlytePropeller | +| flytepropeller.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flytepropeller.serviceMonitor | object | `{"enabled":false,"interval":"60s","labels":{},"scrapeTimeout":"30s"}` | Settings for flytepropeller service monitor | +| flytepropeller.serviceMonitor.enabled | bool | `false` | If enabled create the flyetepropeller service monitor | +| flytepropeller.serviceMonitor.interval | string | `"60s"` | Sets the interval at which metrics will be scraped by prometheus | +| flytepropeller.serviceMonitor.labels | object | `{}` | Sets the labels for the service monitor which are required by the prometheus to auto-detect the service monitor and start scrapping the metrics | +| flytepropeller.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out | +| flytepropeller.terminationMessagePolicy | string | `"FallbackToLogsOnError"` | Error reporting | +| flytepropeller.tolerations | list | `[]` | tolerations for Flytepropeller deployment | +| flytescheduler.additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. | +| flytescheduler.additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. | +| flytescheduler.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. | +| flytescheduler.affinity | object | `{}` | affinity for Flytescheduler deployment | +| flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment | +| flytescheduler.image.tag | string | `"v1.13.2"` | Docker image tag | +| flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment | +| flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods | +| flytescheduler.podEnv | object | `{}` | Additional Flytescheduler container environment variables | +| flytescheduler.podLabels | object | `{}` | Labels for Flytescheduler pods | +| flytescheduler.priorityClassName | string | `""` | Sets priorityClassName for flyte scheduler pod(s). | +| flytescheduler.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytescheduler deployment | +| flytescheduler.runPrecheck | bool | `true` | Whether to inject an init container which waits on flyteadmin | +| flytescheduler.secrets | object | `{}` | | +| flytescheduler.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for flytescheduler pod(s). | +| flytescheduler.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Flytescheduler | +| flytescheduler.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flytescheduler pods | +| flytescheduler.serviceAccount.create | bool | `true` | Should a service account be created for Flytescheduler | +| flytescheduler.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flytescheduler.tolerations | list | `[]` | tolerations for Flytescheduler deployment | +| secrets.adminOauthClientCredentials.clientId | string | `"flytepropeller"` | | +| secrets.adminOauthClientCredentials.clientSecret | string | `"foobar"` | | +| secrets.adminOauthClientCredentials.enabled | bool | `true` | | +| sparkoperator | object | `{"enabled":false,"plugin_config":{"plugins":{"spark":{"spark-config-default":[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]}}}}` | Optional: Spark Plugin using the Spark Operator | +| sparkoperator.enabled | bool | `false` | - enable or disable Sparkoperator deployment installation | +| sparkoperator.plugin_config | object | `{"plugins":{"spark":{"spark-config-default":[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]}}}` | Spark plugin configuration | +| sparkoperator.plugin_config.plugins.spark.spark-config-default | list | `[{"spark.hadoop.fs.s3a.aws.credentials.provider":"com.amazonaws.auth.DefaultAWSCredentialsProviderChain"},{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]` | Spark default configuration | +| storage | object | `{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"enableMultiContainer":false,"gcs":null,"limits":{"maxDownloadMBs":10},"s3":{"accessKey":"","authType":"iam","region":"us-east-1","secretKey":""},"type":"sandbox"}` | ---------------------------------------------------- STORAGE SETTINGS | +| storage.bucketName | string | `"my-s3-bucket"` | bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. | +| storage.custom | object | `{}` | Settings for storage type custom. See https://github.com/graymeta/stow for supported storage providers/settings. | +| storage.enableMultiContainer | bool | `false` | toggles multi-container storage config | +| storage.gcs | string | `nil` | settings for storage type gcs | +| storage.limits | object | `{"maxDownloadMBs":10}` | default limits being applied to storage config | +| storage.s3 | object | `{"accessKey":"","authType":"iam","region":"us-east-1","secretKey":""}` | settings for storage type s3 | +| storage.s3.accessKey | string | `""` | AWS IAM user access key ID to use for S3 bucket auth, only used if authType is set to accesskey | +| storage.s3.authType | string | `"iam"` | type of authentication to use for S3 buckets, can either be iam or accesskey | +| storage.s3.secretKey | string | `""` | AWS IAM user secret access key to use for S3 bucket auth, only used if authType is set to accesskey | +| storage.type | string | `"sandbox"` | Sets the storage type. Supported values are sandbox, s3, gcs and custom. | +| webhook.autoscaling.enabled | bool | `false` | | +| webhook.autoscaling.maxReplicas | int | `10` | | +| webhook.autoscaling.metrics[0].resource.name | string | `"cpu"` | | +| webhook.autoscaling.metrics[0].resource.target.averageUtilization | int | `80` | | +| webhook.autoscaling.metrics[0].resource.target.type | string | `"Utilization"` | | +| webhook.autoscaling.metrics[0].type | string | `"Resource"` | | +| webhook.autoscaling.metrics[1].resource.name | string | `"memory"` | | +| webhook.autoscaling.metrics[1].resource.target.averageUtilization | int | `80` | | +| webhook.autoscaling.metrics[1].resource.target.type | string | `"Utilization"` | | +| webhook.autoscaling.metrics[1].type | string | `"Resource"` | | +| webhook.autoscaling.minReplicas | int | `1` | | +| webhook.enabled | bool | `true` | enable or disable secrets webhook | +| webhook.nodeSelector | object | `{}` | nodeSelector for webhook deployment | +| webhook.podAnnotations | object | `{}` | Annotations for webhook pods | +| webhook.podEnv | object | `{}` | Additional webhook container environment variables | +| webhook.podLabels | object | `{}` | Labels for webhook pods | +| webhook.priorityClassName | string | `""` | Sets priorityClassName for webhook pod | +| webhook.prometheus.enabled | bool | `false` | | +| webhook.resources.requests.cpu | string | `"200m"` | | +| webhook.resources.requests.ephemeral-storage | string | `"500Mi"` | | +| webhook.resources.requests.memory | string | `"500Mi"` | | +| webhook.securityContext | object | `{"fsGroup":65534,"fsGroupChangePolicy":"Always","runAsNonRoot":true,"runAsUser":1001,"seLinuxOptions":{"type":"spc_t"}}` | Sets securityContext for webhook pod(s). | +| webhook.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for the webhook | +| webhook.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for the webhook | +| webhook.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to the webhook | +| webhook.serviceAccount.create | bool | `true` | Should a service account be created for the webhook | +| webhook.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| workflow_notifications | object | `{"config":{},"enabled":false}` | **Optional Component** Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of various events in their workflows | +| workflow_scheduler | object | `{"config":{},"enabled":false,"type":""}` | **Optional Component** Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, this module, you will not have scheduled launchplans / workflows. Docs: https://docs.flyte.org/en/latest/howto/enable_and_use_schedules.html#setting-up-scheduled-workflows | diff --git a/charts/flyte-core/values-eks.yaml b/charts/flyte-core/values-eks.yaml index 904a30bc78..c9015b8f68 100644 --- a/charts/flyte-core/values-eks.yaml +++ b/charts/flyte-core/values-eks.yaml @@ -4,10 +4,10 @@ userSettings: certificateArn: dbPassword: rdsHost: -# These two storage buckets could be the same or you could specify different buckets if required. Both keys are required. -# Learn more https://docs.flyte.org/en/latest/concepts/data_management.html#understand-how-flyte-handles-data - bucketName: - rawDataBucketName: + # These two storage buckets could be the same or you could specify different buckets if required. Both keys are required. + # Learn more https://docs.flyte.org/en/latest/concepts/data_management.html#understand-how-flyte-handles-data + bucketName: + rawDataBucketName: logGroup: redisHostUrl: redisHostKey: @@ -270,7 +270,6 @@ configmap: # hostPath: "{{ .Values.userSettings.redisHostUrl }}" # hostKey: "{{ .Values.userSettings.redisHostKey }}" - enabled_plugins: # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) tasks: @@ -280,7 +279,7 @@ configmap: # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo @@ -289,12 +288,11 @@ configmap: # - sagemaker_training default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array # sagemaker_custom_training_task: sagemaker_custom_training # sagemaker_custom_training_job_task: sagemaker_custom_training - # -- Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. # Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various # logging engines @@ -342,9 +340,9 @@ workflow_notifications: subject: "Flyte: {{ project }}/{{ domain }}/{{ launch_plan.name }} has {{ phase }}" sender: "flyte@example.com" body: | - "Execution {{ workflow.project }}/{{ workflow.domain }}/{{ workflow.name }}/{{ name }} has {{ phase }}. - Details: https://flyte.example.com/console/projects/{{ project }}/domains/{{ domain }}/executions/{{ name }}. - {{ error }}" + "Execution {{ workflow.project }}/{{ workflow.domain }}/{{ workflow.name }}/{{ name }} has {{ phase }}. + Details: https://flyte.example.com/console/projects/{{ project }}/domains/{{ domain }}/executions/{{ name }}. + {{ error }}" # -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic # cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain @@ -413,8 +411,6 @@ cluster_resource_manager: limits.cpu: {{ projectQuotaCpu }} limits.memory: {{ projectQuotaMemory }} - - # # SPARKOPERATOR # diff --git a/charts/flyte-core/values-gcp.yaml b/charts/flyte-core/values-gcp.yaml index b6b0342ab3..d34e70bb45 100644 --- a/charts/flyte-core/values-gcp.yaml +++ b/charts/flyte-core/values-gcp.yaml @@ -5,10 +5,10 @@ userSettings: googleProjectId: dbHost: dbPassword: -# These two storage buckets could be the same or you could specify different buckets if required. Both keys are required. -# Learn more https://docs.flyte.org/en/latest/concepts/data_management.html#understand-how-flyte-handles-data - bucketName: - rawDataBucketName: + # These two storage buckets could be the same or you could specify different buckets if required. Both keys are required. + # Learn more https://docs.flyte.org/en/latest/concepts/data_management.html#understand-how-flyte-handles-data + bucketName: + rawDataBucketName: hostName: # @@ -294,16 +294,15 @@ configmap: # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array - # -- Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. # Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various # logging engines diff --git a/charts/flyte-core/values-keycloak-idp-flyteclients-without-browser.yaml b/charts/flyte-core/values-keycloak-idp-flyteclients-without-browser.yaml index 1e088ea4b7..862b974f37 100644 --- a/charts/flyte-core/values-keycloak-idp-flyteclients-without-browser.yaml +++ b/charts/flyte-core/values-keycloak-idp-flyteclients-without-browser.yaml @@ -287,7 +287,6 @@ flyteconsole: # -- Sets priorityClassName for flyte console pod(s). priorityClassName: "" - # It will enable the redoc route in ingress deployRedoc: false @@ -427,37 +426,37 @@ db: configmap: clusters: labelClusterMap: {} -# labelClusterMap: -# team1: -# - id: testcluster -# weight: 1 -# team2: -# - id: testcluster2 -# weight: 0.5 -# - id: testcluster3 -# weight: 0.5 + # labelClusterMap: + # team1: + # - id: testcluster + # weight: 1 + # team2: + # - id: testcluster2 + # weight: 0.5 + # - id: testcluster3 + # weight: 0.5 clusterConfigs: [] -# clusterConfigs: -# - name: "testcluster" -# endpoint: "testcluster_endpoint" -# auth: -# type: "file_path" -# tokenPath: "/path/to/testcluster/token" -# certPath: "/path/to/testcluster/cert" -# - name: "testcluster2" -# endpoint: "testcluster2_endpoint" -# enabled: true -# auth: -# type: "file_path" -# tokenPath: "/path/to/testcluster2/token" -# certPath: "/path/to/testcluster2/cert" -# - name: "testcluster3" -# endpoint: "testcluster3_endpoint" -# enabled: true -# auth: -# type: "file_path" -# tokenPath: "/path/to/testcluster3/token" -# certPath: "/path/to/testcluster3/cert" + # clusterConfigs: + # - name: "testcluster" + # endpoint: "testcluster_endpoint" + # auth: + # type: "file_path" + # tokenPath: "/path/to/testcluster/token" + # certPath: "/path/to/testcluster/cert" + # - name: "testcluster2" + # endpoint: "testcluster2_endpoint" + # enabled: true + # auth: + # type: "file_path" + # tokenPath: "/path/to/testcluster2/token" + # certPath: "/path/to/testcluster2/cert" + # - name: "testcluster3" + # endpoint: "testcluster3_endpoint" + # enabled: true + # auth: + # type: "file_path" + # tokenPath: "/path/to/testcluster3/token" + # certPath: "/path/to/testcluster3/cert" # -- Configuration for Flyte console UI console: @@ -527,14 +526,12 @@ configmap: # Controls app authentication config appAuth: - authServerType: External externalAuthServer: allowedAudience: flytepropeller baseUrl: "" # populate this value only if different from userauth baseUrl metadataUrl: ".well-known/openid-configuration" - thirdPartyConfig: flyteClient: audience: flytepropeller @@ -672,13 +669,13 @@ configmap: # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array # -- Kubernetes specific Flyte configuration diff --git a/charts/flyte-core/values.yaml b/charts/flyte-core/values.yaml index 9faaed731a..7253e2219b 100755 --- a/charts/flyte-core/values.yaml +++ b/charts/flyte-core/values.yaml @@ -15,8 +15,8 @@ flyteadmin: replicaCount: 1 image: # -- Docker image for Flyteadmin deployment - repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE - tag: v1.13.2 # FLYTEADMIN_TAG + repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE + tag: v1.13.2 # FLYTEADMIN_TAG pullPolicy: IfNotPresent # -- Additional flyteadmin container environment variables # @@ -142,9 +142,9 @@ flytescheduler: runPrecheck: true image: # -- Docker image for Flytescheduler deployment - repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE + repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE # -- Docker image tag - tag: v1.13.2 # FLYTESCHEDULER_TAG + tag: v1.13.2 # FLYTESCHEDULER_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Flytescheduler deployment @@ -208,9 +208,9 @@ datacatalog: replicaCount: 1 image: # -- Docker image for Datacatalog deployment - repository: cr.flyte.org/flyteorg/datacatalog # DATACATALOG_IMAGE + repository: cr.flyte.org/flyteorg/datacatalog # DATACATALOG_IMAGE # -- Docker image tag - tag: v1.13.2 # DATACATALOG_TAG + tag: v1.13.2 # DATACATALOG_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Datacatalog deployment @@ -306,8 +306,8 @@ flytepropeller: replicaCount: 1 image: # -- Docker image for Flytepropeller deployment - repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE - tag: v1.13.2 # FLYTEPROPELLER_TAG + repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE + tag: v1.13.2 # FLYTEPROPELLER_TAG pullPolicy: IfNotPresent # -- Default resources requests and limits for Flytepropeller deployment resources: @@ -394,8 +394,8 @@ flyteconsole: replicaCount: 1 image: # -- Docker image for Flyteconsole deployment - repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE - tag: v1.17.1 # FLYTECONSOLE_TAG + repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE + tag: v1.17.1 # FLYTECONSOLE_TAG pullPolicy: IfNotPresent # -- Default resources requests and limits for Flyteconsole deployment resources: @@ -426,8 +426,9 @@ flyteconsole: enabled: false tracking_id: "G-0QW4DJWJ20" # -- Sets priorityClassName for flyte console pod(s). - priorityClassName: "" - # -- ImagePullSecrets to assign to the Flyteconsole deployment + priorityClassName: + "" + # -- ImagePullSecrets to assign to the Flyteconsole deployment imagePullSecrets: [] # -- Sets securityContext for flyteconsole pod(s). securityContext: @@ -801,7 +802,7 @@ configmap: # -- Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) co-pilot: name: flyte-copilot- - image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 # FLYTECOPILOT_IMAGE + image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 # FLYTECOPILOT_IMAGE start-timeout: 30s # -- Core propeller configuration @@ -860,13 +861,13 @@ configmap: # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array # -- Uncomment to enable task type that uses Flyte Agent # bigquery_query_job_task: agent-service @@ -940,7 +941,7 @@ external_events: # writing to fifo sns topics. topicName: "arn:aws:sns:us-east-2:123456:123-my-topic" eventTypes: - - all # Or workflow, node, task. Or "*" + - all # Or workflow, node, task. Or "*" # Cloud events are used to send events (unprocessed, as Admin see them) in cloud event format to # an SNS topic (or gcp equivalent) @@ -951,7 +952,7 @@ cloud_events: # writing to fifo sns topics. topicName: "arn:aws:sns:us-east-2:123456:123-my-topic" eventTypes: - - all # Or workflow, node, task. Or "*" + - all # Or workflow, node, task. Or "*" type: aws # -- Configuration for sending cloud events to AWS SNS aws: diff --git a/charts/flyte/README.md b/charts/flyte/README.md index 944c624ab6..f481e91cfa 100644 --- a/charts/flyte/README.md +++ b/charts/flyte/README.md @@ -6,17 +6,18 @@ A Helm chart for Flyte Sandbox ## Requirements -| Repository | Name | Version | -|------------|------|---------| -| file://../flyte-core | flyte(flyte-core) | v0.1.10 | -| https://charts.bitnami.com/bitnami | contour | 7.10.1 | -| https://helm.dask.org | daskoperator(dask-kubernetes-operator) | 2022.12.0 | -| https://kubeflow.github.io/spark-operator | sparkoperator(spark-operator) | 1.1.15 | -| https://kubernetes.github.io/dashboard/ | kubernetes-dashboard | 4.0.2 | +| Repository | Name | Version | +| ----------------------------------------- | -------------------------------------- | --------- | +| file://../flyte-core | flyte(flyte-core) | v0.1.10 | +| https://charts.bitnami.com/bitnami | contour | 7.10.1 | +| https://helm.dask.org | daskoperator(dask-kubernetes-operator) | 2022.12.0 | +| https://kubeflow.github.io/spark-operator | sparkoperator(spark-operator) | 1.1.15 | +| https://kubernetes.github.io/dashboard/ | kubernetes-dashboard | 4.0.2 | **NOTE:** Flyte sandbox helm chart is deprecated, From now follow the sandbox [docs](https://docs.flyte.org/en/latest/deployment/sandbox.html) for installing it on cloud ### SANDBOX INSTALLATION: + - [Install helm 3](https://helm.sh/docs/intro/install/) - Install Flyte sandbox: @@ -34,11 +35,13 @@ helm diff upgrade -f values-sandbox.yaml flyte . ``` Then apply your changes: + ```bash helm upgrade -f values-sandbox.yaml flyte . ``` #### Alternative: Generate raw kubernetes yaml with helm template + - `helm template --name-template=flyte-sandbox . -n flyte -f values-sandbox.yaml > flyte_generated_sandbox.yaml` - Deploy the manifest `kubectl apply -f flyte_generated_sandbox.yaml` @@ -46,207 +49,208 @@ helm upgrade -f values-sandbox.yaml flyte . - If running on minikube, get flyte host using `minikube service contour -n heptio-contour --url`. And then visit `http:///console` ### CONFIGURATION NOTES: + - The docker images, their tags and other default parameters are configured in `values.yaml` file. - Each Flyte installation type should have separate `values-*.yaml` file: for sandbox, EKS and etc. The configuration in `values.yaml` and the chosen config `values-*.yaml` are merged when generating the deployment manifest. ## Values -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| contour.affinity | object | `{}` | affinity for Contour deployment | -| contour.contour.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Contour | -| contour.contour.resources.limits | object | `{"cpu":"100m","memory":"100Mi"}` | Limits are the maximum set of resources needed for this pod | -| contour.contour.resources.requests | object | `{"cpu":"10m","memory":"50Mi"}` | Requests are the minimum set of resources needed for this pod | -| contour.enabled | bool | `true` | - enable or disable Contour deployment installation | -| contour.envoy.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Envoy | -| contour.envoy.resources.limits | object | `{"cpu":"100m","memory":"100Mi"}` | Limits are the maximum set of resources needed for this pod | -| contour.envoy.resources.requests | object | `{"cpu":"10m","memory":"50Mi"}` | Requests are the minimum set of resources needed for this pod | -| contour.envoy.service.nodePorts.http | int | `30081` | | -| contour.envoy.service.ports.http | int | `80` | | -| contour.envoy.service.type | string | `"NodePort"` | | -| contour.nodeSelector | object | `{}` | nodeSelector for Contour deployment | -| contour.podAnnotations | object | `{}` | Annotations for Contour pods | -| contour.replicaCount | int | `1` | Replicas count for Contour deployment | -| contour.serviceAccountAnnotations | object | `{}` | Annotations for ServiceAccount attached to Contour pods | -| contour.tolerations | list | `[]` | tolerations for Contour deployment | -| daskoperator | object | `{"enabled":false}` | Optional: Dask Plugin using the Dask Operator | -| daskoperator.enabled | bool | `false` | - enable or disable the dask operator deployment installation | -| flyte | object | `{"cluster_resource_manager":{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"service_account_name":"flyteadmin","templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]},"common":{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console"},"enabled":true,"host":"","separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":true}},"configmap":{"adminServer":{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}},"catalog":{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}},"console":{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"},"copilot":{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}},"core":{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}},"datacatalogServer":{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}},"domain":{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]},"enabled_plugins":{"tasks":{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}}},"k8s":{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}},"logger":{"logger":{"level":5,"show-source":true}},"remoteData":{"remoteData":{"region":"us-east-1","scheme":"local","signedUrls":{"durationMinutes":3}}},"resource_manager":{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}},"task_logs":{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}},"task_resource_defaults":{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}},"datacatalog":{"affinity":{},"configPath":"/etc/datacatalog/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/datacatalog","tag":"v1.13.2"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"db":{"admin":{"database":{"dbname":"flyteadmin","host":"postgres","port":5432,"username":"postgres"}},"datacatalog":{"database":{"dbname":"datacatalog","host":"postgres","port":5432,"username":"postgres"}}},"deployRedoc":true,"flyteadmin":{"additionalVolumeMounts":[],"additionalVolumes":[],"affinity":{},"configPath":"/etc/flyte/config/*.yaml","env":[],"image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flyteadmin","tag":"v1.13.2"},"initialProjects":["flytesnacks","flytetester","flyteexamples"],"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"secrets":{},"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"flyteconsole":{"affinity":{},"ga":{"enabled":true,"tracking_id":"G-0QW4DJWJ20"},"image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flyteconsole","tag":"v1.17.1"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"500m","memory":"275Mi"},"requests":{"cpu":"10m","memory":"250Mi"}},"service":{"annotations":{},"type":"ClusterIP"},"tolerations":[]},"flytepropeller":{"affinity":{},"configPath":"/etc/flyte/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flytepropeller","tag":"v1.13.2"},"manager":false,"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"flytescheduler":{"affinity":{},"configPath":"/etc/flyte/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flytescheduler","tag":"v1.13.2"},"nodeSelector":{},"podAnnotations":{},"resources":{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"secrets":{},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"storage":{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"gcs":null,"s3":{"region":"us-east-1"},"type":"sandbox"},"webhook":{"enabled":true,"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]}},"workflow_notifications":{"config":{},"enabled":false},"workflow_scheduler":{"enabled":true,"type":"native"}}` | ------------------------------------------------------------------- Core System settings This section consists of Core components of Flyte and their deployment settings. This includes FlyteAdmin service, Datacatalog, FlytePropeller and Flyteconsole | -| flyte.cluster_resource_manager | object | `{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"service_account_name":"flyteadmin","templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]}` | Configuration for the Cluster resource manager component. This is an optional component, that enables automatic cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain | -| flyte.cluster_resource_manager.config.cluster_resources | object | `{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}` | ClusterResource parameters Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. | -| flyte.cluster_resource_manager.config.cluster_resources.standaloneDeployment | bool | `false` | Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints | -| flyte.cluster_resource_manager.enabled | bool | `true` | Enables the Cluster resource manager component | -| flyte.cluster_resource_manager.service_account_name | string | `"flyteadmin"` | Service account name to run with | -| flyte.cluster_resource_manager.templates | list | `[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]` | Resource templates that should be applied | -| flyte.cluster_resource_manager.templates[0] | object | `{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"}` | Template for namespaces resources | -| flyte.common | object | `{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console"},"enabled":true,"host":"","separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":true}}` | ---------------------------------------------- COMMON SETTINGS | -| flyte.common.databaseSecret.name | string | `""` | Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret | -| flyte.common.databaseSecret.secretManifest | object | `{}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets | -| flyte.common.flyteNamespaceTemplate.enabled | bool | `false` | - Enable or disable creating Flyte namespace in template. Enable when using helm as template-engine only. Disable when using `helm install ...`. | -| flyte.common.ingress.albSSLRedirect | bool | `false` | - albSSLRedirect adds a special route for ssl redirect. Only useful in combination with the AWS LoadBalancer Controller. | -| flyte.common.ingress.annotations | object | `{"nginx.ingress.kubernetes.io/app-root":"/console"}` | - Ingress annotations applied to both HTTP and GRPC ingresses. | -| flyte.common.ingress.enabled | bool | `true` | - Enable or disable creating Ingress for Flyte. Relevant to disable when using e.g. Istio as ingress controller. | -| flyte.common.ingress.host | string | `""` | - Ingress hostname | -| flyte.common.ingress.separateGrpcIngress | bool | `false` | - separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx. | -| flyte.common.ingress.separateGrpcIngressAnnotations | object | `{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"}` | - Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled. | -| flyte.common.ingress.tls | object | `{"enabled":false}` | - TLS Settings | -| flyte.common.ingress.webpackHMR | bool | `true` | - Enable or disable HMR route to flyteconsole. This is useful only for frontend development. | -| flyte.configmap | object | `{"adminServer":{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}},"catalog":{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}},"console":{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"},"copilot":{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}},"core":{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}},"datacatalogServer":{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}},"domain":{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]},"enabled_plugins":{"tasks":{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}}},"k8s":{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}},"logger":{"logger":{"level":5,"show-source":true}},"remoteData":{"remoteData":{"region":"us-east-1","scheme":"local","signedUrls":{"durationMinutes":3}}},"resource_manager":{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}},"task_logs":{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}},"task_resource_defaults":{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}}` | ----------------------------------------------------------------- CONFIGMAPS SETTINGS | -| flyte.configmap.adminServer | object | `{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}}` | FlyteAdmin server configuration | -| flyte.configmap.adminServer.auth | object | `{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}}` | Authentication configuration | -| flyte.configmap.adminServer.server.security.secure | bool | `false` | Controls whether to serve requests over SSL/TLS. | -| flyte.configmap.adminServer.server.security.useAuth | bool | `false` | Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. | -| flyte.configmap.catalog | object | `{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}}` | Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) | -| flyte.configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI | -| flyte.configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration | -| flyte.configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) | -| flyte.configmap.core | object | `{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration | -| flyte.configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). | -| flyte.configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config | -| flyte.configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. | -| flyte.configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | -| flyte.configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | -| flyte.configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins | -| flyte.configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}}` | Kubernetes specific Flyte configuration | -| flyte.configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) | -| flyte.configmap.logger | object | `{"logger":{"level":5,"show-source":true}}` | Logger configuration | -| flyte.configmap.resource_manager | object | `{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}}` | Resource manager configuration | -| flyte.configmap.resource_manager.propeller | object | `{"resourcemanager":{"redis":null,"type":"noop"}}` | resource manager configuration | -| flyte.configmap.task_logs | object | `{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}}` | Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various logging engines | -| flyte.configmap.task_logs.plugins.logs.cloudwatch-enabled | bool | `false` | One option is to enable cloudwatch logging for EKS, update the region and log group accordingly | -| flyte.configmap.task_resource_defaults | object | `{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}` | Task default resources configuration Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). | -| flyte.configmap.task_resource_defaults.task_resources | object | `{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}` | Task default resources parameters | -| flyte.datacatalog.affinity | object | `{}` | affinity for Datacatalog deployment | -| flyte.datacatalog.configPath | string | `"/etc/datacatalog/config/*.yaml"` | Default regex string for searching configuration files | -| flyte.datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flyte.datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment | -| flyte.datacatalog.image.tag | string | `"v1.13.2"` | Docker image tag | -| flyte.datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment | -| flyte.datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods | -| flyte.datacatalog.replicaCount | int | `1` | Replicas count for Datacatalog deployment | -| flyte.datacatalog.resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Datacatalog deployment | -| flyte.datacatalog.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"}` | Service settings for Datacatalog | -| flyte.datacatalog.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Datacatalog | -| flyte.datacatalog.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Datacatalog pods | -| flyte.datacatalog.serviceAccount.create | bool | `true` | Should a service account be created for Datacatalog | -| flyte.datacatalog.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyte.datacatalog.tolerations | list | `[]` | tolerations for Datacatalog deployment | -| flyte.flyteadmin.affinity | object | `{}` | affinity for Flyteadmin deployment | -| flyte.flyteadmin.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flyte.flyteadmin.env | list | `[]` | Additional flyteadmin container environment variables e.g. SendGrid's API key - name: SENDGRID_API_KEY value: "" e.g. secret environment variable (you can combine it with .additionalVolumes): - name: SENDGRID_API_KEY valueFrom: secretKeyRef: name: sendgrid-secret key: api_key | -| flyte.flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flyte.flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment | -| flyte.flyteadmin.image.tag | string | `"v1.13.2"` | Docker image tag | -| flyte.flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create | -| flyte.flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment | -| flyte.flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods | -| flyte.flyteadmin.replicaCount | int | `1` | Replicas count for Flyteadmin deployment | -| flyte.flyteadmin.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flyteadmin deployment | -| flyte.flyteadmin.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"}` | Service settings for Flyteadmin | -| flyte.flyteadmin.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlyteAdmin | -| flyte.flyteadmin.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flyteadmin pods | -| flyte.flyteadmin.serviceAccount.create | bool | `true` | Should a service account be created for flyteadmin | -| flyte.flyteadmin.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyte.flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment | -| flyte.flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment | -| flyte.flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flyte.flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment | -| flyte.flyteconsole.image.tag | string | `"v1.17.1"` | Docker image tag | -| flyte.flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment | -| flyte.flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods | -| flyte.flyteconsole.replicaCount | int | `1` | Replicas count for Flyteconsole deployment | -| flyte.flyteconsole.resources | object | `{"limits":{"cpu":"500m","memory":"275Mi"},"requests":{"cpu":"10m","memory":"250Mi"}}` | Default resources requests and limits for Flyteconsole deployment | -| flyte.flyteconsole.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Flyteconsole | -| flyte.flyteconsole.tolerations | list | `[]` | tolerations for Flyteconsole deployment | -| flyte.flytepropeller.affinity | object | `{}` | affinity for Flytepropeller deployment | -| flyte.flytepropeller.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flyte.flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flyte.flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment | -| flyte.flytepropeller.image.tag | string | `"v1.13.2"` | Docker image tag | -| flyte.flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment | -| flyte.flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods | -| flyte.flytepropeller.replicaCount | int | `1` | Replicas count for Flytepropeller deployment | -| flyte.flytepropeller.resources | object | `{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytepropeller deployment | -| flyte.flytepropeller.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlytePropeller | -| flyte.flytepropeller.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to FlytePropeller pods | -| flyte.flytepropeller.serviceAccount.create | bool | `true` | Should a service account be created for FlytePropeller | -| flyte.flytepropeller.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyte.flytepropeller.tolerations | list | `[]` | tolerations for Flytepropeller deployment | -| flyte.flytescheduler.affinity | object | `{}` | affinity for Flytescheduler deployment | -| flyte.flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | -| flyte.flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| flyte.flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment | -| flyte.flytescheduler.image.tag | string | `"v1.13.2"` | Docker image tag | -| flyte.flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment | -| flyte.flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods | -| flyte.flytescheduler.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytescheduler deployment | -| flyte.flytescheduler.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Flytescheduler | -| flyte.flytescheduler.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flytescheduler pods | -| flyte.flytescheduler.serviceAccount.create | bool | `true` | Should a service account be created for Flytescheduler | -| flyte.flytescheduler.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyte.flytescheduler.tolerations | list | `[]` | tolerations for Flytescheduler deployment | -| flyte.storage | object | `{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"gcs":null,"s3":{"region":"us-east-1"},"type":"sandbox"}` | ---------------------------------------------------- STORAGE SETTINGS | -| flyte.storage.bucketName | string | `"my-s3-bucket"` | bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. | -| flyte.storage.custom | object | `{}` | Settings for storage type custom. See https://github.com/graymeta/stow for supported storage providers/settings. | -| flyte.storage.gcs | string | `nil` | settings for storage type gcs | -| flyte.storage.s3 | object | `{"region":"us-east-1"}` | settings for storage type s3 | -| flyte.storage.type | string | `"sandbox"` | Sets the storage type. Supported values are sandbox, s3, gcs and custom. | -| flyte.webhook.enabled | bool | `true` | enable or disable secrets webhook | -| flyte.webhook.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for the webhook | -| flyte.webhook.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for the webhook | -| flyte.webhook.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to the webhook | -| flyte.webhook.serviceAccount.create | bool | `true` | Should a service account be created for the webhook | -| flyte.webhook.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | -| flyte.workflow_notifications | object | `{"config":{},"enabled":false}` | **Optional Component** Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of various events in their workflows | -| flyte.workflow_scheduler | object | `{"enabled":true,"type":"native"}` | **Optional Component** Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, this module, you will not have scheduled launchplans / workflows. Docs: https://docs.flyte.org/en/latest/howto/enable_and_use_schedules.html#setting-up-scheduled-workflows | -| flyteagent.enabled | bool | `true` | | -| kubernetes-dashboard.enabled | bool | `true` | | -| kubernetes-dashboard.extraArgs[0] | string | `"--enable-skip-login"` | | -| kubernetes-dashboard.extraArgs[1] | string | `"--enable-insecure-login"` | | -| kubernetes-dashboard.extraArgs[2] | string | `"--disable-settings-authorizer"` | | -| kubernetes-dashboard.protocolHttp | bool | `true` | | -| kubernetes-dashboard.rbac.clusterReadOnlyRole | bool | `true` | | -| kubernetes-dashboard.service.externalPort | int | `30082` | | -| kubernetes-dashboard.service.nodePort | int | `30082` | | -| kubernetes-dashboard.service.type | string | `"NodePort"` | | -| minio.affinity | object | `{}` | affinity for Minio deployment | -| minio.enabled | bool | `true` | - enable or disable Minio deployment installation | -| minio.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| minio.image.repository | string | `"ecr.flyte.org/bitnami/minio"` | Docker image for Minio deployment | -| minio.image.tag | string | `"2021.10.13-debian-10-r0"` | Docker image tag | -| minio.nodeSelector | object | `{}` | nodeSelector for Minio deployment | -| minio.podAnnotations | object | `{}` | Annotations for Minio pods | -| minio.replicaCount | int | `1` | Replicas count for Minio deployment | -| minio.resources | object | `{"limits":{"cpu":"200m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Minio deployment | -| minio.resources.limits | object | `{"cpu":"200m","memory":"512Mi"}` | Limits are the maximum set of resources needed for this pod | -| minio.resources.requests | object | `{"cpu":"10m","memory":"128Mi"}` | Requests are the minimum set of resources needed for this pod | -| minio.service | object | `{"annotations":{},"type":"NodePort"}` | Service settings for Minio | -| minio.tolerations | list | `[]` | tolerations for Minio deployment | -| postgres.affinity | object | `{}` | affinity for Postgres deployment | -| postgres.enabled | bool | `true` | - enable or disable Postgres deployment installation | -| postgres.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| postgres.image.repository | string | `"ecr.flyte.org/ubuntu/postgres"` | Docker image for Postgres deployment | -| postgres.image.tag | string | `"13-21.04_beta"` | Docker image tag | -| postgres.nodeSelector | object | `{}` | nodeSelector for Postgres deployment | -| postgres.podAnnotations | object | `{}` | Annotations for Postgres pods | -| postgres.replicaCount | int | `1` | Replicas count for Postgres deployment | -| postgres.resources | object | `{"limits":{"cpu":"1000m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Postgres deployment | -| postgres.service | object | `{"annotations":{},"type":"NodePort"}` | Service settings for Postgres | -| postgres.tolerations | list | `[]` | tolerations for Postgres deployment | -| redis | object | `{"enabled":false}` | --------------------------------------------- REDIS SETTINGS | -| redis.enabled | bool | `false` | - enable or disable Redis Statefulset installation | -| redoc.affinity | object | `{}` | affinity for redoc deployment | -| redoc.enabled | bool | `true` | - enable or disable redoc deployment installation | -| redoc.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | -| redoc.image.repository | string | `"docker.io/redocly/redoc"` | Docker image for redoc deployment | -| redoc.image.tag | string | `"latest"` | Docker image tag | -| redoc.nodeSelector | object | `{}` | nodeSelector for redoc deployment | -| redoc.podAnnotations | object | `{}` | Annotations for redoc pods | -| redoc.replicaCount | int | `1` | Replicas count for redoc deployment | -| redoc.resources | object | `{"limits":{"cpu":"200m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for redoc deployment | -| redoc.resources.limits | object | `{"cpu":"200m","memory":"512Mi"}` | Limits are the maximum set of resources needed for this pod | -| redoc.resources.requests | object | `{"cpu":"10m","memory":"128Mi"}` | Requests are the minimum set of resources needed for this pod | -| redoc.service | object | `{"type":"ClusterIP"}` | Service settings for redoc | -| redoc.tolerations | list | `[]` | tolerations for redoc deployment | -| sparkoperator | object | `{"enabled":false}` | Optional: Spark Plugin using the Spark Operator | -| sparkoperator.enabled | bool | `false` | - enable or disable Sparkoperator deployment installation | +| Key | Type | Default | Description | +| ---------------------------------------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| contour.affinity | object | `{}` | affinity for Contour deployment | +| contour.contour.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Contour | +| contour.contour.resources.limits | object | `{"cpu":"100m","memory":"100Mi"}` | Limits are the maximum set of resources needed for this pod | +| contour.contour.resources.requests | object | `{"cpu":"10m","memory":"50Mi"}` | Requests are the minimum set of resources needed for this pod | +| contour.enabled | bool | `true` | - enable or disable Contour deployment installation | +| contour.envoy.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Envoy | +| contour.envoy.resources.limits | object | `{"cpu":"100m","memory":"100Mi"}` | Limits are the maximum set of resources needed for this pod | +| contour.envoy.resources.requests | object | `{"cpu":"10m","memory":"50Mi"}` | Requests are the minimum set of resources needed for this pod | +| contour.envoy.service.nodePorts.http | int | `30081` | | +| contour.envoy.service.ports.http | int | `80` | | +| contour.envoy.service.type | string | `"NodePort"` | | +| contour.nodeSelector | object | `{}` | nodeSelector for Contour deployment | +| contour.podAnnotations | object | `{}` | Annotations for Contour pods | +| contour.replicaCount | int | `1` | Replicas count for Contour deployment | +| contour.serviceAccountAnnotations | object | `{}` | Annotations for ServiceAccount attached to Contour pods | +| contour.tolerations | list | `[]` | tolerations for Contour deployment | +| daskoperator | object | `{"enabled":false}` | Optional: Dask Plugin using the Dask Operator | +| daskoperator.enabled | bool | `false` | - enable or disable the dask operator deployment installation | +| flyte | object | `{"cluster_resource_manager":{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"service_account_name":"flyteadmin","templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]},"common":{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console"},"enabled":true,"host":"","separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":true}},"configmap":{"adminServer":{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}},"catalog":{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}},"console":{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"},"copilot":{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}},"core":{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}},"datacatalogServer":{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}},"domain":{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]},"enabled_plugins":{"tasks":{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}}},"k8s":{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}},"logger":{"logger":{"level":5,"show-source":true}},"remoteData":{"remoteData":{"region":"us-east-1","scheme":"local","signedUrls":{"durationMinutes":3}}},"resource_manager":{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}},"task_logs":{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}},"task_resource_defaults":{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}},"datacatalog":{"affinity":{},"configPath":"/etc/datacatalog/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/datacatalog","tag":"v1.13.2"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"db":{"admin":{"database":{"dbname":"flyteadmin","host":"postgres","port":5432,"username":"postgres"}},"datacatalog":{"database":{"dbname":"datacatalog","host":"postgres","port":5432,"username":"postgres"}}},"deployRedoc":true,"flyteadmin":{"additionalVolumeMounts":[],"additionalVolumes":[],"affinity":{},"configPath":"/etc/flyte/config/*.yaml","env":[],"image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flyteadmin","tag":"v1.13.2"},"initialProjects":["flytesnacks","flytetester","flyteexamples"],"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"secrets":{},"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"flyteconsole":{"affinity":{},"ga":{"enabled":true,"tracking_id":"G-0QW4DJWJ20"},"image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flyteconsole","tag":"v1.17.1"},"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"500m","memory":"275Mi"},"requests":{"cpu":"10m","memory":"250Mi"}},"service":{"annotations":{},"type":"ClusterIP"},"tolerations":[]},"flytepropeller":{"affinity":{},"configPath":"/etc/flyte/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flytepropeller","tag":"v1.13.2"},"manager":false,"nodeSelector":{},"podAnnotations":{},"replicaCount":1,"resources":{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"flytescheduler":{"affinity":{},"configPath":"/etc/flyte/config/*.yaml","image":{"pullPolicy":"IfNotPresent","repository":"cr.flyte.org/flyteorg/flytescheduler","tag":"v1.13.2"},"nodeSelector":{},"podAnnotations":{},"resources":{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}},"secrets":{},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]},"tolerations":[]},"storage":{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"gcs":null,"s3":{"region":"us-east-1"},"type":"sandbox"},"webhook":{"enabled":true,"service":{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"},"serviceAccount":{"annotations":{},"create":true,"imagePullSecrets":[]}},"workflow_notifications":{"config":{},"enabled":false},"workflow_scheduler":{"enabled":true,"type":"native"}}` | ------------------------------------------------------------------- Core System settings This section consists of Core components of Flyte and their deployment settings. This includes FlyteAdmin service, Datacatalog, FlytePropeller and Flyteconsole | +| flyte.cluster_resource_manager | object | `{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"service_account_name":"flyteadmin","templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]}` | Configuration for the Cluster resource manager component. This is an optional component, that enables automatic cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain | +| flyte.cluster_resource_manager.config.cluster_resources | object | `{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","refreshInterval":"5m","standaloneDeployment":false,"templatePath":"/etc/flyte/clusterresource/templates"}` | ClusterResource parameters Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. | +| flyte.cluster_resource_manager.config.cluster_resources.standaloneDeployment | bool | `false` | Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints | +| flyte.cluster_resource_manager.enabled | bool | `true` | Enables the Cluster resource manager component | +| flyte.cluster_resource_manager.service_account_name | string | `"flyteadmin"` | Service account name to run with | +| flyte.cluster_resource_manager.templates | list | `[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]` | Resource templates that should be applied | +| flyte.cluster_resource_manager.templates[0] | object | `{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"}` | Template for namespaces resources | +| flyte.common | object | `{"databaseSecret":{"name":"","secretManifest":{}},"flyteNamespaceTemplate":{"enabled":false},"ingress":{"albSSLRedirect":false,"annotations":{"nginx.ingress.kubernetes.io/app-root":"/console"},"enabled":true,"host":"","separateGrpcIngress":false,"separateGrpcIngressAnnotations":{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"},"tls":{"enabled":false},"webpackHMR":true}}` | ---------------------------------------------- COMMON SETTINGS | +| flyte.common.databaseSecret.name | string | `""` | Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret | +| flyte.common.databaseSecret.secretManifest | object | `{}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets | +| flyte.common.flyteNamespaceTemplate.enabled | bool | `false` | - Enable or disable creating Flyte namespace in template. Enable when using helm as template-engine only. Disable when using `helm install ...`. | +| flyte.common.ingress.albSSLRedirect | bool | `false` | - albSSLRedirect adds a special route for ssl redirect. Only useful in combination with the AWS LoadBalancer Controller. | +| flyte.common.ingress.annotations | object | `{"nginx.ingress.kubernetes.io/app-root":"/console"}` | - Ingress annotations applied to both HTTP and GRPC ingresses. | +| flyte.common.ingress.enabled | bool | `true` | - Enable or disable creating Ingress for Flyte. Relevant to disable when using e.g. Istio as ingress controller. | +| flyte.common.ingress.host | string | `""` | - Ingress hostname | +| flyte.common.ingress.separateGrpcIngress | bool | `false` | - separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx. | +| flyte.common.ingress.separateGrpcIngressAnnotations | object | `{"nginx.ingress.kubernetes.io/backend-protocol":"GRPC"}` | - Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled. | +| flyte.common.ingress.tls | object | `{"enabled":false}` | - TLS Settings | +| flyte.common.ingress.webpackHMR | bool | `true` | - Enable or disable HMR route to flyteconsole. This is useful only for frontend development. | +| flyte.configmap | object | `{"adminServer":{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}},"catalog":{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}},"console":{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"},"copilot":{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}},"core":{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}},"datacatalogServer":{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}},"domain":{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]},"enabled_plugins":{"tasks":{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}}},"k8s":{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}},"logger":{"logger":{"level":5,"show-source":true}},"remoteData":{"remoteData":{"region":"us-east-1","scheme":"local","signedUrls":{"durationMinutes":3}}},"resource_manager":{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}},"task_logs":{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}},"task_resource_defaults":{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}}` | ----------------------------------------------------------------- CONFIGMAPS SETTINGS | +| flyte.configmap.adminServer | object | `{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":2,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpc":{"port":8089},"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}}` | FlyteAdmin server configuration | +| flyte.configmap.adminServer.auth | object | `{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"http://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}}` | Authentication configuration | +| flyte.configmap.adminServer.server.security.secure | bool | `false` | Controls whether to serve requests over SSL/TLS. | +| flyte.configmap.adminServer.server.security.useAuth | bool | `false` | Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. | +| flyte.configmap.catalog | object | `{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}}` | Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) | +| flyte.configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI | +| flyte.configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration | +| flyte.configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) | +| flyte.configmap.core | object | `{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration | +| flyte.configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). | +| flyte.configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config | +| flyte.configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. | +| flyte.configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | +| flyte.configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sensor":"agent-service","uploader":"uploader"},"enabled-plugins":["container","uploader","k8s-array","agent-service","echo"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | +| flyte.configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","uploader","k8s-array","agent-service","echo"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker\*, athena if you install the backend plugins | +| flyte.configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}}}` | Kubernetes specific Flyte configuration | +| flyte.configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-from-configmaps":[],"default-env-from-secrets":[],"default-env-vars":[{"FLYTE_AWS_ENDPOINT":"http://minio.flyte:9000"},{"FLYTE_AWS_ACCESS_KEY_ID":"minio"},{"FLYTE_AWS_SECRET_ACCESS_KEY":"miniostorage"}],"default-memory":"200Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) | +| flyte.configmap.logger | object | `{"logger":{"level":5,"show-source":true}}` | Logger configuration | +| flyte.configmap.resource_manager | object | `{"propeller":{"resourcemanager":{"redis":null,"type":"noop"}}}` | Resource manager configuration | +| flyte.configmap.resource_manager.propeller | object | `{"resourcemanager":{"redis":null,"type":"noop"}}` | resource manager configuration | +| flyte.configmap.task_logs | object | `{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":true,"kubernetes-template-uri":"http://localhost:30082/#/log/{{ \"{{\" }} .namespace {{ \"}}\" }}/{{ \"{{\" }} .podName {{ \"}}\" }}/pod?namespace={{ \"{{\" }} .namespace {{ \"}}\" }}"}}}` | Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various logging engines | +| flyte.configmap.task_logs.plugins.logs.cloudwatch-enabled | bool | `false` | One option is to enable cloudwatch logging for EKS, update the region and log group accordingly | +| flyte.configmap.task_resource_defaults | object | `{"task_resources":{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}}` | Task default resources configuration Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). | +| flyte.configmap.task_resource_defaults.task_resources | object | `{"defaults":{"cpu":"100m","memory":"200Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"1Gi","storage":"20Mi"}}` | Task default resources parameters | +| flyte.datacatalog.affinity | object | `{}` | affinity for Datacatalog deployment | +| flyte.datacatalog.configPath | string | `"/etc/datacatalog/config/*.yaml"` | Default regex string for searching configuration files | +| flyte.datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flyte.datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment | +| flyte.datacatalog.image.tag | string | `"v1.13.2"` | Docker image tag | +| flyte.datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment | +| flyte.datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods | +| flyte.datacatalog.replicaCount | int | `1` | Replicas count for Datacatalog deployment | +| flyte.datacatalog.resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Datacatalog deployment | +| flyte.datacatalog.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"}` | Service settings for Datacatalog | +| flyte.datacatalog.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Datacatalog | +| flyte.datacatalog.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Datacatalog pods | +| flyte.datacatalog.serviceAccount.create | bool | `true` | Should a service account be created for Datacatalog | +| flyte.datacatalog.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyte.datacatalog.tolerations | list | `[]` | tolerations for Datacatalog deployment | +| flyte.flyteadmin.affinity | object | `{}` | affinity for Flyteadmin deployment | +| flyte.flyteadmin.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flyte.flyteadmin.env | list | `[]` | Additional flyteadmin container environment variables e.g. SendGrid's API key - name: SENDGRID_API_KEY value: "" e.g. secret environment variable (you can combine it with .additionalVolumes): - name: SENDGRID_API_KEY valueFrom: secretKeyRef: name: sendgrid-secret key: api_key | +| flyte.flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flyte.flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment | +| flyte.flyteadmin.image.tag | string | `"v1.13.2"` | Docker image tag | +| flyte.flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create | +| flyte.flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment | +| flyte.flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods | +| flyte.flyteadmin.replicaCount | int | `1` | Replicas count for Flyteadmin deployment | +| flyte.flyteadmin.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flyteadmin deployment | +| flyte.flyteadmin.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"}` | Service settings for Flyteadmin | +| flyte.flyteadmin.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlyteAdmin | +| flyte.flyteadmin.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flyteadmin pods | +| flyte.flyteadmin.serviceAccount.create | bool | `true` | Should a service account be created for flyteadmin | +| flyte.flyteadmin.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyte.flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment | +| flyte.flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment | +| flyte.flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flyte.flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment | +| flyte.flyteconsole.image.tag | string | `"v1.17.1"` | Docker image tag | +| flyte.flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment | +| flyte.flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods | +| flyte.flyteconsole.replicaCount | int | `1` | Replicas count for Flyteconsole deployment | +| flyte.flyteconsole.resources | object | `{"limits":{"cpu":"500m","memory":"275Mi"},"requests":{"cpu":"10m","memory":"250Mi"}}` | Default resources requests and limits for Flyteconsole deployment | +| flyte.flyteconsole.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Flyteconsole | +| flyte.flyteconsole.tolerations | list | `[]` | tolerations for Flyteconsole deployment | +| flyte.flytepropeller.affinity | object | `{}` | affinity for Flytepropeller deployment | +| flyte.flytepropeller.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flyte.flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flyte.flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment | +| flyte.flytepropeller.image.tag | string | `"v1.13.2"` | Docker image tag | +| flyte.flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment | +| flyte.flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods | +| flyte.flytepropeller.replicaCount | int | `1` | Replicas count for Flytepropeller deployment | +| flyte.flytepropeller.resources | object | `{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytepropeller deployment | +| flyte.flytepropeller.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for FlytePropeller | +| flyte.flytepropeller.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to FlytePropeller pods | +| flyte.flytepropeller.serviceAccount.create | bool | `true` | Should a service account be created for FlytePropeller | +| flyte.flytepropeller.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyte.flytepropeller.tolerations | list | `[]` | tolerations for Flytepropeller deployment | +| flyte.flytescheduler.affinity | object | `{}` | affinity for Flytescheduler deployment | +| flyte.flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flyte.flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| flyte.flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment | +| flyte.flytescheduler.image.tag | string | `"v1.13.2"` | Docker image tag | +| flyte.flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment | +| flyte.flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods | +| flyte.flytescheduler.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytescheduler deployment | +| flyte.flytescheduler.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for Flytescheduler | +| flyte.flytescheduler.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flytescheduler pods | +| flyte.flytescheduler.serviceAccount.create | bool | `true` | Should a service account be created for Flytescheduler | +| flyte.flytescheduler.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyte.flytescheduler.tolerations | list | `[]` | tolerations for Flytescheduler deployment | +| flyte.storage | object | `{"bucketName":"my-s3-bucket","cache":{"maxSizeMBs":0,"targetGCPercent":70},"custom":{},"gcs":null,"s3":{"region":"us-east-1"},"type":"sandbox"}` | ---------------------------------------------------- STORAGE SETTINGS | +| flyte.storage.bucketName | string | `"my-s3-bucket"` | bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. | +| flyte.storage.custom | object | `{}` | Settings for storage type custom. See https://github.com/graymeta/stow for supported storage providers/settings. | +| flyte.storage.gcs | string | `nil` | settings for storage type gcs | +| flyte.storage.s3 | object | `{"region":"us-east-1"}` | settings for storage type s3 | +| flyte.storage.type | string | `"sandbox"` | Sets the storage type. Supported values are sandbox, s3, gcs and custom. | +| flyte.webhook.enabled | bool | `true` | enable or disable secrets webhook | +| flyte.webhook.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for the webhook | +| flyte.webhook.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for the webhook | +| flyte.webhook.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to the webhook | +| flyte.webhook.serviceAccount.create | bool | `true` | Should a service account be created for the webhook | +| flyte.webhook.serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account | +| flyte.workflow_notifications | object | `{"config":{},"enabled":false}` | **Optional Component** Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of various events in their workflows | +| flyte.workflow_scheduler | object | `{"enabled":true,"type":"native"}` | **Optional Component** Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, this module, you will not have scheduled launchplans / workflows. Docs: https://docs.flyte.org/en/latest/howto/enable_and_use_schedules.html#setting-up-scheduled-workflows | +| flyteagent.enabled | bool | `true` | | +| kubernetes-dashboard.enabled | bool | `true` | | +| kubernetes-dashboard.extraArgs[0] | string | `"--enable-skip-login"` | | +| kubernetes-dashboard.extraArgs[1] | string | `"--enable-insecure-login"` | | +| kubernetes-dashboard.extraArgs[2] | string | `"--disable-settings-authorizer"` | | +| kubernetes-dashboard.protocolHttp | bool | `true` | | +| kubernetes-dashboard.rbac.clusterReadOnlyRole | bool | `true` | | +| kubernetes-dashboard.service.externalPort | int | `30082` | | +| kubernetes-dashboard.service.nodePort | int | `30082` | | +| kubernetes-dashboard.service.type | string | `"NodePort"` | | +| minio.affinity | object | `{}` | affinity for Minio deployment | +| minio.enabled | bool | `true` | - enable or disable Minio deployment installation | +| minio.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| minio.image.repository | string | `"ecr.flyte.org/bitnami/minio"` | Docker image for Minio deployment | +| minio.image.tag | string | `"2021.10.13-debian-10-r0"` | Docker image tag | +| minio.nodeSelector | object | `{}` | nodeSelector for Minio deployment | +| minio.podAnnotations | object | `{}` | Annotations for Minio pods | +| minio.replicaCount | int | `1` | Replicas count for Minio deployment | +| minio.resources | object | `{"limits":{"cpu":"200m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Minio deployment | +| minio.resources.limits | object | `{"cpu":"200m","memory":"512Mi"}` | Limits are the maximum set of resources needed for this pod | +| minio.resources.requests | object | `{"cpu":"10m","memory":"128Mi"}` | Requests are the minimum set of resources needed for this pod | +| minio.service | object | `{"annotations":{},"type":"NodePort"}` | Service settings for Minio | +| minio.tolerations | list | `[]` | tolerations for Minio deployment | +| postgres.affinity | object | `{}` | affinity for Postgres deployment | +| postgres.enabled | bool | `true` | - enable or disable Postgres deployment installation | +| postgres.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| postgres.image.repository | string | `"ecr.flyte.org/ubuntu/postgres"` | Docker image for Postgres deployment | +| postgres.image.tag | string | `"13-21.04_beta"` | Docker image tag | +| postgres.nodeSelector | object | `{}` | nodeSelector for Postgres deployment | +| postgres.podAnnotations | object | `{}` | Annotations for Postgres pods | +| postgres.replicaCount | int | `1` | Replicas count for Postgres deployment | +| postgres.resources | object | `{"limits":{"cpu":"1000m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Postgres deployment | +| postgres.service | object | `{"annotations":{},"type":"NodePort"}` | Service settings for Postgres | +| postgres.tolerations | list | `[]` | tolerations for Postgres deployment | +| redis | object | `{"enabled":false}` | --------------------------------------------- REDIS SETTINGS | +| redis.enabled | bool | `false` | - enable or disable Redis Statefulset installation | +| redoc.affinity | object | `{}` | affinity for redoc deployment | +| redoc.enabled | bool | `true` | - enable or disable redoc deployment installation | +| redoc.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy | +| redoc.image.repository | string | `"docker.io/redocly/redoc"` | Docker image for redoc deployment | +| redoc.image.tag | string | `"latest"` | Docker image tag | +| redoc.nodeSelector | object | `{}` | nodeSelector for redoc deployment | +| redoc.podAnnotations | object | `{}` | Annotations for redoc pods | +| redoc.replicaCount | int | `1` | Replicas count for redoc deployment | +| redoc.resources | object | `{"limits":{"cpu":"200m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for redoc deployment | +| redoc.resources.limits | object | `{"cpu":"200m","memory":"512Mi"}` | Limits are the maximum set of resources needed for this pod | +| redoc.resources.requests | object | `{"cpu":"10m","memory":"128Mi"}` | Requests are the minimum set of resources needed for this pod | +| redoc.service | object | `{"type":"ClusterIP"}` | Service settings for redoc | +| redoc.tolerations | list | `[]` | tolerations for redoc deployment | +| sparkoperator | object | `{"enabled":false}` | Optional: Spark Plugin using the Spark Operator | +| sparkoperator.enabled | bool | `false` | - enable or disable Sparkoperator deployment installation | diff --git a/charts/flyte/values.yaml b/charts/flyte/values.yaml index 8231f5bda1..5c551e6988 100755 --- a/charts/flyte/values.yaml +++ b/charts/flyte/values.yaml @@ -13,9 +13,9 @@ flyte: replicaCount: 1 image: # -- Docker image for Flyteadmin deployment - repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE + repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE # -- Docker image tag - tag: v1.13.2 # FLYTEADMIN_TAG + tag: v1.13.2 # FLYTEADMIN_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Additional flyteadmin container environment variables @@ -81,9 +81,9 @@ flyte: flytescheduler: image: # -- Docker image for Flytescheduler deployment - repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE + repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE # -- Docker image tag - tag: v1.13.2 # FLYTESCHEDULER_TAG + tag: v1.13.2 # FLYTESCHEDULER_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Flytescheduler deployment @@ -126,9 +126,9 @@ flyte: replicaCount: 1 image: # -- Docker image for Datacatalog deployment - repository: cr.flyte.org/flyteorg/datacatalog # DATACATALOG_IMAGE + repository: cr.flyte.org/flyteorg/datacatalog # DATACATALOG_IMAGE # -- Docker image tag - tag: v1.13.2 # DATACATALOG_TAG + tag: v1.13.2 # DATACATALOG_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Datacatalog deployment @@ -175,9 +175,9 @@ flyte: manager: false image: # -- Docker image for Flytepropeller deployment - repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE + repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE # -- Docker image tag - tag: v1.13.2 # FLYTEPROPELLER_TAG + tag: v1.13.2 # FLYTEPROPELLER_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Flytepropeller deployment @@ -219,9 +219,9 @@ flyte: replicaCount: 1 image: # -- Docker image for Flyteconsole deployment - repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE + repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE # -- Docker image tag - tag: v1.17.1 # FLYTECONSOLE_TAG + tag: v1.17.1 # FLYTECONSOLE_TAG # -- Docker image pull policy pullPolicy: IfNotPresent # -- Default resources requests and limits for Flyteconsole deployment @@ -473,7 +473,7 @@ flyte: # -- Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) co-pilot: name: flyte-copilot- - image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 # FLYTECOPILOT_IMAGE + image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 # FLYTECOPILOT_IMAGE start-timeout: 30s # -- Core propeller configuration @@ -527,17 +527,16 @@ flyte: # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array sensor: agent-service - # -- Kubernetes specific Flyte configuration k8s: plugins: diff --git a/deployment/eks/flyte_aws_scheduler_helm_generated.yaml b/deployment/eks/flyte_aws_scheduler_helm_generated.yaml index d7cb3500d6..a900c688f2 100644 --- a/deployment/eks/flyte_aws_scheduler_helm_generated.yaml +++ b/deployment/eks/flyte_aws_scheduler_helm_generated.yaml @@ -5,12 +5,12 @@ kind: ServiceAccount metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/datacatalog/rbac.yaml @@ -19,12 +19,12 @@ kind: ServiceAccount metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/propeller/rbac.yaml @@ -33,12 +33,12 @@ kind: ServiceAccount metadata: name: flytepropeller namespace: flyte - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -74,7 +74,7 @@ kind: Secret metadata: name: db-pass stringData: - pass.txt: '' + pass.txt: "" type: Opaque --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -92,7 +92,7 @@ kind: ConfigMap metadata: name: flyte-admin-clusters-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -109,20 +109,20 @@ kind: ConfigMap metadata: name: flyte-admin-base-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -130,7 +130,7 @@ data: name: staging - id: production name: production - server.yaml: | + server.yaml: | auth: appAuth: thirdPartyConfig: @@ -173,13 +173,13 @@ data: - '*' secure: false useAuth: false - remoteData.yaml: | + remoteData.yaml: | remoteData: region: us-east-1 scheme: local signedUrls: durationMinutes: 3 - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -192,7 +192,7 @@ data: cache: max_size_mbs: 1024 target_gc_percent: 70 - task_resource_defaults.yaml: | + task_resource_defaults.yaml: | task_resources: defaults: cpu: 1000m @@ -203,7 +203,7 @@ data: gpu: 1 memory: 1Gi storage: 2000Mi - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -230,7 +230,7 @@ data: refreshInterval: 5m standaloneDeployment: false templatePath: /etc/flyte/clusterresource/templates - scheduler.yaml: | + scheduler.yaml: | scheduler: eventScheduler: region: '' @@ -252,13 +252,13 @@ kind: ConfigMap metadata: name: clusterresource-template namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - aa_namespace.yaml: | + aa_namespace.yaml: | apiVersion: v1 kind: Namespace metadata: @@ -266,8 +266,8 @@ data: spec: finalizers: - kubernetes - - aab_default_service_account.yaml: | + + aab_default_service_account.yaml: | apiVersion: v1 kind: ServiceAccount metadata: @@ -275,8 +275,8 @@ data: namespace: {{ namespace }} annotations: eks.amazonaws.com/role-arn: {{ defaultIamRole }} - - ab_project_resource_quota.yaml: | + + ab_project_resource_quota.yaml: | apiVersion: v1 kind: ResourceQuota metadata: @@ -293,13 +293,13 @@ kind: ConfigMap metadata: name: flyte-clusterresourcesync-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -326,14 +326,14 @@ data: refreshInterval: 5m standaloneDeployment: false templatePath: /etc/flyte/clusterresource/templates - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -352,12 +352,12 @@ kind: ConfigMap metadata: name: flyte-console-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm -data: +data: BASE_URL: /console CONFIG_DIR: /etc/flyte/config --- @@ -367,20 +367,20 @@ kind: ConfigMap metadata: name: datacatalog-config namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - server.yaml: | + server.yaml: | application: grpcPort: 8089 grpcServerReflection: true @@ -391,7 +391,7 @@ data: metrics-scope: datacatalog profiler-port: 10254 storage-prefix: metadata/datacatalog - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -411,13 +411,13 @@ kind: ConfigMap metadata: name: flyte-propeller-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -427,19 +427,19 @@ data: capacity: 1000 rate: 500 type: admin - catalog.yaml: | + catalog.yaml: | catalog-cache: endpoint: datacatalog:89 insecure: true type: datacatalog - copilot.yaml: | + copilot.yaml: | plugins: k8s: co-pilot: image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 name: flyte-copilot- start-timeout: 30s - core.yaml: | + core.yaml: | manager: pod-application: flytepropeller pod-template-container-name: flytepropeller @@ -485,30 +485,30 @@ data: webhook: certDir: /etc/webhook/certs serviceName: flyte-pod-webhook - enabled_plugins.yaml: | + enabled_plugins.yaml: | tasks: task-plugins: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo - k8s.yaml: | + k8s.yaml: | plugins: k8s: default-cpus: 100m default-env-vars: [] default-memory: 100Mi - resource_manager.yaml: | + resource_manager.yaml: | propeller: resourcemanager: type: noop - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -521,7 +521,7 @@ data: cache: max_size_mbs: 1024 target_gc_percent: 70 - task_logs.yaml: | + task_logs.yaml: | plugins: logs: cloudwatch-enabled: true @@ -558,103 +558,103 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flyteadmin - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - - limitranges - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + - limitranges + verbs: + - "*" --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -# Allow RO access to PODS -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -# Allow Event recording access -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -# Allow Access All plugin objects -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -# Allow Access to CRD -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -# Allow Access to all resources under flyte.lyft.com -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + # Allow RO access to PODS + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + # Allow Event recording access + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + # Allow Access All plugin objects + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + # Allow Access to CRD + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + # Allow Access to all resources under flyte.lyft.com + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a ClusterRole for the webhook @@ -683,7 +683,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flyteadmin-binding - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -693,16 +693,16 @@ roleRef: kind: ClusterRole name: flyte-flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -712,9 +712,9 @@ roleRef: kind: ClusterRole name: flyte-flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a binding from Role -> ServiceAccount @@ -738,12 +738,12 @@ kind: Service metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: type: ClusterIP @@ -768,7 +768,7 @@ spec: protocol: TCP appProtocol: TCP port: 10254 - selector: + selector: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte --- @@ -778,7 +778,7 @@ kind: Service metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -786,12 +786,12 @@ metadata: spec: type: ClusterIP ports: - - name: http - port: 80 - protocol: TCP - appProtocol: TCP - targetPort: 8080 - selector: + - name: http + port: 80 + protocol: TCP + appProtocol: TCP + targetPort: 8080 + selector: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte --- @@ -801,25 +801,25 @@ kind: Service metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: type: NodePort ports: - - name: http - port: 88 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 89 - protocol: TCP - targetPort: 8089 - selector: + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte --- @@ -830,7 +830,7 @@ kind: Service metadata: name: flyte-pod-webhook namespace: flyte - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: selector: @@ -847,7 +847,7 @@ kind: Deployment metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -855,20 +855,20 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "c943b200cd0bed97fe456c0c713dd79cdc4e22133495cac89db3fc55e9b79c7" - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -877,11 +877,11 @@ spec: type: spc_t initContainers: - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - run + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: run-migrations @@ -890,19 +890,19 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - seed-projects - - flytesnacks - - flytetester - - flyteexamples + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: seed-projects @@ -911,16 +911,16 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - clusterresource - - sync + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: sync-cluster-resources @@ -929,21 +929,21 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets - name: generate-secrets image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" command: ["/bin/sh", "-c"] args: [ - "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", ] securityContext: allowPrivilegeEscalation: false @@ -960,81 +960,95 @@ spec: fieldRef: fieldPath: metadata.namespace containers: - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flyteadmin - ports: - - containerPort: 8088 - - containerPort: 8089 - - containerPort: 10254 - readinessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 15 - livenessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 20 - periodSeconds: 5 - resources: - limits: - cpu: 250m - ephemeral-storage: 200Mi - memory: 500Mi - requests: - cpu: 50m - ephemeral-storage: 200Mi - memory: 200Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /srv/flyte - name: shared-data - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + - containerPort: 10254 + readinessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 15 + livenessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 20 + periodSeconds: 5 + resources: + limits: + cpu: 250m + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets serviceAccountName: flyteadmin volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - emptyDir: {} - name: scratch - - configMap: - name: flyte-admin-base-config - name: base-config-volume - - projected: - sources: - - configMap: - name: flyte-admin-base-config - - configMap: - name: flyte-admin-clusters-config - name: clusters-config-volume - - configMap: - name: clusterresource-template - name: resource-templates - - name: admin-secrets - secret: - secretName: flyte-admin-secrets - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - emptyDir: {} + name: scratch + - configMap: + name: flyte-admin-base-config + name: base-config-volume + - projected: + sources: + - configMap: + name: flyte-admin-base-config + - configMap: + name: flyte-admin-clusters-config + name: clusters-config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: admin-secrets + secret: + secretName: flyte-admin-secrets + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteadmin - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteadmin + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/clusterresourcesync/deployment.yaml apiVersion: apps/v1 @@ -1042,7 +1056,7 @@ kind: Deployment metadata: name: syncresources namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1050,7 +1064,7 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte template: @@ -1059,7 +1073,7 @@ spec: configChecksum: "55ce597c10b17ef6e891f0c9242b17aafb3d7b4e4e414d0a5078d71ad9c804f" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1076,12 +1090,12 @@ spec: imagePullPolicy: "IfNotPresent" name: sync-cluster-resources volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume serviceAccountName: flyteadmin volumes: - name: db-pass @@ -1100,7 +1114,7 @@ kind: Deployment metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1108,58 +1122,58 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "2f930e1732c47d0849f79f9a8d06262ec97597a217bbf2337ae4f2938402ee0" - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true runAsUser: 1000 seLinuxOptions: type: spc_t containers: - - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" - imagePullPolicy: "IfNotPresent" - name: flyteconsole - envFrom: - - configMapRef: - name: flyte-console-config - ports: - - containerPort: 8080 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 250m - memory: 250Mi - requests: - cpu: 10m - memory: 50Mi - volumeMounts: - - mountPath: /srv/flyte - name: shared-data + - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 250m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data volumes: - - emptyDir: {} - name: shared-data - affinity: + - emptyDir: {} + name: shared-data + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteconsole - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteconsole + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/datacatalog/deployment.yaml apiVersion: apps/v1 @@ -1167,7 +1181,7 @@ kind: Deployment metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1175,20 +1189,20 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "ded28f3a68d22eb8e5af14a44cc0d14326f10060405268aac5a3665fb86c8bc" - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 1001 fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true @@ -1196,72 +1210,72 @@ spec: seLinuxOptions: type: spc_t initContainers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - migrate - - run - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: run-migrations - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] containers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: datacatalog - ports: - - containerPort: 8080 - - containerPort: 8089 - - containerPort: 10254 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 1 - ephemeral-storage: 200Mi - memory: 500Mi - requests: - cpu: 500m - ephemeral-storage: 200Mi - memory: 200Mi - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8080 + - containerPort: 8089 + - containerPort: 10254 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 1 + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 500m + ephemeral-storage: 200Mi + memory: 200Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume serviceAccountName: datacatalog volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - configMap: - name: datacatalog-config - name: config-volume - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: datacatalog - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: datacatalog + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/deployment.yaml apiVersion: apps/v1 @@ -1269,7 +1283,7 @@ kind: Deployment metadata: namespace: flyte name: flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1277,7 +1291,7 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte template: @@ -1286,66 +1300,66 @@ spec: configChecksum: "6572aa999f8e6842b4dba120e12e6ccb8cdfa506373de2a267b62a63146ccde" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 priorityClassName: system-cluster-critical containers: - - command: - - flytepropeller - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytepropeller - ports: - - containerPort: 10254 - resources: - limits: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - requests: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: auth - mountPath: /etc/secrets/ - terminationMessagePolicy: "FallbackToLogsOnError" + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + requests: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + terminationMessagePolicy: "FallbackToLogsOnError" serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth - affinity: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flytepropeller - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/webhook.yaml # Create the actual deployment @@ -1371,7 +1385,7 @@ spec: prometheus.io/path: "/metrics" prometheus.io/port: "10254" spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -1380,32 +1394,32 @@ spec: type: spc_t serviceAccountName: flyte-pod-webhook initContainers: - - name: generate-secrets - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - command: - - flytepropeller - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config + - name: generate-secrets + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config containers: - name: webhook image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" @@ -1426,7 +1440,7 @@ spec: fieldRef: fieldPath: metadata.namespace ports: - - containerPort: 9443 + - containerPort: 9443 securityContext: allowPrivilegeEscalation: false capabilities: @@ -1457,10 +1471,11 @@ kind: Ingress metadata: name: flyte-core namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -1470,7 +1485,7 @@ metadata: nginx.ingress.kubernetes.io/app-root: /console nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - http: paths: @@ -1626,11 +1641,12 @@ kind: Ingress metadata: name: flyte-core-grpc namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/backend-protocol-version: GRPC - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -1641,7 +1657,7 @@ metadata: nginx.ingress.kubernetes.io/backend-protocol: GRPC nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - host: null http: @@ -1653,7 +1669,7 @@ spec: # path: /* # pathType: ImplementationSpecific # - + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. - path: /flyteidl.service.SignalService pathType: ImplementationSpecific diff --git a/deployment/eks/flyte_helm_dataplane_generated.yaml b/deployment/eks/flyte_helm_dataplane_generated.yaml index 682d1cef01..ce7215cb1e 100644 --- a/deployment/eks/flyte_helm_dataplane_generated.yaml +++ b/deployment/eks/flyte_helm_dataplane_generated.yaml @@ -5,12 +5,12 @@ kind: ServiceAccount metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/propeller/rbac.yaml @@ -19,12 +19,12 @@ kind: ServiceAccount metadata: name: flytepropeller namespace: flyte - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -51,7 +51,7 @@ kind: Secret metadata: name: db-pass stringData: - pass.txt: '' + pass.txt: "" type: Opaque --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -69,13 +69,13 @@ kind: ConfigMap metadata: name: flyte-propeller-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -85,19 +85,19 @@ data: capacity: 1000 rate: 500 type: admin - catalog.yaml: | + catalog.yaml: | catalog-cache: endpoint: datacatalog:89 insecure: true type: datacatalog - copilot.yaml: | + copilot.yaml: | plugins: k8s: co-pilot: image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 name: flyte-copilot- start-timeout: 30s - core.yaml: | + core.yaml: | manager: pod-application: flytepropeller pod-template-container-name: flytepropeller @@ -143,30 +143,30 @@ data: webhook: certDir: /etc/webhook/certs serviceName: flyte-pod-webhook - enabled_plugins.yaml: | + enabled_plugins.yaml: | tasks: task-plugins: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo - k8s.yaml: | + k8s.yaml: | plugins: k8s: default-cpus: 100m default-env-vars: [] default-memory: 100Mi - resource_manager.yaml: | + resource_manager.yaml: | propeller: resourcemanager: type: noop - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -179,7 +179,7 @@ data: cache: max_size_mbs: 1024 target_gc_percent: 70 - task_logs.yaml: | + task_logs.yaml: | plugins: logs: cloudwatch-enabled: true @@ -216,103 +216,103 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flyteadmin - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - - limitranges - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + - limitranges + verbs: + - "*" --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -# Allow RO access to PODS -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -# Allow Event recording access -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -# Allow Access All plugin objects -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -# Allow Access to CRD -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -# Allow Access to all resources under flyte.lyft.com -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + # Allow RO access to PODS + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + # Allow Event recording access + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + # Allow Access All plugin objects + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + # Allow Access to CRD + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + # Allow Access to all resources under flyte.lyft.com + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a ClusterRole for the webhook @@ -341,7 +341,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flyteadmin-binding - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -351,16 +351,16 @@ roleRef: kind: ClusterRole name: flyte-flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -370,9 +370,9 @@ roleRef: kind: ClusterRole name: flyte-flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a binding from Role -> ServiceAccount @@ -397,7 +397,7 @@ kind: Service metadata: name: flyte-pod-webhook namespace: flyte - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: selector: @@ -414,7 +414,7 @@ kind: Deployment metadata: namespace: flyte name: flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -422,7 +422,7 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte template: @@ -431,66 +431,66 @@ spec: configChecksum: "6572aa999f8e6842b4dba120e12e6ccb8cdfa506373de2a267b62a63146ccde" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 priorityClassName: system-cluster-critical containers: - - command: - - flytepropeller - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytepropeller - ports: - - containerPort: 10254 - resources: - limits: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - requests: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: auth - mountPath: /etc/secrets/ - terminationMessagePolicy: "FallbackToLogsOnError" + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + requests: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + terminationMessagePolicy: "FallbackToLogsOnError" serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth - affinity: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flytepropeller - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/webhook.yaml # Create the actual deployment @@ -516,7 +516,7 @@ spec: prometheus.io/path: "/metrics" prometheus.io/port: "10254" spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -525,32 +525,32 @@ spec: type: spc_t serviceAccountName: flyte-pod-webhook initContainers: - - name: generate-secrets - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - command: - - flytepropeller - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config + - name: generate-secrets + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config containers: - name: webhook image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" @@ -571,7 +571,7 @@ spec: fieldRef: fieldPath: metadata.namespace ports: - - containerPort: 9443 + - containerPort: 9443 securityContext: allowPrivilegeEscalation: false capabilities: @@ -602,10 +602,11 @@ kind: Ingress metadata: name: flyte-core namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -615,7 +616,7 @@ metadata: nginx.ingress.kubernetes.io/app-root: /console nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - http: paths: @@ -771,11 +772,12 @@ kind: Ingress metadata: name: flyte-core-grpc namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/backend-protocol-version: GRPC - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -786,7 +788,7 @@ metadata: nginx.ingress.kubernetes.io/backend-protocol: GRPC nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - host: null http: @@ -798,7 +800,7 @@ spec: # path: /* # pathType: ImplementationSpecific # - + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. - path: /flyteidl.service.SignalService pathType: ImplementationSpecific diff --git a/deployment/eks/flyte_helm_generated.yaml b/deployment/eks/flyte_helm_generated.yaml index 5e0ae72ec2..0b8be75daf 100644 --- a/deployment/eks/flyte_helm_generated.yaml +++ b/deployment/eks/flyte_helm_generated.yaml @@ -5,12 +5,12 @@ kind: ServiceAccount metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/datacatalog/rbac.yaml @@ -19,12 +19,12 @@ kind: ServiceAccount metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/flytescheduler/sa.yaml @@ -33,7 +33,7 @@ kind: ServiceAccount metadata: name: flytescheduler namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -45,12 +45,12 @@ kind: ServiceAccount metadata: name: flytepropeller namespace: flyte - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -86,7 +86,7 @@ kind: Secret metadata: name: db-pass stringData: - pass.txt: '' + pass.txt: "" type: Opaque --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -104,7 +104,7 @@ kind: ConfigMap metadata: name: flyte-admin-clusters-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -121,20 +121,20 @@ kind: ConfigMap metadata: name: flyte-admin-base-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -142,7 +142,7 @@ data: name: staging - id: production name: production - server.yaml: | + server.yaml: | auth: appAuth: thirdPartyConfig: @@ -185,13 +185,13 @@ data: - '*' secure: false useAuth: false - remoteData.yaml: | + remoteData.yaml: | remoteData: region: us-east-1 scheme: local signedUrls: durationMinutes: 3 - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -204,7 +204,7 @@ data: cache: max_size_mbs: 1024 target_gc_percent: 70 - task_resource_defaults.yaml: | + task_resource_defaults.yaml: | task_resources: defaults: cpu: 1000m @@ -215,7 +215,7 @@ data: gpu: 1 memory: 1Gi storage: 2000Mi - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -249,13 +249,13 @@ kind: ConfigMap metadata: name: clusterresource-template namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - aa_namespace.yaml: | + aa_namespace.yaml: | apiVersion: v1 kind: Namespace metadata: @@ -263,8 +263,8 @@ data: spec: finalizers: - kubernetes - - aab_default_service_account.yaml: | + + aab_default_service_account.yaml: | apiVersion: v1 kind: ServiceAccount metadata: @@ -272,8 +272,8 @@ data: namespace: {{ namespace }} annotations: eks.amazonaws.com/role-arn: {{ defaultIamRole }} - - ab_project_resource_quota.yaml: | + + ab_project_resource_quota.yaml: | apiVersion: v1 kind: ResourceQuota metadata: @@ -290,13 +290,13 @@ kind: ConfigMap metadata: name: flyte-clusterresourcesync-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -323,14 +323,14 @@ data: refreshInterval: 5m standaloneDeployment: false templatePath: /etc/flyte/clusterresource/templates - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -349,12 +349,12 @@ kind: ConfigMap metadata: name: flyte-console-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm -data: +data: BASE_URL: /console CONFIG_DIR: /etc/flyte/config --- @@ -364,20 +364,20 @@ kind: ConfigMap metadata: name: datacatalog-config namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - server.yaml: | + server.yaml: | application: grpcPort: 8089 grpcServerReflection: true @@ -388,7 +388,7 @@ data: metrics-scope: datacatalog profiler-port: 10254 storage-prefix: metadata/datacatalog - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -408,13 +408,13 @@ kind: ConfigMap metadata: name: flyte-scheduler-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -424,14 +424,14 @@ data: capacity: 1000 rate: 500 type: admin - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - server.yaml: | + server.yaml: | scheduler: metricsScope: 'flyte:' profilerPort: 10254 @@ -442,13 +442,13 @@ kind: ConfigMap metadata: name: flyte-propeller-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -458,19 +458,19 @@ data: capacity: 1000 rate: 500 type: admin - catalog.yaml: | + catalog.yaml: | catalog-cache: endpoint: datacatalog:89 insecure: true type: datacatalog - copilot.yaml: | + copilot.yaml: | plugins: k8s: co-pilot: image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 name: flyte-copilot- start-timeout: 30s - core.yaml: | + core.yaml: | manager: pod-application: flytepropeller pod-template-container-name: flytepropeller @@ -516,30 +516,30 @@ data: webhook: certDir: /etc/webhook/certs serviceName: flyte-pod-webhook - enabled_plugins.yaml: | + enabled_plugins.yaml: | tasks: task-plugins: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo - k8s.yaml: | + k8s.yaml: | plugins: k8s: default-cpus: 100m default-env-vars: [] default-memory: 100Mi - resource_manager.yaml: | + resource_manager.yaml: | propeller: resourcemanager: type: noop - storage.yaml: | + storage.yaml: | storage: type: s3 container: "" @@ -552,7 +552,7 @@ data: cache: max_size_mbs: 1024 target_gc_percent: 70 - task_logs.yaml: | + task_logs.yaml: | plugins: logs: cloudwatch-enabled: true @@ -589,103 +589,103 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flyteadmin - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - - limitranges - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + - limitranges + verbs: + - "*" --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -# Allow RO access to PODS -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -# Allow Event recording access -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -# Allow Access All plugin objects -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -# Allow Access to CRD -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -# Allow Access to all resources under flyte.lyft.com -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + # Allow RO access to PODS + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + # Allow Event recording access + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + # Allow Access All plugin objects + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + # Allow Access to CRD + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + # Allow Access to all resources under flyte.lyft.com + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a ClusterRole for the webhook @@ -714,7 +714,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flyteadmin-binding - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -724,16 +724,16 @@ roleRef: kind: ClusterRole name: flyte-flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -743,9 +743,9 @@ roleRef: kind: ClusterRole name: flyte-flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a binding from Role -> ServiceAccount @@ -769,12 +769,12 @@ kind: Service metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: type: ClusterIP @@ -799,7 +799,7 @@ spec: protocol: TCP appProtocol: TCP port: 10254 - selector: + selector: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte --- @@ -809,7 +809,7 @@ kind: Service metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -817,12 +817,12 @@ metadata: spec: type: ClusterIP ports: - - name: http - port: 80 - protocol: TCP - appProtocol: TCP - targetPort: 8080 - selector: + - name: http + port: 80 + protocol: TCP + appProtocol: TCP + targetPort: 8080 + selector: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte --- @@ -832,25 +832,25 @@ kind: Service metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: type: NodePort ports: - - name: http - port: 88 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 89 - protocol: TCP - targetPort: 8089 - selector: + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte --- @@ -861,7 +861,7 @@ kind: Service metadata: name: flyte-pod-webhook namespace: flyte - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: selector: @@ -878,7 +878,7 @@ kind: Deployment metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -886,20 +886,20 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "391e8e126d669f751ac1a03de0b45fe7969a0fe58f3dfead9bb7be1b5d951ff" - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -908,11 +908,11 @@ spec: type: spc_t initContainers: - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - run + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: run-migrations @@ -921,19 +921,19 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - seed-projects - - flytesnacks - - flytetester - - flyteexamples + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: seed-projects @@ -942,16 +942,16 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - clusterresource - - sync + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: sync-cluster-resources @@ -960,21 +960,21 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets - name: generate-secrets image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" command: ["/bin/sh", "-c"] args: [ - "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", ] securityContext: allowPrivilegeEscalation: false @@ -991,81 +991,95 @@ spec: fieldRef: fieldPath: metadata.namespace containers: - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flyteadmin - ports: - - containerPort: 8088 - - containerPort: 8089 - - containerPort: 10254 - readinessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 15 - livenessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 20 - periodSeconds: 5 - resources: - limits: - cpu: 250m - ephemeral-storage: 200Mi - memory: 500Mi - requests: - cpu: 50m - ephemeral-storage: 200Mi - memory: 200Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /srv/flyte - name: shared-data - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + - containerPort: 10254 + readinessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 15 + livenessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 20 + periodSeconds: 5 + resources: + limits: + cpu: 250m + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets serviceAccountName: flyteadmin volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - emptyDir: {} - name: scratch - - configMap: - name: flyte-admin-base-config - name: base-config-volume - - projected: - sources: - - configMap: - name: flyte-admin-base-config - - configMap: - name: flyte-admin-clusters-config - name: clusters-config-volume - - configMap: - name: clusterresource-template - name: resource-templates - - name: admin-secrets - secret: - secretName: flyte-admin-secrets - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - emptyDir: {} + name: scratch + - configMap: + name: flyte-admin-base-config + name: base-config-volume + - projected: + sources: + - configMap: + name: flyte-admin-base-config + - configMap: + name: flyte-admin-clusters-config + name: clusters-config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: admin-secrets + secret: + secretName: flyte-admin-secrets + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteadmin - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteadmin + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/clusterresourcesync/deployment.yaml apiVersion: apps/v1 @@ -1073,7 +1087,7 @@ kind: Deployment metadata: name: syncresources namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1081,7 +1095,7 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte template: @@ -1090,7 +1104,7 @@ spec: configChecksum: "55ce597c10b17ef6e891f0c9242b17aafb3d7b4e4e414d0a5078d71ad9c804f" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1107,12 +1121,12 @@ spec: imagePullPolicy: "IfNotPresent" name: sync-cluster-resources volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume serviceAccountName: flyteadmin volumes: - name: db-pass @@ -1131,7 +1145,7 @@ kind: Deployment metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1139,58 +1153,58 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "2f930e1732c47d0849f79f9a8d06262ec97597a217bbf2337ae4f2938402ee0" - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true runAsUser: 1000 seLinuxOptions: type: spc_t containers: - - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" - imagePullPolicy: "IfNotPresent" - name: flyteconsole - envFrom: - - configMapRef: - name: flyte-console-config - ports: - - containerPort: 8080 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 250m - memory: 250Mi - requests: - cpu: 10m - memory: 50Mi - volumeMounts: - - mountPath: /srv/flyte - name: shared-data + - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 250m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data volumes: - - emptyDir: {} - name: shared-data - affinity: + - emptyDir: {} + name: shared-data + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteconsole - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteconsole + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/datacatalog/deployment.yaml apiVersion: apps/v1 @@ -1198,7 +1212,7 @@ kind: Deployment metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1206,20 +1220,20 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "ded28f3a68d22eb8e5af14a44cc0d14326f10060405268aac5a3665fb86c8bc" - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 1001 fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true @@ -1227,72 +1241,72 @@ spec: seLinuxOptions: type: spc_t initContainers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - migrate - - run - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: run-migrations - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] containers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: datacatalog - ports: - - containerPort: 8080 - - containerPort: 8089 - - containerPort: 10254 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 1 - ephemeral-storage: 200Mi - memory: 500Mi - requests: - cpu: 500m - ephemeral-storage: 200Mi - memory: 200Mi - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8080 + - containerPort: 8089 + - containerPort: 10254 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 1 + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 500m + ephemeral-storage: 200Mi + memory: 200Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume serviceAccountName: datacatalog volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - configMap: - name: datacatalog-config - name: config-volume - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: datacatalog - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: datacatalog + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/flytescheduler/deployment.yaml apiVersion: apps/v1 @@ -1300,7 +1314,7 @@ kind: Deployment metadata: name: flytescheduler namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1308,20 +1322,20 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "391e8e126d669f751ac1a03de0b45fe7969a0fe58f3dfead9bb7be1b5d951ff" - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -1329,69 +1343,69 @@ spec: seLinuxOptions: type: spc_t initContainers: - - command: - - flytescheduler - - precheck - - --config - - /etc/flyte/config/*.yaml - image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytescheduler-check - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: config-volume - - name: auth - mountPath: /etc/secrets/ + - command: + - flytescheduler + - precheck + - --config + - /etc/flyte/config/*.yaml + image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytescheduler-check + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ containers: - - command: - - flytescheduler - - run - - --config - - /etc/flyte/config/*.yaml - image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytescheduler - ports: - - containerPort: 10254 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 250m - ephemeral-storage: 100Mi - memory: 500Mi - requests: - cpu: 10m - ephemeral-storage: 50Mi - memory: 50Mi - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: config-volume - - name: auth - mountPath: /etc/secrets/ + - command: + - flytescheduler + - run + - --config + - /etc/flyte/config/*.yaml + image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytescheduler + ports: + - containerPort: 10254 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ serviceAccountName: flytescheduler volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - configMap: - name: flyte-scheduler-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-scheduler-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth --- # Source: flyte-core/templates/propeller/deployment.yaml apiVersion: apps/v1 @@ -1399,7 +1413,7 @@ kind: Deployment metadata: namespace: flyte name: flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1407,7 +1421,7 @@ metadata: spec: replicas: 2 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte template: @@ -1416,66 +1430,66 @@ spec: configChecksum: "6572aa999f8e6842b4dba120e12e6ccb8cdfa506373de2a267b62a63146ccde" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 priorityClassName: system-cluster-critical containers: - - command: - - flytepropeller - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytepropeller - ports: - - containerPort: 10254 - resources: - limits: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - requests: - cpu: 1 - ephemeral-storage: 1Gi - memory: 2Gi - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: auth - mountPath: /etc/secrets/ - terminationMessagePolicy: "FallbackToLogsOnError" + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + requests: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + terminationMessagePolicy: "FallbackToLogsOnError" serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth - affinity: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flytepropeller - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/webhook.yaml # Create the actual deployment @@ -1501,7 +1515,7 @@ spec: prometheus.io/path: "/metrics" prometheus.io/port: "10254" spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -1510,32 +1524,32 @@ spec: type: spc_t serviceAccountName: flyte-pod-webhook initContainers: - - name: generate-secrets - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - command: - - flytepropeller - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config + - name: generate-secrets + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config containers: - name: webhook image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" @@ -1556,7 +1570,7 @@ spec: fieldRef: fieldPath: metadata.namespace ports: - - containerPort: 9443 + - containerPort: 9443 securityContext: allowPrivilegeEscalation: false capabilities: @@ -1587,10 +1601,11 @@ kind: Ingress metadata: name: flyte-core namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -1600,7 +1615,7 @@ metadata: nginx.ingress.kubernetes.io/app-root: /console nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - http: paths: @@ -1756,11 +1771,12 @@ kind: Ingress metadata: name: flyte-core-grpc namespace: flyte - annotations: - alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: + '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/backend-protocol-version: GRPC - alb.ingress.kubernetes.io/certificate-arn: '' + alb.ingress.kubernetes.io/certificate-arn: "" alb.ingress.kubernetes.io/group.name: flyte alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing @@ -1771,7 +1787,7 @@ metadata: nginx.ingress.kubernetes.io/backend-protocol: GRPC nginx.ingress.kubernetes.io/service-upstream: "true" spec: - ingressClassName: + ingressClassName: rules: - host: null http: @@ -1783,7 +1799,7 @@ spec: # path: /* # pathType: ImplementationSpecific # - + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. - path: /flyteidl.service.SignalService pathType: ImplementationSpecific diff --git a/deployment/gcp/flyte_helm_dataplane_generated.yaml b/deployment/gcp/flyte_helm_dataplane_generated.yaml index 8196b38520..c1d113fcee 100644 --- a/deployment/gcp/flyte_helm_dataplane_generated.yaml +++ b/deployment/gcp/flyte_helm_dataplane_generated.yaml @@ -5,12 +5,12 @@ kind: ServiceAccount metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: iam.gke.io/gcp-service-account: gsa-flyteadmin@.iam.gserviceaccount.com --- # Source: flyte-core/templates/propeller/rbac.yaml @@ -19,12 +19,12 @@ kind: ServiceAccount metadata: name: flytepropeller namespace: flyte - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: iam.gke.io/gcp-service-account: gsa-flytepropeller@.iam.gserviceaccount.com --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -51,7 +51,7 @@ kind: Secret metadata: name: db-pass stringData: - pass.txt: '' + pass.txt: "" type: Opaque --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -69,13 +69,13 @@ kind: ConfigMap metadata: name: flyte-propeller-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -85,19 +85,19 @@ data: capacity: 1000 rate: 500 type: admin - catalog.yaml: | + catalog.yaml: | catalog-cache: endpoint: datacatalog:89 insecure: true type: datacatalog - copilot.yaml: | + copilot.yaml: | plugins: k8s: co-pilot: image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 name: flyte-copilot- start-timeout: 30s - core.yaml: | + core.yaml: | manager: pod-application: flytepropeller pod-template-container-name: flytepropeller @@ -143,30 +143,30 @@ data: webhook: certDir: /etc/webhook/certs serviceName: flyte-pod-webhook - enabled_plugins.yaml: | + enabled_plugins.yaml: | tasks: task-plugins: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo - k8s.yaml: | + k8s.yaml: | plugins: k8s: default-cpus: 100m default-env-vars: [] default-memory: 100Mi - resource_manager.yaml: | + resource_manager.yaml: | propeller: resourcemanager: type: noop - storage.yaml: | + storage.yaml: | storage: type: stow stow: @@ -182,7 +182,7 @@ data: cache: max_size_mbs: 0 target_gc_percent: 70 - task_logs.yaml: | + task_logs.yaml: | plugins: k8s-array: logs: @@ -224,103 +224,103 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flyteadmin - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - - limitranges - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + - limitranges + verbs: + - "*" --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -# Allow RO access to PODS -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -# Allow Event recording access -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -# Allow Access All plugin objects -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -# Allow Access to CRD -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -# Allow Access to all resources under flyte.lyft.com -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + # Allow RO access to PODS + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + # Allow Event recording access + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + # Allow Access All plugin objects + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + # Allow Access to CRD + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + # Allow Access to all resources under flyte.lyft.com + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a ClusterRole for the webhook @@ -349,7 +349,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flyteadmin-binding - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -359,16 +359,16 @@ roleRef: kind: ClusterRole name: flyte-flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -378,9 +378,9 @@ roleRef: kind: ClusterRole name: flyte-flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a binding from Role -> ServiceAccount @@ -405,7 +405,7 @@ kind: Service metadata: name: flyte-pod-webhook namespace: flyte - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: selector: @@ -422,7 +422,7 @@ kind: Deployment metadata: namespace: flyte name: flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -430,7 +430,7 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte template: @@ -439,65 +439,65 @@ spec: configChecksum: "8562f7f608d4936e13f6ad70c18c7c095068e742243e7f380f89694d2182110" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 containers: - - command: - - flytepropeller - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytepropeller - ports: - - containerPort: 10254 - resources: - limits: - cpu: 500m - ephemeral-storage: 2Gi - memory: 1Gi - requests: - cpu: 50m - ephemeral-storage: 2Gi - memory: 1Gi - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: auth - mountPath: /etc/secrets/ - terminationMessagePolicy: "FallbackToLogsOnError" + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 500m + ephemeral-storage: 2Gi + memory: 1Gi + requests: + cpu: 50m + ephemeral-storage: 2Gi + memory: 1Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + terminationMessagePolicy: "FallbackToLogsOnError" serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth - affinity: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flytepropeller - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/webhook.yaml # Create the actual deployment @@ -523,7 +523,7 @@ spec: prometheus.io/path: "/metrics" prometheus.io/port: "10254" spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -532,32 +532,32 @@ spec: type: spc_t serviceAccountName: flyte-pod-webhook initContainers: - - name: generate-secrets - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - command: - - flytepropeller - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config + - name: generate-secrets + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config containers: - name: webhook image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" @@ -578,7 +578,7 @@ spec: fieldRef: fieldPath: metadata.namespace ports: - - containerPort: 9443 + - containerPort: 9443 securityContext: allowPrivilegeEscalation: false capabilities: @@ -609,14 +609,14 @@ kind: Ingress metadata: name: flyte-core namespace: flyte - annotations: + annotations: cert-manager.io/issuer: letsencrypt-production kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/app-root: /console nginx.ingress.kubernetes.io/service-upstream: "true" nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: - ingressClassName: + ingressClassName: rules: - http: paths: @@ -755,12 +755,12 @@ spec: name: flyteadmin port: number: 80 - host: '' + host: "" tls: - secretName: flyte-flyte-tls hosts: - - '' - + - "" + # Certain ingress controllers like nginx cannot serve HTTP 1 and GRPC with a single ingress because GRPC can only # enabled on the ingress object, not on backend services (GRPC annotation is set on the ingress, not on the services). --- @@ -770,7 +770,7 @@ kind: Ingress metadata: name: flyte-core-grpc namespace: flyte - annotations: + annotations: cert-manager.io/issuer: letsencrypt-production kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/app-root: /console @@ -778,13 +778,13 @@ metadata: nginx.ingress.kubernetes.io/service-upstream: "true" nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: - ingressClassName: + ingressClassName: rules: - - host: '' + - host: "" http: paths: # - + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. - path: /flyteidl.service.SignalService pathType: ImplementationSpecific @@ -873,4 +873,4 @@ spec: tls: - secretName: flyte-flyte-tls hosts: - - '' + - "" diff --git a/deployment/gcp/flyte_helm_generated.yaml b/deployment/gcp/flyte_helm_generated.yaml index ce1f64c1df..452ed223c6 100644 --- a/deployment/gcp/flyte_helm_generated.yaml +++ b/deployment/gcp/flyte_helm_generated.yaml @@ -5,12 +5,12 @@ kind: ServiceAccount metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: iam.gke.io/gcp-service-account: gsa-flyteadmin@.iam.gserviceaccount.com --- # Source: flyte-core/templates/datacatalog/rbac.yaml @@ -19,12 +19,12 @@ kind: ServiceAccount metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: iam.gke.io/gcp-service-account: gsa-datacatalog@.iam.gserviceaccount.com --- # Source: flyte-core/templates/flytescheduler/sa.yaml @@ -33,7 +33,7 @@ kind: ServiceAccount metadata: name: flytescheduler namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -45,12 +45,12 @@ kind: ServiceAccount metadata: name: flytepropeller namespace: flyte - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: iam.gke.io/gcp-service-account: gsa-flytepropeller@.iam.gserviceaccount.com --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -86,7 +86,7 @@ kind: Secret metadata: name: db-pass stringData: - pass.txt: '' + pass.txt: "" type: Opaque --- # Source: flyte-core/templates/propeller/webhook.yaml @@ -104,7 +104,7 @@ kind: ConfigMap metadata: name: flyte-admin-clusters-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -121,20 +121,20 @@ kind: ConfigMap metadata: name: flyte-admin-base-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -142,7 +142,7 @@ data: name: staging - id: production name: production - server.yaml: | + server.yaml: | auth: appAuth: thirdPartyConfig: @@ -185,15 +185,15 @@ data: - '*' secure: false useAuth: false - remoteData.yaml: | + remoteData.yaml: | remoteData: scheme: gcs signedUrls: durationMinutes: 3 - namespace_config.yaml: | + namespace_config.yaml: | namespace_mapping: template: '{{ domain }}' - storage.yaml: | + storage.yaml: | storage: type: stow stow: @@ -209,7 +209,7 @@ data: cache: max_size_mbs: 0 target_gc_percent: 70 - task_resource_defaults.yaml: | + task_resource_defaults.yaml: | task_resources: defaults: cpu: 500m @@ -220,7 +220,7 @@ data: gpu: 1 memory: 1Gi storage: 2000Mi - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -254,13 +254,13 @@ kind: ConfigMap metadata: name: clusterresource-template namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - aa_namespace.yaml: | + aa_namespace.yaml: | apiVersion: v1 kind: Namespace metadata: @@ -268,8 +268,8 @@ data: spec: finalizers: - kubernetes - - aab_default_service_account.yaml: | + + aab_default_service_account.yaml: | apiVersion: v1 kind: ServiceAccount metadata: @@ -279,8 +279,8 @@ data: # Needed for gcp workload identity to function # https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity iam.gke.io/gcp-service-account: {{ gsa }} - - ab_project_resource_quota.yaml: | + + ab_project_resource_quota.yaml: | apiVersion: v1 kind: ResourceQuota metadata: @@ -297,13 +297,13 @@ kind: ConfigMap metadata: name: flyte-clusterresourcesync-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - cluster_resources.yaml: | + cluster_resources.yaml: | cluster_resources: customData: - production: @@ -330,14 +330,14 @@ data: refreshInterval: 5m standaloneDeployment: false templatePath: /etc/flyte/clusterresource/templates - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - domain.yaml: | + domain.yaml: | domains: - id: development name: development @@ -345,7 +345,7 @@ data: name: staging - id: production name: production - namespace_config.yaml: | + namespace_config.yaml: | namespace_mapping: template: '{{ domain }}' clusters.yaml: | @@ -359,12 +359,12 @@ kind: ConfigMap metadata: name: flyte-console-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm -data: +data: BASE_URL: /console CONFIG_DIR: /etc/flyte/config --- @@ -374,20 +374,20 @@ kind: ConfigMap metadata: name: datacatalog-config namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - server.yaml: | + server.yaml: | application: grpcPort: 8089 grpcServerReflection: true @@ -398,7 +398,7 @@ data: metrics-scope: datacatalog profiler-port: 10254 storage-prefix: metadata/datacatalog - storage.yaml: | + storage.yaml: | storage: type: stow stow: @@ -421,13 +421,13 @@ kind: ConfigMap metadata: name: flyte-scheduler-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -437,14 +437,14 @@ data: capacity: 1000 rate: 500 type: admin - db.yaml: | + db.yaml: | database: dbname: flyteadmin host: '' passwordPath: /etc/db/pass.txt port: 5432 username: flyteadmin - server.yaml: | + server.yaml: | scheduler: metricsScope: 'flyte:' profilerPort: 10254 @@ -455,13 +455,13 @@ kind: ConfigMap metadata: name: flyte-propeller-config namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm data: - admin.yaml: | + admin.yaml: | admin: clientId: 'flytepropeller' clientSecretLocation: /etc/secrets/client_secret @@ -471,19 +471,19 @@ data: capacity: 1000 rate: 500 type: admin - catalog.yaml: | + catalog.yaml: | catalog-cache: endpoint: datacatalog:89 insecure: true type: datacatalog - copilot.yaml: | + copilot.yaml: | plugins: k8s: co-pilot: image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 name: flyte-copilot- start-timeout: 30s - core.yaml: | + core.yaml: | manager: pod-application: flytepropeller pod-template-container-name: flytepropeller @@ -529,30 +529,30 @@ data: webhook: certDir: /etc/webhook/certs serviceName: flyte-pod-webhook - enabled_plugins.yaml: | + enabled_plugins.yaml: | tasks: task-plugins: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo - k8s.yaml: | + k8s.yaml: | plugins: k8s: default-cpus: 100m default-env-vars: [] default-memory: 100Mi - resource_manager.yaml: | + resource_manager.yaml: | propeller: resourcemanager: type: noop - storage.yaml: | + storage.yaml: | storage: type: stow stow: @@ -568,7 +568,7 @@ data: cache: max_size_mbs: 0 target_gc_percent: 70 - task_logs.yaml: | + task_logs.yaml: | plugins: k8s-array: logs: @@ -610,103 +610,103 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flyteadmin - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - - limitranges - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + - limitranges + verbs: + - "*" --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm rules: -# Allow RO access to PODS -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -# Allow Event recording access -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -# Allow Access All plugin objects -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -# Allow Access to CRD -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -# Allow Access to all resources under flyte.lyft.com -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + # Allow RO access to PODS + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + # Allow Event recording access + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + # Allow Access All plugin objects + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + # Allow Access to CRD + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + # Allow Access to all resources under flyte.lyft.com + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a ClusterRole for the webhook @@ -735,7 +735,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flyteadmin-binding - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -745,16 +745,16 @@ roleRef: kind: ClusterRole name: flyte-flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- # Source: flyte-core/templates/propeller/rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: flyte-flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -764,9 +764,9 @@ roleRef: kind: ClusterRole name: flyte-flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- # Source: flyte-core/templates/propeller/webhook.yaml # Create a binding from Role -> ServiceAccount @@ -790,12 +790,12 @@ kind: Service metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: cloud.google.com/app-protocols: '{"grpc":"HTTP2"}' projectcontour.io/upstream-protocol.h2c: grpc spec: @@ -821,7 +821,7 @@ spec: protocol: TCP appProtocol: TCP port: 10254 - selector: + selector: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte --- @@ -831,7 +831,7 @@ kind: Service metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -839,12 +839,12 @@ metadata: spec: type: ClusterIP ports: - - name: http - port: 80 - protocol: TCP - appProtocol: TCP - targetPort: 8080 - selector: + - name: http + port: 80 + protocol: TCP + appProtocol: TCP + targetPort: 8080 + selector: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte --- @@ -854,26 +854,26 @@ kind: Service metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm - annotations: + annotations: cloud.google.com/app-protocols: '{"grpc":"HTTP2"}' projectcontour.io/upstream-protocol.h2c: grpc spec: type: NodePort ports: - - name: http - port: 88 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 89 - protocol: TCP - targetPort: 8089 - selector: + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte --- @@ -884,7 +884,7 @@ kind: Service metadata: name: flyte-pod-webhook namespace: flyte - annotations: + annotations: projectcontour.io/upstream-protocol.h2c: grpc spec: selector: @@ -901,7 +901,7 @@ kind: Deployment metadata: name: flyteadmin namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -909,20 +909,20 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "20a517901c6b6f01f47e968fa15ca51f6d9522e728ecace8b48553eb428cde6" - labels: + labels: app.kubernetes.io/name: flyteadmin app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -931,11 +931,11 @@ spec: type: spc_t initContainers: - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - run + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: run-migrations @@ -944,19 +944,19 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - seed-projects - - flytesnacks - - flytetester - - flyteexamples + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: seed-projects @@ -965,16 +965,16 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: base-config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: base-config-volume - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - clusterresource - - sync + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" name: sync-cluster-resources @@ -983,21 +983,21 @@ spec: capabilities: drop: ["ALL"] volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets - name: generate-secrets image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" imagePullPolicy: "IfNotPresent" command: ["/bin/sh", "-c"] args: [ - "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --name flyte-admin-secrets --fromPath /etc/scratch/secrets", ] securityContext: allowPrivilegeEscalation: false @@ -1014,81 +1014,95 @@ spec: fieldRef: fieldPath: metadata.namespace containers: - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flyteadmin - ports: - - containerPort: 8088 - - containerPort: 8089 - - containerPort: 10254 - readinessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 15 - livenessProbe: - exec: - command: [ "sh", "-c", "reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ \"$reply\" -lt 200 -o \"$reply\" -ge 400 ]; then exit 1; fi;","grpc_health_probe", "-addr=:8089"] - initialDelaySeconds: 20 - periodSeconds: 5 - resources: - limits: - cpu: 250m - ephemeral-storage: 2Gi - memory: 500Mi - requests: - cpu: 500m - ephemeral-storage: 2Gi - memory: 1G - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /srv/flyte - name: shared-data - - mountPath: /etc/flyte/config - name: clusters-config-volume - - mountPath: /etc/secrets/ - name: admin-secrets + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/flyteadmin:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + - containerPort: 10254 + readinessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 15 + livenessProbe: + exec: + command: + [ + "sh", + "-c", + 'reply=$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:8088/healthcheck); if [ "$reply" -lt 200 -o "$reply" -ge 400 ]; then exit 1; fi;', + "grpc_health_probe", + "-addr=:8089", + ] + initialDelaySeconds: 20 + periodSeconds: 5 + resources: + limits: + cpu: 250m + ephemeral-storage: 2Gi + memory: 500Mi + requests: + cpu: 500m + ephemeral-storage: 2Gi + memory: 1G + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: clusters-config-volume + - mountPath: /etc/secrets/ + name: admin-secrets serviceAccountName: flyteadmin volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - emptyDir: {} - name: scratch - - configMap: - name: flyte-admin-base-config - name: base-config-volume - - projected: - sources: - - configMap: - name: flyte-admin-base-config - - configMap: - name: flyte-admin-clusters-config - name: clusters-config-volume - - configMap: - name: clusterresource-template - name: resource-templates - - name: admin-secrets - secret: - secretName: flyte-admin-secrets - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - emptyDir: {} + name: scratch + - configMap: + name: flyte-admin-base-config + name: base-config-volume + - projected: + sources: + - configMap: + name: flyte-admin-base-config + - configMap: + name: flyte-admin-clusters-config + name: clusters-config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: admin-secrets + secret: + secretName: flyte-admin-secrets + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteadmin - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteadmin + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/clusterresourcesync/deployment.yaml apiVersion: apps/v1 @@ -1096,7 +1110,7 @@ kind: Deployment metadata: name: syncresources namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1104,7 +1118,7 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte template: @@ -1113,7 +1127,7 @@ spec: configChecksum: "dc18f5d54e0770c574e6b0693724047e22063030259104eebb554398d63209f" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flyteclusterresourcesync app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1130,12 +1144,12 @@ spec: imagePullPolicy: "IfNotPresent" name: sync-cluster-resources volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume serviceAccountName: flyteadmin volumes: - name: db-pass @@ -1154,7 +1168,7 @@ kind: Deployment metadata: name: flyteconsole namespace: flyte - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1162,58 +1176,58 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "2f930e1732c47d0849f79f9a8d06262ec97597a217bbf2337ae4f2938402ee0" - labels: + labels: app.kubernetes.io/name: flyteconsole app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true runAsUser: 1000 seLinuxOptions: type: spc_t containers: - - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" - imagePullPolicy: "IfNotPresent" - name: flyteconsole - envFrom: - - configMapRef: - name: flyte-console-config - ports: - - containerPort: 8080 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 250m - memory: 250Mi - requests: - cpu: 10m - memory: 50Mi - volumeMounts: - - mountPath: /srv/flyte - name: shared-data + - image: "cr.flyte.org/flyteorg/flyteconsole:v1.17.1" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 250m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data volumes: - - emptyDir: {} - name: shared-data - affinity: + - emptyDir: {} + name: shared-data + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flyteconsole - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteconsole + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/datacatalog/deployment.yaml apiVersion: apps/v1 @@ -1221,7 +1235,7 @@ kind: Deployment metadata: name: datacatalog namespace: flyte - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1229,20 +1243,20 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "537b12b49584e5eb9da85bbf0d8d8d21d8edce0560b0b53f595485f2cdb1cb6" - labels: + labels: app.kubernetes.io/name: datacatalog app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 1001 fsGroupChangePolicy: OnRootMismatch runAsNonRoot: true @@ -1250,72 +1264,72 @@ spec: seLinuxOptions: type: spc_t initContainers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - migrate - - run - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: run-migrations - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] containers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - serve - image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: datacatalog - ports: - - containerPort: 8080 - - containerPort: 8089 - - containerPort: 10254 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 500m - ephemeral-storage: 2Gi - memory: 500Mi - requests: - cpu: 50m - ephemeral-storage: 2Gi - memory: 200Mi - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/datacatalog/config - name: config-volume + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "cr.flyte.org/flyteorg/datacatalog:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8080 + - containerPort: 8089 + - containerPort: 10254 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 500m + ephemeral-storage: 2Gi + memory: 500Mi + requests: + cpu: 50m + ephemeral-storage: 2Gi + memory: 200Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume serviceAccountName: datacatalog volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - configMap: - name: datacatalog-config - name: config-volume - affinity: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: datacatalog - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: datacatalog + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/flytescheduler/deployment.yaml apiVersion: apps/v1 @@ -1323,7 +1337,7 @@ kind: Deployment metadata: name: flytescheduler namespace: flyte - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1331,20 +1345,20 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte template: metadata: annotations: configChecksum: "20a517901c6b6f01f47e968fa15ca51f6d9522e728ecace8b48553eb428cde6" - labels: + labels: app.kubernetes.io/name: flytescheduler app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -1352,69 +1366,69 @@ spec: seLinuxOptions: type: spc_t initContainers: - - command: - - flytescheduler - - precheck - - --config - - /etc/flyte/config/*.yaml - image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytescheduler-check - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: config-volume - - name: auth - mountPath: /etc/secrets/ + - command: + - flytescheduler + - precheck + - --config + - /etc/flyte/config/*.yaml + image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytescheduler-check + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ containers: - - command: - - flytescheduler - - run - - --config - - /etc/flyte/config/*.yaml - image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytescheduler - ports: - - containerPort: 10254 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - resources: - limits: - cpu: 250m - ephemeral-storage: 100Mi - memory: 500Mi - requests: - cpu: 10m - ephemeral-storage: 50Mi - memory: 50Mi - volumeMounts: - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/flyte/config - name: config-volume - - name: auth - mountPath: /etc/secrets/ + - command: + - flytescheduler + - run + - --config + - /etc/flyte/config/*.yaml + image: "cr.flyte.org/flyteorg/flytescheduler:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytescheduler + ports: + - containerPort: 10254 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ serviceAccountName: flytescheduler volumes: - - name: db-pass - secret: - secretName: db-pass - - emptyDir: {} - name: shared-data - - configMap: - name: flyte-scheduler-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-scheduler-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth --- # Source: flyte-core/templates/propeller/deployment.yaml apiVersion: apps/v1 @@ -1422,7 +1436,7 @@ kind: Deployment metadata: namespace: flyte name: flytepropeller - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 @@ -1430,7 +1444,7 @@ metadata: spec: replicas: 1 selector: - matchLabels: + matchLabels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte template: @@ -1439,65 +1453,65 @@ spec: configChecksum: "8562f7f608d4936e13f6ad70c18c7c095068e742243e7f380f89694d2182110" prometheus.io/path: "/metrics" prometheus.io/port: "10254" - labels: + labels: app.kubernetes.io/name: flytepropeller app.kubernetes.io/instance: flyte helm.sh/chart: flyte-core-v0.1.10 app.kubernetes.io/managed-by: Helm spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 containers: - - command: - - flytepropeller - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - name: flytepropeller - ports: - - containerPort: 10254 - resources: - limits: - cpu: 500m - ephemeral-storage: 2Gi - memory: 1Gi - requests: - cpu: 50m - ephemeral-storage: 2Gi - memory: 1Gi - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: auth - mountPath: /etc/secrets/ - terminationMessagePolicy: "FallbackToLogsOnError" + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 500m + ephemeral-storage: 2Gi + memory: 1Gi + requests: + cpu: 50m + ephemeral-storage: 2Gi + memory: 1Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + terminationMessagePolicy: "FallbackToLogsOnError" serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth - affinity: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth + affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchLabels: - app.kubernetes.io/name: flytepropeller - topologyKey: kubernetes.io/hostname + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname --- # Source: flyte-core/templates/propeller/webhook.yaml # Create the actual deployment @@ -1523,7 +1537,7 @@ spec: prometheus.io/path: "/metrics" prometheus.io/port: "10254" spec: - securityContext: + securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsNonRoot: true @@ -1532,32 +1546,32 @@ spec: type: spc_t serviceAccountName: flyte-pod-webhook initContainers: - - name: generate-secrets - image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" - imagePullPolicy: "IfNotPresent" - command: - - flytepropeller - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config + - name: generate-secrets + image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config containers: - name: webhook image: "cr.flyte.org/flyteorg/flytepropeller:v1.13.2" @@ -1578,7 +1592,7 @@ spec: fieldRef: fieldPath: metadata.namespace ports: - - containerPort: 9443 + - containerPort: 9443 securityContext: allowPrivilegeEscalation: false capabilities: @@ -1609,14 +1623,14 @@ kind: Ingress metadata: name: flyte-core namespace: flyte - annotations: + annotations: cert-manager.io/issuer: letsencrypt-production kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/app-root: /console nginx.ingress.kubernetes.io/service-upstream: "true" nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: - ingressClassName: + ingressClassName: rules: - http: paths: @@ -1755,12 +1769,12 @@ spec: name: flyteadmin port: number: 80 - host: '' + host: "" tls: - secretName: flyte-flyte-tls hosts: - - '' - + - "" + # Certain ingress controllers like nginx cannot serve HTTP 1 and GRPC with a single ingress because GRPC can only # enabled on the ingress object, not on backend services (GRPC annotation is set on the ingress, not on the services). --- @@ -1770,7 +1784,7 @@ kind: Ingress metadata: name: flyte-core-grpc namespace: flyte - annotations: + annotations: cert-manager.io/issuer: letsencrypt-production kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/app-root: /console @@ -1778,13 +1792,13 @@ metadata: nginx.ingress.kubernetes.io/service-upstream: "true" nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: - ingressClassName: + ingressClassName: rules: - - host: '' + - host: "" http: paths: # - + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. - path: /flyteidl.service.SignalService pathType: ImplementationSpecific @@ -1873,4 +1887,4 @@ spec: tls: - secretName: flyte-flyte-tls hosts: - - '' + - "" diff --git a/deployment/sandbox-binary/flyte_sandbox_binary_helm_generated.yaml b/deployment/sandbox-binary/flyte_sandbox_binary_helm_generated.yaml index 6fafa61550..035928299e 100644 --- a/deployment/sandbox-binary/flyte_sandbox_binary_helm_generated.yaml +++ b/deployment/sandbox-binary/flyte_sandbox_binary_helm_generated.yaml @@ -106,10 +106,10 @@ data: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo @@ -164,79 +164,79 @@ metadata: annotations: rules: - apiGroups: - - "" + - "" resources: - - namespaces - - resourcequotas - - secrets - - serviceaccounts + - namespaces + - resourcequotas + - secrets + - serviceaccounts verbs: - - create - - get - - list - - patch - - update + - create + - get + - list + - patch + - update - apiGroups: - - "" + - "" resources: - - pods + - pods verbs: - - create - - delete - - get - - list - - patch - - update - - watch + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - - "" + - "" resources: - - events + - events verbs: - - create - - delete - - patch - - update + - create + - delete + - patch + - update - apiGroups: - - "" + - "" resources: - - podtemplates + - podtemplates verbs: - - get - - list - - watch + - get + - list + - watch - apiGroups: - - flyte.lyft.com + - flyte.lyft.com resources: - - flyteworkflows + - flyteworkflows verbs: - - create - - delete - - deletecollection - - get - - list - - patch - - post - - update - - watch + - create + - delete + - deletecollection + - get + - list + - patch + - post + - update + - watch - apiGroups: - - apiextensions.k8s.io + - apiextensions.k8s.io resources: - - customresourcedefinitions + - customresourcedefinitions verbs: - - create - - get - - list + - create + - get + - list - apiGroups: - - admissionregistration.k8s.io + - admissionregistration.k8s.io resources: - - mutatingwebhookconfigurations + - mutatingwebhookconfigurations verbs: - - create - - get - - list - - patch - - update + - create + - get + - list + - patch + - update --- # Source: flyte-binary/templates/clusterrolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -430,14 +430,14 @@ spec: - name: cluster-resource-templates projected: sources: - - configMap: - name: flyte-flyte-binary-cluster-resource-templates + - configMap: + name: flyte-flyte-binary-cluster-resource-templates - name: config projected: sources: - - configMap: - name: flyte-flyte-binary-config - - secret: - name: flyte-flyte-binary-config-secret + - configMap: + name: flyte-flyte-binary-config + - secret: + name: flyte-flyte-binary-config-secret - name: state emptyDir: {} diff --git a/deployment/sandbox/flyte_helm_generated.yaml b/deployment/sandbox/flyte_helm_generated.yaml index 22b4855352..d3645163b2 100644 --- a/deployment/sandbox/flyte_helm_generated.yaml +++ b/deployment/sandbox/flyte_helm_generated.yaml @@ -643,10 +643,10 @@ data: container: container container_array: k8s-array sensor: agent-service - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 0f3124faff..25e4b12094 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -13,17 +13,17 @@ spec: kind: FlyteWorkflow plural: flyteworkflows shortNames: - - fly + - fly singular: flyteworkflow scope: Namespaced versions: - - name: v1alpha1 - schema: - openAPIV3Schema: - type: object - x-kubernetes-preserve-unknown-fields: true - served: true - storage: true + - name: v1alpha1 + schema: + openAPIV3Schema: + type: object + x-kubernetes-preserve-unknown-fields: true + served: true + storage: true --- apiVersion: v1 kind: ServiceAccount @@ -55,18 +55,18 @@ metadata: name: flyte-pod-webhook namespace: flyte rules: -- apiGroups: - - '*' - resources: - - mutatingwebhookconfigurations - - secrets - - pods - - replicasets/finalizers - verbs: - - get - - create - - update - - patch + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + - replicasets/finalizers + verbs: + - get + - create + - update + - patch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -74,85 +74,85 @@ metadata: name: flyteadmin namespace: flyte rules: -- apiGroups: - - "" - - flyte.lyft.com - - rbac.authorization.k8s.io - resources: - - configmaps - - flyteworkflows - - namespaces - - pods - - resourcequotas - - roles - - rolebindings - - secrets - - services - - serviceaccounts - - spark-role - verbs: - - '*' + - apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + verbs: + - "*" --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: flytepropeller rules: -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - update - - delete - - patch -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch - - create - - update - - delete - - patch -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - - create - - delete - - update -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - - flyteworkflows/finalizers - verbs: - - get - - list - - watch - - create - - update - - delete - - patch - - post - - deletecollection + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -164,9 +164,9 @@ roleRef: kind: ClusterRole name: flyte-pod-webhook subjects: -- kind: ServiceAccount - name: flyte-pod-webhook - namespace: flyte + - kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -178,9 +178,9 @@ roleRef: kind: ClusterRole name: flyteadmin subjects: -- kind: ServiceAccount - name: flyteadmin - namespace: flyte + - kind: ServiceAccount + name: flyteadmin + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -192,9 +192,9 @@ roleRef: kind: ClusterRole name: flytepropeller subjects: -- kind: ServiceAccount - name: flytepropeller - namespace: flyte + - kind: ServiceAccount + name: flytepropeller + namespace: flyte --- apiVersion: v1 data: @@ -427,13 +427,13 @@ data: task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array k8s.yaml: | plugins: @@ -542,14 +542,14 @@ metadata: namespace: flyte spec: ports: - - name: http - port: 88 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 89 - protocol: TCP - targetPort: 8089 + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 selector: app: datacatalog --- @@ -562,10 +562,10 @@ metadata: namespace: flyte spec: ports: - - name: https - port: 443 - protocol: TCP - targetPort: 9443 + - name: https + port: 443 + protocol: TCP + targetPort: 9443 selector: app: flyte-pod-webhook --- @@ -578,18 +578,18 @@ metadata: namespace: flyte spec: ports: - - name: redoc - port: 87 - protocol: TCP - targetPort: 8087 - - name: http - port: 80 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 81 - protocol: TCP - targetPort: 8089 + - name: redoc + port: 87 + protocol: TCP + targetPort: 8087 + - name: http + port: 80 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 81 + protocol: TCP + targetPort: 8089 selector: app: flyteadmin --- @@ -601,10 +601,10 @@ metadata: spec: externalName: minio ports: - - name: minio-api - port: 9000 - - name: minio-console - port: 9001 + - name: minio-api + port: 9000 + - name: minio-console + port: 9001 selector: app: minio --- @@ -615,7 +615,7 @@ metadata: namespace: flyte spec: ports: - - port: 5432 + - port: 5432 selector: app: postgres --- @@ -643,51 +643,51 @@ spec: app.kubernetes.io/version: 0.3.0 spec: containers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - serve - image: cr.flyte.org/flyteorg/datacatalog:v1.0.51 - imagePullPolicy: IfNotPresent - name: datacatalog - ports: - - containerPort: 8088 - - containerPort: 8089 - volumeMounts: - - mountPath: /etc/datacatalog/config - name: config-volume - - mountPath: /etc/db - name: db-pass + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: cr.flyte.org/flyteorg/datacatalog:v1.0.51 + imagePullPolicy: IfNotPresent + name: datacatalog + ports: + - containerPort: 8088 + - containerPort: 8089 + volumeMounts: + - mountPath: /etc/datacatalog/config + name: config-volume + - mountPath: /etc/db + name: db-pass initContainers: - - command: - - datacatalog - - --config - - /etc/datacatalog/config/*.yaml - - migrate - - run - image: cr.flyte.org/flyteorg/datacatalog:v1.0.51 - imagePullPolicy: IfNotPresent - name: run-migrations - volumeMounts: - - mountPath: /etc/datacatalog/config - name: config-volume - - mountPath: /etc/db - name: db-pass + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: cr.flyte.org/flyteorg/datacatalog:v1.0.51 + imagePullPolicy: IfNotPresent + name: run-migrations + volumeMounts: + - mountPath: /etc/datacatalog/config + name: config-volume + - mountPath: /etc/db + name: db-pass securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 serviceAccountName: datacatalog volumes: - - emptyDir: {} - name: shared-data - - configMap: - name: datacatalog-config-64k8dg9gck - name: config-volume - - name: db-pass - secret: - secretName: db-pass-9dgchhk2bm + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config-64k8dg9gck + name: config-volume + - name: db-pass + secret: + secretName: db-pass-9dgchhk2bm --- apiVersion: apps/v1 kind: Deployment @@ -712,66 +712,66 @@ spec: app.kubernetes.io/version: 0.5.13 spec: containers: - - args: - - webhook - - --config - - /etc/flyte/config/*.yaml - command: - - flytepropeller - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 - imagePullPolicy: IfNotPresent - name: webhook - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - readOnly: true - - mountPath: /etc/webhook/certs - name: webhook-certs - readOnly: true + - args: + - webhook + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + readOnly: true + - mountPath: /etc/webhook/certs + name: webhook-certs + readOnly: true initContainers: - - args: - - webhook - - init-certs - - --config - - /etc/flyte/config/*.yaml - command: - - flytepropeller - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume + - args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 serviceAccountName: flyte-pod-webhook volumes: - - configMap: - name: flyte-propeller-config-hhtmttc8b4 - name: config-volume - - name: webhook-certs - secret: - secretName: flyte-pod-webhook + - configMap: + name: flyte-propeller-config-hhtmttc8b4 + name: config-volume + - name: webhook-certs + secret: + secretName: flyte-pod-webhook --- apiVersion: apps/v1 kind: Deployment @@ -797,146 +797,146 @@ spec: app.kubernetes.io/version: 0.4.13 spec: containers: - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - serve - image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 - imagePullPolicy: IfNotPresent - name: flyteadmin - ports: - - containerPort: 8088 - - containerPort: 8089 - resources: - limits: - cpu: "0.1" - ephemeral-storage: 100Mi - memory: 200Mi - volumeMounts: - - mountPath: /srv/flyte - name: shared-data - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/db - name: db-pass - - mountPath: /etc/secrets/ - name: auth - - command: - - sh - - -c - - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh - env: - - name: PAGE_TITLE - value: Flyte Admin OpenAPI - - name: SPEC_URL - value: /api/v1/openapi - - name: PORT - value: "8087" - image: docker.io/redocly/redoc - imagePullPolicy: IfNotPresent - name: redoc - ports: - - containerPort: 8087 - resources: - limits: - cpu: "0.1" - memory: 200Mi + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 + imagePullPolicy: IfNotPresent + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: "0.1" + ephemeral-storage: 100Mi + memory: 200Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/secrets/ + name: auth + - command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: Flyte Admin OpenAPI + - name: SPEC_URL + value: /api/v1/openapi + - name: PORT + value: "8087" + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + name: redoc + ports: + - containerPort: 8087 + resources: + limits: + cpu: "0.1" + memory: 200Mi initContainers: - - command: - - sh - - -c - - until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done; - image: ecr.flyte.org/ubuntu/postgres:13-21.04_beta - name: check-db-ready - securityContext: - runAsUser: 999 - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - run - image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 - imagePullPolicy: IfNotPresent - name: run-migrations - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/db - name: db-pass - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - migrate - - seed-projects - - flytetester - - flytesnacks - image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 - imagePullPolicy: IfNotPresent - name: seed-projects - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/db - name: db-pass - - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - clusterresource - - sync - image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 - imagePullPolicy: IfNotPresent - name: sync-cluster-resources - volumeMounts: - - mountPath: /etc/flyte/clusterresource/templates - name: resource-templates - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/db - name: db-pass - - args: - - flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/scratch/secrets - command: - - /bin/sh - - -c - env: - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/scratch - name: scratch + - command: + - sh + - -c + - until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done; + image: ecr.flyte.org/ubuntu/postgres:13-21.04_beta + name: check-db-ready + securityContext: + runAsUser: 999 + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run + image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 + imagePullPolicy: IfNotPresent + name: run-migrations + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/db + name: db-pass + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytetester + - flytesnacks + image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 + imagePullPolicy: IfNotPresent + name: seed-projects + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/db + name: db-pass + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 + imagePullPolicy: IfNotPresent + name: sync-cluster-resources + volumeMounts: + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/db + name: db-pass + - args: + - flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/scratch/secrets && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/scratch/secrets + command: + - /bin/sh + - -c + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: cr.flyte.org/flyteorg/flyteadmin:v1.1.118 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/scratch + name: scratch securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 serviceAccountName: flyteadmin volumes: - - configMap: - name: clusterresource-template-dk5mbchdmt - name: resource-templates - - emptyDir: {} - name: shared-data - - emptyDir: {} - name: scratch - - configMap: - name: flyte-admin-config-hc64g2ct6h - name: config-volume - - name: db-pass - secret: - secretName: db-pass-9dgchhk2bm - - name: auth - secret: - secretName: flyte-admin-auth + - configMap: + name: clusterresource-template-dk5mbchdmt + name: resource-templates + - emptyDir: {} + name: shared-data + - emptyDir: {} + name: scratch + - configMap: + name: flyte-admin-config-hc64g2ct6h + name: config-volume + - name: db-pass + secret: + secretName: db-pass-9dgchhk2bm + - name: auth + secret: + secretName: flyte-admin-auth --- apiVersion: apps/v1 kind: Deployment @@ -961,42 +961,42 @@ spec: app.kubernetes.io/version: 0.7.1 spec: containers: - - args: - - --config - - /etc/flyte/config/*.yaml - command: - - flytepropeller - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 - imagePullPolicy: IfNotPresent - name: flytepropeller - ports: - - containerPort: 10254 - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/secrets/ - name: auth + - args: + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: cr.flyte.org/flyteorg/flytepropeller:v1.1.116 + imagePullPolicy: IfNotPresent + name: flytepropeller + ports: + - containerPort: 10254 + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/ + name: auth securityContext: fsGroup: 65534 fsGroupChangePolicy: Always runAsUser: 1001 serviceAccountName: flytepropeller volumes: - - configMap: - name: flyte-propeller-config-hhtmttc8b4 - name: config-volume - - name: auth - secret: - secretName: flyte-secret-auth + - configMap: + name: flyte-propeller-config-hhtmttc8b4 + name: config-volume + - name: auth + secret: + secretName: flyte-secret-auth --- apiVersion: apps/v1 kind: Deployment @@ -1022,53 +1022,53 @@ spec: app.kubernetes.io/version: 0.3.4 spec: containers: - - command: - - flytescheduler - - run - - --config - - /etc/flyte/config/*.yaml - image: cr.flyte.org/flyteorg/flytescheduler:v0.6.49 - imagePullPolicy: IfNotPresent - name: flytescheduler - resources: - limits: - cpu: 250m - ephemeral-storage: 100Mi - memory: 500Mi - requests: - cpu: 10m - ephemeral-storage: 50Mi - memory: 50Mi - volumeMounts: - - mountPath: /etc/secrets/ - name: auth - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/db - name: db-pass + - command: + - flytescheduler + - run + - --config + - /etc/flyte/config/*.yaml + image: cr.flyte.org/flyteorg/flytescheduler:v0.6.49 + imagePullPolicy: IfNotPresent + name: flytescheduler + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + - mountPath: /etc/secrets/ + name: auth + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/db + name: db-pass initContainers: - - command: - - flytescheduler - - precheck - - --config - - /etc/flyte/config/*.yaml - image: cr.flyte.org/flyteorg/flytescheduler:v0.6.49 - imagePullPolicy: IfNotPresent - name: flytescheduler-check - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume + - command: + - flytescheduler + - precheck + - --config + - /etc/flyte/config/*.yaml + image: cr.flyte.org/flyteorg/flytescheduler:v0.6.49 + imagePullPolicy: IfNotPresent + name: flytescheduler-check + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume serviceAccountName: flyteadmin volumes: - - configMap: - name: flyte-scheduler-config - name: config-volume - - name: db-pass - secret: - secretName: db-pass-9dgchhk2bm - - name: auth - secret: - secretName: flyte-secret-auth + - configMap: + name: flyte-scheduler-config + name: config-volume + - name: db-pass + secret: + secretName: db-pass-9dgchhk2bm + - name: auth + secret: + secretName: flyte-secret-auth --- apiVersion: apps/v1 kind: Deployment @@ -1085,30 +1085,30 @@ spec: app: minio spec: containers: - - env: - - name: MINIO_ACCESS_KEY - value: minio - - name: MINIO_SECRET_KEY - value: miniostorage - - name: MINIO_DEFAULT_BUCKETS - value: my-s3-bucket - image: ecr.flyte.org/bitnami/minio:2021.10.13-debian-10-r0 - name: minio - ports: - - containerPort: 9000 + - env: + - name: MINIO_ACCESS_KEY + value: minio + - name: MINIO_SECRET_KEY + value: miniostorage + - name: MINIO_DEFAULT_BUCKETS + value: my-s3-bucket + image: ecr.flyte.org/bitnami/minio:2021.10.13-debian-10-r0 name: minio - - containerPort: 9001 - name: minio-console - volumeMounts: - - mountPath: /data - name: minio-storage + ports: + - containerPort: 9000 + name: minio + - containerPort: 9001 + name: minio-console + volumeMounts: + - mountPath: /data + name: minio-storage securityContext: fsGroup: 1001 fsGroupChangePolicy: OnRootMismatch runAsUser: 1001 volumes: - - emptyDir: {} - name: minio-storage + - emptyDir: {} + name: minio-storage --- apiVersion: apps/v1 kind: Deployment @@ -1125,20 +1125,20 @@ spec: app: postgres spec: containers: - - env: - - name: POSTGRES_HOST_AUTH_METHOD - value: trust - image: ecr.flyte.org/ubuntu/postgres:13-21.04_beta - name: postgres - ports: - - containerPort: 5432 + - env: + - name: POSTGRES_HOST_AUTH_METHOD + value: trust + image: ecr.flyte.org/ubuntu/postgres:13-21.04_beta name: postgres - volumeMounts: - - mountPath: /var/lib/postgresql/data - name: postgres-storage + ports: + - containerPort: 5432 + name: postgres + volumeMounts: + - mountPath: /var/lib/postgresql/data + name: postgres-storage volumes: - - emptyDir: {} - name: postgres-storage + - emptyDir: {} + name: postgres-storage --- apiVersion: networking.k8s.io/v1 kind: Ingress @@ -1149,12 +1149,12 @@ metadata: namespace: flyte spec: rules: - - http: - paths: - - backend: - service: - name: minio - port: - number: 9001 - path: /minio - pathType: ImplementationSpecific + - http: + paths: + - backend: + service: + name: minio + port: + number: 9001 + path: /minio + pathType: ImplementationSpecific diff --git a/docker/sandbox-bundled/manifests/complete-agent.yaml b/docker/sandbox-bundled/manifests/complete-agent.yaml index f0c418ad95..57377d3741 100644 --- a/docker/sandbox-bundled/manifests/complete-agent.yaml +++ b/docker/sandbox-bundled/manifests/complete-agent.yaml @@ -39,7 +39,7 @@ metadata: name: flyte-sandbox-minio namespace: flyte secrets: -- name: flyte-sandbox-minio + - name: flyte-sandbox-minio --- apiVersion: v1 kind: ServiceAccount @@ -64,48 +64,48 @@ metadata: name: flyte-sandbox-kubernetes-dashboard namespace: flyte rules: -- apiGroups: - - "" - resourceNames: - - kubernetes-dashboard-key-holder - - kubernetes-dashboard-certs - - kubernetes-dashboard-csrf - resources: - - secrets - verbs: - - get - - update - - delete -- apiGroups: - - "" - resourceNames: - - kubernetes-dashboard-settings - resources: - - configmaps - verbs: - - get - - update -- apiGroups: - - "" - resourceNames: - - heapster - - dashboard-metrics-scraper - resources: - - services - verbs: - - proxy -- apiGroups: - - "" - resourceNames: - - heapster - - 'http:heapster:' - - 'https:heapster:' - - dashboard-metrics-scraper - - http:dashboard-metrics-scraper - resources: - - services/proxy - verbs: - - get + - apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-key-holder + - kubernetes-dashboard-certs + - kubernetes-dashboard-csrf + resources: + - secrets + verbs: + - get + - update + - delete + - apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-settings + resources: + - configmaps + verbs: + - get + - update + - apiGroups: + - "" + resourceNames: + - heapster + - dashboard-metrics-scraper + resources: + - services + verbs: + - proxy + - apiGroups: + - "" + resourceNames: + - heapster + - "http:heapster:" + - "https:heapster:" + - dashboard-metrics-scraper + - http:dashboard-metrics-scraper + resources: + - services/proxy + verbs: + - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -119,86 +119,86 @@ metadata: name: flyte-sandbox-cluster-role namespace: flyte rules: -- apiGroups: - - "" - resources: - - namespaces - - resourcequotas - - secrets - - serviceaccounts - verbs: - - create - - get - - list - - patch - - update -- apiGroups: - - "" - resources: - - pods - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - delete - - patch - - update -- apiGroups: - - "" - resources: - - podtemplates - verbs: - - get - - list - - watch -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - verbs: - - create - - delete - - deletecollection - - get - - list - - patch - - post - - update - - watch -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - create - - get - - list -- apiGroups: - - admissionregistration.k8s.io - resources: - - mutatingwebhookconfigurations - verbs: - - create - - get - - list - - patch - - update -- apiGroups: - - '*' - resources: - - '*' - verbs: - - '*' + - apiGroups: + - "" + resources: + - namespaces + - resourcequotas + - secrets + - serviceaccounts + verbs: + - create + - get + - list + - patch + - update + - apiGroups: + - "" + resources: + - pods + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - create + - delete + - patch + - update + - apiGroups: + - "" + resources: + - podtemplates + verbs: + - get + - list + - watch + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + verbs: + - create + - delete + - deletecollection + - get + - list + - patch + - post + - update + - watch + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - list + - apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + verbs: + - create + - get + - list + - patch + - update + - apiGroups: + - "*" + resources: + - "*" + verbs: + - "*" --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -211,130 +211,130 @@ metadata: helm.sh/chart: kubernetes-dashboard-6.0.0 name: flyte-sandbox-kubernetes-dashboard-readonly rules: -- apiGroups: - - "" - resources: - - configmaps - - endpoints - - persistentvolumeclaims - - pods - - replicationcontrollers - - replicationcontrollers/scale - - serviceaccounts - - services - - nodes - - persistentvolumeclaims - - persistentvolumes - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - bindings - - events - - limitranges - - namespaces/status - - pods/log - - pods/status - - replicationcontrollers/status - - resourcequotas - - resourcequotas/status - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - namespaces - verbs: - - get - - list - - watch -- apiGroups: - - apps - resources: - - daemonsets - - deployments - - deployments/scale - - replicasets - - replicasets/scale - - statefulsets - verbs: - - get - - list - - watch -- apiGroups: - - autoscaling - resources: - - horizontalpodautoscalers - verbs: - - get - - list - - watch -- apiGroups: - - batch - resources: - - cronjobs - - jobs - verbs: - - get - - list - - watch -- apiGroups: - - extensions - resources: - - daemonsets - - deployments - - deployments/scale - - ingresses - - networkpolicies - - replicasets - - replicasets/scale - - replicationcontrollers/scale - verbs: - - get - - list - - watch -- apiGroups: - - policy - resources: - - poddisruptionbudgets - verbs: - - get - - list - - watch -- apiGroups: - - networking.k8s.io - resources: - - networkpolicies - - ingresses - verbs: - - get - - list - - watch -- apiGroups: - - storage.k8s.io - resources: - - storageclasses - - volumeattachments - verbs: - - get - - list - - watch -- apiGroups: - - rbac.authorization.k8s.io - resources: - - clusterrolebindings - - clusterroles - - roles - - rolebindings - verbs: - - get - - list - - watch + - apiGroups: + - "" + resources: + - configmaps + - endpoints + - persistentvolumeclaims + - pods + - replicationcontrollers + - replicationcontrollers/scale + - serviceaccounts + - services + - nodes + - persistentvolumeclaims + - persistentvolumes + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - bindings + - events + - limitranges + - namespaces/status + - pods/log + - pods/status + - replicationcontrollers/status + - resourcequotas + - resourcequotas/status + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - namespaces + verbs: + - get + - list + - watch + - apiGroups: + - apps + resources: + - daemonsets + - deployments + - deployments/scale + - replicasets + - replicasets/scale + - statefulsets + verbs: + - get + - list + - watch + - apiGroups: + - autoscaling + resources: + - horizontalpodautoscalers + verbs: + - get + - list + - watch + - apiGroups: + - batch + resources: + - cronjobs + - jobs + verbs: + - get + - list + - watch + - apiGroups: + - extensions + resources: + - daemonsets + - deployments + - deployments/scale + - ingresses + - networkpolicies + - replicasets + - replicasets/scale + - replicationcontrollers/scale + verbs: + - get + - list + - watch + - apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - networkpolicies + - ingresses + verbs: + - get + - list + - watch + - apiGroups: + - storage.k8s.io + resources: + - storageclasses + - volumeattachments + verbs: + - get + - list + - watch + - apiGroups: + - rbac.authorization.k8s.io + resources: + - clusterrolebindings + - clusterroles + - roles + - rolebindings + verbs: + - get + - list + - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding @@ -352,9 +352,9 @@ roleRef: kind: Role name: flyte-sandbox-kubernetes-dashboard subjects: -- kind: ServiceAccount - name: flyte-sandbox-kubernetes-dashboard - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox-kubernetes-dashboard + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -372,9 +372,9 @@ roleRef: kind: ClusterRole name: flyte-sandbox-cluster-role subjects: -- kind: ServiceAccount - name: flyte-sandbox - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -391,9 +391,9 @@ roleRef: kind: ClusterRole name: flyte-sandbox-kubernetes-dashboard-readonly subjects: -- kind: ServiceAccount - name: flyte-sandbox-kubernetes-dashboard - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox-kubernetes-dashboard + namespace: flyte --- apiVersion: v1 data: @@ -458,10 +458,10 @@ data: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo @@ -922,11 +922,11 @@ metadata: namespace: flyte spec: ports: - - name: http-5000 - nodePort: 30000 - port: 5000 - protocol: TCP - targetPort: 5000 + - name: http-5000 + nodePort: 30000 + port: 5000 + protocol: TCP + targetPort: 5000 selector: app: docker-registry release: flyte-sandbox @@ -945,10 +945,10 @@ metadata: namespace: flyte spec: ports: - - name: grpc - nodePort: null - port: 8089 - targetPort: grpc + - name: grpc + nodePort: null + port: 8089 + targetPort: grpc selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -968,10 +968,10 @@ metadata: namespace: flyte spec: ports: - - name: http - nodePort: null - port: 8088 - targetPort: http + - name: http + nodePort: null + port: 8088 + targetPort: http selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -993,9 +993,9 @@ metadata: namespace: flyte spec: ports: - - name: http - port: 80 - targetPort: http + - name: http + port: 80 + targetPort: http selector: app.kubernetes.io/component: kubernetes-dashboard app.kubernetes.io/instance: flyte-sandbox @@ -1015,13 +1015,13 @@ metadata: spec: externalTrafficPolicy: Cluster ports: - - name: minio-api - nodePort: 30002 - port: 9000 - targetPort: minio-api - - name: minio-console - port: 9001 - targetPort: minio-console + - name: minio-api + nodePort: 30002 + port: 9000 + targetPort: minio-api + - name: minio-console + port: 9001 + targetPort: minio-console selector: app.kubernetes.io/instance: flyte-sandbox app.kubernetes.io/name: minio @@ -1041,10 +1041,10 @@ metadata: spec: externalTrafficPolicy: Cluster ports: - - name: tcp-postgresql - nodePort: 30001 - port: 5432 - targetPort: tcp-postgresql + - name: tcp-postgresql + nodePort: 30001 + port: 5432 + targetPort: tcp-postgresql selector: app.kubernetes.io/component: primary app.kubernetes.io/instance: flyte-sandbox @@ -1067,9 +1067,9 @@ metadata: spec: clusterIP: None ports: - - name: tcp-postgresql - port: 5432 - targetPort: tcp-postgresql + - name: tcp-postgresql + port: 5432 + targetPort: tcp-postgresql publishNotReadyAddresses: true selector: app.kubernetes.io/component: primary @@ -1090,10 +1090,10 @@ metadata: namespace: flyte spec: ports: - - name: http - nodePort: 30080 - port: 8000 - protocol: TCP + - name: http + nodePort: 30080 + port: 8000 + protocol: TCP selector: app.kubernetes.io/component: proxy app.kubernetes.io/instance: flyte-sandbox @@ -1113,9 +1113,9 @@ metadata: namespace: flyte spec: ports: - - name: webhook - port: 443 - targetPort: webhook + - name: webhook + port: 443 + targetPort: webhook selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -1136,11 +1136,11 @@ metadata: namespace: flyte spec: ports: - - appProtocol: TCP - name: agent-grpc - port: 8000 - protocol: TCP - targetPort: agent-grpc + - appProtocol: TCP + name: agent-grpc + port: 8000 + protocol: TCP + targetPort: agent-grpc selector: app.kubernetes.io/instance: flyte-sandbox app.kubernetes.io/name: flyteagent @@ -1159,7 +1159,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce capacity: storage: 1Gi hostPath: @@ -1179,7 +1179,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce capacity: storage: 1Gi hostPath: @@ -1199,7 +1199,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce resources: requests: storage: 1Gi @@ -1219,7 +1219,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce resources: requests: storage: 1Gi @@ -1258,83 +1258,83 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - args: - - start - - --config - - /etc/flyte/config.d/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: flyte-binary:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: /healthcheck - port: http - initialDelaySeconds: 30 - name: flyte - ports: - - containerPort: 8088 - name: http - - containerPort: 8089 - name: grpc - - containerPort: 9443 - name: webhook - readinessProbe: - httpGet: - path: /healthcheck - port: http - initialDelaySeconds: 30 - volumeMounts: - - mountPath: /etc/flyte/cluster-resource-templates - name: cluster-resource-templates - - mountPath: /etc/flyte/config.d - name: config - - mountPath: /var/run/flyte - name: state + - args: + - start + - --config + - /etc/flyte/config.d/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: flyte-binary:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: /healthcheck + port: http + initialDelaySeconds: 30 + name: flyte + ports: + - containerPort: 8088 + name: http + - containerPort: 8089 + name: grpc + - containerPort: 9443 + name: webhook + readinessProbe: + httpGet: + path: /healthcheck + port: http + initialDelaySeconds: 30 + volumeMounts: + - mountPath: /etc/flyte/cluster-resource-templates + name: cluster-resource-templates + - mountPath: /etc/flyte/config.d + name: config + - mountPath: /var/run/flyte + name: state initContainers: - - args: - - | - until pg_isready \ - -h flyte-sandbox-postgresql \ - -p 5432 \ - -U postgres - do - echo waiting for database - sleep 0.1 - done - command: - - sh - - -ec - image: bitnami/postgresql:sandbox - imagePullPolicy: Never - name: wait-for-db + - args: + - | + until pg_isready \ + -h flyte-sandbox-postgresql \ + -p 5432 \ + -U postgres + do + echo waiting for database + sleep 0.1 + done + command: + - sh + - -ec + image: bitnami/postgresql:sandbox + imagePullPolicy: Never + name: wait-for-db serviceAccountName: flyte-sandbox volumes: - - name: cluster-resource-templates - projected: - sources: - - configMap: - name: flyte-sandbox-cluster-resource-templates - - configMap: - name: flyte-sandbox-extra-cluster-resource-templates - - name: config - projected: - sources: - - configMap: - name: flyte-sandbox-config - - secret: - name: flyte-sandbox-config-secret - - configMap: - name: flyte-sandbox-extra-config - - emptyDir: {} - name: state + - name: cluster-resource-templates + projected: + sources: + - configMap: + name: flyte-sandbox-cluster-resource-templates + - configMap: + name: flyte-sandbox-extra-cluster-resource-templates + - name: config + projected: + sources: + - configMap: + name: flyte-sandbox-config + - secret: + name: flyte-sandbox-config-secret + - configMap: + name: flyte-sandbox-extra-config + - emptyDir: {} + name: state --- apiVersion: apps/v1 kind: Deployment @@ -1362,36 +1362,36 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - args: - - --addr - - unix:///run/buildkit/buildkitd.sock - - --addr - - tcp://0.0.0.0:30003 - image: moby/buildkit:sandbox - imagePullPolicy: Never - livenessProbe: - exec: - command: - - buildctl - - debug - - workers - initialDelaySeconds: 5 - periodSeconds: 30 - name: buildkit - ports: - - containerPort: 30003 - name: tcp - protocol: TCP - readinessProbe: - exec: - command: - - buildctl - - debug - - workers - initialDelaySeconds: 5 - periodSeconds: 30 - securityContext: - privileged: true + - args: + - --addr + - unix:///run/buildkit/buildkitd.sock + - --addr + - tcp://0.0.0.0:30003 + image: moby/buildkit:sandbox + imagePullPolicy: Never + livenessProbe: + exec: + command: + - buildctl + - debug + - workers + initialDelaySeconds: 5 + periodSeconds: 30 + name: buildkit + ports: + - containerPort: 30003 + name: tcp + protocol: TCP + readinessProbe: + exec: + command: + - buildctl + - debug + - workers + initialDelaySeconds: 5 + periodSeconds: 30 + securityContext: + privileged: true dnsPolicy: ClusterFirstWithHostNet hostNetwork: true --- @@ -1422,46 +1422,46 @@ spec: release: flyte-sandbox spec: containers: - - command: - - /bin/registry - - serve - - /etc/docker/registry/config.yml - env: - - name: REGISTRY_HTTP_SECRET - valueFrom: - secretKeyRef: - key: haSharedSecret - name: flyte-sandbox-docker-registry-secret - - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY - value: /var/lib/registry - image: registry:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: / - port: 5000 - name: docker-registry - ports: - - containerPort: 5000 - readinessProbe: - httpGet: - path: / - port: 5000 - resources: {} - volumeMounts: - - mountPath: /etc/docker/registry - name: flyte-sandbox-docker-registry-config - - mountPath: /var/lib/registry/ - name: data + - command: + - /bin/registry + - serve + - /etc/docker/registry/config.yml + env: + - name: REGISTRY_HTTP_SECRET + valueFrom: + secretKeyRef: + key: haSharedSecret + name: flyte-sandbox-docker-registry-secret + - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY + value: /var/lib/registry + image: registry:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: / + port: 5000 + name: docker-registry + ports: + - containerPort: 5000 + readinessProbe: + httpGet: + path: / + port: 5000 + resources: {} + volumeMounts: + - mountPath: /etc/docker/registry + name: flyte-sandbox-docker-registry-config + - mountPath: /var/lib/registry/ + name: data securityContext: fsGroup: 1000 runAsUser: 1000 volumes: - - configMap: + - configMap: + name: flyte-sandbox-docker-registry-config name: flyte-sandbox-docker-registry-config - name: flyte-sandbox-docker-registry-config - - emptyDir: {} - name: data + - emptyDir: {} + name: data --- apiVersion: apps/v1 kind: Deployment @@ -1499,52 +1499,52 @@ spec: helm.sh/chart: kubernetes-dashboard-6.0.0 spec: containers: - - args: - - --namespace=flyte - - --metrics-provider=none - - --enable-insecure-login - - --enable-skip-login - image: kubernetesui/dashboard:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: / - port: 9090 - scheme: HTTP - initialDelaySeconds: 30 - timeoutSeconds: 30 - name: kubernetes-dashboard - ports: - - containerPort: 9090 - name: http - protocol: TCP - resources: - limits: - cpu: 2 - memory: 200Mi - requests: - cpu: 100m - memory: 200Mi - securityContext: - allowPrivilegeEscalation: false - readOnlyRootFilesystem: true - runAsGroup: 2001 - runAsUser: 1001 - volumeMounts: - - mountPath: /certs - name: kubernetes-dashboard-certs - - mountPath: /tmp - name: tmp-volume + - args: + - --namespace=flyte + - --metrics-provider=none + - --enable-insecure-login + - --enable-skip-login + image: kubernetesui/dashboard:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: / + port: 9090 + scheme: HTTP + initialDelaySeconds: 30 + timeoutSeconds: 30 + name: kubernetes-dashboard + ports: + - containerPort: 9090 + name: http + protocol: TCP + resources: + limits: + cpu: 2 + memory: 200Mi + requests: + cpu: 100m + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsGroup: 2001 + runAsUser: 1001 + volumeMounts: + - mountPath: /certs + name: kubernetes-dashboard-certs + - mountPath: /tmp + name: tmp-volume securityContext: seccompProfile: type: RuntimeDefault serviceAccountName: flyte-sandbox-kubernetes-dashboard volumes: - - name: kubernetes-dashboard-certs - secret: - secretName: flyte-sandbox-kubernetes-dashboard-certs - - emptyDir: {} - name: tmp-volume + - name: kubernetes-dashboard-certs + secret: + secretName: flyte-sandbox-kubernetes-dashboard-certs + - emptyDir: {} + name: tmp-volume --- apiVersion: apps/v1 kind: Deployment @@ -1578,103 +1578,103 @@ spec: podAffinity: null podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app.kubernetes.io/instance: flyte-sandbox - app.kubernetes.io/name: minio - topologyKey: kubernetes.io/hostname - weight: 1 + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: flyte-sandbox + app.kubernetes.io/name: minio + topologyKey: kubernetes.io/hostname + weight: 1 containers: - - env: - - name: BITNAMI_DEBUG - value: "false" - - name: MINIO_SCHEME - value: http - - name: MINIO_FORCE_NEW_KEYS - value: "no" - - name: MINIO_ROOT_USER - valueFrom: - secretKeyRef: - key: root-user - name: flyte-sandbox-minio - - name: MINIO_ROOT_PASSWORD - valueFrom: - secretKeyRef: - key: root-password - name: flyte-sandbox-minio - - name: MINIO_DEFAULT_BUCKETS - value: my-s3-bucket - - name: MINIO_BROWSER - value: "on" - - name: MINIO_PROMETHEUS_AUTH_TYPE - value: public - - name: MINIO_CONSOLE_PORT_NUMBER - value: "9001" - - name: MINIO_BROWSER_REDIRECT_URL - value: http://localhost:30080/minio - envFrom: null - image: docker.io/bitnami/minio:sandbox - imagePullPolicy: Never - livenessProbe: - failureThreshold: 5 - httpGet: - path: /minio/health/live - port: minio-api - scheme: HTTP - initialDelaySeconds: 5 - periodSeconds: 5 - successThreshold: 1 - timeoutSeconds: 5 - name: minio - ports: - - containerPort: 9000 - name: minio-api - protocol: TCP - - containerPort: 9001 - name: minio-console - protocol: TCP - readinessProbe: - failureThreshold: 5 - initialDelaySeconds: 5 - periodSeconds: 5 - successThreshold: 1 - tcpSocket: - port: minio-api - timeoutSeconds: 1 - resources: - limits: {} - requests: {} - securityContext: - runAsNonRoot: true - runAsUser: 1001 - volumeMounts: - - mountPath: /data - name: data + - env: + - name: BITNAMI_DEBUG + value: "false" + - name: MINIO_SCHEME + value: http + - name: MINIO_FORCE_NEW_KEYS + value: "no" + - name: MINIO_ROOT_USER + valueFrom: + secretKeyRef: + key: root-user + name: flyte-sandbox-minio + - name: MINIO_ROOT_PASSWORD + valueFrom: + secretKeyRef: + key: root-password + name: flyte-sandbox-minio + - name: MINIO_DEFAULT_BUCKETS + value: my-s3-bucket + - name: MINIO_BROWSER + value: "on" + - name: MINIO_PROMETHEUS_AUTH_TYPE + value: public + - name: MINIO_CONSOLE_PORT_NUMBER + value: "9001" + - name: MINIO_BROWSER_REDIRECT_URL + value: http://localhost:30080/minio + envFrom: null + image: docker.io/bitnami/minio:sandbox + imagePullPolicy: Never + livenessProbe: + failureThreshold: 5 + httpGet: + path: /minio/health/live + port: minio-api + scheme: HTTP + initialDelaySeconds: 5 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 5 + name: minio + ports: + - containerPort: 9000 + name: minio-api + protocol: TCP + - containerPort: 9001 + name: minio-console + protocol: TCP + readinessProbe: + failureThreshold: 5 + initialDelaySeconds: 5 + periodSeconds: 5 + successThreshold: 1 + tcpSocket: + port: minio-api + timeoutSeconds: 1 + resources: + limits: {} + requests: {} + securityContext: + runAsNonRoot: true + runAsUser: 1001 + volumeMounts: + - mountPath: /data + name: data initContainers: - - command: - - /bin/bash - - -ec - - | - chown -R 1001:1001 /data - image: docker.io/bitnami/os-shell:sandbox - imagePullPolicy: Never - name: volume-permissions - resources: - limits: {} - requests: {} - securityContext: - runAsUser: 0 - volumeMounts: - - mountPath: /data - name: data + - command: + - /bin/bash + - -ec + - | + chown -R 1001:1001 /data + image: docker.io/bitnami/os-shell:sandbox + imagePullPolicy: Never + name: volume-permissions + resources: + limits: {} + requests: {} + securityContext: + runAsUser: 0 + volumeMounts: + - mountPath: /data + name: data securityContext: fsGroup: 1001 serviceAccountName: flyte-sandbox-minio volumes: - - name: data - persistentVolumeClaim: - claimName: flyte-sandbox-minio-storage + - name: data + persistentVolumeClaim: + claimName: flyte-sandbox-minio-storage --- apiVersion: apps/v1 kind: Deployment @@ -1702,26 +1702,26 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - image: envoyproxy/envoy:sandbox - imagePullPolicy: Never - livenessProbe: - initialDelaySeconds: 30 - tcpSocket: - port: http - name: proxy - ports: - - containerPort: 8000 - name: http - readinessProbe: - tcpSocket: - port: http - volumeMounts: - - mountPath: /etc/envoy - name: config + - image: envoyproxy/envoy:sandbox + imagePullPolicy: Never + livenessProbe: + initialDelaySeconds: 30 + tcpSocket: + port: http + name: proxy + ports: + - containerPort: 8000 + name: http + readinessProbe: + tcpSocket: + port: http + volumeMounts: + - mountPath: /etc/envoy + name: config volumes: - - configMap: - name: flyte-sandbox-proxy-config - name: config + - configMap: + name: flyte-sandbox-proxy-config + name: config --- apiVersion: apps/v1 kind: Deployment @@ -1748,48 +1748,48 @@ spec: helm.sh/chart: flyteagent-v0.1.10 spec: containers: - - command: - - pyflyte - - serve - - agent - env: - - name: FLYTE_AWS_ENDPOINT - value: http://flyte-sandbox-minio.flyte:9000 - - name: FLYTE_AWS_ACCESS_KEY_ID - value: minio - - name: FLYTE_AWS_SECRET_ACCESS_KEY - value: miniostorage - image: cr.flyte.org/flyteorg/flyteagent:1.13.6 - imagePullPolicy: IfNotPresent - name: flyteagent - ports: - - containerPort: 8000 - name: agent-grpc - readinessProbe: - grpc: - port: 8000 - initialDelaySeconds: 1 - periodSeconds: 3 - resources: - limits: - cpu: 500m - ephemeral-storage: 200Mi - memory: 300Mi - requests: - cpu: 500m - ephemeral-storage: 200Mi - memory: 200Mi - securityContext: - allowPrivilegeEscalation: false - volumeMounts: - - mountPath: /etc/secrets + - command: + - pyflyte + - serve + - agent + env: + - name: FLYTE_AWS_ENDPOINT + value: http://flyte-sandbox-minio.flyte:9000 + - name: FLYTE_AWS_ACCESS_KEY_ID + value: minio + - name: FLYTE_AWS_SECRET_ACCESS_KEY + value: miniostorage + image: cr.flyte.org/flyteorg/flyteagent:1.13.6 + imagePullPolicy: IfNotPresent name: flyteagent + ports: + - containerPort: 8000 + name: agent-grpc + readinessProbe: + grpc: + port: 8000 + initialDelaySeconds: 1 + periodSeconds: 3 + resources: + limits: + cpu: 500m + ephemeral-storage: 200Mi + memory: 300Mi + requests: + cpu: 500m + ephemeral-storage: 200Mi + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + volumeMounts: + - mountPath: /etc/secrets + name: flyteagent securityContext: {} serviceAccountName: flyteagent volumes: - - name: flyteagent - secret: - secretName: flyteagent + - name: flyteagent + secret: + secretName: flyteagent --- apiVersion: apps/v1 kind: StatefulSet @@ -1825,128 +1825,128 @@ spec: podAffinity: null podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app.kubernetes.io/component: primary - app.kubernetes.io/instance: flyte-sandbox - app.kubernetes.io/name: postgresql - topologyKey: kubernetes.io/hostname - weight: 1 + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/component: primary + app.kubernetes.io/instance: flyte-sandbox + app.kubernetes.io/name: postgresql + topologyKey: kubernetes.io/hostname + weight: 1 containers: - - env: - - name: BITNAMI_DEBUG - value: "false" - - name: POSTGRESQL_PORT_NUMBER - value: "5432" - - name: POSTGRESQL_VOLUME_DIR - value: /bitnami/postgresql - - name: PGDATA - value: /bitnami/postgresql/data - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - key: postgres-password - name: flyte-sandbox-postgresql - - name: POSTGRESQL_ENABLE_LDAP - value: "no" - - name: POSTGRESQL_ENABLE_TLS - value: "no" - - name: POSTGRESQL_LOG_HOSTNAME - value: "false" - - name: POSTGRESQL_LOG_CONNECTIONS - value: "false" - - name: POSTGRESQL_LOG_DISCONNECTIONS - value: "false" - - name: POSTGRESQL_PGAUDIT_LOG_CATALOG - value: "off" - - name: POSTGRESQL_CLIENT_MIN_MESSAGES - value: error - - name: POSTGRESQL_SHARED_PRELOAD_LIBRARIES - value: pgaudit - image: docker.io/bitnami/postgresql:sandbox - imagePullPolicy: Never - livenessProbe: - exec: - command: - - /bin/sh - - -c - - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 - failureThreshold: 6 - initialDelaySeconds: 30 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - name: postgresql - ports: - - containerPort: 5432 - name: tcp-postgresql - readinessProbe: - exec: - command: - - /bin/sh - - -c - - -e - - | - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 - [ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f /bitnami/postgresql/.initialized ] - failureThreshold: 6 - initialDelaySeconds: 5 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - resources: - limits: {} - requests: - cpu: 250m - memory: 256Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - runAsGroup: 0 - runAsNonRoot: true - runAsUser: 1001 - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /bitnami/postgresql - name: data + - env: + - name: BITNAMI_DEBUG + value: "false" + - name: POSTGRESQL_PORT_NUMBER + value: "5432" + - name: POSTGRESQL_VOLUME_DIR + value: /bitnami/postgresql + - name: PGDATA + value: /bitnami/postgresql/data + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + key: postgres-password + name: flyte-sandbox-postgresql + - name: POSTGRESQL_ENABLE_LDAP + value: "no" + - name: POSTGRESQL_ENABLE_TLS + value: "no" + - name: POSTGRESQL_LOG_HOSTNAME + value: "false" + - name: POSTGRESQL_LOG_CONNECTIONS + value: "false" + - name: POSTGRESQL_LOG_DISCONNECTIONS + value: "false" + - name: POSTGRESQL_PGAUDIT_LOG_CATALOG + value: "off" + - name: POSTGRESQL_CLIENT_MIN_MESSAGES + value: error + - name: POSTGRESQL_SHARED_PRELOAD_LIBRARIES + value: pgaudit + image: docker.io/bitnami/postgresql:sandbox + imagePullPolicy: Never + livenessProbe: + exec: + command: + - /bin/sh + - -c + - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 + failureThreshold: 6 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + name: postgresql + ports: + - containerPort: 5432 + name: tcp-postgresql + readinessProbe: + exec: + command: + - /bin/sh + - -c + - -e + - | + exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 + [ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f /bitnami/postgresql/.initialized ] + failureThreshold: 6 + initialDelaySeconds: 5 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + resources: + limits: {} + requests: + cpu: 250m + memory: 256Mi + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + runAsGroup: 0 + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + volumeMounts: + - mountPath: /bitnami/postgresql + name: data hostIPC: false hostNetwork: false initContainers: - - command: - - /bin/sh - - -ec - - | - chown 1001:1001 /bitnami/postgresql - mkdir -p /bitnami/postgresql/data - chmod 700 /bitnami/postgresql/data - find /bitnami/postgresql -mindepth 1 -maxdepth 1 -not -name "conf" -not -name ".snapshot" -not -name "lost+found" | \ - xargs -r chown -R 1001:1001 - image: docker.io/bitnami/os-shell:sandbox - imagePullPolicy: Never - name: init-chmod-data - resources: - limits: {} - requests: {} - securityContext: - runAsGroup: 0 - runAsNonRoot: false - runAsUser: 0 - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /bitnami/postgresql - name: data + - command: + - /bin/sh + - -ec + - | + chown 1001:1001 /bitnami/postgresql + mkdir -p /bitnami/postgresql/data + chmod 700 /bitnami/postgresql/data + find /bitnami/postgresql -mindepth 1 -maxdepth 1 -not -name "conf" -not -name ".snapshot" -not -name "lost+found" | \ + xargs -r chown -R 1001:1001 + image: docker.io/bitnami/os-shell:sandbox + imagePullPolicy: Never + name: init-chmod-data + resources: + limits: {} + requests: {} + securityContext: + runAsGroup: 0 + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + volumeMounts: + - mountPath: /bitnami/postgresql + name: data securityContext: fsGroup: 1001 serviceAccountName: default volumes: - - name: data - persistentVolumeClaim: - claimName: flyte-sandbox-db-storage + - name: data + persistentVolumeClaim: + claimName: flyte-sandbox-db-storage updateStrategy: rollingUpdate: {} type: RollingUpdate diff --git a/docker/sandbox-bundled/manifests/complete.yaml b/docker/sandbox-bundled/manifests/complete.yaml index c9464636af..1703f5523d 100644 --- a/docker/sandbox-bundled/manifests/complete.yaml +++ b/docker/sandbox-bundled/manifests/complete.yaml @@ -39,7 +39,7 @@ metadata: name: flyte-sandbox-minio namespace: flyte secrets: -- name: flyte-sandbox-minio + - name: flyte-sandbox-minio --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role @@ -53,48 +53,48 @@ metadata: name: flyte-sandbox-kubernetes-dashboard namespace: flyte rules: -- apiGroups: - - "" - resourceNames: - - kubernetes-dashboard-key-holder - - kubernetes-dashboard-certs - - kubernetes-dashboard-csrf - resources: - - secrets - verbs: - - get - - update - - delete -- apiGroups: - - "" - resourceNames: - - kubernetes-dashboard-settings - resources: - - configmaps - verbs: - - get - - update -- apiGroups: - - "" - resourceNames: - - heapster - - dashboard-metrics-scraper - resources: - - services - verbs: - - proxy -- apiGroups: - - "" - resourceNames: - - heapster - - 'http:heapster:' - - 'https:heapster:' - - dashboard-metrics-scraper - - http:dashboard-metrics-scraper - resources: - - services/proxy - verbs: - - get + - apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-key-holder + - kubernetes-dashboard-certs + - kubernetes-dashboard-csrf + resources: + - secrets + verbs: + - get + - update + - delete + - apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-settings + resources: + - configmaps + verbs: + - get + - update + - apiGroups: + - "" + resourceNames: + - heapster + - dashboard-metrics-scraper + resources: + - services + verbs: + - proxy + - apiGroups: + - "" + resourceNames: + - heapster + - "http:heapster:" + - "https:heapster:" + - dashboard-metrics-scraper + - http:dashboard-metrics-scraper + resources: + - services/proxy + verbs: + - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -108,86 +108,86 @@ metadata: name: flyte-sandbox-cluster-role namespace: flyte rules: -- apiGroups: - - "" - resources: - - namespaces - - resourcequotas - - secrets - - serviceaccounts - verbs: - - create - - get - - list - - patch - - update -- apiGroups: - - "" - resources: - - pods - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - delete - - patch - - update -- apiGroups: - - "" - resources: - - podtemplates - verbs: - - get - - list - - watch -- apiGroups: - - flyte.lyft.com - resources: - - flyteworkflows - verbs: - - create - - delete - - deletecollection - - get - - list - - patch - - post - - update - - watch -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - create - - get - - list -- apiGroups: - - admissionregistration.k8s.io - resources: - - mutatingwebhookconfigurations - verbs: - - create - - get - - list - - patch - - update -- apiGroups: - - '*' - resources: - - '*' - verbs: - - '*' + - apiGroups: + - "" + resources: + - namespaces + - resourcequotas + - secrets + - serviceaccounts + verbs: + - create + - get + - list + - patch + - update + - apiGroups: + - "" + resources: + - pods + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - create + - delete + - patch + - update + - apiGroups: + - "" + resources: + - podtemplates + verbs: + - get + - list + - watch + - apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + verbs: + - create + - delete + - deletecollection + - get + - list + - patch + - post + - update + - watch + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - list + - apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + verbs: + - create + - get + - list + - patch + - update + - apiGroups: + - "*" + resources: + - "*" + verbs: + - "*" --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -200,130 +200,130 @@ metadata: helm.sh/chart: kubernetes-dashboard-6.0.0 name: flyte-sandbox-kubernetes-dashboard-readonly rules: -- apiGroups: - - "" - resources: - - configmaps - - endpoints - - persistentvolumeclaims - - pods - - replicationcontrollers - - replicationcontrollers/scale - - serviceaccounts - - services - - nodes - - persistentvolumeclaims - - persistentvolumes - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - bindings - - events - - limitranges - - namespaces/status - - pods/log - - pods/status - - replicationcontrollers/status - - resourcequotas - - resourcequotas/status - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - namespaces - verbs: - - get - - list - - watch -- apiGroups: - - apps - resources: - - daemonsets - - deployments - - deployments/scale - - replicasets - - replicasets/scale - - statefulsets - verbs: - - get - - list - - watch -- apiGroups: - - autoscaling - resources: - - horizontalpodautoscalers - verbs: - - get - - list - - watch -- apiGroups: - - batch - resources: - - cronjobs - - jobs - verbs: - - get - - list - - watch -- apiGroups: - - extensions - resources: - - daemonsets - - deployments - - deployments/scale - - ingresses - - networkpolicies - - replicasets - - replicasets/scale - - replicationcontrollers/scale - verbs: - - get - - list - - watch -- apiGroups: - - policy - resources: - - poddisruptionbudgets - verbs: - - get - - list - - watch -- apiGroups: - - networking.k8s.io - resources: - - networkpolicies - - ingresses - verbs: - - get - - list - - watch -- apiGroups: - - storage.k8s.io - resources: - - storageclasses - - volumeattachments - verbs: - - get - - list - - watch -- apiGroups: - - rbac.authorization.k8s.io - resources: - - clusterrolebindings - - clusterroles - - roles - - rolebindings - verbs: - - get - - list - - watch + - apiGroups: + - "" + resources: + - configmaps + - endpoints + - persistentvolumeclaims + - pods + - replicationcontrollers + - replicationcontrollers/scale + - serviceaccounts + - services + - nodes + - persistentvolumeclaims + - persistentvolumes + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - bindings + - events + - limitranges + - namespaces/status + - pods/log + - pods/status + - replicationcontrollers/status + - resourcequotas + - resourcequotas/status + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - namespaces + verbs: + - get + - list + - watch + - apiGroups: + - apps + resources: + - daemonsets + - deployments + - deployments/scale + - replicasets + - replicasets/scale + - statefulsets + verbs: + - get + - list + - watch + - apiGroups: + - autoscaling + resources: + - horizontalpodautoscalers + verbs: + - get + - list + - watch + - apiGroups: + - batch + resources: + - cronjobs + - jobs + verbs: + - get + - list + - watch + - apiGroups: + - extensions + resources: + - daemonsets + - deployments + - deployments/scale + - ingresses + - networkpolicies + - replicasets + - replicasets/scale + - replicationcontrollers/scale + verbs: + - get + - list + - watch + - apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - networkpolicies + - ingresses + verbs: + - get + - list + - watch + - apiGroups: + - storage.k8s.io + resources: + - storageclasses + - volumeattachments + verbs: + - get + - list + - watch + - apiGroups: + - rbac.authorization.k8s.io + resources: + - clusterrolebindings + - clusterroles + - roles + - rolebindings + verbs: + - get + - list + - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding @@ -341,9 +341,9 @@ roleRef: kind: Role name: flyte-sandbox-kubernetes-dashboard subjects: -- kind: ServiceAccount - name: flyte-sandbox-kubernetes-dashboard - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox-kubernetes-dashboard + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -361,9 +361,9 @@ roleRef: kind: ClusterRole name: flyte-sandbox-cluster-role subjects: -- kind: ServiceAccount - name: flyte-sandbox - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox + namespace: flyte --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -380,9 +380,9 @@ roleRef: kind: ClusterRole name: flyte-sandbox-kubernetes-dashboard-readonly subjects: -- kind: ServiceAccount - name: flyte-sandbox-kubernetes-dashboard - namespace: flyte + - kind: ServiceAccount + name: flyte-sandbox-kubernetes-dashboard + namespace: flyte --- apiVersion: v1 data: @@ -447,10 +447,10 @@ data: default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - echo @@ -895,11 +895,11 @@ metadata: namespace: flyte spec: ports: - - name: http-5000 - nodePort: 30000 - port: 5000 - protocol: TCP - targetPort: 5000 + - name: http-5000 + nodePort: 30000 + port: 5000 + protocol: TCP + targetPort: 5000 selector: app: docker-registry release: flyte-sandbox @@ -918,10 +918,10 @@ metadata: namespace: flyte spec: ports: - - name: grpc - nodePort: null - port: 8089 - targetPort: grpc + - name: grpc + nodePort: null + port: 8089 + targetPort: grpc selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -941,10 +941,10 @@ metadata: namespace: flyte spec: ports: - - name: http - nodePort: null - port: 8088 - targetPort: http + - name: http + nodePort: null + port: 8088 + targetPort: http selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -966,9 +966,9 @@ metadata: namespace: flyte spec: ports: - - name: http - port: 80 - targetPort: http + - name: http + port: 80 + targetPort: http selector: app.kubernetes.io/component: kubernetes-dashboard app.kubernetes.io/instance: flyte-sandbox @@ -988,13 +988,13 @@ metadata: spec: externalTrafficPolicy: Cluster ports: - - name: minio-api - nodePort: 30002 - port: 9000 - targetPort: minio-api - - name: minio-console - port: 9001 - targetPort: minio-console + - name: minio-api + nodePort: 30002 + port: 9000 + targetPort: minio-api + - name: minio-console + port: 9001 + targetPort: minio-console selector: app.kubernetes.io/instance: flyte-sandbox app.kubernetes.io/name: minio @@ -1014,10 +1014,10 @@ metadata: spec: externalTrafficPolicy: Cluster ports: - - name: tcp-postgresql - nodePort: 30001 - port: 5432 - targetPort: tcp-postgresql + - name: tcp-postgresql + nodePort: 30001 + port: 5432 + targetPort: tcp-postgresql selector: app.kubernetes.io/component: primary app.kubernetes.io/instance: flyte-sandbox @@ -1040,9 +1040,9 @@ metadata: spec: clusterIP: None ports: - - name: tcp-postgresql - port: 5432 - targetPort: tcp-postgresql + - name: tcp-postgresql + port: 5432 + targetPort: tcp-postgresql publishNotReadyAddresses: true selector: app.kubernetes.io/component: primary @@ -1063,10 +1063,10 @@ metadata: namespace: flyte spec: ports: - - name: http - nodePort: 30080 - port: 8000 - protocol: TCP + - name: http + nodePort: 30080 + port: 8000 + protocol: TCP selector: app.kubernetes.io/component: proxy app.kubernetes.io/instance: flyte-sandbox @@ -1086,9 +1086,9 @@ metadata: namespace: flyte spec: ports: - - name: webhook - port: 443 - targetPort: webhook + - name: webhook + port: 443 + targetPort: webhook selector: app.kubernetes.io/component: flyte-binary app.kubernetes.io/instance: flyte-sandbox @@ -1108,7 +1108,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce capacity: storage: 1Gi hostPath: @@ -1128,7 +1128,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce capacity: storage: 1Gi hostPath: @@ -1148,7 +1148,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce resources: requests: storage: 1Gi @@ -1168,7 +1168,7 @@ metadata: namespace: flyte spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce resources: requests: storage: 1Gi @@ -1207,83 +1207,83 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - args: - - start - - --config - - /etc/flyte/config.d/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - image: flyte-binary:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: /healthcheck - port: http - initialDelaySeconds: 30 - name: flyte - ports: - - containerPort: 8088 - name: http - - containerPort: 8089 - name: grpc - - containerPort: 9443 - name: webhook - readinessProbe: - httpGet: - path: /healthcheck - port: http - initialDelaySeconds: 30 - volumeMounts: - - mountPath: /etc/flyte/cluster-resource-templates - name: cluster-resource-templates - - mountPath: /etc/flyte/config.d - name: config - - mountPath: /var/run/flyte - name: state + - args: + - start + - --config + - /etc/flyte/config.d/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: flyte-binary:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: /healthcheck + port: http + initialDelaySeconds: 30 + name: flyte + ports: + - containerPort: 8088 + name: http + - containerPort: 8089 + name: grpc + - containerPort: 9443 + name: webhook + readinessProbe: + httpGet: + path: /healthcheck + port: http + initialDelaySeconds: 30 + volumeMounts: + - mountPath: /etc/flyte/cluster-resource-templates + name: cluster-resource-templates + - mountPath: /etc/flyte/config.d + name: config + - mountPath: /var/run/flyte + name: state initContainers: - - args: - - | - until pg_isready \ - -h flyte-sandbox-postgresql \ - -p 5432 \ - -U postgres - do - echo waiting for database - sleep 0.1 - done - command: - - sh - - -ec - image: bitnami/postgresql:sandbox - imagePullPolicy: Never - name: wait-for-db + - args: + - | + until pg_isready \ + -h flyte-sandbox-postgresql \ + -p 5432 \ + -U postgres + do + echo waiting for database + sleep 0.1 + done + command: + - sh + - -ec + image: bitnami/postgresql:sandbox + imagePullPolicy: Never + name: wait-for-db serviceAccountName: flyte-sandbox volumes: - - name: cluster-resource-templates - projected: - sources: - - configMap: - name: flyte-sandbox-cluster-resource-templates - - configMap: - name: flyte-sandbox-extra-cluster-resource-templates - - name: config - projected: - sources: - - configMap: - name: flyte-sandbox-config - - secret: - name: flyte-sandbox-config-secret - - configMap: - name: flyte-sandbox-extra-config - - emptyDir: {} - name: state + - name: cluster-resource-templates + projected: + sources: + - configMap: + name: flyte-sandbox-cluster-resource-templates + - configMap: + name: flyte-sandbox-extra-cluster-resource-templates + - name: config + projected: + sources: + - configMap: + name: flyte-sandbox-config + - secret: + name: flyte-sandbox-config-secret + - configMap: + name: flyte-sandbox-extra-config + - emptyDir: {} + name: state --- apiVersion: apps/v1 kind: Deployment @@ -1311,36 +1311,36 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - args: - - --addr - - unix:///run/buildkit/buildkitd.sock - - --addr - - tcp://0.0.0.0:30003 - image: moby/buildkit:sandbox - imagePullPolicy: Never - livenessProbe: - exec: - command: - - buildctl - - debug - - workers - initialDelaySeconds: 5 - periodSeconds: 30 - name: buildkit - ports: - - containerPort: 30003 - name: tcp - protocol: TCP - readinessProbe: - exec: - command: - - buildctl - - debug - - workers - initialDelaySeconds: 5 - periodSeconds: 30 - securityContext: - privileged: true + - args: + - --addr + - unix:///run/buildkit/buildkitd.sock + - --addr + - tcp://0.0.0.0:30003 + image: moby/buildkit:sandbox + imagePullPolicy: Never + livenessProbe: + exec: + command: + - buildctl + - debug + - workers + initialDelaySeconds: 5 + periodSeconds: 30 + name: buildkit + ports: + - containerPort: 30003 + name: tcp + protocol: TCP + readinessProbe: + exec: + command: + - buildctl + - debug + - workers + initialDelaySeconds: 5 + periodSeconds: 30 + securityContext: + privileged: true dnsPolicy: ClusterFirstWithHostNet hostNetwork: true --- @@ -1371,46 +1371,46 @@ spec: release: flyte-sandbox spec: containers: - - command: - - /bin/registry - - serve - - /etc/docker/registry/config.yml - env: - - name: REGISTRY_HTTP_SECRET - valueFrom: - secretKeyRef: - key: haSharedSecret - name: flyte-sandbox-docker-registry-secret - - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY - value: /var/lib/registry - image: registry:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: / - port: 5000 - name: docker-registry - ports: - - containerPort: 5000 - readinessProbe: - httpGet: - path: / - port: 5000 - resources: {} - volumeMounts: - - mountPath: /etc/docker/registry - name: flyte-sandbox-docker-registry-config - - mountPath: /var/lib/registry/ - name: data + - command: + - /bin/registry + - serve + - /etc/docker/registry/config.yml + env: + - name: REGISTRY_HTTP_SECRET + valueFrom: + secretKeyRef: + key: haSharedSecret + name: flyte-sandbox-docker-registry-secret + - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY + value: /var/lib/registry + image: registry:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: / + port: 5000 + name: docker-registry + ports: + - containerPort: 5000 + readinessProbe: + httpGet: + path: / + port: 5000 + resources: {} + volumeMounts: + - mountPath: /etc/docker/registry + name: flyte-sandbox-docker-registry-config + - mountPath: /var/lib/registry/ + name: data securityContext: fsGroup: 1000 runAsUser: 1000 volumes: - - configMap: + - configMap: + name: flyte-sandbox-docker-registry-config name: flyte-sandbox-docker-registry-config - name: flyte-sandbox-docker-registry-config - - emptyDir: {} - name: data + - emptyDir: {} + name: data --- apiVersion: apps/v1 kind: Deployment @@ -1448,52 +1448,52 @@ spec: helm.sh/chart: kubernetes-dashboard-6.0.0 spec: containers: - - args: - - --namespace=flyte - - --metrics-provider=none - - --enable-insecure-login - - --enable-skip-login - image: kubernetesui/dashboard:sandbox - imagePullPolicy: Never - livenessProbe: - httpGet: - path: / - port: 9090 - scheme: HTTP - initialDelaySeconds: 30 - timeoutSeconds: 30 - name: kubernetes-dashboard - ports: - - containerPort: 9090 - name: http - protocol: TCP - resources: - limits: - cpu: 2 - memory: 200Mi - requests: - cpu: 100m - memory: 200Mi - securityContext: - allowPrivilegeEscalation: false - readOnlyRootFilesystem: true - runAsGroup: 2001 - runAsUser: 1001 - volumeMounts: - - mountPath: /certs - name: kubernetes-dashboard-certs - - mountPath: /tmp - name: tmp-volume + - args: + - --namespace=flyte + - --metrics-provider=none + - --enable-insecure-login + - --enable-skip-login + image: kubernetesui/dashboard:sandbox + imagePullPolicy: Never + livenessProbe: + httpGet: + path: / + port: 9090 + scheme: HTTP + initialDelaySeconds: 30 + timeoutSeconds: 30 + name: kubernetes-dashboard + ports: + - containerPort: 9090 + name: http + protocol: TCP + resources: + limits: + cpu: 2 + memory: 200Mi + requests: + cpu: 100m + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsGroup: 2001 + runAsUser: 1001 + volumeMounts: + - mountPath: /certs + name: kubernetes-dashboard-certs + - mountPath: /tmp + name: tmp-volume securityContext: seccompProfile: type: RuntimeDefault serviceAccountName: flyte-sandbox-kubernetes-dashboard volumes: - - name: kubernetes-dashboard-certs - secret: - secretName: flyte-sandbox-kubernetes-dashboard-certs - - emptyDir: {} - name: tmp-volume + - name: kubernetes-dashboard-certs + secret: + secretName: flyte-sandbox-kubernetes-dashboard-certs + - emptyDir: {} + name: tmp-volume --- apiVersion: apps/v1 kind: Deployment @@ -1527,103 +1527,103 @@ spec: podAffinity: null podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app.kubernetes.io/instance: flyte-sandbox - app.kubernetes.io/name: minio - topologyKey: kubernetes.io/hostname - weight: 1 + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: flyte-sandbox + app.kubernetes.io/name: minio + topologyKey: kubernetes.io/hostname + weight: 1 containers: - - env: - - name: BITNAMI_DEBUG - value: "false" - - name: MINIO_SCHEME - value: http - - name: MINIO_FORCE_NEW_KEYS - value: "no" - - name: MINIO_ROOT_USER - valueFrom: - secretKeyRef: - key: root-user - name: flyte-sandbox-minio - - name: MINIO_ROOT_PASSWORD - valueFrom: - secretKeyRef: - key: root-password - name: flyte-sandbox-minio - - name: MINIO_DEFAULT_BUCKETS - value: my-s3-bucket - - name: MINIO_BROWSER - value: "on" - - name: MINIO_PROMETHEUS_AUTH_TYPE - value: public - - name: MINIO_CONSOLE_PORT_NUMBER - value: "9001" - - name: MINIO_BROWSER_REDIRECT_URL - value: http://localhost:30080/minio - envFrom: null - image: docker.io/bitnami/minio:sandbox - imagePullPolicy: Never - livenessProbe: - failureThreshold: 5 - httpGet: - path: /minio/health/live - port: minio-api - scheme: HTTP - initialDelaySeconds: 5 - periodSeconds: 5 - successThreshold: 1 - timeoutSeconds: 5 - name: minio - ports: - - containerPort: 9000 - name: minio-api - protocol: TCP - - containerPort: 9001 - name: minio-console - protocol: TCP - readinessProbe: - failureThreshold: 5 - initialDelaySeconds: 5 - periodSeconds: 5 - successThreshold: 1 - tcpSocket: - port: minio-api - timeoutSeconds: 1 - resources: - limits: {} - requests: {} - securityContext: - runAsNonRoot: true - runAsUser: 1001 - volumeMounts: - - mountPath: /data - name: data + - env: + - name: BITNAMI_DEBUG + value: "false" + - name: MINIO_SCHEME + value: http + - name: MINIO_FORCE_NEW_KEYS + value: "no" + - name: MINIO_ROOT_USER + valueFrom: + secretKeyRef: + key: root-user + name: flyte-sandbox-minio + - name: MINIO_ROOT_PASSWORD + valueFrom: + secretKeyRef: + key: root-password + name: flyte-sandbox-minio + - name: MINIO_DEFAULT_BUCKETS + value: my-s3-bucket + - name: MINIO_BROWSER + value: "on" + - name: MINIO_PROMETHEUS_AUTH_TYPE + value: public + - name: MINIO_CONSOLE_PORT_NUMBER + value: "9001" + - name: MINIO_BROWSER_REDIRECT_URL + value: http://localhost:30080/minio + envFrom: null + image: docker.io/bitnami/minio:sandbox + imagePullPolicy: Never + livenessProbe: + failureThreshold: 5 + httpGet: + path: /minio/health/live + port: minio-api + scheme: HTTP + initialDelaySeconds: 5 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 5 + name: minio + ports: + - containerPort: 9000 + name: minio-api + protocol: TCP + - containerPort: 9001 + name: minio-console + protocol: TCP + readinessProbe: + failureThreshold: 5 + initialDelaySeconds: 5 + periodSeconds: 5 + successThreshold: 1 + tcpSocket: + port: minio-api + timeoutSeconds: 1 + resources: + limits: {} + requests: {} + securityContext: + runAsNonRoot: true + runAsUser: 1001 + volumeMounts: + - mountPath: /data + name: data initContainers: - - command: - - /bin/bash - - -ec - - | - chown -R 1001:1001 /data - image: docker.io/bitnami/os-shell:sandbox - imagePullPolicy: Never - name: volume-permissions - resources: - limits: {} - requests: {} - securityContext: - runAsUser: 0 - volumeMounts: - - mountPath: /data - name: data + - command: + - /bin/bash + - -ec + - | + chown -R 1001:1001 /data + image: docker.io/bitnami/os-shell:sandbox + imagePullPolicy: Never + name: volume-permissions + resources: + limits: {} + requests: {} + securityContext: + runAsUser: 0 + volumeMounts: + - mountPath: /data + name: data securityContext: fsGroup: 1001 serviceAccountName: flyte-sandbox-minio volumes: - - name: data - persistentVolumeClaim: - claimName: flyte-sandbox-minio-storage + - name: data + persistentVolumeClaim: + claimName: flyte-sandbox-minio-storage --- apiVersion: apps/v1 kind: Deployment @@ -1651,26 +1651,26 @@ spec: app.kubernetes.io/name: flyte-sandbox spec: containers: - - image: envoyproxy/envoy:sandbox - imagePullPolicy: Never - livenessProbe: - initialDelaySeconds: 30 - tcpSocket: - port: http - name: proxy - ports: - - containerPort: 8000 - name: http - readinessProbe: - tcpSocket: - port: http - volumeMounts: - - mountPath: /etc/envoy - name: config + - image: envoyproxy/envoy:sandbox + imagePullPolicy: Never + livenessProbe: + initialDelaySeconds: 30 + tcpSocket: + port: http + name: proxy + ports: + - containerPort: 8000 + name: http + readinessProbe: + tcpSocket: + port: http + volumeMounts: + - mountPath: /etc/envoy + name: config volumes: - - configMap: - name: flyte-sandbox-proxy-config - name: config + - configMap: + name: flyte-sandbox-proxy-config + name: config --- apiVersion: apps/v1 kind: StatefulSet @@ -1706,128 +1706,128 @@ spec: podAffinity: null podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app.kubernetes.io/component: primary - app.kubernetes.io/instance: flyte-sandbox - app.kubernetes.io/name: postgresql - topologyKey: kubernetes.io/hostname - weight: 1 + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/component: primary + app.kubernetes.io/instance: flyte-sandbox + app.kubernetes.io/name: postgresql + topologyKey: kubernetes.io/hostname + weight: 1 containers: - - env: - - name: BITNAMI_DEBUG - value: "false" - - name: POSTGRESQL_PORT_NUMBER - value: "5432" - - name: POSTGRESQL_VOLUME_DIR - value: /bitnami/postgresql - - name: PGDATA - value: /bitnami/postgresql/data - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - key: postgres-password - name: flyte-sandbox-postgresql - - name: POSTGRESQL_ENABLE_LDAP - value: "no" - - name: POSTGRESQL_ENABLE_TLS - value: "no" - - name: POSTGRESQL_LOG_HOSTNAME - value: "false" - - name: POSTGRESQL_LOG_CONNECTIONS - value: "false" - - name: POSTGRESQL_LOG_DISCONNECTIONS - value: "false" - - name: POSTGRESQL_PGAUDIT_LOG_CATALOG - value: "off" - - name: POSTGRESQL_CLIENT_MIN_MESSAGES - value: error - - name: POSTGRESQL_SHARED_PRELOAD_LIBRARIES - value: pgaudit - image: docker.io/bitnami/postgresql:sandbox - imagePullPolicy: Never - livenessProbe: - exec: - command: - - /bin/sh - - -c - - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 - failureThreshold: 6 - initialDelaySeconds: 30 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - name: postgresql - ports: - - containerPort: 5432 - name: tcp-postgresql - readinessProbe: - exec: - command: - - /bin/sh - - -c - - -e - - | - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 - [ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f /bitnami/postgresql/.initialized ] - failureThreshold: 6 - initialDelaySeconds: 5 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - resources: - limits: {} - requests: - cpu: 250m - memory: 256Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - runAsGroup: 0 - runAsNonRoot: true - runAsUser: 1001 - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /bitnami/postgresql - name: data + - env: + - name: BITNAMI_DEBUG + value: "false" + - name: POSTGRESQL_PORT_NUMBER + value: "5432" + - name: POSTGRESQL_VOLUME_DIR + value: /bitnami/postgresql + - name: PGDATA + value: /bitnami/postgresql/data + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + key: postgres-password + name: flyte-sandbox-postgresql + - name: POSTGRESQL_ENABLE_LDAP + value: "no" + - name: POSTGRESQL_ENABLE_TLS + value: "no" + - name: POSTGRESQL_LOG_HOSTNAME + value: "false" + - name: POSTGRESQL_LOG_CONNECTIONS + value: "false" + - name: POSTGRESQL_LOG_DISCONNECTIONS + value: "false" + - name: POSTGRESQL_PGAUDIT_LOG_CATALOG + value: "off" + - name: POSTGRESQL_CLIENT_MIN_MESSAGES + value: error + - name: POSTGRESQL_SHARED_PRELOAD_LIBRARIES + value: pgaudit + image: docker.io/bitnami/postgresql:sandbox + imagePullPolicy: Never + livenessProbe: + exec: + command: + - /bin/sh + - -c + - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 + failureThreshold: 6 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + name: postgresql + ports: + - containerPort: 5432 + name: tcp-postgresql + readinessProbe: + exec: + command: + - /bin/sh + - -c + - -e + - | + exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432 + [ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f /bitnami/postgresql/.initialized ] + failureThreshold: 6 + initialDelaySeconds: 5 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + resources: + limits: {} + requests: + cpu: 250m + memory: 256Mi + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + runAsGroup: 0 + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + volumeMounts: + - mountPath: /bitnami/postgresql + name: data hostIPC: false hostNetwork: false initContainers: - - command: - - /bin/sh - - -ec - - | - chown 1001:1001 /bitnami/postgresql - mkdir -p /bitnami/postgresql/data - chmod 700 /bitnami/postgresql/data - find /bitnami/postgresql -mindepth 1 -maxdepth 1 -not -name "conf" -not -name ".snapshot" -not -name "lost+found" | \ - xargs -r chown -R 1001:1001 - image: docker.io/bitnami/os-shell:sandbox - imagePullPolicy: Never - name: init-chmod-data - resources: - limits: {} - requests: {} - securityContext: - runAsGroup: 0 - runAsNonRoot: false - runAsUser: 0 - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /bitnami/postgresql - name: data + - command: + - /bin/sh + - -ec + - | + chown 1001:1001 /bitnami/postgresql + mkdir -p /bitnami/postgresql/data + chmod 700 /bitnami/postgresql/data + find /bitnami/postgresql -mindepth 1 -maxdepth 1 -not -name "conf" -not -name ".snapshot" -not -name "lost+found" | \ + xargs -r chown -R 1001:1001 + image: docker.io/bitnami/os-shell:sandbox + imagePullPolicy: Never + name: init-chmod-data + resources: + limits: {} + requests: {} + securityContext: + runAsGroup: 0 + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + volumeMounts: + - mountPath: /bitnami/postgresql + name: data securityContext: fsGroup: 1001 serviceAccountName: default volumes: - - name: data - persistentVolumeClaim: - claimName: flyte-sandbox-db-storage + - name: data + persistentVolumeClaim: + claimName: flyte-sandbox-db-storage updateStrategy: rollingUpdate: {} type: RollingUpdate diff --git a/docs/community/troubleshoot.rst b/docs/community/troubleshoot.rst index 2a1b620515..5d1281e967 100644 --- a/docs/community/troubleshoot.rst +++ b/docs/community/troubleshoot.rst @@ -170,7 +170,7 @@ Please add ``spark`` to the list of `enabled-plugins` in the config yaml file. F task-plugins: enabled-plugins: - container - - sidecar + - uploader - K8S-ARRAY - spark default-for-task-types: diff --git a/docs/deployment/agents/airflow.rst b/docs/deployment/agents/airflow.rst index 174967e20c..4d8e5b30c5 100644 --- a/docs/deployment/agents/airflow.rst +++ b/docs/deployment/agents/airflow.rst @@ -27,7 +27,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -50,12 +50,12 @@ Specify agent configuration # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array airflow: agent-service diff --git a/docs/deployment/agents/bigquery.rst b/docs/deployment/agents/bigquery.rst index 0d30d0d3b3..6be5750028 100644 --- a/docs/deployment/agents/bigquery.rst +++ b/docs/deployment/agents/bigquery.rst @@ -35,7 +35,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -58,12 +58,12 @@ Specify agent configuration # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array bigquery_query_job_task: agent-service diff --git a/docs/deployment/agents/chatgpt.rst b/docs/deployment/agents/chatgpt.rst index 9483d95d35..ec5e234aa2 100644 --- a/docs/deployment/agents/chatgpt.rst +++ b/docs/deployment/agents/chatgpt.rst @@ -26,7 +26,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -58,12 +58,12 @@ Specify agent configuration # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array chatgpt: agent-service plugins: diff --git a/docs/deployment/agents/databricks.rst b/docs/deployment/agents/databricks.rst index b419144021..c6c287e291 100644 --- a/docs/deployment/agents/databricks.rst +++ b/docs/deployment/agents/databricks.rst @@ -146,11 +146,11 @@ Specify agent configuration default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader databricks: agent-service enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service @@ -165,7 +165,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -185,12 +185,12 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array databricks: agent-service diff --git a/docs/deployment/agents/openai_batch.rst b/docs/deployment/agents/openai_batch.rst index 8e1c622b73..2765c91a5e 100644 --- a/docs/deployment/agents/openai_batch.rst +++ b/docs/deployment/agents/openai_batch.rst @@ -25,7 +25,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -46,12 +46,12 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array openai-batch: agent-service diff --git a/docs/deployment/agents/sagemaker_inference.rst b/docs/deployment/agents/sagemaker_inference.rst index 3f03e08f55..a5d243c516 100644 --- a/docs/deployment/agents/sagemaker_inference.rst +++ b/docs/deployment/agents/sagemaker_inference.rst @@ -25,7 +25,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -47,12 +47,12 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array boto: agent-service sagemaker-endpoint: agent-service diff --git a/docs/deployment/agents/sensor.rst b/docs/deployment/agents/sensor.rst index 312e34bcd1..c508c7b944 100644 --- a/docs/deployment/agents/sensor.rst +++ b/docs/deployment/agents/sensor.rst @@ -62,7 +62,7 @@ Enable the sensor agent by adding the following config to the relevant YAML file task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -85,12 +85,12 @@ Enable the sensor agent by adding the following config to the relevant YAML file # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array sensor: agent-service diff --git a/docs/deployment/agents/snowflake.rst b/docs/deployment/agents/snowflake.rst index dad62a3795..a9ec2ba6d4 100644 --- a/docs/deployment/agents/snowflake.rst +++ b/docs/deployment/agents/snowflake.rst @@ -41,7 +41,7 @@ Specify agent configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: @@ -64,12 +64,12 @@ Specify agent configuration # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array snowflake: agent-service diff --git a/docs/deployment/configuration/generated/flyteadmin_config.rst b/docs/deployment/configuration/generated/flyteadmin_config.rst index c8bbe4bd32..1219e983c3 100644 --- a/docs/deployment/configuration/generated/flyteadmin_config.rst +++ b/docs/deployment/configuration/generated/flyteadmin_config.rst @@ -3320,7 +3320,7 @@ config.FlyteCoPilotConfig name (string) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Flyte co-pilot sidecar container name prefix. (additional bits will be added after this) +Flyte co-pilot uploader container name prefix. (additional bits will be added after this) **Default Value**: diff --git a/docs/deployment/configuration/generated/flytepropeller_config.rst b/docs/deployment/configuration/generated/flytepropeller_config.rst index fc78d202ca..8909a28be7 100644 --- a/docs/deployment/configuration/generated/flytepropeller_config.rst +++ b/docs/deployment/configuration/generated/flytepropeller_config.rst @@ -1356,7 +1356,7 @@ ray (`ray.Config`_) stackdriver-logresourcename: "" stackdriver-template-uri: "" templates: null - logsSidecar: null + logsUploader: null remoteClusterConfig: auth: caCertPath: "" @@ -2744,7 +2744,7 @@ config.FlyteCoPilotConfig name (string) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Flyte co-pilot sidecar container name prefix. (additional bits will be added after this) +Flyte co-pilot uploader container name prefix. (additional bits will be added after this) **Default Value**: @@ -3742,7 +3742,7 @@ logs (`logs.LogConfig`_) templates: null -logsSidecar (v1.Container) +logsUploader (v1.Container) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" **Default Value**: @@ -5857,7 +5857,7 @@ AWS Secret Manager config. requests: cpu: 200m memory: 500Mi - sidecarImage: docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4 + uploaderImage: docker.io/amazon/aws-secrets-manager-secret-uploader:v0.1.4 gcpSecretManager (`config.GCPSecretManagerConfig`_) @@ -5876,7 +5876,7 @@ GCP Secret Manager config. requests: cpu: 200m memory: 500Mi - sidecarImage: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine + uploaderImage: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine vaultSecretManager (`config.VaultSecretManagerConfig`_) @@ -5896,16 +5896,16 @@ Vault Secret Manager config. config.AWSSecretManagerConfig ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -sidecarImage (string) +uploaderImage (string) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Specifies the sidecar docker image to use +Specifies the uploader docker image to use **Default Value**: .. code-block:: yaml - docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4 + docker.io/amazon/aws-secrets-manager-secret-uploader:v0.1.4 resources (`v1.ResourceRequirements`_) @@ -5961,10 +5961,10 @@ claims ([]v1.ResourceClaim) config.GCPSecretManagerConfig ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -sidecarImage (string) +uploaderImage (string) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Specifies the sidecar docker image to use +Specifies the uploader docker image to use **Default Value**: diff --git a/docs/deployment/configuration/generated/scheduler_config.rst b/docs/deployment/configuration/generated/scheduler_config.rst index 8904155e7c..ed30d84781 100644 --- a/docs/deployment/configuration/generated/scheduler_config.rst +++ b/docs/deployment/configuration/generated/scheduler_config.rst @@ -3320,7 +3320,7 @@ config.FlyteCoPilotConfig name (string) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Flyte co-pilot sidecar container name prefix. (additional bits will be added after this) +Flyte co-pilot uploader container name prefix. (additional bits will be added after this) **Default Value**: diff --git a/docs/deployment/plugins/aws/athena.rst b/docs/deployment/plugins/aws/athena.rst index 34edafc4bd..498a74efb3 100644 --- a/docs/deployment/plugins/aws/athena.rst +++ b/docs/deployment/plugins/aws/athena.rst @@ -31,7 +31,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - athena default-for-task-types: @@ -51,12 +51,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - athena default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array athena: athena diff --git a/docs/deployment/plugins/aws/sagemaker.rst b/docs/deployment/plugins/aws/sagemaker.rst index 6411ee6c2b..4e8606e0c6 100644 --- a/docs/deployment/plugins/aws/sagemaker.rst +++ b/docs/deployment/plugins/aws/sagemaker.rst @@ -41,13 +41,13 @@ Please make sure that the propeller has the correct service account for Sagemake # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - sagemaker_training - sagemaker_hyperparameter_tuning default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array Upgrade the Flyte Helm release diff --git a/docs/deployment/plugins/gcp/bigquery.rst b/docs/deployment/plugins/gcp/bigquery.rst index 03b21e02e1..1de371c3c9 100644 --- a/docs/deployment/plugins/gcp/bigquery.rst +++ b/docs/deployment/plugins/gcp/bigquery.rst @@ -31,7 +31,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - bigquery default-for-task-types: @@ -54,12 +54,12 @@ Specify plugin configuration # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend enabled-plugins: - container - - sidecar + - uploader - k8s-array - bigquery default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array bigquery_query_job_task: bigquery diff --git a/docs/deployment/plugins/k8s/index.rst b/docs/deployment/plugins/k8s/index.rst index 64fbb41136..62fe78bd7f 100644 --- a/docs/deployment/plugins/k8s/index.rst +++ b/docs/deployment/plugins/k8s/index.rst @@ -127,7 +127,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - pytorch default-for-task-types: @@ -148,12 +148,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - pytorch default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array pytorch: pytorch @@ -174,7 +174,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - tensorflow default-for-task-types: @@ -194,12 +194,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - tensorflow default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array tensorflow: tensorflow @@ -220,7 +220,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - mpi default-for-task-types: @@ -240,12 +240,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - mpi default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array mpi: mpi @@ -265,7 +265,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - ray default-for-task-types: @@ -290,12 +290,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - ray default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array ray: ray plugins: @@ -323,7 +323,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - spark default-for-task-types: @@ -489,7 +489,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - spark default-for-task-types: @@ -612,12 +612,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - spark default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array spark: spark cluster_resource_manager: @@ -724,12 +724,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - spark default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array spark: spark cluster_resource_manager: @@ -951,13 +951,13 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - agent-service - spark default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array spark: spark @@ -976,7 +976,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - dask default-for-task-types: @@ -996,12 +996,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - dask default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array dask: dask diff --git a/docs/deployment/plugins/webapi/databricks.rst b/docs/deployment/plugins/webapi/databricks.rst index 69f710677c..8e6ec39f70 100644 --- a/docs/deployment/plugins/webapi/databricks.rst +++ b/docs/deployment/plugins/webapi/databricks.rst @@ -140,11 +140,11 @@ Specify plugin configuration default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader spark: databricks enabled-plugins: - container - - sidecar + - uploader - k8s-array - databricks plugins: @@ -196,7 +196,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - databricks default-for-task-types: @@ -228,12 +228,12 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - databricks default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array spark: databricks databricks: diff --git a/docs/deployment/plugins/webapi/snowflake.rst b/docs/deployment/plugins/webapi/snowflake.rst index 80ef2305d0..5d1e40930d 100644 --- a/docs/deployment/plugins/webapi/snowflake.rst +++ b/docs/deployment/plugins/webapi/snowflake.rst @@ -57,12 +57,12 @@ Specify plugin configuration default-for-task-types: container: container container_array: k8s-array - sidecar: sidecar + uploader: uploader snowflake: snowflake enabled-plugins: - container - k8s-array - - sidecar + - uploader - snowflake .. group-tab:: Helm chart @@ -76,7 +76,7 @@ Specify plugin configuration task-plugins: enabled-plugins: - container - - sidecar + - uploader - k8s-array - snowflake default-for-task-types: @@ -100,12 +100,12 @@ Specify plugin configuration # plugins enabled-plugins: - container - - sidecar + - uploader - k8s-array - snowflake default-for-task-types: container: container - sidecar: sidecar + uploader: uploader container_array: k8s-array snowflake: snowflake diff --git a/docs/user_guide/productionizing/secrets.md b/docs/user_guide/productionizing/secrets.md index 538dc5d5ad..6e91877a5e 100644 --- a/docs/user_guide/productionizing/secrets.md +++ b/docs/user_guide/productionizing/secrets.md @@ -23,7 +23,7 @@ running container. - Install [kubectl](https://kubernetes.io/docs/tasks/tools/). - Have access to a Flyte cluster, for e.g. with `flytectl demo start` as described {ref}`here `. -::: + ::: The first step to using secrets in Flyte is to create one on the backend. By default, Flyte uses the K8s-native secrets manager, which we'll use in this @@ -102,10 +102,11 @@ Never print secret values! The example above is just for demonstration purposes. ::: :::{note} + - In case Flyte fails to access the secret, an error is raised. - The `Secret` group and key are required parameters during declaration and usage. Failure to specify will cause a {py:class}`ValueError`. -::: + ::: ### Multiple keys grouped into one secret @@ -224,7 +225,6 @@ argument names for the username and password. You can then use the `sql_query` task inside a workflow to grab data and perform downstream transformations on it. - ## How secrets injection works The rest of this page describes how secrets injection works under the hood. @@ -290,10 +290,9 @@ The global secrets take precedence over any secret discoverable by the secret ma The following secret managers are available at the time of writing: - [K8s secrets](https://kubernetes.io/docs/concepts/configuration/secret/#creating-a-secret) (**default**): `flyte-pod-webhook` will try to look for a K8s secret named after the secret Group and retrieve the value for the secret Key. -- [AWS Secret Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html): `flyte-pod-webhook` will add the AWS Secret Manager sidecar container to a task Pod which will mount the secret. +- [AWS Secret Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html): `flyte-pod-webhook` will add the AWS Secret Manager uploader container to a task Pod which will mount the secret. - [Vault Agent Injector](https://developer.hashicorp.com/vault/tutorials/getting-started/getting-started-first-secret#write-a-secret) : `flyte-pod-webhook` will annotate the task Pod with the respective Vault annotations that trigger an existing Vault Agent Injector to retrieve the specified secret Key from a vault path defined as secret Group. - When using the K8s secret manager plugin, which is enabled by default, the secrets need to be available in the same namespace as the task execution (for example `flytesnacks-development`). K8s secrets can be mounted as either files or injected as environment variables into the task pod, so if you need to make larger files available to the task, then this might be the better option. @@ -307,7 +306,7 @@ When using the AWS secret management plugin, secrets need to be specified by nam ### Vault secrets manager -When using the Vault secret manager, make sure you have Vault Agent deployed on your cluster as described in this [step-by-step tutorial](https://learn.hashicorp.com/tutorials/vault/kubernetes-sidecar). +When using the Vault secret manager, make sure you have Vault Agent deployed on your cluster as described in this [step-by-step tutorial](https://learn.hashicorp.com/tutorials/vault/kubernetes-uploader). Vault secrets can only be mounted as files and will become available under `"/etc/flyte/secrets/SECRET_GROUP/SECRET_NAME"`. Vault comes with various secrets engines. Currently Flyte supports working with both version 1 and 2 of the `Key Vault engine ` as well as the `databases secrets engine `. diff --git a/flyteadmin/pkg/repositories/transformers/resource_test.go b/flyteadmin/pkg/repositories/transformers/resource_test.go index a1ef2cacef..79e8e0bf0b 100644 --- a/flyteadmin/pkg/repositories/transformers/resource_test.go +++ b/flyteadmin/pkg/repositories/transformers/resource_test.go @@ -87,21 +87,21 @@ func TestMergeUpdateProjectDomainAttributes(t *testing.T) { mergeUpdatedModel, err := MergeUpdatePluginAttributes(context.Background(), existingModel, admin.MatchableResource_PLUGIN_OVERRIDE, &repoInterfaces.ResourceID{}, testutils.GetPluginOverridesAttributes(map[string][]string{ - "sidecar": {"plugin_c"}, - "hive": {"plugin_d"}, + "uploader": {"plugin_c"}, + "hive": {"plugin_d"}, }), ) assert.NoError(t, err) var updatedAttributes admin.MatchingAttributes err = proto.Unmarshal(mergeUpdatedModel.Attributes, &updatedAttributes) assert.NoError(t, err) - var sawPythonTask, sawSidecarTask, sawHiveTask bool + var sawPythonTask, sawUploaderTask, sawHiveTask bool for _, override := range updatedAttributes.GetPluginOverrides().GetOverrides() { if override.GetTaskType() == "python" { sawPythonTask = true assert.EqualValues(t, []string{"plugin_a"}, override.GetPluginId()) - } else if override.GetTaskType() == "sidecar" { - sawSidecarTask = true + } else if override.GetTaskType() == "uploader" { + sawUploaderTask = true assert.EqualValues(t, []string{"plugin_c"}, override.GetPluginId()) } else if override.GetTaskType() == "hive" { sawHiveTask = true @@ -109,7 +109,7 @@ func TestMergeUpdateProjectDomainAttributes(t *testing.T) { } } assert.True(t, sawPythonTask, "Missing python task from finalized attributes") - assert.True(t, sawSidecarTask, "Missing sidecar task from finalized attributes") + assert.True(t, sawUploaderTask, "Missing uploader task from finalized attributes") assert.True(t, sawHiveTask, "Missing hive task from finalized attributes") }) t.Run("unsupported resource type", func(t *testing.T) { @@ -184,21 +184,21 @@ func TestMergeUpdateWorkflowAttributes(t *testing.T) { Domain: resourceDomain, Workflow: resourceWorkflow, MatchingAttributes: testutils.GetPluginOverridesAttributes(map[string][]string{ - "sidecar": {"plugin_c"}, - "hive": {"plugin_d"}, + "uploader": {"plugin_c"}, + "hive": {"plugin_d"}, }), }) assert.NoError(t, err) var updatedAttributes admin.MatchingAttributes err = proto.Unmarshal(mergeUpdatedModel.Attributes, &updatedAttributes) assert.NoError(t, err) - var sawPythonTask, sawSidecarTask, sawHiveTask bool + var sawPythonTask, sawUploaderTask, sawHiveTask bool for _, override := range updatedAttributes.GetPluginOverrides().GetOverrides() { if override.GetTaskType() == "python" { sawPythonTask = true assert.EqualValues(t, []string{"plugin_a"}, override.GetPluginId()) - } else if override.GetTaskType() == "sidecar" { - sawSidecarTask = true + } else if override.GetTaskType() == "uploader" { + sawUploaderTask = true assert.EqualValues(t, []string{"plugin_c"}, override.GetPluginId()) } else if override.GetTaskType() == "hive" { sawHiveTask = true @@ -206,7 +206,7 @@ func TestMergeUpdateWorkflowAttributes(t *testing.T) { } } assert.True(t, sawPythonTask, "Missing python task from finalized attributes") - assert.True(t, sawSidecarTask, "Missing sidecar task from finalized attributes") + assert.True(t, sawUploaderTask, "Missing uploader task from finalized attributes") assert.True(t, sawHiveTask, "Missing hive task from finalized attributes") }) t.Run("unsupported resource type", func(t *testing.T) { diff --git a/flyteadmin/pkg/repositories/transformers/task_execution_test.go b/flyteadmin/pkg/repositories/transformers/task_execution_test.go index 461ab39c8c..68b7fca470 100644 --- a/flyteadmin/pkg/repositories/transformers/task_execution_test.go +++ b/flyteadmin/pkg/repositories/transformers/task_execution_test.go @@ -268,7 +268,7 @@ func TestCreateTaskExecutionModelQueued(t *testing.T) { }, OccurredAt: taskEventOccurredAtProto, Reason: "Task was scheduled", - TaskType: "sidecar", + TaskType: "uploader", }, }, StorageClient: ds, @@ -287,7 +287,7 @@ func TestCreateTaskExecutionModelQueued(t *testing.T) { Message: "Task was scheduled", }, }, - TaskType: "sidecar", + TaskType: "uploader", } expectedClosureBytes, err := proto.Marshal(expectedClosure) diff --git a/flytecopilot/README.md b/flytecopilot/README.md index 92304e947f..f52fe75e8b 100644 --- a/flytecopilot/README.md +++ b/flytecopilot/README.md @@ -1,11 +1,12 @@ # Flyte CoPilot ## Overview -Flyte CoPilot provides a sidecar that understand Flyte Metadata Format as specified in FlyteIDL and make it possible to run arbitrary containers in Flyte. -This is achieved using `flyte-copilot` a binary that runs in 2 modes, - -*Downloader* - Downloads the metadata and any other data (if configured) to a provided path. In kubernetes this path could be a shared volume. - - *Sidecar* - Monitors the process and uploads any data that is generated by the process in a prescribed path/ - + +Flyte CoPilot provides a uploader that understand Flyte Metadata Format as specified in FlyteIDL and make it possible to run arbitrary containers in Flyte. +This is achieved using `flyte-copilot` a binary that runs in 2 modes, -_Downloader_ - Downloads the metadata and any other data (if configured) to a provided path. In kubernetes this path could be a shared volume. + +- _Uploader_ - Monitors the process and uploads any data that is generated by the process in a prescribed path/ + ## Mode: Downloader ```bash @@ -13,40 +14,40 @@ $ flyte-copilot downloader ``` In K8s `flyte-copilot downloader` can be run as part of the init containers with the download volume mounted. This guarantees that the metadata and any data (if configured) -is downloaded before the main container starts up. +is downloaded before the main container starts up. + +## Mode: Uploader -## Mode: Sidecar - As a sidecar process, that runs in parallel with the main container/process, the goal is to - 1. identify the main container - 2. Wait for the main container to start up - 3. Wait for the main container to exit - 4. Copy the data to remote store (especially the metadata) - 5. Exit +As a uploader process, that runs in parallel with the main container/process, the goal is to + +1. identify the main container +2. Wait for the main container to start up +3. Wait for the main container to exit +4. Copy the data to remote store (especially the metadata) +5. Exit ```bash -$ flyte-copilot sidecar -``` +$ flyte-copilot uploader +``` ### Raw notes - Solution 1: - poll Kubeapi. - - Works perfectly fine, but too much load on kubeapi - Solution 2: - Create a protocol. Main container will exit and write a _SUCCESS file to a known location - - problem in the case of oom or random exits. Uploader will be stuck. We could use a timeout? and in the sidecar just kill the pod, when the main exits unhealthy? +Solution 1: +poll Kubeapi. - Works perfectly fine, but too much load on kubeapi - Solution 3: - Use shared process namespace. This allows all pids in a pod to share the namespace. Thus pids can see each other. +Solution 2: +Create a protocol. Main container will exit and write a \_SUCCESS file to a known location - problem in the case of oom or random exits. Uploader will be stuck. We could use a timeout? and in the uploader just kill the pod, when the main exits unhealthy? + +Solution 3: +Use shared process namespace. This allows all pids in a pod to share the namespace. Thus pids can see each other. Problems: How to identify the main container? - Container id is not known ahead of time and container name -> Pid mapping is not possible? - How to wait for main container to start up. One solution for both, call kubeapi and get pod info and find the container id - + Note: we can poll /proc/pid/cgroup file (it contains the container id) so we can create a blind container id to pid mapping. Then somehow get the main container id Once we know the main container, waiting for it to exit is simple and implemented Copying data is simple and implemented - diff --git a/flytecopilot/cmd/sidecar.go b/flytecopilot/cmd/upload.go similarity index 92% rename from flytecopilot/cmd/sidecar.go rename to flytecopilot/cmd/upload.go index 179d6362f8..c69f0257dc 100644 --- a/flytecopilot/cmd/sidecar.go +++ b/flytecopilot/cmd/upload.go @@ -30,7 +30,7 @@ type UploadOptions struct { metaOutputName string // The remote prefix where all the raw outputs should be uploaded of the form s3://bucket/prefix/ remoteOutputsRawPrefix string - // Local directory path where the sidecar should look for outputs. + // Local directory path where the uploader should look for outputs. localDirectoryPath string // Non primitive types will be dumped in this output format metadataFormat string @@ -73,7 +73,7 @@ func (u *UploadOptions) uploader(ctx context.Context) error { outputInterface := iface.GetOutputs() if iface.GetOutputs() == nil || iface.Outputs.Variables == nil || len(iface.GetOutputs().GetVariables()) == 0 { - logger.Infof(ctx, "Empty output interface received. Assuming void outputs. Sidecar will exit immediately.") + logger.Infof(ctx, "Empty output interface received. Assuming void outputs. Uploader will exit immediately.") return nil } @@ -140,7 +140,7 @@ func (u *UploadOptions) uploader(ctx context.Context) error { return nil } -func (u *UploadOptions) Sidecar(ctx context.Context) error { +func (u *UploadOptions) Uploader(ctx context.Context) error { if err := u.uploader(ctx); err != nil { logger.Errorf(ctx, "Uploading failed, err %s", err) @@ -160,11 +160,11 @@ func NewUploadCommand(opts *RootOptions) *cobra.Command { // deleteCmd represents the delete command uploadCmd := &cobra.Command{ - Use: "sidecar ", + Use: "uploader ", Short: "uploads flyteData from the localpath to a remote dir.", Long: `Currently it looks at the outputs.pb and creates one file per variable.`, RunE: func(cmd *cobra.Command, args []string) error { - return uploadOptions.Sidecar(context.Background()) + return uploadOptions.Uploader(context.Background()) }, } @@ -177,8 +177,8 @@ func NewUploadCommand(opts *RootOptions) *cobra.Command { uploadCmd.Flags().DurationVarP(&uploadOptions.timeout, "timeout", "t", time.Hour*1, "Max time to allow for uploads to complete, default is 1H") uploadCmd.Flags().BytesBase64VarP(&uploadOptions.typedInterface, "interface", "i", nil, "Typed Interface - core.TypedInterface, base64 encoded string of the serialized protobuf") uploadCmd.Flags().DurationVarP(&uploadOptions.containerStartTimeout, "start-timeout", "", 0, "Max time to allow for container to startup. 0 indicates wait for ever.") - uploadCmd.Flags().StringVarP(&uploadOptions.startWatcherType, "start-watcher-type", "", containerwatcher.WatcherTypeSharedProcessNS, fmt.Sprintf("Sidecar will wait for container before starting upload process. Watcher type makes the type configurable. Available Type %+v", containerwatcher.AllWatcherTypes)) - uploadCmd.Flags().StringVarP(&uploadOptions.exitWatcherType, "exit-watcher-type", "", containerwatcher.WatcherTypeSharedProcessNS, fmt.Sprintf("Sidecar will wait for completion of the container before starting upload process. Watcher type makes the type configurable. Available Type %+v", containerwatcher.AllWatcherTypes)) + uploadCmd.Flags().StringVarP(&uploadOptions.startWatcherType, "start-watcher-type", "", containerwatcher.WatcherTypeSharedProcessNS, fmt.Sprintf("Uploader will wait for container before starting upload process. Watcher type makes the type configurable. Available Type %+v", containerwatcher.AllWatcherTypes)) + uploadCmd.Flags().StringVarP(&uploadOptions.exitWatcherType, "exit-watcher-type", "", containerwatcher.WatcherTypeSharedProcessNS, fmt.Sprintf("Uploader will wait for completion of the container before starting upload process. Watcher type makes the type configurable. Available Type %+v", containerwatcher.AllWatcherTypes)) uploadCmd.Flags().StringVarP(&uploadOptions.containerInfo.Name, "watch-container", "", "", "For KubeAPI watcher, Wait for this container to exit.") uploadCmd.Flags().StringVarP(&uploadOptions.containerInfo.Namespace, "namespace", "", "", "For KubeAPI watcher, Namespace of the pod [optional]") uploadCmd.Flags().StringVarP(&uploadOptions.containerInfo.PodName, "pod-name", "", "", "For KubeAPI watcher, Name of the pod [optional].") diff --git a/flytecopilot/cmd/sidecar_test.go b/flytecopilot/cmd/upload_test.go similarity index 97% rename from flytecopilot/cmd/sidecar_test.go rename to flytecopilot/cmd/upload_test.go index 2932e6fa9c..5828feedd5 100644 --- a/flytecopilot/cmd/sidecar_test.go +++ b/flytecopilot/cmd/upload_test.go @@ -45,7 +45,7 @@ func TestUploadOptions_Upload(t *testing.T) { localDirectoryPath: tmpDir, } - assert.NoError(t, uopts.Sidecar(ctx)) + assert.NoError(t, uopts.Uploader(ctx)) }) t.Run("uploadBlobType-FileNotFound", func(t *testing.T) { @@ -91,7 +91,7 @@ func TestUploadOptions_Upload(t *testing.T) { ok, err := containerwatcher.FileExists(success) assert.NoError(t, err) assert.True(t, ok, "successfile not created") - assert.NoError(t, uopts.Sidecar(ctx)) + assert.NoError(t, uopts.Uploader(ctx)) v, err := store.Head(ctx, "/output/errors.pb") assert.NoError(t, err) assert.True(t, v.Exists()) diff --git a/flyteplugins/go/tasks/pluginmachinery/bundle/fail_fast.go b/flyteplugins/go/tasks/pluginmachinery/bundle/fail_fast.go index 0447b488cd..64d79118ef 100644 --- a/flyteplugins/go/tasks/pluginmachinery/bundle/fail_fast.go +++ b/flyteplugins/go/tasks/pluginmachinery/bundle/fail_fast.go @@ -53,7 +53,7 @@ func init() { // can reference in other handler definitions. // NOTE: these should match the constants defined flytekit taskTypes := []core.TaskType{ - "container", "sidecar", "container_array", "hive", "presto", "spark", "pytorch", + "container", "uploader", "container_array", "hive", "presto", "spark", "pytorch", "sagemaker_custom_training_job_task", "sagemaker_training_job_task", "sagemaker_hyperparameter_tuning_job_task", } pluginMachinery.PluginRegistry().RegisterCorePlugin( diff --git a/flyteplugins/go/tasks/pluginmachinery/catalog/async_client.go b/flyteplugins/go/tasks/pluginmachinery/catalog/async_client.go index 7e309396fc..8be9e10bc3 100644 --- a/flyteplugins/go/tasks/pluginmachinery/catalog/async_client.go +++ b/flyteplugins/go/tasks/pluginmachinery/catalog/async_client.go @@ -40,7 +40,7 @@ type Future interface { GetResponseError() error } -// Catalog Sidecar future to represent async process of uploading catalog artifacts. +// Catalog Uploader future to represent async process of uploading catalog artifacts. type UploadFuture interface { Future } diff --git a/flyteplugins/go/tasks/pluginmachinery/catalog/async_client_impl_test.go b/flyteplugins/go/tasks/pluginmachinery/catalog/async_client_impl_test.go index db32841ee7..699c9484af 100644 --- a/flyteplugins/go/tasks/pluginmachinery/catalog/async_client_impl_test.go +++ b/flyteplugins/go/tasks/pluginmachinery/catalog/async_client_impl_test.go @@ -162,11 +162,11 @@ func TestAsyncClientImpl_Upload(t *testing.T) { } gotPutFuture, err := c.Upload(ctx, tt.requests...) if (err != nil) != tt.wantErr { - t.Errorf("AsyncClientImpl.Sidecar() error = %v, wantErr %v", err, tt.wantErr) + t.Errorf("AsyncClientImpl.Uploader() error = %v, wantErr %v", err, tt.wantErr) return } if !reflect.DeepEqual(gotPutFuture, tt.wantPutFuture) { - t.Errorf("AsyncClientImpl.Sidecar() = %v, want %v", gotPutFuture, tt.wantPutFuture) + t.Errorf("AsyncClientImpl.Uploader() = %v, want %v", gotPutFuture, tt.wantPutFuture) } expectedWorkItemIDs := []string{inputHash1, inputHash2} gottenWorkItemIDs := make([]string, 0) diff --git a/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/config.go b/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/config.go index eb19015586..b47515e8e8 100644 --- a/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/config.go +++ b/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/config.go @@ -217,10 +217,10 @@ type K8sPluginConfig struct { } // FlyteCoPilotConfig specifies configuration for the Flyte CoPilot system. FlyteCoPilot, allows running flytekit-less containers -// in K8s, where the IO is managed by the FlyteCoPilot sidecar process. +// in K8s, where the IO is managed by the FlyteCoPilot uploader process. type FlyteCoPilotConfig struct { - // Co-pilot sidecar container name - NamePrefix string `json:"name" pflag:",Flyte co-pilot sidecar container name prefix. (additional bits will be added after this)"` + // Co-pilot uploader container name + NamePrefix string `json:"name" pflag:",Flyte co-pilot uploader container name prefix. (additional bits will be added after this)"` // Docker image FQN where co-pilot binary is installed Image string `json:"image" pflag:",Flyte co-pilot Docker Image FQN"` // Default Input Path for every task execution that uses co-pilot. This is used only if a input path is not provided by the user and inputs are required for the task @@ -231,9 +231,9 @@ type FlyteCoPilotConfig struct { InputVolumeName string `json:"input-vol-name" pflag:",Name of the data volume that is created for storing inputs"` // Name of the output volume OutputVolumeName string `json:"output-vol-name" pflag:",Name of the data volume that is created for storing outputs"` - // Time for which the sidecar container should wait after starting up, for the primary process to appear. If it does not show up in this time + // Time for which the uploader container should wait after starting up, for the primary process to appear. If it does not show up in this time // the process will be assumed to be dead or in a terminal condition and will trigger an abort. - StartTimeout config2.Duration `json:"start-timeout" pflag:"-,Time for which the sidecar should wait on startup before assuming the primary container to have failed startup."` + StartTimeout config2.Duration `json:"start-timeout" pflag:"-,Time for which the uploader should wait on startup before assuming the primary container to have failed startup."` // Resources for CoPilot Containers CPU string `json:"cpu" pflag:",Used to set cpu for co-pilot containers"` Memory string `json:"memory" pflag:",Used to set memory for co-pilot containers"` diff --git a/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/k8spluginconfig_flags.go b/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/k8spluginconfig_flags.go index 4652d0bfd4..c5e1a195cf 100755 --- a/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/k8spluginconfig_flags.go +++ b/flyteplugins/go/tasks/pluginmachinery/flytek8s/config/k8spluginconfig_flags.go @@ -54,7 +54,7 @@ func (cfg K8sPluginConfig) GetPFlagSet(prefix string) *pflag.FlagSet { cmdFlags.String(fmt.Sprintf("%v%v", prefix, "default-cpus"), defaultK8sConfig.DefaultCPURequest.String(), "Defines a default value for cpu for containers if not specified.") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "default-memory"), defaultK8sConfig.DefaultMemoryRequest.String(), "Defines a default value for memory for containers if not specified.") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "scheduler-name"), defaultK8sConfig.SchedulerName, "Defines scheduler name.") - cmdFlags.String(fmt.Sprintf("%v%v", prefix, "co-pilot.name"), defaultK8sConfig.CoPilot.NamePrefix, "Flyte co-pilot sidecar container name prefix. (additional bits will be added after this)") + cmdFlags.String(fmt.Sprintf("%v%v", prefix, "co-pilot.name"), defaultK8sConfig.CoPilot.NamePrefix, "Flyte co-pilot uploader container name prefix. (additional bits will be added after this)") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "co-pilot.image"), defaultK8sConfig.CoPilot.Image, "Flyte co-pilot Docker Image FQN") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "co-pilot.default-input-path"), defaultK8sConfig.CoPilot.DefaultInputDataPath, "Default path where the volume should be mounted") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "co-pilot.default-output-path"), defaultK8sConfig.CoPilot.DefaultOutputPath, "Default path where the volume should be mounted") diff --git a/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot.go b/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot.go index 427dae9978..d6ebd7dd16 100644 --- a/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot.go +++ b/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot.go @@ -20,8 +20,8 @@ import ( ) const ( - flyteSidecarContainerName = "sidecar" - flyteInitContainerName = "downloader" + flyteUploaderContainerName = "uploader" + flyteInitContainerName = "downloader" ) var pTraceCapability = v1.Capability("SYS_PTRACE") @@ -88,16 +88,16 @@ func CopilotCommandArgs(storageConfig *storage.Config) []string { }...) } -func SidecarCommandArgs(fromLocalPath string, outputPrefix, rawOutputPath storage.DataReference, startTimeout time.Duration, iface *core.TypedInterface) ([]string, error) { +func UploaderCommandArgs(fromLocalPath string, outputPrefix, rawOutputPath storage.DataReference, startTimeout time.Duration, iface *core.TypedInterface) ([]string, error) { if iface == nil { - return nil, fmt.Errorf("interface is required for CoPilot Sidecar") + return nil, fmt.Errorf("interface is required for CoPilot Uploader") } b, err := proto.Marshal(iface) if err != nil { return nil, errors.Wrap(err, "failed to marshal given core.TypedInterface") } return []string{ - "sidecar", + "uploader", "--start-timeout", startTimeout.String(), "--to-raw-output", @@ -263,15 +263,15 @@ func AddCoPilotToPod(ctx context.Context, cfg config.FlyteCoPilotConfig, coPilot coPilotPod.Volumes = append(coPilotPod.Volumes, DataVolume(cfg.OutputVolumeName, size)) // Lets add the Inputs init container - args, err := SidecarCommandArgs(outPath, outputPaths.GetOutputPrefixPath(), outputPaths.GetRawOutputPrefix(), cfg.StartTimeout.Duration, iFace) + args, err := UploaderCommandArgs(outPath, outputPaths.GetOutputPrefixPath(), outputPaths.GetRawOutputPrefix(), cfg.StartTimeout.Duration, iFace) if err != nil { return primaryInitContainerName, err } - sidecar, err := FlyteCoPilotContainer(flyteSidecarContainerName, cfg, args, outputsVolumeMount) + uploader, err := FlyteCoPilotContainer(flyteUploaderContainerName, cfg, args, outputsVolumeMount) if err != nil { return primaryInitContainerName, err } - coPilotPod.Containers = append(coPilotPod.Containers, sidecar) + coPilotPod.Containers = append(coPilotPod.Containers, uploader) } } diff --git a/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot_test.go b/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot_test.go index 182354d07c..bc45c2b3ab 100644 --- a/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot_test.go +++ b/flyteplugins/go/tasks/pluginmachinery/flytek8s/copilot_test.go @@ -142,8 +142,8 @@ func TestDownloadCommandArgs(t *testing.T) { } } -func TestSidecarCommandArgs(t *testing.T) { - _, err := SidecarCommandArgs("", "", "", time.Second*10, nil) +func TestUploaderCommandArgs(t *testing.T) { + _, err := UploaderCommandArgs("", "", "", time.Second*10, nil) assert.Error(t, err) iFace := &core.TypedInterface{ @@ -154,9 +154,9 @@ func TestSidecarCommandArgs(t *testing.T) { }, }, } - d, err := SidecarCommandArgs("/from", "s3://output-meta", "s3://raw-output", time.Second*10, iFace) + d, err := UploaderCommandArgs("/from", "s3://output-meta", "s3://raw-output", time.Second*10, iFace) assert.NoError(t, err) - expected := []string{"sidecar", "--start-timeout", "10s", "--to-raw-output", "s3://raw-output", "--to-output-prefix", "s3://output-meta", "--from-local-dir", "/from", "--interface", ""} + expected := []string{"uploader", "--start-timeout", "10s", "--to-raw-output", "s3://raw-output", "--to-output-prefix", "s3://output-meta", "--from-local-dir", "/from", "--interface", ""} if assert.Len(t, d, len(expected)) { for i := 0; i < len(expected)-1; i++ { assert.Equal(t, expected[i], d[i]) @@ -254,7 +254,7 @@ func assertPodHasCoPilot(t *testing.T, cfg config.FlyteCoPilotConfig, pilot *cor cntr := c assertContainerHasVolumeMounts(t, cfg, pilot, iFace, &cntr) } else { - if c.Name == cfg.NamePrefix+flyteInitContainerName || c.Name == cfg.NamePrefix+flyteSidecarContainerName { + if c.Name == cfg.NamePrefix+flyteInitContainerName || c.Name == cfg.NamePrefix+flyteUploaderContainerName { if iFace != nil { vmap := map[string]v1.VolumeMount{} for _, v := range c.VolumeMounts { diff --git a/flyteplugins/go/tasks/pluginmachinery/flytek8s/pod_helper.go b/flyteplugins/go/tasks/pluginmachinery/flytek8s/pod_helper.go index 53acac5512..520ecf97ff 100644 --- a/flyteplugins/go/tasks/pluginmachinery/flytek8s/pod_helper.go +++ b/flyteplugins/go/tasks/pluginmachinery/flytek8s/pod_helper.go @@ -858,7 +858,7 @@ func DemystifySuccess(status v1.PodStatus, info pluginsCore.TaskInfo) (pluginsCo // DeterminePrimaryContainerPhase as the name suggests, given all the containers, will return a pluginsCore.PhaseInfo object // corresponding to the phase of the primaryContainer which is identified using the provided name. -// This is useful in case of sidecars or pod jobs, where Flyte will monitor successful exit of a single container. +// This is useful in case of uploaders or pod jobs, where Flyte will monitor successful exit of a single container. func DeterminePrimaryContainerPhase(primaryContainerName string, statuses []v1.ContainerStatus, info *pluginsCore.TaskInfo) pluginsCore.PhaseInfo { for _, s := range statuses { if s.Name == primaryContainerName { diff --git a/flyteplugins/go/tasks/plugins/array/k8s/subtask_exec_context.go b/flyteplugins/go/tasks/plugins/array/k8s/subtask_exec_context.go index d0e483257d..743c680088 100644 --- a/flyteplugins/go/tasks/plugins/array/k8s/subtask_exec_context.go +++ b/flyteplugins/go/tasks/plugins/array/k8s/subtask_exec_context.go @@ -84,7 +84,7 @@ func NewSubTaskExecutionContext(ctx context.Context, tCtx pluginsCore.TaskExecut if subtaskTemplate.GetContainer() != nil { subtaskTemplate.Type = podPlugin.ContainerTaskType } else if taskTemplate.GetK8SPod() != nil { - subtaskTemplate.Type = podPlugin.SidecarTaskType + subtaskTemplate.Type = podPlugin.UploaderTaskType } arrayInputReader := array.GetInputReader(tCtx, taskTemplate) diff --git a/flyteplugins/go/tasks/plugins/k8s/kfoperators/common/common_operator_test.go b/flyteplugins/go/tasks/plugins/k8s/kfoperators/common/common_operator_test.go index 9196c788cc..191f95a678 100644 --- a/flyteplugins/go/tasks/plugins/k8s/kfoperators/common/common_operator_test.go +++ b/flyteplugins/go/tasks/plugins/k8s/kfoperators/common/common_operator_test.go @@ -230,7 +230,7 @@ func dummyPodSpec() v1.PodSpec { Containers: []v1.Container{ { Name: "primary container", - Args: []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_sidecar_tasks", "--task-name", "simple_sidecar_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, + Args: []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_uploader_tasks", "--task-name", "simple_uploader_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, Image: "dummy-image", Resources: v1.ResourceRequirements{ Limits: v1.ResourceList{ @@ -294,7 +294,7 @@ func TestOverrideContainerSpecEmptyFields(t *testing.T) { assert.NoError(t, err) assert.Equal(t, 2, len(podSpec.Containers)) assert.Equal(t, "dummy-image", podSpec.Containers[0].Image) - assert.Equal(t, []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_sidecar_tasks", "--task-name", "simple_sidecar_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, podSpec.Containers[0].Args) + assert.Equal(t, []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_uploader_tasks", "--task-name", "simple_uploader_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, podSpec.Containers[0].Args) } func dummyTaskContext() pluginsCore.TaskExecutionContext { diff --git a/flyteplugins/go/tasks/plugins/k8s/pod/plugin.go b/flyteplugins/go/tasks/plugins/k8s/pod/plugin.go index 60b0d5c8d5..2d59fb0e9e 100644 --- a/flyteplugins/go/tasks/plugins/k8s/pod/plugin.go +++ b/flyteplugins/go/tasks/plugins/k8s/pod/plugin.go @@ -23,13 +23,13 @@ const ( podTaskType = "pod" pythonTaskType = "python-task" rawContainerTaskType = "raw-container" - SidecarTaskType = "sidecar" + UploaderTaskType = "uploader" ) -// Why, you might wonder do we recreate the generated go struct generated from the plugins.SidecarJob proto? Because +// Why, you might wonder do we recreate the generated go struct generated from the plugins.UploaderJob proto? Because // although we unmarshal the task custom json, the PodSpec itself is not generated from a proto definition, -// but a proper go struct defined in k8s libraries. Therefore we only unmarshal the sidecar as a json, rather than jsonpb. -type sidecarJob struct { +// but a proper go struct defined in k8s libraries. Therefore we only unmarshal the uploader as a json, rather than jsonpb. +type uploaderJob struct { PodSpec *v1.PodSpec PrimaryContainerName string Annotations map[string]string @@ -59,27 +59,27 @@ func (p plugin) BuildResource(ctx context.Context, taskCtx pluginsCore.TaskExecu } primaryContainerName := "" - if taskTemplate.GetType() == SidecarTaskType && taskTemplate.GetTaskTypeVersion() == 0 { - // handles pod tasks when they are defined as Sidecar tasks and marshal the podspec using k8s proto. - sidecarJob := sidecarJob{} - err := utils.UnmarshalStructToObj(taskTemplate.GetCustom(), &sidecarJob) + if taskTemplate.GetType() == UploaderTaskType && taskTemplate.GetTaskTypeVersion() == 0 { + // handles pod tasks when they are defined as Uploader tasks and marshal the podspec using k8s proto. + uploaderJob := uploaderJob{} + err := utils.UnmarshalStructToObj(taskTemplate.GetCustom(), &uploaderJob) if err != nil { return nil, pluginserrors.Errorf(pluginserrors.BadTaskSpecification, "invalid TaskSpecification [%v], Err: [%v]", taskTemplate.GetCustom(), err.Error()) } - if sidecarJob.PodSpec == nil { + if uploaderJob.PodSpec == nil { return nil, pluginserrors.Errorf(pluginserrors.BadTaskSpecification, "invalid TaskSpecification, nil PodSpec [%v]", taskTemplate.GetCustom()) } - podSpec = sidecarJob.PodSpec + podSpec = uploaderJob.PodSpec // get primary container name - primaryContainerName = sidecarJob.PrimaryContainerName + primaryContainerName = uploaderJob.PrimaryContainerName // update annotations and labels - objectMeta.Annotations = utils.UnionMaps(objectMeta.Annotations, sidecarJob.Annotations) - objectMeta.Labels = utils.UnionMaps(objectMeta.Labels, sidecarJob.Labels) - } else if taskTemplate.GetType() == SidecarTaskType && taskTemplate.GetTaskTypeVersion() == 1 { + objectMeta.Annotations = utils.UnionMaps(objectMeta.Annotations, uploaderJob.Annotations) + objectMeta.Labels = utils.UnionMaps(objectMeta.Labels, uploaderJob.Labels) + } else if taskTemplate.GetType() == UploaderTaskType && taskTemplate.GetTaskTypeVersion() == 1 { // handles pod tasks that marshal the pod spec to the task custom. err := utils.UnmarshalStructToObj(taskTemplate.GetCustom(), &podSpec) if err != nil { @@ -119,10 +119,10 @@ func (p plugin) BuildResource(ctx context.Context, taskCtx pluginsCore.TaskExecu return nil, err } - // set primaryContainerKey annotation if this is a Sidecar task or, as an optimization, if there is only a single + // set primaryContainerKey annotation if this is a Uploader task or, as an optimization, if there is only a single // container. this plugin marks the task complete if the primary Container is complete, so if there is only one // container we can mark the task as complete before the Pod has been marked complete. - if taskTemplate.GetType() == SidecarTaskType || len(podSpec.Containers) == 1 { + if taskTemplate.GetType() == UploaderTaskType || len(podSpec.Containers) == 1 { objectMeta.Annotations[flytek8s.PrimaryContainerKey] = primaryContainerName } @@ -247,10 +247,10 @@ func (plugin) GetProperties() k8s.PluginProperties { } func init() { - // Register ContainerTaskType and SidecarTaskType plugin entries. These separate task types + // Register ContainerTaskType and UploaderTaskType plugin entries. These separate task types // still exist within the system, only now both are evaluated using the same internal pod plugin // instance. This simplifies migration as users may keep the same configuration but are - // seamlessly transitioned from separate container and sidecar plugins to a single pod plugin. + // seamlessly transitioned from separate container and uploader plugins to a single pod plugin. pluginmachinery.PluginRegistry().RegisterK8sPlugin( k8s.PluginEntry{ ID: ContainerTaskType, @@ -262,8 +262,8 @@ func init() { pluginmachinery.PluginRegistry().RegisterK8sPlugin( k8s.PluginEntry{ - ID: SidecarTaskType, - RegisteredTaskTypes: []pluginsCore.TaskType{SidecarTaskType}, + ID: UploaderTaskType, + RegisteredTaskTypes: []pluginsCore.TaskType{UploaderTaskType}, ResourceToWatch: &v1.Pod{}, Plugin: DefaultPodPlugin, IsDefault: false, @@ -273,7 +273,7 @@ func init() { pluginmachinery.PluginRegistry().RegisterK8sPlugin( k8s.PluginEntry{ ID: podTaskType, - RegisteredTaskTypes: []pluginsCore.TaskType{ContainerTaskType, pythonTaskType, rawContainerTaskType, SidecarTaskType}, + RegisteredTaskTypes: []pluginsCore.TaskType{ContainerTaskType, pythonTaskType, rawContainerTaskType, UploaderTaskType}, ResourceToWatch: &v1.Pod{}, Plugin: DefaultPodPlugin, IsDefault: true, diff --git a/flyteplugins/go/tasks/plugins/k8s/pod/testdata/sidecar_custom b/flyteplugins/go/tasks/plugins/k8s/pod/testdata/sidecar_custom deleted file mode 100755 index 00b01208a7..0000000000 --- a/flyteplugins/go/tasks/plugins/k8s/pod/testdata/sidecar_custom +++ /dev/null @@ -1,54 +0,0 @@ -{ - "podSpec": { - "restartPolicy": "OnFailure", - "containers": [{ - "name": "a container", - "image": "foo", - "args": ["pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_sidecar_tasks", "--task-name", "simple_sidecar_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"], - "volumeMounts": [{ - "mountPath": "some/where", - "name": "volume mount" - }], - "env": [{ - "name": "FLYTE_INTERNAL_CONFIGURATION_PATH", - "value": "flytekit.config" - }, { - "name": "FLYTE_INTERNAL_PROJECT", - "value": "" - }, { - "name": "foo", - "value": "bar" - }, { - "name": "FLYTE_INTERNAL_DOMAIN", - "value": "" - }, { - "name": "FLYTE_INTERNAL_VERSION", - "value": "" - }] - }, { - "name": "another container" - }], - "volumes": [{ - "volumeSource": { - "emptyDir": { - "sizeLimit": { - "string": "10G" - }, - "medium": "Memory" - } - }, - "name": "dshm" - }], - "tolerations": [{ - "key": "my toleration key", - "value": "my toleration value" - }] - }, - "primaryContainerName": "a container", - "annotations": { - "a1": "a1" - }, - "labels": { - "b1": "b1" - } -} diff --git a/flyteplugins/go/tasks/plugins/k8s/pod/testdata/uploader_custom b/flyteplugins/go/tasks/plugins/k8s/pod/testdata/uploader_custom new file mode 100755 index 0000000000..4eafbea34b --- /dev/null +++ b/flyteplugins/go/tasks/plugins/k8s/pod/testdata/uploader_custom @@ -0,0 +1,79 @@ +{ + "podSpec": { + "restartPolicy": "OnFailure", + "containers": [ + { + "name": "a container", + "image": "foo", + "args": [ + "pyflyte-execute", + "--task-module", + "tests.flytekit.unit.sdk.tasks.test_uploader_tasks", + "--task-name", + "simple_uploader_task", + "--inputs", + "{{.input}}", + "--output-prefix", + "{{.outputPrefix}}" + ], + "volumeMounts": [ + { + "mountPath": "some/where", + "name": "volume mount" + } + ], + "env": [ + { + "name": "FLYTE_INTERNAL_CONFIGURATION_PATH", + "value": "flytekit.config" + }, + { + "name": "FLYTE_INTERNAL_PROJECT", + "value": "" + }, + { + "name": "foo", + "value": "bar" + }, + { + "name": "FLYTE_INTERNAL_DOMAIN", + "value": "" + }, + { + "name": "FLYTE_INTERNAL_VERSION", + "value": "" + } + ] + }, + { + "name": "another container" + } + ], + "volumes": [ + { + "volumeSource": { + "emptyDir": { + "sizeLimit": { + "string": "10G" + }, + "medium": "Memory" + } + }, + "name": "dshm" + } + ], + "tolerations": [ + { + "key": "my toleration key", + "value": "my toleration value" + } + ] + }, + "primaryContainerName": "a container", + "annotations": { + "a1": "a1" + }, + "labels": { + "b1": "b1" + } +} \ No newline at end of file diff --git a/flyteplugins/go/tasks/plugins/k8s/pod/sidecar_test.go b/flyteplugins/go/tasks/plugins/k8s/pod/uploader_test.go similarity index 86% rename from flyteplugins/go/tasks/plugins/k8s/pod/sidecar_test.go rename to flyteplugins/go/tasks/plugins/k8s/pod/uploader_test.go index e6b8b0478b..8ccbfecbf8 100644 --- a/flyteplugins/go/tasks/plugins/k8s/pod/sidecar_test.go +++ b/flyteplugins/go/tasks/plugins/k8s/pod/uploader_test.go @@ -30,7 +30,7 @@ import ( const ResourceNvidiaGPU = "nvidia.com/gpu" -var sidecarResourceRequirements = &v1.ResourceRequirements{ +var uploaderResourceRequirements = &v1.ResourceRequirements{ Limits: v1.ResourceList{ v1.ResourceCPU: resource.MustParse("2048m"), v1.ResourceEphemeralStorage: resource.MustParse("100M"), @@ -38,23 +38,23 @@ var sidecarResourceRequirements = &v1.ResourceRequirements{ }, } -func getSidecarTaskTemplateForTest(sideCarJob sidecarJob) *core.TaskTemplate { - sidecarJSON, err := json.Marshal(&sideCarJob) +func getUploaderTaskTemplateForTest(sideCarJob uploaderJob) *core.TaskTemplate { + uploaderJSON, err := json.Marshal(&sideCarJob) if err != nil { panic(err) } structObj := structpb.Struct{} - err = json.Unmarshal(sidecarJSON, &structObj) + err = json.Unmarshal(uploaderJSON, &structObj) if err != nil { panic(err) } return &core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, Custom: &structObj, } } -func dummySidecarTaskMetadata(resources *v1.ResourceRequirements, extendedResources *core.ExtendedResources) pluginsCore.TaskExecutionMetadata { +func dummyUploaderTaskMetadata(resources *v1.ResourceRequirements, extendedResources *core.ExtendedResources) pluginsCore.TaskExecutionMetadata { taskMetadata := &pluginsCoreMock.TaskExecutionMetadata{} taskMetadata.On("GetNamespace").Return("test-namespace") taskMetadata.On("GetAnnotations").Return(map[string]string{"annotation-1": "val1"}) @@ -98,9 +98,9 @@ func dummySidecarTaskMetadata(resources *v1.ResourceRequirements, extendedResour return taskMetadata } -func getDummySidecarTaskContext(taskTemplate *core.TaskTemplate, resources *v1.ResourceRequirements, extendedResources *core.ExtendedResources) pluginsCore.TaskExecutionContext { +func getDummyUploaderTaskContext(taskTemplate *core.TaskTemplate, resources *v1.ResourceRequirements, extendedResources *core.ExtendedResources) pluginsCore.TaskExecutionContext { taskCtx := &pluginsCoreMock.TaskExecutionContext{} - dummyTaskMetadata := dummySidecarTaskMetadata(resources, extendedResources) + dummyTaskMetadata := dummyUploaderTaskMetadata(resources, extendedResources) inputReader := &pluginsIOMock.InputReader{} inputReader.OnGetInputPrefixPath().Return("test-data-prefix") inputReader.OnGetInputPath().Return("test-data-reference") @@ -133,7 +133,7 @@ func getPodSpec() v1.PodSpec { Containers: []v1.Container{ { Name: "primary container", - Args: []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_sidecar_tasks", "--task-name", "simple_sidecar_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, + Args: []string{"pyflyte-execute", "--task-module", "tests.flytekit.unit.sdk.tasks.test_uploader_tasks", "--task-name", "simple_uploader_task", "--inputs", "{{.input}}", "--output-prefix", "{{.outputPrefix}}"}, Resources: v1.ResourceRequirements{ Limits: v1.ResourceList{ "cpu": resource.MustParse("2"), @@ -192,7 +192,7 @@ func checkTolerations(t *testing.T, res client.Object, gpuTol v1.Toleration) { } } -func TestBuildSidecarResource_TaskType2(t *testing.T) { +func TestBuildUploaderResource_TaskType2(t *testing.T) { podSpec := getPodSpec() b, err := json.Marshal(podSpec) @@ -206,7 +206,7 @@ func TestBuildSidecarResource_TaskType2(t *testing.T) { } task := core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, TaskTypeVersion: 2, Config: map[string]string{ flytek8s.PrimaryContainerKey: "primary container", @@ -248,7 +248,7 @@ func TestBuildSidecarResource_TaskType2(t *testing.T) { DefaultMemoryRequest: resource.MustParse("1024Mi"), GpuResourceName: ResourceNvidiaGPU, })) - taskCtx := getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) res, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.Nil(t, err) assert.EqualValues(t, map[string]string{ @@ -288,9 +288,9 @@ func TestBuildSidecarResource_TaskType2(t *testing.T) { assert.Equal(t, expectedGPURes, res.(*v1.Pod).Spec.Containers[1].Resources.Limits[ResourceNvidiaGPU]) } -func TestBuildSidecarResource_TaskType2_Invalid_Spec(t *testing.T) { +func TestBuildUploaderResource_TaskType2_Invalid_Spec(t *testing.T) { task := core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, TaskTypeVersion: 2, Config: map[string]string{ flytek8s.PrimaryContainerKey: "primary container", @@ -309,12 +309,12 @@ func TestBuildSidecarResource_TaskType2_Invalid_Spec(t *testing.T) { }, } - taskCtx := getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) _, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.EqualError(t, err, "[BadTaskSpecification] Pod tasks with task type version > 1 should specify their target as a K8sPod with a defined pod spec") } -func TestBuildSidecarResource_TaskType1(t *testing.T) { +func TestBuildUploaderResource_TaskType1(t *testing.T) { podSpec := getPodSpec() b, err := json.Marshal(podSpec) @@ -328,7 +328,7 @@ func TestBuildSidecarResource_TaskType1(t *testing.T) { } task := core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, Custom: structObj, TaskTypeVersion: 1, Config: map[string]string{ @@ -357,7 +357,7 @@ func TestBuildSidecarResource_TaskType1(t *testing.T) { DefaultCPURequest: resource.MustParse("1024m"), DefaultMemoryRequest: resource.MustParse("1024Mi"), })) - taskCtx := getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) res, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.Nil(t, err) assert.EqualValues(t, map[string]string{ @@ -409,7 +409,7 @@ func TestBuildSideResource_TaskType1_InvalidSpec(t *testing.T) { } task := core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, Custom: structObj, TaskTypeVersion: 1, } @@ -422,35 +422,35 @@ func TestBuildSideResource_TaskType1_InvalidSpec(t *testing.T) { DefaultCPURequest: resource.MustParse("1024m"), DefaultMemoryRequest: resource.MustParse("1024Mi"), })) - taskCtx := getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) _, err = DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.EqualError(t, err, "[BadTaskSpecification] invalid TaskSpecification, config needs to be non-empty and include missing [primary_container_name] key") task.Config = map[string]string{ "foo": "bar", } - taskCtx = getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx = getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) _, err = DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.EqualError(t, err, "[BadTaskSpecification] invalid TaskSpecification, config missing [primary_container_name] key in [map[foo:bar]]") } -func TestBuildSidecarResource(t *testing.T) { +func TestBuildUploaderResource(t *testing.T) { dir, err := os.Getwd() if err != nil { t.Fatal(err) } - sidecarCustomJSON, err := ioutil.ReadFile(path.Join(dir, "testdata", "sidecar_custom")) + uploaderCustomJSON, err := ioutil.ReadFile(path.Join(dir, "testdata", "uploader_custom")) if err != nil { - t.Fatal(sidecarCustomJSON) + t.Fatal(uploaderCustomJSON) } - sidecarCustom := structpb.Struct{} - if err := json.Unmarshal(sidecarCustomJSON, &sidecarCustom); err != nil { + uploaderCustom := structpb.Struct{} + if err := json.Unmarshal(uploaderCustomJSON, &uploaderCustom); err != nil { t.Fatal(err) } task := core.TaskTemplate{ - Type: SidecarTaskType, - Custom: &sidecarCustom, + Type: UploaderTaskType, + Custom: &uploaderCustom, } tolGPU := v1.Toleration{ @@ -474,7 +474,7 @@ func TestBuildSidecarResource(t *testing.T) { DefaultCPURequest: resource.MustParse("1024m"), DefaultMemoryRequest: resource.MustParse("1024Mi"), })) - taskCtx := getDummySidecarTaskContext(&task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&task, uploaderResourceRequirements, nil) res, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.Nil(t, err) assert.EqualValues(t, map[string]string{ @@ -510,8 +510,8 @@ func TestBuildSidecarResource(t *testing.T) { assert.Equal(t, expectedEphemeralStorageLimit.Value(), res.(*v1.Pod).Spec.Containers[0].Resources.Limits.StorageEphemeral().Value()) } -func TestBuildSidecarReosurceMissingAnnotationsAndLabels(t *testing.T) { - sideCarJob := sidecarJob{ +func TestBuildUploaderReosurceMissingAnnotationsAndLabels(t *testing.T) { + sideCarJob := uploaderJob{ PrimaryContainerName: "PrimaryContainer", PodSpec: &v1.PodSpec{ Containers: []v1.Container{ @@ -522,17 +522,17 @@ func TestBuildSidecarReosurceMissingAnnotationsAndLabels(t *testing.T) { }, } - task := getSidecarTaskTemplateForTest(sideCarJob) + task := getUploaderTaskTemplateForTest(sideCarJob) - taskCtx := getDummySidecarTaskContext(task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(task, uploaderResourceRequirements, nil) resp, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.NoError(t, err) assert.EqualValues(t, map[string]string{}, resp.GetLabels()) assert.EqualValues(t, map[string]string{"primary_container_name": "PrimaryContainer"}, resp.GetAnnotations()) } -func TestBuildSidecarResourceMissingPrimary(t *testing.T) { - sideCarJob := sidecarJob{ +func TestBuildUploaderResourceMissingPrimary(t *testing.T) { + sideCarJob := uploaderJob{ PrimaryContainerName: "PrimaryContainer", PodSpec: &v1.PodSpec{ Containers: []v1.Container{ @@ -543,14 +543,14 @@ func TestBuildSidecarResourceMissingPrimary(t *testing.T) { }, } - task := getSidecarTaskTemplateForTest(sideCarJob) + task := getUploaderTaskTemplateForTest(sideCarJob) - taskCtx := getDummySidecarTaskContext(task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(task, uploaderResourceRequirements, nil) _, err := DefaultPodPlugin.BuildResource(context.TODO(), taskCtx) assert.True(t, errors.Is(err, errors2.Errorf("BadTaskSpecification", ""))) } -func TestBuildSidecarResource_ExtendedResources(t *testing.T) { +func TestBuildUploaderResource_ExtendedResources(t *testing.T) { assert.NoError(t, config.SetK8sPluginConfig(&config.K8sPluginConfig{ GpuDeviceNodeLabel: "gpu-node-label", GpuPartitionSizeNodeLabel: "gpu-partition-size", @@ -672,7 +672,7 @@ func TestBuildSidecarResource_ExtendedResources(t *testing.T) { }{ { "v0", - *getSidecarTaskTemplateForTest(sidecarJob{ + *getUploaderTaskTemplateForTest(uploaderJob{ PrimaryContainerName: podSpec.Containers[0].Name, PodSpec: &podSpec, }), @@ -680,7 +680,7 @@ func TestBuildSidecarResource_ExtendedResources(t *testing.T) { { "v1", core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, Custom: structObj, TaskTypeVersion: 1, Config: map[string]string{ @@ -691,7 +691,7 @@ func TestBuildSidecarResource_ExtendedResources(t *testing.T) { { "v2", core.TaskTemplate{ - Type: SidecarTaskType, + Type: UploaderTaskType, TaskTypeVersion: 2, Config: map[string]string{ flytek8s.PrimaryContainerKey: podSpec.Containers[0].Name, @@ -710,7 +710,7 @@ func TestBuildSidecarResource_ExtendedResources(t *testing.T) { t.Run(tCfg.name+" "+f.name, func(t *testing.T) { taskTemplate := tCfg.taskTemplate taskTemplate.ExtendedResources = f.extendedResourcesBase - taskContext := getDummySidecarTaskContext(&taskTemplate, f.resources, f.extendedResourcesOverride) + taskContext := getDummyUploaderTaskContext(&taskTemplate, f.resources, f.extendedResourcesOverride) r, err := DefaultPodPlugin.BuildResource(context.TODO(), taskContext) assert.Nil(t, err) assert.NotNil(t, r) @@ -732,8 +732,8 @@ func TestBuildSidecarResource_ExtendedResources(t *testing.T) { } } -func TestGetTaskSidecarStatus(t *testing.T) { - sideCarJob := sidecarJob{ +func TestGetTaskUploaderStatus(t *testing.T) { + sideCarJob := uploaderJob{ PrimaryContainerName: "PrimaryContainer", PodSpec: &v1.PodSpec{ Containers: []v1.Container{ @@ -744,7 +744,7 @@ func TestGetTaskSidecarStatus(t *testing.T) { }, } - task := getSidecarTaskTemplateForTest(sideCarJob) + task := getUploaderTaskTemplateForTest(sideCarJob) var testCases = map[v1.PodPhase]pluginsCore.Phase{ v1.PodSucceeded: pluginsCore.PhaseSuccess, @@ -762,7 +762,7 @@ func TestGetTaskSidecarStatus(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "PrimaryContainer", }) - taskCtx := getDummySidecarTaskContext(task, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(task, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, expectedTaskPhase, phaseInfo.Phase(), @@ -770,7 +770,7 @@ func TestGetTaskSidecarStatus(t *testing.T) { } } -func TestDemystifiedSidecarStatus_PrimaryFailed(t *testing.T) { +func TestDemystifiedUploaderStatus_PrimaryFailed(t *testing.T) { res := &v1.Pod{ Status: v1.PodStatus{ Phase: v1.PodRunning, @@ -789,13 +789,13 @@ func TestDemystifiedSidecarStatus_PrimaryFailed(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "Primary", }) - taskCtx := getDummySidecarTaskContext(&core.TaskTemplate{}, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&core.TaskTemplate{}, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, pluginsCore.PhaseRetryableFailure, phaseInfo.Phase()) } -func TestDemystifiedSidecarStatus_PrimarySucceeded(t *testing.T) { +func TestDemystifiedUploaderStatus_PrimarySucceeded(t *testing.T) { res := &v1.Pod{ Status: v1.PodStatus{ Phase: v1.PodRunning, @@ -814,13 +814,13 @@ func TestDemystifiedSidecarStatus_PrimarySucceeded(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "Primary", }) - taskCtx := getDummySidecarTaskContext(&core.TaskTemplate{}, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&core.TaskTemplate{}, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, pluginsCore.PhaseSuccess, phaseInfo.Phase()) } -func TestDemystifiedSidecarStatus_PrimaryRunning(t *testing.T) { +func TestDemystifiedUploaderStatus_PrimaryRunning(t *testing.T) { res := &v1.Pod{ Status: v1.PodStatus{ Phase: v1.PodRunning, @@ -839,13 +839,13 @@ func TestDemystifiedSidecarStatus_PrimaryRunning(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "Primary", }) - taskCtx := getDummySidecarTaskContext(&core.TaskTemplate{}, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&core.TaskTemplate{}, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, pluginsCore.PhaseRunning, phaseInfo.Phase()) } -func TestDemystifiedSidecarStatus_PrimaryMissing(t *testing.T) { +func TestDemystifiedUploaderStatus_PrimaryMissing(t *testing.T) { res := &v1.Pod{ Spec: v1.PodSpec{ Containers: []v1.Container{ @@ -866,13 +866,13 @@ func TestDemystifiedSidecarStatus_PrimaryMissing(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "Primary", }) - taskCtx := getDummySidecarTaskContext(&core.TaskTemplate{}, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&core.TaskTemplate{}, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, pluginsCore.PhasePermanentFailure, phaseInfo.Phase()) } -func TestDemystifiedSidecarStatus_PrimaryNotExistsYet(t *testing.T) { +func TestDemystifiedUploaderStatus_PrimaryNotExistsYet(t *testing.T) { res := &v1.Pod{ Spec: v1.PodSpec{ Containers: []v1.Container{ @@ -893,7 +893,7 @@ func TestDemystifiedSidecarStatus_PrimaryNotExistsYet(t *testing.T) { res.SetAnnotations(map[string]string{ flytek8s.PrimaryContainerKey: "Primary", }) - taskCtx := getDummySidecarTaskContext(&core.TaskTemplate{}, sidecarResourceRequirements, nil) + taskCtx := getDummyUploaderTaskContext(&core.TaskTemplate{}, uploaderResourceRequirements, nil) phaseInfo, err := DefaultPodPlugin.GetTaskPhase(context.TODO(), taskCtx, res) assert.Nil(t, err) assert.Equal(t, pluginsCore.PhaseRunning, phaseInfo.Phase()) diff --git a/flyteplugins/go/tasks/plugins/k8s/ray/config.go b/flyteplugins/go/tasks/plugins/k8s/ray/config.go index 3f79ed958e..9bc4bc551a 100644 --- a/flyteplugins/go/tasks/plugins/k8s/ray/config.go +++ b/flyteplugins/go/tasks/plugins/k8s/ray/config.go @@ -82,7 +82,7 @@ type Config struct { // Remote Ray Cluster Config RemoteClusterConfig pluginmachinery.ClusterConfig `json:"remoteClusterConfig" pflag:"Configuration of remote K8s cluster for ray jobs"` Logs logs.LogConfig `json:"logs" pflag:"-,Log configuration for ray jobs"` - LogsSidecar *v1.Container `json:"logsSidecar" pflag:"-,Sidecar to inject into head pods for capturing ray job logs"` + LogsUploader *v1.Container `json:"logsUploader" pflag:"-,Uploader to inject into head pods for capturing ray job logs"` DashboardURLTemplate *tasklog.TemplateLogPlugin `json:"dashboardURLTemplate" pflag:"-,Template for URL of Ray dashboard running on a head node."` Defaults DefaultConfig `json:"defaults" pflag:"-,Default configuration for ray jobs"` EnableUsageStats bool `json:"enableUsageStats" pflag:",Enable usage stats for ray jobs. These stats are submitted to usage-stats.ray.io per https://docs.ray.io/en/latest/cluster/usage-stats.html"` diff --git a/flyteplugins/go/tasks/plugins/k8s/ray/ray.go b/flyteplugins/go/tasks/plugins/k8s/ray/ray.go index c1e5dd264d..f499e3fb68 100644 --- a/flyteplugins/go/tasks/plugins/k8s/ray/ray.go +++ b/flyteplugins/go/tasks/plugins/k8s/ray/ray.go @@ -272,12 +272,12 @@ func convertBase64RuntimeEnvToYaml(s string) (string, error) { return string(y), nil } -func injectLogsSidecar(primaryContainer *v1.Container, podSpec *v1.PodSpec) { +func injectLogsUploader(primaryContainer *v1.Container, podSpec *v1.PodSpec) { cfg := GetConfig() - if cfg.LogsSidecar == nil { + if cfg.LogsUploader == nil { return } - sidecar := cfg.LogsSidecar.DeepCopy() + uploader := cfg.LogsUploader.DeepCopy() // Ray logs integration var rayStateVolMount *v1.VolumeMount @@ -306,27 +306,27 @@ func injectLogsSidecar(primaryContainer *v1.Container, podSpec *v1.PodSpec) { primaryContainer.VolumeMounts = append(primaryContainer.VolumeMounts, volMount) rayStateVolMount = &volMount } - // We need to mirror the ray state volume mount into the sidecar as readonly, + // We need to mirror the ray state volume mount into the uploader as readonly, // so that we can read the logs written by the head node. readOnlyRayStateVolMount := *rayStateVolMount.DeepCopy() readOnlyRayStateVolMount.ReadOnly = true - // Update volume mounts on sidecar + // Update volume mounts on uploader // If one already exists with the desired mount path, simply replace it. Otherwise, - // add it to sidecar's volume mounts. - foundExistingSidecarVolMount := false - for idx, vm := range sidecar.VolumeMounts { + // add it to uploader's volume mounts. + foundExistingUploaderVolMount := false + for idx, vm := range uploader.VolumeMounts { if vm.MountPath == rayStateMountPath { - foundExistingSidecarVolMount = true - sidecar.VolumeMounts[idx] = readOnlyRayStateVolMount + foundExistingUploaderVolMount = true + uploader.VolumeMounts[idx] = readOnlyRayStateVolMount } } - if !foundExistingSidecarVolMount { - sidecar.VolumeMounts = append(sidecar.VolumeMounts, readOnlyRayStateVolMount) + if !foundExistingUploaderVolMount { + uploader.VolumeMounts = append(uploader.VolumeMounts, readOnlyRayStateVolMount) } - // Add sidecar to containers - podSpec.Containers = append(podSpec.Containers, *sidecar) + // Add uploader to containers + podSpec.Containers = append(podSpec.Containers, *uploader) } func buildHeadPodTemplate(primaryContainer *v1.Container, basePodSpec *v1.PodSpec, objectMeta *metav1.ObjectMeta, taskCtx pluginsCore.TaskExecutionContext, spec *plugins.HeadGroupSpec) (v1.PodTemplateSpec, error) { @@ -366,8 +366,8 @@ func buildHeadPodTemplate(primaryContainer *v1.Container, basePodSpec *v1.PodSpe primaryContainer.Ports = append(primaryContainer.Ports, ports...) - // Inject a sidecar for capturing and exposing Ray job logs - injectLogsSidecar(primaryContainer, basePodSpec) + // Inject a uploader for capturing and exposing Ray job logs + injectLogsUploader(primaryContainer, basePodSpec) basePodSpec, err := mergeCustomPodSpec(primaryContainer, basePodSpec, spec.GetK8SPod()) if err != nil { diff --git a/flyteplugins/go/tasks/plugins/k8s/ray/ray_test.go b/flyteplugins/go/tasks/plugins/k8s/ray/ray_test.go index 708939485b..eb9165c47e 100644 --- a/flyteplugins/go/tasks/plugins/k8s/ray/ray_test.go +++ b/flyteplugins/go/tasks/plugins/k8s/ray/ray_test.go @@ -592,16 +592,16 @@ func TestDefaultStartParameters(t *testing.T) { assert.Equal(t, ray.Spec.RayClusterSpec.WorkerGroupSpecs[0].Template.Spec.Tolerations, toleration) } -func TestInjectLogsSidecar(t *testing.T) { +func TestInjectLogsUploader(t *testing.T) { rayJobObj := transformRayJobToCustomObj(dummyRayCustomObj()) params := []struct { name string taskTemplate core.TaskTemplate // primaryContainerName string - logsSidecarCfg *corev1.Container + logsUploaderCfg *corev1.Container expectedVolumes []corev1.Volume expectedPrimaryContainerVolumeMounts []corev1.VolumeMount - expectedLogsSidecarVolumeMounts []corev1.VolumeMount + expectedLogsUploaderVolumeMounts []corev1.VolumeMount }{ { "container target", @@ -616,7 +616,7 @@ func TestInjectLogsSidecar(t *testing.T) { Custom: rayJobObj, }, &corev1.Container{ - Name: "logs-sidecar", + Name: "logs-uploader", Image: "test-image", }, []corev1.Volume{ @@ -642,7 +642,7 @@ func TestInjectLogsSidecar(t *testing.T) { }, }, { - "container target with no sidecar", + "container target with no uploader", core.TaskTemplate{ Id: &core.Identifier{Name: "ray-id"}, Target: &core.TaskTemplate_Container{ @@ -676,7 +676,7 @@ func TestInjectLogsSidecar(t *testing.T) { }, }, &corev1.Container{ - Name: "logs-sidecar", + Name: "logs-uploader", Image: "test-image", }, []corev1.Volume{ @@ -733,7 +733,7 @@ func TestInjectLogsSidecar(t *testing.T) { }, }, &corev1.Container{ - Name: "logs-sidecar", + Name: "logs-uploader", Image: "test-image", }, []corev1.Volume{ @@ -763,7 +763,7 @@ func TestInjectLogsSidecar(t *testing.T) { for _, p := range params { t.Run(p.name, func(t *testing.T) { assert.NoError(t, SetConfig(&Config{ - LogsSidecar: p.logsSidecarCfg, + LogsUploader: p.logsUploaderCfg, })) taskContext := dummyRayTaskContext(&p.taskTemplate, resourceRequirements, nil, "", serviceAccount) rayJobResourceHandler := rayJobResourceHandler{} @@ -780,7 +780,7 @@ func TestInjectLogsSidecar(t *testing.T) { // Check containers and respective volume mounts foundPrimaryContainer := false - foundLogsSidecar := false + foundLogsUploader := false for _, cnt := range headPodSpec.Containers { if cnt.Name == "ray-head" { foundPrimaryContainer = true @@ -790,17 +790,17 @@ func TestInjectLogsSidecar(t *testing.T) { cnt.VolumeMounts, ) } - if p.logsSidecarCfg != nil && cnt.Name == p.logsSidecarCfg.Name { - foundLogsSidecar = true + if p.logsUploaderCfg != nil && cnt.Name == p.logsUploaderCfg.Name { + foundLogsUploader = true assert.EqualValues( t, - p.expectedLogsSidecarVolumeMounts, + p.expectedLogsUploaderVolumeMounts, cnt.VolumeMounts, ) } } assert.Equal(t, true, foundPrimaryContainer) - assert.Equal(t, p.logsSidecarCfg != nil, foundLogsSidecar) + assert.Equal(t, p.logsUploaderCfg != nil, foundLogsUploader) }) } } diff --git a/flytepropeller/pkg/controller/nodes/task/k8s/plugin_collector.go b/flytepropeller/pkg/controller/nodes/task/k8s/plugin_collector.go index f0681a6bca..ca9d1219a8 100644 --- a/flytepropeller/pkg/controller/nodes/task/k8s/plugin_collector.go +++ b/flytepropeller/pkg/controller/nodes/task/k8s/plugin_collector.go @@ -106,7 +106,7 @@ func (r *ResourceLevelMonitor) RunCollectorOnce(ctx context.Context) { // This struct is here to ensure that we do not create more than one of these monitors for a given GVK. It wouldn't necessarily break // anything, but it's a waste of compute cycles to compute counts multiple times. This can happen if multiple plugins create the same -// underlying K8s resource type. If two plugins both created Pods (ie sidecar and container), without this we would launch two +// underlying K8s resource type. If two plugins both created Pods (ie uploader and container), without this we would launch two // ResourceLevelMonitor's, have two goroutines spinning, etc. type ResourceMonitorIndex struct { lock *sync.Mutex diff --git a/flytepropeller/pkg/controller/workflow/executor_test.go b/flytepropeller/pkg/controller/workflow/executor_test.go index 187aac7ead..051d3bd793 100644 --- a/flytepropeller/pkg/controller/workflow/executor_test.go +++ b/flytepropeller/pkg/controller/workflow/executor_test.go @@ -235,7 +235,7 @@ func TestWorkflowExecutor_HandleFlyteWorkflow_Error(t *testing.T) { "python-task": "pod", "container": "pod", "raw-container": "pod", - "sidecar": "pod", + "uploader": "pod", } store := createInmemoryDataStore(t, scope.NewSubScope("data_store")) diff --git a/flytepropeller/pkg/webhook/aws_secret_manager.go b/flytepropeller/pkg/webhook/aws_secret_manager.go index ad5e8c48f4..45250bc0df 100644 --- a/flytepropeller/pkg/webhook/aws_secret_manager.go +++ b/flytepropeller/pkg/webhook/aws_secret_manager.go @@ -15,19 +15,19 @@ import ( ) const ( - // AWSSecretArnEnvVar defines the environment variable name to use to specify to the sidecar container which secret + // AWSSecretArnEnvVar defines the environment variable name to use to specify to the uploader container which secret // to pull. AWSSecretArnEnvVar = "SECRET_ARN" - // AWSSecretFilenameEnvVar defines the environment variable name to use to specify to the sidecar container where + // AWSSecretFilenameEnvVar defines the environment variable name to use to specify to the uploader container where // to store the secret. AWSSecretFilenameEnvVar = "SECRET_FILENAME" // AWSSecretsVolumeName defines the static name of the volume used for mounting/sharing secrets between init-container - // sidecar and the rest of the containers in the pod. + // uploader and the rest of the containers in the pod. AWSSecretsVolumeName = "aws-secret-vol" // #nosec - // AWS SideCar Docker Container expects the mount to always be under /tmp + // AWS Uploader Docker Container expects the mount to always be under /tmp AWSInitContainerMountPath = "/tmp" ) @@ -36,7 +36,7 @@ var ( AWSSecretMountPathPrefix = []string{string(os.PathSeparator), "etc", "flyte", "secrets"} ) -// AWSSecretManagerInjector allows injecting of secrets from AWS Secret Manager as files. It uses AWS-provided SideCar +// AWSSecretManagerInjector allows injecting of secrets from AWS Secret Manager as files. It uses AWS-provided Uploader // as an init-container to download the secret and save it to a local volume shared with all other containers in the pod. // It supports multiple secrets to be mounted but that will result into adding an init container for each secret. // The role/serviceaccount used to run the Pod must have permissions to pull the secret from AWS Secret Manager. @@ -80,7 +80,7 @@ func (i AWSSecretManagerInjector) Inject(ctx context.Context, secret *core.Secre } p.Spec.Volumes = appendVolumeIfNotExists(p.Spec.Volumes, vol) - p.Spec.InitContainers = append(p.Spec.InitContainers, createAWSSidecarContainer(i.cfg, p, secret)) + p.Spec.InitContainers = append(p.Spec.InitContainers, createAWSUploaderContainer(i.cfg, p, secret)) secretVolumeMount := corev1.VolumeMount{ Name: AWSSecretsVolumeName, @@ -120,9 +120,9 @@ func (i AWSSecretManagerInjector) Inject(ctx context.Context, secret *core.Secre return p, true, nil } -func createAWSSidecarContainer(cfg config.AWSSecretManagerConfig, p *corev1.Pod, secret *core.Secret) corev1.Container { +func createAWSUploaderContainer(cfg config.AWSSecretManagerConfig, p *corev1.Pod, secret *core.Secret) corev1.Container { return corev1.Container{ - Image: cfg.SidecarImage, + Image: cfg.UploaderImage, // Create a unique name to allow multiple secrets to be mounted. Name: formatAWSInitContainerName(len(p.Spec.InitContainers)), VolumeMounts: []corev1.VolumeMount{ diff --git a/flytepropeller/pkg/webhook/aws_secret_manager_test.go b/flytepropeller/pkg/webhook/aws_secret_manager_test.go index cb0c9ddba4..b752ee6fce 100644 --- a/flytepropeller/pkg/webhook/aws_secret_manager_test.go +++ b/flytepropeller/pkg/webhook/aws_secret_manager_test.go @@ -40,7 +40,7 @@ func TestAWSSecretManagerInjector_Inject(t *testing.T) { InitContainers: []corev1.Container{ { Name: "aws-pull-secret-0", - Image: "docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4", + Image: "docker.io/amazon/aws-secrets-manager-secret-uploader:v0.1.4", Env: []corev1.EnvVar{ { Name: "SECRET_ARN", diff --git a/flytepropeller/pkg/webhook/config/config.go b/flytepropeller/pkg/webhook/config/config.go index 71e901ad5b..bf812a9b56 100644 --- a/flytepropeller/pkg/webhook/config/config.go +++ b/flytepropeller/pkg/webhook/config/config.go @@ -24,7 +24,7 @@ var ( ListenPort: 9443, SecretManagerType: SecretManagerTypeK8s, AWSSecretManagerConfig: AWSSecretManagerConfig{ - SidecarImage: "docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4", + UploaderImage: "docker.io/amazon/aws-secrets-manager-secret-uploader:v0.1.4", Resources: corev1.ResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceMemory: resource.MustParse("500Mi"), @@ -37,7 +37,7 @@ var ( }, }, GCPSecretManagerConfig: GCPSecretManagerConfig{ - SidecarImage: "gcr.io/google.com/cloudsdktool/cloud-sdk:alpine", + UploaderImage: "gcr.io/google.com/cloudsdktool/cloud-sdk:alpine", Resources: corev1.ResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceMemory: resource.MustParse("500Mi"), @@ -110,13 +110,13 @@ func (c Config) ExpandCertDir() string { } type AWSSecretManagerConfig struct { - SidecarImage string `json:"sidecarImage" pflag:",Specifies the sidecar docker image to use"` - Resources corev1.ResourceRequirements `json:"resources" pflag:"-,Specifies resource requirements for the init container."` + UploaderImage string `json:"uploaderImage" pflag:",Specifies the uploader docker image to use"` + Resources corev1.ResourceRequirements `json:"resources" pflag:"-,Specifies resource requirements for the init container."` } type GCPSecretManagerConfig struct { - SidecarImage string `json:"sidecarImage" pflag:",Specifies the sidecar docker image to use"` - Resources corev1.ResourceRequirements `json:"resources" pflag:"-,Specifies resource requirements for the init container."` + UploaderImage string `json:"uploaderImage" pflag:",Specifies the uploader docker image to use"` + Resources corev1.ResourceRequirements `json:"resources" pflag:"-,Specifies resource requirements for the init container."` } type VaultSecretManagerConfig struct { diff --git a/flytepropeller/pkg/webhook/config/config_flags.go b/flytepropeller/pkg/webhook/config/config_flags.go index 089bc00641..39a6e616e2 100755 --- a/flytepropeller/pkg/webhook/config/config_flags.go +++ b/flytepropeller/pkg/webhook/config/config_flags.go @@ -57,8 +57,8 @@ func (cfg Config) GetPFlagSet(prefix string) *pflag.FlagSet { cmdFlags.String(fmt.Sprintf("%v%v", prefix, "serviceName"), DefaultConfig.ServiceName, "The name of the webhook service.") cmdFlags.Int32(fmt.Sprintf("%v%v", prefix, "servicePort"), DefaultConfig.ServicePort, "The port on the service that hosting webhook.") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "secretName"), DefaultConfig.SecretName, "Secret name to write generated certs to.") - cmdFlags.String(fmt.Sprintf("%v%v", prefix, "awsSecretManager.sidecarImage"), DefaultConfig.AWSSecretManagerConfig.SidecarImage, "Specifies the sidecar docker image to use") - cmdFlags.String(fmt.Sprintf("%v%v", prefix, "gcpSecretManager.sidecarImage"), DefaultConfig.GCPSecretManagerConfig.SidecarImage, "Specifies the sidecar docker image to use") + cmdFlags.String(fmt.Sprintf("%v%v", prefix, "awsSecretManager.uploaderImage"), DefaultConfig.AWSSecretManagerConfig.UploaderImage, "Specifies the uploader docker image to use") + cmdFlags.String(fmt.Sprintf("%v%v", prefix, "gcpSecretManager.uploaderImage"), DefaultConfig.GCPSecretManagerConfig.UploaderImage, "Specifies the uploader docker image to use") cmdFlags.String(fmt.Sprintf("%v%v", prefix, "vaultSecretManager.role"), DefaultConfig.VaultSecretManagerConfig.Role, "Specifies the vault role to use") return cmdFlags } diff --git a/flytepropeller/pkg/webhook/config/config_flags_test.go b/flytepropeller/pkg/webhook/config/config_flags_test.go index 613a0f6a3b..b09d3adca5 100755 --- a/flytepropeller/pkg/webhook/config/config_flags_test.go +++ b/flytepropeller/pkg/webhook/config/config_flags_test.go @@ -197,28 +197,28 @@ func TestConfig_SetFlags(t *testing.T) { } }) }) - t.Run("Test_awsSecretManager.sidecarImage", func(t *testing.T) { + t.Run("Test_awsSecretManager.uploaderImage", func(t *testing.T) { t.Run("Override", func(t *testing.T) { testValue := "1" - cmdFlags.Set("awsSecretManager.sidecarImage", testValue) - if vString, err := cmdFlags.GetString("awsSecretManager.sidecarImage"); err == nil { - testDecodeJson_Config(t, fmt.Sprintf("%v", vString), &actual.AWSSecretManagerConfig.SidecarImage) + cmdFlags.Set("awsSecretManager.uploaderImage", testValue) + if vString, err := cmdFlags.GetString("awsSecretManager.uploaderImage"); err == nil { + testDecodeJson_Config(t, fmt.Sprintf("%v", vString), &actual.AWSSecretManagerConfig.UploaderImage) } else { assert.FailNow(t, err.Error()) } }) }) - t.Run("Test_gcpSecretManager.sidecarImage", func(t *testing.T) { + t.Run("Test_gcpSecretManager.uploaderImage", func(t *testing.T) { t.Run("Override", func(t *testing.T) { testValue := "1" - cmdFlags.Set("gcpSecretManager.sidecarImage", testValue) - if vString, err := cmdFlags.GetString("gcpSecretManager.sidecarImage"); err == nil { - testDecodeJson_Config(t, fmt.Sprintf("%v", vString), &actual.GCPSecretManagerConfig.SidecarImage) + cmdFlags.Set("gcpSecretManager.uploaderImage", testValue) + if vString, err := cmdFlags.GetString("gcpSecretManager.uploaderImage"); err == nil { + testDecodeJson_Config(t, fmt.Sprintf("%v", vString), &actual.GCPSecretManagerConfig.UploaderImage) } else { assert.FailNow(t, err.Error()) diff --git a/flytepropeller/pkg/webhook/gcp_secret_manager.go b/flytepropeller/pkg/webhook/gcp_secret_manager.go index 4db4a0d3ab..6983cfaf8d 100644 --- a/flytepropeller/pkg/webhook/gcp_secret_manager.go +++ b/flytepropeller/pkg/webhook/gcp_secret_manager.go @@ -16,7 +16,7 @@ import ( const ( // GCPSecretsVolumeName defines the static name of the volume used for mounting/sharing secrets between init-container - // sidecar and the rest of the containers in the pod. + // uploader and the rest of the containers in the pod. GCPSecretsVolumeName = "gcp-secret-vol" // #nosec ) @@ -26,7 +26,7 @@ var ( ) // GCPSecretManagerInjector allows injecting of secrets from GCP Secret Manager as files. It uses a Google Cloud -// SDK SideCar as an init-container to download the secret and save it to a local volume shared with all other +// SDK Uploader as an init-container to download the secret and save it to a local volume shared with all other // containers in the pod. It supports multiple secrets to be mounted but that will result into adding an init // container for each secret. The Google serviceaccount (GSA) associated with the Pod, either via Workload // Identity (recommended) or the underlying node's serviceacccount, must have permissions to pull the secret @@ -83,7 +83,7 @@ func (i GCPSecretManagerInjector) Inject(ctx context.Context, secret *core.Secre } p.Spec.Volumes = appendVolumeIfNotExists(p.Spec.Volumes, vol) - p.Spec.InitContainers = append(p.Spec.InitContainers, createGCPSidecarContainer(i.cfg, p, secret)) + p.Spec.InitContainers = append(p.Spec.InitContainers, createGCPUploaderContainer(i.cfg, p, secret)) secretVolumeMount := corev1.VolumeMount{ Name: GCPSecretsVolumeName, @@ -123,9 +123,9 @@ func (i GCPSecretManagerInjector) Inject(ctx context.Context, secret *core.Secre return p, true, nil } -func createGCPSidecarContainer(cfg config.GCPSecretManagerConfig, p *corev1.Pod, secret *core.Secret) corev1.Container { +func createGCPUploaderContainer(cfg config.GCPSecretManagerConfig, p *corev1.Pod, secret *core.Secret) corev1.Container { return corev1.Container{ - Image: cfg.SidecarImage, + Image: cfg.UploaderImage, // Create a unique name to allow multiple secrets to be mounted. Name: formatGCPInitContainerName(len(p.Spec.InitContainers)), Command: formatGCPSecretAccessCommand(secret), diff --git a/flytepropeller/pkg/webhook/pod.go b/flytepropeller/pkg/webhook/pod.go index 9e3a300bf6..23be4c3b1c 100644 --- a/flytepropeller/pkg/webhook/pod.go +++ b/flytepropeller/pkg/webhook/pod.go @@ -24,7 +24,7 @@ // stated in the flyteIdl.Secret object) into the Pod. There is no validation that the secret exist and is available // to the Pod at this point. If the secret is not accessible, the Pod will fail with ContainerCreationConfigError and // will be retried. -// - For Vault secrets, it'll inject the right annotations to trigger Vault's own sidecar/webhook to mount the secret. +// - For Vault secrets, it'll inject the right annotations to trigger Vault's own uploader/webhook to mount the secret. package webhook import ( diff --git a/flytepropeller/propeller-config.yaml b/flytepropeller/propeller-config.yaml index 4a04dc5087..9b94d3f6f6 100644 --- a/flytepropeller/propeller-config.yaml +++ b/flytepropeller/propeller-config.yaml @@ -31,7 +31,7 @@ tasks: task-plugins: enabled-plugins: - container - - sidecar + - uploader - K8S-ARRAY - echo - agent-service