From afe078395ce24392b1b7f6155bc425bfc44c818e Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 21 Sep 2023 13:15:20 +0000 Subject: [PATCH 001/124] branch:HPCC-27615-original-folder-structure. Constructed the original folder structure by replacing the subfolders of modules, i.e. aks, logging, storage, and vnet, with the folders in the root by the same names, i.e. aks, logging, storage, and vnet. Also, placed the contents of the folder, 'hpcc' in the root and deleted the folder. --- aks/LICENSE => LICENSE | 0 README.md | 750 +++++++++++++++++- ...tfvars.example => dali.auto.tfvars.example | 0 hpcc/data.tf => data.tf | 0 ...example => eclccserver.auto.tfvars.example | 0 hpcc/hpcc.tf => hpcc.tf | 3 +- hpcc/README.md | 732 ----------------- hpcc/locals.tf => locals.tf | 0 hpcc/main.tf => main.tf | 0 ...tfvars.example => misc.auto.tfvars.example | 0 {hpcc => modules/aks}/LICENSE | 0 {aks => modules/aks}/aks.auto.tfvars.example | 0 {aks => modules/aks}/aks.tf | 4 +- {aks => modules/aks}/automation.tf | 0 {aks => modules/aks}/data.tf | 0 {aks => modules/aks}/locals.tf | 0 {aks => modules/aks}/main.tf | 0 {aks => modules/aks}/misc.auto.tfvars.example | 0 {aks => modules/aks}/outputs.tf | 0 {aks => modules/aks}/providers.tf | 0 {aks => modules/aks}/scripts/start_stop.ps1 | 0 {aks => modules/aks}/variables.tf | 0 {aks => modules/aks}/versions.tf | 0 {logging => modules/logging}/LICENSE | 0 {logging => modules/logging}/data.tf | 0 .../logging}/elastic4hpcc.auto.tfvars.example | 0 {logging => modules/logging}/locals.tf | 0 .../log_analytics.auto.tfvars.example | 0 {logging => modules/logging}/logging.tf | 0 {logging => modules/logging}/main.tf | 0 .../logging}/misc.auto.tfvars.example | 0 {logging => modules/logging}/outputs.tf | 0 {logging => modules/logging}/providers.tf | 0 {logging => modules/logging}/variables.tf | 0 {logging => modules/logging}/versions.tf | 0 {storage => modules/storage}/README.md | 0 {storage => modules/storage}/data.tf | 0 {storage => modules/storage}/locals.tf | 0 {storage => modules/storage}/main.tf | 3 +- .../storage}/misc.auto.tfvars.example | 0 {storage => modules/storage}/outputs.tf | 0 {storage => modules/storage}/providers.tf | 0 .../storage}/storage.auto.tfvars.example | 0 {storage => modules/storage}/variables.tf | 0 {storage => modules/storage}/versions.tf | 0 {vnet => modules/vnet}/LICENSE | 0 {vnet => modules/vnet}/README.md | 0 {vnet => modules/vnet}/data.tf | 0 {vnet => modules/vnet}/locals.tf | 0 {vnet => modules/vnet}/main.tf | 0 .../vnet}/misc.auto.tfvars.example | 0 {vnet => modules/vnet}/outputs.tf | 0 {vnet => modules/vnet}/providers.tf | 0 {vnet => modules/vnet}/variables.tf | 0 {vnet => modules/vnet}/versions.tf | 0 {vnet => modules/vnet}/vnet.tf | 0 hpcc/outputs.tf => outputs.tf | 0 hpcc/providers.tf => providers.tf | 0 ...fvars.example => roxie.auto.tfvars.example | 0 ...fvars.example => sasha.auto.tfvars.example | 0 ...tfvars.example => thor.auto.tfvars.example | 0 hpcc/variables.tf => variables.tf | 0 hpcc/versions.tf => versions.tf | 0 63 files changed, 738 insertions(+), 754 deletions(-) rename aks/LICENSE => LICENSE (100%) rename hpcc/dali.auto.tfvars.example => dali.auto.tfvars.example (100%) rename hpcc/data.tf => data.tf (100%) rename hpcc/eclccserver.auto.tfvars.example => eclccserver.auto.tfvars.example (100%) rename hpcc/hpcc.tf => hpcc.tf (95%) delete mode 100644 hpcc/README.md rename hpcc/locals.tf => locals.tf (100%) rename hpcc/main.tf => main.tf (100%) rename hpcc/misc.auto.tfvars.example => misc.auto.tfvars.example (100%) rename {hpcc => modules/aks}/LICENSE (100%) rename {aks => modules/aks}/aks.auto.tfvars.example (100%) rename {aks => modules/aks}/aks.tf (95%) rename {aks => modules/aks}/automation.tf (100%) rename {aks => modules/aks}/data.tf (100%) rename {aks => modules/aks}/locals.tf (100%) rename {aks => modules/aks}/main.tf (100%) rename {aks => modules/aks}/misc.auto.tfvars.example (100%) rename {aks => modules/aks}/outputs.tf (100%) rename {aks => modules/aks}/providers.tf (100%) rename {aks => modules/aks}/scripts/start_stop.ps1 (100%) rename {aks => modules/aks}/variables.tf (100%) rename {aks => modules/aks}/versions.tf (100%) rename {logging => modules/logging}/LICENSE (100%) rename {logging => modules/logging}/data.tf (100%) rename {logging => modules/logging}/elastic4hpcc.auto.tfvars.example (100%) rename {logging => modules/logging}/locals.tf (100%) rename {logging => modules/logging}/log_analytics.auto.tfvars.example (100%) rename {logging => modules/logging}/logging.tf (100%) rename {logging => modules/logging}/main.tf (100%) rename {logging => modules/logging}/misc.auto.tfvars.example (100%) rename {logging => modules/logging}/outputs.tf (100%) rename {logging => modules/logging}/providers.tf (100%) rename {logging => modules/logging}/variables.tf (100%) rename {logging => modules/logging}/versions.tf (100%) rename {storage => modules/storage}/README.md (100%) rename {storage => modules/storage}/data.tf (100%) rename {storage => modules/storage}/locals.tf (100%) rename {storage => modules/storage}/main.tf (62%) rename {storage => modules/storage}/misc.auto.tfvars.example (100%) rename {storage => modules/storage}/outputs.tf (100%) rename {storage => modules/storage}/providers.tf (100%) rename {storage => modules/storage}/storage.auto.tfvars.example (100%) rename {storage => modules/storage}/variables.tf (100%) rename {storage => modules/storage}/versions.tf (100%) rename {vnet => modules/vnet}/LICENSE (100%) rename {vnet => modules/vnet}/README.md (100%) rename {vnet => modules/vnet}/data.tf (100%) rename {vnet => modules/vnet}/locals.tf (100%) rename {vnet => modules/vnet}/main.tf (100%) rename {vnet => modules/vnet}/misc.auto.tfvars.example (100%) rename {vnet => modules/vnet}/outputs.tf (100%) rename {vnet => modules/vnet}/providers.tf (100%) rename {vnet => modules/vnet}/variables.tf (100%) rename {vnet => modules/vnet}/versions.tf (100%) rename {vnet => modules/vnet}/vnet.tf (100%) rename hpcc/outputs.tf => outputs.tf (100%) rename hpcc/providers.tf => providers.tf (100%) rename hpcc/roxie.auto.tfvars.example => roxie.auto.tfvars.example (100%) rename hpcc/sasha.auto.tfvars.example => sasha.auto.tfvars.example (100%) rename hpcc/thor.auto.tfvars.example => thor.auto.tfvars.example (100%) rename hpcc/variables.tf => variables.tf (100%) rename hpcc/versions.tf => versions.tf (100%) diff --git a/aks/LICENSE b/LICENSE similarity index 100% rename from aks/LICENSE rename to LICENSE diff --git a/README.md b/README.md index f3128ac..67ef75a 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,732 @@ -# Deploy HPCC Systems Platform with Terraform - -This set of Terraform examples deploys all the available features that come with the HPCC Systems OSS Terraform modules. - -## Order of deployment -| Order | Name | Required | -| ----- | --------- | :------: | -| 1 | `VNet` | yes | -| 2 | `AKS` | yes | -| 3 | `Storage` | no | -| 4 | `Logging` | no | -| 5 | `AKS` | no | -| 6 | `HPCC` | yes | - -## Modules -| Name | Source | Used in | -| :---------------------------------: | :----: | :-----: | -| `terraform-azurerm-virtual-network` | | `VNet` | \ No newline at end of file +# Azure - HPCC AKS Root Module +
+ +This module is intended as an example for development and test systems only. It can be used as a blueprint to develop your own production version that meets your organization's security requirements. +
+
+ +## Introduction + +This module deploys an HPCC AKS cluster using remote modules that are listed below. +
+ +## Remote Modules +These are the list of all the remote modules. + +| Name | Description | URL | Required | +| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------- | :------: | +| subscription | Queries enabled azure subscription from host machine | https://github.com/Azure-Terraform/terraform-azurerm-subscription-data.git | yes | +| naming | Enforces naming conventions | - | yes | +| metadata | Provides metadata | https://github.com/Azure-Terraform/terraform-azurerm-metadata.git | yes | +| resource_group | Creates a resource group | https://github.com/Azure-Terraform/terraform-azurerm-resource-group.git | yes | +| virtual_network | Creates a virtual network | https://github.com/Azure-Terraform/terraform-azurerm-virtual-network.git | yes | +| kubernetes | Creates an Azure Kubernetes Service Cluster | https://github.com/Azure-Terraform/terraform-azurerm-kubernetes.git | yes | +
+ +## Supported Arguments +
+ +### The `admin` block: +This block contains information on the user who is deploying the cluster. This is used as tags and part of some resource names to identify who deployed a given resource and how to contact that user. This block is required. + +| Name | Description | Type | Default | Required | +| ----- | ---------------------------- | ------ | ------- | :------: | +| name | Name of the admin. | string | - | yes | +| email | Email address for the admin. | string | - | yes | + +
+Usage Example: +
+ + admin = { + name = "Example" + email = "example@hpccdemo.com" + } +
+ +### The `disable_naming_conventions` block: +When set to `true`, this attribute drops the naming conventions set forth by the python module. This attribute is optional. + + | Name | Description | Type | Default | Required | + | -------------------------- | --------------------------- | ---- | ------- | :------: | + | disable_naming_conventions | Disable naming conventions. | bool | `false` | no | +
+ +### The `metadata` block: +TThe arguments in this block are used as tags and part of resources’ names. This block can be omitted when disable_naming_conventions is set to `true`. + + | Name | Description | Type | Default | Required | + | ------------------- | ---------------------------- | ------ | ------- | :------: | + | project_name | Name of the project. | string | "" | yes | + | product_name | Name of the product. | string | hpcc | no | + | business_unit | Name of your bussiness unit. | string | "" | no | + | environment | Name of the environment. | string | "" | no | + | market | Name of market. | string | "" | no | + | product_group | Name of product group. | string | "" | no | + | resource_group_type | Resource group type. | string | "" | no | + | sre_team | Name of SRE team. | string | "" | no | + | subscription_type | Subscription type. | string | "" | no | +
+ +Usage Example: +
+ + metadata = { + project = "hpccdemo" + product_name = "example" + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = "contoso" + resource_group_type = "app" + sre_team = "hpccplatform" + subscription_type = "dev" + } + +
+ +### The `tags` argument: +The tag attribute can be used for additional tags. The tags must be key value pairs. This block is optional. + + | Name | Description | Type | Default | Required | + | ---- | ------------------------- | ----------- | ------- | :------: | + | tags | Additional resource tags. | map(string) | admin | no | +
+ +### The `resource_group` block: +This block creates a resource group (like a folder) for your resources. This block is required. + + | Name | Description | Type | Default | Required | + | ----------- | ----------------------------------------------------------------- | ---- | ------- | :------: | + | unique_name | Will concatenate a number at the end of your resource group name. | bool | `true` | yes | +
+ +Usage Example: +
+ + resource_group = { + unique_name = true + } + +
+ +### The `virtual_network` block: +This block imports metadata of a virtual network deployed outside of this project. This block is optional. + + | Name | Description | Type | Default | Required | + | ----------------- | --------------------------------------- | ------ | ------- | :------: | + | private_subnet_id | The ID of the private subnet. | string | - | yes | + | public_subnet_id | The ID of the public subnet. | string | - | yes | + | route_table_id | The ID of the route table for the AKS. | string | - | yes | + | location | The location of the virtual network | string | - | yes | +
+ +Usage Example: +
+ + virtual_network = { + private_subnet_id = "" + public_subnet_id = "" + route_table_id = "" + location = "" + } + +
+ +## The `node_pools` block: +The `node-pools` block supports the following arguments:
+`system` - (Required) The system or default node pool. This node pool hosts the system pods by default. The possible arguments for this block are defined below. + +`addpool` - (Required) The additional node pool configuration. This block name is changeable and must be unique across all additional node pools. At least one additional node pool is required. The possible arguments for this block are defined below. + +### The `system` block: +This block creates a system node pool. This block is required. + +| Name | Optional, Required | Description | +| --------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | +| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | +| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | +| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | +| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | +| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | +| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | +| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | +| max_pods | Optional | The maximum number of pods that can run on each agent. | +| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | +| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | +| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | +| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | +| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | +| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | +| tags | Optional | A mapping of tags to assign to the Node Pool. | +| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | +
+ +### The `addpool` block: +This block creates additional node pools. This block is optional. + +| Name | Optional, Required | Description | +| ---------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| node_taints | Optional | A list of Kubernetes taints which should be applied to nodes in the agent pool (e.g key=value:NoSchedule). Changing this forces a new resource to be created. | +| max_surge | Required | The maximum number or percentage of nodes which will be added to the Node Pool size during an upgrade. | +| eviction_policy | Optional | The Eviction Policy which should be used for Virtual Machines within the Virtual Machine Scale Set powering this Node Pool. Possible values are Deallocate and Delete. Will only be used when priority is set to Spot. Changing this forces a new resource to be created. | +| os_type | Optional | The Operating System which should be used for this Node Pool. Changing this forces a new resource to be created. Possible values are Linux and Windows. Defaults to Linux. | +| priority | Optional | The Priority for Virtual Machines within the Virtual Machine Scale Set that powers this Node Pool. Possible values are Regular and Spot. Defaults to Regular. Changing this forces a new resource to be created. | +| proximity_placement_group_id | Optional | The ID of the Proximity Placement Group where the Virtual Machine Scale Set that powers this Node Pool will be placed. Changing this forces a new resource to be created. | +| spot_max_price | Optional | The maximum price you're willing to pay in USD per Virtual Machine. Valid values are -1 (the current on-demand price for a Virtual Machine) or a positive value with up to five decimal places. Changing this forces a new resource to be created. | +| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | +| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | +| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | +| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | +| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | +| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | +| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | +| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | +| max_pods | Optional | The maximum number of pods that can run on each agent. | +| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | +| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | +| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | +| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | +| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | +| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | +| tags | Optional | A mapping of tags to assign to the Node Pool. | +| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | +
+ +Usage Example: +
+ + node_pools = { + system = { + vm_size = "Standard_D4_v4" + node_count = 1 + enable_auto_scaling = true + only_critical_addons_enabled = true + min_count = 1 + max_count = 1 + availability_zones = [] + subnet = "private" + enable_host_encryption = false + enable_node_public_ip = false + os_disk_type = "Managed" + type = "VirtualMachineScaleSets" + # max_pods = 10 + # node_labels = {"engine" = "roxie", "engine" = "roxie"} + # orchestrator_version = "2.9.0" + # os_disk_size_gb = 100 + # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} + + } + + addpool1 = { + vm_size = "Standard_D4_v4" + enable_auto_scaling = true + node_count = 2 + min_count = 1 + max_count = 2 + availability_zones = [] + subnet = "public" + priority = "Regular" + spot_max_price = -1 + max_surge = "1" + os_type = "Linux" + priority = "Regular" + enable_host_encryption = false + enable_node_public_ip = false + only_critical_addons_enabled = false + os_disk_type = "Managed" + type = "VirtualMachineScaleSets" + # orchestrator_version = "2.9.0" + # os_disk_size_gb = 100 + # max_pods = 20 + # node_labels = {"engine" = "roxie", "engine" = "roxie"} + # eviction_policy = "Spot" + # node_taints = ["mytaint1", "mytaint2"] + # proximity_placement_group_id = "my_proximity_placement_group_id" + # spot_max_price = 1 + # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} + } + + addpool2 = { + vm_size = "Standard_D4_v4" + enable_auto_scaling = true + node_count = 2 + min_count = 1 + max_count = 2 + availability_zones = [] + subnet = "public" + priority = "Regular" + spot_max_price = -1 + max_surge = "1" + os_type = "Linux" + priority = "Regular" + enable_host_encryption = false + enable_node_public_ip = false + only_critical_addons_enabled = false + os_disk_type = "Managed" + type = "VirtualMachineScaleSets" + # orchestrator_version = "2.9.0" + # os_disk_size_gb = 100 + # max_pods = 20 + # node_labels = {"engine" = "roxie", "engine" = "roxie"} + # eviction_policy = "Spot" + # node_taints = ["mytaint1", "mytaint2"] + # proximity_placement_group_id = "my_proximity_placement_group_id" + # spot_max_price = 1 + # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} + } + } +
+ +### The `disable_helm` argument: +This block disable helm deployments by Terraform. This block is optional and will stop HPCC from being installed. + + | Name | Description | Type | Default | Required | + | ------------ | -------------------------------------- | ---- | ------- | :------: | + | disable_helm | Disable Helm deployments by Terraform. | bool | `false` | no | +
+ +### The `hpcc` block: +This block deploys the HPCC helm chart. This block is optional. + + | Name | Description | Type | Default | Required | + | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------------------------------ | :------: | + | local_chart | Path to local chart directory name or tgz file. Example1: ~/HPCC-Platform/helm/hpcc Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz | string | null | no | + | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | string | null | no | + | namespace | Namespace to use. | string | default | no | + | name | Release name of the chart. | string | myhpcck8s | no | + | values | List of desired state files to use similar to -f in CLI. | list(string) | values-retained-azurefile.yaml | no | + | version | Version of the HPCC chart. | string | latest | yes | + | image_root | Image root to use. | string | hpccsystems | no | + | image_name | Image name to use. | string | platform-core | no | + | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | + | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | + | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | + | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | + | force_update | Force resource update through delete/recreate if needed. | bool | false | no | + | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | + | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | + | max_history | Maximum number of release versions stored per release. | number | 0 | no | + | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | + | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | + | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | + | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | + | lint | Run the helm chart linter during the plan. | bool | false | no | + | expose_eclwatch | Expose ECLWatch to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | +
+ + Usage Example: +
+ + hpcc = { + expose_eclwatch = true + name = "myhpcck8s" + atomic = true + recreate_pods = false + reuse_values = false + reset_values = false + force_update = false + namespace = "default" + cleanup_on_fail = false + disable_openapi_validation = false + max_history = 0 + wait = true + dependency_update = true + timeout = 900 + wait_for_jobs = false + lint = false + remote_chart = "https://hpcc-systems.github.io/helm-chart" + # local_chart = "/Users/foo/work/demo/helm-chart/helm/hpcc" #Other examples: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz + # version = "8.6.14-rc2" + # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] + # image_root = "west.lexisnexisrisk.com" + # image_name = "platform-core-ln" + # image_version = "8.6.18-rc1" + } + +
+ +### The `storage` block: +This block deploys the HPCC persistent volumes. This block is required. + + | Name | Description | Type | Default | Valid Options | Required | + | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------- | ---------------- | :---------: | + | default | Use AKS provided storage accounts? | bool | `false` | `true` , `false` | no | + | version | The version of the storage chart. | string | 0.1.0 | | no | + | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-azurefile-0.1.0.tgz | string | null | no | + | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | name | Release name of the chart. | string | `myhpcck8s` | no | + | values | List of desired state files to use similar to -f in CLI. | list(string) | [] | no | + | storage_accounts | The storage account to use. | object | Queries attributes' values from storage_accounts module | - | no | + | version | Version of the storage chart. | string | 0.1.0 | no | + | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | + | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | + | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | + | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | + | force_update | Force resource update through delete/recreate if needed. | bool | false | no | + | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | + | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | + | max_history | Maximum number of release versions stored per release. | number | 0 | no | + | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | + | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | + | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 600 | no | + | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | + | lint | Run the helm chart linter during the plan. | bool | false | no | +
+ +#### The `storage_accounts` block: +This block deploys the HPCC persistent volumes. This block is required. + + | Name | Description | Type | Default | Valid Options | Required | + | ------------------- | -------------------------------------------------------------------- | ------------ | --------------------------- | ------------- | :------: | + | name | Name of the storage account | string | - | - | yes | + | resource_group_name | The name of the resource group in which the storage account belongs. | string | - | - | yes | + | subscription_id | The ID of the subscription in which the storage account belongs. | string | Admin's active subscription | - | no | + | shares | The list of shares in the storage account | list(object) | - | - | yes | + | | +
+ +#### The `shares` block: +This block defines the list of shares in the storage account. This block is required. + + | Name | Description | Type | Default | Valid Options | Required | + | -------- | ------------------------------------- | ------ | ------- | ------------- | :------: | + | name | The name of the share. | string | - | - | yes | + | sub_path | The sub path for the HPCC data plane. | string | - | - | yes | + | category | The category for the HPCC data plane | string | - | - | yes | + | sku | The sku for the HPCC data plane. | string | - | - | yes | + | quota | The size of the share in Gigabytes | number | - | - | yes | + +Usage Example: +
+ + storage = { + default = false + atomic = true + recreate_pods = false + reuse_values = false + reset_values = false + force_update = false + namespace = "default" + cleanup_on_fail = false + disable_openapi_validation = false + max_history = 0 + wait = true + dependency_update = true + timeout = 600 + wait_for_jobs = false + lint = false + remote_chart = "https://hpcc-systems.github.io/helm-chart" + # local_chart = "/Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile" + # version = "0.1.0" + # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] + + /* + storage_accounts = { + # do not change the key names + dali = { + name = "dalikxgt" + resource_group_name = "app-storageaccount-sandbox-eastus-79735" + + shares = { + dali = { + name = "dalishare" + sub_path = "dalistorage" //do not change this value + category = "dali" //do not change this value + sku = "Premium_LRS" + quota = 100 + } + } + } + + sasha = { + name = "sashakxgt" + resource_group_name = "app-storageaccount-sandbox-eastus-79735" + + shares = { + sasha = { + name = "sashashare" + sub_path = "sasha" //do not change this value + category = "sasha" //do not change this value + sku = "Standard_LRS" + quota = 100 + } + } + } + + common = { + name = "commonkxgt" + resource_group_name = "app-storageaccount-sandbox-eastus-79735" + + shares = { + data = { + name = "datashare" + sub_path = "hpcc-data" //do not change this value + category = "data" //do not change this value + sku = "Standard_LRS" + quota = 100 + } + + dll = { + name = "dllshare" + sub_path = "queries" //do not change this value + category = "dll" //do not change this value + sku = "Standard_LRS" + quota = 100 + } + + mydropzone = { + name = "mydropzoneshare" + sub_path = "dropzone" //do not change this value + category = "lz" //do not change this value + sku = "Standard_LRS" + quota = 100 + } + } + } + } + */ + } +
+ +### The `elastic4hpcclogs` block: +This block deploys the elastic4hpcclogs chart. This block is optional. + + | Name | Description | Type | Default | Required | + | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ----------------------- | :------: | + | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/managed/logging/elastic Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/elastic4hpcclogs-1.2.10.tgz | string | null | no | + | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | enable | Enable elastic4hpcclogs | bool | `true` | no | + | name | Release name of the chart. | string | myelastic4hpcclogs | no | + | version | The version of the elastic4hpcclogs | string | 1.2.8 | | no | + | values | List of desired state files to use similar to -f in CLI. | list(string) | - | no | + | version | Version of the elastic4hpcclogs chart. | string | 1.2.1 | no | + | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | + | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | + | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | + | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | + | force_update | Force resource update through delete/recreate if needed. | bool | false | no | + | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | + | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | + | max_history | Maximum number of release versions stored per release. | number | 0 | no | + | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | + | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | + | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | + | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | + | lint | Run the helm chart linter during the plan. | bool | false | no | + | expose | Expose myelastic4hpcclogs-kibana service to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | +
+ +Usage Example: +
+ + elastic4hpcclogs = { + enable = true + expose = true + name = "myelastic4hpcclogs" + atomic = true + recreate_pods = false + reuse_values = false + reset_values = false + force_update = false + namespace = "default" + cleanup_on_fail = false + disable_openapi_validation = false + max_history = 0 + wait = true + dependency_update = true + timeout = 300 + wait_for_jobs = false + lint = false + remote_chart = "https://hpcc-systems.github.io/helm-chart" + #local_chart = "/Users/godji/work/demo/helm-chart/helm/managed/logging/elastic" + # version = "1.2.10" + # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] + } +
+ +### The `registry` block: +This block authenticates a private Docker repository. This block is optional. + + | Name | Description | Type | Default | Required | + | -------- | -------------------------------------------------------------------------- | ------ | ------- | :------: | + | server | The server address of the private Docker repository. | string | - | yes | + | username | The username for the private Docker repository account. | string | - | yes | + | password | The password, token, or API key for the private Docker repository account. | string | - | yes | +
+ +Usage Example: +
+ + registry = { + password = "" + server = "" + username = "" + } +
+ +### The `auto_connect` argument: +This block automatically connect your cluster to your local machine similarly to `az aks get-credentials`. + + | Name | Description | Type | Default | Required | + | ------------ | --------------------------------------------------------------------------------------------------------- | ---- | ------- | :------: | + | auto_connect | Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context. | bool | `false` | no | +
+ +## Outputs + +| Name | Description | +| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| aks_login | Get access credentials for the managed Kubernetes cluster. | +| recommendations | A list of security and cost recommendations for this deployment. Your environment has to have been deployed for several hours before Azure provides recommendations. | +
+ +## Usage +### Deploy the Virtual Network Module +
    +
  1. + +Clone this repo: `git clone https://github.com/gfortil/terraform-azurerm-hpcc.git`.
    2. + +
  2. Linux and MacOS
    3. +
      +
    1. + +Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `cp examples/admin.tfvars .`
      3. +
    +
  3. Windows OS
    4. +
      +
    1. + +Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `copy examples\admin.tfvars .`
      3. +
    +
  4. + +Open `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    5. +
  5. + +Set attributes to your preferred values.
    6. +
  6. + +Save `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    7. +
  7. + +Run `terraform init`. This step is only required before your first `terraform apply`.
    8. +
  8. + +Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    9. +
  9. + +Type `yes` if you didn't pass the flag `-auto-approve`.
    10. +
+ +### Deploy the Storage Account Module +
    +
  1. Linux and MacOS
    2. +
      +
    1. + +Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `cp examples/admin.tfvars .`
      3. +
    +
  2. Windows OS
    3. +
      +
    1. + +Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `copy examples\admin.tfvars .`
      3. +
    +
  3. + +Open `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    4. +
  4. + +Set attributes to your preferred values.
    5. +
  5. + +Save `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    6. +
  6. + +Run `terraform init`. This step is only required before your first `terraform apply`.
    7. +
  7. + +Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. +
  8. + +Type `yes` if you didn't pass the flag `-auto-approve`.
    9. +
+ +### Deploy the AKS Module +
    +
  1. Linux and MacOS
    2. +
      +
    1. + +Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc: `cp examples/admin.tfvars .`
      3. +
    +
  2. Windows OS
    3. +
      +
    1. + +Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. +
    2. + +Copy examples/admin.tfvars to terraform-azurerm-hpcc: `copy examples\admin.tfvars .`
      3. +
    +
  3. + +Open `terraform-azurerm-hpcc/admin.tfvars` file.
    4. +
  4. + +Set attributes to your preferred values.
    5. +
  5. + +Save `terraform-azurerm-hpcc/admin.tfvars` file.
    6. +
  6. + +Run `terraform init`. This step is only required before your first `terraform apply`.
    7. +
  7. + +Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. +
  8. + +Type `yes` if you didn't pass the flag `-auto-approve`.
    9. +
  9. + +If `auto_connect = true` (in admin.tfvars), skip this step.
    10. +
      +
    1. + +Copy aks_login command.
      2. +
    2. + +Run aks_login in your command line.
      3. +
    3. + +Accept to overwrite your current context.
      4. +
    +
  10. + +List pods: `kubectl get pods`.
    11. +
  11. + +Get ECLWatch external IP: `kubectl get svc --field-selector metadata.name=eclwatch | awk 'NR==2 {print $4}'`.
    12. +
  12. + +Delete cluster: `terraform destroy -var-file=admin.tfvars` or `terraform destroy -var-file=admin.tfvars -auto-approve`.
    13. +
  13. + +Type: `yes` if flag `-auto-approve` was not set.
    14. +
diff --git a/hpcc/dali.auto.tfvars.example b/dali.auto.tfvars.example similarity index 100% rename from hpcc/dali.auto.tfvars.example rename to dali.auto.tfvars.example diff --git a/hpcc/data.tf b/data.tf similarity index 100% rename from hpcc/data.tf rename to data.tf diff --git a/hpcc/eclccserver.auto.tfvars.example b/eclccserver.auto.tfvars.example similarity index 100% rename from hpcc/eclccserver.auto.tfvars.example rename to eclccserver.auto.tfvars.example diff --git a/hpcc/hpcc.tf b/hpcc.tf similarity index 95% rename from hpcc/hpcc.tf rename to hpcc.tf index 82a047f..7cf71b5 100644 --- a/hpcc/hpcc.tf +++ b/hpcc.tf @@ -9,7 +9,8 @@ resource "kubernetes_namespace" "hpcc" { } module "hpcc" { - source = "github.com/gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "github.com/gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" environment = var.metadata.environment productname = var.metadata.product_name diff --git a/hpcc/README.md b/hpcc/README.md deleted file mode 100644 index 67ef75a..0000000 --- a/hpcc/README.md +++ /dev/null @@ -1,732 +0,0 @@ -# Azure - HPCC AKS Root Module -
- -This module is intended as an example for development and test systems only. It can be used as a blueprint to develop your own production version that meets your organization's security requirements. -
-
- -## Introduction - -This module deploys an HPCC AKS cluster using remote modules that are listed below. -
- -## Remote Modules -These are the list of all the remote modules. - -| Name | Description | URL | Required | -| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------- | :------: | -| subscription | Queries enabled azure subscription from host machine | https://github.com/Azure-Terraform/terraform-azurerm-subscription-data.git | yes | -| naming | Enforces naming conventions | - | yes | -| metadata | Provides metadata | https://github.com/Azure-Terraform/terraform-azurerm-metadata.git | yes | -| resource_group | Creates a resource group | https://github.com/Azure-Terraform/terraform-azurerm-resource-group.git | yes | -| virtual_network | Creates a virtual network | https://github.com/Azure-Terraform/terraform-azurerm-virtual-network.git | yes | -| kubernetes | Creates an Azure Kubernetes Service Cluster | https://github.com/Azure-Terraform/terraform-azurerm-kubernetes.git | yes | -
- -## Supported Arguments -
- -### The `admin` block: -This block contains information on the user who is deploying the cluster. This is used as tags and part of some resource names to identify who deployed a given resource and how to contact that user. This block is required. - -| Name | Description | Type | Default | Required | -| ----- | ---------------------------- | ------ | ------- | :------: | -| name | Name of the admin. | string | - | yes | -| email | Email address for the admin. | string | - | yes | - -
-Usage Example: -
- - admin = { - name = "Example" - email = "example@hpccdemo.com" - } -
- -### The `disable_naming_conventions` block: -When set to `true`, this attribute drops the naming conventions set forth by the python module. This attribute is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------- | ---- | ------- | :------: | - | disable_naming_conventions | Disable naming conventions. | bool | `false` | no | -
- -### The `metadata` block: -TThe arguments in this block are used as tags and part of resources’ names. This block can be omitted when disable_naming_conventions is set to `true`. - - | Name | Description | Type | Default | Required | - | ------------------- | ---------------------------- | ------ | ------- | :------: | - | project_name | Name of the project. | string | "" | yes | - | product_name | Name of the product. | string | hpcc | no | - | business_unit | Name of your bussiness unit. | string | "" | no | - | environment | Name of the environment. | string | "" | no | - | market | Name of market. | string | "" | no | - | product_group | Name of product group. | string | "" | no | - | resource_group_type | Resource group type. | string | "" | no | - | sre_team | Name of SRE team. | string | "" | no | - | subscription_type | Subscription type. | string | "" | no | -
- -Usage Example: -
- - metadata = { - project = "hpccdemo" - product_name = "example" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "contoso" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - } - -
- -### The `tags` argument: -The tag attribute can be used for additional tags. The tags must be key value pairs. This block is optional. - - | Name | Description | Type | Default | Required | - | ---- | ------------------------- | ----------- | ------- | :------: | - | tags | Additional resource tags. | map(string) | admin | no | -
- -### The `resource_group` block: -This block creates a resource group (like a folder) for your resources. This block is required. - - | Name | Description | Type | Default | Required | - | ----------- | ----------------------------------------------------------------- | ---- | ------- | :------: | - | unique_name | Will concatenate a number at the end of your resource group name. | bool | `true` | yes | -
- -Usage Example: -
- - resource_group = { - unique_name = true - } - -
- -### The `virtual_network` block: -This block imports metadata of a virtual network deployed outside of this project. This block is optional. - - | Name | Description | Type | Default | Required | - | ----------------- | --------------------------------------- | ------ | ------- | :------: | - | private_subnet_id | The ID of the private subnet. | string | - | yes | - | public_subnet_id | The ID of the public subnet. | string | - | yes | - | route_table_id | The ID of the route table for the AKS. | string | - | yes | - | location | The location of the virtual network | string | - | yes | -
- -Usage Example: -
- - virtual_network = { - private_subnet_id = "" - public_subnet_id = "" - route_table_id = "" - location = "" - } - -
- -## The `node_pools` block: -The `node-pools` block supports the following arguments:
-`system` - (Required) The system or default node pool. This node pool hosts the system pods by default. The possible arguments for this block are defined below. - -`addpool` - (Required) The additional node pool configuration. This block name is changeable and must be unique across all additional node pools. At least one additional node pool is required. The possible arguments for this block are defined below. - -### The `system` block: -This block creates a system node pool. This block is required. - -| Name | Optional, Required | Description | -| --------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -### The `addpool` block: -This block creates additional node pools. This block is optional. - -| Name | Optional, Required | Description | -| ---------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| node_taints | Optional | A list of Kubernetes taints which should be applied to nodes in the agent pool (e.g key=value:NoSchedule). Changing this forces a new resource to be created. | -| max_surge | Required | The maximum number or percentage of nodes which will be added to the Node Pool size during an upgrade. | -| eviction_policy | Optional | The Eviction Policy which should be used for Virtual Machines within the Virtual Machine Scale Set powering this Node Pool. Possible values are Deallocate and Delete. Will only be used when priority is set to Spot. Changing this forces a new resource to be created. | -| os_type | Optional | The Operating System which should be used for this Node Pool. Changing this forces a new resource to be created. Possible values are Linux and Windows. Defaults to Linux. | -| priority | Optional | The Priority for Virtual Machines within the Virtual Machine Scale Set that powers this Node Pool. Possible values are Regular and Spot. Defaults to Regular. Changing this forces a new resource to be created. | -| proximity_placement_group_id | Optional | The ID of the Proximity Placement Group where the Virtual Machine Scale Set that powers this Node Pool will be placed. Changing this forces a new resource to be created. | -| spot_max_price | Optional | The maximum price you're willing to pay in USD per Virtual Machine. Valid values are -1 (the current on-demand price for a Virtual Machine) or a positive value with up to five decimal places. Changing this forces a new resource to be created. | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -Usage Example: -
- - node_pools = { - system = { - vm_size = "Standard_D4_v4" - node_count = 1 - enable_auto_scaling = true - only_critical_addons_enabled = true - min_count = 1 - max_count = 1 - availability_zones = [] - subnet = "private" - enable_host_encryption = false - enable_node_public_ip = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # max_pods = 10 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - - } - - addpool1 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - - addpool2 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - } -
- -### The `disable_helm` argument: -This block disable helm deployments by Terraform. This block is optional and will stop HPCC from being installed. - - | Name | Description | Type | Default | Required | - | ------------ | -------------------------------------- | ---- | ------- | :------: | - | disable_helm | Disable Helm deployments by Terraform. | bool | `false` | no | -
- -### The `hpcc` block: -This block deploys the HPCC helm chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------------------------------ | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: ~/HPCC-Platform/helm/hpcc Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | string | null | no | - | namespace | Namespace to use. | string | default | no | - | name | Release name of the chart. | string | myhpcck8s | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | values-retained-azurefile.yaml | no | - | version | Version of the HPCC chart. | string | latest | yes | - | image_root | Image root to use. | string | hpccsystems | no | - | image_name | Image name to use. | string | platform-core | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose_eclwatch | Expose ECLWatch to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- - Usage Example: -
- - hpcc = { - expose_eclwatch = true - name = "myhpcck8s" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 900 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/hpcc" #Other examples: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz - # version = "8.6.14-rc2" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - # image_root = "west.lexisnexisrisk.com" - # image_name = "platform-core-ln" - # image_version = "8.6.18-rc1" - } - -
- -### The `storage` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------- | ---------------- | :---------: | - | default | Use AKS provided storage accounts? | bool | `false` | `true` , `false` | no | - | version | The version of the storage chart. | string | 0.1.0 | | no | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-azurefile-0.1.0.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | name | Release name of the chart. | string | `myhpcck8s` | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | [] | no | - | storage_accounts | The storage account to use. | object | Queries attributes' values from storage_accounts module | - | no | - | version | Version of the storage chart. | string | 0.1.0 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 600 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | -
- -#### The `storage_accounts` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | ------------------- | -------------------------------------------------------------------- | ------------ | --------------------------- | ------------- | :------: | - | name | Name of the storage account | string | - | - | yes | - | resource_group_name | The name of the resource group in which the storage account belongs. | string | - | - | yes | - | subscription_id | The ID of the subscription in which the storage account belongs. | string | Admin's active subscription | - | no | - | shares | The list of shares in the storage account | list(object) | - | - | yes | - | | -
- -#### The `shares` block: -This block defines the list of shares in the storage account. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------- | ------------------------------------- | ------ | ------- | ------------- | :------: | - | name | The name of the share. | string | - | - | yes | - | sub_path | The sub path for the HPCC data plane. | string | - | - | yes | - | category | The category for the HPCC data plane | string | - | - | yes | - | sku | The sku for the HPCC data plane. | string | - | - | yes | - | quota | The size of the share in Gigabytes | number | - | - | yes | - -Usage Example: -
- - storage = { - default = false - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 600 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile" - # version = "0.1.0" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - - /* - storage_accounts = { - # do not change the key names - dali = { - name = "dalikxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - dali = { - name = "dalishare" - sub_path = "dalistorage" //do not change this value - category = "dali" //do not change this value - sku = "Premium_LRS" - quota = 100 - } - } - } - - sasha = { - name = "sashakxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - sasha = { - name = "sashashare" - sub_path = "sasha" //do not change this value - category = "sasha" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - - common = { - name = "commonkxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - data = { - name = "datashare" - sub_path = "hpcc-data" //do not change this value - category = "data" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - dll = { - name = "dllshare" - sub_path = "queries" //do not change this value - category = "dll" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - mydropzone = { - name = "mydropzoneshare" - sub_path = "dropzone" //do not change this value - category = "lz" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - } - */ - } -
- -### The `elastic4hpcclogs` block: -This block deploys the elastic4hpcclogs chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ----------------------- | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/managed/logging/elastic Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/elastic4hpcclogs-1.2.10.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | enable | Enable elastic4hpcclogs | bool | `true` | no | - | name | Release name of the chart. | string | myelastic4hpcclogs | no | - | version | The version of the elastic4hpcclogs | string | 1.2.8 | | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | - | no | - | version | Version of the elastic4hpcclogs chart. | string | 1.2.1 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose | Expose myelastic4hpcclogs-kibana service to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- -Usage Example: -
- - elastic4hpcclogs = { - enable = true - expose = true - name = "myelastic4hpcclogs" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 300 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - #local_chart = "/Users/godji/work/demo/helm-chart/helm/managed/logging/elastic" - # version = "1.2.10" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - } -
- -### The `registry` block: -This block authenticates a private Docker repository. This block is optional. - - | Name | Description | Type | Default | Required | - | -------- | -------------------------------------------------------------------------- | ------ | ------- | :------: | - | server | The server address of the private Docker repository. | string | - | yes | - | username | The username for the private Docker repository account. | string | - | yes | - | password | The password, token, or API key for the private Docker repository account. | string | - | yes | -
- -Usage Example: -
- - registry = { - password = "" - server = "" - username = "" - } -
- -### The `auto_connect` argument: -This block automatically connect your cluster to your local machine similarly to `az aks get-credentials`. - - | Name | Description | Type | Default | Required | - | ------------ | --------------------------------------------------------------------------------------------------------- | ---- | ------- | :------: | - | auto_connect | Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context. | bool | `false` | no | -
- -## Outputs - -| Name | Description | -| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| aks_login | Get access credentials for the managed Kubernetes cluster. | -| recommendations | A list of security and cost recommendations for this deployment. Your environment has to have been deployed for several hours before Azure provides recommendations. | -
- -## Usage -### Deploy the Virtual Network Module -
    -
  1. - -Clone this repo: `git clone https://github.com/gfortil/terraform-azurerm-hpcc.git`.
    2. - -
  2. Linux and MacOS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `cp examples/admin.tfvars .`
      3. -
    -
  3. Windows OS
    4. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `copy examples\admin.tfvars .`
      3. -
    -
  4. - -Open `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    5. -
  5. - -Set attributes to your preferred values.
    6. -
  6. - -Save `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    7. -
  7. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    8. -
  8. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    9. -
  9. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    10. -
- -### Deploy the Storage Account Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
- -### Deploy the AKS Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
  9. - -If `auto_connect = true` (in admin.tfvars), skip this step.
    10. -
      -
    1. - -Copy aks_login command.
      2. -
    2. - -Run aks_login in your command line.
      3. -
    3. - -Accept to overwrite your current context.
      4. -
    -
  10. - -List pods: `kubectl get pods`.
    11. -
  11. - -Get ECLWatch external IP: `kubectl get svc --field-selector metadata.name=eclwatch | awk 'NR==2 {print $4}'`.
    12. -
  12. - -Delete cluster: `terraform destroy -var-file=admin.tfvars` or `terraform destroy -var-file=admin.tfvars -auto-approve`.
    13. -
  13. - -Type: `yes` if flag `-auto-approve` was not set.
    14. -
diff --git a/hpcc/locals.tf b/locals.tf similarity index 100% rename from hpcc/locals.tf rename to locals.tf diff --git a/hpcc/main.tf b/main.tf similarity index 100% rename from hpcc/main.tf rename to main.tf diff --git a/hpcc/misc.auto.tfvars.example b/misc.auto.tfvars.example similarity index 100% rename from hpcc/misc.auto.tfvars.example rename to misc.auto.tfvars.example diff --git a/hpcc/LICENSE b/modules/aks/LICENSE similarity index 100% rename from hpcc/LICENSE rename to modules/aks/LICENSE diff --git a/aks/aks.auto.tfvars.example b/modules/aks/aks.auto.tfvars.example similarity index 100% rename from aks/aks.auto.tfvars.example rename to modules/aks/aks.auto.tfvars.example diff --git a/aks/aks.tf b/modules/aks/aks.tf similarity index 95% rename from aks/aks.tf rename to modules/aks/aks.tf index 3c317a0..65d08a3 100644 --- a/aks/aks.tf +++ b/modules/aks/aks.tf @@ -20,8 +20,8 @@ module "aks" { depends_on = [random_string.string] - source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - # source = "../../../terraform-azurerm-aks" + #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" providers = { kubernetes = kubernetes.default diff --git a/aks/automation.tf b/modules/aks/automation.tf similarity index 100% rename from aks/automation.tf rename to modules/aks/automation.tf diff --git a/aks/data.tf b/modules/aks/data.tf similarity index 100% rename from aks/data.tf rename to modules/aks/data.tf diff --git a/aks/locals.tf b/modules/aks/locals.tf similarity index 100% rename from aks/locals.tf rename to modules/aks/locals.tf diff --git a/aks/main.tf b/modules/aks/main.tf similarity index 100% rename from aks/main.tf rename to modules/aks/main.tf diff --git a/aks/misc.auto.tfvars.example b/modules/aks/misc.auto.tfvars.example similarity index 100% rename from aks/misc.auto.tfvars.example rename to modules/aks/misc.auto.tfvars.example diff --git a/aks/outputs.tf b/modules/aks/outputs.tf similarity index 100% rename from aks/outputs.tf rename to modules/aks/outputs.tf diff --git a/aks/providers.tf b/modules/aks/providers.tf similarity index 100% rename from aks/providers.tf rename to modules/aks/providers.tf diff --git a/aks/scripts/start_stop.ps1 b/modules/aks/scripts/start_stop.ps1 similarity index 100% rename from aks/scripts/start_stop.ps1 rename to modules/aks/scripts/start_stop.ps1 diff --git a/aks/variables.tf b/modules/aks/variables.tf similarity index 100% rename from aks/variables.tf rename to modules/aks/variables.tf diff --git a/aks/versions.tf b/modules/aks/versions.tf similarity index 100% rename from aks/versions.tf rename to modules/aks/versions.tf diff --git a/logging/LICENSE b/modules/logging/LICENSE similarity index 100% rename from logging/LICENSE rename to modules/logging/LICENSE diff --git a/logging/data.tf b/modules/logging/data.tf similarity index 100% rename from logging/data.tf rename to modules/logging/data.tf diff --git a/logging/elastic4hpcc.auto.tfvars.example b/modules/logging/elastic4hpcc.auto.tfvars.example similarity index 100% rename from logging/elastic4hpcc.auto.tfvars.example rename to modules/logging/elastic4hpcc.auto.tfvars.example diff --git a/logging/locals.tf b/modules/logging/locals.tf similarity index 100% rename from logging/locals.tf rename to modules/logging/locals.tf diff --git a/logging/log_analytics.auto.tfvars.example b/modules/logging/log_analytics.auto.tfvars.example similarity index 100% rename from logging/log_analytics.auto.tfvars.example rename to modules/logging/log_analytics.auto.tfvars.example diff --git a/logging/logging.tf b/modules/logging/logging.tf similarity index 100% rename from logging/logging.tf rename to modules/logging/logging.tf diff --git a/logging/main.tf b/modules/logging/main.tf similarity index 100% rename from logging/main.tf rename to modules/logging/main.tf diff --git a/logging/misc.auto.tfvars.example b/modules/logging/misc.auto.tfvars.example similarity index 100% rename from logging/misc.auto.tfvars.example rename to modules/logging/misc.auto.tfvars.example diff --git a/logging/outputs.tf b/modules/logging/outputs.tf similarity index 100% rename from logging/outputs.tf rename to modules/logging/outputs.tf diff --git a/logging/providers.tf b/modules/logging/providers.tf similarity index 100% rename from logging/providers.tf rename to modules/logging/providers.tf diff --git a/logging/variables.tf b/modules/logging/variables.tf similarity index 100% rename from logging/variables.tf rename to modules/logging/variables.tf diff --git a/logging/versions.tf b/modules/logging/versions.tf similarity index 100% rename from logging/versions.tf rename to modules/logging/versions.tf diff --git a/storage/README.md b/modules/storage/README.md similarity index 100% rename from storage/README.md rename to modules/storage/README.md diff --git a/storage/data.tf b/modules/storage/data.tf similarity index 100% rename from storage/data.tf rename to modules/storage/data.tf diff --git a/storage/locals.tf b/modules/storage/locals.tf similarity index 100% rename from storage/locals.tf rename to modules/storage/locals.tf diff --git a/storage/main.tf b/modules/storage/main.tf similarity index 62% rename from storage/main.tf rename to modules/storage/main.tf index 8fb9353..dd1de4c 100644 --- a/storage/main.tf +++ b/modules/storage/main.tf @@ -1,5 +1,6 @@ module "storage" { - source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" owner = var.owner disable_naming_conventions = var.disable_naming_conventions diff --git a/storage/misc.auto.tfvars.example b/modules/storage/misc.auto.tfvars.example similarity index 100% rename from storage/misc.auto.tfvars.example rename to modules/storage/misc.auto.tfvars.example diff --git a/storage/outputs.tf b/modules/storage/outputs.tf similarity index 100% rename from storage/outputs.tf rename to modules/storage/outputs.tf diff --git a/storage/providers.tf b/modules/storage/providers.tf similarity index 100% rename from storage/providers.tf rename to modules/storage/providers.tf diff --git a/storage/storage.auto.tfvars.example b/modules/storage/storage.auto.tfvars.example similarity index 100% rename from storage/storage.auto.tfvars.example rename to modules/storage/storage.auto.tfvars.example diff --git a/storage/variables.tf b/modules/storage/variables.tf similarity index 100% rename from storage/variables.tf rename to modules/storage/variables.tf diff --git a/storage/versions.tf b/modules/storage/versions.tf similarity index 100% rename from storage/versions.tf rename to modules/storage/versions.tf diff --git a/vnet/LICENSE b/modules/vnet/LICENSE similarity index 100% rename from vnet/LICENSE rename to modules/vnet/LICENSE diff --git a/vnet/README.md b/modules/vnet/README.md similarity index 100% rename from vnet/README.md rename to modules/vnet/README.md diff --git a/vnet/data.tf b/modules/vnet/data.tf similarity index 100% rename from vnet/data.tf rename to modules/vnet/data.tf diff --git a/vnet/locals.tf b/modules/vnet/locals.tf similarity index 100% rename from vnet/locals.tf rename to modules/vnet/locals.tf diff --git a/vnet/main.tf b/modules/vnet/main.tf similarity index 100% rename from vnet/main.tf rename to modules/vnet/main.tf diff --git a/vnet/misc.auto.tfvars.example b/modules/vnet/misc.auto.tfvars.example similarity index 100% rename from vnet/misc.auto.tfvars.example rename to modules/vnet/misc.auto.tfvars.example diff --git a/vnet/outputs.tf b/modules/vnet/outputs.tf similarity index 100% rename from vnet/outputs.tf rename to modules/vnet/outputs.tf diff --git a/vnet/providers.tf b/modules/vnet/providers.tf similarity index 100% rename from vnet/providers.tf rename to modules/vnet/providers.tf diff --git a/vnet/variables.tf b/modules/vnet/variables.tf similarity index 100% rename from vnet/variables.tf rename to modules/vnet/variables.tf diff --git a/vnet/versions.tf b/modules/vnet/versions.tf similarity index 100% rename from vnet/versions.tf rename to modules/vnet/versions.tf diff --git a/vnet/vnet.tf b/modules/vnet/vnet.tf similarity index 100% rename from vnet/vnet.tf rename to modules/vnet/vnet.tf diff --git a/hpcc/outputs.tf b/outputs.tf similarity index 100% rename from hpcc/outputs.tf rename to outputs.tf diff --git a/hpcc/providers.tf b/providers.tf similarity index 100% rename from hpcc/providers.tf rename to providers.tf diff --git a/hpcc/roxie.auto.tfvars.example b/roxie.auto.tfvars.example similarity index 100% rename from hpcc/roxie.auto.tfvars.example rename to roxie.auto.tfvars.example diff --git a/hpcc/sasha.auto.tfvars.example b/sasha.auto.tfvars.example similarity index 100% rename from hpcc/sasha.auto.tfvars.example rename to sasha.auto.tfvars.example diff --git a/hpcc/thor.auto.tfvars.example b/thor.auto.tfvars.example similarity index 100% rename from hpcc/thor.auto.tfvars.example rename to thor.auto.tfvars.example diff --git a/hpcc/variables.tf b/variables.tf similarity index 100% rename from hpcc/variables.tf rename to variables.tf diff --git a/hpcc/versions.tf b/versions.tf similarity index 100% rename from hpcc/versions.tf rename to versions.tf From 5a51f032f1b7e59cea5dfeef27659d406deff1da Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 21 Sep 2023 20:05:01 +0000 Subject: [PATCH 002/124] branch:HPCC-27615-fixes-deploy-without-external-storage. Now can deploy hpcc without external storage. --- hpcc.tf | 8 ++++++-- locals.tf | 10 +++++----- main.tf | 2 +- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/hpcc.tf b/hpcc.tf index 7cf71b5..a1688af 100644 --- a/hpcc.tf +++ b/hpcc.tf @@ -10,7 +10,9 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "github.com/gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "/home/azureuser/godji/opinionated-terraform-azurerm-hpcc" + source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" environment = var.metadata.environment productname = var.metadata.product_name @@ -51,8 +53,10 @@ module "hpcc" { subnet_ids = merge({ aks = local.subnet_ids.aks }) } + internal_storage_enabled = local.internal_storage_enabled + data_storage_config = { - internal = local.external_storage_config == null ? { + internal = (local.external_storage_config == null) || (local.internal_storage_enabled == true) ? { blob_nfs = { data_plane_count = var.data_storage_config.internal.blob_nfs.data_plane_count storage_account_settings = { diff --git a/locals.tf b/locals.tf index b81b494..40b4509 100644 --- a/locals.tf +++ b/locals.tf @@ -21,11 +21,11 @@ locals { # external_services_storage_exists = fileexists("${path.module}/modules/storage/data/config.json") || var.external_services_storage_config != null - get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null - get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null + get_vnet_config = fileexists("${path.module}/modules/vnet/data/config.json") ? jsondecode(file("${path.module}/modules/vnet/data/config.json")) : null + get_aks_config = fileexists("${path.module}/modules/aks/data/config.json") ? jsondecode(file("${path.module}/modules/aks/data/config.json")) : null + get_storage_config = local.external_storage_exists ? jsondecode(file("${path.module}/modules/storage/data/config.json")) : null - external_storage_exists = fileexists("../storage/data/config.json") || var.external_storage_config != null + external_storage_exists = fileexists("${path.module}/modules/storage/data/config.json") || var.external_storage_config != null subnet_ids = try({ for k, v in var.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${var.use_existing_vnet.name}/subnets/${v.name}" @@ -37,7 +37,7 @@ locals { domain = coalesce(var.internal_domain, format("us-%s.%s.azure.lnrsg.io", "var.metadata.product_name", "dev")) - internal_storage_enabled = local.external_storage_exists == true && var.ignore_external_storage == true ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true + internal_storage_enabled = (local.external_storage_exists == true) && (var.ignore_external_storage == true) ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true # external_services_storage_enabled = local.external_services_storage_exists == true && var.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && var.ignore_external_services_storage == true ? false : true hpcc_namespace = var.hpcc_namespace.existing_namespace != null ? var.hpcc_namespace.existing_namespace : var.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("${path.module}/logging/data/hpcc_namespace.txt") ? file("${path.module}/logging/data/hpcc_namespace.txt") : "default" diff --git a/main.tf b/main.tf index b9846df..aa36ca0 100644 --- a/main.tf +++ b/main.tf @@ -31,7 +31,7 @@ module "metadata" { } resource "null_resource" "launch_svc_url" { - for_each = module.hpcc.hpcc_status == "deployed" ? local.svc_domains : {} + for_each = (module.hpcc.hpcc_status == "deployed") && (var.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} provisioner "local-exec" { command = local.is_windows_os ? "Start-Process ${each.value}" : "open ${each.value} || xdg-open ${each.value}" From 35b1a4375d7009e74046df3f6e6557a9eaa67473 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 21 Sep 2023 20:58:40 +0000 Subject: [PATCH 003/124] branch:HPCC-27615-my-local-has-flat-structure --- .gitignore | 20 +++++++++---------- LICENSE => aks/LICENSE | 0 {modules/aks => aks}/aks.auto.tfvars.example | 0 {modules/aks => aks}/aks.tf | 0 {modules/aks => aks}/automation.tf | 0 {modules/aks => aks}/data.tf | 0 {modules/aks => aks}/locals.tf | 0 {modules/aks => aks}/main.tf | 0 {modules/aks => aks}/misc.auto.tfvars.example | 0 {modules/aks => aks}/outputs.tf | 0 {modules/aks => aks}/providers.tf | 0 {modules/aks => aks}/scripts/start_stop.ps1 | 0 {modules/aks => aks}/variables.tf | 0 {modules/aks => aks}/versions.tf | 0 {modules/aks => hpcc}/LICENSE | 0 .../dali.auto.tfvars.example | 0 data.tf => hpcc/data.tf | 0 .../eclccserver.auto.tfvars.example | 0 hpcc.tf => hpcc/hpcc.tf | 0 locals.tf => hpcc/locals.tf | 10 +++++----- main.tf => hpcc/main.tf | 0 .../misc.auto.tfvars.example | 0 outputs.tf => hpcc/outputs.tf | 0 providers.tf => hpcc/providers.tf | 0 .../roxie.auto.tfvars.example | 0 .../sasha.auto.tfvars.example | 0 .../thor.auto.tfvars.example | 0 variables.tf => hpcc/variables.tf | 0 versions.tf => hpcc/versions.tf | 0 {modules/logging => logging}/LICENSE | 0 {modules/logging => logging}/data.tf | 0 .../elastic4hpcc.auto.tfvars.example | 0 {modules/logging => logging}/locals.tf | 0 .../log_analytics.auto.tfvars.example | 0 {modules/logging => logging}/logging.tf | 0 {modules/logging => logging}/main.tf | 0 .../misc.auto.tfvars.example | 0 {modules/logging => logging}/outputs.tf | 0 {modules/logging => logging}/providers.tf | 0 {modules/logging => logging}/variables.tf | 0 {modules/logging => logging}/versions.tf | 0 {modules/storage => storage}/README.md | 0 {modules/storage => storage}/data.tf | 0 {modules/storage => storage}/locals.tf | 0 {modules/storage => storage}/main.tf | 0 .../misc.auto.tfvars.example | 0 {modules/storage => storage}/outputs.tf | 0 {modules/storage => storage}/providers.tf | 0 .../storage.auto.tfvars.example | 0 {modules/storage => storage}/variables.tf | 0 {modules/storage => storage}/versions.tf | 0 {modules/vnet => vnet}/LICENSE | 0 {modules/vnet => vnet}/README.md | 0 {modules/vnet => vnet}/data.tf | 0 {modules/vnet => vnet}/locals.tf | 0 {modules/vnet => vnet}/main.tf | 0 .../vnet => vnet}/misc.auto.tfvars.example | 0 {modules/vnet => vnet}/outputs.tf | 0 {modules/vnet => vnet}/providers.tf | 0 {modules/vnet => vnet}/variables.tf | 0 {modules/vnet => vnet}/versions.tf | 0 {modules/vnet => vnet}/vnet.tf | 0 62 files changed, 15 insertions(+), 15 deletions(-) rename LICENSE => aks/LICENSE (100%) rename {modules/aks => aks}/aks.auto.tfvars.example (100%) rename {modules/aks => aks}/aks.tf (100%) rename {modules/aks => aks}/automation.tf (100%) rename {modules/aks => aks}/data.tf (100%) rename {modules/aks => aks}/locals.tf (100%) rename {modules/aks => aks}/main.tf (100%) rename {modules/aks => aks}/misc.auto.tfvars.example (100%) rename {modules/aks => aks}/outputs.tf (100%) rename {modules/aks => aks}/providers.tf (100%) rename {modules/aks => aks}/scripts/start_stop.ps1 (100%) rename {modules/aks => aks}/variables.tf (100%) rename {modules/aks => aks}/versions.tf (100%) rename {modules/aks => hpcc}/LICENSE (100%) rename dali.auto.tfvars.example => hpcc/dali.auto.tfvars.example (100%) rename data.tf => hpcc/data.tf (100%) rename eclccserver.auto.tfvars.example => hpcc/eclccserver.auto.tfvars.example (100%) rename hpcc.tf => hpcc/hpcc.tf (100%) rename locals.tf => hpcc/locals.tf (83%) rename main.tf => hpcc/main.tf (100%) rename misc.auto.tfvars.example => hpcc/misc.auto.tfvars.example (100%) rename outputs.tf => hpcc/outputs.tf (100%) rename providers.tf => hpcc/providers.tf (100%) rename roxie.auto.tfvars.example => hpcc/roxie.auto.tfvars.example (100%) rename sasha.auto.tfvars.example => hpcc/sasha.auto.tfvars.example (100%) rename thor.auto.tfvars.example => hpcc/thor.auto.tfvars.example (100%) rename variables.tf => hpcc/variables.tf (100%) rename versions.tf => hpcc/versions.tf (100%) rename {modules/logging => logging}/LICENSE (100%) rename {modules/logging => logging}/data.tf (100%) rename {modules/logging => logging}/elastic4hpcc.auto.tfvars.example (100%) rename {modules/logging => logging}/locals.tf (100%) rename {modules/logging => logging}/log_analytics.auto.tfvars.example (100%) rename {modules/logging => logging}/logging.tf (100%) rename {modules/logging => logging}/main.tf (100%) rename {modules/logging => logging}/misc.auto.tfvars.example (100%) rename {modules/logging => logging}/outputs.tf (100%) rename {modules/logging => logging}/providers.tf (100%) rename {modules/logging => logging}/variables.tf (100%) rename {modules/logging => logging}/versions.tf (100%) rename {modules/storage => storage}/README.md (100%) rename {modules/storage => storage}/data.tf (100%) rename {modules/storage => storage}/locals.tf (100%) rename {modules/storage => storage}/main.tf (100%) rename {modules/storage => storage}/misc.auto.tfvars.example (100%) rename {modules/storage => storage}/outputs.tf (100%) rename {modules/storage => storage}/providers.tf (100%) rename {modules/storage => storage}/storage.auto.tfvars.example (100%) rename {modules/storage => storage}/variables.tf (100%) rename {modules/storage => storage}/versions.tf (100%) rename {modules/vnet => vnet}/LICENSE (100%) rename {modules/vnet => vnet}/README.md (100%) rename {modules/vnet => vnet}/data.tf (100%) rename {modules/vnet => vnet}/locals.tf (100%) rename {modules/vnet => vnet}/main.tf (100%) rename {modules/vnet => vnet}/misc.auto.tfvars.example (100%) rename {modules/vnet => vnet}/outputs.tf (100%) rename {modules/vnet => vnet}/providers.tf (100%) rename {modules/vnet => vnet}/variables.tf (100%) rename {modules/vnet => vnet}/versions.tf (100%) rename {modules/vnet => vnet}/vnet.tf (100%) diff --git a/.gitignore b/.gitignore index 241c253..9f2e093 100644 --- a/.gitignore +++ b/.gitignore @@ -2,34 +2,34 @@ **/.terraform/* # .tfstate files -*.tfstate -*.tfstate.* +**/*.tfstate +**/*.tfstate.* # Crash log files -crash.log +**/crash.log # Ignore any .tfvars files that are generated automatically for each Terraform run. Most # .tfvars files are managed as part of configuration and so should be included in # version control. # # example.tfvars -*.tfvars -*.json +**/*.tfvars +**/*.json # Ignore data files **/data # Ignore override files as they are usually used to override resources locally and so # are not checked in -override.tf -override.tf.json -*_override.tf -*_override.tf.json +**/override.tf +**/override.tf.json +**/*_override.tf +**/*_override.tf.json # Include override files you do wish to add to version control using negated pattern # # !example_override.tf -.terraform.*.hcl +**/.terraform.*.hcl # Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan # example: *tfplan* diff --git a/LICENSE b/aks/LICENSE similarity index 100% rename from LICENSE rename to aks/LICENSE diff --git a/modules/aks/aks.auto.tfvars.example b/aks/aks.auto.tfvars.example similarity index 100% rename from modules/aks/aks.auto.tfvars.example rename to aks/aks.auto.tfvars.example diff --git a/modules/aks/aks.tf b/aks/aks.tf similarity index 100% rename from modules/aks/aks.tf rename to aks/aks.tf diff --git a/modules/aks/automation.tf b/aks/automation.tf similarity index 100% rename from modules/aks/automation.tf rename to aks/automation.tf diff --git a/modules/aks/data.tf b/aks/data.tf similarity index 100% rename from modules/aks/data.tf rename to aks/data.tf diff --git a/modules/aks/locals.tf b/aks/locals.tf similarity index 100% rename from modules/aks/locals.tf rename to aks/locals.tf diff --git a/modules/aks/main.tf b/aks/main.tf similarity index 100% rename from modules/aks/main.tf rename to aks/main.tf diff --git a/modules/aks/misc.auto.tfvars.example b/aks/misc.auto.tfvars.example similarity index 100% rename from modules/aks/misc.auto.tfvars.example rename to aks/misc.auto.tfvars.example diff --git a/modules/aks/outputs.tf b/aks/outputs.tf similarity index 100% rename from modules/aks/outputs.tf rename to aks/outputs.tf diff --git a/modules/aks/providers.tf b/aks/providers.tf similarity index 100% rename from modules/aks/providers.tf rename to aks/providers.tf diff --git a/modules/aks/scripts/start_stop.ps1 b/aks/scripts/start_stop.ps1 similarity index 100% rename from modules/aks/scripts/start_stop.ps1 rename to aks/scripts/start_stop.ps1 diff --git a/modules/aks/variables.tf b/aks/variables.tf similarity index 100% rename from modules/aks/variables.tf rename to aks/variables.tf diff --git a/modules/aks/versions.tf b/aks/versions.tf similarity index 100% rename from modules/aks/versions.tf rename to aks/versions.tf diff --git a/modules/aks/LICENSE b/hpcc/LICENSE similarity index 100% rename from modules/aks/LICENSE rename to hpcc/LICENSE diff --git a/dali.auto.tfvars.example b/hpcc/dali.auto.tfvars.example similarity index 100% rename from dali.auto.tfvars.example rename to hpcc/dali.auto.tfvars.example diff --git a/data.tf b/hpcc/data.tf similarity index 100% rename from data.tf rename to hpcc/data.tf diff --git a/eclccserver.auto.tfvars.example b/hpcc/eclccserver.auto.tfvars.example similarity index 100% rename from eclccserver.auto.tfvars.example rename to hpcc/eclccserver.auto.tfvars.example diff --git a/hpcc.tf b/hpcc/hpcc.tf similarity index 100% rename from hpcc.tf rename to hpcc/hpcc.tf diff --git a/locals.tf b/hpcc/locals.tf similarity index 83% rename from locals.tf rename to hpcc/locals.tf index 40b4509..a96e457 100644 --- a/locals.tf +++ b/hpcc/locals.tf @@ -19,13 +19,13 @@ locals { tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) - # external_services_storage_exists = fileexists("${path.module}/modules/storage/data/config.json") || var.external_services_storage_config != null + # external_services_storage_exists = fileexists("../storage/data/config.json") || var.external_services_storage_config != null - get_vnet_config = fileexists("${path.module}/modules/vnet/data/config.json") ? jsondecode(file("${path.module}/modules/vnet/data/config.json")) : null - get_aks_config = fileexists("${path.module}/modules/aks/data/config.json") ? jsondecode(file("${path.module}/modules/aks/data/config.json")) : null - get_storage_config = local.external_storage_exists ? jsondecode(file("${path.module}/modules/storage/data/config.json")) : null + get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null + get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null + get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null - external_storage_exists = fileexists("${path.module}/modules/storage/data/config.json") || var.external_storage_config != null + external_storage_exists = fileexists("../storage/data/config.json") || var.external_storage_config != null subnet_ids = try({ for k, v in var.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${var.use_existing_vnet.name}/subnets/${v.name}" diff --git a/main.tf b/hpcc/main.tf similarity index 100% rename from main.tf rename to hpcc/main.tf diff --git a/misc.auto.tfvars.example b/hpcc/misc.auto.tfvars.example similarity index 100% rename from misc.auto.tfvars.example rename to hpcc/misc.auto.tfvars.example diff --git a/outputs.tf b/hpcc/outputs.tf similarity index 100% rename from outputs.tf rename to hpcc/outputs.tf diff --git a/providers.tf b/hpcc/providers.tf similarity index 100% rename from providers.tf rename to hpcc/providers.tf diff --git a/roxie.auto.tfvars.example b/hpcc/roxie.auto.tfvars.example similarity index 100% rename from roxie.auto.tfvars.example rename to hpcc/roxie.auto.tfvars.example diff --git a/sasha.auto.tfvars.example b/hpcc/sasha.auto.tfvars.example similarity index 100% rename from sasha.auto.tfvars.example rename to hpcc/sasha.auto.tfvars.example diff --git a/thor.auto.tfvars.example b/hpcc/thor.auto.tfvars.example similarity index 100% rename from thor.auto.tfvars.example rename to hpcc/thor.auto.tfvars.example diff --git a/variables.tf b/hpcc/variables.tf similarity index 100% rename from variables.tf rename to hpcc/variables.tf diff --git a/versions.tf b/hpcc/versions.tf similarity index 100% rename from versions.tf rename to hpcc/versions.tf diff --git a/modules/logging/LICENSE b/logging/LICENSE similarity index 100% rename from modules/logging/LICENSE rename to logging/LICENSE diff --git a/modules/logging/data.tf b/logging/data.tf similarity index 100% rename from modules/logging/data.tf rename to logging/data.tf diff --git a/modules/logging/elastic4hpcc.auto.tfvars.example b/logging/elastic4hpcc.auto.tfvars.example similarity index 100% rename from modules/logging/elastic4hpcc.auto.tfvars.example rename to logging/elastic4hpcc.auto.tfvars.example diff --git a/modules/logging/locals.tf b/logging/locals.tf similarity index 100% rename from modules/logging/locals.tf rename to logging/locals.tf diff --git a/modules/logging/log_analytics.auto.tfvars.example b/logging/log_analytics.auto.tfvars.example similarity index 100% rename from modules/logging/log_analytics.auto.tfvars.example rename to logging/log_analytics.auto.tfvars.example diff --git a/modules/logging/logging.tf b/logging/logging.tf similarity index 100% rename from modules/logging/logging.tf rename to logging/logging.tf diff --git a/modules/logging/main.tf b/logging/main.tf similarity index 100% rename from modules/logging/main.tf rename to logging/main.tf diff --git a/modules/logging/misc.auto.tfvars.example b/logging/misc.auto.tfvars.example similarity index 100% rename from modules/logging/misc.auto.tfvars.example rename to logging/misc.auto.tfvars.example diff --git a/modules/logging/outputs.tf b/logging/outputs.tf similarity index 100% rename from modules/logging/outputs.tf rename to logging/outputs.tf diff --git a/modules/logging/providers.tf b/logging/providers.tf similarity index 100% rename from modules/logging/providers.tf rename to logging/providers.tf diff --git a/modules/logging/variables.tf b/logging/variables.tf similarity index 100% rename from modules/logging/variables.tf rename to logging/variables.tf diff --git a/modules/logging/versions.tf b/logging/versions.tf similarity index 100% rename from modules/logging/versions.tf rename to logging/versions.tf diff --git a/modules/storage/README.md b/storage/README.md similarity index 100% rename from modules/storage/README.md rename to storage/README.md diff --git a/modules/storage/data.tf b/storage/data.tf similarity index 100% rename from modules/storage/data.tf rename to storage/data.tf diff --git a/modules/storage/locals.tf b/storage/locals.tf similarity index 100% rename from modules/storage/locals.tf rename to storage/locals.tf diff --git a/modules/storage/main.tf b/storage/main.tf similarity index 100% rename from modules/storage/main.tf rename to storage/main.tf diff --git a/modules/storage/misc.auto.tfvars.example b/storage/misc.auto.tfvars.example similarity index 100% rename from modules/storage/misc.auto.tfvars.example rename to storage/misc.auto.tfvars.example diff --git a/modules/storage/outputs.tf b/storage/outputs.tf similarity index 100% rename from modules/storage/outputs.tf rename to storage/outputs.tf diff --git a/modules/storage/providers.tf b/storage/providers.tf similarity index 100% rename from modules/storage/providers.tf rename to storage/providers.tf diff --git a/modules/storage/storage.auto.tfvars.example b/storage/storage.auto.tfvars.example similarity index 100% rename from modules/storage/storage.auto.tfvars.example rename to storage/storage.auto.tfvars.example diff --git a/modules/storage/variables.tf b/storage/variables.tf similarity index 100% rename from modules/storage/variables.tf rename to storage/variables.tf diff --git a/modules/storage/versions.tf b/storage/versions.tf similarity index 100% rename from modules/storage/versions.tf rename to storage/versions.tf diff --git a/modules/vnet/LICENSE b/vnet/LICENSE similarity index 100% rename from modules/vnet/LICENSE rename to vnet/LICENSE diff --git a/modules/vnet/README.md b/vnet/README.md similarity index 100% rename from modules/vnet/README.md rename to vnet/README.md diff --git a/modules/vnet/data.tf b/vnet/data.tf similarity index 100% rename from modules/vnet/data.tf rename to vnet/data.tf diff --git a/modules/vnet/locals.tf b/vnet/locals.tf similarity index 100% rename from modules/vnet/locals.tf rename to vnet/locals.tf diff --git a/modules/vnet/main.tf b/vnet/main.tf similarity index 100% rename from modules/vnet/main.tf rename to vnet/main.tf diff --git a/modules/vnet/misc.auto.tfvars.example b/vnet/misc.auto.tfvars.example similarity index 100% rename from modules/vnet/misc.auto.tfvars.example rename to vnet/misc.auto.tfvars.example diff --git a/modules/vnet/outputs.tf b/vnet/outputs.tf similarity index 100% rename from modules/vnet/outputs.tf rename to vnet/outputs.tf diff --git a/modules/vnet/providers.tf b/vnet/providers.tf similarity index 100% rename from modules/vnet/providers.tf rename to vnet/providers.tf diff --git a/modules/vnet/variables.tf b/vnet/variables.tf similarity index 100% rename from modules/vnet/variables.tf rename to vnet/variables.tf diff --git a/modules/vnet/versions.tf b/vnet/versions.tf similarity index 100% rename from modules/vnet/versions.tf rename to vnet/versions.tf diff --git a/modules/vnet/vnet.tf b/vnet/vnet.tf similarity index 100% rename from modules/vnet/vnet.tf rename to vnet/vnet.tf From f31f1c7b4f47ba1e69b53f2b0cf9a1d2761b7418 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 22 Sep 2023 16:11:53 +0000 Subject: [PATCH 004/124] branch:HPCC-27615-easy-deploy. Get is a merge of HPCC-27615 latest and branch 'tlh-bug-fixes-and-personalize-easy-deploy-v0' --- .gitignore.swp | Bin 0 -> 12288 bytes aks/variables.tf | 14 + hpcc/hpcc.tf | 84 +-- hpcc/lite-locals.tf | 531 +++++++++++++++ hpcc/lite-variables.tf | 227 +++++++ hpcc/lite.auto.tfvars.example | 237 +++++++ hpcc/locals.tf | 47 +- hpcc/main.tf | 20 +- hpcc/misc.auto.tfvars.example | 205 ------ hpcc/thor.auto.tfvars.example | 4 +- hpcc/variables.tf | 1151 --------------------------------- 11 files changed, 1092 insertions(+), 1428 deletions(-) create mode 100644 .gitignore.swp create mode 100755 hpcc/lite-locals.tf create mode 100644 hpcc/lite-variables.tf create mode 100755 hpcc/lite.auto.tfvars.example delete mode 100644 hpcc/misc.auto.tfvars.example delete mode 100644 hpcc/variables.tf diff --git a/.gitignore.swp b/.gitignore.swp new file mode 100644 index 0000000000000000000000000000000000000000..8c9c0f0cf78d086f4947d620debde28ac73ae27a GIT binary patch literal 12288 zcmeI2O>5LZ7{{l@qpfdWVCsTkV6$z#NWoLR6zomKn@p40B$}Ox^Rjk}FQ5n>J=dG! zRnRXWi0HLm?8$>?!O!4-l5F~dm0p!6@Mkv9JoC)|`6ZXIv$1${ZJl3^ml&?ojJ@5S zlRNV#*urzh#FjVGTWP}FUoKbX{Zx+oWug3Le>^%W^-))FfF*uZQD>9{xJgO+gi2xBeCV@G2X?bZeB=^oQ@N;Kx9CIC|5dk7V1c(3; zAOb{y2oM1x@IMf6)eL)#MrPY)dhIxKXl!q3AOb{y2oM1xKm>>Y5g-CYfCvx)B0vQG zK>{LWY;cmXT`2tjKmGpyaf-38&<^wgdJnyZUO_LQr_dAVBD6ct*eB>C^bUFhy@a-* zXV63F0dxU61Izm&lwyS1>8Vllga;`yc!TLmnD+6iVUc+w zTwvVT;x^a5O!duW< zF<;l#)qN&L*gP(E2IH&YH&A7b^_GJhT8GMUR6kTSSKl^$)<5@U@4bwJ@YY`o)M2^X NJ1vY%oHm8D>^D4SR>}YX literal 0 HcmV?d00001 diff --git a/aks/variables.tf b/aks/variables.tf index d2fa6f6..082ed75 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -1,3 +1,17 @@ +variable "availability_zones" { + description = "Availability zones to use for the node groups." + type = list(number) + nullable = false + default = [1] +} + +variable "tags" { + description = "Tags to apply to all resources." + type = map(string) + nullable = false + default = {} +} + variable "owner" { description = "Information for the user who administers the deployment." type = object({ diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index a1688af..9000f48 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,55 +1,63 @@ resource "kubernetes_namespace" "hpcc" { - count = var.hpcc_namespace.create_namespace && !fileexists("${path.module}/modules/logging/data/hpcc_namespace.txt") ? 1 : 0 + count = var.hpcc_namespace.create_namespace && !fileexists("../logging/data/hpcc_namespace.txt") ? 1 : 0 metadata { - labels = var.hpcc_namespace.labels - name = "${substr(trimspace(var.owner.name), 0, 5)}${random_integer.random.result}" - # generate_name = "${trimspace(var.owner.name)}" + labels = try(var.hpcc_namespace.labels,{}) + + generate_name = "${var.hpcc_namespace.prefix_name}${trimspace(local.owner.name)}" } } +/*resource "kubernetes_namespace" "hpcc" { + count = (var.hpcc_namespace == []) || !var.hpcc_namespace.create_namespace || fileexists("../logging/data/hpcc_namespace.txt") ? 0 : 1 + + metadata { + labels = try(var.hpcc_namespace.labels,{}) + name = "${substr(trimspace(local.owner.name), 0, 5)}${random_integer.random.result}" + # generate_name = "${trimspace(local.owner.name)}" + } +}*/ + module "hpcc" { - #source = "github.com/gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - #source = "/home/azureuser/godji/opinionated-terraform-azurerm-hpcc" source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - environment = var.metadata.environment - productname = var.metadata.product_name + environment = local.metadata.environment + productname = local.metadata.product_name - internal_domain = var.internal_domain + internal_domain = local.internal_domain cluster_name = local.get_aks_config.cluster_name hpcc_container = { - image_name = var.hpcc_container != null ? var.hpcc_container.image_name : null - image_root = var.hpcc_container != null ? var.hpcc_container.image_root : null - version = var.hpcc_container != null ? var.hpcc_container.version : null - custom_chart_version = var.hpcc_container != null ? var.hpcc_container.custom_chart_version : null - custom_image_version = var.hpcc_container != null ? var.hpcc_container.custom_image_version : null + image_name = local.hpcc_container != null ? local.hpcc_container.image_name : null + image_root = local.hpcc_container != null ? local.hpcc_container.image_root : null + version = local.hpcc_container != null ? local.hpcc_container.version : null + custom_chart_version = local.hpcc_container != null ? local.hpcc_container.custom_chart_version : null + custom_image_version = local.hpcc_container != null ? local.hpcc_container.custom_image_version : null } - hpcc_container_registry_auth = var.hpcc_container_registry_auth != null ? { - password = var.hpcc_container_registry_auth.password - username = var.hpcc_container_registry_auth.username + hpcc_container_registry_auth = local.hpcc_container_registry_auth != null ? { + password = local.hpcc_container_registry_auth.password + username = local.hpcc_container_registry_auth.username } : null install_blob_csi_driver = false //Disable CSI driver resource_group_name = local.get_aks_config.resource_group_name - location = var.metadata.location + location = local.metadata.location tags = module.metadata.tags # namespace = local.hpcc_namespace namespace = { create_namespace = false name = local.hpcc_namespace - labels = var.hpcc_namespace.labels + labels = try(var.hpcc_namespace.labels,{}) } admin_services_storage_account_settings = { - replication_type = var.admin_services_storage_account_settings.replication_type - authorized_ip_ranges = merge(var.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) - delete_protection = var.admin_services_storage_account_settings.delete_protection + replication_type = local.admin_services_storage_account_settings.replication_type + authorized_ip_ranges = merge(local.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) + delete_protection = local.admin_services_storage_account_settings.delete_protection subnet_ids = merge({ aks = local.subnet_ids.aks }) } @@ -58,18 +66,18 @@ module "hpcc" { data_storage_config = { internal = (local.external_storage_config == null) || (local.internal_storage_enabled == true) ? { blob_nfs = { - data_plane_count = var.data_storage_config.internal.blob_nfs.data_plane_count + data_plane_count = local.data_storage_config.internal.blob_nfs.data_plane_count storage_account_settings = { - replication_type = var.data_storage_config.internal.blob_nfs.storage_account_settings.replication_type - authorized_ip_ranges = merge(var.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) - delete_protection = var.data_storage_config.internal.blob_nfs.storage_account_settings.delete_protection + replication_type = local.data_storage_config.internal.blob_nfs.storage_account_settings.replication_type + authorized_ip_ranges = merge(local.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) + delete_protection = local.data_storage_config.internal.blob_nfs.storage_account_settings.delete_protection subnet_ids = merge({ aks = local.subnet_ids.aks }) } } - } : null + } : null # external = local.internal_data_storage_enabled ? null : { - # blob_nfs = local.get_storage_config != null ? local.get_storage_config.data_storage_planes : var.data_storage_config.external.blob_nfs + # blob_nfs = local.get_storage_config != null ? local.get_storage_config.data_storage_planes : local.data_storage_config.external.blob_nfs # hpcc = null # } external = null @@ -77,13 +85,13 @@ module "hpcc" { external_storage_config = local.external_storage_config - spill_volumes = var.spill_volumes - roxie_config = var.roxie_config - thor_config = var.thor_config - vault_config = var.vault_config - eclccserver_settings = var.eclccserver_settings - spray_service_settings = var.spray_service_settings - admin_services_node_selector = { all = { workload = var.spray_service_settings.nodeSelector } } + spill_volumes = local.spill_volumes + roxie_config = local.roxie_config + thor_config = local.thor_config + vault_config = local.vault_config + eclccserver_settings = local.eclccserver_settings + spray_service_settings = local.spray_service_settings + admin_services_node_selector = { all = { workload = local.spray_service_settings.nodeSelector } } esp_remoteclients = { @@ -101,7 +109,7 @@ module "hpcc" { } - helm_chart_timeout = var.helm_chart_timeout - helm_chart_files_overrides = concat(var.helm_chart_files_overrides, fileexists("${path.module}/modules/logging/data/logaccess_body.yaml") ? ["${path.module}/modules/logging/data/logaccess_body.yaml"] : []) - ldap_config = var.ldap_config + helm_chart_timeout = local.helm_chart_timeout + helm_chart_files_overrides = concat(local.helm_chart_files_overrides, fileexists("../logging/data/logaccess_body.yaml") ? ["../logging/data/logaccess_body.yaml"] : []) + ldap_config = local.ldap_config } diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf new file mode 100755 index 0000000..03ab423 --- /dev/null +++ b/hpcc/lite-locals.tf @@ -0,0 +1,531 @@ +locals { + helm_chart_timeout=600 + #hpcc_version = "8.6.20" + + owner = { + name = var.admin_username + email = var.admin_email + } + + metadata = { + project = var.product_name + product_name = var.product_name + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = "tlhhpcc" + resource_group_type = "app" + sre_team = var.product_name + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.azure_region # Acceptable values: eastus, centralus + } + + tags = merge(local.metadata.additional_tags, var.extra_tags) + + # # disable_naming_conventions - Disable naming conventions + # # disable_naming_conventions = true + disable_naming_conventions = false + + # # auto_launch_eclwatch - Automatically launch ECLWatch web interface. + #auto_launch_eclwatch = true + auto_launch_svc = { + eclwatch = false + } + + # azure_auth = { + # # AAD_CLIENT_ID = "" + # # AAD_CLIENT_SECRET = "" + # # AAD_TENANT_ID = "" + # # AAD_PRINCIPAL_ID = "" + # SUBSCRIPTION_ID = "" + # } + + # hpcc_container = { + # version = "9.2.0" + # image_name = "platform-core-ln" + # image_root = "jfrog.com/glb-docker-virtual" + # # custom_chart_version = "9.2.0-rc1" + # # custom_image_version = "9.2.0-demo" + # } + + # hpcc_container_registry_auth = { + # username = "value" + # password = "value" + # } + + internal_domain = var.dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io + + external = {} + # external = { + # blob_nfs = [{ + # container_id = "" + # container_name = "" + # id = "" + # resource_group_name = var.storage_account_resource_group_name + # storage_account_id = "" + # storage_account_name = var.storage_account_name + # }] + # # hpc_cache = [{ + # # id = "" + # # path = "" + # # server = "" + # }] + # hpcc = [{ + # name = "" + # planes = list(object({ + # local = "" + # remote = "" + # })) + # service = "" + # }] + # } + + admin_services_storage_account_settings = { + replication_type = "ZRS" #LRS only if using HPC Cache + authorized_ip_ranges = { + "default" = "0.0.0.0/0" //must be public IP + } + + delete_protection = false + } + + azure_log_analytics_creds = { + scope = null + object_id = "" //AAD_PRINCIPAL_ID + } + + data_storage_config = { + internal = { + blob_nfs = { + data_plane_count = 2 + storage_account_settings = { + replication_type = "ZRS" + delete_protection = false + } + } + # hpc_cache = { + # enabled = false + # size = "small" + # cache_update_frequency = "3h" + # storage_account_data_planes = null + # } + } + external = null + } + + + spill_volumes = { + spill = { + name = "spill" + size = 300 + prefix = "/var/lib/HPCCSystems/spill" + host_path = "/mnt" + storage_class = "spill" + access_mode = "ReadWriteOnce" + } + } + + spray_service_settings = { + replicas = 6 + nodeSelector = "spraypool" #"spraypool" + } + + # ldap = { + # ldap_server = "" //Server IP + # dali = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # esp = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # } + + #======================================== + # defaults in godji original variables.tf + expose_services = false + + auto_connect = false + + use_existing_vnet = null + + hpcc_enabled = true + + helm_chart_strings_overrides = [] + + helm_chart_files_overrides = [] + + vault_config = null + + hpcc_container = null + + hpcc_container_registry_auth = null + + roxie_config = [ + { + disabled = (var.enable_roxie == true)? false : true + name = "roxie" + nodeSelector = {} + numChannels = 2 + prefix = "roxie" + replicas = 2 + serverReplicas = 0 + acePoolSize = 6 + actResetLogPeriod = 0 + affinity = 0 + allFilesDynamic = false + blindLogging = false + blobCacheMem = 0 + callbackRetries = 3 + callbackTimeout = 500 + checkCompleted = true + checkFileDate = false + checkPrimaries = true + clusterWidth = 1 + copyResources = true + coresPerQuery = 0 + crcResources = false + dafilesrvLookupTimeout = 10000 + debugPermitted = true + defaultConcatPreload = 0 + defaultFetchPreload = 0 + defaultFullKeyedJoinPreload = 0 + defaultHighPriorityTimeLimit = 0 + defaultHighPriorityTimeWarning = 30000 + defaultKeyedJoinPreload = 0 + defaultLowPriorityTimeLimit = 0 + defaultLowPriorityTimeWarning = 90000 + defaultMemoryLimit = 1073741824 + defaultParallelJoinPreload = 0 + defaultPrefetchProjectPreload = 10 + defaultSLAPriorityTimeLimit = 0 + defaultSLAPriorityTimeWarning = 30000 + defaultStripLeadingWhitespace = false + diskReadBufferSize = 65536 + doIbytiDelay = true + egress = "engineEgress" + enableHeartBeat = false + enableKeyDiff = false + enableSysLog = false + fastLaneQueue = true + fieldTranslationEnabled = "payload" + flushJHtreeCacheOnOOM = true + forceStdLog = false + highTimeout = 2000 + ignoreMissingFiles = false + indexReadChunkSize = 60000 + initIbytiDelay = 10 + jumboFrames = false + lazyOpen = true + leafCacheMem = 500 + linuxYield = false + localFilesExpire = 1 + localSlave = false + logFullQueries = false + logQueueDrop = 32 + logQueueLen = 512 + lowTimeout = 10000 + maxBlockSize = 1000000000 + maxHttpConnectionRequests = 1 + maxLocalFilesOpen = 4000 + maxLockAttempts = 5 + maxRemoteFilesOpen = 100 + memTraceLevel = 1 + memTraceSizeLimit = 0 + memoryStatsInterval = 60 + minFreeDiskSpace = 6442450944 + minIbytiDelay = 2 + minLocalFilesOpen = 2000 + minRemoteFilesOpen = 50 + miscDebugTraceLevel = 0 + monitorDaliFileServer = false + nodeCacheMem = 1000 + nodeCachePreload = false + parallelAggregate = 0 + parallelLoadQueries = 1 + perChannelFlowLimit = 50 + pingInterval = 0 + preabortIndexReadsThreshold = 100 + preabortKeyedJoinsThreshold = 100 + preloadOnceData = true + prestartSlaveThreads = false + remoteFilesExpire = 3600 + roxieMulticastEnabled = false + serverSideCacheSize = 0 + serverThreads = 100 + simpleLocalKeyedJoins = true + sinkMode = "sequential" + slaTimeout = 2000 + slaveConfig = "simple" + slaveThreads = 30 + soapTraceLevel = 1 + socketCheckInterval = 5000 + statsExpiryTime = 3600 + systemMonitorInterval = 60000 + totalMemoryLimit = "5368709120" + traceLevel = 1 + traceRemoteFiles = false + trapTooManyActiveQueries = true + udpAdjustThreadPriorities = true + udpFlowAckTimeout = 10 + udpFlowSocketsSize = 33554432 + udpInlineCollation = true + udpInlineCollationPacketLimit = 50 + udpLocalWriteSocketSize = 16777216 + udpMaxPermitDeadTimeouts = 100 + udpMaxRetryTimedoutReqs = 10 + udpMaxSlotsPerClient = 100 + udpMulticastBufferSize = 33554432 + udpOutQsPriority = 5 + udpQueueSize = 1000 + udpRecvFlowTimeout = 2000 + udpRequestToSendAckTimeout = 500 + udpResendTimeout = 100 + udpRequestToSendTimeout = 2000 + udpResendEnabled = true + udpRetryBusySenders = 0 + udpSendCompletedInData = false + udpSendQueueSize = 500 + udpSnifferEnabled = false + udpTraceLevel = 0 + useAeron = false + useDynamicServers = false + useHardLink = false + useLogQueue = true + useMemoryMappedIndexes = false + useRemoteResources = false + useTreeCopy = false + services = [ + { + name = "roxie" + servicePort = 9876 + listenQueue = 200 + numThreads = 30 + visibility = "local" + annotations = {} + } + ] + topoServer = { + replicas = 1 + } + channelResources = { + cpu = "1" + memory = "4G" + } + } + ] + + eclagent_settings = { + hthor = { + replicas = 1 + maxActive = 4 + prefix = "hthor" + use_child_process = false + type = "hthor" + spillPlane = "spill" + resources = { + cpu = "1" + memory = "4G" + } + egress = "engineEgress" + cost = { + perCpu = 1 + } + }, + } + + eclccserver_settings = { + "myeclccserver" = { + useChildProcesses = false + maxActive = 4 + egress = "engineEgress" + replicas = 1 + childProcessTimeLimit = 10 + resources = { + cpu = "1" + memory = "4G" + } + legacySyntax = false + options = [] + cost = { + perCpu = 1 + } + } } + + dali_settings = { + coalescer = { + interval = 24 + at = "* * * * *" + minDeltaSize = 50000 + resources = { + cpu = "1" + memory = "4G" + } + } + resources = { + cpu = "2" + memory = "8G" + } + maxStartupTime = 1200 + } + + dfuserver_settings = { + maxJobs = 3 + resources = { + cpu = "1" + memory = "2G" + } + } + + sasha_config = { + disabled = false + wu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 6 + limit = 400 + cutoff = 3 + backup = 0 + at = "* * * * *" + throttle = 0 + retryinterval = 6 + keepResultFiles = false + # egress = "engineEgress" + } + + dfuwu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 24 + limit = 100 + cutoff = 14 + at = "* * * * *" + throttle = 0 + # egress = "engineEgress" + } + + dfurecovery-archiver = { + disabled = false + interval = 12 + limit = 20 + cutoff = 4 + at = "* * * * *" + # egress = "engineEgress" + } + + file-expiry = { + disabled = false + interval = 1 + at = "* * * * *" + persistExpiryDefault = 7 + expiryDefault = 4 + user = "sasha" + # egress = "engineEgress" + } + } + + ldap_config = null + + ldap_tunables = { + cacheTimeout = 5 + checkScopeScans = false + ldapTimeoutSecs = 131 + maxConnections = 10 + passwordExpirationWarningDays = 10 + sharedCache = true + } + + install_blob_csi_driver = true + + remote_storage_plane = null + + onprem_lz_settings = {} + + ignore_external_storage = true + + admin_services_node_selector = {} + + thor_config = [{ + disabled = ((var.thor_num_workers == 0 ) || (var.thor_num_workers == null))? true : false + name = "thor" + prefix = "thor" + numWorkers = var.thor_num_workers + keepJobs = "none" + maxJobs = var.thor_max_jobs + maxGraphs = 2 + maxGraphStartupTime = 172800 + numWorkersPerPod = 1 + nodeSelector = {} + egress = "engineEgress" + tolerations_value = "thorpool" + managerResources = { + cpu = 1 + memory = "2G" + } + workerResources = { + cpu = 3 + memory = "4G" + } + workerMemory = { + query = "3G" + thirdParty = "500M" + } + eclAgentResources = { + cpu = 1 + memory = "2G" + } + cost = { + perCpu = 1 + } + }] + + admin_services_storage = { + dali = { + size = 100 + type = "azurefiles" + } + debug = { + size = 100 + type = "blobnfs" + } + dll = { + size = 100 + type = "blobnfs" + } + lz = { + size = var.storage_lz_gb + type = "blobnfs" + } + sasha = { + size = 100 + type = "blobnfs" + } + } + #======================================== +} diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf new file mode 100644 index 0000000..4a2d34a --- /dev/null +++ b/hpcc/lite-variables.tf @@ -0,0 +1,227 @@ +############################################################################### +# Prompted variables (user will be asked to supply them at plan/apply time +# if a .tfvars file is not supplied); there are no default values +############################################################################### + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + +variable "admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool +} + +variable "enable_rbac_ad" { + description = "REQUIRED. Enable RBAC and AD integration for AKS?\nThis provides additional security for accessing the Kubernetes cluster and settings (not HPCC Systems' settings).\nValue type: boolean\nRecommended value: true\nExample entry: true" + type = bool +} + +variable "enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "REQUIRED. The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported." + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } +} + +variable "max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} + +variable "product_name" { + type = string + description = "REQUIRED. Abbreviated product name, suitable for use in Azure naming.\nMust be 3-16, all lowercase or numeric characters.\nExample entry: myproduct" + validation { + condition = can(regex("^[a-z][a-z0-9]{2,15}$", var.product_name)) + error_message = "Value must be [a-z0-9]{3,16}." + } +} + +variable "storage_data_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_data_gb >= 10 + error_message = "Value must be 10 or more." + } +} + +variable "storage_lz_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for the landing zone in gigabytes.\nMust be 1 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_lz_gb >= 1 + error_message = "Value must be 1 or more." + } +} + +variable "thor_max_jobs" { + type = number + description = "REQUIRED. The maximum number of simultaneous Thor jobs allowed.\nMust be 1 or more." + validation { + condition = var.thor_max_jobs >= 1 + error_message = "Value must be 1 or more." + } +} + +variable "thor_num_workers" { + type = number + description = "REQUIRED. The number of Thor workers to allocate.\nMust be 1 or more." + validation { + condition = var.thor_num_workers >= 1 + error_message = "Value must be 1 or more." + } +} + +############################################################################### +# Optional variables +############################################################################### + +variable "authn_htpasswd_filename" { + type = string + description = "OPTIONAL. If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it.\nA corollary is that persistent storage is enabled.\nAn empty string indicates that htpasswd is not to be used for authentication.\nExample entry: htpasswd.txt" + default = "" +} + +variable "hpcc_image_name" { + type = string + description = "REQUIRED. The global image name of the HPCC docker image to deploy.\nMust be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"].\nDefault value: platform-core" + default = "platform-core" + validation { + condition = contains(["platform-core", "platform-ml", "platform-gnn"], var.hpcc_image_name) + error_message = "Value must be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"]." + } +} + +/*variable "hpcc_namespace" { + type = string + description = "REQUIRED. The Kubernetes namespace in which to install the HPCC modules (if enabled).\nDefault value: default" + default = "default" + validation { + condition = var.hpcc_namespace != "" + error_message = "Namespace must be a non-empty string." + } +}*/ +/*variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + existing_namespace = optional(string) + labels = optional(map(string), { name = "hpcc" }) + create_namespace = optional(bool, true) + }) + default = {} +}*/ +variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + prefix_name = string + labels = map(string) + create_namespace = bool + }) + default = { + prefix_name = "hpcc" + labels = { + name = "hpcc" + } + create_namespace = false + } +} + +variable "enable_premium_storage" { + type = bool + description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." + default = false +} + +variable "storage_account_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" + default = "" +} + +variable "storage_account_resource_group_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." + default = "" +} diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example new file mode 100755 index 0000000..d4586d1 --- /dev/null +++ b/hpcc/lite.auto.tfvars.example @@ -0,0 +1,237 @@ +# To have a dns zone and an A record, included in your apply, the following 3 +# variables, i.e. 'a_record_name', 'dns_zone_name', and +# dns_zone_resource_group_name must be set to non-blank or non-null values. +#----------------------------------------------------------------------------- + +# Name of the A record, of following dns zone, where the ecl watch ip is placed +# This A record will be created and therefore should not exist in the following +# dns zone. +# Example entry: "my-product". This should be something project specific rather +# than something generic. + +a_record_name="" + +#----------------------------------------------------------------------------- + +# Name of an existing dns zone. +# Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" + +dns_zone_name="" + +#----------------------------------------------------------------------------- + +# Name of the resource group of the above dns zone. +# Example entry: "app-dns-prod-eastus2" + +dns_zone_resource_group_name="" + +#------------------------------------------------------------------------------ + +# Abbreviated product name, suitable for use in Azure naming. +# Must be 3-16 characters in length, all lowercase letters or numbers, no spaces. +# Value type: string +# Example entry: "my-product" + +product_name="tlhhpcc" + +#------------------------------------------------------------------------------ + +# The version of HPCC Systems to install. +# Only versions in nn.nn.nn format are supported. +# Value type: string + +hpcc_version="8.6.14" +#hpcc_version="latest" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. +#hpcc_version="8.10.40" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. + +#------------------------------------------------------------------------------ + +# Enable ROXIE? +# This will also expose port 8002 on the cluster. +# Value type: boolean +# Example entry: false + +enable_roxie=false + +#------------------------------------------------------------------------------ + +# Enable RBAC and AD integration for AKS? +# This provides additional security for accessing the Kubernetes cluster and settings (not HPCC Systems' settings). +# Value type: boolean +# Recommended value: true +# Example entry: true + +enable_rbac_ad=false + +#------------------------------------------------------------------------------ + +# Enable code security? +# If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. +# Value type: boolean +# Example entry: false + +enable_code_security=true + +#------------------------------------------------------------------------------ + +# The number of Thor workers to allocate. +# Must be 1 or more. + +thor_num_workers=2 + +#------------------------------------------------------------------------------ + +# The maximum number of simultaneous Thor jobs allowed. +# Must be 1 or more. + +thor_max_jobs=2 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for the landing zone in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_lz_gb=25 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for data in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_data_gb=100 + +#------------------------------------------------------------------------------ + +# Map of name => value tags that can will be associated with the cluster. +# Format is '{"name"="value" [, "name"="value"]*}'. +# The 'name' portion must be unique. +# To add no tags, use '{}'. +# Value type: map of string + +extra_tags={} + +#------------------------------------------------------------------------------ + +# The VM size for each node in the HPCC Systems node pool. +# Recommend "Standard_B4ms" or better. +# See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. +# Value type: string + +node_size="Standard_B8ms" + +#------------------------------------------------------------------------------ + +# The maximum number of VM nodes to allocate for the HPCC Systems node pool. +# Must be 2 or more. +# Value type: integer + +max_node_count=4 + +#------------------------------------------------------------------------------ + +# Email address of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jane.doe@hpccsystems.com" + +admin_email="tlhumphrey2@gmail.com" + +#------------------------------------------------------------------------------ + +# Name of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "Jane Doe" + +admin_name="Timothy Humphrey" + +#------------------------------------------------------------------------------ + +# Username of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jdoe" + +admin_username="tlhumphrey2" + +#------------------------------------------------------------------------------ + +# The Azure region abbreviation in which to create these resources. +# Must be one of ["eastus", "eastus2", "centralus"]. +# Value type: string +# Example entry: "eastus" + +azure_region="eastus" + +#------------------------------------------------------------------------------ + +# Map of name => CIDR IP addresses that can administrate this AKS. +# Format is '{"name"="cidr" [, "name"="cidr"]*}'. +# The 'name' portion must be unique. +# To add no CIDR addresses, use '{}'. +# The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. +# Value type: map of string + +admin_ip_cidr_map={} + +#------------------------------------------------------------------------------ + +# List of explicit CIDR addresses that can access this HPCC Systems cluster. +# To allow public access, specify "0.0.0.0/0". +# To add no CIDR addresses, use '[]'. +# Value type: list of string + +hpcc_user_ip_cidr_list=[ "20.14.220.189/32", "66.241.32.0/24"] + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a resource group for the storage account. +# Value type: string +# Example entry: "my-product-sa" + +storage_account_name="" + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its resource group name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a name for the storage account. +# Value type: string + +storage_account_resource_group_name="" + +#============================================================================== +# Optional settings +#============================================================================== + +# The global image name of the HPCC docker image to deploy. +# Must be one of ["platform-core", "platform-ml", "platform-gnn"]. +# Default value: "platform-core" + +# hpcc_image_name="platform-core" + +#------------------------------------------------------------------------------ + +# The Kubernetes namespace in which to install the HPCC modules (if enabled). +# Default value: "default" + +# hpcc_namespace="default" + +#------------------------------------------------------------------------------ + +# If true, premium ($$$) storage will be used for the following storage shares: Dali. +# OPTIONAL, defaults to false. + +enable_premium_storage=false + +#------------------------------------------------------------------------------ + +# If you would like to use htpasswd to authenticate users to the cluster, enter +# the filename of the htpasswd file. This file should be uploaded to the Azure +# 'dllsshare' file share in order for the HPCC processes to find it. +# A corollary is that persistent storage is enabled. +# An empty string indicates that htpasswd is not to be used for authentication. +# Example entry: "htpasswd.txt" + +authn_htpasswd_filename="" diff --git a/hpcc/locals.tf b/hpcc/locals.tf index a96e457..1e3dd9a 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -4,45 +4,46 @@ locals { AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id } - names = var.disable_naming_conventions ? merge( + names = try(local.disable_naming_conventions, false) ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.metadata.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) - - # external_services_storage_exists = fileexists("../storage/data/config.json") || var.external_services_storage_config != null + # external_services_storage_exists = fileexists("../storage/data/config.json") || local.external_services_storage_config != null get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null + #get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null + get_storage_config = fileexists("../storage/data/config.json") ? jsondecode(file("../storage/data/config.json")) : null - external_storage_exists = fileexists("../storage/data/config.json") || var.external_storage_config != null + external_storage_exists = fileexists("../storage/data/config.json") || local.external_storage_config != null subnet_ids = try({ - for k, v in var.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${var.use_existing_vnet.name}/subnets/${v.name}" + for k, v in local.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${local.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${local.use_existing_vnet.name}/subnets/${v.name}" }, { aks = local.get_vnet_config.private_subnet_id }) - location = var.use_existing_vnet != null ? var.use_existing_vnet.location : local.get_vnet_config.location + location = local.use_existing_vnet != null ? local.use_existing_vnet.location : local.get_vnet_config.location - # hpcc_chart_major_minor_point_version = var.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", var.helm_chart_version) : "master" + # hpcc_chart_major_minor_point_version = local.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", local.helm_chart_version) : "master" - domain = coalesce(var.internal_domain, format("us-%s.%s.azure.lnrsg.io", "var.metadata.product_name", "dev")) + domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) - internal_storage_enabled = (local.external_storage_exists == true) && (var.ignore_external_storage == true) ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true - # external_services_storage_enabled = local.external_services_storage_exists == true && var.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && var.ignore_external_services_storage == true ? false : true + internal_storage_enabled = (local.external_storage_exists == true) && (local.ignore_external_storage == true) ? true : local.external_storage_exists == true && local.ignore_external_storage == false ? false : true + #internal_storage_enabled = local.external_storage_exists == true && local.ignore_external_storage == true ? true : local.external_storage_exists == true && local.ignore_external_storage == false ? false : true + # external_services_storage_enabled = local.external_services_storage_exists == true && local.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && local.ignore_external_services_storage == true ? false : true - hpcc_namespace = var.hpcc_namespace.existing_namespace != null ? var.hpcc_namespace.existing_namespace : var.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("${path.module}/logging/data/hpcc_namespace.txt") ? file("${path.module}/logging/data/hpcc_namespace.txt") : "default" + #hpcc_namespace = local.hpcc_namespace.existing_namespace != null ? local.hpcc_namespace.existing_namespace : local.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("../logging/data/hpcc_namespace.txt") ? file("../logging/data/hpcc_namespace.txt") : "default" + hpcc_namespace = "default" - external_storage_config = local.get_storage_config != null && var.ignore_external_storage == false ? [ + external_storage_config = local.get_storage_config != null && local.ignore_external_storage == false ? [ for plane in local.get_storage_config.external_storage_config : { category = plane.category @@ -58,6 +59,6 @@ locals { } ] : [] - svc_domains = { eclwatch = var.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } + svc_domains = { eclwatch = local.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true } diff --git a/hpcc/main.tf b/hpcc/main.tf index aa36ca0..253197b 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -17,21 +17,21 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market + market = local.metadata.market location = local.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = module.subscription.output.subscription_id - project = var.metadata.project + project = local.metadata.project } resource "null_resource" "launch_svc_url" { - for_each = (module.hpcc.hpcc_status == "deployed") && (var.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} + for_each = (module.hpcc.hpcc_status == "deployed") && (local.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} provisioner "local-exec" { command = local.is_windows_os ? "Start-Process ${each.value}" : "open ${each.value} || xdg-open ${each.value}" diff --git a/hpcc/misc.auto.tfvars.example b/hpcc/misc.auto.tfvars.example deleted file mode 100644 index c313216..0000000 --- a/hpcc/misc.auto.tfvars.example +++ /dev/null @@ -1,205 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing", "enclosed resource" = "hpcc" } - location = "eastus" # Acceptable values: eastus, centralus -} - -# # disable_naming_conventions - Disable naming conventions -# # disable_naming_conventions = true - -# # auto_launch_svc - Automatically launch ECLWatch web interface. -auto_launch_svc = { - eclwatch = true -} - -# azure_auth = { -# # AAD_CLIENT_ID = "" -# # AAD_CLIENT_SECRET = "" -# # AAD_TENANT_ID = "" -# # AAD_PRINCIPAL_ID = "" -# SUBSCRIPTION_ID = "" -# } - -# hpcc_container = { -# version = "9.2.0" -# image_name = "platform-core-ln" -# image_root = "jfrog.com/glb-docker-virtual" -# # custom_chart_version = "9.2.0-rc1" -# # custom_image_version = "9.2.0-demo" -# } - -# hpcc_container_registry_auth = { -# username = "value" -# password = "value" -# } - -internal_domain = "" // Example: hpccplatform-dev.azure.com - -# external = { -# blob_nfs = [{ -# container_id = "" -# container_name = "" -# id = "" -# resource_group_name = "" -# storage_account_id = "" -# storage_account_name = "" -# }] -# hpcc = [{ -# name = "" -# planes = list(object({ -# local = "" -# remote = "" -# })) -# service = "" -# }] -# } - -admin_services_storage_account_settings = { - replication_type = "ZRS" #LRS only if using HPC Cache - # authorized_ip_ranges = { - # "default" = "0.0.0.0/0" //must be public IP - # } - - delete_protection = false -} - -azure_log_analytics_creds = { - scope = null - object_id = "" //AAD_PRINCIPAL_ID -} - -hpcc_namespace = { - # existing_namespace = "" - labels = { - name = "hpcc" - } - create_namespace = true -} - -data_storage_config = { - internal = { - blob_nfs = { - data_plane_count = 2 - storage_account_settings = { - replication_type = "ZRS" - delete_protection = false - } - } - } - external = null -} - -# external_services_storage_config = [ -# { -# category = "dali" -# container_name = "hpcc-dali" -# path = "dalistorage" -# plane_name = "dali" -# size = 100 -# storage_type = "azurefiles" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "debug" -# container_name = "hpcc-debug" -# path = "debug" -# plane_name = "debug" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "dll" -# container_name = "hpcc-dll" -# path = "queries" -# plane_name = "dll" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "lz" -# container_name = "hpcc-mydropzone" -# path = "mydropzone" -# plane_name = "mydropzone" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "sasha" -# container_name = "hpcc-sasha" -# path = "sashastorage" -# plane_name = "sasha" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# } -# ] - -ignore_external_data_storage = false -ignore_external_services_storage = false - -spill_volumes = { - spill = { - name = "spill" - size = 300 - prefix = "/var/lib/HPCCSystems/spill" - host_path = "/mnt" - storage_class = "spill" - access_mode = "ReadWriteOnce" - } -} - -spray_service_settings = { - replicas = 6 - nodeSelector = "spraypool" #"spraypool" -} - -# ldap = { -# ldap_server = "" //Server IP -# dali = { -# hpcc_admin_password = "" -# hpcc_admin_username = "" -# ldap_admin_password = "" -# ldap_admin_username = "" -# adminGroupName = "HPCC-Admins" -# filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# } -# esp = { -# hpcc_admin_password = "" -# hpcc_admin_username = "" -# ldap_admin_password = "" -# ldap_admin_username = "" -# adminGroupName = "HPCC-Admins" -# filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# } -# } diff --git a/hpcc/thor.auto.tfvars.example b/hpcc/thor.auto.tfvars.example index 34ff029..d44d438 100644 --- a/hpcc/thor.auto.tfvars.example +++ b/hpcc/thor.auto.tfvars.example @@ -11,7 +11,9 @@ thor_config = [{ maxGraphs = 2 maxGraphStartupTime = 172800 numWorkersPerPod = 1 - # nodeSelector = {} + nodeSelector = {} + egress = "engineEgress" + tolerations_value = "thorpool" managerResources = { cpu = 1 memory = "2G" diff --git a/hpcc/variables.tf b/hpcc/variables.tf deleted file mode 100644 index 60de38c..0000000 --- a/hpcc/variables.tf +++ /dev/null @@ -1,1151 +0,0 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - -# variable "azure_auth" { -# description = "Azure authentication" -# type = object({ -# AAD_CLIENT_ID = optional(string) -# AAD_CLIENT_SECRET = optional(string) -# AAD_TENANT_ID = optional(string) -# AAD_PRINCIPAL_ID = optional(string) -# SUBSCRIPTION_ID = string -# }) - -# nullable = false -# } - -variable "expose_services" { - description = "Expose ECLWatch and elastic4hpcclogs to the Internet. This is not secure. Please consider before using it." - type = bool - default = false -} - -variable "auto_launch_svc" { - description = "Auto launch HPCC services." - type = object({ - eclwatch = bool - }) - default = { - eclwatch = true - } -} - -variable "auto_connect" { - description = "Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context." - type = bool - default = false -} - -variable "disable_naming_conventions" { - description = "Naming convention module." - type = bool - default = false -} - -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - -variable "use_existing_vnet" { - description = "Information about the existing VNet to use. Overrides vnet variable." - type = object({ - name = string - resource_group_name = string - route_table_name = string - location = string - - subnets = object({ - aks = object({ - name = string - }) - }) - }) - - default = null -} - -## HPCC Helm Release -####################### -variable "hpcc_enabled" { - description = "Is HPCC Platform deployment enabled?" - type = bool - default = true -} - -variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - existing_namespace = optional(string) - labels = optional(map(string), { name = "hpcc" }) - create_namespace = optional(bool, true) - }) - default = {} -} - -variable "helm_chart_strings_overrides" { - description = "Helm chart values as strings, in yaml format, to be merged last." - type = list(string) - default = [] -} - -variable "helm_chart_files_overrides" { - description = "Helm chart values files, in yaml format, to be merged." - type = list(string) - default = [] -} - -variable "helm_chart_timeout" { - description = "Helm timeout for hpcc chart." - type = number - default = 300 -} - -variable "hpcc_container" { - description = "HPCC container information (if version is set to null helm chart version is used)." - type = object({ - image_name = optional(string) - image_root = optional(string) - version = optional(string) - custom_chart_version = optional(string) - custom_image_version = optional(string) - }) - - default = null -} - -variable "hpcc_container_registry_auth" { - description = "Registry authentication for HPCC container." - type = object({ - password = string - username = string - }) - default = null - sensitive = true -} - -variable "vault_config" { - description = "Input for vault secrets." - type = object({ - git = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })), - ecl = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })), - ecluser = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })) - esp = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })) - }) - default = null -} - -## Roxie Config -################## -variable "roxie_config" { - description = "Configuration for Roxie(s)." - type = list(object({ - disabled = bool - name = string - nodeSelector = map(string) - numChannels = number - prefix = string - replicas = number - serverReplicas = number - acePoolSize = number - actResetLogPeriod = number - affinity = number - allFilesDynamic = bool - blindLogging = bool - blobCacheMem = number - callbackRetries = number - callbackTimeout = number - checkCompleted = bool - checkPrimaries = bool - checkFileDate = bool - clusterWidth = number - copyResources = bool - coresPerQuery = number - crcResources = bool - dafilesrvLookupTimeout = number - debugPermitted = bool - defaultConcatPreload = number - defaultFetchPreload = number - defaultFullKeyedJoinPreload = number - defaultHighPriorityTimeLimit = number - defaultHighPriorityTimeWarning = number - defaultKeyedJoinPreload = number - defaultLowPriorityTimeLimit = number - defaultLowPriorityTimeWarning = number - defaultMemoryLimit = number - defaultParallelJoinPreload = number - defaultPrefetchProjectPreload = number - defaultSLAPriorityTimeLimit = number - defaultSLAPriorityTimeWarning = number - defaultStripLeadingWhitespace = bool - diskReadBufferSize = number - doIbytiDelay = bool - egress = string - enableHeartBeat = bool - enableKeyDiff = bool - enableSysLog = bool - fastLaneQueue = bool - fieldTranslationEnabled = string - flushJHtreeCacheOnOOM = bool - forceStdLog = bool - highTimeout = number - ignoreMissingFiles = bool - indexReadChunkSize = number - initIbytiDelay = number - jumboFrames = bool - lazyOpen = bool - leafCacheMem = number - linuxYield = bool - localFilesExpire = number - localSlave = bool - logFullQueries = bool - logQueueDrop = number - logQueueLen = number - lowTimeout = number - maxBlockSize = number - maxHttpConnectionRequests = number - maxLocalFilesOpen = number - maxLockAttempts = number - maxRemoteFilesOpen = number - memTraceLevel = number - memTraceSizeLimit = number - memoryStatsInterval = number - minFreeDiskSpace = number - minIbytiDelay = number - minLocalFilesOpen = number - minRemoteFilesOpen = number - miscDebugTraceLevel = number - monitorDaliFileServer = bool - nodeCacheMem = number - nodeCachePreload = bool - parallelAggregate = number - parallelLoadQueries = number - perChannelFlowLimit = number - pingInterval = number - preabortIndexReadsThreshold = number - preabortKeyedJoinsThreshold = number - preloadOnceData = bool - prestartSlaveThreads = bool - remoteFilesExpire = number - roxieMulticastEnabled = bool - serverSideCacheSize = number - serverThreads = number - simpleLocalKeyedJoins = bool - sinkMode = string - slaTimeout = number - slaveConfig = string - slaveThreads = number - soapTraceLevel = number - socketCheckInterval = number - statsExpiryTime = number - systemMonitorInterval = number - traceLevel = number - traceRemoteFiles = bool - totalMemoryLimit = string - trapTooManyActiveQueries = bool - udpAdjustThreadPriorities = bool - udpFlowAckTimeout = number - udpFlowSocketsSize = number - udpInlineCollation = bool - udpInlineCollationPacketLimit = number - udpLocalWriteSocketSize = number - udpMaxPermitDeadTimeouts = number - udpMaxRetryTimedoutReqs = number - udpMaxSlotsPerClient = number - udpMulticastBufferSize = number - udpOutQsPriority = number - udpQueueSize = number - udpRecvFlowTimeout = number - udpRequestToSendAckTimeout = number - udpResendTimeout = number - udpRequestToSendTimeout = number - udpResendEnabled = bool - udpRetryBusySenders = number - udpSendCompletedInData = bool - udpSendQueueSize = number - udpSnifferEnabled = bool - udpTraceLevel = number - useAeron = bool - useDynamicServers = bool - useHardLink = bool - useLogQueue = bool - useRemoteResources = bool - useMemoryMappedIndexes = bool - useTreeCopy = bool - services = list(object({ - name = string - servicePort = number - listenQueue = number - numThreads = number - visibility = string - annotations = optional(map(string)) - })) - topoServer = object({ - replicas = number - }) - channelResources = object({ - cpu = string - memory = string - }) - })) - - default = [ - { - disabled = false - name = "roxie" - nodeSelector = {} - numChannels = 2 - prefix = "roxie" - replicas = 2 - serverReplicas = 0 - acePoolSize = 6 - actResetLogPeriod = 0 - affinity = 0 - allFilesDynamic = false - blindLogging = false - blobCacheMem = 0 - callbackRetries = 3 - callbackTimeout = 500 - checkCompleted = true - checkFileDate = false - checkPrimaries = true - clusterWidth = 1 - copyResources = true - coresPerQuery = 0 - crcResources = false - dafilesrvLookupTimeout = 10000 - debugPermitted = true - defaultConcatPreload = 0 - defaultFetchPreload = 0 - defaultFullKeyedJoinPreload = 0 - defaultHighPriorityTimeLimit = 0 - defaultHighPriorityTimeWarning = 30000 - defaultKeyedJoinPreload = 0 - defaultLowPriorityTimeLimit = 0 - defaultLowPriorityTimeWarning = 90000 - defaultMemoryLimit = 1073741824 - defaultParallelJoinPreload = 0 - defaultPrefetchProjectPreload = 10 - defaultSLAPriorityTimeLimit = 0 - defaultSLAPriorityTimeWarning = 30000 - defaultStripLeadingWhitespace = false - diskReadBufferSize = 65536 - doIbytiDelay = true - egress = "engineEgress" - enableHeartBeat = false - enableKeyDiff = false - enableSysLog = false - fastLaneQueue = true - fieldTranslationEnabled = "payload" - flushJHtreeCacheOnOOM = true - forceStdLog = false - highTimeout = 2000 - ignoreMissingFiles = false - indexReadChunkSize = 60000 - initIbytiDelay = 10 - jumboFrames = false - lazyOpen = true - leafCacheMem = 500 - linuxYield = false - localFilesExpire = 1 - localSlave = false - logFullQueries = false - logQueueDrop = 32 - logQueueLen = 512 - lowTimeout = 10000 - maxBlockSize = 1000000000 - maxHttpConnectionRequests = 1 - maxLocalFilesOpen = 4000 - maxLockAttempts = 5 - maxRemoteFilesOpen = 100 - memTraceLevel = 1 - memTraceSizeLimit = 0 - memoryStatsInterval = 60 - minFreeDiskSpace = 6442450944 - minIbytiDelay = 2 - minLocalFilesOpen = 2000 - minRemoteFilesOpen = 50 - miscDebugTraceLevel = 0 - monitorDaliFileServer = false - nodeCacheMem = 1000 - nodeCachePreload = false - parallelAggregate = 0 - parallelLoadQueries = 1 - perChannelFlowLimit = 50 - pingInterval = 0 - preabortIndexReadsThreshold = 100 - preabortKeyedJoinsThreshold = 100 - preloadOnceData = true - prestartSlaveThreads = false - remoteFilesExpire = 3600 - roxieMulticastEnabled = false - serverSideCacheSize = 0 - serverThreads = 100 - simpleLocalKeyedJoins = true - sinkMode = "sequential" - slaTimeout = 2000 - slaveConfig = "simple" - slaveThreads = 30 - soapTraceLevel = 1 - socketCheckInterval = 5000 - statsExpiryTime = 3600 - systemMonitorInterval = 60000 - totalMemoryLimit = "5368709120" - traceLevel = 1 - traceRemoteFiles = false - trapTooManyActiveQueries = true - udpAdjustThreadPriorities = true - udpFlowAckTimeout = 10 - udpFlowSocketsSize = 33554432 - udpInlineCollation = true - udpInlineCollationPacketLimit = 50 - udpLocalWriteSocketSize = 16777216 - udpMaxPermitDeadTimeouts = 100 - udpMaxRetryTimedoutReqs = 10 - udpMaxSlotsPerClient = 100 - udpMulticastBufferSize = 33554432 - udpOutQsPriority = 5 - udpQueueSize = 1000 - udpRecvFlowTimeout = 2000 - udpRequestToSendAckTimeout = 500 - udpResendTimeout = 100 - udpRequestToSendTimeout = 2000 - udpResendEnabled = true - udpRetryBusySenders = 0 - udpSendCompletedInData = false - udpSendQueueSize = 500 - udpSnifferEnabled = false - udpTraceLevel = 0 - useAeron = false - useDynamicServers = false - useHardLink = false - useLogQueue = true - useMemoryMappedIndexes = false - useRemoteResources = false - useTreeCopy = false - services = [ - { - name = "roxie" - servicePort = 9876 - listenQueue = 200 - numThreads = 30 - visibility = "local" - annotations = {} - } - ] - topoServer = { - replicas = 1 - } - channelResources = { - cpu = "1" - memory = "4G" - } - } - ] -} - -## Thor Config -################## -variable "thor_config" { - description = "Configurations for Thor." - type = list(object( - { - disabled = bool - eclAgentResources = optional(object({ - cpu = string - memory = string - } - ), - { - cpu = 1 - memory = "2G" - }) - keepJobs = optional(string, "none") - managerResources = optional(object({ - cpu = string - memory = string - }), - { - cpu = 1 - memory = "2G" - }) - maxGraphs = optional(number, 2) - maxJobs = optional(number, 4) - maxGraphStartupTime = optional(number, 172800) - name = optional(string, "thor") - nodeSelector = optional(map(string), { workload = "thorpool" }) - numWorkers = optional(number, 2) - numWorkersPerPod = optional(number, 1) - prefix = optional(string, "thor") - egress = optional(string, "engineEgress") - tolerations_value = optional(string, "thorpool") - workerMemory = optional(object({ - query = string - thirdParty = string - }), - { - query = "3G" - thirdParty = "500M" - }) - workerResources = optional(object({ - cpu = string - memory = string - }), - { - cpu = 3 - memory = "4G" - }) - cost = object({ - perCpu = number - }) - })) - - default = null -} - -## ECL Agent Config -####################### -variable "eclagent_settings" { - description = "eclagent settings" - type = map(object({ - replicas = number - maxActive = number - prefix = string - use_child_process = bool - spillPlane = optional(string, "spill") - type = string - resources = object({ - cpu = string - memory = string - }) - cost = object({ - perCpu = number - }) - egress = optional(string) - })) - - default = { - hthor = { - replicas = 1 - maxActive = 4 - prefix = "hthor" - use_child_process = false - type = "hthor" - spillPlane = "spill" - resources = { - cpu = "1" - memory = "4G" - } - egress = "engineEgress" - cost = { - perCpu = 1 - } - }, - } -} - -## ECLCCServer Config -######################## -variable "eclccserver_settings" { - description = "Set cpu and memory values of the eclccserver. Toggle use_child_process to true to enable eclccserver child processes." - type = map(object({ - useChildProcesses = optional(bool, false) - replicas = optional(number, 1) - maxActive = optional(number, 4) - egress = optional(string, "engineEgress") - gitUsername = optional(string, "") - defaultRepo = optional(string, "") - defaultRepoVersion = optional(string, "") - resources = optional(object({ - cpu = string - memory = string - })) - cost = object({ - perCpu = number - }) - listen_queue = optional(list(string), []) - childProcessTimeLimit = optional(number, 10) - gitUsername = optional(string, "") - legacySyntax = optional(bool, false) - options = optional(list(object({ - name = string - value = string - }))) - })) - - default = { - "myeclccserver" = { - useChildProcesses = false - maxActive = 4 - egress = "engineEgress" - replicas = 1 - childProcessTimeLimit = 10 - resources = { - cpu = "1" - memory = "4G" - } - legacySyntax = false - options = [] - cost = { - perCpu = 1 - } - } } -} - -## Dali Config -################## -variable "dali_settings" { - description = "dali settings" - type = object({ - coalescer = object({ - interval = number - at = string - minDeltaSize = number - resources = object({ - cpu = string - memory = string - }) - }) - resources = object({ - cpu = string - memory = string - }) - maxStartupTime = number - }) - - default = { - coalescer = { - interval = 24 - at = "* * * * *" - minDeltaSize = 50000 - resources = { - cpu = "1" - memory = "4G" - } - } - resources = { - cpu = "2" - memory = "8G" - } - maxStartupTime = 1200 - } -} - -## DFU Server Config -######################## -variable "dfuserver_settings" { - description = "DFUServer settings" - type = object({ - maxJobs = number - resources = object({ - cpu = string - memory = string - }) - }) - - default = { - maxJobs = 3 - resources = { - cpu = "1" - memory = "2G" - } - } -} - -## Spray Service Config -######################### -variable "spray_service_settings" { - description = "spray services settings" - type = object({ - replicas = number - nodeSelector = string - }) - - default = { - replicas = 3 - nodeSelector = "servpool" #"spraypool" - } -} - -## Sasha Config -################## -variable "sasha_config" { - description = "Configuration for Sasha." - type = object({ - disabled = bool - wu-archiver = object({ - disabled = bool - service = object({ - servicePort = number - }) - plane = string - interval = number - limit = number - cutoff = number - backup = number - at = string - throttle = number - retryinterval = number - keepResultFiles = bool - # egress = string - }) - - dfuwu-archiver = object({ - disabled = bool - service = object({ - servicePort = number - }) - plane = string - interval = number - limit = number - cutoff = number - at = string - throttle = number - # egress = string - }) - - dfurecovery-archiver = object({ - disabled = bool - interval = number - limit = number - cutoff = number - at = string - # egress = string - }) - - file-expiry = object({ - disabled = bool - interval = number - at = string - persistExpiryDefault = number - expiryDefault = number - user = string - # egress = string - }) - }) - - default = { - disabled = false - wu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 6 - limit = 400 - cutoff = 3 - backup = 0 - at = "* * * * *" - throttle = 0 - retryinterval = 6 - keepResultFiles = false - # egress = "engineEgress" - } - - dfuwu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 24 - limit = 100 - cutoff = 14 - at = "* * * * *" - throttle = 0 - # egress = "engineEgress" - } - - dfurecovery-archiver = { - disabled = false - interval = 12 - limit = 20 - cutoff = 4 - at = "* * * * *" - # egress = "engineEgress" - } - - file-expiry = { - disabled = false - interval = 1 - at = "* * * * *" - persistExpiryDefault = 7 - expiryDefault = 4 - user = "sasha" - # egress = "engineEgress" - } - } -} - -## LDAP Config -################## -variable "ldap_config" { - description = "LDAP settings for dali and esp services." - type = object({ - dali = object({ - adminGroupName = string - filesBasedn = string - groupsBasedn = string - hpcc_admin_password = string - hpcc_admin_username = string - ldap_admin_password = string - ldap_admin_username = string - ldapAdminVaultId = string - resourcesBasedn = string - sudoersBasedn = string - systemBasedn = string - usersBasedn = string - workunitsBasedn = string - ldapCipherSuite = string - }) - esp = object({ - adminGroupName = string - filesBasedn = string - groupsBasedn = string - ldap_admin_password = string - ldap_admin_username = string - ldapAdminVaultId = string - resourcesBasedn = string - sudoersBasedn = string - systemBasedn = string - usersBasedn = string - workunitsBasedn = string - ldapCipherSuite = string - }) - ldap_server = string - }) - - default = null - sensitive = true -} - -variable "ldap_tunables" { - description = "Tunable settings for LDAP." - type = object({ - cacheTimeout = number - checkScopeScans = bool - ldapTimeoutSecs = number - maxConnections = number - passwordExpirationWarningDays = number - sharedCache = bool - }) - - default = { - cacheTimeout = 5 - checkScopeScans = false - ldapTimeoutSecs = 131 - maxConnections = 10 - passwordExpirationWarningDays = 10 - sharedCache = true - } -} - -## Data Storage Config -######################### -variable "install_blob_csi_driver" { - description = "Install blob-csi-drivers on the cluster." - type = bool - default = true -} - -variable "spill_volumes" { - description = "Map of objects to create Spill Volumes" - type = map(object({ - name = string # "Name of spill volume to be created." - size = number # "Size of spill volume to be created (in GB)." - prefix = string # "Prefix of spill volume to be created." - host_path = string # "Host path on spill volume to be created." - storage_class = string # "Storage class of spill volume to be used." - access_mode = string # "Access mode of spill volume to be used." - })) - - default = { - "spill" = { - name = "spill" - size = 300 - prefix = "/var/lib/HPCCSystems/spill" - host_path = "/mnt" - storage_class = "spill" - access_mode = "ReadWriteOnce" - } - } -} - -variable "data_storage_config" { - description = "Data plane config for HPCC." - type = object({ - internal = optional(object({ - blob_nfs = object({ - data_plane_count = number - storage_account_settings = object({ - # authorized_ip_ranges = map(string) - delete_protection = bool - replication_type = string - # subnet_ids = map(string) - blob_soft_delete_retention_days = optional(number) - container_soft_delete_retention_days = optional(number) - }) - }) - })) - - external = optional(object({ - blob_nfs = list(object({ - container_id = string - container_name = string - id = string - resource_group_name = string - storage_account_id = string - storage_account_name = string - })) - hpcc = list(object({ - name = string - planes = list(object({ - local = string - remote = string - })) - service = string - })) - })) - }) - - default = null - # default = { - # internal = { - # blob_nfs = { - # data_plane_count = 1 - # storage_account_settings = { - # # authorized_ip_ranges = {} - # delete_protection = false - # replication_type = "ZRS" - # # subnet_ids = {} - # blob_soft_delete_retention_days = 7 - # container_soft_delete_retention_days = 7 - # } - # } - # } - # external = null - # } -} - -variable "external_storage_config" { - description = "External services storage config." - type = list(object({ - category = string - container_name = string - path = string - plane_name = string - protocol = string - resource_group = string - size = number - storage_account = string - storage_type = string - prefix_name = string - })) - - default = null -} - -variable "remote_storage_plane" { - description = "Input for attaching remote storage plane" - type = map(object({ - dfs_service_name = string - dfs_secret_name = string - target_storage_accounts = map(object({ - name = string - prefix = string - })) - })) - - default = null -} - -variable "onprem_lz_settings" { - description = "Input for allowing OnPrem LZ." - type = map(object({ - prefix = string - hosts = list(string) - })) - - default = {} -} - -variable "admin_services_storage" { - description = "PV sizes for admin service planes in gigabytes (storage billed only as consumed)." - type = object({ - dali = object({ - size = number - type = string - }) - debug = object({ - size = number - type = string - }) - dll = object({ - size = number - type = string - }) - lz = object({ - size = number - type = string - }) - sasha = object({ - size = number - type = string - }) - }) - - default = { - dali = { - size = 100 - type = "azurefiles" - } - debug = { - size = 100 - type = "blobnfs" - } - dll = { - size = 100 - type = "blobnfs" - } - lz = { - size = 100 - type = "blobnfs" - } - sasha = { - size = 100 - type = "blobnfs" - } - } - - validation { - condition = length([for k, v in var.admin_services_storage : v.type if !contains(["azurefiles", "blobnfs"], v.type)]) == 0 - error_message = "The type must be either \"azurefiles\" or \"blobnfs\"." - } - - validation { - condition = length([for k, v in var.admin_services_storage : v.size if v.type == "azurefiles" && v.size < 100]) == 0 - error_message = "Size must be at least 100 for \"azurefiles\" type." - } -} - -variable "admin_services_storage_account_settings" { - description = "Settings for admin services storage account." - type = object({ - authorized_ip_ranges = optional(map(string)) - delete_protection = bool - replication_type = string - # subnet_ids = map(string) - blob_soft_delete_retention_days = optional(number) - container_soft_delete_retention_days = optional(number) - file_share_retention_days = optional(number) - }) - - default = { - authorized_ip_ranges = {} - delete_protection = false - replication_type = "ZRS" - subnet_ids = {} - blob_soft_delete_retention_days = 7 - container_soft_delete_retention_days = 7 - file_share_retention_days = 7 - } -} - -variable "ignore_external_storage" { - description = "Should storage created using the storage module or var.external_storage_config be ignored?" - type = bool - default = false -} - -## Node Selector -#################### -variable "admin_services_node_selector" { - description = "Node selector for admin services pods." - type = map(map(string)) - default = {} - - validation { - condition = length([for service in keys(var.admin_services_node_selector) : - service if !contains(["all", "dali", "esp", "eclagent", "eclccserver"], service)]) == 0 - error_message = "The keys must be one of \"all\", \"dali\", \"esp\", \"eclagent\" or \"eclccserver\"." - } -} - -## DNS -######### -variable "internal_domain" { - description = "DNS Domain name" - type = string - default = null -} From c11ef097bb30304e5ad095d6bb4c4e96479bb2d7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 22 Sep 2023 16:17:21 +0000 Subject: [PATCH 005/124] branch: HPCC-27615-easy-deploy. This is a merge of HPCC-27615 and branch: tlh-bug-fixes-and-personalize-easy-deploy-v0. Also, in hpcc/hpcc.tf module 'hpcc' now sources Godji's opinionated hpcc instead my my local clone of it. As of this date, 2023/09/22, Godji has not merged a change I made to my local opinionated. So, hpcc will not deploy until he merges or you switch to sourcing a local clone of my opnionated hpcc. --- hpcc/hpcc.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 9000f48..7fe8db0 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -19,8 +19,8 @@ resource "kubernetes_namespace" "hpcc" { }*/ module "hpcc" { - #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" + source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" environment = local.metadata.environment productname = local.metadata.product_name From 977e10771f0e05430970ba0739ee50a16cd67106 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 26 Sep 2023 20:05:47 +0000 Subject: [PATCH 006/124] branch:new-variable-enable_thor. Now one must set 'enable_thor=true' if one wants a thor cluster --- hpcc/hpcc.tf | 4 ++-- hpcc/lite-locals.tf | 2 +- hpcc/lite-variables.tf | 5 +++++ hpcc/lite.auto.tfvars.example | 7 +++++++ 4 files changed, 15 insertions(+), 3 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 7fe8db0..9000f48 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -19,8 +19,8 @@ resource "kubernetes_namespace" "hpcc" { }*/ module "hpcc" { - source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" + #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" environment = local.metadata.environment productname = local.metadata.product_name diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 03ab423..880b09b 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -472,7 +472,7 @@ locals { admin_services_node_selector = {} thor_config = [{ - disabled = ((var.thor_num_workers == 0 ) || (var.thor_num_workers == null))? true : false + disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true name = "thor" prefix = "thor" numWorkers = var.thor_num_workers diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index 4a2d34a..fc900fc 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -2,6 +2,11 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool +} + variable "a_record_name" { type = string diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example index d4586d1..9e2d6b5 100755 --- a/hpcc/lite.auto.tfvars.example +++ b/hpcc/lite.auto.tfvars.example @@ -74,6 +74,13 @@ enable_code_security=true #------------------------------------------------------------------------------ +# If you want a thor cluster then 'enable_thor' must be set to true +# Otherwise it is set to false + +enable_thor=true + +#------------------------------------------------------------------------------ + # The number of Thor workers to allocate. # Must be 1 or more. From 1b54ae741f887b7897edc13b857a7be455f73742 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 27 Sep 2023 18:42:11 +0000 Subject: [PATCH 007/124] branch:fix-roxie-so-port-18002-is-used --- hpcc/hpcc.tf | 1 + hpcc/lite-locals.tf | 22 ++++++++++++---------- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 9000f48..84552bb 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -86,6 +86,7 @@ module "hpcc" { external_storage_config = local.external_storage_config spill_volumes = local.spill_volumes + enable_roxie = var.enable_roxie roxie_config = local.roxie_config thor_config = local.thor_config vault_config = local.vault_config diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 880b09b..5ee0dc9 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -161,6 +161,17 @@ locals { # } # } + roxie_internal_service = { + name = "iroxie" + servicePort = 9876 + listenQueue = 200 + numThreads = 30 + visibility = "local" + annotations = {} + } + + roxie_services = [local.roxie_internal_service] + #======================================== # defaults in godji original variables.tf expose_services = false @@ -315,16 +326,7 @@ locals { useMemoryMappedIndexes = false useRemoteResources = false useTreeCopy = false - services = [ - { - name = "roxie" - servicePort = 9876 - listenQueue = 200 - numThreads = 30 - visibility = "local" - annotations = {} - } - ] + services = local.roxie_services topoServer = { replicas = 1 } From eba6f63ccf55adc799b13575609c9b22e2082d65 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 28 Sep 2023 20:51:19 +0000 Subject: [PATCH 008/124] branch:add-htpasswd-support --- hpcc/hpcc.tf | 2 ++ hpcc/lite-variables.tf | 5 ----- hpcc/lite.auto.tfvars.example | 10 ---------- 3 files changed, 2 insertions(+), 15 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 84552bb..02e2346 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -113,4 +113,6 @@ module "hpcc" { helm_chart_timeout = local.helm_chart_timeout helm_chart_files_overrides = concat(local.helm_chart_files_overrides, fileexists("../logging/data/logaccess_body.yaml") ? ["../logging/data/logaccess_body.yaml"] : []) ldap_config = local.ldap_config + + authn_htpasswd_filename = var.authn_htpasswd_filename } diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index fc900fc..c2e119a 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -69,11 +69,6 @@ variable "enable_code_security" { type = bool } -variable "enable_rbac_ad" { - description = "REQUIRED. Enable RBAC and AD integration for AKS?\nThis provides additional security for accessing the Kubernetes cluster and settings (not HPCC Systems' settings).\nValue type: boolean\nRecommended value: true\nExample entry: true" - type = bool -} - variable "enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example index 9e2d6b5..58c347d 100755 --- a/hpcc/lite.auto.tfvars.example +++ b/hpcc/lite.auto.tfvars.example @@ -55,16 +55,6 @@ enable_roxie=false #------------------------------------------------------------------------------ -# Enable RBAC and AD integration for AKS? -# This provides additional security for accessing the Kubernetes cluster and settings (not HPCC Systems' settings). -# Value type: boolean -# Recommended value: true -# Example entry: true - -enable_rbac_ad=false - -#------------------------------------------------------------------------------ - # Enable code security? # If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. # Value type: boolean From 58268fa72353cd7ac921ead9352430da17547dba Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 4 Oct 2023 19:01:54 +0000 Subject: [PATCH 009/124] branch:add-ecl-code-security. Plus, , setup 'storage_data_lz' --- aks/variables.tf | 7 ------- hpcc/hpcc.tf | 3 +++ 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/aks/variables.tf b/aks/variables.tf index 082ed75..8cbe85b 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -1,10 +1,3 @@ -variable "availability_zones" { - description = "Availability zones to use for the node groups." - type = list(number) - nullable = false - default = [1] -} - variable "tags" { description = "Tags to apply to all resources." type = map(string) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 02e2346..e562661 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -41,6 +41,8 @@ module "hpcc" { username = local.hpcc_container_registry_auth.username } : null + storage_data_gb = var.storage_data_gb + install_blob_csi_driver = false //Disable CSI driver resource_group_name = local.get_aks_config.resource_group_name @@ -114,5 +116,6 @@ module "hpcc" { helm_chart_files_overrides = concat(local.helm_chart_files_overrides, fileexists("../logging/data/logaccess_body.yaml") ? ["../logging/data/logaccess_body.yaml"] : []) ldap_config = local.ldap_config + enable_code_security = var.enable_code_security authn_htpasswd_filename = var.authn_htpasswd_filename } From fbafd5d420d6e021d897fe11c7183b734ebb5394 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 4 Oct 2023 21:45:34 +0000 Subject: [PATCH 010/124] branch:add-ecl-code-security --- hpcc/hpcc.tf | 5 +++-- hpcc/main.tf | 4 ++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index e562661..698ee96 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -19,8 +19,9 @@ resource "kubernetes_namespace" "hpcc" { }*/ module "hpcc" { - #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" + source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" environment = local.metadata.environment productname = local.metadata.product_name diff --git a/hpcc/main.tf b/hpcc/main.tf index 253197b..9f61b5d 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -30,11 +30,11 @@ module "metadata" { project = local.metadata.project } -resource "null_resource" "launch_svc_url" { +/*resource "null_resource" "launch_svc_url" { for_each = (module.hpcc.hpcc_status == "deployed") && (local.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} provisioner "local-exec" { command = local.is_windows_os ? "Start-Process ${each.value}" : "open ${each.value} || xdg-open ${each.value}" interpreter = local.is_windows_os ? ["PowerShell", "-Command"] : ["/bin/bash", "-c"] } -} +}*/ From 226139f55cc93fe68054d202c12b207e7196c7c9 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 6 Oct 2023 20:04:02 +0000 Subject: [PATCH 011/124] branch:add-terraform-to-deploy-everything --- hpcc/hpcc.tf | 4 +- hpcc/main.tf | 10 ++ lite.auto.tfvars.example | 234 ++++++++++++++++++++++++++++++++++++ main.tf | 24 ++++ providers.tf | 5 + scripts/deploy.sh | 19 +++ scripts/destroy.sh | 17 +++ scripts/hpcc_is_deployed.sh | 8 ++ 8 files changed, 319 insertions(+), 2 deletions(-) create mode 100755 lite.auto.tfvars.example create mode 100644 main.tf create mode 100644 providers.tf create mode 100755 scripts/deploy.sh create mode 100755 scripts/destroy.sh create mode 100755 scripts/hpcc_is_deployed.sh diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 698ee96..4b0ece6 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -19,9 +19,9 @@ resource "kubernetes_namespace" "hpcc" { }*/ module "hpcc" { - source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" environment = local.metadata.environment productname = local.metadata.product_name diff --git a/hpcc/main.tf b/hpcc/main.tf index 9f61b5d..bfcc940 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -38,3 +38,13 @@ module "metadata" { interpreter = local.is_windows_os ? ["PowerShell", "-Command"] : ["/bin/bash", "-c"] } }*/ + +resource "null_resource" "launch_svc_url" { + count = (module.hpcc.hpcc_status == "deployed") ? 1 : 0 + + provisioner "local-exec" { + command = "../scripts/hpcc_is_deployed.sh" + } + + depends_on = [ module.hpcc ] +} diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example new file mode 100755 index 0000000..58c347d --- /dev/null +++ b/lite.auto.tfvars.example @@ -0,0 +1,234 @@ +# To have a dns zone and an A record, included in your apply, the following 3 +# variables, i.e. 'a_record_name', 'dns_zone_name', and +# dns_zone_resource_group_name must be set to non-blank or non-null values. +#----------------------------------------------------------------------------- + +# Name of the A record, of following dns zone, where the ecl watch ip is placed +# This A record will be created and therefore should not exist in the following +# dns zone. +# Example entry: "my-product". This should be something project specific rather +# than something generic. + +a_record_name="" + +#----------------------------------------------------------------------------- + +# Name of an existing dns zone. +# Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" + +dns_zone_name="" + +#----------------------------------------------------------------------------- + +# Name of the resource group of the above dns zone. +# Example entry: "app-dns-prod-eastus2" + +dns_zone_resource_group_name="" + +#------------------------------------------------------------------------------ + +# Abbreviated product name, suitable for use in Azure naming. +# Must be 3-16 characters in length, all lowercase letters or numbers, no spaces. +# Value type: string +# Example entry: "my-product" + +product_name="tlhhpcc" + +#------------------------------------------------------------------------------ + +# The version of HPCC Systems to install. +# Only versions in nn.nn.nn format are supported. +# Value type: string + +hpcc_version="8.6.14" +#hpcc_version="latest" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. +#hpcc_version="8.10.40" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. + +#------------------------------------------------------------------------------ + +# Enable ROXIE? +# This will also expose port 8002 on the cluster. +# Value type: boolean +# Example entry: false + +enable_roxie=false + +#------------------------------------------------------------------------------ + +# Enable code security? +# If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. +# Value type: boolean +# Example entry: false + +enable_code_security=true + +#------------------------------------------------------------------------------ + +# If you want a thor cluster then 'enable_thor' must be set to true +# Otherwise it is set to false + +enable_thor=true + +#------------------------------------------------------------------------------ + +# The number of Thor workers to allocate. +# Must be 1 or more. + +thor_num_workers=2 + +#------------------------------------------------------------------------------ + +# The maximum number of simultaneous Thor jobs allowed. +# Must be 1 or more. + +thor_max_jobs=2 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for the landing zone in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_lz_gb=25 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for data in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_data_gb=100 + +#------------------------------------------------------------------------------ + +# Map of name => value tags that can will be associated with the cluster. +# Format is '{"name"="value" [, "name"="value"]*}'. +# The 'name' portion must be unique. +# To add no tags, use '{}'. +# Value type: map of string + +extra_tags={} + +#------------------------------------------------------------------------------ + +# The VM size for each node in the HPCC Systems node pool. +# Recommend "Standard_B4ms" or better. +# See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. +# Value type: string + +node_size="Standard_B8ms" + +#------------------------------------------------------------------------------ + +# The maximum number of VM nodes to allocate for the HPCC Systems node pool. +# Must be 2 or more. +# Value type: integer + +max_node_count=4 + +#------------------------------------------------------------------------------ + +# Email address of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jane.doe@hpccsystems.com" + +admin_email="tlhumphrey2@gmail.com" + +#------------------------------------------------------------------------------ + +# Name of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "Jane Doe" + +admin_name="Timothy Humphrey" + +#------------------------------------------------------------------------------ + +# Username of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jdoe" + +admin_username="tlhumphrey2" + +#------------------------------------------------------------------------------ + +# The Azure region abbreviation in which to create these resources. +# Must be one of ["eastus", "eastus2", "centralus"]. +# Value type: string +# Example entry: "eastus" + +azure_region="eastus" + +#------------------------------------------------------------------------------ + +# Map of name => CIDR IP addresses that can administrate this AKS. +# Format is '{"name"="cidr" [, "name"="cidr"]*}'. +# The 'name' portion must be unique. +# To add no CIDR addresses, use '{}'. +# The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. +# Value type: map of string + +admin_ip_cidr_map={} + +#------------------------------------------------------------------------------ + +# List of explicit CIDR addresses that can access this HPCC Systems cluster. +# To allow public access, specify "0.0.0.0/0". +# To add no CIDR addresses, use '[]'. +# Value type: list of string + +hpcc_user_ip_cidr_list=[ "20.14.220.189/32", "66.241.32.0/24"] + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a resource group for the storage account. +# Value type: string +# Example entry: "my-product-sa" + +storage_account_name="" + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its resource group name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a name for the storage account. +# Value type: string + +storage_account_resource_group_name="" + +#============================================================================== +# Optional settings +#============================================================================== + +# The global image name of the HPCC docker image to deploy. +# Must be one of ["platform-core", "platform-ml", "platform-gnn"]. +# Default value: "platform-core" + +# hpcc_image_name="platform-core" + +#------------------------------------------------------------------------------ + +# The Kubernetes namespace in which to install the HPCC modules (if enabled). +# Default value: "default" + +# hpcc_namespace="default" + +#------------------------------------------------------------------------------ + +# If true, premium ($$$) storage will be used for the following storage shares: Dali. +# OPTIONAL, defaults to false. + +enable_premium_storage=false + +#------------------------------------------------------------------------------ + +# If you would like to use htpasswd to authenticate users to the cluster, enter +# the filename of the htpasswd file. This file should be uploaded to the Azure +# 'dllsshare' file share in order for the HPCC processes to find it. +# A corollary is that persistent storage is enabled. +# An empty string indicates that htpasswd is not to be used for authentication. +# Example entry: "htpasswd.txt" + +authn_htpasswd_filename="" diff --git a/main.tf b/main.tf new file mode 100644 index 0000000..8d9634d --- /dev/null +++ b/main.tf @@ -0,0 +1,24 @@ +resource "null_resource" "deploy_vnet" { + + provisioner "local-exec" { + command = "scripts/deploy.sh vnet" + } +} + +resource "null_resource" "deploy_aks" { + + provisioner "local-exec" { + command = "scripts/deploy.sh aks" + } + + depends_on = [ null_resource.deploy_vnet ] +} + +resource "null_resource" "deploy_hpcc" { + + provisioner "local-exec" { + command = "scripts/deploy.sh hpcc" + } + + depends_on = [ null_resource.deploy_aks ] +} diff --git a/providers.tf b/providers.tf new file mode 100644 index 0000000..e27a382 --- /dev/null +++ b/providers.tf @@ -0,0 +1,5 @@ +provider "azurerm" { + features {} + use_cli = true + storage_use_azuread = true +} diff --git a/scripts/deploy.sh b/scripts/deploy.sh new file mode 100755 index 0000000..10a61a2 --- /dev/null +++ b/scripts/deploy.sh @@ -0,0 +1,19 @@ +#!/bin/bash +cd $1; +name=$(basename `pwd`) +if [ "$name" == "hpcc" ];then + if [ -e "../lite.auto.tfvars" ];then + cp -v ../lite.auto.tfvars . + else + echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." + exit 1 + fi +fi +plan=`/home/azureuser/mkplan ${name}_deployment.plan` +if [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi +echo "=============== Deploying $name. Executing 'terraform init' ==============="; +terraform init +echo "=============== Deploying $name. Executing 'terraform plan -out=$plan' ==============="; +terraform plan -out=$plan +echo "=============== Deploying $name. Executing 'terraform apply $plan' ==============="; +terraform apply $plan diff --git a/scripts/destroy.sh b/scripts/destroy.sh new file mode 100755 index 0000000..5a45df3 --- /dev/null +++ b/scripts/destroy.sh @@ -0,0 +1,17 @@ +#!/bin/bash +if [ "$1" == "vnet" ];then + scripts/destroy.sh hpcc && scripts/destroy.sh aks +elif [ "$1" == "aks" ];then + scripts/destroy.sh hpcc +fi +cd $1; +name=$(basename `pwd`) +plan=`/home/azureuser/mkplan ${name}_destroy.plan` +if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi + +echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; +terraform destroy -auto-approve +rm -vr data +cd .. +r=`terraform state list|egrep "_$name"` +terraform state rm $r diff --git a/scripts/hpcc_is_deployed.sh b/scripts/hpcc_is_deployed.sh new file mode 100755 index 0000000..ee6a7d2 --- /dev/null +++ b/scripts/hpcc_is_deployed.sh @@ -0,0 +1,8 @@ +#!/bin/sh +name=$(basename `pwd`) +if [ -d "data" ] && [ -f "data/config.json" ]; then + echo "Complete! $name is already deployed";exit 0; +else + if [ ! -d "data" ];then mkdir data; fi + touch data/config.json +fi From 347ceef0e9340b76f28f317df4683459c7185ed0 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 9 Oct 2023 20:30:55 +0000 Subject: [PATCH 012/124] branch:add-terraform-to-deploy-everything --- lite-variables.tf | 227 ++++++++++++++++++++++++++++++++++++++++++++++ scripts/deploy.sh | 25 +++-- 2 files changed, 246 insertions(+), 6 deletions(-) create mode 100644 lite-variables.tf diff --git a/lite-variables.tf b/lite-variables.tf new file mode 100644 index 0000000..c2e119a --- /dev/null +++ b/lite-variables.tf @@ -0,0 +1,227 @@ +############################################################################### +# Prompted variables (user will be asked to supply them at plan/apply time +# if a .tfvars file is not supplied); there are no default values +############################################################################### +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool +} + + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + +variable "admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool +} + +variable "enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "REQUIRED. The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported." + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } +} + +variable "max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} + +variable "product_name" { + type = string + description = "REQUIRED. Abbreviated product name, suitable for use in Azure naming.\nMust be 3-16, all lowercase or numeric characters.\nExample entry: myproduct" + validation { + condition = can(regex("^[a-z][a-z0-9]{2,15}$", var.product_name)) + error_message = "Value must be [a-z0-9]{3,16}." + } +} + +variable "storage_data_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_data_gb >= 10 + error_message = "Value must be 10 or more." + } +} + +variable "storage_lz_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for the landing zone in gigabytes.\nMust be 1 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_lz_gb >= 1 + error_message = "Value must be 1 or more." + } +} + +variable "thor_max_jobs" { + type = number + description = "REQUIRED. The maximum number of simultaneous Thor jobs allowed.\nMust be 1 or more." + validation { + condition = var.thor_max_jobs >= 1 + error_message = "Value must be 1 or more." + } +} + +variable "thor_num_workers" { + type = number + description = "REQUIRED. The number of Thor workers to allocate.\nMust be 1 or more." + validation { + condition = var.thor_num_workers >= 1 + error_message = "Value must be 1 or more." + } +} + +############################################################################### +# Optional variables +############################################################################### + +variable "authn_htpasswd_filename" { + type = string + description = "OPTIONAL. If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it.\nA corollary is that persistent storage is enabled.\nAn empty string indicates that htpasswd is not to be used for authentication.\nExample entry: htpasswd.txt" + default = "" +} + +variable "hpcc_image_name" { + type = string + description = "REQUIRED. The global image name of the HPCC docker image to deploy.\nMust be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"].\nDefault value: platform-core" + default = "platform-core" + validation { + condition = contains(["platform-core", "platform-ml", "platform-gnn"], var.hpcc_image_name) + error_message = "Value must be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"]." + } +} + +/*variable "hpcc_namespace" { + type = string + description = "REQUIRED. The Kubernetes namespace in which to install the HPCC modules (if enabled).\nDefault value: default" + default = "default" + validation { + condition = var.hpcc_namespace != "" + error_message = "Namespace must be a non-empty string." + } +}*/ +/*variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + existing_namespace = optional(string) + labels = optional(map(string), { name = "hpcc" }) + create_namespace = optional(bool, true) + }) + default = {} +}*/ +variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + prefix_name = string + labels = map(string) + create_namespace = bool + }) + default = { + prefix_name = "hpcc" + labels = { + name = "hpcc" + } + create_namespace = false + } +} + +variable "enable_premium_storage" { + type = bool + description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." + default = false +} + +variable "storage_account_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" + default = "" +} + +variable "storage_account_resource_group_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." + default = "" +} diff --git a/scripts/deploy.sh b/scripts/deploy.sh index 10a61a2..65919f1 100755 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -1,4 +1,17 @@ #!/bin/bash +#======================================================================== +function assert_fail () { + echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" + if "$@"; then + echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo + else + echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + rm -vr data + exit 1 + fi +} +#======================================================================== + cd $1; name=$(basename `pwd`) if [ "$name" == "hpcc" ];then @@ -11,9 +24,9 @@ if [ "$name" == "hpcc" ];then fi plan=`/home/azureuser/mkplan ${name}_deployment.plan` if [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi -echo "=============== Deploying $name. Executing 'terraform init' ==============="; -terraform init -echo "=============== Deploying $name. Executing 'terraform plan -out=$plan' ==============="; -terraform plan -out=$plan -echo "=============== Deploying $name. Executing 'terraform apply $plan' ==============="; -terraform apply $plan +echo "=============== Deploy $name. Executing 'terraform init' ==============="; +assert_fail terraform init +echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; +assert_fail terraform plan -out=$plan +echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; +assert_fail terraform apply $plan From e5f5853cdd1aa49a58144a99fd6fbd1b8f9f6b88 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 12 Oct 2023 02:07:36 +0000 Subject: [PATCH 013/124] branch:aks-is-now-using-easy-deploy-variables --- aks/aks.tf | 19 +-- aks/lite-locals.tf | 56 +++++++ aks/lite-variables.tf | 54 +++++++ aks/locals.tf | 18 +-- aks/main.tf | 18 +-- aks/variables.tf | 145 ------------------ hpcc/lite-locals.tf | 6 +- hpcc/lite-variables.tf | 52 +++---- hpcc/main.tf | 2 +- lite-variables.tf | 52 +++---- lite.auto.tfvars.example | 39 ++--- main.tf | 6 +- scripts/deploy | 82 ++++++++++ scripts/deploy.sh | 32 ---- scripts/destroy | 31 ++++ scripts/destroy.sh | 17 -- scripts/extract-aks-variables | 14 ++ scripts/get_rg_from_file | 13 ++ .../{hpcc_is_deployed.sh => hpcc_is_deployed} | 0 storage/main.tf | 3 +- 20 files changed, 351 insertions(+), 308 deletions(-) create mode 100644 aks/lite-locals.tf create mode 100644 aks/lite-variables.tf create mode 100755 scripts/deploy delete mode 100755 scripts/deploy.sh create mode 100755 scripts/destroy delete mode 100755 scripts/destroy.sh create mode 100755 scripts/extract-aks-variables create mode 100755 scripts/get_rg_from_file rename scripts/{hpcc_is_deployed.sh => hpcc_is_deployed} (100%) diff --git a/aks/aks.tf b/aks/aks.tf index 65d08a3..6a18306 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -22,6 +22,7 @@ module "aks" { depends_on = [random_string.string] #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "/home/azureuser/temp/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default @@ -29,7 +30,7 @@ module "aks" { kubectl = kubectl.default } - location = var.metadata.location + location = local.metadata.location resource_group_name = module.resource_groups["azure_kubernetes_service"].name cluster_name = local.cluster_name @@ -45,7 +46,7 @@ module "aks" { subnet_name = try(var.use_existing_vnet.subnets.aks.name, "aks-hpcc-private") route_table_name = try(var.use_existing_vnet.route_table_name, local.get_vnet_config.route_table_name) - dns_resource_group_lookup = { "${var.internal_domain}" = var.dns_resource_group } + dns_resource_group_lookup = { "${local.internal_domain}" = local.dns_resource_group } admin_group_object_ids = [data.azuread_group.subscription_owner.object_id] @@ -55,15 +56,15 @@ module "aks" { node_groups = var.node_groups core_services_config = { - alertmanager = var.core_services_config.alertmanager - coredns = var.core_services_config.coredns - external_dns = var.core_services_config.external_dns - cert_manager = var.core_services_config.cert_manager + alertmanager = local.core_services_config.alertmanager + coredns = local.core_services_config.coredns + external_dns = local.core_services_config.external_dns + cert_manager = local.core_services_config.cert_manager ingress_internal_core = { - domain = var.core_services_config.ingress_internal_core.domain - subdomain_suffix = "${var.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(var.owner.name)}" // dns record suffix - public_dns = var.core_services_config.ingress_internal_core.public_dns + domain = local.core_services_config.ingress_internal_core.domain + subdomain_suffix = "${local.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(local.owner.name)}" // dns record suffix + public_dns = local.core_services_config.ingress_internal_core.public_dns } } diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf new file mode 100644 index 0000000..adee9f2 --- /dev/null +++ b/aks/lite-locals.tf @@ -0,0 +1,56 @@ +locals { + internal_domain = var.aks_dns_zone_name + + dns_resource_group = var.aks_dns_zone_resource_group_name + + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + metadata = { + project = "tlhhpccplatform" + product_name = "tlhhpccplatform" + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = "tlhhpcc" + resource_group_type = "app" + sre_team = "tlhhpccplatform" + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + core_services_config = { + alertmanager = { + smtp_host = "smtp-hostname.ds:25" + smtp_from = var.aks_admin_email + routes = [] + receivers = [] + } + + # coredns = { + # forward_zones = { + # "" = "" + # } + # } + coredns = {} + + external_dns = { + public_domain_filters = [var.aks_dns_zone_name] + } + + # cert_manager = { + # acme_dns_zones = [var.aks_dns_zone_name] + # default_issuer_name = "zerossl" + # } + cert_manager = {} + + ingress_internal_core = { + domain = var.aks_dns_zone_name + subdomain_suffix = "hpcc" // dns record suffix //must be unique accross subscription + public_dns = true + } + } +} diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf new file mode 100644 index 0000000..7183c29 --- /dev/null +++ b/aks/lite-variables.tf @@ -0,0 +1,54 @@ +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "aks_dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/aks/locals.tf b/aks/locals.tf index c8cb340..842fabc 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -6,18 +6,18 @@ locals { names = var.disable_naming_conventions ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.metadata.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) + tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null diff --git a/aks/main.tf b/aks/main.tf index cfbe822..f96d390 100644 --- a/aks/main.tf +++ b/aks/main.tf @@ -24,17 +24,17 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market + market = local.metadata.market location = local.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = module.subscription.output.subscription_id - project = var.metadata.project + project = local.metadata.project } module "resource_groups" { diff --git a/aks/variables.tf b/aks/variables.tf index 8cbe85b..b51ae0e 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -5,25 +5,6 @@ variable "tags" { default = {} } -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - # variable "azure_auth" { # description = "Azure authentication" # type = object({ @@ -49,37 +30,6 @@ variable "disable_naming_conventions" { default = false } -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - variable "resource_groups" { description = "Resource group module variables." type = any @@ -112,18 +62,6 @@ variable "use_existing_vnet" { default = null } -## DNS -######### -variable "internal_domain" { - description = "DNS Domain name" - type = string -} - -variable "dns_resource_group" { - description = "DNS resource group name" - type = string -} - ## Other AKS Vars ################## variable "cluster_ordinal" { @@ -190,89 +128,6 @@ variable "node_groups" { default = {} } -variable "core_services_config" { - description = "Core service configuration." - type = object({ - alertmanager = object({ - smtp_host = string - smtp_from = string - receivers = optional(list(object({ - name = string - email_configs = optional(any, []) - opsgenie_configs = optional(any, []) - pagerduty_configs = optional(any, []) - pushover_configs = optional(any, []) - slack_configs = optional(any, []) - sns_configs = optional(any, []) - victorops_configs = optional(any, []) - webhook_configs = optional(any, []) - wechat_configs = optional(any, []) - telegram_configs = optional(any, []) - }))) - routes = optional(list(object({ - receiver = string - group_by = optional(list(string)) - continue = optional(bool) - matchers = list(string) - group_wait = optional(string) - group_interval = optional(string) - repeat_interval = optional(string) - mute_time_intervals = optional(list(string)) - # active_time_intervals = optional(list(string)) - }))) - }) - cert_manager = optional(object({ - acme_dns_zones = optional(list(string)) - additional_issuers = optional(map(any)) - default_issuer_kind = optional(string) - default_issuer_name = optional(string) - })) - coredns = optional(object({ - forward_zones = optional(map(any)) - })) - external_dns = optional(object({ - additional_sources = optional(list(string)) - private_domain_filters = optional(list(string)) - public_domain_filters = optional(list(string)) - })) - fluentd = optional(object({ - image_repository = optional(string) - image_tag = optional(string) - additional_env = optional(map(string)) - debug = optional(bool) - filters = optional(string) - route_config = optional(list(object({ - match = string - label = string - copy = optional(bool) - config = string - }))) - routes = optional(string) - outputs = optional(string) - })) - grafana = optional(object({ - admin_password = optional(string) - additional_plugins = optional(list(string)) - additional_data_sources = optional(list(any)) - })) - ingress_internal_core = optional(object({ - domain = string - subdomain_suffix = optional(string) - lb_source_cidrs = optional(list(string)) - lb_subnet_name = optional(string) - public_dns = optional(bool) - })) - prometheus = optional(object({ - remote_write = optional(any) - })) - storage = optional(object({ - file = optional(bool, true) - blob = optional(bool, false) - }), {}) - }) - nullable = false -} - variable "experimental" { description = "Configure experimental features." type = object({ diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 5ee0dc9..f08c95c 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -4,7 +4,7 @@ locals { owner = { name = var.admin_username - email = var.admin_email + email = var.aks_admin_email } metadata = { @@ -18,7 +18,7 @@ locals { sre_team = var.product_name subscription_type = "dev" additional_tags = { "justification" = "testing" } - location = var.azure_region # Acceptable values: eastus, centralus + location = var.aks_azure_region # Acceptable values: eastus, centralus } tags = merge(local.metadata.additional_tags, var.extra_tags) @@ -54,7 +54,7 @@ locals { # password = "value" # } - internal_domain = var.dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io + internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io external = {} # external = { diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index c2e119a..1e2a70f 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -14,34 +14,16 @@ variable "a_record_name" { default = "" } -variable "dns_zone_resource_group_name" { - type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" -} - -variable "dns_zone_name" { - type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" -} - -variable "admin_email" { +variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" validation { - condition = length(regexall("^[^@]+@[^@]+$", var.admin_email)) > 0 + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 error_message = "Value must at least look like a valid email address." } } -variable "admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "admin_name" { +variable "aks_admin_name" { type = string description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" } @@ -55,11 +37,11 @@ variable "admin_username" { } } -variable "azure_region" { +variable "aks_azure_region" { type = string description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" validation { - condition = contains(["eastus", "eastus2", "centralus"], var.azure_region) + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." } } @@ -80,6 +62,18 @@ variable "extra_tags" { default = {} } +variable "aks_dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "aks_dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + variable "hpcc_user_ip_cidr_list" { description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." type = list(string) @@ -95,16 +89,22 @@ variable "hpcc_version" { } } -variable "max_node_count" { +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { type = number description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." validation { - condition = var.max_node_count >= 2 + condition = var.aks_max_node_count >= 2 error_message = "Value must be 2 or more." } } -variable "node_size" { +variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } diff --git a/hpcc/main.tf b/hpcc/main.tf index bfcc940..1c98a21 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -43,7 +43,7 @@ resource "null_resource" "launch_svc_url" { count = (module.hpcc.hpcc_status == "deployed") ? 1 : 0 provisioner "local-exec" { - command = "../scripts/hpcc_is_deployed.sh" + command = "../scripts/hpcc_is_deployed" } depends_on = [ module.hpcc ] diff --git a/lite-variables.tf b/lite-variables.tf index c2e119a..1e2a70f 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -14,34 +14,16 @@ variable "a_record_name" { default = "" } -variable "dns_zone_resource_group_name" { - type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" -} - -variable "dns_zone_name" { - type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" -} - -variable "admin_email" { +variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" validation { - condition = length(regexall("^[^@]+@[^@]+$", var.admin_email)) > 0 + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 error_message = "Value must at least look like a valid email address." } } -variable "admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "admin_name" { +variable "aks_admin_name" { type = string description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" } @@ -55,11 +37,11 @@ variable "admin_username" { } } -variable "azure_region" { +variable "aks_azure_region" { type = string description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" validation { - condition = contains(["eastus", "eastus2", "centralus"], var.azure_region) + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." } } @@ -80,6 +62,18 @@ variable "extra_tags" { default = {} } +variable "aks_dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "aks_dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + variable "hpcc_user_ip_cidr_list" { description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." type = list(string) @@ -95,16 +89,22 @@ variable "hpcc_version" { } } -variable "max_node_count" { +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { type = number description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." validation { - condition = var.max_node_count >= 2 + condition = var.aks_max_node_count >= 2 error_message = "Value must be 2 or more." } } -variable "node_size" { +variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 58c347d..dfd2f71 100755 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -1,29 +1,16 @@ -# To have a dns zone and an A record, included in your apply, the following 3 -# variables, i.e. 'a_record_name', 'dns_zone_name', and -# dns_zone_resource_group_name must be set to non-blank or non-null values. -#----------------------------------------------------------------------------- - -# Name of the A record, of following dns zone, where the ecl watch ip is placed -# This A record will be created and therefore should not exist in the following -# dns zone. -# Example entry: "my-product". This should be something project specific rather -# than something generic. - -a_record_name="" - #----------------------------------------------------------------------------- # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" -dns_zone_name="" +aks_dns_zone_name="" #----------------------------------------------------------------------------- # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" -dns_zone_resource_group_name="" +aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ @@ -40,9 +27,7 @@ product_name="tlhhpcc" # Only versions in nn.nn.nn format are supported. # Value type: string -hpcc_version="8.6.14" -#hpcc_version="latest" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. -#hpcc_version="8.10.40" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. +hpcc_version="8.10.1" # Currently not used #------------------------------------------------------------------------------ @@ -60,7 +45,7 @@ enable_roxie=false # Value type: boolean # Example entry: false -enable_code_security=true +enable_code_security=false #------------------------------------------------------------------------------ @@ -107,7 +92,7 @@ storage_data_gb=100 # To add no tags, use '{}'. # Value type: map of string -extra_tags={} +extra_tags={} # Currently not used #------------------------------------------------------------------------------ @@ -116,7 +101,7 @@ extra_tags={} # See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. # Value type: string -node_size="Standard_B8ms" +aks_node_size="Standard_B8ms" #------------------------------------------------------------------------------ @@ -124,7 +109,7 @@ node_size="Standard_B8ms" # Must be 2 or more. # Value type: integer -max_node_count=4 +aks_max_node_count=4 #------------------------------------------------------------------------------ @@ -132,7 +117,7 @@ max_node_count=4 # Value type: string # Example entry: "jane.doe@hpccsystems.com" -admin_email="tlhumphrey2@gmail.com" +aks_admin_email="jane.doe@hpccsystems.com" #------------------------------------------------------------------------------ @@ -140,7 +125,7 @@ admin_email="tlhumphrey2@gmail.com" # Value type: string # Example entry: "Jane Doe" -admin_name="Timothy Humphrey" +aks_admin_name="Jane Doe" #------------------------------------------------------------------------------ @@ -148,7 +133,7 @@ admin_name="Timothy Humphrey" # Value type: string # Example entry: "jdoe" -admin_username="tlhumphrey2" +admin_username="jdoe" #------------------------------------------------------------------------------ @@ -157,7 +142,7 @@ admin_username="tlhumphrey2" # Value type: string # Example entry: "eastus" -azure_region="eastus" +aks_azure_region="eastus" #------------------------------------------------------------------------------ @@ -168,7 +153,7 @@ azure_region="eastus" # The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. # Value type: map of string -admin_ip_cidr_map={} +aks_admin_ip_cidr_map={} #------------------------------------------------------------------------------ diff --git a/main.tf b/main.tf index 8d9634d..1551462 100644 --- a/main.tf +++ b/main.tf @@ -1,14 +1,14 @@ resource "null_resource" "deploy_vnet" { provisioner "local-exec" { - command = "scripts/deploy.sh vnet" + command = "scripts/deploy vnet" } } resource "null_resource" "deploy_aks" { provisioner "local-exec" { - command = "scripts/deploy.sh aks" + command = "scripts/deploy aks" } depends_on = [ null_resource.deploy_vnet ] @@ -17,7 +17,7 @@ resource "null_resource" "deploy_aks" { resource "null_resource" "deploy_hpcc" { provisioner "local-exec" { - command = "scripts/deploy.sh hpcc" + command = "scripts/deploy hpcc" } depends_on = [ null_resource.deploy_aks ] diff --git a/scripts/deploy b/scripts/deploy new file mode 100755 index 0000000..b9a1754 --- /dev/null +++ b/scripts/deploy @@ -0,0 +1,82 @@ +#!/bin/bash +#======================================================================== +function assert_fail () { + echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" + if "$@"; then + echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo + else + echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + rm -vrf data + exit 1 + fi +} +#======================================================================== +# make sure data/config.json doesn't exist if kubernetes cluster doesn't exist +# Delete both hpcc/data and aks/data if kubernetes cluster doesn't exist +# If kubernetes cluster does exist but there are no pods in the default namespace +# then delete only hpcc/data +ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` +if [[ "$ns" == *"Unable to connect to the server"* ]];then + # force rm data/config.json in hpcc and aks directories + echo "Forcibly delete hpcc/data and aks/data" + rm -vrf hpcc/data; rm -vrf aks/data +fi +p=`kubectl get pods 2>&1` +if [[ "$p" == *"No resources found in default namespace"* ]];then + # force rm data/config.json in hpcc only + echo "Forcibly delete hpcc/data only" + rm -vrf hpcc/data +fi + +# See if vnet/data/config.json exists. If it does then from the file get +# the resource group name and then check to see if resource group exists. +# if it doesn't exists then delete vnet/data/config.json +if [ -e "vnet/data/config.json" ];then + # Get resource group name from file + rg=`scripts/get_rg_from_file vnet/data/config.json` + if [ $(az group exists --name $rg) = false ]; then + echo "vnet resource group, \"$rg\" does not exists. So deleting vnet/data/config.json" + rm -vrf vnet/data +else + echo "vnet resource group, \"$rg\" does exists. So NOT deleting vnet/data/config.json" + fi +fi +#------------------------------------------------------------------------ +cd $1; + +# put the root directory's lite.auto.tfvars (either all of part) in either aks or hpcc +# directory. +name=$(basename `pwd`) +if [ -e "../lite.auto.tfvars" ] && [ -e "/tmp/${name}.lite.auto.tfvars" ];then + diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` +fi +if [ "$name" == "hpcc" ];then + if [ -e "../lite.auto.tfvars" ];then + cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars + cp -v ../lite-variables.tf . + else + echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." + exit 1 + fi +elif [ "$name" == "aks" ];then + if [ -e "../lite.auto.tfvars" ];then + egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars + ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf + else + echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." + exit 1 + fi +fi +#------------------------------------------------------------------------ +plan=`/home/azureuser/mkplan ${name}_deployment.plan` +if [ "$diff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi +if [ "$name" != "vnet" ];then + cp -v /tmp/${name}.lite.auto.tfvars . +fi + +echo "=============== Deploy $name. Executing 'terraform init' ==============="; +assert_fail terraform init +echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; +assert_fail terraform plan -out=$plan +echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; +assert_fail terraform apply $plan diff --git a/scripts/deploy.sh b/scripts/deploy.sh deleted file mode 100755 index 65919f1..0000000 --- a/scripts/deploy.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash -#======================================================================== -function assert_fail () { - echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" - if "$@"; then - echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo - else - echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo - rm -vr data - exit 1 - fi -} -#======================================================================== - -cd $1; -name=$(basename `pwd`) -if [ "$name" == "hpcc" ];then - if [ -e "../lite.auto.tfvars" ];then - cp -v ../lite.auto.tfvars . - else - echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." - exit 1 - fi -fi -plan=`/home/azureuser/mkplan ${name}_deployment.plan` -if [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi -echo "=============== Deploy $name. Executing 'terraform init' ==============="; -assert_fail terraform init -echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; -assert_fail terraform plan -out=$plan -echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; -assert_fail terraform apply $plan diff --git a/scripts/destroy b/scripts/destroy new file mode 100755 index 0000000..d4f9ff2 --- /dev/null +++ b/scripts/destroy @@ -0,0 +1,31 @@ +#!/bin/bash +#======================================================================== +function assert_fail () { + echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" + if "$@"; then + echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo + else + echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + rm -vrf data + exit 1 + fi +} +#======================================================================== + +if [ "$1" == "vnet" ];then + assert_fail scripts/destroy hpcc + assert_fail scripts/destroy aks +elif [ "$1" == "aks" ];then + assert_fail scripts/destroy hpcc +fi +cd $1; +name=$(basename `pwd`) +plan=`/home/azureuser/mkplan ${name}_destroy.plan` +if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi + +echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; +assert_fail terraform destroy -auto-approve +rm -vr data +cd .. +r=`terraform state list|egrep "_$name"` +terraform state rm $r diff --git a/scripts/destroy.sh b/scripts/destroy.sh deleted file mode 100755 index 5a45df3..0000000 --- a/scripts/destroy.sh +++ /dev/null @@ -1,17 +0,0 @@ -#!/bin/bash -if [ "$1" == "vnet" ];then - scripts/destroy.sh hpcc && scripts/destroy.sh aks -elif [ "$1" == "aks" ];then - scripts/destroy.sh hpcc -fi -cd $1; -name=$(basename `pwd`) -plan=`/home/azureuser/mkplan ${name}_destroy.plan` -if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi - -echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; -terraform destroy -auto-approve -rm -vr data -cd .. -r=`terraform state list|egrep "_$name"` -terraform state rm $r diff --git a/scripts/extract-aks-variables b/scripts/extract-aks-variables new file mode 100755 index 0000000..66b7e3c --- /dev/null +++ b/scripts/extract-aks-variables @@ -0,0 +1,14 @@ +#!/usr/bin/perl +if ( scalar(@ARGV) > 0 ){ + $variable_file = shift @ARGV; +} else{ + die "FATAL ERROR: Variable file name must be given on command line\n"; +} +undef $/; +open(IN, $variable_file) || die "Can't open variable file, \"$variable_file\"\n"; +$_ = ; +close(IN); + +@variable_definitions = m/\bvariable \"aks_[^\"]+\".+?\n}/sg; + +print join("\n\n",@variable_definitions),"\n"; diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file new file mode 100755 index 0000000..598d3c7 --- /dev/null +++ b/scripts/get_rg_from_file @@ -0,0 +1,13 @@ +#!/usr/bin/perl +if ( scalar(@ARGV) > 0 ){ + $config_file = shift @ARGV; +} else{ + die "FATAL ERROR: config file path must be given on command line\n"; +} +undef $/; +open(IN, $config_file) || die "Can't open config file, \"$config_file\"\n"; +$_ = ; +close(IN); + +$rg = $1 if ( /"resource_group_name":"([^"]+)"/ ); +print $rg; diff --git a/scripts/hpcc_is_deployed.sh b/scripts/hpcc_is_deployed similarity index 100% rename from scripts/hpcc_is_deployed.sh rename to scripts/hpcc_is_deployed diff --git a/storage/main.tf b/storage/main.tf index dd1de4c..d2c1578 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,6 +1,7 @@ module "storage" { #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" owner = var.owner disable_naming_conventions = var.disable_naming_conventions From 7fb2c898301f60c499a725fdc6144a85863ed167 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 12 Oct 2023 18:33:50 +0000 Subject: [PATCH 014/124] branch:aks-is-now-using-easy-deploy-variables --- scripts/deploy | 4 ++-- storage/main.tf | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/deploy b/scripts/deploy index b9a1754..1e29944 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -16,13 +16,13 @@ function assert_fail () { # If kubernetes cluster does exist but there are no pods in the default namespace # then delete only hpcc/data ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` -if [[ "$ns" == *"Unable to connect to the server"* ]];then +if [[ "$ns" == *"Unable"* ]];then # force rm data/config.json in hpcc and aks directories echo "Forcibly delete hpcc/data and aks/data" rm -vrf hpcc/data; rm -vrf aks/data fi p=`kubectl get pods 2>&1` -if [[ "$p" == *"No resources found in default namespace"* ]];then +if [[ "$p" == *"Unable"* ]] || [[ "$p" == *"No resources found in default namespace"* ]];then # force rm data/config.json in hpcc only echo "Forcibly delete hpcc/data only" rm -vrf hpcc/data diff --git a/storage/main.tf b/storage/main.tf index d2c1578..ac8c323 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,7 +1,7 @@ module "storage" { #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" + source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" owner = var.owner disable_naming_conventions = var.disable_naming_conventions From 0f56039a641c36cc88c74cc39a09cdc6c2a6b2dd Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 13 Oct 2023 19:08:57 +0000 Subject: [PATCH 015/124] branch:aks-is-now-using-easy-deploy-variables. Changed scripts/deploy so lite.auto.tfvars is correctly copied to hpcc and/or aks directories. --- scripts/deploy | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/scripts/deploy b/scripts/deploy index 1e29944..fe48dfb 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,4 +1,9 @@ #!/bin/bash +if [ "$1" != "" ];then + name=$1 +else + echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'aks' or 'hpcc'. EXITING.";exit 1; +fi #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" @@ -21,8 +26,8 @@ if [[ "$ns" == *"Unable"* ]];then echo "Forcibly delete hpcc/data and aks/data" rm -vrf hpcc/data; rm -vrf aks/data fi -p=`kubectl get pods 2>&1` -if [[ "$p" == *"Unable"* ]] || [[ "$p" == *"No resources found in default namespace"* ]];then +pods=`kubectl get pods 2>&1` +if [[ "$pods" == *"Unable"* ]] || [[ "$pods" == *"No resources found in default namespace"* ]];then # force rm data/config.json in hpcc only echo "Forcibly delete hpcc/data only" rm -vrf hpcc/data @@ -38,34 +43,32 @@ if [ -e "vnet/data/config.json" ];then echo "vnet resource group, \"$rg\" does not exists. So deleting vnet/data/config.json" rm -vrf vnet/data else - echo "vnet resource group, \"$rg\" does exists. So NOT deleting vnet/data/config.json" + echo "vnet resource group, \"$rg\" DOES exists. So NOT deleting vnet/data/config.json" fi fi #------------------------------------------------------------------------ -cd $1; +cd $name; # cd into vnet or aks or hpcc # put the root directory's lite.auto.tfvars (either all of part) in either aks or hpcc # directory. -name=$(basename `pwd`) -if [ -e "../lite.auto.tfvars" ] && [ -e "/tmp/${name}.lite.auto.tfvars" ];then - diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` +if [ -e "../lite.auto.tfvars" ];then + if [ -e "/tmp/${name}.lite.auto.tfvars" ];then + diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` + else + diff="" + fi +else + echo "The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; fi if [ "$name" == "hpcc" ];then - if [ -e "../lite.auto.tfvars" ];then + echo "Coping root's lite.auto.tfvars to /tmp and $name directory." cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars + cp -v ../lite.auto.tfvars . cp -v ../lite-variables.tf . - else - echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." - exit 1 - fi elif [ "$name" == "aks" ];then - if [ -e "../lite.auto.tfvars" ];then egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars + egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf - else - echo "ERROR: The file 'lite.auto.tfvars' file must exist in the root directory and it does not. So, we exit with an error." - exit 1 - fi fi #------------------------------------------------------------------------ plan=`/home/azureuser/mkplan ${name}_deployment.plan` @@ -80,3 +83,4 @@ echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ====== assert_fail terraform plan -out=$plan echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; assert_fail terraform apply $plan +rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform From e14e9eb8cb5b6b4ded271ed280319edc99517a7f Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 17 Oct 2023 21:17:52 +0000 Subject: [PATCH 016/124] branch:aks-is-now-using-easy-deploy-variables --- aks/aks.tf | 10 +++++++--- aks/lite-locals.tf | 2 ++ scripts/deploy | 2 ++ 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 6a18306..92101d9 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -21,8 +21,9 @@ module "aks" { depends_on = [random_string.string] #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" #source = "/home/azureuser/temp/terraform-azurerm-aks" + source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" providers = { kubernetes = kubernetes.default @@ -63,7 +64,7 @@ module "aks" { ingress_internal_core = { domain = local.core_services_config.ingress_internal_core.domain - subdomain_suffix = "${local.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(local.owner.name)}" // dns record suffix + subdomain_suffix = "${local.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(local.owner_name_initials)}" // dns record suffix public_dns = local.core_services_config.ingress_internal_core.public_dns } } @@ -75,10 +76,13 @@ module "aks" { blob = { enabled = true } } - logging = var.logging + # tlh logging = var.logging + logging = null experimental = { oms_agent = var.hpcc_log_analytics_enabled || var.experimental.oms_agent oms_agent_log_analytics_workspace_id = fileexists("../logging/data/workspace_resource_id.txt") ? file("../logging/data/workspace_resource_id.txt") : var.experimental.oms_agent_log_analytics_workspace_id != null ? var.experimental.oms_agent_log_analytics_workspace_id : null + #tlh tried this oms_agent = null + #tlh tried this oms_agent_log_analytics_workspace_id = null } } diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf index adee9f2..32e813c 100644 --- a/aks/lite-locals.tf +++ b/aks/lite-locals.tf @@ -8,6 +8,8 @@ locals { email = var.aks_admin_email } + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + metadata = { project = "tlhhpccplatform" product_name = "tlhhpccplatform" diff --git a/scripts/deploy b/scripts/deploy index fe48dfb..6b903a6 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,4 +1,5 @@ #!/bin/bash +repodir=`dirname `pwd`/deploy|sed "s/\/scripts\/*//"` if [ "$1" != "" ];then name=$1 else @@ -83,4 +84,5 @@ echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ====== assert_fail terraform plan -out=$plan echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; assert_fail terraform apply $plan +cd $repodir rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform From dbe5cef83d88b5ec8bcc6614fe612614eeae80c4 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 18 Oct 2023 20:53:52 +0000 Subject: [PATCH 017/124] branch:aks-is-now-using-easy-deploy-variables --- hpcc/roxie.auto.tfvars.example | 11 ----------- scripts/deploy | 4 +++- 2 files changed, 3 insertions(+), 12 deletions(-) delete mode 100644 hpcc/roxie.auto.tfvars.example diff --git a/hpcc/roxie.auto.tfvars.example b/hpcc/roxie.auto.tfvars.example deleted file mode 100644 index cfc1bf4..0000000 --- a/hpcc/roxie.auto.tfvars.example +++ /dev/null @@ -1,11 +0,0 @@ -# Roxie Settings -################# - -roxie_config = [{ - disabled = false - traceLevel = 3 - - services = [{ - annotations = { "service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path" = "/control/alive" } - }] -}] diff --git a/scripts/deploy b/scripts/deploy index 6b903a6..35756dc 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,5 +1,5 @@ #!/bin/bash -repodir=`dirname `pwd`/deploy|sed "s/\/scripts\/*//"` +repodir=$(echo `pwd`|sed "s/\/scripts\/*//") if [ "$1" != "" ];then name=$1 else @@ -13,6 +13,8 @@ function assert_fail () { else echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo rm -vrf data + cd $repodir + rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform exit 1 fi } From 2203f42428f80fa2f9946f4f62e944eeb30c6ba4 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 20 Oct 2023 18:10:05 +0000 Subject: [PATCH 018/124] branch:few-changes-20231020 --- .gitignore.swp | Bin 12288 -> 0 bytes aks/aks.auto.tfvars.example | 40 ++------------------ aks/aks.tf | 7 ++-- aks/automation.tf | 8 ++-- aks/lite-locals.tf | 8 ++-- aks/locals.tf | 42 ++++++++++++++++++++- aks/variables.tf | 21 ----------- hpcc/lite-locals.tf | 12 +++--- hpcc/lite.auto.tfvars.example | 39 ++++++------------- hpcc/main.tf | 9 ++--- hpcc/thor.auto.tfvars.example | 36 ------------------ scripts/deploy | 68 ++++++++++++++++++++-------------- scripts/destroy | 6 ++- scripts/get_rg_from_file | 4 +- scripts/hpcc_is_deployed | 3 -- scripts/mkplan | 26 +++++++++++++ vnet/lite-variables.tf | 54 +++++++++++++++++++++++++++ vnet/locals.tf | 45 +++++++++++++++++----- vnet/main.tf | 22 +++++------ vnet/misc.auto.tfvars.example | 24 ------------ vnet/variables.tf | 61 ------------------------------ 21 files changed, 251 insertions(+), 284 deletions(-) delete mode 100644 .gitignore.swp delete mode 100644 hpcc/thor.auto.tfvars.example create mode 100755 scripts/mkplan create mode 100644 vnet/lite-variables.tf delete mode 100644 vnet/misc.auto.tfvars.example diff --git a/.gitignore.swp b/.gitignore.swp deleted file mode 100644 index 8c9c0f0cf78d086f4947d620debde28ac73ae27a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 12288 zcmeI2O>5LZ7{{l@qpfdWVCsTkV6$z#NWoLR6zomKn@p40B$}Ox^Rjk}FQ5n>J=dG! zRnRXWi0HLm?8$>?!O!4-l5F~dm0p!6@Mkv9JoC)|`6ZXIv$1${ZJl3^ml&?ojJ@5S zlRNV#*urzh#FjVGTWP}FUoKbX{Zx+oWug3Le>^%W^-))FfF*uZQD>9{xJgO+gi2xBeCV@G2X?bZeB=^oQ@N;Kx9CIC|5dk7V1c(3; zAOb{y2oM1x@IMf6)eL)#MrPY)dhIxKXl!q3AOb{y2oM1xKm>>Y5g-CYfCvx)B0vQG zK>{LWY;cmXT`2tjKmGpyaf-38&<^wgdJnyZUO_LQr_dAVBD6ct*eB>C^bUFhy@a-* zXV63F0dxU61Izm&lwyS1>8Vllga;`yc!TLmnD+6iVUc+w zTwvVT;x^a5O!duW< zF<;l#)qN&L*gP(E2IH&YH&A7b^_GJhT8GMUR6kTSSKl^$)<5@U@4bwJ@YY`o)M2^X NJ1vY%oHm8D>^D4SR>}YX diff --git a/aks/aks.auto.tfvars.example b/aks/aks.auto.tfvars.example index ab557cf..cce1afe 100644 --- a/aks/aks.auto.tfvars.example +++ b/aks/aks.auto.tfvars.example @@ -1,15 +1,11 @@ -cluster_version = "1.25" +cluster_version = "1.26" cluster_ordinal = 1 //cluster name suffix sku_tier = "free" -dns_resource_group = "app-dns-prod-eastus2" -internal_domain = "my-dns-zone.io" hpcc_log_analytics_enabled = false rbac_bindings = { cluster_admin_users = { - # "service_principal1" = "", - # "user1" = "" - # "user2" = "" + # "admin" = "", } @@ -17,39 +13,9 @@ rbac_bindings = { cluster_view_groups = [] } -core_services_config = { - alertmanager = { - smtp_host = "smtp-hostname.ds:25" - smtp_from = "demo@lexisnexisrisk.com" - routes = [] - receivers = [] - } - - # coredns = { - # forward_zones = { - # "" = "" - # } - # } - - external_dns = { - public_domain_filters = ["my-dns-zone.io"] - } - - # cert_manager = { - # acme_dns_zones = ["my-dns-zone.io"] - # default_issuer_name = "zerossl" - # } - - ingress_internal_core = { - domain = "my-dns-zone.io" - subdomain_suffix = "hpcc" // dns record suffix //must be unique accross subscription - public_dns = true - } -} - cluster_endpoint_access_cidrs = ["0.0.0.0/0"] -availability_zones = [1,2] +availability_zones = [1] node_groups = { thorpool = { diff --git a/aks/aks.tf b/aks/aks.tf index 92101d9..281bc17 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -21,9 +21,8 @@ module "aks" { depends_on = [random_string.string] #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - #source = "/home/azureuser/temp/terraform-azurerm-aks" - source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" + source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" providers = { kubernetes = kubernetes.default @@ -53,7 +52,7 @@ module "aks" { rbac_bindings = var.rbac_bindings - availability_zones = var.availability_zones + #availability_zones = var.availability_zones node_groups = var.node_groups core_services_config = { diff --git a/aks/automation.tf b/aks/automation.tf index 45e80f9..f6bfae5 100644 --- a/aks/automation.tf +++ b/aks/automation.tf @@ -1,11 +1,11 @@ resource "azurerm_automation_account" "automation_account" { - name = var.aks_automation.automation_account_name + name = local.aks_automation.automation_account_name location = local.location resource_group_name = module.resource_groups["azure_kubernetes_service"].name sku_name = var.sku_name tags = local.tags - # local_authentication_enabled = var.aks_automation.local_authentication_enabled - public_network_access_enabled = var.aks_automation.public_network_access_enabled + # local_authentication_enabled = local.aks_automation.local_authentication_enabled + public_network_access_enabled = local.aks_automation.public_network_access_enabled identity { type = "SystemAssigned" @@ -60,6 +60,6 @@ resource "azurerm_automation_job_schedule" "job_schedule" { resourcename = module.aks.cluster_name resourcegroupname = module.resource_groups["azure_kubernetes_service"].name operation = each.value.operation - automationaccount = var.aks_automation.automation_account_name + automationaccount = local.aks_automation.automation_account_name } } diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf index 32e813c..57c7a99 100644 --- a/aks/lite-locals.tf +++ b/aks/lite-locals.tf @@ -11,14 +11,14 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) metadata = { - project = "tlhhpccplatform" - product_name = "tlhhpccplatform" + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) business_unit = "commercial" environment = "sandbox" market = "us" - product_group = "tlhhpcc" + product_group = format("%shpcc", local.owner_name_initials) resource_group_type = "app" - sre_team = "tlhhpccplatform" + sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" additional_tags = { "justification" = "testing" } location = var.aks_azure_region # Acceptable values: eastus, centralus diff --git a/aks/locals.tf b/aks/locals.tf index 842fabc..6cd6c5b 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -1,4 +1,42 @@ +resource "random_string" "name" { + length = 3 + special = false + numeric = false + upper = false +} + locals { + aks_automation = { + local_authentication_enabled = false + public_network_access_enabled = false + automation_account_name = "aks-stop-demo-${random_string.name.result}" + + schedule = [ + { + schedule_name = "aks_stop" + description = "Stops the AKS weekday nights at 6PM EST" + runbook_name = "aks_startstop_runbook" + frequency = "Week" //OneTime, Day, Hour, Week, or Month. + interval = "1" //cannot be set when frequency is `OneTime` + operation = "stop" + daylight_saving = true + start_time = "18:00" // At least 5 minutes in the future + week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] + }, + # { + # schedule_name = "aks_start" + # description = "Starts the AKS weekday nights at 6AM EST" + # runbook_name = "aks_startstop_runbook" + # frequency = "Week" //OneTime, Day, Hour, Week, or Month. + # interval = "1" //cannot be set when frequency is `OneTime` + # operation = "start" + # daylight_saving = true + # start_time = "06:00" // At least 5 minutes in the future + # week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] + # } + ] + } + azure_auth_env = { AZURE_TENANT_ID = data.azurerm_client_config.current.tenant_id AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id @@ -47,10 +85,10 @@ locals { tomorrow = formatdate("YYYY-MM-DD", timeadd(local.current_time, "24h")) # today = formatdate("YYYY-MM-DD", timeadd(local.current_time, "1h")) - utc_offset = var.aks_automation.schedule[0].daylight_saving ? 4 : 5 + utc_offset = local.aks_automation.schedule[0].daylight_saving ? 4 : 5 script = { for item in fileset("${path.root}/scripts", "*") : (item) => file("${path.root}/scripts/${item}") } - schedule = { for s in var.aks_automation.schedule : "${s.schedule_name}" => s } + schedule = { for s in local.aks_automation.schedule : "${s.schedule_name}" => s } az_command = "az aks get-credentials --name ${local.cluster_name} --resource-group ${module.resource_groups["azure_kubernetes_service"].name} --admin --overwrite-existing" is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true diff --git a/aks/variables.tf b/aks/variables.tf index b51ae0e..12cbce3 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -160,27 +160,6 @@ variable "runbook" { default = [{}] } -variable "aks_automation" { - description = "Arguments to automate the Azure Kubernetes Cluster" - type = object({ - automation_account_name = string - local_authentication_enabled = optional(bool, false) - public_network_access_enabled = optional(bool, false) - - schedule = list(object({ - description = optional(string, "Stop the Kubernetes cluster.") - schedule_name = optional(string, "aks_stop") - runbook_name = optional(string, "aks_startstop_runbook") # name of the runbook - frequency = string - interval = string - start_time = string - week_days = list(string) - operation = optional(string, "stop") - daylight_saving = optional(bool, false) - })) - }) -} - variable "timezone" { description = "Name of timezone" type = string diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index f08c95c..8966c2a 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -1,5 +1,5 @@ locals { - helm_chart_timeout=600 + helm_chart_timeout=300 #hpcc_version = "8.6.20" owner = { @@ -7,15 +7,17 @@ locals { email = var.aks_admin_email } + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + metadata = { - project = var.product_name - product_name = var.product_name + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) business_unit = "commercial" environment = "sandbox" market = "us" - product_group = "tlhhpcc" + product_group = format("%shpcc", local.owner_name_initials) resource_group_type = "app" - sre_team = var.product_name + sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" additional_tags = { "justification" = "testing" } location = var.aks_azure_region # Acceptable values: eastus, centralus diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example index 58c347d..dfd2f71 100755 --- a/hpcc/lite.auto.tfvars.example +++ b/hpcc/lite.auto.tfvars.example @@ -1,29 +1,16 @@ -# To have a dns zone and an A record, included in your apply, the following 3 -# variables, i.e. 'a_record_name', 'dns_zone_name', and -# dns_zone_resource_group_name must be set to non-blank or non-null values. -#----------------------------------------------------------------------------- - -# Name of the A record, of following dns zone, where the ecl watch ip is placed -# This A record will be created and therefore should not exist in the following -# dns zone. -# Example entry: "my-product". This should be something project specific rather -# than something generic. - -a_record_name="" - #----------------------------------------------------------------------------- # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" -dns_zone_name="" +aks_dns_zone_name="" #----------------------------------------------------------------------------- # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" -dns_zone_resource_group_name="" +aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ @@ -40,9 +27,7 @@ product_name="tlhhpcc" # Only versions in nn.nn.nn format are supported. # Value type: string -hpcc_version="8.6.14" -#hpcc_version="latest" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. -#hpcc_version="8.10.40" # SUCCESSFULLY BRINGS UP HPCC CLUSTER BUT CANNOT EXECUTE ECL ON IT. +hpcc_version="8.10.1" # Currently not used #------------------------------------------------------------------------------ @@ -60,7 +45,7 @@ enable_roxie=false # Value type: boolean # Example entry: false -enable_code_security=true +enable_code_security=false #------------------------------------------------------------------------------ @@ -107,7 +92,7 @@ storage_data_gb=100 # To add no tags, use '{}'. # Value type: map of string -extra_tags={} +extra_tags={} # Currently not used #------------------------------------------------------------------------------ @@ -116,7 +101,7 @@ extra_tags={} # See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. # Value type: string -node_size="Standard_B8ms" +aks_node_size="Standard_B8ms" #------------------------------------------------------------------------------ @@ -124,7 +109,7 @@ node_size="Standard_B8ms" # Must be 2 or more. # Value type: integer -max_node_count=4 +aks_max_node_count=4 #------------------------------------------------------------------------------ @@ -132,7 +117,7 @@ max_node_count=4 # Value type: string # Example entry: "jane.doe@hpccsystems.com" -admin_email="tlhumphrey2@gmail.com" +aks_admin_email="jane.doe@hpccsystems.com" #------------------------------------------------------------------------------ @@ -140,7 +125,7 @@ admin_email="tlhumphrey2@gmail.com" # Value type: string # Example entry: "Jane Doe" -admin_name="Timothy Humphrey" +aks_admin_name="Jane Doe" #------------------------------------------------------------------------------ @@ -148,7 +133,7 @@ admin_name="Timothy Humphrey" # Value type: string # Example entry: "jdoe" -admin_username="tlhumphrey2" +admin_username="jdoe" #------------------------------------------------------------------------------ @@ -157,7 +142,7 @@ admin_username="tlhumphrey2" # Value type: string # Example entry: "eastus" -azure_region="eastus" +aks_azure_region="eastus" #------------------------------------------------------------------------------ @@ -168,7 +153,7 @@ azure_region="eastus" # The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. # Value type: map of string -admin_ip_cidr_map={} +aks_admin_ip_cidr_map={} #------------------------------------------------------------------------------ diff --git a/hpcc/main.tf b/hpcc/main.tf index 1c98a21..dbb417e 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -39,12 +39,9 @@ module "metadata" { } }*/ -resource "null_resource" "launch_svc_url" { - count = (module.hpcc.hpcc_status == "deployed") ? 1 : 0 - - provisioner "local-exec" { - command = "../scripts/hpcc_is_deployed" - } +resource "local_file" "configjson" { + content = "hpcc was successfully deployed!" + filename = "${path.module}/data/config.json" depends_on = [ module.hpcc ] } diff --git a/hpcc/thor.auto.tfvars.example b/hpcc/thor.auto.tfvars.example deleted file mode 100644 index d44d438..0000000 --- a/hpcc/thor.auto.tfvars.example +++ /dev/null @@ -1,36 +0,0 @@ -# Thor Settings -################ - -thor_config = [{ - disabled = false - name = "thor" - prefix = "thor" - numWorkers = 5 - keepJobs = "none" - maxJobs = 4 - maxGraphs = 2 - maxGraphStartupTime = 172800 - numWorkersPerPod = 1 - nodeSelector = {} - egress = "engineEgress" - tolerations_value = "thorpool" - managerResources = { - cpu = 1 - memory = "2G" - } - workerResources = { - cpu = 3 - memory = "4G" - } - workerMemory = { - query = "3G" - thirdParty = "500M" - } - eclAgentResources = { - cpu = 1 - memory = "2G" - } - cost = { - perCpu = 1 - } -}] diff --git a/scripts/deploy b/scripts/deploy index 35756dc..b601c34 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,4 +1,5 @@ #!/bin/bash +thisdir=$(d=`dirname $0`;cd $d;pwd;cd ..) repodir=$(echo `pwd`|sed "s/\/scripts\/*//") if [ "$1" != "" ];then name=$1 @@ -7,11 +8,11 @@ else fi #======================================================================== function assert_fail () { - echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" + echo ">>>>>>>>>>>>>>> EXECUTING: $*" if "$@"; then - echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo + echo;echo ">>>>>>>>>>>>>>> Successful: $*";echo else - echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + echo;echo ">>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo rm -vrf data cd $repodir rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform @@ -21,60 +22,69 @@ function assert_fail () { #======================================================================== # make sure data/config.json doesn't exist if kubernetes cluster doesn't exist # Delete both hpcc/data and aks/data if kubernetes cluster doesn't exist -# If kubernetes cluster does exist but there are no pods in the default namespace -# then delete only hpcc/data ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` +pods=`kubectl get pods 2>&1` if [[ "$ns" == *"Unable"* ]];then # force rm data/config.json in hpcc and aks directories echo "Forcibly delete hpcc/data and aks/data" - rm -vrf hpcc/data; rm -vrf aks/data -fi -pods=`kubectl get pods 2>&1` -if [[ "$pods" == *"Unable"* ]] || [[ "$pods" == *"No resources found in default namespace"* ]];then + rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. + rm -vrf aks/data;cd aks; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. +# If kubernetes cluster does exist but there are no pods in the default namespace +# then delete only hpcc/data +elif [[ "$pods" == *"Unable"* ]] || [[ "$pods" == *"No resources found in default namespace"* ]];then # force rm data/config.json in hpcc only echo "Forcibly delete hpcc/data only" - rm -vrf hpcc/data + rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. fi - -# See if vnet/data/config.json exists. If it does then from the file get +#------------------------------------------------------------------------ +# See if $dir/data/config.json exists. If it does then from the file get # the resource group name and then check to see if resource group exists. -# if it doesn't exists then delete vnet/data/config.json -if [ -e "vnet/data/config.json" ];then - # Get resource group name from file - rg=`scripts/get_rg_from_file vnet/data/config.json` - if [ $(az group exists --name $rg) = false ]; then - echo "vnet resource group, \"$rg\" does not exists. So deleting vnet/data/config.json" - rm -vrf vnet/data -else - echo "vnet resource group, \"$rg\" DOES exists. So NOT deleting vnet/data/config.json" +# if it doesn't exists then delete $dir/data/config.json +#------------------------------------------------------------------------ +for dir in aks vnet;do + if [ -e "$dir/data/config.json" ];then + # Get resource group name from file + rg=`scripts/get_rg_from_file $dir/data/config.json` + if [ $(az group exists --name $rg) = false ]; then + echo "In $0. vnet resource group, \"$rg\" does not exists. So deleting vnet/data and any tfstate files id $dir." + rm -vrf $dir/data + cd $dir; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. + else + echo "In $0. vnet resource group, \"$rg\" DOES exists. So NOT deleting vnet/data." + fi fi -fi +done #------------------------------------------------------------------------ + cd $name; # cd into vnet or aks or hpcc # put the root directory's lite.auto.tfvars (either all of part) in either aks or hpcc # directory. if [ -e "../lite.auto.tfvars" ];then + # Check of there has been a change since last apply. if [ -e "/tmp/${name}.lite.auto.tfvars" ];then diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` else diff="" fi else - echo "The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; + echo "In $0. The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; fi if [ "$name" == "hpcc" ];then echo "Coping root's lite.auto.tfvars to /tmp and $name directory." cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars cp -v ../lite.auto.tfvars . cp -v ../lite-variables.tf . -elif [ "$name" == "aks" ];then +elif [ "$name" == "aks" ] || [ "$name" == "vnet" ];then egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf fi #------------------------------------------------------------------------ -plan=`/home/azureuser/mkplan ${name}_deployment.plan` +if [ ! -d "$HOME/tflogs" ];then + mkdir $HOME/tflogs +fi +plan=`$thisdir/mkplan deploy_${name}.plan` if [ "$diff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi if [ "$name" != "vnet" ];then cp -v /tmp/${name}.lite.auto.tfvars . @@ -86,5 +96,9 @@ echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ====== assert_fail terraform plan -out=$plan echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; assert_fail terraform apply $plan -cd $repodir -rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform +#r=`pwd` +#if [ "$name" == "hpcc" ];then +# cd $repodir +# rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform +# cd $r +#fi diff --git a/scripts/destroy b/scripts/destroy index d4f9ff2..943d204 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -1,4 +1,5 @@ #!/bin/bash +thisdir=$(d=`dirname $0`;cd $d;pwd;cd ..) #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" @@ -20,7 +21,10 @@ elif [ "$1" == "aks" ];then fi cd $1; name=$(basename `pwd`) -plan=`/home/azureuser/mkplan ${name}_destroy.plan` +if [ ! -d "$HOME/tflogs" ];then + mkdir $HOME/tflogs +fi +plan=`$thisdir/mkplan ${name}_destroy.plan` if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file index 598d3c7..a338358 100755 --- a/scripts/get_rg_from_file +++ b/scripts/get_rg_from_file @@ -2,10 +2,10 @@ if ( scalar(@ARGV) > 0 ){ $config_file = shift @ARGV; } else{ - die "FATAL ERROR: config file path must be given on command line\n"; + die "FATAL ERROR: In $0. Config file path must be given on command line\n"; } undef $/; -open(IN, $config_file) || die "Can't open config file, \"$config_file\"\n"; +open(IN, $config_file) || die "In $0. Can't open config file, \"$config_file\"\n"; $_ = ; close(IN); diff --git a/scripts/hpcc_is_deployed b/scripts/hpcc_is_deployed index ee6a7d2..4768cf2 100755 --- a/scripts/hpcc_is_deployed +++ b/scripts/hpcc_is_deployed @@ -2,7 +2,4 @@ name=$(basename `pwd`) if [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; -else - if [ ! -d "data" ];then mkdir data; fi - touch data/config.json fi diff --git a/scripts/mkplan b/scripts/mkplan new file mode 100755 index 0000000..9169053 --- /dev/null +++ b/scripts/mkplan @@ -0,0 +1,26 @@ +#!//usr/bin/perl +$tmpl = "/home/azureuser/tflogs/-.plan"; +$month_date = `date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'`; +$timezone_difference=6; +$H = `date +'%H'`;chomp $H; +$H = $H-$timezone_difference; +$H = sprintf("%02d",$H); +$M = `date +'%M'`;chomp $M; +$M = sprintf("%02d",$M); +$month_date =~ s/\s+//g; +$month_date = "$month_date-$H$M"; +$repo_name = ""; +if (scalar(@ARGV)>0){ + $repo_name = shift @ARGV; +} else { + die "FATAL ERROR. Plan name must be given on command line after 'mkplan'."; +} +$repo_name =~ s/^\/home\/azureuser\/tflogs\///; +$repo_name =~ s/\-\d{8}\-\d{1,2}\.?\d{1,2}\.plan//; +$repo_name =~ s/\//-/g; + +$_ = $tmpl; +s//$repo_name/; +s//$month_date/; +print "$_\n"; + diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf new file mode 100644 index 0000000..7183c29 --- /dev/null +++ b/vnet/lite-variables.tf @@ -0,0 +1,54 @@ +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "aks_dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/vnet/locals.tf b/vnet/locals.tf index 776e751..344ca8e 100644 --- a/vnet/locals.tf +++ b/vnet/locals.tf @@ -1,18 +1,45 @@ locals { + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + resource_groups = { + virtual_network = { + tags = { "enclosed resource" = "open source vnet" } + } + } + names = var.disable_naming_conventions ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.resource_groups.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) + tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) private_subnet_id = module.virtual_network.aks.hpcc.subnets["private"].id diff --git a/vnet/main.tf b/vnet/main.tf index b64712c..7ff07dd 100644 --- a/vnet/main.tf +++ b/vnet/main.tf @@ -12,23 +12,23 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market - location = var.metadata.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + market = local.metadata.market + location = local.metadata.location + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = data.azurerm_subscription.current.id - project = var.metadata.project + project = local.metadata.project } module "resource_groups" { source = "github.com/Azure-Terraform/terraform-azurerm-resource-group.git?ref=v2.1.0" - for_each = var.resource_groups + for_each = local.resource_groups unique_name = true location = module.metadata.location diff --git a/vnet/misc.auto.tfvars.example b/vnet/misc.auto.tfvars.example deleted file mode 100644 index 1838b44..0000000 --- a/vnet/misc.auto.tfvars.example +++ /dev/null @@ -1,24 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -resource_groups = { - virtual_network = { - tags = { "enclosed resource" = "open source vnet" } - } -} diff --git a/vnet/variables.tf b/vnet/variables.tf index fd5d060..4d334a2 100644 --- a/vnet/variables.tf +++ b/vnet/variables.tf @@ -1,66 +1,5 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - variable "disable_naming_conventions" { description = "Naming convention module." type = bool default = false } - -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - -variable "resource_groups" { - description = "Resource group module variables." - type = any - - default = { - azure_kubernetes_service = { - tags = { "apps" = "vnet" } - } - } -} From c54dad883e744d194aba63e372c83389d9ce08e7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sat, 28 Oct 2023 13:55:51 +0000 Subject: [PATCH 019/124] branch:HPCC-27615-easy-deploy-bryan1 --- aks/aks.tf | 8 +++++--- aks/automation.tf | 2 ++ aks/variables.tf | 2 +- hpcc/hpcc.tf | 2 ++ hpcc/locals.tf | 1 - 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 281bc17..41c9e47 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -21,8 +21,10 @@ module "aks" { depends_on = [random_string.string] #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" + source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" + #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default @@ -75,8 +77,8 @@ module "aks" { blob = { enabled = true } } - # tlh logging = var.logging - logging = null + logging = var.logging + #logging = null experimental = { oms_agent = var.hpcc_log_analytics_enabled || var.experimental.oms_agent diff --git a/aks/automation.tf b/aks/automation.tf index f6bfae5..382bd7f 100644 --- a/aks/automation.tf +++ b/aks/automation.tf @@ -1,3 +1,4 @@ +/* resource "azurerm_automation_account" "automation_account" { name = local.aks_automation.automation_account_name location = local.location @@ -63,3 +64,4 @@ resource "azurerm_automation_job_schedule" "job_schedule" { automationaccount = local.aks_automation.automation_account_name } } +*/ diff --git a/aks/variables.tf b/aks/variables.tf index 12cbce3..9314091 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -80,7 +80,7 @@ variable "sku_tier" { description = "Pricing tier for the Azure Kubernetes Service managed cluster; \"free\" & \"paid\" are supported. For production clusters or clusters with more than 10 nodes this should be set to \"paid\"." type = string nullable = false - default = "free" + default = "FREE" validation { condition = contains(["free", "paid"], var.sku_tier) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 4b0ece6..9f1df8d 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -23,6 +23,8 @@ module "hpcc" { #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + depends_on = [ local.get_aks_config ] + environment = local.metadata.environment productname = local.metadata.product_name diff --git a/hpcc/locals.tf b/hpcc/locals.tf index 1e3dd9a..8a2475e 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -21,7 +21,6 @@ locals { get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - #get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null get_storage_config = fileexists("../storage/data/config.json") ? jsondecode(file("../storage/data/config.json")) : null external_storage_exists = fileexists("../storage/data/config.json") || local.external_storage_config != null From f077f15fc13208f40542f04b503954552794c586 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sat, 28 Oct 2023 16:08:28 +0000 Subject: [PATCH 020/124] branch:HPCC-27615-easy-deploy-bryan1-w-oss --- aks/aks.tf | 4 ++-- aks/variables.tf | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 41c9e47..acfdad4 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -23,8 +23,8 @@ module "aks" { #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" - source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" - #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" + source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default diff --git a/aks/variables.tf b/aks/variables.tf index 9314091..1802700 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -77,14 +77,14 @@ variable "cluster_version" { } variable "sku_tier" { - description = "Pricing tier for the Azure Kubernetes Service managed cluster; \"free\" & \"paid\" are supported. For production clusters or clusters with more than 10 nodes this should be set to \"paid\"." + description = "Pricing tier for the Azure Kubernetes Service managed cluster; \"FREE\" & \"PAID\" are supported. For production clusters or clusters with more than 10 nodes this should be set to \"PAID\"." type = string nullable = false default = "FREE" validation { - condition = contains(["free", "paid"], var.sku_tier) - error_message = "Available SKU tiers are \"free\" or \"paid\"." + condition = contains(["FREE", "PAID"], var.sku_tier) + error_message = "Available SKU tiers are \"FREE\" or \"PAID\"." } } From 1868452c0fa789e44477dd3bf01c28e1be9fec88 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sat, 28 Oct 2023 16:54:25 +0000 Subject: [PATCH 021/124] branch:HPCC-27615-easy-deploy-bryan1-w-oss-zones --- aks/aks.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aks/aks.tf b/aks/aks.tf index acfdad4..fd8dc8b 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -54,7 +54,7 @@ module "aks" { rbac_bindings = var.rbac_bindings - #availability_zones = var.availability_zones + availability_zones = var.availability_zones node_groups = var.node_groups core_services_config = { From 1523b644c8a7e9536648f7f9b9a6ab3e9eed8c95 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 30 Oct 2023 16:52:25 +0000 Subject: [PATCH 022/124] branch:HPCC-27615-easy-deploy-bryan1-w-oss-zones. Now can optionally enable oss --- aks/aks.tf | 2 ++ aks/variables.tf | 7 +++++++ hpcc/lite-locals.tf | 2 +- storage/main.tf | 3 ++- 4 files changed, 12 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index fd8dc8b..abf6d5a 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -41,6 +41,8 @@ module "aks" { # for v1.6.2 aks: sku_tier_paid = false sku_tier = var.sku_tier + logging_monitoring_enabled = var.logging_monitoring_enabled + cluster_endpoint_access_cidrs = var.cluster_endpoint_access_cidrs virtual_network_resource_group_name = try(var.use_existing_vnet.resource_group_name, local.get_vnet_config.resource_group_name) diff --git a/aks/variables.tf b/aks/variables.tf index 1802700..ce2f470 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -200,6 +200,13 @@ variable "cluster_endpoint_access_cidrs" { } } +variable "logging_monitoring_enabled" { + description = "If true then logging and monitoring will occur else it will not." + type = bool + nullable = false + default = false +} + variable "logging" { description = "Logging configuration." type = object({ diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 8966c2a..9443bf2 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -471,7 +471,7 @@ locals { onprem_lz_settings = {} - ignore_external_storage = true + ignore_external_storage = false admin_services_node_selector = {} diff --git a/storage/main.tf b/storage/main.tf index ac8c323..97facb1 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,7 +1,8 @@ module "storage" { #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" + source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" owner = var.owner disable_naming_conventions = var.disable_naming_conventions From 8e0e2f1c25e6eae576dd51c5501b8eb03e0cb307 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 30 Oct 2023 16:52:25 +0000 Subject: [PATCH 023/124] branch:HPCC-27615-easy-deploy-bryan1-w-oss-zones. Now can optionally enable logging and monitoring. --- aks/aks.tf | 2 ++ aks/variables.tf | 7 +++++++ hpcc/lite-locals.tf | 2 +- storage/main.tf | 3 ++- 4 files changed, 12 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index fd8dc8b..abf6d5a 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -41,6 +41,8 @@ module "aks" { # for v1.6.2 aks: sku_tier_paid = false sku_tier = var.sku_tier + logging_monitoring_enabled = var.logging_monitoring_enabled + cluster_endpoint_access_cidrs = var.cluster_endpoint_access_cidrs virtual_network_resource_group_name = try(var.use_existing_vnet.resource_group_name, local.get_vnet_config.resource_group_name) diff --git a/aks/variables.tf b/aks/variables.tf index 1802700..ce2f470 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -200,6 +200,13 @@ variable "cluster_endpoint_access_cidrs" { } } +variable "logging_monitoring_enabled" { + description = "If true then logging and monitoring will occur else it will not." + type = bool + nullable = false + default = false +} + variable "logging" { description = "Logging configuration." type = object({ diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 8966c2a..9443bf2 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -471,7 +471,7 @@ locals { onprem_lz_settings = {} - ignore_external_storage = true + ignore_external_storage = false admin_services_node_selector = {} diff --git a/storage/main.tf b/storage/main.tf index ac8c323..97facb1 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,7 +1,8 @@ module "storage" { #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" + source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" owner = var.owner disable_naming_conventions = var.disable_naming_conventions From d2ee6500cf51e4d5c71749c6b4fecf952dbd569c Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 30 Oct 2023 19:49:47 +0000 Subject: [PATCH 024/124] branch: HPCC-27615-easy-deploy-bryan1-w-oss-zones --- aks/aks.tf | 2 +- aks/variables.tf | 2 +- lite.auto.tfvars.example | 8 ++++++++ 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index abf6d5a..7993c8a 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -41,7 +41,7 @@ module "aks" { # for v1.6.2 aks: sku_tier_paid = false sku_tier = var.sku_tier - logging_monitoring_enabled = var.logging_monitoring_enabled + logging_monitoring_enabled = var.aks_logging_monitoring_enabled cluster_endpoint_access_cidrs = var.cluster_endpoint_access_cidrs diff --git a/aks/variables.tf b/aks/variables.tf index ce2f470..692d97d 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -200,7 +200,7 @@ variable "cluster_endpoint_access_cidrs" { } } -variable "logging_monitoring_enabled" { +variable "aks_logging_monitoring_enabled" { description = "If true then logging and monitoring will occur else it will not." type = bool nullable = false diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index dfd2f71..61884cc 100755 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -217,3 +217,11 @@ enable_premium_storage=false # Example entry: "htpasswd.txt" authn_htpasswd_filename="" + +#------------------------------------------------------------------------------ + +# You can optionally turn logging and monitoring on or off by setting the +# following variable, where true means logging and monitoring will be done +# while false means it won't be done. + +aks_logging_monitoring_enabled=false From 50f7050cfdb5e89d7e91c29518b337d48c241ff2 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 31 Oct 2023 20:35:27 +0000 Subject: [PATCH 025/124] branch:HPCC-27615-easy-deploy-bryan2-root-sto-applied-initials-added. Using initials now to prefix some metadata variables. Also, now can deploy storage from root. --- aks/aks.tf | 1 + aks/locals.tf | 4 +-- hpcc/lite-locals.tf | 2 -- hpcc/lite-variables.tf | 6 +++- hpcc/locals.tf | 6 ++-- lite-variables.tf | 6 +++- lite.auto.tfvars.example | 8 +++++ main.tf | 24 +++++++++++++- scripts/deploy | 49 ++++++++++++++--------------- scripts/external_storage | 16 ++++++++++ scripts/get_rg_from_file | 2 +- scripts/hpcc_is_deployed | 5 --- scripts/mkplan | 15 +++++++-- storage/lite-variables.tf | 54 ++++++++++++++++++++++++++++++++ storage/locals.tf | 21 +++++++++++++ storage/main.tf | 4 +-- storage/misc.auto.tfvars.example | 23 -------------- storage/variables.tf | 50 ----------------------------- 18 files changed, 178 insertions(+), 118 deletions(-) create mode 100755 scripts/external_storage delete mode 100755 scripts/hpcc_is_deployed create mode 100644 storage/lite-variables.tf delete mode 100644 storage/misc.auto.tfvars.example diff --git a/aks/aks.tf b/aks/aks.tf index 7993c8a..892cb76 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -22,6 +22,7 @@ module "aks" { depends_on = [random_string.string] #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=OSS" #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" diff --git a/aks/locals.tf b/aks/locals.tf index 6cd6c5b..bcf180f 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -14,13 +14,13 @@ locals { schedule = [ { schedule_name = "aks_stop" - description = "Stops the AKS weekday nights at 6PM EST" + description = "Stops the AKS weekday nights at 6PM MST" runbook_name = "aks_startstop_runbook" frequency = "Week" //OneTime, Day, Hour, Week, or Month. interval = "1" //cannot be set when frequency is `OneTime` operation = "stop" daylight_saving = true - start_time = "18:00" // At least 5 minutes in the future + start_time = "20:00" // At least 5 minutes in the future week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] }, # { diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 9443bf2..3bae99f 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -471,8 +471,6 @@ locals { onprem_lz_settings = {} - ignore_external_storage = false - admin_services_node_selector = {} thor_config = [{ diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index 1e2a70f..f2edec7 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -2,12 +2,16 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### +variable "ignore_external_storage" { + description = "If you definitely want ephemeral storage instead of external, this should be true." + type = bool + default = true +} variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." type = bool } - variable "a_record_name" { type = string description = "OPTIONAL: dns zone A record name" diff --git a/hpcc/locals.tf b/hpcc/locals.tf index 8a2475e..939bd42 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -35,14 +35,14 @@ locals { domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) - internal_storage_enabled = (local.external_storage_exists == true) && (local.ignore_external_storage == true) ? true : local.external_storage_exists == true && local.ignore_external_storage == false ? false : true - #internal_storage_enabled = local.external_storage_exists == true && local.ignore_external_storage == true ? true : local.external_storage_exists == true && local.ignore_external_storage == false ? false : true + internal_storage_enabled = (local.external_storage_exists == true) && (var.ignore_external_storage == true) ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true + #internal_storage_enabled = local.external_storage_exists == true && var.ignore_external_storage == true ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true # external_services_storage_enabled = local.external_services_storage_exists == true && local.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && local.ignore_external_services_storage == true ? false : true #hpcc_namespace = local.hpcc_namespace.existing_namespace != null ? local.hpcc_namespace.existing_namespace : local.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("../logging/data/hpcc_namespace.txt") ? file("../logging/data/hpcc_namespace.txt") : "default" hpcc_namespace = "default" - external_storage_config = local.get_storage_config != null && local.ignore_external_storage == false ? [ + external_storage_config = local.get_storage_config != null && var.ignore_external_storage == false ? [ for plane in local.get_storage_config.external_storage_config : { category = plane.category diff --git a/lite-variables.tf b/lite-variables.tf index 1e2a70f..f2edec7 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -2,12 +2,16 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### +variable "ignore_external_storage" { + description = "If you definitely want ephemeral storage instead of external, this should be true." + type = bool + default = true +} variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." type = bool } - variable "a_record_name" { type = string description = "OPTIONAL: dns zone A record name" diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 61884cc..a5382bc 100755 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -225,3 +225,11 @@ authn_htpasswd_filename="" # while false means it won't be done. aks_logging_monitoring_enabled=false + +#------------------------------------------------------------------------------ + +# If you definitely want ephemeral storage instead of external storage then +# set this variable to true otherwise set it to false. + +ignore_external_storage=true + diff --git a/main.tf b/main.tf index 1551462..618f156 100644 --- a/main.tf +++ b/main.tf @@ -14,11 +14,33 @@ resource "null_resource" "deploy_aks" { depends_on = [ null_resource.deploy_vnet ] } +resource "null_resource" "deploy_storage" { + count = (var.ignore_external_storage == false)? 1 : 0 + + provisioner "local-exec" { + command = "scripts/deploy storage" + } + + #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] + depends_on = [ null_resource.deploy_vnet ] +} + +resource "null_resource" "external_storage" { + count = (var.ignore_external_storage == false)? 1 : 0 + + provisioner "local-exec" { + command = "scripts/external_storage ${path.module} ${var.ignore_external_storage}" + } + + #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] + depends_on = [ null_resource.deploy_vnet ] +} + resource "null_resource" "deploy_hpcc" { provisioner "local-exec" { command = "scripts/deploy hpcc" } - depends_on = [ null_resource.deploy_aks ] + depends_on = [ null_resource.deploy_aks, null_resource.deploy_vnet, null_resource.external_storage ] } diff --git a/scripts/deploy b/scripts/deploy index b601c34..4cf921c 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,10 +1,10 @@ #!/bin/bash -thisdir=$(d=`dirname $0`;cd $d;pwd;cd ..) -repodir=$(echo `pwd`|sed "s/\/scripts\/*//") -if [ "$1" != "" ];then +thisdir=$(dirname $0) +repodir=$(echo $thisdir|sed "s/\/scripts\/*//") +if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then name=$1 else - echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'aks' or 'hpcc'. EXITING.";exit 1; + echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'storage' or 'aks' or 'hpcc'. EXITING.";exit 1; fi #======================================================================== function assert_fail () { @@ -20,13 +20,12 @@ function assert_fail () { fi } #======================================================================== -# make sure data/config.json doesn't exist if kubernetes cluster doesn't exist +# If kubernetes cluster doesn't exist then make sure aks/data/config.json +# and hpcc/data/config.json doesn't exist # Delete both hpcc/data and aks/data if kubernetes cluster doesn't exist ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` pods=`kubectl get pods 2>&1` -if [[ "$ns" == *"Unable"* ]];then - # force rm data/config.json in hpcc and aks directories - echo "Forcibly delete hpcc/data and aks/data" +if [[ "$ns" == *"Unable"* ]];then # kubenetes doesn't exist of there are no namespaces rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. rm -vrf aks/data;cd aks; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. # If kubernetes cluster does exist but there are no pods in the default namespace @@ -37,31 +36,33 @@ elif [[ "$pods" == *"Unable"* ]] || [[ "$pods" == *"No resources found in defaul rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. fi #------------------------------------------------------------------------ -# See if $dir/data/config.json exists. If it does then from the file get +# See if $dir/data/config.json exists, where $dir is vnet or aks or storage. +# If $dir/data/config.json does exist then from the file get # the resource group name and then check to see if resource group exists. # if it doesn't exists then delete $dir/data/config.json #------------------------------------------------------------------------ -for dir in aks vnet;do +for dir in aks vnet storage;do if [ -e "$dir/data/config.json" ];then # Get resource group name from file - rg=`scripts/get_rg_from_file $dir/data/config.json` - if [ $(az group exists --name $rg) = false ]; then - echo "In $0. vnet resource group, \"$rg\" does not exists. So deleting vnet/data and any tfstate files id $dir." + rg=`$thisdir/get_rg_from_file $dir/data/config.json` + rgexist=`az group exists --name $rg` + if [ "$rgexist" == "false" ]; then + echo "In $0. $dir resource group, \"$rg\" does not exists. So deleting vnet/data and any tfstate files id $dir." rm -vrf $dir/data cd $dir; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. else - echo "In $0. vnet resource group, \"$rg\" DOES exists. So NOT deleting vnet/data." + echo "In $0. $dir resource group, \"$rg\" DOES exists. So NOT deleting vnet/data." fi fi done #------------------------------------------------------------------------ -cd $name; # cd into vnet or aks or hpcc +cd $name; # cd into vnet or storage or aks or hpcc # put the root directory's lite.auto.tfvars (either all of part) in either aks or hpcc # directory. if [ -e "../lite.auto.tfvars" ];then - # Check of there has been a change since last apply. + # Check if there has been a change since last apply. if [ -e "/tmp/${name}.lite.auto.tfvars" ];then diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` else @@ -75,7 +76,7 @@ if [ "$name" == "hpcc" ];then cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars cp -v ../lite.auto.tfvars . cp -v ../lite-variables.tf . -elif [ "$name" == "aks" ] || [ "$name" == "vnet" ];then +elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];then egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf @@ -94,11 +95,9 @@ echo "=============== Deploy $name. Executing 'terraform init' ==============="; assert_fail terraform init echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; assert_fail terraform plan -out=$plan -echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; -assert_fail terraform apply $plan -#r=`pwd` -#if [ "$name" == "hpcc" ];then -# cd $repodir -# rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform -# cd $r -#fi +if [ -e "$plan" ];then + echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; + assert_fail terraform apply -auto-approve $plan +else + echo "@@@@@@@@@@ No changes. Your infrastructure matches the configuration. So, terraform apply for $name will not be done. @@@@@@@@@@" +fi diff --git a/scripts/external_storage b/scripts/external_storage new file mode 100755 index 0000000..6894420 --- /dev/null +++ b/scripts/external_storage @@ -0,0 +1,16 @@ +#!/usr/bin/perl +$repopath = shift @ARGV; +$ignore_external_storage = shift @ARGV; + +#print "DEBUG: {\"repopath\" : \"$repopath\", \"ignore_external_storage\" : \"$ignore_external_storage\"}\n"; + +if ( $ignore_external_storage eq "true" ){ + #print "DEBUG: EXITING because ignore_external_storage is true.\n"; + exit 0; +}else{ + #print "DEBUG: EXITING because ignore_external_storage is NOT true.\n"; + while ( ! -e "$repopath/storage/data/config.json" ) { + sleep 10; + } + exit 0; +} diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file index a338358..43ab9f3 100755 --- a/scripts/get_rg_from_file +++ b/scripts/get_rg_from_file @@ -9,5 +9,5 @@ open(IN, $config_file) || die "In $0. Can't open config file, \"$config_file\"\n $_ = ; close(IN); -$rg = $1 if ( /"resource_group_name":"([^"]+)"/ ); +$rg = $1 if ( /"resource_group(?:_name)?":\s*"([^"]+)"/s ); print $rg; diff --git a/scripts/hpcc_is_deployed b/scripts/hpcc_is_deployed deleted file mode 100755 index 4768cf2..0000000 --- a/scripts/hpcc_is_deployed +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh -name=$(basename `pwd`) -if [ -d "data" ] && [ -f "data/config.json" ]; then - echo "Complete! $name is already deployed";exit 0; -fi diff --git a/scripts/mkplan b/scripts/mkplan index 9169053..dd8cc0e 100755 --- a/scripts/mkplan +++ b/scripts/mkplan @@ -1,9 +1,16 @@ #!//usr/bin/perl +$HOME = $ENV{'HOME'}; $tmpl = "/home/azureuser/tflogs/-.plan"; $month_date = `date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'`; $timezone_difference=6; $H = `date +'%H'`;chomp $H; -$H = $H-$timezone_difference; +#print STDERR "DEBUG: H=\"$H\"\n"; +if ( $H <= $timezone_difference ){ + $H = 25 - $timezone_difference; +}else{ + $H = $H-$timezone_difference; +} +#print STDERR "DEBUG: After timezone difference: H=\"$H\"\n"; $H = sprintf("%02d",$H); $M = `date +'%M'`;chomp $M; $M = sprintf("%02d",$M); @@ -13,7 +20,10 @@ $repo_name = ""; if (scalar(@ARGV)>0){ $repo_name = shift @ARGV; } else { - die "FATAL ERROR. Plan name must be given on command line after 'mkplan'."; + $cdir=`pwd`;chomp $cdir; + $reHOME = $HOME; $reHOME =~ s/(\/)/\\$1/g; + $repo_name = $cdir; $repo_name =~ s/^$reHOME\///; + #print "DEBUG: No arguments: cdir=\"$cdir\", reHOME=\"$reHOME\", repo_name=\"$repo_name\"\n"; } $repo_name =~ s/^\/home\/azureuser\/tflogs\///; $repo_name =~ s/\-\d{8}\-\d{1,2}\.?\d{1,2}\.plan//; @@ -22,5 +32,6 @@ $repo_name =~ s/\//-/g; $_ = $tmpl; s//$repo_name/; s//$month_date/; +print STDERR "$_\n"; print "$_\n"; diff --git a/storage/lite-variables.tf b/storage/lite-variables.tf new file mode 100644 index 0000000..7183c29 --- /dev/null +++ b/storage/lite-variables.tf @@ -0,0 +1,54 @@ +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "OPTIONAL: Name of the resource group containing the dns zone." + default = "" +} + +variable "aks_dns_zone_name" { + type = string + description = "OPTIONAL: dns zone name. The name of existing dns zone." + default = "" +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/storage/locals.tf b/storage/locals.tf index 4e26af1..ca2a8be 100644 --- a/storage/locals.tf +++ b/storage/locals.tf @@ -1,4 +1,25 @@ locals { + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null # get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null diff --git a/storage/main.tf b/storage/main.tf index 97facb1..02c221b 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -4,9 +4,9 @@ module "storage" { #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" - owner = var.owner + owner = local.owner disable_naming_conventions = var.disable_naming_conventions - metadata = var.metadata + metadata = local.metadata subnet_ids = local.subnet_ids storage_accounts = var.storage_accounts } diff --git a/storage/misc.auto.tfvars.example b/storage/misc.auto.tfvars.example deleted file mode 100644 index f01112f..0000000 --- a/storage/misc.auto.tfvars.example +++ /dev/null @@ -1,23 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -# disable_naming_conventions = false # true will enforce all the arguments of the metadata block above - -# Provide an existing virtual network deployed outside of this project - diff --git a/storage/variables.tf b/storage/variables.tf index 8333e88..e5e3946 100644 --- a/storage/variables.tf +++ b/storage/variables.tf @@ -1,59 +1,9 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - variable "disable_naming_conventions" { description = "Naming convention module." type = bool default = false } -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - variable "virtual_network" { description = "Subnet IDs" type = list(object({ From 108641a55d04149dce7e32caf76b43d03e2d1bbf Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 31 Oct 2023 20:48:00 +0000 Subject: [PATCH 026/124] branch:HPCC-27615-easy-deploy-bryan2-root-sto-applied-initials-added. Using initials now to prefix some metadata variables. Also, now can deploy storage from root. --- hpcc/locals.tf | 8 ++++---- lite-variables.tf | 6 +++--- main.tf | 6 +++--- scripts/external_storage | 6 +++--- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/hpcc/locals.tf b/hpcc/locals.tf index 939bd42..e9ae039 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -35,14 +35,14 @@ locals { domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) - internal_storage_enabled = (local.external_storage_exists == true) && (var.ignore_external_storage == true) ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true - #internal_storage_enabled = local.external_storage_exists == true && var.ignore_external_storage == true ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true - # external_services_storage_enabled = local.external_services_storage_exists == true && local.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && local.ignore_external_services_storage == true ? false : true + internal_storage_enabled = (local.external_storage_exists == true) && (var.external_storage_desired == false) ? true : local.external_storage_exists == true && var.external_storage_desired == true ? false : true + #internal_storage_enabled = local.external_storage_exists == true && var.external_storage_desired == false ? true : local.external_storage_exists == true && var.external_storage_desired == true ? false : true + # external_services_storage_enabled = local.external_services_storage_exists == true && local.external_storage_desired == true ? true : local.external_services_storage_exists == true && local.external_storage_desired == false ? false : true #hpcc_namespace = local.hpcc_namespace.existing_namespace != null ? local.hpcc_namespace.existing_namespace : local.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("../logging/data/hpcc_namespace.txt") ? file("../logging/data/hpcc_namespace.txt") : "default" hpcc_namespace = "default" - external_storage_config = local.get_storage_config != null && var.ignore_external_storage == false ? [ + external_storage_config = local.get_storage_config != null && var.external_storage_desired == true ? [ for plane in local.get_storage_config.external_storage_config : { category = plane.category diff --git a/lite-variables.tf b/lite-variables.tf index f2edec7..03280fa 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -2,10 +2,10 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### -variable "ignore_external_storage" { - description = "If you definitely want ephemeral storage instead of external, this should be true." +variable "external_storage_desired" { + description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" type = bool - default = true + default = false } variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." diff --git a/main.tf b/main.tf index 618f156..9e30bc3 100644 --- a/main.tf +++ b/main.tf @@ -15,7 +15,7 @@ resource "null_resource" "deploy_aks" { } resource "null_resource" "deploy_storage" { - count = (var.ignore_external_storage == false)? 1 : 0 + count = (var.external_storage_desired == true)? 1 : 0 provisioner "local-exec" { command = "scripts/deploy storage" @@ -26,10 +26,10 @@ resource "null_resource" "deploy_storage" { } resource "null_resource" "external_storage" { - count = (var.ignore_external_storage == false)? 1 : 0 + count = (var.external_storage_desired == true)? 1 : 0 provisioner "local-exec" { - command = "scripts/external_storage ${path.module} ${var.ignore_external_storage}" + command = "scripts/external_storage ${path.module} ${var.external_storage_desired}" } #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] diff --git a/scripts/external_storage b/scripts/external_storage index 6894420..65cca62 100755 --- a/scripts/external_storage +++ b/scripts/external_storage @@ -1,10 +1,10 @@ #!/usr/bin/perl $repopath = shift @ARGV; -$ignore_external_storage = shift @ARGV; +$external_storage_desired = shift @ARGV; -#print "DEBUG: {\"repopath\" : \"$repopath\", \"ignore_external_storage\" : \"$ignore_external_storage\"}\n"; +#print "DEBUG: {\"repopath\" : \"$repopath\", \"external_storage_desired\" : \"$external_storage_desired\"}\n"; -if ( $ignore_external_storage eq "true" ){ +if ( $external_storage_desired eq "false" ){ #print "DEBUG: EXITING because ignore_external_storage is true.\n"; exit 0; }else{ From 19e114a62b27463e23b81fe364976e1569232165 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 1 Nov 2023 13:49:26 +0000 Subject: [PATCH 027/124] branch:HPCC-27615-easy-deploy-bryan2-root-sto-applied-initials-added2 --- aks/aks.tf | 3 +- hpcc/hpcc.tf | 3 +- hpcc/lite-variables.tf | 43 ++--------------------------- lite-variables.tf | 37 ------------------------- lite.auto.tfvars.example | 59 ++++++++++++++++------------------------ scripts/deploy | 4 ++- scripts/mkplan | 1 - storage/main.tf | 3 +- 8 files changed, 35 insertions(+), 118 deletions(-) mode change 100755 => 100644 lite.auto.tfvars.example diff --git a/aks/aks.tf b/aks/aks.tf index 892cb76..b162f18 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -25,7 +25,8 @@ module "aks" { #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=OSS" #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" - source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" providers = { kubernetes = kubernetes.default diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 9f1df8d..ab8932f 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,7 +21,8 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" depends_on = [ local.get_aks_config ] diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index f2edec7..ee36489 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -2,10 +2,10 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### -variable "ignore_external_storage" { - description = "If you definitely want ephemeral storage instead of external, this should be true." +variable "external_storage_desired" { + description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" type = bool - default = true + default = false } variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." @@ -113,15 +113,6 @@ variable "aks_node_size" { description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } -variable "product_name" { - type = string - description = "REQUIRED. Abbreviated product name, suitable for use in Azure naming.\nMust be 3-16, all lowercase or numeric characters.\nExample entry: myproduct" - validation { - condition = can(regex("^[a-z][a-z0-9]{2,15}$", var.product_name)) - error_message = "Value must be [a-z0-9]{3,16}." - } -} - variable "storage_data_gb" { type = number description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." @@ -168,34 +159,6 @@ variable "authn_htpasswd_filename" { default = "" } -variable "hpcc_image_name" { - type = string - description = "REQUIRED. The global image name of the HPCC docker image to deploy.\nMust be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"].\nDefault value: platform-core" - default = "platform-core" - validation { - condition = contains(["platform-core", "platform-ml", "platform-gnn"], var.hpcc_image_name) - error_message = "Value must be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"]." - } -} - -/*variable "hpcc_namespace" { - type = string - description = "REQUIRED. The Kubernetes namespace in which to install the HPCC modules (if enabled).\nDefault value: default" - default = "default" - validation { - condition = var.hpcc_namespace != "" - error_message = "Namespace must be a non-empty string." - } -}*/ -/*variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - existing_namespace = optional(string) - labels = optional(map(string), { name = "hpcc" }) - create_namespace = optional(bool, true) - }) - default = {} -}*/ variable "hpcc_namespace" { description = "Kubernetes namespace where resources will be created." type = object({ diff --git a/lite-variables.tf b/lite-variables.tf index 03280fa..ee36489 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -113,15 +113,6 @@ variable "aks_node_size" { description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } -variable "product_name" { - type = string - description = "REQUIRED. Abbreviated product name, suitable for use in Azure naming.\nMust be 3-16, all lowercase or numeric characters.\nExample entry: myproduct" - validation { - condition = can(regex("^[a-z][a-z0-9]{2,15}$", var.product_name)) - error_message = "Value must be [a-z0-9]{3,16}." - } -} - variable "storage_data_gb" { type = number description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." @@ -168,34 +159,6 @@ variable "authn_htpasswd_filename" { default = "" } -variable "hpcc_image_name" { - type = string - description = "REQUIRED. The global image name of the HPCC docker image to deploy.\nMust be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"].\nDefault value: platform-core" - default = "platform-core" - validation { - condition = contains(["platform-core", "platform-ml", "platform-gnn"], var.hpcc_image_name) - error_message = "Value must be one of [\"platform-core\", \"platform-ml\", \"platform-gnn\"]." - } -} - -/*variable "hpcc_namespace" { - type = string - description = "REQUIRED. The Kubernetes namespace in which to install the HPCC modules (if enabled).\nDefault value: default" - default = "default" - validation { - condition = var.hpcc_namespace != "" - error_message = "Namespace must be a non-empty string." - } -}*/ -/*variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - existing_namespace = optional(string) - labels = optional(map(string), { name = "hpcc" }) - create_namespace = optional(bool, true) - }) - default = {} -}*/ variable "hpcc_namespace" { description = "Kubernetes namespace where resources will be created." type = object({ diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example old mode 100755 new mode 100644 index a5382bc..db909a7 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -1,25 +1,28 @@ #----------------------------------------------------------------------------- +# Name of the A record, of following dns zone, where the ecl watch ip is placed +# This A record will be created and therefore should not exist in the following +# dns zone. +# Example entry: "my-product". This should be something project specific rather +# than something generic. + +a_record_name="" + +#----------------------------------------------------------------------------- + # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" +# REQUIRED -aks_dns_zone_name="" +aks_dns_zone_name="" #----------------------------------------------------------------------------- # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" +# REQUIRED -aks_dns_zone_resource_group_name="" - -#------------------------------------------------------------------------------ - -# Abbreviated product name, suitable for use in Azure naming. -# Must be 3-16 characters in length, all lowercase letters or numbers, no spaces. -# Value type: string -# Example entry: "my-product" - -product_name="tlhhpcc" +aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ @@ -27,7 +30,7 @@ product_name="tlhhpcc" # Only versions in nn.nn.nn format are supported. # Value type: string -hpcc_version="8.10.1" # Currently not used +hpcc_version="8.6.14" #------------------------------------------------------------------------------ @@ -36,7 +39,7 @@ hpcc_version="8.10.1" # Currently not used # Value type: boolean # Example entry: false -enable_roxie=false +enable_roxie=true #------------------------------------------------------------------------------ @@ -45,7 +48,7 @@ enable_roxie=false # Value type: boolean # Example entry: false -enable_code_security=false +enable_code_security=true #------------------------------------------------------------------------------ @@ -91,8 +94,9 @@ storage_data_gb=100 # The 'name' portion must be unique. # To add no tags, use '{}'. # Value type: map of string +# Example: extra_tags={ "owner"="Jane Doe", "owner_email"="jane.doe@gmail.com" } -extra_tags={} # Currently not used +extra_tags={} #------------------------------------------------------------------------------ @@ -117,7 +121,7 @@ aks_max_node_count=4 # Value type: string # Example entry: "jane.doe@hpccsystems.com" -aks_admin_email="jane.doe@hpccsystems.com" +aks_admin_email="jane.doe@gmail.com" #------------------------------------------------------------------------------ @@ -162,7 +166,7 @@ aks_admin_ip_cidr_map={} # To add no CIDR addresses, use '[]'. # Value type: list of string -hpcc_user_ip_cidr_list=[ "20.14.220.189/32", "66.241.32.0/24"] +hpcc_user_ip_cidr_list=[] #------------------------------------------------------------------------------ @@ -183,16 +187,6 @@ storage_account_name="" storage_account_resource_group_name="" -#============================================================================== -# Optional settings -#============================================================================== - -# The global image name of the HPCC docker image to deploy. -# Must be one of ["platform-core", "platform-ml", "platform-gnn"]. -# Default value: "platform-core" - -# hpcc_image_name="platform-core" - #------------------------------------------------------------------------------ # The Kubernetes namespace in which to install the HPCC modules (if enabled). @@ -220,16 +214,9 @@ authn_htpasswd_filename="" #------------------------------------------------------------------------------ -# You can optionally turn logging and monitoring on or off by setting the -# following variable, where true means logging and monitoring will be done -# while false means it won't be done. - -aks_logging_monitoring_enabled=false - -#------------------------------------------------------------------------------ - # If you definitely want ephemeral storage instead of external storage then # set this variable to true otherwise set it to false. -ignore_external_storage=true +external_storage_desired=false +#------------------------------------------------------------------------------ diff --git a/scripts/deploy b/scripts/deploy index 4cf921c..2d1aacc 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,5 +1,5 @@ #!/bin/bash -thisdir=$(dirname $0) +thisdir=`pwd`/`dirname $0` repodir=$(echo $thisdir|sed "s/\/scripts\/*//") if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then name=$1 @@ -85,7 +85,9 @@ fi if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi +#echo "DEBUG: thisdir=\"$thisdir\", repodir=\"$repodir\", name=\"$name\"" plan=`$thisdir/mkplan deploy_${name}.plan` +#echo "DEBUG: plan=\"$plan\"";exit 1 if [ "$diff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi if [ "$name" != "vnet" ];then cp -v /tmp/${name}.lite.auto.tfvars . diff --git a/scripts/mkplan b/scripts/mkplan index dd8cc0e..c54df21 100755 --- a/scripts/mkplan +++ b/scripts/mkplan @@ -32,6 +32,5 @@ $repo_name =~ s/\//-/g; $_ = $tmpl; s//$repo_name/; s//$month_date/; -print STDERR "$_\n"; print "$_\n"; diff --git a/storage/main.tf b/storage/main.tf index 02c221b..6efa56a 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -2,7 +2,8 @@ module "storage" { #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" - source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" + #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" + source = "git@github.com:hpccsystems-solutions-lab/terraform-azurerm-hpcc-storage.git?ref=HPCC-27615-add-rm-0000-cidr" owner = local.owner disable_naming_conventions = var.disable_naming_conventions From 5e3a1ec06da86df2dba1497b8545946293c506bb Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 1 Nov 2023 17:09:15 +0000 Subject: [PATCH 028/124] branch:HPCC-27615-easy-deploy-bryan3-roxiepool-optional --- aks/aks.tf | 2 +- aks/lite-variables.tf | 6 ++ aks/locals.tf | 82 +++++++++++++++++++ ...ks.auto.tfvars.example => node_groups.txt} | 26 +++--- aks/variables.tf | 29 ------- hpcc/hpcc.tf | 2 +- hpcc/lite-locals.tf | 2 +- hpcc/lite-variables.tf | 2 +- hpcc/lite.auto.tfvars.example | 57 +++++++------ lite-variables.tf | 2 +- lite.auto.tfvars.example | 2 +- 11 files changed, 140 insertions(+), 72 deletions(-) rename aks/{aks.auto.tfvars.example => node_groups.txt} (72%) diff --git a/aks/aks.tf b/aks/aks.tf index b162f18..09e6ba4 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -59,7 +59,7 @@ module "aks" { rbac_bindings = var.rbac_bindings availability_zones = var.availability_zones - node_groups = var.node_groups + node_groups = local.node_groups core_services_config = { alertmanager = local.core_services_config.alertmanager diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf index 7183c29..eadfc65 100644 --- a/aks/lite-variables.tf +++ b/aks/lite-variables.tf @@ -1,3 +1,9 @@ +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" diff --git a/aks/locals.tf b/aks/locals.tf index bcf180f..03fe395 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -6,6 +6,88 @@ resource "random_string" "name" { } locals { + roxiepool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" + node_type_version = "v2" + #node_size = "2xlarge" + node_size = "large" + single_group = false + min_capacity = 1 + max_capacity = 3 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "roxiepool" + } + taints = [] + tags = {} + } + + node_groups0 = { + thorpool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" # gp, gpd, mem, memd, stor + node_type_version = "v2" # v1, v2 + #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + single_group = false + min_capacity = 3 + max_capacity = 6 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "thorpool" + } + taints = [] + tags = {} + }, + + servpool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gpd" + node_type_version = "v1" + #node_size = "4xlarge" + node_size = "2xlarge" + single_group = false + min_capacity = 1 + max_capacity = 3 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "servpool" + } + taints = [] + tags = {} + }, + + spraypool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" + node_type_version = "v1" + node_size = "2xlarge" + #node_size = "1xlarge" # NOT ALLOWED + #node_size = "4xlarge" + single_group = false + min_capacity = 3 + max_capacity = 6 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "spraypool" + "spray-service" = "spraypool" + } + taints = [] + tags = {} + } + } + + node_groups = var.aks_enable_roxie? merge( local.node_groups0, { roxiepool = local.roxiepool } ) : local.node_groups0 + aks_automation = { local_authentication_enabled = false public_network_access_enabled = false diff --git a/aks/aks.auto.tfvars.example b/aks/node_groups.txt similarity index 72% rename from aks/aks.auto.tfvars.example rename to aks/node_groups.txt index cce1afe..b3623c6 100644 --- a/aks/aks.auto.tfvars.example +++ b/aks/node_groups.txt @@ -1,12 +1,13 @@ cluster_version = "1.26" cluster_ordinal = 1 //cluster name suffix -sku_tier = "free" +sku_tier = "FREE" hpcc_log_analytics_enabled = false rbac_bindings = { cluster_admin_users = { - # "admin" = "", - + # "service_principal1" = "", + # "user1" = "" + "admin" = "35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9" } cluster_view_users = {} @@ -23,10 +24,11 @@ node_groups = { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = 3 + max_capacity = 6 # placement_group_key = null labels = { "lnrs.io/tier" = "standard" @@ -41,7 +43,8 @@ node_groups = { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - node_size = "2xlarge" + #node_size = "2xlarge" + node_size = "large" single_group = false min_capacity = 1 max_capacity = 3 @@ -59,7 +62,8 @@ node_groups = { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - node_size = "4xlarge" + #node_size = "4xlarge" + node_size = "2xlarge" single_group = false min_capacity = 1 max_capacity = 3 @@ -78,9 +82,11 @@ node_groups = { node_type = "gp" node_type_version = "v1" node_size = "2xlarge" + #node_size = "1xlarge" # NOT ALLOWED + #node_size = "4xlarge" single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = 3 + max_capacity = 6 # placement_group_key = null labels = { "lnrs.io/tier" = "standard" diff --git a/aks/variables.tf b/aks/variables.tf index 692d97d..472477b 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -99,35 +99,6 @@ variable "rbac_bindings" { default = {} } -variable "node_groups" { - description = "Node groups to configure." - type = map(object({ - node_arch = optional(string) - node_os = optional(string) - node_type = optional(string) - node_type_variant = optional(string) - node_type_version = optional(string) - node_size = string - single_group = optional(bool) - min_capacity = optional(number) - max_capacity = number - os_config = optional(map(any)) - ultra_ssd = optional(bool) - placement_group_key = optional(string) - max_pods = optional(number) - max_surge = optional(string) - labels = optional(map(string)) - taints = optional(list(object({ - key = string - value = string - effect = string - }))) - tags = optional(map(string)) - })) - nullable = false - default = {} -} - variable "experimental" { description = "Configure experimental features." type = object({ diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index ab8932f..5ed2592 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -92,7 +92,7 @@ module "hpcc" { external_storage_config = local.external_storage_config spill_volumes = local.spill_volumes - enable_roxie = var.enable_roxie + enable_roxie = var.aks_enable_roxie roxie_config = local.roxie_config thor_config = local.thor_config vault_config = local.vault_config diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 3bae99f..22b1ae0 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -196,7 +196,7 @@ locals { roxie_config = [ { - disabled = (var.enable_roxie == true)? false : true + disabled = (var.aks_enable_roxie == true)? false : true name = "roxie" nodeSelector = {} numChannels = 2 diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index ee36489..a0db654 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -55,7 +55,7 @@ variable "enable_code_security" { type = bool } -variable "enable_roxie" { +variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool } diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example index dfd2f71..8373caa 100755 --- a/hpcc/lite.auto.tfvars.example +++ b/hpcc/lite.auto.tfvars.example @@ -1,25 +1,28 @@ #----------------------------------------------------------------------------- +# Name of the A record, of following dns zone, where the ecl watch ip is placed +# This A record will be created and therefore should not exist in the following +# dns zone. +# Example entry: "my-product". This should be something project specific rather +# than something generic. + +a_record_name="" + +#----------------------------------------------------------------------------- + # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" +# REQUIRED -aks_dns_zone_name="" +aks_dns_zone_name="" #----------------------------------------------------------------------------- # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" +# REQUIRED -aks_dns_zone_resource_group_name="" - -#------------------------------------------------------------------------------ - -# Abbreviated product name, suitable for use in Azure naming. -# Must be 3-16 characters in length, all lowercase letters or numbers, no spaces. -# Value type: string -# Example entry: "my-product" - -product_name="tlhhpcc" +aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ @@ -27,7 +30,7 @@ product_name="tlhhpcc" # Only versions in nn.nn.nn format are supported. # Value type: string -hpcc_version="8.10.1" # Currently not used +hpcc_version="8.6.14" #------------------------------------------------------------------------------ @@ -36,7 +39,7 @@ hpcc_version="8.10.1" # Currently not used # Value type: boolean # Example entry: false -enable_roxie=false +aks_enable_roxie=true #------------------------------------------------------------------------------ @@ -45,7 +48,7 @@ enable_roxie=false # Value type: boolean # Example entry: false -enable_code_security=false +enable_code_security=true #------------------------------------------------------------------------------ @@ -91,8 +94,9 @@ storage_data_gb=100 # The 'name' portion must be unique. # To add no tags, use '{}'. # Value type: map of string +# Example: extra_tags={ "owner"="Jane Doe", "owner_email"="jane.doe@gmail.com" } -extra_tags={} # Currently not used +extra_tags={} #------------------------------------------------------------------------------ @@ -117,7 +121,7 @@ aks_max_node_count=4 # Value type: string # Example entry: "jane.doe@hpccsystems.com" -aks_admin_email="jane.doe@hpccsystems.com" +aks_admin_email="jane.doe@gmail.com" #------------------------------------------------------------------------------ @@ -162,7 +166,7 @@ aks_admin_ip_cidr_map={} # To add no CIDR addresses, use '[]'. # Value type: list of string -hpcc_user_ip_cidr_list=[ "20.14.220.189/32", "66.241.32.0/24"] +hpcc_user_ip_cidr_list=[] #------------------------------------------------------------------------------ @@ -183,16 +187,6 @@ storage_account_name="" storage_account_resource_group_name="" -#============================================================================== -# Optional settings -#============================================================================== - -# The global image name of the HPCC docker image to deploy. -# Must be one of ["platform-core", "platform-ml", "platform-gnn"]. -# Default value: "platform-core" - -# hpcc_image_name="platform-core" - #------------------------------------------------------------------------------ # The Kubernetes namespace in which to install the HPCC modules (if enabled). @@ -217,3 +211,12 @@ enable_premium_storage=false # Example entry: "htpasswd.txt" authn_htpasswd_filename="" + +#------------------------------------------------------------------------------ + +# If you definitely want ephemeral storage instead of external storage then +# set this variable to true otherwise set it to false. + +external_storage_desired=false + +#------------------------------------------------------------------------------ diff --git a/lite-variables.tf b/lite-variables.tf index ee36489..a0db654 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -55,7 +55,7 @@ variable "enable_code_security" { type = bool } -variable "enable_roxie" { +variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool } diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index db909a7..8373caa 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -39,7 +39,7 @@ hpcc_version="8.6.14" # Value type: boolean # Example entry: false -enable_roxie=true +aks_enable_roxie=true #------------------------------------------------------------------------------ From 7065e89e066c5ce81f00e0e32c7d315273467ce7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 2 Nov 2023 13:31:42 +0000 Subject: [PATCH 029/124] branch:HPCC-27615-easy-deploy-bryan4-placing-auto.tfvars-files-aks-storage --- aks/lite-variables.tf | 9 +++-- aks/misc.auto.tfvars.example | 68 ------------------------------------ aks/variables.tf | 7 ---- hpcc/lite-variables.tf | 12 +++++++ lite-variables.tf | 25 ++++++++++--- lite.auto.tfvars.example | 13 +++++++ main.tf | 5 ++- scripts/deploy | 14 +++++++- storage/lite-variables.tf | 11 ++++++ vnet/lite-variables.tf | 11 ++++++ 10 files changed, 90 insertions(+), 85 deletions(-) delete mode 100644 aks/misc.auto.tfvars.example diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf index eadfc65..7a098d1 100644 --- a/aks/lite-variables.tf +++ b/aks/lite-variables.tf @@ -1,5 +1,5 @@ -variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool default = false } @@ -27,6 +27,11 @@ variable "aks_azure_region" { } } +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool +} + variable "aks_dns_zone_resource_group_name" { type = string description = "OPTIONAL: Name of the resource group containing the dns zone." diff --git a/aks/misc.auto.tfvars.example b/aks/misc.auto.tfvars.example deleted file mode 100644 index c1040d3..0000000 --- a/aks/misc.auto.tfvars.example +++ /dev/null @@ -1,68 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -resource_groups = { - azure_kubernetes_service = { - tags = { "enclosed resource" = "open source aks" } - } - # azure_log_analytics_workspace = { - # tags = { "enclosed resource" = "azure log analytics workspace" } - # } -} - -# # auto_connect - Automatically connect to the kubernetes cluster from the host machine. -auto_connect = true - -# # disable_naming_conventions - Disable naming conventions -# # disable_naming_conventions = true - -# azure_auth = { -# # AAD_CLIENT_ID = "" -# # AAD_CLIENT_SECRET = "" -# # AAD_TENANT_ID = "" -# # AAD_PRINCIPAL_ID = "" -# SUBSCRIPTION_ID = "" -# } - -aks_automation = { - local_authentication_enabled = false - public_network_access_enabled = false - automation_account_name = "aks-stop-demo-5" - - schedule = [ - { - schedule_name = "aks_stop" - description = "Stops the AKS weekday nights at 6PM EST" - frequency = "Week" //OneTime, Day, Hour, Week, or Month. - interval = "1" //cannot be set when frequency is `OneTime` - daylight_saving = true - start_time = "18:00" // At least 5 minutes in the future - week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - }, - # { - # schedule_name = "aks_start" - # description = "Starts the AKS weekday nights at 6AM EST" - # frequency = "Week" //OneTime, Day, Hour, Week, or Month. - # interval = "1" //cannot be set when frequency is `OneTime` - # daylight_saving = true - # start_time = "06:00" // At least 5 minutes in the future - # week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - # } - ] -} \ No newline at end of file diff --git a/aks/variables.tf b/aks/variables.tf index 472477b..a44d7fe 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -171,13 +171,6 @@ variable "cluster_endpoint_access_cidrs" { } } -variable "aks_logging_monitoring_enabled" { - description = "If true then logging and monitoring will occur else it will not." - type = bool - nullable = false - default = false -} - variable "logging" { description = "Logging configuration." type = object({ diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index a0db654..9bc8750 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -2,11 +2,23 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + variable "external_storage_desired" { description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" type = bool default = false } + variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." type = bool diff --git a/lite-variables.tf b/lite-variables.tf index a0db654..ccc45bb 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -2,14 +2,27 @@ # Prompted variables (user will be asked to supply them at plan/apply time # if a .tfvars file is not supplied); there are no default values ############################################################################### +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + variable "external_storage_desired" { description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" type = bool default = false } + variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." type = bool + default = true } variable "a_record_name" { @@ -53,11 +66,13 @@ variable "aks_azure_region" { variable "enable_code_security" { description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" type = bool + default = false } variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool + default = false } variable "extra_tags" { @@ -68,14 +83,12 @@ variable "extra_tags" { variable "aks_dns_zone_resource_group_name" { type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" + description = "REQUIRED. Name of the resource group containing the dns zone." } variable "aks_dns_zone_name" { type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" + description = "REQUIRED. dns zone name. The name of existing dns zone." } variable "hpcc_user_ip_cidr_list" { @@ -120,6 +133,7 @@ variable "storage_data_gb" { condition = var.storage_data_gb >= 10 error_message = "Value must be 10 or more." } + default = 100 } variable "storage_lz_gb" { @@ -129,6 +143,7 @@ variable "storage_lz_gb" { condition = var.storage_lz_gb >= 1 error_message = "Value must be 1 or more." } + default = 25 } variable "thor_max_jobs" { @@ -138,6 +153,7 @@ variable "thor_max_jobs" { condition = var.thor_max_jobs >= 1 error_message = "Value must be 1 or more." } + default = 2 } variable "thor_num_workers" { @@ -147,6 +163,7 @@ variable "thor_num_workers" { condition = var.thor_num_workers >= 1 error_message = "Value must be 1 or more." } + default = 2 } ############################################################################### diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 8373caa..615bf5b 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -220,3 +220,16 @@ authn_htpasswd_filename="" external_storage_desired=false #------------------------------------------------------------------------------ + +# This variable enable you to ask for logging and monitoring of the kubernetes +# and hpcc cluster (true means enable logging and monitoring, false means don't. + +aks_logging_monitoring_enabled=false + +#------------------------------------------------------------------------------ + +# Put your azure account id here. It will look like the following: +# 6c5edc79-34fd-333a-9b59-61ec21d7e42d + +my_azure_id="" + diff --git a/main.tf b/main.tf index 9e30bc3..3bc29e3 100644 --- a/main.tf +++ b/main.tf @@ -8,7 +8,7 @@ resource "null_resource" "deploy_vnet" { resource "null_resource" "deploy_aks" { provisioner "local-exec" { - command = "scripts/deploy aks" + command = "scripts/deploy aks ${var.my_azure_id}" } depends_on = [ null_resource.deploy_vnet ] @@ -21,8 +21,7 @@ resource "null_resource" "deploy_storage" { command = "scripts/deploy storage" } - #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] - depends_on = [ null_resource.deploy_vnet ] + depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] } resource "null_resource" "external_storage" { diff --git a/scripts/deploy b/scripts/deploy index 2d1aacc..57f1228 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -6,6 +6,15 @@ if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then else echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'storage' or 'aks' or 'hpcc'. EXITING.";exit 1; fi +if [ "$1" == "aks" ];then + if [ "$2" != "" ];then + my_azure_id=$2 + cp -v $thisdir/needed-auto.tfvars-files/aks/*.auto.tfvars $repodir/aks + sed -i "s//$my_azure_id/" $repodir/aks/aks.auto.tfvars + else + echo "In $0. First argument was \"$name\". There should be a 2nd argument (my_azure_id). But it was missing. EXITING.";exit 1; + fi +fi #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>> EXECUTING: $*" @@ -59,7 +68,7 @@ done cd $name; # cd into vnet or storage or aks or hpcc -# put the root directory's lite.auto.tfvars (either all of part) in either aks or hpcc +# put the root directory's lite.auto.tfvars (either all or part) in either aks or hpcc # directory. if [ -e "../lite.auto.tfvars" ];then # Check if there has been a change since last apply. @@ -80,6 +89,9 @@ elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];t egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf + if [ "$name" == "storage" ];then + cp -v $thisdir/needed-auto.tfvars-files/storage/*.auto.tfvars . + fi fi #------------------------------------------------------------------------ if [ ! -d "$HOME/tflogs" ];then diff --git a/storage/lite-variables.tf b/storage/lite-variables.tf index 7183c29..7a098d1 100644 --- a/storage/lite-variables.tf +++ b/storage/lite-variables.tf @@ -1,3 +1,9 @@ +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" @@ -21,6 +27,11 @@ variable "aks_azure_region" { } } +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool +} + variable "aks_dns_zone_resource_group_name" { type = string description = "OPTIONAL: Name of the resource group containing the dns zone." diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf index 7183c29..7a098d1 100644 --- a/vnet/lite-variables.tf +++ b/vnet/lite-variables.tf @@ -1,3 +1,9 @@ +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" @@ -21,6 +27,11 @@ variable "aks_azure_region" { } } +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool +} + variable "aks_dns_zone_resource_group_name" { type = string description = "OPTIONAL: Name of the resource group containing the dns zone." From ad678e4ea4fce34c7b357926eb370aa3cdd1800f Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 2 Nov 2023 15:12:33 +0000 Subject: [PATCH 030/124] branch:HPCC-27615-easy-deploy-bryan4-placing-auto.tfvars-files-aks-storage --- .gitignore | 3 +- scripts/deploy | 5 +- .../aks/aks.auto.tfvars.example | 19 +++ .../aks/misc.auto.tfvars.example | 22 +++ .../storage/storage.auto.tfvars.example | 152 ++++++++++++++++++ 5 files changed, 198 insertions(+), 3 deletions(-) create mode 100644 scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example create mode 100644 scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example create mode 100644 scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example diff --git a/.gitignore b/.gitignore index 9f2e093..3d735d3 100644 --- a/.gitignore +++ b/.gitignore @@ -13,7 +13,8 @@ # version control. # # example.tfvars -**/*.tfvars +*.tfvars +*/*.tfvars **/*.json # Ignore data files diff --git a/scripts/deploy b/scripts/deploy index 57f1228..a663f42 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -9,7 +9,8 @@ fi if [ "$1" == "aks" ];then if [ "$2" != "" ];then my_azure_id=$2 - cp -v $thisdir/needed-auto.tfvars-files/aks/*.auto.tfvars $repodir/aks + cp -v $thisdir/needed-auto-tfvars-files/aks/aks.auto.tfvars.example $repodir/aks/aks.auto.tfvars + cp -v $thisdir/needed-auto-tfvars-files/aks/misc.auto.tfvars.example $repodir/aks/misc.auto.tfvars sed -i "s//$my_azure_id/" $repodir/aks/aks.auto.tfvars else echo "In $0. First argument was \"$name\". There should be a 2nd argument (my_azure_id). But it was missing. EXITING.";exit 1; @@ -90,7 +91,7 @@ elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];t egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf if [ "$name" == "storage" ];then - cp -v $thisdir/needed-auto.tfvars-files/storage/*.auto.tfvars . + cp -v $thisdir/needed-auto-tfvars-files/storage/storage.auto.tfvars.example ./storage.auto.tfvars fi fi #------------------------------------------------------------------------ diff --git a/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example new file mode 100644 index 0000000..685ebd7 --- /dev/null +++ b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example @@ -0,0 +1,19 @@ +cluster_version = "1.26" # latest version +cluster_ordinal = 1 //cluster name suffix +sku_tier = "FREE" +hpcc_log_analytics_enabled = false + +rbac_bindings = { + cluster_admin_users = { + # "service_principal1" = "", + # "user1" = "" + "admin" = "" + } + + cluster_view_users = {} + cluster_view_groups = [] +} + +cluster_endpoint_access_cidrs = ["0.0.0.0/0"] + +availability_zones = [1] diff --git a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example new file mode 100644 index 0000000..ed01f69 --- /dev/null +++ b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example @@ -0,0 +1,22 @@ +resource_groups = { # rg + azure_kubernetes_service = { + tags = { "enclosed resource" = "open source aks" } + } + # azure_log_analytics_workspace = { + # tags = { "enclosed resource" = "azure log analytics workspace" } + # } +} + +# # auto_connect - Automatically connect to the kubernetes cluster from the host machine. +auto_connect = true + +# # disable_naming_conventions - Disable naming conventions +# # disable_naming_conventions = true + +# azure_auth = { +# # AAD_CLIENT_ID = "" +# # AAD_CLIENT_SECRET = "" +# # AAD_TENANT_ID = "" +# # AAD_PRINCIPAL_ID = "" +# SUBSCRIPTION_ID = "" +# } diff --git a/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example new file mode 100644 index 0000000..38e6a1b --- /dev/null +++ b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example @@ -0,0 +1,152 @@ +storage_accounts = { # storage account + adminsvc1 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "adminsvc1" + storage_type = "azurefiles" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + subnet_ids = {} + file_share_retention_days = 7 + access_tier = "Hot" + account_kind = "FileStorage" + account_tier = "Premium" + + planes = { + dali = { + category = "dali" + name = "dali" + sub_path = "dalistorage" + size = 100 + sku = "" + rwmany = true + protocol = "nfs" + } + } + } + + adminsvc2 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "adminsvc2" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + account_tier = "Standard" + + planes = { + dll = { + category = "dll" + name = "dll" + sub_path = "queries" + size = 100 + sku = "" + rwmany = true + } + + lz = { + category = "lz" + name = "mydropzone" + sub_path = "dropzone" + size = 100 + sku = "" + rwmany = true + } + + sasha = { + category = "sasha" + name = "sasha" + sub_path = "sashastorage" + size = 100 + sku = "" + rwmany = true + } + + debug = { + category = "debug" + name = "debug" + sub_path = "debug" + size = 100 + sku = "" + rwmany = true + } + } + } + + data1 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "data1" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + #replication_type = "GRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + #account_kind = "BlobStorage" + account_tier = "Standard" + + planes = { + data = { + category = "data" + name = "data" + sub_path = "hpcc-data" + size = 100 + sku = "" + rwmany = true + } + } + } + + data2 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "data2" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + #replication_type = "LRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + #account_kind = "BlobStorage" + account_tier = "Standard" + + planes = { + data = { + category = "data" + name = "data" + sub_path = "hpcc-data" + size = 100 + sku = "" + rwmany = true + } + } + } +} From aca0ced83f1be8e0bfd3537f38d057b75bc9d729 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 2 Nov 2023 19:21:34 +0000 Subject: [PATCH 031/124] branch:HPCC-27615-easy-deploy-bryan4-placing-auto.tfvars-files-aks-storage. Corrected comments about 'external_storage_desired' variable. --- lite-variables.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-variables.tf b/lite-variables.tf index ccc45bb..81db473 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -14,7 +14,7 @@ variable "aks_logging_monitoring_enabled" { } variable "external_storage_desired" { - description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" type = bool default = false } From 1c346ee99f6e653c387534d3602ab1ad915a01eb Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 2 Nov 2023 19:26:40 +0000 Subject: [PATCH 032/124] branch:HPCC-27615-easy-deploy-bryan4-placing-auto.tfvars-files-aks-storage. Fixed comments about 'external_storage_desired' variables. --- lite.auto.tfvars.example | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 615bf5b..2f56d86 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -214,7 +214,7 @@ authn_htpasswd_filename="" #------------------------------------------------------------------------------ -# If you definitely want ephemeral storage instead of external storage then +# If you want external storage instead of ephemeral storage then # set this variable to true otherwise set it to false. external_storage_desired=false From e7963550aa6a29f4ce5d00422c96bf1de7189100 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sat, 4 Nov 2023 17:31:13 +0000 Subject: [PATCH 033/124] branch:HPCC-27615-easy-deploy-bryan5-miscellaneous-changes --- README.md | 855 ++++++----------------------------------- aks/lite-variables.tf | 7 +- aks/outputs.tf | 2 + hpcc/.outputs.tf.swp | Bin 0 -> 12288 bytes hpcc/lite-variables.tf | 15 +- hpcc/main.tf | 7 - hpcc/outputs.tf | 19 +- storage/outputs.tf | 2 + vnet/lite-variables.tf | 7 +- vnet/outputs.tf | 2 + 10 files changed, 158 insertions(+), 758 deletions(-) create mode 100644 hpcc/.outputs.tf.swp diff --git a/README.md b/README.md index 67ef75a..6431c14 100644 --- a/README.md +++ b/README.md @@ -1,732 +1,123 @@ -# Azure - HPCC AKS Root Module -
- -This module is intended as an example for development and test systems only. It can be used as a blueprint to develop your own production version that meets your organization's security requirements. -
-
- -## Introduction - -This module deploys an HPCC AKS cluster using remote modules that are listed below. -
- -## Remote Modules -These are the list of all the remote modules. - -| Name | Description | URL | Required | -| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------- | :------: | -| subscription | Queries enabled azure subscription from host machine | https://github.com/Azure-Terraform/terraform-azurerm-subscription-data.git | yes | -| naming | Enforces naming conventions | - | yes | -| metadata | Provides metadata | https://github.com/Azure-Terraform/terraform-azurerm-metadata.git | yes | -| resource_group | Creates a resource group | https://github.com/Azure-Terraform/terraform-azurerm-resource-group.git | yes | -| virtual_network | Creates a virtual network | https://github.com/Azure-Terraform/terraform-azurerm-virtual-network.git | yes | -| kubernetes | Creates an Azure Kubernetes Service Cluster | https://github.com/Azure-Terraform/terraform-azurerm-kubernetes.git | yes | -
- -## Supported Arguments -
- -### The `admin` block: -This block contains information on the user who is deploying the cluster. This is used as tags and part of some resource names to identify who deployed a given resource and how to contact that user. This block is required. - -| Name | Description | Type | Default | Required | -| ----- | ---------------------------- | ------ | ------- | :------: | -| name | Name of the admin. | string | - | yes | -| email | Email address for the admin. | string | - | yes | - -
-Usage Example: -
- - admin = { - name = "Example" - email = "example@hpccdemo.com" - } -
- -### The `disable_naming_conventions` block: -When set to `true`, this attribute drops the naming conventions set forth by the python module. This attribute is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------- | ---- | ------- | :------: | - | disable_naming_conventions | Disable naming conventions. | bool | `false` | no | -
- -### The `metadata` block: -TThe arguments in this block are used as tags and part of resources’ names. This block can be omitted when disable_naming_conventions is set to `true`. - - | Name | Description | Type | Default | Required | - | ------------------- | ---------------------------- | ------ | ------- | :------: | - | project_name | Name of the project. | string | "" | yes | - | product_name | Name of the product. | string | hpcc | no | - | business_unit | Name of your bussiness unit. | string | "" | no | - | environment | Name of the environment. | string | "" | no | - | market | Name of market. | string | "" | no | - | product_group | Name of product group. | string | "" | no | - | resource_group_type | Resource group type. | string | "" | no | - | sre_team | Name of SRE team. | string | "" | no | - | subscription_type | Subscription type. | string | "" | no | -
- -Usage Example: -
- - metadata = { - project = "hpccdemo" - product_name = "example" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "contoso" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - } - -
- -### The `tags` argument: -The tag attribute can be used for additional tags. The tags must be key value pairs. This block is optional. - - | Name | Description | Type | Default | Required | - | ---- | ------------------------- | ----------- | ------- | :------: | - | tags | Additional resource tags. | map(string) | admin | no | -
- -### The `resource_group` block: -This block creates a resource group (like a folder) for your resources. This block is required. - - | Name | Description | Type | Default | Required | - | ----------- | ----------------------------------------------------------------- | ---- | ------- | :------: | - | unique_name | Will concatenate a number at the end of your resource group name. | bool | `true` | yes | -
- -Usage Example: -
- - resource_group = { - unique_name = true - } - -
- -### The `virtual_network` block: -This block imports metadata of a virtual network deployed outside of this project. This block is optional. - - | Name | Description | Type | Default | Required | - | ----------------- | --------------------------------------- | ------ | ------- | :------: | - | private_subnet_id | The ID of the private subnet. | string | - | yes | - | public_subnet_id | The ID of the public subnet. | string | - | yes | - | route_table_id | The ID of the route table for the AKS. | string | - | yes | - | location | The location of the virtual network | string | - | yes | -
- -Usage Example: -
- - virtual_network = { - private_subnet_id = "" - public_subnet_id = "" - route_table_id = "" - location = "" - } - -
- -## The `node_pools` block: -The `node-pools` block supports the following arguments:
-`system` - (Required) The system or default node pool. This node pool hosts the system pods by default. The possible arguments for this block are defined below. - -`addpool` - (Required) The additional node pool configuration. This block name is changeable and must be unique across all additional node pools. At least one additional node pool is required. The possible arguments for this block are defined below. - -### The `system` block: -This block creates a system node pool. This block is required. - -| Name | Optional, Required | Description | -| --------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -### The `addpool` block: -This block creates additional node pools. This block is optional. - -| Name | Optional, Required | Description | -| ---------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| node_taints | Optional | A list of Kubernetes taints which should be applied to nodes in the agent pool (e.g key=value:NoSchedule). Changing this forces a new resource to be created. | -| max_surge | Required | The maximum number or percentage of nodes which will be added to the Node Pool size during an upgrade. | -| eviction_policy | Optional | The Eviction Policy which should be used for Virtual Machines within the Virtual Machine Scale Set powering this Node Pool. Possible values are Deallocate and Delete. Will only be used when priority is set to Spot. Changing this forces a new resource to be created. | -| os_type | Optional | The Operating System which should be used for this Node Pool. Changing this forces a new resource to be created. Possible values are Linux and Windows. Defaults to Linux. | -| priority | Optional | The Priority for Virtual Machines within the Virtual Machine Scale Set that powers this Node Pool. Possible values are Regular and Spot. Defaults to Regular. Changing this forces a new resource to be created. | -| proximity_placement_group_id | Optional | The ID of the Proximity Placement Group where the Virtual Machine Scale Set that powers this Node Pool will be placed. Changing this forces a new resource to be created. | -| spot_max_price | Optional | The maximum price you're willing to pay in USD per Virtual Machine. Valid values are -1 (the current on-demand price for a Virtual Machine) or a positive value with up to five decimal places. Changing this forces a new resource to be created. | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -Usage Example: -
- - node_pools = { - system = { - vm_size = "Standard_D4_v4" - node_count = 1 - enable_auto_scaling = true - only_critical_addons_enabled = true - min_count = 1 - max_count = 1 - availability_zones = [] - subnet = "private" - enable_host_encryption = false - enable_node_public_ip = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # max_pods = 10 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - - } - - addpool1 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - - addpool2 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - } -
- -### The `disable_helm` argument: -This block disable helm deployments by Terraform. This block is optional and will stop HPCC from being installed. - - | Name | Description | Type | Default | Required | - | ------------ | -------------------------------------- | ---- | ------- | :------: | - | disable_helm | Disable Helm deployments by Terraform. | bool | `false` | no | -
- -### The `hpcc` block: -This block deploys the HPCC helm chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------------------------------ | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: ~/HPCC-Platform/helm/hpcc Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | string | null | no | - | namespace | Namespace to use. | string | default | no | - | name | Release name of the chart. | string | myhpcck8s | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | values-retained-azurefile.yaml | no | - | version | Version of the HPCC chart. | string | latest | yes | - | image_root | Image root to use. | string | hpccsystems | no | - | image_name | Image name to use. | string | platform-core | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose_eclwatch | Expose ECLWatch to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- - Usage Example: -
- - hpcc = { - expose_eclwatch = true - name = "myhpcck8s" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 900 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/hpcc" #Other examples: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz - # version = "8.6.14-rc2" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - # image_root = "west.lexisnexisrisk.com" - # image_name = "platform-core-ln" - # image_version = "8.6.18-rc1" - } - -
- -### The `storage` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------- | ---------------- | :---------: | - | default | Use AKS provided storage accounts? | bool | `false` | `true` , `false` | no | - | version | The version of the storage chart. | string | 0.1.0 | | no | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-azurefile-0.1.0.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | name | Release name of the chart. | string | `myhpcck8s` | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | [] | no | - | storage_accounts | The storage account to use. | object | Queries attributes' values from storage_accounts module | - | no | - | version | Version of the storage chart. | string | 0.1.0 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 600 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | -
- -#### The `storage_accounts` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | ------------------- | -------------------------------------------------------------------- | ------------ | --------------------------- | ------------- | :------: | - | name | Name of the storage account | string | - | - | yes | - | resource_group_name | The name of the resource group in which the storage account belongs. | string | - | - | yes | - | subscription_id | The ID of the subscription in which the storage account belongs. | string | Admin's active subscription | - | no | - | shares | The list of shares in the storage account | list(object) | - | - | yes | - | | -
- -#### The `shares` block: -This block defines the list of shares in the storage account. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------- | ------------------------------------- | ------ | ------- | ------------- | :------: | - | name | The name of the share. | string | - | - | yes | - | sub_path | The sub path for the HPCC data plane. | string | - | - | yes | - | category | The category for the HPCC data plane | string | - | - | yes | - | sku | The sku for the HPCC data plane. | string | - | - | yes | - | quota | The size of the share in Gigabytes | number | - | - | yes | - -Usage Example: -
- - storage = { - default = false - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 600 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile" - # version = "0.1.0" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - - /* - storage_accounts = { - # do not change the key names - dali = { - name = "dalikxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - dali = { - name = "dalishare" - sub_path = "dalistorage" //do not change this value - category = "dali" //do not change this value - sku = "Premium_LRS" - quota = 100 - } - } - } - - sasha = { - name = "sashakxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - sasha = { - name = "sashashare" - sub_path = "sasha" //do not change this value - category = "sasha" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - - common = { - name = "commonkxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - data = { - name = "datashare" - sub_path = "hpcc-data" //do not change this value - category = "data" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - dll = { - name = "dllshare" - sub_path = "queries" //do not change this value - category = "dll" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - mydropzone = { - name = "mydropzoneshare" - sub_path = "dropzone" //do not change this value - category = "lz" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - } - */ - } -
- -### The `elastic4hpcclogs` block: -This block deploys the elastic4hpcclogs chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ----------------------- | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/managed/logging/elastic Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/elastic4hpcclogs-1.2.10.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | enable | Enable elastic4hpcclogs | bool | `true` | no | - | name | Release name of the chart. | string | myelastic4hpcclogs | no | - | version | The version of the elastic4hpcclogs | string | 1.2.8 | | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | - | no | - | version | Version of the elastic4hpcclogs chart. | string | 1.2.1 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose | Expose myelastic4hpcclogs-kibana service to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- -Usage Example: -
- - elastic4hpcclogs = { - enable = true - expose = true - name = "myelastic4hpcclogs" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 300 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - #local_chart = "/Users/godji/work/demo/helm-chart/helm/managed/logging/elastic" - # version = "1.2.10" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - } -
- -### The `registry` block: -This block authenticates a private Docker repository. This block is optional. - - | Name | Description | Type | Default | Required | - | -------- | -------------------------------------------------------------------------- | ------ | ------- | :------: | - | server | The server address of the private Docker repository. | string | - | yes | - | username | The username for the private Docker repository account. | string | - | yes | - | password | The password, token, or API key for the private Docker repository account. | string | - | yes | -
- -Usage Example: -
- - registry = { - password = "" - server = "" - username = "" - } -
- -### The `auto_connect` argument: -This block automatically connect your cluster to your local machine similarly to `az aks get-credentials`. - - | Name | Description | Type | Default | Required | - | ------------ | --------------------------------------------------------------------------------------------------------- | ---- | ------- | :------: | - | auto_connect | Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context. | bool | `false` | no | -
- -## Outputs - -| Name | Description | -| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| aks_login | Get access credentials for the managed Kubernetes cluster. | -| recommendations | A list of security and cost recommendations for this deployment. Your environment has to have been deployed for several hours before Azure provides recommendations. | -
- -## Usage -### Deploy the Virtual Network Module -
    -
  1. - -Clone this repo: `git clone https://github.com/gfortil/terraform-azurerm-hpcc.git`.
    2. - -
  2. Linux and MacOS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `cp examples/admin.tfvars .`
      3. -
    -
  3. Windows OS
    4. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `copy examples\admin.tfvars .`
      3. -
    -
  4. - -Open `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    5. -
  5. - -Set attributes to your preferred values.
    6. -
  6. - -Save `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    7. -
  7. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    8. -
  8. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    9. -
  9. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    10. -
- -### Deploy the Storage Account Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
- -### Deploy the AKS Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
  9. - -If `auto_connect = true` (in admin.tfvars), skip this step.
    10. -
      -
    1. - -Copy aks_login command.
      2. -
    2. - -Run aks_login in your command line.
      3. -
    3. - -Accept to overwrite your current context.
      4. -
    -
  10. - -List pods: `kubectl get pods`.
    11. -
  11. - -Get ECLWatch external IP: `kubectl get svc --field-selector metadata.name=eclwatch | awk 'NR==2 {print $4}'`.
    12. -
  12. - -Delete cluster: `terraform destroy -var-file=admin.tfvars` or `terraform destroy -var-file=admin.tfvars -auto-approve`.
    13. -
  13. - -Type: `yes` if flag `-auto-approve` was not set.
    14. -
+# Deploy HPCC Systems on Azure under Kubernetes + +This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure. The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. + +The HPCC Systems cluster this module creates uses ephemeral storage (meaning, the storage will be deleted if the cluster is deleted) or you can ask for Persistent Storage. See the section titled [Persistent Storage](#persistent_storage), below. + +This repo is a fork of the excellent work performed by Godson Fortil. The original can be found at [https://github.com/gfortil/terraform-azurerm-hpcc/tree/HPCC-27615]. + +## Requirements + +* **Terraform** This is a Terraform module, so you need to have Terraform installed on your system. Instructions for downloading and installing Terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of Terraform, as that is needed to accommodate some of the large random numbers used for IDs in the Terraform modules. + +* **helm** Helm is used to deploy the HPCC Systems processes under Kubernetes. Instructions for downloading and installing Helm are at [https://helm.sh/docs/intro/install](https://helm.sh/docs/intro/install/). + +* **kubectl** The Kubernetes client (kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. + +* **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within kubernetes clusters. TL;DR: Make sure you have the command line tools installed. + +* This module will create an AKS cluster in your current **default** Azure subscription. You can view your current subscriptions, and determine which is the default, using the `az account list --output table` command. To set a default subscription, use `az account set --subscription "My_Subscription"`. + +* To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. + +## Installing/Using This Module + +1. If necessary, login to Azure. + * From the command line, this is usually accomplished with the `az login` command. +1. Clone this repo to your local system and change current directory. + * `git clone -b HPCC-27615-easy-deploy https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` + * `cd terraform-azurerm-hpcc-lite` +1. Issue `terraform init` to initialize the Terraform modules. +1. Decide how you want to supply option values to the module during invocation. There are three possibilities: + 1. Invoke the `terraform apply` command and enter values for each option as Terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. + 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to do that is to copy the sample file and then edit the copy: + * `cp lite.auto.tfvars.example lite.auto.tfvars` + 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. +1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). + +At the end of a successful deployment several items are shown: +* The URL used to access ECL Watch. +* The deployment azure resource group. + +## Available Options + +Options have data types. The ones used in this module are: +* string + * Typical string enclosed by quotes + * Example + * `"value"` +* number + * Integer number; do not quote + * Example + * `1234` +* boolean + * true or false (not quoted) +* map of string + * List of key/value pairs, delimited by commas + * Both key and value should be a quoted string + * Entire map is enclosed by braces + * Example with two key/value pairs + * `{"key1" = "value1", "key2" = "value2"}` + * Empty value is `{}` +* list of string + * List of values, delimited by commas + * A value is a quoted string + * Entire list is enclosed in brackets + * Example with two values + * `["value1", "value2"]` + * Empty value is `[]` + +The following options should be set in your `lite.auto.tfvars` file (or entered interactively, if you choose to not create a file). Only a few of them have default values (as noted); the rest are required. The 'Updateable' column indicates whether, for any given option, it is possible to successfully apply the update against an already-running HPCC k8s cluster. + +|Option|Type|Description| +|:-----|:---|:----------| +| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | +| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | +| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | +| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | +| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | +| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | +| `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | +| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | +| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | +| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | +| `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | +| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | +| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | +| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | +| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | +| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | +| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | +| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | +| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | + +## Useful Things + +* Useful `kubectl` commands once the cluster is deployed: + * `kubectl get pods` + * Shows Kubernetes pods for the current cluster. + * `kubectl get services` + * Show the current services running on the pods on the current cluster. + * `kubectl config get-contexts` + * Show the saved kubectl contexts. A context contains login and reference information for a remote Kubernetes cluster. A kubectl command typically relays information about the current context. + * `kubectl config use-context ` + * Make \ context the current context for future kubectl commands. + * `kubectl config unset contexts.` + * Delete context named \. + * Note that when you delete the current context, kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. +* Note that `terraform destroy` does not delete the kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. +* If a deployment fails and you want to start over, you have two options: + * Immediately issue a `terraform destroy` command and let Terraform clean up. + * Clean up the resources by hand: + * Delete the Azure resource group manually, such as through the Azure Portal. + * Note that there are two resource groups, if the deployment got far enough. Examples: + * `app-thhpccplatform-sandbox-eastus-68255` + * `mc_tf-zrms-default-aks-1` + * The first one contains the Kubernetes service that created the second one (services that support Kubernetes). So, if you delete only the first resource group, the second resource group will be deleted automatically. + * Delete all Terraform state files using `rm *.tfstate*` + * Then, of course, fix whatever caused the deployment to fail. +* If you want to completely reset Terraform, issue `rm -rf .terraform* *.tfstate*` and then `terraform init`. diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf index 7a098d1..e88b5b1 100644 --- a/aks/lite-variables.tf +++ b/aks/lite-variables.tf @@ -30,18 +30,17 @@ variable "aks_azure_region" { variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool + default = false } variable "aks_dns_zone_resource_group_name" { type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" + description = "REQUIRED. Name of the resource group containing the dns zone." } variable "aks_dns_zone_name" { type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" + description = "REQUIRED. dns zone name. The name of existing dns zone." } variable "aks_admin_ip_cidr_map" { diff --git a/aks/outputs.tf b/aks/outputs.tf index 5f97637..2926b6b 100644 --- a/aks/outputs.tf +++ b/aks/outputs.tf @@ -23,4 +23,6 @@ output "cluster_resource_group_name" { resource "local_file" "output" { content = local.config filename = "${path.module}/data/config.json" + + depends_on = [ module.aks ] } diff --git a/hpcc/.outputs.tf.swp b/hpcc/.outputs.tf.swp new file mode 100644 index 0000000000000000000000000000000000000000..8bf023153193346b9fb48a21968779468fc4835f GIT binary patch literal 12288 zcmeI2&5G1O5XXBx39>5s0&T0{SCX0W;IOb4ML}Fa7e@~wp*N|_L_6u&AEUF&C_azR z;2S7jyn6RBtWG8>$}D?Sse=D>GF_?q*YB3ldxOV!@AGjyWEiIzyE(Z7H@===@fF78 zzBk}4n6P(9I5mrMm`AFt{p)C6jn?G8`B5<0GYeTK4vdk7HdWN_;5uuv4C^>Z!d0TZ zYrM0uE7p{xzli`5I0^ypifFvau8)R;kbYjA<7dxIj`Aes5CI}U1c(3;AOb{y2oM1x z@ShNH%?5jmvm9H^gP#D}*!XQ--O@q?hyW2F0z`la5CI}U1c(3;AOb{y2>gQtWX{-^ zO~wW%(LBEV_wWDTPBHcYWl=9t4eB21IO;RzenNdjy+^%6?V=u{MyOEh5$Y<6f(Q@+ zB0vO)01+SpM1Tko0U|&I{&oV(KZ0?dFT)QsQ0F$)HQ(k>d8Ko&APxt%`7?(5MX8{c z72vM8aL`CMi@TgXHuo1jo_lICinqk!Q5Ob54?+Q-e$&AaH)J@(^ccXP&zf*H)*hn59VMqQ#P(_ z#}W@riJsgHcsq(N+Uw3u73}s^cv~eFtUIjE#m}6y&*?g?@fZ%kOLs{?rd~;xg|qa9 z^vcB-?bdKS7!Jg>xioPYAg^t@uWM*MUp~zA?OQwiNvOsryF0&_>zkzBsNm;2;M?74 literal 0 HcmV?d00001 diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index 9bc8750..81db473 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -14,7 +14,7 @@ variable "aks_logging_monitoring_enabled" { } variable "external_storage_desired" { - description = "If you definitely want ephemeral storage instead of external, this should be false. For external storage this should be true" + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" type = bool default = false } @@ -22,6 +22,7 @@ variable "external_storage_desired" { variable "enable_thor" { description = "REQUIRED. If you want a thor cluster." type = bool + default = true } variable "a_record_name" { @@ -65,11 +66,13 @@ variable "aks_azure_region" { variable "enable_code_security" { description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" type = bool + default = false } variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool + default = false } variable "extra_tags" { @@ -80,14 +83,12 @@ variable "extra_tags" { variable "aks_dns_zone_resource_group_name" { type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" + description = "REQUIRED. Name of the resource group containing the dns zone." } variable "aks_dns_zone_name" { type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" + description = "REQUIRED. dns zone name. The name of existing dns zone." } variable "hpcc_user_ip_cidr_list" { @@ -132,6 +133,7 @@ variable "storage_data_gb" { condition = var.storage_data_gb >= 10 error_message = "Value must be 10 or more." } + default = 100 } variable "storage_lz_gb" { @@ -141,6 +143,7 @@ variable "storage_lz_gb" { condition = var.storage_lz_gb >= 1 error_message = "Value must be 1 or more." } + default = 25 } variable "thor_max_jobs" { @@ -150,6 +153,7 @@ variable "thor_max_jobs" { condition = var.thor_max_jobs >= 1 error_message = "Value must be 1 or more." } + default = 2 } variable "thor_num_workers" { @@ -159,6 +163,7 @@ variable "thor_num_workers" { condition = var.thor_num_workers >= 1 error_message = "Value must be 1 or more." } + default = 2 } ############################################################################### diff --git a/hpcc/main.tf b/hpcc/main.tf index dbb417e..9f61b5d 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -38,10 +38,3 @@ module "metadata" { interpreter = local.is_windows_os ? ["PowerShell", "-Command"] : ["/bin/bash", "-c"] } }*/ - -resource "local_file" "configjson" { - content = "hpcc was successfully deployed!" - filename = "${path.module}/data/config.json" - - depends_on = [ module.hpcc ] -} diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index 6e18fea..c028b3f 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -1,9 +1,16 @@ -output "hpcc_namespace" { - description = "The namespace where the HPCC Platform is deployed." - value = local.hpcc_namespace +output "eclwatch_url" { + description = "Print the ECL Watch URL." + value = format("eclwatch-default.%s:18010",var.aks_dns_zone_name) } -output "eclwatch" { - description = "Print the ECL Watch domain out." - value = local.svc_domains.eclwatch +output "deployment_resource_group" { + description = "Print the name of the deployment resource group." + value = local.get_aks_config.resource_group_name +} + +resource "local_file" "config" { + content = "hpcc successfully deployed" + filename = "${path.module}/data/config.json" + + depends_on = [ module.hpcc ] } diff --git a/storage/outputs.tf b/storage/outputs.tf index 49e6fc4..9e5808a 100644 --- a/storage/outputs.tf +++ b/storage/outputs.tf @@ -1,4 +1,6 @@ resource "local_file" "config" { content = module.storage.config filename = "${path.module}/data/config.json" + + depends_on = [ module.storage ] } diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf index 7a098d1..e88b5b1 100644 --- a/vnet/lite-variables.tf +++ b/vnet/lite-variables.tf @@ -30,18 +30,17 @@ variable "aks_azure_region" { variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool + default = false } variable "aks_dns_zone_resource_group_name" { type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" + description = "REQUIRED. Name of the resource group containing the dns zone." } variable "aks_dns_zone_name" { type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" + description = "REQUIRED. dns zone name. The name of existing dns zone." } variable "aks_admin_ip_cidr_map" { diff --git a/vnet/outputs.tf b/vnet/outputs.tf index 7c79d01..c4bf74b 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -29,6 +29,8 @@ output "vnet_name" { resource "local_file" "output" { content = local.config filename = "${path.module}/data/config.json" + + depends_on = [ module.virtual_network ] } From 3588eec9752247423bfbdde1431c83eb352256be Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sun, 5 Nov 2023 18:02:58 +0000 Subject: [PATCH 034/124] branch:HPCC-27615-easy-deploy-bryan5-miscellaneous-changes. Added documentation. --- documentation/hpcc-tf-for-developers.md | 170 ++++++++++++++++++++++++ hpcc/.outputs.tf.swp | Bin 12288 -> 0 bytes 2 files changed, 170 insertions(+) create mode 100755 documentation/hpcc-tf-for-developers.md delete mode 100644 hpcc/.outputs.tf.swp diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md new file mode 100755 index 0000000..733888a --- /dev/null +++ b/documentation/hpcc-tf-for-developers.md @@ -0,0 +1,170 @@ +# For Developers: Tutorial of HPCC Easy Deploy Terraform + +This tutorial explains the terraform that deploys HPCC Systems on an azure kubernetes service (aks). The terraform was designed to enable one to deploy HPCC Systems easily. +The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/tree/HPCC-27615-easy-deploy]) + +From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system within these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. + +The following sections will explain the terraform in root directory and all subdirectories. + +## Root Directory +Here is the root directory's contents (blue names are subdirectories) and a description of each entry: + + +|Entry Name|Description| +|:-----|:----------| +| `lite-variables.tf` | Contains all input variables | +| `lite.auto.tfvars.example` |Is an example .auto.tfvars file | +| `main.tf` | Contains most of the terraform that deploys all components of system | +| `providers.tf` | Contains one provider, azurerm | +| `scripts` | Directory containing scripts used in deployment | +| `aks` | Directory containing terraform to deploy `aks` | +| `hpcc` | Directory containing terraform to deploy `hpcc` | +| `storage` | Directory containing terraform to deploy external or persistent `storage` | +| `vnet` | Directory containing terraform to deploy virtual network used by `aks` | + +The subfolders, except for `scripts`, create components needed by the full system. + +The following table shows all the variables in the file, `lite-variables.tf`, and their types. Plus, the table gives a description of each variable. + +|Variable|Type|Description| +|:-----|:---|:----------| +| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | +| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | +| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | +| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | +| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | +| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | +| `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | +| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | +| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | +| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | +| `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | +| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | +| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | +| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | +| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | +| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | +| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | +| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | +| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | + +The following table gives the name of each of the 5 `null_resource` in `main.tf` and gives a short description of what each does. + +|null_resource name|description| +|:-----------------|:----------| +| `deploy_vnet` | deploys aks' virtual network | +| `deploy_aks` | deploys aks | +| `deploy_storage` | deploys persistent storage | +| `external_storage` | waits for deployment of presistent storage | +| `deploy_hpcc` | deploys hpcc | + +## scripts subdirectory + +|scripts subdirectory entry name|description| +|:--------------------------------|:----------| +| `deploy` | Deploys any of the components, i.e. aks, hpcc, storage, or vnet | +| `destroy` | Deploys a single component, i.e. aks, hpcc, storage, or vnet. This script destorys 1) the component whose name is given on the command line after `deploy`, e.g. `destroy vnet`, and 2) any components that depends of the component given on the command line after `destroy`, e.g. before `vnet` is destroyed both `hpcc` and `aks` would be destroyed. | +| `external_storage` | Waits for presistent storage to be created (or if ephemeral storage is used this scripts exits) NOTE: HPCC is not deployed until `external_storage` exits successfully. | +| `extract-aks-variables` | the `deploy` script uses this script to copy from root directory the `lite-variables.tf` file contents used to deploy a component. | +| `get_rg_from_file` | Outputs the resource group name in the `config.json` file given on the command line | +| `mkplan` | Make a unique name for the file that will contain the terraform plan of a component being deployed. | +| `needed-auto-tfvars-files` | Directory containing .auto.tfvars files needed by the `aks` and `storage` components. | + +## aks subdirectory + +|aks subdirectory entry name|description| +|:------------------------------|:----------| +| `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` and one if its parameters comes from the variable, `my_azure_id` which is the object id of the user's azure account. | +| `aks.tf` | This file contains must of the terraform needed to deploy `aks`. | +| `automation.tf` | This file contains the terraform for scheduling the stopping or starting of the kubernetes cluster. | +| `data`<\font> | This directory and its contents, `config.json`, are created after the `aks` cluster is successfully deployed. | +| `data.tf` | This file contains `data` statements that gets resources needed that already exist. | +| `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform was forked, all the variables in this file were input variables defined in `variables.tf`. | +| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This file was copied to the `aks` directory by the `deploy` script. | +| `lite.auto.tfvars` | This file contains all the variables (and their values) whose name beings with `aks_`. These variables and their values are copied from the lite.auto.tfvars file in the root directory. The copy is done by the script, `deploy`. | +| `locals.tf` | This file contains local variables that were originally in Godson Fortil's repository. | +| `main.tf` | This file contains resources and modules needed for the deployment. They are: `resource "random_integer" "int`, `resource "random_string" "string`, `module "subscription`, `module "naming`, `module "metadata`, `module "resource_groups`, `resource "null_resource" "az`. | +| `misc.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. | +| `outputs.tf` | This file contains `output` statement which outputs the following: `advisor_recommendations`,`aks_login`,`cluster_name`,`hpcc_log_analytics_enabled`,`cluster_resource_group_name`. | +| `providers.tf` | This file contains the following providers: `azurerm`,`azuread`,`kubernetes`,`kubernetes`,`kubectl`,`kubectl`,`helm`,`helm`,`shell`. | +| `variables.tf` | This file contains the variables described in the next table. | +| `versions.tf` | This file gives the version needed of each provider. | + + +## hpcc subdirectory + +## storage subdirectory + +## vnet subdirectory + + +Now you'll notice it has choice called no or resource and I've given names to each one of these floyd v-net for example deploy AKs deploy hpcc noticed that inside of these resources some of them have depends clauses the reason for the depends Clause is as an example aKs will not start until or venet completes and the depend on and your notice also that I have for hpcc those that depends on there and it won't start until the AKs is up and running + +There's a photo called Scripps and in it are several Scripts that ploy is used by the main to to deploy the various components of of the terraform boy is a bash script that will deploy if you + +specify as an argument hpcc it will it will it will it + + +And then it doesn't in it a plan reply and it does that for each one of these the AKs hpcc is exactly the same + +Each one of these directories will have a file in it that ends with example these all of these are actually Auto efr's filed and what you'll do with those is you will copy it into a file called well that will end with auto.tfvars you will change things to make it your own so as an example maybe I should do that +As an example of the first variable in this light.auto.tfvar's file is aksdns Zone name in the example file it doesn't have a DSN Zone name there but you have to put one there so it would be a DS Zone that you have created or somebody has created for you + +In the v-net folder you will find several.tf files refine an example file which is a template file you will have to make a auto.tfvar's file using it you also see that there may be a directory called Data and what happens there is once the v-net is up and running a file called config.json is placed in that older data + +If you look at the providers file is really only two providers is one called random then there's one called Azure RM random you will find will basically create the strings are random numbers for you and as your RM is play the base for any Azure functions a terraform as your functions + +Question +To the question is it's like a library each one of these is like libraries are these providers the same thing as like a library or like an import of some sort + + + +The variables files as you would expect contain input variables to to the the module in this case v-net module along with those when it defines the variable find those variables and give it a type type can be an integer string they can be an object if you look in this case here owner is actually an object which + + +Thelocals.tf file is very similar to the variables file in that it defines variables also the big difference is if it defines a variable here it can use variables that were in the variables.tf file to divide it for example this very first one the variables called name but it uses disabled naming conventions which is a variable when you reference these when you when you reference a variable that comes in the comes in the variable.tf file append to it variable. or pre-pin to it variable. variable. if it's a local variable you will say local. and then the name of the variable + +Let's look at the data file in the data file there are three data items I'm not quite sure what to say about these the only the only one I really know about is this one right here and what happens there is it actually goes out to uRL that URL will actually return the host name of whatever machine is executing the it's the host public IP that returns of the machine that has executing the terraform + + +Question +Okay go ahead let's try it the data HTTP host IP section of code that retrieves the the hosts IP address this save the IP address somewhere for it to be used does it save it in the URL variable +The answer to your question is yes it saves the public IP in the variable called UR to continue with that if you want if you want to get a hold of that stay as a data.hp http. host_ip.url for you to get a hold of it + +All right let's look at maine.tf +You'll notice in maine.tf there are several modules one called subscription one called names one called metadata there's also one called Resource Group that's it + + + +Outputs.tf the outputs.tf file will contain output statements and each of the output statements will basically when this module finishes it will it will output these outputs to either the person who ran the terraform or could be the module that that executed the terraform at the module that executed this particular module in this case vnet + +Versions.tf +The versions.tf file will give you versions that you need in order to execute the terraform if you'll notice in this file here they have a version statement the version statement for example the very first one it says the version is less than or equal to 2.99.0 if you if you if you were to use a version of as your RM that was larger than that that it would say it was three it wouldn't work it would give you an error the same thing with the others we've got adam and we also have + +The AKs module it will contain again some templates ending with the word example and of course again these will be converted into auto.tfvar's file by copying them and then making them owned by making them your own by changing the what the what the variables values are +Notice also that this folder has a data directory just like the vnet had saying this data directory is used in the same way as in vnet in other words when the AKs is completely when the AKs is completely deployed it will place a eurasian.jsonfile in the data directory +For both the v-neck aKs there is useful information inside of the config jason for example for the v-net config file both AKs and hpcc terraforms use animation in the config Dot jason file for vnet + +I think so + +In the AKs folder we have the terraform that that deploys the AKs as your resources + +Inside of main.tf you're fine that there's some resources and modules in in here you'll see for most of these these subdirectories are they will always have a have a module called subscription there are always having a module called metadata naming horse group these are all but they may have others in this case here we have a null resource called call AZ + + +The most important in this AKs directory is the aks.tf file and as you can guess aks.tf does the deployment of a kubernetes cluster + +And now in the hpcc you find the terraform that deploys an hpcc cluster as all the others this has very similar file names at least as there's a there's one example file so a +Your notice that there are several template files ending with the word example in so in this folder in this directory but most of them are currently not being used because they have a zero zero size so there's nothing in there + + +Let's look at the main.tf file what you're going to see is very very similar to the mains in the other directory is the v-net directory the AKs directory you have subscription naming metadata now the no resources different to have also where is resources Resource Group should be in here new + +The most important file in this directory is the hpcc.tf file as you can guess this deploys pCC cluster + + diff --git a/hpcc/.outputs.tf.swp b/hpcc/.outputs.tf.swp deleted file mode 100644 index 8bf023153193346b9fb48a21968779468fc4835f..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 12288 zcmeI2&5G1O5XXBx39>5s0&T0{SCX0W;IOb4ML}Fa7e@~wp*N|_L_6u&AEUF&C_azR z;2S7jyn6RBtWG8>$}D?Sse=D>GF_?q*YB3ldxOV!@AGjyWEiIzyE(Z7H@===@fF78 zzBk}4n6P(9I5mrMm`AFt{p)C6jn?G8`B5<0GYeTK4vdk7HdWN_;5uuv4C^>Z!d0TZ zYrM0uE7p{xzli`5I0^ypifFvau8)R;kbYjA<7dxIj`Aes5CI}U1c(3;AOb{y2oM1x z@ShNH%?5jmvm9H^gP#D}*!XQ--O@q?hyW2F0z`la5CI}U1c(3;AOb{y2>gQtWX{-^ zO~wW%(LBEV_wWDTPBHcYWl=9t4eB21IO;RzenNdjy+^%6?V=u{MyOEh5$Y<6f(Q@+ zB0vO)01+SpM1Tko0U|&I{&oV(KZ0?dFT)QsQ0F$)HQ(k>d8Ko&APxt%`7?(5MX8{c z72vM8aL`CMi@TgXHuo1jo_lICinqk!Q5Ob54?+Q-e$&AaH)J@(^ccXP&zf*H)*hn59VMqQ#P(_ z#}W@riJsgHcsq(N+Uw3u73}s^cv~eFtUIjE#m}6y&*?g?@fZ%kOLs{?rd~;xg|qa9 z^vcB-?bdKS7!Jg>xioPYAg^t@uWM*MUp~zA?OQwiNvOsryF0&_>zkzBsNm;2;M?74 From 8ec2aab21ffdddcf7fce26a123b20726d889ae46 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sun, 5 Nov 2023 19:26:59 +0000 Subject: [PATCH 035/124] branch:HPCC-27615-easy-deploy-bryan5-miscellaneous-changes. Updated developers documentation --- documentation/hpcc-tf-for-developers.md | 204 ++++++++++++++++++++++++ 1 file changed, 204 insertions(+) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index 733888a..e3a4b54 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -168,3 +168,207 @@ Let's look at the main.tf file what you're going to see is very very similar to The most important file in this directory is the hpcc.tf file as you can guess this deploys pCC cluster +## Appendix A Resources Create by HPCC Deployment + +| Resources Created by HPCC Deployment | +| :----------------------------------------------------------- | +| `local_file.config.json` | +| `random_integer.random` | +| `module.hpcc.azurerm_storage_account.azurefiles_admin_services[0]` | +| `module.hpcc.azurerm_storage_account.blob_nfs_admin_services[0]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["debug"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["dll"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["mydropzone"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["sasha"]` | +| `module.hpcc.azurerm_storage_share.azurefiles_admin_services["dali"]` | +| `module.hpcc.helm_release.hpcc` | +| `module.hpcc.kubernetes_persistent_volume.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume.spill["spill"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.spill["spill"]` | +| `module.hpcc.kubernetes_secret.azurefiles_admin_services[0]` | +| `module.hpcc.kubernetes_storage_class.premium_zrs_file_share_storage_class[0]` | +| `module.hpcc.random_string.random` | +| `module.hpcc.random_uuid.volume_handle` | +| `module.hpcc.module.certificates.kubectl_manifest.default_issuer` | +| `module.hpcc.module.certificates.kubectl_manifest.local_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.remote_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.signing_certificate` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["2"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | + +## Appendix B Resources Created by aks Deployment + +|Resources Created by aks Deployment| +|:------------------------------------------------------------------------------------------------| +| `data.azuread_group.subscription_owner` | +| `data.azurerm_advisor_recommendations.advisor` | +| `data.azurerm_client_config.current` | +| `data.azurerm_subscription.current` | +| `data.http.host_ip` | +| `local_file.output` | +| `null_resource.az[0]` | +| `random_integer.int` | +| `random_string.name` | +| `random_string.string` | +| `module.aks.data.azurerm_subscription.current` | +| `module.aks.kubernetes_config_map.terraform_modules` | +| `module.aks.kubernetes_config_map_v1_data.terraform_modules` | +| `module.aks.terraform_data.creation_metadata` | +| `module.aks.terraform_data.immutable_inputs` | +| `module.aks.time_static.timestamp` | +| `module.aks.module.cluster.data.azurerm_client_config.current` | +| `module.aks.module.cluster.data.azurerm_kubernetes_cluster.default` | +| `module.aks.module.cluster.data.azurerm_kubernetes_service_versions.default` | +| `module.aks.module.cluster.data.azurerm_monitor_diagnostic_categories.default` | +| `module.aks.module.cluster.data.azurerm_public_ip.outbound[0]` | +| `module.aks.module.cluster.azurerm_kubernetes_cluster.default` | +| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_network` | +| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_route_table[0]` | +| `module.aks.module.cluster.azurerm_user_assigned_identity.default` | +| `module.aks.module.cluster.terraform_data.maintenance_control_plane_start_date` | +| `module.aks.module.cluster.terraform_data.maintenance_nodes_start_date` | +| `module.aks.module.cluster.time_sleep.modify` | +| `module.aks.module.cluster_version_tag.shell_script.default` | +| `module.aks.module.core_config.kubernetes_labels.system_namespace["default"]` | +| `module.aks.module.core_config.kubernetes_labels.system_namespace["kube-system"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["cert-manager"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["dns"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["ingress-core-internal"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["logging"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["monitoring"]` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_cluster` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_node` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_virtual_machine_contributor_node` | +| `module.aks.module.core_config.module.aad_pod_identity.helm_release.aad_pod_identity` | +| `module.aks.module.core_config.module.aad_pod_identity.time_sleep.finalizer_wait` | +| `module.aks.module.core_config.module.cert_manager.helm_release.default` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt_staging"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["zerossl"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["configmap-dashboard-cert-manager.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["poddistributionbudget-cert-manager-webhook.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["prometheusrule-certmanager.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubernetes_secret.zerossl_eabsecret` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_federated_identity_credential.default["system:serviceaccount:cert-manager:cert-manager"]` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_role_assignment.default[0]` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_user_assigned_identity.default` | +| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_files["prometheusrule-coredns.yaml"]` | +| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_objects["coredns_custom"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureassignedidentities.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentities.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentitybindings.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azurepodidentityexceptions.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificaterequests.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificates.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["challenges.acme.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["clusterissuers.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["issuers.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["orders.acme.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["external-dns"].kubectl_manifest.crds["dnsendpoints.externaldns.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagerconfigs.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagers.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["podmonitors.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["probes.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusagents.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheuses.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusrules.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["scrapeconfigs.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["servicemonitors.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["thanosrulers.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.external_dns.helm_release.public[0]` | +| `module.aks.module.core_config.module.external_dns.kubectl_manifest.resource_files["configmap-dashboard-external-dns.yaml"]` | +| `module.aks.module.core_config.module.external_dns.kubernetes_secret.public_config[0]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_federated_identity_credential.default["system:serviceaccount:dns:external-dns-public"]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[0]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[1]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_user_assigned_identity.default` | +| `module.aks.module.core_config.module.ingress_internal_core.helm_release.default` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.certificate` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["configmap-dashboard-ingress-nginx-core-internal.yaml"]` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["prometheusrule-ingress-nginx-core-internal.yaml"]` | +| `module.aks.module.core_config.module.ingress_internal_core.time_sleep.lb_detach` | +| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0.shell_script.default` | +| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0-rc_1.shell_script.default` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-retain"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-retain"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-retain"]` | +| `module.aks.module.node_groups.module.bootstrap_node_group_hack.shell_script.default` | +| `module.aks.module.node_groups.module.system_node_groups["system1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["servpool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["spraypool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["thorpool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.rbac.azurerm_role_assignment.cluster_user["35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9"]` | +| `module.aks.module.rbac.kubernetes_cluster_role.aggregate_to_view[0]` | +| `module.aks.module.rbac.kubernetes_cluster_role_binding.cluster_admin[0]` | +| `module.metadata.data.azurerm_subscription.current` | +| `module.resource_groups["azure_kubernetes_service"].azurerm_resource_group.rg` | +| `module.resource_groups["azure_kubernetes_service"].random_integer.suffix[0]` | +| `module.subscription.data.azurerm_subscription.selected` | + +## Appendix C Resources Created by Deployment of vnet + + + +| Resources Created by Deployment of vnet | +| :----------------------------------------------------------- | +| `data.azurerm_advisor_recommendations.advisor` | +| `data.azurerm_subscription.current` | +| `data.http.host_ip` | +| `local_file.output` | +| `module.metadata.data.azurerm_subscription.current` | +| `module.resource_groups["virtual_network"].azurerm_resource_group.rg` | +| `module.resource_groups["virtual_network"].random_integer.suffix[0]` | +| `module.subscription.data.azurerm_subscription.selected` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-internet"]` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-local-vnet-10-1-0-0-21"]` | +| `module.virtual_network.azurerm_route_table.aks_route_table["hpcc"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-private"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-public"]` | +| `module.virtual_network.azurerm_virtual_network.vnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-private"].azurerm_subnet.subnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-public"].azurerm_subnet.subnet` | + +## Appendix D. Resources Created by Deployment of storage + +|Resources Created by Depolyment of storage| +|:------------------------------------------------------------------------------------| +| `local_file.config.json` | +| `module.storage.azurerm_storage_account.azurefiles["adminsvc1"]` | +| `module.storage.azurerm_storage_account.blobnfs["adminsvc2"]` | +| `module.storage.azurerm_storage_account.blobnfs["data1"]` | +| `module.storage.azurerm_storage_account.blobnfs["data2"]` | +| `module.storage.azurerm_storage_container.blobnfs["1"]` | +| `module.storage.azurerm_storage_container.blobnfs["2"]` | +| `module.storage.azurerm_storage_container.blobnfs["3"]` | +| `module.storage.azurerm_storage_container.blobnfs["4"]` | +| `module.storage.azurerm_storage_container.blobnfs["5"]` | +| `module.storage.azurerm_storage_container.blobnfs["6"]` | +| `module.storage.azurerm_storage_share.azurefiles["0"]` | +| `module.storage.null_resource.remove0000_from_azurefile["adminsvc1"]` | +| `module.storage.null_resource.remove0000_from_blobfs["adminsvc2"]` | +| `module.storage.null_resource.remove0000_from_blobfs["data1"]` | +| `module.storage.null_resource.remove0000_from_blobfs["data2"]` | +| `module.storage.random_string.random` | +| `module.storage.module.resource_groups["storage_accounts"].azurerm_resource_group.rg` | +| `module.storage.module.resource_groups["storage_accounts"].random_integer.suffix[0]` | From 498d908b3827fdf3e87a08091a6733299aed0d67 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 6 Nov 2023 20:38:44 +0000 Subject: [PATCH 036/124] branch:HPCC-27615-easy-deploy-bryan6-restrict-hpcc-access --- hpcc/hpcc.tf | 3 +++ hpcc/index.html | 1 + 2 files changed, 4 insertions(+) create mode 100644 hpcc/index.html diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 5ed2592..0fb9cc8 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -45,6 +45,9 @@ module "hpcc" { username = local.hpcc_container_registry_auth.username } : null + + hpcc_user_ip_cidr_list = var.hpcc_user_ip_cidr_list + storage_data_gb = var.storage_data_gb install_blob_csi_driver = false //Disable CSI driver diff --git a/hpcc/index.html b/hpcc/index.html new file mode 100644 index 0000000..7937d2e --- /dev/null +++ b/hpcc/index.html @@ -0,0 +1 @@ +20.96.202.148 \ No newline at end of file From 81afffd7976b922790219831280596e789b8a6f2 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 6 Nov 2023 21:13:04 +0000 Subject: [PATCH 037/124] branch:HPCC-27615-easy-deploy-bryan6-restrict-hpcc-access. Updated user's documentation --- README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) mode change 100644 => 100755 README.md diff --git a/README.md b/README.md old mode 100644 new mode 100755 index 6431c14..cf96256 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # Deploy HPCC Systems on Azure under Kubernetes -This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure. The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. +This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. -The HPCC Systems cluster this module creates uses ephemeral storage (meaning, the storage will be deleted if the cluster is deleted) or you can ask for Persistent Storage. See the section titled [Persistent Storage](#persistent_storage), below. +The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent_storage), below. This repo is a fork of the excellent work performed by Godson Fortil. The original can be found at [https://github.com/gfortil/terraform-azurerm-hpcc/tree/HPCC-27615]. @@ -95,6 +95,10 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | | `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | +## Persistent Storage + +To get persistent storage, i.e. storage that is not deleted when the hpcc cluster is deleted, set the variable, external_storage_desired, to true. + ## Useful Things * Useful `kubectl` commands once the cluster is deployed: From 902ec17ecf1d3d1fd57134118345d3c990a0a9f3 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 6 Nov 2023 21:18:12 +0000 Subject: [PATCH 038/124] branch:HPCC-27615-easy-deploy-bryan6-restrict-hpcc-access. Updating User's Documentation --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index cf96256..c05ceb9 100755 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. -The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent_storage), below. +The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. This repo is a fork of the excellent work performed by Godson Fortil. The original can be found at [https://github.com/gfortil/terraform-azurerm-hpcc/tree/HPCC-27615]. From 682612e17abf38192bab6aca8b894436c68d40c7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 6 Nov 2023 21:28:41 +0000 Subject: [PATCH 039/124] branch:HPCC-27615-easy-deploy-bryan6-restrict-hpcc-access. Updated User's Documentation. --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index c05ceb9..02b003d 100755 --- a/README.md +++ b/README.md @@ -30,12 +30,12 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi 1. Issue `terraform init` to initialize the Terraform modules. 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: 1. Invoke the `terraform apply` command and enter values for each option as Terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. - 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to do that is to copy the sample file and then edit the copy: + 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to do that is to copy the example file and then edit the copy: * `cp lite.auto.tfvars.example lite.auto.tfvars` 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. 1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). -At the end of a successful deployment several items are shown: +At the end of a successful deployment these items are output: * The URL used to access ECL Watch. * The deployment azure resource group. @@ -67,7 +67,7 @@ Options have data types. The ones used in this module are: * `["value1", "value2"]` * Empty value is `[]` -The following options should be set in your `lite.auto.tfvars` file (or entered interactively, if you choose to not create a file). Only a few of them have default values (as noted); the rest are required. The 'Updateable' column indicates whether, for any given option, it is possible to successfully apply the update against an already-running HPCC k8s cluster. +The following options should be set in your `lite.auto.tfvars` file (or entered interactively, if you choose to not create a file). Only a few of them have default values. The rest are required. The 'Updateable' column indicates whether, for any given option, it is possible to successfully apply the update against an already-running HPCC k8s cluster. |Option|Type|Description| |:-----|:---|:----------| From 2417b9a7b8b34f5a616e4b1390f629e3a375203d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:08:55 +0000 Subject: [PATCH 040/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation --- documentation/hpcc-tf-for-developers.md | 264 ++++++++++++------------ hpcc/dali.auto.tfvars.example | 0 hpcc/eclccserver.auto.tfvars.example | 0 hpcc/lite.auto.tfvars.example | 222 -------------------- hpcc/sasha.auto.tfvars.example | 0 5 files changed, 131 insertions(+), 355 deletions(-) delete mode 100644 hpcc/dali.auto.tfvars.example delete mode 100644 hpcc/eclccserver.auto.tfvars.example delete mode 100755 hpcc/lite.auto.tfvars.example delete mode 100644 hpcc/sasha.auto.tfvars.example diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index e3a4b54..c103f60 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -3,12 +3,12 @@ This tutorial explains the terraform that deploys HPCC Systems on an azure kubernetes service (aks). The terraform was designed to enable one to deploy HPCC Systems easily. The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/tree/HPCC-27615-easy-deploy]) -From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system within these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. +From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system from these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. The following sections will explain the terraform in root directory and all subdirectories. ## Root Directory -Here is the root directory's contents (blue names are subdirectories) and a description of each entry: +Here is the root directory's contents (**blue** names are subdirectories) and a description of each entry: |Entry Name|Description| @@ -23,9 +23,8 @@ Here is the root directory's contents (blue names are | `storage` | Directory containing terraform to deploy external or persistent `storage` | | `vnet` | Directory containing terraform to deploy virtual network used by `aks` | -The subfolders, except for `scripts`, create components needed by the full system. -The following table shows all the variables in the file, `lite-variables.tf`, and their types. Plus, the table gives a description of each variable. +The following table shows all the variables in the file, `lite-variables.tf`, and their types. Plus, the table gives a description of each variable. Also, when one deploys from the root directory the `deploy` script puts these variables (or some of them) in the subdirectory where the deployment takes place. |Variable|Type|Description| |:-----|:---|:----------| @@ -53,6 +52,7 @@ The following table shows all the variables in the file, `lite-variables.tf`, an | `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | | `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | + The following table gives the name of each of the 5 `null_resource` in `main.tf` and gives a short description of what each does. |null_resource name|description| @@ -63,29 +63,36 @@ The following table gives the name of each of the 5 `null_resource` in `main.tf` | `external_storage` | waits for deployment of presistent storage | | `deploy_hpcc` | deploys hpcc | +The subfolders, except for `scripts`, create components needed by the full system. + ## scripts subdirectory |scripts subdirectory entry name|description| |:--------------------------------|:----------| -| `deploy` | Deploys any of the components, i.e. aks, hpcc, storage, or vnet | -| `destroy` | Deploys a single component, i.e. aks, hpcc, storage, or vnet. This script destorys 1) the component whose name is given on the command line after `deploy`, e.g. `destroy vnet`, and 2) any components that depends of the component given on the command line after `destroy`, e.g. before `vnet` is destroyed both `hpcc` and `aks` would be destroyed. | +| `deploy` | Used by each of the `deploy` `null_resource`s in main.tf. This script deploys any of the components, i.e. aks, hpcc, storage, or vnet | +| `destroy` | Destroys a single component, i.e. aks, hpcc, storage, or vnet. This script destorys 1) the component whose name is given on the command line after `deploy`, e.g. `destroy vnet`, and 2) any components that depends on it, e.g. before `vnet` is destroyed both `hpcc` and `aks` would be destroyed. | | `external_storage` | Waits for presistent storage to be created (or if ephemeral storage is used this scripts exits) NOTE: HPCC is not deployed until `external_storage` exits successfully. | -| `extract-aks-variables` | the `deploy` script uses this script to copy from root directory the `lite-variables.tf` file contents used to deploy a component. | +| `extract-aks-variables` | the `deploy` script uses this script to copy from root directory the `lite-variables.tf` file contents used to deploy aks. | | `get_rg_from_file` | Outputs the resource group name in the `config.json` file given on the command line | -| `mkplan` | Make a unique name for the file that will contain the terraform plan of a component being deployed. | +| `mkplan` | Makes a unique name for the file that will contain the terraform plan of a component being deployed. | | `needed-auto-tfvars-files` | Directory containing .auto.tfvars files needed by the `aks` and `storage` components. | ## aks subdirectory +The following table tells what files and subdirectories and in the `aks` subdirectory. The deployment of an `aks`happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an `aks`. Also, if you deploy `aks` manually you do it from this directory. + +What is deployed by this subdirectory and their order is given in Appendix A (#appendix_a). + + |aks subdirectory entry name|description| |:------------------------------|:----------| -| `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` and one if its parameters comes from the variable, `my_azure_id` which is the object id of the user's azure account. | -| `aks.tf` | This file contains must of the terraform needed to deploy `aks`. | -| `automation.tf` | This file contains the terraform for scheduling the stopping or starting of the kubernetes cluster. | +| `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` is one of this file's variables which contains the variable, `my_azure_id` which is the object id of the user's azure account. This variable is given its value by the script `deploy`.| +| `aks.tf` | This file contains most of the terraform needed to deploy `aks`. The main module in this file is the `aks` module. | +| `automation.tf` | This file contains the terraform for scheduling the stopping and/or starting of the kubernetes cluster. | | `data`<\font> | This directory and its contents, `config.json`, are created after the `aks` cluster is successfully deployed. | | `data.tf` | This file contains `data` statements that gets resources needed that already exist. | | `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform was forked, all the variables in this file were input variables defined in `variables.tf`. | -| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This file was copied to the `aks` directory by the `deploy` script. | +| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `aks`. This file was copied to the `aks` directory by the `deploy` script. | | `lite.auto.tfvars` | This file contains all the variables (and their values) whose name beings with `aks_`. These variables and their values are copied from the lite.auto.tfvars file in the root directory. The copy is done by the script, `deploy`. | | `locals.tf` | This file contains local variables that were originally in Godson Fortil's repository. | | `main.tf` | This file contains resources and modules needed for the deployment. They are: `resource "random_integer" "int`, `resource "random_string" "string`, `module "subscription`, `module "naming`, `module "metadata`, `module "resource_groups`, `resource "null_resource" "az`. | @@ -95,123 +102,70 @@ The following table gives the name of each of the 5 `null_resource` in `main.tf` | `variables.tf` | This file contains the variables described in the next table. | | `versions.tf` | This file gives the version needed of each provider. | - ## hpcc subdirectory -## storage subdirectory - -## vnet subdirectory - - -Now you'll notice it has choice called no or resource and I've given names to each one of these floyd v-net for example deploy AKs deploy hpcc noticed that inside of these resources some of them have depends clauses the reason for the depends Clause is as an example aKs will not start until or venet completes and the depend on and your notice also that I have for hpcc those that depends on there and it won't start until the AKs is up and running - -There's a photo called Scripps and in it are several Scripts that ploy is used by the main to to deploy the various components of of the terraform boy is a bash script that will deploy if you - -specify as an argument hpcc it will it will it will it - - -And then it doesn't in it a plan reply and it does that for each one of these the AKs hpcc is exactly the same - -Each one of these directories will have a file in it that ends with example these all of these are actually Auto efr's filed and what you'll do with those is you will copy it into a file called well that will end with auto.tfvars you will change things to make it your own so as an example maybe I should do that -As an example of the first variable in this light.auto.tfvar's file is aksdns Zone name in the example file it doesn't have a DSN Zone name there but you have to put one there so it would be a DS Zone that you have created or somebody has created for you - -In the v-net folder you will find several.tf files refine an example file which is a template file you will have to make a auto.tfvar's file using it you also see that there may be a directory called Data and what happens there is once the v-net is up and running a file called config.json is placed in that older data - -If you look at the providers file is really only two providers is one called random then there's one called Azure RM random you will find will basically create the strings are random numbers for you and as your RM is play the base for any Azure functions a terraform as your functions +The following table tells what files and subdirectories and in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. -Question -To the question is it's like a library each one of these is like libraries are these providers the same thing as like a library or like an import of some sort +What is deployed by this subdirectory and their order is given in Appendix B (#appendix_b). +|hpcc subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `hpcc.tf` | Contains the `hpcc` module which does most of the work of deploying an hpcc cluster. | +| `lite-locals.tf` | Contains variables that use lite-variables.tf variables. The contents was in .auto.tfvars of Godson's terraform-azurerm-hpcc, branch HPCC-27615 (which this terraform is a fork). | +| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | +| `lite.auto.tfvars` | Contains alls the variables used for easy deployment with values of the user. This file is copied in the hpcc directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of the hpcc cluster. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains modules and resources needed for the deployment of the hpcc cluster. | +| `outputs.tf` | Contains output statements that show the user important information, like the eclwatch url and the resource group used by most of the resources. | +| `providers.tf` | Contains providers needed for the hpcc cluster deployment. Also, some of these providers (`kubernetes` and `kubectl`) get credentials for the kubernetes cluster for authenication. | +| `versions.tf` | Contains the versions needed for all providers. | +| `data` | This directory contains the file `config.json` which is created when the hpcc cluster successfully deploys. | +## storage subdirectory -The variables files as you would expect contain input variables to to the the module in this case v-net module along with those when it defines the variable find those variables and give it a type type can be an integer string they can be an object if you look in this case here owner is actually an object which - - -Thelocals.tf file is very similar to the variables file in that it defines variables also the big difference is if it defines a variable here it can use variables that were in the variables.tf file to divide it for example this very first one the variables called name but it uses disabled naming conventions which is a variable when you reference these when you when you reference a variable that comes in the comes in the variable.tf file append to it variable. or pre-pin to it variable. variable. if it's a local variable you will say local. and then the name of the variable - -Let's look at the data file in the data file there are three data items I'm not quite sure what to say about these the only the only one I really know about is this one right here and what happens there is it actually goes out to uRL that URL will actually return the host name of whatever machine is executing the it's the host public IP that returns of the machine that has executing the terraform - - -Question -Okay go ahead let's try it the data HTTP host IP section of code that retrieves the the hosts IP address this save the IP address somewhere for it to be used does it save it in the URL variable -The answer to your question is yes it saves the public IP in the variable called UR to continue with that if you want if you want to get a hold of that stay as a data.hp http. host_ip.url for you to get a hold of it - -All right let's look at maine.tf -You'll notice in maine.tf there are several modules one called subscription one called names one called metadata there's also one called Resource Group that's it - - - -Outputs.tf the outputs.tf file will contain output statements and each of the output statements will basically when this module finishes it will it will output these outputs to either the person who ran the terraform or could be the module that that executed the terraform at the module that executed this particular module in this case vnet - -Versions.tf -The versions.tf file will give you versions that you need in order to execute the terraform if you'll notice in this file here they have a version statement the version statement for example the very first one it says the version is less than or equal to 2.99.0 if you if you if you were to use a version of as your RM that was larger than that that it would say it was three it wouldn't work it would give you an error the same thing with the others we've got adam and we also have - -The AKs module it will contain again some templates ending with the word example and of course again these will be converted into auto.tfvar's file by copying them and then making them owned by making them your own by changing the what the what the variables values are -Notice also that this folder has a data directory just like the vnet had saying this data directory is used in the same way as in vnet in other words when the AKs is completely when the AKs is completely deployed it will place a eurasian.jsonfile in the data directory -For both the v-neck aKs there is useful information inside of the config jason for example for the v-net config file both AKs and hpcc terraforms use animation in the config Dot jason file for vnet +The following table tells what files and subdirectories and in the `storage` subdirectory. The deployment of an `storage` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `storage`. Also, if you deploy `storage` manually you do it from this directory. -I think so +What is deployed by this subdirectory and their order is given in Appendix C (#appendix_c). -In the AKs folder we have the terraform that that deploys the AKs as your resources +|storage subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `storage`. This file was copied to the `aks` directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of the `storage`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains only the `storage` module | +| `outputs.tf` | Contains only the resource `local_file` which outputs to a file config.json. This is done only when on a successful deployment of `storage`. | +| `providers.tf` | Contains only 2 providers: azurerm and azuread | +| `storage.auto.tfvars` | Contains variables that describe the storage accounts that are created. This file is copied to the `storage` directory by the `deploy` script. | +| `variables.tf` | Contains variables needed for `storage` deployment. | +| `versions.tf` | dummy description text | +| `data` | This directory contains the file `config.json` which is created when the external storage successfully deploys. | -Inside of main.tf you're fine that there's some resources and modules in in here you'll see for most of these these subdirectories are they will always have a have a module called subscription there are always having a module called metadata naming horse group these are all but they may have others in this case here we have a null resource called call AZ +## vnet subdirectory -The most important in this AKs directory is the aks.tf file and as you can guess aks.tf does the deployment of a kubernetes cluster +The following table tells what files and subdirectories are in the `vnet` subdirectory. The deployment of an `vnet` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `vnet`. Also, if you deploy `vnet` manually you do it from this directory. -And now in the hpcc you find the terraform that deploys an hpcc cluster as all the others this has very similar file names at least as there's a there's one example file so a -Your notice that there are several template files ending with the word example in so in this folder in this directory but most of them are currently not being used because they have a zero zero size so there's nothing in there +What is deployed by this subdirectory and their order is given in Appendix D (#appendix_d). +|vnet subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | +| `lite.auto.tfvars` | Contains all the variables used for easy deployment with values of the user. This file is copied in the `vnet` directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of `vnet`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains modules and resources needed for the deployment of `vnet` | +| `outputs.tf` | Contains several output statements that output important information to the deployer. Also, this file contains an output state that outputs a file, config.json. This file is only output if there is a successful deployment of `vnet`. | +| `providers.tf` | Contains 2 providers: `random` and `azurerm`. | +| `variables.tf` | Contains only one variable, `disable_naming_conventions`. | +| `versions.tf` | Contains the required versions of `terraform`, `azurerm` and `random`. | +| `vnet.tf` | Contains the module `virtual_network` which deploys the virtual network used by `aks`, `hpcc`, and `storage`. | +| `data` | This directory contains the file `config.json` which is created when the `vnet` is successfully deploys. | -Let's look at the main.tf file what you're going to see is very very similar to the mains in the other directory is the v-net directory the AKs directory you have subscription naming metadata now the no resources different to have also where is resources Resource Group should be in here new +​ -The most important file in this directory is the hpcc.tf file as you can guess this deploys pCC cluster - -## Appendix A Resources Create by HPCC Deployment - -| Resources Created by HPCC Deployment | -| :----------------------------------------------------------- | -| `local_file.config.json` | -| `random_integer.random` | -| `module.hpcc.azurerm_storage_account.azurefiles_admin_services[0]` | -| `module.hpcc.azurerm_storage_account.blob_nfs_admin_services[0]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["debug"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["dll"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["mydropzone"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["sasha"]` | -| `module.hpcc.azurerm_storage_share.azurefiles_admin_services["dali"]` | -| `module.hpcc.helm_release.hpcc` | -| `module.hpcc.kubernetes_persistent_volume.azurefiles["dali"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-1"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-2"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["debug"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["dll"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["mydropzone"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["sasha"]` | -| `module.hpcc.kubernetes_persistent_volume.spill["spill"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.azurefiles["dali"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-1"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-2"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["debug"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["dll"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["mydropzone"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["sasha"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.spill["spill"]` | -| `module.hpcc.kubernetes_secret.azurefiles_admin_services[0]` | -| `module.hpcc.kubernetes_storage_class.premium_zrs_file_share_storage_class[0]` | -| `module.hpcc.random_string.random` | -| `module.hpcc.random_uuid.volume_handle` | -| `module.hpcc.module.certificates.kubectl_manifest.default_issuer` | -| `module.hpcc.module.certificates.kubectl_manifest.local_certificate` | -| `module.hpcc.module.certificates.kubectl_manifest.remote_certificate` | -| `module.hpcc.module.certificates.kubectl_manifest.signing_certificate` | -| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["1"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["2"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["1"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | - -## Appendix B Resources Created by aks Deployment +## Appendix A. Resources Created by aks Deployment |Resources Created by aks Deployment| |:------------------------------------------------------------------------------------------------| @@ -326,30 +280,52 @@ The most important file in this directory is the hpcc.tf file as you can guess t | `module.resource_groups["azure_kubernetes_service"].random_integer.suffix[0]` | | `module.subscription.data.azurerm_subscription.selected` | -## Appendix C Resources Created by Deployment of vnet - +## Appendix B. Resources Created by HPCC Deployment -| Resources Created by Deployment of vnet | +| Resources Created by HPCC Deployment | | :----------------------------------------------------------- | -| `data.azurerm_advisor_recommendations.advisor` | -| `data.azurerm_subscription.current` | -| `data.http.host_ip` | -| `local_file.output` | -| `module.metadata.data.azurerm_subscription.current` | -| `module.resource_groups["virtual_network"].azurerm_resource_group.rg` | -| `module.resource_groups["virtual_network"].random_integer.suffix[0]` | -| `module.subscription.data.azurerm_subscription.selected` | -| `module.virtual_network.azurerm_route.aks_route["hpcc-internet"]` | -| `module.virtual_network.azurerm_route.aks_route["hpcc-local-vnet-10-1-0-0-21"]` | -| `module.virtual_network.azurerm_route_table.aks_route_table["hpcc"]` | -| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-private"]` | -| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-public"]` | -| `module.virtual_network.azurerm_virtual_network.vnet` | -| `module.virtual_network.module.aks_subnet["aks-hpcc-private"].azurerm_subnet.subnet` | -| `module.virtual_network.module.aks_subnet["aks-hpcc-public"].azurerm_subnet.subnet` | +| `local_file.config.json` | +| `random_integer.random` | +| `module.hpcc.azurerm_storage_account.azurefiles_admin_services[0]` | +| `module.hpcc.azurerm_storage_account.blob_nfs_admin_services[0]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["debug"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["dll"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["mydropzone"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["sasha"]` | +| `module.hpcc.azurerm_storage_share.azurefiles_admin_services["dali"]` | +| `module.hpcc.helm_release.hpcc` | +| `module.hpcc.kubernetes_persistent_volume.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume.spill["spill"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.spill["spill"]` | +| `module.hpcc.kubernetes_secret.azurefiles_admin_services[0]` | +| `module.hpcc.kubernetes_storage_class.premium_zrs_file_share_storage_class[0]` | +| `module.hpcc.random_string.random` | +| `module.hpcc.random_uuid.volume_handle` | +| `module.hpcc.module.certificates.kubectl_manifest.default_issuer` | +| `module.hpcc.module.certificates.kubectl_manifest.local_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.remote_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.signing_certificate` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["2"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | + -## Appendix D. Resources Created by Deployment of storage +## Appendix C. Resources Created by Deployment of storage |Resources Created by Depolyment of storage| |:------------------------------------------------------------------------------------| @@ -372,3 +348,25 @@ The most important file in this directory is the hpcc.tf file as you can guess t | `module.storage.random_string.random` | | `module.storage.module.resource_groups["storage_accounts"].azurerm_resource_group.rg` | | `module.storage.module.resource_groups["storage_accounts"].random_integer.suffix[0]` | + +## Appendix D. Resources Created by Deployment of vnet + + +| Resources Created by Deployment of vnet | +| :----------------------------------------------------------- | +| `data.azurerm_advisor_recommendations.advisor` | +| `data.azurerm_subscription.current` | +| `data.http.host_ip` | +| `local_file.output` | +| `module.metadata.data.azurerm_subscription.current` | +| `module.resource_groups["virtual_network"].azurerm_resource_group.rg` | +| `module.resource_groups["virtual_network"].random_integer.suffix[0]` | +| `module.subscription.data.azurerm_subscription.selected` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-internet"]` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-local-vnet-10-1-0-0-21"]` | +| `module.virtual_network.azurerm_route_table.aks_route_table["hpcc"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-private"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-public"]` | +| `module.virtual_network.azurerm_virtual_network.vnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-private"].azurerm_subnet.subnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-public"].azurerm_subnet.subnet` | diff --git a/hpcc/dali.auto.tfvars.example b/hpcc/dali.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 diff --git a/hpcc/eclccserver.auto.tfvars.example b/hpcc/eclccserver.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 diff --git a/hpcc/lite.auto.tfvars.example b/hpcc/lite.auto.tfvars.example deleted file mode 100755 index 8373caa..0000000 --- a/hpcc/lite.auto.tfvars.example +++ /dev/null @@ -1,222 +0,0 @@ -#----------------------------------------------------------------------------- - -# Name of the A record, of following dns zone, where the ecl watch ip is placed -# This A record will be created and therefore should not exist in the following -# dns zone. -# Example entry: "my-product". This should be something project specific rather -# than something generic. - -a_record_name="" - -#----------------------------------------------------------------------------- - -# Name of an existing dns zone. -# Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" -# REQUIRED - -aks_dns_zone_name="" - -#----------------------------------------------------------------------------- - -# Name of the resource group of the above dns zone. -# Example entry: "app-dns-prod-eastus2" -# REQUIRED - -aks_dns_zone_resource_group_name="" - -#------------------------------------------------------------------------------ - -# The version of HPCC Systems to install. -# Only versions in nn.nn.nn format are supported. -# Value type: string - -hpcc_version="8.6.14" - -#------------------------------------------------------------------------------ - -# Enable ROXIE? -# This will also expose port 8002 on the cluster. -# Value type: boolean -# Example entry: false - -aks_enable_roxie=true - -#------------------------------------------------------------------------------ - -# Enable code security? -# If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. -# Value type: boolean -# Example entry: false - -enable_code_security=true - -#------------------------------------------------------------------------------ - -# If you want a thor cluster then 'enable_thor' must be set to true -# Otherwise it is set to false - -enable_thor=true - -#------------------------------------------------------------------------------ - -# The number of Thor workers to allocate. -# Must be 1 or more. - -thor_num_workers=2 - -#------------------------------------------------------------------------------ - -# The maximum number of simultaneous Thor jobs allowed. -# Must be 1 or more. - -thor_max_jobs=2 - -#------------------------------------------------------------------------------ - -# The amount of storage reserved for the landing zone in gigabytes. -# Must be 1 or more. -# If a storage account is defined (see below) then this value is ignored. - -storage_lz_gb=25 - -#------------------------------------------------------------------------------ - -# The amount of storage reserved for data in gigabytes. -# Must be 1 or more. -# If a storage account is defined (see below) then this value is ignored. - -storage_data_gb=100 - -#------------------------------------------------------------------------------ - -# Map of name => value tags that can will be associated with the cluster. -# Format is '{"name"="value" [, "name"="value"]*}'. -# The 'name' portion must be unique. -# To add no tags, use '{}'. -# Value type: map of string -# Example: extra_tags={ "owner"="Jane Doe", "owner_email"="jane.doe@gmail.com" } - -extra_tags={} - -#------------------------------------------------------------------------------ - -# The VM size for each node in the HPCC Systems node pool. -# Recommend "Standard_B4ms" or better. -# See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. -# Value type: string - -aks_node_size="Standard_B8ms" - -#------------------------------------------------------------------------------ - -# The maximum number of VM nodes to allocate for the HPCC Systems node pool. -# Must be 2 or more. -# Value type: integer - -aks_max_node_count=4 - -#------------------------------------------------------------------------------ - -# Email address of the administrator of this HPCC Systems cluster. -# Value type: string -# Example entry: "jane.doe@hpccsystems.com" - -aks_admin_email="jane.doe@gmail.com" - -#------------------------------------------------------------------------------ - -# Name of the administrator of this HPCC Systems cluster. -# Value type: string -# Example entry: "Jane Doe" - -aks_admin_name="Jane Doe" - -#------------------------------------------------------------------------------ - -# Username of the administrator of this HPCC Systems cluster. -# Value type: string -# Example entry: "jdoe" - -admin_username="jdoe" - -#------------------------------------------------------------------------------ - -# The Azure region abbreviation in which to create these resources. -# Must be one of ["eastus", "eastus2", "centralus"]. -# Value type: string -# Example entry: "eastus" - -aks_azure_region="eastus" - -#------------------------------------------------------------------------------ - -# Map of name => CIDR IP addresses that can administrate this AKS. -# Format is '{"name"="cidr" [, "name"="cidr"]*}'. -# The 'name' portion must be unique. -# To add no CIDR addresses, use '{}'. -# The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. -# Value type: map of string - -aks_admin_ip_cidr_map={} - -#------------------------------------------------------------------------------ - -# List of explicit CIDR addresses that can access this HPCC Systems cluster. -# To allow public access, specify "0.0.0.0/0". -# To add no CIDR addresses, use '[]'. -# Value type: list of string - -hpcc_user_ip_cidr_list=[] - -#------------------------------------------------------------------------------ - -# If you are attaching to an existing storage account, put its name here. -# Leave as an empty string if you do not have a storage account. -# If you put something here then you must also define a resource group for the storage account. -# Value type: string -# Example entry: "my-product-sa" - -storage_account_name="" - -#------------------------------------------------------------------------------ - -# If you are attaching to an existing storage account, put its resource group name here. -# Leave as an empty string if you do not have a storage account. -# If you put something here then you must also define a name for the storage account. -# Value type: string - -storage_account_resource_group_name="" - -#------------------------------------------------------------------------------ - -# The Kubernetes namespace in which to install the HPCC modules (if enabled). -# Default value: "default" - -# hpcc_namespace="default" - -#------------------------------------------------------------------------------ - -# If true, premium ($$$) storage will be used for the following storage shares: Dali. -# OPTIONAL, defaults to false. - -enable_premium_storage=false - -#------------------------------------------------------------------------------ - -# If you would like to use htpasswd to authenticate users to the cluster, enter -# the filename of the htpasswd file. This file should be uploaded to the Azure -# 'dllsshare' file share in order for the HPCC processes to find it. -# A corollary is that persistent storage is enabled. -# An empty string indicates that htpasswd is not to be used for authentication. -# Example entry: "htpasswd.txt" - -authn_htpasswd_filename="" - -#------------------------------------------------------------------------------ - -# If you definitely want ephemeral storage instead of external storage then -# set this variable to true otherwise set it to false. - -external_storage_desired=false - -#------------------------------------------------------------------------------ diff --git a/hpcc/sasha.auto.tfvars.example b/hpcc/sasha.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 From d8aa06c47315776bbff6de8e481cc384028b8e78 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:20:46 +0000 Subject: [PATCH 041/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation --- documentation/hpcc-tf-for-developers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index c103f60..84f4e2b 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -81,7 +81,7 @@ The subfolders, except for `scripts`, create components needed by the full syste The following table tells what files and subdirectories and in the `aks` subdirectory. The deployment of an `aks`happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an `aks`. Also, if you deploy `aks` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in Appendix A (#appendix_a). +What is deployed by this subdirectory and their order is given in [_(Appendix A)_](#Appendix-A) |aks subdirectory entry name|description| From bf1a64d29f4f869fb06bdab45e39ecea16a84518 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:34:43 +0000 Subject: [PATCH 042/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation --- documentation/hpcc-tf-for-developers.md | 26 +++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index 84f4e2b..cbc2533 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -81,7 +81,7 @@ The subfolders, except for `scripts`, create components needed by the full syste The following table tells what files and subdirectories and in the `aks` subdirectory. The deployment of an `aks`happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an `aks`. Also, if you deploy `aks` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in [_(Appendix A)_](#Appendix-A) +What is deployed by this subdirectory and their order is given in [_Appendix A_](#Appendix-A). |aks subdirectory entry name|description| @@ -106,7 +106,7 @@ What is deployed by this subdirectory and their order is given in [_(Appendix A The following table tells what files and subdirectories and in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. -What is deployed by this subdirectory and their order is given in Appendix B (#appendix_b). +What is deployed by this subdirectory and their order is given in [_Appendix B_](#Appendix-B). |hpcc subdirectory entry name|description| |:--------------------------------|:----------| @@ -126,7 +126,7 @@ What is deployed by this subdirectory and their order is given in Appendix B (#a The following table tells what files and subdirectories and in the `storage` subdirectory. The deployment of an `storage` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `storage`. Also, if you deploy `storage` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in Appendix C (#appendix_c). +What is deployed by this subdirectory and their order is given in [_Appendix C_](#Appendix-C). |storage subdirectory entry name|description| |:--------------------------------|:----------| @@ -146,7 +146,7 @@ What is deployed by this subdirectory and their order is given in Appendix C (#a The following table tells what files and subdirectories are in the `vnet` subdirectory. The deployment of an `vnet` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `vnet`. Also, if you deploy `vnet` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in Appendix D (#appendix_d). +What is deployed by this subdirectory and their order is given in [_Appendix D_](#Appendix-D). |vnet subdirectory entry name|description| |:--------------------------------|:----------| @@ -164,8 +164,11 @@ What is deployed by this subdirectory and their order is given in Appendix D (#a ​ +## Appendix A + +**Resources Created by aks Deployment** + -## Appendix A. Resources Created by aks Deployment |Resources Created by aks Deployment| |:------------------------------------------------------------------------------------------------| @@ -280,7 +283,9 @@ What is deployed by this subdirectory and their order is given in Appendix D (#a | `module.resource_groups["azure_kubernetes_service"].random_integer.suffix[0]` | | `module.subscription.data.azurerm_subscription.selected` | -## Appendix B. Resources Created by HPCC Deployment +## Appendix B + +**Resources Created by HPCC Deployment** | Resources Created by HPCC Deployment | @@ -325,7 +330,10 @@ What is deployed by this subdirectory and their order is given in Appendix D (#a | `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | -## Appendix C. Resources Created by Deployment of storage +## Appendix C + +**Resources Created by Deployment of storage** + |Resources Created by Depolyment of storage| |:------------------------------------------------------------------------------------| @@ -349,7 +357,9 @@ What is deployed by this subdirectory and their order is given in Appendix D (#a | `module.storage.module.resource_groups["storage_accounts"].azurerm_resource_group.rg` | | `module.storage.module.resource_groups["storage_accounts"].random_integer.suffix[0]` | -## Appendix D. Resources Created by Deployment of vnet +## Appendix D + +**Resources Created by Deployment of vnet** | Resources Created by Deployment of vnet | From 41be7fa28e6da15a8c5d8baf0dae0500aec24a58 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:40:56 +0000 Subject: [PATCH 043/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation --- documentation/hpcc-tf-for-developers.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index cbc2533..f9121c6 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -166,9 +166,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix D_ ## Appendix A -**Resources Created by aks Deployment** - - |Resources Created by aks Deployment| |:------------------------------------------------------------------------------------------------| @@ -285,8 +282,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix D_ ## Appendix B -**Resources Created by HPCC Deployment** - | Resources Created by HPCC Deployment | | :----------------------------------------------------------- | @@ -332,8 +327,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix D_ ## Appendix C -**Resources Created by Deployment of storage** - |Resources Created by Depolyment of storage| |:------------------------------------------------------------------------------------| @@ -359,8 +352,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix D_ ## Appendix D -**Resources Created by Deployment of vnet** - | Resources Created by Deployment of vnet | | :----------------------------------------------------------- | From 02fe05b2dc4d817ef17de5236a2f899c7615bb95 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:49:36 +0000 Subject: [PATCH 044/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Updated README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 02b003d..e2c2800 100755 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # Deploy HPCC Systems on Azure under Kubernetes +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/blob/HPCC-27615-easy-deploy/documentation/hpcc-tf-for-developers.md]. + This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. From 5bc6502c8a2b944333876a040ccc5ef716f1b331 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:53:20 +0000 Subject: [PATCH 045/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Updated README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e2c2800..29de0c7 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/blob/HPCC-27615-easy-deploy/documentation/hpcc-tf-for-developers.md]. +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/HPCC-27615-easy-deploy/documentation/hpcc-tf-for-developers.md]. This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. From a8a8170f5a0714c9c61f68db01c664152fc7f8f7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 19:56:03 +0000 Subject: [PATCH 046/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Updated README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 29de0c7..adc52cb 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/HPCC-27615-easy-deploy/documentation/hpcc-tf-for-developers.md]. +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [documentation/hpcc-tf-for-developers.md]. This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. From 882a39034a131aa266c1143af3c30cf431b1287a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 20:25:34 +0000 Subject: [PATCH 047/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index adc52cb..49017b4 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [documentation/hpcc-tf-for-developers.md]. +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [here](documentation/hpcc-tf-for-developers.md). This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. From 345453efd647d978ccc8e9fe77cf69f417438c1a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 7 Nov 2023 20:28:30 +0000 Subject: [PATCH 048/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 49017b4..0aea139 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found at [here](documentation/hpcc-tf-for-developers.md). +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. From 986595a63828027b3cce0a48c60d3ae2d82e0005 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 9 Nov 2023 21:42:38 +0000 Subject: [PATCH 049/124] branch:HPCC-27615-easy-deploy-bryan8-cleanup-and-make-clearer --- hpcc/data.tf | 5 +++++ hpcc/hpcc.tf | 24 +++++++++++------------- hpcc/locals.tf | 38 +++++++++++++++++++------------------- hpcc/outputs.tf | 4 ++++ lite-variables.tf | 2 +- lite.auto.tfvars.example | 3 +-- scripts/deploy | 17 +++++++++-------- storage/lite-variables.tf | 7 +++---- 8 files changed, 53 insertions(+), 47 deletions(-) diff --git a/hpcc/data.tf b/hpcc/data.tf index ae24590..4a92803 100644 --- a/hpcc/data.tf +++ b/hpcc/data.tf @@ -11,3 +11,8 @@ data "azuread_group" "subscription_owner" { data "azurerm_client_config" "current" { } + +data "local_file" "aks" { + filename = "../aks/data/config.json" +} + diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 0fb9cc8..047d2da 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,16 +21,15 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" - source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" - - depends_on = [ local.get_aks_config ] + source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment productname = local.metadata.product_name internal_domain = local.internal_domain - cluster_name = local.get_aks_config.cluster_name + #cluster_name = local.get_aks_config.cluster_name + cluster_name = jsondecode(file("../aks/data/config.json")).cluster_name hpcc_container = { image_name = local.hpcc_container != null ? local.hpcc_container.image_name : null @@ -45,11 +44,8 @@ module "hpcc" { username = local.hpcc_container_registry_auth.username } : null - hpcc_user_ip_cidr_list = var.hpcc_user_ip_cidr_list - storage_data_gb = var.storage_data_gb - install_blob_csi_driver = false //Disable CSI driver resource_group_name = local.get_aks_config.resource_group_name @@ -63,6 +59,9 @@ module "hpcc" { labels = try(var.hpcc_namespace.labels,{}) } + #----------------------------------------------------------------------- + # Storage variables (internal (ephemeral) or external) + #- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - admin_services_storage_account_settings = { replication_type = local.admin_services_storage_account_settings.replication_type authorized_ip_ranges = merge(local.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) @@ -72,8 +71,10 @@ module "hpcc" { internal_storage_enabled = local.internal_storage_enabled + storage_data_gb = var.storage_data_gb + data_storage_config = { - internal = (local.external_storage_config == null) || (local.internal_storage_enabled == true) ? { + internal = (local.internal_storage_enabled == true) ? { blob_nfs = { data_plane_count = local.data_storage_config.internal.blob_nfs.data_plane_count storage_account_settings = { @@ -85,14 +86,11 @@ module "hpcc" { } } : null - # external = local.internal_data_storage_enabled ? null : { - # blob_nfs = local.get_storage_config != null ? local.get_storage_config.data_storage_planes : local.data_storage_config.external.blob_nfs - # hpcc = null - # } external = null } external_storage_config = local.external_storage_config + #----------------------------------------------------------------------- spill_volumes = local.spill_volumes enable_roxie = var.aks_enable_roxie diff --git a/hpcc/locals.tf b/hpcc/locals.tf index e9ae039..e3fe976 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -4,6 +4,8 @@ locals { AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id } + hpcc_namespace = "default" + names = try(local.disable_naming_conventions, false) ? merge( { business_unit = local.metadata.business_unit @@ -17,30 +19,17 @@ locals { local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - # external_services_storage_exists = fileexists("../storage/data/config.json") || local.external_services_storage_config != null - get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - get_storage_config = fileexists("../storage/data/config.json") ? jsondecode(file("../storage/data/config.json")) : null - - external_storage_exists = fileexists("../storage/data/config.json") || local.external_storage_config != null - - subnet_ids = try({ - for k, v in local.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${local.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${local.use_existing_vnet.name}/subnets/${v.name}" - }, { aks = local.get_vnet_config.private_subnet_id }) - - location = local.use_existing_vnet != null ? local.use_existing_vnet.location : local.get_vnet_config.location - - # hpcc_chart_major_minor_point_version = local.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", local.helm_chart_version) : "master" - domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) + #--------------------------------------------------------------------------------------------------------------------------- + # Setup storage (either external storage of internal (ephemeral) storage + #- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + get_storage_config = fileexists("../storage/data/config.json") ? jsondecode(file("../storage/data/config.json")) : null - internal_storage_enabled = (local.external_storage_exists == true) && (var.external_storage_desired == false) ? true : local.external_storage_exists == true && var.external_storage_desired == true ? false : true - #internal_storage_enabled = local.external_storage_exists == true && var.external_storage_desired == false ? true : local.external_storage_exists == true && var.external_storage_desired == true ? false : true - # external_services_storage_enabled = local.external_services_storage_exists == true && local.external_storage_desired == true ? true : local.external_services_storage_exists == true && local.external_storage_desired == false ? false : true + external_storage_exists = local.external_storage_config != null - #hpcc_namespace = local.hpcc_namespace.existing_namespace != null ? local.hpcc_namespace.existing_namespace : local.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("../logging/data/hpcc_namespace.txt") ? file("../logging/data/hpcc_namespace.txt") : "default" - hpcc_namespace = "default" + internal_storage_enabled = var.external_storage_desired == true ? false : true external_storage_config = local.get_storage_config != null && var.external_storage_desired == true ? [ for plane in local.get_storage_config.external_storage_config : @@ -57,6 +46,17 @@ locals { prefix_name = plane.prefix_name } ] : [] + #--------------------------------------------------------------------------------------------------------------------------- + + subnet_ids = try({ + for k, v in local.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${local.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${local.use_existing_vnet.name}/subnets/${v.name}" + }, { aks = local.get_vnet_config.private_subnet_id }) + + location = local.use_existing_vnet != null ? local.use_existing_vnet.location : local.get_vnet_config.location + + # hpcc_chart_major_minor_point_version = local.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", local.helm_chart_version) : "master" + + domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) svc_domains = { eclwatch = local.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index c028b3f..e75e2f0 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -8,6 +8,10 @@ output "deployment_resource_group" { value = local.get_aks_config.resource_group_name } +output "external_storage_config_exists" { + value = fileexists("../storage/data/config.json") ? true : false +} + resource "local_file" "config" { content = "hpcc successfully deployed" filename = "${path.module}/data/config.json" diff --git a/lite-variables.tf b/lite-variables.tf index 81db473..a58c4bb 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -92,7 +92,7 @@ variable "aks_dns_zone_name" { } variable "hpcc_user_ip_cidr_list" { - description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, Value should be [] add [\"0.0.0.0/0\"]." type = list(string) default = [] } diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 2f56d86..39b5b8a 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -162,8 +162,7 @@ aks_admin_ip_cidr_map={} #------------------------------------------------------------------------------ # List of explicit CIDR addresses that can access this HPCC Systems cluster. -# To allow public access, specify "0.0.0.0/0". -# To add no CIDR addresses, use '[]'. +# To allow public access, value should be ["0.0.0.0/0"] or []. # Value type: list of string hpcc_user_ip_cidr_list=[] diff --git a/scripts/deploy b/scripts/deploy index a663f42..00f9ae7 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -74,9 +74,9 @@ cd $name; # cd into vnet or storage or aks or hpcc if [ -e "../lite.auto.tfvars" ];then # Check if there has been a change since last apply. if [ -e "/tmp/${name}.lite.auto.tfvars" ];then - diff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` + tfvarsdiff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` else - diff="" + tfvarsdiff="" fi else echo "In $0. The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; @@ -98,13 +98,14 @@ fi if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi -#echo "DEBUG: thisdir=\"$thisdir\", repodir=\"$repodir\", name=\"$name\"" + plan=`$thisdir/mkplan deploy_${name}.plan` -#echo "DEBUG: plan=\"$plan\"";exit 1 -if [ "$diff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi -if [ "$name" != "vnet" ];then - cp -v /tmp/${name}.lite.auto.tfvars . -fi + +if [ "$tfvarsdiff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi + +#if [ "$name" != "vnet" ];then +# cp -v /tmp/${name}.lite.auto.tfvars . +#fi echo "=============== Deploy $name. Executing 'terraform init' ==============="; assert_fail terraform init diff --git a/storage/lite-variables.tf b/storage/lite-variables.tf index 7a098d1..e88b5b1 100644 --- a/storage/lite-variables.tf +++ b/storage/lite-variables.tf @@ -30,18 +30,17 @@ variable "aks_azure_region" { variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool + default = false } variable "aks_dns_zone_resource_group_name" { type = string - description = "OPTIONAL: Name of the resource group containing the dns zone." - default = "" + description = "REQUIRED. Name of the resource group containing the dns zone." } variable "aks_dns_zone_name" { type = string - description = "OPTIONAL: dns zone name. The name of existing dns zone." - default = "" + description = "REQUIRED. dns zone name. The name of existing dns zone." } variable "aks_admin_ip_cidr_map" { From fc879fe8ca0339fc988a09c8896db83d9a44286d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 9 Nov 2023 22:07:03 +0000 Subject: [PATCH 050/124] branch:HPCC-27615-easy-deploy-bryan8-pods-assigned-nodepools --- hpcc/hpcc.tf | 3 ++- hpcc/lite-locals.tf | 15 +++++++++++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 047d2da..a496689 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -99,7 +99,8 @@ module "hpcc" { vault_config = local.vault_config eclccserver_settings = local.eclccserver_settings spray_service_settings = local.spray_service_settings - admin_services_node_selector = { all = { workload = local.spray_service_settings.nodeSelector } } + # tlh 20231109 admin_services_node_selector = { all = { workload = local.spray_service_settings.nodeSelector } } + admin_services_node_selector = { all = { workload = "servpool" } } esp_remoteclients = { diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 22b1ae0..867f922 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -130,7 +130,7 @@ locals { spray_service_settings = { replicas = 6 - nodeSelector = "spraypool" #"spraypool" + nodeSelector = "spraypool" } # ldap = { @@ -198,8 +198,9 @@ locals { { disabled = (var.aks_enable_roxie == true)? false : true name = "roxie" - nodeSelector = {} - numChannels = 2 + nodeSelector = { workload = "roxiepool" } + # tlh 20231109 numChannels = 2 + numChannels = 1 prefix = "roxie" replicas = 2 serverReplicas = 0 @@ -351,6 +352,7 @@ locals { cpu = "1" memory = "4G" } + nodeSelector = { workload = "servpool" } egress = "engineEgress" cost = { perCpu = 1 @@ -369,6 +371,7 @@ locals { cpu = "1" memory = "4G" } + nodeSelector = { workload = "servpool" } legacySyntax = false options = [] cost = { @@ -381,6 +384,7 @@ locals { interval = 24 at = "* * * * *" minDeltaSize = 50000 + nodeSelector = { workload = "servpool" } resources = { cpu = "1" memory = "4G" @@ -395,6 +399,7 @@ locals { dfuserver_settings = { maxJobs = 3 + nodeSelector = { workload = "servpool" } resources = { cpu = "1" memory = "2G" @@ -403,6 +408,7 @@ locals { sasha_config = { disabled = false + nodeSelector = { workload = "servpool" } wu-archiver = { disabled = false service = { @@ -483,7 +489,8 @@ locals { maxGraphs = 2 maxGraphStartupTime = 172800 numWorkersPerPod = 1 - nodeSelector = {} + #nodeSelector = {} + nodeSelector = { workload = "thorpool" } egress = "engineEgress" tolerations_value = "thorpool" managerResources = { From 9f522e2984e43ccf8d1d028732ba332522b2f1c7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 10 Nov 2023 13:11:28 +0000 Subject: [PATCH 051/124] branch:HPCC-27615-easy-deploy-bryan8-pods-assigned-nodepools. hpcc.tf sources github repo --- hpcc/hpcc.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index a496689..cac7615 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,8 +21,8 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" - #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment productname = local.metadata.product_name From ba5343297b96811405675e02f5b3bb0cb600c780 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 10 Nov 2023 18:40:02 +0000 Subject: [PATCH 052/124] branch:no-ephemeral-storage-when-external-used --- hpcc/main.tf | 9 +++++++++ hpcc/scripts/delete_ephemeral_storage_accounts | 16 ++++++++++++++++ scripts/destroy | 4 ++++ 3 files changed, 29 insertions(+) create mode 100755 hpcc/scripts/delete_ephemeral_storage_accounts diff --git a/hpcc/main.tf b/hpcc/main.tf index 9f61b5d..c436605 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -30,6 +30,15 @@ module "metadata" { project = local.metadata.project } +resource "null_resource" "delete_ephemeral_storage_accounts" { + count = var.external_storage_desired && (local.external_storage_config != []) ? 1 : 0 + + provisioner "local-exec" { + command = "scripts/delete_ephemeral_storage_accounts ${local.get_aks_config.resource_group_name}" + } + depends_on = [module.hpcc] +} + /*resource "null_resource" "launch_svc_url" { for_each = (module.hpcc.hpcc_status == "deployed") && (local.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} diff --git a/hpcc/scripts/delete_ephemeral_storage_accounts b/hpcc/scripts/delete_ephemeral_storage_accounts new file mode 100755 index 0000000..0b78f2c --- /dev/null +++ b/hpcc/scripts/delete_ephemeral_storage_accounts @@ -0,0 +1,16 @@ +#!/bin/bash +if [ "$1" != "" ];then + rg=$1 +else + echo "$0 has no arguments. It must of 1 argument that is the name of a resource group. EXITING.";exit 1; +fi +echo "In $0. Inputted resource groups is \"$rg\""; +sleep 20; +estos=`az resource list --resource-group $rg|egrep "id\":.*storageAccounts\/hpcc"|sed "s/^ *\"id\": \"//"|sed "s/\", *$//"` +if [ "$estos" == "" ];then + echo "In $0 There are no hpcc storage accounts in the resource group, $rg. EXITING.";exit; +fi +for s in $estos;do + echo "Deleting storage account: $s" + az storage account delete --ids $s -y +done diff --git a/scripts/destroy b/scripts/destroy index 943d204..18c97f3 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -14,6 +14,7 @@ function assert_fail () { #======================================================================== if [ "$1" == "vnet" ];then + assert_fail kubectl delete pods --all --force assert_fail scripts/destroy hpcc assert_fail scripts/destroy aks elif [ "$1" == "aks" ];then @@ -21,6 +22,9 @@ elif [ "$1" == "aks" ];then fi cd $1; name=$(basename `pwd`) +if [ "$name" == "hpcc" ];then + assert_fail kubectl delete pods --all --force +fi if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi From cf68e8148102c878724ac296f4e2cc7fa6d18a3b Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 10 Nov 2023 19:09:00 +0000 Subject: [PATCH 053/124] branch:no-ephemeral-storage-when-external-used. Fixed paths in scripts/destroy --- scripts/destroy | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/destroy b/scripts/destroy index 18c97f3..8ef1f21 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -1,5 +1,7 @@ #!/bin/bash -thisdir=$(d=`dirname $0`;cd $d;pwd;cd ..) +thisdir=$(d=`dirname $0`;cd $d;pwd) +repodir=`echo $thisdir|sed "s/\/scripts//"` +#echo "DEBUG: thisdir=\"$thisdir\", repodir=\"$repodir\", Directory where destroy takes place: \"$repodir/$1\"";exit #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" @@ -20,7 +22,7 @@ if [ "$1" == "vnet" ];then elif [ "$1" == "aks" ];then assert_fail scripts/destroy hpcc fi -cd $1; +cd $repodir/$1; name=$(basename `pwd`) if [ "$name" == "hpcc" ];then assert_fail kubectl delete pods --all --force From 31396de5356f03adfe451e33c8c922ebac0af666 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Sun, 12 Nov 2023 18:46:34 +0000 Subject: [PATCH 054/124] branch:HPCC-27615-easy-deploy-bryan7-developers-documentation. Updated repo's link. --- README.md | 2 +- documentation/hpcc-tf-for-developers.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0aea139..d5309cb 100755 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi 1. If necessary, login to Azure. * From the command line, this is usually accomplished with the `az login` command. 1. Clone this repo to your local system and change current directory. - * `git clone -b HPCC-27615-easy-deploy https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` + * `git clone -b https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` * `cd terraform-azurerm-hpcc-lite` 1. Issue `terraform init` to initialize the Terraform modules. 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index f9121c6..5637e59 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -1,7 +1,7 @@ # For Developers: Tutorial of HPCC Easy Deploy Terraform This tutorial explains the terraform that deploys HPCC Systems on an azure kubernetes service (aks). The terraform was designed to enable one to deploy HPCC Systems easily. -The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite/tree/HPCC-27615-easy-deploy]) +The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite]) From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system from these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. From d17572eaf66a2f285428aa53ff6b0f57d430495a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 16 Nov 2023 18:42:32 +0000 Subject: [PATCH 055/124] branch:HPCC-27615-easy-deploy-bryan9-variable-eclwatch-a-record --- hpcc/hpcc.tf | 5 +++-- hpcc/outputs.tf | 2 +- lite-variables.tf | 4 ++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index cac7615..ace7302 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,12 +21,13 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" - source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" + source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment productname = local.metadata.product_name + a_record_name = var.a_record_name internal_domain = local.internal_domain #cluster_name = local.get_aks_config.cluster_name cluster_name = jsondecode(file("../aks/data/config.json")).cluster_name diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index e75e2f0..655cea3 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -1,6 +1,6 @@ output "eclwatch_url" { description = "Print the ECL Watch URL." - value = format("eclwatch-default.%s:18010",var.aks_dns_zone_name) + value = format("%s.%s:18010",var.a_record_name, var.aks_dns_zone_name) } output "deployment_resource_group" { diff --git a/lite-variables.tf b/lite-variables.tf index a58c4bb..dba7ce8 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -27,8 +27,8 @@ variable "enable_thor" { variable "a_record_name" { type = string - description = "OPTIONAL: dns zone A record name" - default = "" + description = "REQUIRED: dns zone A record name for eclwatch" + default = "eclwatch-default" } variable "aks_admin_email" { From fcc5365efffdb2131a2f16f67e9e94ac5dba0608 Mon Sep 17 00:00:00 2001 From: Timothy L Humphrey Date: Thu, 16 Nov 2023 12:06:28 -0700 Subject: [PATCH 056/124] Update hpcc.tf Make sure 'source' in hpcc.tf is the solutions-lab opinionated hpcc instead of my local copy of it. --- hpcc/hpcc.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index ace7302..ddccb6a 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,8 +21,8 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" - #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment productname = local.metadata.product_name From a76e5247e71588d9d8a2fe465c4732763a9d9f54 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 16 Nov 2023 19:22:22 +0000 Subject: [PATCH 057/124] branch:HPCC-27615-easy-deploy-bryan10-added-hpcc_version --- hpcc/hpcc.tf | 6 +++--- hpcc/lite-variables.tf | 3 ++- lite-variables.tf | 9 +++++---- lite.auto.tfvars.example | 2 +- 4 files changed, 11 insertions(+), 9 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index ace7302..3a9cf47 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -21,16 +21,16 @@ resource "kubernetes_namespace" "hpcc" { module "hpcc" { #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" - #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment productname = local.metadata.product_name a_record_name = var.a_record_name internal_domain = local.internal_domain - #cluster_name = local.get_aks_config.cluster_name cluster_name = jsondecode(file("../aks/data/config.json")).cluster_name + hpcc_version = var.hpcc_version hpcc_container = { image_name = local.hpcc_container != null ? local.hpcc_container.image_name : null diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index 81db473..a14c506 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -98,12 +98,13 @@ variable "hpcc_user_ip_cidr_list" { } variable "hpcc_version" { - description = "REQUIRED. The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported." + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" type = string validation { condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." } + default = "latest" } variable "aks_admin_ip_cidr_map" { diff --git a/lite-variables.tf b/lite-variables.tf index dba7ce8..a14c506 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -27,8 +27,8 @@ variable "enable_thor" { variable "a_record_name" { type = string - description = "REQUIRED: dns zone A record name for eclwatch" - default = "eclwatch-default" + description = "OPTIONAL: dns zone A record name" + default = "" } variable "aks_admin_email" { @@ -92,18 +92,19 @@ variable "aks_dns_zone_name" { } variable "hpcc_user_ip_cidr_list" { - description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, Value should be [] add [\"0.0.0.0/0\"]." + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." type = list(string) default = [] } variable "hpcc_version" { - description = "REQUIRED. The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported." + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" type = string validation { condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." } + default = "latest" } variable "aks_admin_ip_cidr_map" { diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 39b5b8a..3eeb70d 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -30,7 +30,7 @@ aks_dns_zone_resource_group_name=" Date: Fri, 17 Nov 2023 08:19:50 -0600 Subject: [PATCH 058/124] Tims' Modifications Signed-off-by: Dan S. Camper --- .gitignore | 19 +- README.md | 147 ++- aks/aks.tf | 34 +- aks/automation.tf | 10 +- aks/lite-locals.tf | 58 + aks/lite-variables.tf | 64 + aks/locals.tf | 142 +- aks/main.tf | 18 +- aks/misc.auto.tfvars.example | 68 - ...ks.auto.tfvars.example => node_groups.txt} | 60 +- aks/outputs.tf | 2 + aks/variables.tf | 206 +-- documentation/hpcc-tf-for-developers.md | 373 ++++++ hpcc/README.md | 732 ----------- hpcc/dali.auto.tfvars.example | 0 hpcc/data.tf | 5 + hpcc/eclccserver.auto.tfvars.example | 0 hpcc/hpcc.tf | 112 +- hpcc/index.html | 1 + hpcc/lite-locals.tf | 540 ++++++++ hpcc/lite-variables.tf | 212 +++ hpcc/locals.tf | 60 +- hpcc/main.tf | 33 +- hpcc/misc.auto.tfvars.example | 205 --- hpcc/outputs.tf | 23 +- hpcc/roxie.auto.tfvars.example | 11 - hpcc/sasha.auto.tfvars.example | 0 .../scripts/delete_ephemeral_storage_accounts | 16 + hpcc/thor.auto.tfvars.example | 34 - hpcc/variables.tf | 1151 ----------------- lite-variables.tf | 212 +++ lite.auto.tfvars.example | 234 ++++ main.tf | 45 + providers.tf | 5 + scripts/deploy | 119 ++ scripts/destroy | 41 + scripts/external_storage | 16 + scripts/extract-aks-variables | 14 + scripts/get_rg_from_file | 13 + scripts/mkplan | 36 + .../aks/aks.auto.tfvars.example | 19 + .../aks/misc.auto.tfvars.example | 22 + .../storage/storage.auto.tfvars.example | 152 +++ storage/lite-variables.tf | 64 + storage/locals.tf | 21 + storage/main.tf | 10 +- storage/misc.auto.tfvars.example | 23 - storage/outputs.tf | 2 + storage/variables.tf | 50 - vnet/lite-variables.tf | 64 + vnet/locals.tf | 45 +- vnet/main.tf | 22 +- vnet/misc.auto.tfvars.example | 24 - vnet/outputs.tf | 2 + vnet/variables.tf | 61 - 55 files changed, 2875 insertions(+), 2777 deletions(-) mode change 100644 => 100755 README.md create mode 100644 aks/lite-locals.tf create mode 100644 aks/lite-variables.tf delete mode 100644 aks/misc.auto.tfvars.example rename aks/{aks.auto.tfvars.example => node_groups.txt} (62%) create mode 100755 documentation/hpcc-tf-for-developers.md delete mode 100644 hpcc/README.md delete mode 100644 hpcc/dali.auto.tfvars.example delete mode 100644 hpcc/eclccserver.auto.tfvars.example create mode 100644 hpcc/index.html create mode 100755 hpcc/lite-locals.tf create mode 100644 hpcc/lite-variables.tf delete mode 100644 hpcc/misc.auto.tfvars.example delete mode 100644 hpcc/roxie.auto.tfvars.example delete mode 100644 hpcc/sasha.auto.tfvars.example create mode 100755 hpcc/scripts/delete_ephemeral_storage_accounts delete mode 100644 hpcc/thor.auto.tfvars.example delete mode 100644 hpcc/variables.tf create mode 100644 lite-variables.tf create mode 100644 lite.auto.tfvars.example create mode 100644 main.tf create mode 100644 providers.tf create mode 100755 scripts/deploy create mode 100755 scripts/destroy create mode 100755 scripts/external_storage create mode 100755 scripts/extract-aks-variables create mode 100755 scripts/get_rg_from_file create mode 100755 scripts/mkplan create mode 100644 scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example create mode 100644 scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example create mode 100644 scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example create mode 100644 storage/lite-variables.tf delete mode 100644 storage/misc.auto.tfvars.example create mode 100644 vnet/lite-variables.tf delete mode 100644 vnet/misc.auto.tfvars.example diff --git a/.gitignore b/.gitignore index 241c253..3d735d3 100644 --- a/.gitignore +++ b/.gitignore @@ -2,11 +2,11 @@ **/.terraform/* # .tfstate files -*.tfstate -*.tfstate.* +**/*.tfstate +**/*.tfstate.* # Crash log files -crash.log +**/crash.log # Ignore any .tfvars files that are generated automatically for each Terraform run. Most # .tfvars files are managed as part of configuration and so should be included in @@ -14,22 +14,23 @@ crash.log # # example.tfvars *.tfvars -*.json +*/*.tfvars +**/*.json # Ignore data files **/data # Ignore override files as they are usually used to override resources locally and so # are not checked in -override.tf -override.tf.json -*_override.tf -*_override.tf.json +**/override.tf +**/override.tf.json +**/*_override.tf +**/*_override.tf.json # Include override files you do wish to add to version control using negated pattern # # !example_override.tf -.terraform.*.hcl +**/.terraform.*.hcl # Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan # example: *tfplan* diff --git a/README.md b/README.md old mode 100644 new mode 100755 index f3128ac..d5309cb --- a/README.md +++ b/README.md @@ -1,18 +1,129 @@ -# Deploy HPCC Systems Platform with Terraform - -This set of Terraform examples deploys all the available features that come with the HPCC Systems OSS Terraform modules. - -## Order of deployment -| Order | Name | Required | -| ----- | --------- | :------: | -| 1 | `VNet` | yes | -| 2 | `AKS` | yes | -| 3 | `Storage` | no | -| 4 | `Logging` | no | -| 5 | `AKS` | no | -| 6 | `HPCC` | yes | - -## Modules -| Name | Source | Used in | -| :---------------------------------: | :----: | :-----: | -| `terraform-azurerm-virtual-network` | | `VNet` | \ No newline at end of file +# Deploy HPCC Systems on Azure under Kubernetes + +NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). + +This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. + +The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. + +This repo is a fork of the excellent work performed by Godson Fortil. The original can be found at [https://github.com/gfortil/terraform-azurerm-hpcc/tree/HPCC-27615]. + +## Requirements + +* **Terraform** This is a Terraform module, so you need to have Terraform installed on your system. Instructions for downloading and installing Terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of Terraform, as that is needed to accommodate some of the large random numbers used for IDs in the Terraform modules. + +* **helm** Helm is used to deploy the HPCC Systems processes under Kubernetes. Instructions for downloading and installing Helm are at [https://helm.sh/docs/intro/install](https://helm.sh/docs/intro/install/). + +* **kubectl** The Kubernetes client (kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. + +* **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within kubernetes clusters. TL;DR: Make sure you have the command line tools installed. + +* This module will create an AKS cluster in your current **default** Azure subscription. You can view your current subscriptions, and determine which is the default, using the `az account list --output table` command. To set a default subscription, use `az account set --subscription "My_Subscription"`. + +* To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. + +## Installing/Using This Module + +1. If necessary, login to Azure. + * From the command line, this is usually accomplished with the `az login` command. +1. Clone this repo to your local system and change current directory. + * `git clone -b https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` + * `cd terraform-azurerm-hpcc-lite` +1. Issue `terraform init` to initialize the Terraform modules. +1. Decide how you want to supply option values to the module during invocation. There are three possibilities: + 1. Invoke the `terraform apply` command and enter values for each option as Terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. + 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to do that is to copy the example file and then edit the copy: + * `cp lite.auto.tfvars.example lite.auto.tfvars` + 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. +1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). + +At the end of a successful deployment these items are output: +* The URL used to access ECL Watch. +* The deployment azure resource group. + +## Available Options + +Options have data types. The ones used in this module are: +* string + * Typical string enclosed by quotes + * Example + * `"value"` +* number + * Integer number; do not quote + * Example + * `1234` +* boolean + * true or false (not quoted) +* map of string + * List of key/value pairs, delimited by commas + * Both key and value should be a quoted string + * Entire map is enclosed by braces + * Example with two key/value pairs + * `{"key1" = "value1", "key2" = "value2"}` + * Empty value is `{}` +* list of string + * List of values, delimited by commas + * A value is a quoted string + * Entire list is enclosed in brackets + * Example with two values + * `["value1", "value2"]` + * Empty value is `[]` + +The following options should be set in your `lite.auto.tfvars` file (or entered interactively, if you choose to not create a file). Only a few of them have default values. The rest are required. The 'Updateable' column indicates whether, for any given option, it is possible to successfully apply the update against an already-running HPCC k8s cluster. + +|Option|Type|Description| +|:-----|:---|:----------| +| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | +| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | +| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | +| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | +| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | +| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | +| `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | +| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | +| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | +| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | +| `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | +| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | +| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | +| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | +| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | +| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | +| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | +| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | +| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | + +## Persistent Storage + +To get persistent storage, i.e. storage that is not deleted when the hpcc cluster is deleted, set the variable, external_storage_desired, to true. + +## Useful Things + +* Useful `kubectl` commands once the cluster is deployed: + * `kubectl get pods` + * Shows Kubernetes pods for the current cluster. + * `kubectl get services` + * Show the current services running on the pods on the current cluster. + * `kubectl config get-contexts` + * Show the saved kubectl contexts. A context contains login and reference information for a remote Kubernetes cluster. A kubectl command typically relays information about the current context. + * `kubectl config use-context ` + * Make \ context the current context for future kubectl commands. + * `kubectl config unset contexts.` + * Delete context named \. + * Note that when you delete the current context, kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. +* Note that `terraform destroy` does not delete the kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. +* If a deployment fails and you want to start over, you have two options: + * Immediately issue a `terraform destroy` command and let Terraform clean up. + * Clean up the resources by hand: + * Delete the Azure resource group manually, such as through the Azure Portal. + * Note that there are two resource groups, if the deployment got far enough. Examples: + * `app-thhpccplatform-sandbox-eastus-68255` + * `mc_tf-zrms-default-aks-1` + * The first one contains the Kubernetes service that created the second one (services that support Kubernetes). So, if you delete only the first resource group, the second resource group will be deleted automatically. + * Delete all Terraform state files using `rm *.tfstate*` + * Then, of course, fix whatever caused the deployment to fail. +* If you want to completely reset Terraform, issue `rm -rf .terraform* *.tfstate*` and then `terraform init`. diff --git a/aks/aks.tf b/aks/aks.tf index 3c317a0..09e6ba4 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -20,8 +20,13 @@ module "aks" { depends_on = [random_string.string] - source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - # source = "../../../terraform-azurerm-aks" + #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=OSS" + #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" + #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" + #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" providers = { kubernetes = kubernetes.default @@ -29,7 +34,7 @@ module "aks" { kubectl = kubectl.default } - location = var.metadata.location + location = local.metadata.location resource_group_name = module.resource_groups["azure_kubernetes_service"].name cluster_name = local.cluster_name @@ -38,6 +43,8 @@ module "aks" { # for v1.6.2 aks: sku_tier_paid = false sku_tier = var.sku_tier + logging_monitoring_enabled = var.aks_logging_monitoring_enabled + cluster_endpoint_access_cidrs = var.cluster_endpoint_access_cidrs virtual_network_resource_group_name = try(var.use_existing_vnet.resource_group_name, local.get_vnet_config.resource_group_name) @@ -45,25 +52,25 @@ module "aks" { subnet_name = try(var.use_existing_vnet.subnets.aks.name, "aks-hpcc-private") route_table_name = try(var.use_existing_vnet.route_table_name, local.get_vnet_config.route_table_name) - dns_resource_group_lookup = { "${var.internal_domain}" = var.dns_resource_group } + dns_resource_group_lookup = { "${local.internal_domain}" = local.dns_resource_group } admin_group_object_ids = [data.azuread_group.subscription_owner.object_id] rbac_bindings = var.rbac_bindings availability_zones = var.availability_zones - node_groups = var.node_groups + node_groups = local.node_groups core_services_config = { - alertmanager = var.core_services_config.alertmanager - coredns = var.core_services_config.coredns - external_dns = var.core_services_config.external_dns - cert_manager = var.core_services_config.cert_manager + alertmanager = local.core_services_config.alertmanager + coredns = local.core_services_config.coredns + external_dns = local.core_services_config.external_dns + cert_manager = local.core_services_config.cert_manager ingress_internal_core = { - domain = var.core_services_config.ingress_internal_core.domain - subdomain_suffix = "${var.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(var.owner.name)}" // dns record suffix - public_dns = var.core_services_config.ingress_internal_core.public_dns + domain = local.core_services_config.ingress_internal_core.domain + subdomain_suffix = "${local.core_services_config.ingress_internal_core.subdomain_suffix}${trimspace(local.owner_name_initials)}" // dns record suffix + public_dns = local.core_services_config.ingress_internal_core.public_dns } } @@ -75,9 +82,12 @@ module "aks" { } logging = var.logging + #logging = null experimental = { oms_agent = var.hpcc_log_analytics_enabled || var.experimental.oms_agent oms_agent_log_analytics_workspace_id = fileexists("../logging/data/workspace_resource_id.txt") ? file("../logging/data/workspace_resource_id.txt") : var.experimental.oms_agent_log_analytics_workspace_id != null ? var.experimental.oms_agent_log_analytics_workspace_id : null + #tlh tried this oms_agent = null + #tlh tried this oms_agent_log_analytics_workspace_id = null } } diff --git a/aks/automation.tf b/aks/automation.tf index 45e80f9..382bd7f 100644 --- a/aks/automation.tf +++ b/aks/automation.tf @@ -1,11 +1,12 @@ +/* resource "azurerm_automation_account" "automation_account" { - name = var.aks_automation.automation_account_name + name = local.aks_automation.automation_account_name location = local.location resource_group_name = module.resource_groups["azure_kubernetes_service"].name sku_name = var.sku_name tags = local.tags - # local_authentication_enabled = var.aks_automation.local_authentication_enabled - public_network_access_enabled = var.aks_automation.public_network_access_enabled + # local_authentication_enabled = local.aks_automation.local_authentication_enabled + public_network_access_enabled = local.aks_automation.public_network_access_enabled identity { type = "SystemAssigned" @@ -60,6 +61,7 @@ resource "azurerm_automation_job_schedule" "job_schedule" { resourcename = module.aks.cluster_name resourcegroupname = module.resource_groups["azure_kubernetes_service"].name operation = each.value.operation - automationaccount = var.aks_automation.automation_account_name + automationaccount = local.aks_automation.automation_account_name } } +*/ diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf new file mode 100644 index 0000000..57c7a99 --- /dev/null +++ b/aks/lite-locals.tf @@ -0,0 +1,58 @@ +locals { + internal_domain = var.aks_dns_zone_name + + dns_resource_group = var.aks_dns_zone_resource_group_name + + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + core_services_config = { + alertmanager = { + smtp_host = "smtp-hostname.ds:25" + smtp_from = var.aks_admin_email + routes = [] + receivers = [] + } + + # coredns = { + # forward_zones = { + # "" = "" + # } + # } + coredns = {} + + external_dns = { + public_domain_filters = [var.aks_dns_zone_name] + } + + # cert_manager = { + # acme_dns_zones = [var.aks_dns_zone_name] + # default_issuer_name = "zerossl" + # } + cert_manager = {} + + ingress_internal_core = { + domain = var.aks_dns_zone_name + subdomain_suffix = "hpcc" // dns record suffix //must be unique accross subscription + public_dns = true + } + } +} diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf new file mode 100644 index 0000000..e88b5b1 --- /dev/null +++ b/aks/lite-variables.tf @@ -0,0 +1,64 @@ +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "REQUIRED. Name of the resource group containing the dns zone." +} + +variable "aks_dns_zone_name" { + type = string + description = "REQUIRED. dns zone name. The name of existing dns zone." +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/aks/locals.tf b/aks/locals.tf index c8cb340..03fe395 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -1,4 +1,124 @@ +resource "random_string" "name" { + length = 3 + special = false + numeric = false + upper = false +} + locals { + roxiepool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" + node_type_version = "v2" + #node_size = "2xlarge" + node_size = "large" + single_group = false + min_capacity = 1 + max_capacity = 3 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "roxiepool" + } + taints = [] + tags = {} + } + + node_groups0 = { + thorpool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" # gp, gpd, mem, memd, stor + node_type_version = "v2" # v1, v2 + #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + single_group = false + min_capacity = 3 + max_capacity = 6 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "thorpool" + } + taints = [] + tags = {} + }, + + servpool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gpd" + node_type_version = "v1" + #node_size = "4xlarge" + node_size = "2xlarge" + single_group = false + min_capacity = 1 + max_capacity = 3 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "servpool" + } + taints = [] + tags = {} + }, + + spraypool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" + node_type_version = "v1" + node_size = "2xlarge" + #node_size = "1xlarge" # NOT ALLOWED + #node_size = "4xlarge" + single_group = false + min_capacity = 3 + max_capacity = 6 + # placement_group_key = null + labels = { + "lnrs.io/tier" = "standard" + "workload" = "spraypool" + "spray-service" = "spraypool" + } + taints = [] + tags = {} + } + } + + node_groups = var.aks_enable_roxie? merge( local.node_groups0, { roxiepool = local.roxiepool } ) : local.node_groups0 + + aks_automation = { + local_authentication_enabled = false + public_network_access_enabled = false + automation_account_name = "aks-stop-demo-${random_string.name.result}" + + schedule = [ + { + schedule_name = "aks_stop" + description = "Stops the AKS weekday nights at 6PM MST" + runbook_name = "aks_startstop_runbook" + frequency = "Week" //OneTime, Day, Hour, Week, or Month. + interval = "1" //cannot be set when frequency is `OneTime` + operation = "stop" + daylight_saving = true + start_time = "20:00" // At least 5 minutes in the future + week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] + }, + # { + # schedule_name = "aks_start" + # description = "Starts the AKS weekday nights at 6AM EST" + # runbook_name = "aks_startstop_runbook" + # frequency = "Week" //OneTime, Day, Hour, Week, or Month. + # interval = "1" //cannot be set when frequency is `OneTime` + # operation = "start" + # daylight_saving = true + # start_time = "06:00" // At least 5 minutes in the future + # week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] + # } + ] + } + azure_auth_env = { AZURE_TENANT_ID = data.azurerm_client_config.current.tenant_id AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id @@ -6,18 +126,18 @@ locals { names = var.disable_naming_conventions ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.metadata.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) + tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null @@ -47,10 +167,10 @@ locals { tomorrow = formatdate("YYYY-MM-DD", timeadd(local.current_time, "24h")) # today = formatdate("YYYY-MM-DD", timeadd(local.current_time, "1h")) - utc_offset = var.aks_automation.schedule[0].daylight_saving ? 4 : 5 + utc_offset = local.aks_automation.schedule[0].daylight_saving ? 4 : 5 script = { for item in fileset("${path.root}/scripts", "*") : (item) => file("${path.root}/scripts/${item}") } - schedule = { for s in var.aks_automation.schedule : "${s.schedule_name}" => s } + schedule = { for s in local.aks_automation.schedule : "${s.schedule_name}" => s } az_command = "az aks get-credentials --name ${local.cluster_name} --resource-group ${module.resource_groups["azure_kubernetes_service"].name} --admin --overwrite-existing" is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true diff --git a/aks/main.tf b/aks/main.tf index cfbe822..f96d390 100644 --- a/aks/main.tf +++ b/aks/main.tf @@ -24,17 +24,17 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market + market = local.metadata.market location = local.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = module.subscription.output.subscription_id - project = var.metadata.project + project = local.metadata.project } module "resource_groups" { diff --git a/aks/misc.auto.tfvars.example b/aks/misc.auto.tfvars.example deleted file mode 100644 index c1040d3..0000000 --- a/aks/misc.auto.tfvars.example +++ /dev/null @@ -1,68 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -resource_groups = { - azure_kubernetes_service = { - tags = { "enclosed resource" = "open source aks" } - } - # azure_log_analytics_workspace = { - # tags = { "enclosed resource" = "azure log analytics workspace" } - # } -} - -# # auto_connect - Automatically connect to the kubernetes cluster from the host machine. -auto_connect = true - -# # disable_naming_conventions - Disable naming conventions -# # disable_naming_conventions = true - -# azure_auth = { -# # AAD_CLIENT_ID = "" -# # AAD_CLIENT_SECRET = "" -# # AAD_TENANT_ID = "" -# # AAD_PRINCIPAL_ID = "" -# SUBSCRIPTION_ID = "" -# } - -aks_automation = { - local_authentication_enabled = false - public_network_access_enabled = false - automation_account_name = "aks-stop-demo-5" - - schedule = [ - { - schedule_name = "aks_stop" - description = "Stops the AKS weekday nights at 6PM EST" - frequency = "Week" //OneTime, Day, Hour, Week, or Month. - interval = "1" //cannot be set when frequency is `OneTime` - daylight_saving = true - start_time = "18:00" // At least 5 minutes in the future - week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - }, - # { - # schedule_name = "aks_start" - # description = "Starts the AKS weekday nights at 6AM EST" - # frequency = "Week" //OneTime, Day, Hour, Week, or Month. - # interval = "1" //cannot be set when frequency is `OneTime` - # daylight_saving = true - # start_time = "06:00" // At least 5 minutes in the future - # week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - # } - ] -} \ No newline at end of file diff --git a/aks/aks.auto.tfvars.example b/aks/node_groups.txt similarity index 62% rename from aks/aks.auto.tfvars.example rename to aks/node_groups.txt index ab557cf..b3623c6 100644 --- a/aks/aks.auto.tfvars.example +++ b/aks/node_groups.txt @@ -1,55 +1,22 @@ -cluster_version = "1.25" +cluster_version = "1.26" cluster_ordinal = 1 //cluster name suffix -sku_tier = "free" -dns_resource_group = "app-dns-prod-eastus2" -internal_domain = "my-dns-zone.io" +sku_tier = "FREE" hpcc_log_analytics_enabled = false rbac_bindings = { cluster_admin_users = { # "service_principal1" = "", # "user1" = "" - # "user2" = "" - + "admin" = "35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9" } cluster_view_users = {} cluster_view_groups = [] } -core_services_config = { - alertmanager = { - smtp_host = "smtp-hostname.ds:25" - smtp_from = "demo@lexisnexisrisk.com" - routes = [] - receivers = [] - } - - # coredns = { - # forward_zones = { - # "" = "" - # } - # } - - external_dns = { - public_domain_filters = ["my-dns-zone.io"] - } - - # cert_manager = { - # acme_dns_zones = ["my-dns-zone.io"] - # default_issuer_name = "zerossl" - # } - - ingress_internal_core = { - domain = "my-dns-zone.io" - subdomain_suffix = "hpcc" // dns record suffix //must be unique accross subscription - public_dns = true - } -} - cluster_endpoint_access_cidrs = ["0.0.0.0/0"] -availability_zones = [1,2] +availability_zones = [1] node_groups = { thorpool = { @@ -57,10 +24,11 @@ node_groups = { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = 3 + max_capacity = 6 # placement_group_key = null labels = { "lnrs.io/tier" = "standard" @@ -75,7 +43,8 @@ node_groups = { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - node_size = "2xlarge" + #node_size = "2xlarge" + node_size = "large" single_group = false min_capacity = 1 max_capacity = 3 @@ -93,7 +62,8 @@ node_groups = { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - node_size = "4xlarge" + #node_size = "4xlarge" + node_size = "2xlarge" single_group = false min_capacity = 1 max_capacity = 3 @@ -112,9 +82,11 @@ node_groups = { node_type = "gp" node_type_version = "v1" node_size = "2xlarge" + #node_size = "1xlarge" # NOT ALLOWED + #node_size = "4xlarge" single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = 3 + max_capacity = 6 # placement_group_key = null labels = { "lnrs.io/tier" = "standard" diff --git a/aks/outputs.tf b/aks/outputs.tf index 5f97637..2926b6b 100644 --- a/aks/outputs.tf +++ b/aks/outputs.tf @@ -23,4 +23,6 @@ output "cluster_resource_group_name" { resource "local_file" "output" { content = local.config filename = "${path.module}/data/config.json" + + depends_on = [ module.aks ] } diff --git a/aks/variables.tf b/aks/variables.tf index d2fa6f6..a44d7fe 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -1,20 +1,8 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } +variable "tags" { + description = "Tags to apply to all resources." + type = map(string) + nullable = false + default = {} } # variable "azure_auth" { @@ -42,37 +30,6 @@ variable "disable_naming_conventions" { default = false } -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - variable "resource_groups" { description = "Resource group module variables." type = any @@ -105,18 +62,6 @@ variable "use_existing_vnet" { default = null } -## DNS -######### -variable "internal_domain" { - description = "DNS Domain name" - type = string -} - -variable "dns_resource_group" { - description = "DNS resource group name" - type = string -} - ## Other AKS Vars ################## variable "cluster_ordinal" { @@ -132,14 +77,14 @@ variable "cluster_version" { } variable "sku_tier" { - description = "Pricing tier for the Azure Kubernetes Service managed cluster; \"free\" & \"paid\" are supported. For production clusters or clusters with more than 10 nodes this should be set to \"paid\"." + description = "Pricing tier for the Azure Kubernetes Service managed cluster; \"FREE\" & \"PAID\" are supported. For production clusters or clusters with more than 10 nodes this should be set to \"PAID\"." type = string nullable = false - default = "free" + default = "FREE" validation { - condition = contains(["free", "paid"], var.sku_tier) - error_message = "Available SKU tiers are \"free\" or \"paid\"." + condition = contains(["FREE", "PAID"], var.sku_tier) + error_message = "Available SKU tiers are \"FREE\" or \"PAID\"." } } @@ -154,118 +99,6 @@ variable "rbac_bindings" { default = {} } -variable "node_groups" { - description = "Node groups to configure." - type = map(object({ - node_arch = optional(string) - node_os = optional(string) - node_type = optional(string) - node_type_variant = optional(string) - node_type_version = optional(string) - node_size = string - single_group = optional(bool) - min_capacity = optional(number) - max_capacity = number - os_config = optional(map(any)) - ultra_ssd = optional(bool) - placement_group_key = optional(string) - max_pods = optional(number) - max_surge = optional(string) - labels = optional(map(string)) - taints = optional(list(object({ - key = string - value = string - effect = string - }))) - tags = optional(map(string)) - })) - nullable = false - default = {} -} - -variable "core_services_config" { - description = "Core service configuration." - type = object({ - alertmanager = object({ - smtp_host = string - smtp_from = string - receivers = optional(list(object({ - name = string - email_configs = optional(any, []) - opsgenie_configs = optional(any, []) - pagerduty_configs = optional(any, []) - pushover_configs = optional(any, []) - slack_configs = optional(any, []) - sns_configs = optional(any, []) - victorops_configs = optional(any, []) - webhook_configs = optional(any, []) - wechat_configs = optional(any, []) - telegram_configs = optional(any, []) - }))) - routes = optional(list(object({ - receiver = string - group_by = optional(list(string)) - continue = optional(bool) - matchers = list(string) - group_wait = optional(string) - group_interval = optional(string) - repeat_interval = optional(string) - mute_time_intervals = optional(list(string)) - # active_time_intervals = optional(list(string)) - }))) - }) - cert_manager = optional(object({ - acme_dns_zones = optional(list(string)) - additional_issuers = optional(map(any)) - default_issuer_kind = optional(string) - default_issuer_name = optional(string) - })) - coredns = optional(object({ - forward_zones = optional(map(any)) - })) - external_dns = optional(object({ - additional_sources = optional(list(string)) - private_domain_filters = optional(list(string)) - public_domain_filters = optional(list(string)) - })) - fluentd = optional(object({ - image_repository = optional(string) - image_tag = optional(string) - additional_env = optional(map(string)) - debug = optional(bool) - filters = optional(string) - route_config = optional(list(object({ - match = string - label = string - copy = optional(bool) - config = string - }))) - routes = optional(string) - outputs = optional(string) - })) - grafana = optional(object({ - admin_password = optional(string) - additional_plugins = optional(list(string)) - additional_data_sources = optional(list(any)) - })) - ingress_internal_core = optional(object({ - domain = string - subdomain_suffix = optional(string) - lb_source_cidrs = optional(list(string)) - lb_subnet_name = optional(string) - public_dns = optional(bool) - })) - prometheus = optional(object({ - remote_write = optional(any) - })) - storage = optional(object({ - file = optional(bool, true) - blob = optional(bool, false) - }), {}) - }) - nullable = false -} - variable "experimental" { description = "Configure experimental features." type = object({ @@ -298,27 +131,6 @@ variable "runbook" { default = [{}] } -variable "aks_automation" { - description = "Arguments to automate the Azure Kubernetes Cluster" - type = object({ - automation_account_name = string - local_authentication_enabled = optional(bool, false) - public_network_access_enabled = optional(bool, false) - - schedule = list(object({ - description = optional(string, "Stop the Kubernetes cluster.") - schedule_name = optional(string, "aks_stop") - runbook_name = optional(string, "aks_startstop_runbook") # name of the runbook - frequency = string - interval = string - start_time = string - week_days = list(string) - operation = optional(string, "stop") - daylight_saving = optional(bool, false) - })) - }) -} - variable "timezone" { description = "Name of timezone" type = string diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md new file mode 100755 index 0000000..5637e59 --- /dev/null +++ b/documentation/hpcc-tf-for-developers.md @@ -0,0 +1,373 @@ +# For Developers: Tutorial of HPCC Easy Deploy Terraform + +This tutorial explains the terraform that deploys HPCC Systems on an azure kubernetes service (aks). The terraform was designed to enable one to deploy HPCC Systems easily. +The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite]) + +From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system from these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. + +The following sections will explain the terraform in root directory and all subdirectories. + +## Root Directory +Here is the root directory's contents (**blue** names are subdirectories) and a description of each entry: + + +|Entry Name|Description| +|:-----|:----------| +| `lite-variables.tf` | Contains all input variables | +| `lite.auto.tfvars.example` |Is an example .auto.tfvars file | +| `main.tf` | Contains most of the terraform that deploys all components of system | +| `providers.tf` | Contains one provider, azurerm | +| `scripts` | Directory containing scripts used in deployment | +| `aks` | Directory containing terraform to deploy `aks` | +| `hpcc` | Directory containing terraform to deploy `hpcc` | +| `storage` | Directory containing terraform to deploy external or persistent `storage` | +| `vnet` | Directory containing terraform to deploy virtual network used by `aks` | + + +The following table shows all the variables in the file, `lite-variables.tf`, and their types. Plus, the table gives a description of each variable. Also, when one deploys from the root directory the `deploy` script puts these variables (or some of them) in the subdirectory where the deployment takes place. + +|Variable|Type|Description| +|:-----|:---|:----------| +| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | +| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | +| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | +| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | +| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | +| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | +| `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | +| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | +| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | +| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | +| `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | +| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | +| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | +| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | +| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | +| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | +| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | +| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | +| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | +| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | + + +The following table gives the name of each of the 5 `null_resource` in `main.tf` and gives a short description of what each does. + +|null_resource name|description| +|:-----------------|:----------| +| `deploy_vnet` | deploys aks' virtual network | +| `deploy_aks` | deploys aks | +| `deploy_storage` | deploys persistent storage | +| `external_storage` | waits for deployment of presistent storage | +| `deploy_hpcc` | deploys hpcc | + +The subfolders, except for `scripts`, create components needed by the full system. + +## scripts subdirectory + +|scripts subdirectory entry name|description| +|:--------------------------------|:----------| +| `deploy` | Used by each of the `deploy` `null_resource`s in main.tf. This script deploys any of the components, i.e. aks, hpcc, storage, or vnet | +| `destroy` | Destroys a single component, i.e. aks, hpcc, storage, or vnet. This script destorys 1) the component whose name is given on the command line after `deploy`, e.g. `destroy vnet`, and 2) any components that depends on it, e.g. before `vnet` is destroyed both `hpcc` and `aks` would be destroyed. | +| `external_storage` | Waits for presistent storage to be created (or if ephemeral storage is used this scripts exits) NOTE: HPCC is not deployed until `external_storage` exits successfully. | +| `extract-aks-variables` | the `deploy` script uses this script to copy from root directory the `lite-variables.tf` file contents used to deploy aks. | +| `get_rg_from_file` | Outputs the resource group name in the `config.json` file given on the command line | +| `mkplan` | Makes a unique name for the file that will contain the terraform plan of a component being deployed. | +| `needed-auto-tfvars-files` | Directory containing .auto.tfvars files needed by the `aks` and `storage` components. | + +## aks subdirectory + +The following table tells what files and subdirectories and in the `aks` subdirectory. The deployment of an `aks`happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an `aks`. Also, if you deploy `aks` manually you do it from this directory. + +What is deployed by this subdirectory and their order is given in [_Appendix A_](#Appendix-A). + + +|aks subdirectory entry name|description| +|:------------------------------|:----------| +| `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` is one of this file's variables which contains the variable, `my_azure_id` which is the object id of the user's azure account. This variable is given its value by the script `deploy`.| +| `aks.tf` | This file contains most of the terraform needed to deploy `aks`. The main module in this file is the `aks` module. | +| `automation.tf` | This file contains the terraform for scheduling the stopping and/or starting of the kubernetes cluster. | +| `data`<\font> | This directory and its contents, `config.json`, are created after the `aks` cluster is successfully deployed. | +| `data.tf` | This file contains `data` statements that gets resources needed that already exist. | +| `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform was forked, all the variables in this file were input variables defined in `variables.tf`. | +| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `aks`. This file was copied to the `aks` directory by the `deploy` script. | +| `lite.auto.tfvars` | This file contains all the variables (and their values) whose name beings with `aks_`. These variables and their values are copied from the lite.auto.tfvars file in the root directory. The copy is done by the script, `deploy`. | +| `locals.tf` | This file contains local variables that were originally in Godson Fortil's repository. | +| `main.tf` | This file contains resources and modules needed for the deployment. They are: `resource "random_integer" "int`, `resource "random_string" "string`, `module "subscription`, `module "naming`, `module "metadata`, `module "resource_groups`, `resource "null_resource" "az`. | +| `misc.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. | +| `outputs.tf` | This file contains `output` statement which outputs the following: `advisor_recommendations`,`aks_login`,`cluster_name`,`hpcc_log_analytics_enabled`,`cluster_resource_group_name`. | +| `providers.tf` | This file contains the following providers: `azurerm`,`azuread`,`kubernetes`,`kubernetes`,`kubectl`,`kubectl`,`helm`,`helm`,`shell`. | +| `variables.tf` | This file contains the variables described in the next table. | +| `versions.tf` | This file gives the version needed of each provider. | + +## hpcc subdirectory + +The following table tells what files and subdirectories and in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. + +What is deployed by this subdirectory and their order is given in [_Appendix B_](#Appendix-B). + +|hpcc subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `hpcc.tf` | Contains the `hpcc` module which does most of the work of deploying an hpcc cluster. | +| `lite-locals.tf` | Contains variables that use lite-variables.tf variables. The contents was in .auto.tfvars of Godson's terraform-azurerm-hpcc, branch HPCC-27615 (which this terraform is a fork). | +| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | +| `lite.auto.tfvars` | Contains alls the variables used for easy deployment with values of the user. This file is copied in the hpcc directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of the hpcc cluster. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains modules and resources needed for the deployment of the hpcc cluster. | +| `outputs.tf` | Contains output statements that show the user important information, like the eclwatch url and the resource group used by most of the resources. | +| `providers.tf` | Contains providers needed for the hpcc cluster deployment. Also, some of these providers (`kubernetes` and `kubectl`) get credentials for the kubernetes cluster for authenication. | +| `versions.tf` | Contains the versions needed for all providers. | +| `data` | This directory contains the file `config.json` which is created when the hpcc cluster successfully deploys. | + +## storage subdirectory + +The following table tells what files and subdirectories and in the `storage` subdirectory. The deployment of an `storage` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `storage`. Also, if you deploy `storage` manually you do it from this directory. + +What is deployed by this subdirectory and their order is given in [_Appendix C_](#Appendix-C). + +|storage subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `storage`. This file was copied to the `aks` directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of the `storage`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains only the `storage` module | +| `outputs.tf` | Contains only the resource `local_file` which outputs to a file config.json. This is done only when on a successful deployment of `storage`. | +| `providers.tf` | Contains only 2 providers: azurerm and azuread | +| `storage.auto.tfvars` | Contains variables that describe the storage accounts that are created. This file is copied to the `storage` directory by the `deploy` script. | +| `variables.tf` | Contains variables needed for `storage` deployment. | +| `versions.tf` | dummy description text | +| `data` | This directory contains the file `config.json` which is created when the external storage successfully deploys. | + +## vnet subdirectory + + +The following table tells what files and subdirectories are in the `vnet` subdirectory. The deployment of an `vnet` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `vnet`. Also, if you deploy `vnet` manually you do it from this directory. + +What is deployed by this subdirectory and their order is given in [_Appendix D_](#Appendix-D). + +|vnet subdirectory entry name|description| +|:--------------------------------|:----------| +| `data.tf` | Contains `data` statements providing information about existing resources. | +| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | +| `lite.auto.tfvars` | Contains all the variables used for easy deployment with values of the user. This file is copied in the `vnet` directory by the `deploy` script. | +| `locals.tf` | Contains local variables used in the deployment of `vnet`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | +| `main.tf` | Contains modules and resources needed for the deployment of `vnet` | +| `outputs.tf` | Contains several output statements that output important information to the deployer. Also, this file contains an output state that outputs a file, config.json. This file is only output if there is a successful deployment of `vnet`. | +| `providers.tf` | Contains 2 providers: `random` and `azurerm`. | +| `variables.tf` | Contains only one variable, `disable_naming_conventions`. | +| `versions.tf` | Contains the required versions of `terraform`, `azurerm` and `random`. | +| `vnet.tf` | Contains the module `virtual_network` which deploys the virtual network used by `aks`, `hpcc`, and `storage`. | +| `data` | This directory contains the file `config.json` which is created when the `vnet` is successfully deploys. | + +​ + +## Appendix A + + +|Resources Created by aks Deployment| +|:------------------------------------------------------------------------------------------------| +| `data.azuread_group.subscription_owner` | +| `data.azurerm_advisor_recommendations.advisor` | +| `data.azurerm_client_config.current` | +| `data.azurerm_subscription.current` | +| `data.http.host_ip` | +| `local_file.output` | +| `null_resource.az[0]` | +| `random_integer.int` | +| `random_string.name` | +| `random_string.string` | +| `module.aks.data.azurerm_subscription.current` | +| `module.aks.kubernetes_config_map.terraform_modules` | +| `module.aks.kubernetes_config_map_v1_data.terraform_modules` | +| `module.aks.terraform_data.creation_metadata` | +| `module.aks.terraform_data.immutable_inputs` | +| `module.aks.time_static.timestamp` | +| `module.aks.module.cluster.data.azurerm_client_config.current` | +| `module.aks.module.cluster.data.azurerm_kubernetes_cluster.default` | +| `module.aks.module.cluster.data.azurerm_kubernetes_service_versions.default` | +| `module.aks.module.cluster.data.azurerm_monitor_diagnostic_categories.default` | +| `module.aks.module.cluster.data.azurerm_public_ip.outbound[0]` | +| `module.aks.module.cluster.azurerm_kubernetes_cluster.default` | +| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_network` | +| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_route_table[0]` | +| `module.aks.module.cluster.azurerm_user_assigned_identity.default` | +| `module.aks.module.cluster.terraform_data.maintenance_control_plane_start_date` | +| `module.aks.module.cluster.terraform_data.maintenance_nodes_start_date` | +| `module.aks.module.cluster.time_sleep.modify` | +| `module.aks.module.cluster_version_tag.shell_script.default` | +| `module.aks.module.core_config.kubernetes_labels.system_namespace["default"]` | +| `module.aks.module.core_config.kubernetes_labels.system_namespace["kube-system"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["cert-manager"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["dns"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["ingress-core-internal"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["logging"]` | +| `module.aks.module.core_config.kubernetes_namespace.default["monitoring"]` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_cluster` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_node` | +| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_virtual_machine_contributor_node` | +| `module.aks.module.core_config.module.aad_pod_identity.helm_release.aad_pod_identity` | +| `module.aks.module.core_config.module.aad_pod_identity.time_sleep.finalizer_wait` | +| `module.aks.module.core_config.module.cert_manager.helm_release.default` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt_staging"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["zerossl"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["configmap-dashboard-cert-manager.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["poddistributionbudget-cert-manager-webhook.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["prometheusrule-certmanager.yaml"]` | +| `module.aks.module.core_config.module.cert_manager.kubernetes_secret.zerossl_eabsecret` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_federated_identity_credential.default["system:serviceaccount:cert-manager:cert-manager"]` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_role_assignment.default[0]` | +| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_user_assigned_identity.default` | +| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_files["prometheusrule-coredns.yaml"]` | +| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_objects["coredns_custom"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureassignedidentities.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentities.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentitybindings.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azurepodidentityexceptions.aadpodidentity.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificaterequests.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificates.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["challenges.acme.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["clusterissuers.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["issuers.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["orders.acme.cert-manager.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["external-dns"].kubectl_manifest.crds["dnsendpoints.externaldns.k8s.io.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagerconfigs.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagers.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["podmonitors.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["probes.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusagents.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheuses.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusrules.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["scrapeconfigs.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["servicemonitors.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["thanosrulers.monitoring.coreos.com.yaml"]` | +| `module.aks.module.core_config.module.external_dns.helm_release.public[0]` | +| `module.aks.module.core_config.module.external_dns.kubectl_manifest.resource_files["configmap-dashboard-external-dns.yaml"]` | +| `module.aks.module.core_config.module.external_dns.kubernetes_secret.public_config[0]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_federated_identity_credential.default["system:serviceaccount:dns:external-dns-public"]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[0]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[1]` | +| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_user_assigned_identity.default` | +| `module.aks.module.core_config.module.ingress_internal_core.helm_release.default` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.certificate` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["configmap-dashboard-ingress-nginx-core-internal.yaml"]` | +| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["prometheusrule-ingress-nginx-core-internal.yaml"]` | +| `module.aks.module.core_config.module.ingress_internal_core.time_sleep.lb_detach` | +| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0.shell_script.default` | +| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0-rc_1.shell_script.default` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-retain"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-retain"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-delete"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-ephemeral"]` | +| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-retain"]` | +| `module.aks.module.node_groups.module.bootstrap_node_group_hack.shell_script.default` | +| `module.aks.module.node_groups.module.system_node_groups["system1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["servpool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["spraypool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.node_groups.module.user_node_groups["thorpool1"].azurerm_kubernetes_cluster_node_pool.default` | +| `module.aks.module.rbac.azurerm_role_assignment.cluster_user["35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9"]` | +| `module.aks.module.rbac.kubernetes_cluster_role.aggregate_to_view[0]` | +| `module.aks.module.rbac.kubernetes_cluster_role_binding.cluster_admin[0]` | +| `module.metadata.data.azurerm_subscription.current` | +| `module.resource_groups["azure_kubernetes_service"].azurerm_resource_group.rg` | +| `module.resource_groups["azure_kubernetes_service"].random_integer.suffix[0]` | +| `module.subscription.data.azurerm_subscription.selected` | + +## Appendix B + + +| Resources Created by HPCC Deployment | +| :----------------------------------------------------------- | +| `local_file.config.json` | +| `random_integer.random` | +| `module.hpcc.azurerm_storage_account.azurefiles_admin_services[0]` | +| `module.hpcc.azurerm_storage_account.blob_nfs_admin_services[0]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["debug"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["dll"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["mydropzone"]` | +| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["sasha"]` | +| `module.hpcc.azurerm_storage_share.azurefiles_admin_services["dali"]` | +| `module.hpcc.helm_release.hpcc` | +| `module.hpcc.kubernetes_persistent_volume.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume.spill["spill"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.azurefiles["dali"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-1"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-2"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["debug"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["dll"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["mydropzone"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["sasha"]` | +| `module.hpcc.kubernetes_persistent_volume_claim.spill["spill"]` | +| `module.hpcc.kubernetes_secret.azurefiles_admin_services[0]` | +| `module.hpcc.kubernetes_storage_class.premium_zrs_file_share_storage_class[0]` | +| `module.hpcc.random_string.random` | +| `module.hpcc.random_uuid.volume_handle` | +| `module.hpcc.module.certificates.kubectl_manifest.default_issuer` | +| `module.hpcc.module.certificates.kubectl_manifest.local_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.remote_certificate` | +| `module.hpcc.module.certificates.kubectl_manifest.signing_certificate` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["2"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["1"]` | +| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | + + +## Appendix C + + +|Resources Created by Depolyment of storage| +|:------------------------------------------------------------------------------------| +| `local_file.config.json` | +| `module.storage.azurerm_storage_account.azurefiles["adminsvc1"]` | +| `module.storage.azurerm_storage_account.blobnfs["adminsvc2"]` | +| `module.storage.azurerm_storage_account.blobnfs["data1"]` | +| `module.storage.azurerm_storage_account.blobnfs["data2"]` | +| `module.storage.azurerm_storage_container.blobnfs["1"]` | +| `module.storage.azurerm_storage_container.blobnfs["2"]` | +| `module.storage.azurerm_storage_container.blobnfs["3"]` | +| `module.storage.azurerm_storage_container.blobnfs["4"]` | +| `module.storage.azurerm_storage_container.blobnfs["5"]` | +| `module.storage.azurerm_storage_container.blobnfs["6"]` | +| `module.storage.azurerm_storage_share.azurefiles["0"]` | +| `module.storage.null_resource.remove0000_from_azurefile["adminsvc1"]` | +| `module.storage.null_resource.remove0000_from_blobfs["adminsvc2"]` | +| `module.storage.null_resource.remove0000_from_blobfs["data1"]` | +| `module.storage.null_resource.remove0000_from_blobfs["data2"]` | +| `module.storage.random_string.random` | +| `module.storage.module.resource_groups["storage_accounts"].azurerm_resource_group.rg` | +| `module.storage.module.resource_groups["storage_accounts"].random_integer.suffix[0]` | + +## Appendix D + + +| Resources Created by Deployment of vnet | +| :----------------------------------------------------------- | +| `data.azurerm_advisor_recommendations.advisor` | +| `data.azurerm_subscription.current` | +| `data.http.host_ip` | +| `local_file.output` | +| `module.metadata.data.azurerm_subscription.current` | +| `module.resource_groups["virtual_network"].azurerm_resource_group.rg` | +| `module.resource_groups["virtual_network"].random_integer.suffix[0]` | +| `module.subscription.data.azurerm_subscription.selected` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-internet"]` | +| `module.virtual_network.azurerm_route.aks_route["hpcc-local-vnet-10-1-0-0-21"]` | +| `module.virtual_network.azurerm_route_table.aks_route_table["hpcc"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-private"]` | +| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-public"]` | +| `module.virtual_network.azurerm_virtual_network.vnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-private"].azurerm_subnet.subnet` | +| `module.virtual_network.module.aks_subnet["aks-hpcc-public"].azurerm_subnet.subnet` | diff --git a/hpcc/README.md b/hpcc/README.md deleted file mode 100644 index 67ef75a..0000000 --- a/hpcc/README.md +++ /dev/null @@ -1,732 +0,0 @@ -# Azure - HPCC AKS Root Module -
- -This module is intended as an example for development and test systems only. It can be used as a blueprint to develop your own production version that meets your organization's security requirements. -
-
- -## Introduction - -This module deploys an HPCC AKS cluster using remote modules that are listed below. -
- -## Remote Modules -These are the list of all the remote modules. - -| Name | Description | URL | Required | -| --------------- | ---------------------------------------------------- | -------------------------------------------------------------------------- | :------: | -| subscription | Queries enabled azure subscription from host machine | https://github.com/Azure-Terraform/terraform-azurerm-subscription-data.git | yes | -| naming | Enforces naming conventions | - | yes | -| metadata | Provides metadata | https://github.com/Azure-Terraform/terraform-azurerm-metadata.git | yes | -| resource_group | Creates a resource group | https://github.com/Azure-Terraform/terraform-azurerm-resource-group.git | yes | -| virtual_network | Creates a virtual network | https://github.com/Azure-Terraform/terraform-azurerm-virtual-network.git | yes | -| kubernetes | Creates an Azure Kubernetes Service Cluster | https://github.com/Azure-Terraform/terraform-azurerm-kubernetes.git | yes | -
- -## Supported Arguments -
- -### The `admin` block: -This block contains information on the user who is deploying the cluster. This is used as tags and part of some resource names to identify who deployed a given resource and how to contact that user. This block is required. - -| Name | Description | Type | Default | Required | -| ----- | ---------------------------- | ------ | ------- | :------: | -| name | Name of the admin. | string | - | yes | -| email | Email address for the admin. | string | - | yes | - -
-Usage Example: -
- - admin = { - name = "Example" - email = "example@hpccdemo.com" - } -
- -### The `disable_naming_conventions` block: -When set to `true`, this attribute drops the naming conventions set forth by the python module. This attribute is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------- | ---- | ------- | :------: | - | disable_naming_conventions | Disable naming conventions. | bool | `false` | no | -
- -### The `metadata` block: -TThe arguments in this block are used as tags and part of resources’ names. This block can be omitted when disable_naming_conventions is set to `true`. - - | Name | Description | Type | Default | Required | - | ------------------- | ---------------------------- | ------ | ------- | :------: | - | project_name | Name of the project. | string | "" | yes | - | product_name | Name of the product. | string | hpcc | no | - | business_unit | Name of your bussiness unit. | string | "" | no | - | environment | Name of the environment. | string | "" | no | - | market | Name of market. | string | "" | no | - | product_group | Name of product group. | string | "" | no | - | resource_group_type | Resource group type. | string | "" | no | - | sre_team | Name of SRE team. | string | "" | no | - | subscription_type | Subscription type. | string | "" | no | -
- -Usage Example: -
- - metadata = { - project = "hpccdemo" - product_name = "example" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "contoso" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - } - -
- -### The `tags` argument: -The tag attribute can be used for additional tags. The tags must be key value pairs. This block is optional. - - | Name | Description | Type | Default | Required | - | ---- | ------------------------- | ----------- | ------- | :------: | - | tags | Additional resource tags. | map(string) | admin | no | -
- -### The `resource_group` block: -This block creates a resource group (like a folder) for your resources. This block is required. - - | Name | Description | Type | Default | Required | - | ----------- | ----------------------------------------------------------------- | ---- | ------- | :------: | - | unique_name | Will concatenate a number at the end of your resource group name. | bool | `true` | yes | -
- -Usage Example: -
- - resource_group = { - unique_name = true - } - -
- -### The `virtual_network` block: -This block imports metadata of a virtual network deployed outside of this project. This block is optional. - - | Name | Description | Type | Default | Required | - | ----------------- | --------------------------------------- | ------ | ------- | :------: | - | private_subnet_id | The ID of the private subnet. | string | - | yes | - | public_subnet_id | The ID of the public subnet. | string | - | yes | - | route_table_id | The ID of the route table for the AKS. | string | - | yes | - | location | The location of the virtual network | string | - | yes | -
- -Usage Example: -
- - virtual_network = { - private_subnet_id = "" - public_subnet_id = "" - route_table_id = "" - location = "" - } - -
- -## The `node_pools` block: -The `node-pools` block supports the following arguments:
-`system` - (Required) The system or default node pool. This node pool hosts the system pods by default. The possible arguments for this block are defined below. - -`addpool` - (Required) The additional node pool configuration. This block name is changeable and must be unique across all additional node pools. At least one additional node pool is required. The possible arguments for this block are defined below. - -### The `system` block: -This block creates a system node pool. This block is required. - -| Name | Optional, Required | Description | -| --------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -### The `addpool` block: -This block creates additional node pools. This block is optional. - -| Name | Optional, Required | Description | -| ---------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| node_taints | Optional | A list of Kubernetes taints which should be applied to nodes in the agent pool (e.g key=value:NoSchedule). Changing this forces a new resource to be created. | -| max_surge | Required | The maximum number or percentage of nodes which will be added to the Node Pool size during an upgrade. | -| eviction_policy | Optional | The Eviction Policy which should be used for Virtual Machines within the Virtual Machine Scale Set powering this Node Pool. Possible values are Deallocate and Delete. Will only be used when priority is set to Spot. Changing this forces a new resource to be created. | -| os_type | Optional | The Operating System which should be used for this Node Pool. Changing this forces a new resource to be created. Possible values are Linux and Windows. Defaults to Linux. | -| priority | Optional | The Priority for Virtual Machines within the Virtual Machine Scale Set that powers this Node Pool. Possible values are Regular and Spot. Defaults to Regular. Changing this forces a new resource to be created. | -| proximity_placement_group_id | Optional | The ID of the Proximity Placement Group where the Virtual Machine Scale Set that powers this Node Pool will be placed. Changing this forces a new resource to be created. | -| spot_max_price | Optional | The maximum price you're willing to pay in USD per Virtual Machine. Valid values are -1 (the current on-demand price for a Virtual Machine) or a positive value with up to five decimal places. Changing this forces a new resource to be created. | -| vm_size | Optional | The size of the Virtual Machine, such as Standard_A4_v2. | -| node_count | Optional | The initial number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000 and between min_count and max_count. | -| enable_auto_scalling | Optional | Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | -| min_count | Optional | The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| max_count | Optional | The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. Can only be set when enable_auto_scalling is set to true. | -| availability_zones | Optional | A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created. | -| enable_host_encryption | Optional | Should the nodes in the Default Node Pool have host encryption enabled? Defaults to false. Can only be enabled on new node pools. Requires VirtualMachineScaleSets as VM type. Can only be enabled in Azure regions that support server-side encryption of Azure managed disks and only with specific supported VM sizes. | -| enable_node_public_ip | Optional | Should nodes in this Node Pool have a Public IP Address? Defaults to false. | -| max_pods | Optional | The maximum number of pods that can run on each agent. | -| node_labels | Optional | A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. | -| only_critical_addons_enable | Optional | Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. When set to true, only system pods will be scheduled on the system node pool. | -| orchestrator_version | Optional | Version of Kubernetes used for the Agents. If not specified, the latest recommended version will be used at provisioning time (but won't auto-upgrade). | -| os_disk_size_gb | Optional | The size of the OS Disk which should be used for each agent in the Node Pool. | -| os_disk_type | Optional | The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. | -| type | Optional | The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | -| tags | Optional | A mapping of tags to assign to the Node Pool. | -| subnet | Optional | The ID of a Subnet where the Kubernetes Node Pool should exist. | -
- -Usage Example: -
- - node_pools = { - system = { - vm_size = "Standard_D4_v4" - node_count = 1 - enable_auto_scaling = true - only_critical_addons_enabled = true - min_count = 1 - max_count = 1 - availability_zones = [] - subnet = "private" - enable_host_encryption = false - enable_node_public_ip = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # max_pods = 10 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - - } - - addpool1 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - - addpool2 = { - vm_size = "Standard_D4_v4" - enable_auto_scaling = true - node_count = 2 - min_count = 1 - max_count = 2 - availability_zones = [] - subnet = "public" - priority = "Regular" - spot_max_price = -1 - max_surge = "1" - os_type = "Linux" - priority = "Regular" - enable_host_encryption = false - enable_node_public_ip = false - only_critical_addons_enabled = false - os_disk_type = "Managed" - type = "VirtualMachineScaleSets" - # orchestrator_version = "2.9.0" - # os_disk_size_gb = 100 - # max_pods = 20 - # node_labels = {"engine" = "roxie", "engine" = "roxie"} - # eviction_policy = "Spot" - # node_taints = ["mytaint1", "mytaint2"] - # proximity_placement_group_id = "my_proximity_placement_group_id" - # spot_max_price = 1 - # tags = {"mynodepooltag1" = "mytagvalue1", "mynodepooltag2" = "mytagvalue2"} - } - } -
- -### The `disable_helm` argument: -This block disable helm deployments by Terraform. This block is optional and will stop HPCC from being installed. - - | Name | Description | Type | Default | Required | - | ------------ | -------------------------------------- | ---- | ------- | :------: | - | disable_helm | Disable Helm deployments by Terraform. | bool | `false` | no | -
- -### The `hpcc` block: -This block deploys the HPCC helm chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------------------------------ | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: ~/HPCC-Platform/helm/hpcc Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | string | null | no | - | namespace | Namespace to use. | string | default | no | - | name | Release name of the chart. | string | myhpcck8s | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | values-retained-azurefile.yaml | no | - | version | Version of the HPCC chart. | string | latest | yes | - | image_root | Image root to use. | string | hpccsystems | no | - | image_name | Image name to use. | string | platform-core | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose_eclwatch | Expose ECLWatch to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- - Usage Example: -
- - hpcc = { - expose_eclwatch = true - name = "myhpcck8s" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 900 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/hpcc" #Other examples: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-8.6.16-rc1.tgz - # version = "8.6.14-rc2" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - # image_root = "west.lexisnexisrisk.com" - # image_name = "platform-core-ln" - # image_version = "8.6.18-rc1" - } - -
- -### The `storage` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------- | ---------------- | :---------: | - | default | Use AKS provided storage accounts? | bool | `false` | `true` , `false` | no | - | version | The version of the storage chart. | string | 0.1.0 | | no | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/hpcc-azurefile-0.1.0.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | name | Release name of the chart. | string | `myhpcck8s` | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | [] | no | - | storage_accounts | The storage account to use. | object | Queries attributes' values from storage_accounts module | - | no | - | version | Version of the storage chart. | string | 0.1.0 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 600 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | -
- -#### The `storage_accounts` block: -This block deploys the HPCC persistent volumes. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | ------------------- | -------------------------------------------------------------------- | ------------ | --------------------------- | ------------- | :------: | - | name | Name of the storage account | string | - | - | yes | - | resource_group_name | The name of the resource group in which the storage account belongs. | string | - | - | yes | - | subscription_id | The ID of the subscription in which the storage account belongs. | string | Admin's active subscription | - | no | - | shares | The list of shares in the storage account | list(object) | - | - | yes | - | | -
- -#### The `shares` block: -This block defines the list of shares in the storage account. This block is required. - - | Name | Description | Type | Default | Valid Options | Required | - | -------- | ------------------------------------- | ------ | ------- | ------------- | :------: | - | name | The name of the share. | string | - | - | yes | - | sub_path | The sub path for the HPCC data plane. | string | - | - | yes | - | category | The category for the HPCC data plane | string | - | - | yes | - | sku | The sku for the HPCC data plane. | string | - | - | yes | - | quota | The size of the share in Gigabytes | number | - | - | yes | - -Usage Example: -
- - storage = { - default = false - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 600 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - # local_chart = "/Users/foo/work/demo/helm-chart/helm/examples/azure/hpcc-azurefile" - # version = "0.1.0" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - - /* - storage_accounts = { - # do not change the key names - dali = { - name = "dalikxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - dali = { - name = "dalishare" - sub_path = "dalistorage" //do not change this value - category = "dali" //do not change this value - sku = "Premium_LRS" - quota = 100 - } - } - } - - sasha = { - name = "sashakxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - sasha = { - name = "sashashare" - sub_path = "sasha" //do not change this value - category = "sasha" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - - common = { - name = "commonkxgt" - resource_group_name = "app-storageaccount-sandbox-eastus-79735" - - shares = { - data = { - name = "datashare" - sub_path = "hpcc-data" //do not change this value - category = "data" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - dll = { - name = "dllshare" - sub_path = "queries" //do not change this value - category = "dll" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - - mydropzone = { - name = "mydropzoneshare" - sub_path = "dropzone" //do not change this value - category = "lz" //do not change this value - sku = "Standard_LRS" - quota = 100 - } - } - } - } - */ - } -
- -### The `elastic4hpcclogs` block: -This block deploys the elastic4hpcclogs chart. This block is optional. - - | Name | Description | Type | Default | Required | - | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ----------------------- | :------: | - | local_chart | Path to local chart directory name or tgz file. Example1: /Users/foo/work/demo/helm-chart/helm/managed/logging/elastic Example2: https://github.com/hpcc-systems/helm-chart/raw/master/docs/elastic4hpcclogs-1.2.10.tgz | string | null | no | - | remote_chart | URL of the remote chart. Example: https://hpcc-systems.github.io/helm-chart | enable | Enable elastic4hpcclogs | bool | `true` | no | - | name | Release name of the chart. | string | myelastic4hpcclogs | no | - | version | The version of the elastic4hpcclogs | string | 1.2.8 | | no | - | values | List of desired state files to use similar to -f in CLI. | list(string) | - | no | - | version | Version of the elastic4hpcclogs chart. | string | 1.2.1 | no | - | atomic | If set, installation process purges chart on fail. The `wait` flag will be set automatically if `atomic` is used. | bool | false | no | - | recreate_pods | Perform pods restart during upgrade/rollback. | bool | false | no | - | reuse_values | When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. | bool | false | no | - | reset_values | When upgrading, reset the values to the ones built into the chart. | bool | false | no | - | force_update | Force resource update through delete/recreate if needed. | bool | false | no | - | cleanup_on_fail | Allow deletion of new resources created in this upgrade when upgrade fails. | bool | false | no | - | disable_openapi_validation | If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. | bool | false | no | - | max_history | Maximum number of release versions stored per release. | number | 0 | no | - | wait | Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` . | bool | true | no | - | dependency_update | Runs helm dependency update before installing the chart. | bool | false | no | - | timeout | Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). | number | 900 | no | - | wait_for_jobs | If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. | bool | false | no | - | lint | Run the helm chart linter during the plan. | bool | false | no | - | expose | Expose myelastic4hpcclogs-kibana service to the internet. This can cause the service to hang on pending state if external IPs are blocked by your organization's cloud policies. | bool | true | no | -
- -Usage Example: -
- - elastic4hpcclogs = { - enable = true - expose = true - name = "myelastic4hpcclogs" - atomic = true - recreate_pods = false - reuse_values = false - reset_values = false - force_update = false - namespace = "default" - cleanup_on_fail = false - disable_openapi_validation = false - max_history = 0 - wait = true - dependency_update = true - timeout = 300 - wait_for_jobs = false - lint = false - remote_chart = "https://hpcc-systems.github.io/helm-chart" - #local_chart = "/Users/godji/work/demo/helm-chart/helm/managed/logging/elastic" - # version = "1.2.10" - # values = ["/Users/foo/mycustomvalues1.yaml", "/Users/foo/mycustomvalues2.yaml"] - } -
- -### The `registry` block: -This block authenticates a private Docker repository. This block is optional. - - | Name | Description | Type | Default | Required | - | -------- | -------------------------------------------------------------------------- | ------ | ------- | :------: | - | server | The server address of the private Docker repository. | string | - | yes | - | username | The username for the private Docker repository account. | string | - | yes | - | password | The password, token, or API key for the private Docker repository account. | string | - | yes | -
- -Usage Example: -
- - registry = { - password = "" - server = "" - username = "" - } -
- -### The `auto_connect` argument: -This block automatically connect your cluster to your local machine similarly to `az aks get-credentials`. - - | Name | Description | Type | Default | Required | - | ------------ | --------------------------------------------------------------------------------------------------------- | ---- | ------- | :------: | - | auto_connect | Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context. | bool | `false` | no | -
- -## Outputs - -| Name | Description | -| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| aks_login | Get access credentials for the managed Kubernetes cluster. | -| recommendations | A list of security and cost recommendations for this deployment. Your environment has to have been deployed for several hours before Azure provides recommendations. | -
- -## Usage -### Deploy the Virtual Network Module -
    -
  1. - -Clone this repo: `git clone https://github.com/gfortil/terraform-azurerm-hpcc.git`.
    2. - -
  2. Linux and MacOS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `cp examples/admin.tfvars .`
      3. -
    -
  3. Windows OS
    4. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/virtual_network: `cd terraform-azurerm-hpcc/modules/virtual_network`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/virtual_network: `copy examples\admin.tfvars .`
      3. -
    -
  4. - -Open `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    5. -
  5. - -Set attributes to your preferred values.
    6. -
  6. - -Save `terraform-azurerm-hpcc/modules/virtual_network/admin.tfvars` file.
    7. -
  7. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    8. -
  8. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    9. -
  9. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    10. -
- -### Deploy the Storage Account Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc/modules/storage_accounts: `cd terraform-azurerm-hpcc/modules/storage_accounts`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc/modules/storage_accounts: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/modules/storage_accounts/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
- -### Deploy the AKS Module -
    -
  1. Linux and MacOS
    2. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `cp examples/admin.tfvars .`
      3. -
    -
  2. Windows OS
    3. -
      -
    1. - -Change directory to terraform-azurerm-hpcc: `cd terraform-azurerm-hpcc`
      2. -
    2. - -Copy examples/admin.tfvars to terraform-azurerm-hpcc: `copy examples\admin.tfvars .`
      3. -
    -
  3. - -Open `terraform-azurerm-hpcc/admin.tfvars` file.
    4. -
  4. - -Set attributes to your preferred values.
    5. -
  5. - -Save `terraform-azurerm-hpcc/admin.tfvars` file.
    6. -
  6. - -Run `terraform init`. This step is only required before your first `terraform apply`.
    7. -
  7. - -Run `terraform apply -var-file=admin.tfvars` or `terraform apply -var-file=admin.tfvars -auto-approve`.
    8. -
  8. - -Type `yes` if you didn't pass the flag `-auto-approve`.
    9. -
  9. - -If `auto_connect = true` (in admin.tfvars), skip this step.
    10. -
      -
    1. - -Copy aks_login command.
      2. -
    2. - -Run aks_login in your command line.
      3. -
    3. - -Accept to overwrite your current context.
      4. -
    -
  10. - -List pods: `kubectl get pods`.
    11. -
  11. - -Get ECLWatch external IP: `kubectl get svc --field-selector metadata.name=eclwatch | awk 'NR==2 {print $4}'`.
    12. -
  12. - -Delete cluster: `terraform destroy -var-file=admin.tfvars` or `terraform destroy -var-file=admin.tfvars -auto-approve`.
    13. -
  13. - -Type: `yes` if flag `-auto-approve` was not set.
    14. -
diff --git a/hpcc/dali.auto.tfvars.example b/hpcc/dali.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 diff --git a/hpcc/data.tf b/hpcc/data.tf index ae24590..4a92803 100644 --- a/hpcc/data.tf +++ b/hpcc/data.tf @@ -11,3 +11,8 @@ data "azuread_group" "subscription_owner" { data "azurerm_client_config" "current" { } + +data "local_file" "aks" { + filename = "../aks/data/config.json" +} + diff --git a/hpcc/eclccserver.auto.tfvars.example b/hpcc/eclccserver.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 82a047f..3a9cf47 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,84 +1,107 @@ resource "kubernetes_namespace" "hpcc" { - count = var.hpcc_namespace.create_namespace && !fileexists("${path.module}/modules/logging/data/hpcc_namespace.txt") ? 1 : 0 + count = var.hpcc_namespace.create_namespace && !fileexists("../logging/data/hpcc_namespace.txt") ? 1 : 0 metadata { - labels = var.hpcc_namespace.labels - name = "${substr(trimspace(var.owner.name), 0, 5)}${random_integer.random.result}" - # generate_name = "${trimspace(var.owner.name)}" + labels = try(var.hpcc_namespace.labels,{}) + + generate_name = "${var.hpcc_namespace.prefix_name}${trimspace(local.owner.name)}" } } +/*resource "kubernetes_namespace" "hpcc" { + count = (var.hpcc_namespace == []) || !var.hpcc_namespace.create_namespace || fileexists("../logging/data/hpcc_namespace.txt") ? 0 : 1 + + metadata { + labels = try(var.hpcc_namespace.labels,{}) + name = "${substr(trimspace(local.owner.name), 0, 5)}${random_integer.random.result}" + # generate_name = "${trimspace(local.owner.name)}" + } +}*/ + module "hpcc" { - source = "github.com/gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" + #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" - environment = var.metadata.environment - productname = var.metadata.product_name + environment = local.metadata.environment + productname = local.metadata.product_name - internal_domain = var.internal_domain - cluster_name = local.get_aks_config.cluster_name + a_record_name = var.a_record_name + internal_domain = local.internal_domain + cluster_name = jsondecode(file("../aks/data/config.json")).cluster_name + hpcc_version = var.hpcc_version hpcc_container = { - image_name = var.hpcc_container != null ? var.hpcc_container.image_name : null - image_root = var.hpcc_container != null ? var.hpcc_container.image_root : null - version = var.hpcc_container != null ? var.hpcc_container.version : null - custom_chart_version = var.hpcc_container != null ? var.hpcc_container.custom_chart_version : null - custom_image_version = var.hpcc_container != null ? var.hpcc_container.custom_image_version : null + image_name = local.hpcc_container != null ? local.hpcc_container.image_name : null + image_root = local.hpcc_container != null ? local.hpcc_container.image_root : null + version = local.hpcc_container != null ? local.hpcc_container.version : null + custom_chart_version = local.hpcc_container != null ? local.hpcc_container.custom_chart_version : null + custom_image_version = local.hpcc_container != null ? local.hpcc_container.custom_image_version : null } - hpcc_container_registry_auth = var.hpcc_container_registry_auth != null ? { - password = var.hpcc_container_registry_auth.password - username = var.hpcc_container_registry_auth.username + hpcc_container_registry_auth = local.hpcc_container_registry_auth != null ? { + password = local.hpcc_container_registry_auth.password + username = local.hpcc_container_registry_auth.username } : null + hpcc_user_ip_cidr_list = var.hpcc_user_ip_cidr_list + install_blob_csi_driver = false //Disable CSI driver resource_group_name = local.get_aks_config.resource_group_name - location = var.metadata.location + location = local.metadata.location tags = module.metadata.tags # namespace = local.hpcc_namespace namespace = { create_namespace = false name = local.hpcc_namespace - labels = var.hpcc_namespace.labels + labels = try(var.hpcc_namespace.labels,{}) } + #----------------------------------------------------------------------- + # Storage variables (internal (ephemeral) or external) + #- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - admin_services_storage_account_settings = { - replication_type = var.admin_services_storage_account_settings.replication_type - authorized_ip_ranges = merge(var.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) - delete_protection = var.admin_services_storage_account_settings.delete_protection + replication_type = local.admin_services_storage_account_settings.replication_type + authorized_ip_ranges = merge(local.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) + delete_protection = local.admin_services_storage_account_settings.delete_protection subnet_ids = merge({ aks = local.subnet_ids.aks }) } + internal_storage_enabled = local.internal_storage_enabled + + storage_data_gb = var.storage_data_gb + data_storage_config = { - internal = local.external_storage_config == null ? { + internal = (local.internal_storage_enabled == true) ? { blob_nfs = { - data_plane_count = var.data_storage_config.internal.blob_nfs.data_plane_count + data_plane_count = local.data_storage_config.internal.blob_nfs.data_plane_count storage_account_settings = { - replication_type = var.data_storage_config.internal.blob_nfs.storage_account_settings.replication_type - authorized_ip_ranges = merge(var.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) - delete_protection = var.data_storage_config.internal.blob_nfs.storage_account_settings.delete_protection + replication_type = local.data_storage_config.internal.blob_nfs.storage_account_settings.replication_type + authorized_ip_ranges = merge(local.admin_services_storage_account_settings.authorized_ip_ranges, { host_ip = data.http.host_ip.response_body }) + delete_protection = local.data_storage_config.internal.blob_nfs.storage_account_settings.delete_protection subnet_ids = merge({ aks = local.subnet_ids.aks }) } } - } : null + } : null - # external = local.internal_data_storage_enabled ? null : { - # blob_nfs = local.get_storage_config != null ? local.get_storage_config.data_storage_planes : var.data_storage_config.external.blob_nfs - # hpcc = null - # } external = null } external_storage_config = local.external_storage_config - - spill_volumes = var.spill_volumes - roxie_config = var.roxie_config - thor_config = var.thor_config - vault_config = var.vault_config - eclccserver_settings = var.eclccserver_settings - spray_service_settings = var.spray_service_settings - admin_services_node_selector = { all = { workload = var.spray_service_settings.nodeSelector } } + #----------------------------------------------------------------------- + + spill_volumes = local.spill_volumes + enable_roxie = var.aks_enable_roxie + roxie_config = local.roxie_config + thor_config = local.thor_config + vault_config = local.vault_config + eclccserver_settings = local.eclccserver_settings + spray_service_settings = local.spray_service_settings + # tlh 20231109 admin_services_node_selector = { all = { workload = local.spray_service_settings.nodeSelector } } + admin_services_node_selector = { all = { workload = "servpool" } } esp_remoteclients = { @@ -96,7 +119,10 @@ module "hpcc" { } - helm_chart_timeout = var.helm_chart_timeout - helm_chart_files_overrides = concat(var.helm_chart_files_overrides, fileexists("${path.module}/modules/logging/data/logaccess_body.yaml") ? ["${path.module}/modules/logging/data/logaccess_body.yaml"] : []) - ldap_config = var.ldap_config + helm_chart_timeout = local.helm_chart_timeout + helm_chart_files_overrides = concat(local.helm_chart_files_overrides, fileexists("../logging/data/logaccess_body.yaml") ? ["../logging/data/logaccess_body.yaml"] : []) + ldap_config = local.ldap_config + + enable_code_security = var.enable_code_security + authn_htpasswd_filename = var.authn_htpasswd_filename } diff --git a/hpcc/index.html b/hpcc/index.html new file mode 100644 index 0000000..7937d2e --- /dev/null +++ b/hpcc/index.html @@ -0,0 +1 @@ +20.96.202.148 \ No newline at end of file diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf new file mode 100755 index 0000000..867f922 --- /dev/null +++ b/hpcc/lite-locals.tf @@ -0,0 +1,540 @@ +locals { + helm_chart_timeout=300 + #hpcc_version = "8.6.20" + + owner = { + name = var.admin_username + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + tags = merge(local.metadata.additional_tags, var.extra_tags) + + # # disable_naming_conventions - Disable naming conventions + # # disable_naming_conventions = true + disable_naming_conventions = false + + # # auto_launch_eclwatch - Automatically launch ECLWatch web interface. + #auto_launch_eclwatch = true + auto_launch_svc = { + eclwatch = false + } + + # azure_auth = { + # # AAD_CLIENT_ID = "" + # # AAD_CLIENT_SECRET = "" + # # AAD_TENANT_ID = "" + # # AAD_PRINCIPAL_ID = "" + # SUBSCRIPTION_ID = "" + # } + + # hpcc_container = { + # version = "9.2.0" + # image_name = "platform-core-ln" + # image_root = "jfrog.com/glb-docker-virtual" + # # custom_chart_version = "9.2.0-rc1" + # # custom_image_version = "9.2.0-demo" + # } + + # hpcc_container_registry_auth = { + # username = "value" + # password = "value" + # } + + internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io + + external = {} + # external = { + # blob_nfs = [{ + # container_id = "" + # container_name = "" + # id = "" + # resource_group_name = var.storage_account_resource_group_name + # storage_account_id = "" + # storage_account_name = var.storage_account_name + # }] + # # hpc_cache = [{ + # # id = "" + # # path = "" + # # server = "" + # }] + # hpcc = [{ + # name = "" + # planes = list(object({ + # local = "" + # remote = "" + # })) + # service = "" + # }] + # } + + admin_services_storage_account_settings = { + replication_type = "ZRS" #LRS only if using HPC Cache + authorized_ip_ranges = { + "default" = "0.0.0.0/0" //must be public IP + } + + delete_protection = false + } + + azure_log_analytics_creds = { + scope = null + object_id = "" //AAD_PRINCIPAL_ID + } + + data_storage_config = { + internal = { + blob_nfs = { + data_plane_count = 2 + storage_account_settings = { + replication_type = "ZRS" + delete_protection = false + } + } + # hpc_cache = { + # enabled = false + # size = "small" + # cache_update_frequency = "3h" + # storage_account_data_planes = null + # } + } + external = null + } + + + spill_volumes = { + spill = { + name = "spill" + size = 300 + prefix = "/var/lib/HPCCSystems/spill" + host_path = "/mnt" + storage_class = "spill" + access_mode = "ReadWriteOnce" + } + } + + spray_service_settings = { + replicas = 6 + nodeSelector = "spraypool" + } + + # ldap = { + # ldap_server = "" //Server IP + # dali = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # esp = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # } + + roxie_internal_service = { + name = "iroxie" + servicePort = 9876 + listenQueue = 200 + numThreads = 30 + visibility = "local" + annotations = {} + } + + roxie_services = [local.roxie_internal_service] + + #======================================== + # defaults in godji original variables.tf + expose_services = false + + auto_connect = false + + use_existing_vnet = null + + hpcc_enabled = true + + helm_chart_strings_overrides = [] + + helm_chart_files_overrides = [] + + vault_config = null + + hpcc_container = null + + hpcc_container_registry_auth = null + + roxie_config = [ + { + disabled = (var.aks_enable_roxie == true)? false : true + name = "roxie" + nodeSelector = { workload = "roxiepool" } + # tlh 20231109 numChannels = 2 + numChannels = 1 + prefix = "roxie" + replicas = 2 + serverReplicas = 0 + acePoolSize = 6 + actResetLogPeriod = 0 + affinity = 0 + allFilesDynamic = false + blindLogging = false + blobCacheMem = 0 + callbackRetries = 3 + callbackTimeout = 500 + checkCompleted = true + checkFileDate = false + checkPrimaries = true + clusterWidth = 1 + copyResources = true + coresPerQuery = 0 + crcResources = false + dafilesrvLookupTimeout = 10000 + debugPermitted = true + defaultConcatPreload = 0 + defaultFetchPreload = 0 + defaultFullKeyedJoinPreload = 0 + defaultHighPriorityTimeLimit = 0 + defaultHighPriorityTimeWarning = 30000 + defaultKeyedJoinPreload = 0 + defaultLowPriorityTimeLimit = 0 + defaultLowPriorityTimeWarning = 90000 + defaultMemoryLimit = 1073741824 + defaultParallelJoinPreload = 0 + defaultPrefetchProjectPreload = 10 + defaultSLAPriorityTimeLimit = 0 + defaultSLAPriorityTimeWarning = 30000 + defaultStripLeadingWhitespace = false + diskReadBufferSize = 65536 + doIbytiDelay = true + egress = "engineEgress" + enableHeartBeat = false + enableKeyDiff = false + enableSysLog = false + fastLaneQueue = true + fieldTranslationEnabled = "payload" + flushJHtreeCacheOnOOM = true + forceStdLog = false + highTimeout = 2000 + ignoreMissingFiles = false + indexReadChunkSize = 60000 + initIbytiDelay = 10 + jumboFrames = false + lazyOpen = true + leafCacheMem = 500 + linuxYield = false + localFilesExpire = 1 + localSlave = false + logFullQueries = false + logQueueDrop = 32 + logQueueLen = 512 + lowTimeout = 10000 + maxBlockSize = 1000000000 + maxHttpConnectionRequests = 1 + maxLocalFilesOpen = 4000 + maxLockAttempts = 5 + maxRemoteFilesOpen = 100 + memTraceLevel = 1 + memTraceSizeLimit = 0 + memoryStatsInterval = 60 + minFreeDiskSpace = 6442450944 + minIbytiDelay = 2 + minLocalFilesOpen = 2000 + minRemoteFilesOpen = 50 + miscDebugTraceLevel = 0 + monitorDaliFileServer = false + nodeCacheMem = 1000 + nodeCachePreload = false + parallelAggregate = 0 + parallelLoadQueries = 1 + perChannelFlowLimit = 50 + pingInterval = 0 + preabortIndexReadsThreshold = 100 + preabortKeyedJoinsThreshold = 100 + preloadOnceData = true + prestartSlaveThreads = false + remoteFilesExpire = 3600 + roxieMulticastEnabled = false + serverSideCacheSize = 0 + serverThreads = 100 + simpleLocalKeyedJoins = true + sinkMode = "sequential" + slaTimeout = 2000 + slaveConfig = "simple" + slaveThreads = 30 + soapTraceLevel = 1 + socketCheckInterval = 5000 + statsExpiryTime = 3600 + systemMonitorInterval = 60000 + totalMemoryLimit = "5368709120" + traceLevel = 1 + traceRemoteFiles = false + trapTooManyActiveQueries = true + udpAdjustThreadPriorities = true + udpFlowAckTimeout = 10 + udpFlowSocketsSize = 33554432 + udpInlineCollation = true + udpInlineCollationPacketLimit = 50 + udpLocalWriteSocketSize = 16777216 + udpMaxPermitDeadTimeouts = 100 + udpMaxRetryTimedoutReqs = 10 + udpMaxSlotsPerClient = 100 + udpMulticastBufferSize = 33554432 + udpOutQsPriority = 5 + udpQueueSize = 1000 + udpRecvFlowTimeout = 2000 + udpRequestToSendAckTimeout = 500 + udpResendTimeout = 100 + udpRequestToSendTimeout = 2000 + udpResendEnabled = true + udpRetryBusySenders = 0 + udpSendCompletedInData = false + udpSendQueueSize = 500 + udpSnifferEnabled = false + udpTraceLevel = 0 + useAeron = false + useDynamicServers = false + useHardLink = false + useLogQueue = true + useMemoryMappedIndexes = false + useRemoteResources = false + useTreeCopy = false + services = local.roxie_services + topoServer = { + replicas = 1 + } + channelResources = { + cpu = "1" + memory = "4G" + } + } + ] + + eclagent_settings = { + hthor = { + replicas = 1 + maxActive = 4 + prefix = "hthor" + use_child_process = false + type = "hthor" + spillPlane = "spill" + resources = { + cpu = "1" + memory = "4G" + } + nodeSelector = { workload = "servpool" } + egress = "engineEgress" + cost = { + perCpu = 1 + } + }, + } + + eclccserver_settings = { + "myeclccserver" = { + useChildProcesses = false + maxActive = 4 + egress = "engineEgress" + replicas = 1 + childProcessTimeLimit = 10 + resources = { + cpu = "1" + memory = "4G" + } + nodeSelector = { workload = "servpool" } + legacySyntax = false + options = [] + cost = { + perCpu = 1 + } + } } + + dali_settings = { + coalescer = { + interval = 24 + at = "* * * * *" + minDeltaSize = 50000 + nodeSelector = { workload = "servpool" } + resources = { + cpu = "1" + memory = "4G" + } + } + resources = { + cpu = "2" + memory = "8G" + } + maxStartupTime = 1200 + } + + dfuserver_settings = { + maxJobs = 3 + nodeSelector = { workload = "servpool" } + resources = { + cpu = "1" + memory = "2G" + } + } + + sasha_config = { + disabled = false + nodeSelector = { workload = "servpool" } + wu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 6 + limit = 400 + cutoff = 3 + backup = 0 + at = "* * * * *" + throttle = 0 + retryinterval = 6 + keepResultFiles = false + # egress = "engineEgress" + } + + dfuwu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 24 + limit = 100 + cutoff = 14 + at = "* * * * *" + throttle = 0 + # egress = "engineEgress" + } + + dfurecovery-archiver = { + disabled = false + interval = 12 + limit = 20 + cutoff = 4 + at = "* * * * *" + # egress = "engineEgress" + } + + file-expiry = { + disabled = false + interval = 1 + at = "* * * * *" + persistExpiryDefault = 7 + expiryDefault = 4 + user = "sasha" + # egress = "engineEgress" + } + } + + ldap_config = null + + ldap_tunables = { + cacheTimeout = 5 + checkScopeScans = false + ldapTimeoutSecs = 131 + maxConnections = 10 + passwordExpirationWarningDays = 10 + sharedCache = true + } + + install_blob_csi_driver = true + + remote_storage_plane = null + + onprem_lz_settings = {} + + admin_services_node_selector = {} + + thor_config = [{ + disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true + name = "thor" + prefix = "thor" + numWorkers = var.thor_num_workers + keepJobs = "none" + maxJobs = var.thor_max_jobs + maxGraphs = 2 + maxGraphStartupTime = 172800 + numWorkersPerPod = 1 + #nodeSelector = {} + nodeSelector = { workload = "thorpool" } + egress = "engineEgress" + tolerations_value = "thorpool" + managerResources = { + cpu = 1 + memory = "2G" + } + workerResources = { + cpu = 3 + memory = "4G" + } + workerMemory = { + query = "3G" + thirdParty = "500M" + } + eclAgentResources = { + cpu = 1 + memory = "2G" + } + cost = { + perCpu = 1 + } + }] + + admin_services_storage = { + dali = { + size = 100 + type = "azurefiles" + } + debug = { + size = 100 + type = "blobnfs" + } + dll = { + size = 100 + type = "blobnfs" + } + lz = { + size = var.storage_lz_gb + type = "blobnfs" + } + sasha = { + size = 100 + type = "blobnfs" + } + } + #======================================== +} diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf new file mode 100644 index 0000000..a14c506 --- /dev/null +++ b/hpcc/lite-variables.tf @@ -0,0 +1,212 @@ +############################################################################### +# Prompted variables (user will be asked to supply them at plan/apply time +# if a .tfvars file is not supplied); there are no default values +############################################################################### +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + +variable "external_storage_desired" { + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" + type = bool + default = false +} + +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool + default = true +} + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool + default = false +} + +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "REQUIRED. Name of the resource group containing the dns zone." +} + +variable "aks_dns_zone_name" { + type = string + description = "REQUIRED. dns zone name. The name of existing dns zone." +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } + default = "latest" +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} + +variable "storage_data_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_data_gb >= 10 + error_message = "Value must be 10 or more." + } + default = 100 +} + +variable "storage_lz_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for the landing zone in gigabytes.\nMust be 1 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_lz_gb >= 1 + error_message = "Value must be 1 or more." + } + default = 25 +} + +variable "thor_max_jobs" { + type = number + description = "REQUIRED. The maximum number of simultaneous Thor jobs allowed.\nMust be 1 or more." + validation { + condition = var.thor_max_jobs >= 1 + error_message = "Value must be 1 or more." + } + default = 2 +} + +variable "thor_num_workers" { + type = number + description = "REQUIRED. The number of Thor workers to allocate.\nMust be 1 or more." + validation { + condition = var.thor_num_workers >= 1 + error_message = "Value must be 1 or more." + } + default = 2 +} + +############################################################################### +# Optional variables +############################################################################### + +variable "authn_htpasswd_filename" { + type = string + description = "OPTIONAL. If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it.\nA corollary is that persistent storage is enabled.\nAn empty string indicates that htpasswd is not to be used for authentication.\nExample entry: htpasswd.txt" + default = "" +} + +variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + prefix_name = string + labels = map(string) + create_namespace = bool + }) + default = { + prefix_name = "hpcc" + labels = { + name = "hpcc" + } + create_namespace = false + } +} + +variable "enable_premium_storage" { + type = bool + description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." + default = false +} + +variable "storage_account_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" + default = "" +} + +variable "storage_account_resource_group_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." + default = "" +} diff --git a/hpcc/locals.tf b/hpcc/locals.tf index b81b494..e3fe976 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -4,45 +4,34 @@ locals { AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id } - names = var.disable_naming_conventions ? merge( + hpcc_namespace = "default" + + names = try(local.disable_naming_conventions, false) ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.metadata.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) - - # external_services_storage_exists = fileexists("${path.module}/modules/storage/data/config.json") || var.external_services_storage_config != null - get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - get_storage_config = local.external_storage_exists ? jsondecode(file("../storage/data/config.json")) : null - - external_storage_exists = fileexists("../storage/data/config.json") || var.external_storage_config != null - - subnet_ids = try({ - for k, v in var.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${var.use_existing_vnet.name}/subnets/${v.name}" - }, { aks = local.get_vnet_config.private_subnet_id }) - - location = var.use_existing_vnet != null ? var.use_existing_vnet.location : local.get_vnet_config.location - # hpcc_chart_major_minor_point_version = var.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", var.helm_chart_version) : "master" + #--------------------------------------------------------------------------------------------------------------------------- + # Setup storage (either external storage of internal (ephemeral) storage + #- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + get_storage_config = fileexists("../storage/data/config.json") ? jsondecode(file("../storage/data/config.json")) : null - domain = coalesce(var.internal_domain, format("us-%s.%s.azure.lnrsg.io", "var.metadata.product_name", "dev")) + external_storage_exists = local.external_storage_config != null - internal_storage_enabled = local.external_storage_exists == true && var.ignore_external_storage == true ? true : local.external_storage_exists == true && var.ignore_external_storage == false ? false : true - # external_services_storage_enabled = local.external_services_storage_exists == true && var.ignore_external_services_storage == false ? true : local.external_services_storage_exists == true && var.ignore_external_services_storage == true ? false : true + internal_storage_enabled = var.external_storage_desired == true ? false : true - hpcc_namespace = var.hpcc_namespace.existing_namespace != null ? var.hpcc_namespace.existing_namespace : var.hpcc_namespace.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : fileexists("${path.module}/logging/data/hpcc_namespace.txt") ? file("${path.module}/logging/data/hpcc_namespace.txt") : "default" - - external_storage_config = local.get_storage_config != null && var.ignore_external_storage == false ? [ + external_storage_config = local.get_storage_config != null && var.external_storage_desired == true ? [ for plane in local.get_storage_config.external_storage_config : { category = plane.category @@ -57,7 +46,18 @@ locals { prefix_name = plane.prefix_name } ] : [] + #--------------------------------------------------------------------------------------------------------------------------- + + subnet_ids = try({ + for k, v in local.use_existing_vnet.subnets : k => "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${local.use_existing_vnet.resource_group_name}/providers/Microsoft.Network/virtualNetworks/${local.use_existing_vnet.name}/subnets/${v.name}" + }, { aks = local.get_vnet_config.private_subnet_id }) + + location = local.use_existing_vnet != null ? local.use_existing_vnet.location : local.get_vnet_config.location + + # hpcc_chart_major_minor_point_version = local.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", local.helm_chart_version) : "master" + + domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) - svc_domains = { eclwatch = var.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } + svc_domains = { eclwatch = local.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true } diff --git a/hpcc/main.tf b/hpcc/main.tf index b9846df..c436605 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -17,24 +17,33 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market + market = local.metadata.market location = local.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = module.subscription.output.subscription_id - project = var.metadata.project + project = local.metadata.project } -resource "null_resource" "launch_svc_url" { - for_each = module.hpcc.hpcc_status == "deployed" ? local.svc_domains : {} +resource "null_resource" "delete_ephemeral_storage_accounts" { + count = var.external_storage_desired && (local.external_storage_config != []) ? 1 : 0 + + provisioner "local-exec" { + command = "scripts/delete_ephemeral_storage_accounts ${local.get_aks_config.resource_group_name}" + } + depends_on = [module.hpcc] +} + +/*resource "null_resource" "launch_svc_url" { + for_each = (module.hpcc.hpcc_status == "deployed") && (local.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} provisioner "local-exec" { command = local.is_windows_os ? "Start-Process ${each.value}" : "open ${each.value} || xdg-open ${each.value}" interpreter = local.is_windows_os ? ["PowerShell", "-Command"] : ["/bin/bash", "-c"] } -} +}*/ diff --git a/hpcc/misc.auto.tfvars.example b/hpcc/misc.auto.tfvars.example deleted file mode 100644 index c313216..0000000 --- a/hpcc/misc.auto.tfvars.example +++ /dev/null @@ -1,205 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing", "enclosed resource" = "hpcc" } - location = "eastus" # Acceptable values: eastus, centralus -} - -# # disable_naming_conventions - Disable naming conventions -# # disable_naming_conventions = true - -# # auto_launch_svc - Automatically launch ECLWatch web interface. -auto_launch_svc = { - eclwatch = true -} - -# azure_auth = { -# # AAD_CLIENT_ID = "" -# # AAD_CLIENT_SECRET = "" -# # AAD_TENANT_ID = "" -# # AAD_PRINCIPAL_ID = "" -# SUBSCRIPTION_ID = "" -# } - -# hpcc_container = { -# version = "9.2.0" -# image_name = "platform-core-ln" -# image_root = "jfrog.com/glb-docker-virtual" -# # custom_chart_version = "9.2.0-rc1" -# # custom_image_version = "9.2.0-demo" -# } - -# hpcc_container_registry_auth = { -# username = "value" -# password = "value" -# } - -internal_domain = "" // Example: hpccplatform-dev.azure.com - -# external = { -# blob_nfs = [{ -# container_id = "" -# container_name = "" -# id = "" -# resource_group_name = "" -# storage_account_id = "" -# storage_account_name = "" -# }] -# hpcc = [{ -# name = "" -# planes = list(object({ -# local = "" -# remote = "" -# })) -# service = "" -# }] -# } - -admin_services_storage_account_settings = { - replication_type = "ZRS" #LRS only if using HPC Cache - # authorized_ip_ranges = { - # "default" = "0.0.0.0/0" //must be public IP - # } - - delete_protection = false -} - -azure_log_analytics_creds = { - scope = null - object_id = "" //AAD_PRINCIPAL_ID -} - -hpcc_namespace = { - # existing_namespace = "" - labels = { - name = "hpcc" - } - create_namespace = true -} - -data_storage_config = { - internal = { - blob_nfs = { - data_plane_count = 2 - storage_account_settings = { - replication_type = "ZRS" - delete_protection = false - } - } - } - external = null -} - -# external_services_storage_config = [ -# { -# category = "dali" -# container_name = "hpcc-dali" -# path = "dalistorage" -# plane_name = "dali" -# size = 100 -# storage_type = "azurefiles" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "debug" -# container_name = "hpcc-debug" -# path = "debug" -# plane_name = "debug" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "dll" -# container_name = "hpcc-dll" -# path = "queries" -# plane_name = "dll" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "lz" -# container_name = "hpcc-mydropzone" -# path = "mydropzone" -# plane_name = "mydropzone" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# }, -# { -# category = "sasha" -# container_name = "hpcc-sasha" -# path = "sashastorage" -# plane_name = "sasha" -# size = 100 -# storage_type = "blobnfs" -# storage_account = "" -# resource_group_name = "" -# } -# ] - -ignore_external_data_storage = false -ignore_external_services_storage = false - -spill_volumes = { - spill = { - name = "spill" - size = 300 - prefix = "/var/lib/HPCCSystems/spill" - host_path = "/mnt" - storage_class = "spill" - access_mode = "ReadWriteOnce" - } -} - -spray_service_settings = { - replicas = 6 - nodeSelector = "spraypool" #"spraypool" -} - -# ldap = { -# ldap_server = "" //Server IP -# dali = { -# hpcc_admin_password = "" -# hpcc_admin_username = "" -# ldap_admin_password = "" -# ldap_admin_username = "" -# adminGroupName = "HPCC-Admins" -# filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# } -# esp = { -# hpcc_admin_password = "" -# hpcc_admin_username = "" -# ldap_admin_password = "" -# ldap_admin_username = "" -# adminGroupName = "HPCC-Admins" -# filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" -# workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" -# } -# } diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index 6e18fea..655cea3 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -1,9 +1,20 @@ -output "hpcc_namespace" { - description = "The namespace where the HPCC Platform is deployed." - value = local.hpcc_namespace +output "eclwatch_url" { + description = "Print the ECL Watch URL." + value = format("%s.%s:18010",var.a_record_name, var.aks_dns_zone_name) } -output "eclwatch" { - description = "Print the ECL Watch domain out." - value = local.svc_domains.eclwatch +output "deployment_resource_group" { + description = "Print the name of the deployment resource group." + value = local.get_aks_config.resource_group_name +} + +output "external_storage_config_exists" { + value = fileexists("../storage/data/config.json") ? true : false +} + +resource "local_file" "config" { + content = "hpcc successfully deployed" + filename = "${path.module}/data/config.json" + + depends_on = [ module.hpcc ] } diff --git a/hpcc/roxie.auto.tfvars.example b/hpcc/roxie.auto.tfvars.example deleted file mode 100644 index cfc1bf4..0000000 --- a/hpcc/roxie.auto.tfvars.example +++ /dev/null @@ -1,11 +0,0 @@ -# Roxie Settings -################# - -roxie_config = [{ - disabled = false - traceLevel = 3 - - services = [{ - annotations = { "service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path" = "/control/alive" } - }] -}] diff --git a/hpcc/sasha.auto.tfvars.example b/hpcc/sasha.auto.tfvars.example deleted file mode 100644 index e69de29..0000000 diff --git a/hpcc/scripts/delete_ephemeral_storage_accounts b/hpcc/scripts/delete_ephemeral_storage_accounts new file mode 100755 index 0000000..0b78f2c --- /dev/null +++ b/hpcc/scripts/delete_ephemeral_storage_accounts @@ -0,0 +1,16 @@ +#!/bin/bash +if [ "$1" != "" ];then + rg=$1 +else + echo "$0 has no arguments. It must of 1 argument that is the name of a resource group. EXITING.";exit 1; +fi +echo "In $0. Inputted resource groups is \"$rg\""; +sleep 20; +estos=`az resource list --resource-group $rg|egrep "id\":.*storageAccounts\/hpcc"|sed "s/^ *\"id\": \"//"|sed "s/\", *$//"` +if [ "$estos" == "" ];then + echo "In $0 There are no hpcc storage accounts in the resource group, $rg. EXITING.";exit; +fi +for s in $estos;do + echo "Deleting storage account: $s" + az storage account delete --ids $s -y +done diff --git a/hpcc/thor.auto.tfvars.example b/hpcc/thor.auto.tfvars.example deleted file mode 100644 index 34ff029..0000000 --- a/hpcc/thor.auto.tfvars.example +++ /dev/null @@ -1,34 +0,0 @@ -# Thor Settings -################ - -thor_config = [{ - disabled = false - name = "thor" - prefix = "thor" - numWorkers = 5 - keepJobs = "none" - maxJobs = 4 - maxGraphs = 2 - maxGraphStartupTime = 172800 - numWorkersPerPod = 1 - # nodeSelector = {} - managerResources = { - cpu = 1 - memory = "2G" - } - workerResources = { - cpu = 3 - memory = "4G" - } - workerMemory = { - query = "3G" - thirdParty = "500M" - } - eclAgentResources = { - cpu = 1 - memory = "2G" - } - cost = { - perCpu = 1 - } -}] diff --git a/hpcc/variables.tf b/hpcc/variables.tf deleted file mode 100644 index 60de38c..0000000 --- a/hpcc/variables.tf +++ /dev/null @@ -1,1151 +0,0 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - -# variable "azure_auth" { -# description = "Azure authentication" -# type = object({ -# AAD_CLIENT_ID = optional(string) -# AAD_CLIENT_SECRET = optional(string) -# AAD_TENANT_ID = optional(string) -# AAD_PRINCIPAL_ID = optional(string) -# SUBSCRIPTION_ID = string -# }) - -# nullable = false -# } - -variable "expose_services" { - description = "Expose ECLWatch and elastic4hpcclogs to the Internet. This is not secure. Please consider before using it." - type = bool - default = false -} - -variable "auto_launch_svc" { - description = "Auto launch HPCC services." - type = object({ - eclwatch = bool - }) - default = { - eclwatch = true - } -} - -variable "auto_connect" { - description = "Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context." - type = bool - default = false -} - -variable "disable_naming_conventions" { - description = "Naming convention module." - type = bool - default = false -} - -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - -variable "use_existing_vnet" { - description = "Information about the existing VNet to use. Overrides vnet variable." - type = object({ - name = string - resource_group_name = string - route_table_name = string - location = string - - subnets = object({ - aks = object({ - name = string - }) - }) - }) - - default = null -} - -## HPCC Helm Release -####################### -variable "hpcc_enabled" { - description = "Is HPCC Platform deployment enabled?" - type = bool - default = true -} - -variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - existing_namespace = optional(string) - labels = optional(map(string), { name = "hpcc" }) - create_namespace = optional(bool, true) - }) - default = {} -} - -variable "helm_chart_strings_overrides" { - description = "Helm chart values as strings, in yaml format, to be merged last." - type = list(string) - default = [] -} - -variable "helm_chart_files_overrides" { - description = "Helm chart values files, in yaml format, to be merged." - type = list(string) - default = [] -} - -variable "helm_chart_timeout" { - description = "Helm timeout for hpcc chart." - type = number - default = 300 -} - -variable "hpcc_container" { - description = "HPCC container information (if version is set to null helm chart version is used)." - type = object({ - image_name = optional(string) - image_root = optional(string) - version = optional(string) - custom_chart_version = optional(string) - custom_image_version = optional(string) - }) - - default = null -} - -variable "hpcc_container_registry_auth" { - description = "Registry authentication for HPCC container." - type = object({ - password = string - username = string - }) - default = null - sensitive = true -} - -variable "vault_config" { - description = "Input for vault secrets." - type = object({ - git = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })), - ecl = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })), - ecluser = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })) - esp = map(object({ - name = optional(string) - url = optional(string) - kind = optional(string) - vault_namespace = optional(string) - role_id = optional(string) - secret_name = optional(string) # Should match the secret name created in the corresponding vault_secrets variable - })) - }) - default = null -} - -## Roxie Config -################## -variable "roxie_config" { - description = "Configuration for Roxie(s)." - type = list(object({ - disabled = bool - name = string - nodeSelector = map(string) - numChannels = number - prefix = string - replicas = number - serverReplicas = number - acePoolSize = number - actResetLogPeriod = number - affinity = number - allFilesDynamic = bool - blindLogging = bool - blobCacheMem = number - callbackRetries = number - callbackTimeout = number - checkCompleted = bool - checkPrimaries = bool - checkFileDate = bool - clusterWidth = number - copyResources = bool - coresPerQuery = number - crcResources = bool - dafilesrvLookupTimeout = number - debugPermitted = bool - defaultConcatPreload = number - defaultFetchPreload = number - defaultFullKeyedJoinPreload = number - defaultHighPriorityTimeLimit = number - defaultHighPriorityTimeWarning = number - defaultKeyedJoinPreload = number - defaultLowPriorityTimeLimit = number - defaultLowPriorityTimeWarning = number - defaultMemoryLimit = number - defaultParallelJoinPreload = number - defaultPrefetchProjectPreload = number - defaultSLAPriorityTimeLimit = number - defaultSLAPriorityTimeWarning = number - defaultStripLeadingWhitespace = bool - diskReadBufferSize = number - doIbytiDelay = bool - egress = string - enableHeartBeat = bool - enableKeyDiff = bool - enableSysLog = bool - fastLaneQueue = bool - fieldTranslationEnabled = string - flushJHtreeCacheOnOOM = bool - forceStdLog = bool - highTimeout = number - ignoreMissingFiles = bool - indexReadChunkSize = number - initIbytiDelay = number - jumboFrames = bool - lazyOpen = bool - leafCacheMem = number - linuxYield = bool - localFilesExpire = number - localSlave = bool - logFullQueries = bool - logQueueDrop = number - logQueueLen = number - lowTimeout = number - maxBlockSize = number - maxHttpConnectionRequests = number - maxLocalFilesOpen = number - maxLockAttempts = number - maxRemoteFilesOpen = number - memTraceLevel = number - memTraceSizeLimit = number - memoryStatsInterval = number - minFreeDiskSpace = number - minIbytiDelay = number - minLocalFilesOpen = number - minRemoteFilesOpen = number - miscDebugTraceLevel = number - monitorDaliFileServer = bool - nodeCacheMem = number - nodeCachePreload = bool - parallelAggregate = number - parallelLoadQueries = number - perChannelFlowLimit = number - pingInterval = number - preabortIndexReadsThreshold = number - preabortKeyedJoinsThreshold = number - preloadOnceData = bool - prestartSlaveThreads = bool - remoteFilesExpire = number - roxieMulticastEnabled = bool - serverSideCacheSize = number - serverThreads = number - simpleLocalKeyedJoins = bool - sinkMode = string - slaTimeout = number - slaveConfig = string - slaveThreads = number - soapTraceLevel = number - socketCheckInterval = number - statsExpiryTime = number - systemMonitorInterval = number - traceLevel = number - traceRemoteFiles = bool - totalMemoryLimit = string - trapTooManyActiveQueries = bool - udpAdjustThreadPriorities = bool - udpFlowAckTimeout = number - udpFlowSocketsSize = number - udpInlineCollation = bool - udpInlineCollationPacketLimit = number - udpLocalWriteSocketSize = number - udpMaxPermitDeadTimeouts = number - udpMaxRetryTimedoutReqs = number - udpMaxSlotsPerClient = number - udpMulticastBufferSize = number - udpOutQsPriority = number - udpQueueSize = number - udpRecvFlowTimeout = number - udpRequestToSendAckTimeout = number - udpResendTimeout = number - udpRequestToSendTimeout = number - udpResendEnabled = bool - udpRetryBusySenders = number - udpSendCompletedInData = bool - udpSendQueueSize = number - udpSnifferEnabled = bool - udpTraceLevel = number - useAeron = bool - useDynamicServers = bool - useHardLink = bool - useLogQueue = bool - useRemoteResources = bool - useMemoryMappedIndexes = bool - useTreeCopy = bool - services = list(object({ - name = string - servicePort = number - listenQueue = number - numThreads = number - visibility = string - annotations = optional(map(string)) - })) - topoServer = object({ - replicas = number - }) - channelResources = object({ - cpu = string - memory = string - }) - })) - - default = [ - { - disabled = false - name = "roxie" - nodeSelector = {} - numChannels = 2 - prefix = "roxie" - replicas = 2 - serverReplicas = 0 - acePoolSize = 6 - actResetLogPeriod = 0 - affinity = 0 - allFilesDynamic = false - blindLogging = false - blobCacheMem = 0 - callbackRetries = 3 - callbackTimeout = 500 - checkCompleted = true - checkFileDate = false - checkPrimaries = true - clusterWidth = 1 - copyResources = true - coresPerQuery = 0 - crcResources = false - dafilesrvLookupTimeout = 10000 - debugPermitted = true - defaultConcatPreload = 0 - defaultFetchPreload = 0 - defaultFullKeyedJoinPreload = 0 - defaultHighPriorityTimeLimit = 0 - defaultHighPriorityTimeWarning = 30000 - defaultKeyedJoinPreload = 0 - defaultLowPriorityTimeLimit = 0 - defaultLowPriorityTimeWarning = 90000 - defaultMemoryLimit = 1073741824 - defaultParallelJoinPreload = 0 - defaultPrefetchProjectPreload = 10 - defaultSLAPriorityTimeLimit = 0 - defaultSLAPriorityTimeWarning = 30000 - defaultStripLeadingWhitespace = false - diskReadBufferSize = 65536 - doIbytiDelay = true - egress = "engineEgress" - enableHeartBeat = false - enableKeyDiff = false - enableSysLog = false - fastLaneQueue = true - fieldTranslationEnabled = "payload" - flushJHtreeCacheOnOOM = true - forceStdLog = false - highTimeout = 2000 - ignoreMissingFiles = false - indexReadChunkSize = 60000 - initIbytiDelay = 10 - jumboFrames = false - lazyOpen = true - leafCacheMem = 500 - linuxYield = false - localFilesExpire = 1 - localSlave = false - logFullQueries = false - logQueueDrop = 32 - logQueueLen = 512 - lowTimeout = 10000 - maxBlockSize = 1000000000 - maxHttpConnectionRequests = 1 - maxLocalFilesOpen = 4000 - maxLockAttempts = 5 - maxRemoteFilesOpen = 100 - memTraceLevel = 1 - memTraceSizeLimit = 0 - memoryStatsInterval = 60 - minFreeDiskSpace = 6442450944 - minIbytiDelay = 2 - minLocalFilesOpen = 2000 - minRemoteFilesOpen = 50 - miscDebugTraceLevel = 0 - monitorDaliFileServer = false - nodeCacheMem = 1000 - nodeCachePreload = false - parallelAggregate = 0 - parallelLoadQueries = 1 - perChannelFlowLimit = 50 - pingInterval = 0 - preabortIndexReadsThreshold = 100 - preabortKeyedJoinsThreshold = 100 - preloadOnceData = true - prestartSlaveThreads = false - remoteFilesExpire = 3600 - roxieMulticastEnabled = false - serverSideCacheSize = 0 - serverThreads = 100 - simpleLocalKeyedJoins = true - sinkMode = "sequential" - slaTimeout = 2000 - slaveConfig = "simple" - slaveThreads = 30 - soapTraceLevel = 1 - socketCheckInterval = 5000 - statsExpiryTime = 3600 - systemMonitorInterval = 60000 - totalMemoryLimit = "5368709120" - traceLevel = 1 - traceRemoteFiles = false - trapTooManyActiveQueries = true - udpAdjustThreadPriorities = true - udpFlowAckTimeout = 10 - udpFlowSocketsSize = 33554432 - udpInlineCollation = true - udpInlineCollationPacketLimit = 50 - udpLocalWriteSocketSize = 16777216 - udpMaxPermitDeadTimeouts = 100 - udpMaxRetryTimedoutReqs = 10 - udpMaxSlotsPerClient = 100 - udpMulticastBufferSize = 33554432 - udpOutQsPriority = 5 - udpQueueSize = 1000 - udpRecvFlowTimeout = 2000 - udpRequestToSendAckTimeout = 500 - udpResendTimeout = 100 - udpRequestToSendTimeout = 2000 - udpResendEnabled = true - udpRetryBusySenders = 0 - udpSendCompletedInData = false - udpSendQueueSize = 500 - udpSnifferEnabled = false - udpTraceLevel = 0 - useAeron = false - useDynamicServers = false - useHardLink = false - useLogQueue = true - useMemoryMappedIndexes = false - useRemoteResources = false - useTreeCopy = false - services = [ - { - name = "roxie" - servicePort = 9876 - listenQueue = 200 - numThreads = 30 - visibility = "local" - annotations = {} - } - ] - topoServer = { - replicas = 1 - } - channelResources = { - cpu = "1" - memory = "4G" - } - } - ] -} - -## Thor Config -################## -variable "thor_config" { - description = "Configurations for Thor." - type = list(object( - { - disabled = bool - eclAgentResources = optional(object({ - cpu = string - memory = string - } - ), - { - cpu = 1 - memory = "2G" - }) - keepJobs = optional(string, "none") - managerResources = optional(object({ - cpu = string - memory = string - }), - { - cpu = 1 - memory = "2G" - }) - maxGraphs = optional(number, 2) - maxJobs = optional(number, 4) - maxGraphStartupTime = optional(number, 172800) - name = optional(string, "thor") - nodeSelector = optional(map(string), { workload = "thorpool" }) - numWorkers = optional(number, 2) - numWorkersPerPod = optional(number, 1) - prefix = optional(string, "thor") - egress = optional(string, "engineEgress") - tolerations_value = optional(string, "thorpool") - workerMemory = optional(object({ - query = string - thirdParty = string - }), - { - query = "3G" - thirdParty = "500M" - }) - workerResources = optional(object({ - cpu = string - memory = string - }), - { - cpu = 3 - memory = "4G" - }) - cost = object({ - perCpu = number - }) - })) - - default = null -} - -## ECL Agent Config -####################### -variable "eclagent_settings" { - description = "eclagent settings" - type = map(object({ - replicas = number - maxActive = number - prefix = string - use_child_process = bool - spillPlane = optional(string, "spill") - type = string - resources = object({ - cpu = string - memory = string - }) - cost = object({ - perCpu = number - }) - egress = optional(string) - })) - - default = { - hthor = { - replicas = 1 - maxActive = 4 - prefix = "hthor" - use_child_process = false - type = "hthor" - spillPlane = "spill" - resources = { - cpu = "1" - memory = "4G" - } - egress = "engineEgress" - cost = { - perCpu = 1 - } - }, - } -} - -## ECLCCServer Config -######################## -variable "eclccserver_settings" { - description = "Set cpu and memory values of the eclccserver. Toggle use_child_process to true to enable eclccserver child processes." - type = map(object({ - useChildProcesses = optional(bool, false) - replicas = optional(number, 1) - maxActive = optional(number, 4) - egress = optional(string, "engineEgress") - gitUsername = optional(string, "") - defaultRepo = optional(string, "") - defaultRepoVersion = optional(string, "") - resources = optional(object({ - cpu = string - memory = string - })) - cost = object({ - perCpu = number - }) - listen_queue = optional(list(string), []) - childProcessTimeLimit = optional(number, 10) - gitUsername = optional(string, "") - legacySyntax = optional(bool, false) - options = optional(list(object({ - name = string - value = string - }))) - })) - - default = { - "myeclccserver" = { - useChildProcesses = false - maxActive = 4 - egress = "engineEgress" - replicas = 1 - childProcessTimeLimit = 10 - resources = { - cpu = "1" - memory = "4G" - } - legacySyntax = false - options = [] - cost = { - perCpu = 1 - } - } } -} - -## Dali Config -################## -variable "dali_settings" { - description = "dali settings" - type = object({ - coalescer = object({ - interval = number - at = string - minDeltaSize = number - resources = object({ - cpu = string - memory = string - }) - }) - resources = object({ - cpu = string - memory = string - }) - maxStartupTime = number - }) - - default = { - coalescer = { - interval = 24 - at = "* * * * *" - minDeltaSize = 50000 - resources = { - cpu = "1" - memory = "4G" - } - } - resources = { - cpu = "2" - memory = "8G" - } - maxStartupTime = 1200 - } -} - -## DFU Server Config -######################## -variable "dfuserver_settings" { - description = "DFUServer settings" - type = object({ - maxJobs = number - resources = object({ - cpu = string - memory = string - }) - }) - - default = { - maxJobs = 3 - resources = { - cpu = "1" - memory = "2G" - } - } -} - -## Spray Service Config -######################### -variable "spray_service_settings" { - description = "spray services settings" - type = object({ - replicas = number - nodeSelector = string - }) - - default = { - replicas = 3 - nodeSelector = "servpool" #"spraypool" - } -} - -## Sasha Config -################## -variable "sasha_config" { - description = "Configuration for Sasha." - type = object({ - disabled = bool - wu-archiver = object({ - disabled = bool - service = object({ - servicePort = number - }) - plane = string - interval = number - limit = number - cutoff = number - backup = number - at = string - throttle = number - retryinterval = number - keepResultFiles = bool - # egress = string - }) - - dfuwu-archiver = object({ - disabled = bool - service = object({ - servicePort = number - }) - plane = string - interval = number - limit = number - cutoff = number - at = string - throttle = number - # egress = string - }) - - dfurecovery-archiver = object({ - disabled = bool - interval = number - limit = number - cutoff = number - at = string - # egress = string - }) - - file-expiry = object({ - disabled = bool - interval = number - at = string - persistExpiryDefault = number - expiryDefault = number - user = string - # egress = string - }) - }) - - default = { - disabled = false - wu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 6 - limit = 400 - cutoff = 3 - backup = 0 - at = "* * * * *" - throttle = 0 - retryinterval = 6 - keepResultFiles = false - # egress = "engineEgress" - } - - dfuwu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 24 - limit = 100 - cutoff = 14 - at = "* * * * *" - throttle = 0 - # egress = "engineEgress" - } - - dfurecovery-archiver = { - disabled = false - interval = 12 - limit = 20 - cutoff = 4 - at = "* * * * *" - # egress = "engineEgress" - } - - file-expiry = { - disabled = false - interval = 1 - at = "* * * * *" - persistExpiryDefault = 7 - expiryDefault = 4 - user = "sasha" - # egress = "engineEgress" - } - } -} - -## LDAP Config -################## -variable "ldap_config" { - description = "LDAP settings for dali and esp services." - type = object({ - dali = object({ - adminGroupName = string - filesBasedn = string - groupsBasedn = string - hpcc_admin_password = string - hpcc_admin_username = string - ldap_admin_password = string - ldap_admin_username = string - ldapAdminVaultId = string - resourcesBasedn = string - sudoersBasedn = string - systemBasedn = string - usersBasedn = string - workunitsBasedn = string - ldapCipherSuite = string - }) - esp = object({ - adminGroupName = string - filesBasedn = string - groupsBasedn = string - ldap_admin_password = string - ldap_admin_username = string - ldapAdminVaultId = string - resourcesBasedn = string - sudoersBasedn = string - systemBasedn = string - usersBasedn = string - workunitsBasedn = string - ldapCipherSuite = string - }) - ldap_server = string - }) - - default = null - sensitive = true -} - -variable "ldap_tunables" { - description = "Tunable settings for LDAP." - type = object({ - cacheTimeout = number - checkScopeScans = bool - ldapTimeoutSecs = number - maxConnections = number - passwordExpirationWarningDays = number - sharedCache = bool - }) - - default = { - cacheTimeout = 5 - checkScopeScans = false - ldapTimeoutSecs = 131 - maxConnections = 10 - passwordExpirationWarningDays = 10 - sharedCache = true - } -} - -## Data Storage Config -######################### -variable "install_blob_csi_driver" { - description = "Install blob-csi-drivers on the cluster." - type = bool - default = true -} - -variable "spill_volumes" { - description = "Map of objects to create Spill Volumes" - type = map(object({ - name = string # "Name of spill volume to be created." - size = number # "Size of spill volume to be created (in GB)." - prefix = string # "Prefix of spill volume to be created." - host_path = string # "Host path on spill volume to be created." - storage_class = string # "Storage class of spill volume to be used." - access_mode = string # "Access mode of spill volume to be used." - })) - - default = { - "spill" = { - name = "spill" - size = 300 - prefix = "/var/lib/HPCCSystems/spill" - host_path = "/mnt" - storage_class = "spill" - access_mode = "ReadWriteOnce" - } - } -} - -variable "data_storage_config" { - description = "Data plane config for HPCC." - type = object({ - internal = optional(object({ - blob_nfs = object({ - data_plane_count = number - storage_account_settings = object({ - # authorized_ip_ranges = map(string) - delete_protection = bool - replication_type = string - # subnet_ids = map(string) - blob_soft_delete_retention_days = optional(number) - container_soft_delete_retention_days = optional(number) - }) - }) - })) - - external = optional(object({ - blob_nfs = list(object({ - container_id = string - container_name = string - id = string - resource_group_name = string - storage_account_id = string - storage_account_name = string - })) - hpcc = list(object({ - name = string - planes = list(object({ - local = string - remote = string - })) - service = string - })) - })) - }) - - default = null - # default = { - # internal = { - # blob_nfs = { - # data_plane_count = 1 - # storage_account_settings = { - # # authorized_ip_ranges = {} - # delete_protection = false - # replication_type = "ZRS" - # # subnet_ids = {} - # blob_soft_delete_retention_days = 7 - # container_soft_delete_retention_days = 7 - # } - # } - # } - # external = null - # } -} - -variable "external_storage_config" { - description = "External services storage config." - type = list(object({ - category = string - container_name = string - path = string - plane_name = string - protocol = string - resource_group = string - size = number - storage_account = string - storage_type = string - prefix_name = string - })) - - default = null -} - -variable "remote_storage_plane" { - description = "Input for attaching remote storage plane" - type = map(object({ - dfs_service_name = string - dfs_secret_name = string - target_storage_accounts = map(object({ - name = string - prefix = string - })) - })) - - default = null -} - -variable "onprem_lz_settings" { - description = "Input for allowing OnPrem LZ." - type = map(object({ - prefix = string - hosts = list(string) - })) - - default = {} -} - -variable "admin_services_storage" { - description = "PV sizes for admin service planes in gigabytes (storage billed only as consumed)." - type = object({ - dali = object({ - size = number - type = string - }) - debug = object({ - size = number - type = string - }) - dll = object({ - size = number - type = string - }) - lz = object({ - size = number - type = string - }) - sasha = object({ - size = number - type = string - }) - }) - - default = { - dali = { - size = 100 - type = "azurefiles" - } - debug = { - size = 100 - type = "blobnfs" - } - dll = { - size = 100 - type = "blobnfs" - } - lz = { - size = 100 - type = "blobnfs" - } - sasha = { - size = 100 - type = "blobnfs" - } - } - - validation { - condition = length([for k, v in var.admin_services_storage : v.type if !contains(["azurefiles", "blobnfs"], v.type)]) == 0 - error_message = "The type must be either \"azurefiles\" or \"blobnfs\"." - } - - validation { - condition = length([for k, v in var.admin_services_storage : v.size if v.type == "azurefiles" && v.size < 100]) == 0 - error_message = "Size must be at least 100 for \"azurefiles\" type." - } -} - -variable "admin_services_storage_account_settings" { - description = "Settings for admin services storage account." - type = object({ - authorized_ip_ranges = optional(map(string)) - delete_protection = bool - replication_type = string - # subnet_ids = map(string) - blob_soft_delete_retention_days = optional(number) - container_soft_delete_retention_days = optional(number) - file_share_retention_days = optional(number) - }) - - default = { - authorized_ip_ranges = {} - delete_protection = false - replication_type = "ZRS" - subnet_ids = {} - blob_soft_delete_retention_days = 7 - container_soft_delete_retention_days = 7 - file_share_retention_days = 7 - } -} - -variable "ignore_external_storage" { - description = "Should storage created using the storage module or var.external_storage_config be ignored?" - type = bool - default = false -} - -## Node Selector -#################### -variable "admin_services_node_selector" { - description = "Node selector for admin services pods." - type = map(map(string)) - default = {} - - validation { - condition = length([for service in keys(var.admin_services_node_selector) : - service if !contains(["all", "dali", "esp", "eclagent", "eclccserver"], service)]) == 0 - error_message = "The keys must be one of \"all\", \"dali\", \"esp\", \"eclagent\" or \"eclccserver\"." - } -} - -## DNS -######### -variable "internal_domain" { - description = "DNS Domain name" - type = string - default = null -} diff --git a/lite-variables.tf b/lite-variables.tf new file mode 100644 index 0000000..a14c506 --- /dev/null +++ b/lite-variables.tf @@ -0,0 +1,212 @@ +############################################################################### +# Prompted variables (user will be asked to supply them at plan/apply time +# if a .tfvars file is not supplied); there are no default values +############################################################################### +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + +variable "external_storage_desired" { + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" + type = bool + default = false +} + +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool + default = true +} + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool + default = false +} + +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "REQUIRED. Name of the resource group containing the dns zone." +} + +variable "aks_dns_zone_name" { + type = string + description = "REQUIRED. dns zone name. The name of existing dns zone." +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } + default = "latest" +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} + +variable "storage_data_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_data_gb >= 10 + error_message = "Value must be 10 or more." + } + default = 100 +} + +variable "storage_lz_gb" { + type = number + description = "REQUIRED. The amount of storage reserved for the landing zone in gigabytes.\nMust be 1 or more.\nIf a storage account is defined (see below) then this value is ignored." + validation { + condition = var.storage_lz_gb >= 1 + error_message = "Value must be 1 or more." + } + default = 25 +} + +variable "thor_max_jobs" { + type = number + description = "REQUIRED. The maximum number of simultaneous Thor jobs allowed.\nMust be 1 or more." + validation { + condition = var.thor_max_jobs >= 1 + error_message = "Value must be 1 or more." + } + default = 2 +} + +variable "thor_num_workers" { + type = number + description = "REQUIRED. The number of Thor workers to allocate.\nMust be 1 or more." + validation { + condition = var.thor_num_workers >= 1 + error_message = "Value must be 1 or more." + } + default = 2 +} + +############################################################################### +# Optional variables +############################################################################### + +variable "authn_htpasswd_filename" { + type = string + description = "OPTIONAL. If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it.\nA corollary is that persistent storage is enabled.\nAn empty string indicates that htpasswd is not to be used for authentication.\nExample entry: htpasswd.txt" + default = "" +} + +variable "hpcc_namespace" { + description = "Kubernetes namespace where resources will be created." + type = object({ + prefix_name = string + labels = map(string) + create_namespace = bool + }) + default = { + prefix_name = "hpcc" + labels = { + name = "hpcc" + } + create_namespace = false + } +} + +variable "enable_premium_storage" { + type = bool + description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." + default = false +} + +variable "storage_account_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" + default = "" +} + +variable "storage_account_resource_group_name" { + type = string + description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." + default = "" +} diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example new file mode 100644 index 0000000..3eeb70d --- /dev/null +++ b/lite.auto.tfvars.example @@ -0,0 +1,234 @@ +#----------------------------------------------------------------------------- + +# Name of the A record, of following dns zone, where the ecl watch ip is placed +# This A record will be created and therefore should not exist in the following +# dns zone. +# Example entry: "my-product". This should be something project specific rather +# than something generic. + +a_record_name="" + +#----------------------------------------------------------------------------- + +# Name of an existing dns zone. +# Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" +# REQUIRED + +aks_dns_zone_name="" + +#----------------------------------------------------------------------------- + +# Name of the resource group of the above dns zone. +# Example entry: "app-dns-prod-eastus2" +# REQUIRED + +aks_dns_zone_resource_group_name="" + +#------------------------------------------------------------------------------ + +# The version of HPCC Systems to install. +# Only versions in nn.nn.nn format are supported. +# Value type: string + +hpcc_version="9.4.4" + +#------------------------------------------------------------------------------ + +# Enable ROXIE? +# This will also expose port 8002 on the cluster. +# Value type: boolean +# Example entry: false + +aks_enable_roxie=true + +#------------------------------------------------------------------------------ + +# Enable code security? +# If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. +# Value type: boolean +# Example entry: false + +enable_code_security=true + +#------------------------------------------------------------------------------ + +# If you want a thor cluster then 'enable_thor' must be set to true +# Otherwise it is set to false + +enable_thor=true + +#------------------------------------------------------------------------------ + +# The number of Thor workers to allocate. +# Must be 1 or more. + +thor_num_workers=2 + +#------------------------------------------------------------------------------ + +# The maximum number of simultaneous Thor jobs allowed. +# Must be 1 or more. + +thor_max_jobs=2 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for the landing zone in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_lz_gb=25 + +#------------------------------------------------------------------------------ + +# The amount of storage reserved for data in gigabytes. +# Must be 1 or more. +# If a storage account is defined (see below) then this value is ignored. + +storage_data_gb=100 + +#------------------------------------------------------------------------------ + +# Map of name => value tags that can will be associated with the cluster. +# Format is '{"name"="value" [, "name"="value"]*}'. +# The 'name' portion must be unique. +# To add no tags, use '{}'. +# Value type: map of string +# Example: extra_tags={ "owner"="Jane Doe", "owner_email"="jane.doe@gmail.com" } + +extra_tags={} + +#------------------------------------------------------------------------------ + +# The VM size for each node in the HPCC Systems node pool. +# Recommend "Standard_B4ms" or better. +# See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. +# Value type: string + +aks_node_size="Standard_B8ms" + +#------------------------------------------------------------------------------ + +# The maximum number of VM nodes to allocate for the HPCC Systems node pool. +# Must be 2 or more. +# Value type: integer + +aks_max_node_count=4 + +#------------------------------------------------------------------------------ + +# Email address of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jane.doe@hpccsystems.com" + +aks_admin_email="jane.doe@gmail.com" + +#------------------------------------------------------------------------------ + +# Name of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "Jane Doe" + +aks_admin_name="Jane Doe" + +#------------------------------------------------------------------------------ + +# Username of the administrator of this HPCC Systems cluster. +# Value type: string +# Example entry: "jdoe" + +admin_username="jdoe" + +#------------------------------------------------------------------------------ + +# The Azure region abbreviation in which to create these resources. +# Must be one of ["eastus", "eastus2", "centralus"]. +# Value type: string +# Example entry: "eastus" + +aks_azure_region="eastus" + +#------------------------------------------------------------------------------ + +# Map of name => CIDR IP addresses that can administrate this AKS. +# Format is '{"name"="cidr" [, "name"="cidr"]*}'. +# The 'name' portion must be unique. +# To add no CIDR addresses, use '{}'. +# The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. +# Value type: map of string + +aks_admin_ip_cidr_map={} + +#------------------------------------------------------------------------------ + +# List of explicit CIDR addresses that can access this HPCC Systems cluster. +# To allow public access, value should be ["0.0.0.0/0"] or []. +# Value type: list of string + +hpcc_user_ip_cidr_list=[] + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a resource group for the storage account. +# Value type: string +# Example entry: "my-product-sa" + +storage_account_name="" + +#------------------------------------------------------------------------------ + +# If you are attaching to an existing storage account, put its resource group name here. +# Leave as an empty string if you do not have a storage account. +# If you put something here then you must also define a name for the storage account. +# Value type: string + +storage_account_resource_group_name="" + +#------------------------------------------------------------------------------ + +# The Kubernetes namespace in which to install the HPCC modules (if enabled). +# Default value: "default" + +# hpcc_namespace="default" + +#------------------------------------------------------------------------------ + +# If true, premium ($$$) storage will be used for the following storage shares: Dali. +# OPTIONAL, defaults to false. + +enable_premium_storage=false + +#------------------------------------------------------------------------------ + +# If you would like to use htpasswd to authenticate users to the cluster, enter +# the filename of the htpasswd file. This file should be uploaded to the Azure +# 'dllsshare' file share in order for the HPCC processes to find it. +# A corollary is that persistent storage is enabled. +# An empty string indicates that htpasswd is not to be used for authentication. +# Example entry: "htpasswd.txt" + +authn_htpasswd_filename="" + +#------------------------------------------------------------------------------ + +# If you want external storage instead of ephemeral storage then +# set this variable to true otherwise set it to false. + +external_storage_desired=false + +#------------------------------------------------------------------------------ + +# This variable enable you to ask for logging and monitoring of the kubernetes +# and hpcc cluster (true means enable logging and monitoring, false means don't. + +aks_logging_monitoring_enabled=false + +#------------------------------------------------------------------------------ + +# Put your azure account id here. It will look like the following: +# 6c5edc79-34fd-333a-9b59-61ec21d7e42d + +my_azure_id="" + diff --git a/main.tf b/main.tf new file mode 100644 index 0000000..3bc29e3 --- /dev/null +++ b/main.tf @@ -0,0 +1,45 @@ +resource "null_resource" "deploy_vnet" { + + provisioner "local-exec" { + command = "scripts/deploy vnet" + } +} + +resource "null_resource" "deploy_aks" { + + provisioner "local-exec" { + command = "scripts/deploy aks ${var.my_azure_id}" + } + + depends_on = [ null_resource.deploy_vnet ] +} + +resource "null_resource" "deploy_storage" { + count = (var.external_storage_desired == true)? 1 : 0 + + provisioner "local-exec" { + command = "scripts/deploy storage" + } + + depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] +} + +resource "null_resource" "external_storage" { + count = (var.external_storage_desired == true)? 1 : 0 + + provisioner "local-exec" { + command = "scripts/external_storage ${path.module} ${var.external_storage_desired}" + } + + #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] + depends_on = [ null_resource.deploy_vnet ] +} + +resource "null_resource" "deploy_hpcc" { + + provisioner "local-exec" { + command = "scripts/deploy hpcc" + } + + depends_on = [ null_resource.deploy_aks, null_resource.deploy_vnet, null_resource.external_storage ] +} diff --git a/providers.tf b/providers.tf new file mode 100644 index 0000000..e27a382 --- /dev/null +++ b/providers.tf @@ -0,0 +1,5 @@ +provider "azurerm" { + features {} + use_cli = true + storage_use_azuread = true +} diff --git a/scripts/deploy b/scripts/deploy new file mode 100755 index 0000000..00f9ae7 --- /dev/null +++ b/scripts/deploy @@ -0,0 +1,119 @@ +#!/bin/bash +thisdir=`pwd`/`dirname $0` +repodir=$(echo $thisdir|sed "s/\/scripts\/*//") +if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then + name=$1 +else + echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'storage' or 'aks' or 'hpcc'. EXITING.";exit 1; +fi +if [ "$1" == "aks" ];then + if [ "$2" != "" ];then + my_azure_id=$2 + cp -v $thisdir/needed-auto-tfvars-files/aks/aks.auto.tfvars.example $repodir/aks/aks.auto.tfvars + cp -v $thisdir/needed-auto-tfvars-files/aks/misc.auto.tfvars.example $repodir/aks/misc.auto.tfvars + sed -i "s//$my_azure_id/" $repodir/aks/aks.auto.tfvars + else + echo "In $0. First argument was \"$name\". There should be a 2nd argument (my_azure_id). But it was missing. EXITING.";exit 1; + fi +fi +#======================================================================== +function assert_fail () { + echo ">>>>>>>>>>>>>>> EXECUTING: $*" + if "$@"; then + echo;echo ">>>>>>>>>>>>>>> Successful: $*";echo + else + echo;echo ">>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + rm -vrf data + cd $repodir + rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform + exit 1 + fi +} +#======================================================================== +# If kubernetes cluster doesn't exist then make sure aks/data/config.json +# and hpcc/data/config.json doesn't exist +# Delete both hpcc/data and aks/data if kubernetes cluster doesn't exist +ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` +pods=`kubectl get pods 2>&1` +if [[ "$ns" == *"Unable"* ]];then # kubenetes doesn't exist of there are no namespaces + rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. + rm -vrf aks/data;cd aks; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. +# If kubernetes cluster does exist but there are no pods in the default namespace +# then delete only hpcc/data +elif [[ "$pods" == *"Unable"* ]] || [[ "$pods" == *"No resources found in default namespace"* ]];then + # force rm data/config.json in hpcc only + echo "Forcibly delete hpcc/data only" + rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. +fi +#------------------------------------------------------------------------ +# See if $dir/data/config.json exists, where $dir is vnet or aks or storage. +# If $dir/data/config.json does exist then from the file get +# the resource group name and then check to see if resource group exists. +# if it doesn't exists then delete $dir/data/config.json +#------------------------------------------------------------------------ +for dir in aks vnet storage;do + if [ -e "$dir/data/config.json" ];then + # Get resource group name from file + rg=`$thisdir/get_rg_from_file $dir/data/config.json` + rgexist=`az group exists --name $rg` + if [ "$rgexist" == "false" ]; then + echo "In $0. $dir resource group, \"$rg\" does not exists. So deleting vnet/data and any tfstate files id $dir." + rm -vrf $dir/data + cd $dir; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. + else + echo "In $0. $dir resource group, \"$rg\" DOES exists. So NOT deleting vnet/data." + fi + fi +done +#------------------------------------------------------------------------ + +cd $name; # cd into vnet or storage or aks or hpcc + +# put the root directory's lite.auto.tfvars (either all or part) in either aks or hpcc +# directory. +if [ -e "../lite.auto.tfvars" ];then + # Check if there has been a change since last apply. + if [ -e "/tmp/${name}.lite.auto.tfvars" ];then + tfvarsdiff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` + else + tfvarsdiff="" + fi +else + echo "In $0. The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; +fi +if [ "$name" == "hpcc" ];then + echo "Coping root's lite.auto.tfvars to /tmp and $name directory." + cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars + cp -v ../lite.auto.tfvars . + cp -v ../lite-variables.tf . +elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];then + egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars + egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars + ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf + if [ "$name" == "storage" ];then + cp -v $thisdir/needed-auto-tfvars-files/storage/storage.auto.tfvars.example ./storage.auto.tfvars + fi +fi +#------------------------------------------------------------------------ +if [ ! -d "$HOME/tflogs" ];then + mkdir $HOME/tflogs +fi + +plan=`$thisdir/mkplan deploy_${name}.plan` + +if [ "$tfvarsdiff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi + +#if [ "$name" != "vnet" ];then +# cp -v /tmp/${name}.lite.auto.tfvars . +#fi + +echo "=============== Deploy $name. Executing 'terraform init' ==============="; +assert_fail terraform init +echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; +assert_fail terraform plan -out=$plan +if [ -e "$plan" ];then + echo "=============== Deploy $name. Executing 'terraform apply $plan' ==============="; + assert_fail terraform apply -auto-approve $plan +else + echo "@@@@@@@@@@ No changes. Your infrastructure matches the configuration. So, terraform apply for $name will not be done. @@@@@@@@@@" +fi diff --git a/scripts/destroy b/scripts/destroy new file mode 100755 index 0000000..8ef1f21 --- /dev/null +++ b/scripts/destroy @@ -0,0 +1,41 @@ +#!/bin/bash +thisdir=$(d=`dirname $0`;cd $d;pwd) +repodir=`echo $thisdir|sed "s/\/scripts//"` +#echo "DEBUG: thisdir=\"$thisdir\", repodir=\"$repodir\", Directory where destroy takes place: \"$repodir/$1\"";exit +#======================================================================== +function assert_fail () { + echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" + if "$@"; then + echo;echo ">>>>>>>>>>>>>>>>>>> Successful: $*";echo + else + echo;echo ">>>>>>>>>>>>>>>>>>> FAILED: $*. EXITING!";echo + rm -vrf data + exit 1 + fi +} +#======================================================================== + +if [ "$1" == "vnet" ];then + assert_fail kubectl delete pods --all --force + assert_fail scripts/destroy hpcc + assert_fail scripts/destroy aks +elif [ "$1" == "aks" ];then + assert_fail scripts/destroy hpcc +fi +cd $repodir/$1; +name=$(basename `pwd`) +if [ "$name" == "hpcc" ];then + assert_fail kubectl delete pods --all --force +fi +if [ ! -d "$HOME/tflogs" ];then + mkdir $HOME/tflogs +fi +plan=`$thisdir/mkplan ${name}_destroy.plan` +if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi + +echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; +assert_fail terraform destroy -auto-approve +rm -vr data +cd .. +r=`terraform state list|egrep "_$name"` +terraform state rm $r diff --git a/scripts/external_storage b/scripts/external_storage new file mode 100755 index 0000000..65cca62 --- /dev/null +++ b/scripts/external_storage @@ -0,0 +1,16 @@ +#!/usr/bin/perl +$repopath = shift @ARGV; +$external_storage_desired = shift @ARGV; + +#print "DEBUG: {\"repopath\" : \"$repopath\", \"external_storage_desired\" : \"$external_storage_desired\"}\n"; + +if ( $external_storage_desired eq "false" ){ + #print "DEBUG: EXITING because ignore_external_storage is true.\n"; + exit 0; +}else{ + #print "DEBUG: EXITING because ignore_external_storage is NOT true.\n"; + while ( ! -e "$repopath/storage/data/config.json" ) { + sleep 10; + } + exit 0; +} diff --git a/scripts/extract-aks-variables b/scripts/extract-aks-variables new file mode 100755 index 0000000..66b7e3c --- /dev/null +++ b/scripts/extract-aks-variables @@ -0,0 +1,14 @@ +#!/usr/bin/perl +if ( scalar(@ARGV) > 0 ){ + $variable_file = shift @ARGV; +} else{ + die "FATAL ERROR: Variable file name must be given on command line\n"; +} +undef $/; +open(IN, $variable_file) || die "Can't open variable file, \"$variable_file\"\n"; +$_ = ; +close(IN); + +@variable_definitions = m/\bvariable \"aks_[^\"]+\".+?\n}/sg; + +print join("\n\n",@variable_definitions),"\n"; diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file new file mode 100755 index 0000000..43ab9f3 --- /dev/null +++ b/scripts/get_rg_from_file @@ -0,0 +1,13 @@ +#!/usr/bin/perl +if ( scalar(@ARGV) > 0 ){ + $config_file = shift @ARGV; +} else{ + die "FATAL ERROR: In $0. Config file path must be given on command line\n"; +} +undef $/; +open(IN, $config_file) || die "In $0. Can't open config file, \"$config_file\"\n"; +$_ = ; +close(IN); + +$rg = $1 if ( /"resource_group(?:_name)?":\s*"([^"]+)"/s ); +print $rg; diff --git a/scripts/mkplan b/scripts/mkplan new file mode 100755 index 0000000..c54df21 --- /dev/null +++ b/scripts/mkplan @@ -0,0 +1,36 @@ +#!//usr/bin/perl +$HOME = $ENV{'HOME'}; +$tmpl = "/home/azureuser/tflogs/-.plan"; +$month_date = `date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'`; +$timezone_difference=6; +$H = `date +'%H'`;chomp $H; +#print STDERR "DEBUG: H=\"$H\"\n"; +if ( $H <= $timezone_difference ){ + $H = 25 - $timezone_difference; +}else{ + $H = $H-$timezone_difference; +} +#print STDERR "DEBUG: After timezone difference: H=\"$H\"\n"; +$H = sprintf("%02d",$H); +$M = `date +'%M'`;chomp $M; +$M = sprintf("%02d",$M); +$month_date =~ s/\s+//g; +$month_date = "$month_date-$H$M"; +$repo_name = ""; +if (scalar(@ARGV)>0){ + $repo_name = shift @ARGV; +} else { + $cdir=`pwd`;chomp $cdir; + $reHOME = $HOME; $reHOME =~ s/(\/)/\\$1/g; + $repo_name = $cdir; $repo_name =~ s/^$reHOME\///; + #print "DEBUG: No arguments: cdir=\"$cdir\", reHOME=\"$reHOME\", repo_name=\"$repo_name\"\n"; +} +$repo_name =~ s/^\/home\/azureuser\/tflogs\///; +$repo_name =~ s/\-\d{8}\-\d{1,2}\.?\d{1,2}\.plan//; +$repo_name =~ s/\//-/g; + +$_ = $tmpl; +s//$repo_name/; +s//$month_date/; +print "$_\n"; + diff --git a/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example new file mode 100644 index 0000000..685ebd7 --- /dev/null +++ b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example @@ -0,0 +1,19 @@ +cluster_version = "1.26" # latest version +cluster_ordinal = 1 //cluster name suffix +sku_tier = "FREE" +hpcc_log_analytics_enabled = false + +rbac_bindings = { + cluster_admin_users = { + # "service_principal1" = "", + # "user1" = "" + "admin" = "" + } + + cluster_view_users = {} + cluster_view_groups = [] +} + +cluster_endpoint_access_cidrs = ["0.0.0.0/0"] + +availability_zones = [1] diff --git a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example new file mode 100644 index 0000000..ed01f69 --- /dev/null +++ b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example @@ -0,0 +1,22 @@ +resource_groups = { # rg + azure_kubernetes_service = { + tags = { "enclosed resource" = "open source aks" } + } + # azure_log_analytics_workspace = { + # tags = { "enclosed resource" = "azure log analytics workspace" } + # } +} + +# # auto_connect - Automatically connect to the kubernetes cluster from the host machine. +auto_connect = true + +# # disable_naming_conventions - Disable naming conventions +# # disable_naming_conventions = true + +# azure_auth = { +# # AAD_CLIENT_ID = "" +# # AAD_CLIENT_SECRET = "" +# # AAD_TENANT_ID = "" +# # AAD_PRINCIPAL_ID = "" +# SUBSCRIPTION_ID = "" +# } diff --git a/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example new file mode 100644 index 0000000..38e6a1b --- /dev/null +++ b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example @@ -0,0 +1,152 @@ +storage_accounts = { # storage account + adminsvc1 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "adminsvc1" + storage_type = "azurefiles" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + subnet_ids = {} + file_share_retention_days = 7 + access_tier = "Hot" + account_kind = "FileStorage" + account_tier = "Premium" + + planes = { + dali = { + category = "dali" + name = "dali" + sub_path = "dalistorage" + size = 100 + sku = "" + rwmany = true + protocol = "nfs" + } + } + } + + adminsvc2 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "adminsvc2" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + account_tier = "Standard" + + planes = { + dll = { + category = "dll" + name = "dll" + sub_path = "queries" + size = 100 + sku = "" + rwmany = true + } + + lz = { + category = "lz" + name = "mydropzone" + sub_path = "dropzone" + size = 100 + sku = "" + rwmany = true + } + + sasha = { + category = "sasha" + name = "sasha" + sub_path = "sashastorage" + size = 100 + sku = "" + rwmany = true + } + + debug = { + category = "debug" + name = "debug" + sub_path = "debug" + size = 100 + sku = "" + rwmany = true + } + } + } + + data1 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "data1" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + #replication_type = "GRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + #account_kind = "BlobStorage" + account_tier = "Standard" + + planes = { + data = { + category = "data" + name = "data" + sub_path = "hpcc-data" + size = 100 + sku = "" + rwmany = true + } + } + } + + data2 = { + delete_protection = false //Set to false to allow deletion + prefix_name = "data2" + storage_type = "blobnfs" + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "209.243.55.98" } + #authorized_ip_ranges = { anyone = "20.96.186.106" } + #authorized_ip_ranges = { anyone = "97.118.251.104" } + #authorized_ip_ranges = { anyone = "0.0.0.0/0" } + authorized_ip_ranges = {} + replication_type = "ZRS" + #replication_type = "LRS" + subnet_ids = {} + blob_soft_delete_retention_days = 7 + container_soft_delete_retention_days = 7 + access_tier = "Hot" + account_kind = "StorageV2" + #account_kind = "BlobStorage" + account_tier = "Standard" + + planes = { + data = { + category = "data" + name = "data" + sub_path = "hpcc-data" + size = 100 + sku = "" + rwmany = true + } + } + } +} diff --git a/storage/lite-variables.tf b/storage/lite-variables.tf new file mode 100644 index 0000000..e88b5b1 --- /dev/null +++ b/storage/lite-variables.tf @@ -0,0 +1,64 @@ +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "REQUIRED. Name of the resource group containing the dns zone." +} + +variable "aks_dns_zone_name" { + type = string + description = "REQUIRED. dns zone name. The name of existing dns zone." +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/storage/locals.tf b/storage/locals.tf index 4e26af1..ca2a8be 100644 --- a/storage/locals.tf +++ b/storage/locals.tf @@ -1,4 +1,25 @@ locals { + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null # get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null diff --git a/storage/main.tf b/storage/main.tf index 8fb9353..6efa56a 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,9 +1,13 @@ module "storage" { - source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" + #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" + #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" + source = "git@github.com:hpccsystems-solutions-lab/terraform-azurerm-hpcc-storage.git?ref=HPCC-27615-add-rm-0000-cidr" - owner = var.owner + owner = local.owner disable_naming_conventions = var.disable_naming_conventions - metadata = var.metadata + metadata = local.metadata subnet_ids = local.subnet_ids storage_accounts = var.storage_accounts } diff --git a/storage/misc.auto.tfvars.example b/storage/misc.auto.tfvars.example deleted file mode 100644 index f01112f..0000000 --- a/storage/misc.auto.tfvars.example +++ /dev/null @@ -1,23 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -# disable_naming_conventions = false # true will enforce all the arguments of the metadata block above - -# Provide an existing virtual network deployed outside of this project - diff --git a/storage/outputs.tf b/storage/outputs.tf index 49e6fc4..9e5808a 100644 --- a/storage/outputs.tf +++ b/storage/outputs.tf @@ -1,4 +1,6 @@ resource "local_file" "config" { content = module.storage.config filename = "${path.module}/data/config.json" + + depends_on = [ module.storage ] } diff --git a/storage/variables.tf b/storage/variables.tf index 8333e88..e5e3946 100644 --- a/storage/variables.tf +++ b/storage/variables.tf @@ -1,59 +1,9 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - variable "disable_naming_conventions" { description = "Naming convention module." type = bool default = false } -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - variable "virtual_network" { description = "Subnet IDs" type = list(object({ diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf new file mode 100644 index 0000000..e88b5b1 --- /dev/null +++ b/vnet/lite-variables.tf @@ -0,0 +1,64 @@ +variable "aks_logging_monitoring_enabled" { + description = "Used to get logging and monitoring of kubernetes and hpcc cluster." + type = bool + default = false +} + +variable "aks_admin_email" { + type = string + description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" + validation { + condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 + error_message = "Value must at least look like a valid email address." + } +} + +variable "aks_admin_name" { + type = string + description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" +} + +variable "aks_azure_region" { + type = string + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" + validation { + condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) + error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." + } +} + +variable "aks_enable_roxie" { + description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + type = bool + default = false +} + +variable "aks_dns_zone_resource_group_name" { + type = string + description = "REQUIRED. Name of the resource group containing the dns zone." +} + +variable "aks_dns_zone_name" { + type = string + description = "REQUIRED. dns zone name. The name of existing dns zone." +} + +variable "aks_admin_ip_cidr_map" { + description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." + type = map(string) + default = {} +} + +variable "aks_max_node_count" { + type = number + description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." + validation { + condition = var.aks_max_node_count >= 2 + error_message = "Value must be 2 or more." + } +} + +variable "aks_node_size" { + type = string + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." +} diff --git a/vnet/locals.tf b/vnet/locals.tf index 776e751..344ca8e 100644 --- a/vnet/locals.tf +++ b/vnet/locals.tf @@ -1,18 +1,45 @@ locals { + owner = { + name = var.aks_admin_name + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + resource_groups = { + virtual_network = { + tags = { "enclosed resource" = "open source vnet" } + } + } + names = var.disable_naming_conventions ? merge( { - business_unit = var.metadata.business_unit - environment = var.metadata.environment - location = var.resource_groups.location - market = var.metadata.market - subscription_type = var.metadata.subscription_type + business_unit = local.metadata.business_unit + environment = local.metadata.environment + location = local.metadata.location + market = local.metadata.market + subscription_type = local.metadata.subscription_type }, - var.metadata.product_group != "" ? { product_group = var.metadata.product_group } : {}, - var.metadata.product_name != "" ? { product_name = var.metadata.product_name } : {}, - var.metadata.resource_group_type != "" ? { resource_group_type = var.metadata.resource_group_type } : {} + local.metadata.product_group != "" ? { product_group = local.metadata.product_group } : {}, + local.metadata.product_name != "" ? { product_name = local.metadata.product_name } : {}, + local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) + tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) private_subnet_id = module.virtual_network.aks.hpcc.subnets["private"].id diff --git a/vnet/main.tf b/vnet/main.tf index b64712c..7ff07dd 100644 --- a/vnet/main.tf +++ b/vnet/main.tf @@ -12,23 +12,23 @@ module "metadata" { naming_rules = module.naming.yaml - market = var.metadata.market - location = var.metadata.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type + market = local.metadata.market + location = local.metadata.location + sre_team = local.metadata.sre_team + environment = local.metadata.environment + product_name = local.metadata.product_name + business_unit = local.metadata.business_unit + product_group = local.metadata.product_group + subscription_type = local.metadata.subscription_type + resource_group_type = local.metadata.resource_group_type subscription_id = data.azurerm_subscription.current.id - project = var.metadata.project + project = local.metadata.project } module "resource_groups" { source = "github.com/Azure-Terraform/terraform-azurerm-resource-group.git?ref=v2.1.0" - for_each = var.resource_groups + for_each = local.resource_groups unique_name = true location = module.metadata.location diff --git a/vnet/misc.auto.tfvars.example b/vnet/misc.auto.tfvars.example deleted file mode 100644 index 1838b44..0000000 --- a/vnet/misc.auto.tfvars.example +++ /dev/null @@ -1,24 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccplatform" - product_name = "hpccplatform" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "hpcc" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = "eastus" # Acceptable values: eastus, centralus -} - -resource_groups = { - virtual_network = { - tags = { "enclosed resource" = "open source vnet" } - } -} diff --git a/vnet/outputs.tf b/vnet/outputs.tf index 7c79d01..c4bf74b 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -29,6 +29,8 @@ output "vnet_name" { resource "local_file" "output" { content = local.config filename = "${path.module}/data/config.json" + + depends_on = [ module.virtual_network ] } diff --git a/vnet/variables.tf b/vnet/variables.tf index fd5d060..4d334a2 100644 --- a/vnet/variables.tf +++ b/vnet/variables.tf @@ -1,66 +1,5 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - variable "disable_naming_conventions" { description = "Naming convention module." type = bool default = false } - -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = map(string) - location = string - }) - - default = { - business_unit = "" - environment = "" - market = "" - product_group = "" - product_name = "hpcc" - project = "" - resource_group_type = "" - sre_team = "" - subscription_type = "" - additional_tags = {} - location = "" - } -} - -variable "resource_groups" { - description = "Resource group module variables." - type = any - - default = { - azure_kubernetes_service = { - tags = { "apps" = "vnet" } - } - } -} From b44fbcb39208f41243074b7055a0507dd3e2c891 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 20 Nov 2023 14:01:13 +0000 Subject: [PATCH 059/124] branch:HPCC-27615-easy-deploy-bryan10-added-hpcc_version_and_misc. Added 'delete pods' to scripts/destroy --- scripts/destroy | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/destroy b/scripts/destroy index 943d204..8b47b69 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -20,7 +20,10 @@ elif [ "$1" == "aks" ];then assert_fail scripts/destroy hpcc fi cd $1; -name=$(basename `pwd`) +name=$1 +if [ "$name" == "hpcc" ];then + assert_fail kubectl delete pods --all --force +fi if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi From 8747f1644b4bc3e950e28f809ae1c541fa753837 Mon Sep 17 00:00:00 2001 From: Timothy L Humphrey Date: Mon, 20 Nov 2023 07:08:50 -0700 Subject: [PATCH 060/124] Update destroy Removed one statement "kubectl delete pods". --- scripts/destroy | 1 - 1 file changed, 1 deletion(-) diff --git a/scripts/destroy b/scripts/destroy index e25feba..c66f964 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -16,7 +16,6 @@ function assert_fail () { #======================================================================== if [ "$1" == "vnet" ];then - assert_fail kubectl delete pods --all --force assert_fail scripts/destroy hpcc assert_fail scripts/destroy aks elif [ "$1" == "aks" ];then From 822acd9563c2c2f4a1906d003b0b7d03a59bb35e Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 21 Nov 2023 00:29:45 +0000 Subject: [PATCH 061/124] branch:main-review-mods-2023-11-17. The easy fixes --- README.md | 30 ++++--- aks/aks.tf | 34 -------- aks/lite-locals.tf | 9 -- aks/lite-variables.tf | 2 + aks/locals.tf | 21 ----- aks/node_groups.txt | 99 --------------------- aks/outputs.tf | 5 -- aks/variables.tf | 13 --- hpcc/hpcc.tf | 27 +----- hpcc/index.html | 1 - hpcc/lite-locals.tf | 91 -------------------- hpcc/lite-variables.tf | 126 ++++++++++++++------------- hpcc/locals.tf | 2 - lite-variables.tf | 126 ++++++++++++++------------- main.tf | 2 +- scripts/destroy | 1 - scripts/external_storage | 25 +++--- scripts/extract-aks-variables | 21 ++--- scripts/get_rg_from_file | 24 +++--- scripts/mkplan | 62 ++++++-------- storage/lite-variables.tf | 64 -------------- storage/locals.tf | 1 - storage/main.tf | 4 - storage/providers.tf | 7 -- storage/storage.auto.tfvars.example | 128 ---------------------------- vnet/lite-variables.tf | 2 + vnet/outputs.tf | 2 - 27 files changed, 206 insertions(+), 723 deletions(-) delete mode 100644 aks/node_groups.txt delete mode 100644 hpcc/index.html delete mode 100644 storage/lite-variables.tf delete mode 100644 storage/storage.auto.tfvars.example diff --git a/README.md b/README.md index d5309cb..02dfdf6 100755 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). +NOTE: A tutorial of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. -The HPCC Systems cluster created by this module uses ephemeral storage (meaning, the storage will be deleted when the cluster is deleted). But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. +The HPCC Systems cluster created by this module uses ephemeral storage, which is the default. This means the storage will be deleted when the cluster is deleted) But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. -This repo is a fork of the excellent work performed by Godson Fortil. The original can be found at [https://github.com/gfortil/terraform-azurerm-hpcc/tree/HPCC-27615]. +This repo is a fork of the excellent work performed by Godson Fortil. The original can be found in branch, HPCC-27615 of [https://github.com/gfortil/Terraform-azurerm-hpcc]. ## Requirements @@ -18,8 +18,6 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within kubernetes clusters. TL;DR: Make sure you have the command line tools installed. -* This module will create an AKS cluster in your current **default** Azure subscription. You can view your current subscriptions, and determine which is the default, using the `az account list --output table` command. To set a default subscription, use `az account set --subscription "My_Subscription"`. - * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. ## Installing/Using This Module @@ -32,14 +30,15 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi 1. Issue `terraform init` to initialize the Terraform modules. 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: 1. Invoke the `terraform apply` command and enter values for each option as Terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. - 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to do that is to copy the example file and then edit the copy: - * `cp lite.auto.tfvars.example lite.auto.tfvars` - 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. + 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to creat `lite.auto.tfvars` is to copy the example file, `lite.auto.tfvars.example`, and then edit the copy: + * `cp -v lite.auto.tfvars.example lite.auto.tfvars` + 1. Use -var arguments on the command line when executing the Terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. 1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). At the end of a successful deployment these items are output: -* The URL used to access ECL Watch. -* The deployment azure resource group. +* The URL used to access ECL Watch, `eclwatch_url`. +* The deployment azure resource group, `deployment_resource_group`. +* Whether there is external storage or not, `external_storage_config_exists`. ## Available Options @@ -78,8 +77,8 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | | `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | | `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | -| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | -| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | +| `aks_dns_zone_name` | string | Name of an existing Azure DNS Zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above Azure DNS Zone. Example entry: "app-dns-prod-eastus2" | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | | `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | | `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | @@ -99,10 +98,15 @@ The following options should be set in your `lite.auto.tfvars` file (or entered ## Persistent Storage -To get persistent storage, i.e. storage that is not deleted when the hpcc cluster is deleted, set the variable, external_storage_desired, to true. +To get persistent storage, i.e. storage that is not deleted when the HPCC cluster is deleted, set the variable, `external_storage_desired`, to true. ## Useful Things +* Useful `az cli` commands: + * `az account list --output table` + * Shows your current subscriptions, and determine which is the default + * `az account set --subscription "My_Subscription"` + * Sets the default subscription * Useful `kubectl` commands once the cluster is deployed: * `kubectl get pods` * Shows Kubernetes pods for the current cluster. diff --git a/aks/aks.tf b/aks/aks.tf index 09e6ba4..028ef32 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,31 +1,5 @@ -# resource "kubernetes_secret" "private_docker_registry" { -# count = can(var.registry.server) && can(var.registry.username) && can(var.registry.password) ? 1 : 0 -# metadata { -# name = "docker-cfg" -# } -# type = "kubernetes.io/dockerconfigjson" -# data = { -# ".dockerconfigjson" = jsonencode({ -# auths = { -# "${var.registry.server}" = { -# "username" = var.registry.username -# "password" = var.registry.password -# "email" = var.admin.email -# "auth" = base64encode("${var.registry.username}:${var.registry.password}") -# } -# } -# }) -# } -# } - module "aks" { depends_on = [random_string.string] - #source = "github.com/gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=HPCC-27615" - #source = "git@github.com:gfortil/terraform-azurerm-aks.git?ref=OSS" - #source = "/home/azureuser/tlhumphrey2/rba-rsg-terraform-azurerm-aks" - #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-aks" - #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" providers = { @@ -82,12 +56,4 @@ module "aks" { } logging = var.logging - #logging = null - - experimental = { - oms_agent = var.hpcc_log_analytics_enabled || var.experimental.oms_agent - oms_agent_log_analytics_workspace_id = fileexists("../logging/data/workspace_resource_id.txt") ? file("../logging/data/workspace_resource_id.txt") : var.experimental.oms_agent_log_analytics_workspace_id != null ? var.experimental.oms_agent_log_analytics_workspace_id : null - #tlh tried this oms_agent = null - #tlh tried this oms_agent_log_analytics_workspace_id = null - } } diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf index 57c7a99..adebbe1 100644 --- a/aks/lite-locals.tf +++ b/aks/lite-locals.tf @@ -32,21 +32,12 @@ locals { receivers = [] } - # coredns = { - # forward_zones = { - # "" = "" - # } - # } coredns = {} external_dns = { public_domain_filters = [var.aks_dns_zone_name] } - # cert_manager = { - # acme_dns_zones = [var.aks_dns_zone_name] - # default_issuer_name = "zerossl" - # } cert_manager = {} ingress_internal_core = { diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf index e88b5b1..b6c4b6e 100644 --- a/aks/lite-variables.tf +++ b/aks/lite-variables.tf @@ -1,3 +1,4 @@ +# All 'aks_' variables are before any other variables. variable "aks_logging_monitoring_enabled" { description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool @@ -62,3 +63,4 @@ variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } +#===== end of aks variables ===== diff --git a/aks/locals.tf b/aks/locals.tf index 03fe395..231113d 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -11,12 +11,10 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - #node_size = "2xlarge" node_size = "large" single_group = false min_capacity = 1 max_capacity = 3 - # placement_group_key = null labels = { "lnrs.io/tier" = "standard" "workload" = "roxiepool" @@ -31,12 +29,10 @@ locals { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge single_group = false min_capacity = 3 max_capacity = 6 - # placement_group_key = null labels = { "lnrs.io/tier" = "standard" "workload" = "thorpool" @@ -50,12 +46,10 @@ locals { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - #node_size = "4xlarge" node_size = "2xlarge" single_group = false min_capacity = 1 max_capacity = 3 - # placement_group_key = null labels = { "lnrs.io/tier" = "standard" "workload" = "servpool" @@ -70,12 +64,9 @@ locals { node_type = "gp" node_type_version = "v1" node_size = "2xlarge" - #node_size = "1xlarge" # NOT ALLOWED - #node_size = "4xlarge" single_group = false min_capacity = 3 max_capacity = 6 - # placement_group_key = null labels = { "lnrs.io/tier" = "standard" "workload" = "spraypool" @@ -105,17 +96,6 @@ locals { start_time = "20:00" // At least 5 minutes in the future week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] }, - # { - # schedule_name = "aks_start" - # description = "Starts the AKS weekday nights at 6AM EST" - # runbook_name = "aks_startstop_runbook" - # frequency = "Week" //OneTime, Day, Hour, Week, or Month. - # interval = "1" //cannot be set when frequency is `OneTime` - # operation = "start" - # daylight_saving = true - # start_time = "06:00" // At least 5 minutes in the future - # week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - # } ] } @@ -165,7 +145,6 @@ locals { current_hour = tonumber(formatdate("HH", local.current_time)) today = formatdate("YYYY-MM-DD", local.current_time) tomorrow = formatdate("YYYY-MM-DD", timeadd(local.current_time, "24h")) - # today = formatdate("YYYY-MM-DD", timeadd(local.current_time, "1h")) utc_offset = local.aks_automation.schedule[0].daylight_saving ? 4 : 5 diff --git a/aks/node_groups.txt b/aks/node_groups.txt deleted file mode 100644 index b3623c6..0000000 --- a/aks/node_groups.txt +++ /dev/null @@ -1,99 +0,0 @@ -cluster_version = "1.26" -cluster_ordinal = 1 //cluster name suffix -sku_tier = "FREE" -hpcc_log_analytics_enabled = false - -rbac_bindings = { - cluster_admin_users = { - # "service_principal1" = "", - # "user1" = "" - "admin" = "35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9" - } - - cluster_view_users = {} - cluster_view_groups = [] -} - -cluster_endpoint_access_cidrs = ["0.0.0.0/0"] - -availability_zones = [1] - -node_groups = { - thorpool = { - ultra_ssd = false - node_os = "ubuntu" - node_type = "gp" # gp, gpd, mem, memd, stor - node_type_version = "v2" # v1, v2 - #node_size = "2xlarge" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge - node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge - single_group = false - min_capacity = 3 - max_capacity = 6 - # placement_group_key = null - labels = { - "lnrs.io/tier" = "standard" - "workload" = "thorpool" - } - taints = [] - tags = {} - }, - - roxiepool = { - ultra_ssd = false - node_os = "ubuntu" - node_type = "gp" - node_type_version = "v2" - #node_size = "2xlarge" - node_size = "large" - single_group = false - min_capacity = 1 - max_capacity = 3 - # placement_group_key = null - labels = { - "lnrs.io/tier" = "standard" - "workload" = "roxiepool" - } - taints = [] - tags = {} - }, - - servpool = { - ultra_ssd = false - node_os = "ubuntu" - node_type = "gpd" - node_type_version = "v1" - #node_size = "4xlarge" - node_size = "2xlarge" - single_group = false - min_capacity = 1 - max_capacity = 3 - # placement_group_key = null - labels = { - "lnrs.io/tier" = "standard" - "workload" = "servpool" - } - taints = [] - tags = {} - }, - - spraypool = { - ultra_ssd = false - node_os = "ubuntu" - node_type = "gp" - node_type_version = "v1" - node_size = "2xlarge" - #node_size = "1xlarge" # NOT ALLOWED - #node_size = "4xlarge" - single_group = false - min_capacity = 3 - max_capacity = 6 - # placement_group_key = null - labels = { - "lnrs.io/tier" = "standard" - "workload" = "spraypool" - "spray-service" = "spraypool" - } - taints = [] - tags = {} - } -} diff --git a/aks/outputs.tf b/aks/outputs.tf index 2926b6b..2c9eaab 100644 --- a/aks/outputs.tf +++ b/aks/outputs.tf @@ -10,11 +10,6 @@ output "cluster_name" { value = module.aks.cluster_name } -output "hpcc_log_analytics_enabled" { - description = "Is Log Analytics enabled for HPCC?" - value = var.hpcc_log_analytics_enabled && fileexists("../logging/data/workspace_resource_id.txt") -} - output "cluster_resource_group_name" { description = "The resource group where the cluster is deployed." value = module.resource_groups["azure_kubernetes_service"].name diff --git a/aks/variables.tf b/aks/variables.tf index a44d7fe..fc51908 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -5,19 +5,6 @@ variable "tags" { default = {} } -# variable "azure_auth" { -# description = "Azure authentication" -# type = object({ -# AAD_CLIENT_ID = optional(string) -# AAD_CLIENT_SECRET = optional(string) -# AAD_TENANT_ID = optional(string) -# AAD_PRINCIPAL_ID = optional(string) -# SUBSCRIPTION_ID = string -# }) - -# nullable = false -# } - variable "auto_connect" { description = "Automatically connect to the Kubernetes cluster from the host machine by overwriting the current context." type = bool diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 3a9cf47..0614a3b 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,27 +1,4 @@ -resource "kubernetes_namespace" "hpcc" { - count = var.hpcc_namespace.create_namespace && !fileexists("../logging/data/hpcc_namespace.txt") ? 1 : 0 - - metadata { - labels = try(var.hpcc_namespace.labels,{}) - - generate_name = "${var.hpcc_namespace.prefix_name}${trimspace(local.owner.name)}" - } -} - -/*resource "kubernetes_namespace" "hpcc" { - count = (var.hpcc_namespace == []) || !var.hpcc_namespace.create_namespace || fileexists("../logging/data/hpcc_namespace.txt") ? 0 : 1 - - metadata { - labels = try(var.hpcc_namespace.labels,{}) - name = "${substr(trimspace(local.owner.name), 0, 5)}${random_integer.random.result}" - # generate_name = "${trimspace(local.owner.name)}" - } -}*/ - module "hpcc" { - #source = "git@github.com:gfortil/opinionated-terraform-azurerm-hpcc?ref=HPCC-27615" - #source = "/home/azureuser/temp/opinionated-terraform-azurerm-hpcc" - #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" environment = local.metadata.environment @@ -53,7 +30,6 @@ module "hpcc" { location = local.metadata.location tags = module.metadata.tags - # namespace = local.hpcc_namespace namespace = { create_namespace = false name = local.hpcc_namespace @@ -100,7 +76,6 @@ module "hpcc" { vault_config = local.vault_config eclccserver_settings = local.eclccserver_settings spray_service_settings = local.spray_service_settings - # tlh 20231109 admin_services_node_selector = { all = { workload = local.spray_service_settings.nodeSelector } } admin_services_node_selector = { all = { workload = "servpool" } } esp_remoteclients = { @@ -120,7 +95,7 @@ module "hpcc" { } helm_chart_timeout = local.helm_chart_timeout - helm_chart_files_overrides = concat(local.helm_chart_files_overrides, fileexists("../logging/data/logaccess_body.yaml") ? ["../logging/data/logaccess_body.yaml"] : []) + helm_chart_files_overrides = local.helm_chart_files_overrides ldap_config = local.ldap_config enable_code_security = var.enable_code_security diff --git a/hpcc/index.html b/hpcc/index.html deleted file mode 100644 index 7937d2e..0000000 --- a/hpcc/index.html +++ /dev/null @@ -1 +0,0 @@ -20.96.202.148 \ No newline at end of file diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 867f922..1a41629 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -1,6 +1,5 @@ locals { helm_chart_timeout=300 - #hpcc_version = "8.6.20" owner = { name = var.admin_username @@ -25,63 +24,15 @@ locals { tags = merge(local.metadata.additional_tags, var.extra_tags) - # # disable_naming_conventions - Disable naming conventions - # # disable_naming_conventions = true disable_naming_conventions = false - # # auto_launch_eclwatch - Automatically launch ECLWatch web interface. - #auto_launch_eclwatch = true auto_launch_svc = { eclwatch = false } - - # azure_auth = { - # # AAD_CLIENT_ID = "" - # # AAD_CLIENT_SECRET = "" - # # AAD_TENANT_ID = "" - # # AAD_PRINCIPAL_ID = "" - # SUBSCRIPTION_ID = "" - # } - - # hpcc_container = { - # version = "9.2.0" - # image_name = "platform-core-ln" - # image_root = "jfrog.com/glb-docker-virtual" - # # custom_chart_version = "9.2.0-rc1" - # # custom_image_version = "9.2.0-demo" - # } - - # hpcc_container_registry_auth = { - # username = "value" - # password = "value" - # } internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io external = {} - # external = { - # blob_nfs = [{ - # container_id = "" - # container_name = "" - # id = "" - # resource_group_name = var.storage_account_resource_group_name - # storage_account_id = "" - # storage_account_name = var.storage_account_name - # }] - # # hpc_cache = [{ - # # id = "" - # # path = "" - # # server = "" - # }] - # hpcc = [{ - # name = "" - # planes = list(object({ - # local = "" - # remote = "" - # })) - # service = "" - # }] - # } admin_services_storage_account_settings = { replication_type = "ZRS" #LRS only if using HPC Cache @@ -106,12 +57,6 @@ locals { delete_protection = false } } - # hpc_cache = { - # enabled = false - # size = "small" - # cache_update_frequency = "3h" - # storage_account_data_planes = null - # } } external = null } @@ -132,36 +77,6 @@ locals { replicas = 6 nodeSelector = "spraypool" } - - # ldap = { - # ldap_server = "" //Server IP - # dali = { - # hpcc_admin_password = "" - # hpcc_admin_username = "" - # ldap_admin_password = "" - # ldap_admin_username = "" - # adminGroupName = "HPCC-Admins" - # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # } - # esp = { - # hpcc_admin_password = "" - # hpcc_admin_username = "" - # ldap_admin_password = "" - # ldap_admin_username = "" - # adminGroupName = "HPCC-Admins" - # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # } - # } roxie_internal_service = { name = "iroxie" @@ -199,7 +114,6 @@ locals { disabled = (var.aks_enable_roxie == true)? false : true name = "roxie" nodeSelector = { workload = "roxiepool" } - # tlh 20231109 numChannels = 2 numChannels = 1 prefix = "roxie" replicas = 2 @@ -423,7 +337,6 @@ locals { throttle = 0 retryinterval = 6 keepResultFiles = false - # egress = "engineEgress" } dfuwu-archiver = { @@ -437,7 +350,6 @@ locals { cutoff = 14 at = "* * * * *" throttle = 0 - # egress = "engineEgress" } dfurecovery-archiver = { @@ -446,7 +358,6 @@ locals { limit = 20 cutoff = 4 at = "* * * * *" - # egress = "engineEgress" } file-expiry = { @@ -456,7 +367,6 @@ locals { persistExpiryDefault = 7 expiryDefault = 4 user = "sasha" - # egress = "engineEgress" } } @@ -489,7 +399,6 @@ locals { maxGraphs = 2 maxGraphStartupTime = 172800 numWorkersPerPod = 1 - #nodeSelector = {} nodeSelector = { workload = "thorpool" } egress = "engineEgress" tolerations_value = "thorpool" diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf index a14c506..879d033 100644 --- a/hpcc/lite-variables.tf +++ b/hpcc/lite-variables.tf @@ -1,36 +1,10 @@ -############################################################################### -# Prompted variables (user will be asked to supply them at plan/apply time -# if a .tfvars file is not supplied); there are no default values -############################################################################### -variable "my_azure_id" { - description = "REQUIRED. The id of your azure account." - type = string -} - +# All 'aks_' variables are before any other variables. variable "aks_logging_monitoring_enabled" { description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool default = false } -variable "external_storage_desired" { - description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" - type = bool - default = false -} - -variable "enable_thor" { - description = "REQUIRED. If you want a thor cluster." - type = bool - default = true -} - -variable "a_record_name" { - type = string - description = "OPTIONAL: dns zone A record name" - default = "" -} - variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" @@ -45,15 +19,6 @@ variable "aks_admin_name" { description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" } -variable "admin_username" { - type = string - description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" - validation { - condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 - error_message = "Value must at least two characters in length and contain no spaces." - } -} - variable "aks_azure_region" { type = string description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" @@ -63,24 +28,12 @@ variable "aks_azure_region" { } } -variable "enable_code_security" { - description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" - type = bool - default = false -} - variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool default = false } -variable "extra_tags" { - description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." - type = map(string) - default = {} -} - variable "aks_dns_zone_resource_group_name" { type = string description = "REQUIRED. Name of the resource group containing the dns zone." @@ -91,22 +44,6 @@ variable "aks_dns_zone_name" { description = "REQUIRED. dns zone name. The name of existing dns zone." } -variable "hpcc_user_ip_cidr_list" { - description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." - type = list(string) - default = [] -} - -variable "hpcc_version" { - description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" - type = string - validation { - condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) - error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." - } - default = "latest" -} - variable "aks_admin_ip_cidr_map" { description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." type = map(string) @@ -126,6 +63,67 @@ variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } +#===== end of aks variables ===== + +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "external_storage_desired" { + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" + type = bool + default = false +} + +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool + default = true +} + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool + default = false +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } + default = "latest" +} variable "storage_data_gb" { type = number diff --git a/hpcc/locals.tf b/hpcc/locals.tf index e3fe976..becb47b 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -54,8 +54,6 @@ locals { location = local.use_existing_vnet != null ? local.use_existing_vnet.location : local.get_vnet_config.location - # hpcc_chart_major_minor_point_version = local.helm_chart_version != null ? regex("[\\d+?.\\d+?.\\d+?]+", local.helm_chart_version) : "master" - domain = coalesce(local.internal_domain, format("us-%s.%s.azure.lnrsg.io", "local.metadata.product_name", "dev")) svc_domains = { eclwatch = local.auto_launch_svc.eclwatch ? "https://eclwatch-${local.hpcc_namespace}.${local.domain}:18010" : null } diff --git a/lite-variables.tf b/lite-variables.tf index a14c506..879d033 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -1,36 +1,10 @@ -############################################################################### -# Prompted variables (user will be asked to supply them at plan/apply time -# if a .tfvars file is not supplied); there are no default values -############################################################################### -variable "my_azure_id" { - description = "REQUIRED. The id of your azure account." - type = string -} - +# All 'aks_' variables are before any other variables. variable "aks_logging_monitoring_enabled" { description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool default = false } -variable "external_storage_desired" { - description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" - type = bool - default = false -} - -variable "enable_thor" { - description = "REQUIRED. If you want a thor cluster." - type = bool - default = true -} - -variable "a_record_name" { - type = string - description = "OPTIONAL: dns zone A record name" - default = "" -} - variable "aks_admin_email" { type = string description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" @@ -45,15 +19,6 @@ variable "aks_admin_name" { description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" } -variable "admin_username" { - type = string - description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" - validation { - condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 - error_message = "Value must at least two characters in length and contain no spaces." - } -} - variable "aks_azure_region" { type = string description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" @@ -63,24 +28,12 @@ variable "aks_azure_region" { } } -variable "enable_code_security" { - description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" - type = bool - default = false -} - variable "aks_enable_roxie" { description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" type = bool default = false } -variable "extra_tags" { - description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." - type = map(string) - default = {} -} - variable "aks_dns_zone_resource_group_name" { type = string description = "REQUIRED. Name of the resource group containing the dns zone." @@ -91,22 +44,6 @@ variable "aks_dns_zone_name" { description = "REQUIRED. dns zone name. The name of existing dns zone." } -variable "hpcc_user_ip_cidr_list" { - description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." - type = list(string) - default = [] -} - -variable "hpcc_version" { - description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" - type = string - validation { - condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) - error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." - } - default = "latest" -} - variable "aks_admin_ip_cidr_map" { description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." type = map(string) @@ -126,6 +63,67 @@ variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } +#===== end of aks variables ===== + +variable "my_azure_id" { + description = "REQUIRED. The id of your azure account." + type = string +} + +variable "external_storage_desired" { + description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" + type = bool + default = false +} + +variable "enable_thor" { + description = "REQUIRED. If you want a thor cluster." + type = bool + default = true +} + +variable "a_record_name" { + type = string + description = "OPTIONAL: dns zone A record name" + default = "" +} + +variable "admin_username" { + type = string + description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" + validation { + condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 + error_message = "Value must at least two characters in length and contain no spaces." + } +} + +variable "enable_code_security" { + description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" + type = bool + default = false +} + +variable "extra_tags" { + description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." + type = map(string) + default = {} +} + +variable "hpcc_user_ip_cidr_list" { + description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." + type = list(string) + default = [] +} + +variable "hpcc_version" { + description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" + type = string + validation { + condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) + error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." + } + default = "latest" +} variable "storage_data_gb" { type = number diff --git a/main.tf b/main.tf index 3bc29e3..b428f4e 100644 --- a/main.tf +++ b/main.tf @@ -24,6 +24,7 @@ resource "null_resource" "deploy_storage" { depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] } +# if external storage is desired, this resource makes sure it exists. 'deploy_hpcc' depends on this. resource "null_resource" "external_storage" { count = (var.external_storage_desired == true)? 1 : 0 @@ -31,7 +32,6 @@ resource "null_resource" "external_storage" { command = "scripts/external_storage ${path.module} ${var.external_storage_desired}" } - #depends_on = [ null_resource.deploy_vnet, null_resource.deploy_aks ] depends_on = [ null_resource.deploy_vnet ] } diff --git a/scripts/destroy b/scripts/destroy index c66f964..765c7f0 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -1,7 +1,6 @@ #!/bin/bash thisdir=$(d=`dirname $0`;cd $d;pwd) repodir=`echo $thisdir|sed "s/\/scripts//"` -#echo "DEBUG: thisdir=\"$thisdir\", repodir=\"$repodir\", Directory where destroy takes place: \"$repodir/$1\"";exit #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" diff --git a/scripts/external_storage b/scripts/external_storage index 65cca62..4ad1b25 100755 --- a/scripts/external_storage +++ b/scripts/external_storage @@ -1,16 +1,15 @@ -#!/usr/bin/perl -$repopath = shift @ARGV; -$external_storage_desired = shift @ARGV; +#!/bin/bash +repopath=$1 +external_storage_desired=$2 -#print "DEBUG: {\"repopath\" : \"$repopath\", \"external_storage_desired\" : \"$external_storage_desired\"}\n"; - -if ( $external_storage_desired eq "false" ){ - #print "DEBUG: EXITING because ignore_external_storage is true.\n"; - exit 0; -}else{ - #print "DEBUG: EXITING because ignore_external_storage is NOT true.\n"; - while ( ! -e "$repopath/storage/data/config.json" ) { +if [ "$external_storage_desired" == "false" ];then + #echo "DEBUG: EXITING because ignore_external_storage is true." + exit 0 +else + #echo "DEBUG: EXITING because ignore_external_storage is NOT true." + while [ ! -e "$repopath/storage/data/config.json" ];do + echo "This file does not exist, yet: $repopath/storage/data/config.json" sleep 10; - } + done exit 0; -} +fi diff --git a/scripts/extract-aks-variables b/scripts/extract-aks-variables index 66b7e3c..30d105e 100755 --- a/scripts/extract-aks-variables +++ b/scripts/extract-aks-variables @@ -1,14 +1,9 @@ -#!/usr/bin/perl -if ( scalar(@ARGV) > 0 ){ - $variable_file = shift @ARGV; -} else{ - die "FATAL ERROR: Variable file name must be given on command line\n"; -} -undef $/; -open(IN, $variable_file) || die "Can't open variable file, \"$variable_file\"\n"; -$_ = ; -close(IN); +#!/bin/bash +if [ $# -gt 0 ];then + variable_file=$1 +else + echo "FATAL ERROR: Variable file name must be given on command line" + exit 1 +fi -@variable_definitions = m/\bvariable \"aks_[^\"]+\".+?\n}/sg; - -print join("\n\n",@variable_definitions),"\n"; +sed '/#===== end of aks variables =====/q' $variable_file diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file index 43ab9f3..473557b 100755 --- a/scripts/get_rg_from_file +++ b/scripts/get_rg_from_file @@ -1,13 +1,13 @@ -#!/usr/bin/perl -if ( scalar(@ARGV) > 0 ){ - $config_file = shift @ARGV; -} else{ - die "FATAL ERROR: In $0. Config file path must be given on command line\n"; -} -undef $/; -open(IN, $config_file) || die "In $0. Can't open config file, \"$config_file\"\n"; -$_ = ; -close(IN); +#!/bin/bash +if [ $# -gt 0 ];then + config_file=$1 +else + echo "FATAL ERROR: In $0. Config file path must be given on command line" + exit 1 +fi -$rg = $1 if ( /"resource_group(?:_name)?":\s*"([^"]+)"/s ); -print $rg; +rg=`sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group_name\": *\"//"|sed "s/\".*$//"` +if [[ "$rg" != *"hpccplatform"* ]];then + rg=`sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group\": *\"//"|sed "s/\".*$//"` +fi +echo $rg diff --git a/scripts/mkplan b/scripts/mkplan index c54df21..feca5a6 100755 --- a/scripts/mkplan +++ b/scripts/mkplan @@ -1,36 +1,28 @@ -#!//usr/bin/perl -$HOME = $ENV{'HOME'}; -$tmpl = "/home/azureuser/tflogs/-.plan"; -$month_date = `date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'`; -$timezone_difference=6; -$H = `date +'%H'`;chomp $H; -#print STDERR "DEBUG: H=\"$H\"\n"; -if ( $H <= $timezone_difference ){ - $H = 25 - $timezone_difference; -}else{ - $H = $H-$timezone_difference; -} -#print STDERR "DEBUG: After timezone difference: H=\"$H\"\n"; -$H = sprintf("%02d",$H); -$M = `date +'%M'`;chomp $M; -$M = sprintf("%02d",$M); -$month_date =~ s/\s+//g; -$month_date = "$month_date-$H$M"; -$repo_name = ""; -if (scalar(@ARGV)>0){ - $repo_name = shift @ARGV; -} else { - $cdir=`pwd`;chomp $cdir; - $reHOME = $HOME; $reHOME =~ s/(\/)/\\$1/g; - $repo_name = $cdir; $repo_name =~ s/^$reHOME\///; - #print "DEBUG: No arguments: cdir=\"$cdir\", reHOME=\"$reHOME\", repo_name=\"$repo_name\"\n"; -} -$repo_name =~ s/^\/home\/azureuser\/tflogs\///; -$repo_name =~ s/\-\d{8}\-\d{1,2}\.?\d{1,2}\.plan//; -$repo_name =~ s/\//-/g; - -$_ = $tmpl; -s//$repo_name/; -s//$month_date/; -print "$_\n"; +#!/bin/bash +month_date=`date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'` +timezone_difference=6 +H=`date +'%H'` +#echo "H=\"$H\", timezone_difference=\"$timezone_difference\"" 1>&2 +if [ $H -le $timezone_difference ];then + H=$((25 - $timezone_difference)); +else + H=$(($H-$timezone_difference)); +fi +#echo "After timezone_difference is subtracted: H=\"$H\"" 1>&2 +printf -v H "%02d" $H +M=`date +'%M'` +printf -v M "%02d" $M +month_date=`echo $month_date|sed "s/ *//g"`; +month_date="$month_date-$H$M"; +repo_name=""; +if [ $# -gt 0 ];then + repo_name=$1 +else + cdir=`pwd` + reHOME=`echo $HOME|sed "s/[/]/./g"` + repo_name=`echo $cdir|sed "s/^$reHOME.//"|sed "s/[/]/-/g"`; + #echo "DEBUG: No arguments: cdir=\"$cdir\", reHOME=\"$reHOME\", repo_name=\"$repo_name\"" 1>&2 +fi +plan_name="$HOME/tflogs/$repo_name-$month_date.plan" +echo $plan_name diff --git a/storage/lite-variables.tf b/storage/lite-variables.tf deleted file mode 100644 index e88b5b1..0000000 --- a/storage/lite-variables.tf +++ /dev/null @@ -1,64 +0,0 @@ -variable "aks_logging_monitoring_enabled" { - description = "Used to get logging and monitoring of kubernetes and hpcc cluster." - type = bool - default = false -} - -variable "aks_admin_email" { - type = string - description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" - validation { - condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 - error_message = "Value must at least look like a valid email address." - } -} - -variable "aks_admin_name" { - type = string - description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" -} - -variable "aks_azure_region" { - type = string - description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" - validation { - condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) - error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." - } -} - -variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" - type = bool - default = false -} - -variable "aks_dns_zone_resource_group_name" { - type = string - description = "REQUIRED. Name of the resource group containing the dns zone." -} - -variable "aks_dns_zone_name" { - type = string - description = "REQUIRED. dns zone name. The name of existing dns zone." -} - -variable "aks_admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "aks_max_node_count" { - type = number - description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." - validation { - condition = var.aks_max_node_count >= 2 - error_message = "Value must be 2 or more." - } -} - -variable "aks_node_size" { - type = string - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." -} diff --git a/storage/locals.tf b/storage/locals.tf index ca2a8be..89be4be 100644 --- a/storage/locals.tf +++ b/storage/locals.tf @@ -21,7 +21,6 @@ locals { } get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null - # get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null virtual_network = var.virtual_network != null ? var.virtual_network : [ { diff --git a/storage/main.tf b/storage/main.tf index 6efa56a..ece147a 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,8 +1,4 @@ module "storage" { - #source = "github.com/gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - #source = "git@github.com:gfortil/terraform-azurerm-hpcc-storage?ref=HPCC-27615" - #source = "/home/azureuser/tlhumphrey2/terraform-azurerm-hpcc-storage" - #source = "/home/azureuser/temp/HPCC-27615/terraform-azurerm-hpcc-storage" source = "git@github.com:hpccsystems-solutions-lab/terraform-azurerm-hpcc-storage.git?ref=HPCC-27615-add-rm-0000-cidr" owner = local.owner diff --git a/storage/providers.tf b/storage/providers.tf index 80eb367..e609c7e 100644 --- a/storage/providers.tf +++ b/storage/providers.tf @@ -5,10 +5,3 @@ provider "azurerm" { } provider "azuread" {} - -# provider "kubernetes" { -# host = local.get_aks_config.kube_admin_config[0].host -# client_certificate = base64decode(local.get_aks_config.kube_admin_config[0].client_certificate) -# client_key = base64decode(local.get_aks_config.kube_admin_config[0].client_key) -# cluster_ca_certificate = base64decode(local.get_aks_config.kube_admin_config[0].cluster_ca_certificate) -# } \ No newline at end of file diff --git a/storage/storage.auto.tfvars.example b/storage/storage.auto.tfvars.example deleted file mode 100644 index fa23ec0..0000000 --- a/storage/storage.auto.tfvars.example +++ /dev/null @@ -1,128 +0,0 @@ -storage_accounts = { - adminsvc1 = { - delete_protection = false //Set to false to allow deletion - prefix_name = "adminsvc1" - storage_type = "azurefiles" - authorized_ip_ranges = {} - replication_type = "ZRS" - subnet_ids = {} - file_share_retention_days = 7 - access_tier = "Hot" - account_kind = "FileStorage" - account_tier = "Premium" - - planes = { - dali = { - category = "dali" - name = "dali" - sub_path = "dalistorage" - size = 100 - sku = "" - rwmany = true - protocol = "nfs" - } - } - } - - adminsvc2 = { - delete_protection = false //Set to false to allow deletion - prefix_name = "adminsvc2" - storage_type = "blobnfs" - authorized_ip_ranges = {} - replication_type = "ZRS" - subnet_ids = {} - blob_soft_delete_retention_days = 7 - container_soft_delete_retention_days = 7 - access_tier = "Hot" - account_kind = "StorageV2" - account_tier = "Standard" - - planes = { - dll = { - category = "dll" - name = "dll" - sub_path = "queries" - size = 100 - sku = "" - rwmany = true - } - - lz = { - category = "lz" - name = "mydropzone" - sub_path = "dropzone" - size = 100 - sku = "" - rwmany = true - } - - sasha = { - category = "sasha" - name = "sasha" - sub_path = "sashastorage" - size = 100 - sku = "" - rwmany = true - } - - debug = { - category = "debug" - name = "debug" - sub_path = "debug" - size = 100 - sku = "" - rwmany = true - } - } - } - - data1 = { - delete_protection = false //Set to false to allow deletion - prefix_name = "data1" - storage_type = "blobnfs" - authorized_ip_ranges = {} - replication_type = "ZRS" - subnet_ids = {} - blob_soft_delete_retention_days = 7 - container_soft_delete_retention_days = 7 - access_tier = "Hot" - account_kind = "StorageV2" - account_tier = "Standard" - - planes = { - data = { - category = "data" - name = "data" - sub_path = "hpcc-data" - size = 100 - sku = "" - rwmany = true - } - } - } - - data2 = { - delete_protection = false //Set to false to allow deletion - prefix_name = "data2" - storage_type = "blobnfs" - authorized_ip_ranges = {} - replication_type = "ZRS" - subnet_ids = {} - blob_soft_delete_retention_days = 7 - container_soft_delete_retention_days = 7 - access_tier = "Hot" - account_kind = "StorageV2" - account_tier = "Standard" - - planes = { - data = { - category = "data" - name = "data" - sub_path = "hpcc-data" - size = 100 - sku = "" - rwmany = true - } - } - } -} diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf index e88b5b1..b6c4b6e 100644 --- a/vnet/lite-variables.tf +++ b/vnet/lite-variables.tf @@ -1,3 +1,4 @@ +# All 'aks_' variables are before any other variables. variable "aks_logging_monitoring_enabled" { description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool @@ -62,3 +63,4 @@ variable "aks_node_size" { type = string description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." } +#===== end of aks variables ===== diff --git a/vnet/outputs.tf b/vnet/outputs.tf index c4bf74b..80bf464 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -32,5 +32,3 @@ resource "local_file" "output" { depends_on = [ module.virtual_network ] } - - From 4d5526d61b67c3498110b4e4d6fc6c2032c124a5 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 21 Nov 2023 19:03:49 +0000 Subject: [PATCH 062/124] branch:main-review-mods-2023-11-17-activate-aks_node_size --- aks/aks.tf | 3 +- aks/lite-variables.tf | 66 ------------- aks/locals.tf | 8 +- hpcc/lite-variables.tf | 210 ----------------------------------------- vnet/lite-variables.tf | 66 ------------- 5 files changed, 6 insertions(+), 347 deletions(-) delete mode 100644 aks/lite-variables.tf delete mode 100644 hpcc/lite-variables.tf delete mode 100644 vnet/lite-variables.tf diff --git a/aks/aks.tf b/aks/aks.tf index 028ef32..3afcf95 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,6 +1,7 @@ module "aks" { depends_on = [random_string.string] - source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" + #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" + source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default diff --git a/aks/lite-variables.tf b/aks/lite-variables.tf deleted file mode 100644 index b6c4b6e..0000000 --- a/aks/lite-variables.tf +++ /dev/null @@ -1,66 +0,0 @@ -# All 'aks_' variables are before any other variables. -variable "aks_logging_monitoring_enabled" { - description = "Used to get logging and monitoring of kubernetes and hpcc cluster." - type = bool - default = false -} - -variable "aks_admin_email" { - type = string - description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" - validation { - condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 - error_message = "Value must at least look like a valid email address." - } -} - -variable "aks_admin_name" { - type = string - description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" -} - -variable "aks_azure_region" { - type = string - description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" - validation { - condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) - error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." - } -} - -variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" - type = bool - default = false -} - -variable "aks_dns_zone_resource_group_name" { - type = string - description = "REQUIRED. Name of the resource group containing the dns zone." -} - -variable "aks_dns_zone_name" { - type = string - description = "REQUIRED. dns zone name. The name of existing dns zone." -} - -variable "aks_admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "aks_max_node_count" { - type = number - description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." - validation { - condition = var.aks_max_node_count >= 2 - error_message = "Value must be 2 or more." - } -} - -variable "aks_node_size" { - type = string - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." -} -#===== end of aks variables ===== diff --git a/aks/locals.tf b/aks/locals.tf index 231113d..cc36e1d 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -11,7 +11,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - node_size = "large" + node_size = var.aks_node_size single_group = false min_capacity = 1 max_capacity = 3 @@ -29,7 +29,7 @@ locals { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - node_size = "large" # large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 18xlarge, 20xlarge, 24xlarge, 26xlarge + node_size = var.aks_node_size single_group = false min_capacity = 3 max_capacity = 6 @@ -46,7 +46,7 @@ locals { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - node_size = "2xlarge" + node_size = var.aks_node_size single_group = false min_capacity = 1 max_capacity = 3 @@ -63,7 +63,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v1" - node_size = "2xlarge" + node_size = var.aks_node_size single_group = false min_capacity = 3 max_capacity = 6 diff --git a/hpcc/lite-variables.tf b/hpcc/lite-variables.tf deleted file mode 100644 index 879d033..0000000 --- a/hpcc/lite-variables.tf +++ /dev/null @@ -1,210 +0,0 @@ -# All 'aks_' variables are before any other variables. -variable "aks_logging_monitoring_enabled" { - description = "Used to get logging and monitoring of kubernetes and hpcc cluster." - type = bool - default = false -} - -variable "aks_admin_email" { - type = string - description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" - validation { - condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 - error_message = "Value must at least look like a valid email address." - } -} - -variable "aks_admin_name" { - type = string - description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" -} - -variable "aks_azure_region" { - type = string - description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" - validation { - condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) - error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." - } -} - -variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" - type = bool - default = false -} - -variable "aks_dns_zone_resource_group_name" { - type = string - description = "REQUIRED. Name of the resource group containing the dns zone." -} - -variable "aks_dns_zone_name" { - type = string - description = "REQUIRED. dns zone name. The name of existing dns zone." -} - -variable "aks_admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "aks_max_node_count" { - type = number - description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." - validation { - condition = var.aks_max_node_count >= 2 - error_message = "Value must be 2 or more." - } -} - -variable "aks_node_size" { - type = string - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." -} -#===== end of aks variables ===== - -variable "my_azure_id" { - description = "REQUIRED. The id of your azure account." - type = string -} - -variable "external_storage_desired" { - description = "If you want external storage instead of ephemeral, this should be true. For ephemeral storage this should be false" - type = bool - default = false -} - -variable "enable_thor" { - description = "REQUIRED. If you want a thor cluster." - type = bool - default = true -} - -variable "a_record_name" { - type = string - description = "OPTIONAL: dns zone A record name" - default = "" -} - -variable "admin_username" { - type = string - description = "REQUIRED. Username of the administrator of this HPCC Systems cluster.\nExample entry: jdoe" - validation { - condition = length(var.admin_username) > 1 && length(regexall(" ", var.admin_username)) == 0 - error_message = "Value must at least two characters in length and contain no spaces." - } -} - -variable "enable_code_security" { - description = "REQUIRED. Enable code security?\nIf true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc.\nExample entry: false" - type = bool - default = false -} - -variable "extra_tags" { - description = "OPTIONAL. Map of name => value tags that can will be associated with the cluster.\nFormat is '{\"name\"=\"value\" [, \"name\"=\"value\"]*}'.\nThe 'name' portion must be unique.\nTo add no tags, enter '{}'. This is OPTIONAL and defaults to an empty string map." - type = map(string) - default = {} -} - -variable "hpcc_user_ip_cidr_list" { - description = "OPTIONAL. List of additional CIDR addresses that can access this HPCC Systems cluster.\nDefault value is '[]' which means no CIDR addresses.\nTo open to the internet, add \"0.0.0.0/0\"." - type = list(string) - default = [] -} - -variable "hpcc_version" { - description = "The version of HPCC Systems to install.\nOnly versions in nn.nn.nn format are supported. Default is 'latest'" - type = string - validation { - condition = (var.hpcc_version == "latest") || can(regex("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(-rc\\d{1,3})?$", var.hpcc_version)) - error_message = "Value must be 'latest' OR in nn.nn.nn format and 8.6.0 or higher." - } - default = "latest" -} - -variable "storage_data_gb" { - type = number - description = "REQUIRED. The amount of storage reserved for data in gigabytes.\nMust be 10 or more.\nIf a storage account is defined (see below) then this value is ignored." - validation { - condition = var.storage_data_gb >= 10 - error_message = "Value must be 10 or more." - } - default = 100 -} - -variable "storage_lz_gb" { - type = number - description = "REQUIRED. The amount of storage reserved for the landing zone in gigabytes.\nMust be 1 or more.\nIf a storage account is defined (see below) then this value is ignored." - validation { - condition = var.storage_lz_gb >= 1 - error_message = "Value must be 1 or more." - } - default = 25 -} - -variable "thor_max_jobs" { - type = number - description = "REQUIRED. The maximum number of simultaneous Thor jobs allowed.\nMust be 1 or more." - validation { - condition = var.thor_max_jobs >= 1 - error_message = "Value must be 1 or more." - } - default = 2 -} - -variable "thor_num_workers" { - type = number - description = "REQUIRED. The number of Thor workers to allocate.\nMust be 1 or more." - validation { - condition = var.thor_num_workers >= 1 - error_message = "Value must be 1 or more." - } - default = 2 -} - -############################################################################### -# Optional variables -############################################################################### - -variable "authn_htpasswd_filename" { - type = string - description = "OPTIONAL. If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it.\nA corollary is that persistent storage is enabled.\nAn empty string indicates that htpasswd is not to be used for authentication.\nExample entry: htpasswd.txt" - default = "" -} - -variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - prefix_name = string - labels = map(string) - create_namespace = bool - }) - default = { - prefix_name = "hpcc" - labels = { - name = "hpcc" - } - create_namespace = false - } -} - -variable "enable_premium_storage" { - type = bool - description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." - default = false -} - -variable "storage_account_name" { - type = string - description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" - default = "" -} - -variable "storage_account_resource_group_name" { - type = string - description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." - default = "" -} diff --git a/vnet/lite-variables.tf b/vnet/lite-variables.tf deleted file mode 100644 index b6c4b6e..0000000 --- a/vnet/lite-variables.tf +++ /dev/null @@ -1,66 +0,0 @@ -# All 'aks_' variables are before any other variables. -variable "aks_logging_monitoring_enabled" { - description = "Used to get logging and monitoring of kubernetes and hpcc cluster." - type = bool - default = false -} - -variable "aks_admin_email" { - type = string - description = "REQUIRED. Email address of the administrator of this HPCC Systems cluster.\nExample entry: jane.doe@hpccsystems.com" - validation { - condition = length(regexall("^[^@]+@[^@]+$", var.aks_admin_email)) > 0 - error_message = "Value must at least look like a valid email address." - } -} - -variable "aks_admin_name" { - type = string - description = "REQUIRED. Name of the administrator of this HPCC Systems cluster.\nExample entry: Jane Doe" -} - -variable "aks_azure_region" { - type = string - description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" - validation { - condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) - error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." - } -} - -variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" - type = bool - default = false -} - -variable "aks_dns_zone_resource_group_name" { - type = string - description = "REQUIRED. Name of the resource group containing the dns zone." -} - -variable "aks_dns_zone_name" { - type = string - description = "REQUIRED. dns zone name. The name of existing dns zone." -} - -variable "aks_admin_ip_cidr_map" { - description = "OPTIONAL. Map of name => CIDR IP addresses that can administrate this AKS.\nFormat is '{\"name\"=\"cidr\" [, \"name\"=\"cidr\"]*}'.\nThe 'name' portion must be unique.\nDefault value is '{}' means no CIDR addresses.\nThe corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user." - type = map(string) - default = {} -} - -variable "aks_max_node_count" { - type = number - description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." - validation { - condition = var.aks_max_node_count >= 2 - error_message = "Value must be 2 or more." - } -} - -variable "aks_node_size" { - type = string - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." -} -#===== end of aks variables ===== From 3aab1f8c49b49e7701f82a1761c7ae36fdaa4ecc Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 21 Nov 2023 19:56:32 +0000 Subject: [PATCH 063/124] branch:main-review-mods-2023-11-17-activate-aks_node_size --- aks/aks.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 3afcf95..286511f 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,7 +1,7 @@ module "aks" { depends_on = [random_string.string] - #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" - source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" + #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default From 2a6805bc655249b54a4974d93de0c84b77b3111d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 22 Nov 2023 13:48:34 +0000 Subject: [PATCH 064/124] branch:main-review-mods-2023-11-17-activate-aks_node_size. aks_node_size=Standard_D4s_v4 --- lite.auto.tfvars.example | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 3eeb70d..8a4556d 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -105,7 +105,7 @@ extra_tags={} # See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. # Value type: string -aks_node_size="Standard_B8ms" +aks_node_size="Standard_D4s_v4" #------------------------------------------------------------------------------ From 6809ae20933614c06bed11177cf6fa8cb8a3b491 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 22 Nov 2023 20:30:16 +0000 Subject: [PATCH 065/124] branch:main-review-mods-2023-11-17-deploy-hpcc-depends-on-storage --- main.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/main.tf b/main.tf index b428f4e..16fba90 100644 --- a/main.tf +++ b/main.tf @@ -41,5 +41,5 @@ resource "null_resource" "deploy_hpcc" { command = "scripts/deploy hpcc" } - depends_on = [ null_resource.deploy_aks, null_resource.deploy_vnet, null_resource.external_storage ] + depends_on = [ null_resource.deploy_aks, null_resource.deploy_vnet, null_resource.external_storage, null_resource.deploy_storage ] } From 2e5e4f470e87bbdf47b2b560de795e12c0ff2e44 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 24 Nov 2023 16:09:38 +0000 Subject: [PATCH 066/124] branch:main-review-mods-2023-11-17-aks_node-sizes-now-object --- aks/locals.tf | 8 ++++---- hpcc/lite-locals.tf | 2 +- lite-variables.tf | 9 +++++++-- lite.auto.tfvars.example | 2 +- 4 files changed, 13 insertions(+), 8 deletions(-) diff --git a/aks/locals.tf b/aks/locals.tf index cc36e1d..f17baa7 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -11,7 +11,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - node_size = var.aks_node_size + node_size = var.aks_node_sizes.roxie single_group = false min_capacity = 1 max_capacity = 3 @@ -29,7 +29,7 @@ locals { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - node_size = var.aks_node_size + node_size = var.aks_node_sizes.thor single_group = false min_capacity = 3 max_capacity = 6 @@ -46,7 +46,7 @@ locals { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - node_size = var.aks_node_size + node_size = var.aks_node_sizes.serv single_group = false min_capacity = 1 max_capacity = 3 @@ -63,7 +63,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v1" - node_size = var.aks_node_size + node_size = var.aks_node_sizes.spray single_group = false min_capacity = 3 max_capacity = 6 diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 1a41629..5044cc3 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -1,5 +1,5 @@ locals { - helm_chart_timeout=300 + helm_chart_timeout=600 owner = { name = var.admin_username diff --git a/lite-variables.tf b/lite-variables.tf index 879d033..0fbe951 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -59,9 +59,14 @@ variable "aks_max_node_count" { } } -variable "aks_node_size" { - type = string +variable "aks_node_sizes" { description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." + type = object({ + roxie = optional(string, "2xlarge") + serv = optional(string, "4xlarge") + spray = optional(string, "2xlarge") + thor = optional(string, "2xlarge") + }) } #===== end of aks variables ===== diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 8a4556d..5f6d794 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -105,7 +105,7 @@ extra_tags={} # See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. # Value type: string -aks_node_size="Standard_D4s_v4" +aks_node_sizes = { roxie = "2xlarge", serv = "4xlarge", spray = "2xlarge", thor = "2xlarge" } #------------------------------------------------------------------------------ From 290d02340c70c5207a929be46693866d63ca3176 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 02:46:10 +0000 Subject: [PATCH 067/124] Remove input variables not used --- lite-variables.tf | 21 ----- lite.auto.tfvars.example | 114 +++++++++++++--------------- scripts/change-source-statements.sh | 7 ++ scripts/external_storage | 2 +- 4 files changed, 62 insertions(+), 82 deletions(-) create mode 100755 scripts/change-source-statements.sh diff --git a/lite-variables.tf b/lite-variables.tf index 0fbe951..ce9c47b 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -50,15 +50,6 @@ variable "aks_admin_ip_cidr_map" { default = {} } -variable "aks_max_node_count" { - type = number - description = "REQUIRED. The maximum number of VM nodes to allocate for the HPCC Systems node pool.\nMust be 2 or more." - validation { - condition = var.aks_max_node_count >= 2 - error_message = "Value must be 2 or more." - } -} - variable "aks_node_sizes" { description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." type = object({ @@ -201,15 +192,3 @@ variable "enable_premium_storage" { description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." default = false } - -variable "storage_account_name" { - type = string - description = "OPTIONAL. If you are attaching to an existing storage account, enter its name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a resource group for the storage account.\nExample entry: my-product-sa" - default = "" -} - -variable "storage_account_resource_group_name" { - type = string - description = "OPTIONAL. If you are attaching to an existing storage account, enter its resource group name here.\nLeave blank if you do not have a storage account.\nIf you enter something here then you must also enter a name for the storage account." - default = "" -} diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 5f6d794..236778a 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -5,30 +5,36 @@ # dns zone. # Example entry: "my-product". This should be something project specific rather # than something generic. +# Value type: string +# Updateable: Y -a_record_name="" +a_record_name="" #----------------------------------------------------------------------------- # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" -# REQUIRED +# Value type: string +# Updateable: N -aks_dns_zone_name="" +aks_dns_zone_name="" #----------------------------------------------------------------------------- # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" -# REQUIRED +# Value type: string +# Updateable: N -aks_dns_zone_resource_group_name="" + +aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ # The version of HPCC Systems to install. # Only versions in nn.nn.nn format are supported. # Value type: string +# Updateable: Y hpcc_version="9.4.4" @@ -36,24 +42,28 @@ hpcc_version="9.4.4" # Enable ROXIE? # This will also expose port 8002 on the cluster. -# Value type: boolean # Example entry: false +# Value type: boolean +# Updateable: Y -aks_enable_roxie=true +aks_enable_roxie=false #------------------------------------------------------------------------------ # Enable code security? # If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. -# Value type: boolean # Example entry: false +# Value type: boolean +# Updateable: Y -enable_code_security=true +enable_code_security=false #------------------------------------------------------------------------------ # If you want a thor cluster then 'enable_thor' must be set to true # Otherwise it is set to false +# Value type: boolean +# Updateable: Y enable_thor=true @@ -61,6 +71,8 @@ enable_thor=true # The number of Thor workers to allocate. # Must be 1 or more. +# Value type: number +# Updateable: Y thor_num_workers=2 @@ -68,6 +80,8 @@ thor_num_workers=2 # The maximum number of simultaneous Thor jobs allowed. # Must be 1 or more. +# Value type: number +# Updateable: Y thor_max_jobs=2 @@ -76,6 +90,8 @@ thor_max_jobs=2 # The amount of storage reserved for the landing zone in gigabytes. # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. +# Value type: number +# Updateable: Y storage_lz_gb=25 @@ -84,6 +100,8 @@ storage_lz_gb=25 # The amount of storage reserved for data in gigabytes. # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. +# Value type: number +# Updateable: Y storage_data_gb=100 @@ -94,7 +112,7 @@ storage_data_gb=100 # The 'name' portion must be unique. # To add no tags, use '{}'. # Value type: map of string -# Example: extra_tags={ "owner"="Jane Doe", "owner_email"="jane.doe@gmail.com" } +# Updateable: Y extra_tags={} @@ -104,38 +122,34 @@ extra_tags={} # Recommend "Standard_B4ms" or better. # See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. # Value type: string +# Updateable: N aks_node_sizes = { roxie = "2xlarge", serv = "4xlarge", spray = "2xlarge", thor = "2xlarge" } #------------------------------------------------------------------------------ -# The maximum number of VM nodes to allocate for the HPCC Systems node pool. -# Must be 2 or more. -# Value type: integer - -aks_max_node_count=4 - -#------------------------------------------------------------------------------ - # Email address of the administrator of this HPCC Systems cluster. -# Value type: string # Example entry: "jane.doe@hpccsystems.com" +# Value type: string +# Updateable: Y -aks_admin_email="jane.doe@gmail.com" +aks_admin_email="jane.doe@hpccsystems.com" #------------------------------------------------------------------------------ # Name of the administrator of this HPCC Systems cluster. -# Value type: string # Example entry: "Jane Doe" +# Value type: string +# Updateable: Y aks_admin_name="Jane Doe" #------------------------------------------------------------------------------ # Username of the administrator of this HPCC Systems cluster. -# Value type: string # Example entry: "jdoe" +# Value type: string +# Updateable: N admin_username="jdoe" @@ -143,8 +157,9 @@ admin_username="jdoe" # The Azure region abbreviation in which to create these resources. # Must be one of ["eastus", "eastus2", "centralus"]. -# Value type: string # Example entry: "eastus" +# Value type: string +# Updateable: N aks_azure_region="eastus" @@ -156,58 +171,29 @@ aks_azure_region="eastus" # To add no CIDR addresses, use '{}'. # The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. # Value type: map of string +# Updateable: Y aks_admin_ip_cidr_map={} #------------------------------------------------------------------------------ # List of explicit CIDR addresses that can access this HPCC Systems cluster. -# To allow public access, value should be ["0.0.0.0/0"] or []. +# To allow public access, set value to ["0.0.0.0/0"] or []. # Value type: list of string +# Updateable: Y hpcc_user_ip_cidr_list=[] #------------------------------------------------------------------------------ -# If you are attaching to an existing storage account, put its name here. -# Leave as an empty string if you do not have a storage account. -# If you put something here then you must also define a resource group for the storage account. -# Value type: string -# Example entry: "my-product-sa" - -storage_account_name="" - -#------------------------------------------------------------------------------ - -# If you are attaching to an existing storage account, put its resource group name here. -# Leave as an empty string if you do not have a storage account. -# If you put something here then you must also define a name for the storage account. -# Value type: string - -storage_account_resource_group_name="" - -#------------------------------------------------------------------------------ - -# The Kubernetes namespace in which to install the HPCC modules (if enabled). -# Default value: "default" - -# hpcc_namespace="default" - -#------------------------------------------------------------------------------ - -# If true, premium ($$$) storage will be used for the following storage shares: Dali. -# OPTIONAL, defaults to false. - -enable_premium_storage=false - -#------------------------------------------------------------------------------ - # If you would like to use htpasswd to authenticate users to the cluster, enter # the filename of the htpasswd file. This file should be uploaded to the Azure # 'dllsshare' file share in order for the HPCC processes to find it. # A corollary is that persistent storage is enabled. # An empty string indicates that htpasswd is not to be used for authentication. # Example entry: "htpasswd.txt" +# Value type: string +# Updateable: Y authn_htpasswd_filename="" @@ -215,6 +201,8 @@ authn_htpasswd_filename="" # If you want external storage instead of ephemeral storage then # set this variable to true otherwise set it to false. +# Value type: boolean +# Updateable: Y external_storage_desired=false @@ -222,13 +210,19 @@ external_storage_desired=false # This variable enable you to ask for logging and monitoring of the kubernetes # and hpcc cluster (true means enable logging and monitoring, false means don't. +# Value type: boolean +# Updateable: N aks_logging_monitoring_enabled=false #------------------------------------------------------------------------------ -# Put your azure account id here. It will look like the following: -# 6c5edc79-34fd-333a-9b59-61ec21d7e42d +# Your azure account object id. Find this on azure portal, by going to 'users' +# then search for your name and click on it. The account object id is called +# 'Object ID'. There is a link next to it that lets you copy it. +# Value type: string +# Updateable: N -my_azure_id="" +my_azure_id="" +#------------------------------------------------------------------------------ diff --git a/scripts/change-source-statements.sh b/scripts/change-source-statements.sh new file mode 100755 index 0000000..9f93b41 --- /dev/null +++ b/scripts/change-source-statements.sh @@ -0,0 +1,7 @@ +#!/bin/bash +sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" hpcc/hpcc.tf +sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" aks/aks.tf +sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" storage/main.tf +sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" hpcc/hpcc.tf +sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" aks/aks.tf +sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" storage/main.tf diff --git a/scripts/external_storage b/scripts/external_storage index 4ad1b25..3334373 100755 --- a/scripts/external_storage +++ b/scripts/external_storage @@ -9,7 +9,7 @@ else #echo "DEBUG: EXITING because ignore_external_storage is NOT true." while [ ! -e "$repopath/storage/data/config.json" ];do echo "This file does not exist, yet: $repopath/storage/data/config.json" - sleep 10; + sleep 20; done exit 0; fi From 801e2790b02f078c32686894e6843a404dbc1bef Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 15:07:57 +0000 Subject: [PATCH 068/124] Added column 'updatable' to table of options in README.md --- README.md | 50 +++++++++++++++++++++++++------------------------- 1 file changed, 25 insertions(+), 25 deletions(-) mode change 100755 => 100644 README.md diff --git a/README.md b/README.md old mode 100755 new mode 100644 index 02dfdf6..8c8d9b3 --- a/README.md +++ b/README.md @@ -70,31 +70,31 @@ Options have data types. The ones used in this module are: The following options should be set in your `lite.auto.tfvars` file (or entered interactively, if you choose to not create a file). Only a few of them have default values. The rest are required. The 'Updateable' column indicates whether, for any given option, it is possible to successfully apply the update against an already-running HPCC k8s cluster. -|Option|Type|Description| -|:-----|:---|:----------| -| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | -| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | -| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | -| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | -| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | -| `aks_dns_zone_name` | string | Name of an existing Azure DNS Zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | -| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above Azure DNS Zone. Example entry: "app-dns-prod-eastus2" | -| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | -| `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | -| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | -| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | -| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | -| `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | -| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | -| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | -| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | -| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | -| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | -| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | -| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | -| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | -| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | -| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | +|Option|Type|Description|Updatable| +|:-----|:---|:----------|:--------| +| `a_record_name` | string | Name of the A record, of following dns zone, where the ecl watch ip is placed This A record will be created and therefore should not exist in the following dns zone. Example entry: "my-product". This should be something project specific rather than something generic. | Y | +| `admin_username` | string | Username of the administrator of this HPCC Systems cluster. Example entry: "jdoe" | N | +| `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | Y | +| `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | Y | +| `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | Y | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | N | +| `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | N | +| `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | +| `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | +| `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | +| `aks_node_sizes ` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | N | +| `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | Y | +| `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | Y | +| `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | Y | +| `external_storage_desired` | boolean | If you want external storage instead of ephemeral storage then set this variable to true otherwise set it to false. | Y | +| `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | Y | +| `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, set value to ["0.0.0.0/0"] or []. | Y | +| `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | Y | +| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | N | +| `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | Y | +| `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | Y | +| `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | Y | +| `thor_num_workers` | number | The number of Thor workers to allocate. Must be 1 or more. | Y | ## Persistent Storage From 075d3aafe7d30efa7b6f083fb12a84575357cdea Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 15:21:45 +0000 Subject: [PATCH 069/124] Removed automation.tf from aks --- aks/automation.tf | 67 ----------------------------------------------- 1 file changed, 67 deletions(-) delete mode 100644 aks/automation.tf diff --git a/aks/automation.tf b/aks/automation.tf deleted file mode 100644 index 382bd7f..0000000 --- a/aks/automation.tf +++ /dev/null @@ -1,67 +0,0 @@ -/* -resource "azurerm_automation_account" "automation_account" { - name = local.aks_automation.automation_account_name - location = local.location - resource_group_name = module.resource_groups["azure_kubernetes_service"].name - sku_name = var.sku_name - tags = local.tags - # local_authentication_enabled = local.aks_automation.local_authentication_enabled - public_network_access_enabled = local.aks_automation.public_network_access_enabled - - identity { - type = "SystemAssigned" - } -} - -resource "azurerm_role_assignment" "role_assignment" { - #scope = "${data.azurerm_subscription.primary.id}/resourceGroups/${module.resource_groups["azure_kubernetes_service"].name}" - scope = data.azurerm_subscription.current.id - role_definition_name = "Contributor" - principal_id = azurerm_automation_account.automation_account.identity[0].principal_id -} - -resource "azurerm_automation_runbook" "runbook" { - for_each = local.runbook - name = each.value.runbook_name - runbook_type = each.value.runbook_type - content = local.script[each.value.script_name] - location = local.location - resource_group_name = module.resource_groups["azure_kubernetes_service"].name - automation_account_name = azurerm_automation_account.automation_account.name - description = "Runbook for script ${each.value.script_name}" - log_progress = var.log_progress - log_verbose = var.log_verbose - tags = local.tags -} - -resource "azurerm_automation_schedule" "schedule" { - for_each = local.schedule - automation_account_name = azurerm_automation_account.automation_account.name - description = each.value.description - frequency = each.value.frequency - interval = each.value.frequency == "OneTime" ? null : each.value.interval - month_days = each.value.frequency != "Month" ? null : each.value.month_days - week_days = each.value.frequency != "Week" ? null : each.value.week_days - name = each.value.schedule_name - resource_group_name = module.resource_groups["azure_kubernetes_service"].name - # TODO: this assumes the timezone is "America/New_York" and doesn't account for DST - should be fixed - start_time = length(each.value.start_time) == 5 && contains(["Week", "Month"], each.value.frequency) ? contains(each.value.week_days, local.current_day) && tonumber(substr(each.value.start_time, 0, 2)) >= local.current_hour ? "${local.today}T${each.value.start_time}:15-0${local.utc_offset}:00" : "${local.tomorrow}T${each.value.start_time}:15-0${local.utc_offset}:00" : "${local.today}T${each.value.start_time}:15-0${local.utc_offset}:00" - timezone = var.timezone - # expiry_time = each.value.expiry_time -} - -resource "azurerm_automation_job_schedule" "job_schedule" { - for_each = local.schedule - runbook_name = azurerm_automation_runbook.runbook[each.value.runbook_name].name - schedule_name = azurerm_automation_schedule.schedule[each.value.schedule_name].name - resource_group_name = module.resource_groups["azure_kubernetes_service"].name - automation_account_name = azurerm_automation_account.automation_account.name - - parameters = { - resourcename = module.aks.cluster_name - resourcegroupname = module.resource_groups["azure_kubernetes_service"].name - operation = each.value.operation - automationaccount = local.aks_automation.automation_account_name - } -} -*/ From fb61631bd88e4da9afebf9e89dc44bd278492811 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 15:29:51 +0000 Subject: [PATCH 070/124] Increased values of 'managerResources'. NOTE: Dan's 'managerResources' has same values as this 'managerResources'. --- hpcc/lite-locals.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 5044cc3..49a4d6a 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -403,8 +403,8 @@ locals { egress = "engineEgress" tolerations_value = "thorpool" managerResources = { - cpu = 1 - memory = "2G" + cpu = 2 + memory = "4G" } workerResources = { cpu = 3 From c5d78de3a51f15acd7353bf3ee55faceed7bf2e1 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 15:32:54 +0000 Subject: [PATCH 071/124] Increased values of 'workerResources' to match Dan's --- hpcc/lite-locals.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 49a4d6a..7c45385 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -407,7 +407,7 @@ locals { memory = "4G" } workerResources = { - cpu = 3 + cpu = 4 memory = "4G" } workerMemory = { From 99d7fcfd5bbbf0b99a45926cb6632844692d9f5d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 15:37:42 +0000 Subject: [PATCH 072/124] Increased cpu of hthor resources to 2. This is higher than what Dan has. --- hpcc/lite-locals.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index 7c45385..da35930 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -263,7 +263,7 @@ locals { type = "hthor" spillPlane = "spill" resources = { - cpu = "1" + cpu = "2" memory = "4G" } nodeSelector = { workload = "servpool" } From 416b4e43b19a47a28dd118a5b5ab9439ea58c07f Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 20:58:23 +0000 Subject: [PATCH 073/124] Eliminated metadata variables not used --- aks/lite-locals.tf | 1 - aks/locals.tf | 2 +- hpcc/lite-locals.tf | 3 +- logging/LICENSE | 21 ---- logging/data.tf | 2 - logging/elastic4hpcc.auto.tfvars.example | 1 - logging/locals.tf | 5 - logging/log_analytics.auto.tfvars.example | 15 --- logging/logging.tf | 47 --------- logging/main.tf | 49 --------- logging/misc.auto.tfvars.example | 25 ----- logging/outputs.tf | 36 ------- logging/providers.tf | 33 ------ logging/variables.tf | 116 ---------------------- logging/versions.tf | 15 --- vnet/locals.tf | 3 +- 16 files changed, 3 insertions(+), 371 deletions(-) delete mode 100644 logging/LICENSE delete mode 100644 logging/data.tf delete mode 100644 logging/elastic4hpcc.auto.tfvars.example delete mode 100644 logging/locals.tf delete mode 100644 logging/log_analytics.auto.tfvars.example delete mode 100644 logging/logging.tf delete mode 100644 logging/main.tf delete mode 100644 logging/misc.auto.tfvars.example delete mode 100644 logging/outputs.tf delete mode 100644 logging/providers.tf delete mode 100644 logging/variables.tf delete mode 100644 logging/versions.tf diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf index adebbe1..04e2ebe 100644 --- a/aks/lite-locals.tf +++ b/aks/lite-locals.tf @@ -20,7 +20,6 @@ locals { resource_group_type = "app" sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" - additional_tags = { "justification" = "testing" } location = var.aks_azure_region # Acceptable values: eastus, centralus } diff --git a/aks/locals.tf b/aks/locals.tf index f17baa7..2699bef 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -117,7 +117,7 @@ locals { local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) + tags = { "owner" = local.owner.name, "owner_email" = local.owner.email } get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index da35930..f674764 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -18,11 +18,10 @@ locals { resource_group_type = "app" sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" - additional_tags = { "justification" = "testing" } location = var.aks_azure_region # Acceptable values: eastus, centralus } - tags = merge(local.metadata.additional_tags, var.extra_tags) + tags = var.extra_tags disable_naming_conventions = false diff --git a/logging/LICENSE b/logging/LICENSE deleted file mode 100644 index f596f91..0000000 --- a/logging/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2023 HPCC Systems® - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/logging/data.tf b/logging/data.tf deleted file mode 100644 index 5584491..0000000 --- a/logging/data.tf +++ /dev/null @@ -1,2 +0,0 @@ -data "azurerm_subscription" "current" { -} diff --git a/logging/elastic4hpcc.auto.tfvars.example b/logging/elastic4hpcc.auto.tfvars.example deleted file mode 100644 index e7672ed..0000000 --- a/logging/elastic4hpcc.auto.tfvars.example +++ /dev/null @@ -1 +0,0 @@ -elastic4hpcclogs = {} diff --git a/logging/locals.tf b/logging/locals.tf deleted file mode 100644 index 0e4a7bc..0000000 --- a/logging/locals.tf +++ /dev/null @@ -1,5 +0,0 @@ -locals { - tags = merge(var.metadata.additional_tags, { "owner" = var.owner.name, "owner_email" = var.owner.email }) - get_aks_config = fileexists("../aks/data/config.json") ? jsondecode(file("../aks/data/config.json")) : null - hpcc_namespace = var.hpcc.existing_namespace != null ? var.hpcc.existing_namespace : var.hpcc.create_namespace == true ? kubernetes_namespace.hpcc[0].metadata[0].name : "default" -} diff --git a/logging/log_analytics.auto.tfvars.example b/logging/log_analytics.auto.tfvars.example deleted file mode 100644 index 24a7787..0000000 --- a/logging/log_analytics.auto.tfvars.example +++ /dev/null @@ -1,15 +0,0 @@ -azure_log_analytics_workspace = { - name = "my-hpcc-log-analytics-workspace" - internet_query_enabled = true - internet_ingestion_enabled = true - tags = { - "app" = "hpcc" - } -} - -azure_log_analytics_creds = { - AAD_CLIENT_ID = "" - AAD_CLIENT_SECRET = "" - AAD_PRINCIPAL_ID = "" - AAD_TENANT_ID = "" -} diff --git a/logging/logging.tf b/logging/logging.tf deleted file mode 100644 index e6fec02..0000000 --- a/logging/logging.tf +++ /dev/null @@ -1,47 +0,0 @@ -module "logging" { - source = "github.com/gfortil/terraform-azurerm-hpcc-logging.git?ref=HPCC-29420" - - azure_log_analytics_workspace = var.azure_log_analytics_workspace != null ? { - unique_name = true - daily_quota_gb = var.azure_log_analytics_workspace.daily_quota_gb - internet_ingestion_enabled = var.azure_log_analytics_workspace.internet_ingestion_enabled - internet_query_enabled = var.azure_log_analytics_workspace.internet_query_enabled - location = var.location - name = var.azure_log_analytics_workspace.name - resource_group_name = module.resource_group.name - reservation_capacity_in_gb_per_day = var.azure_log_analytics_workspace.reservation_capacity_in_gb_per_day - retention_in_days = var.azure_log_analytics_workspace.retention_in_days - sku = var.azure_log_analytics_workspace.sku - use_existing_workspace = var.azure_log_analytics_workspace.use_existing_workspace - tags = merge(local.tags, var.azure_log_analytics_workspace.tags) - } : null - - // Should be set as an environment variable or stored in a key vault - azure_log_analytics_creds = var.azure_log_analytics_creds - - hpcc = { - namespace = local.hpcc_namespace - version = var.hpcc.version - } - - elastic4hpcclogs = var.azure_log_analytics_workspace == null ? { - internet_enabled = var.elastic4hpcclogs.internet_enabled - name = var.elastic4hpcclogs.name - atomic = var.elastic4hpcclogs.atomic - recreate_pods = var.elastic4hpcclogs.recreate_pods - reuse_values = var.elastic4hpcclogs.reuse_values - reset_values = var.elastic4hpcclogs.reset_values - force_update = var.elastic4hpcclogs.force_update - cleanup_on_fail = var.elastic4hpcclogs.cleanup_on_fail - disable_openapi_validation = var.elastic4hpcclogs.disable_openapi_validation - max_history = var.elastic4hpcclogs.max_history - wait = var.elastic4hpcclogs.wait - dependency_update = var.elastic4hpcclogs.dependency_update - timeout = var.elastic4hpcclogs.timeout - wait_for_jobs = var.elastic4hpcclogs.wait_for_jobs - lint = var.elastic4hpcclogs.lint - remote_chart = var.elastic4hpcclogs.remote_chart - local_chart = var.elastic4hpcclogs.local_chart - version = var.elastic4hpcclogs.version - } : null -} diff --git a/logging/main.tf b/logging/main.tf deleted file mode 100644 index 40948c2..0000000 --- a/logging/main.tf +++ /dev/null @@ -1,49 +0,0 @@ -module "subscription" { - source = "github.com/Azure-Terraform/terraform-azurerm-subscription-data.git?ref=v1.0.0" - subscription_id = data.azurerm_subscription.current.subscription_id -} - -module "naming" { - source = "github.com/Azure-Terraform/example-naming-template.git?ref=v1.0.0" -} - -module "metadata" { - source = "github.com/Azure-Terraform/terraform-azurerm-metadata.git?ref=v1.5.1" - - naming_rules = module.naming.yaml - - market = var.metadata.market - location = var.location - sre_team = var.metadata.sre_team - environment = var.metadata.environment - product_name = var.metadata.product_name - business_unit = var.metadata.business_unit - product_group = var.metadata.product_group - subscription_type = var.metadata.subscription_type - resource_group_type = var.metadata.resource_group_type - subscription_id = data.azurerm_subscription.current.subscription_id - project = var.metadata.project -} - -module "resource_group" { - source = "github.com/Azure-Terraform/terraform-azurerm-resource-group.git?ref=v2.0.0" - - unique_name = true - location = var.location - names = module.metadata.names - tags = local.tags -} - -resource "kubernetes_namespace" "hpcc" { - count = var.hpcc.create_namespace ? 1 : 0 - - metadata { - annotations = { - description = "Created by the logging module." - } - - labels = var.hpcc.labels - - generate_name = "${trimspace(var.owner.name)}" - } -} diff --git a/logging/misc.auto.tfvars.example b/logging/misc.auto.tfvars.example deleted file mode 100644 index 1c62736..0000000 --- a/logging/misc.auto.tfvars.example +++ /dev/null @@ -1,25 +0,0 @@ -owner = { - name = "demo" - email = "demo@lexisnexisrisk.com" -} - -metadata = { - project = "hpccdemo" - product_name = "aks" - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = "contoso" - resource_group_type = "app" - sre_team = "hpccplatform" - subscription_type = "dev" - additional_tags = { "justification" = "testing", "enclosed resource" = "open source logs" } -} - -location = "eastus2" - -hpcc = { - # existing_namespace = "" - version = "9.2.12-rc1" - create_namespace = true -} diff --git a/logging/outputs.tf b/logging/outputs.tf deleted file mode 100644 index 6fc8231..0000000 --- a/logging/outputs.tf +++ /dev/null @@ -1,36 +0,0 @@ -output "logaccess_body" { - description = "logaccess configuration to apply to the HPCC helm deployment." - value = module.logging.logaccess_body -} - -output "workspace_resource_id" { - description = "The resource ID of the workspace" - value = module.logging.workspace_resource_id -} - -output "workspace_id" { - description = "The Azure Analytics Workspace ID" - value = module.logging.workspace_id -} - -output "hpcc_namespace" { - description = "The namespace where the Kubernetes secret has been created and in which HPCC must be deployed." - value = var.hpcc.create_namespace ? kubernetes_namespace.hpcc[0].metadata[0].name : var.hpcc.namespace_prefix -} - -resource "local_file" "logaccess_body" { - content = module.logging.logaccess_body - filename = "${path.module}/data/logaccess_body.yaml" -} - -resource "local_file" "workspace_resource_id" { - content = module.logging.workspace_resource_id - filename = "${path.module}/data/workspace_resource_id.txt" -} - -resource "local_file" "hpcc_namespace" { - count = var.hpcc.create_namespace ? 1 : 0 - - content = var.hpcc.create_namespace ? kubernetes_namespace.hpcc[0].metadata[0].name : var.hpcc.namespace_prefix - filename = "${path.module}/data/hpcc_namespace.txt" -} diff --git a/logging/providers.tf b/logging/providers.tf deleted file mode 100644 index 3af0a7d..0000000 --- a/logging/providers.tf +++ /dev/null @@ -1,33 +0,0 @@ -provider "azurerm" { - features {} - use_cli = true -} - -provider "http" {} - -provider "helm" { - alias = "default" - - kubernetes { - host = module.aks.cluster_endpoint - cluster_ca_certificate = base64decode(module.aks.cluster_certificate_authority_data) - - exec { - api_version = "client.authentication.k8s.io/v1beta1" - command = "kubelogin" - args = ["get-token", "--server-id", "6dae42f8-4368-4678-94ff-3960e28e3630", "--login", "azurecli"] - env = local.azure_auth_env - } - } - - experiments { - manifest = true - } -} - -provider "kubernetes" { - host = local.get_aks_config.kube_admin_config[0].host - client_certificate = base64decode(local.get_aks_config.kube_admin_config[0].client_certificate) - client_key = base64decode(local.get_aks_config.kube_admin_config[0].client_key) - cluster_ca_certificate = base64decode(local.get_aks_config.kube_admin_config[0].cluster_ca_certificate) -} diff --git a/logging/variables.tf b/logging/variables.tf deleted file mode 100644 index bd35a38..0000000 --- a/logging/variables.tf +++ /dev/null @@ -1,116 +0,0 @@ -variable "owner" { - description = "Information for the user who administers the deployment." - type = object({ - name = string - email = string - }) - - validation { - condition = try( - regex("hpccdemo", var.owner.name) != "hpccdemo", true - ) && try( - regex("hpccdemo", var.owner.email) != "hpccdemo", true - ) && try( - regex("@example.com", var.owner.email) != "@example.com", true - ) - error_message = "Your name and email are required in the owner block and must not contain hpccdemo or @example.com." - } -} - -variable "metadata" { - description = "Metadata module variables." - type = object({ - market = string - sre_team = string - environment = string - product_name = string - business_unit = string - product_group = string - subscription_type = string - resource_group_type = string - project = string - additional_tags = optional(map(string)) - }) - - nullable = false -} - -variable "location" { - description = "Azure location" - type = string - - default = "eastus" -} - -variable "azure_log_analytics_workspace" { - description = "Azure log analytics workspace attributes" - type = object({ - unique_name = optional(bool) - name = string - daily_quota_gb = optional(number) - internet_ingestion_enabled = optional(bool) - internet_query_enabled = optional(bool) - reservation_capacity_in_gb_per_day = optional(number) - retention_in_days = optional(number) - sku = optional(string) - tags = optional(map(string)) - use_existing_workspace = optional(object({ - name = string - resource_group_name = string - })) - }) - - default = null -} - -variable "azure_log_analytics_creds" { - description = "Credentials for the Azure log analytics workspace" - type = object({ - AAD_TENANT_ID = string - AAD_CLIENT_ID = string - AAD_CLIENT_SECRET = string - AAD_PRINCIPAL_ID = string - - }) - - sensitive = true - default = null -} - -variable "hpcc" { - description = "HPCC Platform attributes" - type = object({ - version = optional(string, "latest") - existing_namespace = optional(string) - labels = optional(object({ name = string }), { name = "hpcc" }) - create_namespace = optional(bool, false) - }) - - nullable = false -} - -variable "elastic4hpcclogs" { - description = "The attributes for elastic4hpcclogs." - type = object({ - internet_enabled = optional(bool, true) - name = optional(string, "myelastic4hpcclogs") - atomic = optional(bool) - recreate_pods = optional(bool) - reuse_values = optional(bool) - reset_values = optional(bool) - force_update = optional(bool) - cleanup_on_fail = optional(bool) - disable_openapi_validation = optional(bool) - max_history = optional(number) - wait = optional(bool, true) - dependency_update = optional(bool, true) - timeout = optional(number, 300) - wait_for_jobs = optional(bool) - lint = optional(bool) - remote_chart = optional(string, "https://hpcc-systems.github.io/helm-chart") - local_chart = optional(string) - version = optional(string, "latest") - }) - - default = null -} diff --git a/logging/versions.tf b/logging/versions.tf deleted file mode 100644 index e5a15eb..0000000 --- a/logging/versions.tf +++ /dev/null @@ -1,15 +0,0 @@ -terraform { - required_version = ">= 1.4.6" - - required_providers { - azurerm = { - source = "hashicorp/azurerm" - version = ">= 3.63.0" - } - - http = { - source = "hashicorp/http" - version = ">=3.2.1" - } - } -} diff --git a/vnet/locals.tf b/vnet/locals.tf index 344ca8e..0d11607 100644 --- a/vnet/locals.tf +++ b/vnet/locals.tf @@ -16,7 +16,6 @@ locals { resource_group_type = "app" sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" - additional_tags = { "justification" = "testing" } location = var.aks_azure_region # Acceptable values: eastus, centralus } @@ -39,7 +38,7 @@ locals { local.metadata.resource_group_type != "" ? { resource_group_type = local.metadata.resource_group_type } : {} ) : module.metadata.names - tags = merge(local.metadata.additional_tags, { "owner" = local.owner.name, "owner_email" = local.owner.email }) + tags = { "owner" = local.owner.name, "owner_email" = local.owner.email } private_subnet_id = module.virtual_network.aks.hpcc.subnets["private"].id From 0274223deb6ea0063c3cec16ea0034adf29360aa Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 21:12:54 +0000 Subject: [PATCH 074/124] Removed metadata from aks, hpcc, and vnet (not in storage). Now it is copied from root directory to each of these directories. --- aks/lite-locals.tf | 13 ------------- hpcc/lite-locals.tf | 13 ------------- metadata.tf | 14 ++++++++++++++ scripts/deploy | 3 +++ vnet/locals.tf | 13 ------------- 5 files changed, 17 insertions(+), 39 deletions(-) create mode 100644 metadata.tf diff --git a/aks/lite-locals.tf b/aks/lite-locals.tf index 04e2ebe..cf18cdf 100644 --- a/aks/lite-locals.tf +++ b/aks/lite-locals.tf @@ -10,19 +10,6 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - metadata = { - project = format("%shpccplatform", local.owner_name_initials) - product_name = format("%shpccplatform", local.owner_name_initials) - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = format("%shpcc", local.owner_name_initials) - resource_group_type = "app" - sre_team = format("%shpccplatform", local.owner_name_initials) - subscription_type = "dev" - location = var.aks_azure_region # Acceptable values: eastus, centralus - } - core_services_config = { alertmanager = { smtp_host = "smtp-hostname.ds:25" diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf index f674764..e9725f7 100755 --- a/hpcc/lite-locals.tf +++ b/hpcc/lite-locals.tf @@ -8,19 +8,6 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - metadata = { - project = format("%shpccplatform", local.owner_name_initials) - product_name = format("%shpccplatform", local.owner_name_initials) - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = format("%shpcc", local.owner_name_initials) - resource_group_type = "app" - sre_team = format("%shpccplatform", local.owner_name_initials) - subscription_type = "dev" - location = var.aks_azure_region # Acceptable values: eastus, centralus - } - tags = var.extra_tags disable_naming_conventions = false diff --git a/metadata.tf b/metadata.tf new file mode 100644 index 0000000..69b53f9 --- /dev/null +++ b/metadata.tf @@ -0,0 +1,14 @@ +locals { + metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + location = var.aks_azure_region # Acceptable values: eastus, centralus + } +} diff --git a/scripts/deploy b/scripts/deploy index 00f9ae7..c420950 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -69,6 +69,9 @@ done cd $name; # cd into vnet or storage or aks or hpcc +echo "Copy metadata.tf in root directory to this directory." +cp -v ../metadata.tf . + # put the root directory's lite.auto.tfvars (either all or part) in either aks or hpcc # directory. if [ -e "../lite.auto.tfvars" ];then diff --git a/vnet/locals.tf b/vnet/locals.tf index 0d11607..0f5b365 100644 --- a/vnet/locals.tf +++ b/vnet/locals.tf @@ -6,19 +6,6 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - metadata = { - project = format("%shpccplatform", local.owner_name_initials) - product_name = format("%shpccplatform", local.owner_name_initials) - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = format("%shpcc", local.owner_name_initials) - resource_group_type = "app" - sre_team = format("%shpccplatform", local.owner_name_initials) - subscription_type = "dev" - location = var.aks_azure_region # Acceptable values: eastus, centralus - } - resource_groups = { virtual_network = { tags = { "enclosed resource" = "open source vnet" } From bfd5142c9f10afd52f710a32f1a4383dfc5b39f4 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 21:16:23 +0000 Subject: [PATCH 075/124] Removed metadata from aks, hpcc, storage, and vnet (not in storage). Now it is copied from root directory to each of these directories. --- storage/locals.tf | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/storage/locals.tf b/storage/locals.tf index 89be4be..458572b 100644 --- a/storage/locals.tf +++ b/storage/locals.tf @@ -6,20 +6,6 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - metadata = { - project = format("%shpccplatform", local.owner_name_initials) - product_name = format("%shpccplatform", local.owner_name_initials) - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = format("%shpcc", local.owner_name_initials) - resource_group_type = "app" - sre_team = format("%shpccplatform", local.owner_name_initials) - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = var.aks_azure_region # Acceptable values: eastus, centralus - } - get_vnet_config = fileexists("../vnet/data/config.json") ? jsondecode(file("../vnet/data/config.json")) : null virtual_network = var.virtual_network != null ? var.virtual_network : [ From 5617f980cd0757d5d7d5ae81997bd8ff7d207aff Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 5 Dec 2023 21:36:06 +0000 Subject: [PATCH 076/124] Removed metadata from aks, hpcc, storage, and vnet. Now it is copied from ../scripts/needed-auto-tfvars-files/metadata.tf to each of these directories. --- scripts/deploy | 2 +- metadata.tf => scripts/needed-auto-tfvars-files/metadata.tf | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename metadata.tf => scripts/needed-auto-tfvars-files/metadata.tf (100%) diff --git a/scripts/deploy b/scripts/deploy index c420950..3d3f6f4 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -70,7 +70,7 @@ done cd $name; # cd into vnet or storage or aks or hpcc echo "Copy metadata.tf in root directory to this directory." -cp -v ../metadata.tf . +cp -v ../scripts/needed-auto-tfvars-files/metadata.tf . # put the root directory's lite.auto.tfvars (either all or part) in either aks or hpcc # directory. diff --git a/metadata.tf b/scripts/needed-auto-tfvars-files/metadata.tf similarity index 100% rename from metadata.tf rename to scripts/needed-auto-tfvars-files/metadata.tf From 6471329380ec8ca98095a07e5826fd557b5b6cc7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 6 Dec 2023 19:27:23 +0000 Subject: [PATCH 077/124] Added new variable, 'aks_capacity' which defines the minimum and maximum nodes of each node pool. --- README.md | 3 ++- aks/locals.tf | 16 ++++++++-------- lite-variables.tf | 14 ++++++++++++++ 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 8c8d9b3..1992071 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,8 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | | `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | -| `aks_node_sizes ` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | N | +| `aks_node_sizes ` | map of string | The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = "xlarge", serv = "2xlarge", spray = "xlarge", thor = "xlarge" }'. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | N | +| `aks_capacity ` | map of number | The min and max number of nodes of each node pool in the HPCC Systems. Example format is '{ roxie_min = 1, roxie_max = 3, serv_min = 1, serv_max = 3, spray_min = 1, spray_max = 3, thor_min = 1, thor_max = 3}'. | N | | `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | Y | | `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | Y | | `enable_thor` | boolean | If you want a thor cluster then 'enable_thor' must be set to true Otherwise it is set to false | Y | diff --git a/aks/locals.tf b/aks/locals.tf index 2699bef..dad8de3 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -13,8 +13,8 @@ locals { node_type_version = "v2" node_size = var.aks_node_sizes.roxie single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = var.aks_capacity.roxie_min + max_capacity = var.aks_capacity.roxie_max labels = { "lnrs.io/tier" = "standard" "workload" = "roxiepool" @@ -31,8 +31,8 @@ locals { node_type_version = "v2" # v1, v2 node_size = var.aks_node_sizes.thor single_group = false - min_capacity = 3 - max_capacity = 6 + min_capacity = var.aks_capacity.thor_min + max_capacity = var.aks_capacity.thor_max labels = { "lnrs.io/tier" = "standard" "workload" = "thorpool" @@ -48,8 +48,8 @@ locals { node_type_version = "v1" node_size = var.aks_node_sizes.serv single_group = false - min_capacity = 1 - max_capacity = 3 + min_capacity = var.aks_capacity.serv_min + max_capacity = var.aks_capacity.serv_max labels = { "lnrs.io/tier" = "standard" "workload" = "servpool" @@ -65,8 +65,8 @@ locals { node_type_version = "v1" node_size = var.aks_node_sizes.spray single_group = false - min_capacity = 3 - max_capacity = 6 + min_capacity = var.aks_capacity.spray_min + max_capacity = var.aks_capacity.spray_max labels = { "lnrs.io/tier" = "standard" "workload" = "spraypool" diff --git a/lite-variables.tf b/lite-variables.tf index ce9c47b..7f510bf 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -59,6 +59,20 @@ variable "aks_node_sizes" { thor = optional(string, "2xlarge") }) } + +variable "aks_capacity" { + description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." + type = object({ + roxie_min = optional(number, 1) + roxie_max = optional(number, 3) + serv_min = optional(number, 1) + serv_max = optional(number, 3) + spray_min = optional(number, 3) + spray_max = optional(number, 6) + thor_min = optional(number, 3) + thor_max = optional(number, 6) + }) +} #===== end of aks variables ===== variable "my_azure_id" { From ab729603df60cd9b8a905f4f5d42074798e315b1 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 6 Dec 2023 19:52:24 +0000 Subject: [PATCH 078/124] In lite-variables.tf, I changed the descriptions of 'aks_node_sizes' and 'aks_capacity'. Now they match what is in README.md --- lite-variables.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lite-variables.tf b/lite-variables.tf index 7f510bf..5f57bcc 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -51,7 +51,7 @@ variable "aks_admin_ip_cidr_map" { } variable "aks_node_sizes" { - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." + description = "The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = "xlarge", serv = "2xlarge", spray = "xlarge", thor = "xlarge" }'.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." type = object({ roxie = optional(string, "2xlarge") serv = optional(string, "4xlarge") @@ -61,7 +61,7 @@ variable "aks_node_sizes" { } variable "aks_capacity" { - description = "REQUIRED. The VM size for each node in the HPCC Systems node pool.\nRecommend \"Standard_B4ms\" or better.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." + description = "The min and max number of nodes of each node pool in the HPCC Systems. Example format is '{ roxie_min = 1, roxie_max = 3, serv_min = 1, serv_max = 3, spray_min = 1, spray_max = 3, thor_min = 1, thor_max = 3}'." type = object({ roxie_min = optional(number, 1) roxie_max = optional(number, 3) From 96bbe2c529863f5490f65c8d38f37bc22f9adfec Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 6 Dec 2023 20:09:56 +0000 Subject: [PATCH 079/124] In all bash scripts, replaced with --- hpcc/scripts/delete_ephemeral_storage_accounts | 2 +- scripts/deploy | 14 +++++++------- scripts/destroy | 10 +++++----- scripts/get_rg_from_file | 4 ++-- scripts/mkplan | 16 ++++++++-------- 5 files changed, 23 insertions(+), 23 deletions(-) diff --git a/hpcc/scripts/delete_ephemeral_storage_accounts b/hpcc/scripts/delete_ephemeral_storage_accounts index 0b78f2c..3d58cbc 100755 --- a/hpcc/scripts/delete_ephemeral_storage_accounts +++ b/hpcc/scripts/delete_ephemeral_storage_accounts @@ -6,7 +6,7 @@ else fi echo "In $0. Inputted resource groups is \"$rg\""; sleep 20; -estos=`az resource list --resource-group $rg|egrep "id\":.*storageAccounts\/hpcc"|sed "s/^ *\"id\": \"//"|sed "s/\", *$//"` +estos=$(az resource list --resource-group $rg|egrep "id\":.*storageAccounts\/hpcc"|sed "s/^ *\"id\": \"//"|sed "s/\", *$//") if [ "$estos" == "" ];then echo "In $0 There are no hpcc storage accounts in the resource group, $rg. EXITING.";exit; fi diff --git a/scripts/deploy b/scripts/deploy index 3d3f6f4..afa13bb 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -1,5 +1,5 @@ #!/bin/bash -thisdir=`pwd`/`dirname $0` +thisdir=$(pwd)/$(dirname $0) repodir=$(echo $thisdir|sed "s/\/scripts\/*//") if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then name=$1 @@ -33,8 +33,8 @@ function assert_fail () { # If kubernetes cluster doesn't exist then make sure aks/data/config.json # and hpcc/data/config.json doesn't exist # Delete both hpcc/data and aks/data if kubernetes cluster doesn't exist -ns=`kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//"` -pods=`kubectl get pods 2>&1` +ns=$(kubectl get ns 2>&1|egrep -v NAME|sed "s/ *.*$//") +pods=$(kubectl get pods 2>&1) if [[ "$ns" == *"Unable"* ]];then # kubenetes doesn't exist of there are no namespaces rm -vrf hpcc/data;cd hpcc; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. rm -vrf aks/data;cd aks; rm *.tfstate*;rm .terraform.lock.hcl ;sudo rm -r .terraform;cd .. @@ -54,8 +54,8 @@ fi for dir in aks vnet storage;do if [ -e "$dir/data/config.json" ];then # Get resource group name from file - rg=`$thisdir/get_rg_from_file $dir/data/config.json` - rgexist=`az group exists --name $rg` + rg=$($thisdir/get_rg_from_file $dir/data/config.json) + rgexist=$(az group exists --name $rg) if [ "$rgexist" == "false" ]; then echo "In $0. $dir resource group, \"$rg\" does not exists. So deleting vnet/data and any tfstate files id $dir." rm -vrf $dir/data @@ -77,7 +77,7 @@ cp -v ../scripts/needed-auto-tfvars-files/metadata.tf . if [ -e "../lite.auto.tfvars" ];then # Check if there has been a change since last apply. if [ -e "/tmp/${name}.lite.auto.tfvars" ];then - tfvarsdiff=`diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars` + tfvarsdiff=$(diff /tmp/${name}.lite.auto.tfvars ../lite.auto.tfvars) else tfvarsdiff="" fi @@ -102,7 +102,7 @@ if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi -plan=`$thisdir/mkplan deploy_${name}.plan` +plan=$($thisdir/mkplan deploy_${name}.plan) if [ "$tfvarsdiff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi diff --git a/scripts/destroy b/scripts/destroy index cd44dc8..b1bd96c 100755 --- a/scripts/destroy +++ b/scripts/destroy @@ -1,6 +1,6 @@ #!/bin/bash -thisdir=$(d=`dirname $0`;cd $d;pwd) -repodir=`echo $thisdir|sed "s/\/scripts//"` +thisdir=$(d=$(dirname $0);cd $d;pwd) +repodir=$(echo $thisdir|sed "s/\/scripts//") #======================================================================== function assert_fail () { echo ">>>>>>>>>>>>>>>>>>> EXECUTING: $*" @@ -21,19 +21,19 @@ elif [ "$1" == "aks" ];then assert_fail scripts/destroy hpcc fi cd $repodir/$1; -name=$(basename `pwd`) +name=$(basename $(pwd)) if [ "$name" == "hpcc" ];then assert_fail kubectl delete pods --all --force fi if [ ! -d "$HOME/tflogs" ];then mkdir $HOME/tflogs fi -plan=`$thisdir/mkplan ${name}_destroy.plan` +plan=$($thisdir/mkplan ${name}_destroy.plan) if [ ! -d "data" ] || [ ! -f "data/config.json" ]; then echo "$name is already destroyed";exit 0; fi echo "=============== Destroying $name. Executing 'terraform destroy' ==============="; assert_fail terraform destroy -auto-approve rm -vr data cd .. -r=`terraform state list|egrep "_$name"` +r=$(terraform state list|egrep "_$name") terraform state rm $r diff --git a/scripts/get_rg_from_file b/scripts/get_rg_from_file index 473557b..334b506 100755 --- a/scripts/get_rg_from_file +++ b/scripts/get_rg_from_file @@ -6,8 +6,8 @@ else exit 1 fi -rg=`sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group_name\": *\"//"|sed "s/\".*$//"` +rg=$(sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group_name\": *\"//"|sed "s/\".*$//") if [[ "$rg" != *"hpccplatform"* ]];then - rg=`sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group\": *\"//"|sed "s/\".*$//"` + rg=$(sed "/\"resource_group\": *\"|\"resource_group_name\": *\"/q" $config_file|sed "s/^.*\"resource_group\": *\"//"|sed "s/\".*$//") fi echo $rg diff --git a/scripts/mkplan b/scripts/mkplan index feca5a6..8cb4a58 100755 --- a/scripts/mkplan +++ b/scripts/mkplan @@ -1,7 +1,7 @@ #!/bin/bash -month_date=`date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d'` +month_date=$(date -d "$D" '+%Y';date -d "$D" '+%m';date -d "$D" '+%d') timezone_difference=6 -H=`date +'%H'` +H=$(date +'%H') #echo "H=\"$H\", timezone_difference=\"$timezone_difference\"" 1>&2 if [ $H -le $timezone_difference ];then H=$((25 - $timezone_difference)); @@ -10,17 +10,17 @@ else fi #echo "After timezone_difference is subtracted: H=\"$H\"" 1>&2 printf -v H "%02d" $H -M=`date +'%M'` +M=$(date +'%M') printf -v M "%02d" $M -month_date=`echo $month_date|sed "s/ *//g"`; -month_date="$month_date-$H$M"; +month_date=$(echo $month_date|sed "s/ *//g") +month_date="$month_date-$H$M" repo_name=""; if [ $# -gt 0 ];then repo_name=$1 else - cdir=`pwd` - reHOME=`echo $HOME|sed "s/[/]/./g"` - repo_name=`echo $cdir|sed "s/^$reHOME.//"|sed "s/[/]/-/g"`; + cdir=$(pwd) + reHOME=$(echo $HOME|sed "s/[/]/./g") + repo_name=$(echo $cdir|sed "s/^$reHOME.//"|sed "s/[/]/-/g") #echo "DEBUG: No arguments: cdir=\"$cdir\", reHOME=\"$reHOME\", repo_name=\"$repo_name\"" 1>&2 fi plan_name="$HOME/tflogs/$repo_name-$month_date.plan" From 4d88cc9b99fc2b9e454ef13d690e422fa79dac47 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 6 Dec 2023 20:15:55 +0000 Subject: [PATCH 080/124] Removed referenced branch from all source statements, since the branch is the default branch. --- aks/aks.tf | 2 +- hpcc/hpcc.tf | 2 +- storage/main.tf | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 286511f..9f90c94 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,6 +1,6 @@ module "aks" { depends_on = [random_string.string] - source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git?ref=make-logging-and-monitoring-optional" + source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 0614a3b..81b1839 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,5 +1,5 @@ module "hpcc" { - source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git?ref=add-ecl-code-security-misc" + source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" environment = local.metadata.environment productname = local.metadata.product_name diff --git a/storage/main.tf b/storage/main.tf index ece147a..36c2f84 100644 --- a/storage/main.tf +++ b/storage/main.tf @@ -1,5 +1,5 @@ module "storage" { - source = "git@github.com:hpccsystems-solutions-lab/terraform-azurerm-hpcc-storage.git?ref=HPCC-27615-add-rm-0000-cidr" + source = "git@github.com:hpccsystems-solutions-lab/terraform-azurerm-hpcc-storage.git" owner = local.owner disable_naming_conventions = var.disable_naming_conventions From 91266e92a6a1914252ef2e7abbd588877a0c00b3 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 7 Dec 2023 14:11:19 +0000 Subject: [PATCH 081/124] In lite-variables.tf, line 54 added '\' before each " in description. --- lite-variables.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-variables.tf b/lite-variables.tf index 5f57bcc..8901881 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -51,7 +51,7 @@ variable "aks_admin_ip_cidr_map" { } variable "aks_node_sizes" { - description = "The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = "xlarge", serv = "2xlarge", spray = "xlarge", thor = "xlarge" }'.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." + description = "The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = \"xlarge\", serv = \"2xlarge\", spray = \"xlarge\", thor = \"xlarge\" }'.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." type = object({ roxie = optional(string, "2xlarge") serv = optional(string, "4xlarge") From f45720c18d3b4c9c87eff1d1b88490d6306ea692 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 7 Dec 2023 15:23:40 +0000 Subject: [PATCH 082/124] In hpcc/outputs.tf, prefixed eclwatch url with 'https://'. In lite-variables.tf, added default value to aks_capacity. --- hpcc/outputs.tf | 2 +- lite-variables.tf | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index 655cea3..06d2a93 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -1,6 +1,6 @@ output "eclwatch_url" { description = "Print the ECL Watch URL." - value = format("%s.%s:18010",var.a_record_name, var.aks_dns_zone_name) + value = format("https://%s.%s:18010",var.a_record_name, var.aks_dns_zone_name) } output "deployment_resource_group" { diff --git a/lite-variables.tf b/lite-variables.tf index 8901881..7ce8407 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -72,6 +72,16 @@ variable "aks_capacity" { thor_min = optional(number, 3) thor_max = optional(number, 6) }) + default = { + roxie_min = 1 + roxie_max = 3 + serv_min = 1 + serv_max = 3 + spray_min = 3 + spray_max = 6 + thor_min = 3 + thor_max = 6 + } } #===== end of aks variables ===== From 34056101b415ce7d25eba706b16cc528497c3021 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 7 Dec 2023 17:55:06 +0000 Subject: [PATCH 083/124] In README.md, documented outpuss of hpcc, aks, vnet. There are no outputs of storage. Also, made sure all outputs have 'description' --- README.md | 22 ++++++++++++++++++---- aks/outputs.tf | 8 +++++--- hpcc/outputs.tf | 1 + storage/outputs.tf | 1 + vnet/outputs.tf | 10 +++++++++- 5 files changed, 34 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 1992071..2bb0daf 100644 --- a/README.md +++ b/README.md @@ -35,10 +35,24 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi 1. Use -var arguments on the command line when executing the Terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. 1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). -At the end of a successful deployment these items are output: -* The URL used to access ECL Watch, `eclwatch_url`. -* The deployment azure resource group, `deployment_resource_group`. -* Whether there is external storage or not, `external_storage_config_exists`. +At the end of a successful deployment these items are output for aks, hpcc, and vnet: +* aks + * Advisor recommendations or 'none', `advisor_recommendations`. + * Location of the aks credentials, `aks_login`. + * Name of the Azure Kubernetes Service, `cluster_name`. + * Resource group where the cluster is deployed, `cluster_resource_group_name`. +* hpcc + * The URL used to access ECL Watch, `eclwatch_url`. + * The deployment azure resource group, `deployment_resource_group`. + * Whether there is external storage or not, `external_storage_config_exists`. +* vnet + * Advisor recommendations or 'none', `advisor_recommendations`. + * ID of private subnet, `private_subnet_id`. + * ID of public subnet, `public_subnet_id`. + * ID of route table, `route_table_id`. + * Route table name, `route_table_name`. + * Virtual network resource group name, `resource_group_name`. + * Virtual network name, `vnet_name`. ## Available Options diff --git a/aks/outputs.tf b/aks/outputs.tf index 2c9eaab..ba98d66 100644 --- a/aks/outputs.tf +++ b/aks/outputs.tf @@ -1,17 +1,19 @@ output "advisor_recommendations" { - value = data.azurerm_advisor_recommendations.advisor.recommendations + description = "Advisor recommendations or 'none'" + value = data.azurerm_advisor_recommendations.advisor.recommendations == tolist([])? "none" : data.azurerm_advisor_recommendations.advisor.recommendations } output "aks_login" { + description = "Location of the aks credentials" value = "az aks get-credentials --name ${module.aks.cluster_name} --resource-group ${module.resource_groups["azure_kubernetes_service"].name}" } output "cluster_name" { - description = "The name of the Azure Kubernetes Service." + description = "Name of the Azure Kubernetes Service" value = module.aks.cluster_name } output "cluster_resource_group_name" { - description = "The resource group where the cluster is deployed." + description = "Resource group where the cluster is deployed" value = module.resource_groups["azure_kubernetes_service"].name } diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index 06d2a93..7b66964 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -9,6 +9,7 @@ output "deployment_resource_group" { } output "external_storage_config_exists" { + description = "Print whether external storage exists, i.e. true of false." value = fileexists("../storage/data/config.json") ? true : false } diff --git a/storage/outputs.tf b/storage/outputs.tf index 9e5808a..f5cbbaa 100644 --- a/storage/outputs.tf +++ b/storage/outputs.tf @@ -1,4 +1,5 @@ resource "local_file" "config" { + description = "File containing configuration of external storage." content = module.storage.config filename = "${path.module}/data/config.json" diff --git a/vnet/outputs.tf b/vnet/outputs.tf index 80bf464..08abc72 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -1,32 +1,40 @@ output "advisor_recommendations" { - value = data.azurerm_advisor_recommendations.advisor.recommendations + description = "Advisor recommendations or 'none'." + value = data.azurerm_advisor_recommendations.advisor.recommendations == tolist([])? "none" : data.azurerm_advisor_recommendations.advisor.recommendations } output "private_subnet_id" { + description = "ID of private subnet." value = module.virtual_network.aks.hpcc.subnets["private"].id } output "public_subnet_id" { + description = "ID of public subnet." value = module.virtual_network.aks.hpcc.subnets["public"].id } output "route_table_id" { + description = "ID of route table." value = module.virtual_network.aks.hpcc.route_table_id } output "route_table_name" { + description = "Route table name." value = "${module.virtual_network.vnet.resource_group_name}-aks-hpcc-routetable" } output "resource_group_name" { + description = "Virtual network resource group name." value = module.virtual_network.vnet.resource_group_name } output "vnet_name" { + description = "Virtual network name." value = module.virtual_network.vnet.name } resource "local_file" "output" { + description = "File containing the configuration of the virtual network." content = local.config filename = "${path.module}/data/config.json" From 6e6b6c3ef954863fc6175e92f14e9d93428ea170 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 8 Dec 2023 14:37:41 +0000 Subject: [PATCH 084/124] Change default node size for spray pool from 2xlarge to large. --- lite-variables.tf | 2 +- lite.auto.tfvars.example | 2 +- vnet/outputs.tf | 1 - 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/lite-variables.tf b/lite-variables.tf index 7ce8407..b292c08 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -55,7 +55,7 @@ variable "aks_node_sizes" { type = object({ roxie = optional(string, "2xlarge") serv = optional(string, "4xlarge") - spray = optional(string, "2xlarge") + spray = optional(string, "large") thor = optional(string, "2xlarge") }) } diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 236778a..1dd425c 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -124,7 +124,7 @@ extra_tags={} # Value type: string # Updateable: N -aks_node_sizes = { roxie = "2xlarge", serv = "4xlarge", spray = "2xlarge", thor = "2xlarge" } +aks_node_sizes = { roxie = "2xlarge", serv = "4xlarge", spray = "large", thor = "2xlarge" } #------------------------------------------------------------------------------ diff --git a/vnet/outputs.tf b/vnet/outputs.tf index 08abc72..d63d65a 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -34,7 +34,6 @@ output "vnet_name" { } resource "local_file" "output" { - description = "File containing the configuration of the virtual network." content = local.config filename = "${path.module}/data/config.json" From 6ece380b024aa14b68635b4be17390162d8b291e Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 8 Dec 2023 14:47:03 +0000 Subject: [PATCH 085/124] Reduced the size of nodes in each node pool. --- lite-variables.tf | 6 +++--- lite.auto.tfvars.example | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lite-variables.tf b/lite-variables.tf index b292c08..152f9ee 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -53,10 +53,10 @@ variable "aks_admin_ip_cidr_map" { variable "aks_node_sizes" { description = "The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = \"xlarge\", serv = \"2xlarge\", spray = \"xlarge\", thor = \"xlarge\" }'.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." type = object({ - roxie = optional(string, "2xlarge") - serv = optional(string, "4xlarge") + roxie = optional(string, "xlarge") + serv = optional(string, "2xlarge") spray = optional(string, "large") - thor = optional(string, "2xlarge") + thor = optional(string, "xlarge") }) } diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 1dd425c..215301f 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -124,7 +124,7 @@ extra_tags={} # Value type: string # Updateable: N -aks_node_sizes = { roxie = "2xlarge", serv = "4xlarge", spray = "large", thor = "2xlarge" } +aks_node_sizes = { roxie = "xlarge", serv = "2xlarge", spray = "large", thor = "xlarge" } #------------------------------------------------------------------------------ From eae050fc50c013d6b594553f1af367a35a3d3550 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 8 Dec 2023 17:37:37 +0000 Subject: [PATCH 086/124] Fixed value of output 'advisor_recommendations' for both aks/outputs.tf and vnet\outputs.tf --- aks/outputs.tf | 2 +- lite-variables.tf | 12 ++++++------ vnet/outputs.tf | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/aks/outputs.tf b/aks/outputs.tf index ba98d66..908d8a1 100644 --- a/aks/outputs.tf +++ b/aks/outputs.tf @@ -1,6 +1,6 @@ output "advisor_recommendations" { description = "Advisor recommendations or 'none'" - value = data.azurerm_advisor_recommendations.advisor.recommendations == tolist([])? "none" : data.azurerm_advisor_recommendations.advisor.recommendations + value = data.azurerm_advisor_recommendations.advisor.recommendations } output "aks_login" { description = "Location of the aks credentials" diff --git a/lite-variables.tf b/lite-variables.tf index 152f9ee..021c32f 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -63,23 +63,23 @@ variable "aks_node_sizes" { variable "aks_capacity" { description = "The min and max number of nodes of each node pool in the HPCC Systems. Example format is '{ roxie_min = 1, roxie_max = 3, serv_min = 1, serv_max = 3, spray_min = 1, spray_max = 3, thor_min = 1, thor_max = 3}'." type = object({ - roxie_min = optional(number, 1) + roxie_min = optional(number, 0) roxie_max = optional(number, 3) serv_min = optional(number, 1) serv_max = optional(number, 3) - spray_min = optional(number, 3) + spray_min = optional(number, 0) spray_max = optional(number, 6) - thor_min = optional(number, 3) + thor_min = optional(number, 0) thor_max = optional(number, 6) }) default = { - roxie_min = 1 + roxie_min = 0 roxie_max = 3 serv_min = 1 serv_max = 3 - spray_min = 3 + spray_min = 0 spray_max = 6 - thor_min = 3 + thor_min = 0 thor_max = 6 } } diff --git a/vnet/outputs.tf b/vnet/outputs.tf index d63d65a..497f99c 100644 --- a/vnet/outputs.tf +++ b/vnet/outputs.tf @@ -1,6 +1,6 @@ output "advisor_recommendations" { description = "Advisor recommendations or 'none'." - value = data.azurerm_advisor_recommendations.advisor.recommendations == tolist([])? "none" : data.azurerm_advisor_recommendations.advisor.recommendations + value = data.azurerm_advisor_recommendations.advisor.recommendations } output "private_subnet_id" { From 11554af16086ee864a1a8fbba53955c2b89550b4 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 11 Dec 2023 14:42:02 +0000 Subject: [PATCH 087/124] Calculates max capacity of thorpool. Set thor cpu and ram. --- aks/locals.tf | 16 +- hpcc/lite-locals.tf | 435 --------------------------------------- lite-variables.tf | 45 ++-- lite.auto.tfvars.example | 10 + main.tf | 2 +- scripts/deploy | 36 ++-- 6 files changed, 66 insertions(+), 478 deletions(-) delete mode 100755 hpcc/lite-locals.tf diff --git a/aks/locals.tf b/aks/locals.tf index dad8de3..5f4a9c9 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -13,8 +13,8 @@ locals { node_type_version = "v2" node_size = var.aks_node_sizes.roxie single_group = false - min_capacity = var.aks_capacity.roxie_min - max_capacity = var.aks_capacity.roxie_max + min_capacity = 1 + max_capacity = 3 labels = { "lnrs.io/tier" = "standard" "workload" = "roxiepool" @@ -31,8 +31,8 @@ locals { node_type_version = "v2" # v1, v2 node_size = var.aks_node_sizes.thor single_group = false - min_capacity = var.aks_capacity.thor_min - max_capacity = var.aks_capacity.thor_max + min_capacity = 1 + max_capacity = var.aks_thorpool_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "thorpool" @@ -48,8 +48,8 @@ locals { node_type_version = "v1" node_size = var.aks_node_sizes.serv single_group = false - min_capacity = var.aks_capacity.serv_min - max_capacity = var.aks_capacity.serv_max + min_capacity = 1 + max_capacity = 3 labels = { "lnrs.io/tier" = "standard" "workload" = "servpool" @@ -65,8 +65,8 @@ locals { node_type_version = "v1" node_size = var.aks_node_sizes.spray single_group = false - min_capacity = var.aks_capacity.spray_min - max_capacity = var.aks_capacity.spray_max + min_capacity = 0 + max_capacity = 6 labels = { "lnrs.io/tier" = "standard" "workload" = "spraypool" diff --git a/hpcc/lite-locals.tf b/hpcc/lite-locals.tf deleted file mode 100755 index e9725f7..0000000 --- a/hpcc/lite-locals.tf +++ /dev/null @@ -1,435 +0,0 @@ -locals { - helm_chart_timeout=600 - - owner = { - name = var.admin_username - email = var.aks_admin_email - } - - owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - - tags = var.extra_tags - - disable_naming_conventions = false - - auto_launch_svc = { - eclwatch = false - } - - internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io - - external = {} - - admin_services_storage_account_settings = { - replication_type = "ZRS" #LRS only if using HPC Cache - authorized_ip_ranges = { - "default" = "0.0.0.0/0" //must be public IP - } - - delete_protection = false - } - - azure_log_analytics_creds = { - scope = null - object_id = "" //AAD_PRINCIPAL_ID - } - - data_storage_config = { - internal = { - blob_nfs = { - data_plane_count = 2 - storage_account_settings = { - replication_type = "ZRS" - delete_protection = false - } - } - } - external = null - } - - - spill_volumes = { - spill = { - name = "spill" - size = 300 - prefix = "/var/lib/HPCCSystems/spill" - host_path = "/mnt" - storage_class = "spill" - access_mode = "ReadWriteOnce" - } - } - - spray_service_settings = { - replicas = 6 - nodeSelector = "spraypool" - } - - roxie_internal_service = { - name = "iroxie" - servicePort = 9876 - listenQueue = 200 - numThreads = 30 - visibility = "local" - annotations = {} - } - - roxie_services = [local.roxie_internal_service] - - #======================================== - # defaults in godji original variables.tf - expose_services = false - - auto_connect = false - - use_existing_vnet = null - - hpcc_enabled = true - - helm_chart_strings_overrides = [] - - helm_chart_files_overrides = [] - - vault_config = null - - hpcc_container = null - - hpcc_container_registry_auth = null - - roxie_config = [ - { - disabled = (var.aks_enable_roxie == true)? false : true - name = "roxie" - nodeSelector = { workload = "roxiepool" } - numChannels = 1 - prefix = "roxie" - replicas = 2 - serverReplicas = 0 - acePoolSize = 6 - actResetLogPeriod = 0 - affinity = 0 - allFilesDynamic = false - blindLogging = false - blobCacheMem = 0 - callbackRetries = 3 - callbackTimeout = 500 - checkCompleted = true - checkFileDate = false - checkPrimaries = true - clusterWidth = 1 - copyResources = true - coresPerQuery = 0 - crcResources = false - dafilesrvLookupTimeout = 10000 - debugPermitted = true - defaultConcatPreload = 0 - defaultFetchPreload = 0 - defaultFullKeyedJoinPreload = 0 - defaultHighPriorityTimeLimit = 0 - defaultHighPriorityTimeWarning = 30000 - defaultKeyedJoinPreload = 0 - defaultLowPriorityTimeLimit = 0 - defaultLowPriorityTimeWarning = 90000 - defaultMemoryLimit = 1073741824 - defaultParallelJoinPreload = 0 - defaultPrefetchProjectPreload = 10 - defaultSLAPriorityTimeLimit = 0 - defaultSLAPriorityTimeWarning = 30000 - defaultStripLeadingWhitespace = false - diskReadBufferSize = 65536 - doIbytiDelay = true - egress = "engineEgress" - enableHeartBeat = false - enableKeyDiff = false - enableSysLog = false - fastLaneQueue = true - fieldTranslationEnabled = "payload" - flushJHtreeCacheOnOOM = true - forceStdLog = false - highTimeout = 2000 - ignoreMissingFiles = false - indexReadChunkSize = 60000 - initIbytiDelay = 10 - jumboFrames = false - lazyOpen = true - leafCacheMem = 500 - linuxYield = false - localFilesExpire = 1 - localSlave = false - logFullQueries = false - logQueueDrop = 32 - logQueueLen = 512 - lowTimeout = 10000 - maxBlockSize = 1000000000 - maxHttpConnectionRequests = 1 - maxLocalFilesOpen = 4000 - maxLockAttempts = 5 - maxRemoteFilesOpen = 100 - memTraceLevel = 1 - memTraceSizeLimit = 0 - memoryStatsInterval = 60 - minFreeDiskSpace = 6442450944 - minIbytiDelay = 2 - minLocalFilesOpen = 2000 - minRemoteFilesOpen = 50 - miscDebugTraceLevel = 0 - monitorDaliFileServer = false - nodeCacheMem = 1000 - nodeCachePreload = false - parallelAggregate = 0 - parallelLoadQueries = 1 - perChannelFlowLimit = 50 - pingInterval = 0 - preabortIndexReadsThreshold = 100 - preabortKeyedJoinsThreshold = 100 - preloadOnceData = true - prestartSlaveThreads = false - remoteFilesExpire = 3600 - roxieMulticastEnabled = false - serverSideCacheSize = 0 - serverThreads = 100 - simpleLocalKeyedJoins = true - sinkMode = "sequential" - slaTimeout = 2000 - slaveConfig = "simple" - slaveThreads = 30 - soapTraceLevel = 1 - socketCheckInterval = 5000 - statsExpiryTime = 3600 - systemMonitorInterval = 60000 - totalMemoryLimit = "5368709120" - traceLevel = 1 - traceRemoteFiles = false - trapTooManyActiveQueries = true - udpAdjustThreadPriorities = true - udpFlowAckTimeout = 10 - udpFlowSocketsSize = 33554432 - udpInlineCollation = true - udpInlineCollationPacketLimit = 50 - udpLocalWriteSocketSize = 16777216 - udpMaxPermitDeadTimeouts = 100 - udpMaxRetryTimedoutReqs = 10 - udpMaxSlotsPerClient = 100 - udpMulticastBufferSize = 33554432 - udpOutQsPriority = 5 - udpQueueSize = 1000 - udpRecvFlowTimeout = 2000 - udpRequestToSendAckTimeout = 500 - udpResendTimeout = 100 - udpRequestToSendTimeout = 2000 - udpResendEnabled = true - udpRetryBusySenders = 0 - udpSendCompletedInData = false - udpSendQueueSize = 500 - udpSnifferEnabled = false - udpTraceLevel = 0 - useAeron = false - useDynamicServers = false - useHardLink = false - useLogQueue = true - useMemoryMappedIndexes = false - useRemoteResources = false - useTreeCopy = false - services = local.roxie_services - topoServer = { - replicas = 1 - } - channelResources = { - cpu = "1" - memory = "4G" - } - } - ] - - eclagent_settings = { - hthor = { - replicas = 1 - maxActive = 4 - prefix = "hthor" - use_child_process = false - type = "hthor" - spillPlane = "spill" - resources = { - cpu = "2" - memory = "4G" - } - nodeSelector = { workload = "servpool" } - egress = "engineEgress" - cost = { - perCpu = 1 - } - }, - } - - eclccserver_settings = { - "myeclccserver" = { - useChildProcesses = false - maxActive = 4 - egress = "engineEgress" - replicas = 1 - childProcessTimeLimit = 10 - resources = { - cpu = "1" - memory = "4G" - } - nodeSelector = { workload = "servpool" } - legacySyntax = false - options = [] - cost = { - perCpu = 1 - } - } } - - dali_settings = { - coalescer = { - interval = 24 - at = "* * * * *" - minDeltaSize = 50000 - nodeSelector = { workload = "servpool" } - resources = { - cpu = "1" - memory = "4G" - } - } - resources = { - cpu = "2" - memory = "8G" - } - maxStartupTime = 1200 - } - - dfuserver_settings = { - maxJobs = 3 - nodeSelector = { workload = "servpool" } - resources = { - cpu = "1" - memory = "2G" - } - } - - sasha_config = { - disabled = false - nodeSelector = { workload = "servpool" } - wu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 6 - limit = 400 - cutoff = 3 - backup = 0 - at = "* * * * *" - throttle = 0 - retryinterval = 6 - keepResultFiles = false - } - - dfuwu-archiver = { - disabled = false - service = { - servicePort = 8877 - } - plane = "sasha" - interval = 24 - limit = 100 - cutoff = 14 - at = "* * * * *" - throttle = 0 - } - - dfurecovery-archiver = { - disabled = false - interval = 12 - limit = 20 - cutoff = 4 - at = "* * * * *" - } - - file-expiry = { - disabled = false - interval = 1 - at = "* * * * *" - persistExpiryDefault = 7 - expiryDefault = 4 - user = "sasha" - } - } - - ldap_config = null - - ldap_tunables = { - cacheTimeout = 5 - checkScopeScans = false - ldapTimeoutSecs = 131 - maxConnections = 10 - passwordExpirationWarningDays = 10 - sharedCache = true - } - - install_blob_csi_driver = true - - remote_storage_plane = null - - onprem_lz_settings = {} - - admin_services_node_selector = {} - - thor_config = [{ - disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true - name = "thor" - prefix = "thor" - numWorkers = var.thor_num_workers - keepJobs = "none" - maxJobs = var.thor_max_jobs - maxGraphs = 2 - maxGraphStartupTime = 172800 - numWorkersPerPod = 1 - nodeSelector = { workload = "thorpool" } - egress = "engineEgress" - tolerations_value = "thorpool" - managerResources = { - cpu = 2 - memory = "4G" - } - workerResources = { - cpu = 4 - memory = "4G" - } - workerMemory = { - query = "3G" - thirdParty = "500M" - } - eclAgentResources = { - cpu = 1 - memory = "2G" - } - cost = { - perCpu = 1 - } - }] - - admin_services_storage = { - dali = { - size = 100 - type = "azurefiles" - } - debug = { - size = 100 - type = "blobnfs" - } - dll = { - size = 100 - type = "blobnfs" - } - lz = { - size = var.storage_lz_gb - type = "blobnfs" - } - sasha = { - size = 100 - type = "blobnfs" - } - } - #======================================== -} diff --git a/lite-variables.tf b/lite-variables.tf index 021c32f..b4a1970 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -58,31 +58,24 @@ variable "aks_node_sizes" { spray = optional(string, "large") thor = optional(string, "xlarge") }) -} + validation { + condition = (length(regexall("^[24]*x*large", var.aks_node_sizes.roxie)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.serv)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.spray)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.thor)) == 1) -variable "aks_capacity" { - description = "The min and max number of nodes of each node pool in the HPCC Systems. Example format is '{ roxie_min = 1, roxie_max = 3, serv_min = 1, serv_max = 3, spray_min = 1, spray_max = 3, thor_min = 1, thor_max = 3}'." - type = object({ - roxie_min = optional(number, 0) - roxie_max = optional(number, 3) - serv_min = optional(number, 1) - serv_max = optional(number, 3) - spray_min = optional(number, 0) - spray_max = optional(number, 6) - thor_min = optional(number, 0) - thor_max = optional(number, 6) - }) + error_message = "All aks_node_sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." + } default = { - roxie_min = 0 - roxie_max = 3 - serv_min = 1 - serv_max = 3 - spray_min = 0 - spray_max = 6 - thor_min = 0 - thor_max = 6 + roxie = "xlarge" + serv = "2xlarge" + spray = "large" + thor = "xlarge" } } + +variable "aks_thorpool_max_capacity" { + type = number + description = "The max capacity (or node count) of the thorpool. This is calculated and an argument for terraform plan and appy for aks." + default = 2 +} #===== end of aks variables ===== variable "my_azure_id" { @@ -185,6 +178,16 @@ variable "thor_num_workers" { default = 2 } +variable "thor_worker_cpus" { + type = number + description = "The number of CPUs each Thor worker should have.\nMust be 2, 4, 8 or 16. Also, this should be less than 'cpu' in thor node_size." + validation { + condition = ((var.thor_worker_cpus == 2) || (var.thor_worker_cpus == 4) || (var.thor_worker_cpus == 8) || (var.thor_worker_cpus == 16)) + error_message = "Value must be 2, 4, 8, or 16." + } + default = 2 +} + ############################################################################### # Optional variables ############################################################################### diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 215301f..733b069 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -87,6 +87,16 @@ thor_max_jobs=2 #------------------------------------------------------------------------------ +# The number of CPUs each Thor worker should have. +# Must be 2, 4, 8 or 16. Also, this should be less than 'cpu' in thor node_size. +# REQUIRED +# Value type: number +# Updateable: N + +thor_worker_cpus=2 + +#------------------------------------------------------------------------------ + # The amount of storage reserved for the landing zone in gigabytes. # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. diff --git a/main.tf b/main.tf index 16fba90..6bfef65 100644 --- a/main.tf +++ b/main.tf @@ -8,7 +8,7 @@ resource "null_resource" "deploy_vnet" { resource "null_resource" "deploy_aks" { provisioner "local-exec" { - command = "scripts/deploy aks ${var.my_azure_id}" + command = "scripts/deploy aks ${var.my_azure_id} ${local.thorpool_max_capacity}" } depends_on = [ null_resource.deploy_vnet ] diff --git a/scripts/deploy b/scripts/deploy index afa13bb..816929f 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -3,18 +3,28 @@ thisdir=$(pwd)/$(dirname $0) repodir=$(echo $thisdir|sed "s/\/scripts\/*//") if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then name=$1 + if [ "$name" == "aks" ];then + if [ "$2" != "" ];then + my_azure_id=$2 + else + echo "ERROR: name=\"$name\". Need a 2nd argument (my_azure_id) but none given.";exit 1; + fi + fi + if [ "$name" == "aks" ];then + if [ "$3" != "" ];then + thorpool_max_capacity=$3 + else + echo "ERROR: name=\"$name\". Need a 3rd argument (thorpool max capacity) but none given.";exit 1; + fi + fi else echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'storage' or 'aks' or 'hpcc'. EXITING.";exit 1; fi if [ "$1" == "aks" ];then - if [ "$2" != "" ];then - my_azure_id=$2 - cp -v $thisdir/needed-auto-tfvars-files/aks/aks.auto.tfvars.example $repodir/aks/aks.auto.tfvars - cp -v $thisdir/needed-auto-tfvars-files/aks/misc.auto.tfvars.example $repodir/aks/misc.auto.tfvars - sed -i "s//$my_azure_id/" $repodir/aks/aks.auto.tfvars - else - echo "In $0. First argument was \"$name\". There should be a 2nd argument (my_azure_id). But it was missing. EXITING.";exit 1; - fi + my_azure_id=$2 + cp -v $thisdir/needed-auto-tfvars-files/aks/aks.auto.tfvars.example $repodir/aks/aks.auto.tfvars + cp -v $thisdir/needed-auto-tfvars-files/aks/misc.auto.tfvars.example $repodir/aks/misc.auto.tfvars + sed -i "s//$my_azure_id/" $repodir/aks/aks.auto.tfvars fi #======================================================================== function assert_fail () { @@ -85,14 +95,18 @@ else echo "In $0. The root directory does not have a file called 'lite.aute.tfvars'. It must. EXITING";exit 1; fi if [ "$name" == "hpcc" ];then - echo "Coping root's lite.auto.tfvars to /tmp and $name directory." + echo "Coping root's lite.auto.tfvars to /tmp and $name directory. Also, coping root's lite-locals.tf and lite-variables.tf to $name directory." cp -v ../lite.auto.tfvars /tmp/${name}.lite.auto.tfvars cp -v ../lite.auto.tfvars . + cp -v ../lite-locals.tf . cp -v ../lite-variables.tf . elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];then egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf + if [ "$name" == "aks" ];then + echo "aks_thorpool_max_capacity=$thorpool_max_capacity" >> lite.auto.tfvars + fi if [ "$name" == "storage" ];then cp -v $thisdir/needed-auto-tfvars-files/storage/storage.auto.tfvars.example ./storage.auto.tfvars fi @@ -106,10 +120,6 @@ plan=$($thisdir/mkplan deploy_${name}.plan) if [ "$tfvarsdiff" == "" ] && [ -d "data" ] && [ -f "data/config.json" ]; then echo "Complete! $name is already deployed";exit 0; fi -#if [ "$name" != "vnet" ];then -# cp -v /tmp/${name}.lite.auto.tfvars . -#fi - echo "=============== Deploy $name. Executing 'terraform init' ==============="; assert_fail terraform init echo "=============== Deploy $name. Executing 'terraform plan -out=$plan' ==============="; From 5c33a803757363fa5746d1c52d0d9c956a16b934 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 12 Dec 2023 14:25:56 +0000 Subject: [PATCH 088/124] Calculates max capacity of thorpool. Set thor cpu and ram. Added to root, lite-locals.tf --- lite-locals.tf | 598 +++++++++++++++++++++++++++++++++++++++++++++++++ scripts/deploy | 4 +- 2 files changed, 601 insertions(+), 1 deletion(-) create mode 100644 lite-locals.tf diff --git a/lite-locals.tf b/lite-locals.tf new file mode 100644 index 0000000..a6aaf2f --- /dev/null +++ b/lite-locals.tf @@ -0,0 +1,598 @@ +output "thor_max_jobs" { + value = var.thor_max_jobs +} +output "thor_num_workers" { + value = var.thor_num_workers +} +output "thor_node_size" { + value = var.aks_node_sizes.thor +} +output "thor_ns_spec" { + value = local.ns_spec[var.aks_node_sizes.thor] +} +output "thor_worker_cpus" { + value = var.thor_worker_cpus +} +output "thorWorkersPerNode" { + value = "local.ns_spec[${var.aks_node_sizes.thor}].cpu / var.thor_worker_cpus = ${local.thorWorkersPerNode}" +} +output "thor_worker_ram" { + value = "local.ns_spec[${var.aks_node_sizes.thor}].ram / local.thorWorkersPerNode = ${local.thor_worker_ram}" +} +output "nodesPer1Job" { + value = "var.thor_num_workers / local.thorWorkersPerNode = ${local.nodesPer1Job}" +} +output "thorpool_max_capacity" { + value = "local.nodesPer1Job * var.thor_max_jobs = ${local.thorpool_max_capacity}" +} +locals { + ns_spec = { + "large" = { + cpu = 2 + ram = 8 + } + "xlarge" = { + cpu = 4 + ram = 16 + } + "2xlarge" = { + cpu = 8 + ram = 32 + } + "4xlarge" = { + cpu = 16 + ram = 64 + } + } + + twpn = "${ local.ns_spec[var.aks_node_sizes.thor].cpu / var.thor_worker_cpus }" + thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${var.aks_node_sizes.thor}].cpu, ${local.ns_spec[var.aks_node_sizes.thor].cpu}, is not a multiple of var.thor_worker_cpus, ${var.thor_worker_cpus}." + + twr = "${local.ns_spec[var.aks_node_sizes.thor].ram / local.thorWorkersPerNode }" + thor_worker_ram = ceil(local.twr) == local.twr? local.twr : "local.thor_worker_ram, ${local.twr}, is not an integer because local.ns_spec[${var.aks_node_sizes.thor}].ram, ${local.ns_spec[var.aks_node_sizes.thor].ram}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." + + np1j = "${var.thor_num_workers / local.thorWorkersPerNode }" + nodesPer1Job = ceil(local.np1j) == local.np1j? local.np1j : "local.nodesPer1Job, ${local.np1j}, is not an integer because var.thor_num_workers, ${var.thor_num_workers}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." + + thorpool_max_capacity = ceil("${ local.nodesPer1Job * var.thor_max_jobs }") + + helm_chart_timeout=300 + #hpcc_version = "8.6.20" + + owner = { + name = var.admin_username + email = var.aks_admin_email + } + + owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) + + /*metadata = { + project = format("%shpccplatform", local.owner_name_initials) + product_name = format("%shpccplatform", local.owner_name_initials) + business_unit = "commercial" + environment = "sandbox" + market = "us" + product_group = format("%shpcc", local.owner_name_initials) + resource_group_type = "app" + sre_team = format("%shpccplatform", local.owner_name_initials) + subscription_type = "dev" + additional_tags = { "justification" = "testing" } + location = var.aks_azure_region # Acceptable values: eastus, centralus + } + + tags = merge(local.metadata.additional_tags, var.extra_tags) + */ + + # # disable_naming_conventions - Disable naming conventions + # # disable_naming_conventions = true + disable_naming_conventions = false + + # # auto_launch_eclwatch - Automatically launch ECLWatch web interface. + #auto_launch_eclwatch = true + auto_launch_svc = { + eclwatch = false + } + + # azure_auth = { + # # AAD_CLIENT_ID = "" + # # AAD_CLIENT_SECRET = "" + # # AAD_TENANT_ID = "" + # # AAD_PRINCIPAL_ID = "" + # SUBSCRIPTION_ID = "" + # } + + # hpcc_container = { + # version = "9.2.0" + # image_name = "platform-core-ln" + # image_root = "jfrog.com/glb-docker-virtual" + # # custom_chart_version = "9.2.0-rc1" + # # custom_image_version = "9.2.0-demo" + # } + + # hpcc_container_registry_auth = { + # username = "value" + # password = "value" + # } + + internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io + + external = {} + # external = { + # blob_nfs = [{ + # container_id = "" + # container_name = "" + # id = "" + # resource_group_name = var.storage_account_resource_group_name + # storage_account_id = "" + # storage_account_name = var.storage_account_name + # }] + # # hpc_cache = [{ + # # id = "" + # # path = "" + # # server = "" + # }] + # hpcc = [{ + # name = "" + # planes = list(object({ + # local = "" + # remote = "" + # })) + # service = "" + # }] + # } + + admin_services_storage_account_settings = { + replication_type = "ZRS" #LRS only if using HPC Cache + authorized_ip_ranges = { + "default" = "0.0.0.0/0" //must be public IP + } + + delete_protection = false + } + + azure_log_analytics_creds = { + scope = null + object_id = "" //AAD_PRINCIPAL_ID + } + + data_storage_config = { + internal = { + blob_nfs = { + data_plane_count = 2 + storage_account_settings = { + replication_type = "ZRS" + delete_protection = false + } + } + # hpc_cache = { + # enabled = false + # size = "small" + # cache_update_frequency = "3h" + # storage_account_data_planes = null + # } + } + external = null + } + + + spill_volumes = { + spill = { + name = "spill" + size = 300 + prefix = "/var/lib/HPCCSystems/spill" + host_path = "/mnt" + storage_class = "spill" + access_mode = "ReadWriteOnce" + } + } + + spray_service_settings = { + replicas = 6 + nodeSelector = "spraypool" + } + + # ldap = { + # ldap_server = "" //Server IP + # dali = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # esp = { + # hpcc_admin_password = "" + # hpcc_admin_username = "" + # ldap_admin_password = "" + # ldap_admin_username = "" + # adminGroupName = "HPCC-Admins" + # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" + # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" + # } + # } + + roxie_internal_service = { + name = "iroxie" + servicePort = 9876 + listenQueue = 200 + numThreads = 30 + visibility = "local" + annotations = {} + } + + roxie_services = [local.roxie_internal_service] + + #======================================== + # defaults in godji original variables.tf + expose_services = false + + auto_connect = false + + use_existing_vnet = null + + hpcc_enabled = true + + helm_chart_strings_overrides = [] + + helm_chart_files_overrides = [] + + vault_config = null + + hpcc_container = null + + hpcc_container_registry_auth = null + + roxie_config = [ + { + disabled = (var.aks_enable_roxie == true)? false : true + name = "roxie" + nodeSelector = { workload = "roxiepool" } + # tlh 20231109 numChannels = 2 + numChannels = 1 + prefix = "roxie" + replicas = 2 + serverReplicas = 0 + acePoolSize = 6 + actResetLogPeriod = 0 + affinity = 0 + allFilesDynamic = false + blindLogging = false + blobCacheMem = 0 + callbackRetries = 3 + callbackTimeout = 500 + checkCompleted = true + checkFileDate = false + checkPrimaries = true + clusterWidth = 1 + copyResources = true + coresPerQuery = 0 + crcResources = false + dafilesrvLookupTimeout = 10000 + debugPermitted = true + defaultConcatPreload = 0 + defaultFetchPreload = 0 + defaultFullKeyedJoinPreload = 0 + defaultHighPriorityTimeLimit = 0 + defaultHighPriorityTimeWarning = 30000 + defaultKeyedJoinPreload = 0 + defaultLowPriorityTimeLimit = 0 + defaultLowPriorityTimeWarning = 90000 + defaultMemoryLimit = 1073741824 + defaultParallelJoinPreload = 0 + defaultPrefetchProjectPreload = 10 + defaultSLAPriorityTimeLimit = 0 + defaultSLAPriorityTimeWarning = 30000 + defaultStripLeadingWhitespace = false + diskReadBufferSize = 65536 + doIbytiDelay = true + egress = "engineEgress" + enableHeartBeat = false + enableKeyDiff = false + enableSysLog = false + fastLaneQueue = true + fieldTranslationEnabled = "payload" + flushJHtreeCacheOnOOM = true + forceStdLog = false + highTimeout = 2000 + ignoreMissingFiles = false + indexReadChunkSize = 60000 + initIbytiDelay = 10 + jumboFrames = false + lazyOpen = true + leafCacheMem = 500 + linuxYield = false + localFilesExpire = 1 + localSlave = false + logFullQueries = false + logQueueDrop = 32 + logQueueLen = 512 + lowTimeout = 10000 + maxBlockSize = 1000000000 + maxHttpConnectionRequests = 1 + maxLocalFilesOpen = 4000 + maxLockAttempts = 5 + maxRemoteFilesOpen = 100 + memTraceLevel = 1 + memTraceSizeLimit = 0 + memoryStatsInterval = 60 + minFreeDiskSpace = 6442450944 + minIbytiDelay = 2 + minLocalFilesOpen = 2000 + minRemoteFilesOpen = 50 + miscDebugTraceLevel = 0 + monitorDaliFileServer = false + nodeCacheMem = 1000 + nodeCachePreload = false + parallelAggregate = 0 + parallelLoadQueries = 1 + perChannelFlowLimit = 50 + pingInterval = 0 + preabortIndexReadsThreshold = 100 + preabortKeyedJoinsThreshold = 100 + preloadOnceData = true + prestartSlaveThreads = false + remoteFilesExpire = 3600 + roxieMulticastEnabled = false + serverSideCacheSize = 0 + serverThreads = 100 + simpleLocalKeyedJoins = true + sinkMode = "sequential" + slaTimeout = 2000 + slaveConfig = "simple" + slaveThreads = 30 + soapTraceLevel = 1 + socketCheckInterval = 5000 + statsExpiryTime = 3600 + systemMonitorInterval = 60000 + totalMemoryLimit = "5368709120" + traceLevel = 1 + traceRemoteFiles = false + trapTooManyActiveQueries = true + udpAdjustThreadPriorities = true + udpFlowAckTimeout = 10 + udpFlowSocketsSize = 33554432 + udpInlineCollation = true + udpInlineCollationPacketLimit = 50 + udpLocalWriteSocketSize = 16777216 + udpMaxPermitDeadTimeouts = 100 + udpMaxRetryTimedoutReqs = 10 + udpMaxSlotsPerClient = 100 + udpMulticastBufferSize = 33554432 + udpOutQsPriority = 5 + udpQueueSize = 1000 + udpRecvFlowTimeout = 2000 + udpRequestToSendAckTimeout = 500 + udpResendTimeout = 100 + udpRequestToSendTimeout = 2000 + udpResendEnabled = true + udpRetryBusySenders = 0 + udpSendCompletedInData = false + udpSendQueueSize = 500 + udpSnifferEnabled = false + udpTraceLevel = 0 + useAeron = false + useDynamicServers = false + useHardLink = false + useLogQueue = true + useMemoryMappedIndexes = false + useRemoteResources = false + useTreeCopy = false + services = local.roxie_services + topoServer = { + replicas = 1 + } + channelResources = { + cpu = "1" + memory = "4G" + } + } + ] + + eclagent_settings = { + hthor = { + replicas = 1 + maxActive = 4 + prefix = "hthor" + use_child_process = false + type = "hthor" + spillPlane = "spill" + resources = { + cpu = "1" + memory = "4G" + } + nodeSelector = { workload = "servpool" } + egress = "engineEgress" + cost = { + perCpu = 1 + } + }, + } + + eclccserver_settings = { + "myeclccserver" = { + useChildProcesses = false + maxActive = 4 + egress = "engineEgress" + replicas = 1 + childProcessTimeLimit = 10 + resources = { + cpu = "1" + memory = "4G" + } + nodeSelector = { workload = "servpool" } + legacySyntax = false + options = [] + cost = { + perCpu = 1 + } + } } + + dali_settings = { + coalescer = { + interval = 24 + at = "* * * * *" + minDeltaSize = 50000 + nodeSelector = { workload = "servpool" } + resources = { + cpu = "1" + memory = "4G" + } + } + resources = { + cpu = "2" + memory = "8G" + } + maxStartupTime = 1200 + } + + dfuserver_settings = { + maxJobs = 3 + nodeSelector = { workload = "servpool" } + resources = { + cpu = "1" + memory = "2G" + } + } + + sasha_config = { + disabled = false + nodeSelector = { workload = "servpool" } + wu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 6 + limit = 400 + cutoff = 3 + backup = 0 + at = "* * * * *" + throttle = 0 + retryinterval = 6 + keepResultFiles = false + # egress = "engineEgress" + } + + dfuwu-archiver = { + disabled = false + service = { + servicePort = 8877 + } + plane = "sasha" + interval = 24 + limit = 100 + cutoff = 14 + at = "* * * * *" + throttle = 0 + # egress = "engineEgress" + } + + dfurecovery-archiver = { + disabled = false + interval = 12 + limit = 20 + cutoff = 4 + at = "* * * * *" + # egress = "engineEgress" + } + + file-expiry = { + disabled = false + interval = 1 + at = "* * * * *" + persistExpiryDefault = 7 + expiryDefault = 4 + user = "sasha" + # egress = "engineEgress" + } + } + + ldap_config = null + + ldap_tunables = { + cacheTimeout = 5 + checkScopeScans = false + ldapTimeoutSecs = 131 + maxConnections = 10 + passwordExpirationWarningDays = 10 + sharedCache = true + } + + install_blob_csi_driver = true + + remote_storage_plane = null + + onprem_lz_settings = {} + + admin_services_node_selector = {} + + thor_config = [{ + disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true + name = "thor" + prefix = "thor" + numWorkers = var.thor_num_workers + keepJobs = "none" + maxJobs = var.thor_max_jobs + maxGraphs = 2 + maxGraphStartupTime = 172800 + numWorkersPerPod = 1 + #nodeSelector = {} + nodeSelector = { workload = "thorpool" } + egress = "engineEgress" + tolerations_value = "thorpool" + managerResources = { + cpu = 1 + memory = "2G" + } + workerResources = { + cpu = 3 + memory = "4G" + } + workerMemory = { + query = "3G" + thirdParty = "500M" + } + eclAgentResources = { + cpu = 1 + memory = "2G" + } + cost = { + perCpu = 1 + } + }] + + admin_services_storage = { + dali = { + size = 100 + type = "azurefiles" + } + debug = { + size = 100 + type = "blobnfs" + } + dll = { + size = 100 + type = "blobnfs" + } + lz = { + size = var.storage_lz_gb + type = "blobnfs" + } + sasha = { + size = 100 + type = "blobnfs" + } + } + #======================================== +} diff --git a/scripts/deploy b/scripts/deploy index 816929f..a1caa64 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -5,7 +5,8 @@ if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then name=$1 if [ "$name" == "aks" ];then if [ "$2" != "" ];then - my_azure_id=$2 + my_azure_id=$2 + echo "Second argument, i.e. \$2(or my_azure_id)=\"$2\"" else echo "ERROR: name=\"$name\". Need a 2nd argument (my_azure_id) but none given.";exit 1; fi @@ -13,6 +14,7 @@ if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then if [ "$name" == "aks" ];then if [ "$3" != "" ];then thorpool_max_capacity=$3 + #echo "DEBUG: name=\"$name\", \$3(or thorpool_max_capacity)=\"$3\"";exit else echo "ERROR: name=\"$name\". Need a 3rd argument (thorpool max capacity) but none given.";exit 1; fi From 8dd9ebfa784b403d6141f31962368d06413171cb Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 12 Dec 2023 15:25:31 +0000 Subject: [PATCH 089/124] In both hpcc/hpcc.tf and aks/aks.tf, changed source statements so value begins with 'https://' instead of 'git@'. --- aks/aks.tf | 3 ++- hpcc/hpcc.tf | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 9f90c94..63e0a5c 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,6 +1,7 @@ module "aks" { depends_on = [random_string.string] - source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" + #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" + source = "https://github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 81b1839..f9d1091 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,5 +1,6 @@ module "hpcc" { - source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" + #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" + source = "https://github.com/hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" environment = local.metadata.environment productname = local.metadata.product_name From a00e254ee0cb73aa5552b7ddd1ada0f1a8edcb7d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 12 Dec 2023 15:46:06 +0000 Subject: [PATCH 090/124] In both hpcc/hpcc.tf and aks/aks.tf, removed prefix 'git@' from source statement and changed ':' to '/'. --- aks/aks.tf | 2 +- hpcc/hpcc.tf | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index 63e0a5c..bc4b51f 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,7 +1,7 @@ module "aks" { depends_on = [random_string.string] #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" - source = "https://github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" + source = "github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index f9d1091..08a2c08 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,6 +1,6 @@ module "hpcc" { #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" - source = "https://github.com/hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" + source = "github.com/hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" environment = local.metadata.environment productname = local.metadata.product_name From 9f48ecef7593f42dcf00e8fbeef047a018145060 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 12 Dec 2023 20:28:37 +0000 Subject: [PATCH 091/124] Changed variable 'aks_node_sizes' to individual string variables: roxie_node_size, serv_node_size, spray_node_size, and thor_node_size. --- README.md | 6 +++- aks/aks.tf | 7 +++-- aks/locals.tf | 8 +++--- lite-locals.tf | 23 +++++++++------ lite-variables.tf | 61 +++++++++++++++++++++++++++------------- lite.auto.tfvars.example | 34 +++++++++++++++++++--- 6 files changed, 100 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index 2bb0daf..a33ea1b 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. +* If you run the terraform on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. ## Installing/Using This Module @@ -96,7 +97,10 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | | `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | -| `aks_node_sizes ` | map of string | The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = "xlarge", serv = "2xlarge", spray = "xlarge", thor = "xlarge" }'. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | N | +| `aks_roxie_node_size ` | string | The VM size for each roxie node in the HPCC Systems. Example format `aks_roxie_node-size`="xlarge".| N | +| `aks_serv_node_size ` | string | The VM size for each serv node in the HPCC Systems. Example format `aks_serv_node-size`="2xlarge".| N | +| `aks_spray_node_size ` | string | The VM size for each spray node in the HPCC Systems. Example format `aks_spray_node-size`="2xlarge".| N | +| `aks_thor_node_size ` | string | The VM size for each thor node in the HPCC Systems. Example format `aks_thor_node-size`="2xlarge".| N | | `aks_capacity ` | map of number | The min and max number of nodes of each node pool in the HPCC Systems. Example format is '{ roxie_min = 1, roxie_max = 3, serv_min = 1, serv_max = 3, spray_min = 1, spray_max = 3, thor_min = 1, thor_max = 3}'. | N | | `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | Y | | `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | Y | diff --git a/aks/aks.tf b/aks/aks.tf index bc4b51f..f8f1431 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,8 +1,8 @@ module "aks" { depends_on = [random_string.string] #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" - source = "github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" - #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + #source = "github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" + source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default @@ -30,7 +30,8 @@ module "aks" { dns_resource_group_lookup = { "${local.internal_domain}" = local.dns_resource_group } - admin_group_object_ids = [data.azuread_group.subscription_owner.object_id] + #admin_group_object_ids = [data.azuread_group.subscription_owner.object_id] + admin_group_object_ids = null rbac_bindings = var.rbac_bindings diff --git a/aks/locals.tf b/aks/locals.tf index 5f4a9c9..fab327c 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -11,7 +11,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v2" - node_size = var.aks_node_sizes.roxie + node_size = var.aks_roxie_node_size single_group = false min_capacity = 1 max_capacity = 3 @@ -29,7 +29,7 @@ locals { node_os = "ubuntu" node_type = "gp" # gp, gpd, mem, memd, stor node_type_version = "v2" # v1, v2 - node_size = var.aks_node_sizes.thor + node_size = var.aks_thor_node_size single_group = false min_capacity = 1 max_capacity = var.aks_thorpool_max_capacity @@ -46,7 +46,7 @@ locals { node_os = "ubuntu" node_type = "gpd" node_type_version = "v1" - node_size = var.aks_node_sizes.serv + node_size = var.aks_serv_node_size single_group = false min_capacity = 1 max_capacity = 3 @@ -63,7 +63,7 @@ locals { node_os = "ubuntu" node_type = "gp" node_type_version = "v1" - node_size = var.aks_node_sizes.spray + node_size = var.aks_spray_node_size single_group = false min_capacity = 0 max_capacity = 6 diff --git a/lite-locals.tf b/lite-locals.tf index a6aaf2f..97ffdb4 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -5,19 +5,19 @@ output "thor_num_workers" { value = var.thor_num_workers } output "thor_node_size" { - value = var.aks_node_sizes.thor + value = local.aks_node_sizes.thor } output "thor_ns_spec" { - value = local.ns_spec[var.aks_node_sizes.thor] + value = local.ns_spec[local.aks_node_sizes.thor] } output "thor_worker_cpus" { value = var.thor_worker_cpus } output "thorWorkersPerNode" { - value = "local.ns_spec[${var.aks_node_sizes.thor}].cpu / var.thor_worker_cpus = ${local.thorWorkersPerNode}" + value = "local.ns_spec[${local.aks_node_sizes.thor}].cpu / var.thor_worker_cpus = ${local.thorWorkersPerNode}" } output "thor_worker_ram" { - value = "local.ns_spec[${var.aks_node_sizes.thor}].ram / local.thorWorkersPerNode = ${local.thor_worker_ram}" + value = "local.ns_spec[${local.aks_node_sizes.thor}].ram / local.thorWorkersPerNode = ${local.thor_worker_ram}" } output "nodesPer1Job" { value = "var.thor_num_workers / local.thorWorkersPerNode = ${local.nodesPer1Job}" @@ -26,6 +26,13 @@ output "thorpool_max_capacity" { value = "local.nodesPer1Job * var.thor_max_jobs = ${local.thorpool_max_capacity}" } locals { + aks_node_sizes = { + roxie = var.aks_roxie_node_size + serv = var.aks_serv_node_size + spray = var.aks_spray_node_size + thor = var.aks_thor_node_size + } + ns_spec = { "large" = { cpu = 2 @@ -45,11 +52,11 @@ locals { } } - twpn = "${ local.ns_spec[var.aks_node_sizes.thor].cpu / var.thor_worker_cpus }" - thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${var.aks_node_sizes.thor}].cpu, ${local.ns_spec[var.aks_node_sizes.thor].cpu}, is not a multiple of var.thor_worker_cpus, ${var.thor_worker_cpus}." + twpn = "${ local.ns_spec[local.aks_node_sizes.thor].cpu / var.thor_worker_cpus }" + thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].cpu, ${local.ns_spec[local.aks_node_sizes.thor].cpu}, is not a multiple of var.thor_worker_cpus, ${var.thor_worker_cpus}." - twr = "${local.ns_spec[var.aks_node_sizes.thor].ram / local.thorWorkersPerNode }" - thor_worker_ram = ceil(local.twr) == local.twr? local.twr : "local.thor_worker_ram, ${local.twr}, is not an integer because local.ns_spec[${var.aks_node_sizes.thor}].ram, ${local.ns_spec[var.aks_node_sizes.thor].ram}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." + twr = "${local.ns_spec[local.aks_node_sizes.thor].ram / local.thorWorkersPerNode }" + thor_worker_ram = ceil(local.twr) == local.twr? local.twr : "local.thor_worker_ram, ${local.twr}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].ram, ${local.ns_spec[local.aks_node_sizes.thor].ram}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." np1j = "${var.thor_num_workers / local.thorWorkersPerNode }" nodesPer1Job = ceil(local.np1j) == local.np1j? local.np1j : "local.nodesPer1Job, ${local.np1j}, is not an integer because var.thor_num_workers, ${var.thor_num_workers}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." diff --git a/lite-variables.tf b/lite-variables.tf index b4a1970..ee5c262 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -50,31 +50,54 @@ variable "aks_admin_ip_cidr_map" { default = {} } -variable "aks_node_sizes" { - description = "The VM size for each node of each node pool in the HPCC Systems. Example format is '{ roxie = \"xlarge\", serv = \"2xlarge\", spray = \"xlarge\", thor = \"xlarge\" }'.\nSee https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information." - type = object({ - roxie = optional(string, "xlarge") - serv = optional(string, "2xlarge") - spray = optional(string, "large") - thor = optional(string, "xlarge") - }) +variable "aks_thorpool_max_capacity" { + type = number + description = "The max capacity (or node count) of the thorpool. This is calculated and an argument for terraform plan and appy for aks." + default = 2 +} + +variable "aks_roxie_node_size" { + type = string + description = "The size of the roxie nodes. Possibilities are 'large', 'xlarge', '2xlarge', and '4xlarge'." + validation { + condition = (length(regexall("^[24]*x*large", var.aks_roxie_node_size)) == 1) + + error_message = "All node sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." + } + default = "xlarge" +} + +variable "aks_serv_node_size" { + type = string + description = "The size of the serv nodes. Possibilities are 'large', 'xlarge', '2xlarge', and '4xlarge'." validation { - condition = (length(regexall("^[24]*x*large", var.aks_node_sizes.roxie)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.serv)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.spray)) == 1) && (length(regexall("^[24]*x*large", var.aks_node_sizes.thor)) == 1) + condition = (length(regexall("^[24]*x*large", var.aks_serv_node_size)) == 1) - error_message = "All aks_node_sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." + error_message = "All node sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." } - default = { - roxie = "xlarge" - serv = "2xlarge" - spray = "large" - thor = "xlarge" + default = "2xlarge" +} + +variable "aks_spray_node_size" { + type = string + description = "The size of the spray nodes. Possibilities are 'large', 'xlarge', '2xlarge', and '4xlarge'." + validation { + condition = (length(regexall("^[24]*x*large", var.aks_spray_node_size)) == 1) + + error_message = "All node sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." } + default = "large" } -variable "aks_thorpool_max_capacity" { - type = number - description = "The max capacity (or node count) of the thorpool. This is calculated and an argument for terraform plan and appy for aks." - default = 2 +variable "aks_thor_node_size" { + type = string + description = "The size of the thor nodes. Possibilities are 'large', 'xlarge', '2xlarge', and '4xlarge'." + validation { + condition = (length(regexall("^[24]*x*large", var.aks_thor_node_size)) == 1) + + error_message = "All node sizes must be one of the following: large, xlarge, 2xlarge, or 4xlarge." + } + default = "xlarge" } #===== end of aks variables ===== diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 733b069..58d12cb 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -128,13 +128,39 @@ extra_tags={} #------------------------------------------------------------------------------ -# The VM size for each node in the HPCC Systems node pool. -# Recommend "Standard_B4ms" or better. -# See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. +# The VM size for each roxie node in the HPCC Systems node pool. +# Possible values are: large, xlarge, 2xlarge, and 4xlarge # Value type: string # Updateable: N -aks_node_sizes = { roxie = "xlarge", serv = "2xlarge", spray = "large", thor = "xlarge" } +aks_roxie_node_size = "xlarge" + +#------------------------------------------------------------------------------ + +# The VM size for each serv node in the HPCC Systems node pool. +# Possible values are: large, xlarge, 2xlarge, and 4xlarge +# Value type: string +# Updateable: N + +aks_serv_node_size = "2xlarge" + +#------------------------------------------------------------------------------ + +# The VM size for each spray node in the HPCC Systems node pool. +# Possible values are: large, xlarge, 2xlarge, and 4xlarge +# Value type: string +# Updateable: N + +aks_spray_node_size = "large" + +#------------------------------------------------------------------------------ + +# The VM size for each thor node in the HPCC Systems node pool. +# Possible values are: large, xlarge, 2xlarge, and 4xlarge +# Value type: string +# Updateable: N + +aks_thor_node_size = "xlarge" #------------------------------------------------------------------------------ From f4847f520476370bcd5a419c09117da7d1e8a572 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 12 Dec 2023 21:21:49 +0000 Subject: [PATCH 092/124] Changed 'source' in aks/aks.tf. Now it points to github repo --- aks/aks.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index f8f1431..cec915b 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -1,8 +1,8 @@ module "aks" { depends_on = [random_string.string] #source = "git@github.com:hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" - #source = "github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" - source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" + source = "github.com/hpccsystems-solutions-lab/tlh-oss-terraform-azurerm-aks.git" + #source = "/home/azureuser/temp/OSS/terraform-azurerm-aks" providers = { kubernetes = kubernetes.default From 01f09ed0099a68cbdecbb6592003f97a2ba3d7c2 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 16:45:36 +0000 Subject: [PATCH 093/124] To redone lite-locals.tf, added workerResources cpu and memory. --- lite-locals.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lite-locals.tf b/lite-locals.tf index 97ffdb4..f73510d 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -563,8 +563,8 @@ locals { memory = "2G" } workerResources = { - cpu = 3 - memory = "4G" + cpu = var.thor_worker_cpus + memory = local.thor_worker_ram } workerMemory = { query = "3G" From b30f1a598767238b3a0614cbc85b0a2a90d4c4c2 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 16:49:14 +0000 Subject: [PATCH 094/124] Removed commented-out code from hpcc/main.tf --- hpcc/main.tf | 9 --------- 1 file changed, 9 deletions(-) diff --git a/hpcc/main.tf b/hpcc/main.tf index c436605..88c9157 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -38,12 +38,3 @@ resource "null_resource" "delete_ephemeral_storage_accounts" { } depends_on = [module.hpcc] } - -/*resource "null_resource" "launch_svc_url" { - for_each = (module.hpcc.hpcc_status == "deployed") && (local.auto_launch_svc.eclwatch == true) ? local.svc_domains : {} - - provisioner "local-exec" { - command = local.is_windows_os ? "Start-Process ${each.value}" : "open ${each.value} || xdg-open ${each.value}" - interpreter = local.is_windows_os ? ["PowerShell", "-Command"] : ["/bin/bash", "-c"] - } -}*/ From b83083c5c98d12132862aa4414344adc8c7ac37a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 17:08:53 +0000 Subject: [PATCH 095/124] Removed variable 'hpcc_namespace' --- hpcc/hpcc.tf | 2 +- lite-variables.tf | 16 ---------------- 2 files changed, 1 insertion(+), 17 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index 08a2c08..b25c961 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -34,7 +34,7 @@ module "hpcc" { namespace = { create_namespace = false name = local.hpcc_namespace - labels = try(var.hpcc_namespace.labels,{}) + labels = { name = "hpcc" } } #----------------------------------------------------------------------- diff --git a/lite-variables.tf b/lite-variables.tf index ee5c262..e8c57f4 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -221,22 +221,6 @@ variable "authn_htpasswd_filename" { default = "" } -variable "hpcc_namespace" { - description = "Kubernetes namespace where resources will be created." - type = object({ - prefix_name = string - labels = map(string) - create_namespace = bool - }) - default = { - prefix_name = "hpcc" - labels = { - name = "hpcc" - } - create_namespace = false - } -} - variable "enable_premium_storage" { type = bool description = "OPTIONAL. If true, premium ($$$) storage will be used for the following storage shares: Dali.\nDefaults to false." From 185c760a3b491698d2978ec62c0d538e4c9aa12a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 17:50:28 +0000 Subject: [PATCH 096/124] Removed 18010 from output of eclwatch URL. Also, changed opinionated hpcc so it uses 443 instead of 18010. --- hpcc/outputs.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hpcc/outputs.tf b/hpcc/outputs.tf index 7b66964..81f1964 100644 --- a/hpcc/outputs.tf +++ b/hpcc/outputs.tf @@ -1,6 +1,6 @@ output "eclwatch_url" { description = "Print the ECL Watch URL." - value = format("https://%s.%s:18010",var.a_record_name, var.aks_dns_zone_name) + value = format("https://%s.%s",var.a_record_name, var.aks_dns_zone_name) } output "deployment_resource_group" { From b7724838141324e5a93ec53562ce0aa284027743 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 17:55:43 +0000 Subject: [PATCH 097/124] Capitalized Kubernetes everywhere in README.md --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index a33ea1b..0d44c34 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ NOTE: A tutorial of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). -This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. +This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. The HPCC Systems cluster created by this module uses ephemeral storage, which is the default. This means the storage will be deleted when the cluster is deleted) But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. @@ -16,7 +16,7 @@ This repo is a fork of the excellent work performed by Godson Fortil. The origi * **kubectl** The Kubernetes client (kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. -* **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within kubernetes clusters. TL;DR: Make sure you have the command line tools installed. +* **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. * If you run the terraform on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. @@ -96,7 +96,7 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | N | | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | -| `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | +| `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the Kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | | `aks_roxie_node_size ` | string | The VM size for each roxie node in the HPCC Systems. Example format `aks_roxie_node-size`="xlarge".| N | | `aks_serv_node_size ` | string | The VM size for each serv node in the HPCC Systems. Example format `aks_serv_node-size`="2xlarge".| N | | `aks_spray_node_size ` | string | The VM size for each spray node in the HPCC Systems. Example format `aks_spray_node-size`="2xlarge".| N | From 3eb4e6ec81e27a330f686385a74a108a0b4c80e5 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 17:58:55 +0000 Subject: [PATCH 098/124] Deleted paragraph 'This repo is a fork of the excellent work performed by Godson Fortil. ...' --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 0d44c34..7780b1c 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,6 @@ This is a slightly-opinionated Terraform module for deploying an HPCC Systems cl The HPCC Systems cluster created by this module uses ephemeral storage, which is the default. This means the storage will be deleted when the cluster is deleted) But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. -This repo is a fork of the excellent work performed by Godson Fortil. The original can be found in branch, HPCC-27615 of [https://github.com/gfortil/Terraform-azurerm-hpcc]. - ## Requirements * **Terraform** This is a Terraform module, so you need to have Terraform installed on your system. Instructions for downloading and installing Terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of Terraform, as that is needed to accommodate some of the large random numbers used for IDs in the Terraform modules. From 76386ea442fdc0b8b3806b871c192f3124bbb296 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 18:03:46 +0000 Subject: [PATCH 099/124] Make sure all these are capitalized in README.md when used as product name: Kubernetes, Helm, Github, and Kubectl. --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7780b1c..a1adaad 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * **helm** Helm is used to deploy the HPCC Systems processes under Kubernetes. Instructions for downloading and installing Helm are at [https://helm.sh/docs/intro/install](https://helm.sh/docs/intro/install/). -* **kubectl** The Kubernetes client (kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. +* **kubectl** The Kubernetes client (Kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. @@ -135,8 +135,8 @@ To get persistent storage, i.e. storage that is not deleted when the HPCC cluste * Make \ context the current context for future kubectl commands. * `kubectl config unset contexts.` * Delete context named \. - * Note that when you delete the current context, kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. -* Note that `terraform destroy` does not delete the kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. + * Note that when you delete the current context, Kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. +* Note that `terraform destroy` does not delete the Kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. * If a deployment fails and you want to start over, you have two options: * Immediately issue a `terraform destroy` command and let Terraform clean up. * Clean up the resources by hand: From 4f37c4328ff9a1617e572beb452d533f742864e5 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 18:14:37 +0000 Subject: [PATCH 100/124] Throughout README.md changed 'terraform' to 'terraform code' --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a1adaad..688a34a 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. -* If you run the terraform on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. +* If you run the terraform code on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. ## Installing/Using This Module From 85b2beff04c38eb77a9093fe5e54ffc77e7164dc Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 18:36:19 +0000 Subject: [PATCH 101/124] All fixes for Dan's comments about hpcc-tf-for-developers.md are in this commit. --- documentation/hpcc-tf-for-developers.md | 250 ++---------------------- 1 file changed, 18 insertions(+), 232 deletions(-) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index 5637e59..bb76c47 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -1,11 +1,11 @@ # For Developers: Tutorial of HPCC Easy Deploy Terraform -This tutorial explains the terraform that deploys HPCC Systems on an azure kubernetes service (aks). The terraform was designed to enable one to deploy HPCC Systems easily. -The terraform can be found on github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite]) +This tutorial explains the terraform code that deploys HPCC Systems on an azure Kubernetes service (aks). The terraform code was designed to enable one to deploy HPCC Systems easily. +The terraform code can be found on Github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite]) From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system from these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. -The following sections will explain the terraform in root directory and all subdirectories. +The following sections will explain the terraform code in root directory and all subdirectories. ## Root Directory Here is the root directory's contents (**blue** names are subdirectories) and a description of each entry: @@ -15,13 +15,13 @@ Here is the root directory's contents (**blue** names |:-----|:----------| | `lite-variables.tf` | Contains all input variables | | `lite.auto.tfvars.example` |Is an example .auto.tfvars file | -| `main.tf` | Contains most of the terraform that deploys all components of system | +| `main.tf` | Contains most of the terraform code that deploys all components of system | | `providers.tf` | Contains one provider, azurerm | | `scripts` | Directory containing scripts used in deployment | -| `aks` | Directory containing terraform to deploy `aks` | -| `hpcc` | Directory containing terraform to deploy `hpcc` | -| `storage` | Directory containing terraform to deploy external or persistent `storage` | -| `vnet` | Directory containing terraform to deploy virtual network used by `aks` | +| `aks` | Directory containing terraform code to deploy `aks` | +| `hpcc` | Directory containing terraform code to deploy `hpcc` | +| `storage` | Directory containing terraform code to deploy external or persistent `storage` | +| `vnet` | Directory containing terraform code to deploy virtual network used by `aks` | The following table shows all the variables in the file, `lite-variables.tf`, and their types. Plus, the table gives a description of each variable. Also, when one deploys from the root directory the `deploy` script puts these variables (or some of them) in the subdirectory where the deployment takes place. @@ -37,7 +37,10 @@ The following table shows all the variables in the file, `lite-variables.tf`, an | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | | `aks_max_node_count` | number | The maximum number of VM nodes to allocate for the HPCC Systems node pool. Must be 2 or more. | -| `aks_node_size` | string | The VM size for each node in the HPCC Systems node pool. Recommend "Standard_B4ms" or better. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes-general for more information. | +| `aks_roxie_node_size ` | string | The VM size for each roxie node in the HPCC Systems. Example format `aks_roxie_node-size`="xlarge".| +| `aks_serv_node_size ` | string | The VM size for each serv node in the HPCC Systems. Example format `aks_serv_node-size`="2xlarge".| +| `aks_spray_node_size ` | string | The VM size for each spray node in the HPCC Systems. Example format `aks_spray_node-size`="2xlarge".| +| `aks_thor_node_size ` | string | The VM size for each thor node in the HPCC Systems. Example format `aks_thor_node-size`="2xlarge".| | `authn_htpasswd_filename` | string | If you would like to use htpasswd to authenticate users to the cluster, enter the filename of the htpasswd file. This file should be uploaded to the Azure 'dllsshare' file share in order for the HPCC processes to find it. A corollary is that persistent storage is enabled. An empty string indicates that htpasswd is not to be used for authentication. Example entry: "htpasswd.txt" | | `enable_code_security` | boolean | Enable code security? If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. Example entry: false | | `enable_premium_storage` | boolean | If true, premium ($$$) storage will be used for the following storage shares: Dali. OPTIONAL, defaults to false. | @@ -74,24 +77,21 @@ The subfolders, except for `scripts`, create components needed by the full syste | `external_storage` | Waits for presistent storage to be created (or if ephemeral storage is used this scripts exits) NOTE: HPCC is not deployed until `external_storage` exits successfully. | | `extract-aks-variables` | the `deploy` script uses this script to copy from root directory the `lite-variables.tf` file contents used to deploy aks. | | `get_rg_from_file` | Outputs the resource group name in the `config.json` file given on the command line | -| `mkplan` | Makes a unique name for the file that will contain the terraform plan of a component being deployed. | +| `mkplan` | Makes a unique name for the file that will contain the `terraform plan` of a component being deployed. | | `needed-auto-tfvars-files` | Directory containing .auto.tfvars files needed by the `aks` and `storage` components. | ## aks subdirectory The following table tells what files and subdirectories and in the `aks` subdirectory. The deployment of an `aks`happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an `aks`. Also, if you deploy `aks` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in [_Appendix A_](#Appendix-A). - - |aks subdirectory entry name|description| |:------------------------------|:----------| | `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` is one of this file's variables which contains the variable, `my_azure_id` which is the object id of the user's azure account. This variable is given its value by the script `deploy`.| -| `aks.tf` | This file contains most of the terraform needed to deploy `aks`. The main module in this file is the `aks` module. | -| `automation.tf` | This file contains the terraform for scheduling the stopping and/or starting of the kubernetes cluster. | +| `aks.tf` | This file contains most of the terraform code needed to deploy `aks`. The main module in this file is the `aks` module. | +| `automation.tf` | This file contains the terraform code for scheduling the stopping and/or starting of the Kubernetes cluster. | | `data`<\font> | This directory and its contents, `config.json`, are created after the `aks` cluster is successfully deployed. | | `data.tf` | This file contains `data` statements that gets resources needed that already exist. | -| `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform was forked, all the variables in this file were input variables defined in `variables.tf`. | +| `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform code was forked, all the variables in this file were input variables defined in `variables.tf`. | | `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `aks`. This file was copied to the `aks` directory by the `deploy` script. | | `lite.auto.tfvars` | This file contains all the variables (and their values) whose name beings with `aks_`. These variables and their values are copied from the lite.auto.tfvars file in the root directory. The copy is done by the script, `deploy`. | | `locals.tf` | This file contains local variables that were originally in Godson Fortil's repository. | @@ -106,19 +106,17 @@ What is deployed by this subdirectory and their order is given in [_Appendix A_ The following table tells what files and subdirectories and in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. -What is deployed by this subdirectory and their order is given in [_Appendix B_](#Appendix-B). - |hpcc subdirectory entry name|description| |:--------------------------------|:----------| | `data.tf` | Contains `data` statements providing information about existing resources. | | `hpcc.tf` | Contains the `hpcc` module which does most of the work of deploying an hpcc cluster. | -| `lite-locals.tf` | Contains variables that use lite-variables.tf variables. The contents was in .auto.tfvars of Godson's terraform-azurerm-hpcc, branch HPCC-27615 (which this terraform is a fork). | +| `lite-locals.tf` | Contains variables that use lite-variables.tf variables. The contents was in .auto.tfvars of Godson's terraform-azurerm-hpcc, branch HPCC-27615 (which this terraform code is a fork). | | `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | | `lite.auto.tfvars` | Contains alls the variables used for easy deployment with values of the user. This file is copied in the hpcc directory by the `deploy` script. | | `locals.tf` | Contains local variables used in the deployment of the hpcc cluster. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | | `main.tf` | Contains modules and resources needed for the deployment of the hpcc cluster. | | `outputs.tf` | Contains output statements that show the user important information, like the eclwatch url and the resource group used by most of the resources. | -| `providers.tf` | Contains providers needed for the hpcc cluster deployment. Also, some of these providers (`kubernetes` and `kubectl`) get credentials for the kubernetes cluster for authenication. | +| `providers.tf` | Contains providers needed for the hpcc cluster deployment. Also, some of these providers (`kubernetes` and `kubectl`) get credentials for the Kubernetes cluster for authenication. | | `versions.tf` | Contains the versions needed for all providers. | | `data` | This directory contains the file `config.json` which is created when the hpcc cluster successfully deploys. | @@ -126,8 +124,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix B_ The following table tells what files and subdirectories and in the `storage` subdirectory. The deployment of an `storage` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `storage`. Also, if you deploy `storage` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in [_Appendix C_](#Appendix-C). - |storage subdirectory entry name|description| |:--------------------------------|:----------| | `data.tf` | Contains `data` statements providing information about existing resources. | @@ -146,8 +142,6 @@ What is deployed by this subdirectory and their order is given in [_Appendix C_] The following table tells what files and subdirectories are in the `vnet` subdirectory. The deployment of an `vnet` happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy `vnet`. Also, if you deploy `vnet` manually you do it from this directory. -What is deployed by this subdirectory and their order is given in [_Appendix D_](#Appendix-D). - |vnet subdirectory entry name|description| |:--------------------------------|:----------| | `data.tf` | Contains `data` statements providing information about existing resources. | @@ -163,211 +157,3 @@ What is deployed by this subdirectory and their order is given in [_Appendix D_ | `data` | This directory contains the file `config.json` which is created when the `vnet` is successfully deploys. | ​ - -## Appendix A - - -|Resources Created by aks Deployment| -|:------------------------------------------------------------------------------------------------| -| `data.azuread_group.subscription_owner` | -| `data.azurerm_advisor_recommendations.advisor` | -| `data.azurerm_client_config.current` | -| `data.azurerm_subscription.current` | -| `data.http.host_ip` | -| `local_file.output` | -| `null_resource.az[0]` | -| `random_integer.int` | -| `random_string.name` | -| `random_string.string` | -| `module.aks.data.azurerm_subscription.current` | -| `module.aks.kubernetes_config_map.terraform_modules` | -| `module.aks.kubernetes_config_map_v1_data.terraform_modules` | -| `module.aks.terraform_data.creation_metadata` | -| `module.aks.terraform_data.immutable_inputs` | -| `module.aks.time_static.timestamp` | -| `module.aks.module.cluster.data.azurerm_client_config.current` | -| `module.aks.module.cluster.data.azurerm_kubernetes_cluster.default` | -| `module.aks.module.cluster.data.azurerm_kubernetes_service_versions.default` | -| `module.aks.module.cluster.data.azurerm_monitor_diagnostic_categories.default` | -| `module.aks.module.cluster.data.azurerm_public_ip.outbound[0]` | -| `module.aks.module.cluster.azurerm_kubernetes_cluster.default` | -| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_network` | -| `module.aks.module.cluster.azurerm_role_assignment.network_contributor_route_table[0]` | -| `module.aks.module.cluster.azurerm_user_assigned_identity.default` | -| `module.aks.module.cluster.terraform_data.maintenance_control_plane_start_date` | -| `module.aks.module.cluster.terraform_data.maintenance_nodes_start_date` | -| `module.aks.module.cluster.time_sleep.modify` | -| `module.aks.module.cluster_version_tag.shell_script.default` | -| `module.aks.module.core_config.kubernetes_labels.system_namespace["default"]` | -| `module.aks.module.core_config.kubernetes_labels.system_namespace["kube-system"]` | -| `module.aks.module.core_config.kubernetes_namespace.default["cert-manager"]` | -| `module.aks.module.core_config.kubernetes_namespace.default["dns"]` | -| `module.aks.module.core_config.kubernetes_namespace.default["ingress-core-internal"]` | -| `module.aks.module.core_config.kubernetes_namespace.default["logging"]` | -| `module.aks.module.core_config.kubernetes_namespace.default["monitoring"]` | -| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_cluster` | -| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_managed_identity_operator_node` | -| `module.aks.module.core_config.module.aad_pod_identity.azurerm_role_assignment.k8s_virtual_machine_contributor_node` | -| `module.aks.module.core_config.module.aad_pod_identity.helm_release.aad_pod_identity` | -| `module.aks.module.core_config.module.aad_pod_identity.time_sleep.finalizer_wait` | -| `module.aks.module.core_config.module.cert_manager.helm_release.default` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt"]` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["letsencrypt_staging"]` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.issuers["zerossl"]` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["configmap-dashboard-cert-manager.yaml"]` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["poddistributionbudget-cert-manager-webhook.yaml"]` | -| `module.aks.module.core_config.module.cert_manager.kubectl_manifest.resource_files["prometheusrule-certmanager.yaml"]` | -| `module.aks.module.core_config.module.cert_manager.kubernetes_secret.zerossl_eabsecret` | -| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_federated_identity_credential.default["system:serviceaccount:cert-manager:cert-manager"]` | -| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_role_assignment.default[0]` | -| `module.aks.module.core_config.module.cert_manager.module.identity.azurerm_user_assigned_identity.default` | -| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_files["prometheusrule-coredns.yaml"]` | -| `module.aks.module.core_config.module.coredns.kubectl_manifest.resource_objects["coredns_custom"]` | -| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureassignedidentities.aadpodidentity.k8s.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentities.aadpodidentity.k8s.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azureidentitybindings.aadpodidentity.k8s.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["aad-pod-identity"].kubectl_manifest.crds["azurepodidentityexceptions.aadpodidentity.k8s.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificaterequests.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["certificates.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["challenges.acme.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["clusterissuers.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["issuers.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["cert-manager"].kubectl_manifest.crds["orders.acme.cert-manager.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["external-dns"].kubectl_manifest.crds["dnsendpoints.externaldns.k8s.io.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagerconfigs.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["alertmanagers.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["podmonitors.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["probes.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusagents.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheuses.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["prometheusrules.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["scrapeconfigs.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["servicemonitors.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.crds.module.crds["kube-prometheus-stack"].kubectl_manifest.crds["thanosrulers.monitoring.coreos.com.yaml"]` | -| `module.aks.module.core_config.module.external_dns.helm_release.public[0]` | -| `module.aks.module.core_config.module.external_dns.kubectl_manifest.resource_files["configmap-dashboard-external-dns.yaml"]` | -| `module.aks.module.core_config.module.external_dns.kubernetes_secret.public_config[0]` | -| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_federated_identity_credential.default["system:serviceaccount:dns:external-dns-public"]` | -| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[0]` | -| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_role_assignment.default[1]` | -| `module.aks.module.core_config.module.external_dns.module.identity_public[0].azurerm_user_assigned_identity.default` | -| `module.aks.module.core_config.module.ingress_internal_core.helm_release.default` | -| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.certificate` | -| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["configmap-dashboard-ingress-nginx-core-internal.yaml"]` | -| `module.aks.module.core_config.module.ingress_internal_core.kubectl_manifest.resource_files["prometheusrule-ingress-nginx-core-internal.yaml"]` | -| `module.aks.module.core_config.module.ingress_internal_core.time_sleep.lb_detach` | -| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0.shell_script.default` | -| `module.aks.module.core_config.module.pre_upgrade.module.v1_0_0-rc_1.shell_script.default` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-delete"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-ephemeral"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-retain"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-delete"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-ephemeral"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-premium-ssd-v2-retain"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-delete"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-ephemeral"]` | -| `module.aks.module.core_config.module.storage.kubernetes_storage_class.default["azure-disk-standard-ssd-retain"]` | -| `module.aks.module.node_groups.module.bootstrap_node_group_hack.shell_script.default` | -| `module.aks.module.node_groups.module.system_node_groups["system1"].azurerm_kubernetes_cluster_node_pool.default` | -| `module.aks.module.node_groups.module.user_node_groups["servpool1"].azurerm_kubernetes_cluster_node_pool.default` | -| `module.aks.module.node_groups.module.user_node_groups["spraypool1"].azurerm_kubernetes_cluster_node_pool.default` | -| `module.aks.module.node_groups.module.user_node_groups["thorpool1"].azurerm_kubernetes_cluster_node_pool.default` | -| `module.aks.module.rbac.azurerm_role_assignment.cluster_user["35cbdc79-7ef5-4d2c-9b59-61ec21d76aa9"]` | -| `module.aks.module.rbac.kubernetes_cluster_role.aggregate_to_view[0]` | -| `module.aks.module.rbac.kubernetes_cluster_role_binding.cluster_admin[0]` | -| `module.metadata.data.azurerm_subscription.current` | -| `module.resource_groups["azure_kubernetes_service"].azurerm_resource_group.rg` | -| `module.resource_groups["azure_kubernetes_service"].random_integer.suffix[0]` | -| `module.subscription.data.azurerm_subscription.selected` | - -## Appendix B - - -| Resources Created by HPCC Deployment | -| :----------------------------------------------------------- | -| `local_file.config.json` | -| `random_integer.random` | -| `module.hpcc.azurerm_storage_account.azurefiles_admin_services[0]` | -| `module.hpcc.azurerm_storage_account.blob_nfs_admin_services[0]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["debug"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["dll"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["mydropzone"]` | -| `module.hpcc.azurerm_storage_container.blob_nfs_admin_services["sasha"]` | -| `module.hpcc.azurerm_storage_share.azurefiles_admin_services["dali"]` | -| `module.hpcc.helm_release.hpcc` | -| `module.hpcc.kubernetes_persistent_volume.azurefiles["dali"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-1"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["data-2"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["debug"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["dll"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["mydropzone"]` | -| `module.hpcc.kubernetes_persistent_volume.blob_nfs["sasha"]` | -| `module.hpcc.kubernetes_persistent_volume.spill["spill"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.azurefiles["dali"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-1"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["data-2"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["debug"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["dll"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["mydropzone"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.blob_nfs["sasha"]` | -| `module.hpcc.kubernetes_persistent_volume_claim.spill["spill"]` | -| `module.hpcc.kubernetes_secret.azurefiles_admin_services[0]` | -| `module.hpcc.kubernetes_storage_class.premium_zrs_file_share_storage_class[0]` | -| `module.hpcc.random_string.random` | -| `module.hpcc.random_uuid.volume_handle` | -| `module.hpcc.module.certificates.kubectl_manifest.default_issuer` | -| `module.hpcc.module.certificates.kubectl_manifest.local_certificate` | -| `module.hpcc.module.certificates.kubectl_manifest.remote_certificate` | -| `module.hpcc.module.certificates.kubectl_manifest.signing_certificate` | -| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["1"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_account.default["2"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["1"]` | -| `module.hpcc.module.data_storage[0].azurerm_storage_container.hpcc_data["2"]` | - - -## Appendix C - - -|Resources Created by Depolyment of storage| -|:------------------------------------------------------------------------------------| -| `local_file.config.json` | -| `module.storage.azurerm_storage_account.azurefiles["adminsvc1"]` | -| `module.storage.azurerm_storage_account.blobnfs["adminsvc2"]` | -| `module.storage.azurerm_storage_account.blobnfs["data1"]` | -| `module.storage.azurerm_storage_account.blobnfs["data2"]` | -| `module.storage.azurerm_storage_container.blobnfs["1"]` | -| `module.storage.azurerm_storage_container.blobnfs["2"]` | -| `module.storage.azurerm_storage_container.blobnfs["3"]` | -| `module.storage.azurerm_storage_container.blobnfs["4"]` | -| `module.storage.azurerm_storage_container.blobnfs["5"]` | -| `module.storage.azurerm_storage_container.blobnfs["6"]` | -| `module.storage.azurerm_storage_share.azurefiles["0"]` | -| `module.storage.null_resource.remove0000_from_azurefile["adminsvc1"]` | -| `module.storage.null_resource.remove0000_from_blobfs["adminsvc2"]` | -| `module.storage.null_resource.remove0000_from_blobfs["data1"]` | -| `module.storage.null_resource.remove0000_from_blobfs["data2"]` | -| `module.storage.random_string.random` | -| `module.storage.module.resource_groups["storage_accounts"].azurerm_resource_group.rg` | -| `module.storage.module.resource_groups["storage_accounts"].random_integer.suffix[0]` | - -## Appendix D - - -| Resources Created by Deployment of vnet | -| :----------------------------------------------------------- | -| `data.azurerm_advisor_recommendations.advisor` | -| `data.azurerm_subscription.current` | -| `data.http.host_ip` | -| `local_file.output` | -| `module.metadata.data.azurerm_subscription.current` | -| `module.resource_groups["virtual_network"].azurerm_resource_group.rg` | -| `module.resource_groups["virtual_network"].random_integer.suffix[0]` | -| `module.subscription.data.azurerm_subscription.selected` | -| `module.virtual_network.azurerm_route.aks_route["hpcc-internet"]` | -| `module.virtual_network.azurerm_route.aks_route["hpcc-local-vnet-10-1-0-0-21"]` | -| `module.virtual_network.azurerm_route_table.aks_route_table["hpcc"]` | -| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-private"]` | -| `module.virtual_network.azurerm_subnet_route_table_association.aks["aks-hpcc-public"]` | -| `module.virtual_network.azurerm_virtual_network.vnet` | -| `module.virtual_network.module.aks_subnet["aks-hpcc-private"].azurerm_subnet.subnet` | -| `module.virtual_network.module.aks_subnet["aks-hpcc-public"].azurerm_subnet.subnet` | From e6ff10cd6ddb41e8bb167fcd0ea7647aadf07674 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 18:57:03 +0000 Subject: [PATCH 102/124] In lite-locals.tf, deleted all terraform code that was commented-out. --- lite-locals.tf | 82 -------------------------------------------------- 1 file changed, 82 deletions(-) diff --git a/lite-locals.tf b/lite-locals.tf index f73510d..caf179d 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -64,7 +64,6 @@ locals { thorpool_max_capacity = ceil("${ local.nodesPer1Job * var.thor_max_jobs }") helm_chart_timeout=300 - #hpcc_version = "8.6.20" owner = { name = var.admin_username @@ -100,53 +99,8 @@ locals { eclwatch = false } - # azure_auth = { - # # AAD_CLIENT_ID = "" - # # AAD_CLIENT_SECRET = "" - # # AAD_TENANT_ID = "" - # # AAD_PRINCIPAL_ID = "" - # SUBSCRIPTION_ID = "" - # } - - # hpcc_container = { - # version = "9.2.0" - # image_name = "platform-core-ln" - # image_root = "jfrog.com/glb-docker-virtual" - # # custom_chart_version = "9.2.0-rc1" - # # custom_image_version = "9.2.0-demo" - # } - - # hpcc_container_registry_auth = { - # username = "value" - # password = "value" - # } - internal_domain = var.aks_dns_zone_name // Example: hpcczone.us-hpccsystems-dev.azure.lnrsg.io - external = {} - # external = { - # blob_nfs = [{ - # container_id = "" - # container_name = "" - # id = "" - # resource_group_name = var.storage_account_resource_group_name - # storage_account_id = "" - # storage_account_name = var.storage_account_name - # }] - # # hpc_cache = [{ - # # id = "" - # # path = "" - # # server = "" - # }] - # hpcc = [{ - # name = "" - # planes = list(object({ - # local = "" - # remote = "" - # })) - # service = "" - # }] - # } admin_services_storage_account_settings = { replication_type = "ZRS" #LRS only if using HPC Cache @@ -171,12 +125,6 @@ locals { delete_protection = false } } - # hpc_cache = { - # enabled = false - # size = "small" - # cache_update_frequency = "3h" - # storage_account_data_planes = null - # } } external = null } @@ -197,36 +145,6 @@ locals { replicas = 6 nodeSelector = "spraypool" } - - # ldap = { - # ldap_server = "" //Server IP - # dali = { - # hpcc_admin_password = "" - # hpcc_admin_username = "" - # ldap_admin_password = "" - # ldap_admin_username = "" - # adminGroupName = "HPCC-Admins" - # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # } - # esp = { - # hpcc_admin_password = "" - # hpcc_admin_username = "" - # ldap_admin_password = "" - # ldap_admin_username = "" - # adminGroupName = "HPCC-Admins" - # filesBasedn = "ou=files,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # groupsBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # resourcesBasedn = "ou=smc,ou=espservices,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # systemBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # usersBasedn = "OU=AADDC Users,dc=z0lpf,dc=onmicrosoft,dc=com" - # workunitsBasedn = "ou=workunits,ou=eclHPCCSysUser,dc=z0lpf,dc=onmicrosoft,dc=com" - # } - # } roxie_internal_service = { name = "iroxie" From 38ef8771816b3e246874a3c57afb984e410b5751 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 21:42:23 +0000 Subject: [PATCH 103/124] All fixes for all Dan's review in 1:15pm email today. --- README.md | 22 ++++++++--------- aks/locals.tf | 4 ++-- documentation/hpcc-tf-for-developers.md | 24 ++++++------------- lite-locals.tf | 12 ++++++---- lite-variables.tf | 10 -------- lite.auto.tfvars.example | 12 +--------- .../aks/misc.auto.tfvars.example | 11 --------- .../storage/storage.auto.tfvars.example | 24 ------------------- 8 files changed, 28 insertions(+), 91 deletions(-) diff --git a/README.md b/README.md index 688a34a..db061f9 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,17 @@ NOTE: A tutorial of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). -This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. +This is a slightly-opinionated terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. The HPCC Systems cluster created by this module uses ephemeral storage, which is the default. This means the storage will be deleted when the cluster is deleted) But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. ## Requirements -* **Terraform** This is a Terraform module, so you need to have Terraform installed on your system. Instructions for downloading and installing Terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of Terraform, as that is needed to accommodate some of the large random numbers used for IDs in the Terraform modules. +* **terraform** This is a terraform module, so you need to have terraform installed on your system. Instructions for downloading and installing terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of terraform, as that is needed to accommodate some of the large random numbers used for IDs in the terraform modules. * **helm** Helm is used to deploy the HPCC Systems processes under Kubernetes. Instructions for downloading and installing Helm are at [https://helm.sh/docs/intro/install](https://helm.sh/docs/intro/install/). -* **kubectl** The Kubernetes client (Kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. +* **kubectl** The Kubernetes client (kubectl) is also required so you can inspect and manage the Azure Kubernetes cluster. Instructions for download and installing that can be found at [https://kubernetes.io/releases/download/](https://kubernetes.io/releases/download/). Make sure you have version 1.22.0 or later. * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. @@ -26,12 +26,12 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is 1. Clone this repo to your local system and change current directory. * `git clone -b https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` * `cd terraform-azurerm-hpcc-lite` -1. Issue `terraform init` to initialize the Terraform modules. +1. Issue `terraform init` to initialize the terraform modules. 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: - 1. Invoke the `terraform apply` command and enter values for each option as Terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. + 1. Invoke the `terraform apply` command and enter values for each option as terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to creat `lite.auto.tfvars` is to copy the example file, `lite.auto.tfvars.example`, and then edit the copy: * `cp -v lite.auto.tfvars.example lite.auto.tfvars` - 1. Use -var arguments on the command line when executing the Terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. + 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. 1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). At the end of a successful deployment these items are output for aks, hpcc, and vnet: @@ -135,16 +135,16 @@ To get persistent storage, i.e. storage that is not deleted when the HPCC cluste * Make \ context the current context for future kubectl commands. * `kubectl config unset contexts.` * Delete context named \. - * Note that when you delete the current context, Kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. -* Note that `terraform destroy` does not delete the Kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. + * Note that when you delete the current context, kubectl does not select another context as the current context. Instead, no context will be current. You must use `kubectl config use-context ` to make another context current. +* Note that `terraform destroy` does not delete the kubectl context. You need to use `kubectl config unset contexts.` to get rid of the context from your local system. * If a deployment fails and you want to start over, you have two options: - * Immediately issue a `terraform destroy` command and let Terraform clean up. + * Immediately issue a `terraform destroy` command and let terraform clean up. * Clean up the resources by hand: * Delete the Azure resource group manually, such as through the Azure Portal. * Note that there are two resource groups, if the deployment got far enough. Examples: * `app-thhpccplatform-sandbox-eastus-68255` * `mc_tf-zrms-default-aks-1` * The first one contains the Kubernetes service that created the second one (services that support Kubernetes). So, if you delete only the first resource group, the second resource group will be deleted automatically. - * Delete all Terraform state files using `rm *.tfstate*` + * Delete all terraform state files using `rm *.tfstate*` * Then, of course, fix whatever caused the deployment to fail. -* If you want to completely reset Terraform, issue `rm -rf .terraform* *.tfstate*` and then `terraform init`. +* If you want to completely reset terraform, issue `rm -rf .terraform* *.tfstate*` and then `terraform init`. diff --git a/aks/locals.tf b/aks/locals.tf index fab327c..feb8ac9 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -13,7 +13,7 @@ locals { node_type_version = "v2" node_size = var.aks_roxie_node_size single_group = false - min_capacity = 1 + min_capacity = 0 max_capacity = 3 labels = { "lnrs.io/tier" = "standard" @@ -31,7 +31,7 @@ locals { node_type_version = "v2" # v1, v2 node_size = var.aks_thor_node_size single_group = false - min_capacity = 1 + min_capacity = 0 max_capacity = var.aks_thorpool_max_capacity labels = { "lnrs.io/tier" = "standard" diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index bb76c47..07e9511 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -13,6 +13,7 @@ Here is the root directory's contents (**blue** names |Entry Name|Description| |:-----|:----------| +| `lite-locals.tf` | Contains variables used my `hpcc` and `aks` | | `lite-variables.tf` | Contains all input variables | | `lite.auto.tfvars.example` |Is an example .auto.tfvars file | | `main.tf` | Contains most of the terraform code that deploys all components of system | @@ -86,39 +87,31 @@ The following table tells what files and subdirectories and in the `aks` subdire |aks subdirectory entry name|description| |:------------------------------|:----------| -| `aks.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. This file contains `rbac_bindings` is one of this file's variables which contains the variable, `my_azure_id` which is the object id of the user's azure account. This variable is given its value by the script `deploy`.| | `aks.tf` | This file contains most of the terraform code needed to deploy `aks`. The main module in this file is the `aks` module. | -| `automation.tf` | This file contains the terraform code for scheduling the stopping and/or starting of the Kubernetes cluster. | -| `data`<\font> | This directory and its contents, `config.json`, are created after the `aks` cluster is successfully deployed. | | `data.tf` | This file contains `data` statements that gets resources needed that already exist. | | `lite-locals.tf` | This file contains local variables that need variables given in lite.auto.tfvars. In Godson Fortil's repository, which this terraform code was forked, all the variables in this file were input variables defined in `variables.tf`. | -| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `aks`. This file was copied to the `aks` directory by the `deploy` script. | -| `lite.auto.tfvars` | This file contains all the variables (and their values) whose name beings with `aks_`. These variables and their values are copied from the lite.auto.tfvars file in the root directory. The copy is done by the script, `deploy`. | | `locals.tf` | This file contains local variables that were originally in Godson Fortil's repository. | | `main.tf` | This file contains resources and modules needed for the deployment. They are: `resource "random_integer" "int`, `resource "random_string" "string`, `module "subscription`, `module "naming`, `module "metadata`, `module "resource_groups`, `resource "null_resource" "az`. | -| `misc.auto.tfvars` | This file is copied to the `aks` subdirectory when the `deploy` script is executed to deploy `aks`. | | `outputs.tf` | This file contains `output` statement which outputs the following: `advisor_recommendations`,`aks_login`,`cluster_name`,`hpcc_log_analytics_enabled`,`cluster_resource_group_name`. | | `providers.tf` | This file contains the following providers: `azurerm`,`azuread`,`kubernetes`,`kubernetes`,`kubectl`,`kubectl`,`helm`,`helm`,`shell`. | | `variables.tf` | This file contains the variables described in the next table. | | `versions.tf` | This file gives the version needed of each provider. | +| `scripts` | This directory contains scripts used in the `aks` terraform code. | ## hpcc subdirectory -The following table tells what files and subdirectories and in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. +The following table tells what files and subdirectories are in the hpcc subdirectory. The deployment of an hpcc cluster happens in this directory. If one deploys from the root directory, the `deploy` script goes to this directory to deploy an hpcc cluster. Also, if you deploy an hpcc cluster manually you do it from this directory. |hpcc subdirectory entry name|description| |:--------------------------------|:----------| | `data.tf` | Contains `data` statements providing information about existing resources. | | `hpcc.tf` | Contains the `hpcc` module which does most of the work of deploying an hpcc cluster. | -| `lite-locals.tf` | Contains variables that use lite-variables.tf variables. The contents was in .auto.tfvars of Godson's terraform-azurerm-hpcc, branch HPCC-27615 (which this terraform code is a fork). | -| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | -| `lite.auto.tfvars` | Contains alls the variables used for easy deployment with values of the user. This file is copied in the hpcc directory by the `deploy` script. | | `locals.tf` | Contains local variables used in the deployment of the hpcc cluster. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | | `main.tf` | Contains modules and resources needed for the deployment of the hpcc cluster. | | `outputs.tf` | Contains output statements that show the user important information, like the eclwatch url and the resource group used by most of the resources. | | `providers.tf` | Contains providers needed for the hpcc cluster deployment. Also, some of these providers (`kubernetes` and `kubectl`) get credentials for the Kubernetes cluster for authenication. | | `versions.tf` | Contains the versions needed for all providers. | -| `data` | This directory contains the file `config.json` which is created when the hpcc cluster successfully deploys. | +| `scripts` | This directory contains scripts used by the `hpcc` terraform code. | ## storage subdirectory @@ -126,16 +119,14 @@ The following table tells what files and subdirectories and in the `storage` sub |storage subdirectory entry name|description| |:--------------------------------|:----------| +| `README.md` | Contains a description of the `storage` module. | | `data.tf` | Contains `data` statements providing information about existing resources. | -| `lite-variables.tf` | This file contains the definition of all variables in `lite.auto.tfvars`. This is a subset of the root directory's lite-variables.tf use by `storage`. This file was copied to the `aks` directory by the `deploy` script. | | `locals.tf` | Contains local variables used in the deployment of the `storage`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | | `main.tf` | Contains only the `storage` module | | `outputs.tf` | Contains only the resource `local_file` which outputs to a file config.json. This is done only when on a successful deployment of `storage`. | | `providers.tf` | Contains only 2 providers: azurerm and azuread | -| `storage.auto.tfvars` | Contains variables that describe the storage accounts that are created. This file is copied to the `storage` directory by the `deploy` script. | | `variables.tf` | Contains variables needed for `storage` deployment. | | `versions.tf` | dummy description text | -| `data` | This directory contains the file `config.json` which is created when the external storage successfully deploys. | ## vnet subdirectory @@ -144,9 +135,8 @@ The following table tells what files and subdirectories are in the `vnet` subdir |vnet subdirectory entry name|description| |:--------------------------------|:----------| +| `README.md` | Contains a description of the `vnet` module. | | `data.tf` | Contains `data` statements providing information about existing resources. | -| `lite-variables.tf` | Contains all variables used for easy deployment. This file is copied in the hpcc directory by the `deploy` script. | -| `lite.auto.tfvars` | Contains all the variables used for easy deployment with values of the user. This file is copied in the `vnet` directory by the `deploy` script. | | `locals.tf` | Contains local variables used in the deployment of `vnet`. Variables in this file also use some of the easy deploy variables in lite-variables.tf. | | `main.tf` | Contains modules and resources needed for the deployment of `vnet` | | `outputs.tf` | Contains several output statements that output important information to the deployer. Also, this file contains an output state that outputs a file, config.json. This file is only output if there is a successful deployment of `vnet`. | @@ -154,6 +144,6 @@ The following table tells what files and subdirectories are in the `vnet` subdir | `variables.tf` | Contains only one variable, `disable_naming_conventions`. | | `versions.tf` | Contains the required versions of `terraform`, `azurerm` and `random`. | | `vnet.tf` | Contains the module `virtual_network` which deploys the virtual network used by `aks`, `hpcc`, and `storage`. | -| `data` | This directory contains the file `config.json` which is created when the `vnet` is successfully deploys. | + ​ diff --git a/lite-locals.tf b/lite-locals.tf index caf179d..bf8ada8 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -11,10 +11,10 @@ output "thor_ns_spec" { value = local.ns_spec[local.aks_node_sizes.thor] } output "thor_worker_cpus" { - value = var.thor_worker_cpus + value = local.thor_worker_cpus } output "thorWorkersPerNode" { - value = "local.ns_spec[${local.aks_node_sizes.thor}].cpu / var.thor_worker_cpus = ${local.thorWorkersPerNode}" + value = "local.ns_spec[${local.aks_node_sizes.thor}].cpu / local.thor_worker_cpus = ${local.thorWorkersPerNode}" } output "thor_worker_ram" { value = "local.ns_spec[${local.aks_node_sizes.thor}].ram / local.thorWorkersPerNode = ${local.thor_worker_ram}" @@ -26,6 +26,8 @@ output "thorpool_max_capacity" { value = "local.nodesPer1Job * var.thor_max_jobs = ${local.thorpool_max_capacity}" } locals { + thor_worker_cpus = 2 + aks_node_sizes = { roxie = var.aks_roxie_node_size serv = var.aks_serv_node_size @@ -52,8 +54,8 @@ locals { } } - twpn = "${ local.ns_spec[local.aks_node_sizes.thor].cpu / var.thor_worker_cpus }" - thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].cpu, ${local.ns_spec[local.aks_node_sizes.thor].cpu}, is not a multiple of var.thor_worker_cpus, ${var.thor_worker_cpus}." + twpn = "${ local.ns_spec[local.aks_node_sizes.thor].cpu / local.thor_worker_cpus }" + thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].cpu, ${local.ns_spec[local.aks_node_sizes.thor].cpu}, is not a multiple of local.thor_worker_cpus, ${local.thor_worker_cpus}." twr = "${local.ns_spec[local.aks_node_sizes.thor].ram / local.thorWorkersPerNode }" thor_worker_ram = ceil(local.twr) == local.twr? local.twr : "local.thor_worker_ram, ${local.twr}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].ram, ${local.ns_spec[local.aks_node_sizes.thor].ram}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." @@ -481,7 +483,7 @@ locals { memory = "2G" } workerResources = { - cpu = var.thor_worker_cpus + cpu = local.thor_worker_cpus memory = local.thor_worker_ram } workerMemory = { diff --git a/lite-variables.tf b/lite-variables.tf index e8c57f4..5c5c4df 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -201,16 +201,6 @@ variable "thor_num_workers" { default = 2 } -variable "thor_worker_cpus" { - type = number - description = "The number of CPUs each Thor worker should have.\nMust be 2, 4, 8 or 16. Also, this should be less than 'cpu' in thor node_size." - validation { - condition = ((var.thor_worker_cpus == 2) || (var.thor_worker_cpus == 4) || (var.thor_worker_cpus == 8) || (var.thor_worker_cpus == 16)) - error_message = "Value must be 2, 4, 8, or 16." - } - default = 2 -} - ############################################################################### # Optional variables ############################################################################### diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 58d12cb..4c77489 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -32,7 +32,7 @@ aks_dns_zone_resource_group_name="" #------------------------------------------------------------------------------ # The version of HPCC Systems to install. -# Only versions in nn.nn.nn format are supported. +# Only versions in nn.nn.nn format the 'latest' are supported. # Value type: string # Updateable: Y @@ -87,16 +87,6 @@ thor_max_jobs=2 #------------------------------------------------------------------------------ -# The number of CPUs each Thor worker should have. -# Must be 2, 4, 8 or 16. Also, this should be less than 'cpu' in thor node_size. -# REQUIRED -# Value type: number -# Updateable: N - -thor_worker_cpus=2 - -#------------------------------------------------------------------------------ - # The amount of storage reserved for the landing zone in gigabytes. # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. diff --git a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example index ed01f69..1e73c5a 100644 --- a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example +++ b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example @@ -9,14 +9,3 @@ resource_groups = { # rg # # auto_connect - Automatically connect to the kubernetes cluster from the host machine. auto_connect = true - -# # disable_naming_conventions - Disable naming conventions -# # disable_naming_conventions = true - -# azure_auth = { -# # AAD_CLIENT_ID = "" -# # AAD_CLIENT_SECRET = "" -# # AAD_TENANT_ID = "" -# # AAD_PRINCIPAL_ID = "" -# SUBSCRIPTION_ID = "" -# } diff --git a/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example index 38e6a1b..feaec01 100644 --- a/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example +++ b/scripts/needed-auto-tfvars-files/storage/storage.auto.tfvars.example @@ -3,11 +3,6 @@ storage_accounts = { # storage account delete_protection = false //Set to false to allow deletion prefix_name = "adminsvc1" storage_type = "azurefiles" - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "209.243.55.98" } - #authorized_ip_ranges = { anyone = "20.96.186.106" } - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "0.0.0.0/0" } authorized_ip_ranges = {} replication_type = "ZRS" subnet_ids = {} @@ -33,11 +28,6 @@ storage_accounts = { # storage account delete_protection = false //Set to false to allow deletion prefix_name = "adminsvc2" storage_type = "blobnfs" - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "209.243.55.98" } - #authorized_ip_ranges = { anyone = "20.96.186.106" } - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "0.0.0.0/0" } authorized_ip_ranges = {} replication_type = "ZRS" subnet_ids = {} @@ -90,20 +80,13 @@ storage_accounts = { # storage account delete_protection = false //Set to false to allow deletion prefix_name = "data1" storage_type = "blobnfs" - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "209.243.55.98" } - #authorized_ip_ranges = { anyone = "20.96.186.106" } - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "0.0.0.0/0" } authorized_ip_ranges = {} replication_type = "ZRS" - #replication_type = "GRS" subnet_ids = {} blob_soft_delete_retention_days = 7 container_soft_delete_retention_days = 7 access_tier = "Hot" account_kind = "StorageV2" - #account_kind = "BlobStorage" account_tier = "Standard" planes = { @@ -122,20 +105,13 @@ storage_accounts = { # storage account delete_protection = false //Set to false to allow deletion prefix_name = "data2" storage_type = "blobnfs" - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "209.243.55.98" } - #authorized_ip_ranges = { anyone = "20.96.186.106" } - #authorized_ip_ranges = { anyone = "97.118.251.104" } - #authorized_ip_ranges = { anyone = "0.0.0.0/0" } authorized_ip_ranges = {} replication_type = "ZRS" - #replication_type = "LRS" subnet_ids = {} blob_soft_delete_retention_days = 7 container_soft_delete_retention_days = 7 access_tier = "Hot" account_kind = "StorageV2" - #account_kind = "BlobStorage" account_tier = "Standard" planes = { From 6e366a71edb8d35afb59ec525d2b79c0c1ad9391 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 13 Dec 2023 21:46:29 +0000 Subject: [PATCH 104/124] In hpcc-tf-for-developers.md, capitalizes all Azure --- documentation/hpcc-tf-for-developers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index 07e9511..7ee4ede 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -1,6 +1,6 @@ # For Developers: Tutorial of HPCC Easy Deploy Terraform -This tutorial explains the terraform code that deploys HPCC Systems on an azure Kubernetes service (aks). The terraform code was designed to enable one to deploy HPCC Systems easily. +This tutorial explains the terraform code that deploys HPCC Systems on an Azure Kubernetes service (aks). The terraform code was designed to enable one to deploy HPCC Systems easily. The terraform code can be found on Github. Here is a link to it ([https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite]) From the root directory of the repository one can deploy all components of the HPCC cluster. Also, one can deploy individual components of the system from these subdirectories: `vnet`, `storage`, `aks`, and `hpcc`. If you want to deploy the individual components manually, here is the order you should do the deployment: 1st `vnet`, 2nd `storage` (if you want persistent storage), 3rd `aks`, and finally `hpcc`. @@ -50,7 +50,7 @@ The following table shows all the variables in the file, `lite-variables.tf`, an | `extra_tags` | map of string | Map of name => value tags that can will be associated with the cluster. Format is '{"name"="value" [, "name"="value"]*}'. The 'name' portion must be unique. To add no tags, use '{}'. | | `hpcc_user_ip_cidr_list` | list of string | List of explicit CIDR addresses that can access this HPCC Systems cluster. To allow public access, specify "0.0.0.0/0". To add no CIDR addresses, use '[]'. | | `hpcc_version` | string | The version of HPCC Systems to install. Only versions in nn.nn.nn format are supported. | -| `my_azure_id` | string | Your azure account object id. Find this on azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | +| `my_azure_id` | string | Your Azure account object id. Find this on Azure portal, by going to 'users' then search for your name and click on it. The account object id is called 'Object ID'. There is a link next to it that lets you copy it. | | `storage_data_gb` | number | The amount of storage reserved for data in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | | `storage_lz_gb` | number | The amount of storage reserved for the landing zone in gigabytes. Must be 1 or more. If a storage account is defined (see below) then this value is ignored. | | `thor_max_jobs` | number | The maximum number of simultaneous Thor jobs allowed. Must be 1 or more. | From e74c3b69cc48ce295c0deb3476d4a22a8ebf680c Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 14 Dec 2023 15:39:48 +0000 Subject: [PATCH 105/124] Dan's review fixes in email dated 12/14/2023 7:38am --- README.md | 10 ++-- aks/locals.tf | 23 -------- lite-locals.tf | 56 +----------------- lite-variables.tf | 2 +- lite.auto.tfvars.example | 57 +++++++++---------- scripts/change-source-statements.sh | 7 --- .../aks/misc.auto.tfvars.example | 3 - 7 files changed, 35 insertions(+), 123 deletions(-) delete mode 100755 scripts/change-source-statements.sh diff --git a/README.md b/README.md index db061f9..eee13ff 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,13 @@ NOTE: A tutorial of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). -This is a slightly-opinionated terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. +This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. The HPCC Systems cluster created by this module uses ephemeral storage, which is the default. This means the storage will be deleted when the cluster is deleted) But, you can also have Persistent Storage. See the section titled [Persistent Storage](#persistent-storage), below. ## Requirements -* **terraform** This is a terraform module, so you need to have terraform installed on your system. Instructions for downloading and installing terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of terraform, as that is needed to accommodate some of the large random numbers used for IDs in the terraform modules. +* **terraform** This is a Terraform module, so you need to have terraform installed on your system. Instructions for downloading and installing terraform can be found at [https://www.terraform.io/downloads.html](https://www.terraform.io/downloads.html). Do make sure you install a 64-bit version of terraform, as that is needed to accommodate some of the large random numbers used for IDs in the Terraform modules. * **helm** Helm is used to deploy the HPCC Systems processes under Kubernetes. Instructions for downloading and installing Helm are at [https://helm.sh/docs/intro/install](https://helm.sh/docs/intro/install/). @@ -24,12 +24,12 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is 1. If necessary, login to Azure. * From the command line, this is usually accomplished with the `az login` command. 1. Clone this repo to your local system and change current directory. - * `git clone -b https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` + * `git clone https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` * `cd terraform-azurerm-hpcc-lite` -1. Issue `terraform init` to initialize the terraform modules. +1. Issue `terraform init` to initialize the Terraform modules. 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: 1. Invoke the `terraform apply` command and enter values for each option as terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. - 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to creat `lite.auto.tfvars` is to copy the example file, `lite.auto.tfvars.example`, and then edit the copy: + 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to create `lite.auto.tfvars` is to copy the example file, `lite.auto.tfvars.example`, and then edit the copy: * `cp -v lite.auto.tfvars.example lite.auto.tfvars` 1. Use -var arguments on the command line when executing the terraform tool to set each of the values found in the .tfvars file. This method is useful if you are driving the creation of the cluster from a script. 1. After the Kubernetes cluster is deployed, your local `kubectl` tool can be used to interact with it. At some point during the deployment `kubectl` will acquire the login credentials for the cluster and it will be the current context (so any `kubectl` commands you enter will be directed to that cluster by default). diff --git a/aks/locals.tf b/aks/locals.tf index feb8ac9..3e01df0 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -79,26 +79,6 @@ locals { node_groups = var.aks_enable_roxie? merge( local.node_groups0, { roxiepool = local.roxiepool } ) : local.node_groups0 - aks_automation = { - local_authentication_enabled = false - public_network_access_enabled = false - automation_account_name = "aks-stop-demo-${random_string.name.result}" - - schedule = [ - { - schedule_name = "aks_stop" - description = "Stops the AKS weekday nights at 6PM MST" - runbook_name = "aks_startstop_runbook" - frequency = "Week" //OneTime, Day, Hour, Week, or Month. - interval = "1" //cannot be set when frequency is `OneTime` - operation = "stop" - daylight_saving = true - start_time = "20:00" // At least 5 minutes in the future - week_days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] - }, - ] - } - azure_auth_env = { AZURE_TENANT_ID = data.azurerm_client_config.current.tenant_id AZURE_SUBSCRIPTION_ID = data.azurerm_client_config.current.subscription_id @@ -146,10 +126,7 @@ locals { today = formatdate("YYYY-MM-DD", local.current_time) tomorrow = formatdate("YYYY-MM-DD", timeadd(local.current_time, "24h")) - utc_offset = local.aks_automation.schedule[0].daylight_saving ? 4 : 5 - script = { for item in fileset("${path.root}/scripts", "*") : (item) => file("${path.root}/scripts/${item}") } - schedule = { for s in local.aks_automation.schedule : "${s.schedule_name}" => s } az_command = "az aks get-credentials --name ${local.cluster_name} --resource-group ${module.resource_groups["azure_kubernetes_service"].name} --admin --overwrite-existing" is_windows_os = substr(pathexpand("~"), 0, 1) == "/" ? false : true diff --git a/lite-locals.tf b/lite-locals.tf index bf8ada8..1392bb0 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -1,30 +1,3 @@ -output "thor_max_jobs" { - value = var.thor_max_jobs -} -output "thor_num_workers" { - value = var.thor_num_workers -} -output "thor_node_size" { - value = local.aks_node_sizes.thor -} -output "thor_ns_spec" { - value = local.ns_spec[local.aks_node_sizes.thor] -} -output "thor_worker_cpus" { - value = local.thor_worker_cpus -} -output "thorWorkersPerNode" { - value = "local.ns_spec[${local.aks_node_sizes.thor}].cpu / local.thor_worker_cpus = ${local.thorWorkersPerNode}" -} -output "thor_worker_ram" { - value = "local.ns_spec[${local.aks_node_sizes.thor}].ram / local.thorWorkersPerNode = ${local.thor_worker_ram}" -} -output "nodesPer1Job" { - value = "var.thor_num_workers / local.thorWorkersPerNode = ${local.nodesPer1Job}" -} -output "thorpool_max_capacity" { - value = "local.nodesPer1Job * var.thor_max_jobs = ${local.thorpool_max_capacity}" -} locals { thor_worker_cpus = 2 @@ -63,7 +36,7 @@ locals { np1j = "${var.thor_num_workers / local.thorWorkersPerNode }" nodesPer1Job = ceil(local.np1j) == local.np1j? local.np1j : "local.nodesPer1Job, ${local.np1j}, is not an integer because var.thor_num_workers, ${var.thor_num_workers}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." - thorpool_max_capacity = ceil("${ local.nodesPer1Job * var.thor_max_jobs }") + thorpool_max_capacity = ceil("${ ceil(local.nodesPer1Job) * ceil(var.thor_max_jobs) }") helm_chart_timeout=300 @@ -74,29 +47,8 @@ locals { owner_name_initials = lower(join("",[for x in split(" ",local.owner.name): substr(x,0,1)])) - /*metadata = { - project = format("%shpccplatform", local.owner_name_initials) - product_name = format("%shpccplatform", local.owner_name_initials) - business_unit = "commercial" - environment = "sandbox" - market = "us" - product_group = format("%shpcc", local.owner_name_initials) - resource_group_type = "app" - sre_team = format("%shpccplatform", local.owner_name_initials) - subscription_type = "dev" - additional_tags = { "justification" = "testing" } - location = var.aks_azure_region # Acceptable values: eastus, centralus - } - - tags = merge(local.metadata.additional_tags, var.extra_tags) - */ - - # # disable_naming_conventions - Disable naming conventions - # # disable_naming_conventions = true disable_naming_conventions = false - # # auto_launch_eclwatch - Automatically launch ECLWatch web interface. - #auto_launch_eclwatch = true auto_launch_svc = { eclwatch = false } @@ -184,7 +136,6 @@ locals { disabled = (var.aks_enable_roxie == true)? false : true name = "roxie" nodeSelector = { workload = "roxiepool" } - # tlh 20231109 numChannels = 2 numChannels = 1 prefix = "roxie" replicas = 2 @@ -408,7 +359,6 @@ locals { throttle = 0 retryinterval = 6 keepResultFiles = false - # egress = "engineEgress" } dfuwu-archiver = { @@ -422,7 +372,6 @@ locals { cutoff = 14 at = "* * * * *" throttle = 0 - # egress = "engineEgress" } dfurecovery-archiver = { @@ -431,7 +380,6 @@ locals { limit = 20 cutoff = 4 at = "* * * * *" - # egress = "engineEgress" } file-expiry = { @@ -441,7 +389,6 @@ locals { persistExpiryDefault = 7 expiryDefault = 4 user = "sasha" - # egress = "engineEgress" } } @@ -474,7 +421,6 @@ locals { maxGraphs = 2 maxGraphStartupTime = 172800 numWorkersPerPod = 1 - #nodeSelector = {} nodeSelector = { workload = "thorpool" } egress = "engineEgress" tolerations_value = "thorpool" diff --git a/lite-variables.tf b/lite-variables.tf index 5c5c4df..c7d600e 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -188,7 +188,7 @@ variable "thor_max_jobs" { condition = var.thor_max_jobs >= 1 error_message = "Value must be 1 or more." } - default = 2 + default = 1 } variable "thor_num_workers" { diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index 4c77489..ba823ab 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -6,7 +6,7 @@ # Example entry: "my-product". This should be something project specific rather # than something generic. # Value type: string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) a_record_name="" @@ -15,7 +15,7 @@ a_record_name="" # Name of an existing dns zone. # Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_dns_zone_name="" @@ -24,7 +24,7 @@ aks_dns_zone_name="" # Name of the resource group of the above dns zone. # Example entry: "app-dns-prod-eastus2" # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_dns_zone_resource_group_name="" @@ -34,9 +34,9 @@ aks_dns_zone_resource_group_name="" # The version of HPCC Systems to install. # Only versions in nn.nn.nn format the 'latest' are supported. # Value type: string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) -hpcc_version="9.4.4" +hpcc_version="latest" #------------------------------------------------------------------------------ @@ -44,7 +44,7 @@ hpcc_version="9.4.4" # This will also expose port 8002 on the cluster. # Example entry: false # Value type: boolean -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) aks_enable_roxie=false @@ -54,7 +54,7 @@ aks_enable_roxie=false # If true, only signed ECL code will be allowed to create embedded language functions, use PIPE(), etc. # Example entry: false # Value type: boolean -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) enable_code_security=false @@ -63,7 +63,7 @@ enable_code_security=false # If you want a thor cluster then 'enable_thor' must be set to true # Otherwise it is set to false # Value type: boolean -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) enable_thor=true @@ -72,7 +72,7 @@ enable_thor=true # The number of Thor workers to allocate. # Must be 1 or more. # Value type: number -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) thor_num_workers=2 @@ -81,9 +81,9 @@ thor_num_workers=2 # The maximum number of simultaneous Thor jobs allowed. # Must be 1 or more. # Value type: number -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) -thor_max_jobs=2 +thor_max_jobs=1 #------------------------------------------------------------------------------ @@ -91,7 +91,7 @@ thor_max_jobs=2 # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. # Value type: number -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) storage_lz_gb=25 @@ -101,7 +101,7 @@ storage_lz_gb=25 # Must be 1 or more. # If a storage account is defined (see below) then this value is ignored. # Value type: number -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) storage_data_gb=100 @@ -112,7 +112,7 @@ storage_data_gb=100 # The 'name' portion must be unique. # To add no tags, use '{}'. # Value type: map of string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) extra_tags={} @@ -121,7 +121,7 @@ extra_tags={} # The VM size for each roxie node in the HPCC Systems node pool. # Possible values are: large, xlarge, 2xlarge, and 4xlarge # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_roxie_node_size = "xlarge" @@ -130,7 +130,7 @@ aks_roxie_node_size = "xlarge" # The VM size for each serv node in the HPCC Systems node pool. # Possible values are: large, xlarge, 2xlarge, and 4xlarge # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_serv_node_size = "2xlarge" @@ -139,7 +139,7 @@ aks_serv_node_size = "2xlarge" # The VM size for each spray node in the HPCC Systems node pool. # Possible values are: large, xlarge, 2xlarge, and 4xlarge # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_spray_node_size = "large" @@ -148,7 +148,7 @@ aks_spray_node_size = "large" # The VM size for each thor node in the HPCC Systems node pool. # Possible values are: large, xlarge, 2xlarge, and 4xlarge # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_thor_node_size = "xlarge" @@ -157,7 +157,7 @@ aks_thor_node_size = "xlarge" # Email address of the administrator of this HPCC Systems cluster. # Example entry: "jane.doe@hpccsystems.com" # Value type: string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) aks_admin_email="jane.doe@hpccsystems.com" @@ -166,7 +166,7 @@ aks_admin_email="jane.doe@hpccsystems.com" # Name of the administrator of this HPCC Systems cluster. # Example entry: "Jane Doe" # Value type: string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) aks_admin_name="Jane Doe" @@ -175,17 +175,16 @@ aks_admin_name="Jane Doe" # Username of the administrator of this HPCC Systems cluster. # Example entry: "jdoe" # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) admin_username="jdoe" #------------------------------------------------------------------------------ # The Azure region abbreviation in which to create these resources. -# Must be one of ["eastus", "eastus2", "centralus"]. # Example entry: "eastus" # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_azure_region="eastus" @@ -197,7 +196,7 @@ aks_azure_region="eastus" # To add no CIDR addresses, use '{}'. # The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. # Value type: map of string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) aks_admin_ip_cidr_map={} @@ -206,7 +205,7 @@ aks_admin_ip_cidr_map={} # List of explicit CIDR addresses that can access this HPCC Systems cluster. # To allow public access, set value to ["0.0.0.0/0"] or []. # Value type: list of string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) hpcc_user_ip_cidr_list=[] @@ -219,7 +218,7 @@ hpcc_user_ip_cidr_list=[] # An empty string indicates that htpasswd is not to be used for authentication. # Example entry: "htpasswd.txt" # Value type: string -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) authn_htpasswd_filename="" @@ -228,7 +227,7 @@ authn_htpasswd_filename="" # If you want external storage instead of ephemeral storage then # set this variable to true otherwise set it to false. # Value type: boolean -# Updateable: Y +# Updatable: Y (Y means YES, is updatable) external_storage_desired=false @@ -237,7 +236,7 @@ external_storage_desired=false # This variable enable you to ask for logging and monitoring of the kubernetes # and hpcc cluster (true means enable logging and monitoring, false means don't. # Value type: boolean -# Updateable: N +# Updatable: N (N means NO, not updatable) aks_logging_monitoring_enabled=false @@ -247,7 +246,7 @@ aks_logging_monitoring_enabled=false # then search for your name and click on it. The account object id is called # 'Object ID'. There is a link next to it that lets you copy it. # Value type: string -# Updateable: N +# Updatable: N (N means NO, not updatable) my_azure_id="" diff --git a/scripts/change-source-statements.sh b/scripts/change-source-statements.sh deleted file mode 100755 index 9f93b41..0000000 --- a/scripts/change-source-statements.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash -sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" hpcc/hpcc.tf -sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" aks/aks.tf -sed -i "s/^\( *source *= *\"git\@github.com:.*\.git\)[?]ref=.*$/\1\"/" storage/main.tf -sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" hpcc/hpcc.tf -sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" aks/aks.tf -sed -i "s/^\( *source *= *\"\)git\@\(github.com\):/\1git::https:\/\/\2\//" storage/main.tf diff --git a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example index 1e73c5a..714bb43 100644 --- a/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example +++ b/scripts/needed-auto-tfvars-files/aks/misc.auto.tfvars.example @@ -2,9 +2,6 @@ resource_groups = { # rg azure_kubernetes_service = { tags = { "enclosed resource" = "open source aks" } } - # azure_log_analytics_workspace = { - # tags = { "enclosed resource" = "azure log analytics workspace" } - # } } # # auto_connect - Automatically connect to the kubernetes cluster from the host machine. From 44cedc40eda5afb74bffbfd7f20f0b715caf94c7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 15 Dec 2023 16:58:58 +0000 Subject: [PATCH 106/124] Removed azuread_group.subscription_owner from aks/aks.tf and aks/data.tf. This is not available on must personal azure account. --- aks/aks.tf | 1 - aks/data.tf | 4 ---- 2 files changed, 5 deletions(-) diff --git a/aks/aks.tf b/aks/aks.tf index cec915b..f609661 100644 --- a/aks/aks.tf +++ b/aks/aks.tf @@ -30,7 +30,6 @@ module "aks" { dns_resource_group_lookup = { "${local.internal_domain}" = local.dns_resource_group } - #admin_group_object_ids = [data.azuread_group.subscription_owner.object_id] admin_group_object_ids = null rbac_bindings = var.rbac_bindings diff --git a/aks/data.tf b/aks/data.tf index de069dc..9ed1b2a 100644 --- a/aks/data.tf +++ b/aks/data.tf @@ -11,9 +11,5 @@ data "http" "host_ip" { data "azurerm_subscription" "current" { } -data "azuread_group" "subscription_owner" { - display_name = "ris-azr-group-${data.azurerm_subscription.current.display_name}-owner" -} - data "azurerm_client_config" "current" { } From 97f7eefa80e25b890c7160a2e22ec2a2f0288e69 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 18 Dec 2023 20:51:30 +0000 Subject: [PATCH 107/124] Make 1 or 4 nodepools optional. Added aks_4nodepools --- README.md | 1 + aks/locals.tf | 19 ++++++++++++++++++- hpcc/hpcc.tf | 2 +- lite-locals.tf | 18 +++++++++--------- lite-variables.tf | 6 ++++++ lite.auto.tfvars.example | 9 +++++++++ 6 files changed, 44 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index eee13ff..ad4ac60 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,7 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | | `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the Kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | +| `aks_4nodepools ` | boolean | Determines whether 1 or 4 nodepools are use -- 4 used if true otherwise 2 used. (default is false). | N | | `aks_roxie_node_size ` | string | The VM size for each roxie node in the HPCC Systems. Example format `aks_roxie_node-size`="xlarge".| N | | `aks_serv_node_size ` | string | The VM size for each serv node in the HPCC Systems. Example format `aks_serv_node-size`="2xlarge".| N | | `aks_spray_node_size ` | string | The VM size for each spray node in the HPCC Systems. Example format `aks_spray_node-size`="2xlarge".| N | diff --git a/aks/locals.tf b/aks/locals.tf index 3e01df0..8cac167 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -77,7 +77,24 @@ locals { } } - node_groups = var.aks_enable_roxie? merge( local.node_groups0, { roxiepool = local.roxiepool } ) : local.node_groups0 + hpccpool = { + ultra_ssd = false + node_os = "ubuntu" + node_type = "gp" # gp, gpd, mem, memd, stor + node_type_version = "v2" # v1, v2 + node_size = var.aks_serv_node_size + single_group = false + min_capacity = 4 + max_capacity = 385 + labels = { + "lnrs.io/tier" = "standard" + "workload" = "hpccpool" + } + taints = [] + tags = {} + } + + node_groups = var.aks_4nodepools? (var.aks_enable_roxie? merge( local.node_groups0, { roxiepool = local.roxiepool } ) : local.node_groups0) : { hpccpool = local.hpccpool } azure_auth_env = { AZURE_TENANT_ID = data.azurerm_client_config.current.tenant_id diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index b25c961..c5b4600 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -77,7 +77,7 @@ module "hpcc" { vault_config = local.vault_config eclccserver_settings = local.eclccserver_settings spray_service_settings = local.spray_service_settings - admin_services_node_selector = { all = { workload = "servpool" } } + admin_services_node_selector = var.aks_4nodepools? { all = { workload = "servpool" } } : { all = { workload = "hpccpool" } } esp_remoteclients = { diff --git a/lite-locals.tf b/lite-locals.tf index 1392bb0..71a1559 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -96,8 +96,8 @@ locals { } spray_service_settings = { - replicas = 6 - nodeSelector = "spraypool" + replicas = 1 + nodeSelector = var.aks_4nodepools? "spraypool" : "hpccpool" } roxie_internal_service = { @@ -135,7 +135,7 @@ locals { { disabled = (var.aks_enable_roxie == true)? false : true name = "roxie" - nodeSelector = { workload = "roxiepool" } + nodeSelector = var.aks_4nodepools? { workload = "roxiepool" } : { workload = "hpccpool" } numChannels = 1 prefix = "roxie" replicas = 2 @@ -288,7 +288,7 @@ locals { cpu = "1" memory = "4G" } - nodeSelector = { workload = "servpool" } + nodeSelector = var.aks_4nodepools? { workload = "servpool" } : { workload = "hpccpool" } egress = "engineEgress" cost = { perCpu = 1 @@ -307,7 +307,7 @@ locals { cpu = "1" memory = "4G" } - nodeSelector = { workload = "servpool" } + nodeSelector = var.aks_4nodepools? { workload = "servpool" } : { workload = "hpccpool" } legacySyntax = false options = [] cost = { @@ -320,7 +320,7 @@ locals { interval = 24 at = "* * * * *" minDeltaSize = 50000 - nodeSelector = { workload = "servpool" } + nodeSelector = var.aks_4nodepools? { workload = "servpool" } : { workload = "hpccpool" } resources = { cpu = "1" memory = "4G" @@ -335,7 +335,7 @@ locals { dfuserver_settings = { maxJobs = 3 - nodeSelector = { workload = "servpool" } + nodeSelector = var.aks_4nodepools? { workload = "servpool" } : { workload = "hpccpool" } resources = { cpu = "1" memory = "2G" @@ -344,7 +344,7 @@ locals { sasha_config = { disabled = false - nodeSelector = { workload = "servpool" } + nodeSelector = var.aks_4nodepools? { workload = "servpool" } : { workload = "hpccpool" } wu-archiver = { disabled = false service = { @@ -421,7 +421,7 @@ locals { maxGraphs = 2 maxGraphStartupTime = 172800 numWorkersPerPod = 1 - nodeSelector = { workload = "thorpool" } + nodeSelector = var.aks_4nodepools? { workload = "thorpool" } : { workload = "hpccpool" } egress = "engineEgress" tolerations_value = "thorpool" managerResources = { diff --git a/lite-variables.tf b/lite-variables.tf index c7d600e..b7e4361 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -1,4 +1,10 @@ # All 'aks_' variables are before any other variables. +variable "aks_4nodepools" { + description = "If true 4 nodepools are used. If false 2 are used." + type = bool + default = false +} + variable "aks_logging_monitoring_enabled" { description = "Used to get logging and monitoring of kubernetes and hpcc cluster." type = bool diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index ba823ab..dc31e8b 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -224,6 +224,15 @@ authn_htpasswd_filename="" #------------------------------------------------------------------------------ +# If 'aks_4nodepools` is true then 4 nodepools are used. Otherwise only 1 is +# used. Using 4 nodepools increases multi-processing and the cost. +# Value type: boolean +# Updatable: N (Y means YES, is updatable) + +aks_4nodepools=false + +#------------------------------------------------------------------------------ + # If you want external storage instead of ephemeral storage then # set this variable to true otherwise set it to false. # Value type: boolean From 4ba65e286754ee44df21e589b17cb2b20d20469f Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 19 Dec 2023 15:09:16 +0000 Subject: [PATCH 108/124] Removed all error messages in 'thorpool_max_capacity' calculations and replaced with floor values and '1' if result is 0. --- lite-locals.tf | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/lite-locals.tf b/lite-locals.tf index 71a1559..124d162 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -27,16 +27,16 @@ locals { } } - twpn = "${ local.ns_spec[local.aks_node_sizes.thor].cpu / local.thor_worker_cpus }" - thorWorkersPerNode = ceil(local.twpn) == local.twpn? local.twpn : "local.thorWorkersPerNode, ${local.twpn}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].cpu, ${local.ns_spec[local.aks_node_sizes.thor].cpu}, is not a multiple of local.thor_worker_cpus, ${local.thor_worker_cpus}." + twpn = floor("${ local.ns_spec[local.aks_node_sizes.thor].cpu / local.thor_worker_cpus }") + thorWorkersPerNode = local.twpn > 0? local.twpn : "local.thor_worker_cpus, ${local.thor_worker_cpus}, is larger then the number of CPUs on the selected node size, ${local.ns_spec[local.aks_node_sizes.thor].cpu}. That is, a large node size is needed." - twr = "${local.ns_spec[local.aks_node_sizes.thor].ram / local.thorWorkersPerNode }" - thor_worker_ram = ceil(local.twr) == local.twr? local.twr : "local.thor_worker_ram, ${local.twr}, is not an integer because local.ns_spec[${local.aks_node_sizes.thor}].ram, ${local.ns_spec[local.aks_node_sizes.thor].ram}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." + twr = floor("${local.ns_spec[local.aks_node_sizes.thor].ram / local.thorWorkersPerNode }") + thor_worker_ram = local.twr > 0? local.twr : 1 - np1j = "${var.thor_num_workers / local.thorWorkersPerNode }" - nodesPer1Job = ceil(local.np1j) == local.np1j? local.np1j : "local.nodesPer1Job, ${local.np1j}, is not an integer because var.thor_num_workers, ${var.thor_num_workers}, is not a multiple of local.thorWorkersPerNode, ${local.thorWorkersPerNode}." + np1j = floor("${var.thor_num_workers / local.thorWorkersPerNode }") + nodesPer1Job = local.np1j > 0? local.np1j : 1 - thorpool_max_capacity = ceil("${ ceil(local.nodesPer1Job) * ceil(var.thor_max_jobs) }") + thorpool_max_capacity = "${ local.nodesPer1Job * var.thor_max_jobs }" helm_chart_timeout=300 From a8cef3e4fae2d70d1ab8e62939a5d1b07fb951df Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 19 Dec 2023 15:33:42 +0000 Subject: [PATCH 109/124] Removed all occurrences of region restriction. --- README.md | 2 +- documentation/hpcc-tf-for-developers.md | 2 +- lite-variables.tf | 6 +----- 3 files changed, 3 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index ad4ac60..3dae5cf 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | Y | | `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | Y | | `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | Y | -| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | N | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Example entry: "eastus" | N | | `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | N | | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | N | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | diff --git a/documentation/hpcc-tf-for-developers.md b/documentation/hpcc-tf-for-developers.md index 7ee4ede..5fabb12 100755 --- a/documentation/hpcc-tf-for-developers.md +++ b/documentation/hpcc-tf-for-developers.md @@ -33,7 +33,7 @@ The following table shows all the variables in the file, `lite-variables.tf`, an | `aks_admin_email` | string | Email address of the administrator of this HPCC Systems cluster. Example entry: "jane.doe@hpccsystems.com" | | `aks_admin_ip_cidr_map` | map of string | Map of name => CIDR IP addresses that can administrate this AKS. Format is '{"name"="cidr" [, "name"="cidr"]*}'. The 'name' portion must be unique. To add no CIDR addresses, use '{}'. The corporate network and your current IP address will be added automatically, and these addresses will have access to the HPCC cluster as a user. | | `aks_admin_name` | string | Name of the administrator of this HPCC Systems cluster. Example entry: "Jane Doe" | -| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Must be one of ["eastus", "eastus2", "centralus"]. Example entry: "eastus" | +| `aks_azure_region` | string | The Azure region abbreviation in which to create these resources. Example entry: "eastus" | | `aks_dns_zone_name` | string | Name of an existing dns zone. Example entry: "hpcczone.us-hpccsystems-dev.azure.lnrsg.io" | | `aks_dns_zone_resource_group_name` | string | Name of the resource group of the above dns zone. Example entry: "app-dns-prod-eastus2" | | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | diff --git a/lite-variables.tf b/lite-variables.tf index b7e4361..8a8c83d 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -27,11 +27,7 @@ variable "aks_admin_name" { variable "aks_azure_region" { type = string - description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nMust be one of [\"eastus\", \"eastus2\", \"centralus\"].\nExample entry: eastus2" - validation { - condition = contains(["eastus", "eastus2", "centralus"], var.aks_azure_region) - error_message = "Value must be one of [\"eastus\", \"eastus2\", \"centralus\"]." - } + description = "REQUIRED. The Azure region abbreviation in which to create these resources.\nExample entry: eastus2" } variable "aks_enable_roxie" { From d899526e09fbb0b8c41c7cfba093e9d53d53f10d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 19 Dec 2023 15:37:27 +0000 Subject: [PATCH 110/124] In lite-variables.tf, changed 8002 to 18002 --- lite-variables.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-variables.tf b/lite-variables.tf index 8a8c83d..9f46fa2 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -31,7 +31,7 @@ variable "aks_azure_region" { } variable "aks_enable_roxie" { - description = "REQUIRED. Enable ROXIE?\nThis will also expose port 8002 on the cluster.\nExample entry: false" + description = "REQUIRED if you want roxie. Enable ROXIE?\nThis will also expose port 18002 on the cluster.\nExample entry: false" type = bool default = false } From 225509315b3d782cfdb958a2e20e758d18fe505b Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 19 Dec 2023 15:40:16 +0000 Subject: [PATCH 111/124] In lite.auto.tfvars.example, changed 8002 to 18002 --- lite.auto.tfvars.example | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite.auto.tfvars.example b/lite.auto.tfvars.example index dc31e8b..db412a2 100644 --- a/lite.auto.tfvars.example +++ b/lite.auto.tfvars.example @@ -41,7 +41,7 @@ hpcc_version="latest" #------------------------------------------------------------------------------ # Enable ROXIE? -# This will also expose port 8002 on the cluster. +# This will also expose port 18002 on the cluster. # Example entry: false # Value type: boolean # Updatable: Y (Y means YES, is updatable) From a5e3dc5d01117d417b382ac12ea0e81886d7a475 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 19 Dec 2023 16:36:27 +0000 Subject: [PATCH 112/124] Removed all error messages in 'thorpool_max_capacity' calculations and replaced with floor values and '1' if result is 0. --- lite-locals.tf | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/lite-locals.tf b/lite-locals.tf index 124d162..c91f4e9 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -28,13 +28,12 @@ locals { } twpn = floor("${ local.ns_spec[local.aks_node_sizes.thor].cpu / local.thor_worker_cpus }") - thorWorkersPerNode = local.twpn > 0? local.twpn : "local.thor_worker_cpus, ${local.thor_worker_cpus}, is larger then the number of CPUs on the selected node size, ${local.ns_spec[local.aks_node_sizes.thor].cpu}. That is, a large node size is needed." + thorWorkersPerNode = local.twpn > 0? local.twpn : 1 twr = floor("${local.ns_spec[local.aks_node_sizes.thor].ram / local.thorWorkersPerNode }") thor_worker_ram = local.twr > 0? local.twr : 1 - np1j = floor("${var.thor_num_workers / local.thorWorkersPerNode }") - nodesPer1Job = local.np1j > 0? local.np1j : 1 + nodesPer1Job = ceil("${var.thor_num_workers / local.thorWorkersPerNode }") thorpool_max_capacity = "${ local.nodesPer1Job * var.thor_max_jobs }" From abbb3bfb7919e931ab9d23ec059cd66e78ec739c Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 20 Dec 2023 15:12:18 +0000 Subject: [PATCH 113/124] To metadata.tf of storage, added 'additional_tags'. Plus, removed 'description' from outputs.tf --- scripts/deploy | 3 +++ scripts/needed-auto-tfvars-files/metadata.tf | 1 + storage/outputs.tf | 1 - 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/deploy b/scripts/deploy index a1caa64..3d8414d 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -83,6 +83,9 @@ cd $name; # cd into vnet or storage or aks or hpcc echo "Copy metadata.tf in root directory to this directory." cp -v ../scripts/needed-auto-tfvars-files/metadata.tf . +if [ "$name" == "storage" ];then + sed -i "s/#additional_tags/additional_tags/" metadata.tf +fi # put the root directory's lite.auto.tfvars (either all or part) in either aks or hpcc # directory. diff --git a/scripts/needed-auto-tfvars-files/metadata.tf b/scripts/needed-auto-tfvars-files/metadata.tf index 69b53f9..e04bd52 100644 --- a/scripts/needed-auto-tfvars-files/metadata.tf +++ b/scripts/needed-auto-tfvars-files/metadata.tf @@ -10,5 +10,6 @@ locals { sre_team = format("%shpccplatform", local.owner_name_initials) subscription_type = "dev" location = var.aks_azure_region # Acceptable values: eastus, centralus + #additional_tags = {} } } diff --git a/storage/outputs.tf b/storage/outputs.tf index f5cbbaa..9e5808a 100644 --- a/storage/outputs.tf +++ b/storage/outputs.tf @@ -1,5 +1,4 @@ resource "local_file" "config" { - description = "File containing configuration of external storage." content = module.storage.config filename = "${path.module}/data/config.json" From 9cf176599d59de3d45fe9ef30fba038bad7093c6 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 27 Dec 2023 21:22:36 +0000 Subject: [PATCH 114/124] 'workerResources' memory. Added 'G'. Caused thor container error. --- hpcc/hpcc.tf | 1 + hpcc/locals.tf | 2 +- hpcc/main.tf | 4 ++-- lite-locals.tf | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/hpcc/hpcc.tf b/hpcc/hpcc.tf index c5b4600..cc0c9dd 100644 --- a/hpcc/hpcc.tf +++ b/hpcc/hpcc.tf @@ -1,6 +1,7 @@ module "hpcc" { #source = "git@github.com:hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" source = "github.com/hpccsystems-solutions-lab/tlh-opinionated-terraform-azurerm-hpcc.git" + #source = "/home/azureuser/tlhumphrey2/RBA-terraform-azurerm-hpcc" environment = local.metadata.environment productname = local.metadata.product_name diff --git a/hpcc/locals.tf b/hpcc/locals.tf index becb47b..9f57295 100644 --- a/hpcc/locals.tf +++ b/hpcc/locals.tf @@ -45,7 +45,7 @@ locals { storage_type = plane.storage_type prefix_name = plane.prefix_name } - ] : [] + ] : [] #--------------------------------------------------------------------------------------------------------------------------- subnet_ids = try({ diff --git a/hpcc/main.tf b/hpcc/main.tf index 88c9157..debdf88 100644 --- a/hpcc/main.tf +++ b/hpcc/main.tf @@ -30,11 +30,11 @@ module "metadata" { project = local.metadata.project } -resource "null_resource" "delete_ephemeral_storage_accounts" { +/*resource "null_resource" "delete_ephemeral_storage_accounts" { count = var.external_storage_desired && (local.external_storage_config != []) ? 1 : 0 provisioner "local-exec" { command = "scripts/delete_ephemeral_storage_accounts ${local.get_aks_config.resource_group_name}" } depends_on = [module.hpcc] -} +}*/ diff --git a/lite-locals.tf b/lite-locals.tf index c91f4e9..f52d655 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -429,7 +429,7 @@ locals { } workerResources = { cpu = local.thor_worker_cpus - memory = local.thor_worker_ram + memory = format("%dG", local.thor_worker_ram) } workerMemory = { query = "3G" From f58b8b785cd71c3dcd8683d34db50e259fc5a86d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 3 Jan 2024 20:44:35 +0000 Subject: [PATCH 115/124] Added scripts/extract-aks-tfvars to properly extra 'aks_' variables from lite.auto.tfvars --- scripts/deploy | 4 ++-- scripts/extract-aks-tfvars | 43 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 2 deletions(-) create mode 100755 scripts/extract-aks-tfvars diff --git a/scripts/deploy b/scripts/deploy index 3d8414d..5d2ca7c 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -106,8 +106,8 @@ if [ "$name" == "hpcc" ];then cp -v ../lite-locals.tf . cp -v ../lite-variables.tf . elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];then - egrep "^aks_" ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars - egrep "^aks_" ../lite.auto.tfvars > lite.auto.tfvars + ../scripts/extract-aks-tfvars ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars + ../scripts/extract-aks-tfvars ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf if [ "$name" == "aks" ];then echo "aks_thorpool_max_capacity=$thorpool_max_capacity" >> lite.auto.tfvars diff --git a/scripts/extract-aks-tfvars b/scripts/extract-aks-tfvars new file mode 100755 index 0000000..6e332e8 --- /dev/null +++ b/scripts/extract-aks-tfvars @@ -0,0 +1,43 @@ +#!/bin/bash +# USAGE: scripts/extract-aks-tfvars lite.auto.tfvars +declare -A endbracket; +endbracket["{"]="}"; +endbracket["["]="]"; +if [ $# -le 0 ];then + echo "FATAL ERROR: $0: lite.auto.tfvars file name must be given on command line" + exit 1 +fi +tfvarsfile=$1; +while read line;do + re1='(aks_[A-Za-z0-9_][A-Za-z0-9_]* *= *)(..*)' + if [[ "$line" =~ ^$re1 ]];then + if [ "$var_plus_equal_plus_value" != "" ];then + echo $var_plus_equal_plus_value; + endbrack=""; + var_plus_equal_plus_value=""; + fi + var_plus_equal=${BASH_REMATCH[1]} + val=${BASH_REMATCH[2]} + var_plus_equal_plus_value="$var_plus_equal$val"; + re2='\([\{\[]\)' + result=$(echo $val|sed "s/$re2 *$/\1/") + #echo "DEBUG: var_plus_equal={$var_plus_equal}, val={$val}, result={$result}" + if [ "$result" == "{" ] || [ "$result" == "[" ];then + endbrack=${endbracket[$result]} + #echo "DEBUG: result is either '{' or '[': {$result}, endbrack={$endbrack}" + else + echo $var_plus_equal_plus_value; + fi + elif [ "$endbrack" == "}" ] || [ "$endbrack" == "]" ];then + var_plus_equal_plus_value="$var_plus_equal_plus_value $line"; + if [[ "$line" =~ "$endbrack" ]];then + echo $var_plus_equal_plus_value; + endbrack=""; + var_plus_equal_plus_value=""; + fi + else + endbrack=""; + var_plus_equal_plus_value=""; + fi + #echo "DEBUG: End of while read" +done <$tfvarsfile From 9cdfc88ccbc3767dc88d7c1744d58d5fe74d338f Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 3 Jan 2024 20:56:07 +0000 Subject: [PATCH 116/124] In lite-variables.tf, no longer says REQUIRED for aks_enable_roxie. --- lite-variables.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-variables.tf b/lite-variables.tf index 9f46fa2..639da20 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -31,7 +31,7 @@ variable "aks_azure_region" { } variable "aks_enable_roxie" { - description = "REQUIRED if you want roxie. Enable ROXIE?\nThis will also expose port 18002 on the cluster.\nExample entry: false" + description = "If you want roxie then this variable must be set to true. Enable ROXIE?\nThis will also expose port 18002 on the cluster.\nExample entry: false" type = bool default = false } From 81eca56d96befcdf4148fbb58a13f91ea19e429a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 3 Jan 2024 21:46:25 +0000 Subject: [PATCH 117/124] Changed workerMemory.query to same value as workerResources.memory. --- lite-locals.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-locals.tf b/lite-locals.tf index f52d655..6e7b001 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -432,7 +432,7 @@ locals { memory = format("%dG", local.thor_worker_ram) } workerMemory = { - query = "3G" + query = format("%dG", local.thor_worker_ram) thirdParty = "500M" } eclAgentResources = { From ed598d89156627285d6dae242736bcf6e292351d Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Thu, 4 Jan 2024 21:40:39 +0000 Subject: [PATCH 118/124] Created 'aks_nodepools_max_capacity'. max_capacity of all hpcc nodepools are set to this. Also, fixed workerMemory.query and workerMemory.thirdParty. --- README.md | 1 + aks/locals.tf | 12 ++++++------ lite-locals.tf | 9 +++++---- lite-variables.tf | 6 +++--- main.tf | 2 +- scripts/deploy | 11 ----------- 6 files changed, 16 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 3dae5cf..ec8acac 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,7 @@ The following options should be set in your `lite.auto.tfvars` file (or entered | `aks_enable_roxie` | boolean | Enable ROXIE? This will also expose port 8002 on the cluster. Example entry: false | Y | | `aks_logging_monitoring_enabled` | boolean | This variable enable you to ask for logging and monitoring of the Kubernetes and hpcc cluster (true means enable logging and monitoring, false means don't. | N | | `aks_4nodepools ` | boolean | Determines whether 1 or 4 nodepools are use -- 4 used if true otherwise 2 used. (default is false). | N | +| `aks_nodepools_max_capacity` | string | The maximum number of nodes of every hpcc nodepool.| N | | `aks_roxie_node_size ` | string | The VM size for each roxie node in the HPCC Systems. Example format `aks_roxie_node-size`="xlarge".| N | | `aks_serv_node_size ` | string | The VM size for each serv node in the HPCC Systems. Example format `aks_serv_node-size`="2xlarge".| N | | `aks_spray_node_size ` | string | The VM size for each spray node in the HPCC Systems. Example format `aks_spray_node-size`="2xlarge".| N | diff --git a/aks/locals.tf b/aks/locals.tf index 8cac167..aff8403 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -14,7 +14,7 @@ locals { node_size = var.aks_roxie_node_size single_group = false min_capacity = 0 - max_capacity = 3 + max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "roxiepool" @@ -32,7 +32,7 @@ locals { node_size = var.aks_thor_node_size single_group = false min_capacity = 0 - max_capacity = var.aks_thorpool_max_capacity + max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "thorpool" @@ -49,7 +49,7 @@ locals { node_size = var.aks_serv_node_size single_group = false min_capacity = 1 - max_capacity = 3 + max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "servpool" @@ -66,7 +66,7 @@ locals { node_size = var.aks_spray_node_size single_group = false min_capacity = 0 - max_capacity = 6 + max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "spraypool" @@ -84,8 +84,8 @@ locals { node_type_version = "v2" # v1, v2 node_size = var.aks_serv_node_size single_group = false - min_capacity = 4 - max_capacity = 385 + min_capacity = 1 + max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" "workload" = "hpccpool" diff --git a/lite-locals.tf b/lite-locals.tf index 6e7b001..ee73284 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -35,8 +35,6 @@ locals { nodesPer1Job = ceil("${var.thor_num_workers / local.thorWorkersPerNode }") - thorpool_max_capacity = "${ local.nodesPer1Job * var.thor_max_jobs }" - helm_chart_timeout=300 owner = { @@ -410,6 +408,9 @@ locals { admin_services_node_selector = {} + workerMemory_thirdParty = "500" + workerMemory_query = "${local.thor_worker_ram - local.workerMemory_thirdParty}" + thor_config = [{ disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true name = "thor" @@ -432,8 +433,8 @@ locals { memory = format("%dG", local.thor_worker_ram) } workerMemory = { - query = format("%dG", local.thor_worker_ram) - thirdParty = "500M" + query = format(%dG", local.workerMemory_query) + thirdParty = format(%dG", local.workerMemory_thirdParty) } eclAgentResources = { cpu = 1 diff --git a/lite-variables.tf b/lite-variables.tf index 639da20..28dd9ed 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -52,10 +52,10 @@ variable "aks_admin_ip_cidr_map" { default = {} } -variable "aks_thorpool_max_capacity" { +variable "aks_nodepools_max_capacity" { type = number - description = "The max capacity (or node count) of the thorpool. This is calculated and an argument for terraform plan and appy for aks." - default = 2 + description = "The max capacity (or maximum node count) of all hopcc nodepools." + default = 400 } variable "aks_roxie_node_size" { diff --git a/main.tf b/main.tf index 6bfef65..16fba90 100644 --- a/main.tf +++ b/main.tf @@ -8,7 +8,7 @@ resource "null_resource" "deploy_vnet" { resource "null_resource" "deploy_aks" { provisioner "local-exec" { - command = "scripts/deploy aks ${var.my_azure_id} ${local.thorpool_max_capacity}" + command = "scripts/deploy aks ${var.my_azure_id}" } depends_on = [ null_resource.deploy_vnet ] diff --git a/scripts/deploy b/scripts/deploy index 5d2ca7c..97bfb8b 100755 --- a/scripts/deploy +++ b/scripts/deploy @@ -11,14 +11,6 @@ if [ "$1" != "" ] && [[ $1 =~ hpcc|aks|vnet|storage ]];then echo "ERROR: name=\"$name\". Need a 2nd argument (my_azure_id) but none given.";exit 1; fi fi - if [ "$name" == "aks" ];then - if [ "$3" != "" ];then - thorpool_max_capacity=$3 - #echo "DEBUG: name=\"$name\", \$3(or thorpool_max_capacity)=\"$3\"";exit - else - echo "ERROR: name=\"$name\". Need a 3rd argument (thorpool max capacity) but none given.";exit 1; - fi - fi else echo "$0 has no arguments. It must of 1 argument that is 'vnet' or 'storage' or 'aks' or 'hpcc'. EXITING.";exit 1; fi @@ -109,9 +101,6 @@ elif [ "$name" == "aks" ] || [ "$name" == "vnet" ] || [ "$name" == "storage" ];t ../scripts/extract-aks-tfvars ../lite.auto.tfvars > /tmp/${name}.lite.auto.tfvars ../scripts/extract-aks-tfvars ../lite.auto.tfvars > lite.auto.tfvars ../scripts/extract-aks-variables ../lite-variables.tf > lite-variables.tf - if [ "$name" == "aks" ];then - echo "aks_thorpool_max_capacity=$thorpool_max_capacity" >> lite.auto.tfvars - fi if [ "$name" == "storage" ];then cp -v $thisdir/needed-auto-tfvars-files/storage/storage.auto.tfvars.example ./storage.auto.tfvars fi From 05f46ff35bc421dea3abdbf1fb956c79c6e4816c Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 5 Jan 2024 15:38:09 +0000 Subject: [PATCH 119/124] Corrected 'aks_nodepools_max_capacity' code of lite-locals.tf and lite-variables.tf --- lite-locals.tf | 4 ++-- lite-variables.tf | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lite-locals.tf b/lite-locals.tf index ee73284..0a82cea 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -433,8 +433,8 @@ locals { memory = format("%dG", local.thor_worker_ram) } workerMemory = { - query = format(%dG", local.workerMemory_query) - thirdParty = format(%dG", local.workerMemory_thirdParty) + query = format("%dG", local.workerMemory_query) + thirdParty = format("%dG", local.workerMemory_thirdParty) } eclAgentResources = { cpu = 1 diff --git a/lite-variables.tf b/lite-variables.tf index 28dd9ed..7e3d736 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -55,7 +55,7 @@ variable "aks_admin_ip_cidr_map" { variable "aks_nodepools_max_capacity" { type = number description = "The max capacity (or maximum node count) of all hopcc nodepools." - default = 400 + default = 385 } variable "aks_roxie_node_size" { From 09cad2bb5bdf9322d166b90da6eba62fc650dac6 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Fri, 5 Jan 2024 21:49:42 +0000 Subject: [PATCH 120/124] In README.md, minimum vCPU requirements given. In aks/locals.tf, min_capacity changed for hpccpool & servpool. In lite-locals.tf, Corrected workerMemory values. --- README.md | 5 ++++- aks/locals.tf | 4 ++-- lite-locals.tf | 5 +++-- lite-variables.tf | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index ec8acac..7464b10 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Deploy HPCC Systems on Azure under Kubernetes -NOTE: A tutorial of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). +NOTE: Documentation of this Terraform for the developer, or others who are interested, can be found [here](documentation/hpcc-tf-for-developers.md). This is a slightly-opinionated Terraform module for deploying an HPCC Systems cluster on Azure's Kubernetes service (aks). The goal is to provide a simple method for deploying a cluster from scratch, with only the most important options to consider. @@ -17,6 +17,9 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. +* You need a minimum of 28 vCPUs available on `azure` and `aks_serv_node_size` must be at least `xlarge`. The following `az` command will tell you the maximum number of vCPUs you can use. And, the 2nd `az` command, below, gives you the number of vCPUs you have already used region `eastus` (replace `eastus` with the name of the region you are using). The first `az` command result minus the second `az` command result gives you the number of vCPUs available for you to use. + * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f5` + * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f4` * If you run the terraform code on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. ## Installing/Using This Module diff --git a/aks/locals.tf b/aks/locals.tf index aff8403..c54ea7a 100644 --- a/aks/locals.tf +++ b/aks/locals.tf @@ -48,7 +48,7 @@ locals { node_type_version = "v1" node_size = var.aks_serv_node_size single_group = false - min_capacity = 1 + min_capacity = 3 max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" @@ -84,7 +84,7 @@ locals { node_type_version = "v2" # v1, v2 node_size = var.aks_serv_node_size single_group = false - min_capacity = 1 + min_capacity = 3 max_capacity = var.aks_nodepools_max_capacity labels = { "lnrs.io/tier" = "standard" diff --git a/lite-locals.tf b/lite-locals.tf index 0a82cea..18a74fa 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -409,7 +409,8 @@ locals { admin_services_node_selector = {} workerMemory_thirdParty = "500" - workerMemory_query = "${local.thor_worker_ram - local.workerMemory_thirdParty}" + wmq = "${(local.thor_worker_ram * 1000000000) - (local.workerMemory_thirdParty * 1000000)}" + workerMemory_query = floor("${local.wmq / 1000000000}") thor_config = [{ disabled = (var.enable_thor == true) || (var.enable_thor == null)? false : true @@ -434,7 +435,7 @@ locals { } workerMemory = { query = format("%dG", local.workerMemory_query) - thirdParty = format("%dG", local.workerMemory_thirdParty) + thirdParty = format("%dM", local.workerMemory_thirdParty) } eclAgentResources = { cpu = 1 diff --git a/lite-variables.tf b/lite-variables.tf index 7e3d736..799bdd6 100644 --- a/lite-variables.tf +++ b/lite-variables.tf @@ -1,6 +1,6 @@ # All 'aks_' variables are before any other variables. variable "aks_4nodepools" { - description = "If true 4 nodepools are used. If false 2 are used." + description = "If true 4 nodepools are used. If false 1 are used." type = bool default = false } From 4d1b78f973caa0be444ae1badfeae5d16b92d6f3 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Mon, 8 Jan 2024 15:24:59 +0000 Subject: [PATCH 121/124] Added in README.md: 1) info about the directory, and what is in it, and 2) the need for 28 vCPUs before deploying HPCC. --- README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7464b10..57288f2 100644 --- a/README.md +++ b/README.md @@ -17,9 +17,10 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. -* You need a minimum of 28 vCPUs available on `azure` and `aks_serv_node_size` must be at least `xlarge`. The following `az` command will tell you the maximum number of vCPUs you can use. And, the 2nd `az` command, below, gives you the number of vCPUs you have already used region `eastus` (replace `eastus` with the name of the region you are using). The first `az` command result minus the second `az` command result gives you the number of vCPUs available for you to use. - * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f5` - * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f4` + * You need a minimum of 28 vCPUs available on `azure` and `aks_serv_node_size` must be at least `xlarge`. The following `az` command will tell you the maximum number of vCPUs you can use. And, the 2nd `az` command, below, gives you the number of vCPUs you have already used in region `eastus` (replace `eastus` with the name of the region you are using). Furthermore, you can get the number of vCPUs available for you to use by subtracting the result of the 2nd `az` command, below, from the result of the first `az` command. + * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f5` + * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f4` + * If you run the terraform code on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. ## Installing/Using This Module @@ -30,6 +31,9 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * `git clone https://github.com/hpccsystems-solutions-lab/terraform-azurerm-hpcc-lite.git` * `cd terraform-azurerm-hpcc-lite` 1. Issue `terraform init` to initialize the Terraform modules. +2. Issue `terraform apply` + This command will do a `terraform init`, `terraform plan` and `terraform apply` for each of the subsystems needed, i.e. `vnet`, `aks`, `storage`, and `hpcc` (the `storage` subsystem is deployed only if you set `external_storage_desired=true`). The order that these subsystems are deploy is: `vnet`, `aks`, `storage`, and `hpcc`. + For each subsystem, `terraform` creates a `plan` file which is stored in the directory: `~/tflogs` (note: if this directory doesn't exist, it is created automatically). 1. Decide how you want to supply option values to the module during invocation. There are three possibilities: 1. Invoke the `terraform apply` command and enter values for each option as terraform prompts for it, then enter `yes` at the final prompt to begin building the cluster. 1. **Recommended:** Create a `lite.auto.tfvars` file containing the values for each option, invoke `terraform apply`, then enter `yes` at the final prompt to begin building the cluster. The easiest way to create `lite.auto.tfvars` is to copy the example file, `lite.auto.tfvars.example`, and then edit the copy: From e4a8cc4cd9c6724ce769e617ed744c05256da75a Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 10 Jan 2024 18:40:45 +0000 Subject: [PATCH 122/124] In scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example, changed availability_zone from 1 to 2. Why? In eastus2, getting error message saying hpccpool could not be created because constrints too restrictive. --- aks/variables.tf | 2 +- scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/aks/variables.tf b/aks/variables.tf index fc51908..2daf996 100644 --- a/aks/variables.tf +++ b/aks/variables.tf @@ -238,5 +238,5 @@ variable "availability_zones" { description = "Availability zones to use for the node groups." type = list(number) nullable = false - default = [1] + default = [2] } diff --git a/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example index 685ebd7..cbae554 100644 --- a/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example +++ b/scripts/needed-auto-tfvars-files/aks/aks.auto.tfvars.example @@ -16,4 +16,4 @@ rbac_bindings = { cluster_endpoint_access_cidrs = ["0.0.0.0/0"] -availability_zones = [1] +availability_zones = [2] From 431fad0417587bebed49bff61b4b1ae7c7ce83f4 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 10 Jan 2024 19:22:36 +0000 Subject: [PATCH 123/124] In lite-locals.tf, increased 'helm_chart_timeout' from 300 to 600. Why? 300 was not enough time for HPCC to fully deploy on my azure vm Standard_B2s --- lite-locals.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lite-locals.tf b/lite-locals.tf index 18a74fa..7d4e71d 100644 --- a/lite-locals.tf +++ b/lite-locals.tf @@ -35,7 +35,7 @@ locals { nodesPer1Job = ceil("${var.thor_num_workers / local.thorWorkersPerNode }") - helm_chart_timeout=300 + helm_chart_timeout=600 owner = { name = var.admin_username From 3c7c641f2c47cfd5ee09431814f19fb16855f8ec Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 10 Jan 2024 20:40:27 +0000 Subject: [PATCH 124/124] In README.md, said that 'jq' and 'kubelogin' are required (i.e. they must be installed). --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 57288f2..69a7816 100644 --- a/README.md +++ b/README.md @@ -17,10 +17,12 @@ The HPCC Systems cluster created by this module uses ephemeral storage, which is * **Azure CLI** To work with Azure, you will need to install the Azure Command Line tools. Instructions can be found at [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Even if you think you won't be working with Azure, this module does leverage the command line tools to manipulate network security groups within Kubernetes clusters. TL;DR: Make sure you have the command line tools installed. * To successfully create everything you will need to have Azure's `Contributor` role plus access to `Microsoft.Authorization/*/Write` and `Microsoft.Authorization/*/Delete` permissions on your subscription. You may have to create a custom role for this. Of course, Azure's `Owner` role includes everything so if you're the subscription's owner then you're good to go. - * You need a minimum of 28 vCPUs available on `azure` and `aks_serv_node_size` must be at least `xlarge`. The following `az` command will tell you the maximum number of vCPUs you can use. And, the 2nd `az` command, below, gives you the number of vCPUs you have already used in region `eastus` (replace `eastus` with the name of the region you are using). Furthermore, you can get the number of vCPUs available for you to use by subtracting the result of the 2nd `az` command, below, from the result of the first `az` command. +* You need a minimum of 28 vCPUs available on `azure` and `aks_serv_node_size` must be at least `xlarge`. The following `az` command will tell you the maximum number of vCPUs you can use. And, the 2nd `az` command, below, gives you the number of vCPUs you have already used in region `eastus` (replace `eastus` with the name of the region you are using). Furthermore, you can get the number of vCPUs available for you to use by subtracting the result of the 2nd `az` command, below, from the result of the first `az` command. * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f5` * `az vm list-usage --location "eastus" -o table|grep "Total Regional vCPUs"|sed "s/ */\t/g"|cut -f4` - +* You need to make sure `jq` and `kubelogin` are installed on your linux machine. You can determine if they are by using the `which` command, e.g. `which jq` returns `jq`s path if it is installed. The following commands can be used to install '`jq` and `kubelogin`, respectively: + * `sudo apt-get install jq` + * `sudo az aks install-cli` * If you run the terraform code on an azure VM, then the azure VM must have EncryptionAtHost enabled. You can do this by: 1) Stopping your azure VM; 2) click on `Disk` in the Overview of the azure VM; 3) click on the tab, `Additional Settings`; 4) selecting `yes` radio button under `Encryption at host`. ## Installing/Using This Module