Skip to content

Commit d950902

Browse files
irur2bit
iru
and
r2bit
authored
feat: parametrize ecs-task cpu/memory limits + aws module 4.0 bump changes (#64)
* feat(connector): add ability to set CPU and mempry limits for ECS task * chore: expose ecs cpu/mem variables to examples * chore: upgrade bucket acl/lifecycle/versioning usage to 4.0 * chore: testing minor changes Co-authored-by: Janar K <[email protected]>
1 parent f7774a2 commit d950902

File tree

21 files changed

+109
-31
lines changed

21 files changed

+109
-31
lines changed

‎README.md

+4
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,10 @@ It may take some time, but you should see logs detecting the new image in the EC
144144

145145
## Troubleshooting
146146

147+
### Q: Getting error "Error: failed creating ECS Task Definition: ClientException: No Fargate configuration exists for given values.
148+
A: Your ECS task_size values aren't valid for Fargate. Specifically, your mem_limit value is too big for the cpu_limit you specified
149+
S: Check [supported task cpu and memory values](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html)
150+
147151
### Q: Getting error "404 Invalid parameter: TopicArn" when trying to reuse an existing cloudtrail-sns
148152

149153
```text

‎examples-internal/organizational-k8s-threat-reuse_cloudtrail_s3/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ Notice that:
8181

8282
| Name | Version |
8383
|------|---------|
84-
| <a name="provider_aws"></a> [aws](#provider\_aws) | 3.74.1 |
84+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.0.0 |
8585
| <a name="provider_helm"></a> [helm](#provider\_helm) | 2.4.1 |
8686

8787
## Modules

‎examples/organizational/README.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -74,15 +74,15 @@ Notice that:
7474
| Name | Version |
7575
|------|---------|
7676
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
77-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.62.0 |
77+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
7878
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.29 |
7979

8080
## Providers
8181

8282
| Name | Version |
8383
|------|---------|
84-
| <a name="provider_aws"></a> [aws](#provider\_aws) | 3.74.1 |
85-
| <a name="provider_aws.member"></a> [aws.member](#provider\_aws.member) | 3.74.1 |
84+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.0.0 |
85+
| <a name="provider_aws.member"></a> [aws.member](#provider\_aws.member) | 4.0.0 |
8686

8787
## Modules
8888

@@ -122,6 +122,8 @@ Notice that:
122122
| <a name="input_deploy_image_scanning_ecr"></a> [deploy\_image\_scanning\_ecr](#input\_deploy\_image\_scanning\_ecr) | true/false whether to deploy the image scanning on ECR pushed images | `bool` | `true` | no |
123123
| <a name="input_deploy_image_scanning_ecs"></a> [deploy\_image\_scanning\_ecs](#input\_deploy\_image\_scanning\_ecs) | true/false whether to deploy the image scanning on ECS running images | `bool` | `true` | no |
124124
| <a name="input_ecs_cluster_name"></a> [ecs\_cluster\_name](#input\_ecs\_cluster\_name) | Name of a pre-existing ECS (elastic container service) cluster. If defaulted, a new ECS cluster/VPC/Security Group will be created. For both options, ECS location will/must be within the `sysdig_secure_for_cloud_member_account_id` parameter accountID | `string` | `"create"` | no |
125+
| <a name="input_ecs_task_cpu"></a> [ecs\_task\_cpu](#input\_ecs\_task\_cpu) | Amount of CPU (in CPU units) to reserve for cloud-connector task | `string` | `"256"` | no |
126+
| <a name="input_ecs_task_memory"></a> [ecs\_task\_memory](#input\_ecs\_task\_memory) | Amount of memory (in megabytes) to reserve for cloud-connector task | `string` | `"512"` | no |
125127
| <a name="input_ecs_vpc_id"></a> [ecs\_vpc\_id](#input\_ecs\_vpc\_id) | ID of the VPC where the workload is to be deployed. Defaulted to be created when `ecs_cluster_name is not provided.` | `string` | `"create"` | no |
126128
| <a name="input_ecs_vpc_region_azs"></a> [ecs\_vpc\_region\_azs](#input\_ecs\_vpc\_region\_azs) | List of Availability Zones for ECS VPC creation. e.g.: ["apne1-az1", "apne1-az2"]. If defaulted, two of the default 'aws\_availability\_zones' datasource will be taken | `list(string)` | `[]` | no |
127129
| <a name="input_ecs_vpc_subnets_private_ids"></a> [ecs\_vpc\_subnets\_private\_ids](#input\_ecs\_vpc\_subnets\_private\_ids) | List of VPC subnets where workload is to be deployed. Defaulted to be created when `ecs_cluster_name is not provided.` | `list(string)` | `[]` | no |

‎examples/organizational/main.tf

+2
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,8 @@ module "cloud_connector" {
8080
ecs_cluster_name = local.ecs_cluster_name
8181
ecs_vpc_id = local.ecs_vpc_id
8282
ecs_vpc_subnets_private_ids = local.ecs_vpc_subnets_private_ids
83+
ecs_task_cpu = var.ecs_task_cpu
84+
ecs_task_memory = var.ecs_task_memory
8385

8486
tags = var.tags
8587
depends_on = [local.cloudtrail_sns_arn, module.ssm]

‎examples/organizational/variables.tf

+14
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,20 @@ variable "ecs_vpc_region_azs" {
120120
default = []
121121
}
122122

123+
# Configure CPU and memory in pairs.
124+
# See: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size
125+
variable "ecs_task_cpu" {
126+
type = string
127+
description = "Amount of CPU (in CPU units) to reserve for cloud-connector task"
128+
default = "256"
129+
}
130+
131+
variable "ecs_task_memory" {
132+
type = string
133+
description = "Amount of memory (in megabytes) to reserve for cloud-connector task"
134+
default = "512"
135+
}
136+
123137

124138

125139
#

‎examples/organizational/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ terraform {
22
required_version = ">= 0.15.0"
33
required_providers {
44
aws = {
5-
version = ">= 3.62.0"
5+
version = ">= 4.0.0"
66
}
77
sysdig = {
88
source = "sysdiglabs/sysdig"

‎examples/single-account-k8s/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ Notice that:
7272

7373
| Name | Version |
7474
|------|---------|
75-
| <a name="provider_aws"></a> [aws](#provider\_aws) | 3.74.1 |
75+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.0.0 |
7676
| <a name="provider_helm"></a> [helm](#provider\_helm) | 2.4.1 |
7777

7878
## Modules

‎examples/single-account/README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Notice that:
5050
| Name | Version |
5151
|------|---------|
5252
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
53-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.62.0 |
53+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
5454
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.29 |
5555

5656
## Providers
@@ -86,6 +86,8 @@ No resources.
8686
| <a name="input_deploy_image_scanning_ecr"></a> [deploy\_image\_scanning\_ecr](#input\_deploy\_image\_scanning\_ecr) | true/false whether to deploy the image scanning on ECR pushed images | `bool` | `true` | no |
8787
| <a name="input_deploy_image_scanning_ecs"></a> [deploy\_image\_scanning\_ecs](#input\_deploy\_image\_scanning\_ecs) | true/false whether to deploy the image scanning on ECS running images | `bool` | `true` | no |
8888
| <a name="input_ecs_cluster_name"></a> [ecs\_cluster\_name](#input\_ecs\_cluster\_name) | Name of a pre-existing ECS (elastic container service) cluster. If defaulted, a new ECS cluster/VPC/Security Group will be created | `string` | `"create"` | no |
89+
| <a name="input_ecs_task_cpu"></a> [ecs\_task\_cpu](#input\_ecs\_task\_cpu) | Amount of CPU (in CPU units) to reserve for cloud-connector task | `string` | `"256"` | no |
90+
| <a name="input_ecs_task_memory"></a> [ecs\_task\_memory](#input\_ecs\_task\_memory) | Amount of memory (in megabytes) to reserve for cloud-connector task | `string` | `"512"` | no |
8991
| <a name="input_ecs_vpc_id"></a> [ecs\_vpc\_id](#input\_ecs\_vpc\_id) | ID of the VPC where the workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `string` | `"create"` | no |
9092
| <a name="input_ecs_vpc_region_azs"></a> [ecs\_vpc\_region\_azs](#input\_ecs\_vpc\_region\_azs) | List of Availability Zones for ECS VPC creation. e.g.: ["apne1-az1", "apne1-az2"]. If defaulted, two of the default 'aws\_availability\_zones' datasource will be taken | `list(string)` | `[]` | no |
9193
| <a name="input_ecs_vpc_subnets_private_ids"></a> [ecs\_vpc\_subnets\_private\_ids](#input\_ecs\_vpc\_subnets\_private\_ids) | List of VPC subnets where workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `list(string)` | `[]` | no |

‎examples/single-account/main.tf

+2-1
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,8 @@ module "cloud_connector" {
4949
ecs_cluster_name = local.ecs_cluster_name
5050
ecs_vpc_id = local.ecs_vpc_id
5151
ecs_vpc_subnets_private_ids = local.ecs_vpc_subnets_private_ids
52-
52+
ecs_task_cpu = var.ecs_task_cpu
53+
ecs_task_memory = var.ecs_task_memory
5354

5455
tags = var.tags
5556
depends_on = [local.cloudtrail_sns_arn, module.ssm]

‎examples/single-account/variables.tf

+13
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,19 @@ variable "ecs_vpc_region_azs" {
6060
default = []
6161
}
6262

63+
# Configure CPU and memory in pairs.
64+
# See: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size
65+
variable "ecs_task_cpu" {
66+
type = string
67+
description = "Amount of CPU (in CPU units) to reserve for cloud-connector task"
68+
default = "256"
69+
}
70+
71+
variable "ecs_task_memory" {
72+
type = string
73+
description = "Amount of memory (in megabytes) to reserve for cloud-connector task"
74+
default = "512"
75+
}
6376

6477
#
6578
# scanning configuration

‎examples/single-account/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ terraform {
22
required_version = ">= 0.15.0"
33
required_providers {
44
aws = {
5-
version = ">= 3.62.0"
5+
version = ">= 4.0.0"
66
}
77
sysdig = {
88
source = "sysdiglabs/sysdig"

‎examples/trigger-events/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Notice that:
4747

4848
| Name | Version |
4949
|------|---------|
50-
| <a name="provider_aws"></a> [aws](#provider\_aws) | 3.74.1 |
50+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.0.0 |
5151

5252
## Modules
5353

‎modules/infrastructure/cloudtrail/README.md

+4-2
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@
66
| Name | Version |
77
|------|---------|
88
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
9-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.50.0 |
9+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
1010

1111
## Providers
1212

1313
| Name | Version |
1414
|------|---------|
15-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.50.0 |
15+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0.0 |
1616

1717
## Modules
1818

@@ -26,6 +26,8 @@ No modules.
2626
| [aws_kms_alias.kms](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_alias) | resource |
2727
| [aws_kms_key.cloudtrail_kms](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
2828
| [aws_s3_bucket.cloudtrail](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
29+
| [aws_s3_bucket_acl.cloudtrail](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
30+
| [aws_s3_bucket_lifecycle_configuration.cloudtrail](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration) | resource |
2931
| [aws_s3_bucket_policy.cloudtrail_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) | resource |
3032
| [aws_s3_bucket_public_access_block.cloudtrail](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
3133
| [aws_sns_topic.cloudtrail](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) | resource |

‎modules/infrastructure/cloudtrail/s3.tf

+13-4
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,27 @@
11
resource "aws_s3_bucket" "cloudtrail" {
22
bucket = "${var.name}-${data.aws_caller_identity.me.account_id}"
3-
acl = "private"
43
force_destroy = true
4+
tags = var.tags
5+
}
6+
7+
resource "aws_s3_bucket_lifecycle_configuration" "cloudtrail" {
8+
bucket = aws_s3_bucket.cloudtrail.id
59

6-
lifecycle_rule {
7-
enabled = true
10+
rule {
11+
id = "expire in ${var.s3_bucket_expiration_days} days"
12+
status = "Enabled"
813
expiration {
914
days = var.s3_bucket_expiration_days
1015
}
1116
}
12-
tags = var.tags
1317
}
1418

1519

20+
resource "aws_s3_bucket_acl" "cloudtrail" {
21+
bucket = aws_s3_bucket.cloudtrail.id
22+
acl = "private"
23+
}
24+
1625

1726
# --------------------------
1827
# iam, acl

‎modules/infrastructure/cloudtrail/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ terraform {
22
required_version = ">= 0.15.0"
33
required_providers {
44
aws = {
5-
version = ">= 3.50.0"
5+
version = ">= 4.0.0"
66
}
77
}
88
}

‎modules/services/cloud-connector/README.md

+7-3
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,13 @@ A task deployed on an **ECS deployment** will detect events in your infrastructu
88
| Name | Version |
99
|------|---------|
1010
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
11-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.50.0 |
11+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
1212

1313
## Providers
1414

1515
| Name | Version |
1616
|------|---------|
17-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.50.0 |
17+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0.0 |
1818

1919
## Modules
2020

@@ -39,8 +39,10 @@ A task deployed on an **ECS deployment** will detect events in your infrastructu
3939
| [aws_iam_role_policy.task_read_parameters](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource |
4040
| [aws_iam_role_policy.trigger_scan](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource |
4141
| [aws_s3_bucket.s3_config_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
42-
| [aws_s3_bucket_object.config](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object) | resource |
42+
| [aws_s3_bucket_acl.s3_config_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
4343
| [aws_s3_bucket_public_access_block.s3_config_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
44+
| [aws_s3_bucket_versioning.s3_config_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) | resource |
45+
| [aws_s3_object.config](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_object) | resource |
4446
| [aws_security_group.sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
4547
| [aws_caller_identity.me](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
4648
| [aws_ecs_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ecs_cluster) | data source |
@@ -71,6 +73,8 @@ A task deployed on an **ECS deployment** will detect events in your infrastructu
7173
| <a name="input_connector_ecs_task_role_name"></a> [connector\_ecs\_task\_role\_name](#input\_connector\_ecs\_task\_role\_name) | Default ecs cloudconnector task role name | `string` | `"ECSTaskRole"` | no |
7274
| <a name="input_deploy_image_scanning_ecr"></a> [deploy\_image\_scanning\_ecr](#input\_deploy\_image\_scanning\_ecr) | true/false whether to deploy the image scanning on ECR pushed images | `bool` | `true` | no |
7375
| <a name="input_deploy_image_scanning_ecs"></a> [deploy\_image\_scanning\_ecs](#input\_deploy\_image\_scanning\_ecs) | true/false whether to deploy the image scanning on ECS running images | `bool` | `true` | no |
76+
| <a name="input_ecs_task_cpu"></a> [ecs\_task\_cpu](#input\_ecs\_task\_cpu) | Amount of CPU (in CPU units) to reserve for cloud-connector task | `string` | `"256"` | no |
77+
| <a name="input_ecs_task_memory"></a> [ecs\_task\_memory](#input\_ecs\_task\_memory) | Amount of memory (in megabytes) to reserve for cloud-connector task | `string` | `"512"` | no |
7478
| <a name="input_extra_env_vars"></a> [extra\_env\_vars](#input\_extra\_env\_vars) | Extra environment variables for the Cloud Connector deployment | `map(string)` | `{}` | no |
7579
| <a name="input_image"></a> [image](#input\_image) | Image of the cloud connector to deploy | `string` | `"quay.io/sysdig/cloud-connector:latest"` | no |
7680
| <a name="input_is_organizational"></a> [is\_organizational](#input\_is\_organizational) | whether secure-for-cloud should be deployed in an organizational setup | `bool` | `false` | no |

‎modules/services/cloud-connector/ecs-service.tf

+2-2
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,8 @@ resource "aws_ecs_task_definition" "task_definition" {
2424
network_mode = "awsvpc"
2525
execution_role_arn = aws_iam_role.execution.arn # ARN of the task execution role that the Amazon ECS container agent and the Docker daemon can assume
2626
task_role_arn = local.ecs_task_role_arn # ARN of IAM role that allows your Amazon ECS container task to make calls to other AWS resource-group.
27-
cpu = "256"
28-
memory = "512"
27+
cpu = var.ecs_task_cpu
28+
memory = var.ecs_task_memory
2929

3030
container_definitions = jsonencode([
3131
{

‎modules/services/cloud-connector/s3-config.tf

+4-3
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@ locals {
22
s3_bucket_config_id = aws_s3_bucket.s3_config_bucket.id
33
}
44

5-
resource "aws_s3_bucket_object" "config" {
6-
bucket = local.s3_bucket_config_id
7-
key = "cloud-connector.yaml"
5+
resource "aws_s3_object" "config" {
6+
bucket = local.s3_bucket_config_id
7+
key = "cloud-connector.yaml"
8+
89
content = local.default_config
910
tags = var.tags
1011
}

‎modules/services/cloud-connector/s3.tf

+14-4
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,22 @@ data "aws_caller_identity" "me" {}
22

33
resource "aws_s3_bucket" "s3_config_bucket" {
44
bucket = "${var.name}-${data.aws_caller_identity.me.account_id}-config"
5-
acl = "private"
65
force_destroy = true
7-
versioning {
8-
enabled = true
6+
tags = var.tags
7+
}
8+
9+
10+
resource "aws_s3_bucket_acl" "s3_config_bucket" {
11+
bucket = aws_s3_bucket.s3_config_bucket.id
12+
acl = "private"
13+
}
14+
15+
16+
resource "aws_s3_bucket_versioning" "s3_config_bucket" {
17+
bucket = aws_s3_bucket.s3_config_bucket.id
18+
versioning_configuration {
19+
status = "Enabled"
920
}
10-
tags = var.tags
1121
}
1222

1323

‎modules/services/cloud-connector/variables.tf

+15-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,6 @@ variable "sns_topic_arn" {
4747
# optionals - with default
4848
#---------------------------------
4949

50-
5150
#
5251
# module composition
5352
#
@@ -84,6 +83,21 @@ variable "organizational_config" {
8483
#
8584
# module config
8685
#
86+
87+
# Configure CPU and memory in pairs.
88+
# See: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size
89+
variable "ecs_task_cpu" {
90+
type = string
91+
description = "Amount of CPU (in CPU units) to reserve for cloud-connector task"
92+
default = "256"
93+
}
94+
95+
variable "ecs_task_memory" {
96+
type = string
97+
description = "Amount of memory (in megabytes) to reserve for cloud-connector task"
98+
default = "512"
99+
}
100+
87101
variable "connector_ecs_task_role_name" {
88102
type = string
89103
default = "ECSTaskRole"

‎modules/services/cloud-connector/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ terraform {
22
required_version = ">= 0.15.0"
33
required_providers {
44
aws = {
5-
version = ">= 3.50.0"
5+
version = ">= 4.0.0"
66
}
77
}
88
}

0 commit comments

Comments
 (0)