Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: ECS EC2 Windows OTEL sidecar example/demonstration. #107

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions examples/ecs-ec2-windows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Demonstration/Example deployment of ECS EC2 Open Telemetry Agent for Windows

## Usage

To run this example you need to save this code in Terraform file, and change the values according to your settings.

For parameter details, see [ECS EC2 Windows demo module README](../../modules/ecs-ec2-windows/README.md)

```hcl
module "ecs_ec2_windows_demo" {
source = "../../modules/ecs-ec2-windows"
ecs_cluster_name = var.ecs_cluster_name
coralogix_region = "Singapore"
api_key = var.api_key
security_group_id = var.security_group_id
subnet_ids = var.subnet_ids
}
```

now execute:
```bash
$ terraform init
$ terraform plan
$ terraform apply
```

Run `terraform destroy` when you don't need these resources.
25 changes: 25 additions & 0 deletions examples/ecs-ec2-windows/ecs-ec2-windows.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
variable "api_key" {
type = string
sensitive = true
}

variable "ecs_cluster_name" {
type = string
}

variable "security_group_id" {
type = string
}

variable "subnet_ids" {
type = list
}

module "ecs_ec2_windows_demo" {
source = "../../modules/ecs-ec2-windows"
ecs_cluster_name = var.ecs_cluster_name
coralogix_region = "Singapore"
api_key = var.api_key
security_group_id = var.security_group_id
subnet_ids = var.subnet_ids
}
83 changes: 83 additions & 0 deletions modules/ecs-ec2-windows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Demonstration/Example deployment of ECS EC2 Open Telemetry Agent for Windows

Terraform module to launch an example ECS Service illustrating the deployment of a demo Windows container application alongside an [Coralogix Opentelemetry Collector](https://hub.docker.com/r/coralogixrepo/coralogix-otel-collector) running in a sidecar Windows container to collect logs and metrics from applications within the ECS Task, and forwarding to Coralogix. The defatult sample application logs to STDOUT every second, and you can replace this example your own container application image. This example requires deployment into existing AWS ECS Cluster on EC2 Windows container instances.

This example is intended for demonstration and instructional purposes only. It should not be deployed directly into production. The example may be customized to suit the user's requirements.

## Usage

Provisions a demonstration ECS Service having 1 Task consisting of 1 example Windows application and 1 Windows OTEL Collector as a sidecar.

```hcl
module "ecs_ec2_windows_demo" {
source = "../../modules/ecs-ec2-windows"
ecs_cluster_name = "ecs-cluster-name"
coralogix_region = "Europe"|"Europe2"|"India"|"Singapore"|"US"|"US2"
api_key = var.api_key
security_group_id = var.security_group_id
subnet_ids = var.subnet_ids
application_name = "Coralogix Application Name"
subsystem_name = "Coralogix Subsystem Name"
custom_domain = "[optional] custom Coralogix domain"
app_image = "[optional] User-provided demo App as a Windows container image, to demonstrate collection of console logs and metrics. If omitted, defaults to a provided sample Windows logging app."
otel_image = "[optional] Coralogix Open Telemetry distribution Windows image name and tag."
otel_config_file = "[optional] file path to custom OTEL collector config file"
}
```

#### Windows Sample Application
The default Windows sample application logs the following text every 1 second:
> "Hello from console writer. __N__"

Where __N__ is an incrementing number counter.

#### Verification
To verify successful deployment:
* Verify the logs are captured on your Coralogix logs console.
* Verify the metrics for the containers can be displayed on Coralogix Grafana.

#### Integrating the OTEL Collector into your application.

For logging to work, a Windows application should do either of the following:
1. Log to STDOUT. The OTEL Collector sidecar has been configured to collect docker container logs from the host.
2. Mount the log file volume to the OTEL Collector sidecar, and include the log location into the OTEL [filelog receiver configuration](./otel_ecs_ec2_win.config.yaml).
3. Instrument your application for OTEL, configured to export to the sidecar OTLP endpoints, which for 'awsvpc' networking, are at http://localhost:4317 and http://localhost:4318

## Requirements

| Name | Version |
|------|---------|
| terraform | ~> 1.6.0 |
| aws | >= 5.0.0 |

## Providers

| Name | Version |
|------|---------|
| aws | >= 5.0.0 |

## Resources

| Name | Type |
|------|------|
| [aws_ecs_service.demo_service](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) | resource |
| [aws_ecs_task_definition.demo_task_definition](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource |
| [aws_iam_role.ecsTaskExecutionRole](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy.ecs_awslogs_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| api\_key | The Send-Your-Data API key for your Coralogix account. See: https://coralogix.com/docs/send-your-data-api-key/ | `string` | n/a | yes |
| app\_image | Optional user-provided demo App as a Windows container image, to demonstrate collection of console logs and metrics. If omitted, defaults to a provided sample Windows logging app. | `string` | `""` | no |
| application\_name | Optional Application name as Coralogix metadata. | `string` | `"ECS-Windows-Demo"` | no |
| coralogix\_region | The region of the Coralogix endpoint domain: [Europe, Europe2, India, Singapore, US, US2, Custom]. If "Custom" then __custom\_domain__ parameter must be specified. | `string` | n/a | yes |
| custom\_domain | Optional Coralogix custom domain, e.g. "private.coralogix.com" Private Link domain. If specified, overrides the public domain corresponding to the __coralogix\_region__ parameter. | `string` | `null` | no |
| ecs\_cluster\_name | Name of the AWS ECS Cluster to deploy the demonstration ECS Service, consisting of 1 Coralogix OTEL Collector and 1 sample app as Windows containers in the task. Supports EC2 Windows instances only, not Fargate. | `string` | n/a | yes |
| otel\_config\_file | Optional file path to a custom opentelemetry configuration file. Defaults to an embedded configuration. | `string` | `null` | no |
| otel\_image | Optional Coralogix Open Telemetry distribution Windows image name and tag. | `string` | `"coralogixrepo/coralogix-otel-collector:0.1.0-windowsserver-1809"` | no |
| security\_group\_id | Security Group ID to deploy the ECS Service into. Must be in the same VPC as the ECS Cluster. | `string` | n/a | yes |
| subnet\_ids | List of subnet IDs to deploy the ECS Service into. Must be in the same VPC as the ECS Cluster. | `list` | n/a | yes |
| subsystem\_name | Optional Subsystem name as Coralogix metadata. | `string` | `"ECS-Windows-Demo"` | no |
114 changes: 114 additions & 0 deletions modules/ecs-ec2-windows/container_definitions.tftpl.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
[
%{ if app_image != "" }
{
"name" : "custom_demo_app",
"image" : "${app_image}",
"cpu" : 512,
"memory" : 1024,
"essential" : true,
"portMappings" : [
{
"containerPort" : 80,
"hostPort" : 80
}
],
"dependsOn": [
{
"containerName": "coralogix-otel-agent",
"condition": "HEALTHY"
}
]
},
%{else}
{
"name" : "demo_app",
"image" : "mcr.microsoft.com/windows/servercore:ltsc2019",
"cpu" : 512,
"memory" : 1024,
"essential" : true,
"entryPoint" : [
"Powershell",
"-Command"
],
"command" : [
"$count=1;while(1) { Write-Host \"Hello from console writer.\" $count; sleep 1; $count=$count+1;}"
],
"dependsOn": [
{
"containerName": "coralogix-otel-agent",
"condition": "HEALTHY"
}
]
},
%{ endif }
{
"name" : "coralogix-otel-agent",
"image" : "${otel_image}",
"essential" : true,
"cpu" : 512,
"memory" : 1024,
"mountPoints" : [
{
"sourceVolume" : "hostfs",
"containerPath" : "C:\\hostfs",
"readOnly" : true
}
],
"portMappings" : [
{
"containerPort" : 4317,
"hostPort" : 4317
},
{
"containerPort" : 4318,
"hostPort" : 4318
},
{
"containerPort" : 8888,
"hostPort" : 8888
},
{
"containerPort" : 13133,
"hostPort" : 13133
}
],
"environment" : [
{
"name" : "CORALOGIX_DOMAIN",
"value" : "${coralogix_domain}"
},
{
"name" : "PRIVATE_KEY",
"value" : "${api_key}"
},
{
"name" : "APP_NAME",
"value" : "${application_name}"
},
{
"name" : "SUB_SYS",
"value" : "${subsystem_name}"
},
{
"name" : "OTEL_CONFIG",
"value" : ${jsonencode(otel_config)}
}
],
"healthCheck" : {
"command" : ["CMD-SHELL", "curl localhost:13133"],
"startPeriod" : 60,
"interval" : 30,
"timeout" : 5,
"retries" : 5
},
"logConfiguration" : {
"logDriver" : "awslogs",
"options" : {
"awslogs-group" : "/ecs/coralogix-otel-agent",
"awslogs-region" : "${region}",
"awslogs-stream-prefix" : "coralogix-otel-agent",
"awslogs-create-group" : "true"
}
}
}
]
112 changes: 112 additions & 0 deletions modules/ecs-ec2-windows/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
data "aws_region" "current" {}

locals {
name = "coralogix-monitor-demo"
coralogix_region_domain_map = {
"europe" = "coralogix.com"
"europe2" = "eu2.coralogix.com"
"india" = "coralogix.in"
"singapore" = "coralogixsg.com"
"us" = "coralogix.us"
"us2" = "cx498.coralogix.com"
"custom" = null
}
coralogix_domain = coalesce(var.custom_domain, local.coralogix_region_domain_map[lower(var.coralogix_region)])
otel_config_file_path = coalesce(var.otel_config_file, "${path.module}/otel_ecs_ec2_win.config.yaml")
otel_config = templatefile(local.otel_config_file_path, {})
}

resource "aws_iam_role" "ecsTaskExecutionRole" {
name = "${local.name}-ecs-task-execution-role"
path = "/"
managed_policy_arns = [
"arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
]
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Sid = ""
Effect = "Allow"
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
},
]
})
}

resource "aws_iam_role_policy" "ecs_awslogs_policy" {
name = "${local.name}-ecs-awslogs-policy"
role = aws_iam_role.ecsTaskExecutionRole.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow",
Action = [
"logs:CreateLogStream",
"logs:CreateLogGroup",
"logs:DescribeLogStreams",
"logs:PutLogEvents"
],
Resource = "arn:aws:logs:*:*:log-group:/ecs/*"
},
]
})
}

resource "aws_ecs_task_definition" "demo_task_definition" {
family = "${local.name}-ec2-windows"
execution_role_arn = aws_iam_role.ecsTaskExecutionRole.arn
task_role_arn = aws_iam_role.ecsTaskExecutionRole.arn
cpu = 1024
memory = 2048
requires_compatibilities = ["EC2"]
network_mode = "awsvpc"
volume {
name = "hostfs"
host_path = "C:\\"
}
tags = {
"ecs:taskDefinition:createdFrom" = "terraform"
}
container_definitions = templatefile("${path.module}/container_definitions.tftpl.json",{
region = data.aws_region.current.name
otel_image = var.otel_image
app_image = var.app_image
coralogix_domain = local.coralogix_domain,
application_name = var.application_name,
subsystem_name = var.subsystem_name,
api_key = var.api_key
otel_config = local.otel_config
}
)
}

resource "aws_ecs_service" "demo_service" {
name = "${local.name}-service"
cluster = var.ecs_cluster_name
launch_type = "EC2"
task_definition = aws_ecs_task_definition.demo_task_definition.arn
scheduling_strategy = "REPLICA"
deployment_maximum_percent = 100
deployment_minimum_healthy_percent = 0
desired_count = 1
enable_ecs_managed_tags = true
deployment_circuit_breaker {
enable = true
rollback = true
}
deployment_controller {
type = "ECS"
}
service_connect_configuration {
enabled = false
}
network_configuration {
subnets = var.subnet_ids
security_groups = [var.security_group_id]
}
}
Loading