Add database provisioner, ALB and bastion host (#28)

- Add database provisioner, currently support H2 and rds-postgresql.
- Add an AWS ALB so that we can access the SkyWalking UI via ALB,
  instead of doing port forward.
- For security reason, add a bastion host and all operations to OAP and
  UI instances are done via bastion host as a proxy.
- Simplify some command line options by adding them into files.
- Generate Terraform docs for inputs, outputs, resources, etc.

Author: kezhenxu94

.gitignore

@@ -18,3 +18,4 @@ aws/terraform.tfstate.backup
 ansible/local.var.yaml
 ansible/inventory
 !ansible/inventory/template
+.terraform.tfstate.lock.info

README.md

@@ -42,16 +42,7 @@ terraform init
 
 The script is designed with modularity and reusability in mind. Various parameters like region, instance count, instance type, etc., are exposed as variables for easier customization.
 
-#### Variables:
-
-| Variable Name       | Description                                          | Default Value               |
-|---------------------|------------------------------------------------------|-----------------------------|
-| `oap_instance_count`| Number of SkyWalking OAP instances                   | `1`                         |
-| `ui_instance_count` | Number of SkyWalking UI instances                    | `1`                         |
-| `region`            | AWS region where resources will be provisioned       | `us-east-1`                 |
-| `instance_type`     | AWS instance type for SkyWalking OAP and UI          | `t2.medium`                 |
-| `public_key_path`   | Path where the SSH key for instances will be stored  | `~/.ssh`                    |
-| `extra_tags`        | Additional tags that can be applied to all resources | `{}`                        |
+For the full configuration list, please refer to [the doc](/aws/README.md).
 
 To modify the default values, you can create a `terraform.tfvars` file in the same directory as your Terraform script:
 
@@ -60,7 +51,6 @@ oap_instance_count = 2
 ui_instance_count  = 2
 region             = "us-west-1"
 instance_type      = "t2.large"
-public_key_path    = "/path/to/your/desired/location"
 extra_tags         = {
   "Environment" = "Production"
 }
@@ -75,17 +65,24 @@ terraform plan
 terraform apply
 ```
 
+After all the resources are created, you can head to the
+[Ansible part](#ansible) to start deploying SkyWalking.
+
 ### 4. Accessing the Resources
 
-Once the resources are created:
+#### SSH into bastion host (Optional)
 
-- **SkyWalking OAP and UI instances**: You can SSH into the instances using the generated key pair. The public IPs of these instances are stored in local files (`oap-server` and `ui-server` respectively) under the `ansible/inventory/` directory, relative to the module's path.
+You don't usually need to SSH into the bastion host, but if you want to, you can
+SSH into the bastion host with the command:
 
-```bash
-ssh -i /path/to/skywalking.pem ec2-user@<INSTANCE_PUBLIC_IP>
+```shell
+KEY_FILE=$(terraform output -raw ssh-user-key-file)
+BASTION_IP=$(terraform output -json bastion_ips | jq -r '.[0]')
+
+ssh -i "$KEY_FILE" ec2-user@"$BASTION_IP"
 ```
 
-- **Security Groups**: Two security groups are created:
+- **Security Attention**: two security rules are created for the bastion host:
   - `ssh-access`: Allows SSH access from any IP (`0.0.0.0/0`). **Please note** that this is potentially insecure and you should restrict the IP range wherever possible.
   - `public-egress-access`: Allows egress access to the internet for the instances.
 
@@ -118,26 +115,18 @@ This guide provides steps on using Ansible to install Apache SkyWalking on AWS i
 
 ## Instructions
 
-### 1. Change diroectory and set the SSH Key File Path
+### 1. Change diroectory
 
-Save the SSH key file path generated by Terraform to a variable for future use:
-
-```
+```shell
 cd ../ansible/
-SSH_KEY_FILE=$(terraform -chdir=../aws output -raw ssh-user-key-file)
-echo $SSH_KEY_FILE
 ```
 
-**Expected Output**:
-
-You should see a file path similar to: `/Users/kezhenxu94/.ssh/skywalking.pem`
-
 ### 2. Test Connectivity to the EC2 Instances
 
 Before installing SkyWalking, ensure that you can connect to the EC2 instances:
 
 ```
-ANSIBLE_HOST_KEY_CHECKING=False ansible -m ping all -u ec2-user --private-key "$SSH_KEY_FILE"
+ansible -m ping all -u ec2-user
 ```
 
 **Expected Output**:
@@ -165,7 +154,7 @@ You should see output for each IP with a `SUCCESS` status:
 After confirming connectivity, proceed to install Apache SkyWalking using the Ansible playbook:
 
 ```
-ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u ec2-user --private-key "$SSH_KEY_FILE" playbooks/install-skywalking.yml
+ansible-playbook -u ec2-user playbooks/install-skywalking.yml -i inventory/skywalking.yaml
 ```
 
 ### 4. Configurations
@@ -196,3 +185,16 @@ skywalking_ui_environment: {}
 skywalking_oap_environment: {}
 
 ```
+
+### 5. Accessing SkyWalking UI!
+
+After the installation is complete, you can go back to the aws folder and get
+the ALB domain name address that can be used to  access the SkyWalking UI:
+
+```shell
+cd ../aws
+terraform output -raw alb_dns_name
+```
+
+And you can open your browser and access the SkyWalking UI with the address.
+

ansible/inventory/template/skywalking.yaml.tftpl

@@ -16,21 +16,35 @@
 # specific language governing permissions and limitations
 # under the License.
 #
+
+proxy:
+  ${bastion.public_ip}:
+
 skywalking:
+  vars:
+    ansible_ssh_private_key_file: ${private_key_file}
+    ansible_ssh_user: ec2-user
+    ansible_ssh_common_args: '-o StrictHostKeyChecking=no -o ProxyCommand="ssh -i ${private_key_file} -o StrictHostKeyChecking=no -W %h:%p -q ec2-user@${bastion.public_ip}"'
   children:
     skywalking_oap:
     skywalking_ui:
 
 skywalking_oap:
   hosts:
 %{ for oap in oap_instances ~}
-    ${oap.public_ip}:
-      private_ip: ${oap.private_ip}
+    ${oap.private_ip}:
 %{ endfor ~}
+  vars:
+    database:
+      type: ${database_type}
+      host: ${database_host}
+      port: ${database_port}
+      name: ${database_name}
+      user: ${database_user}
+      password: ${database_password}
 
 skywalking_ui:
   hosts:
 %{ for ui in ui_instances ~}
-    ${ui.public_ip}:
-      private_ip: ${ui.private_ip}
+    ${ui.private_ip}:
 %{ endfor ~}

ansible/roles/skywalking/tasks/main.yml

@@ -52,15 +52,15 @@
     src: skywalking-ui.env.j2
     dest: /home/skywalking/webapp.env
     owner: skywalking
-    mode: "0660"
+    mode: "0600"
   when: inventory_hostname in groups['skywalking_ui']
 
 - name: Generate environment file for OAP service
   template:
     src: skywalking-oap.env.j2
     dest: /home/skywalking/oap.env
     owner: skywalking
-    mode: "0660"
+    mode: "0600"
   when: inventory_hostname in groups['skywalking_oap']
 
 - name: Check hostgroup size
@@ -69,7 +69,7 @@
     oap_init_node: "{{ [groups['skywalking_oap'][0]] }}"
 
 - name: Run the OAPSericeInit script
-  command: "sudo -u skywalking /usr/local/skywalking/bin/oapServiceInit.sh" 
+  command: "sudo -u skywalking -- sh -c 'set -a; source /home/skywalking/oap.env; set +a; /usr/local/skywalking/bin/oapServiceInit.sh'"
   when: inventory_hostname in oap_init_node
 
 - name: Generate systemd unit file for oap service

ansible/roles/skywalking/templates/skywalking-oap.env.j2

@@ -16,6 +16,20 @@
 # specific language governing permissions and limitations
 # under the License.
 #
+
+{% set database = hostvars[inventory_hostname]["database"] %}
+{% set storage = database['type'] %}
+
+{% if storage and (storage | length) %}
+SW_STORAGE={{ storage | regex_replace('^rds-', '')}}
+{% endif %}
+
+{% if "postgresql" in storage %}
+SW_JDBC_URL=jdbc:postgresql://{{ database["host"] }}:{{ database["port"] }}/{{ database["name"] }}
+SW_DATA_SOURCE_USER={{ database['user'] }}
+SW_DATA_SOURCE_PASSWORD={{ database['password'] }}
+{% endif %}
+
 {% for key, value in skywalking_oap_environment.items() %}
 {{ key }}="{{ value }}"
 {% endfor %}

ansible/roles/skywalking/templates/skywalking-oap.service.j2

@@ -21,7 +21,8 @@ After=network.target
 Type=simple
 User=skywalking
 Group=skywalking
-ExecStart=/usr/local/skywalking/bin/oapService.sh
+EnvironmentFile=/home/skywalking/oap.env
+ExecStart=/usr/local/skywalking/bin/oapServiceNoInit.sh
 TimeoutSec=300
 KillMode=process
 ExecReload=/bin/kill -HUP $MAINPID

ansible/roles/skywalking/templates/skywalking-ui.env.j2

@@ -20,6 +20,6 @@
 {{ key }}="{{ value }}"
 {% endfor %}
 
-SW_OAP_ADDRESS="{% for host in groups['skywalking_oap'] %}http://{{ hostvars[host].private_ip }}:{{ skywalking_ui_environment['SW_CORE_GRPC_PORT'] | default ('12800') }}{% if not loop.last %},{% endif %}{% endfor %}"
-SW_ZIPKIN_ADDRESS="{% for host in groups['skywalking_oap'] %}http://{{ hostvars[host].private_ip }}:{{ skywalking_ui_environment['SW_QUERY_ZIPKIN_REST_PORT'] | default ('9412') }}{% if not loop.last %},{% endif %}{% endfor %}"
+SW_OAP_ADDRESS="{% for host in groups['skywalking_oap'] %}http://{{ hostvars[host].inventory_hostname }}:{{ skywalking_ui_environment['SW_CORE_GRPC_PORT'] | default ('12800') }}{% if not loop.last %},{% endif %}{% endfor %}"
+SW_ZIPKIN_ADDRESS="{% for host in groups['skywalking_oap'] %}http://{{ hostvars[host].inventory_hostname }}:{{ skywalking_ui_environment['SW_QUERY_ZIPKIN_REST_PORT'] | default ('9412') }}{% if not loop.last %},{% endif %}{% endfor %}"
 

aws/README.md

@@ -0,0 +1,84 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.10.0 |
+| <a name="provider_local"></a> [local](#provider\_local) | 2.4.0 |
+| <a name="provider_random"></a> [random](#provider\_random) | 3.5.1 |
+| <a name="provider_tls"></a> [tls](#provider\_tls) | 4.0.4 |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_alb"></a> [alb](#module\_alb) | terraform-aws-modules/alb/aws | ~> 8.0 |
+| <a name="module_rds"></a> [rds](#module\_rds) | terraform-aws-modules/rds/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [aws_instance.bastion](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) | resource |
+| [aws_instance.skywalking-oap](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) | resource |
+| [aws_instance.skywalking-ui](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) | resource |
+| [aws_key_pair.ssh-user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/key_pair) | resource |
+| [aws_security_group.allow_apps](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group.bastion](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group.public-egress-access](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group.skywalking-oap](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group.skywalking-ui](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [local_file.inventories](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
+| [local_file.ssh-user](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
+| [random_password.rds_password](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource |
+| [tls_private_key.ssh-user](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) | resource |
+| [aws_ami.amazon-linux](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
+| [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_access_key"></a> [access\_key](#input\_access\_key) | Access key of the AWS account, if you have configured AWS CLI, you can leave it empty. | `string` | `""` | no |
+| <a name="input_bastion_enabled"></a> [bastion\_enabled](#input\_bastion\_enabled) | Enable bastion host, if you want to access the instances via SSH, you must enable it. | `bool` | `true` | no |
+| <a name="input_bastion_instance_type"></a> [bastion\_instance\_type](#input\_bastion\_instance\_type) | CPU, memory, storage and networking capacity for bastion host | `string` | `"t2.micro"` | no |
+| <a name="input_cidr"></a> [cidr](#input\_cidr) | CIDR for database tier | `string` | `"11.0.0.0/16"` | no |
+| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | Name of the cluster | `string` | `"skywalking-cluster"` | no |
+| <a name="input_database_subnets"></a> [database\_subnets](#input\_database\_subnets) | CIDR used for database subnets | `set(string)` | <pre>[<br>  "11.0.104.0/24",<br>  "11.0.105.0/24",<br>  "11.0.106.0/24"<br>]</pre> | no |
+| <a name="input_db_instance_class"></a> [db\_instance\_class](#input\_db\_instance\_class) | Instance class for the database | `string` | `"db.t3.medium"` | no |
+| <a name="input_db_max_storage_size"></a> [db\_max\_storage\_size](#input\_db\_max\_storage\_size) | Maximum storage size for the database, in GB | `number` | `100` | no |
+| <a name="input_db_name"></a> [db\_name](#input\_db\_name) | Name of the database | `string` | `"skywalking"` | no |
+| <a name="input_db_password"></a> [db\_password](#input\_db\_password) | Password for the database, if not set, a random password will be generated. | `string` | `null` | no |
+| <a name="input_db_storage_size"></a> [db\_storage\_size](#input\_db\_storage\_size) | Storage size for the database, in GB | `number` | `5` | no |
+| <a name="input_db_username"></a> [db\_username](#input\_db\_username) | Username for the database | `string` | `"skywalking"` | no |
+| <a name="input_extra_tags"></a> [extra\_tags](#input\_extra\_tags) | Additional tags to be added to all resources | `map(string)` | `{}` | no |
+| <a name="input_instance_type"></a> [instance\_type](#input\_instance\_type) | CPU, memory, storage and networking capacity for OAP and UI instances | `string` | `"t2.medium"` | no |
+| <a name="input_oap_instance_count"></a> [oap\_instance\_count](#input\_oap\_instance\_count) | Number of OAP instances, if you want to use H2 storage, you must set it to 1. | `number` | `1` | no |
+| <a name="input_private_subnets"></a> [private\_subnets](#input\_private\_subnets) | CIDR used for private subnets | `set(string)` | <pre>[<br>  "11.0.1.0/24",<br>  "11.0.2.0/24",<br>  "11.0.3.0/24"<br>]</pre> | no |
+| <a name="input_public_key_path"></a> [public\_key\_path](#input\_public\_key\_path) | Path to store the key file for SSH access to the instances. | `string` | `"~/.ssh"` | no |
+| <a name="input_public_subnets"></a> [public\_subnets](#input\_public\_subnets) | CIDR used for public subnets | `set(string)` | <pre>[<br>  "11.0.101.0/24",<br>  "11.0.102.0/24",<br>  "11.0.103.0/24"<br>]</pre> | no |
+| <a name="input_region"></a> [region](#input\_region) | Physical location for clustered data centers. | `string` | `"us-east-1"` | no |
+| <a name="input_secret_key"></a> [secret\_key](#input\_secret\_key) | Secret key of the AWS account, if you have configured AWS CLI, you can leave it empty. | `string` | `""` | no |
+| <a name="input_storage"></a> [storage](#input\_storage) | Storage type for SkyWalking OAP, can be 'h2', or 'rds-postgresql' | `string` | `"rds-postgresql"` | no |
+| <a name="input_ui_instance_count"></a> [ui\_instance\_count](#input\_ui\_instance\_count) | Number of UI instances | `number` | `1` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_alb_dns_name"></a> [alb\_dns\_name](#output\_alb\_dns\_name) | The domain name of the ALB that can be used to access SkyWalking UI. |
+| <a name="output_bastion_ips"></a> [bastion\_ips](#output\_bastion\_ips) | The public IP that can be used to SSH into the bastion host. |
+| <a name="output_database_address"></a> [database\_address](#output\_database\_address) | The database address |
+| <a name="output_database_name"></a> [database\_name](#output\_database\_name) | The database name |
+| <a name="output_database_password"></a> [database\_password](#output\_database\_password) | The database password |
+| <a name="output_database_port"></a> [database\_port](#output\_database\_port) | The database port |
+| <a name="output_database_username"></a> [database\_username](#output\_database\_username) | The database username |
+| <a name="output_skywalking_oap_ips"></a> [skywalking\_oap\_ips](#output\_skywalking\_oap\_ips) | The private IPs of the OAP instances |
+| <a name="output_skywalking_ui_ips"></a> [skywalking\_ui\_ips](#output\_skywalking\_ui\_ips) | The IPs of the SkyWalking UI instances |
+| <a name="output_ssh-user-key-file"></a> [ssh-user-key-file](#output\_ssh-user-key-file) | The SSH key file that can be used to connect to the bastion instance. |
+<!-- END_TF_DOCS -->

aws/alb-main.tf

@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+module "alb" {
+  source  = "terraform-aws-modules/alb/aws"
+  version = "~> 8.0"
+
+  name = var.cluster_name
+
+  load_balancer_type = "application"
+
+  vpc_id          = module.vpc.vpc_id
+  subnets         = module.vpc.public_subnets
+  security_groups = [module.vpc.default_security_group_id]
+
+  security_group_rules = {
+    ingress_all_http = {
+      type        = "ingress"
+      from_port   = 80
+      to_port     = 80
+      protocol    = "tcp"
+      description = "Allow HTTP traffic"
+      cidr_blocks = ["0.0.0.0/0"]
+    }
+    egress_all = {
+      type        = "egress"
+      from_port   = 0
+      to_port     = 0
+      protocol    = "-1"
+      cidr_blocks = ["0.0.0.0/0"]
+    }
+  }
+
+  target_groups = [
+    {
+      name_prefix      = substr(var.cluster_name, 0, 6)
+      backend_protocol = "HTTP"
+      backend_port     = 8080
+      target_type      = "instance"
+      targets = [
+        for i, ui in aws_instance.skywalking-ui : {
+          target_id = ui.id
+          port      = 8080
+        }
+      ]
+    }
+  ]
+
+  http_tcp_listeners = [
+    {
+      port               = 80
+      protocol           = "HTTP"
+      target_group_index = 0
+    }
+  ]
+
+  tags = var.extra_tags
+}