From 683ea90f7eeb7f04a96c0622f4a66fa680a1aefc Mon Sep 17 00:00:00 2001 From: Khai Do <3697686+zaro0508@users.noreply.github.com> Date: Fri, 17 May 2024 15:24:36 -0700 Subject: [PATCH 1/5] update docs (#122) Add info on setting up github self hosted runners --- README.md | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 54 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 605c340..37d2749 100644 --- a/README.md +++ b/README.md @@ -57,8 +57,8 @@ Context specific secrets for each environment that corresponds to a git branch ( ## Self hosted runners [agora2-infra] repository deploys a bastian host in AWS for each environment which have access to -the databases. We manually configure a [Github self-hosted runner] for each bastian host, -a label is applied to each runner to match the corresponding deployment branch name (develop/staging/prod). +the databases. We manually configure a [Github self-hosted runner](https://docs.github.com/en/actions/hosting-your-own-runners) +for each bastian host, a label is applied to each runner to match the corresponding git branch name (develop/staging/prod). Each runner corresponds to an environment which corresponds to a git branch. The update is executed from these runners. When a push happens on a branch (i.e. develop), the update is executed on the `agora-bastian-develop` runner which in turn updates the development database. @@ -67,6 +67,58 @@ is executed on the `agora-bastian-develop` runner which in turn updates the deve ![alt text][self_hosted_runners] +### Setup self hosted runners + +Github self hosted runners are deployed with a [Sceptre template config file])(https://github.com/Sage-Bionetworks/agora2-infra/blob/main/config/agoradev/develop/agora-bastian.yaml). + +Self Hosted Runner setup: +* Deploy the template to the Agora AWS account. +* Login to AWS console and goto `EC2 -> select the deployed instance -> Connect -> Session Manager -> Connect` to gain ssh access to the instance. +* Follow the instructions to install the [Github self hosted runner](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners#adding-a-self-hosted-runner-to-a-repository). We installed it to the `/home/ssm-user/actions-runner` folder. +* Run the `config.sh` script to configure the runner. !! Important !! Make sure to set the runner `name` and `label` corresponding to the desired deployment environment (develop/staging/prod).. +```text +sh-4.2$ pwd +/home/ssm-user/actions-runner + +sh-4.2$ ./config.sh --url https://github.com/Sage-Bionetworks/agora-data-manager --token XXXXXXXXXXXXXXXXX6VLI + +-------------------------------------------------------------------------------- +| ____ _ _ _ _ _ _ _ _ | +| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ | +| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| | +| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ | +| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ | +| | +| Self-hosted runner registration | +| | +-------------------------------------------------------------------------------- + +# Authentication + + +√ Connected to GitHub + +# Runner Registration + +Enter the name of the runner group to add this runner to: [press Enter for Default] + +Enter the name of runner: [press Enter for ip-10-XXX-XXX-XXX] agora-bastian-prod + +This runner will have the following labels: 'self-hosted', 'Linux', 'X64' +Enter any additional labels (ex. label-1,label-2): [press Enter to skip] prod + +√ Runner successfully added +√ Runner connection is good + +# Runner settings + +Enter name of work folder: [press Enter for _work] + +√ Settings Saved. +``` +* Setup the [GH runner agent to run as a service](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/configuring-the-self-hosted-runner-application-as-a-service) +* Run the agent and then check the [GH Runners page](https://github.com/Sage-Bionetworks/agora-data-manager/settings/actions/runners) to make sure that the runner is in `Idle` status. + [db_update]: agora-db-update.drawio.png "update diagram" [github_secrets]: github_secrets.png "github secrets screen" [self_hosted_runners]: self-hosted-runners.png "self hosted runners" From 94801ff7ebbe66ea343561125001d119404582c6 Mon Sep 17 00:00:00 2001 From: Jessica Date: Fri, 25 Oct 2024 12:26:37 -0700 Subject: [PATCH 2/5] Push data RC mv69 to dev --- data-manifest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-manifest.json b/data-manifest.json index 8b74810..ed3e1bf 100644 --- a/data-manifest.json +++ b/data-manifest.json @@ -1,5 +1,5 @@ { - "data-version": "68", + "data-version": "69", "data-manifest-id": "syn13363290", "team-images-id": "syn12861877" } From bb3b6f81ed9aac7d9942abba5176753395197a85 Mon Sep 17 00:00:00 2001 From: Jessica Date: Mon, 28 Oct 2024 12:06:26 -0700 Subject: [PATCH 3/5] IT-3964: Add workaround for Node20 runners --- .github/workflows/main.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 5e1e214..6217ab8 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -21,6 +21,7 @@ jobs: # variables in context environments are setup in github to match branch names environment: name: ${{ github.ref_name }} + ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true steps: # use older checkout version due to https://github.com/dawidd6/action-download-artifact/issues/261 From 0a317873ba3605a6e04eea5a6c766212265f9f98 Mon Sep 17 00:00:00 2001 From: Jessica Date: Mon, 28 Oct 2024 12:48:07 -0700 Subject: [PATCH 4/5] IT-3964: Move new config to jobs.deploy.env --- .github/workflows/main.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 6217ab8..dfcb575 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -21,6 +21,7 @@ jobs: # variables in context environments are setup in github to match branch names environment: name: ${{ github.ref_name }} + env: ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true steps: From 789cecd7116fb9205ecca6dd5fcd1973ee5b8e33 Mon Sep 17 00:00:00 2001 From: Jessica Date: Mon, 28 Oct 2024 13:04:16 -0700 Subject: [PATCH 5/5] IT-3964: Move config to global level --- .github/workflows/main.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index dfcb575..ce511ee 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -6,6 +6,9 @@ on: push: branches: ['develop', 'staging', 'prod' ] +env: + ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true + jobs: tests: runs-on: ubuntu-latest @@ -21,8 +24,6 @@ jobs: # variables in context environments are setup in github to match branch names environment: name: ${{ github.ref_name }} - env: - ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true steps: # use older checkout version due to https://github.com/dawidd6/action-download-artifact/issues/261