Skip to content

Commit

Permalink
Release/new infrastructure/0001 (#93)
Browse files Browse the repository at this point in the history
* Initial work to make create jenkins job configurable.
* added new job to update config.xml to s3
* added new global properties to pass variables from centralized
location to ( terraform and user-scripts ).
* removed local parameters from create jenkins job
* updated names for variables to more meaningful names
* updated cidr terraform vars to be list.
* grouped terraform security group ingress rules to be list instead of
strings.  Grouped by ingress rule function ( http, https, etc.. )

* making the jenkins git repo a global parameter

* update to add dsm-url to user-script ( install-docker.sh )

* update to variable name in user-script

* removing quotes for ami-id so it can pass variable

* config updates for repos and git hashes

* adding script to initialize jenkins

* rework done on jenkins configurations and deployment
* making maven more agnostic to version being deployed
* making terraform configurable
* IAC imporovements to Docker, Terraform and Jenkins
* cleaning up variable naming standards for configurations

* will be doing cleanup on jobs to use naming standard for variables

* removing the .12 terraform experimental variable validation =(

* removing a sg

* fixing up some variable naming standards

* updating maven variable name

* more naming standards.

* and some more...

* updating location of user-script to more configurable

* removing line terminator

* user-script update

* pulling some naming standards into user-script

* docker args clean up

* changing some things for it to be more environment agnostic

* keystore is fine

* missing xml closure

* Update for destroy job to use new vars

* useful gitignore to update jobs

* gitignore

* Latest jobs from dev work

* Adding Maven apt-get work to ICA work

* Minor naming convention update and deactivating non-fisma provider to
prevent it from being applied.

* Adding a script that should be able to initialize a jenkins server from
an ec2.

* latest state of jobs for configuring dev

* changes to initialize jenkins

* Latest jobs before

* cleanup user-data, lookup AMI

* more changes for deploy

* more cleanup

* fix

* fixes

* boolean -> bool

* syntax

* pass git commit

* syntax

* use git repo

* update destroy job

* fix jenkins creation

* aws binary

* more job changes

* quotes

* combine into one policy statement

* fix assume role code

* fix jobs

* bucket policy

* fix hpds job

* more fix

* Add open access param

* Syncing jobs with current open-pic-sure state

* update aws cli path

shouldn't need to use an explicit path

* Removing unused variable

* Workaround for python introducing externally-managed-environment

* Updating Script Approvals to be sha512

* Add missing IDP provider in teardown

* [ALS-4884] Add analytics_id to terraform destroy and apply (#68)

* [ALS-4998] Add new env vars to terraform

* syncing jenkins jobs from current state to test in auth

* update for analytics, open access and project id

* syncing job changes.

* rebase off open-picsure

* add more changes

* merge conflict

* duplication

* cleanup

* declaring program variable

* fix path

* Add proj to name

* program

* Remove ssh

* Feature/swap stacks (#72)

* Update for including aws-cli functionality to perform swapping the green
blue environments.

* no need to use /usr/bin

* Some logging for clarity in console.

* forcing a new hash

for some reason Terraform does not like the previous commit.

No terraform changes were made.

This commit terraform works 8964fee
This commit terraform breaks 43fe6ac

---------

Co-authored-by: Tom <[email protected]>

* global var for httpd staging vhost

* remove unused var

* Feature/fix jenkins state  (#75)

* Initial changes for fixing jenkins destroying itself

* removing git commit from sg name - will be a random string

* wget for jenkins archive fix

* resource not data

* create before destroying.

* need to handle tags better.
* Should create this tag even if we use aws cli to update it.
* if terraform refreshes tags it will delete unmanaged tags.

* just var

* cap

* moving to a green / blue stack method.

* Green will be auto-promoted to blue on successful init
* blue should remain functional until successful init
* blue should be demoted to a
* destroy job will always just destroy green.
* leaving blue / green explict for now.

* fixes
* improving initcomplete tag lookup
* method to rollback blue to green.

* lowercase true

* using arguments

* pkcs12 as env var in container instead of user-script

* removing build and just loading and running image

* one dollar sign

* remove misc code

* moving build to jenkins job and out of user-script

* config.xml.override didn't override..

* testing new create new jenkins job

* removing bucket policy job

* deleting unused jobs
* bye bye update bucket policy

* this is no longer used

* Feature/build container in jenkins job (#76)

* using arguments

* pkcs12 as env var in container instead of user-script

* removing build and just loading and running image

* one dollar sign

* remove misc code

* moving build to jenkins job and out of user-script

* config.xml.override didn't override..

* testing new create new jenkins job

* removing bucket policy job

* deleting unused jobs
* bye bye update bucket policy

* this is no longer used

* added automatic dns swap for rollback and checking condition of green
state while rolling back and deploying new server

* fixing json flag

---------

Co-authored-by: Tom <[email protected]>

* fixing conditional

should be true statements for conditional

* Removing reliance in jenkins to use the stack_s3_bucket.

* This way the CI state is not bound to the application's s3 bucket.

---------

Co-authored-by: Tom <[email protected]>

* latest updates: (#83)

* tga priv_ips need to be stored in an Array not string to iterate over
* adding tag_manager_id variable

Co-authored-by: Tom <[email protected]>

* removing unused variable

* Sync last open release with auth-migration release (#88)

* [ALS-5164] Create new open access job (#86)

[ALS-5164] Update deployment pipeline
Our deployment pipeline to conditionally run the Open Access build.

[ALS-5164] Add logic to handle potential errors
These cases can happen if its not open access or a version of open access is used that doesn't contain psamaui.

* Need to stage this file so it's always available for open. (#87)

* Fence mapping is now needed for all environments.

---------

Co-authored-by: Gcolon021 <[email protected]>

* [ALS-5279] - Adding Role Strategy Plugin (#89)

* # Disable plugin.  Making dockerfile more configurable.
* added configuration to disable setup wizard
* added configurations to make args optional in order to make container more portable.

* Swap user around

* moving debian sources

* some bash cleanup

* trying this way

* Should do that stuff as entrypoints removing

* adding cleanup and some ideas

* adding role-strategy plugin

* removing auto skip setup wizard.

* Syncing jobs from auth-dev

---------

Co-authored-by: Tom <[email protected]>

* ALS-5153 (#90)

# improving RDS strategies
* added functions to dynamically handle different scenarios for managing RDS instances.
* Strategies are stubs atm for standalone.  Should add functionality to these blocks.
* added continous assume role to await init to avoid assumed role timeout.  The job itself will handle init timeout. set to 4 hrs currently.  Build out timer in the job.

---------

Co-authored-by: Tom <[email protected]>

* Sourcing scripts to rollback jenkins (#91)


---------

Co-authored-by: Tom <[email protected]>

* [ALS-5344] Banner config now uploads to S3 (#92)

[ALS-5344] Banner config now uploads to S3 (#92)

* Updating readme documentation

---------

Co-authored-by: Tom <[email protected]>
Co-authored-by: bp85 <[email protected]>
Co-authored-by: gcolon021 <[email protected]>
Co-authored-by: Gcolon021 <[email protected]>
  • Loading branch information
5 people authored Dec 7, 2023
1 parent 10929e8 commit c210705
Show file tree
Hide file tree
Showing 65 changed files with 2,941 additions and 1,879 deletions.
177 changes: 41 additions & 136 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,146 +1,51 @@
Prerequisites:

AWS CLI installed and configured with admin access to the C&C account.
Terraform installed
Git installed


Create new S3 bucket for new Jenkins instance to use setting the following options DURING CREATION some can't be set after
- bucket should be named using the following template : avillach-biodatacatalyst-deployments-<Random 7 hex digits>
- Object Locking must be enabled
- Encryption should be AES-256
- Enable Object-level logging as secrets are stored in this bucket
- Enable versioning
- Server access logging enabled (hms-dbmi-cnc-cloudtrail, no target prefix)

Set Bucket Policy in the Permissions section for the bucket to the following after replacing __BUCKET_NAME__ with the bucket name:

-----------------------------------------------------
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::191687121306:role/hms-dbmi-cnc-role"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObjectAcl",
"s3:GetObjectTagging",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::__BUCKET_NAME__/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::752463128620:role/system/jenkins-s3-role"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObjectAcl",
"s3:GetObjectTagging",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::__BUCKET_NAME__/*"
}
]
}
-----------------------------------------------------




Clone https://github.com/hms-dbmi/avillachlab-jenkins

Run the following commands after replacing all __VARIABLE_NAME__ entries with their correct values for the environment:

-----------------------------------------------------

cd dev-jenkins-terraform
env > env.txt
terraform init
terraform apply -auto-approve \
-var "git-commit=__GIT_COMMIT_FOR_JENKINS_REPO__" \
-var "stack-s3-bucket=__S3_BUCKET_NAME_YOU_CREATED__" \
-var "stack-id=__S3_BUCKET_NAME_SUFFIX__" \
-var "subnet-id=__JENKINS_SUBNET_ID__" \
-var "vpc-id=__JENKINS_VPC_ID__" \
-var "instance-profile-name=__JENKINS_INSTANCE_PROFILE_NAME__" \
-var "access-cidr=__JENKINS_ACCESS_CIDR__" \
-var "provisioning-cidr=__JENKINS_PROVISIONING_CIDR__"

aws s3 --sse=AES256 cp terraform.tfstate s3://${stack_s3_bucket}/jenkins_state_${GIT_COMMIT}/terraform.tfstate
aws s3 --sse=AES256 cp env.txt s3://${stack_s3_bucket}/jenkins_state_${GIT_COMMIT}/last_env.txt

INSTANCE_ID=`terraform state show aws_instance.dev-jenkins | grep "\"i-[a-f0-9]" | cut -f 2 -d "=" | sed 's/"//g'`

while [ -z $(/usr/local/bin/aws --region=us-east-1 ec2 describe-tags --filters "Name=resource-id,Values=${INSTANCE_ID}" | grep InitComplete) ];do echo "still initializing";sleep 10;done

echo "http://`terraform state show aws_instance.dev-jenkins | grep private_ip | cut -f 2 -d "=" | sed 's/\"//g' | sed 's/ //g'`"

-----------------------------------------------------


Set stack_s3_bucket Value to new S3 bucket name in new Jenkins
- Manage Jenkins > Configure System
- under "Global properties" set stack_s3_bucket to the new bucket created in the first step

Add the following arn as a trusted entity in the hms-dbmi-cnc-role in the prod account:
- https://console.aws.amazon.com/iam/home?region=us-east-1#/roles/hms-dbmi-cnc-role?section=trust
- example template :

{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:sts::752463128620:assumed-role/jenkins-s3-role/< instance id of the jenkins ec2 you just created >"
},
"Action": "sts:AssumeRole",
"Condition": {}
}

- example :

{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:sts::752463128620:assumed-role/jenkins-s3-role/i-0615f53dd368cbdfc"
},
"Action": "sts:AssumeRole",
"Condition": {}
}

Switch to Jenkins Configuration View ( DO NOT QUEUE UP THE JOBS! Wait for each to complete successfully before going on to the next. )

Run Jenkins Build "Create stack_variables.tf files"
Run Jenkins Build "Update VPC Settings" after confirming the following:
- confirm that the R53_Zone_ID is correct for the prod account Route 53 Zone
- confirm that the vpc and subnet group names are correct for the prod account
Run Jenkins Build "Update PIC-SURE Token Introspection Token"
Run Jenkins Build "Update Fence Client Credentials"
- provide the correct Fence Client ID and Client Secret as provided by the Fence team
Run Jenkins Build "Update HTTPD Certs and Key"
- provide the correct Cert, Chain and Key file for the production HTTPD server
# Overview

Welcome to the base folder of your Jenkins project. This folder contains two subfolders, each dedicated to a specific aspect of your Jenkins infrastructure: `jenkins-docker` and `jenkins-terraform`. Below, you'll find an overview of each component along with links to their respective README files for detailed information.

Switch to the Deployment View
## Jenkins Docker (`jenkins-docker`)

Run Jenkins Build Check For Updates
- The first time this runs it will take about 1.5 hours because it has to rekey the data.
This section focuses on the Dockerization of Jenkins, incorporating additional tools and configurations to enhance its functionality. The Jenkins Docker image is extended from the official LTS image, making it a versatile and powerful solution. For more information, refer to the [Jenkins Docker README](jenkins-docker/README.md).

Run Jenkins Build Swap Stacks
- This will point the internal production CNAME at the current stage environment
- The current stage environment becomes prod and the current prod environment becomes stage
## Jenkins Terraform (`jenkins-terraform`)

Run Jenkins Build Check For Updates
- This time it should only take about a half hour because the data has already been rekeyed.
Here, Terraform is leveraged to deploy and manage Jenkins infrastructure on AWS. The README provides insights into the variables, backend configuration, and outputs defined in the Terraform files. Detailed information about variables, Terraform backend configuration, and outputs can be found in the [Jenkins Terraform README](jenkins-terraform/README.md).

Feel free to explore each component to understand their configurations, usage, and any additional details you may need for managing your Jenkins environment efficiently. If you have any questions or need assistance, don't hesitate to reach out.

## Table of Contents

- [Jenkins Docker](#jenkins-docker)
- [Jenkins Terraform](#jenkins-terraform)

Feel free to dive into the respective sections for detailed information and instructions.

# Jenkins Docker (`jenkins-docker`)

This section focuses on the Dockerization of Jenkins, incorporating additional tools and configurations to enhance its functionality. The Jenkins Docker image is extended from the official LTS image, making it a versatile and powerful solution.

## [Jenkins Docker README](jenkins-docker/README.md)

- [Features](jenkins-docker/README.md#features)
- [Prerequisites](jenkins-docker/README.md#prerequisites)
- [Building the Image](jenkins-docker/README.md#building-the-image)
- [Configuration](jenkins-docker/README.md#configuration)
- [Usage](jenkins-docker/README.md#usage)
- [Cleanup](jenkins-docker/README.md#cleanup)
- [Contributing](jenkins-docker/README.md#contributing)
- [License](jenkins-docker/README.md#license)

Feel free to explore the Jenkins Docker README for comprehensive information on configuring, building, and using the Docker image.

# Jenkins Terraform (`jenkins-terraform`)

Here, Terraform is leveraged to deploy and manage Jenkins infrastructure on AWS. The README provides insights into the variables, backend configuration, and outputs defined in the Terraform files.

## [Jenkins Terraform README](jenkins-terraform/README.md)

- [Variables](jenkins-terraform/README.md#variables)
- [Terraform Backend](jenkins-terraform/README.md#terraform-backend)
- [Outputs](jenkins-terraform/README.md#outputs)

Feel free to explore the Jenkins Terraform README for detailed information on variables, Terraform backend configuration, and outputs.

Feel free to explore each component to understand their configurations, usage, and any additional details you may need for managing your Jenkins environment efficiently. If you have any questions or need assistance, don't hesitate to reach out.
Loading

0 comments on commit c210705

Please sign in to comment.