Jenkins Infra Datadog monitoring

Table of Contents

• Requirements
• Monitors

This repository is meant to hold configuration files of Jenkins infrastructure monitoring stuff.

This repository hosts the infrastructure-as-code definition for all the DataDog monitoring resources for the Jenkins Infrastructure Project.

Requirements

• An access to the Jenkins Datadog account for the Jenkins Infrastructure at https://jenkins.datadoghq.com.

• The requirements (of the shared tools) listed at shared-tools/terraform#requirements

• The shared-tools common repository as a git submodule should be initialized and up-to-date:

  • On a freshly cloned Terraform repository, execute the following command to obtain the shared tools:

git submodule update --init --recursive

• Otherwise you should update to to the latest main branch:

cd ./.shared-tools
git pull origin main
cd ../

• The Azure Credential Secret value to allow the Terraform AzureRM Backend to access remote Azure storage:

  • Must be specified in the environment variable TF_BACKEND_ARM_CLIENT_SECRET

  • The secret value should never be written in clear (not in a text file, not in the terminal). For instance on macOS, store it in your Keychain access and set the variable with export TF_BACKEND_ARM_CLIENT_SECRET="$(security find-generic-password -a jenkins-tf-datadog-backend-arm-secret -w)"

• If you intend to work with the production, the environment variable TF_VAR_environment must be set:

export TF_VAR_environment=production

Monitors

DataDog monitors are configured through Terraform with the datadog provider.

Once you’ve fulfilled the Requirements, you may execute any command from https://github.com/jenkins-infra/shared-tools/blob/main/terraform/README.adoc#available-commands by adding the correct flag --directory pointing to .shared-tools/terraform/:

make --directory=.shared-tools/terraform help
make --directory=.shared-tools/terraform lint
# ...

A usual change (add/update/remove a datadog monitor for instance) to this repository looks like the following:

• Fork the repository and clone it locally

• Follow the Requirements steps to obtain the shared tools

• Start by running a full make --directory=.shared-tools/terraform validate command to ensure that you work on a sane base (should generate a report TXT file with no changes to be applied)

• Edit the Terraform project files

• Run the command make --directory=.shared-tools/terraform validate again to ensure that your changes are OK

• Commit, push and open a pull request to let the Jenkins pipeline run the test + plan (as per https://github.com/jenkins-infra/shared-tools/blob/main/terraform/README.adoc#jenkins-pipeline)