Evergreen.py

A client library for the Evergreen API written in python. Currently supports the V2 version of the API. For more details, see https://github.com/evergreen-ci/evergreen/wiki/REST-V2-Usage.

Table of contents

1. Description
2. Getting Help
3. Dependencies
4. Installation
5. Usage
6. Documentation
7. Contributor's Guide
   • Setting up a local development environment
   • Linting/formatting
   • Running tests
   • Automatically running checks on commit
   • Versioning
   • Code Review
   • Deployment

Description

This is a Python client library for interacting with Evergreen and Evergreen objects. It currently only supports the V2 version of Evergreen's api. It can be used either by Python code in a separate application or on the command line to get data about Evergreen objects quickly and easily.

Getting Help

What's the right channel to ask my question?

If you have a question about evergreen.py, please mention @dag-on-call in slack channel #evergreen-users, or email us at [email protected].

How can I request a change/report a bug in evergreen.py?

Create a DEVPROD ticket.

What should I include in my ticket or #evergreen-users question?

Since #evergreen-users questions are interrupts, please include as much information as possible. This can help avoid long information-gathering threads.

Please include the following:

• Motivation for Request
  • provide us the motivation for this change.
• Context
  • provide some background contexts for this issue.
• Description
  • provide some descriptions on how this issue happened.

Dependencies

• Python 3.9-3.13

Installation

$ pip install evergreen.py

Usage

This client can be used either in code or directly via the command line.

In code:

>> from evergreen.api import EvgAuth, EvergreenApi
>> api = EvergreenApi.get_api(EvgAuth('your.username', '***'))
>> project = api.project_by_id('mongodb-mongo-master')
>> project.display_name
'MongoDB (master)'

Cli:

$ evg-api --json list-hosts
{
    "host_id": "host num 0",
    "host_url": "host.num.com",
    "distro": {
        "distro_id": "ubuntu1804-build",
        "provider": "static",
        "image_id": ""
    },
    "provisioned": true,
    "started_by": "mci",
    "host_type": "",
    "user": "mci-exec",
    "status": "running",
    "running_task": {
        "task_id": null,
        "name": null,
        "dispatch_time": null,
        "version_id": null,
        "build_id": null
    },
    "user_host": false
}

The patch_from_diff API requires the Evergreen CLI to be installed. Add the following to the host's DOCKERFILE:

RUN wget https://evergreen.mongodb.com/clients/linux_amd64/evergreen
RUN chmod +x evergreen
ENV PATH="/project:$PATH"

You will need to provide an .evergreen.yml file with credentials to use the CLI. Assuming you are using the web-app chart this can be done by mounting kubernetes secrets in your pod.

Store the secret in the cluster:

kubectl create secret generic <secret_name> --from-file .evergreen.yml --namespace <namespace>

In environments/deployment.yml configure the file to be mounted and linked to the correct location:

volumeSecrets:
  - name: <secret_name>
    path: /etc/secrets
lifecycle:
  postStart:
    type: exec
    command:
      - /bin/sh
      - -c
      - ln -sf /etc/secrets/.evergreen.yml

Documentation

You can find the documentation here.

Contributor's Guide

Setting up a local development environment

Requirements

• Poetry 1.1 or later

You will need Evergreen credentials on your local machine to use this library or the attached CLI. You can set up your credentials by following the link here.

Linting/formatting

This project uses black and isort for linting/formatting.

poetry run black src tests
poetry run isort src tests

Running tests

poetry run pytest

There are a few tests that are slow running. These tests are not run by default, but can be included by setting the env variable RUN_SLOW_TESTS to any value.

$ RUN_SLOW_TEST=1 poetry run pytest

To get code coverage information:

$ poetry run pytest --cov=src --cov-report=html

Changes to doc site building

Docs are built with sphinx by their recommended GitHub Action. This action is configured to use a requirements.txt from the docs subdirectory so it doesn't know anything about Poetry and Poetry doesn't know anything about Sphinx deps. But this environment also needs to know about the runtime dependencies of the evergreen package in order to auto-document the Python modules from docstrings.

This is an outline of the process you'd use to update the docs requirements.txt

poetry run python3 -m venv docs-env
. docs-env/bin/activate
pip install -r docs/requirements.txt
# if you need to change any docs-only dependencies (like sphinx or extensions/themes), upgrade them now with pip install

If you have upgraded any material runtime dependencies of the evergreen package with poetry, follow this section as well

deactivate
poetry export -f requirements.txt --output docs/runtime_requirements.txt
. docs-env/bin/activate
pip install -r docs/runtime_requirements.txt
rm docs/runtime_requirements.txt

Check that the doc site still builds

pushd docs
sphinx-build -W source build
# poke around the build output directory
popd

Finally to re-pin the docs dependencies

pip freeze > docs/requirements.txt
git add docs/requirements.txt

Automatically running checks on commit

This project has pre-commit configured. Pre-commit will run configured checks at git commit time. To enable pre-commit on your local repository run:

$ poetry run pre-commit install

Versioning

Before deploying a new version, please update the CHANGELOG.md file with a description of what is being changed.

Deployment to PyPi are done automatically on merges to master. In order to avoid overwriting a previous deploy, the version should be updated on all changes. The semver versioning scheme should be used for determining the version number.

The version is found in the pyproject.toml file.

Code Review

This project uses the GitHub merge queue. Click "Merge when ready" as soon as you'd like.

Deployment

Deployment to PyPi is automatically triggered on merges to master.