Drifty

Prevent observability drift in your PRs

⚠️ WIP ⚠️

Local Development

1. Copy the example environment file:

   cp .env.example .env

2. Configure your .env file:

   • Monitor Provider: Your provider's API credentials
   • AI Provider: Any OpenAI-compatible API works (OpenRouter, Together AI, Ollama, etc.)

3. Make sure you have AWS credentials configured (e.g., aws sso login)

4. Run:

   cargo run -- sync

Usage

drifty sync

Syncs your monitors to S3 Vectors for drift detection.

• ✅ Checks if a bucket exists, creates one if not
• ✅ Checks if an index for your provider exists, creates one if not
• ✅ Fetches monitors from your provider
• ✅ Generates embeddings
• ✅ Uploads embeddings to S3 Vectors

Monitors are cached for 1 month. Pass -f or --force to refresh.

drifty check (wip!)

Checks your PR for potential monitoring drift.

• Chunks your PR
• Generates embeddings
• Compares against your synced monitors

AWS / GitHub Actions Setup (OIDC)

Drifty uses S3 Vectors to store monitor embeddings. For CI/CD, we recommend using OIDC instead of long-lived access keys.

Installation

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/joswayski/drifty/releases/latest/download/drifty-installer.sh | sh

Click to expand setup instructions

Step 1: Create an IAM policy for permissions

1. Go to IAM → Policies → Create policy

2. Either use the IAM.example.json or create a new policy with the following permissions:

   • s3vectors:CreateIndex
   • s3vectors:PutVectors
   • s3vectors:GetIndex
   • s3vectors:GetVectorBucket
   • s3vectors:GetVectors
   • s3vectors:CreateVectorBucket
   • s3vectors:QueryVectors

   Make sure to include the name of the bucket that you will use, and to add a wildcard on the indexes under resources:

Step 2: Create the OIDC Identity Provider

1. Go to IAM → Identity providers → Add provider
2. Select OpenID Connect
3. Provider URL: https://token.actions.githubusercontent.com
4. Audience: sts.amazonaws.com
5. Click Add provider

Step 3: Create an IAM Role

1. Go to IAM → Roles → Create role
2. Select Web identity
3. Identity provider: token.actions.githubusercontent.com
4. Audience: sts.amazonaws.com
5. Add your organization and repository name

6. Click Next, attach the policy you created in Step 1.

7. Name the role (e.g., GitHubActionsDriftyRole)

Step 4: Configure Your Workflow

Create a GitHub action like the one you see here. Make sure to add your environment variables as secrets in your GitHub repo settings.

Supported Providers

• Datadog ✅

Contributions welcome for other providers!