Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS-1166][DOCS-880][DOCS-1167][DOCS-1168] Update automation docs #1084

Merged
merged 63 commits into from
Mar 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
2e08a16
[DOCS-1166] Update all possible links and references to old Model Reg…
mdlinville Jan 16, 2025
582d4db
[DOCS-880][DOCS-1167][DOCS-1168] Consolidate project-scoped and model…
mdlinville Feb 4, 2025
643b6c3
Style
mdlinville Feb 5, 2025
064288d
Markdown mistake
mdlinville Feb 5, 2025
f4bfe72
Fix links
mdlinville Feb 6, 2025
0f55a71
Fix another broken link
mdlinville Feb 6, 2025
a37335e
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 6, 2025
22d0ccc
Merge branch 'main' into automation_updates
mdlinville Feb 6, 2025
db6d7fa
Merge branch 'main' into automation_updates
mdlinville Feb 7, 2025
5b201a4
Merge branch 'main' into automation_updates
mdlinville Feb 11, 2025
3633fa7
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 11, 2025
7a801c0
Stephen's round 1 feedback
mdlinville Feb 11, 2025
c526e25
Apply suggestions from code review
mdlinville Feb 11, 2025
2e74a54
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 14, 2025
387b8dd
Remove mention of run metrics automations
mdlinville Feb 14, 2025
ca18948
Feedback
mdlinville Feb 14, 2025
173901d
Merge remote-tracking branch 'origin/automation_updates' into automat…
mdlinville Feb 14, 2025
a6a5958
Merge branch 'main' into automation_updates
mdlinville Feb 18, 2025
1e45d26
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 18, 2025
fdbac71
Adjust link to secrets page
mdlinville Feb 18, 2025
8e12500
Merge remote-tracking branch 'origin/automation_updates' into automat…
mdlinville Feb 18, 2025
1a02164
Fill out remaining sections in the Secrets page
mdlinville Feb 19, 2025
eef5f44
Apply suggestions from code review
mdlinville Feb 19, 2025
8cb89f6
Feedback
mdlinville Feb 19, 2025
98a2a20
Merge remote-tracking branch 'origin/automation_updates' into automat…
mdlinville Feb 19, 2025
398af6d
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 19, 2025
eef2399
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Feb 26, 2025
78869e6
Stephen's open feedback, some light restructuring
mdlinville Feb 26, 2025
44d731b
Apply suggestions from code review
mdlinville Feb 26, 2025
4b49470
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 6, 2025
1791cf9
Apply suggestions from code review
mdlinville Mar 6, 2025
d671203
Update content/guides/core/secrets.md
mdlinville Mar 6, 2025
318ffd5
Merge remote-tracking branch 'origin/automation_updates' into automat…
mdlinville Mar 6, 2025
b423d1b
Much of Noa's first feedback
mdlinville Mar 6, 2025
abf2dd8
Update links after recent changes to main
mdlinville Mar 6, 2025
ad32b0b
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 7, 2025
d1c88f5
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 17, 2025
0e30930
Chunk automations content into multiple pages
mdlinville Mar 18, 2025
239bb2c
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 18, 2025
5e032a0
Content updates
mdlinville Mar 19, 2025
6fb2362
Webhook updates
mdlinville Mar 21, 2025
725bab6
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 21, 2025
c5f79b6
Merge branch 'main' into automation_updates
mdlinville Mar 24, 2025
8f2a6f6
Merge branch 'main' into automation_updates
mdlinville Mar 25, 2025
9e8070b
Apply suggestions from code review
mdlinville Mar 26, 2025
1767129
Apply suggestions from code review
mdlinville Mar 26, 2025
dc24798
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 26, 2025
07797f0
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 26, 2025
e5d0533
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 26, 2025
f73c8f9
More Noa feedback
mdlinville Mar 26, 2025
f55d101
Noa's feedback about scopes and events
mdlinville Mar 26, 2025
64d514f
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville Mar 28, 2025
4de1043
Noa's final feedback
mdlinville Mar 28, 2025
5ce190d
Typo
mdlinville Mar 28, 2025
001e1a5
Update content/guides/core/automations/create-automations/_index.md
mdlinville Mar 28, 2025
e16f353
Noa and Stephen's feedback
mdlinville Mar 29, 2025
39001ee
Add an include for Enterprise Only, use it in the automation pages
mdlinville Mar 31, 2025
4027246
Update user and team settings
mdlinville Mar 31, 2025
bd09da6
Revert wandb.alert changes
mdlinville Mar 31, 2025
a63fee7
Add a Cloud-only include and use it instead
mdlinville Mar 31, 2025
f46eede
Nit
mdlinville Mar 31, 2025
803b7e2
Apply suggestions from code review
mdlinville Mar 31, 2025
1e259a1
Merge branch 'main' into automation_updates
mdlinville Mar 31, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,15 @@ Hit `CTRL+C` in the terminal that is showing `hugo` activity to interrupt the se
```markdown
{{< prism file="/webhook_test.sh" title="webhook_test.sh">}}{{< /prism >}}
```
- We are _experimenting_ with using `readfile` for includes. If you run into issues, report it in a Github issue.

Add a Markdown file with no front matter to `content/_includes/`. Subdirectories are supported. Include the file using the [`readfile`](https://www.docsy.dev/docs/adding-content/shortcodes/#reuse-documentation) shortcode. For example:
```markdown
{{< readfile file="/_includes/enterprise-only.md" >}}
```

- If you change an include, the `hugo serve` incremental build does not pick up the change. Stop and restart `hugo serve`.
- Hugo and Docsy shortcodes are **not** supported inside the include file.
## Editing style

Style overrides are in `/assets/scss/_variables_project.scss`. Here we can override all the styles that ship with the Docsy theme. O
Expand Down
2 changes: 2 additions & 0 deletions content/_includes/enterprise-cloud-only.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This feature requires a [Pro or Enterprise plan](https://wandb.ai/site/pricing/).

2 changes: 2 additions & 0 deletions content/_includes/enterprise-only.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This feature is available for all [Enterprise](https://wandb.ai/site/pricing/) licenses.

3 changes: 2 additions & 1 deletion content/guides/core/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ W&B Core provides capabilities across the entire ML lifecycle. With W&B Core, yo
- Explore and evaluate data and metrics using [interactive, configurable visualizations]({{< relref "/guides/models/tables/" >}}).
- [Document and share]({{< relref "./reports/" >}}) insights across the entire organization by generating live reports in digestible, visual formats that are easily understood by non-technical stakeholders.
- [Query and create visualizations of your data]({{< relref "/guides/models/app/features/panels/query-panels/" >}}) that serve your custom needs.
- Configure custom automations that trigger key workflows for [model CI/CD]({{< relref "/guides/core/automations/model-registry-automations.md" >}}).
- [Protect sensitive strings using secrets]({{< relref "/guides/core/secrets.md" >}}).
- Configure automations that trigger key workflows for [model CI/CD]({{< relref "/guides/core/automations/" >}}).
5 changes: 2 additions & 3 deletions content/guides/core/artifacts/_index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
---
description: Overview of what W&B Artifacts are, how they work, and how to get started
using W&B Artifacts.
description: Overview of W&B Artifacts, how they work, and how to get started using them.
menu:
default:
identifier: artifacts
Expand Down Expand Up @@ -86,6 +85,6 @@ You can pass a custom path into the `root` [parameter]({{< relref "/ref/python/a

## Next steps
* Learn how to [version]({{< relref "./create-a-new-artifact-version.md" >}}) and [update]({{< relref "./update-an-artifact.md" >}}) artifacts.
* Learn how to trigger downstream workflows in response to changes to your artifacts with [artifact automation]({{< relref "/guides/core/automations/project-scoped-automations/" >}}).
* Learn how to trigger downstream workflows or notify a Slack channel in response to changes to your artifacts with [automations]({{< relref "/guides/core/automations/" >}}).
* Learn about the [registry]({{< relref "/guides/core/registry/" >}}), a space that houses trained models.
* Explore the [Python SDK]({{< relref "/ref/python/artifact.md" >}}) and [CLI]({{< relref "/ref/cli/wandb-artifact/" >}}) reference guides.
32 changes: 32 additions & 0 deletions content/guides/core/automations/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,36 @@ menu:
title: Automations
weight: 4
---
{{% pageinfo color="info" %}}
{{< readfile file="/_includes/enterprise-cloud-only.md" >}}
{{% /pageinfo %}}

This page describes _automations_ in W&B. [Create an automation]({{< relref "create-automations/" >}}) to trigger workflow steps, such as automated model testing and deployment, based on an event in W&B, such as when an [artifact]({{< relref "/guides/core/artifacts" >}}) artifact version is created.

For example, an automation can post to a Slack channel when a new version is created, or can run a webhook to trigger automated testing when the `production` alias is added to an artifact.

## Overview
An automation can run when a specific [event]({{< relref "automation-events.md" >}}) occurs in a registry or project.

For an artifact in a [Registry]({{< relref "/guides/core/registry/">}}), you can configure an automation to run:
- When a new artifact version is linked to a collection. For example, trigger testing and validation workflows for new candidate models.
- When an alias is added to an artifact version. For example, trigger a deployment workflow when an alias is added to a model version.

For an artifact in a [project]({{< relref "/guides/models/track/project-page.md" >}}), you can configure an automation to run:
- When a new version is added to an artifact. For example, start a training job when a new version of a dataset artifact is added to a given collection.
- When an alias is added to an artifact version. For example, trigger a PII redaction workflow when the alias "redaction" is added to a dataset artifact.

For more details, refer to [Automation events and scopes]({{< relref "automation-events.md" >}}).

To [create an automation]({{< relref "create-automations/" >}}), you:

1. If required, configure [secrets]({{< relref "/guides/core/secrets.md" >}}) for sensitive strings the automation requires, such as access tokens, passwords, or sensitive configuration details. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations to securely pass credentials or tokens to the webhook's external service without exposing it in plain text or hard-coding it in the webhook's payload.
1. Configure the webhook or Slack notification to authorize W&B to post to Slack or run the webhook on your behalf. A single automation action (webhook or Slack notification) can be used by multiple automations. These actions are defined in your **Team Settings**.
1. In the project or registry, create the automation:
1. Define the [event]({{< relref "#automation-events" >}}) to watch for, such as when a new artifact version is added.
1. Define the action to take when the event occurs (posting to a Slack channel or running a webhook). For a webhook, specify a secret to use for the access token and/or a secret to send with the payload, if required.

## Next steps
- [Create an automation]({{< relref "create-automations/" >}}).
- Learn about [Automation events and scopes]({{< relref "automation-events.md" >}}).
- [Create a secret]({{< relref "/guides/core/secrets.md" >}}).
55 changes: 55 additions & 0 deletions content/guides/core/automations/automation-events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
menu:
default:
identifier: automation-scopes
parent: automations
title: Automation events and scopes
weight: 2
---
{{% pageinfo color="info" %}}
{{< readfile file="/_includes/enterprise-cloud-only.md" >}}
{{% /pageinfo %}}

An automation can start when a specific event occurs within a project's or registrie's scope. The *scope* of a project refers to [INSERT tech def of scope]. This page describes the events that can trigger an automation within each scope.

Learn more about automations in the [Automations overview]({{< relref "/guides/core/automations/" >}}) or [Create an automation]({{< relref "create-automations/" >}}).

## Registry
This section describes the scopes and events for an automation in a [Registry]({{< relref "/guides/core/registry/">}}).

1. Navigate to the **Registry** App at https://wandb.ai/registry/.
1. Click the name of a registry, then view and create automations in the **Automations** tab.

Learn more about [creating automations]({{< relref "create-automations/" >}}).

### Scopes
You can create a Registry automation at these scopes:
- [Registry]({{< relref "/guides/core/registry/">}}) level: The automation watches for the event taking place on any collection within a specific registry, including collections added in the future.
- Collection level: A single collection in a specific registry.

### Events
A Registry automation can watch for these events:
- **Linking a new artifact to a collection**: Test and validate new models or datasets when they are added to a registry.
- **Adding a new alias to a version of an artifact**: Trigger a specific step of your workflow when a new artifact version has a specific alias applied. For example, deploy a model when it has the `production` alias applied.

## Project
This section describes the scopes and events for an automation in a [project]({{< relref "/guides/models/track/project-page.md" >}}).

1. Navigate to your W&B project on the W&B App at `https://wandb.ai/<team>/<project-name>`.
1. View and create automations in the **Automations** tab.

Learn more about [creating automations]({{< relref "create-automations/" >}}).

### Scopes
You can create a project automation at these scopes:
- Project level: The automation watches for the event taking place on any collection in the project.
- Collection level: All collections in the project that match the filter you specify.

### Events
A project automation can watch for these events:
- **A new version of an artifact is created in a collection**: Apply recurring actions to each version of an artifact. Specifying a collection is optional. For example, start a training job when a new dataset artifact version is created.
- **An artifact alias is added**: Trigger a specific step of your workflow when a new artifact version in a project or collection has a specific alias applied. For example, run a series of downstream processing steps when an artifact has the `test-set-quality-check` alias applied.

## Next steps
- [Create a Slack automation]({{< relref "create-automations/slack.md" >}})
- [Create a webhook automation]({{< relref "create-automations/webhook.md" >}})
49 changes: 49 additions & 0 deletions content/guides/core/automations/create-automations/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
menu:
default:
identifier: create-automations
parent: automations
title: Create an automation
weight: 1
---
{{% pageinfo color="info" %}}
{{< readfile file="/_includes/enterprise-cloud-only.md" >}}
{{% /pageinfo %}}

This page gives an overview of creating and managing W&B [automations]({{< relref "/guides/core/automations/">}}). For more detailed instructions, refer to [Create a Slack automation]({{< relref "/guides/core/automations/create-automations/slack.md" >}}) or [Create a webhook automation]({{< relref "/guides/core/automations/create-automations/webhook.md" >}}).

{{% alert %}}
Looking for companion tutorials for automations?
- [Learn to automatically triggers a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw).
- [Watch a video demonstrating automatically deploying a model to a Sagemaker endpoint](https://www.youtube.com/watch?v=s5CMj_w3DaQ).
- [Watch a video series introducing automations](https://youtube.com/playlist?list=PLD80i8An1OEGECFPgY-HPCNjXgGu-qGO6&feature=shared).
{{% /alert %}}

## Requirements
- A team admin can create and manage automations for the team's projects, as well as components of their automations, such as webhooks, secrets, or Slack connections. Refer to [Team settings]({{< relref "/guides/models/app/settings-page/team-settings/" >}}).
- To create a registry automation, you must have access to the registry. Refer to [Configure Registry access]({{< relref "/guides/core/registry/configure_registry.md#registry-roles" >}}).
- To create a Slack automation, you must have permission to post to the Slack instance and channel you select.

## Create an automation
Create an automation from the project or registry's **Automations** tab. At a high level, to create an automation, follow these steps:

1. If necessary, [create a W&B secret]({{< relref "/guides/core/secrets.md" >}}) for each sensitive string required by the automation, such as an access token, password, or SSH key. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations.
1. Configure the webhook or Slack notification to authorize W&B to post to Slack or run the webhook on your behalf. A single automation action (webhook or Slack notification) can be used by multiple automations. These actions are defined in your **Team Settings**.
1. In the project or registry, create the automation, which specifies the event to watch for and the action to take (such as posting to Slack or running a webhook). When you create a webhook automation, you configure the payload it send.

For details, refer to:

- [Create a Slack automation]({{< relref "slack.md" >}})
- [Create a webhook automation]({{< relref "webhook.md" >}})

## View and manage automations
View and manage automations from a project or registry's **Automations** tab.

- To view an automation's details, click its name.
- To edit an automation, click its action `...` menu, then click **Edit automation**.
- To delete an automation, click its action `...` menu, then click **Delete automation**.

## Next steps
- [Create a Slack automation]({{< relref "slack.md" >}}).
- [Create a webhook automation]({{< relref "webhook.md" >}}).
- [Create a secret]({{< relref "/guides/core/secrets.md" >}}).
102 changes: 102 additions & 0 deletions content/guides/core/automations/create-automations/slack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
menu:
default:
identifier: create-slack-automations
parent: create-automations
title: Create a Slack automation
weight: 1
---
{{% pageinfo color="info" %}}
{{< readfile file="/_includes/enterprise-cloud-only.md" >}}
{{% /pageinfo %}}

This page shows how to create a Slack [automation]({{< relref "/guides/core/automations/" >}}> ). To create a webhook automation, refer to [Create a webhook automation]({{< relref "/guides/core/automations/create-automations/webhook.md" >}}) instead.

At a high level, to create a Slack automation, you take these steps:
1. [Add a Slack integration]({{< relref "#add-a-slack-channel" >}}), which authorizes W&B to post to the Slack instance and channel.
1. [Create the Slack automation]({{< relref "#create-slack-automation" >}}), which defines the [event]({{< relref "/guides/core/automations/automation-events.md" >}}) to watch for and the channel to post to.

## Connect to Slack
A team admin can add a Slack destination to the team.

1. Log in to W&B and go to Team Settings page.
1. In the **Slack channel integrations** section, click **Connect Slack** to add a new Slack instance. To add a channel for an existing Slack instance, click **New integration**.

If necessary, sign in to Slack in your browser. When prompted, grant W&B permission to post to the Slack channel you select. Read the page, then click **Search for a channel** and begin typing the channel name. Select the channel from the list, then click **Allow**.

1. In Slack, go to the channel you selected. If you see a post like `[Your Slack handle] added an integration to this channel: Weights & Biases`, the integration is configured correctly.

Now you can [create an automation]({{< relref "#create-a-slack-automation" >}}) that notifies the Slack channel you configured.

## View and manage Slack connections
A team admin can view and manage the team's Slack instances and channels.

1. Log in to W&B and go to **Team Settings**.
1. View each Slack destination in the **Slack channel integrations** section.
1. Delete a destination by clicking its trash icon.

## Create an automation
After you [connect your W&B team to Slack]({{< relref "#connect-to-slack" >}}), select **Registry** or **Project**, then follow these steps to create an automation that notifies the Slack channel.

{{< tabpane text=true >}}
{{% tab "Registry" %}}
A Registry admin can create automations in that registry.

1. Log in to W&B.
1. Click the name of a registry to view its details,
1. To create an automation scoped to the registry, click the **Automations** tab, then click **Create automation**. An automation that is scoped to a registry is automatically applied to all of its collections (including those created in the future).

To create an automation scoped only to a specific collection in the registry, click the collection's action `...` menu, then click **Create automation**. Alternatively, while viewing a collection, create an automation for it using the **Create automation** button in the **Automations** section of the collection's details page.
1. Choose the [**Event**]({{< relref "/guides/core/automations/automation-events.md" >}}) to watch for.

Fill in any additional fields that appear, which depend upon the event. For example, if you select **An artifact alias is added**, you must specify the **Alias regex**.

Click **Next step**.
1. Select the team that owns the [Slack integration]({{< relref "#add-a-slack-integration" >}}).
1. Set **Action type** to **Slack notification**. Select the Slack channel, then click **Next step**.
1. Provide a name for the automation. Optionally, provide a description.
1. Click **Create automation**.

{{% /tab %}}
{{% tab "Project" %}}
A W&B admin can create automations in a project.

1. Log in to W&B.
1. Go the project page and click the **Automations** tab.
1. Click **Create automation**.
1. Choose the [**Event**]({{< relref "/guides/core/automations/automation-events.md" >}}) to watch for.

Fill in any additional fields that appear, which depend upon the event. For example, if you select **An artifact alias is added**, you must specify the **Alias regex**.

Click **Next step**.
1. Select the team that owns the [Slack integration]({{< relref "#add-a-slack-integration" >}}).
1. Set **Action type** to **Slack notification**. Select the Slack channel, then click **Next step**.
1. Provide a name for the automation. Optionally, provide a description.
1. Click **Create automation**.

{{% /tab %}}
{{< /tabpane >}}

## View and manage automations

{{< tabpane text=true >}}
{{% tab "Registry" %}}

- Manage the registry's automations from the registry's **Automations** tab.
- Mamage a collection's automations from the **Automations** section of the collection's details page.

From either of these pages, a Registry admin can manage existing automations:
- To view an automation's details, click its name.
- To edit an automation, click its action `...` menu, then click **Edit automation**.
- To delete an automation, click its action `...` menu, then click **Delete automation**. Confiruation is required.


{{% /tab %}}
{{% tab "Project" %}}
A W&B admin can view and manage a project's automations from the project's **Automations** tab.

- To view an automation's details, click its name.
- To edit an automation, click its action `...` menu, then click **Edit automation**.
- To delete an automation, click its action `...` menu, then click **Delete automation**. Confiruation is required.
{{% /tab %}}
{{< /tabpane >}}
Loading