[DOCS-1166][DOCS-880][DOCS-1167][DOCS-1168] Automation updates part 1 of 2 #1084
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Deploying docs with
|
| Latest commit: |
db6d7fa
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://54a9c0c3.docodile.pages.dev |
| Branch Preview URL: | https://automation-updates.docodile.pages.dev |
| Every project is comprised of [Workspaces]({{< relref "/guides/models/track/workspaces.md" >}}) and [Reports]({{< relref "/guides/core/reports/" >}}), and is linked to relevant [Artifacts]({{< relref "/guides/core/artifacts/" >}}), [Sweeps]({{< relref "/guides/models/sweeps/" >}}), [Launch Jobs]({{< relref "/launch/" >}}) and [Automations]({{< relref "/guides/models/automations/" >}}). |
There was a problem hiding this comment.
I think based on tristan's comments elsewhere that we want to remove references to launch in docs.
| An automation's triggering event depends on the automation's scope. | ||
|
|
||
| ### Registry | ||
| For a [registered model]({{< relref "/guides/models/registry/">}}), you can configure an automation to run on these events: |
There was a problem hiding this comment.
I believe there's no restriction to just the registered model types, it's any datatype in any collection.
I'm pretty sure the term "registered model" is not a thing anymore, it's an artifact (!) of when that was the only type of registry, but I might be wrong.
(more instances of this below, but I won't call them out anymore)
| - **Adding a new alias to a version of the registered model**: Trigger a special step of your workflow when a new model version has a label or alias applied. For example, deploy a model when it has the `deploy` alias applied. | ||
|
|
||
| ### Project | ||
| For a model in a project that is not in a registry, you can configure an automation to run on these events: |
There was a problem hiding this comment.
ditto above - not just about models, any artifact.
| - **A new version of an artifact is created in a collection**: Apply recurring actions to each version of an artifact. For example, start a training job when a new dataset artifact version is created. | ||
| - **An artifact alias is added**: Apply recurring actions when a specific alias is applied to an artifact version. For example, run a series of downstream processing steps when an artifact gains the `test-set-quality-check` alias. | ||
|
|
||
| ### Run |
There was a problem hiding this comment.
(I know we discussed in slack, but flagging here since it's convenient) We should keep mentions of the run metric automations out of the docs until the feature is generally released.
| - **Run metric threshold met**: Apply recurring actions based on a run metric's value and whether it is above, below, or exactly equal to the threshold. Optionally, specify how to average results. | ||
| - **Run metric change threshold met**: Apply recurring actions based on whether a run metric's amount of change (absolute or relative) is above, below, or exactly equal to the threshold. Optionally, specify how to average results. | ||
|
|
||
| ## Evant actions |
There was a problem hiding this comment.
| ## Evant actions | |
| ## Event actions |
There was a problem hiding this comment.
I think it's probably worth keeping the concepts for events and actions separate, so I think it's a bit confusing to call something "event actions" :)
I would probably just call them actions
| Specify the secrets you want to use for your webhook automation when you configure the webhook. See the [Configure a webhook]({{< relref "#configure-a-webhook" >}}) section for more information. | ||
|
|
||
| {{% alert %}} | ||
| Once you create a secret, you can access that secret in your W&B workflows with `$`. |
There was a problem hiding this comment.
I don't know what a workflow means here, and I don't really know what this mean. Does it mean that you can reference a secret in a webhook payload using $my_secret_name ?
| 1. Provide a name for the webhook. | ||
| 1. Provide the endpoint URL for the webhook. | ||
| 1. If the webhook authenticates using an access token, set **Access token** to the name of the secret that contains the access token. When you configure an automation that uses this webhook, the access token will be available in the `$ACCESS_TOKEN` environment variable, and the HTTP header will have `Authorization: Bearer $ACCESS_TOKEN`. | ||
| 1. If the webhook validates its payload using a sensitive string, set **Secret** to the name of the secret that contains the sensitive string. |
There was a problem hiding this comment.
So this doesn't really tell the user what happens to this value when we send the webhook. Is it sent in the body of the payload? is it available in the payload as $my_secret_name? is it sent in a header named secret?
So, TODO item for me: figure out what this does (unless @tonyyli-wandb knows off the top of your head?)
| 1. Click **Test** to test the webhook. Optionally, provide a payload to test. Any payload you specify in this step does not persist, and will need to be specified when you [create the webhook automation]({{< relref "#create-webhook-automation" >}}). | ||
|
|
||
| {{% alert %}} | ||
| See the [Troubleshoot your webhook]({{< relref "#troubleshoot-your-webhook" >}}) section to view where the secret and access token are specified in |
There was a problem hiding this comment.
my opinion: we should just remove this alert and fix the above to tell the user :)
| 1. Choose the **Event** which triggers the automation. If applicable, provide options that are specific to the event type. If your project has no registries, registry events will not be available. Click **Next step**. | ||
| 1. Select the team where you added the webhook. | ||
| 1. Set **Action type** to **Webhook**, then select the webhook. | ||
| 1. Provide the payload for the webhook in **Payload**. Refer to the reference for varialbles that you can use. For details, refer to the [Example webhook payloads]({{< relref "#example-webhook-payloads" >}}) section. |
There was a problem hiding this comment.
| 1. Provide the payload for the webhook in **Payload**. Refer to the reference for varialbles that you can use. For details, refer to the [Example webhook payloads]({{< relref "#example-webhook-payloads" >}}) section. | |
| 1. Provide the payload for the webhook in **Payload**. Refer to the reference for variables that you can use. For details, refer to the [Example webhook payloads]({{< relref "#example-webhook-payloads" >}}) section. |
| The following tabs demonstrate example payloads based on common use cases. Within the examples they reference the following keys to refer to condition objects in the payload parameters: | ||
| * `${event_type}` Refers to the type of event that triggered the action. | ||
| * `${event_author}` Refers to the user that triggered the action. | ||
| * `${artifact_version}` Refers to the specific artifact version that triggered the action. Passed as an artifact instance. |
There was a problem hiding this comment.
We need to answer the question: what does it mean that something is 'passed as an artifact instance'?
DOCS-1166 Update all possible links and references to old Model Registry
[DOCS-880]DOCS-1167 Consolidate project-scoped and model registry automations, new Slack notification flow, add redirects
Added @tonyyli-wandb @ibindlish for round 1 of technical review