Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS-1166][DOCS-880][DOCS-1167][DOCS-1168] Automation updates part 1 of 2 #1084

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
3 changes: 1 addition & 2 deletions content/guides/core/artifacts/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,5 @@ You can pass a custom path into the `root` [parameter]({{< relref "/ref/python/a

## Next steps
* Learn how to [version]({{< relref "./create-a-new-artifact-version.md" >}}) and [update]({{< relref "./update-an-artifact.md" >}}) artifacts.
* Learn how to trigger downstream workflows in response to changes to your artifacts with [artifact automation]({{< relref "/guides/models/automations/project-scoped-automations/" >}}).
* Learn about the [registry]({{< relref "/guides/models/registry/" >}}), a space that houses trained models.
* Learn how to trigger downstream workflows or notify a Slack channel in response to changes to your artifacts or registered models with [automations]({{< relref "/guides/models/automations/" >}}).
* Explore the [Python SDK]({{< relref "/ref/python/artifact.md" >}}) and [CLI]({{< relref "/ref/cli/wandb-artifact/" >}}) reference guides.
2 changes: 1 addition & 1 deletion content/guides/hosting/iam/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ For more information, see [Add and manage teams]({{< relref "./access-management
A *Project* is a subscope within a team, that maps to an actual AI project with specific intended outcomes. You may have more than one project within a team. Each project has a visibility mode which determines who can access it.


Every project is comprised of [Workspaces]({{< relref "/guides/models/track/workspaces.md" >}}) and [Reports]({{< relref "/guides/core/reports/" >}}), and is linked to relevant [Artifacts]({{< relref "/guides/core/artifacts/" >}}), [Sweeps]({{< relref "/guides/models/sweeps/" >}}), [Launch Jobs]({{< relref "/launch/" >}}) and [Automations]({{< relref "/guides/models/automations/project-scoped-automations.md" >}}).
Every project is comprised of [Workspaces]({{< relref "/guides/models/track/workspaces.md" >}}) and [Reports]({{< relref "/guides/core/reports/" >}}), and is linked to relevant [Artifacts]({{< relref "/guides/core/artifacts/" >}}), [Sweeps]({{< relref "/guides/models/sweeps/" >}}), and [Automations]({{< relref "/guides/models/automations/" >}}).
25 changes: 14 additions & 11 deletions content/guides/integrations/huggingface.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,14 +183,13 @@ Using TensorFlow? Just swap the PyTorch `Trainer` for the TensorFlow `TFTrainer`
### Turn on model checkpointing


Using W&B's [Artifacts]({{< relref "/guides/core/artifacts/" >}}), you can store up to 100GB of models and datasets for free and then use the W&B [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) to register models to prepare them for staging or deployment in your production environment.
Using [Artifacts]({{< relref "/guides/core/artifacts/" >}}), you can store up to 100GB of models and datasets for free and then use the Weights & Biases [Registry]({{< relref "/guides/models/registry/" >}}). Using Registry, you can register models to explore and evaluate them, prepare them for staging, or deploy them in your production environment.

Logging your Hugging Face model checkpoints to Artifacts can be done by setting the `WANDB_LOG_MODEL` environment variable to one of `end` or `checkpoint` or `false`:
To log your Hugging Face model checkpoints to Artifact, set the `WANDB_LOG_MODEL` environment variable:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

Suggested change
To log your Hugging Face model checkpoints to Artifact, set the `WANDB_LOG_MODEL` environment variable:
To log your Hugging Face model checkpoints to Artifacts, set the `WANDB_LOG_MODEL` environment variable to one of the following:


- **`checkpoint`**: a checkpoint will be uploaded every `args.save_steps` from the [`TrainingArguments`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments).
- **`end`**: the model will be uploaded at the end of training.

Use `WANDB_LOG_MODEL` along with `load_best_model_at_end` to upload the best model at the end of training.
- **`checkpoint`**: Upload a checkpoint every `args.save_steps` from the [`TrainingArguments`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments).
- **`end`**: Upload the model at the end of training, if `load_best_model_at_end` is also set.
- **`false`**: Do not upload the model.


{{< tabpane text=true >}}
Expand Down Expand Up @@ -231,10 +230,10 @@ By default, your model will be saved to W&B Artifacts as `model-{run_id}` when `
However, If you pass a [`run_name`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments.run_name) in your `TrainingArguments`, the model will be saved as `model-{run_name}` or `checkpoint-{run_name}`.
{{% /alert %}}

#### W&B Model Registry
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the **[Model Registry]({{< relref "/guides/models/registry/model_registry/" >}})**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate]({{< relref "/guides/models/automations/project-scoped-automations/#create-a-webhook-automation" >}}) downstream actions with webhooks or jobs.
#### W&B Registry
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **[Registry]({{< relref "/guides/models/registry/" >}})**. Using Registry, you can organize your best models by task, manage the lifecycles of models, track and audit the entire ML lifecyle, and [automate]({{< relref "/guides/models/automations/#create-webhook-automation" >}}) downstream actions with webhooks or jobs.
Comment on lines +233 to +234

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

  • I think the convention in most places is to just refer to it as "Registry" (without the definite article)
  • Launch is deprecated, but also more generally, automations aren't necessarily intended as specific to just webhooks or Launch actions
Suggested change
#### W&B Registry
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **[Registry]({{< relref "/guides/models/registry/" >}})**. Using Registry, you can organize your best models by task, manage the lifecycles of models, track and audit the entire ML lifecyle, and [automate]({{< relref "/guides/models/automations/#create-webhook-automation" >}}) downstream actions with webhooks or jobs.
#### W&B Registry
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team with **[Registry]({{< relref "/guides/models/registry/" >}})**. Using Registry, you can organize your best models by task, manage the lifecycles of models, track and audit the entire ML lifecyle, and [automate]({{< relref "/guides/models/automations/#create-webhook-automation" >}}) downstream actions.


See the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) documentation for how to link a model Artifact to the Model Registry.
To link a model Artifact, refer to [Registry]({{< relref "/guides/models/registry/" >}}).

### Visualise evaluation outputs during training

Expand Down Expand Up @@ -268,9 +267,13 @@ Once you have logged your training results you can explore your results dynamica
## Advanced features and FAQs

### How do I save the best model?
If `load_best_model_at_end=True` is set in the `TrainingArguments` that are passed to the `Trainer`, then W&B will save the best performing model checkpoint to Artifacts.
If your code sets `load_best_model_at_end=True` in the `TrainingArguments` you pass to the `Trainer`, W&B saves the best performing model checkpoint to Artifacts.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

Suggested change
If your code sets `load_best_model_at_end=True` in the `TrainingArguments` you pass to the `Trainer`, W&B saves the best performing model checkpoint to Artifacts.
If you pass `TrainingArguments` with `load_best_model_at_end=True` to your `Trainer`, W&B saves the best-performing model checkpoint to Artifacts.


If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}).
If you save your model checkpoints as Artifacts, you can promote them to the [Registry]({{< relref "/guides/models/registry/" >}}). In Registry, you can:
- Organize your best model versions by ML task.
- Centralize models and share them with your team.
- Stage models for production or bookmark them for further evaluation.
- Trigger downstream CI/CD processes.

### How do I load a saved model?

Expand Down
2 changes: 1 addition & 1 deletion content/guides/integrations/lightning.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,7 +445,7 @@ The model checkpoints you log are viewable through the [W&B Artifacts]({{< relre

To bookmark your best model checkpoints and centralize them across your team, you can link them to the [W&B Model Registry]({{< relref "/guides/models" >}}).

Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate]({{< relref "/guides/models/automations/project-scoped-automations/#create-a-webhook-automation" >}}) downstream actions with webhooks or jobs.
Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate]({{< relref "/guides/models/automations/#create-webhook-automation" >}}) downstream actions with webhooks or jobs.

## Log images, text, and more

Expand Down
2 changes: 1 addition & 1 deletion content/guides/integrations/openai-fine-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ The datasets are visualized as W&B Tables, which allows you to explore, search,

OpenAI gives you an id of the fine-tuned model. Since we don't have access to the model weights, the `WandbLogger` creates a `model_metadata.json` file with all the details (hyperparameters, data file ids, etc.) of the model along with the `fine_tuned_model`` id and is logged as a W&B Artifact.

This model (metadata) artifact can further be linked to a model in the [W&B Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}).
This model (metadata) artifact can further be linked to a model in the [W&B Registry]({{< relref "/guides/models/registry/" >}}).

{{< img src="/images/integrations/openai_model_metadata.png" alt="" >}}

Expand Down
2 changes: 1 addition & 1 deletion content/guides/integrations/spacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ model_log_interval = 1000
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `project_name` | `str`. The name of the W&B Project. The project will be created automatically if it doesn’t exist yet. |
| `remove_config_values` | `List[str]` . A list of values to exclude from the config before it is uploaded to W&B. `[]` by default. |
| `model_log_interval` | `Optional int`. `None` by default. If set, [model versioning]({{< relref "/guides/models/registry/model_registry/" >}}) with [Artifacts]({{< relref "/guides/core/artifacts/" >}})will be enabled. Pass in the number of steps to wait between logging model checkpoints. `None` by default. |
| `model_log_interval` | `Optional int`. `None` by default. If set, enables [model versioning]({{< relref "/guides/models/registry/" >}}) with [Artifacts]({{< relref "/guides/core/artifacts/" >}}). Pass in the number of steps to wait between logging model checkpoints. `None` by default. |
| `log_dataset_dir` | `Optional str`. If passed a path, the dataset will be uploaded as an Artifact at the beginning of training. `None` by default. |
| `entity` | `Optional str` . If passed, the run will be created in the specified entity |
| `run_name` | `Optional str` . If specified, the run will be created with the specified name. |
Expand Down
2 changes: 1 addition & 1 deletion content/guides/integrations/yolov5.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Just follow the links printed to the standard out by wandb.

By passing a few simple command line arguments to YOLO, you can take advantage of even more W&B features.

* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/model_registry/" >}}). At the end of every `save_period` epochs, the model weights will be saved to W&B. The best-performing model on the validation set will be tagged automatically.
* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/" >}}). At the end of every `save_period` epochs, saves the model weights to W&B and tags the best performing model on the validation set.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

Suggested change
* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/" >}}). At the end of every `save_period` epochs, saves the model weights to W&B and tags the best performing model on the validation set.
* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/" >}}). At the end of every `save_period` epochs, W&B will save the model weights and tag the best-performing model on the validation set.

* Turning on the `--upload_dataset` flag will also upload the dataset for data versioning.
* Passing a number to `--bbox_interval` will turn on [data visualization]({{< relref "../" >}}). At the end of every `bbox_interval` epochs, the outputs of the model on the validation set will be uploaded to W&B.

Expand Down
2 changes: 1 addition & 1 deletion content/guides/models/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ With W&B Models, you can:
- Track and visualize all [ML experiments]({{< relref "./track/" >}}).
- Optimize and fine-tune models at scale with [hyperparameter sweeps]({{< relref "./sweeps/" >}}).
- [Maintain a centralized hub of all models]({{< relref "./registry/" >}}), with a seamless handoff point to devops and deployment
- Configure custom automations that trigger key workflows for [model CI/CD]({{< relref "./automations/model-registry-automations.md" >}}).
- Configure custom automations that trigger key workflows for [model CI/CD]({{< relref "./automations/" >}}).

Machine learning practitioners rely on W&B Models as their ML system of record to track and visualize experiments, manage model versions and lineage, and optimize hyperparameters.
12 changes: 6 additions & 6 deletions content/guides/models/app/settings-page/teams.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,19 +68,19 @@ Team settings allow you to manage the settings for your team and its members. Wi
| Remove team members | | | X |
| Manage team settings| | | X |

### Model Registry
### Registry
The proceeding table lists permissions that apply to all projects across a given team.

| Permissions | View-Only | Team Member | Model Registry Admin | Team Admin |
| Permissions | View-Only | Team Member | Registry Admin | Team Admin |
| ---------------------------| --------- | ----------- | -------------- | ---------- |
| Add aliases | | X | X | X |
| Add models to the registry | | X | X | X |
| View models in the registry| X | X | X | X |
|Download models | X | X | X | X |
|Add/Remove Registry Admins | | | X | X |
|Add/Remove Protected Aliases| | | X | |
| Download models | X | X | X | X |
|Add or remove Registry Admins | | | X | X |
|Add or remove Protected Aliases| | | X | |

See the [Model Registry]({{< relref "/guides/models/registry/model_registry/access_controls.md" >}}) chapter for more information about protected aliases.
For more details about protected aliases, refer to [Registry Access Controls]({{< relref "/guides/models/registry/model_registry/access_controls.md" >}}).

### Reports
Report permissions grant access to create, view, and edit reports. The proceeding table lists permissions that apply to all reports across a given team.
Expand Down
Loading