-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOCS-1166][DOCS-880][DOCS-1167][DOCS-1168] Automation updates part 1 of 2 #1084
Open
mdlinville
wants to merge
12
commits into
main
Choose a base branch
from
automation_updates
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 2 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
2e08a16
[DOCS-1166] Update all possible links and references to old Model Reg…
mdlinville 582d4db
[DOCS-880][DOCS-1167][DOCS-1168] Consolidate project-scoped and model…
mdlinville 643b6c3
Style
mdlinville 064288d
Markdown mistake
mdlinville f4bfe72
Fix links
mdlinville 0f55a71
Fix another broken link
mdlinville a37335e
Merge remote-tracking branch 'origin/main' into automation_updates
mdlinville 22d0ccc
Merge branch 'main' into automation_updates
mdlinville db6d7fa
Merge branch 'main' into automation_updates
mdlinville 5b201a4
Merge branch 'main' into automation_updates
mdlinville 7a801c0
Stephen's round 1 feedback
mdlinville c526e25
Apply suggestions from code review
mdlinville File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -183,14 +183,13 @@ Using TensorFlow? Just swap the PyTorch `Trainer` for the TensorFlow `TFTrainer` | |||||
### Turn on model checkpointing | ||||||
|
||||||
|
||||||
Using W&B's [Artifacts]({{< relref "/guides/core/artifacts/" >}}), you can store up to 100GB of models and datasets for free and then use the W&B [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) to register models to prepare them for staging or deployment in your production environment. | ||||||
Using [Artifacts]({{< relref "/guides/core/artifacts/" >}}), you can store up to 100GB of models and datasets for free and then use the Weights & Biases [Registry]({{< relref "/guides/models/registry/" >}}). Using Registry, you can register models to explore and evaluate them, prepare them for staging, or deploy them in your production environment. | ||||||
|
||||||
Logging your Hugging Face model checkpoints to Artifacts can be done by setting the `WANDB_LOG_MODEL` environment variable to one of `end` or `checkpoint` or `false`: | ||||||
To log your Hugging Face model checkpoints to Artifact, set the `WANDB_LOG_MODEL` environment variable: | ||||||
|
||||||
- **`checkpoint`**: a checkpoint will be uploaded every `args.save_steps` from the [`TrainingArguments`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments). | ||||||
- **`end`**: the model will be uploaded at the end of training. | ||||||
|
||||||
Use `WANDB_LOG_MODEL` along with `load_best_model_at_end` to upload the best model at the end of training. | ||||||
- **`checkpoint`**: Upload a checkpoint every `args.save_steps` from the [`TrainingArguments`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments). | ||||||
- **`end`**: Upload the model at the end of training, if `load_best_model_at_end` is also set. | ||||||
- **`false`**: Do not upload the model. | ||||||
|
||||||
|
||||||
{{< tabpane text=true >}} | ||||||
|
@@ -231,10 +230,10 @@ By default, your model will be saved to W&B Artifacts as `model-{run_id}` when ` | |||||
However, If you pass a [`run_name`](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments.run_name) in your `TrainingArguments`, the model will be saved as `model-{run_name}` or `checkpoint-{run_name}`. | ||||||
{{% /alert %}} | ||||||
|
||||||
#### W&B Model Registry | ||||||
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the **[Model Registry]({{< relref "/guides/models/registry/model_registry/" >}})**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate]({{< relref "/guides/models/automations/project-scoped-automations/#create-a-webhook-automation" >}}) downstream actions with webhooks or jobs. | ||||||
#### W&B Registry | ||||||
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **[Registry]({{< relref "/guides/models/registry/" >}})**. Using Registry, you can organize your best models by task, manage the lifecycles of models, track and audit the entire ML lifecyle, and [automate]({{< relref "/guides/models/automations/project-scoped-automations/#create-a-webhook-automation" >}}) downstream actions with webhooks or jobs. | ||||||
|
||||||
See the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) documentation for how to link a model Artifact to the Model Registry. | ||||||
To link a model Artifact, refer to [Registry]({{< relref "/guides/models/registry/" >}}). | ||||||
|
||||||
### Visualise evaluation outputs during training | ||||||
|
||||||
|
@@ -268,9 +267,13 @@ Once you have logged your training results you can explore your results dynamica | |||||
## Advanced features and FAQs | ||||||
|
||||||
### How do I save the best model? | ||||||
If `load_best_model_at_end=True` is set in the `TrainingArguments` that are passed to the `Trainer`, then W&B will save the best performing model checkpoint to Artifacts. | ||||||
If your code sets `load_best_model_at_end=True` in the `TrainingArguments` you pass to the `Trainer`, W&B saves the best performing model checkpoint to Artifacts. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion:
Suggested change
|
||||||
|
||||||
If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}). | ||||||
If you save your model checkpoints as Artifacts, you can promote them to the [Registry]({{< relref "/guides/models/registry/" >}}). In Registry, you can: | ||||||
- Organize your best model versions by ML task. | ||||||
- Centralize models and share them with your team. | ||||||
- Stage models for production or bookmark them for further evaluation. | ||||||
- Trigger downstream CI/CD processes. | ||||||
|
||||||
### How do I load a saved model? | ||||||
|
||||||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -34,7 +34,7 @@ Just follow the links printed to the standard out by wandb. | |||||
|
||||||
By passing a few simple command line arguments to YOLO, you can take advantage of even more W&B features. | ||||||
|
||||||
* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/model_registry/" >}}). At the end of every `save_period` epochs, the model weights will be saved to W&B. The best-performing model on the validation set will be tagged automatically. | ||||||
* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/" >}}). At the end of every `save_period` epochs, saves the model weights to W&B and tags the best performing model on the validation set. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion:
Suggested change
|
||||||
* Turning on the `--upload_dataset` flag will also upload the dataset for data versioning. | ||||||
* Passing a number to `--bbox_interval` will turn on [data visualization]({{< relref "../" >}}). At the end of every `bbox_interval` epochs, the outputs of the model on the validation set will be uploaded to W&B. | ||||||
|
||||||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: