Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scarf checker script and CI workflow #2227

Merged
merged 7 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/scarf-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Scarf checker
on:
workflow_dispatch:
pull_request:
types: [opened, synchronize, ready_for_review]
# Run when md files are involved in the PR.
paths:
- '**.md'
jobs:
check-for-scarf:
# Only run on non-draft PRs within the same repository.
if: github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.draft == false
name: check-for-scarf
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/[email protected]

- name: Check for scarf
run: bash scripts/scarf-checker.sh
4 changes: 4 additions & 0 deletions docs/book/deploying-zenml/zenml-cloud/user-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,7 @@ initial Role. The User will be sent an invitation email. If a user is part of an
organization, they can utilize their login on all tenants they have authority to
access.

<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


4 changes: 4 additions & 0 deletions docs/book/reference/how-do-i.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,7 @@ Upgrading your ZenML client package is as simple as running `pip install --upgra
* use a \<YOUR\_COMPONENT\_GOES\_HERE> stack component?

For information on how to use a specific stack component, please refer to [the component guide](../stacks-and-components/component-guide/component-guide.md) which contains all our tips and advice on how to use each integration and component with ZenML.
<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
# Custom stack solutions

<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
Expand Up @@ -201,4 +201,7 @@ delete the project you created.

```bash
gcloud project delete <PROJECT_ID_OR_NUMBER>
```
```<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
Expand Up @@ -241,3 +241,7 @@ zenml stack update -e mlflow_tracker
```

This also applies to all other [stack components](../../../stacks-and-components/component-guide/component-guide.md).
<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,7 @@ are using a remote orchestrator, you would then have to place the updated
versions for the dependencies in a `DockerSettings` object (described in detail
[here](https://docs.zenml.io/advanced-guide/pipelines/containerization#how-to-install-additional-pip-dependencies-or-apt-packages))
which will then make sure everything is working as you need.
<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,7 @@ export ZENML_SUPPRESS_LOGS=false
For more information on server configuration, see the [ZenML Server
documentation](../../../deploying-zenml/zenml-self-hosted/deploy-with-docker.md)
for more, especially the section entitled "ZenML server configuration options".
<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,7 @@ description: Focusing on the advanced usage of ZenML pipelines
# Pipelining features

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><mark style="color:purple;"><strong>Configure steps/pipelines</strong></mark></td><td>Configuring pipelines, steps, and stack components in ZenML.</td><td><a href="configure-steps-pipelines.md">configure-steps-pipelines.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Compose pipelines</strong></mark></td><td>Composing your ZenML pipelines.</td><td><a href="compose-pipelines.md">compose-pipelines.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Schedule pipeline runs</strong></mark></td><td>Planning runs to add automation to your pipelines.</td><td><a href="schedule-pipeline-runs.md">schedule-pipeline-runs.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Fetch metadata within steps</strong></mark></td><td>Accessing meta information in real-time within your pipeline.</td><td><a href="fetch-metadata-within-steps.md">fetch-metadata-within-steps.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Fetch metadata during pipeline composition</strong></mark></td><td>Accessing meta information during pipeline composition.</td><td><a href="fetch-metadata-within-pipeline.md">fetch-metadata-within-pipeline.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Use failure/success hooks</strong></mark></td><td>Running failure and success hooks after step execution.</td><td><a href="use-failure-success-hooks.md">use-failure-success-hooks.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Hyperparameter tuning</strong></mark></td><td>Running a hyperparameter tuning trial with ZenML.</td><td><a href="hyper-parameter-tuning.md">hyper-parameter-tuning.md</a></td></tr></tbody></table>
<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>


24 changes: 24 additions & 0 deletions scripts/format.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,27 @@ ruff $SRC --select F401,F841 --fix --exclude "__init__.py" --isolated
# sorts imports
ruff $SRC --select I --fix --ignore D
ruff format $SRC

set +x
# Adds scarf snippet to docs files where it is missing
# Text to be searched in each file
search_text='https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc'

# Text block to be appended
append_text='<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>

'

# Find all .md files in ./docs/book, excluding toc.md
while IFS= read -r file; do
# Check if the file does not contain the search text
if ! grep -qF "$search_text" "$file"; then
echo "Appending scarf to file '$file'."
# Append the text block and a final blank line
echo "$append_text" >> "$file"
fi
done < <(find docs/book -type f -name '*.md' ! -name 'toc.md')

echo "Any needed / missing scarfs have now been added."
set -x
23 changes: 23 additions & 0 deletions scripts/scarf-checker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash

# Text to be searched in each file
search_text='https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc'

# Flag to track if error is found
error_found=0

# Find all .md files in ./docs/book, excluding toc.md
while IFS= read -r file; do
# Check if the file contains the search text
if ! grep -qF "$search_text" "$file"; then
echo "Error: File '$file' does not contain the scarf snippet."
error_found=1
fi
done < <(find docs/book -type f -name '*.md' ! -name 'toc.md')

# Exit with error if any file did not contain the text
if [ "$error_found" -eq 1 ]; then
exit 1
fi

echo "All files contain the scarf snippet."
Loading