Skip to content

Commit ca80fee

Browse files
atheo89atheo89
committedDec 12, 2024
Update piplock renewal gh action
Remove old obsolete files Include codeserver notebook on the refresh-piplock-files recipe Update refresh-pipfilelock-files to accept optional file directories for updates Update documentation in regards the Pipfile lock generation
1 parent 33f95dd commit ca80fee

8 files changed

+130
-222
lines changed
 

‎.github/workflows/piplock-renewal-2023a.yml

-47
This file was deleted.

‎.github/workflows/piplock-renewal-2023b.yml

-47
This file was deleted.

‎.github/workflows/piplock-renewal-2024a.yml

-47
This file was deleted.

‎.github/workflows/piplock-renewal-main.yml

-47
This file was deleted.

‎.github/workflows/piplock-renewal.yaml

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
# This GitHub action is meant to update the pipfile.locks
3+
name: Pipfile.locks Renewal Action
4+
5+
on: # yamllint disable-line rule:truthy
6+
# Triggers the workflow every Monday at 22pm UTC am 0 22 * * 1
7+
schedule:
8+
- cron: "0 22 * * 1"
9+
workflow_dispatch: # for manual trigger workflow from GH Web UI
10+
inputs:
11+
branch:
12+
description: 'Specify branch'
13+
required: false
14+
default: 'main'
15+
python_version:
16+
description: 'Select Python version to update Pipfile.lock'
17+
required: false
18+
default: '3.11'
19+
type: choice
20+
options:
21+
- '3.11'
22+
- '3.9'
23+
- '3.8'
24+
update_optional_dirs:
25+
description: 'Include optional directories in update'
26+
required: false
27+
default: 'false'
28+
type: choice
29+
options:
30+
- 'true'
31+
- 'false'
32+
33+
jobs:
34+
build:
35+
runs-on: ubuntu-latest
36+
permissions:
37+
contents: write
38+
steps:
39+
# Checkout the specified branch from the specified organization
40+
- name: Checkout code from the specified branch
41+
uses: actions/checkout@v4
42+
with:
43+
ref: ${{ github.event.inputs.branch }}
44+
token: ${{ secrets.GH_ACCESS_TOKEN }}
45+
46+
# Configure Git
47+
- name: Configure Git
48+
run: |
49+
git config --global user.email "github-actions[bot]@users.noreply.github.com"
50+
git config --global user.name "GitHub Actions"
51+
52+
# Setup Python environment with the specified version (or default to '3.11')
53+
- name: Setup Python environment
54+
uses: actions/setup-python@v5
55+
with:
56+
python-version: ${{ github.event.inputs.python_version }}
57+
58+
# Install pipenv
59+
- name: Install pipenv
60+
run: pip install pipenv
61+
62+
# Run makefile recipe to refresh Pipfile.lock and push changes back to the branch
63+
- name: Run make refresh-pipfilelock-files and push the changes back to the branch
64+
run: |
65+
make refresh-pipfilelock-files PYTHON_VERSION=${{ github.event.inputs.python_version }} INCLUDE_OPT_DIRS=${{ github.event.inputs.update_optional_dirs }}
66+
git add .
67+
git commit -m "Update Pipfile.lock files by piplock-renewal.yaml action"
68+
git push origin ${{ github.event.inputs.branch }}

‎CONTRIBUTING.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@ Pull requests are the best way to propose changes to the notebooks repository:
2727
- Create a proper filepath and naming to the corresponding folder
2828
- Add the minimum files you have to add:
2929
- Pipfile with the additional packages
30-
- Generate the Pipfile lock by running `pipenv lock` to the corresponding pipfile directory
3130
- Dockefile with proper instructions
3231
- Kustomization objects to deploy the new notebook into an openshift cluster (Kustomization.yaml, service.yaml, statefulset.yaml)
3332
- Create instructions into Makefile, for example if you derive the new notebooks from minimal then the recipe should be like the following:
@@ -38,6 +37,7 @@ Pull requests are the best way to propose changes to the notebooks repository:
3837
$(call image,$@,jupyter/${NOTEBOOK_NAME}/ubi8-python-3.8,$<)
3938
```
4039
- Add the paths of the new pipfiles under `refresh-pipfilelock-files`
40+
- Run the [piplock-renewal.yaml](https://github.com/opendatahub-io/notebooks/blob/main/.github/workflows/piplock-renewal.yaml) against your fork branch, check [here](https://github.com/opendatahub-io/notebooks/blob/main/README.md) for more info.
4141
- Test the changes locally, by manually running the `$ make jupyter-${NOTEBOOK_NAME}-ubi8-python-3.8` from the terminal.
4242
4343

‎Makefile

+55-33
Original file line numberDiff line numberDiff line change
@@ -650,41 +650,63 @@ validate-rstudio-image: bin/kubectl
650650
continue; \
651651
fi; \
652652

653-
# This is only for the workflow action
653+
# This recipe used mainly from the Pipfile.locks Renewal Action
654+
# Default Python version
655+
PYTHON_VERSION ?= 3.11
656+
ROOT_DIR := $(shell pwd)
657+
BASE_DIRS := base/c9s-python-$(PYTHON_VERSION) \
658+
base/ubi9-python-$(PYTHON_VERSION) \
659+
jupyter/minimal/ubi9-python-$(PYTHON_VERSION) \
660+
jupyter/datascience/ubi9-python-$(PYTHON_VERSION) \
661+
jupyter/pytorch/ubi9-python-$(PYTHON_VERSION) \
662+
jupyter/tensorflow/ubi9-python-$(PYTHON_VERSION) \
663+
jupyter/trustyai/ubi9-python-$(PYTHON_VERSION) \
664+
jupyter/rocm/tensorflow/ubi9-python-$(PYTHON_VERSION) \
665+
jupyter/rocm/pytorch/ubi9-python-$(PYTHON_VERSION) \
666+
codeserver/ubi9-python-$(PYTHON_VERSION) \
667+
runtimes/minimal/ubi9-python-$(PYTHON_VERSION) \
668+
runtimes/datascience/ubi9-python-$(PYTHON_VERSION) \
669+
runtimes/pytorch/ubi9-python-$(PYTHON_VERSION) \
670+
runtimes/tensorflow/ubi9-python-$(PYTHON_VERSION) \
671+
runtimes/rocm-tensorflow/ubi9-python-$(PYTHON_VERSION) \
672+
runtimes/rocm-pytorch/ubi9-python-$(PYTHON_VERSION)
673+
674+
# Default value is false, can be overiden
675+
# The below directories are not supported on tier-1
676+
INCLUDE_OPT_DIRS ?= false
677+
OPT_DIRS := jupyter/intel/ml/ubi9-python-$(PYTHON_VERSION) \
678+
jupyter/intel/pytorch/ubi9-python-$(PYTHON_VERSION) \
679+
jupyter/intel/tensorflow/ubi9-python-$(PYTHON_VERSION) \
680+
intel/runtimes/ml/ubi9-python-$(PYTHON_VERSION) \
681+
intel/runtimes/pytorch/ubi9-python-$(PYTHON_VERSION) \
682+
intel/runtimes/tensorflow/ubi9-python-$(PYTHON_VERSION)
683+
684+
# This recipe gets args, can be used like
685+
# make refresh-pipfilelock-files PYTHON_VERSION=3.11 INCLUDE_OPT_DIRS=false
654686
.PHONY: refresh-pipfilelock-files
655687
refresh-pipfilelock-files:
656-
cd base/ubi9-python-3.9 && pipenv lock
657-
cd base/c9s-python-3.9 && pipenv lock
658-
cd jupyter/minimal/ubi9-python-3.9 && pipenv lock
659-
cd jupyter/datascience/ubi9-python-3.9 && pipenv lock
660-
cd jupyter/pytorch/ubi9-python-3.9 && pipenv lock
661-
cd jupyter/tensorflow/ubi9-python-3.9 && pipenv lock
662-
cd jupyter/trustyai/ubi9-python-3.9 && pipenv lock
663-
cd runtimes/minimal/ubi9-python-3.9 && pipenv lock
664-
cd runtimes/datascience/ubi9-python-3.9 && pipenv lock
665-
cd runtimes/pytorch/ubi9-python-3.9 && pipenv lock
666-
cd runtimes/tensorflow/ubi9-python-3.9 && pipenv lock
667-
cd runtimes/rocm-tensorflow/ubi9-python-3.9 && pipenv lock
668-
cd runtimes/rocm-pytorch/ubi9-python-3.9 && pipenv lock
669-
cd base/c9s-python-3.11 && pipenv lock
670-
cd base/ubi9-python-3.11 && pipenv lock
671-
cd codeserver/ubi9-python-3.11 && pipenv lock
672-
cd jupyter/minimal/ubi9-python-3.11 && pipenv lock
673-
cd jupyter/datascience/ubi9-python-3.11 && pipenv lock
674-
cd jupyter/pytorch/ubi9-python-3.11 && pipenv lock
675-
cd jupyter/tensorflow/ubi9-python-3.11 && pipenv lock
676-
cd jupyter/trustyai/ubi9-python-3.11 && pipenv lock
677-
cd jupyter/rocm/tensorflow/ubi9-python-3.11 && pipenv lock
678-
cd jupyter/rocm/pytorch/ubi9-python-3.11 && pipenv lock
679-
cd runtimes/minimal/ubi9-python-3.11 && pipenv lock
680-
cd runtimes/datascience/ubi9-python-3.11 && pipenv lock
681-
cd runtimes/pytorch/ubi9-python-3.11 && pipenv lock
682-
cd runtimes/tensorflow/ubi9-python-3.11 && pipenv lock
683-
cd runtimes/rocm-tensorflow/ubi9-python-3.11 && pipenv lock
684-
cd runtimes/rocm-pytorch/ubi9-python-3.11 && pipenv lock
685-
686-
687-
688+
@echo "Updating Pipfile.lock files for Python $(PYTHON_VERSION)"
689+
@if [ "$(INCLUDE_OPT_DIRS)" = "true" ]; then \
690+
echo "Including optional directories"; \
691+
DIRS="$(BASE_DIRS) $(OPT_DIRS)"; \
692+
else \
693+
DIRS="$(BASE_DIRS)"; \
694+
fi; \
695+
for dir in $$DIRS; do \
696+
echo "Processing directory: $$dir"; \
697+
cd $(ROOT_DIR); \
698+
if [ -d "$$dir" ]; then \
699+
echo "Updating $(PYTHON_VERSION) Pipfile.lock in $$dir"; \
700+
cd $$dir; \
701+
if [ -f "Pipfile" ]; then \
702+
pipenv lock; \
703+
else \
704+
echo "No Pipfile found in $$dir, skipping."; \
705+
fi; \
706+
else \
707+
echo "Skipping $$dir as it does not exist"; \
708+
fi; \
709+
done
688710

689711
# This is only for the workflow action
690712
# For running manually, set the required environment variables

‎README.md

+6
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,12 @@ Use podman/docker to execute the workbench images as container.
5151
podman run -it -p 8888:8888 quay.io/opendatahub/workbench-images:jupyter-minimal-ubi9-python-3.9-2024a-20240317-6f4c36b
5252
```
5353

54+
### Pipfile.lock Generation
55+
56+
Users can update Pipfile.lock files using the [piplock-renewal.yaml](https://github.com/opendatahub-io/notebooks/blob/main/.github/workflows/piplock-renewal.yaml) GitHub Action. This workflow enables users to specify a target branch for updating and automerging Pipfile.lock files, select the desired Python version for the update as well as to choose whether to include optional directories in the update process. After the action completes, the updated files can be retrieved with a simple git pull.
57+
58+
Note: To ensure the GitHub Action runs successfully, users must add a `GH_ACCESS_TOKEN` secret in their fork.
59+
5460
### Deploy & Test
5561

5662
#### Running Python selftests in Pytest

0 commit comments

Comments
 (0)