Skip to content

Commit 0c452f7

Browse files
mathis-marcotteWendy GaultierMathis Marcotte
authored
Kf1.7 upgrade (#251)
* Update common folder (#227) * Code update Common Folder * added notebook page (#233) Co-authored-by: Mathis Marcotte <[email protected]> * added volume-details-page (#234) Co-authored-by: Mathis Marcotte <[email protected]> * Kf1.7 jupyter apis index frontend (#235) * updated jupyter index files --------- Co-authored-by: Mathis Marcotte <[email protected]> * Update jupyter general (#239) General code in all folders * Kf1.7 jupyter form update (#236) * renamed form-default to form-new * updated form for 1.7 --------- Co-authored-by: Mathis Marcotte <[email protected]> * Kf1.7 jupyter backend (#228) * updated get pvcs request * updated poddefaults return response * added new jupyter endpoints * updated access review spec * added new volume endpoints * updated readme for thunder client --------- Co-authored-by: Mathis Marcotte <[email protected]> * Kf1.7 volume table stuff (#241) * imported volume components --------- Co-authored-by: Mathis Marcotte <[email protected]> * fixes to be able to run jupyter-apis (#242) * fixes to be able to run jupyter-apis * index related fixes * revert change in namespace * fixed some form issues * fixed some issues with the details page * fixed monaco-editor for loading * hid volumes form * fixed the custom image checkbox hiding images * fixed some bugs with the form-image * revert the form images * fixed some more issues * updated go package --------- Co-authored-by: Mathis Marcotte <[email protected]> * fixed vulnerabilities (#246) Co-authored-by: Mathis Marcotte <[email protected]> * deleted useless packa-lock file * reverted change to build-angular package version * Kf1.7 vulnerabilities (#247) * fixed vulnerabilities for common * fixed vulnerabilities for jupyter frontend --------- Co-authored-by: Mathis Marcotte <[email protected]> * Kf1.7 frontend js tests (#248) * fixes to be able to run jupyter-apis * index related fixes * revert change in namespace * fixed some form issues * fixed some issues with the details page * fixed monaco-editor for loading * hid volumes form * fixed the custom image checkbox hiding images * fixed some bugs with the form-image * revert the form images * fixed some more issues * updated go package * fixed some of the issues with the js tests --------- Co-authored-by: Mathis Marcotte <[email protected]> * fixed bug with declarations * set docker image to node 16 * Kf1.7 cypress (#249) * kf1.7 cypress update * fixed cypress tests --------- Co-authored-by: Mathis Marcotte <[email protected]> * fixed semver vulnerability issue * kf1.7 added new workflows (#250) * added new workflows * testing with different triggers * ran prettier to format * fixed lint picking up node_modules * deleted workflow that got replaced * ran prettier on jupyter * initial lint fix for jupyter * write prettier on jupyter * jupyter lint * more formatting and linting * fixed issues * formatting --------- Co-authored-by: Mathis Marcotte <[email protected]> * removed old tslint file * updated golang version * updated go version * updated readme --------- Co-authored-by: Wendy Gaultier <[email protected]> Co-authored-by: Mathis Marcotte <[email protected]>
1 parent 60f1d15 commit 0c452f7

File tree

439 files changed

+63987
-40921
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

439 files changed

+63987
-40921
lines changed

‎.github/workflows/build.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222

2323
- uses: actions/setup-go@v2
2424
with:
25-
go-version: '1.17.7'
25+
go-version: '1.20.5'
2626

2727
- name: Gather dependencies
2828
run: |

‎.github/workflows/common_frontend_tests.yaml

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: Common Frontend Tests
2+
on:
3+
pull_request:
4+
types:
5+
- 'opened'
6+
- 'synchronize'
7+
- 'reopened'
8+
9+
jobs:
10+
frontend-format-lint-check:
11+
name: Check code format and lint
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Check out code
16+
uses: actions/checkout@v3
17+
18+
- name: Setup Node
19+
uses: actions/setup-node@v3
20+
with:
21+
node-version: 16
22+
23+
- name: Check frontend code formatting
24+
run: |
25+
cd frontend/common/kubeflow-common-lib
26+
npm i
27+
npm run format:check
28+
- name: Check frontend code linting
29+
run: |
30+
cd frontend/common/kubeflow-common-lib
31+
npm i
32+
npm run lint-check
33+
34+
frontend-unit-tests:
35+
runs-on: ubuntu-latest
36+
name: Unit tests
37+
steps:
38+
- name: Checkout
39+
uses: actions/checkout@v3
40+
- name: Setup node version to 12
41+
uses: actions/setup-node@v3
42+
with:
43+
node-version: 16
44+
45+
- name: Install Kubeflow common library dependecies
46+
run: |
47+
cd frontend/common/kubeflow-common-lib
48+
npm i
49+
npm run test:prod

‎.github/workflows/frontend_tests.yml

-36
This file was deleted.

‎.github/workflows/jwa_frontend_tests.yaml

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
name: JWA Frontend Tests
2+
on:
3+
pull_request:
4+
types:
5+
- 'opened'
6+
- 'synchronize'
7+
- 'reopened'
8+
9+
jobs:
10+
frontend-format-linting-check:
11+
name: Check code format and lint
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Check out code
16+
uses: actions/checkout@v3
17+
18+
- name: Setup Node
19+
uses: actions/setup-node@v3
20+
with:
21+
node-version: 16
22+
23+
- name: Check frontend formatting
24+
run: |
25+
cd frontend/jupyter
26+
npm i
27+
npm run format:check
28+
29+
- name: Check frontend code linting
30+
run: |
31+
cd frontend/jupyter
32+
npm i
33+
npm run lint-check
34+
35+
frontend-unit-tests:
36+
runs-on: ubuntu-latest
37+
name: Unit tests
38+
steps:
39+
- name: Checkout
40+
uses: actions/checkout@v3
41+
- name: Setup node version to 16
42+
uses: actions/setup-node@v3
43+
with:
44+
node-version: 16
45+
46+
- name: Install Kubeflow common library dependecies
47+
run: |
48+
cd frontend/common/kubeflow-common-lib
49+
npm i
50+
npm run build
51+
npm link ./dist/kubeflow
52+
- name: Install JWA dependencies
53+
run: |
54+
cd frontend/jupyter
55+
npm i
56+
npm link kubeflow
57+
- name: Run unit tests
58+
run: |
59+
cd frontend/jupyter
60+
npm run test:prod
61+
62+
run-ui-tests:
63+
name: UI tests with Cypress
64+
runs-on: ubuntu-20.04
65+
steps:
66+
- name: Checkout
67+
uses: actions/checkout@v3
68+
- name: Setup node version to 16
69+
uses: actions/setup-node@v3
70+
with:
71+
node-version: 16
72+
73+
- name: Install Kubeflow common library dependecies
74+
run: |
75+
cd frontend/common/kubeflow-common-lib
76+
npm i
77+
npm run build
78+
npm link ./dist/kubeflow
79+
- name: Install JWA dependencies
80+
run: |
81+
cd frontend/jupyter
82+
npm i
83+
npm link kubeflow
84+
- name: Serve UI & run Cypress tests in Chrome and Firefox
85+
run: |
86+
cd frontend/jupyter
87+
npm run serve & npx wait-on http://localhost:4200
88+
npm run ui-test-ci-all

‎Dockerfile

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Stage 0: UI Build Stage
2-
FROM node:12-alpine as frontend
2+
FROM node:16-alpine as frontend
33
WORKDIR /src
44
ENV NG_CLI_ANALYTICS "ci"
55
RUN apk add --no-cache --virtual .gyp python3 make g++
@@ -22,7 +22,7 @@ RUN npm run build -- --output-path=./dist/rok --configuration=rok-prod
2222
RUN ./node_modules/.bin/ng build --configuration production --localize
2323

2424
# Stage 1: Build with the golang image
25-
FROM golang:1.17-alpine AS backend
25+
FROM golang:1.20-alpine AS backend
2626
RUN apk add git
2727
WORKDIR /go/src/github.com/StatCan/jupyter-apis
2828
COPY go.mod .

‎README.md

+33-19
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ The front-end is configured to proxy requests to the local API server. It
5959
requires an environment variable (`KF_USER_ID`) to specify the current user –
6060
this is passed to the API server as an HTTP header.
6161

62-
The following can be pasted in a script and executed. This uses the latest node lts version(v16.16.0).
62+
The following can be pasted in a script and executed. This uses the latest node lts/gallium version(v16.20.0) with npm v8(8.19.4).
6363
**NOTE**: `user` is when using vagrant. Use the email adress if it is the dev cluser (please never connect to prod directly)
6464

6565
```
@@ -70,12 +70,18 @@ npm link ./dist/kubeflow
7070
7171
cd ../../jupyter
7272
npm i
73-
npm link kubeflow --legacy-peer-deps
73+
npm link kubeflow
7474
KF_USER_ID=user npm start
7575
```
7676

7777
For the kubecost data to be retrievable, the following will need to be executed `kubectl port-forward -n kubecost-system deployment/kubecost-cost-analyzer 9090`
7878

79+
### Testing backend Rest API
80+
81+
To test the backend, install the [Thunder Client extension](https://marketplace.visualstudio.com/items?itemName=rangav.vscode-thunder-client). After being installed, it should appear in the vs code sidebar (it might require a restart first).
82+
First thing is to make sure that the backend is running. Then, open Thunder Client from the vs code sidebar. You should be able to see multiple requests that can be made against the backend under the "Collections" tab, in the "Golang kubeflow" dropdown. From there, just select a request and hit "Send".
83+
Some requests require certain parameters to have values. Those can be filled up under the "Env" tab.
84+
7985
### Running intergration tests
8086

8187
We use [Cypress](https://www.cypress.io/) to make our end-to-end tests.
@@ -147,26 +153,34 @@ Any push to an open PR that has the auto-deploy label on it allows developers to
147153

148154
Routes are defined in this repository [here](./main.go).
149155

150-
[Upstream](https://github.com/kubeflow/kubeflow/tree/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes), the endpoints are structures via request type (e.g. `GET`, `PUT`, `DELETE`).
156+
[Upstream](https://github.com/kubeflow/kubeflow/tree/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes), the endpoints are structures via request type (e.g. `GET`, `PUT`, `DELETE`).
151157

152158
_Note_
153159

154160
- _that not all endpoints are included in the golang implementation_
155-
- _to find the upstream endpoint, load the [Upstream](https://github.com/kubeflow/kubeflow/tree/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes)
161+
- _to find the upstream endpoint, load the [Upstream](https://github.com/kubeflow/kubeflow/tree/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes)
156162
and use search with the endpoint text!_
157163

158-
| Request Type | Golang Endpoint | Upstream Python Endpoint | Purpose |
159-
| ------------ | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
160-
| GET | /api/config | [/api/config](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L9) | |
161-
| GET | /api/gpus | [/api/gpus](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L52) | |
162-
| GET | /api/storageclasses/default | [/api/storageclasses/default](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/routes/get.py#L26) | |
163-
| GET | /api/namespaces/{namespace}/cost/allocation | Not found | Get the kubecost Allocation API |
164-
| GET | /api/namespaces | [/api/namespaces](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/routes/get.py#L10) | Get the list of namespaces |
165-
| GET | /api/namespaces/{namespace} | Not found | Get namespace metadata |
166-
| GET | /api/namespaces/{namespace}/notebooks | [/api/namespaces/\<namespace\>/notebooks](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L44) | Get the list of notebooks |
167-
| POST | /api/namespaces/{namespace}/notebooks | [/api/namespaces/\<namespace\>/notebooks](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/default/routes/post.py#L11) | Create a notebook |
168-
| DELETE | /api/namespaces/{namespace}/notebooks/{notebook} | [/api/namespaces/\<namespace\>/notebooks/<notebook>](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/delete.py#L9) | Update a notebook |
169-
| PATCH | /api/namespaces/{namespace}/notebooks/{notebook} | [/api/namespaces/\<namespace\>/notebooks/<notebook](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/patch.py#L19) | Delete a notebook |
170-
| GET | /api/namespaces/{namespace}/pvcs | [/api/namespaces/\<namespace\>/pvc](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L15) | List `PVC`s |
171-
| DELETE | /api/namespaces/{namespace}/pvcs/{pvc} | [/api/namespaces/\<namespace\>/pvcs/<pvc>](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/volumes/backend/apps/default/routes/delete.py#L11) | Delete a `PVC` |
172-
| GET | /api/namespaces/{namespace}/poddefaults | [/api/namespaces/\<namespace\>/poddefaults](https://github.com/kubeflow/kubeflow/blob/v1.6.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L25) | Get `PodDefault`s for a given namespace |
164+
| Request Type | Golang Endpoint | Upstream Python Endpoint | Purpose |
165+
| --- | --- | --- | --- |
166+
| GET | /api/config | [/api/config](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L13) | Gets the spawner_ui_config.yaml |
167+
| GET | /api/gpus | [/api/gpus](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L101) | Reads the GPU vendors from the spawner config |
168+
| GET | /api/storageclasses | [/api/storageclasses](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/routes/get.py#L18) | list all storageclasses |
169+
| GET | /api/storageclasses/default | [/api/storageclasses/default](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/routes/get.py#L26) | gets the storage class with the `is-default-class` annotation |
170+
| GET | /api/namespaces/{namespace}/cost/allocation | Not found | Get the kubecost Allocation API |
171+
| GET | /api/namespaces | [/api/namespaces](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/common/backend/kubeflow/kubeflow/crud_backend/routes/get.py#L10) | Get the list of namespaces |
172+
| GET | /api/namespaces/{namespace} | Not found | Get namespace metadata |
173+
| GET | /api/namespaces/{namespace}/notebooks | [/api/namespaces/\<namespace\>/notebooks](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L52) | Get the list of notebooks |
174+
| POST | /api/namespaces/{namespace}/notebooks | [/api/namespaces/\<namespace\>/notebooks](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/default/routes/post.py#L11) | Create a notebook |
175+
| GET | /api/namespaces/{namespace}/notebooks/{notebook} | [/api/namespaces/\<namespace\>/notebooks/\<name\>](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L59) | Get a notebook |
176+
| GET | /api/namespaces/{namespace}/notebooks/{notebook}/pod | [/api/namespaces/\<namespace\>/notebooks/\<notebook_name\>/pod](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L64) | Gets pod of notebook |
177+
| GET | /api/namespaces/{namespace}/notebooks/{notebook}/pod/{pod_name}/logs | [/api/namespaces/\<namespace\>/notebooks/\<notebook_name\>/pod/\<pod_name\>/logs](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L81) | Gets logs of pod of notebook |
178+
| GET | /api/namespaces/{namespace}/notebooks/{notebook}/events | [/api/namespaces/\<namespace\>/notebooks/\<notebook_name\>/events](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L90) | Gets events of notebook |
179+
| DELETE | /api/namespaces/{namespace}/notebooks/{notebook} | [/api/namespaces/\<namespace\>/notebooks/\<notebook\>](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/delete.py#L9) | Delete a notebook |
180+
| PATCH | /api/namespaces/{namespace}/notebooks/{notebook} | [/api/namespaces/\<namespace\>/notebooks/\<notebook\>](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/patch.py#L19) | Update a notebook |
181+
| GET | /api/namespaces/{namespace}/pvcs | [/api/namespaces/\<namespace\>/pvcs](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/volumes/backend/apps/default/routes/get.py#L9) | List `PVC`s |
182+
| GET | /api/namespaces/{namespace}/pvcs/{pvc} | [/api/namespaces/\<namespace\>/pvcs/\<pvc_name\>](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/volumes/backend/apps/default/routes/get.py#L19) | Gets a `PVC` |
183+
| DELETE | /api/namespaces/{namespace}/pvcs/{pvc} | [/api/namespaces/\<namespace\>/pvcs/\<pvc\>](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/volumes/backend/apps/default/routes/delete.py#L11) | Delete a `PVC` |
184+
| GET | /api/namespaces/{namespace}/pvcs/{pvc}/pods | [/api/namespaces/\<namespace\>/pvcs/\<pvc_name\>/pods](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/volumes/backend/apps/default/routes/get.py#L25) | Gets pods of a `PVC` |
185+
| GET | /api/namespaces/{namespace}/pvcs/{pvc}/events | [/api/namespaces/\<namespace\>/pvcs/\<pvc_name\>/events](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/volumes/backend/apps/default/routes/get.py#L32) | Gets events of a `PVC` |
186+
| GET | /api/namespaces/{namespace}/poddefaults | [/api/namespaces/\<namespace\>/poddefaults](https://github.com/kubeflow/kubeflow/blob/v1.7.0/components/crud-web-apps/jupyter/backend/apps/common/routes/get.py#L29) | Get `PodDefault`s for a given namespace |

‎frontend/common/kubeflow-common-lib/.eslintrc.json

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"root": true,
3+
"ignorePatterns": [
4+
"projects/**/*"
5+
],
6+
"overrides": [
7+
{
8+
"files": [
9+
"*.ts"
10+
],
11+
"parserOptions": {
12+
"project": [
13+
"tsconfig.json",
14+
"e2e/tsconfig.json"
15+
],
16+
"createDefaultProgram": true
17+
},
18+
"extends": [
19+
"plugin:@angular-eslint/ng-cli-compat",
20+
"plugin:@angular-eslint/ng-cli-compat--formatting-add-on",
21+
"plugin:@angular-eslint/template/process-inline-templates"
22+
],
23+
"rules": {
24+
"@typescript-eslint/explicit-member-accessibility": [
25+
"off",
26+
{
27+
"accessibility": "explicit"
28+
}
29+
],
30+
"@typescript-eslint/naming-convention": "off",
31+
"@typescript-eslint/member-ordering": "off",
32+
"@angular-eslint/no-input-rename": "off",
33+
"prefer-arrow/prefer-arrow-functions": "off",
34+
"arrow-parens": [
35+
"off",
36+
"always"
37+
],
38+
"import/order": "off"
39+
}
40+
},
41+
{
42+
"files": [
43+
"*.html"
44+
],
45+
"extends": [
46+
"plugin:@angular-eslint/template/recommended"
47+
],
48+
"rules": {}
49+
}
50+
]
51+
}

‎frontend/common/kubeflow-common-lib/angular.json

+13-12
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,13 @@
1717
"options": {
1818
"tsConfig": "projects/kubeflow/tsconfig.lib.json",
1919
"project": "projects/kubeflow/ng-package.json"
20-
}
21-
, "configurations": {
20+
},
21+
"configurations": {
2222
"production": {
2323
"tsConfig": "projects/kubeflow/tsconfig.lib.prod.json"
2424
}
2525
}
26-
},
26+
},
2727
"test": {
2828
"builder": "@angular-devkit/build-angular:karma",
2929
"options": {
@@ -33,18 +33,19 @@
3333
}
3434
},
3535
"lint": {
36-
"builder": "@angular-devkit/build-angular:tslint",
36+
"builder": "@angular-eslint/builder:lint",
3737
"options": {
38-
"tsConfig": [
39-
"projects/kubeflow/tsconfig.lib.json",
40-
"projects/kubeflow/tsconfig.spec.json"
41-
],
42-
"exclude": [
43-
"**/node_modules/**"
38+
"lintFilePatterns": [
39+
"projects/kubeflow/**/*.ts",
40+
"projects/kubeflow/**/*.html"
4441
]
4542
}
4643
}
4744
}
48-
}},
49-
"defaultProject": "kubeflow"
45+
}
46+
},
47+
"defaultProject": "kubeflow",
48+
"cli": {
49+
"defaultCollection": "@angular-eslint/schematics"
50+
}
5051
}

0 commit comments

Comments
 (0)