Skip to content

Commit

Permalink
First adaptations-WIP
Browse files Browse the repository at this point in the history
  • Loading branch information
w-lampaert authored Jan 10, 2024
1 parent 3cc830e commit c9e4121
Showing 1 changed file with 21 additions and 20 deletions.
41 changes: 21 additions & 20 deletions source/leuven/services/openondemand.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ Open OnDemand is available for the Tier-2 Genius and wICE clusters.

You can use this interface by navigating to the `KU Leuven Open OnDemand page`_.
You can log in using your KU Leuven or VSC credentials.
Access is only granted to VSC3 users at the moment.
Once logged in, you'll notice you are connected with your VSC account.

Use
===
Expand All @@ -28,7 +26,7 @@ The KU Leuven Open OnDemand page provides a range of functions:
- Opening a shell on one of the login nodes
- Using interactive apps

All of these functionalities can be used by accessing them through the tabs at the top of the page. In the following, we will describe the different parts in
All of these functionalities can be used by accessing them through the tabs at the top of the page. In the following, we will describe these different parts in
some more detail.

Files
Expand All @@ -38,7 +36,7 @@ This menu provides a file explorer that allows you to navigate files by clicking
available here. General file explorer options like moving, deleting, modifying and creating files or directories are available as well. You can also use this interface
to download and upload files to and from your local machine. Be aware that this is not recommended for very large files.

**Good to know:** the standard 'ctrl+s' sadly does not store your files on Open OnDemand, but will trigger a save on your local machine. Luckily, there is a quite
**Good to know:** the standard 'ctrl+s' does not store your files on Open OnDemand, but will trigger a save on your local machine. Luckily, there is a quite
obvious save button on the editor page.

Jobs
Expand All @@ -54,7 +52,7 @@ The jobs tab has two menus: 'Active Jobs' and 'Job Composer':
Active jobs
-----------

This lists all of your running, queued and completed jobs.
This lists all your running, queued and completed jobs.
Completed jobs will disappear after a couple of minutes.
You have a handy overview of some job details by clicking the large arrow next
to the job id, including things like the node list, the account you used, and
Expand All @@ -76,7 +74,7 @@ Templates
~~~~~~~~~

To enter the Templates menu, you can click on 'Templates' at the top once you are in the 'Job Composer' menu. You can also access this menu by clicking the button 'New
Job'-'From Template'. Once in this menu, you should see a table with 3 System Templates. The resources that are requested in these scripts are the default settings.
Job'-'From Template'. Once in this menu, you should see a table with three System Templates. The resources that are requested in these scripts are the default settings.
The templates:

- CPU job template: a template for jobs on the thin nodes (the default ``batch`` partition). This is also the default template (which you will get when clicking 'From Default Template' under the 'New Job' button in the 'Jobs' menu).
Expand All @@ -86,11 +84,12 @@ The templates:
You can create your own templates from scratch or by copying one of the existing templates.
In both cases you will be redirected to a page where you can provide a
name, the cluster and add some notes.
Once you save this, you will create a new folder in the templates folder
``$VSC_DATA/ondemand/data/sys/myjobs/templates/``.
This information will be saved in the ``manifest.yml`` file that is always present
in each of the template folders.
Which other files will be present in this folder depends on how you created your new template.
To save this, you will need to provide a path to store it in. Ondemand will create a new subdirectory in the path you provided
with the name of your template.

**UPDATE AFTER FIX**
A ``manifest.yml`` file will always be present in a template directory, It contains all the info your provided when setting
up this template. Which other files will be present in this directory depends on how you created your new template.
If you use the 'New Template' button, it will also contain the job
script of the default template, being the CPU job template.
If you use the 'Copy Template' button, the contents of the folder will depend on the
Expand All @@ -106,15 +105,17 @@ upload new files.
These files will be present in each job you create from this template, which means you fully adapt this to the standard set-up you use for your
jobs.

**UPDATE AFTER FIX**

Jobs
~~~~

The functioning of creating jobs is a bit similar to how you create new templates.
Whatever method you choose, you will create a new folder for each job, this time
located at ``$VSC_DATA/ondemand/data/projects/default/``.
The job folders will be numbered in the order you have created them.
The job directories will be numbered in the order you have created them.
**Do not change this folder name as long as you plan on using it from the job menus, as this will break the linking.**
When removing a job, the folder will be deleted as well.
**When removing a job, the directory will be deleted as well.**

To create a job, press the 'New Job' button and choose the option that best suits
your needs.
Expand Down Expand Up @@ -169,7 +170,7 @@ is available in the next chapter.
- Numbers of hours: your walltime (min 1h).
- Number of cores: the amount of cores per node. This defaults to 1.
- Required memory per core in megabytes. This defaults to 3400 MB.
- Number of GPUs. If you request a GPU of the ``gpu`` partition you will get a full A100 GPU. For the ``interactive`` partition on wICE, every GPU is a virtual GPU slice of the available A100 GPUs. One GPU slice is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]:<number> (e.g. A100:2). You can also just request a number of GPUs as <number>. Then you will be appointed the first available GPU types. In practice, both methods are the same for now. This might change if we would decide to add extra GPU types. **The interactive partition only allows you to request max 1 GPU (slice) though.**
- Number of GPUs. Depending on the partition you have requested, you might get a different GPU. For the Genius GPUs, the GPU will be the same as the type specified in the partition (e.g. a NVidia V100 for ``gpu_v100``). For wICE, you can request both a GPU from the ``gpu`` partion (a full NVidia A100) or from the ``interactive`` partition. For the latter, one GPU is a virtual GPU slice of the available A100 GPUs. One GPU slice is the same as 1/7th of a A100 GPU. The default is 0. You can specify the type of GPU as well: [Type]:<number> (e.g. A100:2). You can also just request a number of GPUs as <number>.**The interactive partition only allows you to request max 1 GPU (slice) though.**
- Reservation: if you are part of a reservation, you can also use these nodes with Open Ondemand by specifying your reservation name here.
- Pre-run scriptlet: this allows you to add bash commands to your job before launching the app. This can be used for example for loading extra modules that you need within the app. **Be aware that this feature is still somewhat experimental, and its functionality also depends on the app you are running (mainly RStudio Server has some issues here). If you would like to use this feature, but you run into problems, please contact our helpdesk.**

Expand All @@ -183,7 +184,7 @@ Choosing your resources
Choosing the correct resources for your interactive session is mostly the same as selecting them when launching regular batch jobs. For this reason we strongly
recommend you to have a look at how to specify your resources both on `Genius <https://docs.vscentrum.be/en/latest/leuven/genius_quick_start.html#running-jobs-on-genius>`_ and `wICE <https://docs.vscentrum.be/en/latest/leuven/wice_quick_start.html#running-jobs-on-wice>`_.

As mentioned above, in most cases we recommend using the ``interactive`` partition on wICE for your interactive apps. This partition is meant for lighter work, like
As mentioned above, in most cases we recommend using the ``interactive`` partition on wICE for the interactive apps. This partition is meant for lighter work, like
visualisations, testing and pre- and postprocessing. Using this partition is also free, mainly to encourage you to request these resources for such work, instead
of using any of the other partitions. There are however some limitations on the amount of resources you can request here:

Expand All @@ -205,7 +206,7 @@ your presence to start your code.
Interactive shell
-----------------

This app will launch a shell on (one of the) requested nodes, allowing you to use these compute resources from within a Linux terminal. This is different
This app will launch a shell on (one of) the requested node(s), allowing you to use these compute resources from within a Linux terminal. This is different
than the shell you get in the "Clusters" menu, which directs you towards one of the login nodes.

Jupyter Lab
Expand All @@ -219,7 +220,7 @@ environments and their associated kernels, both the Python and R installation wi
To create any other kernels, first create a `Python <https://docs.vscentrum.be/en/latest/software/python_package_management.html#create-an-environment>`_ or
`R <https://docs.vscentrum.be/en/latest/software/r_package_management.html#creating-an-environment>`_ conda environment. The second step consists out of effectively
creating the kernel in your ``$VSC_HOME/.local`` folder, as Jupyter will look for kernels in this location. The following commands should be excecuted from a shell,
and only need to be done once for the set-up of each new kernel. This starts with first of all activating your conda
and only need to be done once for the set-up of each new kernel. This starts with activating your conda
environment::

source activate <env_name>
Expand Down Expand Up @@ -252,7 +253,7 @@ avoid clogging your ``$VSC_HOME``. You can do this by using the ``lib`` argument

**Remarks:**

- Navigating between your different directories is possible using the file explorer. If you are navigating by clicking the folder, you will notice that you can see all user folders. You do not have access to these, and you will receive an error when you try to open them. You will also notice that you cannot use the same way of navigating after this. Another solution is to click the three dots on the right (...) and enter your file location.
- Navigating between your different directories is possible using the file explorer. If you are navigating by clicking the folder, you will notice that you can see all user folders. You do not have access to these, and you will receive an error when you try to open them. You will also notice that you cannot use the same way of navigating after this. Another solution is to click the three dots on the right (...) and enter your path.
- The 'Tools-Install packages' interface does not allow you to select any other path than the default in your ``$VSC_HOME``. It is recommended to use the ``install.packages`` function instead.
- RStudioServer will by default store the RStudio cache in ``$VSC_HOME/.local/share/rstudio``. This cache can get very large and cause you to exceed the quota of your home directory. To avoid this, you can redirect this cache to your data directory by setting ``$XDG_DATA_HOME`` variables in your ``~/.bashrc``.

Expand All @@ -274,11 +275,11 @@ code-server
This is the browser version of Visual Studio Code. For more information, check out `the official guidelines <https://code.visualstudio.com/docs>`_. As a default,
a Python and a Git module are already loaded, which means you can use both Python and git from a terminal window within code-server. How to open a terminal
window is probably one of the first things you should know: click on the three horizontal lines in the upper left corner, select 'Terminal - New Terminal'. This will
open a shell on the node you are running your session on. Notice that you are start in your ``$VSC_DATA`` folder. You can use this as a regular shell, meaning that you
open a shell on the node you are running your session on. Notice that you are starting in your ``$VSC_DATA`` directory. You can use this as a regular shell, meaning that you
can submit jobs, load modules and so on.

Code-server contains many different options and menus, but only a few will be discussed here. Feel free to explore them. We will however discuss how to set up
code-server to use, any of the compatible languages and use code-server as an IDE. For each of the languages you want to use you need two things: an installation of
code-server to use any of the compatible languages and use code-server as an IDE. For each of the languages you want to use you need two things: an installation of
the specific interpreter and an extension in code-server that allows you to connect to it. The extensions can be found in the 'extensions' menu. In what follows, the
steps for both Python and R are described.

Expand Down

0 comments on commit c9e4121

Please sign in to comment.