Merge branch 'ENCCS:main' into main

Author: qianglise

content/GPU-computing.rst

@@ -1,6 +1,7 @@
 .. _GPU-computing:
 
-GPU computing
+
+GPU Computing
 =============
 
 .. questions::
@@ -22,11 +23,10 @@ GPU computing
    - 40 min exercises
 
 
-GPU Intro
+Introduction to GPU programming
 ---------
 
 
-
 Moore's law
 ^^^^^^^^^^^
 
@@ -61,7 +61,6 @@ with the term *accelerator*. GPU provides much higher instruction throughput
 and memory bandwidth than CPU within a similar price and power envelope.
 
 
-
 How do GPUs differ from CPUs?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -78,7 +77,6 @@ and complex flow control to avoid long memory access latencies,
 both of which are expensive in terms of transistors.
 
 
-
 .. figure:: img/gpu_vs_cpu.png
    :align: center
 
@@ -160,6 +158,7 @@ This workshop will focus on Numba only.
 Numba for GPUs
 --------------
 
+
 Terminology
 ^^^^^^^^^^^
 
@@ -200,8 +199,6 @@ NumPy arrays are transferred between the CPU and the GPU automatically.
      this feature is called dynamic parallelism but Numba does not support it currently
 
 
-
-
 ufunc (gufunc) decorator
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -349,7 +346,8 @@ Alough it is simple to use ufuncs(gfuncs) to run on GPU, the performance is the
 In addition, not all functions can be written as ufuncs in practice. To have much more flexibility, 
 one needs to write a kernel on GPU or device function, which requires more understanding of the GPU programming. 
 
-GPU Programming Model
+
+GPU programming model
 ^^^^^^^^^^^^^^^^^^^^^
 
 Accelerators are a separate main circuit board with the processor, memory, power management, etc., 
@@ -363,6 +361,7 @@ The device code is executed by doing calls to functions (kernels) written specif
 to take advantage of the GPU. The kernel calls are asynchronous, the control is returned 
 to the host after a kernel calls. All kernels are executed sequentially. 
 
+
 GPU Autopsy. Volta GPU
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -470,7 +469,6 @@ For 1D, it is numba.cuda.threadIdx.x + numba.cuda.blockIdx.x * numba.cuda.blockD
    use the GPU computational resources efficiently.
 
 
-
 It is important to notice that the total number of threads in a grid is a multiple of the block size. 
 This is not necessary the case for the problem that we are solving: the length of the vectors 
 can be non-divisible by selected block size. So we either need to make sure that the threads 
@@ -509,13 +507,13 @@ values like 128, 256 or 512 are frequently used
   - it must be large than the number of available (single precision, double precision or integer operation) cores in a SM to fully occupy the SM
 
 
-
-Data and Memory management
+Data and memory management
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 With many cores trying to access the memory simultaneously and with little cache available, 
 the accelerator can run out of memory very quickly. This makes the data and memory management an essential task on the GPU.
 
+
 Data transfer
 ~~~~~~~~~~~~~
 
@@ -565,7 +563,6 @@ CUDA Kernel and device functions are created with the ``numba.cuda.jit`` decorat
 We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thread positions.
 
 
-
 .. demo:: Demo: CUDA kernel
 
    .. tabs::
@@ -668,9 +665,6 @@ We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thr
             :language: ipython
 
 
-
-
-
 .. note:: 
 
    ``numba.cuda.synchronize()`` is used after the kernel launch to make sure the profiling is correct.
@@ -680,8 +674,6 @@ We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thr
    e.g. matmul_numba_gpu.max_blocksize = 32
 
 
-
-
 Optimization
 ------------
 
@@ -836,8 +828,6 @@ Exercises
             $ sbatch job.sh sbatch_matmul_sm.py
 
 
-
-
 .. exercise:: Discrete Laplace Operator
 
    In this exercise, we will work with the discrete Laplace operator.

content/dask.rst

@@ -1,6 +1,7 @@
 .. _dask:
 
-Dask for scalable analytics
+
+Dask for Scalable Analytics
 ===========================
 
 .. objectives::
@@ -14,6 +15,7 @@ Dask for scalable analytics
    - 40 min teaching/type-along
    - 40 min exercises
 
+
 Overview
 --------
 
@@ -31,6 +33,7 @@ tools to work with big data. In addition, Dask can also speeds up
 our analysis by using multiple CPU cores which makes our work run 
 faster on laptop, HPC and cloud platforms.
 
+
 What is Dask?
 -------------
 
@@ -49,7 +52,8 @@ Dask is composed of two parts:
    by schedulers on a single machine or a cluster. From the 
    `Dask documentation <https://docs.dask.org/en/stable/>`__.
 
-Dask Clusters
+
+Dask clusters
 -------------
 
 Dask needs computing resources in order to perform parallel computations. 
@@ -145,8 +149,6 @@ http://localhost:8787/status and can be always queried from commond line by:
    # or 
    client.dashboard_link
 
-
-
 When everything finishes, you can shut down the connected scheduler and workers 
 by calling the :meth:`shutdown` method:
 
@@ -155,9 +157,7 @@ by calling the :meth:`shutdown` method:
    client.shutdown()
 
 
-
-
-Dask Collections
+Dask collections
 ----------------
 
 Dask provides dynamic parallel task scheduling and 
@@ -168,7 +168,7 @@ three main high-level collections:
   - ``dask.bag``: Parallel Python Lists 
 
 
-Dask Arrays
+Dask arrays
 ^^^^^^^^^^^
 
 A Dask array looks and feels a lot like a NumPy array. 
@@ -254,7 +254,8 @@ We can visualize the symbolic operations by calling :meth:`visualize`:
 You can find additional details and examples here 
 https://examples.dask.org/array.html.
 
-Dask Dataframe
+
+Dask dataframe
 ^^^^^^^^^^^^^^
 
 Dask dataframes split a dataframe into partitions along an index and can be used 
@@ -308,7 +309,7 @@ You can find additional details and examples here
 https://examples.dask.org/dataframe.html.
 
 
-Dask Bag
+Dask bag
 ^^^^^^^^
 
 A Dask bag enables processing data that can be represented as a sequence of arbitrary 
@@ -375,8 +376,7 @@ specifically the step where we count words in a text.
    both parallelisation and the ability to utilize RAM on multiple machines.
 
 
-
-Dask Delayed
+Dask delayed
 ^^^^^^^^^^^^
 
 Sometimes problems don't fit into one of the collections like 
@@ -452,7 +452,6 @@ to make them lazy and tasks into a graph which we will run later on parallel har
 	  x.compute()
 
 
-
 Comparison to Spark
 -------------------
 
@@ -798,7 +797,6 @@ Exercises
       plt.plot(tas_sto.year,tas_sto)  # plotting trigers computation
 
 
-
 .. keypoints::
 
    - Dask uses lazy execution

content/guide.rst

@@ -1,6 +1,7 @@
-Instructor's guide
+Instructor's Guide
 ------------------
 
+
 Why teach this lesson
 ^^^^^^^^^^^^^^^^^^^^^
 
@@ -16,6 +17,7 @@ and GPU-ported code in a high-level language like Python can save substantial de
 make HPC resources more accessible to a wider range of researchers, and lead to better 
 overall utilisation of available HPC computing power.
 
+
 Intended learning outcomes
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -43,65 +45,43 @@ Schedule for 3 half-day workshop
 +-------------------+------------------------------------+
 | Time              | Episode                            |
 +===================+====================================+
-| 09:00 - 09:20     | :doc:`motivation`                  | 
+| 09:00 - 09:15     | Welcome and Ice-Breaking Session   | 
 +-------------------+------------------------------------+
-| 09:20 - 10:00     | :doc:`scientific-data`             |
+| 09:15 - 09:30     | :doc:`motivation`                  | 
 +-------------------+------------------------------------+
-| 10:00 - 10:20     | Break                              |
+| 09:30 - 10:20     | :doc:`scientific-data`             |
 +-------------------+------------------------------------+
-| 10:20 - 11:00     | :doc:`stack`                       | 
+| 10:20 - 10:40     | Break                              |
 +-------------------+------------------------------------+
-| 11:00 - 11:20     | Break                              |
+| 10:40 - 11:55     | :doc:`stack`                       | 
 +-------------------+------------------------------------+
-| 11:20 - 12:00     | :doc:`stack`                       | 
+| 11:55 - 12:00     | Q/A and Reflections                | 
 +-------------------+------------------------------------+
 
-
 **Day 2:**
 
 +-------------------+------------------------------------+
 | Time              | Episode                            |
 +===================+====================================+
-| 09:00 - 09:40     | :doc:`parallel-computing`          |
-+-------------------+------------------------------------+
-| 09:40 - 09:50     | Break                              |
-+-------------------+------------------------------------+
-| 09:50 - 10:20     | :doc:`parallel-computing`          |
+| 09:00 - 10:20     | :doc:`parallel-computing`          |
 +-------------------+------------------------------------+
 | 10:20 - 10:40     | Break                              |
 +-------------------+------------------------------------+
-| 10:40 - 11:20     | :doc:`optimization`                |
+| 10:40 - 11:55     | :doc:`optimization`                |
 +-------------------+------------------------------------+
-| 11:20 - 11:30     | Break                              |
+| 11:55 - 12:00     | Q/A and Reflections                | 
 +-------------------+------------------------------------+
-| 11:30 - 12:00     | :doc:`optimization`                |
-+-------------------+------------------------------------+
-
 
 **Day 3:**
 
 +-------------------+------------------------------------+
 | Time              | Episode                            |
 +===================+====================================+
-| 09:00 - 09:40     | :doc:`performance-boosting`        | 
-+-------------------+------------------------------------+
-| 09:40 - 09:50     | Break                              |
+| 09:00 - 10:10     | :doc:`performance-boosting`        | 
 +-------------------+------------------------------------+
-| 09:50 - 10:20     | :doc:`performance-boosting`        | 
-+-------------------+------------------------------------+
-| 10:20 - 10:40     | Break                              |
+| 10:10 - 10:30     | Break                              |
 +-------------------+------------------------------------+
-| 10:40 - 11:20     | :doc:`dask`                        |
+| 10:30 - 11:50     | :doc:`dask`                        |
 +-------------------+------------------------------------+
-| 11:20 - 11:30     | Break                              |
-+-------------------+------------------------------------+
-| 11:30 - 12:00     | :doc:`dask`                        |
-+-------------------+------------------------------------+
-
-
-
-
-
-
-
-
+| 11:50 - 12:00     | Q/A and Summary                    | 
++-------------------+------------------------------------+