regulat lesson materials

Author: code4yonglei

content/GPU-computing.rst

@@ -1,6 +1,7 @@
 .. _GPU-computing:
 
-GPU computing
+
+GPU Computing
 =============
 
 .. questions::
@@ -22,11 +23,10 @@ GPU computing
    - 40 min exercises
 
 
-GPU Intro
+Introduction to GPU programming
 ---------
 
 
-
 Moore's law
 ^^^^^^^^^^^
 
@@ -61,7 +61,6 @@ with the term *accelerator*. GPU provides much higher instruction throughput
 and memory bandwidth than CPU within a similar price and power envelope.
 
 
-
 How do GPUs differ from CPUs?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -78,7 +77,6 @@ and complex flow control to avoid long memory access latencies,
 both of which are expensive in terms of transistors.
 
 
-
 .. figure:: img/gpu_vs_cpu.png
    :align: center
 
@@ -160,6 +158,7 @@ This workshop will focus on Numba only.
 Numba for GPUs
 --------------
 
+
 Terminology
 ^^^^^^^^^^^
 
@@ -200,8 +199,6 @@ NumPy arrays are transferred between the CPU and the GPU automatically.
      this feature is called dynamic parallelism but Numba does not support it currently
 
 
-
-
 ufunc (gufunc) decorator
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -349,7 +346,8 @@ Alough it is simple to use ufuncs(gfuncs) to run on GPU, the performance is the
 In addition, not all functions can be written as ufuncs in practice. To have much more flexibility, 
 one needs to write a kernel on GPU or device function, which requires more understanding of the GPU programming. 
 
-GPU Programming Model
+
+GPU programming model
 ^^^^^^^^^^^^^^^^^^^^^
 
 Accelerators are a separate main circuit board with the processor, memory, power management, etc., 
@@ -363,6 +361,7 @@ The device code is executed by doing calls to functions (kernels) written specif
 to take advantage of the GPU. The kernel calls are asynchronous, the control is returned 
 to the host after a kernel calls. All kernels are executed sequentially. 
 
+
 GPU Autopsy. Volta GPU
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -470,7 +469,6 @@ For 1D, it is numba.cuda.threadIdx.x + numba.cuda.blockIdx.x * numba.cuda.blockD
    use the GPU computational resources efficiently.
 
 
-
 It is important to notice that the total number of threads in a grid is a multiple of the block size. 
 This is not necessary the case for the problem that we are solving: the length of the vectors 
 can be non-divisible by selected block size. So we either need to make sure that the threads 
@@ -509,13 +507,13 @@ values like 128, 256 or 512 are frequently used
   - it must be large than the number of available (single precision, double precision or integer operation) cores in a SM to fully occupy the SM
 
 
-
-Data and Memory management
+Data and memory management
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 With many cores trying to access the memory simultaneously and with little cache available, 
 the accelerator can run out of memory very quickly. This makes the data and memory management an essential task on the GPU.
 
+
 Data transfer
 ~~~~~~~~~~~~~
 
@@ -565,7 +563,6 @@ CUDA Kernel and device functions are created with the ``numba.cuda.jit`` decorat
 We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thread positions.
 
 
-
 .. demo:: Demo: CUDA kernel
 
    .. tabs::
@@ -668,9 +665,6 @@ We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thr
             :language: ipython
 
 
-
-
-
 .. note:: 
 
    ``numba.cuda.synchronize()`` is used after the kernel launch to make sure the profiling is correct.
@@ -680,8 +674,6 @@ We will use Numba function ``numba.cuda.grid(ndim)`` to calculate the global thr
    e.g. matmul_numba_gpu.max_blocksize = 32
 
 
-
-
 Optimization
 ------------
 
@@ -836,8 +828,6 @@ Exercises
             $ sbatch job.sh sbatch_matmul_sm.py
 
 
-
-
 .. exercise:: Discrete Laplace Operator
 
    In this exercise, we will work with the discrete Laplace operator.

content/dask.rst

@@ -1,6 +1,7 @@
 .. _dask:
 
-Dask for scalable analytics
+
+Dask for Scalable Analytics
 ===========================
 
 .. objectives::
@@ -14,6 +15,7 @@ Dask for scalable analytics
    - 40 min teaching/type-along
    - 40 min exercises
 
+
 Overview
 --------
 
@@ -31,6 +33,7 @@ tools to work with big data. In addition, Dask can also speeds up
 our analysis by using multiple CPU cores which makes our work run 
 faster on laptop, HPC and cloud platforms.
 
+
 What is Dask?
 -------------
 
@@ -49,7 +52,8 @@ Dask is composed of two parts:
    by schedulers on a single machine or a cluster. From the 
    `Dask documentation <https://docs.dask.org/en/stable/>`__.
 
-Dask Clusters
+
+Dask clusters
 -------------
 
 Dask needs computing resources in order to perform parallel computations. 
@@ -145,8 +149,6 @@ http://localhost:8787/status and can be always queried from commond line by:
    # or 
    client.dashboard_link
 
-
-
 When everything finishes, you can shut down the connected scheduler and workers 
 by calling the :meth:`shutdown` method:
 
@@ -155,9 +157,7 @@ by calling the :meth:`shutdown` method:
    client.shutdown()
 
 
-
-
-Dask Collections
+Dask collections
 ----------------
 
 Dask provides dynamic parallel task scheduling and 
@@ -168,7 +168,7 @@ three main high-level collections:
   - ``dask.bag``: Parallel Python Lists 
 
 
-Dask Arrays
+Dask arrays
 ^^^^^^^^^^^
 
 A Dask array looks and feels a lot like a NumPy array. 
@@ -254,7 +254,8 @@ We can visualize the symbolic operations by calling :meth:`visualize`:
 You can find additional details and examples here 
 https://examples.dask.org/array.html.
 
-Dask Dataframe
+
+Dask dataframe
 ^^^^^^^^^^^^^^
 
 Dask dataframes split a dataframe into partitions along an index and can be used 
@@ -308,7 +309,7 @@ You can find additional details and examples here
 https://examples.dask.org/dataframe.html.
 
 
-Dask Bag
+Dask bag
 ^^^^^^^^
 
 A Dask bag enables processing data that can be represented as a sequence of arbitrary 
@@ -375,8 +376,7 @@ specifically the step where we count words in a text.
    both parallelisation and the ability to utilize RAM on multiple machines.
 
 
-
-Dask Delayed
+Dask delayed
 ^^^^^^^^^^^^
 
 Sometimes problems don't fit into one of the collections like 
@@ -452,7 +452,6 @@ to make them lazy and tasks into a graph which we will run later on parallel har
 	  x.compute()
 
 
-
 Comparison to Spark
 -------------------
 
@@ -798,7 +797,6 @@ Exercises
       plt.plot(tas_sto.year,tas_sto)  # plotting trigers computation
 
 
-
 .. keypoints::
 
    - Dask uses lazy execution

content/index.rst

@@ -21,15 +21,13 @@ processing on single workstations the focus shifts to profiling and optimising,
 and distributed computing.
 
 
-
 .. prereq::
 
    - Basic experience with Python
    - Basic experience in working in a Linux-like terminal
    - Some prior experience in working with large or small datasets
 
 
-
 .. csv-table::
    :widths: auto
    :delim: ;
@@ -43,7 +41,6 @@ and distributed computing.
    90 min ; :doc:`dask`
 
 
-
 .. toctree::
    :maxdepth: 1
    :caption: Preparation
@@ -80,9 +77,9 @@ and distributed computing.
    guide
 
 
-
 .. _learner-personas:
 
+
 Who is the course for?
 ----------------------
 
@@ -91,8 +88,6 @@ datasets and who want to learn powerful tools and best practices for writing mor
 performant, parallelised, robust and reproducible data analysis pipelines.
 
 
-
-
 About the course
 ----------------
 
@@ -107,16 +102,13 @@ Instructors who wish to teach this lesson can refer to the :doc:`guide` for
 practical advice.
 
 
-
-
 See also
 --------
 
 Each lesson episode has a "See also" section at the end which lists 
 recommended further learning material.
 
 
-
 Credits
 -------
 
@@ -149,6 +141,7 @@ educational material, in particular:
 - `Elegant SciPy <https://github.com/elegant-scipy/notebooks/>`__
 - `A Comprehensive Guide to NumPy Data Types <https://axil.github.io/a-comprehensive-guide-to-numpy-data-types.html>`__
 
+
 Instructional Material
 ^^^^^^^^^^^^^^^^^^^^^^
 
@@ -185,7 +178,6 @@ With the understanding that:
   publicity, privacy, or moral rights may limit how you use the material.
 
 
-
 Software
 ^^^^^^^^
 

content/motivation.rst

@@ -1,5 +1,6 @@
 .. _motivation:
 
+
 Motivation
 ==========
 
@@ -97,11 +98,11 @@ Specifically, the lesson covers:
 - How to measure performance and boost performance of time consuming Python functions
 - Various methods to parallelise Python code
 
-The lesson does not cover the following:
+The lesson does not cover the following episodes but the lesson materials are provided:
 
 - Visualisation techniques
 - Machine learning 
-- GPU related
+- GPU programming
 
 .. keypoints::