Merge pull request #38 from Sparks29032/docs_patch

Finalize diffpy.utils documentation

Author: sbillinge

CHANGELOG.md

@@ -0,0 +1,53 @@
+=============
+Release Notes
+=============
+
+.. current developments
+
+v3.2.3
+====================
+
+**Added:**
+
+* Compatability with Python 3.12.0rc3, 3.11.
+* CI Coverage.
+* New tests for loadData function.
+* loadData function now toggleable. Can return either (a) data read from data blocks or (b) header information stored
+  above the data block.
+
+**Removed:**
+
+* Remove use of pkg_resources (deprecated).
+* No longer use Travis.
+
+
+
+v3.1.0
+====================
+
+**Added:**
+
+* Compatibility with Python 3.10, 3.9, 3.8.
+
+**Removed:**
+
+* Remove the support for Python 3.5, 3.6.
+
+
+
+v3.0.0
+====================
+
+**Added:**
+
+* Compatibility with Python 3.7, 3.6, 3.5 in addition to 2.7.
+
+**Changed:**
+
+* Switch to platform-independent "noarch" Anaconda package.
+
+**Deprecated:**
+
+* Variable `__gitsha__` in the `version` module which was renamed to `__git_commit__`.
+
+

CHANGELOG.rst

@@ -1,3 +1,5 @@
+.. _Parsers Documentation:
+
 diffpy.utils.parsers package
 ============================
 
@@ -6,6 +8,8 @@ diffpy.utils.parsers package
     :undoc-members:
     :show-inheritance:
 
+For a sample data extraction workflow, see :ref:`parsers example<Parsers Example>`.
+
 diffpy.utils.parsers.loaddata module
 ------------------------------------
 

doc/manual/source/api/diffpy.utils.parsers.rst

@@ -0,0 +1,11 @@
+.. _Examples:
+
+:tocdepth: 2
+
+Examples
+########
+Landing page for diffpy.utils examples.
+
+.. toctree::
+    parsersexample
+    resampleexample

doc/manual/source/examples/exampledata/parserdata.zip

@@ -0,0 +1,104 @@
+.. _Parsers Example:
+
+:tocdepth: 2
+
+Parsers Example
+###############
+
+This example will demonstrate how diffpy.utils lets us easily process and serialize files.
+Using the parsers module, we can load file data into simple and easy-to-work-with Python objects.
+
+1) To begin, unzip :download:`parserdata<./exampledata/parserdata.zip>` and take a look at ``data.txt``.
+   Our goal will be to extract and serialize the data table as well as the parameters listed in the header of this file.
+
+2) To get the data table, we will use the ``loadData`` function. The default behavior of this
+   function is to find and extract a data table from a file.::
+
+     from diffpy.utils.parsers import loadData
+     data_table = loadData('<PATH to data.txt>')
+
+   While this will work with most datasets, on our ``data.txt`` file, we got a ``ValueError``. The reason for this is
+   due to the comments ``$ Phase Transition Near This Temperature Range`` and ``--> Note Significant Jump in Rw <--``
+   embedded within the dataset. To fix this, try using the ``comments`` parameter. ::
+
+     data_table = loadData('<PATH to data.txt>', comments=['$', '-->'])
+
+   This parameter tells ``loadData`` that any lines beginning with ``$`` and ``-->`` are just comments and
+   more entries in our data table may follow.
+
+   Here are a few other parameters to test out:
+
+   * ``delimiter=','``: Look for a comma-separated data table. Useful for csv file types.
+     However, since ``data.txt`` is whitespace separated, running ::
+
+       loadData('<PATH to data.txt>', comments=['$', '-->'], delimiter=',')
+
+     returns an empty list.
+   * ``minrows=50``: Only look for data tables with at least 50 rows. Since our data table has much less than that many
+     rows, running ::
+
+       loadData('<PATH to data.txt>', comments=['$', '-->'], minrows=50)
+
+     returns an empty list.
+   * ``usecols=[0, 3]``: Only return the 0th and 3rd columns (zero-indexed) of the data table. For ``data.txt``, this
+     corresponds to the temperature and rw columns. ::
+
+       loadData('<PATH to data.txt>', comments=['$', '-->'], usecols=[0, 3])
+
+3) Next, to get the header information, we can again use ``loadData``,
+   but this time with the ``headers`` parameter enabled. ::
+
+     hdata = loadData('<PATH to data.txt>', comments=['$', '-->'], headers=True)
+
+4) Rather than working with separate ``data_table`` and ``hdata`` objects, it may be easier to combine them into a single
+dictionary. We can do so using the ``serialize_data`` function. ::
+
+     from diffpy.utils.parsers import serialize_data
+     file_data = serialize_data('<PATH to data.txt', hdata, data_table)
+     # File data is a dictionary with a single key
+     # The key is the file name (in our case, 'data.txt')
+     # The entry is a dictionary containing data from hdata and data_table
+     data_dict = file_data['data.txt']
+
+   This dictionary ``data_dict`` contains all entries in ``hdata`` and an additional entry named
+   ``data table`` containing ``data_table``. ::
+
+     here_is_the_data_table = data_dict['data table']
+
+   There is also an option to name columns in the data table and save those columns as entries instead. ::
+
+     data_table_column_names = ['temperature', 'scale', 'stretch', 'rw']  # names of the columns in data.txt
+     file_data = serialize_data('<PATH to data.txt>', hdata, data_table, dt_colnames=data_table_column_names)
+     data_dict = file_data['data.txt']
+
+   Now we can extract specific data table columns from the dictionary. ::
+
+     data_table_temperature_column = data_dict['temperature']
+     data_table_rw_column = data_dict['rw']
+
+5) When we are done working with the data, we can store it on disc for later use. This can also be done using the
+   ``serialize_data`` function with an additional ``serial_file`` parameter.::
+
+     parsed_file_data = serialize_data('<PATH to data.txt>', hdata, data_table, serial_file='<PATH to serialfile.json>')
+
+   The returned value, ``parsed_file_data``, is the dictionary we just added to ``serialfile.json``.
+   To extract the data from the serial file, we use ``deserialize_data''. ::
+
+     from diffpy.utils.parsers import deserialize_data
+     parsed_file_data = deserialize_data('<PATH to serialdata.json>')
+
+6) Finally, ``serialize_data`` allows us to store data from multiple text file in a single serial file. For one last bit
+   of practice, we will extract and add the data from ``moredata.txt`` into the same ``serialdata.json`` file.::
+
+     data_table = loadData('<PATH to moredata.txt>')
+     hdata = loadData('<PATH to moredata.txt>', headers=True)
+     serialize_data('<PATH to moredata.txt>', hdata, data_table, serial_file='<PATH to serialdata.json>')
+
+   The serial file ``serialfile.json`` should now contain two entries: ``data.txt`` and ``moredata.txt``.
+   The data from each file can be accessed using ::
+
+     serial_data = deserialize_data('<PATH to serialdata.json>')
+     data_txt_data = serial_data['data.txt']  # Access data.txt data
+     moredata_txt_data = serial_data['moredata.txt']  # Access moredata.txt data
+
+For more information, check out the :ref:`documentation<Parsers Documentation>` of the ``parsers`` module.

doc/manual/source/examples/examples.rst

@@ -0,0 +1,78 @@
+.. _Resample Example:
+
+:tocdepth: 2
+
+Resampling Example
+##################
+
+This example will demonstrate how we can use diffpy.utils functions to resample a function on a denser grid.
+Specifically, we will resample the grid of one function to match another for us to easily compare the two.
+Then we will show how this resampling method lets us create a perfect reconstruction of certain functions
+given enough datapoints.
+
+1) To start, unzip :download:`parserdata<./exampledata/parserdata.zip>`. Then, load the data table from ``Nickel.gr``
+   and ``NiTarget.gr``. These datasets are based on data from `Atomic Pair Distribution Function Analysis: A Primer
+   <https://global.oup.com/academic/product/atomic-pair-distribution-function-analysis-9780198885801?cc=us&lang=en&>`_.
+   ::
+
+     from diffpy.utils.parsers import loadData
+     nickel_datatable = loadData('<PATH to Nickel.gr>')
+     nitarget_datatable = loadData('<PATH to NiTarget.gr>')
+
+   Each data table has two columns: first is the grid and second is the function value.
+   To extract the columns, we can utilize the serialize function ... ::
+
+     from diffpy.utils.parsers import serialize_data
+     nickel_data = serialize_data('Nickel.gr', {}, nickel_datatable, dt_colnames=['grid', 'func'])
+     nickel_grid = nickel_data['Nickel.gr']['grid']
+     nickel_func = nickel_data['Nickel.gr']['func']
+     target_data = serialize_data('NiTarget.gr', {}, nitarget_datatable, dt_colnames=['grid', 'function'])
+     target_grid = nickel_data['Nickel.gr']['grid']
+     target_func = nickel_data['Nickel.gr']['func']
+
+   ... or you can use any other column extracting method you prefer.
+
+2) If we plot the two on top of each other ::
+
+     import matplotlib.pyplot as plt
+     plt.plot(target_grid, target_func, linewidth=3)
+     plt.plot(nickel_grid, nickel_func, linewidth=1)
+
+   they look pretty similar, but to truly see the difference, we should plot the difference between the two.
+   We may want to run something like ... ::
+
+     import numpy as np
+     difference = np.subtract(target_func, nickel_func)
+
+   ... but this will only produce the right result if the ``target_func`` and ``nickel_func`` are on the same grid.
+   Checking the lengths of ``target_grid`` and ``nickel_grid`` shows that these grids are clearly distinct.
+
+3) However, we can resample the two functions to be on the same grid. Since both functions have grids spanning
+   ``[0, 60]``, let us define a new grid ... ::
+
+     grid = np.linspace(0, 60, 6001)
+
+   ... and use the diffpy.utils ``wsinterp`` function to resample on this grid.::
+
+     from diffpy.utils.parsers import wsinterp
+     nickel_resample = wsinterp(grid, nickel_grid, nickel_func)
+     target_resample = wsinterp(grid, target_grid, target_func)
+
+   We can now plot the difference to see that these two functions are in fact equal.:
+
+     plt.plot(grid, target_resample - nickel_resample)
+
+   This is the desired result as the data in ``Nickel.gr`` is every tenth data point in ``NiTarget.gr``.
+   This also shows us that ``wsinterp`` can help us reconstruct a function from incomplete data.
+
+4) In order for our function reconstruction to be perfect, we require that (a) the function is a Fourier transform of a
+   band-limited dataset and (b) the original grid has enough equally-spaced datapoints based on the Nyquist sampling
+   theorem.
+
+     * If our function :math:`F(r)` is of the form :math:`F(r) = \int_0^{qmax} f(q)e^{-iqr}dq` where :math:`qmax` is
+       the bandlimit, then for a grid spanning :math:`r \in [rmin, rmax]`, the Nyquist sampling theorem tells us we
+       require at least :math:`qmax * (rmin - rmax) / \pi` equally-spaced datapoints.
+
+   In the case of our dataset, our band-limit is ``qmax=25.0`` and our function spans :math:`r \in (0.0, 60.0)`.
+   Thus, our original grid requires :math:`25.0 * 60.0 / \pi < 478`. Since our grid has :math:`601` datapoints, our
+   reconstruction was perfect as shown from the comparison between ``Nickel.gr`` and ``NiTarget.gr``.

doc/manual/source/examples/parsersexample.rst

@@ -7,11 +7,17 @@ diffpy.utils - general purpose shared utilities for the diffpy libraries.
 | Software version |release|.
 | Last updated |today|.
 
-The diffpy.utils package provides functions for extracting array data from
-variously formatted text files and wx GUI utilities used by the PDFgui
-program.  The package also includes interpolation function based on the
-Whittaker-Shannon formula that can be used to resample a PDF or other profile
-function over a new grid.
+The diffpy.utils package provides general functions for extracting data from variously formatted text files as well as
+some PDF-specific functionality. These include wx GUI utilities used by the PDFgui program and an interpolation function
+based on the Whittaker-Shannon formula for resampling a bandlimited PDF or other profile function.
+
+========
+Examples
+========
+Illustrations of when and how one would use various diffpy.utils functions.
+
+* :ref:`File Data Extraction<Parsers Example>`
+* :ref:`Resampling & Data Reconstruction<Resample Example>`
 
 =======
 Authors
@@ -40,6 +46,7 @@ Table of contents
 
    license
    release
+   Examples <examples/examples>
    Package API <api/diffpy.utils>
 
 ======================================

doc/manual/source/examples/resampleexample.rst

@@ -1,3 +1,5 @@
+:tocdepth: 2
+
 .. index:: license
 
 License

doc/manual/source/index.rst

@@ -1,3 +1,5 @@
+:tocdepth: 2
+
 .. index:: release notes
 
-.. mdinclude:: ../../../CHANGELOG.md
+.. include:: ../../../CHANGELOG.rst