added more methods and suggestions

Author: eni-awowale

_toc.yml

@@ -18,6 +18,7 @@ parts:
           - file: fundamentals/01_datastructures
           - file: fundamentals/01.1_creating_data_structures
           - file: fundamentals/01.1_io
+          - file: fundamentals/01_datatree_imerghh.ipynb
       - file: fundamentals/02_labeled_data.md
         sections:
           - file: fundamentals/02.1_indexing_Basic.ipynb

DataTree/DataTree_Tutorial.ipynb renamed to fundamentals/01_datatree_imerghh.ipynb

@@ -15,14 +15,13 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
     "import cartopy.crs as ccrs\n",
     "import matplotlib.pyplot as plt\n",
-    "from xarray import open_datatree\n",
-    "from metpy.plots import ctables"
+    "import xarray as xr"
    ]
   },
   {
@@ -38,17 +37,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7 = open_datatree(\n",
-    "    '~/Downloads/3B-HHR.MS.MRG.3IMERG.20210829-S073000-E075959.0450.V07B.HDF5', engine='h5netcdf'\n",
+    "imerghh_730 = xr.open_datatree(\n",
+    "    '~/xarray-data/imerghh_730.hdf5', engine='h5netcdf'\n",
     ")\n",
-    "gpm_imerghh_7"
+    "imerghh_730"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### List all of the groups with `.groups`"
+    "### Nodes\n",
+    "Groups in a netcdf4 or hdf5 file in the DataTree model are represented as \"nodes\" in the DataTree model.\n",
+    "We can list all of the groups with `.groups`"
    ]
   },
   {
@@ -57,7 +58,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7.groups"
+    "imerghh_730.groups"
    ]
   },
   {
@@ -74,7 +75,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid']\n",
+    "imerghh_730['/Grid']\n",
     "\n",
     "# Returns only the data contained in the \"/Grid\" group"
    ]
@@ -85,7 +86,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid/precipitation']"
+    "imerghh_730['/Grid/precipitation']"
    ]
   },
   {
@@ -94,7 +95,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7.Grid.precipitation"
+    "imerghh_730.Grid.precipitation\n",
+    "\n",
+    "# Method based syntax"
    ]
   },
   {
@@ -110,7 +113,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid/Intermediate'].parent"
+    "imerghh_730['/Grid/Intermediate'].parent"
    ]
   },
   {
@@ -119,7 +122,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7.Grid.children"
+    "imerghh_730.Grid.children"
    ]
   },
   {
@@ -145,7 +148,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7.dims\n",
+    "imerghh_730.dims\n",
     "# Note there are no dimensions, coordinates, or data variables defined at the root node"
    ]
   },
@@ -155,7 +158,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7.attrs"
+    "imerghh_730.attrs"
    ]
   },
   {
@@ -164,7 +167,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid'].dims"
+    "imerghh_730['/Grid'].dims"
    ]
   },
   {
@@ -173,7 +176,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid/Intermediate'].dims"
+    "imerghh_730['/Grid/Intermediate'].dims"
    ]
   },
   {
@@ -182,17 +185,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gpm_imerghh_7['/Grid/Intermediate'].data_vars"
+    "imerghh_730['/Grid/Intermediate'].data_vars"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Plotting precipitation data with DataTree\n",
-    "Xarray’s plotting capabilities are centered around DataArray objects. To plot DataTree objects we access their relevant DataArrays in this case, `gpm_imerghh_7['/Grid/precipitation']`. \n",
-    "\n",
-    "We use the `.where()` method to get a subset of precipitation data over the Gulf of Mexico."
+    "### Creating a DataTree from a dictionary with `DataTree.from_dict()`\n",
+    "You can create a DataTree from a dictionary of `xr.Datasets` objects or `xr.DataTree` objects.\n",
+    "The key of the dictionary is the node/group of the new DataTree object."
    ]
   },
   {
@@ -201,21 +203,18 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "precipitation_subset = gpm_imerghh_7['/Grid/precipitation'].where(\n",
-    "    (gpm_imerghh_7['/Grid/precipitation'].lat >= 20)\n",
-    "    & (gpm_imerghh_7['/Grid/precipitation'].lat <= 35)\n",
-    "    & (gpm_imerghh_7['/Grid/precipitation'].lon >= -110)\n",
-    "    & (gpm_imerghh_7['/Grid/precipitation'].lon <= -78),\n",
-    "    drop=True,\n",
-    ")"
+    "imerghh_830 = xr.open_datatree('~/xarray-data/imerghh_830.hdf5', engine='h5netcdf')\n",
+    "xr.DataTree.from_dict({'time_830': imerghh_830})"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Data masking\n",
-    "We add a data mask to the precipitation values that are zero."
+    "### Using `DataTree.from_dict()` to make a DataTree object\n",
+    "Lets combine our two DataTree objects (`imerghh_730` and `imerghh_830`) at each time stamp with `DataTree.from_dict()`.\n",
+    "All of the groups in the original datasets will remain intact but now we have two additional groups `/time_730` and `/time_830`.\n",
+    "The groups `/Grid` and `/Grid/Intermediate`are nested in ancestor node's `/time_730` and `/time_830` respectively. They are all children of the root node `'/'`"
    ]
   },
   {
@@ -224,33 +223,84 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "precipitation_subset_mask = precipitation_subset.where(precipitation_subset > 0.0)"
+    "combined_imerghh_tree = xr.DataTree.from_dict({'time_730': imerghh_730,\n",
+    "                                               'time_830': imerghh_830})\n",
+    "combined_imerghh_tree"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "combined_imerghh_tree.children"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Add a custom precipitation color map from [metpy](https://unidata.github.io/MetPy/latest/api/generated/metpy.plots.ctables.html)"
+    "### Combining data with DataTree\n",
+    "DataTree objects (like Dataset objects) can contain `DataArray` objects. We can `concat` and `merge` DataArrays in an DataTree along a specified dimension. Lets combine the precipitation data from nodes `/time_730` and `/time_830`. Note these datasets have the same size across their `\"time\"`, `\"lat\"` and `\"lon\"` dimensions.\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "precip_concat = xr.concat([combined_imerghh_tree['time_730/Grid/precipitation'], combined_imerghh_tree['time_830/Grid/precipitation']], dim='time')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Plotting precipitation data with DataTree\n",
+    "Xarray’s plotting capabilities are centered around DataArray objects. To plot DataTree objects we access their relevant DataArrays in this case, our concatenated `DataArray` `precip_concat`. \n",
+    "\n",
+    "We use the `.where()` method to get a subset of precipitation data over the Gulf of Mexico."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
    "metadata": {},
    "outputs": [],
    "source": [
-    "clevs = [0, 1, 2.5, 5, 7.5, 10, 15, 20, 30, 40, 50, 70, 100, 150, 200, 250, 300, 400, 500, 600, 750]\n",
-    "norm, cmap = ctables.registry.get_with_boundaries('precipitation', clevs)\n",
-    "cmap"
+    "precip_concat_sub = precip_concat.where(\n",
+    "    (precip_concat.lat >= 20)\n",
+    "    & (precip_concat.lat <= 35)\n",
+    "    & (precip_concat.lon >= -110)\n",
+    "    & (precip_concat.lon <= -78),\n",
+    "    drop=True,)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Plot the data with `.plot()`\n",
-    "Note since this data is two-dimensional it calls `xarray.plot.pcolormesh()` by default with just the `.plot()` method."
+    "### Data masking\n",
+    "We add a data mask to the precipitation values that are zero. We will use the `.where()` method to get data values greater than 0.0"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "precipitation_subset_mask = precip_concat_sub.where(precip_concat_sub > 0.0)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Plot the data with `.plot()` as a `FacetGrid` object\n",
+    "We can use `xarray.plot.FacetGrid` objects to make plots with multiple axes. Each axes shows the same relationship conditioned on different levels of some dimension, in our case different time stamps. Note since this data is two-dimensional it calls `xarray.plot.pcolormesh()` by default with just the `.plot()` method."
    ]
   },
   {
@@ -259,35 +309,35 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Set the figure size, projection, extent and grid lines to the plot\n",
-    "fig = plt.figure(figsize=(8, 8))\n",
-    "ax = plt.axes(projection=ccrs.PlateCarree())\n",
-    "ax.set_extent([-100, -80, 20, 35])\n",
-    "ax.coastlines()\n",
-    "gl = ax.gridlines(draw_labels=True, linewidth=1, color='black', linestyle='--')\n",
-    "gl.right_labels = False\n",
-    "gl.top_labels = False\n",
-    "\n",
-    "# Get the minimum and maximum values in the array\n",
-    "min = precipitation_subset_mask.min()\n",
-    "max = precipitation_subset_mask.max()\n",
-    "\n",
     "# Plot the precipitation data\n",
-    "precipitation_subset_mask[0].plot(\n",
+    "precip_plot = precipitation_subset_mask.plot(figsize=(12, 6), transform=ccrs.PlateCarree(), subplot_kws={'projection':ccrs.PlateCarree()},\n",
     "    x=\"lon\",\n",
     "    y=\"lat\",\n",
-    "    ax=ax,\n",
-    "    cmap=cmap,\n",
-    "    cbar_kwargs={\"orientation\": \"horizontal\", \"pad\": 0.05},\n",
-    "    vmin=min,\n",
-    "    vmax=max,\n",
+    "    col='time', # The dimension (\"time\") we are faceting our plot on\n",
+    "    col_wrap=2, # Number of subplots\n",
+    "    cmap='jet',\n",
+    "    cbar_kwargs={\"orientation\": \"horizontal\", \"pad\": 0.15, \"shrink\": 0.6},\n",
+    "    vmin=precipitation_subset_mask.min(),\n",
+    "    vmax=precipitation_subset_mask.max(),\n",
+    "\n",
     ")\n",
     "\n",
-    "plt.title('Half-hourly precipitation rate in the Gulf of Mexico on August 29, 2021 at 07:30')"
+    "\n",
+    "for ax in precip_plot.axs.flat:\n",
+    "    ax.set_extent([-100, -80, 20, 35])\n",
+    "    ax.coastlines()\n",
+    "    gl = ax.gridlines(linewidth=1, color='black', linestyle='--')\n",
+    "    gl.left_labels = True\n",
+    "    gl.bottom_labels = True\n"
    ]
   }
  ],
  "metadata": {
+  "kernelspec": {
+   "display_name": "utf-upgrade-datatree",
+   "language": "python",
+   "name": "python3"
+  },
   "language_info": {
    "codemirror_mode": {
     "name": "ipython",
@@ -297,7 +347,8 @@
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
+   "pygments_lexer": "ipython3",
+   "version": "3.12.5"
   }
  },
  "nbformat": 4,