02_WorkingWithData.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Raw data are **sacrosanct**\n",
    "\n",
    "<blockquote class=\"twitter-tweet\" data-lang=\"en-gb\"><p lang=\"en\" dir=\"ltr\"><a href=\"https://twitter.com/tomjwebb?ref_src=twsrc%5Etfw\">@tomjwebb</a> don&#39;t, not even with a barge pole, not for one second, touch or otherwise edit the raw data files. Do any manipulations in script</p>&mdash; Gavin Simpson (@ucfagls) <a href=\"https://twitter.com/ucfagls/status/556107371634634755?ref_src=twsrc%5Etfw\">16 January 2015</a></blockquote> <script async src=\"https://platform.twitter.com/widgets.js\" charset=\"utf-8\"></script> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Getting started with your data \n",
    "\n",
    "### Should I version control my raw data?\n",
    "<div class='info'> The raw data *should* never change. All of the processing or data wrangling must be done in copies of your raw data. </div>\n",
    "\n",
    "<br>\n",
    "However, it is important to consider how you are going to share your data. The [best practices for sharing data on the Web summary](https://www.w3.org/TR/dwbp/#bp-summary) is a good place to get started.\n",
    "\n",
    "You can also explore alternatives like [git annex](https://git-annex.branchable.com/) and [Datalad](http://datalad.org/) to version control your data, or [FigShare](https://figshare.com/) to share your data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Adding raw data\n",
    "\n",
    "We will be using the Kaggle [wine review](https://www.kaggle.com/residentmario/renaming-combining-data/data) dataset.\n",
    "The first step is to download the data and store it in our `data/raw` directory.\n",
    "\n",
    "You should have a copy of the datasets that we will be using already if you followed the installation instructions. \n",
    "\n",
    "Otherwise you can get a copy at [https://drive.google.com/drive/u/1/folders/1b2B0KWS0UAVQqFgzx2R2qMNeiiB98lMe?usp=sharing](https://drive.google.com/drive/u/1/folders/1b2B0KWS0UAVQqFgzx2R2qMNeiiB98lMe?usp=sharing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# You got data... is it enough? Data without documentation has no value\n",
    "### metadata = data about data \n",
    "\n",
    "Information that describes, explains, locates or makes it easier\n",
    "to <strong>find, access, and use</strong> a resource\n",
    "\n",
    "<img src=\"assets/meta.jpg\" alt=\"metadata\" width='300px'>\n",
    "<img src=\"assets/metadata.png\" alt=\"metadata\" >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Adding metadata\n",
    "\n",
    "You want to make sure that all your data has information describing how you got the data, the meaning of the columns, date of last access, data version, etc. \n",
    "\n",
    "For your own use make sure to create at least a `README.txt` file describing the data as best as you can.\n",
    "\n",
    "Create a README.txt ( or .md) file inside the data directory and add the following content or something similar.\n",
    "\n",
    "```\n",
    "Data collected from Kaggle winemag reviews\n",
    "https://www.kaggle.com/residentmario/renaming-combining-data/data\n",
    "\n",
    "Collected on 09/05/2018 by Tania Allard\n",
    "```\n",
    "\n",
    "\n",
    "Commit both the `README` and the data to the repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1> Diving deeper into metadada </h1>\n",
    "\n",
    "What would someone outside your research project (or you in 5 years time) need to know about your data to build on your work.\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "        <th>What</th>\n",
    "        <th>Such as..</th>\n",
    "    </tr>\n",
    "<tr>\n",
    "<td>What is the title of the dataset?</td>\n",
    "    <td>The regulation of emotions in adolescents: age differences\n",
    "        and emotion-specific patterns</td>\n",
    "</tr>\n",
    " <tr>\n",
    "    <td>Are there any related publications or data sets?</td>\n",
    "    <td>Theurel A, Gentaz E (2018) The regulation of emotions in\n",
    "        adolescents: age differences and emotion-specific patterns.\n",
    "        PLOS ONE 13(6): e0195501. <a href=\"https://doi.org/10.1371/journal.pone.0195501\">\n",
    "        https://doi.org/10.1371/journal.pone.0195501</a> </td>\n",
    " </tr>\n",
    "</table>\n",
    "\n",
    "<table>\n",
    "<tr>\n",
    "    <th>Where</th>\n",
    "    <th>Such as..</th>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>Where was the data collected?</td>\n",
    "<td>France, Rhones Alpes</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>Where does the data live?</td>\n",
    "<td>Theurel A, Gentaz E (2018) Data from: The regulation of emotions\n",
    "    in adolescents: age differences and emotion-specific patterns.\n",
    "    Dryad Digital Repository. <a href=\"https://doi.org/10.5061/dryad.n230404\">\n",
    "    https://doi.org/10.5061/dryad.n230404</a> </td>\n",
    "</tr>\n",
    "</table>\n",
    "\n",
    "<table>\n",
    "     <tr>\n",
    "         <th>Who</th>\n",
    "         <th>Such as..</th>\n",
    "     </tr>\n",
    "    <tr>\n",
    "     <td>Who is responsible for the data?</td>\n",
    "     <td>Dr Theurel, Anne</td>\n",
    "    </tr>\n",
    "    <th>When</th>\n",
    "    <th>Such as..</th>\n",
    "    <tr>\n",
    "        <td>Was the data collected? What time span does the data cover?\t</td>\n",
    "        <td>Collected: June 2015. Data coverage: 1932-1944</td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additional resources on schemas and metadata:\n",
    "- [https://guides.lib.unc.edu/c.php?g=8749&p=44504](https://guides.lib.unc.edu/c.php?g=8749&p=44504)\n",
    "- [https://www.lib.ncsu.edu/data-management/metadata#best]*https://www.lib.ncsu.edu/data-management/metadata#best)\n",
    "- [http://mozillascience.github.io/checklist/](http://mozillascience.github.io/checklist/)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href=\"https://fonts.googleapis.com/css?family=Didact+Gothic|Dosis:400,500,700\" rel=\"stylesheet\"><style>\n",
       "@font-face {\n",
       "  font-family: \"Computer Modern\";\n",
       "  src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "/* div.cell{\n",
       "width:800px;\n",
       "margin-left:16% !important;\n",
       "margin-right:auto;\n",
       "} */\n",
       "h1 {\n",
       "  font-family: 'Dosis', \"Helvetica Neue\", Arial, sans-serif;\n",
       "  color: #0B132B;\n",
       "}\n",
       "h2 {\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #1C2541;\n",
       "}\n",
       "h3{\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  margin-top:12px;\n",
       "  margin-bottom: 3px;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "h4{\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "h5 {\n",
       "  font-family: 'Dosis', sans-serif;\n",
       "  color: #40a8a6;\n",
       "}\n",
       "div.text_cell_render{\n",
       "  font-family: 'Didact Gothic',Computer Modern, \"Helvetica Neue\", Arial, Helvetica,\n",
       "  Geneva, sans-serif;\n",
       "  line-height: 130%;\n",
       "  font-size: 110%;\n",
       "  /* width:600px; */\n",
       "  /* margin-left:auto;\n",
       "  margin-right:auto; */\n",
       "}\n",
       "\n",
       ".text_cell_render h1 {\n",
       "  font-weight: 200;\n",
       "  font-size: 30pt;\n",
       "  /* font-size: 50pt */\n",
       "  line-height: 100%;\n",
       "  color:#0B132B;\n",
       "  margin-bottom: 0.5em;\n",
       "  margin-top: 0.5em;\n",
       "  display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h2{\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       ".text_cell_render h3{\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       "\n",
       ".warning{\n",
       "  color: rgb( 240, 20, 20 )\n",
       "}\n",
       "\n",
       "div.warn {\n",
       "  background-color: #FF5A5F;\n",
       "  border-color: #FF5A5F;\n",
       "  border-left: 5px solid #C81D25;\n",
       "  padding: 0.5em;\n",
       "\n",
       "  color: #fff;\n",
       "  opacity: 0.8;\n",
       "}\n",
       "\n",
       "div.info {\n",
       "  background-color: #087E8B;\n",
       "  border-color: #087E8B;\n",
       "  border-left: 5px solid #0B3954;\n",
       "  padding: 0.5em;\n",
       "  color: #fff;\n",
       "  opacity: 0.8;\n",
       "}\n",
       "\n",
       "</style>\n",
       "<script>\n",
       "MathJax.Hub.Config({\n",
       "  TeX: {\n",
       "    extensions: [\"AMSmath.js\"]\n",
       "    },\n",
       "    tex2jax: {\n",
       "      inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
       "      displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
       "      },\n",
       "      displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "      \"HTML-CSS\": {\n",
       "        styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "      }\n",
       "      });\n",
       "      </script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "\n",
    "def css_styling():\n",
    "    styles = open(\"styles/custom.css\", \"r\").read()\n",
    "    return HTML(styles)\n",
    "css_styling()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}