index.json

[{"authors":["admin"],"categories":null,"content":"My research focus is using machine learning tools to analyze large datasets of groundwater measurements compiled from different contaminated sites. By combining the historical information about chemical concentration, geochemical processes, and subsurface structure that has been collected from many sites across the country, I\u0026rsquo;m working on applying tools created for big data analysis that may help us understand groundwater remediation and how to optimize treatment.\nBefore coming to CSU, I worked for over five years in the Remediation Division of the Texas Commission on Environmental Quality. Even before that I went to the University of Texas at Austin and received a BS in Environmental Science. I\u0026rsquo;m interested in solving complex problems to bring contaminated sites back into productive use. In my spare time I\u0026rsquo;m probably in the garden, maybe on my bike, or possibly on a different continent.\nThis website is made in  with the blogdown package using Hugo and the theme \u0026ldquo;Academic\u0026rdquo;. It\u0026rsquo;s hosted on github pages and is all free and open-source. The water drop icon is from freepik.\n","date":1593734400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1593734400,"objectID":"2525497d367e79493fd32b198b28f040","permalink":"/author/liz-mcconnell/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/author/liz-mcconnell/","section":"authors","summary":"My research focus is using machine learning tools to analyze large datasets of groundwater measurements compiled from different contaminated sites. By combining the historical information about chemical concentration, geochemical processes, and subsurface structure that has been collected from many sites across the country, I\u0026rsquo;m working on applying tools created for big data analysis that may help us understand groundwater remediation and how to optimize treatment.","tags":null,"title":"Liz McConnell","type":"authors"},{"authors":["吳恩達"],"categories":null,"content":"吳恩達 is a professor of artificial intelligence at the Stanford AI Lab. His research interests include distributed robotics, mobile computing and programmable matter. He leads the Robotic Neurobiology group, which develops self-reconfiguring robots, systems of self-organizing robots, and mobile sensor networks.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Sed neque elit, tristique placerat feugiat ac, facilisis vitae arcu. Proin eget egestas augue. Praesent ut sem nec arcu pellentesque aliquet. Duis dapibus diam vel metus tempus vulputate.\n","date":1461110400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1555459200,"objectID":"da99cb196019cc5857b9b3e950397ca9","permalink":"/author/%E5%90%B3%E6%81%A9%E9%81%94/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/author/%E5%90%B3%E6%81%A9%E9%81%94/","section":"authors","summary":"吳恩達 is a professor of artificial intelligence at the Stanford AI Lab. His research interests include distributed robotics, mobile computing and programmable matter. He leads the Robotic Neurobiology group, which develops self-reconfiguring robots, systems of self-organizing robots, and mobile sensor networks.","tags":null,"title":"吳恩達","type":"authors"},{"authors":null,"categories":null,"content":"Flexibility This feature can be used for publishing content such as:\n Online courses Project or software documentation Tutorials  The courses folder may be renamed. For example, we can rename it to docs for software/project documentation or tutorials for creating an online course.\nDelete tutorials To remove these pages, delete the courses folder and see below to delete the associated menu link.\nUpdate site menu After renaming or deleting the courses folder, you may wish to update any [[main]] menu links to it by editing your menu configuration at config/_default/menus.toml.\nFor example, if you delete this folder, you can remove the following from your menu configuration:\n[[main]] name = \u0026quot;Courses\u0026quot; url = \u0026quot;courses/\u0026quot; weight = 50  Or, if you are creating a software documentation site, you can rename the courses folder to docs and update the associated Courses menu configuration to:\n[[main]] name = \u0026quot;Docs\u0026quot; url = \u0026quot;docs/\u0026quot; weight = 50  Update the docs menu If you use the docs layout, note that the name of the menu in the front matter should be in the form [menu.X] where X is the folder name. Hence, if you rename the courses/example/ folder, you should also rename the menu definitions in the front matter of files within courses/example/ from [menu.example] to [menu.\u0026lt;NewFolderName\u0026gt;].\n","date":1536451200,"expirydate":-62135596800,"kind":"section","lang":"en","lastmod":1536451200,"objectID":"59c3ce8e202293146a8a934d37a4070b","permalink":"/courses/example/","publishdate":"2018-09-09T00:00:00Z","relpermalink":"/courses/example/","section":"courses","summary":"Learn how to use Academic's docs layout for publishing online courses, software documentation, and tutorials.","tags":null,"title":"Overview","type":"docs"},{"authors":null,"categories":null,"content":"In this tutorial, I\u0026rsquo;ll share my top 10 tips for getting started with Academic:\nTip 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum, sem id dapibus fringilla, tellus enim condimentum arcu, nec volutpat est felis vel metus. Vestibulum sit amet erat at nulla eleifend gravida.\nNullam vel molestie justo. Curabitur vitae efficitur leo. In hac habitasse platea dictumst. Sed pulvinar mauris dui, eget varius purus congue ac. Nulla euismod, lorem vel elementum dapibus, nunc justo porta mi, sed tempus est est vel tellus. Nam et enim eleifend, laoreet sem sit amet, elementum sem. Morbi ut leo congue, maximus velit ut, finibus arcu. In et libero cursus, rutrum risus non, molestie leo. Nullam congue quam et volutpat malesuada. Sed risus tortor, pulvinar et dictum nec, sodales non mi. Phasellus lacinia commodo laoreet. Nam mollis, erat in feugiat consectetur, purus eros egestas tellus, in auctor urna odio at nibh. Mauris imperdiet nisi ac magna convallis, at rhoncus ligula cursus.\nCras aliquam rhoncus ipsum, in hendrerit nunc mattis vitae. Duis vitae efficitur metus, ac tempus leo. Cras nec fringilla lacus. Quisque sit amet risus at ipsum pharetra commodo. Sed aliquam mauris at consequat eleifend. Praesent porta, augue sed viverra bibendum, neque ante euismod ante, in vehicula justo lorem ac eros. Suspendisse augue libero, venenatis eget tincidunt ut, malesuada at lorem. Donec vitae bibendum arcu. Aenean maximus nulla non pretium iaculis. Quisque imperdiet, nulla in pulvinar aliquet, velit quam ultrices quam, sit amet fringilla leo sem vel nunc. Mauris in lacinia lacus.\nSuspendisse a tincidunt lacus. Curabitur at urna sagittis, dictum ante sit amet, euismod magna. Sed rutrum massa id tortor commodo, vitae elementum turpis tempus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean purus turpis, venenatis a ullamcorper nec, tincidunt et massa. Integer posuere quam rutrum arcu vehicula imperdiet. Mauris ullamcorper quam vitae purus congue, quis euismod magna eleifend. Vestibulum semper vel augue eget tincidunt. Fusce eget justo sodales, dapibus odio eu, ultrices lorem. Duis condimentum lorem id eros commodo, in facilisis mauris scelerisque. Morbi sed auctor leo. Nullam volutpat a lacus quis pharetra. Nulla congue rutrum magna a ornare.\nAliquam in turpis accumsan, malesuada nibh ut, hendrerit justo. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque sed erat nec justo posuere suscipit. Donec ut efficitur arcu, in malesuada neque. Nunc dignissim nisl massa, id vulputate nunc pretium nec. Quisque eget urna in risus suscipit ultricies. Pellentesque odio odio, tincidunt in eleifend sed, posuere a diam. Nam gravida nisl convallis semper elementum. Morbi vitae felis faucibus, vulputate orci placerat, aliquet nisi. Aliquam erat volutpat. Maecenas sagittis pulvinar purus, sed porta quam laoreet at.\nTip 2 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum, sem id dapibus fringilla, tellus enim condimentum arcu, nec volutpat est felis vel metus. Vestibulum sit amet erat at nulla eleifend gravida.\nNullam vel molestie justo. Curabitur vitae efficitur leo. In hac habitasse platea dictumst. Sed pulvinar mauris dui, eget varius purus congue ac. Nulla euismod, lorem vel elementum dapibus, nunc justo porta mi, sed tempus est est vel tellus. Nam et enim eleifend, laoreet sem sit amet, elementum sem. Morbi ut leo congue, maximus velit ut, finibus arcu. In et libero cursus, rutrum risus non, molestie leo. Nullam congue quam et volutpat malesuada. Sed risus tortor, pulvinar et dictum nec, sodales non mi. Phasellus lacinia commodo laoreet. Nam mollis, erat in feugiat consectetur, purus eros egestas tellus, in auctor urna odio at nibh. Mauris imperdiet nisi ac magna convallis, at rhoncus ligula cursus.\nCras aliquam rhoncus ipsum, in hendrerit nunc mattis vitae. Duis vitae efficitur metus, ac tempus leo. Cras nec fringilla lacus. Quisque sit amet risus at ipsum pharetra commodo. Sed aliquam mauris at consequat eleifend. Praesent porta, augue sed viverra bibendum, neque ante euismod ante, in vehicula justo lorem ac eros. Suspendisse augue libero, venenatis eget tincidunt ut, malesuada at lorem. Donec vitae bibendum arcu. Aenean maximus nulla non pretium iaculis. Quisque imperdiet, nulla in pulvinar aliquet, velit quam ultrices quam, sit amet fringilla leo sem vel nunc. Mauris in lacinia lacus.\nSuspendisse a tincidunt lacus. Curabitur at urna sagittis, dictum ante sit amet, euismod magna. Sed rutrum massa id tortor commodo, vitae elementum turpis tempus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean purus turpis, venenatis a ullamcorper nec, tincidunt et massa. Integer posuere quam rutrum arcu vehicula imperdiet. Mauris ullamcorper quam vitae purus congue, quis euismod magna eleifend. Vestibulum semper vel augue eget tincidunt. Fusce eget justo sodales, dapibus odio eu, ultrices lorem. Duis condimentum lorem id eros commodo, in facilisis mauris scelerisque. Morbi sed auctor leo. Nullam volutpat a lacus quis pharetra. Nulla congue rutrum magna a ornare.\nAliquam in turpis accumsan, malesuada nibh ut, hendrerit justo. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque sed erat nec justo posuere suscipit. Donec ut efficitur arcu, in malesuada neque. Nunc dignissim nisl massa, id vulputate nunc pretium nec. Quisque eget urna in risus suscipit ultricies. Pellentesque odio odio, tincidunt in eleifend sed, posuere a diam. Nam gravida nisl convallis semper elementum. Morbi vitae felis faucibus, vulputate orci placerat, aliquet nisi. Aliquam erat volutpat. Maecenas sagittis pulvinar purus, sed porta quam laoreet at.\n","date":1557010800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1557010800,"objectID":"74533bae41439377bd30f645c4677a27","permalink":"/courses/example/example1/","publishdate":"2019-05-05T00:00:00+01:00","relpermalink":"/courses/example/example1/","section":"courses","summary":"In this tutorial, I\u0026rsquo;ll share my top 10 tips for getting started with Academic:\nTip 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.","tags":null,"title":"Example Page 1","type":"docs"},{"authors":null,"categories":null,"content":"Here are some more tips for getting started with Academic:\nTip 3 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum, sem id dapibus fringilla, tellus enim condimentum arcu, nec volutpat est felis vel metus. Vestibulum sit amet erat at nulla eleifend gravida.\nNullam vel molestie justo. Curabitur vitae efficitur leo. In hac habitasse platea dictumst. Sed pulvinar mauris dui, eget varius purus congue ac. Nulla euismod, lorem vel elementum dapibus, nunc justo porta mi, sed tempus est est vel tellus. Nam et enim eleifend, laoreet sem sit amet, elementum sem. Morbi ut leo congue, maximus velit ut, finibus arcu. In et libero cursus, rutrum risus non, molestie leo. Nullam congue quam et volutpat malesuada. Sed risus tortor, pulvinar et dictum nec, sodales non mi. Phasellus lacinia commodo laoreet. Nam mollis, erat in feugiat consectetur, purus eros egestas tellus, in auctor urna odio at nibh. Mauris imperdiet nisi ac magna convallis, at rhoncus ligula cursus.\nCras aliquam rhoncus ipsum, in hendrerit nunc mattis vitae. Duis vitae efficitur metus, ac tempus leo. Cras nec fringilla lacus. Quisque sit amet risus at ipsum pharetra commodo. Sed aliquam mauris at consequat eleifend. Praesent porta, augue sed viverra bibendum, neque ante euismod ante, in vehicula justo lorem ac eros. Suspendisse augue libero, venenatis eget tincidunt ut, malesuada at lorem. Donec vitae bibendum arcu. Aenean maximus nulla non pretium iaculis. Quisque imperdiet, nulla in pulvinar aliquet, velit quam ultrices quam, sit amet fringilla leo sem vel nunc. Mauris in lacinia lacus.\nSuspendisse a tincidunt lacus. Curabitur at urna sagittis, dictum ante sit amet, euismod magna. Sed rutrum massa id tortor commodo, vitae elementum turpis tempus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean purus turpis, venenatis a ullamcorper nec, tincidunt et massa. Integer posuere quam rutrum arcu vehicula imperdiet. Mauris ullamcorper quam vitae purus congue, quis euismod magna eleifend. Vestibulum semper vel augue eget tincidunt. Fusce eget justo sodales, dapibus odio eu, ultrices lorem. Duis condimentum lorem id eros commodo, in facilisis mauris scelerisque. Morbi sed auctor leo. Nullam volutpat a lacus quis pharetra. Nulla congue rutrum magna a ornare.\nAliquam in turpis accumsan, malesuada nibh ut, hendrerit justo. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque sed erat nec justo posuere suscipit. Donec ut efficitur arcu, in malesuada neque. Nunc dignissim nisl massa, id vulputate nunc pretium nec. Quisque eget urna in risus suscipit ultricies. Pellentesque odio odio, tincidunt in eleifend sed, posuere a diam. Nam gravida nisl convallis semper elementum. Morbi vitae felis faucibus, vulputate orci placerat, aliquet nisi. Aliquam erat volutpat. Maecenas sagittis pulvinar purus, sed porta quam laoreet at.\nTip 4 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum, sem id dapibus fringilla, tellus enim condimentum arcu, nec volutpat est felis vel metus. Vestibulum sit amet erat at nulla eleifend gravida.\nNullam vel molestie justo. Curabitur vitae efficitur leo. In hac habitasse platea dictumst. Sed pulvinar mauris dui, eget varius purus congue ac. Nulla euismod, lorem vel elementum dapibus, nunc justo porta mi, sed tempus est est vel tellus. Nam et enim eleifend, laoreet sem sit amet, elementum sem. Morbi ut leo congue, maximus velit ut, finibus arcu. In et libero cursus, rutrum risus non, molestie leo. Nullam congue quam et volutpat malesuada. Sed risus tortor, pulvinar et dictum nec, sodales non mi. Phasellus lacinia commodo laoreet. Nam mollis, erat in feugiat consectetur, purus eros egestas tellus, in auctor urna odio at nibh. Mauris imperdiet nisi ac magna convallis, at rhoncus ligula cursus.\nCras aliquam rhoncus ipsum, in hendrerit nunc mattis vitae. Duis vitae efficitur metus, ac tempus leo. Cras nec fringilla lacus. Quisque sit amet risus at ipsum pharetra commodo. Sed aliquam mauris at consequat eleifend. Praesent porta, augue sed viverra bibendum, neque ante euismod ante, in vehicula justo lorem ac eros. Suspendisse augue libero, venenatis eget tincidunt ut, malesuada at lorem. Donec vitae bibendum arcu. Aenean maximus nulla non pretium iaculis. Quisque imperdiet, nulla in pulvinar aliquet, velit quam ultrices quam, sit amet fringilla leo sem vel nunc. Mauris in lacinia lacus.\nSuspendisse a tincidunt lacus. Curabitur at urna sagittis, dictum ante sit amet, euismod magna. Sed rutrum massa id tortor commodo, vitae elementum turpis tempus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean purus turpis, venenatis a ullamcorper nec, tincidunt et massa. Integer posuere quam rutrum arcu vehicula imperdiet. Mauris ullamcorper quam vitae purus congue, quis euismod magna eleifend. Vestibulum semper vel augue eget tincidunt. Fusce eget justo sodales, dapibus odio eu, ultrices lorem. Duis condimentum lorem id eros commodo, in facilisis mauris scelerisque. Morbi sed auctor leo. Nullam volutpat a lacus quis pharetra. Nulla congue rutrum magna a ornare.\nAliquam in turpis accumsan, malesuada nibh ut, hendrerit justo. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque sed erat nec justo posuere suscipit. Donec ut efficitur arcu, in malesuada neque. Nunc dignissim nisl massa, id vulputate nunc pretium nec. Quisque eget urna in risus suscipit ultricies. Pellentesque odio odio, tincidunt in eleifend sed, posuere a diam. Nam gravida nisl convallis semper elementum. Morbi vitae felis faucibus, vulputate orci placerat, aliquet nisi. Aliquam erat volutpat. Maecenas sagittis pulvinar purus, sed porta quam laoreet at.\n","date":1557010800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1557010800,"objectID":"1c2b5a11257c768c90d5050637d77d6a","permalink":"/courses/example/example2/","publishdate":"2019-05-05T00:00:00+01:00","relpermalink":"/courses/example/example2/","section":"courses","summary":"Here are some more tips for getting started with Academic:\nTip 3 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.","tags":null,"title":"Example Page 2","type":"docs"},{"authors":[],"categories":null,"content":" Click on the Slides button above to view the built-in slides feature.   Slides can be added in a few ways:\n Create slides using Academic\u0026rsquo;s Slides feature and link using slides parameter in the front matter of the talk file Upload an existing slide deck to static/ and link using url_slides parameter in the front matter of the talk file Embed your slides (e.g. Google Slides) or presentation video on this page using shortcodes.  Further talk details can easily be added to this page using Markdown and $\\rm \\LaTeX$ math code.\n","date":1906549200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1906549200,"objectID":"96344c08df50a1b693cc40432115cbe3","permalink":"/talk/example/","publishdate":"2017-01-01T00:00:00Z","relpermalink":"/talk/example/","section":"talk","summary":"An example talk using Academic's Markdown slides feature.","tags":[],"title":"Example Talk","type":"talk"},{"authors":null,"categories":"Data Cleaning","content":" I’ve been wrestling with ideas for how to standardize sampling dates for a while now. The data sets I’m working with have fairly regular sampling intervals, but sometimes they aren’t sampled on the same day - and this drives me crazy. In terms of real-world sampling plans it makes total sense - weekends, rainy days, multi-day sampling events - but in terms of cleaning and working with the data it’s a pain. Here I’ve made up some fake data that shows the problem I’m having, then use fuzzyjoin to make it work. In the process I realized I could probably do approximately the same thing using dplyr. It’s a snowy day in Colorado, so let’s get to it.\nlibrary(tidyverse) library(lubridate) well \u0026lt;- t(data.frame(1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4)) #making some fake wells date \u0026lt;- seq.Date(ymd(\u0026quot;1996-01-01\u0026quot;), ymd(\u0026quot;1996-12-01\u0026quot;), by = \u0026quot;quarter\u0026quot;) #making some fake dates date \u0026lt;- rep(date, times = 4) date \u0026lt;- date + sample(-7:7,16,replace=T) concentration \u0026lt;- sample(1:20,16,replace=T) #making some fake concentrations df_a \u0026lt;- cbind.data.frame(well, date, concentration) #first fake dataframe df_a \u0026lt;- remove_rownames(df_a) date \u0026lt;- date + sample(-7:7,16,replace=T) concentration \u0026lt;- sample(1:20,16,replace=T) df_b \u0026lt;- cbind.data.frame(well, date, concentration) #second fake dataframe df_b \u0026lt;- remove_rownames(df_b) df_a #one set of samples ## well date concentration ## 1 1 1996-01-07 13 ## 2 1 1996-03-27 7 ## 3 1 1996-07-02 20 ## 4 1 1996-09-28 17 ## 5 2 1995-12-29 5 ## 6 2 1996-03-30 12 ## 7 2 1996-06-28 19 ## 8 2 1996-10-05 14 ## 9 3 1995-12-30 19 ## 10 3 1996-04-05 13 ## 11 3 1996-07-04 8 ## 12 3 1996-10-03 9 ## 13 4 1996-01-03 11 ## 14 4 1996-04-07 11 ## 15 4 1996-06-30 4 ## 16 4 1996-09-29 5 df_b #another set of samples ## well date concentration ## 1 1 1996-01-06 7 ## 2 1 1996-04-03 7 ## 3 1 1996-06-30 4 ## 4 1 1996-09-29 15 ## 5 2 1995-12-30 1 ## 6 2 1996-03-31 20 ## 7 2 1996-07-02 16 ## 8 2 1996-09-29 15 ## 9 3 1995-12-24 10 ## 10 3 1996-04-10 12 ## 11 3 1996-07-03 12 ## 12 3 1996-10-08 7 ## 13 4 1996-01-08 11 ## 14 4 1996-04-10 9 ## 15 4 1996-06-30 10 ## 16 4 1996-10-01 11 df_ab \u0026lt;- rbind(df_a, df_b) ggplot(df_ab, aes(x=date, y=concentration, color=well)) + geom_point(alpha=0.8) Do you see the problem?!?! I can’t make the neat dataframe with both observations that I want. If I join on exact dates I loose a bunch of data. After some googling I found a package called {fuzzyjoin} that allows you to join columns within a certain tolerance, so they don’t have to exactly match up. Great - that’s what I want. Unfortunately, in this case I want to join by two columns, well and date, but the max_dist argument will apply to both, so since I just made the wells 1-4, if I wanted to capture dates within more than four days of each other all wells would be considered matches. So I fuzzy joined first by date, then filtered for the observations where well.x matches well.y. After some cleaning up, it looks pretty good.\nlibrary(fuzzyjoin) df_c \u0026lt;- difference_inner_join(df_a, df_b, by = \u0026quot;date\u0026quot;, max_dist= 14) #join dates within two weeks df_c \u0026lt;- df_c %\u0026gt;% filter(well.x == well.y) %\u0026gt;% #keep only joined observations from the same well rename(\u0026quot;PCE\u0026quot;=concentration.x, \u0026quot;TCE\u0026quot;=concentration.y, \u0026quot;well\u0026quot;=well.x) %\u0026gt;% #change some names mutate(date = round_date(date.x, unit = \u0026quot;month\u0026quot;)) %\u0026gt;% #round to nearest month select(-c(\u0026quot;well.y\u0026quot;, \u0026quot;date.x\u0026quot;, \u0026quot;date.y\u0026quot;)) #remove duplicate well name head(df_c) ## well PCE TCE date ## 1 1 13 7 1996-01-01 ## 2 1 7 7 1996-04-01 ## 3 1 20 4 1996-07-01 ## 4 1 17 15 1996-10-01 ## 5 2 5 1 1996-01-01 ## 6 2 12 20 1996-04-01 df_c %\u0026gt;% pivot_longer(cols = c(PCE, TCE)) %\u0026gt;% ggplot(aes(x=date, y=value, color=well)) + geom_point(alpha=0.5) When I did the round_date thing above I thought, well, couldn’t I just have rounded by date first, then joined by exact date after? So I did just that and got the same results. This wouldn’t always give the same results, for example if a well was sampled around the middle of the month and one observation was rounded up while another was rounded down. The {fuzzyjoin} method joins relative to the distance between the dates, not based on the rounded date. In most cases it won’t matter, but there’s a small distinction.\ndf_a \u0026lt;- mutate(df_a, date = round_date(date, unit = \u0026quot;month\u0026quot;)) df_b \u0026lt;- mutate(df_b, date = round_date(date, unit = \u0026quot;month\u0026quot;)) #df_c \u0026lt;- inner_join(df_a, df_b, by = date) # ^this causes an error: Error: `by` must be a (named) character vector, list, or NULL, not a `Date` object. df_a$date \u0026lt;- as.double(df_a$date) df_b$date \u0026lt;- as.double(df_b$date) df_c \u0026lt;- inner_join(df_a, df_b, by = \u0026quot;date\u0026quot;) df_c \u0026lt;- df_c %\u0026gt;% filter(well.x==well.y) %\u0026gt;% rename(\u0026quot;PCE\u0026quot;=concentration.x, \u0026quot;TCE\u0026quot;=concentration.y, \u0026quot;well\u0026quot;=well.x) %\u0026gt;% select(-c(\u0026quot;well.y\u0026quot;)) #remove duplicate well name df_c \u0026lt;- df_c %\u0026gt;% mutate(date = as.Date(date, origin = \u0026quot;1970-01-01\u0026quot;)) head(df_c) ## well date PCE TCE ## 1 1 1996-01-01 13 7 ## 2 1 1996-04-01 7 7 ## 3 1 1996-07-01 20 4 ## 4 1 1996-10-01 17 15 ## 5 2 1996-01-01 5 1 ## 6 2 1996-04-01 12 20 df_c %\u0026gt;% pivot_longer(cols = c(PCE, TCE)) %\u0026gt;% ggplot(aes(x=date, y=value, color=well)) + geom_point(alpha=0.5) Now it’s time to apply this to a real dataset and see what problems I’ll run into…because they’re out there…lurking…spooky!\n","date":1603584000,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1603584000,"objectID":"c5d04dbc7932ecda4307fb114eca2b3c","permalink":"/post/fuzzyjoin/","publishdate":"2020-10-25T00:00:00Z","relpermalink":"/post/fuzzyjoin/","section":"post","summary":"I’ve been wrestling with ideas for how to standardize sampling dates for a while now. The data sets I’m working with have fairly regular sampling intervals, but sometimes they aren’t sampled on the same day - and this drives me crazy.","tags":["data cleaning","big data","dplyr","fuzzyjoin","dates"],"title":"Joining by Nearby Dates","type":"post"},{"authors":null,"categories":"R","content":" It’s Tidy Tuesday!! This week the focus is threatened and extinct plants. Certainly an issue worth thinking about. I’m a bit of a plant person, but really who isn’t? Let’s dive in!\n#libraries library(tidyverse) library(tidytuesdayR) library(skimr) library(tidytext) #load data tuesdata \u0026lt;- tidytuesdayR::tt_load(2020, week = 34) ## ## Downloading file 1 of 3: `plants.csv` ## Downloading file 2 of 3: `threats.csv` ## Downloading file 3 of 3: `actions.csv` plants \u0026lt;- tuesdata$plants threats \u0026lt;- tuesdata$threats actions \u0026lt;- tuesdata$actions threat_filtered \u0026lt;- threats %\u0026gt;% filter(threatened == 1) action_filtered \u0026lt;- actions %\u0026gt;% filter(action_taken == 1) threat_filtered %\u0026gt;% count(continent, group, threat_type) %\u0026gt;% ggplot(aes(y = tidytext::reorder_within(threat_type, n, continent), x = n, fill = group)) + geom_col() + tidytext::scale_y_reordered() + facet_wrap(~continent, scales = \u0026quot;free\u0026quot;, ncol = 2) action_filtered %\u0026gt;% count(continent, group, action_type) %\u0026gt;% ggplot(aes(y = tidytext::reorder_within(action_type, n, continent), x = n, fill = group)) + geom_col() + tidytext::scale_y_reordered() + facet_wrap(~continent, scales = \u0026quot;free\u0026quot;, ncol = 2) Some interesting data. I wonder, could see improvement in the status of the plant after an intervention, and are some threats are easy to mitigate while others can’t be stopped?\nThe year_last_seen column is seperated into 20-year chunks and the red_list_caregory has two options, “Extinct”, or “Extinct in the Wild”. I guess a successful re-introduction would be represented by a change from “Extinct in the Wild” to off of the list…but let’s see if that’s actually what’s in the data.\nactions_change \u0026lt;- action_filtered %\u0026gt;% select(c(binomial_name, year_last_seen, red_list_category, action_type)) %\u0026gt;% mutate(date = case_when(year_last_seen == \u0026quot;2000-2020\u0026quot; ~ \u0026quot;2020\u0026quot;, year_last_seen == \u0026quot;1980-1999\u0026quot; ~ \u0026quot;1999\u0026quot;, year_last_seen == \u0026quot;1960-1979\u0026quot; ~ \u0026quot;1979\u0026quot;, year_last_seen == \u0026quot;1940-1959\u0026quot; ~ \u0026quot;1959\u0026quot;, year_last_seen == \u0026quot;1920-1939\u0026quot; ~ \u0026quot;1939\u0026quot;, year_last_seen == \u0026quot;1900-1919\u0026quot; ~ \u0026quot;1919\u0026quot;, year_last_seen == \u0026quot;Before 1900\u0026quot; ~ \u0026quot;1900\u0026quot;, T ~ \u0026quot;NA\u0026quot;)) most_listed \u0026lt;- actions_change %\u0026gt;% count(binomial_name) %\u0026gt;% filter(n \u0026gt;= 2) most_changes \u0026lt;- actions_change %\u0026gt;% filter(binomial_name %in% most_listed$binomial_name) ","date":1597708800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1597708800,"objectID":"b86dd7f3e3ad344be76d39bbf358d738","permalink":"/post/plant_threats/","publishdate":"2020-08-18T00:00:00Z","relpermalink":"/post/plant_threats/","section":"post","summary":"It’s Tidy Tuesday!! This week the focus is threatened and extinct plants. Certainly an issue worth thinking about. I’m a bit of a plant person, but really who isn’t? Let’s dive in!","tags":["tidytuesday","conservation","R","dataviz"],"title":"Threatened Plants #TidyTuesday","type":"post"},{"authors":null,"categories":"R","content":" I’ve been searching for a way to interactively display some geospatial data. After trying out a few things I found {leaflet} - so easy and beautiful! I started out reading this excellent primer, then got to work with specific examples. Let’s take a look and see what it can do -\u0026gt;\n#libraries library(tidyverse) #for manipulating and visualizing data library(leaflet) #for making interactive maps #upload data from Geotracker - LA County URL \u0026lt;- \u0026quot;https://geotracker.waterboards.ca.gov/data_download/geo_by_county/LosAngelesGeoXY.zip\u0026quot; download.file(URL, destfile=\u0026#39;LosAngelesGeoXY.zip\u0026#39;, method=\u0026#39;curl\u0026#39;) unzip(\u0026#39;LosAngelesGeoXY.zip\u0026#39;) LA_XY \u0026lt;- read.delim(\u0026quot;LosAngelesGeoXY.txt\u0026quot;) head(LA_XY) ## COUNTY GLOBAL_ID FIELD_PT_NAME FIELD_PT_CLASS XY_SURVEY_DATE LATITUDE ## 1 Los Angeles T0603701211 GMX-RPZ7 PZ 2001-01-25 34.02664 ## 2 Los Angeles T0603701211 GMX-RPZ8 PZ 2001-07-11 34.01503 ## 3 Los Angeles T0603701211 GMX-RPZ9 PZ 2001-07-11 34.01506 ## 4 Los Angeles T0603701211 KMX-MW1 MW 1999-08-27 34.01560 ## 5 Los Angeles T0603701211 KMX-MW2 MW 1999-08-27 34.01787 ## 6 Los Angeles T0603701211 KMX-MW3 MW 1999-08-27 34.01728 ## LONGITUDE XY_METHOD XY_DATUM XY_ACC_VAL XY_SURVEY_ORG GPS_EQUIP_TYPE ## 1 -118.4214 CGPS NAD83 3 Calvada Survey L530 ## 2 -118.4168 CGPS NAD83 3 Calvada Survey L530 ## 3 -118.4169 CGPS NAD83 3 Calvada Survey L530 ## 4 -118.4266 CGPS NAD83 3 Calvada Survey L530 ## 5 -118.4251 CGPS NAD83 3 Calvada Survey L530 ## 6 -118.4246 CGPS NAD83 3 Calvada Survey L530 ## XY_SURVEY_DESC ## 1 ## 2 ## 3 ## 4 ## 5 ## 6 Similar to the well measurements data, there is documentation on the geotracker for each of these column ids. The GLOBAL_ID represents a site and matches up with the concentration data files. Let’s take a closer look at the FIELD_PT_CLASS.\nLA_XY %\u0026gt;% select(FIELD_PT_CLASS) %\u0026gt;% summary() ## FIELD_PT_CLASS ## : 1 ## BH: 11 ## MW:109 ## PZ: 6 ## SG: 30 There are four codes in our data - BH, MW, PZ, SG. You might guess that BH is borehole and MW is monitoring well. I would guess that PZ stands for piezometer and I’m actually not sure what SG stands for. The documentation lists 33 valid values for FIELD_PT_CLASS, but PZ and SG are not on the list.\nThe date on the documentation is 2005, so maybe these codes have been added in the 15 years since then. A disclaimer at the top of the list states that new values are added occasionally, but the link to see the current list is broken :(\nIn any case, we’ll plot these points and see what we can add to make them useful.\nLA_wells_map \u0026lt;- leaflet(LA_XY) %\u0026gt;% addProviderTiles(\u0026quot;CartoDB.Positron\u0026quot;) %\u0026gt;% #using the CartoDB.Positron tiles; there are other options! addCircleMarkers(lng = ~LONGITUDE, lat = ~LATITUDE, ) #LA_wells_map   Look at that! With just a few lines of code we’ve made a map of points that we can pan and zoom. I was honestly expecting a lot more sites. There are other basemaps that you can use, so take a look at what’s available and pick what’s right for you. At this point my map doesn’t display that much information though, so let’s see what we can improve.\n#make a palette to add colors pal \u0026lt;- colorFactor(topo.colors(5), LA_XY$FIELD_PT_CLASS) LA_wells_map_with_color \u0026lt;- leaflet(LA_XY) %\u0026gt;% addProviderTiles(\u0026quot;CartoDB.Positron\u0026quot;) %\u0026gt;% addCircleMarkers(lng = ~LONGITUDE, lat = ~LATITUDE, label= ~as.character(FIELD_PT_CLASS), #add a label color = ~pal(FIELD_PT_CLASS)) #add colors #LA_wells_map_with_color  This shows which class the points are and when you hover over the class it shows the class code. If you zoom in to the site near San Gabriel, you can see three monitoring wells (the two close to each other are probably the same one) and a pattern of SG class points that is fairly uniform. The site near El Monte is made up only of boreholes, as is the one near Manhattan Beach. The others have a mix of monitoring wells and piezometers.\nThe label field in our call to leaflet creates a tag that shows up when we hover our mouse over it. There’s another field for popup, which will make a box that appears when you click on a point. Let’s add some data and put it in a popup box. To do this we’ll have to get the concentration data for the sites and join by the FIELD_PT_NAME\nURL \u0026lt;- \u0026quot;https://geotracker.waterboards.ca.gov/data_download/edf_by_county/LosAngelesEDF.zip\u0026quot; download.file(URL, destfile=\u0026#39;LosAngelesEDF.zip\u0026#39;, method=\u0026#39;curl\u0026#39;) unzip(\u0026#39;LosAngelesEDF.zip\u0026#39;) LA_EDF \u0026lt;- read.delim(\u0026quot;LosAngelesEDF.txt\u0026quot;) str(LA_EDF) ## \u0026#39;data.frame\u0026#39;: 128666 obs. of 23 variables: ## $ COUNTY : Factor w/ 2 levels \u0026quot;\\032\u0026quot;,\u0026quot;Los Angeles\u0026quot;: 2 2 2 2 2 2 2 2 2 2 ... ## $ GLOBAL_ID : Factor w/ 8 levels \u0026quot;\u0026quot;,\u0026quot;SL603799209\u0026quot;,..: 3 3 3 3 3 3 3 3 3 3 ... ## $ FIELD_PT_NAME: Factor w/ 127 levels \u0026quot;\u0026quot;,\u0026quot;15200\u0026quot;,\u0026quot;15210\u0026quot;,..: 50 31 40 50 45 39 39 39 32 40 ... ## $ LOGDATE : Factor w/ 274 levels \u0026quot;\u0026quot;,\u0026quot;2006-07-10\u0026quot;,..: 56 46 46 56 47 46 46 46 46 46 ... ## $ LOGTIME : int 545 1230 945 850 1130 730 850 715 1215 900 ... ## $ LOGCODE : Factor w/ 10 levels \u0026quot;\u0026quot;,\u0026quot;AEII\u0026quot;,\u0026quot;EAIH\u0026quot;,..: 4 4 4 4 4 4 4 4 4 4 ... ## $ SAMPID : Factor w/ 2537 levels \u0026quot;\u0026quot;,\u0026quot;15200\u0026quot;,\u0026quot;15210\u0026quot;,..: 2062 1835 1954 2061 2033 1934 1935 1936 1852 1952 ... ## $ MATRIX : Factor w/ 7 levels \u0026quot;\u0026quot;,\u0026quot;AX\u0026quot;,\u0026quot;GS\u0026quot;,\u0026quot;IA\u0026quot;,..: 6 6 6 6 6 6 6 6 6 6 ... ## $ LABWO : Factor w/ 45 levels \u0026quot;\u0026quot;,\u0026quot;1609-03\u0026quot;,\u0026quot;1803292\u0026quot;,..: 45 45 45 45 45 45 45 45 45 45 ... ## $ LABCODE : Factor w/ 9 levels \u0026quot;\u0026quot;,\u0026quot;AAC\u0026quot;,\u0026quot;ASLL\u0026quot;,..: 5 5 5 5 5 5 5 5 5 5 ... ## $ LABSAMPID : Factor w/ 2656 levels \u0026quot;\u0026quot;,\u0026quot;060703521\u0026quot;,..: 525 437 443 527 455 439 440 435 436 442 ... ## $ ANMCODE : Factor w/ 24 levels \u0026quot;\u0026quot;,\u0026quot;8260+OX\u0026quot;,\u0026quot;8260FA\u0026quot;,..: 23 23 23 23 23 23 23 23 23 23 ... ## $ LABLOTCTL : Factor w/ 652 levels \u0026quot;\u0026quot;,\u0026quot;060711B01\u0026quot;,..: 144 123 127 144 129 123 123 123 123 123 ... ## $ ANADATE : Factor w/ 384 levels \u0026quot;\u0026quot;,\u0026quot;2006-07-11\u0026quot;,..: 87 72 74 87 76 72 72 72 72 72 ... ## $ BASIS : Factor w/ 7 levels \u0026quot;\u0026quot;,\u0026quot;A\u0026quot;,\u0026quot;D\u0026quot;,\u0026quot;L\u0026quot;,..: 5 5 5 5 5 5 5 5 5 5 ... ## $ PARLABEL : Factor w/ 150 levels \u0026quot;\u0026quot;,\u0026quot;4:2FTS\u0026quot;,\u0026quot;6:2FTS\u0026quot;,..: 80 12 56 141 90 141 100 82 141 68 ... ## $ PARVAL : num 10 1 1 1 10 1 5 1 1 2 ... ## $ PARVQ : Factor w/ 5 levels \u0026quot;\u0026quot;,\u0026quot;\u0026lt;\u0026quot;,\u0026quot;=\u0026quot;,\u0026quot;ND\u0026quot;,..: 2 2 2 2 3 2 2 2 2 2 ... ## $ LABDL : num 5.4 0.24 0.36 0.23 4.3 0.23 1.5 0.26 0.23 0.18 ... ## $ REPDL : num 10 1 1 1 10 1 5 1 1 2 ... ## $ UNITS : Factor w/ 10 levels \u0026quot;\u0026quot;,\u0026quot;MG/KG\u0026quot;,\u0026quot;MG/L\u0026quot;,..: 8 8 8 8 8 8 8 8 8 8 ... ## $ DILFAC : num 1 1 1 1 1 1 5 1 1 1 ... ## $ LNOTE : Factor w/ 72 levels \u0026quot;\u0026quot;,\u0026quot;B\u0026quot;,\u0026quot;B,VCO\u0026quot;,..: 1 1 1 1 68 1 1 1 1 1 ... Ooof that’s a lot of data. The question now is what do we want to put in the popup? Most recent concentration of a small class of chemicals? Maximum concentration? Could we add a plot of the concentration over time? I haven’t been able to get the plot-within-a-plot working (yet), so let’s go with maximum concentration of Trichloroethene (TCE) at each well.\nTCE_conc \u0026lt;- LA_EDF %\u0026gt;% filter(PARLABEL == \u0026quot;TCE\u0026quot;) %\u0026gt;% group_by(FIELD_PT_NAME) %\u0026gt;% mutate(TCE = max(PARVAL)) %\u0026gt;% select(FIELD_PT_NAME, TCE, UNITS) %\u0026gt;% filter(UNITS %in% c(\u0026quot;UG/L\u0026quot;, \u0026quot;MG/L\u0026quot;)) %\u0026gt;% unique() %\u0026gt;% mutate(value = case_when(UNITS == \u0026quot;MG/L\u0026quot; ~ 1000, T ~ 1)) %\u0026gt;% #create a col with conversion factor mutate(TCE = TCE*value) #convert mg/L to ug/L LA_TCE_xy \u0026lt;- TCE_conc %\u0026gt;% inner_join(LA_XY, by = \u0026quot;FIELD_PT_NAME\u0026quot;) %\u0026gt;% select(FIELD_PT_NAME, TCE, LATITUDE, LONGITUDE) TCEpal \u0026lt;- colorNumeric( palette = \u0026quot;YlGnBu\u0026quot;, domain = 0:110) popup1 \u0026lt;- \u0026quot;TCE concentration: \u0026quot; #I bet there\u0026#39;s a better way to do this popup2 \u0026lt;- \u0026quot; ug/L\u0026quot; #Tell me in the comments! LA_TCE_xy$pop1 \u0026lt;- popup1 #Like and subscribe! LA_TCE_xy$pop2 \u0026lt;- popup2 LA_TCE_xy$popup \u0026lt;- paste0(LA_TCE_xy$pop1, LA_TCE_xy$TCE, LA_TCE_xy$pop2) LA_wells_TCE_map \u0026lt;- leaflet(LA_TCE_xy) %\u0026gt;% addProviderTiles(\u0026quot;CartoDB.Positron\u0026quot;) %\u0026gt;% addCircleMarkers(lng = ~LONGITUDE, lat = ~LATITUDE, color = ~TCEpal(TCE), popup = ~popup) %\u0026gt;% #add colors addLegend(pal = TCEpal, values = ~TCE, opacity = 0.7, title = NULL, position = \u0026quot;bottomright\u0026quot;) #LA_wells_TCE_map  You may notice some annoying things about this map; most of the values are very low so they skew the colors to almost zero and the scale starts with highest values on the bottom, which is counterintuitive to me. Also we’ve lost some points - the spatial file had 157 points but when we join it with concentration data we’re left with 69 points.\nOne common way to visualize data with a large range is with a log transformation. Let’s try it out!\nLA_TCE_xy \u0026lt;- LA_TCE_xy %\u0026gt;% #log transform the concentrations mutate(TCE = case_when(TCE == 0 ~ 0.0001, T ~ TCE)) %\u0026gt;% mutate(log_TCE = log(TCE)) logpal \u0026lt;- colorNumeric( palette = \u0026quot;YlGnBu\u0026quot;, domain = -9.2105:14.701) LA_wells_logTCE_map \u0026lt;- leaflet(LA_TCE_xy) %\u0026gt;% addProviderTiles(\u0026quot;CartoDB.Positron\u0026quot;) %\u0026gt;% addCircleMarkers(lng = ~LONGITUDE, lat = ~LATITUDE, color = ~logpal(log_TCE), popup = ~popup) #LA_wells_logTCE_map  If we use the log of the TCE value it’s easier to see the changes in color, though it could still use some fiddling with. I took out the scale becuse the popup still says the real value but the color is now based on a log value. There seem to be some workarounds for this but I didn’t try them out. There are probably better color schemes out there but that’s like entering another world, so I’ll leave it at this.\nI’m sure that there are many ways to improve this map and build on these concepts in leaflet. Take a look at what’s out there an let me know if you discover anything you love!\n","date":1597276800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1597276800,"objectID":"d9d3b89fc0425577707275f09462ac04","permalink":"/post/leaflet/","publishdate":"2020-08-13T00:00:00Z","relpermalink":"/post/leaflet/","section":"post","summary":"I’ve been searching for a way to interactively display some geospatial data. After trying out a few things I found {leaflet} - so easy and beautiful! I started out reading this excellent primer, then got to work with specific examples.","tags":["R Markdown","Geotracker","data mining","geospatial"],"title":"Making Maps with Leaflet in R","type":"post"},{"authors":null,"categories":"Reading","content":" Earlier this summer I got an email about reading groups for grad students hosted by the department of cell and molecular biology. Who wasn’t hungry for a reading group this summer? The first book I signed up to read was Weapons of Math Destruction by Cathy O’Neil. I recognized it because I had heard an NPR interview with the author on the radio. I actually listened to the audiobook because I wasn’t in Fort Collins and libraries were closed (COVID). It was a really engaging story - basically the whole book is composed of examples of “Weapons of Math Destruction”, or WMDs, a term that O’Neil coins in the first chapted to describe models that are important, opaque, and reinforce inequality.\nThe features of WMDs are highlighted through examples (so many examples!) of real algorithms that are in use every day. But O’Neil doesn’t just throw lots of examples of WMDs at us, she also describes models that have some negative aspects, but are not on the whole that harmful, and models that are sound uses of statistics. It’s complicated and forces you to confront how insidious these algorithms are.\nSomething that surprised me was her discussion of credit scores. I was all ready to jump on board against credit scores, but that wasn’t her conclusion in the book. This is not a WMD. The FICO model has a feedback loop, but it corrects itself rather than perpetuates inequality. If many people who are predicted to be risky borrowers pay on time, the model is updated to account for the new data and make better predictions. In addition, the model is fairly transparent - if you are trying to improve your credit score you can easily look up ways to do so. And credit scores are also regulated, so you have legal rights to see your score and know what goes into it.\nIn stark contrast are ubiquitous e-scores used by credit card companies, car dealerships, and advertising agencies that you probably have no idea exist. These scores are created by models that combine lots of seemingly disparate information about you and predict if you’re likely to buy their product. The problem is that the factors the model uses are proxies like zip code, which means that inequality created by racist zoning laws is perpetuated by an algorithm. Predatory for-profit college lead generation companies use algorithms that target people who are earnestly searching for oppurtunities at upward mobility - and make a fortune doing it.\nAnother outraging story was about scheduling software used by low-wage corporations like McDonald’s, Starbucks, and Walmart. They’re designed to optimize saving for the company, so they create ever-changing and erratic schedules that keep workers from qualifying for benefits. In 2014 the New York Times published an article profiling Jannette Navarro, a Starbucks employee and single mother trying to work her way through college. Her life was so dominated by her work schedule that she couldn’t make her classes consistently or plan for any sort of normal day-care schedule. After the story, legislators drafted a bill to regulate scheduling software but it went nowhere.\nI’ll leave you with most shocking statistic I read, related to the insurance industry.\n \u0026quot;In Florida, adults with clean driving records and poor credit scores paid an average of $1,552 more than the same drivers with excellent credit and a drunk driving convition.\n A seemingly inocuous number created by an algorithm is plugged in to another algorithm for an utterly ridiculous outcome.\nIn the discussion group one person said they were vaguely aware that things like this were happening, but didn’t know the extent or specific examples. It does get somewhat repetitive since the author really pounds the ideas into you with tons of examples, but that didn’t bother me. Another person asked that now that we’ve read the book, what can we do about it? We didn’t come up with a lot of answers - be aware and support policies that regulate them. Are there other actions that people can take to combat the proliferation of Weapons of Math Destruction?\n","date":1596412800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1596412800,"objectID":"07c84ad59332cefd5c32921da59b4bd2","permalink":"/post/wmds/","publishdate":"2020-08-03T00:00:00Z","relpermalink":"/post/wmds/","section":"post","summary":"Earlier this summer I got an email about reading groups for grad students hosted by the department of cell and molecular biology. Who wasn’t hungry for a reading group this summer?","tags":["machine learning","big data","ethics","book club"],"title":"Weapons of Math Destruction","type":"post"},{"authors":null,"categories":"R","content":" It took me a few times to get this right - the interface of the USGS website is a rabbit hole of buttons and options for data retrieval. It’s not so hard to get the data you’re interested in, but pay close attention because it’s easy to do the wrong thing. Probably the easiest way is to use the dataRetrieval package. The last commits on their github are from 2 months ago, so it’s pretty up-to-date.\nTo use this package, you need to know a few things about the structure of the data. Each USGS streamflow station has a number - you’ll need to know the number for the station you want. There’s also a number associated with each measured value; 00060 for discharge in feet per second, 00065 for gauge height in feet, 00010 for temperature in degrees celcius, and many more. The last critical thing to know is the date range of the data you’re interested in.\n Station Number Parameter Code Date Range of Interest  Here’s an example from the documentation:\n#install.packages(\u0026quot;dataRetrieval\u0026quot;) #uncomment to install the package library(dataRetrieval) # Choptank River near Greensboro, MD: siteNumber \u0026lt;- \u0026quot;01491000\u0026quot; parameterCd \u0026lt;- \u0026quot;00060\u0026quot; # Discharge startDate \u0026lt;- \u0026quot;2009-10-01\u0026quot; endDate \u0026lt;- \u0026quot;2012-09-30\u0026quot; discharge \u0026lt;- readNWISdv(siteNumber, parameterCd, startDate, endDate) Here’s what the resulting data frame looks like:\nThe fifth column is a quality code - A stands for approved for release by the USGS.\nAnother notable argument to the readNWISdv function is statCd. This allows you to request daily data and specify the statistic used. For example, you can request the daily maximum, mean, median, or total.\nPretty simple, right?\nThere are some other handy functions and options in this library. You’ll notice that the colunm names aren’t really descriptive of the data. You can use another function from the package to rename them. Try this:\ndischarge \u0026lt;- renameNWISColumns(discharge) names(discharge) ## [1] \u0026quot;agency_cd\u0026quot; \u0026quot;site_no\u0026quot; \u0026quot;Date\u0026quot; \u0026quot;Flow\u0026quot; \u0026quot;Flow_cd\u0026quot; That can clean things up a lot - especially if you’re importing many parameters.\nlibrary(ggplot2) ggplot(discharge, aes(x=Date, y=Flow)) + geom_point(alpha = 0.4) What I am interested is the streamflow data, so this is as far as I need to go. You can also import different kinds of USGS data using other functions of the same package. It’s a treasure trove! Check out the documentation to get the full scope of its powers.\nI also came across another package called waterData that can do the same retrieval and maybe some more analysis on the data. I haven’t looked further into this, but the last commits to their github are from 3 years ago, so I choose the dataRetrieval package.\nNow I’ll let you in on the less fun bit, which is the way I originally tried to download the data locally then upload to R. I started out fumbling around the website trying to find the gauge closest to the site I’m interested in. There’s an interactive map on the homepage; you click on a state to get to a more refined map and click around some more to find the gauge you want.\nYou get to a really promising page that gives you some options to select parameters, a date range, and whether you want the output to be a graph, table, or tab-seperated format. This looks good, but is not.\nI chose the tab-seperated option; After a lengthy wait a new tab with the output opened in the browser - so no file download. I guess I could have copied and pasted all of that into a text file, but I decided to search for a way to actually download the file.\nI found a USGS primer on dowloading data (always read the instructions first!), which took some different turns than the path I took previoulsy. The user interface is difficult to navigate and not intuitive, but it’s a great resource for data. I won’t go through the whole process, because it’s mostly just clicking on the right spot and choosing your desired options, but here is the most important part:\nThere are still extra steps of choosing the options in the website, downloading the data, then reading it into R. This is the code I used to read the data once I had downloaded it as a text file and renamed it “streamflow”. If I were to go back, I would not do it this way, but I guess it doesn’t require downloading a new package and is fine if you’re not dowloading a lot of data.\nlibrary(tidyverse) streamflow \u0026lt;- read.table(\u0026quot;streamflow\u0026quot;, skip = 32) #the first 32 lines are metadata colnames(streamflow) \u0026lt;- c(\u0026quot;agency\u0026quot;, \u0026quot;site_no\u0026quot;, \u0026quot;date\u0026quot;, \u0026quot;time\u0026quot;, \u0026quot;timezone\u0026quot;, \u0026quot;Flow\u0026quot;, \u0026quot;quality_code\u0026quot;) #discharge is in cubic feet per second, quality code A means approved streamflow$date \u0026lt;- as.Date(streamflow$date) ggplot(streamflow, aes(x=date, y=Flow)) + geom_point(alpha=0.4) Hope this info helps you avoid wasting time like I did on my first try! Do you know of a better way to get the data into R? Did you encounter any pitfalls?\n","date":1594425600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594425600,"objectID":"e181984d718187faf775ab9f097529f3","permalink":"/post/streamflowdata/","publishdate":"2020-07-11T00:00:00Z","relpermalink":"/post/streamflowdata/","section":"post","summary":"It took me a few times to get this right - the interface of the USGS website is a rabbit hole of buttons and options for data retrieval. It’s not so hard to get the data you’re interested in, but pay close attention because it’s easy to do the wrong thing.","tags":["R Markdown","streamflow","USGS","data mining"],"title":"USGS Data Retrieval","type":"post"},{"authors":["Liz McConnell"],"categories":["R"],"content":" Geotracker is a public database that is used to store environmental data from regulated sites in California. I\u0026rsquo;m going to download some data of a contaminated site and clean it up to try to derive some insights about the site. I picked Alpine county because it is the least populated one and I want to work with a smaller file. Some of the more industrial counties can have thousands of sites.\nURL \u0026lt;- \u0026quot;https://geotracker.waterboards.ca.gov/data_download/edf_by_county/AlpineEDF.zip\u0026quot; download.file(URL, destfile='alpine.zip', method='curl')  After downloading the zipped file I need to unzip it and read in the .txt file. Let\u0026rsquo;s see what\u0026rsquo;s inside.\nunzip('alpine.zip') alpine \u0026lt;- read.delim(\u0026quot;AlpineEDF.txt\u0026quot;) head(alpine)   A data.frame: 6 × 23  COUNTYGLOBAL_IDFIELD_PT_NAMELOGDATELOGTIMELOGCODESAMPIDMATRIXLABWOLABCODE⋯ANADATEBASISPARLABELPARVALPARVQLABDLREPDLUNITSDILFACLNOTE \u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;int\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;⋯\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;chr\u0026gt;   1AlpineT0600300005K1 2012-10-081110CLSRK1-15 SOCVJ0496CLSR⋯2012-10-12WPHCD  0ND0.033 1.0MG/KG1  2AlpineT0600300005KMPUD#22008-12-301330CLSRKMPUD #2 W CSA0056CLSR⋯2009-01-07NMOIL 79= 9.10050.0UG/L 1DU,DU 3AlpineT0600300005KMPUD#22002-04-161203KFR KMPUD #2 WXNA ALPS⋯2002-04-18NMTBE  0ND0.250 0.5UG/L 1  4AlpineT0600300005KMPUD#22002-04-161203KFR KMPUD #2 WXNA ALPS⋯2002-04-18NBZME  0ND0.250 0.5UG/L 1  5AlpineT0600300005KMPUD#22002-10-311805KFR KMPUD WELL #2WXNA ALPS⋯2002-11-04NDCE11 0ND0.500 1.0UG/L 1  6AlpineT0600300005KMPUD#22002-10-311805KFR KMPUD WELL #2WXNA ALPS⋯2002-11-04NBZME  0ND0.250 0.5UG/L 1    There\u0026rsquo;s documentation about all of the fields, but we can already make sense of some of them. GLOBAL_ID represents a site, and FIELD_PT_NAME is a well. PARLABEL is the code for the name of the contaminant and PARVAL is the concentration reported. There\u0026rsquo;s also helpful QA/QC information, and the latitude and longitude of the wells are in a different file on the geotracker website. We\u0026rsquo;ll focus on one site with the most observations reported.\nlibrary(dplyr) alpine %\u0026gt;% group_by(GLOBAL_ID) %\u0026gt;% count() %\u0026gt;% arrange(desc(n))  Attaching package: ‘dplyr’ The following objects are masked from ‘package:stats’: filter, lag The following objects are masked from ‘package:base’: intersect, setdiff, setequal, union   A grouped_df: 7 × 2  GLOBAL_IDn \u0026lt;chr\u0026gt;\u0026lt;int\u0026gt;   T060030001111764 T0600300005 5776 T0600397314 5131 T0600300013 3780 T0600300007 2145 T0600300008 1532   1   Looks like T0600300011 is our site. Let\u0026rsquo;s try to see what the main chemicals of concern and their concentrations are.\nsite \u0026lt;- alpine %\u0026gt;% filter(GLOBAL_ID == \u0026quot;T0600300011\u0026quot;) site %\u0026gt;% group_by(PARLABEL) %\u0026gt;% tally(sort = TRUE) %\u0026gt;% head()   A tibble: 6 × 2  PARLABELn \u0026lt;chr\u0026gt;\u0026lt;int\u0026gt;   BZ 260 BZME 260 EBZ 260 XYLENES1314203 XYLO 203 MTBE 194   The most results came from benzene (BZ), toluene (BZME), ethylbenzene (EBZ), xylene - isomers m \u0026amp; p (XYLENES1314), o-xylene (XYLO), and methyl-tert-butyl ether (MTBE). This is a hydrocarbon site. Now let\u0026rsquo;s try to see which wells are near the source area.\ntop_wells \u0026lt;- site %\u0026gt;% filter(PARLABEL %in% c(\u0026quot;BZ\u0026quot;, \u0026quot;BZME\u0026quot;, \u0026quot;EBZ\u0026quot;, \u0026quot;XYLENES1314\u0026quot;, \u0026quot;XYLO\u0026quot;, \u0026quot;MTBE\u0026quot;)) %\u0026gt;% select(c(\u0026quot;FIELD_PT_NAME\u0026quot;, \u0026quot;LOGDATE\u0026quot;, \u0026quot;MATRIX\u0026quot;, \u0026quot;PARLABEL\u0026quot;, \u0026quot;PARVAL\u0026quot;, \u0026quot;PARVQ\u0026quot;, \u0026quot;LABDL\u0026quot;, \u0026quot;REPDL\u0026quot;, \u0026quot;UNITS\u0026quot;,)) %\u0026gt;% group_by(FIELD_PT_NAME) %\u0026gt;% tally(sort = TRUE) %\u0026gt;% filter(n\u0026gt;100) # the wells with the most data are: top_wells   A tibble: 6 × 2  FIELD_PT_NAMEn \u0026lt;chr\u0026gt;\u0026lt;int\u0026gt;   MW-3167 MW-2161 MWA4157 MW-1155 MWA1143 MWA2143   The most sampled wells are MW-3, MW-2, MWA4, MW-1, MWA1, and MWA2. Naming conventions for wells can mean nothing or can mean a lot, and get wackier the longer a site has been around.\nsite \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME %in% top_wells$FIELD_PT_NAME) %\u0026gt;% filter(PARLABEL %in% c(\u0026quot;BZ\u0026quot;, \u0026quot;BZME\u0026quot;, \u0026quot;EBZ\u0026quot;, \u0026quot;XYLENES1314\u0026quot;, \u0026quot;XYLO\u0026quot;, \u0026quot;MTBE\u0026quot;)) %\u0026gt;% select(c(\u0026quot;FIELD_PT_NAME\u0026quot;, \u0026quot;LOGDATE\u0026quot;, \u0026quot;MATRIX\u0026quot;, \u0026quot;PARLABEL\u0026quot;, \u0026quot;PARVAL\u0026quot;, \u0026quot;PARVQ\u0026quot;, \u0026quot;LABDL\u0026quot;, \u0026quot;REPDL\u0026quot;, \u0026quot;UNITS\u0026quot;,)) head(site)   A data.frame: 6 × 9  FIELD_PT_NAMELOGDATEMATRIXPARLABELPARVALPARVQLABDLREPDLUNITS \u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;chr\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;chr\u0026gt;   1MW-12002-10-08WBZ 0ND5.05.0UG/L 2MW-12003-09-08WBZ 0ND5.05.0UG/L 3MW-12002-11-25WXYLO0ND5.05.0UG/L 4MW-12003-03-05WMTBE0ND5.05.0UG/L 5MW-12003-03-05WEBZ 0ND5.05.0UG/L 6MW-12003-03-05WEBZ 0ND0.50.5UG/L   Let's take a closer look at the units. The first six observations are reported in \u0026mu;g/L, but are they all? First we\u0026rsquo;ll want to clean up the classes - the FIELD_PT_NAME should be a factor and LOGDATE should be a date.\nsite$FIELD_PT_NAME \u0026lt;- as.factor(site$FIELD_PT_NAME) site$LOGDATE \u0026lt;- as.Date(site$LOGDATE) site$MATRIX \u0026lt;- as.factor(site$MATRIX) site$PARLABEL \u0026lt;- as.factor(site$PARLABEL) site$PARVQ \u0026lt;- as.factor(site$PARVQ) site$UNITS \u0026lt;- as.factor(site$UNITS) levels(site$PARLABEL) \u0026lt;- list(Benzene=\u0026quot;BZ\u0026quot;, Toluene=\u0026quot;BZME\u0026quot;, Ethylbenzene=\u0026quot;EBZ\u0026quot;, Xylenes_MP=\u0026quot;XYLENES1314\u0026quot;, Xylene_O=\u0026quot;XYLO\u0026quot;, MTBE=\u0026quot;MTBE\u0026quot;) summary(site)   FIELD_PT_NAME LOGDATE MATRIX PARLABEL MW-1:155 Min. :2002-07-11 W:926 Benzene :179 MW-2:161 1st Qu.:2003-05-27 Toluene :179 MW-3:167 Median :2004-05-05 Ethylbenzene:179 MWA1:143 Mean :2005-11-06 Xylenes_MP :132 MWA2:143 3rd Qu.:2008-05-05 Xylene_O :132 MWA4:157 Max. :2010-04-14 MTBE :125 PARVAL PARVQ LABDL REPDL UNITS Min. : 0.00 = :135 Min. :0.080 Min. : 0.500 UG/L:926 1st Qu.: 0.00 ND:791 1st Qu.:0.380 1st Qu.: 0.500 Median : 0.00 Median :0.500 Median : 0.500 Mean : 20.95 Mean :1.909 Mean : 3.568 3rd Qu.: 0.00 3rd Qu.:5.000 3rd Qu.: 5.000 Max. :1150.00 Max. :9.900 Max. :50.000  Turns out that they are all reported in \u0026mu;g/L, which is good, because that means the concentrations are probably fairly low - remember one \u0026mu;g/L is one part per billion. This isn't always the case - be sure to keep an eye out for units and do conversions as necessary before working with the data. The PARVQ tells us if the chemical was detected in the sample or below the detection limit (ND). For 791 of the observations, the chemical was not detected in the sample, while in only 135 observations a true detected concentration is reported. In cases where the chemical is not detected, is NA, zero, the lab detection limit, report detection limit, or something else reported? site %\u0026gt;% filter(PARVQ == 'ND') %\u0026gt;% head()   A data.frame: 6 × 9  FIELD_PT_NAMELOGDATEMATRIXPARLABELPARVALPARVQLABDLREPDLUNITS \u0026lt;fct\u0026gt;\u0026lt;date\u0026gt;\u0026lt;fct\u0026gt;\u0026lt;fct\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;fct\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;dbl\u0026gt;\u0026lt;fct\u0026gt;   1MW-12002-10-08WBenzene 0ND5.05.0UG/L 2MW-12003-09-08WBenzene 0ND5.05.0UG/L 3MW-12002-11-25WXylene_O 0ND5.05.0UG/L 4MW-12003-03-05WMTBE 0ND5.05.0UG/L 5MW-12003-03-05WEthylbenzene0ND5.05.0UG/L 6MW-12003-03-05WEthylbenzene0ND0.50.5UG/L   It looks like the value reported for non-detected samples is zero. This can cause problems when analyzing the data statistically. Let\u0026rsquo;s visualize what these concentrations look like at MW-1.\nMW1 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MW-1\u0026quot;) MW2 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MW-2\u0026quot;) MW3 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MW-3\u0026quot;) MWA1 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MWA1\u0026quot;) MWA2 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MWA2\u0026quot;) MWA4 \u0026lt;- site %\u0026gt;% filter(FIELD_PT_NAME == \u0026quot;MWA4\u0026quot;) library(ggplot2) ggplot(MW1, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MW-1\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;) ggplot(MW2, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MW-2\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;) ggplot(MW3, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MW-3\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;) ggplot(MWA1, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MWA1\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;) ggplot(MWA2, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MWA2\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;)  At these wells it looks like there were some spikes before 2004, but nothing much going on later. Let\u0026rsquo;s look at the last well from our selection - MWA4.\nggplot(MWA4, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MWA4\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;)  This one has a lot higher concentrations - take a look at the values on the y axis. Yikes! Let\u0026rsquo;s see what the EPA Maximum Contaminant Levels (MCLs) are for these chemicals.\nggplot(MWA4, aes(x = LOGDATE, y = PARVAL, color = PARLABEL)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MWA4\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Concentration in ug/L\u0026quot;) + geom_hline(yintercept=5, linetype=\u0026quot;dashed\u0026quot;, color=\u0026quot;#F8766D\u0026quot;) + geom_hline(yintercept=1000, linetype=\u0026quot;dashed\u0026quot;, color=\u0026quot;#B79F00\u0026quot;) + geom_hline(yintercept=700, linetype=\u0026quot;dashed\u0026quot;, color=\u0026quot;#00BA38\u0026quot;)  There is no MCL for MTBE (there are probably state guidelines) and the MCL for total xylenes is 10,000, which is off this chart. Benzene has the lowest MCL by far, and it is usually the chemical that drives cleanup. It\u0026rsquo;s a little hard for us to see it on this plot, so let\u0026rsquo;s zoom in some more.\nMWA4_benzene \u0026lt;- MWA4 %\u0026gt;% filter(PARLABEL == \u0026quot;Benzene\u0026quot;) ggplot(MWA4_benzene, aes(x = LOGDATE, y = PARVAL, color = \u0026quot;Benzene\u0026quot;,)) + geom_line() + geom_point(alpha = 0.6) + labs(title =\u0026quot;MWA4\u0026quot;, x =\u0026quot;Date\u0026quot;, y = \u0026quot;Benzene Concentration in ug/L\u0026quot;) + geom_hline(yintercept=5, linetype=\u0026quot;dashed\u0026quot;, color=\u0026quot;red\u0026quot;)  Around 2004 this well has almost seven times the MCL of benzene, but it quickly went down to non-detectable or very low for the rest of the recorded period. If I was especially interested in this site I would try to request some of the accompanying reports and perform a Mann-Kendall analysis of trends in the wells, but mostly I wanted to show how you can download data from Geotracker and manipulate it in R.\nThis tutorial is done entirely in a jupyter notebook running an R kernal inside of a docker container. The source files are on github and if you have any questions about using jupyter or docker don\u0026rsquo;t hesitate to contact me!\n","date":1593734400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593734400,"objectID":"42e96a7d091d84d1718a49ca4944826c","permalink":"/post/geotracker/","publishdate":"2020-07-03T00:00:00Z","relpermalink":"/post/geotracker/","section":"post","summary":"Geotracker is a public database that is used to store environmental data from regulated sites in California. I\u0026rsquo;m going to download some data of a contaminated site and clean it up to try to derive some insights about the site.","tags":["R Markdown","jupyter","geotracker","docker"],"title":"Mining Data from California's Geotracker Database","type":"post"},{"authors":null,"categories":"R","content":" In my IoT class last year with Jay Ham we used a website called Dark Sky to get current weather conditions. I’ve been thinking about this recently, since I would like to see if I can match up weather conditions to the changes in the depth to water of wells at a site. I was inspired to look into this based on a talk from Jonathan Kennel from the Univesity of Guelph ( Happy Canada Day! ) and several conversations with my advisor.\nI’ll walk through how I imported the data to R.\nDark Sky is a great resource, however, when I went to re-visit the website I found that they have joined Apple. This means they are no longer creating new accounts. Luckily I already had one from my class last fall. The API is still supported through at least the end of 2021. Later I’ll mention some ways that you could get similar (maybe better) data through other channels.\nThe API allows up to 1000 calls per day for free. Using the Time Machine you can request data from a certain date and location. I focused on hourly data, though it’s probably finer resolution than I need.\n“The hourly data block will contain data points starting at midnight (local time) of the day requested, and continuing until midnight (local time) of the following day.”\nThe docs include a sample API call, which includes your key, the location, and the date requested.\nGET https://api.darksky.net/forecast/0123456789abcdef9876543210fedcba/42.3601,-71.0589,255657600?exclude=currently,flags\nA quick visit to my best friend stack overflow provided a little more clarity about how to use the API in R. The date is in UNIX format. I wanted to start at January 1, 2000, so I used a handy UNIX converter to find my desired date number, 946684800. I replaced the url and now I’m ready to call the API.\n#GET the url req \u0026lt;- httr::GET(\u0026quot;https://api.darksky.net/forecast/{key}/30.012188,-94.024525,946684800?exclude=currently,flags\u0026quot;) req$status_code # should be 200 if it worked. If you get 400, something has gone wrong. # extract req$content cont \u0026lt;- req$content #Convert to char char \u0026lt;- rawToChar(req$content) #Convert to df df \u0026lt;- jsonlite::fromJSON(char) It worked! I removed my private API key, so you’ll have to take my word for it. But now I have a new problem - the call only works for one day at a time. I want a lot of days, so I decided to write a loop.\nOne thing I couldn’t get to work was changing the date inside the string for the API url. I posted in the R for Data Science Slack, and a few minutes later I learned a handy new trick - you can put a variable inside a string by just inserting it with quotes around the variable. Something like this: \u0026quot;https://api.darksky.net/forecast/{key}/30.012188,-94.024525,\u0026quot;,day,\u0026quot;?exclude=currently,flags\u0026quot;\nGreat! I ran the loop and it worked, kinda. It errored out after the first run because rbind could not combine two data frames with different numbers of columns. After looking at the next few days to see which columns were off I saw that it went from 15 to 16 to 17, then back down. Very annoying! They must have added some new info for some days, but this made the data inconsistent so I had to add a select function to the loop. I selected for the 15 columns that were consistent across all days and ran it again. Success!\nHere’s the code I ended up with:\nlibrary(dplyr) #initialize all_hours so there\u0026#39;s something to rbind to df \u0026lt;- jsonlite::fromJSON(paste0(\u0026quot;https://api.darksky.net/forecast/{key}/30.012188,-94.024525,946684800?exclude=currently,flags\u0026quot;)) all_hours \u0026lt;- df$hourly$data # make a vector of unix dates I want (minus the first one, which I already put in all_hours) unix_day \u0026lt;- seq(946771200, 1033084800, by=86400) for (day in unix_day){ df \u0026lt;- jsonlite::fromJSON(paste0(\u0026quot;https://api.darksky.net/forecast/{key}/30.012188,-94.024525,\u0026quot;, day, \u0026quot;?exclude=currently,flags\u0026quot;)) hourly \u0026lt;- select(df$hourly$data, c(cols)) all_hours \u0026lt;- rbind(hourly, all_hours)} #convert unix time to date all_hours$time \u0026lt;- as.POSIXct(all_hours$time, origin=\u0026quot;1970-01-01\u0026quot;) I selected the columns I want and saved them as weather.csv. Let’s zoom in to two rainy days in May 2000.\nlibrary(ggplot2) weather \u0026lt;- read.csv(\u0026quot;weather.csv\u0026quot;) #weather \u0026lt;- weather[2810:2845,] #convert unix time to date weather$time \u0026lt;- as.POSIXct(weather$time, origin=\u0026quot;1970-01-01\u0026quot;) ggplot(weather, aes(x = time, y = precipIntensity, group =1)) + geom_point(alpha = 0.4, color = \u0026quot;blue\u0026quot;) + geom_line(alpha = 0.4, color = \u0026quot;blue\u0026quot;, size = 0.5) + theme_gray() + theme(axis.text.x=element_text(angle=90, hjust=1)) + labs(title =\u0026quot;Precipitation Over Time\u0026quot;, x = \u0026quot;Date\u0026quot;, y = \u0026quot;Precipitation in Inches\u0026quot;) You can clearly see the precipitation-generating storm events over a few months.\nIt will take me a few days of using my 1000 free API calls to cover the period I’m interested in, but overall it was really easy.\nOther Weather Options Clearly getting historical weather data for a certain location can be really useful, but since Dark Sky is no longer creating accounts it’s not a very practical resource.\nThe helpful commenters on the R for Data Science Slack also suggested using NOAA to get the data. This is probably a more robust dataset anyway, but the downside is that they don’t have the option to use lat/long as a location - you have to pick from one of their existing stations. In my case there wasn’t one near the field site I was interested in, but they’re spread out across the country so I bet it will be a great source for a lot of people. There’s a good tutorial on accessing this data at R Open Sci.\nI’ve also hear the Weather Underground has a good API, but it looks like you need to contribute weather data with an IoT device to access it. Cool! But may not be useful to some.\nAre there any other sources of weather data that you know of? Do you have suggestions to improve the approach I used for the Dark Sky data?\n ","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"34180d604b11b2b43719a4bcc5aa421a","permalink":"/post/weather/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/post/weather/","section":"post","summary":"In my IoT class last year with Jay Ham we used a website called Dark Sky to get current weather conditions. I’ve been thinking about this recently, since I would like to see if I can match up weather conditions to the changes in the depth to water of wells at a site.","tags":["R Markdown","weather","data mining","API"],"title":"Accessing Historical Weather Data with Dark Sky API","type":"post"},{"authors":null,"categories":["R"],"content":" If you’re not familiar with Tidy Tuesday, it is a weekly project hosted online by the R for data science community. Every Tuesday a new dataset is released and people are encouraged to explore, analyse, and visualize it in interesting ways. This is my first week exploring tidy tuesday data. Information about the project and datasets is at the tidytuesday github. Before working with this data I watched Julia Silge’s excellent screencast and picked up some great ways to find missing values and recode data.\nA Little History I learned a lot just by looking at the data provided. I was not previously aware of the history captured in the african_names dataset - which lists the names of enslaved people that were freed as they were being illegally smuggled to the Americas. The most names were recorded at the port of Freetown in Sierra Leone before making the trans-atlantic journey. Here’s the description of the dataset excepted on the tidytuesday github page:\nDuring the last 60 years of the trans-Atlantic slave trade, courts around the Atlantic basins condemned over two thousand vessels for engaging in the traffic and recorded the details of captives found on board including their African names. The African Names Database was created from these records, now located in the Registers of Liberated Africans at the Sierra Leone National Archives, Freetown, as well as Series FO84, FO313, CO247 and CO267 held at the British National Archives in London. Links are provided to the ships in the Voyages Database from which the liberated Africans were rescued, as well as to the African Origins site where users can hear the names pronounced and help us identify the languages in which they think the names are used.\n Table 1: Data summary  Name african_names  Number of rows 91490  Number of columns 11  _______________________   Column type frequency:   character 6  numeric 5  ________________________   Group variables None    Variable type: character\n  skim_variable n_missing complete_rate min max empty n_unique whitespace    name 0 1.00 2 24 0 62330 0  gender 12878 0.86 3 5 0 4 0  ship_name 1 1.00 2 59 0 443 0  port_disembark 0 1.00 6 19 0 5 0  port_embark 1126 0.99 4 31 0 59 0  country_origin 79404 0.13 3 31 0 563 0    Variable type: numeric\n  skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist    id 0 1.00 62122.02 51305.07 1.0 22935.25 45822.5 101263.8 199932 ▇▆▃▁▂  voyage_id 0 1.00 17698.25 82016.88 557.0 2443.00 2871.0 3601.0 500082 ▇▁▁▁▁  age 1126 0.99 18.89 8.60 0.5 11.00 20.0 26.0 77 ▆▇▁▁▁  height 4820 0.95 58.61 6.84 0.0 54.00 60.0 64.0 85 ▁▁▂▇▁  year_arrival 0 1.00 1831.40 9.52 1808.0 1826.00 1832.0 1837.0 1862 ▂▆▇▃▁    ## # A tibble: 5 x 2 ## # Groups: port_disembark [5] ## port_disembark n ## \u0026lt;chr\u0026gt; \u0026lt;int\u0026gt; ## 1 Freetown 81009 ## 2 Havana 10058 ## 3 Bahamas unspecified 183 ## 4 Kingston, Jamaica 144 ## 5 St. Helena 96 From this output we can see that country_origin has the most missing data by far. There’s a clue about this in the description of the data above, which mentions the African Origins site, where users can hear the names pronounced and help identify the languages in which they think the names are used.\nSo the people who were liberated were from such different cultures that the original documentarians could not speak the same language or determine where they originally came from. We can see that about 81,000 people were freed in Freetown (This now makes sense - again learning lots here). About 10,000 people were freed in Havana, Cuba, and many less in other locations in the caribbean.\nThe age variable includes entries from a 6-month old child to a 77 year old person. The gender variable has 12,878 missing values and 4 options. I’ll use some of the same techniques as Julia Silge to clean up this data.\nafrican_names %\u0026gt;% group_by(port_disembark, year_arrival) %\u0026gt;% count() %\u0026gt;% arrange(desc(year_arrival)) %\u0026gt;% ggplot(aes(x = year_arrival, y = n, color = port_disembark)) + geom_line(alpha = 0.6, size = 2) + geom_point(alpha = 0.6) We can see that the majority of liberations occured in Freetown. I wonder if the ships had stopped going to Freetown by 1849, or if there was less enforcement, or if they stopped being recorded. Similary, I wonder what the policies were in each of the other ports that made them free enslaved Africans for the time periods reflected in this data.\n ","date":1593223994,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593223994,"objectID":"12411e3f680e40565a389c98e4d8e4eb","permalink":"/post/2020-06-26-ttafamhistory/","publishdate":"2020-06-26T21:13:14-05:00","relpermalink":"/post/2020-06-26-ttafamhistory/","section":"post","summary":"If you’re not familiar with Tidy Tuesday, it is a weekly project hosted online by the R for data science community. Every Tuesday a new dataset is released and people are encouraged to explore, analyse, and visualize it in interesting ways.","tags":["R Markdown","plot","tidytuesday"],"title":"Tidy Tuesday - African American History","type":"post"},{"authors":["Liz McConnell"],"categories":null,"content":" Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.    Click the Slides button above to demo Academic\u0026rsquo;s Markdown slides feature.   Supplementary notes can be added here, including code and math.\n","date":1592956800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1592956800,"objectID":"69425fb10d4db090cfbd46854715582c","permalink":"/publication/conference-paper/","publishdate":"2017-01-01T00:00:00Z","relpermalink":"/publication/conference-paper/","section":"publication","summary":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.","tags":["Source Themes"],"title":"Getting Started","type":"publication"},{"authors":null,"categories":null,"content":"Academic is designed to give technical content creators a seamless experience. You can focus on the content and Academic handles the rest.\nHighlight your code snippets, take notes on math classes, and draw diagrams from textual representation.\nOn this page, you\u0026rsquo;ll find some examples of the types of technical content that can be rendered with Academic.\nExamples Code Academic supports a Markdown extension for highlighting code syntax. You can enable this feature by toggling the highlight option in your config/_default/params.toml file.\n```python import pandas as pd data = pd.read_csv(\u0026quot;data.csv\u0026quot;) data.head() ```  renders as\nimport pandas as pd data = pd.read_csv(\u0026quot;data.csv\u0026quot;) data.head()  Math Academic supports a Markdown extension for $\\LaTeX$ math. You can enable this feature by toggling the math option in your config/_default/params.toml file.\nTo render inline or block math, wrap your LaTeX math with $...$ or $$...$$, respectively.\nExample math block:\n$$\\gamma_{n} = \\frac{ \\left | \\left (\\mathbf x_{n} - \\mathbf x_{n-1} \\right )^T \\left [\\nabla F (\\mathbf x_{n}) - \\nabla F (\\mathbf x_{n-1}) \\right ] \\right |} {\\left \\|\\nabla F(\\mathbf{x}_{n}) - \\nabla F(\\mathbf{x}_{n-1}) \\right \\|^2}$$  renders as\n$$\\gamma_{n} = \\frac{ \\left | \\left (\\mathbf x_{n} - \\mathbf x_{n-1} \\right )^T \\left [\\nabla F (\\mathbf x_{n}) - \\nabla F (\\mathbf x_{n-1}) \\right ] \\right |}{\\left |\\nabla F(\\mathbf{x}_{n}) - \\nabla F(\\mathbf{x}_{n-1}) \\right |^2}$$\nExample inline math $\\nabla F(\\mathbf{x}_{n})$ renders as $\\nabla F(\\mathbf{x}_{n})$.\nExample multi-line math using the \\\\\\\\ math linebreak:\n$$f(k;p_0^*) = \\begin{cases} p_0^* \u0026amp; \\text{if }k=1, \\\\\\\\ 1-p_0^* \u0026amp; \\text {if }k=0.\\end{cases}$$  renders as\n$$f(k;p_0^*) = \\begin{cases} p_0^* \u0026amp; \\text{if }k=1, \\\\\n1-p_0^* \u0026amp; \\text {if }k=0.\\end{cases}$$\nDiagrams Academic supports a Markdown extension for diagrams. You can enable this feature by toggling the diagram option in your config/_default/params.toml file or by adding diagram: true to your page front matter.\nAn example flowchart:\n```mermaid graph TD A[Hard] --\u0026gt;|Text| B(Round) B --\u0026gt; C{Decision} C --\u0026gt;|One| D[Result 1] C --\u0026gt;|Two| E[Result 2] ```  renders as\ngraph TD A[Hard] --\u0026gt;|Text| B(Round) B --\u0026gt; C{Decision} C --\u0026gt;|One| D[Result 1] C --\u0026gt;|Two| E[Result 2]  An example sequence diagram:\n```mermaid sequenceDiagram Alice-\u0026gt;\u0026gt;John: Hello John, how are you? loop Healthcheck John-\u0026gt;\u0026gt;John: Fight against hypochondria end Note right of John: Rational thoughts! John--\u0026gt;\u0026gt;Alice: Great! John-\u0026gt;\u0026gt;Bob: How about you? Bob--\u0026gt;\u0026gt;John: Jolly good! ```  renders as\nsequenceDiagram Alice-\u0026gt;\u0026gt;John: Hello John, how are you? loop Healthcheck John-\u0026gt;\u0026gt;John: Fight against hypochondria end Note right of John: Rational thoughts! John--\u0026gt;\u0026gt;Alice: Great! John-\u0026gt;\u0026gt;Bob: How about you? Bob--\u0026gt;\u0026gt;John: Jolly good!  An example Gantt diagram:\n```mermaid gantt section Section Completed :done, des1, 2014-01-06,2014-01-08 Active :active, des2, 2014-01-07, 3d Parallel 1 : des3, after des1, 1d Parallel 2 : des4, after des1, 1d Parallel 3 : des5, after des3, 1d Parallel 4 : des6, after des4, 1d ```  renders as\ngantt section Section Completed :done, des1, 2014-01-06,2014-01-08 Active :active, des2, 2014-01-07, 3d Parallel 1 : des3, after des1, 1d Parallel 2 : des4, after des1, 1d Parallel 3 : des5, after des3, 1d Parallel 4 : des6, after des4, 1d  An example class diagram:\n```mermaid classDiagram Class01 \u0026lt;|-- AveryLongClass : Cool \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; Class01 Class09 --\u0026gt; C2 : Where am i? Class09 --* C3 Class09 --|\u0026gt; Class07 Class07 : equals() Class07 : Object[] elementData Class01 : size() Class01 : int chimp Class01 : int gorilla class Class10 { \u0026lt;\u0026lt;service\u0026gt;\u0026gt; int id size() } ```  renders as\nclassDiagram Class01 \u0026lt;|-- AveryLongClass : Cool \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; Class01 Class09 --\u0026gt; C2 : Where am i? Class09 --* C3 Class09 --|\u0026gt; Class07 Class07 : equals() Class07 : Object[] elementData Class01 : size() Class01 : int chimp Class01 : int gorilla class Class10 { \u0026lt;\u0026lt;service\u0026gt;\u0026gt; int id size() }  An example state diagram:\n```mermaid stateDiagram [*] --\u0026gt; Still Still --\u0026gt; [*] Still --\u0026gt; Moving Moving --\u0026gt; Still Moving --\u0026gt; Crash Crash --\u0026gt; [*] ```  renders as\nstateDiagram [*] --\u0026gt; Still Still --\u0026gt; [*] Still --\u0026gt; Moving Moving --\u0026gt; Still Moving --\u0026gt; Crash Crash --\u0026gt; [*]  Todo lists You can even write your todo lists in Academic too:\n- [x] Write math example - [x] Write diagram example - [ ] Do something else  renders as\n Write math example Write diagram example Do something else  Tables Represent your data in tables:\n| First Header | Second Header | | ------------- | ------------- | | Content Cell | Content Cell | | Content Cell | Content Cell |  renders as\n   First Header Second Header     Content Cell Content Cell   Content Cell Content Cell    Asides Academic supports a shortcode for asides, also referred to as notices, hints, or alerts. By wrapping a paragraph in {{% alert note %}} ... {{% /alert %}}, it will render as an aside.\n{{% alert note %}} A Markdown aside is useful for displaying notices, hints, or definitions to your readers. {{% /alert %}}  renders as\n A Markdown aside is useful for displaying notices, hints, or definitions to your readers.   Spoilers Add a spoiler to a page to reveal text, such as an answer to a question, after a button is clicked.\n{{\u0026lt; spoiler text=\u0026quot;Click to view the spoiler\u0026quot; \u0026gt;}} You found me! {{\u0026lt; /spoiler \u0026gt;}}  renders as\n Click to view the spoiler  You found me!    Icons Academic enables you to use a wide range of icons from Font Awesome and Academicons in addition to emojis.\nHere are some examples using the icon shortcode to render icons:\n{{\u0026lt; icon name=\u0026quot;terminal\u0026quot; pack=\u0026quot;fas\u0026quot; \u0026gt;}} Terminal {{\u0026lt; icon name=\u0026quot;python\u0026quot; pack=\u0026quot;fab\u0026quot; \u0026gt;}} Python {{\u0026lt; icon name=\u0026quot;r-project\u0026quot; pack=\u0026quot;fab\u0026quot; \u0026gt;}} R  renders as\n  Terminal\n Python\n R\nDid you find this page helpful? Consider sharing it 🙌 ","date":1562889600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1562889600,"objectID":"07e02bccc368a192a0c76c44918396c3","permalink":"/post/writing-technical-content/","publishdate":"2019-07-12T00:00:00Z","relpermalink":"/post/writing-technical-content/","section":"post","summary":"Academic is designed to give technical content creators a seamless experience. You can focus on the content and Academic handles the rest.\nHighlight your code snippets, take notes on math classes, and draw diagrams from textual representation.","tags":null,"title":"Writing technical content in Academic","type":"post"},{"authors":["Liz McConnell"],"categories":null,"content":" Click the Slides button above to demo Academic\u0026rsquo;s Markdown slides feature.   Supplementary notes can be added here, including code and math.\n","date":1554595200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1554595200,"objectID":"557dc08fd4b672a0c08e0a8cf0c9ff7d","permalink":"/publication/preprint/","publishdate":"2017-01-01T00:00:00Z","relpermalink":"/publication/preprint/","section":"publication","summary":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.","tags":["Source Themes"],"title":"An example preprint / working paper","type":"publication"},{"authors":["Liz McConnell"],"categories":[],"content":"from IPython.core.display import Image Image('https://www.python.org/static/community_logos/python-logo-master-v3-TM-flattened.png')  print(\u0026quot;Welcome to Academic!\u0026quot;)  Welcome to Academic!  Install Python and JupyterLab  Install Anaconda which includes Python 3 and JupyterLab.\nAlternatively, install JupyterLab with pip3 install jupyterlab.\nCreate or upload a Jupyter notebook Run the following commands in your Terminal, substituting \u0026lt;MY-WEBSITE-FOLDER\u0026gt; and \u0026lt;SHORT-POST-TITLE\u0026gt; with the file path to your Academic website folder and a short title for your blog post (use hyphens instead of spaces), respectively:\nmkdir -p \u0026lt;MY-WEBSITE-FOLDER\u0026gt;/content/post/\u0026lt;SHORT-POST-TITLE\u0026gt;/ cd \u0026lt;MY-WEBSITE-FOLDER\u0026gt;/content/post/\u0026lt;SHORT-POST-TITLE\u0026gt;/ jupyter lab index.ipynb  The jupyter command above will launch the JupyterLab editor, allowing us to add Academic metadata and write the content.\nEdit your post metadata The first cell of your Jupter notebook will contain your post metadata ( front matter).\nIn Jupter, choose Markdown as the type of the first cell and wrap your Academic metadata in three dashes, indicating that it is YAML front matter:\n--- title: My post's title date: 2019-09-01 # Put any other Academic metadata here... ---  Edit the metadata of your post, using the documentation as a guide to the available options.\nTo set a featured image, place an image named featured into your post\u0026rsquo;s folder.\nFor other tips, such as using math, see the guide on writing content with Academic.\nConvert notebook to Markdown jupyter nbconvert index.ipynb --to markdown --NbConvertApp.output_files_dir=.  Example This post was created with Jupyter. The orginal files can be found at https://github.com/gcushen/hugo-academic/tree/master/exampleSite/content/post/jupyter\n","date":1549324800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1567641600,"objectID":"6e929dc84ed3ef80467b02e64cd2ed64","permalink":"/post/jupyter/","publishdate":"2019-02-05T00:00:00Z","relpermalink":"/post/jupyter/","section":"post","summary":"Learn how to blog in Academic using Jupyter notebooks","tags":[],"title":"Display Jupyter Notebooks with Academic","type":"post"},{"authors":[],"categories":[],"content":"Create slides in Markdown with Academic  Academic | Documentation\n Features  Efficiently write slides in Markdown 3-in-1: Create, Present, and Publish your slides Supports speaker notes Mobile friendly slides   Controls  Next: Right Arrow or Space Previous: Left Arrow Start: Home Finish: End Overview: Esc Speaker notes: S Fullscreen: F Zoom: Alt + Click  PDF Export: E   Code Highlighting Inline code: variable\nCode block:\nporridge = \u0026quot;blueberry\u0026quot; if porridge == \u0026quot;blueberry\u0026quot;: print(\u0026quot;Eating...\u0026quot;)   Math In-line math: $x + y = z$\nBlock math:\n$$ f\\left( x \\right) = ;\\frac{{2\\left( {x + 4} \\right)\\left( {x - 4} \\right)}}{{\\left( {x + 4} \\right)\\left( {x + 1} \\right)}} $$\n Fragments Make content appear incrementally\n{{% fragment %}} One {{% /fragment %}} {{% fragment %}} **Two** {{% /fragment %}} {{% fragment %}} Three {{% /fragment %}}  Press Space to play!\nOne  Two  Three \n A fragment can accept two optional parameters:\n class: use a custom style (requires definition in custom CSS) weight: sets the order in which a fragment appears   Speaker Notes Add speaker notes to your presentation\n{{% speaker_note %}} - Only the speaker can read these notes - Press `S` key to view {{% /speaker_note %}}  Press the S key to view the speaker notes!\n Only the speaker can read these notes Press S key to view    Themes  black: Black background, white text, blue links (default) white: White background, black text, blue links league: Gray background, white text, blue links beige: Beige background, dark text, brown links sky: Blue background, thin dark text, blue links    night: Black background, thick white text, orange links serif: Cappuccino background, gray text, brown links simple: White background, black text, blue links solarized: Cream-colored background, dark green text, blue links   Custom Slide Customize the slide style and background\n{{\u0026lt; slide background-image=\u0026quot;/img/boards.jpg\u0026quot; \u0026gt;}} {{\u0026lt; slide background-color=\u0026quot;#0000FF\u0026quot; \u0026gt;}} {{\u0026lt; slide class=\u0026quot;my-style\u0026quot; \u0026gt;}}   Custom CSS Example Let\u0026rsquo;s make headers navy colored.\nCreate assets/css/reveal_custom.css with:\n.reveal section h1, .reveal section h2, .reveal section h3 { color: navy; }   Questions?  Ask\n Documentation\n","date":1549324800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1549324800,"objectID":"0e6de1a61aa83269ff13324f3167c1a9","permalink":"/slides/example/","publishdate":"2019-02-05T00:00:00Z","relpermalink":"/slides/example/","section":"slides","summary":"An introduction to using Academic's Slides feature.","tags":[],"title":"Slides","type":"slides"},{"authors":null,"categories":null,"content":"","date":1461715200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1461715200,"objectID":"d1311ddf745551c9e117aa4bb7e28516","permalink":"/project/external-project/","publishdate":"2016-04-27T00:00:00Z","relpermalink":"/project/external-project/","section":"project","summary":"An example of linking directly to an external project website using `external_link`.","tags":["Demo"],"title":"External Project","type":"project"},{"authors":null,"categories":null,"content":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum, sem id dapibus fringilla, tellus enim condimentum arcu, nec volutpat est felis vel metus. Vestibulum sit amet erat at nulla eleifend gravida.\nNullam vel molestie justo. Curabitur vitae efficitur leo. In hac habitasse platea dictumst. Sed pulvinar mauris dui, eget varius purus congue ac. Nulla euismod, lorem vel elementum dapibus, nunc justo porta mi, sed tempus est est vel tellus. Nam et enim eleifend, laoreet sem sit amet, elementum sem. Morbi ut leo congue, maximus velit ut, finibus arcu. In et libero cursus, rutrum risus non, molestie leo. Nullam congue quam et volutpat malesuada. Sed risus tortor, pulvinar et dictum nec, sodales non mi. Phasellus lacinia commodo laoreet. Nam mollis, erat in feugiat consectetur, purus eros egestas tellus, in auctor urna odio at nibh. Mauris imperdiet nisi ac magna convallis, at rhoncus ligula cursus.\nCras aliquam rhoncus ipsum, in hendrerit nunc mattis vitae. Duis vitae efficitur metus, ac tempus leo. Cras nec fringilla lacus. Quisque sit amet risus at ipsum pharetra commodo. Sed aliquam mauris at consequat eleifend. Praesent porta, augue sed viverra bibendum, neque ante euismod ante, in vehicula justo lorem ac eros. Suspendisse augue libero, venenatis eget tincidunt ut, malesuada at lorem. Donec vitae bibendum arcu. Aenean maximus nulla non pretium iaculis. Quisque imperdiet, nulla in pulvinar aliquet, velit quam ultrices quam, sit amet fringilla leo sem vel nunc. Mauris in lacinia lacus.\nSuspendisse a tincidunt lacus. Curabitur at urna sagittis, dictum ante sit amet, euismod magna. Sed rutrum massa id tortor commodo, vitae elementum turpis tempus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean purus turpis, venenatis a ullamcorper nec, tincidunt et massa. Integer posuere quam rutrum arcu vehicula imperdiet. Mauris ullamcorper quam vitae purus congue, quis euismod magna eleifend. Vestibulum semper vel augue eget tincidunt. Fusce eget justo sodales, dapibus odio eu, ultrices lorem. Duis condimentum lorem id eros commodo, in facilisis mauris scelerisque. Morbi sed auctor leo. Nullam volutpat a lacus quis pharetra. Nulla congue rutrum magna a ornare.\nAliquam in turpis accumsan, malesuada nibh ut, hendrerit justo. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque sed erat nec justo posuere suscipit. Donec ut efficitur arcu, in malesuada neque. Nunc dignissim nisl massa, id vulputate nunc pretium nec. Quisque eget urna in risus suscipit ultricies. Pellentesque odio odio, tincidunt in eleifend sed, posuere a diam. Nam gravida nisl convallis semper elementum. Morbi vitae felis faucibus, vulputate orci placerat, aliquet nisi. Aliquam erat volutpat. Maecenas sagittis pulvinar purus, sed porta quam laoreet at.\n","date":1461715200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1461715200,"objectID":"8f66d660a9a2edc2d08e68cc30f701f7","permalink":"/project/internal-project/","publishdate":"2016-04-27T00:00:00Z","relpermalink":"/project/internal-project/","section":"project","summary":"An example of using the in-built project page.","tags":["Deep Learning"],"title":"Internal Project","type":"project"},{"authors":["Liz McConnell","吳恩達"],"categories":["Demo","教程"],"content":"Create a free website with Academic using Markdown, Jupyter, or RStudio. Choose a beautiful color theme and build anything with the Page Builder - over 40 widgets, themes, and language packs included!\n Check out the latest demo of what you\u0026rsquo;ll get in less than 10 minutes, or view the showcase of personal, project, and business sites.\n 👉 Get Started 📚 View the documentation 💬 Ask a question on the forum 👥 Chat with the community 🐦 Twitter: @source_themes @GeorgeCushen #MadeWithAcademic 💡 Request a feature or report a bug ⬆️ Updating? View the Update Guide and Release Notes ❤️ Support development of Academic:  ☕️ Donate a coffee 💵 Become a backer on Patreon 🖼️ Decorate your laptop or journal with an Academic sticker 👕 Wear the T-shirt 👩‍💻 Contribute      Academic is mobile first with a responsive design to ensure that your site looks stunning on every device.   Key features:\n Page builder - Create anything with widgets and elements Edit any type of content - Blog posts, publications, talks, slides, projects, and more! Create content in Markdown, Jupyter, or RStudio Plugin System - Fully customizable color and font themes Display Code and Math - Code highlighting and LaTeX math supported Integrations - Google Analytics, Disqus commenting, Maps, Contact Forms, and more! Beautiful Site - Simple and refreshing one page design Industry-Leading SEO - Help get your website found on search engines and social media Media Galleries - Display your images and videos with captions in a customizable gallery Mobile Friendly - Look amazing on every screen with a mobile friendly version of your site Multi-language - 15+ language packs including English, 中文, and Português Multi-user - Each author gets their own profile page Privacy Pack - Assists with GDPR Stand Out - Bring your site to life with animation, parallax backgrounds, and scroll effects One-Click Deployment - No servers. No databases. Only files.  Themes Academic comes with automatic day (light) and night (dark) mode built-in. Alternatively, visitors can choose their preferred mode - click the sun/moon icon in the top right of the Demo to see it in action! Day/night mode can also be disabled by the site admin in params.toml.\n Choose a stunning theme and font for your site. Themes are fully customizable.\nEcosystem   Academic Admin: An admin tool to import publications from BibTeX or import assets for an offline site  Academic Scripts: Scripts to help migrate content to new versions of Academic  Install You can choose from one of the following four methods to install:\n  one-click install using your web browser (recommended)  install on your computer using Git with the Command Prompt/Terminal app  install on your computer by downloading the ZIP files  install on your computer with RStudio  Then personalize and deploy your new site.\nUpdating  View the Update Guide.\nFeel free to star the project on Github to help keep track of updates.\nLicense Copyright 2016-present George Cushen.\nReleased under the MIT license.\n","date":1461110400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1555459200,"objectID":"279b9966ca9cf3121ce924dca452bb1c","permalink":"/post/getting-started/","publishdate":"2016-04-20T00:00:00Z","relpermalink":"/post/getting-started/","section":"post","summary":"Create a beautifully simple website in under 10 minutes.","tags":["Academic","开源"],"title":"Academic: the website builder for Hugo","type":"post"},{"authors":["Liz McConnell","Robert Ford"],"categories":null,"content":" Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.    Click the Slides button above to demo Academic\u0026rsquo;s Markdown slides feature.   Supplementary notes can be added here, including code and math.\n","date":1441065600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1441065600,"objectID":"966884cc0d8ac9e31fab966c4534e973","permalink":"/publication/journal-article/","publishdate":"2017-01-01T00:00:00Z","relpermalink":"/publication/journal-article/","section":"publication","summary":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.","tags":["Source Themes"],"title":"An example journal article","type":"publication"},{"authors":null,"categories":["R"],"content":" R Markdown This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.\nYou can embed an R code chunk like this:\nsummary(cars) ## speed dist ## Min. : 4.0 Min. : 2.00 ## 1st Qu.:12.0 1st Qu.: 26.00 ## Median :15.0 Median : 36.00 ## Mean :15.4 Mean : 42.98 ## 3rd Qu.:19.0 3rd Qu.: 56.00 ## Max. :25.0 Max. :120.00 fit \u0026lt;- lm(dist ~ speed, data = cars) fit ## ## Call: ## lm(formula = dist ~ speed, data = cars) ## ## Coefficients: ## (Intercept) speed ## -17.579 3.932  Including Plots You can also embed plots. See Figure 1 for example:\npar(mar = c(0, 1, 0, 1)) pie( c(280, 60, 20), c(\u0026#39;Sky\u0026#39;, \u0026#39;Sunny side of pyramid\u0026#39;, \u0026#39;Shady side of pyramid\u0026#39;), col = c(\u0026#39;#0292D8\u0026#39;, \u0026#39;#F7EA39\u0026#39;, \u0026#39;#C4B632\u0026#39;), init.angle = -50, border = NA )  Figure 1: A fancy pie chart.   ","date":1437703994,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1437703994,"objectID":"10065deaa3098b0da91b78b48d0efc71","permalink":"/post/2015-07-23-r-rmarkdown/","publishdate":"2015-07-23T21:13:14-05:00","relpermalink":"/post/2015-07-23-r-rmarkdown/","section":"post","summary":"R Markdown This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.","tags":["R Markdown","plot","regression"],"title":"Hello R Markdown","type":"post"}]