using-rmarkdown.Rmd

---
title: "Writing reproducible, re-usable reports, with R + markdown"
author: "Peter Ralph"
date: September 29, 2020
---

```{r setup, include=FALSE}
knitr::opts_knit$set(self.contained=TRUE)
```

Markdown
========


Goal:
-----


Communicate your thoughts to your collaborators and your future self,


. . .

combining text, math, documented code,

. . .

and **no fuss**.



----------------------

No fussing about layout:

 - unformatted text 
 - [markdown](http://commonmark.org/)
 - [LaTeX](http://latex-project.org/intro.html)

Markdown aims to be readable as-is, \
but has methods to produce beautiful output. 

. . .

> A Markdown-formatted document should be publishable as-is, as plain text, without looking like it's been marked up with tags or formatting instructions. -- [John Gruber](http://daringfireball.net/projects/markdown/syntax#philosophy)



No, really, don't think about the layout
----------------------------------------

Don't even *think* about the layout,
just the content.

> [...] it is better to leave document design to document designers, and to let authors get on with writing documents -- [intro to LaTeX](http://latex-project.org/intro.html)



-----------------------------------------

Today's goal:

1. Learn a few simple rules
2. and get going producing content.





Structuring text
----------------

1.  Separate things with empty lines.
2.  Wrap lines whenever you feel like it.

![your laptop?](Erika_9_typewriter.jpg)

<!--
On a typewriter,
carriage return
always does the same thing.

But, it's nice to have different (semantic) levels of spaces.
-->




Source
---------------

````{.markdown}
Text formatting
===============

Inline markup
-------------

**Paragraphs** begin and end 
with empty lines, and are *not* indented.

- **bold** and *emphasis*
- ~~strikeout~~
- super^scripts^ and sub~scripts~
- and `inline_code($fixed.width)`

````

Result
----------------

### Text formatting


#### Inline markup


**Paragraphs** begin and end with empty lines,
and are *not* indented.

- **bold** and *emphasis*
- ~~strikeout~~
- super^scripts^ and sub~scripts~
- and `inline_code($fixed.width)`


Source
----------------

````{.markdown}
Lists must be preceded by an empty line,

1.  ordered or unordered.
2.  Indenting subsequent content
    
    -  will continue the list

        *   and nest
        *   other lists.

3.  *Indenting* means one tab, or four spaces.

After using markdown for a bit, 
go read [pandoc's documentation](http://pandoc.org/README.html).
````


---------------

Lists must be preceded by an empty line,

1.  ordered or unordered.
2.  Indenting subsequent content
    
    -  will continue the list

        *  and nest
        *  other lists.

3.  *Indenting* means one tab, or four spaces.

After using markdown for a bit, 
go read [pandoc's documentation](http://pandoc.org/README.html).



Math
----

```
Math goes between `$`, single ($\pi$), or double: 
$$ \frac{\pi}{4} = \sum_{n=0}^\infty \frac{(-1)^n}{2k+1} .$$
```
Math goes between `$`, single ($\pi$), or double: 
$$\frac{\pi}{4} = \sum_{n=0}^\infty \frac{(-1)^n}{2k+1} .$$


------------------

Even math environments *inside* double dollar signs:
```{.md}
$$
\begin{align}
    x &= (a+b)^2 - (a-b)^2 \\
      &= 4ab
\end{align}
$$
```
\begin{align}
    x &= (a+b)^2 - (a-b)^2 \\
      &= 4ab
\end{align}



Code blocks
-----------

    ```{.r}
    msg <- "Hello, world."
    print(msg)
    ```

**produces**

```{.R}
msg <- "Hello, world."
print(msg)
```

Blockquotes
-----------

    > I also dream about a modern replacement for LaTeX 
    > designed from the ground up to target multiple output formats 
    > (at least PDF, HTML, EPUB). -- [John MacFarlane](http://john.macfarlane.usesthis.com/)

**produces**

> I also dream about a modern replacement for LaTeX 
> designed from the ground up to target multiple output formats 
> (at least PDF, HTML, EPUB). -- [John MacFarlane](http://john.macfarlane.usesthis.com/)


Links and Images
------------------

[Links](http://www.w3.org/) just go in parentheses, 
and can be [internal](#links-and-images).

```{.markdown}
[Links](http://www.w3.org/) just go in parentheses, 
and can be [internal](#links-and-images).
```

----------------------

Images are **the same** but with a `!` in front. \
Setting width and height are optional.

```{.markdown}
![oh, internet](cat-pandoc.jpg){width=300px height=300px}
```

![oh, internet](cat-pandoc.jpg){width=300px height=300px}




Rendering markdown with [pandoc](http://pandoc.org)
==================================================

Get started!
------------

Directions / in Rstudio.

0.  Make a directory for this workshop \
    / start a new project.
1.  Open a text editor \
    / open a new Rmarkdown file and delete everything.
2.  Write some text.
3.  Save it as `notes.Rmd`.

-------------

To compile, either:

 *  Open `R`, run

    ```{.r}
    library(knitr)
    knit2html("notes.Rmd")
    ```

    and open `notes.html` in your web browser.

 *  **or** in `Rstudio`, save the file (with suffix `.Rmd`) and click on *knit*.

*Where is it?*  Use `getwd()` to tell you.


Wait, but the layout?
---------------------

Remember,

> [...] it is better to leave document design to document designers, and to let authors get on with writing documents.

. . .

If you **must**, then:

- tweak the css
- modify a [rmarkdown format](http://rmarkdown.rstudio.com/developer_custom_formats.html)
- or a [pandoc template](http://pandoc.org/README.html#templates)



Gotchas
-------


1.  many things need to begin with an empty line
1.  carriage returns don't (usually) mean anything
1.  but indentations do (*by at least four spaces*)


. . .


For more info see [the documentation](http://pandoc.org/README.html#the-four-space-rule).



Metadata
--------

At the top of your document, add 
```{.yaml}
---
title: YOUR TITLE HERE
author: YOUR NAME HERE
date: August 23, 2017
---
```

... the [YAML](http://yaml.org/) metadata, \
delimited by **exactly three** dashes.


-----------------

**Note:** Besides setting the title,
you can control the **output** in
[many](http://tex.stackexchange.com/questions/139139/adding-headers-and-footers-using-pandoc/139205#139205) 
[ways](http://rmarkdown.rstudio.com/html_document_format.html#pandoc-arguments) 
here.

*For example*: add

```{.yaml}
output_format : pdf_document
```

and render with `rmarkdown::render("notes.md")`.

. . .

**or even:**


```{.yaml}
output_format : word_document
```



Under the hood
--------------

[`pandoc`](http://pandoc.org/README.html)

```{.sh}
pandoc --help
```

![jgm](jgm.png){width=567px height=339px}




Adding in R
===========


-----------

**Goal:**
add R code to the document,
*along with its output*.

. . .

![knitr](knitr-screenshot.png)


Go ahead
--------

Just add a *chunk* of R code, wrapped in

`````{.Rmd}
 ```{r}
 # PUT ARBITRARY R CODE HERE
 ```
`````

. . .

**Try it!**

----------

`````{.Rmd}
Powers of two?

 ```{r}
 2^(0:10)
 ```
`````

. . .

Powers of two?

```{r}
2^(0:10)
```

----------

`````{.Rmd}
How about this?
$$ \lim_{n \to \infty} 4 \sum_{k=1}^n \frac{ (-1)^n }{ 2n+1 } = \pi , $$

 ```{r}
 cumsum( 4 * (-1)^(0:20) / (2*(0:20)+1) )
 ```
`````

. . .

How about this?
$$ \lim_{n \to \infty} 4 \sum_{k=0}^n \frac{ (-1)^n }{ 2n+1 } = \pi , $$


```{r}
cumsum( 4 * (-1)^(0:20) / (2*(0:20)+1) )
```

--------

`````{.Rmd}
 ```{r}
 plot(cumsum( 4 * (-1)^(0:20) / (2*(0:20)+1) ))
 abline(h=pi, col='red')
 ```
`````

. . .


```{r, fig.height=3.5}
plot(cumsum( 4 * (-1)^(0:20) / (2*(0:20)+1) ))
abline(h=pi, col='red')
```

Exercise
--------

Make a short Rmarkdown document
that

- checks that $$1 + 2 + \cdots + n = n(n+1)/2$$
  for every $n$ between 1 and 100

- shows these on a plot

- explains what's being computed

*Useful:* `x = cumsum(1:100)` and `plot(x)` and `lines(y)`.



What's going on
---------------

1.  `knitr` uses a *regular expression* to find code chunks

    -  pulls these out, 
    -  evaluates them one at a time
    -  and inserts "the results" back in

2.  `pandoc` renders the resulting markdown file

    -   with various choices of styling


Chunk options
-------------

Name each chunk,
and set [options](http://yihui.name/knitr/options/)
for [what gets printed](http://yihui.name/knitr/demo/output/)

`````{.Rmd}
 ```{r my_chunk_name, fig.height=4, echo=FALSE}
`````


`echo=(TRUE|FALSE)` 

: include source code in the output?

`results="(markup|asis)"` 

: style the output or not?

`include=(FALSE|TRUE)` 

: include anything in the output?


-------------

Set document defaults up top:

```````{.Rmd}
 ```{r, include=FALSE}
 fig.dim <- 5
 library(knitr)
 opts_chunk$set(
        fig.height=fig.dim,
        fig.width=2*fig.dim,
        fig.align='center'
    )
 ```
```````


Tables
------

One option: use `pander`.

```````{.Rmd}
 ```{r}
 library(pander)
 bases <- table( sample( c("A","C","G","T"), 300, replace=TRUE ) )
 pander(t(bases))
 ```
```````

*note:* the transpose `t( )`

```{r}
 library(pander)
 bases <- table( sample( c("A","C","G","T"), 300, replace=TRUE ) )
 pander(t(bases))
```


Inline code
-----------

`````{.Rmd}
You can 
``r "r"` paste(letters[c(9,14,19,5,18,20)],collapse='')` 
code anywhere.
`````

You can 
`r paste(letters[c(9,14,19,5,18,20)],collapse='')` 
code anywhere.


--------------

Even in the YAML header.

Go change yours!

    ---
    title: "My notes"
    author: "Peter Ralph"
    date: "``r "r"` date()`"
    ---


Online example
==============

--------------

**Goal:** 
Write a function that will generate all sequences of `A`/`C`/`G`/`T` of length $n$
for which no two adjacent letters are the same.

. . .

Here is a [pre-written solution](examples/sequences.html).


Your turn
---------

Download the 
[iris dataset](https://raw.githubusercontent.com/petrelharp/r-markdown-tutorial/master/examples/iris.tsv) 
to a new directory.

or just do

````{.R}
data(iris)
dir.create("examples")
write.table(iris,file="examples/iris.tsv",sep="\t")
````

----------

1.  Read in the data.

2.  Describe the dataset: number of observations, variables, etcetera.

    - inline `R` code (`` ``r "r nrow(iris)"`` ``)

3.  Make a table of the number of observations for each species.

    * `pander()`
    * or `results="asis"` and `print.xtable(xtable( ),type='html')`

4.  Plot the flower dimensions against each other,

    * using `pairs()`, and colored by species.



Templated reports
=================

-----------------

Set up some fake data:
each has 50 observations of two quantitative variables (`age` and `height`)
and a categorical variable (`type`):

```{r make_data, warning=FALSE}
dir.create("examples/thedata")
owd <- setwd("examples/thedata")
for (samp in LETTERS[1:8]) {
    dir.create(samp)
    xy <- data.frame( 
        age=exp(rnorm(50)),
        type=sample(letters[1:3],50,replace=TRUE)
    )
    xy$height <- 5 + runif(1)*xy$age + 3*runif(1)*as.numeric(xy$type) + rnorm(50)
    write.table(xy,file=paste0(samp,"/data.tsv"))
}
setwd(owd)
```

------------------

We now have 10 datasets,
each in a file like [`A/data.tsv`](examples/thedata/A/data.tsv).
Here's what one looks like:
```{r show_data}
xy
```

------------------

We would like to *visualize* each, 
like this:

```{r plot_one_data, echo=FALSE}
xy <- data.frame( 
    age=exp(rnorm(50)),
    type=sample(letters[1:3],50,replace=TRUE)
)
xy$height <- 5 + runif(1)*xy$age + 3*runif(1)*as.numeric(xy$type) + rnorm(50)/3
with(xy, plot( height ~ age, col=type ) )
legend( "topleft", pch=1, legend=levels(xy$type), col=1:nlevels(xy$type) )
```


---------------------

**The template:** `examples/simple-template.Rmd`

``````{.Rmd}
 ---
 title: "Visualization for ``r "r"` getwd()`"
 date: "``r "r"` date()`"
 ---

 ```{r setup, echo=FALSE}
 input.file <- "data.tsv"
 xy <- read.table(input.file)
 ```

 The file ``r "r"` normalizePath(input.file)`
 has ``r "r"` nrow(xy)` observations:

 ```{r}
 plot( height ~ age, col=type, data=xy )
 legend( "topleft", pch=1, col=1:nlevels(xy$type) )
 ```
``````

**Input:** this looks for the file `data.tsv` in the current directory.

-----------------------

**Render it:**

*Option 1:* copy the template into each of the ten directories, and render them there.

. . .

*Option 2:* use my `templater` package.


```{r, eval=FALSE}
library(devtools)
install_github("petrelharp/templater")
library(templater)
dir.names <- file.path("examples/thedata", LETTERS[1:8])
for (input.dir in dir.names) {
    output.file <- file.path(input.dir, "visualization.html")
    render_template("examples/simple-template.Rmd", output=output.file, 
                    change.rootdir=TRUE, quiet=TRUE)
}
```


-------------------------

**Look at them:**

`````{.Rmd}
 ```{r make_links, results="asis"}
 output.files <- file.path("examples/thedata", LETTERS[1:8], "visualization.html")
 links <- paste("[",dir.names,"](",output.files,")",sep='')
 cat( paste("- ", links, "\n"), "\n" )
 ```
`````

```{r make_links, results="asis", echo=FALSE}
dir.names <- file.path("examples/thedata", LETTERS[1:8])
output.files <- file.path("examples/thedata", LETTERS[1:8], "visualization.html")
links <- paste("[",dir.names,"](",output.files,")",sep='')
cat( paste("- ", links, "\n"), "\n" )
```


Another exercise
----------------

**Goal:**
Compare different $k$ with $k$-means on the `iris` dataset.

1.  Make subdirectories, called `iris/k`, for $1 \le k \le 5$,
2.  and in each runs `kmeans` with the appropriate `k`.

Example:
```{r eval=FALSE}
data(iris)
km <- kmeans(iris[,1:4], centers=3)
pairs(iris, col=km$cluster)
```


Other resources
===============


---------------

 -  [Karl Broman's](http://kbroman.org/knitr_knutshell/pages/Rmarkdown.html) intro to Rmarkdown
 -  the extensive, excellent [documentation for pandoc](http://pandoc.org/README.html)
 -  [Stack](http://stackoverflow.com/questions/tagged/rmarkdown)[Overflow](http://stackoverflow.com/questions/tagged/knitr)
 -  my [technical notes](technical-notes.html) I made while writing this up