action-graphics.Rmd

# Graphics {#action-graphics}

```{r, include = FALSE}
source("common.R")
```

We talked briefly about `renderPlot()` in Chapter \@ref(basic-ui); it's a powerful tool for displaying graphics in your app. This chapter will show you how to use it to its full extent, including how to make your plots interactive. You'll also learn two important related functions: `renderCachedPlot()`, which speeds up your app by caching frequently used plots, and `renderImage()`, which allows you to serve images created in other ways.

In this chapter, we'll show a few base graphics, but will focus on ggplot2.

```{r setup}
library(shiny)
library(ggplot2)
```

## Interactivity

One of the coolest things about `plotOutput()` is that it can also be used as an input, allowing you to create interactive graphics. This is a very powerful tool, and has a wide range of applications. Here I'll show the basics, and give you a few pointers about how you might extend for your own purposes.

There are four mouse events that you can respond to on a plot: `click`, `dblClick` (double click), `hover` (fired when the mouse stays in the same place for a little while), and `brush`. To turn these events into Shiny inputs, you supply a string to the corresponding argument, which then creates an input with that name. For example, imagine you wanted to create a plot that responded to click events. You'd set `plotOutput("plot", click = "plot_click")`.

Here's a simple example that displays the location of the mouse click (note that the location is in data coordinates):

```{r}
ui <- basicPage(
  plotOutput("plot", click = "plot_click"),
  verbatimTextOutput("info")
)

server <- function(input, output) {
  output$plot <- renderPlot({
    plot(mtcars$wt, mtcars$mpg)
  }, res = 96)

  output$info <- renderPrint({
    req(input$plot_click)
    x <- round(input$plot_click$x, 2)
    y <- round(input$plot_click$y, 2)
    paste0("[", x, ", ", y, "]")
  })
}
```

(Note the use of `req()`; that makes sure the app doesn't do anything before the first click.)

The following sections describe the events in more details. We'll start with the "point" events, `click`, `dblClick` and `hover`, whose primary data is the `x` and `y` location of the event (in various units). Then we'll discuss the `hover` event, which gives a rectangle defined by its four sides (`xmin`, `xmax`, `ymin`, and `ymax`). I'll give a brief example of using these events to maintain a logical selection vector; we'll come back to the underlying ideas later in Chapter XYZ.

If you're looking for more general interactivity, you might also want to consider the plotly package, as documented in the book *[Interactive web-based data visualization with R, plotly, and shiny](https://plotly-r.com)*, by Carson Sievert.

### Clicking

The point events return a relatively rich data structure. The most important components of the output `x` and `y` components that give the location of the event in data coordinates. But I'm not going to talk about the input, since you'll only need to use it directly in relatively rare situations. Instead, you'll use the `nearPoints()` helper, which finds data points near the event, taking care of a bunch of fiddly details along the way. Here's a simple example of `nearPoints()` in action, showing a table of data about the points near the event:

```{r}
ui <- fluidPage(
  plotOutput("plot", click = clickOpts("click")),
  tableOutput("data")
)
server <- function(input, output, session) {
  output$plot <- renderPlot({
    plot(mtcars$wt, mtcars$mpg)
  }, res = 96)
  
  output$data <- renderTable({
    nearPoints(mtcars, input$click, xvar = "wt", yvar = "mpg")
  })
}
```

Here we give `nearPoints()` four arguments: the data frame the underlies the plot, the input event, and the names of the plotted `x` and `y` variables. If you use ggplot2, you only need to provide the first two arguments since `x_arg` and `y_arg` can be automatically imputed from the plot data structure. For that reason, I'll use ggplot2 throughout the rest of the chapter.

Here's the previous Shiny app reimplemented with ggplot2:

```{r}
ui <- fluidPage(
  plotOutput("plot", click = "plot_click"),
  tableOutput("data")
)
server <- function(input, output, session) {
  output$plot <- renderPlot({
    ggplot(mtcars, aes(wt, mpg)) + geom_point()
  }, res = 96)
  
  output$data <- renderTable({
    nearPoints(mtcars, input$plot_click)
  })
}
```

Another way to use `nearPoints()` is with `allRows = TRUE` and `addDist = TRUE`; that returns the original data frame with two new columns:

-   `dist_` gives the distance between the row and the event (in pixels).
-   `selected_` says whether or not it should be selected (i.e. the logical vector that's returned.

### Other point events

As well as `click`, you can perform the same operations with `hover` and `dblClick`; just change the name of the argument. You can of course use multiple event types on one plot. Just make sure to explain to the user what they can do: one downside of mouse events is that they're not immediately obvious[^1].

If needed, you can get additional control over the events by supplying `clickOpts()`, `dblclickOpts()`, or `hoverOpts()` instead of a string giving the input id. These are rarely needed, so I won't discuss them here; see the documentation for details.

### Brushing

Another way of selecting points on a plot is to use a **brush**, a rectangular selection defined by four edges. Once you've mastered click and `nearPoints()`, using the brush is straighforward: you use the `hover` event and the `brushedPoints()` helper. Here's another simple example that just shows which points have been selected by the brush:

```{r}
ui <- fluidPage(
  plotOutput("plot", brush = "plot_brush"),
  tableOutput("data")
)
server <- function(input, output, session) {
  output$plot <- renderPlot({
    ggplot(mtcars, aes(wt, mpg)) + geom_point()
  }, res = 96)
  
  output$data <- renderTable({
    brushedPoints(mtcars, input$plot_brush)
  })
}
```

If you want to change the colour of the brush, you'll need to use `brushOpts()`. You can also set the `direction` if you want to brush only the `x` or `y` axes. This is useful for (e.g) brushing time series.

### Modifying data

So far we've displayed the results of the interaction in another output. But the true elegance of interactivity comes when you display the changes in the same plot you're interacting with. Unfortunately this requires an advanced reactivity technique that you have yet learned about: `reactiveVal()`. We'll come back to `reactiveVal()` in Chapter \@ref(reactivity-components), but I wanted to show it here because it's such a useful technique. You'll probably need to re-read this section after you've read that chapter, but hopefully even without all the theory you'll get a sense of the potential applications.

As you might guess from the name, `reactiveVal()` is rather similar to `reactive()`. You create a reactive value by calling `reactiveVal()` with its initial value, and retrieve that value in the same way as a reactive:

```{r, eval = FALSE}
val <- reactiveVal(10)
val()
#> [1] 10
```

However, it has a big difference: you can also update it, and all reactive consumers that refer to it will recompute. It uses a special syntax for updating --- you call it like a function (like retrieving the value) but supply an argument that is the new value:

```{r, eval = FALSE}
val(20)
val()
#> [1] 20
```

So updating a reactive value based on its current value looks something like this:

```{r, eval = FALSE}
val(val() + 1)
val()
#> [1] 21
```

Unfortunately if you actually try to run this code in the console you'll get an error because be run in an reactive environment. That makes it challenging to understand and debug, because you'll need to add a `browser()` call to your Shiny app to get into a state where you can explore what's happening. This is one of the challenges we'll come back to later in Chapter \@ref(reactivity-components).

But for now, lets put the challenges of learning `reactiveVal()` aside, and show you why you might bother. Imagine that you want to visualise the distance between a click and the points on the plot. In the app below, we start by creating a reactive value to store those distances, initialising it with a constant that will be used before we click anything. Then we use `observeEvent()` to update the reactive value when the mouse is clicked, and a ggplot that visualises the distance with point size. All up, this looks something like:

```{r}
df <- data.frame(x = rnorm(100), y = rnorm(100))

ui <- fluidPage(
  plotOutput("plot", click = "plot_click")
)
server <- function(input, output, session) {
  dist <- reactiveVal(rep(1, nrow(df)))
  observeEvent(input$plot_click,
    dist(nearPoints(df, input$plot_click, allRows = TRUE, addDist = TRUE)$dist_)  
  )
  
  output$plot <- renderPlot({
    df$dist <- dist()
    ggplot(df, aes(x, y, size = dist)) + 
      geom_point() + 
      scale_size_area(limits = c(0, 1000), max_size = 10, guide = NULL)
  })
}
```

There are two important ggplot2 techniques to note here:

-   I add the distances to the data frame before plotting it. It's good practice to put everything that you're visualising in a single data frame.
-   I set the `limits` to `scale_size_area()` to ensure that sizes are comparable over time. Here, I did a little interactive experimentation to determine right the range, but you can work out the exact details if needed (see exercises below).

Here's a more complicated idea. I want to use a brush to select (and deselect) points on a plot. I just display them using colours on the plot, but you could imagine many other applications. To make this work, I initialise the `reactiveVal()` to a vector of `FALSE`s, then use `brushedPoints()` and `ifelse()` toggle their values: if they were previously excluded they'll be included; if they were previously included, they'll be excluded.

```{r}
ui <- fluidPage(
  plotOutput("plot", brush = "plot_brush"),
  tableOutput("data")
)
server <- function(input, output, session) {
  selected <- reactiveVal(rep(TRUE, nrow(mtcars)))

  observeEvent(input$plot_brush, {
    brushed <- brushedPoints(mtcars, input$plot_brush, allRows = TRUE)$selected_
    selected(ifelse(brushed, !selected(), selected()))
  })

  output$plot <- renderPlot({
    mtcars$sel <- selected()
    ggplot(mtcars, aes(wt, mpg)) + 
      geom_point(aes(colour = sel)) +
      scale_colour_discrete(limits = c("TRUE", "FALSE"))
  }, res = 96)
 
}
```

Again, I set the limits of the scale to ensure that the legend (and colours) don't change after the first click.

### Data flow

Before we move on, it's important to understand the basic data flow in interactive plots in order to understand their limitations. The basic flow is something like this:

1.  Javascript captures mouse event.
2.  Shiny sends the javascript mouse event back to R, invalidating the input.
3.  Downstream reactive consumers are recomputed.
4.  `plotOutput()` generates a new PNG and sends it to the browser.

For local apps, the bottleneck tends to be the time taken to draw the plot. Depending on how complex the plot is, this may take a signficant fraction of a second. But for hosted apps, you also have to take into account the time needed to transmit the event from the browser to the R, and then the rendered plot back from R to the browser.

In general, this means that it's not possible to create Shiny apps where action and response is percieved as instanteous (i.e. the plot updates simultaneously with your action). If you need this level of speed, you'll need to perform more computation in javascript.

## Dynamic height and width

The rest of this chapter is less exciting than interactive graphics, but important to cover somewhere. First, you can make the size of a plot reactive, so it changes size based on the user's interaction with your app. To do this, you supply functions to the `width` and `height` argument. These functions should have no argument and return the desired size in pixels.They are evaluated in a reactive environment so that you can make the size of your plot dynamic.

The following app illustrates the basic idea. It provides two sliders that directly control the size of the plot:

```{r}
ui <- fluidPage(
  sliderInput("height", "height", min = 100, max = 500, value = 250),
  sliderInput("width", "width", min = 100, max = 500, value = 250),
  sliderInput("n", "n", min = 10, max = 100, value = 25),
  plotOutput("plot", width = 250, height = 250)
)
server <- function(input, output, session) {
  output$plot <- renderPlot(
    width = function() input$width,
    height = function() input$height,
    res = 96,
    {
      plot(rnorm(input$n), rnorm(input$n))
    }
  )
}
```

Note that when you resize the plot, the data stays the same. This is the same behaviour as when you resize a Shiny app that contains a plot with a dynamic height/width (e.g. the default width of 100%).

In real cases, you'd use more complicated expressions in the `width` and `height` functions. For example, if you're using a faceted plot in ggplot2, you might use it to increase the size of the plot to keep the individual facet sizes roughly the same (unfortunately there's no easy way to keep them exactly the same because it's currently not possible to find out the size of the fixed elements around the borders of the plot.)

## Cached plots

If you have an app with complicated plots that take a while to draw, and the same plots are seen by many people, you can get some major performance advantages by used plot caching. This is most a matter of changing `renderPlot()` to `renderCachedPlot()`, but there are a few other issues that you need to consider.

The following app uses `renderCachedPlot()` to speed up the rendering of a large scatterplot of the diamonds dataset. If you run it yourself, you'll notice the first time you show each plot, it takes a noticeable fraction of a second to render (because it has to draw \~50,000 points). But if you re-show that same plot, it appears instantly (because it retrieves the plot from the cache).

```{r}
ui <- fluidPage(
  selectInput("x", "X", choices = names(diamonds), selected = "carat"),
  selectInput("y", "Y", choices = names(diamonds), selected = "price"),
  plotOutput("diamonds")
)

server <- function(input, output, session) {
  output$diamonds <- renderCachedPlot({
    ggplot(diamonds, aes(.data[[input$x]], .data[[input$y]])) + 
      geom_point()
  },
  cacheKeyExpr = list(input$x, input$y))
}
```

You'll notice one important difference between `renderPlot()` and `renderCachedPlot()`: a cached plot also needs a `cacheKeyExpr`, an expression that uniquely identifys each plot. This is the most important argument to `renderCachedPlot()` and we'll discuss it in more detail below. We'll also cover two other important arguments:

-   The sizing policy, which ensures that plot is shared even when the sizes are a little different.
-   The scoping, which controls how frequently the cache is used.

Here we'll focus on the big picture; full the full details you can refer to the [Shiny website](https://shiny.rstudio.com/articles/plot-caching.html).

### Cache key

The `cacheKeyExpr` is the most important argument to `renderCachedPlot()` because it determines when the cache can be used. It should return an object (typically a list of simple vectors) that determins the "state" of the plot; whenever that value is the same as a previous value, a cached plot will be used.

Typically the cache key will be a list of input parameters or simple reactives. It is possible to use a dataset as a cache key, but you should avoid using very large datasets because it take some time to reduce and compare them across runs. If you want a plot to invalidate periodically, you can use something like`proc.time()[[3]] %/% 3600`. This is value will change once per hour (3600 s). You can adjust to any constant.

The cache is also affected by the plot size, and the cache scope, as described below.

### Sizing policy

Normally plots are rendered to the exact size needed by the app. But doesn't work very well for caching, because even a pixel difference would require that the plot be re-rendering. Instead, `renderCachePlot()` caches plots with fixed sizes and then resizing as needed. Generally, you won't need to worry about this as the defaults use an exponential rounding strategy determined by the `sizingPolicy` argument. You can see more details in the `sizeGrowthRatio()` documentation.

You may also want to consider setting cached plots to a fixed size in `plotOutput()`. The default value for `height` is already fixed at `"400px"`, but `width` argument is `"100%"`. If you set `width = "400px"` every plot will be exactly the same size, and you'll get the best cache performance.

### Scoping

By default, the plot cache is stored in memory, and shared across all users of the app. If needed, you can override these defaults with:

-   `cache = "session"`: the cache lifetime will tied to a single user. It will be created when the session starts (i.e. when someone first visits you app), and deleted when the session ends.

-   `cache = diskCache(...)`: shares across multiple users, multiple\
    processes, and app-restarts. Beware that restarting the app will no longer clear the cache, so if you change the plotting code, you'll also need to manually reset the cache by deleting the directory.

It's also possible to store in a database, or right your own backend. See <https://shiny.rstudio.com/https://shiny.rstudio.com/articles/plot-caching.html> for more details.

## Images

If you need to display images that you already have saved on disk. These might come from an external source (maybe a directory of photographs), or perhaps you've generated them with another package (e.g. a 3d scene with`rgl::rgl.snapshot()`).

Unlike `renderPlot()`, `renderImage()` must return a list:

-   `src`: (local) path to the image file.

-   `contentType` entry specifying the MIME type of the image. If one is not provided, will try to detect the type from the file extension. You should only need to set this if you're generating images without file extensions, which generally you won't do.

-   Other components like `width`, `height`

-   `class`, `alt` will be added as attributes to the `<img>` tag.

<https://shiny.rstudio.com/articles/images.html>

### Static images

Sample app shows directory full of files.

`www/images`: another option is to put images in the `www/` directory. This is somewhat more efficient as it allows the browser to use caching automatically.

`?shiny::addResourcePath`

> Static files under the www/ directory are automatically made available under a request path that begins with /.

Then use `img()` + `renderUI()` to generate img tags.

### Dynamic images

In most cases, you're better off figuring out how to use `renderPlot()` because it takes care of so many details for you, it's easily cached, and the vast majority of R packages that generate images already provided some way to .

-   Note the `deleteFile` argument. Set it to `TRUE` if you're rendering the image live, and it should be deleted after its been sent to the user. `outfile <- tempfile(fileext = '.png')`

-   You can detect the size of the plot in the browserwith `session$clientData$output_myImage_width` and `session$clientData$output_myImage_height`. (This works for all outputs, but isn't usually needed. ) Also `session$clientData$pixelratio`.

## Exercises

1.  Make a plot with click handle that shows all the data returned in the input.

2.  Make a plot with click, dblclick, hover, and brush output handlers and nicely display the current selection in the sidebar. Plot the plot in the main panel.

3.  Compute the limits of the distance scale using the size of the plot.

```{r}
output_size <- function(id) {
  reactive(c(
    session$clientData[[paste0("output_", id, "_width")]],
    session$clientData[[paste0("output_", id, "_height")]]
  ))
}
```

[^1]: As a general rule, adding explanatory text suggests that your interface is too complex, so is best avoided, where possible. This is the key idea behind "affordances", the idea that an object should suggest naturally how to interact with it as introduced by Don Norman in the *"Design of Everyday Things"*.