I recently worked on a project with the Scottish Government that involved building a dashboard to visualise some data (I won’t bore you with the details beyond this). We had a lot of data to show, and wanted to let users explore it in quite an interactive way, choosing between different ways of cutting up the data and producing customised graphs.

The Shiny framework is a very common choice for this kind of thing, but wasn’t really practical for us for a few reasons.

1. We wanted to restrict access to internal users only
2. …but we also wanted to be able to share it with the odd external stakeholder without too much faff, and
3. it needed to be easy to maintain and update without extensive R or software engineering expertise.

Shiny needs to be deployed to a server configured specifically for this, and doing this in a way that satisfied (1) and (2) would have made (3) very difficult to satisfy. We decided it would be more suitable to create a static HTML document (using R Markdown), which could be uploaded to internal document management systems or just emailed around.

Squeezing interactivity out of R Markdown

As a consequence, I had to figure out how to squeeze as much interactivity as possible out of the static R Markdown format and the tools around it. It went a lot better than I expected, but it took a lot of experimentation.

There some very nice tools available for adding interactivity to static R Markdown documents, but compared to the Shiny ecosystem, there’s nowhere near as much information available on how to make the most of them. Consequently, I believe Shiny often gets used by default in projects that would actually benefit from a (faster, lighter, easier-to-deploy) static HTML implementation, simply because people don’t know this is an option.

Code example

To preserve some of the tricks I discovered, I’ve created an example repository on GitHub, which demonstrates most of the approaches I used in the original project, but without any of the top-secret government data (instead, I’ve used the nycflights13 dataset). Below, I will describe some of the most noteworthy/useful details. You can also see what the end product looks like here.

How the whole thing fits together

All the steps needed to build the dashboard (except installing the required packages) are automated in the create_outputs.R script, which you can run in R Studio (or in a terminal if you’d rather). This will:

• Create an “output” folder, where all the outputs will live
• Run some scripts that import and clean the required data – in this example, this just involves loading an R package, but in the actual project we had a bunch of CSV and Excel files organised in an “input” folder (and stored together with the source code)
• Run some more scripts that create the summary datasets we want to display on the dashboard – these roughly correspond to different pages or plots
• Build the HTML output from the dashboard.Rmd file, where the content and layout of the dashboard itself is defined
• Do some tidying up that is mostly specific to the particular setup we had:
  • The document storage system we used to store the output broke when we uploaded an HTML file on its own (specifically, it injected some mystery JavaScript code that broke the dashboard), so we had to store it as a ZIP file
  • We decided not to use Git for version control, so we also store the source code itself as a ZIP file in the same document storage system (Yes, Git is awesome, I recommend using it whenever possible, but software engineering is the art of compromise and in this context the overhead outweighed the benefit 😢)

Packages

The most important ingredients are:

• R Markdown, which lets you create Word, PDF or HTML documents (among other things) with R plots and tables in them
• flexdashboard, a custom R Markdown format that outputs nice-looking dashboards with multiple pages and configurable layout
• Various htmlwidgets-compatible packages, which let you use various fancy interactive JavaScript-based tools (“widgets”) in an R Markdown document – in particular:
  • Plotly (which I used extensively for plots)
  • DT (which I haven’t used in the example code, but used for a few tables in the actual project)
• crosstalk, an “add-on to the htmlwidgets package” that lets the user filter datasets on the fly (using dropdowns, sliders, etc.), and allows multiple graphs/tables to respond to those changes

Interactivity by filtering data

Pretty much all of the pages use the same general approach. There are some controls on the page (dropdowns, etc.) that allow the user to make choices about what to see. The dataset that we feed into the graph contains every possible combination of settings that the user can choose. Since the controls and the graphs are interconnected, the user only ever sees the subset of the data corresponding to the options they have chosen.

Setting default options

An important caveat about this filtering approach is that if the user left any of the filter controls blank, chaos would ensue, because they would suddenly see lots of different data at once that isn’t meant to be seen together. So setting a default value for every control is very important.

Unfortunately, there’s no built-in way to provide defaults for Crosstalk’s filter controls as of yet. It is, however, possible to do this by including some JavaScript code (as discussed here, on StackOverflow) that asks the browser to change the control to a specific value once the page has loaded:

window.addEventListener("load", function(event) { 
  document.getElementById("insert the same ID string you chose in the filter_select() function in R").getElementsByClassName("selectized")[0].selectize.setValue("insert the value that should be selected by default", false);
});

Because this is a mouthful, I put it in a function at the start of the document, which we can then use to set individual default values a bit more elegantly (but just a bit…) near where the control itself is defined.

This solution is admittedly pretty janky (and could quite conceivably break if changes are made in future versions of Crosstalk), so I really hope that this eventually gets incorporated into Crosstalk itself, as discussed in an open GitHub issue. Another caveat: because of the way this works, the messy, all-the-data version shows up for a split second when the page loads, but because users will generally start on the Background page, this is usually not visible.

Ordering bar graphs

Another thing that took a lot of trial and error to figure out was how to order bar graphs by size (using Plotly), and in particular, how to put two bar graphs side by side, with one ordered by size, and the other just ordered like the first. Some of the approaches I tried for this worked sort of okay for the first but not the second scenario, or broke in unpredictable ways when you changed the settings.

So to save you the days of trial and error: the most robust way to do this turned out to be to sort the dataset in advance (in R), and then use categoryorder = "trace" in Plotly, which means that the values are displayed in exactly the order that they appear in the dataset. I think this is why the side-by-side plots work as well – not because the two plots are talking to each other, but because the plot on the right has been fed data in the same order as the plot on the left. But honestly I’m not sure, because working with Plotly often felt more like programming a trickster god than a computer.

That’s it

I think that covers the main things. If you have any further questions, feel free to email or get in touch via Twitter or Mastodon.