R Markdown publishing options. Source: RStudio and Alison Hill R Markdown resources (illustration by Allison Horst).

1 Introduction

R Markdown is a powerful and flexible tool for reproducible data analysis, collaboration, and publication (e.g. papers, blogs, websites, dashboards, CVs) in R or other languages (Python, SQL, and more).

Click HERE for the workshop intro slides.

1.1 Workshop outline

  1. Review basics of working in R Markdown for data analysis

  2. Customize R Markdown outputs for fun and functionality

  3. A few useful features for research, including:

    • Reference code outputs in text for reproducible and efficient reporting
    • Sourcing external scripts in an Rmd
    • From R Markdown to R scripts and back with spin() and purl()

2 Demo

Note: the only packages we’ll use for this workshop is the tidyverse package.

library(tidyverse)

2.1 R Markdown essentials

We will consider three essential elements of R Markdown in this workshop:

  • YAML
  • Markdown text
  • Code chunks

2.1.1 YAML

The YAML (“YAML ain’t markup language” or “Yet another markup language”) is the document information stored atop the R Markdown document, bounded on each end by ---. When you first create a new .Rmd, that will look something like this:

---
title: "Document title"
author: "Author"
date: "MM/DD/YYYY"
output: html_document
---

We’ll leave it as-is for now, but know that this is where we can quickly set a number of options for the document related to output format, themes, layouts, and more.

2.1.2 Markdown text

Most stuff that is outside of the YAML and code chunks in our .Rmd is our text in markdown. We won’t spend a lot of time customizing text, but here are a few commonly used formatting examples:

Use different numbers of pound signs (hashtags) to start a new line, followed by a space then the text, to create headers. If there is one #, that is a level one (by default, the largest) header. As you increase the number of pound signs to start a line, the header decreases in size.

This written in R Markdown:

### Header 3

…produces a Level 3 (moderately sized) header when knitted.

And we use markdown similarly to update text in other ways, like:

  • A single asterisk on either side of text makes it italic
  • A double asterisk on either side of text makes it bold
  • Use a hat (^) on the end of text to make asuperscript
  • Or an tilde (~) on each end to make asubscript

You could manually format all of your text in markdown. OR (drum roll) you could use the shiny new visual editor in RMarkdown (formal release of RStudio 1.4 announced January 2021). If you are running the newest version of RStudio, there is a compass icon in the top right of the .Rmd tab that allows you to switch between the Visual Editor and plain markdown.

2.1.3 Code chunks

Code chunks are where we’ll add most code in an R Markdown document (though we’ll see later we can also add or refer to R code inline). There are a number of ways to add a code chunk, including:

  • Click on the green + c button in the top bar for the .Rmd, then choose R (or explore options for adding code in other languages)

  • In the menu, click Code > Insert Chunk

  • Use the shortcut (Cmd + Option + I) << my favorite

  • Just type in the start and end gates (```{r} to start, ``` to end)

Within a new code chunk, let’s add some code:

ggplot(data = mtcars, aes(x = wt, y = mpg)) +
  geom_point()

When we knit, the code and graph appear in the document, demonstrating that we can have our text, code and outputs all in the same place - and any outputs will automatically update when we change our code & re-knit.

2.1.4 Aside: chunk options

Chunk options are options designated in the code chunk header that determine what appears or does not appear for each chunk upon knitting (and a lot more, but we’ll start there…). For more information on code chunk options in R Markdown, see Chapter 11 in the R Markdown Cookbook.

Chunk options can be added to individual code chunk headers (within the {r} atop the chunk), or applied globally by adding options to the {r setup} code chunk at the top of the document.

Here are some common chunk options:

  • echo = FALSE: do not show the code in the knitted document
  • include = FALSE: do not include code or any outputs in knitted document
  • message = FALSE: suppress messages when knit
  • warning = FALSE: suppress warnings when knit
  • eval = FALSE: do not evaluate this code

2.2 R Markdown customization

For today, we’ll make changes to the YAML in our .Rmd to fancy our output document. Our customization will be related to our knitted HTML output, using the following:

  • toc: true: add a table of contents based on header hierarchies
  • toc_float: true: make it a floating TOC
  • number_sections: true: add numbered sections based on header hierarchy
  • theme: _____: add a bootstrap theme
  • code_folding: show: code is default shown, but can be hidden if the reader clicks on the ‘Hide’ button created (alternative default is code_folding: hide)
  • code_download: true: add a button atop the knitted document to download the .Rmd!

For free bootstrap themes, visit: https://bootswatch.com/3/

To add options to the YAML, add them as “children” in the html_document subsection. Beware of spacing here: generally add 2 spaces of indentation for each new sublevel in the YAML hierarchy.

---
title: "My title"
subtitle: "And subtitle"
author: "Allison Horst"
output: 
  html_document:
    theme: flatly
    toc: true
    toc_float: true
    number_sections: true
    code_folding: show
    code_download: true
---

Let’s try them out in our R Markdown document!

2.3 R Markdown for research

2.3.1 Inline code

Often, we want to reference code outcomes within our text. For example, we may want to refer to a mean value, like:

  "The mean nitrate concentration in Lake A is 11.2 mg/L." 

The common option of copying and pasting values, however, is tedious and dangerous. A better way is to refer to code outputs directly, so that values are updated automatically when code or data change.

In R Markdown, we add code inline with a single back-tick (`) on either end, and a lowercase r between to start it.

Then we can insert code within our text, which under the hood contains this inline R code to calculate the mean gas mileage for all cars in the mtcars dataset:

r round(mean(mtcars$mpg, na.rm = TRUE), 2)

…to produce this output: “The mean gas mileage of cars in the dataset is 20.09 mm.”

Then, if our data or code ever change, so will our text output.

2.3.2 Sourcing scripts

Sometimes, you’ll want to incorporate something from an R script into an R Markdown document. This might be the case if you have long code, and you might want it to exist in a script instead of cluttering up your .Rmd.

You can script external R scripts like this (from Ch 16.1 of the fantastic R Markdown Cookbook by Yihui Xie, Christophe Dervieux, and Emily Riederer):

source("your-script.R", local = knitr::knit_global())

For example, I can read in my script my_function.R, which contains a function fruit, which looks like this:

fruit <- function(apples, bananas, oranges, berries) {
  apples + bananas + oranges*berries
}

We can make that available to us in our R Markdown document with:

source("my_function.R", local = knitr::knit_global())

Which means then the function is now available for me to use…

fruit(apples = 2, bananas = 1, oranges = 5, berries = 10)
## [1] 53

2.3.3 From R Markdown to R scripts & back with spin() and purl()

Sometimes it can be useful to pull out all code from code chunks in an R Markdown document and condense them into an R script. Or sometimes it’s useful to do the opposite - take information in an R script and add it to an R Markdown document. We can do these with knitr::purl() and knitr::spin(), respectively.

  • Use knitr::purl() to aggregate code from code chunks (not inline code) into an R script
  • Use knitr::spin() to convert an R script to an R Markdown document

2.3.3.1 purl(): R Markdown > R script

Let’s purl() the document you’ve been working in to create an R script!

  • In the R Console, run knitr::purl("your_file_name.Rmd").

You should see a .R script created in the root, with the same name but ending in .R - check it out!

2.3.3.2 spin(): R script > Markdown

See more information on rendering reports from R scripts here. Let’s create a new little R script, and add some information that will be rendered as Markdown syntax.

  • Create a new empty .R script, then copy and paste the following code & annotation into it:
#' Let's make a markdown document (.md) with `knitr::spin()`.
#'
#' This is normal text
#'
#' ## This is a level 2 header
#'
#' *This text is italicized*

#+ 
library(tidyverse)

#+
# An actual comment in the code chunk
ggplot(data = starwars, aes(x = height, y = mass)) +
  geom_point()

#' Then some normal text outside
  • Save that .R script (e.g. as starwars_graph.R).

  • In the Console, run:

knitr::spin("file_name.R")

Notice that the .md and the knitted HTML are saved! That’s how we can render a report straight from an R script.

3 Resources

LS0tCnRpdGxlOiAiUiBNYXJrZG93biBmb3IgZnVuLCBmdW5jdGlvbmFsIGFuZCByZXByb2R1Y2libGUgcmVwb3J0aW5nIgpzdWJ0aXRsZTogIlVyYmFuIERyYWluYWdlIE1vZGVsaW5nIENvbmZlcmVuY2UgKEphbnVhcnkgMjAyMikiCmF1dGhvcjogIkFsbGlzb24gSG9yc3QiCm91dHB1dDogCiAgaHRtbF9kb2N1bWVudDoKICAgIHRoZW1lOiBsdW1lbgogICAgdG9jOiB0cnVlCiAgICB0b2NfZmxvYXQ6IHRydWUKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgY29kZV9mb2xkaW5nOiBzaG93Ci0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGUgPSBGQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFLCBtZXNzYWdlID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRSkKYGBgCgohW1IgTWFya2Rvd24gcHVibGlzaGluZyBvcHRpb25zLiBTb3VyY2U6IFJTdHVkaW8gYW5kIEFsaXNvbiBIaWxsIFIgTWFya2Rvd24gcmVzb3VyY2VzIChpbGx1c3RyYXRpb24gYnkgQWxsaXNvbiBIb3JzdCkuXShybWFya2Rvd25faGVkZ2Vob2dfd2lkZV9zbS5wbmcpCgojIEludHJvZHVjdGlvbgoKW1IgTWFya2Rvd25dKGh0dHBzOi8vcm1hcmtkb3duLnJzdHVkaW8uY29tLykgaXMgYSBwb3dlcmZ1bCBhbmQgZmxleGlibGUgdG9vbCBmb3IgcmVwcm9kdWNpYmxlIGRhdGEgYW5hbHlzaXMsIGNvbGxhYm9yYXRpb24sIGFuZCBwdWJsaWNhdGlvbiAoZS5nLiBwYXBlcnMsIGJsb2dzLCB3ZWJzaXRlcywgZGFzaGJvYXJkcywgQ1ZzKSBpbiBSIG9yIG90aGVyIGxhbmd1YWdlcyAoUHl0aG9uLCBTUUwsIGFuZCBtb3JlKS4KCkNsaWNrIFtIRVJFXShodHRwczovL2RvY3MuZ29vZ2xlLmNvbS9wcmVzZW50YXRpb24vZC9lLzJQQUNYLTF2Ulo3MlFkS19ucS1zNGh3akxZQ2dVbURWV014ZEF4X1d3RDZFcXU5U2dDbWp2aG11aXc2TTRYYnF6Q3FtcUtWUkllYVVyTzFCR0dVLXE2L3B1Yj9zdGFydD1mYWxzZSZsb29wPWZhbHNlJmRlbGF5bXM9MzAwMCkgZm9yIHRoZSB3b3Jrc2hvcCBpbnRybyBzbGlkZXMuCgoKIyMgV29ya3Nob3Agb3V0bGluZQoKMS4gIFJldmlldyBiYXNpY3Mgb2Ygd29ya2luZyBpbiBSIE1hcmtkb3duIGZvciBkYXRhIGFuYWx5c2lzCgoyLiAgQ3VzdG9taXplIFIgTWFya2Rvd24gb3V0cHV0cyBmb3IgZnVuIGFuZCBmdW5jdGlvbmFsaXR5CgozLiAgQSBmZXcgdXNlZnVsIGZlYXR1cmVzIGZvciByZXNlYXJjaCwgaW5jbHVkaW5nOgoKICAgIC0gICBSZWZlcmVuY2UgY29kZSBvdXRwdXRzIGluIHRleHQgZm9yIHJlcHJvZHVjaWJsZSBhbmQgZWZmaWNpZW50IHJlcG9ydGluZwogICAgLSAgIFtTb3VyY2luZyBleHRlcm5hbCBzY3JpcHRzXShodHRwczovL2Jvb2tkb3duLm9yZy95aWh1aS9ybWFya2Rvd24tY29va2Jvb2svc291cmNlLXNjcmlwdC5odG1sKSBpbiBhbiBSbWQKICAgIC0gICBGcm9tIFIgTWFya2Rvd24gdG8gUiBzY3JpcHRzIGFuZCBiYWNrIHdpdGggYHNwaW4oKWAgYW5kIGBwdXJsKClgCgojIERlbW8KCk5vdGU6IHRoZSBvbmx5IHBhY2thZ2VzIHdlJ2xsIHVzZSBmb3IgdGhpcyB3b3Jrc2hvcCBpcyB0aGUgYHRpZHl2ZXJzZWAgcGFja2FnZS4gCgpgYGB7ciwgZWNobyA9IFRSVUUsIHdhcm5pbmcgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKYGBgCgojIyBSIE1hcmtkb3duIGVzc2VudGlhbHMKCldlIHdpbGwgY29uc2lkZXIgdGhyZWUgZXNzZW50aWFsIGVsZW1lbnRzIG9mIFIgTWFya2Rvd24gaW4gdGhpcyB3b3Jrc2hvcDoKCi0gICBZQU1MCi0gICBNYXJrZG93biB0ZXh0Ci0gICBDb2RlIGNodW5rcwoKIyMjIFlBTUwKClRoZSBZQU1MICgiWUFNTCBhaW4ndCBtYXJrdXAgbGFuZ3VhZ2UiIG9yICJZZXQgYW5vdGhlciBtYXJrdXAgbGFuZ3VhZ2UiKSBpcyB0aGUgZG9jdW1lbnQgaW5mb3JtYXRpb24gc3RvcmVkIGF0b3AgdGhlIFIgTWFya2Rvd24gZG9jdW1lbnQsIGJvdW5kZWQgb24gZWFjaCBlbmQgYnkgYC0tLWAuIFdoZW4geW91IGZpcnN0IGNyZWF0ZSBhIG5ldyAuUm1kLCB0aGF0IHdpbGwgbG9vayBzb21ldGhpbmcgbGlrZSB0aGlzOgoKICAgIC0tLQogICAgdGl0bGU6ICJEb2N1bWVudCB0aXRsZSIKICAgIGF1dGhvcjogIkF1dGhvciIKICAgIGRhdGU6ICJNTS9ERC9ZWVlZIgogICAgb3V0cHV0OiBodG1sX2RvY3VtZW50CiAgICAtLS0KCldlJ2xsIGxlYXZlIGl0IGFzLWlzIGZvciBub3csIGJ1dCBrbm93IHRoYXQgdGhpcyBpcyB3aGVyZSB3ZSBjYW4gcXVpY2tseSBzZXQgYSBudW1iZXIgb2Ygb3B0aW9ucyBmb3IgdGhlIGRvY3VtZW50IHJlbGF0ZWQgdG8gb3V0cHV0IGZvcm1hdCwgdGhlbWVzLCBsYXlvdXRzLCBhbmQgbW9yZS4KCiMjIyBNYXJrZG93biB0ZXh0CgpNb3N0IHN0dWZmIHRoYXQgaXMgKm91dHNpZGUqIG9mIHRoZSBZQU1MIGFuZCBjb2RlIGNodW5rcyBpbiBvdXIgLlJtZCBpcyBvdXIgdGV4dCBpbiBtYXJrZG93bi4gV2Ugd29uJ3Qgc3BlbmQgYSBsb3Qgb2YgdGltZSBjdXN0b21pemluZyB0ZXh0LCBidXQgaGVyZSBhcmUgYSBmZXcgY29tbW9ubHkgdXNlZCBmb3JtYXR0aW5nIGV4YW1wbGVzOgoKVXNlIGRpZmZlcmVudCBudW1iZXJzIG9mIHBvdW5kIHNpZ25zIChoYXNodGFncykgdG8gc3RhcnQgYSBuZXcgbGluZSwgZm9sbG93ZWQgYnkgYSBzcGFjZSB0aGVuIHRoZSB0ZXh0LCB0byBjcmVhdGUgaGVhZGVycy4gSWYgdGhlcmUgaXMgb25lIGAjYCwgdGhhdCBpcyBhIGxldmVsIG9uZSAoYnkgZGVmYXVsdCwgdGhlIGxhcmdlc3QpIGhlYWRlci4gQXMgeW91IGluY3JlYXNlIHRoZSBudW1iZXIgb2YgcG91bmQgc2lnbnMgdG8gc3RhcnQgYSBsaW5lLCB0aGUgaGVhZGVyIGRlY3JlYXNlcyBpbiBzaXplLgoKVGhpcyB3cml0dGVuIGluIFIgTWFya2Rvd246CgpcI1wjXCMgSGVhZGVyIDMKCi4uLnByb2R1Y2VzIGEgTGV2ZWwgMyAobW9kZXJhdGVseSBzaXplZCkgaGVhZGVyIHdoZW4ga25pdHRlZC4KCkFuZCB3ZSB1c2UgbWFya2Rvd24gc2ltaWxhcmx5IHRvIHVwZGF0ZSB0ZXh0IGluIG90aGVyIHdheXMsIGxpa2U6CgotICAgQSBzaW5nbGUgYXN0ZXJpc2sgb24gZWl0aGVyIHNpZGUgb2YgdGV4dCBtYWtlcyBpdCAqaXRhbGljKgotICAgQSBkb3VibGUgYXN0ZXJpc2sgb24gZWl0aGVyIHNpZGUgb2YgdGV4dCBtYWtlcyBpdCAqYm9sZCoKLSAgIFVzZSBhIGhhdCAoXikgb24gdGhlIGVuZCBvZiB0ZXh0IHRvIG1ha2UgYV5zdXBlcnNjcmlwdF4KLSAgIE9yIGFuIHRpbGRlIChcfikgb24gZWFjaCBlbmQgdG8gbWFrZSBhfnN1YnNjcmlwdH4KCllvdSBjb3VsZCBtYW51YWxseSBmb3JtYXQgYWxsIG9mIHlvdXIgdGV4dCBpbiBtYXJrZG93bi4gT1IgKGRydW0gcm9sbCkgeW91IGNvdWxkIHVzZSB0aGUgc2hpbnkgbmV3IHZpc3VhbCBlZGl0b3IgaW4gUk1hcmtkb3duIChmb3JtYWwgcmVsZWFzZSBvZiBbUlN0dWRpbyAxLjQgYW5ub3VuY2VkIEphbnVhcnkgMjAyMV0oaHR0cHM6Ly9ibG9nLnJzdHVkaW8uY29tLzIwMjEvMDEvMTkvYW5ub3VuY2luZy1yc3R1ZGlvLTEtNC8pKS4gSWYgeW91IGFyZSBydW5uaW5nIHRoZSBuZXdlc3QgdmVyc2lvbiBvZiBSU3R1ZGlvLCB0aGVyZSBpcyBhIGNvbXBhc3MgaWNvbiBpbiB0aGUgdG9wIHJpZ2h0IG9mIHRoZSAuUm1kIHRhYiB0aGF0IGFsbG93cyB5b3UgdG8gc3dpdGNoIGJldHdlZW4gdGhlIFZpc3VhbCBFZGl0b3IgYW5kIHBsYWluIG1hcmtkb3duLgoKIyMjIENvZGUgY2h1bmtzCgpDb2RlIGNodW5rcyBhcmUgd2hlcmUgd2UnbGwgYWRkIG1vc3QgY29kZSBpbiBhbiBSIE1hcmtkb3duIGRvY3VtZW50ICh0aG91Z2ggd2UnbGwgc2VlIGxhdGVyIHdlIGNhbiBhbHNvIGFkZCBvciByZWZlciB0byBSIGNvZGUgaW5saW5lKS4gVGhlcmUgYXJlIGEgbnVtYmVyIG9mIHdheXMgdG8gYWRkIGEgY29kZSBjaHVuaywgaW5jbHVkaW5nOiAKCi0gQ2xpY2sgb24gdGhlIGdyZWVuIGArIGNgIGJ1dHRvbiBpbiB0aGUgdG9wIGJhciBmb3IgdGhlIC5SbWQsIHRoZW4gY2hvb3NlIGBSYCAob3IgZXhwbG9yZSBvcHRpb25zIGZvciBhZGRpbmcgY29kZSBpbiBvdGhlciBsYW5ndWFnZXMpCgotIEluIHRoZSBtZW51LCBjbGljayBDb2RlID4gSW5zZXJ0IENodW5rCgotIFVzZSB0aGUgc2hvcnRjdXQgKENtZCArIE9wdGlvbiArIEkpIDw8IG15IGZhdm9yaXRlCgotIEp1c3QgdHlwZSBpbiB0aGUgc3RhcnQgYW5kIGVuZCBnYXRlcyAoXGBcYFxgYHtyfWAgdG8gc3RhcnQsIFxgXGBcYCB0byBlbmQpCgpXaXRoaW4gYSBuZXcgY29kZSBjaHVuaywgbGV0J3MgYWRkIHNvbWUgY29kZTogCgpgYGB7ciwgZWNobyA9IFRSVUUsIGV2YWwgPSBGQUxTRX0KZ2dwbG90KGRhdGEgPSBtdGNhcnMsIGFlcyh4ID0gd3QsIHkgPSBtcGcpKSArCiAgZ2VvbV9wb2ludCgpCmBgYAoKV2hlbiB3ZSBrbml0LCB0aGUgY29kZSBhbmQgZ3JhcGggYXBwZWFyIGluIHRoZSBkb2N1bWVudCwgZGVtb25zdHJhdGluZyB0aGF0IHdlIGNhbiBoYXZlIG91ciB0ZXh0LCBjb2RlIGFuZCBvdXRwdXRzIGFsbCBpbiB0aGUgc2FtZSBwbGFjZSAtIGFuZCBhbnkgb3V0cHV0cyB3aWxsIGF1dG9tYXRpY2FsbHkgdXBkYXRlIHdoZW4gd2UgY2hhbmdlIG91ciBjb2RlICYgcmUta25pdC4KCiMjIyBBc2lkZTogY2h1bmsgb3B0aW9ucwoKQ2h1bmsgb3B0aW9ucyBhcmUgb3B0aW9ucyBkZXNpZ25hdGVkIGluIHRoZSBjb2RlIGNodW5rIGhlYWRlciB0aGF0IGRldGVybWluZSB3aGF0IGFwcGVhcnMgb3IgZG9lcyBub3QgYXBwZWFyIGZvciBlYWNoIGNodW5rIHVwb24ga25pdHRpbmcgKGFuZCBhIGxvdCBtb3JlLCBidXQgd2UnbGwgc3RhcnQgdGhlcmUuLi4pLiBGb3IgbW9yZSBpbmZvcm1hdGlvbiBvbiBjb2RlIGNodW5rIG9wdGlvbnMgaW4gUiBNYXJrZG93biwgc2VlIENoYXB0ZXIgMTEgaW4gdGhlIFtSIE1hcmtkb3duIENvb2tib29rXShodHRwczovL2Jvb2tkb3duLm9yZy95aWh1aS9ybWFya2Rvd24tY29va2Jvb2svY2h1bmstb3B0aW9ucy5odG1sKS4KCkNodW5rIG9wdGlvbnMgY2FuIGJlIGFkZGVkIHRvIGluZGl2aWR1YWwgY29kZSBjaHVuayBoZWFkZXJzICh3aXRoaW4gdGhlIGB7cn1gIGF0b3AgdGhlIGNodW5rKSwgb3IgYXBwbGllZCBnbG9iYWxseSBieSBhZGRpbmcgb3B0aW9ucyB0byB0aGUgYHtyIHNldHVwfWAgY29kZSBjaHVuayBhdCB0aGUgdG9wIG9mIHRoZSBkb2N1bWVudC4gCgpIZXJlIGFyZSBzb21lIGNvbW1vbiBjaHVuayBvcHRpb25zOiAKCi0gYGVjaG8gPSBGQUxTRWA6IGRvIG5vdCBzaG93IHRoZSBjb2RlIGluIHRoZSBrbml0dGVkIGRvY3VtZW50Ci0gYGluY2x1ZGUgPSBGQUxTRWA6IGRvIG5vdCBpbmNsdWRlIGNvZGUgb3IgYW55IG91dHB1dHMgaW4ga25pdHRlZCBkb2N1bWVudAotIGBtZXNzYWdlID0gRkFMU0VgOiBzdXBwcmVzcyBtZXNzYWdlcyB3aGVuIGtuaXQKLSBgd2FybmluZyA9IEZBTFNFYDogc3VwcHJlc3Mgd2FybmluZ3Mgd2hlbiBrbml0Ci0gYGV2YWwgPSBGQUxTRWA6IGRvIG5vdCBldmFsdWF0ZSB0aGlzIGNvZGUKCiMjIFIgTWFya2Rvd24gY3VzdG9taXphdGlvbgoKRm9yIHRvZGF5LCB3ZSdsbCBtYWtlIGNoYW5nZXMgdG8gdGhlIFlBTUwgaW4gb3VyIC5SbWQgdG8gZmFuY3kgb3VyIG91dHB1dCBkb2N1bWVudC4gT3VyIGN1c3RvbWl6YXRpb24gd2lsbCBiZSByZWxhdGVkIHRvIG91ciBrbml0dGVkIEhUTUwgb3V0cHV0LCB1c2luZyB0aGUgZm9sbG93aW5nOiAKCi0gYHRvYzogdHJ1ZWA6IGFkZCBhIHRhYmxlIG9mIGNvbnRlbnRzIGJhc2VkIG9uIGhlYWRlciBoaWVyYXJjaGllcwotIGB0b2NfZmxvYXQ6IHRydWVgOiBtYWtlIGl0IGEgZmxvYXRpbmcgVE9DCi0gYG51bWJlcl9zZWN0aW9uczogdHJ1ZWA6IGFkZCBudW1iZXJlZCBzZWN0aW9ucyBiYXNlZCBvbiBoZWFkZXIgaGllcmFyY2h5Ci0gYHRoZW1lOiBfX19fX2A6IGFkZCBhIGJvb3RzdHJhcCB0aGVtZQotIGBjb2RlX2ZvbGRpbmc6IHNob3dgOiBjb2RlIGlzIGRlZmF1bHQgc2hvd24sIGJ1dCBjYW4gYmUgaGlkZGVuIGlmIHRoZSByZWFkZXIgY2xpY2tzIG9uIHRoZSAnSGlkZScgYnV0dG9uIGNyZWF0ZWQgKGFsdGVybmF0aXZlIGRlZmF1bHQgaXMgYGNvZGVfZm9sZGluZzogaGlkZWApCi0gYGNvZGVfZG93bmxvYWQ6IHRydWVgOiBhZGQgYSBidXR0b24gYXRvcCB0aGUga25pdHRlZCBkb2N1bWVudCB0byBkb3dubG9hZCB0aGUgLlJtZCEgCgpGb3IgZnJlZSBib290c3RyYXAgdGhlbWVzLCB2aXNpdDogaHR0cHM6Ly9ib290c3dhdGNoLmNvbS8zLwoKVG8gYWRkIG9wdGlvbnMgdG8gdGhlIFlBTUwsIGFkZCB0aGVtIGFzICJjaGlsZHJlbiIgaW4gdGhlIGBodG1sX2RvY3VtZW50YCBzdWJzZWN0aW9uLiBCZXdhcmUgb2Ygc3BhY2luZyBoZXJlOiBnZW5lcmFsbHkgYWRkIDIgc3BhY2VzIG9mIGluZGVudGF0aW9uIGZvciBlYWNoIG5ldyBzdWJsZXZlbCBpbiB0aGUgWUFNTCBoaWVyYXJjaHkuIAoKYGBgCi0tLQp0aXRsZTogIk15IHRpdGxlIgpzdWJ0aXRsZTogIkFuZCBzdWJ0aXRsZSIKYXV0aG9yOiAiQWxsaXNvbiBIb3JzdCIKb3V0cHV0OiAKICBodG1sX2RvY3VtZW50OgogICAgdGhlbWU6IGZsYXRseQogICAgdG9jOiB0cnVlCiAgICB0b2NfZmxvYXQ6IHRydWUKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgY29kZV9mb2xkaW5nOiBzaG93CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQpgYGAKCkxldCdzIHRyeSB0aGVtIG91dCBpbiBvdXIgUiBNYXJrZG93biBkb2N1bWVudCEgCgojIyBSIE1hcmtkb3duIGZvciByZXNlYXJjaAoKIyMjIElubGluZSBjb2RlCgpPZnRlbiwgd2Ugd2FudCB0byByZWZlcmVuY2UgY29kZSBvdXRjb21lcyAqd2l0aGluKiBvdXIgdGV4dC4gRm9yIGV4YW1wbGUsIHdlIG1heSB3YW50IHRvIHJlZmVyIHRvIGEgbWVhbiB2YWx1ZSwgbGlrZTogCgogICAgICAiVGhlIG1lYW4gbml0cmF0ZSBjb25jZW50cmF0aW9uIGluIExha2UgQSBpcyAxMS4yIG1nL0wuIiAKClRoZSBjb21tb24gb3B0aW9uIG9mIGNvcHlpbmcgYW5kIHBhc3RpbmcgdmFsdWVzLCBob3dldmVyLCBpcyAqKnRlZGlvdXMqKiBhbmQgKipkYW5nZXJvdXMqKi4gQSBiZXR0ZXIgd2F5IGlzIHRvIHJlZmVyIHRvIGNvZGUgb3V0cHV0cyBkaXJlY3RseSwgc28gdGhhdCB2YWx1ZXMgYXJlIHVwZGF0ZWQgYXV0b21hdGljYWxseSB3aGVuIGNvZGUgb3IgZGF0YSBjaGFuZ2UuIAoKSW4gUiBNYXJrZG93biwgd2UgYWRkIGNvZGUgaW5saW5lIHdpdGggYSBzaW5nbGUgYmFjay10aWNrIChgKSBvbiBlaXRoZXIgZW5kLCBhbmQgYSBsb3dlcmNhc2UgciBiZXR3ZWVuIHRvIHN0YXJ0IGl0LgoKVGhlbiB3ZSBjYW4gaW5zZXJ0IGNvZGUgd2l0aGluIG91ciB0ZXh0LCB3aGljaCB1bmRlciB0aGUgaG9vZCBjb250YWlucyB0aGlzIGlubGluZSBSIGNvZGUgdG8gY2FsY3VsYXRlIHRoZSBtZWFuIGdhcyBtaWxlYWdlIGZvciBhbGwgY2FycyBpbiB0aGUgYG10Y2Fyc2AgZGF0YXNldDoKCmBgYApyIHJvdW5kKG1lYW4obXRjYXJzJG1wZywgbmEucm0gPSBUUlVFKSwgMikKYGBgCgouLi50byBwcm9kdWNlIHRoaXMgb3V0cHV0OiAiVGhlIG1lYW4gZ2FzIG1pbGVhZ2Ugb2YgY2FycyBpbiB0aGUgZGF0YXNldCBpcyBgciByb3VuZChtZWFuKG10Y2FycyRtcGcsIG5hLnJtID0gVFJVRSksIDIpYCBtbS4iCgpUaGVuLCBpZiBvdXIgZGF0YSBvciBjb2RlIGV2ZXIgKmNoYW5nZSosIHNvIHdpbGwgb3VyIHRleHQgb3V0cHV0LiAgCgojIyMgU291cmNpbmcgc2NyaXB0cwoKU29tZXRpbWVzLCB5b3UnbGwgd2FudCB0byBpbmNvcnBvcmF0ZSBzb21ldGhpbmcgZnJvbSBhbiBSIHNjcmlwdCBpbnRvIGFuIFIgTWFya2Rvd24gZG9jdW1lbnQuIFRoaXMgbWlnaHQgYmUgdGhlIGNhc2UgaWYgeW91IGhhdmUgbG9uZyBjb2RlLCBhbmQgeW91IG1pZ2h0IHdhbnQgaXQgdG8gZXhpc3QgaW4gYSBzY3JpcHQgaW5zdGVhZCBvZiBjbHV0dGVyaW5nIHVwIHlvdXIgLlJtZC4gCgpZb3UgY2FuIHNjcmlwdCBleHRlcm5hbCBSIHNjcmlwdHMgbGlrZSB0aGlzIChmcm9tIENoIDE2LjEgb2YgdGhlIGZhbnRhc3RpYyBbUiBNYXJrZG93biBDb29rYm9va10oaHR0cHM6Ly9ib29rZG93bi5vcmcveWlodWkvcm1hcmtkb3duLWNvb2tib29rLykgYnkgWWlodWkgWGllLCBDaHJpc3RvcGhlIERlcnZpZXV4LCBhbmQgRW1pbHkgUmllZGVyZXIpOgoKYGBgCnNvdXJjZSgieW91ci1zY3JpcHQuUiIsIGxvY2FsID0ga25pdHI6OmtuaXRfZ2xvYmFsKCkpCmBgYAoKRm9yIGV4YW1wbGUsIEkgY2FuIHJlYWQgaW4gbXkgc2NyaXB0IGBteV9mdW5jdGlvbi5SYCwgd2hpY2ggY29udGFpbnMgYSBmdW5jdGlvbiBgZnJ1aXRgLCB3aGljaCBsb29rcyBsaWtlIHRoaXM6IAoKYGBgCmZydWl0IDwtIGZ1bmN0aW9uKGFwcGxlcywgYmFuYW5hcywgb3JhbmdlcywgYmVycmllcykgewogIGFwcGxlcyArIGJhbmFuYXMgKyBvcmFuZ2VzKmJlcnJpZXMKfQpgYGAKCldlIGNhbiBtYWtlIHRoYXQgYXZhaWxhYmxlIHRvIHVzIGluIG91ciBSIE1hcmtkb3duIGRvY3VtZW50IHdpdGg6CgpgYGB7ciwgZWNobyA9IFRSVUV9CnNvdXJjZSgibXlfZnVuY3Rpb24uUiIsIGxvY2FsID0ga25pdHI6OmtuaXRfZ2xvYmFsKCkpCmBgYAoKV2hpY2ggbWVhbnMgdGhlbiB0aGUgZnVuY3Rpb24gaXMgbm93IGF2YWlsYWJsZSBmb3IgbWUgdG8gdXNlLi4uCgpgYGB7ciwgZWNobyA9IFRSVUV9CmZydWl0KGFwcGxlcyA9IDIsIGJhbmFuYXMgPSAxLCBvcmFuZ2VzID0gNSwgYmVycmllcyA9IDEwKQpgYGAKCiMjIyBGcm9tIFIgTWFya2Rvd24gdG8gUiBzY3JpcHRzICYgYmFjayB3aXRoIGBzcGluKClgIGFuZCBgcHVybCgpYAoKU29tZXRpbWVzIGl0IGNhbiBiZSB1c2VmdWwgdG8gcHVsbCBvdXQgYWxsIGNvZGUgZnJvbSAqY29kZSBjaHVua3MqIGluIGFuIFIgTWFya2Rvd24gZG9jdW1lbnQgYW5kIGNvbmRlbnNlIHRoZW0gaW50byBhbiBSIHNjcmlwdC4gT3Igc29tZXRpbWVzIGl0J3MgdXNlZnVsIHRvIGRvIHRoZSBvcHBvc2l0ZSAtIHRha2UgaW5mb3JtYXRpb24gaW4gYW4gUiBzY3JpcHQgYW5kIGFkZCBpdCB0byBhbiBSIE1hcmtkb3duIGRvY3VtZW50LiBXZSBjYW4gZG8gdGhlc2Ugd2l0aCBga25pdHI6OnB1cmwoKWAgYW5kIGBrbml0cjo6c3BpbigpYCwgcmVzcGVjdGl2ZWx5LiAKCi0gVXNlIGBrbml0cjo6cHVybCgpYCB0byBhZ2dyZWdhdGUgY29kZSBmcm9tICoqY29kZSBjaHVua3MqKiAobm90IGlubGluZSBjb2RlKSBpbnRvIGFuIFIgc2NyaXB0Ci0gVXNlIGBrbml0cjo6c3BpbigpYCB0byBjb252ZXJ0IGFuIFIgc2NyaXB0IHRvIGFuIFIgTWFya2Rvd24gZG9jdW1lbnQKCgojIyMjIGBwdXJsKClgOiBSIE1hcmtkb3duID4gUiBzY3JpcHQKCkxldCdzIGBwdXJsKClgIHRoZSBkb2N1bWVudCB5b3UndmUgYmVlbiB3b3JraW5nIGluIHRvIGNyZWF0ZSBhbiBSIHNjcmlwdCEgCgotIEluIHRoZSBSIENvbnNvbGUsIHJ1biBga25pdHI6OnB1cmwoInlvdXJfZmlsZV9uYW1lLlJtZCIpYC4KCllvdSBzaG91bGQgc2VlIGEgLlIgc2NyaXB0IGNyZWF0ZWQgaW4gdGhlIHJvb3QsIHdpdGggdGhlIHNhbWUgbmFtZSBidXQgZW5kaW5nIGluIC5SIC0gY2hlY2sgaXQgb3V0ISAKCiMjIyMgYHNwaW4oKWA6IFIgc2NyaXB0ID4gTWFya2Rvd24KClNlZSBtb3JlIGluZm9ybWF0aW9uIG9uIHJlbmRlcmluZyByZXBvcnRzIGZyb20gUiBzY3JpcHRzIFtoZXJlXShodHRwczovL2Jvb2tkb3duLm9yZy95aWh1aS9ybWFya2Rvd24tY29va2Jvb2svc3Bpbi5odG1sKS4gTGV0J3MgY3JlYXRlIGEgbmV3IGxpdHRsZSBSIHNjcmlwdCwgYW5kIGFkZCBzb21lIGluZm9ybWF0aW9uIHRoYXQgd2lsbCBiZSByZW5kZXJlZCBhcyBNYXJrZG93biBzeW50YXguIAoKLSBDcmVhdGUgYSBuZXcgZW1wdHkgLlIgc2NyaXB0LCB0aGVuIGNvcHkgYW5kIHBhc3RlIHRoZSBmb2xsb3dpbmcgY29kZSAmIGFubm90YXRpb24gaW50byBpdDogCgpgYGB7ciwgZXZhbCA9IEZBTFNFLCBlY2hvID0gVFJVRX0KCiMnIExldCdzIG1ha2UgYSBtYXJrZG93biBkb2N1bWVudCAoLm1kKSB3aXRoIGBrbml0cjo6c3BpbigpYC4KIycKIycgVGhpcyBpcyBub3JtYWwgdGV4dAojJwojJyAjIyBUaGlzIGlzIGEgbGV2ZWwgMiBoZWFkZXIKIycKIycgKlRoaXMgdGV4dCBpcyBpdGFsaWNpemVkKgoKIysgCmxpYnJhcnkodGlkeXZlcnNlKQoKIysKIyBBbiBhY3R1YWwgY29tbWVudCBpbiB0aGUgY29kZSBjaHVuawpnZ3Bsb3QoZGF0YSA9IHN0YXJ3YXJzLCBhZXMoeCA9IGhlaWdodCwgeSA9IG1hc3MpKSArCiAgZ2VvbV9wb2ludCgpCgojJyBUaGVuIHNvbWUgbm9ybWFsIHRleHQgb3V0c2lkZQoKYGBgCgotIFNhdmUgdGhhdCAuUiBzY3JpcHQgKGUuZy4gYXMgYHN0YXJ3YXJzX2dyYXBoLlJgKS4gCgotIEluIHRoZSBDb25zb2xlLCBydW46IAoKYGBgCmtuaXRyOjpzcGluKCJmaWxlX25hbWUuUiIpCmBgYAoKTm90aWNlIHRoYXQgdGhlIC5tZCBhbmQgdGhlIGtuaXR0ZWQgSFRNTCBhcmUgc2F2ZWQhIFRoYXQncyBob3cgd2UgY2FuIHJlbmRlciBhIHJlcG9ydCBzdHJhaWdodCBmcm9tIGFuIFIgc2NyaXB0LiAKCgojIFJlc291cmNlcwoKLSAgIFtUaGUgUiBNYXJrZG93biBDb29rYm9va10oaHR0cHM6Ly9ib29rZG93bi5vcmcveWlodWkvcm1hcmtkb3duLWNvb2tib29rLykgYnkgWWlodWkgWGllLCBDaHJpc3RvcGhlIERlcnZpZXV4LCBFbWlseSBSaWVkZXJlcgotICAgW1IgTWFya2Rvd246IHRoZSBEZWZpbml0aXZlIEd1aWRlXShodHRwczovL2Jvb2tkb3duLm9yZy95aWh1aS9ybWFya2Rvd24vKSBieSBZaWh1aSBYaWUsIEouIEouIEFsbGFpcmUsIEdhcnJldHQgR3JvbGVtdW5kCi0gICBbUlN0dWRpbydzIFIgTWFya2Rvd24gc2l0ZV0oaHR0cHM6Ly9ybWFya2Rvd24ucnN0dWRpby5jb20vKQotICAgW1IgTWFya2Rvd24gUmVmZXJlbmNlIEd1aWRlXShodHRwczovL3d3dy5yc3R1ZGlvLmNvbS93cC1jb250ZW50L3VwbG9hZHMvMjAxNS8wMy9ybWFya2Rvd24tcmVmZXJlbmNlLnBkZikKLSAgIFtSU3R1ZGlvJ3MgUiBNYXJrZG93biBsZXNzb24gc2VyaWVzXShodHRwczovL3JtYXJrZG93bi5yc3R1ZGlvLmNvbS9sZXNzb24tMS5odG1sKQotICAgQWxpc29uIEhpbGwncyB3b3Jrc2hvcCBvbiBbQWR2YW5jZWQgUiBNYXJrZG93bl0oaHR0cHM6Ly9hbGlzb24ucmJpbmQuaW8vcHJvamVjdC9hZHZhbmNlZC1yLW1hcmtkb3duLykK