Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Data Visualization for Corpus Linguistics:Shiny Framework

Olga Scrivner

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

What You Will Learn

1. Introduction to Visual Analytics

2. Reactive Web Framework

3. Shiny Application

4. Practice

https://languagevariationsuite.wordpress.com/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Data Analytics

“Analytics is the critical technologyneeded to bring value out of data”

(Anonymous)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics

“The science of analytical reasoningfacilitated by visual interactive interfaces”

(Thomas and Cook, 2005)

“Visual analytics integrates new computational andtheory-based tools with innovative interactive techniquesand visual representations to enable human-information

discourse” (Thomas and Cook, 2005)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics and Linguistics

“ Language is a system of relatively arbitrary symbols”(Baker-Shenk and Cokely, 1980)

http://www.tableaufit.com/tableau-conference-linguistics-data-visualization/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics (and Corpus Linguistics)

1. Exploration (Queries)

2. Comprehension (Analysis)

3. Communication (Presentation)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Corpus Linguistics Visualization

http://scimaps.org/home.html

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visualization Tools

Challenges:

I Many tools are built for non-linguistic research

I Limited preprocessing

I Visualization may not fit linguistic data

Solution: Reactive Framework

I Interactive and web-based

I Research-driven customization

I Build your own tool!

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visualization Tools

Challenges:

I Many tools are built for non-linguistic research

I Limited preprocessing

I Visualization may not fit linguistic data

Solution: Reactive Framework

I Interactive and web-based

I Research-driven customization

I Build your own tool!

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Web Framework

“Reactive Systems are highly responsive, giving userseffective interactive feedback”

http://www.reactivemanifesto.org/

http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Architecture

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Framework and Data Science

“The impact of data scientists’ work depends on how wellothers can understand their insights to take further actions”

Benefit 1: Interactive display and manipulation of data

Benefit 2: No installation required

Benefit 3: Easy to develop and share with clients andproject teams

Benefit 4: Open source library

http://datascience.ibm.com/blog/shiny-a-data-scientist-best-friend/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Application

1. Shiny is an R package for building interactive webapplications

2. Open-Sourced by RStudio 11/2012 on CRAN

3. Uses web sockets (new HTTP):

I Interactive communication sessions between the user’sbrowser and a server without having to poll the serverfor a reply

4. Entirely extensible - custom input/output

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Library

http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Gallery - Get Inspired

https://www.rstudio.com/products/shiny/shiny-user-showcase/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Interactive Text Mining Suite (Scrivner et al.2016)

1. Web application for text processing and mining

2. Interactive natural language processing techniques

I Wordstops, stemming, text-preprocessing

3. High customization

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Stopwords

Stopwords (e.g. the, and):

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Manual Removal of Stopwords

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Stemming

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Word Cloud Representation

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Customization

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Format Flexibility

Example: Using Google Books API

Current limitation is 40 books

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Workshop Materials

1. Rstudio

2. R

3. Shiny library

4. Materials:

https:

//languagevariationsuite.wordpress.com/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R software

R is a free software for data analysis, text mining andvisualization.

To install R on Window:

1. Download the binary file for Rhttps://cran.r-project.org/bin/windows/base/

R-3.3.1-win.exe

2. Open the downloaded .exe file and Install R

To install R on Mac:

1. Download the appropriate version of .pkg filehttps://cran.r-project.org/bin/macosx/

2. Open the downloaded .pkg file and Install R

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R Studio

RStudio is a free user interface for R.

1. Install the appropriate RStudio version https:

//www.rstudio.com/products/rstudio/download/

2. Run it to install R-studio

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R Studio Structure

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Installing Packages

In your bottom left window - go to Packages

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Selecting Packages - shiny

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Creating Scripts

Create R File: File → New File → R Script

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Execution - RUN

To execute - click run

In the script (top left window) type:

library(shiny)

Click Run (your cursor can be at the beginning or at the endof the line)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Demo

runExample(“01 hello”)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Demo

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Demo

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

UI.R

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Input/Output

I Input - things user can toggle

I Output - R objects that user can see, often depend oninputs

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Closing App

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Step 1 - Create New Web Shiny App

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Step 2 - Run App

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

UI

shinyUI( fluidPage(

titlePanel(“Old Faithful Geyser Data”),

sidebarLayout(

sidebarPanel(

sliderInput(“bins”,

“Number of bins:”,

min = 1,

max = 50,

value = 30)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

UI

shinyUI( fluidPage(

titlePanel(“Old Faithful Geyser Data”),

sidebarLayout(

sidebarPanel(

sliderInput(“bins”,

“Number of bins:”,

min = 1,

max = 50,

value = 30)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Modifying UI - Practice

I Change slider’s label:Number of bins → Choose a number

I Change slider’s values: max and value

I Save

I RunApp

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

HTML

The UI script simply creates HTML

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

HTML

HTML tags:

http:

//shiny.rstudio.com/articles/tag-glossary.html

I h1() = header1

I br() = line break

I p() = paragraph

I hr() = line

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Adding HTML Tags to UI.R - Practice

shinyUI( fluidPage(

titlePanel(“My Title”),

h3(“My subtitle”),

p(“This is my first app!”),

br(),

hr(),

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Layout

http:

//shiny.rstudio.com/articles/layout-guide.html

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

mainPanel

mainPanel( plotOutput(”distPlot”))

Let’s add 3 tab panels: Plot, Summary, Table

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

tabsetPanel

mainPanel(

tabsetPanel(

tabPanel(“Plot”, plotOutput(“distPlot”)),

tabPanel(“Summary”),

tabPanel(“Table”)

)

)

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Input Data

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Input Data

1. Blog https:

//languagevariationsuite.wordpress.com/

2. Download csv file - movie metadata.csv

3. Place this file into the directory myshiny

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Let’s look at the fileInput() function from ShinyReference page

I Type in your browser: fileInput Shiny

I or go to the Shiny Reference page -https://shiny.rstudio.com/reference/shiny/

latest/fileInput.html

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Scroll down to Examples

I We will add fileInput function inside sidebarPanel

I Copy fileInput function

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Paste it after slider

NB: commas are important!

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Function in Sever.R

Reactive - it changes every time the user uploads new data

I Create a function that reads csv file

I myfile() is a function that will read and return csv files

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Render - Reactive Data

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Function summary in Server.R

output$summary <- renderPrint({

summary(myfile())

})

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Output Function table in Server.R

output$table <- renderDataTable({

myfile()

})

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Linking summary and table with UI.R

tabsetPanel(

tabPanel("Plot", plotOutput("distPlot")),

tabPanel("Summary", verbatimTextOutput("summary")),

tabPanel("Table", dataTableOutput("table"))

)

RunApp

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Linking summary and table with UI.R

tabsetPanel(

tabPanel("Plot", plotOutput("distPlot")),

tabPanel("Summary", verbatimTextOutput("summary")),

tabPanel("Table", dataTableOutput("table"))

)

RunApp

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Testing CSV Upload

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

One more IF-Statement - Server.R

We want to do a histogram for csv file:

if (condition) {do..} else {do...}

output$distPlot <- renderPlot({

if (is.null(input$file1)) {

....

hist(x, breaks = bins, col = ’darkgray’, border =

’white’)

}

else{

x <- myfile()$budget

bins <- seq(min(x), max(x), length.out = input$bins

+ 1)

hist(x, breaks = bins, col=’red’,

main = ’My New Histogram’)

}

})

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

One more IF-Statement - Server.R

We want to do a histogram for csv file:

if (condition) {do..} else {do...}output$distPlot <- renderPlot({

if (is.null(input$file1)) {

....

hist(x, breaks = bins, col = ’darkgray’, border =

’white’)

}

else{

x <- myfile()$budget

bins <- seq(min(x), max(x), length.out = input$bins

+ 1)

hist(x, breaks = bins, col=’red’,

main = ’My New Histogram’)

}

})

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

One more IF-Statement - Server.R

We want to do a histogram for csv file:

if (condition) {do..} else {do...}output$distPlot <- renderPlot({

if (is.null(input$file1)) {

....

hist(x, breaks = bins, col = ’darkgray’, border =

’white’)

}

else{

x <- myfile()$budget

bins <- seq(min(x), max(x), length.out = input$bins

+ 1)

hist(x, breaks = bins, col=’red’,

main = ’My New Histogram’)

}

})

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

RunApp

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Server - Recap

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deployment Options

1. Share server.r and ui.r

2. Host on shinyapps.io

3. Host on Shiny server

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deploy with shinyapps.io

I www.shinyapps.io

I sign up for an account.

I Publish Application button in RStudio and followinstructions

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deploy with shinyapps.io

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

My shinyapps.io

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

My shinyapps.io

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Future of Visualization Tools: CNS

Cyberinfrastructure for Network Science Center at IndianaUniversity - http://cns.iu.edu/

Our current project on Shiny Framework (team OlgaScrivner and Jivitesh Poojary)

I Build Shiny templates for Data Visualization: stage I

https:

//github.com/Jivitesh-Poojary/CNS-Shiny-Apps

I Create user-friendly customizable Shiny dashboards:stage II

I Develop preprocessing plugins for Shiny templates:stage III

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Useful resources

1. Shiny official tutorial -http://shiny.rstudio.com/tutorial

2. Cheat sheet - http://shiny.rstudio.com/images/shiny-cheatsheet.pdf

3. Publish your app free - http://www.shinyapps.io

4. Examples -http://www.showmeshiny.com/

5. Tutorial by Dean Attali - http://deanattali.com/blog/building-shiny-apps-tutorial/

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Thank you!

My email: obscrivn@indiana.edu

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

References

Borner, Katy. 2015. Atlas of Knowledge: Anyone Can Map. TheMIT PressCollins, Christopher. 2012. Bridging the Linguistic VisualizationDivide. LingVis/UNCLH WorkshopGrolemund, Garrett. 2017. How to start with Shiny, Part 1.Workshop

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Credits

http:

//deanattali.com/blog/building-shiny-apps-tutorial/

http://scimaps.org/mapdetail/stream_of_scientific_128

https://github.com/IBMDataScience/dsx-shiny-apps

http://www.slideshare.net/SarahAerni/

data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-science-from-pivotal-open-source-hub-meetup

http://www.unixstickers.com/image/data/stickers/

react/badge/React-JS.sh.png

https://github.com/rstudio/shiny/issues/250

http://www.slideshare.net/ilio-catallo/

spring-mvc-the-basics