Big data meetup_10_9_2013
-
Upload
tanya-cashorali -
Category
Technology
-
view
102 -
download
0
description
Transcript of Big data meetup_10_9_2013
![Page 1: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/1.jpg)
Click to edit Master title style
This presentation, and the information contained therein, is the proprietary and confidential property of ComLinkData, powered by
Comniscient Technologies LLC, and may only be viewed by the individual to which it was initially provided by ComLinkData. By viewing this
presentation, you agree that you will not copy, reproduce or distribute the presentation in whole or in part, without the prior written consent
of ComLinkData. You agree not to forward this presentation (by email or otherwise) to anyone inside or outside your organization, and you will
not save this presentation on a network to which anyone other than you has access (except for information technology professionals who are
subject to confidentiality provisions, in the ordinary course of their responsibilities). You agree to maintain the confidentiality of all of the
information and data in the presentation that is not already in the public domain. You further agree to use the presentation only for the
purpose of evaluating the services offered by ComLinkData, and will not use any of the confidential information in the presentation for any
other purpose, including for the purpose of analyzing aspects of the telecommunications market.
Turning Big Data into Business Applications: An introduction to R Shiny@tanyacash21, @JoshuaWalker_
10/9/2013
![Page 2: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/2.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.2
The Opportunity: Real-time Telecom Insights
50,000 transactions a day18MM+ a year
2.2 Terabytes Disk Space3 years of data and growing
Demographics
Advertising spend
Events
Social Media
Distribution
Device
Real-time Telecom Switching Data
![Page 3: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/3.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.3
Data Flow and Architecture
![Page 4: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/4.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.4
The Front End Framework
Web Application (Front End)
jQuery
AngularJS
d3.js
Twitter Bootstrap
Font Awesome
Javascript, HTML5, CSS3 Java Servlets API(Service Layer)
1. Parse queries2. Fetch data
3. Construct JSON
SQL Server / Amazon Redshift
![Page 5: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/5.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.5
R is Gaining Major Visibility
http://r4stats.com/2013/03/19/r-2012-growth-exceeds-sas-all-time-total/
![Page 6: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/6.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.6
Testable, Reproducible, Scalable
• Testable – RUnit package
• Reproducible – Commented scripts / multiple ways to ingest data (flat files, Excel files, ODBC/JDBC, Rcurl)
• Scalable – doSNOW, doMPI, doMC, foreach packages
![Page 7: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/7.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.7
Developers and Data Geeks Tend to Have Mutually Exclusive Sets of Skills
Web Developers / Designers / Engineers
Data Scientists / Analysts / Researchers
!=
![Page 8: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/8.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.8
Solution: Shiny!
+ =
![Page 9: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/9.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.9
Shiny Features
• Easy to use – readily available widgets, no HTML/Javascript or jQuery knowledge required
• Fast bidirectional communication using websockets
• Uses Twitter Bootstrap
• Reactive Programming Model
![Page 10: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/10.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.10
Reactivity
Inputs on a web page Plots / Tables
![Page 11: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/11.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.11
rCharts
Generates d3.js code
![Page 12: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/12.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.12
googleVis
![Page 13: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/13.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.13
Define HTML UI Instead of ui.R for Even More Flexibility
index.html
ui.R:<application-dir>|-- www |-- index.html|-- server.R
![Page 14: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/14.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.14
Interactive Shiny Workshop
Start up R!
![Page 15: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/15.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.15
Demo of Full Shiny Applications
![Page 16: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/16.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.16
New Method of Collaboration and Product Development in Data-Centric Companies
![Page 17: Big data meetup_10_9_2013](https://reader036.fdocuments.in/reader036/viewer/2022070304/54c64e6b4a79591a6d8b45cb/html5/thumbnails/17.jpg)
Click to edit Master title style
Your viewing this presentation is subject the confidentiality and other terms provided on the first page of the presentation.
© 2013 ComLinkData, powered by Comniscient
Technologies LLC. All rights reserved.17
References
1. http://rcharts.io/2. http://rcharts.io/gallery/3. http://cran.r-project.org/web/packages/googleVis/index.html4. http://cran.r-project.org/web/packages/websockets/index.html5. http://rstudio.github.io/shiny/tutorial/#welcome6. https://groups.google.com/forum/#!forum/shiny-discuss7. http://r4stats.com/2013/03/19/r-2012-growth-exceeds-sas-all-time-total/8. http://www.businessinsider.com/excel-partly-to-blame-for-trading-loss-2013-2