Big Data in Real Time
-
Upload
ron-zavner -
Category
Technology
-
view
193 -
download
4
description
Transcript of Big Data in Real Time
![Page 1: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/1.jpg)
Big Data in REAL TIMERon Zavner
![Page 2: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/2.jpg)
We’re Living in a Real Time World…Homeland Security
Real Time Search
Social
eCommerce
User Tracking & Engagement
Financial Services
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved2
![Page 3: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/3.jpg)
The Flavors of Big Data Analytics
Counting Correlating Research
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved3
![Page 4: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/4.jpg)
It takes a week for users to
send 1 billion tweets
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved4
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
![Page 5: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/5.jpg)
On average,
140 milliontweets get sent every day
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved5
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
![Page 6: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/6.jpg)
The highest throughput to date is
6,939 tweets/sec.® Copyright 2011 Gigaspaces Ltd. All Rights Reserved6
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
![Page 7: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/7.jpg)
460,000 new accounts are created daily
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved7
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
![Page 8: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/8.jpg)
Challenge – Word Count
Word:Count
Tweets
CountCount??® Copyright 2011 Gigaspaces Ltd. All Rights Reserved8
![Page 9: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/9.jpg)
Thousands of tweets per second to process Aggregate counters for each word Latency – less than a second System needs to linearly scale System needs to be fault tolerant Querying & Persisting Data Managing the system
Analyze the Problem
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved9
![Page 10: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/10.jpg)
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved10
Tier Based Architecture?
![Page 11: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/11.jpg)
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved11
Data Grid
![Page 13: Big Data in Real Time](https://reader033.fdocuments.in/reader033/viewer/2022052900/5561e8c8d8b42ab1068b50f2/html5/thumbnails/13.jpg)
1. Just2. Found3. Love
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved13
The 3 Most Popular Words on Twitter?
- August 2012