Twitter Data - University of California,...
Transcript of Twitter Data - University of California,...
![Page 1: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/1.jpg)
Twitter DataGilad Mishne - @gilad
Twitter SearchInfo 290 - Analyzing Big Data With Twitter
UC Berkeley Information SchoolAugust 2012
![Page 2: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/2.jpg)
About Twitter
2
• The fastest, simplest way to communicate
• More than 140M active users• Majority (also) mobile; 60% out of U.S.
• More than 400M twitter.com visitors• More than 400M tweets/day (peak: 25K/sec)
• 1,000 employees (majority in San Francisco)• 50% engineers
![Page 3: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/3.jpg)
Twitter data: text
3
![Page 4: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/4.jpg)
Twitter data: social graph
4Credit: @isaach
![Page 5: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/5.jpg)
Twitter data: time series
5
![Page 6: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/6.jpg)
Twitter data: interest graph
6Credit: @psychemedia
![Page 7: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/7.jpg)
Combined: the pulse of the world
7
![Page 8: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/8.jpg)
What we do with large data
8
![Page 9: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/9.jpg)
Scale
9
![Page 10: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/10.jpg)
Search
10
![Page 11: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/11.jpg)
Recommendations
11
![Page 12: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/12.jpg)
Ads
12
![Page 13: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/13.jpg)
Anti-Spam
13
![Page 14: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/14.jpg)
The Speakers
(Plus, some Twitter features)
14
![Page 15: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/15.jpg)
Twitter Overview
15Embedded links
Expanded Tweets
![Page 16: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/16.jpg)
Hadoop/Pig
16
Replies/conversations
![Page 17: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/17.jpg)
Trends/Streaming
17
Protected accounts
Support for 30+ languages
![Page 18: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/18.jpg)
Search
18Retweets, Favorites
![Page 19: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/19.jpg)
Graphs, Recommendations, Relevance
19Retweets (in the timeline)
![Page 20: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/20.jpg)
Security, anomaly detection
20
Photos
![Page 21: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/21.jpg)
Scalding
21
Geotagging,hashtags
![Page 22: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/22.jpg)
Goals
• Work with real data, on real problems• Learn how it is to work in a place like Twitter• Build something useful• Have a good time!
22
![Page 23: Twitter Data - University of California, Berkeleyblogs.ischool.berkeley.edu/.../2012/08/UCB_TwitterIntro2_Aug23_201… · Twitter Data Gilad Mishne - @gilad Twitter Search Info 290](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecdd0025d3564758e1d0558/html5/thumbnails/23.jpg)
Questions?
23
Follow me: @gilad