Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of...
Transcript of Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of...
![Page 1: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/1.jpg)
Predicting Events based on Social
Media Exposure: from the Arab
Spring to Disease Outbreaks
Rishab Aiyer Ghosh
Co-Founder & Chief Scientist
![Page 2: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/2.jpg)
Headlines, early 2011
![Page 3: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/3.jpg)
Google, Twitter search :-( { not realtime, not fresh - or not ranked, not relevant }
![Page 4: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/4.jpg)
Topsy search :-) { realtime, fresh and relevant }
search result from 2011-10-11 at 15.40 EDT
![Page 5: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/5.jpg)
{ validation : real people! }
search result from February 2011
![Page 6: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/6.jpg)
underlying shift
in information dissemination
{ validation : shift }
![Page 7: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/7.jpg)
Most intelligence historically was from open sources
For financial, business, brand, publishing, government
use...
Previously open sources, public sources, were
published sources: news in publications
{ previously }
![Page 8: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/8.jpg)
Public conversations were private information
Human intelligence - inside information - insider
trading
{ previously }
![Page 9: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/9.jpg)
Public conversations are public information
Publicly accessible
{ / shift }
![Page 10: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/10.jpg)
we can all be
{ public conversations are publicly accessible }
in tahrir square
![Page 11: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/11.jpg)
we can all be
{ public conversations are publicly accessible }
in #tahrir square
![Page 12: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/12.jpg)
{ #tahrir, #jan25, #libya, #bahrain ... } 2011-12-3 at 23.26 PST
![Page 13: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/13.jpg)
{ public conversations are publicly accessible }
reporters
on the ground
we can all have
![Page 14: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/14.jpg)
{ public conversations are publicly accessible }
signal / noise
reporters
on the ground
we can all have
![Page 15: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/15.jpg)
{ public conversations are publicly accessible }
signal / noise
reporters
on the ground
we can all have
Topsy’s platform provides
editing, curation...
![Page 16: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/16.jpg)
{ historical context for back-testing... }
![Page 17: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/17.jpg)
What evidence is there of failed planned protests?
{ ... helps predict future events with quantified confidence }
![Page 18: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/18.jpg)
18
Compared to Google Flu Trends, etc…
What people
search for...
(“Pinterest”)
Mirrors their
public
conversation
s
(source: http://adaptmedia.clarify-
it.com/d/ngpj7j )
![Page 19: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/19.jpg)
Unlike search records, which are based
on private IP addresses, private search
query logs…
There are many more ways you can
analyze public conversations
{ public conversations are publicly accessible }
![Page 20: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/20.jpg)
20
Track activity:
Mentions
Exposure
Discover:
Related new
terms
Monitor
conversations:
Posts
Articles shared
Who is saying
what?
See shared media
ranked by
influence
![Page 21: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/21.jpg)
21
![Page 22: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/22.jpg)
22
Map activity for
terms:
Raw mentions
Share of all
social
mentions
See conversations
from particular
locations
![Page 23: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/23.jpg)
{ tech built in-house, running on 2000 servers }
Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture capable of indexing 1M documents per second
Live Ranking - new citations impact search results/ranking in realtime
Native multi-language support
![Page 24: Predicting Events based on Social Media Exposure: from the ...€¦ · Extensible to all types of social content (Twitter, Facebook, blogs, etc.) Fast Indexing - pipeline architecture](https://reader033.fdocuments.in/reader033/viewer/2022042302/5ecd8d4dff7ebd45234ce7b7/html5/thumbnails/24.jpg)
human
intelligence
social media as
Rishab Aiyer Ghosh Co-Founder & Chief Scientist: [email protected]