Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd...
-
Upload
yahoo-developer-network -
Category
Documents
-
view
3.243 -
download
3
Transcript of Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd...
![Page 1: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/1.jpg)
Todd Papaioannou VP, Cloud Architecture
By SearchNetMedia
HADOOP & THE FUTURE OF CLOUD
COMPUTING
![Page 2: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/2.jpg)
HAPPENING WHAT’S
More publicly available human-generated content
More interactions being tracked (e.g. clickstream data)
More business processes are being digitized
More history being kept
= The Data Exhaust!
Flickr : sub_lime79BigData is here!
![Page 3: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/3.jpg)
THE NOISECUTTING THROUGH
Flickr : Lomo-Cam
LocationSocial
Relationships
ScienceUnderstandingUser Interests
access audience blogs communication
computer internet mass media
people networking technology
![Page 4: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/4.jpg)
INTO INSIGHTSTURNING DATA
machine learningtime series
content clustering
factorization models
logic regression
Flickr : NASA Goddard Photo and Video
algorithmsuser interest prediction
Ad inventory modeling
![Page 5: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/5.jpg)
RELEVANTMAKING IT
Flickr : ogimogi
![Page 6: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/6.jpg)
LIGHTNING-FASTHADOOP:
science + big data + insight = personal relevance = VALUE
TECHNOLOGY
Flickr : DDFic
![Page 7: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/7.jpg)
EVERY CLICKBEHIND
![Page 8: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/8.jpg)
HADOOP
Flickr : Got Sarah
![Page 9: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/9.jpg)
THE PLATFORM EFFECTTHE HADOOP ECOSYSTEM
and other Early AdoptersScale and productize Hadoop
9
Apache Hadoop
Orgs with Internet Scale ProblemsAdd tools / frameworks, enhance Hadoop
Mainstream / Enterprise adoptionFund further development, enhancements
EnhanceHadoopEcosystem
Service Providers Grow ecosystem - Training, support, enhancements
Virtuous Circle!• Investment -> Adoption• Adoption -> Investment
![Page 10: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/10.jpg)
HADOOP IS GOINGMAINSTREAM
2007 2008 2009
10
2010
The Datagraph Blog
![Page 11: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/11.jpg)
11
HADOOP ATYAHOO!
“Where Science meets Data”
HADOOP CLUSTERSTens of thousands of servers
DATA PIPELINES
CONTENT
DIMENSIONAL DATA
PRODUCTS
APPLIED SCIENCE
Data Analytics Content OptimizationContent Enrichment Yahoo! Mail Anti-Spam Advertising ProductsAd Optimization Ad SelectionBig Data Processing & ETL
User Interest Prediction Ad inventory prediction Machine learning - search ranking Machine learning - ad targetingMachine learning - spam filtering
![Page 12: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/12.jpg)
2006 2007 2008 2009 201012
FROM PROJECT TOCORE PLATFORM
Today
38K Servers
170 PB Storage
1M+ Monthly Jobs
Tho
usan
ds o
f Ser
vers
Pet
abyt
es
90
80
70
60
50
40
30
20
10
0
250
200
150
100
50
0
Research
Science Impact
Daily Production
“Behind every click”
![Page 13: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/13.jpg)
13
YAHOO!’S VISIONOPEN SOURCE CLOUD
Open Source Benefits
» Avoid technological dead ends
» Leverage community contributions
» Workforce already trained
Ongoing contributions Yahoo!’s adoption of open source
Future contributions
Cloud serving
Storage
![Page 14: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/14.jpg)
FUTURE HOLD?WHAT DOES THE
By Elsie
![Page 15: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/15.jpg)
MORE BIG
By BionicTeaching
![Page 16: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/16.jpg)
DATA IN THECLOUD
By Fadilfb
![Page 17: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/17.jpg)
PRIVATE CLOUDS
By Zachstern
![Page 18: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/18.jpg)
HYBRID CLOUDS
By Calop
![Page 19: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/19.jpg)
AUTOMATION
![Page 20: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/20.jpg)
CLOUD FABRICS
![Page 21: Apache Hadoop India Summit 2011 Keynote talk "Hadoop & the Future of Cloud Computing" by Todd Papaioannou](https://reader035.fdocuments.in/reader035/viewer/2022062319/5549e4d2b4c9050d488b4a4a/html5/thumbnails/21.jpg)
QUESTIONS?