The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7...
Transcript of The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7...
![Page 1: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/1.jpg)
The Big Deal about Big Data
Agile principles to drive Adoption of Advanced Analytics
Oliver Ratzesberger
VP Information Analytics & Innovation
@ratzesberger
![Page 2: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/2.jpg)
Oliver Ratzesberger – VP Analytics & Innovation
• 20 years in Large scale Data Warehouse
• 7 years at eBay – Analytics Platform
Teradata
Hadoop
200PB of infrastructure – largest commercial database sized for >50PB of raw data
• At Sears Holdings/MetaScale since October 2011
Transforming a legacy icon into an Analytical Competitor.
![Page 3: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/3.jpg)
What is BigData?
PetaBytes of information
Hundreds of Millions of Customers
Complex/Semi/Unstructured Data
NoSQL/MapReduce/MPP/Hadoop
Data Science & Data Visualization
Advanced Algorithms & Predictive Technologies
Natural Language & Image Processing
Sensor Data
Sentiment Analysis
![Page 4: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/4.jpg)
BigData at Sears Holding
3.5PB Teradata 2.5PB Hadoop
>5 Million requests per day
Consolidating all Data Marts into a Single Version of the Truth
![Page 5: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/5.jpg)
Simplicity
Occam’s Razor:
“simpler explanations are …
generally better than more
complex ones”
The simple solution is
easy to explain, implement,
and maintain
![Page 6: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/6.jpg)
Design for the Unknown
“Of design for analytics platforms - Perfect is Wasteful”
Friction to change & code weight are the antithesis of agility
![Page 7: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/7.jpg)
Time to Market ( is everything …)
Are your Analytical needs getting stuck in traffic?
![Page 8: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/8.jpg)
The Foundation
Technology Platform
Storage and processing platforms, Teradata & Hadoop, and data interconnect services
Analytics as a Service (A3S)
Reusable, powerful, and integrated analytics services that automates the actions in an analytics environment.
This enables rapid deployment of a high-quality feature rich collaborative analytics environment that will
empower users to be radically more self sufficient, be more productive, and achieve better results.
Insights Platform
Advanced analytics products with out of the box segmentation, trending, alerting, experimentation, etc.
capabilities supporting extremely large data sets
Ser
vice
s, T
rain
ing
, Su
pp
ort
Dev
elo
per
Pla
tfo
rm
![Page 9: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/9.jpg)
Examples Usecases
Analytics as a Service (A3S)
Insights Platform
nSegment nTrend nAlert nExperiment
Operational Data
Engine
Insights Hub
Monitoring
Virtual Data
Containers
Data Movement
Service
Search
Activity Based
Chargeback
Data Profiling
Services
Security
Best practices
compliance
Database Marketing Loyalty Programs Gamification Store Operations
![Page 10: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/10.jpg)
Example Data Engine for Segmentation
![Page 11: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/11.jpg)
The importance of KPIs
![Page 12: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/12.jpg)
Scrum – Adopting an Agile Methodology
![Page 13: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/13.jpg)
Amount of Change
![Page 14: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/14.jpg)
Competing Priorities in Technology
![Page 15: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/15.jpg)
What is DevOps?
• Blend of
Agile Development AND
Agile Operations
• Software development methods that stress
communication and collaboration
• Developing the 1st line of code with
Operations in mind
![Page 16: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/16.jpg)
Developer Platform
![Page 17: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/17.jpg)
A BigData Organizational Example
Analytics & Innovation
Architecture Operations Business
Applications Product
Management Product
Development Data Science
Labs Offshore COE
CTO Analytics & Innovation
![Page 18: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/18.jpg)
Data Science Labs
Dedicated Data Scientist Labs Organization
Center of Excellence for
• Advanced Algorithms
• Predictive Technologies
• Visualization Technologies
Assigned to the top priority initiatives of the enterprise
![Page 19: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/19.jpg)
Separating GOOD from BAD
SEARS HOLDING CORPORATION COPYRIGHT 2012 19
![Page 20: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/20.jpg)
Consistent Simplicity
SEARS HOLDING CORPORATION COPYRIGHT 2012 20
![Page 21: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/21.jpg)
Data Science - When the AVERAGE is useless
SEARS HOLDING CORPORATION COPYRIGHT 2012 21
![Page 22: The Big Deal about Big Data - CIO Summits Big... · •20 years in Large scale Data Warehouse •7 years at eBay – Analytics Platform Teradata Hadoop 200PB of infrastructure –](https://reader033.fdocuments.in/reader033/viewer/2022042205/5ea6be15c0297a397846df2e/html5/thumbnails/22.jpg)
Questions?
Oliver Ratzesberger
VP Information Analytics & Innovation
@ratzesberger