Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... ·...
Transcript of Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... ·...
![Page 1: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/1.jpg)
Big Data Platform Overview
Alex Hay ([email protected]), Big Data CTPMeridee Lowry ([email protected]), Big Data CTP
April 30th, 2014
![Page 2: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/2.jpg)
Big Data is a Concept
Big Data
2
![Page 3: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/3.jpg)
IBM Big Data and Analytics Offerings
Watson Foundations
Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM
Exploration, landing and
archive
Trusted data
Reporting & interactive analysis
Deep analytics & modeling
Real-time processing & analyticsSTREAMS, DATA REPLICATION
Operational systems
BIGINSIGHTS
PUREDATAHADOOP
DB2, INFORMIX
PUREDATA TRANSACTIONS
PUREDATA ANALYTICS
DB2 BLU
PUREDATA ANALYTICS
DB2 WAREHOUSE
PUREDATA OPERATIONAL
ANALYTICS
Actionable insight
Decision management
Predictive analytics and modeling
Reporting, analysis, content
analytics
Discovery and exploration
SPSS MODELER
COGNOS BICOGNOS TM1
WATSON EXPLORER
SPSS MODELER GOLD
Image and video
Data types
Transaction andapplication data
Machine andsensor data
Enterprise content
Social data
Third-party data
3
![Page 4: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/4.jpg)
Online Retailer Scenarios
4
![Page 5: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/5.jpg)
The Big Outdoors Company
Fictitious company
IBM-generated data for sales
Company stats
144 Products
3,500 Unstructured Data Items
5
![Page 6: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/6.jpg)
Scenario Zero Demo
6
![Page 7: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/7.jpg)
Scenario 1: Populate the Landing ZoneValue: Process and explore data more quickly
Information Movement, Matching & Transformation
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
BigSheets
Actionable
Insights
Explore Landed
data using
BigSheets
7
![Page 8: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/8.jpg)
Scenario One Demo
8
![Page 9: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/9.jpg)
BigSheets - Spreadsheet Interface
9
![Page 10: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/10.jpg)
Scenario 2: Perform Contextual DiscoveryValue: More quickly search all forms of data
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Actionable
Insights
Exploration &
Discovery
10
![Page 11: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/11.jpg)
Bringing Big Data to the Point of Impact
Commenting
Rating
SharedFolders
Tagging
Social Tools
FileSystems
RelationalData
ContentManagement
CRM
SupplyChain
ERP
RSS Feeds
ExternalSources
Cloud
CustomSources
Application/Users
Wa
tso
n E
xp
lore
r
11
BIG DATA PLATFORM
Stream Computing
Hadoop System
![Page 12: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/12.jpg)
Analytics – Clustering
12
![Page 13: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/13.jpg)
Scenario 3: Perform Text Analytics & Merge DataValue: Report and Retain Lineage from Hadoop
Information Movement, Matching & Transformation
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Actionable
Insights
Apply Text
Analytics
Merge Structured
w/ Unstructured
Data
BI & Performance
Management
13
![Page 14: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/14.jpg)
Scenario Three Demo
14
![Page 15: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/15.jpg)
Text Analytics – Processing of Unstructured Data
How it works• Parses unstructured text and detects
meaning with annotators
• Understands the context in which the text is analyzed
• Hundreds of pre-built annotators for names, addresses, phone numbers, and others
Parts of speech support forp English, Spanish, French, German, Portuguese, Dutch, Japanese, Chinese
• Distills structured info from unstructured text
Benefits
• More precise and correct answers
• 50% faster than manual method
• Run faster text analysis
Football World Cup 2010, one team distinguished themselves well, losing to the eventual champions 1-0 in the Final. Early in the second half, Netherlands’
striker, Arjen Robben, had a breakaway, but the keeper for Spain, Iker Casillasmade the save. Winger Andres Iniesta
scored for Spain for the win.
Unstructured text (document, email, etc)
Classification and Insight
15
![Page 16: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/16.jpg)
Scenario 4: Enhanced Product and Consumer Data Value: 360-degree view
Information Movement, Matching & Transformation
Landing, Exploration& Archive
PureData for
Operational
Analytics
Data
Sources
External
Social
BI & Performance
Management
Actionable
Insights
Apply Text
Analytics
with SDA
Enterprise Warehouse
Add value to
existing
Customer data
by applying
Social Profiles
16
![Page 17: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/17.jpg)
Scenario Four Demo
17
![Page 18: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/18.jpg)
Scenario 5: Better decision making using Predictive AnalyticsValue: Predict more often, more granular using more data
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Predictive Analytics
& Modeling
Actionable
Insights
18
PureData for
Operational
Analytics
Enterprise Warehouse
![Page 19: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/19.jpg)
Scenario 6: Real-time data mining & Text Analytics Value: Improve Customer Satisfaction & Sales
Real-Time Analytics
Data
Sources
Structured
Operational
Unstructured
External
Social
Sensor
Geospatial
Time Series
Streaming Predictive Analytics
& Modeling
Actionable
Insights
Real-time
(Predictions)
Real-time Text
Analytics
19
![Page 20: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/20.jpg)
Scenario 7: Encryption and Auditing of BigInsightsValue: Improve Data Security through Governance
Analytic Appliances
Security, Governance and Business Continuity
Landing, Exploration& Archive Enterprise
Warehouse
Data Marts
Data
Sources
Structured
Operational
Unstructured
Sensor
Geospatial
Time Series
BI & Performance
Management
Actionable
Insights
20
![Page 21: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/21.jpg)
IBM Big Data and Analytics Offerings
Watson Foundations
Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM
Exploration, landing and
archive
Trusted data
Reporting & interactive analysis
Deep analytics & modeling
Real-time processing & analyticsSTREAMS, DATA REPLICATION
Operational systems
BIGINSIGHTS
PUREDATAHADOOP
DB2, INFORMIX
PUREDATA TRANSACTIONS
PUREDATA ANALYTICS
DB2 BLU
PUREDATA ANALYTICS
DB2 WAREHOUSE
PUREDATA OPERATIONAL
ANALYTICS
Actionable insight
Decision management
Predictive analytics and modeling
Reporting, analysis, content
analytics
Discovery and exploration
SPSS MODELER
COGNOS BICOGNOS TM1
WATSON EXPLORER
SPSS MODELER GOLD
Image and video
Data types
Transaction andapplication data
Machine andsensor data
Enterprise content
Social data
Third-party data
21
![Page 22: Big Data Platform Overview - Meetupfiles.meetup.com/13471152/Indianapolis BD Developer Day... · 2014-05-02 · Big Data Platform Overview Alex Hay (athay@us.ibm.com), ... SPSS MODELER](https://reader030.fdocuments.in/reader030/viewer/2022040909/5e82714b7e07c64c461fd5ae/html5/thumbnails/22.jpg)
Thank You!
22