Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics &...
Transcript of Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics &...
![Page 1: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/1.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Empowering the Data-Driven OrganizationJeroen Dijkxhoorn, SASLars Slagboom, ABN AMRO
![Page 2: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/2.jpg)
In 5 years from now…Elephants will rule the world
![Page 3: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/3.jpg)
Acting on predictive Decisions will be standard
![Page 4: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/4.jpg)
Real Time Analytics is to blame for a crash
![Page 5: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/5.jpg)
Mobile User Interfacing will be the Standard
![Page 6: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/6.jpg)
Data will be everywhere and Nobody knows where exactly
![Page 7: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/7.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Trends Big Data, Storage, Hadoop & In-memory Technology
$- $20.000 $40.000 $60.000 $80.000 $100.000
Vertica
Teradata
Greenplum
Oracle
Microsoft PDW
Hadoop
Today 2009
Cost of Storage, Memory, Computing • In 2000 a GB of Disk $17 today < $0.07
• In 2000 a GB of Ram $1800 today < $10
• In 2009 a TB of RDBMS was $70K today < $ 20K
Cost per Terabyte
Technology Push: storage costs and CPU speed
![Page 8: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/8.jpg)
To enable analytics in this changing environment, you need to:
Bring the Analytics to the Data…
…and run it in a distributed mode
![Page 9: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/9.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Business pull: two Eras . . .two mindsets
Process-centric
Everything is
forbidden unless it is
permitted
Focus on cost control
Technology constrained
Discovery-centric
Everything is
permitted unless it is
forbidden
Focus on value
Technology empowered
![Page 10: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/10.jpg)
To enable analytics in this changing environment, you need to:
Provide self-service analytic capabilities…
…and automate the decision making process
![Page 11: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/11.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Data-Driven with Analytics as the main enabler
![Page 12: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/12.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
From Data to Decision
TEXT
MANAGE
DATA
EX
PL
OR
E
DA
TA
DEVELOP
MODELS
DE
PL
OY
&
MO
NIT
OR
Challenges:
• Growth in Demand
• Growth of Data
• Access to Talent
• Controlling Cost
Needs:
• Scale the Process
• Avoid Replication
• Increase Productivity
• Decouple Cost & Growth
![Page 13: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/13.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
SAS Directions to address these needs
Scale the Process
SPEED UP THE DATA TO DECISION LIFECYCLE
1. Event Stream Processing
2. High Performance Analytics
3. Decision Management
1
Avoid Replication
MOVE SAS PROCESSING TO THE DATA
1. In-Database Processing
2. Scoring Accelerators
3. Code Accelerators
2
Increase Productivity
PROVIDE INTERACTIVE, SELF-SERVICE INTERFACES
1. Data Loader for Hadoop
2. Visual Analytics, Visual Statistics & In-Memory Statistics
3. Move to responsive web-apps based on HTML5
3
Decouple Cost & Growth
SUPPORT IT COST EFFICIENCY EFFORTS
1. Span data and processing across a Grid or Cluster
2. Virtual Apps to deploy in Private, Public or Hybrid Cloud
3. On-premise deployment within 3 hours
4
![Page 14: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/14.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
![Page 15: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/15.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
![Page 16: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/16.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
…… …
……
on a single platform
annual savings
production time
19 models
€15 billion
−30%
Platform Strategy, Automotive Engineering
![Page 17: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/17.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
…
……
…
…
……
……
Risk
Sales
Partners
Fraud
Controlling
Marketing
Logistics
Purchasing
IT
Production
50% reduction in costs for BI/Analytics
Double the value of BI/Analytics projects
per year
Platform strategy: Basis of the Analytics Factory
![Page 18: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/18.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
![Page 19: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/19.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Standardization Consolidation Industrialization
3 steps towards an Analytics Factory
![Page 20: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/20.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Standardization
• Coming together by agreeing what capabilities to use
Consolidation
• Keeping together by centralizing the platform
Industrialization
• Working together by scaling and speeding up the process
3 steps towards an Analytics Factory
![Page 21: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/21.jpg)
Data en Informatie bij ABN AMRO
![Page 22: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/22.jpg)
Introductie
• ABN AMRO
• Enterprise Data & Information
22
![Page 23: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/23.jpg)
23
Standardization Consolidation Industrialization
![Page 24: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/24.jpg)
Standardization
Kenmerken
• Focus op systeemlandschap
• Iedereen zijn eigen voorkeur
• Data decentraal
Succesfactoren
• Externe druk
• Bedrijfsbreed thema
• Beleid
24
Standardization
![Page 25: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/25.jpg)
Consolidation
Kenmerken
• Focus naar gebruiker
• Waarde van geïntegreerde data wordt onderkent
• Wachttijden in je datawarehouse ontwikkeling
Succesfactoren
• Introductie gebruikersteams
• Vermarkt je datawarehouse en BI omgeving
25
Consolidation
![Page 26: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/26.jpg)
Industrialization
Kenmerken
• Focus op gebruik
• Snellere groei van data dan systemen
• Meer vraag dan aanbod
• Data is een keten
Succesfactoren
• Businessprocessen meenemen in je verandering
• Organiseer bronsystemen
26
Industrialization
![Page 27: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/27.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Marc Lammers:
“50 keer 2% is ook 100%”
![Page 28: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/28.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Back to the elephant…
![Page 29: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/29.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Where is Hadoop being used for?
Hadoop as a Data PlatformHadoop as a core component of next
generation analytical platform
TEXT
MANAGE
DATA
EX
PL
OR
E
DA
TA
DEVELOP
MODELS
DE
PL
OY
&
MO
NIT
OR
![Page 30: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/30.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Usage 1: Hadoop as Data Platform
Initiator
• This paradigm is mostly driven by IT
Drivers
• Increasing costs of data storage
• Increasing volume of data
• Latency to deliver information
Benefits
• Large-scale distributed storage and
batch processing
![Page 31: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/31.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Ingest/Load Data
Cleanse & Transform
Data
Load Data To Other Sources
/ Memory
Metadata Documentation
Usage 1: Hadoop as data platform
• SAS/ACCESS
• SAS Data Management
• SAS Event Stream Processing
• SAS Federation Server
• SAS Data Loader for Hadoop
SAS Data Quality Accelerator for
Hadoop
SAS Code Accelerator for Hadoop
• SAS/ACCESS
• SAS Data Management
• SAS Federation Server
• SAS Metadata Server
![Page 32: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/32.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Usage 2: Hadoop as core of next generation analytical platform
TEXT
MANAGE
DATA
EX
PL
OR
E
DA
TA
DEVELOP
MODELS
DE
PL
OY
&
MO
NIT
OR
Initiator
• This paradigm is mostly driven by business
Drivers
• Increasing question to a variety of different
and additional information
• The need for a flexible data platform to
store, process, and analyze data at any
scale
Benefits
• The business can start thinking big again
when it comes to data
![Page 33: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/33.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Usage 2: Hadoop as core of next generation analytical platform
TEXT
MANAGE
DATA
EX
PL
OR
E
DA
TA
DEVELOP
MODELS
DE
PL
OY
&
MO
NIT
OR
• SAS/ACCESS
• SAS Data Management
• SAS Event Stream Processing
• SAS Federation Server
• SAS Data Loader for Hadoop
SAS Data Quality Accelerator for
Hadoop
SAS Code Accelerator for Hadoop • SAS Visual Analytics
• SAS In-memory
Statistics for Hadoop
• SAS HPA Products
• SAS Visual Statistics
• SAS In-memory Statistics
for Hadoop
• SAS Decision Manager
• SAS Scoring Accelerator for
Hadoop
![Page 34: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/34.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Patterns of using SAS with Hadoop for Analytics & reporting
SAS with Hadoop
Hive
Extract from Hadoop pushing
some SAS pre-processing to
Hadoop
Embedded Process - Push
SAS data processing to
Hadoop with Map Reduce
SAS in Hadoop
Score A Code AImpala
In-Memory Analytics - Use
Hadoop for Storage persistence
and commodity computing.
SAS on Hadoop
HPA LASR
![Page 35: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/35.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
Continuity of Business
Bring SAS processing to the Data
Leverage Hadoop for new Technology offerings
Breadth and depth of modern analytic methods in Hadoop
SAS for Hadoop directions
DIRECTIONAL THEMES
![Page 36: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/36.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.
13.30 Parallel Sessions
• Big Data and Visual Analytics – Rabobank
• Business Analytics – SAS
• Data Management – Ziekenhuis Gelderse Vallei
• Visual Analytics – Mercachem
13.30 Guided Tours
• Visual Analytics
15.45 Parallel Sessions
• Big Data and Visual Analytics – Belastingdienst
• Business Analytics – iBridge/ Randstad
• Data management – DSM
• Visual Analytics – H@nd
Information on breakouts Analytical platform
14.30 What’s Hot Sessions
• Big Data Analytics met Hadoop
• Data Management 3.0: What about Hadoop?
• What’s hot in Data Governance
• Modernisatie: meer mogelijkheden, minder risico’s
• Geavanceerd modelleren met SAS
• What’s new in SAS Visual Analytics 7.1
• Best Practices in Visualisatie en Dashboard design
14.30 Roundtables (max 20 pers.)
• The Analytical Bank
• Data monetization
![Page 37: Empowering the Data-Driven Organization - SAS · Patterns of using SAS with Hadoop for Analytics & reporting SAS with Hadoop Hive Extract from Hadoop pushing some SAS pre-processing](https://reader034.fdocuments.in/reader034/viewer/2022042205/5ea6cbfa634e9909df326d62/html5/thumbnails/37.jpg)
Copyright © 2014, SAS Institute Inc. All rights reserved.