Oracle Big Data Appliance and Big Data SQL for advanced analytics
-
Upload
jdijcks -
Category
Data & Analytics
-
view
701 -
download
4
description
Transcript of Oracle Big Data Appliance and Big Data SQL for advanced analytics
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Big DataChanging the Way You Manage and Analyze Big Data
Jean-Pierre DijcksBig Data Product ManagementServer Technologies
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Use Data
12%
Executives who feel they understand the impact data
will have on their organizations
Produce Data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
From Storing Data to Monetizing Data
*Source : ‘Enterprise Architecture As Strategy: Creating a Foundation for Business Execution’ by J Ross, P. Weill, D. Robertson, HBS Press, 2006
StoringData
ManagingData
MonetizingData
DisparateData Marts
EnterpriseData Warehouse
Big Data Management System
StrategicBusiness
Value of IT
IT Budget
CostCenter
ProfitCenter
100%
84%92%
145%
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Analytics 2.0
Analytics 3.0
Analytics 1.0
• Reporting with limited use of descriptive analytics
• Limited range of tabular data • Batch oriented analysis • Analysis bolted onto limited set
of business processes
• Firms “Competing on Analytics”• Extended analytics to larger and
less structured datasets• Emergence of Big Data into the
commercial world• Recognition of Data Science role
in commercial orgs.
• Platform for monetization• Deeper analysis & more data• Faster test-do-learn iterations• Different types of data & wider
business process coverage• Analysts focus on discovery and
driving business value• “Agile” with operational elements
incorporated into design patterns
Adapted from: Tom Davenport material – Harvard Business Review (2010)
The Path to Monetizing Big Data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
ActionableEvents
Streaming Engine Data Reservoir Enterprise Data & Reporting
Discovery Lab
ActionableMetrics
ActionableData Sets
InputEvents
Execution
Innovation
Discovery Output
Data
Conceptual View
StructuredEnterprise Data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
De PersgroepCreating a linked customer analytics system
Objectives Maximizing customer value Optimizing campaign cost through
Automation and Targeting
Solution Single, rich customer repository based on Big
Data Appliance and NG Data® Lily®
Analytics drive: subscriber management (up-sell/cross-sell, churn,
conservation) editorial use (article engagement, adapt content over
time)
- Toyota Global Vision
Customer Data Store
Digital, RDBMS, External
BDA
Mobile
Web
Subscribers
NG DataLily
Customer Analytics
Phase 1: Improved Data Quality Single View of all Customers improves
customer management
Benefits
Social
CustomerAnalytics & aggegated
data
Oracle Data Warehouse
Business Objects
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
GlobacomImprove Customer Information
Objectives Respond to customer queries in as close to real
time as possible Understand behavior, improve retention, and
increase cross-selling d services
Solution Capture and analyzie >1B CDR’s daily in Oracle Big
Data Appliance Integrate resulting data, using Oracle NoSQL Database
into online systems Leverage xDR Navigator from partner mCentric to
improve first call resolution ratesBDA
mCentric
Save over 35,000 call processing minutes per day
Analyze network events 40x faster
Benefits
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
US-based BankLowering Costs by Simplifying IT Infrastructure
Objectives Comply with regulations requiring more data
to support stress testing Reduce IT costs & streamline processing by
eliminating duplicate data stores
Solution Single, reliable BDA/Exadata-based ODS
supporting all downstream systems Landing zone & archival repository for both
structured & unstructured data Use Exadata as “19th” BDA node
- Toyota Global Vision
Operational Data StoreMainframe,
RDBMS, more
BDA Exadata
• Agile business model
• All data• De-normalized
& Partial-normalized
• Normalized• Aggregate data• EDW
Oracle Enterprise Manager
Oracle Data Integrator
Data Delivery
MasterS1
MasterS2
MasterSn
SOA/APICRMSOther
Faster access to 6x more data Lower costs, simplified architecture and fast
time to value
Benefits
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Enterprise Class Big Data Capabilities
zBY INDUSTRY & LINE OF BUSINESS
BIG
DAT
A AP
PLIC
ATIO
NS
DISCOVERY
BUSI
NES
SAN
ALYT
ICS
BUSINESS ANALYTICS
DATA RESERVOIR
BIG
DAT
AM
ANAG
EMEN
T
DATA WAREHOUSE
SOU
RCES
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Management System
SOU
RCES
Oracle Database
Oracle IndustryModels
Oracle Advanced Analytics
Oracle Spatial & Graph
Big Data Appliance
Cloudera Hadoop
Oracle NoSQL Database
Oracle R Advanced Analytics for Hadoop
Oracle R Distribution
Oracle Database
Oracle Advanced Security
Oracle Advanced Analytics
Oracle Spatial & Graph
Oracle Exadata
Oracle Big DataConnectors
Oracle DataIntegrator
Oracle Big Data SQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Strengths of Both Systems
Tooling maturity
Stringent Non-Functionals
ACID transactions
Security
Variety of data formats
Data sparsity
ETL simplicity
Cost effectively store data
Ingestion rate
Straight Through Processing (STP)
0
5
Hadoop
RDBMS
• Hadoop is good at some things
• Databases are good at others• SQL is very important
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 13
“The implementation of this Big Data solution will help CaixaBank remain at the forefront of innovation in the financial sector, delivering the best and most competitive services to our customers”– Juan Maria Nin, Chief Executive Officer, CaixaBank
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Feedback Loop
Data Management
Big Data Platform
(Hadoop/NoSQL)
Relational Data Warehouse
(OCDM)
Analytic Apps
Customer Experience
Operations
Monetization
Adapters
ETL/ELT Adapters
Real-Time Adapters
ThirdParty
DataSources
Oracle Comms Apps (BSS/OSS)
Oracle Comms Ntwk Products (Tekelec
& Acme)
Other Oracle Apps (CRM, ERP, etc.)
Third Party Sources
Oracle Communications Data ModelReference Architecture
To Other Apps
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted
15
Oracle Big Data SQL – A New Architecture
• Powerful, high-performance SQL on Hadoop– Full Oracle SQL capabilities on Hadoop– SQL query processing local to Hadoop nodes
• Simple data integration of Hadoop and Oracle Database– Single SQL point-of-entry to access all data– Scalable joins between Hadoop and RDBMS data
• Optimized hardware– Balanced Configurations– No bottlenecks
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 16
Two Challenges
1. Make Hadoop easily consumable for customers
2. Enable Oracle SQL on All Data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 17
Recap: Big Data Appliance OverviewBig Data Appliance X4-2
Sun Oracle X4-2L Servers with per server:• 2 * 8 Core Intel Xeon E5 Processors• 64 GB Memory• 48TB Disk space
Integrated Software:• Oracle Linux, Oracle Java VM• Oracle Big Data SQL*• Cloudera Distribution of Apache Hadoop – EDH Edition• Cloudera Manager• Oracle R Distribution• Oracle NoSQL Database
* Oracle Big Data SQL is separately licensed
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 18
Recap: Standard and Modular
Starter Rack is a fully cabled and configured for growth with 6 servers
In-Rack Expansion delivers 6 server modular expansion block
Full Rack delivers optimal blend of capacity and expansion options
Grow by adding rack – up to 18 racks without additional switches
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Appliance
Engineered Systems Benefits
Lower TCO than DIY Hadoop Clusters
Faster Time to Value
Higher Performance out-of-box
Lower Management Overhead
Integrated and Comprehensive Security
Tight Integration with your Infrastructure
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
TCO Data Points: 18 servers (DL380 vs. X4-2L)
864TB Raw Storage 288 Cores 1152GB Total Memory
Cloudera Enterprise Subscription with all options
Subscription vs. Perpetual Equivalent Installation Cost Not calculated:
Soft Cost (people and time to value) Data integration licenses
Engineered Systems Benefits
Year 1 Year 2 Year 3 Year 4 Year 5$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
Oracle BDAHP + ClouderaSavings
List Price Comparisons
Cum
ulati
ve C
ost a
nd S
avin
gs
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Engineered Systems BenefitsBDA 3.0 DIY CDH 5.0
Management Console
Single Command Patching and Upgrade
Full Stack Patching and Upgrading
Automatic Cluster Re-Configuration
Security (AAA) out-of-box
Encryption out-of-box (network and at-rest)
InfiniBand + Optimizations
Stack Tuning (OS, Java, Hadoop)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
What does it mean to engineer a BDA?
Linux Optimization
Java Configuration
Pre-Configured AAA security and Encryption
Pre-Configured Hadoop Settings
Ex: HDFS, Memory and MR Slots
Network Optimizations
Node Configurations (Roles and Growth)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Security Differentiators
Oracle Database BDA 2.5 DIY CDH 4.6
User Authentication
Row Level Access Controls
Monitoring and Auditing
Encryption at Rest
Network Encryption
Masking, Redaction etc.
Column Lvl Access Ctrl
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
BDA Security Overview
Authentication through Kerberos
Authorization through Apache Sentry
Auditing through Oracle Audit Vault
Encryption for Data-at-Rest
Network Encryption
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Exadata+
Oracle Database
Big Data Appliance+
Hadoop & NoSQL
Embrace Innovation and Integrate
UnifyDevelopment languages
SecurityAdministration
SupportWorkload managementLifecycle management
Availability
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 26
Oracle Big Data Management System
One fast SQL query, on all your data.
Oracle SQL on Hadoop and beyond, with a Smart Scan service as in Exadata and the security of Oracle Database
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 27
Big Data SQL
SELECT w.sess_id, c.nameFROM web_logs w, customers cWHERE w.source_country = ‘Brazil’AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
10’s of Gigabytes of Data
Only columns and rows needed to answer query are returned
Hadoop Cluster
Big Data SQL
Oracle Database
CUSTOMERSWEB_LOGS
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 28
Big Data SQL
SELECT w.sess_id, c.nameFROM web_logs w, customers cWHERE w.source_country = ‘Brazil’AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
10’s of Gigabytes of Data
Only columns and rows needed to answer query are returned
Hadoop Cluster
Big Data SQL
Oracle Database
CUSTOMERSWEB_LOGS
SQL Push Down in Big Data SQL
• Hadoop Scans on Unstructured Data• WHERE Clause Evaluation• Column Projection• Bloom Filters for Better Join Performance• JSON Parsing, Data Mining Model Evaluation
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 29