Oracle Big Data Building A Big Data Management...

41
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Effi Psychogiou ECEMEA Big Data Product Director May, 2015 Oracle Big Data Building A Big Data Management System

Transcript of Oracle Big Data Building A Big Data Management...

Page 1: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Effi Psychogiou ECEMEA Big Data Product Director May, 2015

Oracle Big Data Building A Big Data Management System

Page 2: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Page 3: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Big Data Disruption

Digitization × Datafication

Page 4: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Customer Experience

Operational Improvement

New Business Models

44% 30% 26%

Page 5: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Traditional Currency

Photo Film

Postal Services

Printing Press

Record Industry

Digital Currency

Web Publishing

Email Digital Camera

Digital Download

Entire Industries are Being Disrupted

Page 6: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 6

de Persgroep Customer 360 Improve prospecting Better content Improve subscriber retention

Increase Share

Page 7: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Bigger Smarter

Easier

Cheaper Better

Faster VALUE FROM DATA

Page 8: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Why Is Big Data Important? Value Creation

HEALTH CARE AUTOMOTIVE COMMUNICATIONS

“In a big data world, a competitor that fails to sufficiently develop its capabilities will be left behind.”

Reduce Prescription Fraud

Predict Failures

Gain Customer sentiment insight

Offering New Services based on Location

Data

McKinsey Global Institute

RETAIL

Better Predict Product Success

BANKING

Better understand customers

Page 9: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Big Data is a Strategic Initiative, not just a Tactic Why is Big Data Important for Organisations

“…CaixaBank has partnered with Oracle to develop a new technology platform that can help improve the business and enable the bank to anticipate the needs of customers with a 360 view of the customer*”

Juan Maria Nin

CEO CaixaBank

(*) Mentioned in a business review with financial analysts in London, 26 March 2014

Page 10: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 10

CREATE VALUE

SILOS OF INNOVATION SYSTEMS OF RECORD

Page 11: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 11

Preserving Customer Investment & Managing All Types of Data

Fourth Generation Oracle Data Architecture

Generation 4 Generation 3 Generation 2 Generation 1 Database Cloud Service

Multitenant

Internet Architecture

Thin Clients

Client/Server

PCs and Servers

Minicomputer &

Mainframe Database • Big Data In-Memory

• Hadoop & NoSQL • Unstructured Data • Objects, XML

• PL/SQL • Text, Images

• Relational • SQL

Page 12: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | #StrataHadoop - Oracle Big Data Architecture

Use the Right Tool for the Job and benefit from the Power of “AND”

Relational Hadoop NoSQL

Run the Business Integrate existing

systems

Mission-critical tasks

Use existing

investments

Ensure skills relevance

Change the Business

Disrupt competitors

Disintermediate supply

chains

Leverage new paradigms

Exploit new analyses

Scale the Business

Serve data faster

Persist data streams

Meet mobile and device

challenges

Scale-out economically

4th Generation Data Architecture Technologies for Big Data

Page 13: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Execution

Innovation

#StrataHadoop - Oracle Big Data Architecture

4th Generation Data Architecture for Big Data

Warehouse Data Factory Reservoir

Data Streaming

Data Platform

Discovery Lab

Analytics

Enterprise Data

Other Data Sources

Data Streams

Business Data

Social/Log Data

Model First Analytics

• Reporting-oriented • Often enterprise wide

in scope, cross LoB • “you know the

questions to ask”

Reports & Dashboards

Data First Analytics

• Data Exploration • Highly visual and/or

interactive • “you don’t know the

questions to ask”

Discovery

• Telematics • Industry Services • Internet of Things • Sentiment

Data Services

Page 14: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Execution

Innovation

#StrataHadoop - Oracle Big Data Architecture

Comprehensive Oracle Solution for Big Data

Warehouse Factory Reservoir

Data Streaming

Data Platform

Discovery Lab

Analytics

Enterprise Data

Other Data Sources

Data Streams

Business Data

Social/Log Data

Model First Analytics

• Reporting-oriented • Often enterprise wide

in scope, cross LoB • “you know the

questions to ask”

Reports & Dashboards

Data First Analytics

• Data Exploration • Highly visual and/or

interactive • “you don’t know the

questions to ask”

Discovery

• Telematics • Industry Services • Internet of Things • Sentiment

Data Services

Apache Oracle

NoSQL & OEP

Oracle Data Integration & Governance

Oracle Database & Big Data SQL

Oracle R

Oracle Big Data

Discovery

Oracle Business

Intelligence

Oracle Big Data

Discovery

Apache

Page 15: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Execution

Innovation

#StrataHadoop - Oracle Big Data Architecture

Integrated Oracle Engineered Systems for Big Data

Data Streaming

Data Platform

Discovery Lab

Analytics

Enterprise Data

Other Data Sources

Data Streams

Business Data

Social/Log Data

Model First Analytics

• Reporting-oriented • Often enterprise wide

in scope, cross LoB • “you know the

questions to ask”

Reports & Dashboards

Data First Analytics

• Data Exploration • Highly visual and/or

interactive • “you don’t know the

questions to ask”

Discovery

• Telematics • Industry Services • Internet of Things • Sentiment

Data Services

Analytics

Page 16: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Execution

Innovation

#StrataHadoop - Oracle Big Data Architecture

Visionary Oracle Cloud for Big Data

Data Platform

Discovery Lab

Analytics

Enterprise Data

Other Data Sources

Data Streams

Business Data

Social/Log Data

Model First Analytics

• Reporting-oriented • Often enterprise wide

in scope, cross LoB • “you know the

questions to ask”

Reports & Dashboards

Data First Analytics

• Data Exploration • Highly visual and/or

interactive • “you don’t know the

questions to ask”

Discovery

• Telematics • Industry Services • Internet of Things • Sentiment

Data Services

Data Streaming

Big Data Platform as a

Service

Big Data Discovery Platform as a

Service

Page 17: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 17 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

BIG DATA MANAGEMENT

BIG DATA ANALYTICS

BIG DATA APPLICATIONS

BIG DATA INTEGRATION

DATA CAPITAL

Connect And Govern Any Data Streaming + Batch

Simplify Access To All Data

Data Reservoir + Data Warehouse

Discover And Predict, Fast

Discovery + Business Analytics

Accelerate Data-Driven Action Mobile + Web + On-device

Enterprise Class Big Data Capabilities For Organizations Creating Value from the Data Capital

Page 18: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | #StrataHadoop - Oracle Big Data Architecture

Oracle Brings Business Value to Big Data

Enterprise-Grade Capabilities

Discover and Predict – Fast

Govern and Secure All Data

Simplify Access to All Data

Performance Integration Availability Scalability Manageability

Page 19: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Discover and Predict, Fast

19

BIG DATA MANAGEMENT

BIG DATA ANALYTICS

BIG DATA APPLICATIONS

BIG DATA INTEGRATION

DATA CAPITAL

Page 20: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 20

What if we could make data preparation 20% of the effort so you can focus 80% of your time on executing and improving your business?

Inverting Data Access Cycles

Page 21: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Oracle Big Data Discovery The Visual Face of Hadoop – Making data preparation 20% of the effort

find explore transform discover share

#StrataHadoop - Oracle Big Data Architecture

Page 22: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Hadoop / Big Data Appliance

• Oracle R Distribution1

• Oracle R Advanced Analytics for Hadoop2

• SAS High Performance Analytics

• Oracle SQL Analytic functions

1 Included with BDA 2 Included w/Oracle Big Data Connectors

Oracle Database / Exadata

• Oracle Advanced Analytics Option

• SAS High Performance Analytics

• Oracle SQL Analytic functions

Statistical & Predictive Analytics Bring the Analytics to the Data

Page 23: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 23

Stubhub Predictive analytics means winning the fraud arms race

90% Fraud Cut

Page 24: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 24

BIG DATA MANAGEMENT

BIG DATA ANALYTICS

BIG DATA APPLICATIONS

BIG DATA INTEGRATION

DATA CAPITAL

Simplify access to all data

Page 25: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Big Data Management Hadoop + NoSQL + Relational

25

Page 26: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Big Data SQL One Fast Query for All Your Data

26

NoSQL

Page 27: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 27

Extracting Value from Data in Unexpected Ways

Page 28: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 28

BIG DATA MANAGEMENT

BIG DATA ANALYTICS

BIG DATA APPLICATIONS

BIG DATA INTEGRATION

DATA CAPITAL

Govern and Secure All Data

Page 29: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | #StrataHadoop - Oracle Big Data Architecture

Comprehensive Integration and Governance

Fast Load

Speed Layer

Batch Layer

Oracle Data Integrator (Transform)

Oracle GoldenGate (Move & Ingest)

Data Governance Foundation

Enterprise Data Quality (Profile & Cleanse)

Enterprise Metadata Management & Business Glossary (Business Glossary, Data Lineage, Impact Analysis and Data Provenance)

Veridata (Verify)

Data Enrichment (Prepare)

Real-Time Data Movement – Low impact capture, stage in Hadoop – Continuous data availability

Data Transformation – Bulk data movement – Pushdown data processing

Data Governance – Prepare unstructured data – Profile data with sampling – Clean data in real time or batch – Verify data for consistency – Trace lineage of all data – Define glossary of business terms

Page 30: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Securing Hadoop and NoSQL

30

NoSQL

Extending the Oracle Database security umbrella

Page 31: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

18 Sun Oracle X5-2L Servers with per server:

• 2 * 18-core Intel® Xeon® Haswell-EP - fastest Intel processor ever shipped

• 128 GB Memory (SR: 768GB, FR: 2304GB)

• 48TB Disk space (SR: 288TB, FR: 864TB)

Integrated Software (4.1):

• Oracle Linux6.5, Oracle JDK 7u72

• Cloudera Distribution of Apache Hadoop 5.3 – EDH Edition, Cloudera Manager 5.3

• Oracle Big Data SQL 1.1, Oracle R Distribution 3.1.1-2, Oracle NoSQL Database CE 3.2.4

Oracle Big Data Appliance X5-2 Available in 6 / 12 / 18 node Configurations

#StrataHadoop - Oracle Big Data Architecture

Page 32: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | #StrataHadoop - Oracle Big Data Architecture

Proven, Cost Effective Solution

“Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.”

⁻ Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board

21% Cost Savings

33% Faster Time

to Value

Page 33: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 33

Where to Start?

Page 34: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 34

Decide on Your Big Data Approach

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

BIG DATA INTEGRATION

1. Transform the business

2. Lay the foundation

3. Pilot

BIG DATA ANALYTICS

BIG DATA APPLICATIONS

CREATE VALUE FROM DATA

BIG DATA MANAGEMENT

Page 35: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Find Your Big Data Project Project Roadmap

• Define the Question

• Define the Ideal Data Set

• Determine what Data you can Access

• Obtain & Clean the Data

• Perform Exploratory Analysis (identify patterns in the data)

• Perform Statistical Prediction/Modeling (intuition on what will happen)

• Interpret & Challenge Results

• Write Up & Distribute Results

Page 36: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Make Up The Big Data Project Team Skills Profile

Business

Data Analytics

• Motivating Questions

• Hypothesis on Data

• Results Communication

• Maths & Statistics

• Data Exploration

• Prediction Modeling

• Results Interpretation

• Computer Science

• Acquisition & Integration

• Preparation

• Quality Assurance

+ Curiosity

+ Tenacity

Page 37: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Build On Big Data Skills & Technologies Data & Analytics

• Data

– Hadoop & MapReduce

– NoSQL (e.g. Oracle NoSQL Database)

– Database (e.g. Oracle Database)

– Programming Languages: SQL, Java/C++

– Scripting languages: PHP, Python, Perl, JavaScript,

• Analytics – Understanding of statistical data analysis

– Experience with packages such as R, MATLAB, SPSS, SAS, Oracle Advanced Analytics

Page 38: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle BigDataLite Virtual Machine

An integrated environment to help you get started with the Oracle Big Data platform.

Many Oracle Big Data platform components have been installed and configured - allowing you to begin using the system right away.

Download Oracle BigDataLite

http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html

Page 39: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

The Big Opportunity of the Future is Data Capital

39 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | #StrataHadoop - Oracle Big Data Architecture

Page 40: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 40

Oracle Can Help You Find Unique Value From Your Data Capital

Page 41: Oracle Big Data Building A Big Data Management Systemcedawi.org/docs/Effi-Psychogiou-Big-Data-Day-Baku-2015...Real-Time Data Movement –Low impact capture, stage in Hadoop –Continuous