Oracle Big Data - Overview - AIOUG Big Data... · Oracle Big Data - Overview Hariharaputhran...
Transcript of Oracle Big Data - Overview - AIOUG Big Data... · Oracle Big Data - Overview Hariharaputhran...
Oracle Big Data - Overview
Hariharaputhran VaithinathanDirector of MembershipAIOUG
Sai Janakiram PenumuruVice PresidentAIOUG
2
Introduction
Sai Janakiram Penumuruo Fourteen years of experience - Oracle DBA / Oracle Apps DBA / Cloud Technologieso Lead Cloud Architect, AT2C Expert - HPEo Co-Fonder, Vice President, Director of Finance - All India Oracle Users Group (AIOUG)o Oracle ACE Director & Member of TOSCA - Topology and Orchestration Specification for Cloud Applicationso Oracle VM SIG Leader www.oraclevmsig.orgo Blog: www.oadba.com; www.oracle12c.infoo Contacts – [email protected] ; twitter - @sai_penumuru
3
V.Hariharaputhrano Twelve years in Oracle Development / DBA / Big Data / Cloud Technologieso All India Oracle Users Group (AIOUG) Evangelist o Leading AIOUG Chennai Chaptero Blog: www.puthranv.com
4
Plan of action for Next
Overview of Oracle Big Data
To make you an Big Data Expert
Minutes45
Agenda• Data Growth and A New Style of IT
• Defining Big Data: Market Drivers and Trends
• Why Big Data Now?
• Overview of Oracle Big Data Appliance
• Traditional vs Modern Systems
• Hadoop Overview
• HDFS Storage
• Map Reduce Processing
• Oracle Hadoop Integration
6
Internet of People to Internet of Things
QUALITY & CONSISTENCY MAINTAIN & REPAIRSMART SHOPPING MONITOR POLLUTION LEVELS
WILDLIFE PROTECTION FARMING ENERGY
Devices TALK to each other as they become SMART & generate DATA
7
A Real World Example:- Big Data- Micro-transactionsSensor data collected from US commercial jet engines during 1 year
1,041,600,500 TB
20 TB20 terabytes of information per engine every hour
2.5Average duration for US flights in hours
2twin-engine Boeing 737
days in a year
36528,537# of commercial flights in the sky in the United States on any given day
8
1030
This will take us beyond our decimal system
Geopbyte
This will be our digital universe tomorrow…
Brontobyte 1027
1024This is our digital universe today
= 250 trillion of DVDs
Yottabyte 10
21
1.3 ZB of network traffic by 2016
Zettabyte
Information from the Internet of Things: We have gone beyond the decimal system
Today data scientist uses
Yottabytes to
describe how much government data the NSA or FBI have on people altogether.
In the near future,
Brontobyte will be the
measurement to describethe type of sensor data that will be generated from the IoT
1018
1 EB of data is created on the internet each day = 250 million DVDs worth of information. The proposed Square Kilometer
Array telescope will generated an EB of data per day
Exabyte
1012
Terabyte500TB of new data per day are ingested in Facebook databases
1015
PetabyteThe CERN Large Hadron Collider generates 1PB per second
109
Gigabyte106
Megabyte
9
695,000 status updates
98,000+ tweets
698,445 Google searches
1,820TB of data created
11million instant messages
168 million+ emails sent
217 new mobile web users
Growing Internet of Things (IoT)
Pervasive Connectivity
Explosion of Information
Smart Device Expansion
Every 60 seconds
2013
A new style of IT required for IoT solutions
By 2020
40 Trillion GB(2)
10 Million(3)
… for 8 Billion(4)
(1) IDC Directions 2013: Why the Datacenter of the Future Will Leverage a Converged Infrastructure, March 2013, Matt Eastwood ; (2) & (3) IDC Predictions 2012: Competing for 2020, Document 231720, December 2011, Frank Gens; (4) http://en.wikipedia.org
A new era of accelerated innovationForever changing how consumers and businesses interact, enabling new opportunities
30 Billion(1)
DATA
Mobile Apps
Devices
10
What is Big Data? : Regular structured data
Over a longer periodFaster analysis
11
What is Big Data? : Meaning from Unstructured Data
Social media
Images Video Audio
Email Documents
Defining Big Data: Market Drivers and Trends
13
Big Data
Variety
Velocity
Volume
Big data is derived from a variety of sources
14
An intelligent end-to-end approach delivers the right information to the right person at the right time
A day in the life of Big Data
ExecutiveDashboards
Enterprise Search
Customer Interaction
PredictiveAnalytics
WebEngagement
Transactional Operational Strategic
Volume
Variety
Velocity
CRM(Sales)
Web ERP(Procurement)
Supply Chain(Ops)
Machine Generated Data
HR
Social Media Video Audio Email Texts TransactionalData
Word, Excel Logs Clickstream Data
ImagesMGD
Why Big Data Now?
16
How to make Smarter Decisions & Better Predictions
GUT
INTUTIONDATAGUESSBIG
17
Better,Right Time Decisions
Right information. Right person. Right time.
Command of information drives increased business performance
Foresight –What will happen?
Hindsight –what happened?
Insight –
What is happening now?
In Flightinformation
Activeinformation
Inactiveinformation
Right Time Decisions
18
Big Data Use Cases
Today’s Challenge New Data What’s Possible
HealthcareExpensive office visits
Remote patient monitoringPreventive care, reduced
hospitalization
ManufacturingIn-person support
Product sensors Automated diagnosis, support
Location-Based ServicesBased on home zip code
Real time location dataGeo-advertising, traffic, local
search
Public SectorStandardized services
Citizen surveysTailored services,
cost reductions
RetailOne size fits all marketing
Social media Sentiment analysis segmentation
19
Innovative analytic use cases are cutting across structured, unstructured and semi structured data
Big Data opportunities across industries and use cases
Government Telecom Manufacturing Healthcare
• Sentiment analysis
• Social CRM / network analysis
• Churn mitigation
• Brand monitoring
• Cross and Up sell
• Loyalty & promotion analysis
• Web application optimization
• Marketing campaign optimization
• Brand management
• Social media analytics
• Pricing optimization
• Internal risk assessment
• Customer behavior analysis
• Revenue assurance
• Logistics optimization
• Clickstream analysis
• Influencer analysis
• IT infrastructure analysis
• Legal discovery
• Equipment monitoring
• Enterprise search
• Drug development
• Scientific research
• Evidence based medicine
• Healthcare outcomes analysis
• Supply chain optimization
• Defect tracking
• RFID Correlation
• Warranty management
• Broadcast monitoring
• Churn prevention
• Advertising optimization
• Law enforcement
• Counter terrorism
• Traffic flow optimization
Horizontal use cases
Sources: IDC: 2012 “Worldwide Big Data Technology and Services Forecast: 2011-2015, Gartner: 2012 “Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016
Finance
• Fraud detection
• Anti-money laundering
• Risk management
Energy
• Weather forecasting
• Natural resource exploration
Overview ofOracle Big Data Appliance
21
Oracle Big Data Appliance
Sources: http://www.oracle.com/technetwork/database/bigdata-appliance/overview/bigdataappliance-datasheet-1883358.pdf
22
Prepare your host system
Oracle Big Data Lite Virtual Machine
To get started:• Download and install Oracle VM VirtualBox and 7-zip
• Download each of the 7-zip files
• Run the 7-zip extractor on the BigDataLite421.7z.001 file only. This will create the BigDataLite-421.ova VirtualBox appliance file
• In VirtualBox, import BigDataLite-421.ova
• Start BigDataLite-4.2.1
• Log in as oracle/welcome1
Version 4.2.1
Technical Requirements:
• Dedicate 2 cores
• 5 GB memory
• 50GB disk space to the virtual machine
• Install will require ~53 GB disk space including temporary files
23
http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html
Download Oracle Big Data Lite Virtual Machine
File Description
Deployment Guide •Start Here!Deployment Guide provides step-by-step instructions for download and deployment.
BigDataLite421.7z.001 (2147483648 bytes)BigDataLite421.7z.002 (2147483648 bytes)BigDataLite421.7z.003 (2147483648 bytes)BigDataLite421.7z.004 (2147483648 bytes)BigDataLite421.7z.005 (2147483648 bytes)BigDataLite421.7z.006 (2147483648 bytes)BigDataLite421.7z.007 (2147483648 bytes)BigDataLite421.7z.008 (2147483648bytes)BigDataLite421.7z.009 (2147483648 bytes)BigDataLite421.7z.010 (2147483648 bytes)BigDataLite421.7z.011 (2147483648 bytes)BigDataLite421.7z.012 (2147483648 bytes)BigDataLite421.7z.013 (147967130 bytes)
md5sum.txt (346 bytes)
To get started:•Download and install Oracle VM VirtualBox and 7-zip•Download each of the 7-zip files•Run the 7-zip extractor on the BigDataLite421.7z.001 file only. This will create the BigDataLite-4.21.ova VirtualBox appliance file•In VirtualBox, import BigDataLite-421.ova•Start BigDataLite-4.2.1•Log in as oracle/welcome1See the Deployment Guide for details.
Cloudera JDBC Drivers Download and install the Cloudera JDBC drivers to enable Oracle SQL Developer and Data Modeler to connect to Hive.
Oracle Enterprise Linux 6.6Oracle Database 12c Release 1 Enterprise Edition (12.1.0.2) - including Oracle Big Data SQL-enabled external tables, Oracle Multitenant, Oracle Advanced Analytics, Oracle OLAP, Oracle Partitioning, Oracle Spatial and Graph, and more.Cloudera Distribution including Apache Hadoop (CDH5.4.0)Cloudera Manager (5.4.0)Oracle Big Data Discovery 1.1Oracle Big Data Connectors 4.2
Oracle SQL Connector for HDFS 3.3.0Oracle Loader for Hadoop 3.4.0Oracle Data Integrator 12cOracle R Advanced Analytics for Hadoop 2.5.0Oracle XQuery for Hadoop 4.2.0
Oracle NoSQL Database Enterprise Edition 12cR1 (3.3.4)Oracle Big Data Spatial and Graph 1.0Oracle JDeveloper 12c (12.1.3)Oracle SQL Developer and Data Modeler 4.1Oracle Data Integrator 12cR1 (12.1.3.0.1)Oracle GoldenGate 12cOracle R Distribution 3.1.1Oracle Perfect Balance 2.4.0Oracle CopyToBDA 2.0
Version 4.2.1
Thank you