Michael Heuer Senior Specialist Systems Engineer€¦ · Support • Financial Cloudera Hadoop •...

GLOBAL SPONSORS

Michael Heuer Senior Specialist Systems Engineer

Building the Enterprise Data lake

© Copyright 2017 Dell Inc. 2

Customers >8000

>20% YoY Growth Data Lake #1

#1 Scale-Out NAS

Market Leader

>2000 Big Data Analytics

Customers Hadoop Shared Storage #1

DELL EMC Data lake momentum


UNSTRUCTURED DATA GROWTH

Source: IDC

2015

71 EB

Total Capacity Shipped, Worldwide % of Unstructured Data

75%

78%

80%

2016

106 EB

2017

133 EB


“Digital is the main reason just over half of

the companies on the Fortune 500 have

disappeared since the year 2000”

Pierre Nanterme

CEO of Accenture

By 2018, 33% of all industry

leaders will be disrupted by digitally

enabled competitors1 Source: IDC

45% believe there

is a possibility their

business will become

obsolete within the

next 3 – 5 years Source: Dell Technologies

see digital start-ups as a

threat to their business,

now or in the future

Source: Dell Technologies

of the G2000 by 2020 will see the majority of their

business depend on digitally-enhanced products,

services and experiences Source: IDC

Digital disruption


What is digital transformation?


Digital Transformation is the

application of digital capabilities to

products, services, and processes

in order to increase customer

value, optimize operational

efficiency, and develop new

monetization opportunities.


Digital displacement Disruptive business models disintermediate the status quo

Vs.

Books

Vs.

Movies Vs.

Photography

Vs.

Electronics

Vs.

Music Vs.

Online

Vs.

Transportation Vs.

Hospitality


Transforming data into business value

Analytics

Insights

Intelligent Apps

Data 0100 1011 0100

0100 1011 0100 Data Aggregation


Expanding Big Data Requires - A New Approach

1980s Bring Data to Compute

Now Bring Compute to Data

Relative size & complexity

Data Information-centric

businesses use all data:

Multi-structured, internal & external data

of all types

Compute

Compute

Compute

Process-centric businesses use:

• Structured data mainly • Internal data only • “Important” data only

Compute

Compute

Compute

Data

Data

Data

Data


The OLD Way: Bringing Data to Compute

ERP, CRM, RDBMS, Machines Files, Images, Video, Logs, Clickstreams External Data Sources

Data Archives EDWs Marts Search Servers Document Stores Storage

Complex Architecture • Many special-purpose

systems • Moving data around • No complete views

Visibility • Leaving data behind • Risk and compliance • High cost of storage

Time to Data • Up-front modeling • Transforms slow • Transforms lose data

Cost of Analytics • Existing systems strained • No agility • BI backlog


The NEW way: Bring Compute to the Data

HD

FS

Load data from all sources

Keep the history

Business needs for insights

Operations Marketing campaign

Finance

Distill data

Transactional systems

CRM PLM ERP Sensor Network Web Social Media Market Supplier

The “Business Datalake”


ISILON DATA LAKE (HADOOP) RDBMS

MACHINE IOT

STATISTICAL MODELING/NLP VISUALIZATION

TRANSFORM

BI

ORGANIZE MANAGE/ CATALOG

DATA WAREHOUSE STREAM CEP

NEAR REAL-TIME

MODELS MAY TAKE HOUR OR DAYS QUERIES MAY RETURN IN SECONDS OR MINUTES

SECONDS

SEARCH/INDEX

ENTERPRISE LOG ANALYSIS

APPLICATIONS

EMAIL

SOCIAL MEDIA SQL ON HADOOP

OUR BIG DATA LANDSCAPE

3rd PARTY

http://www.tpimagazine.com/siteimage/scale/800/600/353400.png

http://www.bizjournals.com/sanjose/blog/2012/06/san-joses-skytree-2-others-win-at.html?s=image_gallery

http://www.google.com/url?url=http://bigdatavendors.com/top.php&rct=j&frm=1&q=&esrc=s&sa=U&ei=G3U1VdaHDMueyATTk4GoDQ&ved=0CBgQ9QEwAQ&usg=AFQjCNGPF3lS-Cr1fgjnUGhiCaTZrf6Mqg

http://www.google.com/url?url=http://www.sitepoint.com/efficient-chinese-search-elasticsearch/&rct=j&frm=1&q=&esrc=s&sa=U&ei=UXU1Vf7EBI-gyQSz94CQAQ&ved=0CBoQ9QEwAQ&usg=AFQjCNFR3dtP-tI7IxXrTSGFmJdeJNe0Zg

http://www.google.com/url?url=http://pixshark.com/kibana.htm&rct=j&frm=1&q=&esrc=s&sa=U&ei=i3U1Va7LNMWXyATCroDoAg&ved=0CBwQ9QEwAw&usg=AFQjCNG3dAT7sE9gEILc-jRnfJLwJUeXWQ

http://www.google.com/url?url=http://hortonworks.com/blog/the-future-of-apache-storm/&rct=j&frm=1&q=&esrc=s&sa=U&ei=gHY1VcLYC4mPyASYjoHQAQ&ved=0CBwQ9QEwAw&usg=AFQjCNHpG5A9AuTmKKwvOqilSl1ocMQmFQ

http://www.google.com/url?url=http://www.hadoopsphere.com/2015/03/hadoop-high-5-with-ibms-anjul-bhambhri.html&rct=j&frm=1&q=&esrc=s&sa=U&ei=mnY1VamjCI2uyATWooCIAw&ved=0CB4Q9QEwBA&usg=AFQjCNEwNc97BzzIMTHPvCy419nHjKOLhg

http://www.google.com/url?url=http://spark-summit.org/east&rct=j&frm=1&q=&esrc=s&sa=U&ei=0HY1VfipH4n2yQTt6YC4Aw&ved=0CBgQ9QEwAA&usg=AFQjCNG_COnNEYyI9VRZC4SyqIc9l96Mrg

http://www.google.com/url?url=http://altaplana.com/clients.html&rct=j&frm=1&q=&esrc=s&sa=U&ei=-XY1VdXhJoKzyQTW_IGwCg&ved=0CBYQ9QEwAA&usg=AFQjCNF9_URNTrV0E64PWy3S79Ym5Rtw7A

http://www.google.com/url?url=http://en.wikipedia.org/wiki/Greenplum&rct=j&frm=1&q=&esrc=s&sa=U&ei=MOs4VevPJJbfsASAmIHoBg&ved=0CDwQ9QEwEw&usg=AFQjCNEO7zx6us020SM17YnIuR5LKppk2Q

http://www.google.com/url?url=http://www.oracle.com/technetwork/database/exadata/overview/index.html&rct=j&frm=1&q=&esrc=s&sa=U&ei=hus4VeynIriKsQSF6oCAAg&ved=0CBYQ9QEwAA&usg=AFQjCNGNRS-bjm4SUXibEDLFk9H5RhmgHg

http://www.google.com/url?url=http://logo-kid.com/pivotal-hd-logo.htm&rct=j&frm=1&q=&esrc=s&sa=U&ei=n-w4VcvYB4y1sATO7YGoBg&ved=0CBYQ9QEwAA&usg=AFQjCNEeQD_s76YWxnq931MgAcQ-F4RZAg

http://www.google.com/url?url=http://www.cloudera.com/content/cloudera/en/campaign/introducing-impala.html&rct=j&frm=1&q=&esrc=s&sa=U&ei=_e04VeSTLc61sATfrYHYCw&ved=0CBYQ9QEwAA&usg=AFQjCNF28VkVUrV07WSP2qs8S0vERAlC5w

http://www.google.com/url?url=http://hortonworks.com/hadoop/hive/&rct=j&frm=1&q=&esrc=s&sa=U&ei=Oe44VbqmF-u1sATpnIBA&ved=0CBYQ9QEwAA&usg=AFQjCNGhlkNn-JB-ODmCFyT-65304tlSiA

https://www.google.com/url?url=https://hadoopi.wordpress.com/2014/10/25/use-spark-sql-on-sql-developer/&rct=j&frm=1&q=&esrc=s&sa=U&ei=Ze44VeatNteHsQS3voDoBQ&ved=0CBYQ9QEwAA&usg=AFQjCNEZIaFwJhhkthDwP4wcHl-y_k5hQg

http://www.google.com/url?url=http://prestodb.io/&rct=j&frm=1&q=&esrc=s&sa=U&ei=Du84VYWhJuS1sAS8k4DgBQ&ved=0CBYQ9QEwAA&usg=AFQjCNHXjNHV9FjsbsPTTOXu6FbrKW-CTg

https://www.google.com/url?url=https://gigaom.com/2014/09/16/sql-on-hadoop-tech-apache-drill-is-ready-to-use-and-part-of-maprs-distro/&rct=j&frm=1&q=&esrc=s&sa=U&ei=c_E4Vaj7G6K1sATfh4HAAw&ved=0CBYQ9QEwAA&usg=AFQjCNHepqA2fhbLGHvLuT9n6-Yfe8PCiw


Biz Apps

Priorities

Services

Deploy

Compute

Dell EMC

portfolio

Knowledge

Ready Nodes

Ready Bundles

Ready Systems

The Ready Solutions formula


Dell EMC Ready Bundles for Hadoop

End-to-end data

management,

processing and

analytics with

„no-code-needed‟

deployment plus

high double-digit

performance gains

Dell EMC Ready Bundle for Cloudera Hadoop

Capacity optimized

and efficient data

processing and

data lake platform

leveraging Isilon

for shared storage

for Hadoop with

HDFS as a

protocol.

Dell EMC Ready Bundle for Hortonworks with

Isilon

Open Source data

management,

processing and

analytics solution

that efficiently

process multi-

structured data

volumes using

existing tools and

resources

Dell EMC Ready Bundle for

Hortonworks Hadoop

Unlocking data insights for up to 50% better revenue growth

Dell EMC

differentiation

Fully validated and certified

configurations

Reduced Splunk OpEx via

HCI options that eliminate

clustered management

overhead and administration

Rapid, worry-free Hadoop

deployments with exclusive

„no code needed‟ tools

Full-lifecycle, outcome-

focused services Big Data Vision • Accelerator •

Health Check • Deployment •

Support • Financial

Cloudera Hadoop • Splunk • Hortonworks Hadoop

Over 350 customers

for every $1.00 ROI in 6 months

$1.97

Lower data analytics costs

76% ETL time reduction

60%

1

TPC-x Big Bench Scale Factor 10000

Price/Performance

May 2017

“We went from 40,000 transactions per

second to 150,000 transactions per

second after we implemented the new

Dell PowerEdge Servers

Jim Bachesta, VP • Audience Science

“We‟ve completely redesigned how we

capture, store and provision data with the

new Dell Hadoop cluster. We can gather

larger amounts of data, and our analysts

and statisticians can mine that data in

ways they couldn‟t before.”

Tony Giordano, EVP • Merkle

Dell EMC Ready Solutions for Big Data

Michael Heuer Senior Specialist Systems Engineer€¦ · Support • Financial Cloudera Hadoop •...

Documents

Transcript of Michael Heuer Senior Specialist Systems Engineer€¦ · Support • Financial Cloudera Hadoop •...