Data and its Role in Your Digital Transformation
-
Upload
pivotal -
Category
Technology
-
view
77 -
download
1
Transcript of Data and its Role in Your Digital Transformation
![Page 1: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/1.jpg)
![Page 2: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/2.jpg)
The Role of Data in Your Digital TransformationLes KleinField CTO DataPivotal
@LesKlein #PivotalForum #DigitalTransformation #Istanbul #BigData #Analytics
![Page 3: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/3.jpg)
Great software companies leverage Big Data
to fundamentally change the consumer experience and pioneer entirely new business models
![Page 4: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/4.jpg)
4© 2016 Pivotal Software, Inc. All rights reserved.
$4BN
Financial Services
$26BN
Hospitality
$50BN
Transportation
$54BN
Entertainment
$30BN
Automotive
$3.2BN
Industrial Products
CLOUD NATIVE SOFTWARE IS CHANGING INDUSTRIES
Data is Fueling Software
![Page 5: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/5.jpg)
5© Copyright 2015 Pivotal. All rights reserved.
Hundreds of thousands of “trip” events each day
400+ billion of viewing-related events per day
Five billion training data
points for Price Tip feature
Disruptors Use a LOT of Data
![Page 6: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/6.jpg)
6© Copyright 2015 Pivotal. All rights reserved.
“We’ve found that when a host selects a price that’s
within 5% of their tip, they’re nearly 4 times
more likely to get booked”
“The importance of accuracy and efficiency […], will continue to
rise as we expand and improve products like uberPOOL and
beyond.”
“Over 75% of what people watch come
from our recommendations”
Data manifests as features in an app
![Page 7: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/7.jpg)
7© Copyright 2015 Pivotal. All rights reserved.
(Data)Microservices
Loosely coupled services architecture, bounded by context
Cloud-NativePlatforms
Enabling continuous delivery & automated
operations
Open SourceDatabase
Innovation
Extreme scale & performance advantages,
built for the cloud
Machine Learning
Use of predictive analytics to build
smart apps
How are they accomplishing this?
![Page 8: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/8.jpg)
8© Copyright 2015 Pivotal. All rights reserved.
These companies…
Release new features in minutes, multiple times a day
Support a micro-services architecture
Consume a wide range of data sources and protocols
Store and Analyze all their data
Update algorithms and predictive models daily
Continuously ask lots of questions of their data
Modify data pipelines and add processing steps daily
They are Data Driven, are you?
![Page 9: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/9.jpg)
9© Copyright 2015 Pivotal. All rights reserved.
A lot of reports do not make you data
drivenA lot of
dashboards do not make you data
driven A lot of alerts do not make you data
driven
![Page 10: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/10.jpg)
10© Copyright 2015 Pivotal. All rights reserved.
Converging Trends
Innovation
New Data New Processes New Insights
The Journey to the Data-Driven Enterprise
Data Scienceand Machine
LearningBig Data
Internet of Things &
Social Media
![Page 11: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/11.jpg)
11© Copyright 2015 Pivotal. All rights reserved.
HDFS
Data Lake
Ingest Store Analytics
Hard to changeLabor intensive
Inefficient
Coding basedNo real-time informationBased on expensive ETL
Migrating from a Reactive, Static and Constrained Model…
![Page 12: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/12.jpg)
12© Copyright 2015 Pivotal. All rights reserved.
HDFSData LakeExpert System /
Machine Learning
In-Memory Real-Time Data
Continuous LearningContinuous Improvement
Continuous Adapting
Data Stream Pipeline
Multiple Data SourcesReal-Time ProcessingStore Everything
To Pro-Active, Self-Improving, Machine Learning Systems
![Page 13: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/13.jpg)
13© Copyright 2015 Pivotal. All rights reserved.
“Companies need to learn how to catch
people or things in the act of doingsomething and affect the outcome“
PAUL MARITZExecutive Chairman, Pivotal
Real-time andPersonalised Informationin Context is what Wins!
![Page 14: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/14.jpg)
14© 2016 Pivotal Software, Inc. All rights reserved.
What does “Customer” mean to you?
![Page 15: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/15.jpg)
15© 2016 Pivotal Software, Inc. All rights reserved.
The new normal: “an audience of one”DATA DEVICES
Media
Banks
DeliveryServices
Marketers
Government
Individuals Employers
AnalyticServices
Advertising
CatalogCo-ops
ListBrokers
Websites
InformationBrokers
CreditBureausMedia
Archives
GOVERNMENTGOVERNMENT
PHONE/TV
PHONE/TV
INTERNETINTERNETAD
AGENCYAD
AGENCY
RETAILRETAIL
![Page 16: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/16.jpg)
16© Copyright 2015 Pivotal. All rights reserved.
![Page 17: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/17.jpg)
17© Copyright 2015 Pivotal. All rights reserved.
![Page 18: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/18.jpg)
18© 2016 Pivotal Software, Inc. All rights reserved.
Your customers are a new breed
![Page 19: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/19.jpg)
19© 2016 Pivotal Software, Inc. All rights reserved.
Evolving from digital natives to data natives
![Page 20: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/20.jpg)
20© 2016 Pivotal Software, Inc. All rights reserved.
“A data native is someone who expects their world to not just be digital,
but to be smart and to adjust immediately to their taste and habits.”
Beyond tech savvy …..
![Page 21: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/21.jpg)
21© 2016 Pivotal Software, Inc. All rights reserved.
Digital natives program their thermostat
![Page 22: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/22.jpg)
22© 2016 Pivotal Software, Inc. All rights reserved.
Data natives expect the thermostat to program itself
![Page 23: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/23.jpg)
23© 2016 Pivotal Software, Inc. All rights reserved.
Do you know the full value of your data and how to leverage it?
![Page 24: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/24.jpg)
24© 2016 Pivotal Software, Inc. All rights reserved.
µs ms s hour day month yr+
“Fast Data” “Big Data”
year
TraditionalSystems
Pivotal Data Science Labs
Value ofData ($)
Time
Value of data over time…
![Page 25: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/25.jpg)
25© 2016 Pivotal Software, Inc. All rights reserved.
Analytics - the journey so far (BI)
Complexity
Value ofAnalytics ($)
Descriptive Analytics
DiagnosticAnalytics
Whathappened?
Why did ithappen?
Hindsight
Insight
![Page 26: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/26.jpg)
26© 2016 Pivotal Software, Inc. All rights reserved.
Analytics – where you need to be today
Value ofAnalytics ($)
Descriptive Analytics
DiagnosticAnalytics
PredictiveAnalytics
PrescriptiveAnalytics
Whathappened?
Why did ithappen?
What willhappen?
How can wemake it happen?
Hindsight
Insight
Foresight
Complexity
![Page 27: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/27.jpg)
27© 2016 Pivotal Software, Inc. All rights reserved.
Interesting questions are found here
Value ofAnalytics ($)
Descriptive Analytics
DiagnosticAnalytics
PredictiveAnalytics
PrescriptiveAnalytics
Whathappened?
Why did ithappen?
What willhappen?
How can wemake it happen?
Hindsight
Insight
Foresight
Complexity
Data science
![Page 28: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/28.jpg)
28© 2016 Pivotal Software, Inc. All rights reserved.Data Science at Pivotal
Theory-driven approach
‘start with the problemand work towards the data’
‘start with the data and work towards the problem’
Data-driven approach
![Page 29: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/29.jpg)
29© 2016 Pivotal Software, Inc. All rights reserved.Data Science at Pivotal
‘start with the data and work towards the problem’
Data-driven approach
![Page 30: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/30.jpg)
30© 2016 Pivotal Software, Inc. All rights reserved.
User Feedback
Scoping Data Review
Model Building
Feature Review
Feature Engineering
Model Evaluation
Operationalization
Our tried and tested approach helps extract value from data
![Page 31: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/31.jpg)
“THE MAGIC HAPPENS WHEN YOU MARRY THE
TRADITIONAL ENGINEERING APPROACH WITH THE DATA SCIENCE ENABLED BY THE DATA LAKE. IT OPENS UP A WHOLE NEW WORLD OF POSSIBLE
‘WHAT IF’ QUESTIONS.
”DAVE BARTLETT, GE AVIATION
![Page 32: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/32.jpg)
32© 2016 Pivotal Software, Inc. All rights reserved.
IT’S ALL ABOUTHOW YOU
OPERATIONALISE YOUR INSIGHTS
![Page 33: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/33.jpg)
33© 2016 Pivotal Software, Inc. All rights reserved.
Scale-out analytic
database
Model API Cloud Native Application
Platform
Data Sources
0 5
Smart Apps: Models Manifesting as Microservices
![Page 34: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/34.jpg)
34© 2016 Pivotal Software, Inc. All rights reserved.
OSSSmart/Data-driven App Technology Needs
Fast Ingest/Pipelining
• Pipelines to consume streaming and batch data from various endpoints
Speed/Serving Layer
• In-memory data grids for real-time and aggregated data sets
• Low latency
Big Data Analytics Platform
• Scale-out Storage• Ability to process large structured and unstructured data• Advanced SQL query capabilities• Ability to run machine learning at scale
Platform for ScalableWeb/Mobile Apps
• Infrastructure Automation
• Develop, run and manage Web applications without the complexity of building and maintaining the infrastructure
![Page 35: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/35.jpg)
35© 2016 Pivotal Software, Inc. All rights reserved.
Fast Ingest/Pipelining Speed/Serving Layer
Big Data Analytics Platform
Platform for ScalableWeb/Mobile Apps
Pivotal’s Answer for Technology Needs
Spring Cloud Data Flow GemFire
GreenplumDatabase
Pivotal HDB
Pivotal Cloud Foundry
Spring
OSS
![Page 36: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/36.jpg)
36© 2016 Pivotal Software, Inc. All rights reserved.
● Silo’d and aging database systems
● Spaghetti data pipelines
● Expensive, proprietary data management systems
● Lack of structured platforms for continuous software delivery
● Monolithic application architectures
● Batch-oriented data integration
● Limited operationalization of analytics
● Proprietary systems
Today’s Enterprise Challenge
![Page 37: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/37.jpg)
37© 2016 Pivotal Software, Inc. All rights reserved.
Data Programming Model
Cloud-Native Platform
Microservices FrameworkPlatform Runtime
Stream Data Platform
Hadoop Spark DW
Apps & Microservices
DBMS
IMDG
K/V Store
Relational DB
Big Data & Machine Learning
Modern Cloud-Native Data Architecture
![Page 38: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/38.jpg)
38© 2016 Pivotal Software, Inc. All rights reserved.
1. Modernize your data platforms:
• Improve performance and scalability for mission-critical data workloads
• Lower overall TCO by adopting industry-leading open source database technologies on modern cloud platforms
2. Prove out a specific business use-case:
• Build a smart app end-to-end
• Focus on a specific analytical model / data-microservice
• Automate ingestion and data pipelines
Pivotal can help you to:
![Page 39: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/39.jpg)
39© 2016 Pivotal Software, Inc. All rights reserved.
Pivotal Offerings
Big Data Suite:
• Best-in-class open source data management software
• GemFire: In-Memory Data Grid
• Greenplum: Data Warehouse
• Pivotal HDB (Apache HAWQ): Hadoop-Native SQL
Pivotal Cloud Foundry
• Industry’s Leading Cloud-Native Platform
Spring Boot, Cloud and Data Flow
• Modern-Java micro-services framework
Pivotal Labs & Data Science
• Agile software development and machine learning
![Page 40: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/40.jpg)
Let’s build something
MEANINGFULLet’s build something
MEANINGFUL
![Page 41: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/41.jpg)
41© 2016 Pivotal Software, Inc. All rights reserved.
AP
PA Real-World Example
![Page 42: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/42.jpg)
42© 2016 Pivotal Software, Inc. All rights reserved.
AppDevelopment
Data analytics
Cloud-nativeApp platform
Data Science & Model building
DataMicro-service
AP
PA Real-World Example
![Page 43: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/43.jpg)
43© 2016 Pivotal Software, Inc. All rights reserved.
AppDevelopment
Data analytics
Cloud-nativeApp platform
Data Science & Model building
DataMicro-service
AP
P Must support scale-out query processing
Must support scale-out query processing
Must deliver as an API Must deliver as an API
Must embrace agile development, focus on outcomes
Must embrace agile development, focus on outcomes
Must support microservices, agile dev, and connect to big data analytics
Must support microservices, agile dev, and connect to big data analytics
A Real-World Example
![Page 44: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/44.jpg)
44© 2016 Pivotal Software, Inc. All rights reserved.
Major Credit Card Issuer Builds Infrastructure Determining Fraud Markers Across 30 Disparate Sources
Challenge:• Desire to identify potential fraudulent activity within Fortune
100 credit card issuer with 100K employees• Identifying markers across 30 internal data sources caused
lags in reporting• Establish Infrastructure to support Palantir, Platfora and RSA
reporting frameworks
Solution:• Provided ability to store 3 months of data at 60-70TB, interest
in moving to 3-year retention period• Build large-scale security analytics platform to scale with data
needs, analytics and reporting requirements• Support organizational desire to evolve from pure BI to
advanced, predictive analytics
Pivotal Solution includes: GPDB, Hadoop, HAWQ
![Page 45: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/45.jpg)
45© 2016 Pivotal Software, Inc. All rights reserved.
Leading Swiss Bank Re-shapes Business to Better Monetize and Regulate Fraudulent Activity with Big Data
Challenge:• Needed to monetize data around privilege user access and
assure security around fraudulent activity • Reduce the cost of monitoring privileged user access for
regulatory requirements. (currently taking 45 people to do this)• Extend across the enterprise to solve business pains around
faster data as opposed to just longer looks in “Hadoop”
Solution:• Built on top of Big Data Suite to operationalize, automate and
extend across the enterprise • What took 6 hours should now take less than 1 hour with real
time decision-making• Capability of monitoring relationship manager’s activity with
analytics to ensure regulatory compliance
Pivotal Solution includes: Big Data Suite and Data Labs
![Page 46: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/46.jpg)
46© 2016 Pivotal Software, Inc. All rights reserved.
Pivotal Data Innovation Across Industries
RETAIL / TRANSPORT
MEDIA / SERVICES
TELCO / ISP
FINANCIAL SERVICES
HIGH TECH / ELECTRONICS
EDUCATION
GOVERNMENT
OTHER
INSURANCE
AUTO / INDUSTRIAL
HEALTHCARE
MANUFACTURING
![Page 47: Data and its Role in Your Digital Transformation](https://reader031.fdocuments.in/reader031/viewer/2022030305/5873d3301a28ab9d168b69b3/html5/thumbnails/47.jpg)