Forecasting & Big Data Analytics - JPK Group€¦ · BI and Advanced Analytics are only part of the...
Transcript of Forecasting & Big Data Analytics - JPK Group€¦ · BI and Advanced Analytics are only part of the...
JPK
Gro
up
June 14-15 • Boston, MA
Forecasting & Big Data AnalyticsUsing Big Data to Enhance Forecasting and Planning
June 14, 8:30am
View presentation online at:
http://www.jpkgroupsummits.com/attendee3
Paul Legutko – Senior Manager, Enterprise
Intelligence/Digital Analytics at Ernst and Young
Business Forecasting and Analytics Forum
Paul Legutko is a leading expert in digital measurement and analytics and a senior manager
in EY’s Analytics practice. Over the last twelve years as a digital analytics consultant, Paul
has seen the field evolve from raw server log files, web analytics, mobile and social
measurement, and into today’s world of Big Data integration. Working with clients from
many industries – particularly Financial Services, Media & Entertainment, and Pharma --
Paul has pioneered the application of new solutions for understanding customer digital
behavior, digital marketing effectiveness, and how to integrate offline and online data.
Page 1 Forecasting & Big Data
Forecasting & Big
Data Analytics
Using Big Data to Enhance
Forecasting and Planning
with Paul Legutko
Page 2 Forecasting & Big Data
But do you know what big data is?
You know what forecasting is…
Page 3 Forecasting & Big Data
Most people think of it like this…
► It sounds plausible, but doesn’t it make big data sound like
just more of the same?
Big data “Not so big” data
Volume Terabytes/petabytes Megabytes/gigabytes
Variety Unstructured (text, voice, video) Structured/relational
Velocity Data in motion (streaming) Data at rest
Veracity Untrusted/uncleansed Trusted/cleansed
Page 4 Forecasting & Big Data
Traditional data and the Four V’s
Traditional data
(however large, fast, inaccurate or various)
Account
Date Address Name Purchases
Status Opt-in SpendLast
Activity
It doesn’t matter
how large this
gets
It doesn’t matter
how many sources
you have
It doesn’t matter
how fast these
change
Traditional database, reporting and analysis tools can handle it
Page 5 Forecasting & Big Data
But knock things down a level …
Big data
(however small, at rest, accurate or singular)
Account
Call OrderDirect
ContactWeb Visit
And you
introduce three
critical new
dimensions
Sequence
The order of events matters
Time
The time between events matters
Pattern
The pattern of events matters
Page 6 Forecasting & Big Data
The Sequence of events matters
This set of data
Forecast
SaleCallSaw ad Web visit
is completely different than this
Forecast an
Annoyed
Customer
CallWeb visitSale Saw ad
Page 7 Forecasting & Big Data
The time between events matters
These events in time
Forecast
Sale
CallSaw ad Web visit
Yield a different forecast than these events in time
May 3rd May 4th May 7th
Forecast
Unclear
Saw ad
Jan. 2nd
Web visit
Mar. 3rd
Call
July 12th
Page 8 Forecasting & Big Data
And the pattern of events matters
This consumption pattern
WeekendWeekend Weekday
Has the same average but a different forecast
High Use Low Use High Use
Weekday
Low Use
Weekend
Forecast
WeekendWeekend Weekday
Med. Use Med. Use Med. Use
Weekday
Med. Use
Forecast
Unclear
Page 9 Forecasting & Big Data
Traditionally, we’ve handled this by creating sub-forecasts
This consumption pattern
Weekend
High Use
Weekday
Low Use
Weekend
High Use
Weekday
Low Use
Weekend
Forecast
Weekday
Forecast
Page 10 Forecasting & Big Data
But that doesn’t work when patterns are complex or individual
$130
June
$142
July
$145
August
$158
Sep.
$167
Oct.
$187
Nov.
C
1ACCELERATING
$130
June
$82
July
$145
August
$25
Sep.
$135
Oct.
$15
Nov.
C
2 OSCILLATING
$130
June
$150
July
$145
August
$18
Sep.
$25
Oct.
$20
Nov.
C
3 SEASONAL
NO SINGLE AVERAGING STRATEGY WILL YIELD THE RIGHT ANSWER
Page 11 Forecasting & Big Data
It’s true in every paradigm case of big data
► Big Data shifts analytics to patterns in detail-level data.
Read meter
once a quarterUtilit
ies
Read meter
every 10
minutes
Aggregate
stream
Analyze the
flow of usage
by day/time
NewOld
Mass market
Reta
il
Cross channel
customer data
Customized
offers
Learn and
respond
Price by risk
‘strata’Auto
Insura
nce
Sensor data
in car
Analyze
driving
patterns
Price risk at
individual level
Page 12 Forecasting & Big Data
It matters because when sequence, time and pattern of events are important
Relational
data models
break down
SQL is extremely cumbersome and
almost impossible to use when querying
sequence, time between and patterns
Stats tools
are limited
Traditional analysis methods (like
regression and correlation) don’t work
directly
Integration
strategies
break
Traditional joins become multi-joins and
no longer facilitate useful integration,
even when keys are present
Page 13 Forecasting & Big Data
Categorizing Data Sources
Page 14 Forecasting & Big Data
So if you didn’t know…now you know
… so how do you forecast with Big Data?
What is big data
Page 15 Forecasting & Big Data
Three common big data scenarios
You have unstructured data (text, voice or video) from sources
like Twitter, Call-Center Notes, or published video content
1
You need to apply basic or even complex statistical analysis to
identify known patterns (improving, seasonal, etc.) at the
individual level before aggregating
2
You need to identify patterns in the data before you can
understand what it means
3
Page 16 Forecasting & Big Data
Structuring unstructured data
You have unstructured data (text, voice or video) from sources
like Twitter, Call-Center Notes, or published video content
1
Call-Center
Social media
Survey Research
Feedback
Offline research
Once text actions are
categorized they can be
used with traditional
forecasting techniques
Taxonomy
Sentiment
Linguistic Parsing
Machine Learning
Text-Analytics
Page 17 Forecasting & Big Data
Applying analytics to each record
You have structured data from sources. You need to apply basic
or even complex statistical analysis to identify known patterns
(improving, seasonal, etc.) at the individual level before
aggregating
2
Records
Clustering Momentum Pattern ID Deep LearningRegression
Spark or
Hadoop
Create an individual forecast for each record. Then aggregate traditionally.
Page 18 Forecasting & Big Data
Finding potential patterns in the data
You need to identify patterns in the data before you can
understand what it means
3
Activity 1 Activity 2
Use pattern matching techniques to spot and identify behavioral signatures and then categorize them for aggregation.
Page 19 Forecasting & Big Data
The Process has changed…
► Before Big Data
Customer Market Sales
Aggregate
Market Sales
Apply
Analytics
Market Forecast Sales
► Using Big Data
Customer Market Sales
Web Visit Survey Call
Apply
Analytics
Customer Market Sales
Velocity Sentiment Persona
Sale Probability
Aggregate
Market Forecast Sales
Page 20 Forecasting & Big Data
What do you need to do?
1
Big data isn’t just having a lot of data – big data problems occur when
the level of meaning resides above the detail level in the sequence,
order or pattern of the events.
2 When that happens, traditional IT techniques break down
3 And so do forecasting techniques
4To make big data useful, you need to add an extra step to your process
where you categorize the data in ways that allow it to be aggregated.
5
There are three common ways to do that: applying text analytics to
structure unstructured text, applying statistical analysis to identify
individual-level trends, and applying pattern-matching to categorize
behavior.
When you’ve done that, you can forecast with big data.
Page 21 Forecasting & Big Data
So… How do I do this?
Is the future of “Big Data”, “Big Judgment”?
Forecasting with Big Data
Page 22 Forecasting & Big Data
Predictive Analytics and Big Data
Big data and analytics represent a disruptive innovation for
business
Analytics is a strategy, talent, and operations topic, NOT just a
technology and data issue
The value of data – “Big” or “Not So Big” -- lies in the actions
businesses take from the insights obtained…analytics alone has no
value
There are hurdles to overcome to realize value …the hardest ones
are organizational and people related
Page 23 Forecasting & Big Data
There are many, many more tools available today than there were three years ago…
► http://mattturck.com/2016/02/01/big-data-landscape/
Page 24 Forecasting & Big Data
So the tools are there…But you need the right people to
• Ask the right questions• Interpret the data correctly
…This is Advanced Analytics
Forecasting with Big Data
Page 25 Forecasting & Big Data
Case Study
► Organization: The Lydian Empire, 6th century BC, under King Croesus.
► Research Question: “What will happen if Lydia invades Persian territory?”
► Solutions and Vendor Selection: Seven oracles in Africa and Greece.
► Controlled Experiment / POC: “What is King Croesus doing right now?” (answer: cooking tortoise
and lamb stew in a covered bronze cauldron)
► Vendor Selected: The Oracle of Delphi
► Fees: 2 silver & gold mixing bowls, gold statue of Croesus’ baker, 117 gold ingots, a gold lion, and the
Queen’s necklace and girdle.
► First Predictive Output: “If you invade into Persia, you will destroy a mighty empire.”
► Second Predictive Output: “Only if a mule is King of Persia will the operation go badly.”
► Action: Lydia invaded Persian Territory.
► Result: Lydian capital Sardis was sacked. Croesus’ fate is unclear.
Case Study
► Organization:
► Research Question:
► Solutions and Vendor Selection:
► Controlled Experiment / POC:
“________________________________________________________
► Vendor Selected:
► Fees:____________________________________________________________________________
___
► First Predictive Output:
► Second Predictive Output:
► Action:
► Result:
Predictive Analytics is not new
Page 26 Forecasting & Big Data
But more data requires more analyticsBI and Advanced Analytics are only part of the equation & the CFO has a critical role to play
Data
Ins
igh
t &
Ac
tio
n
BI
2nd Gen BI
Advanced Analytics
• You want to know how your campaign performed. You want to see how many widgets you produced and sold.
• Organizational weaknesses start appearing and potential opportunity is jeopardized
• Visualization becomes key to telling a story about what’s going on. Basic analytics are providing insights. Dashboards become more interactive.
• But, underlying data platforms and systems start breaking. Organizational silos start cracking – right people, right skills, right org model?
• Harder to ask the question – How is my business performing?
• Advanced analytics proactively providing insights and driving actions.
• Exploratory analytics becomes key combining advanced analytics and visualization to help you make money, save money and save lives.
• Can I be a real time enterprise?
Operational
(What Happened)
Operational + Digital
(What’s Going On)
All Data + Streaming
(What Should I Do)
Page 27 Forecasting & Big Data
The Role of the CFO is changing – why?Thinking about BI and Data Analytics
► There’s too much at stake for
traditional roles and silos to stand …
Priority Collaboration What’s at Stake
Managing Cybersecurity Health of the enterprise (Risk)
Creating analytics-driven org Being data driven; grow the Top Line
Establishing information strategy,
architecture
Ability to compete and improve
performance
Data
an
d A
na
lytics
Po
we
rTime
Data Acquisition
Data Visualization
Artificial Narrow Intelligence
Artificial General Intelligence
Today
Data Forecasting
Full Report: http://www.ey.com/GL/en/Issues/Managing-finance/EY-CFO-program-partnering-for-
performance-the-CFO-and-the-CEO-driving-and-enabling
Page 28 Forecasting & Big Data
And Organizations must evolve to be data-driven
Full Report: http://www.ey.com/Analytics
Strategy and leadership
► Key findings:
► Analytics is central to business strategy
► Leaders already gain competitive advantage from data and analytics
► Leading enterprises designate leaders to guide initiatives
Data analytics impact index 2015: The ‘Best’ versus the ‘Rest’
Page 29 Forecasting & Big Data
The human element of analyticsTechnical capability often exceeds behavioral ‘alignment’
Analytics Maturity Drivers:
► Strategy and Leadership
► Adoption
► Strategic Alignment
► Competitive Differentiation
► Analytics Production
► Technology
► People Analytics Skills
► Data Management
► Analytics Consumption – Organization
► Governance
► Culture
► Change Management
► Analytics Consumption – Individual
► Decision Making
► Insights to Action
► Value Measurement
Technical
Capability Gap
Value
Creation
Danger
Zone
Behavioral
Alignment Gap
Low
Hig
h
► Culture and
mental models
► Organization and
process design
► Learning and
development
► Incentives /
rewards
Behavioral Alignment
(Analytics Consumption)
Low High
► Data quality
► Infrastructure and tools
► Data science
Technical Capability
(Analytics Production)
Page 30 Forecasting & Big Data
Example: Analytics Visualization and ReportingThe goal of analytics is to democratize knowledge, not data.
Analysts: Understand
what business
questions reporting is
supposed to answer,
and develop the
metrics and KPI’s that
provides this insight.
Modelers: Combine
historical and current
reporting with
forecasting and what-if
capabilities that
transform them from
reports to tools for
running the business.
Architects: Engineer the automation of reporting derived directly from
data systems, using whatever visualization and BI tools make most sense
for the organization.
Visualization Design Skill Domains
► Strategy
► Industry knowledge
► Operations
Business Analyst Skill Domains
► Strategy
► Accounting
► Finance
► Economics
► Marketing
Modeler Skill Domains
► Data Modeling
► Business Intelligence
► Advanced Analytics
► Statistics
► Visualization
Architect Skill Domains
► Data Management
► Data Warehouse
► BI Platform Implementation
► Data Integration and Connectivity
Designers: Condense
metrics and KPIs into
visually appealing, rich,
easily comprehensible
and interactive
dashboards and reports.
Every Forecast is a Collaborative Effort… And they are all analytics professionals
Page 31 Forecasting & Big Data
• Big Data isn’t More Data, it’s Different Data, requiring new methods, tools, and skills to activate.
• Using Big Data Analytics requires• The right tools• The right people• The right culture
Wrapping Up
EY | Assurance | Tax | Transactions | Advisory
About EY
EY is a global leader in assurance, tax, transaction and advisory
services. The insights and quality services we deliver help build trust
and confidence in the capital markets and in economies the world
over. We develop outstanding leaders who team to deliver on our
promises to all of our stakeholders. In so doing, we play a critical
role in building a better working world for our people, for our
clients and for our communities.
EY refers to the global organization, and may refer to one or more,
of the member firms of Ernst & Young Global Limited, each of
which is a separate legal entity. Ernst & Young Global Limited, a UK
company limited by guarantee, does not provide services to clients.
For more information about our organization, please visit ey.com.
Ernst & Young LLP is a client-serving member firm of Ernst & Young
Global Limited operating in the US.
© 2013 Ernst & Young LLP.
All Rights Reserved.
1309-1135041