Demystify Big Data Breakfast Briefing: Martha Bennett, Forrester

Big Data – What’s Hype, What’s Reality?

Martha Bennett Principal Analyst

Hortonworks Breakfast Seminar

London, July 9th, 2013

A review of trends and developments

© 2013 Forrester Research, Inc. Reproduction Prohibited 2

We’re in a data-driven world

Firms recognize the importance of data . . .

Implement a bring-your-own PC, smartphone, and/or tablet strategy

Create a comprehensive mobile and tablet strategy for employees

Shift spending from core systems to applications driving engagement with customers

Create a comprehensive cloud strategy

Develop smart product APIs that improve product & service capabilities

Create a comprehensive mobile and tablet strategy for customers or business partners

Cut overall IT costs due to economic conditions

Reorganize or retrain IT to better align with business outcomes and drive innovation

Help the organization better manage and integrate its partners and suppliers

Improve IT budget performance

Develop new skills to better support emerging technologies and business innovation

Improve IT project delivery performance

Improve the use of data and analytics to improve business decisions and outcomes

13%

17%

18%

18%

19%

19%

20%

26%

27%

27%

28%

32%

37%

4%

6%

6%

7%

7%

8%

7%

7%

8%

8%

9%

11%

18%

High priority

Critical priority

Source: Forrsights Business Decision-Makers Survey, Q4 2012

Base: 3,616 business decision-makers from firms with 100 or more employees

1

2

3

4

5

6

7

. . . and BI is a top investment priority . . .The top seven software applications in firms’ adoption plans by year

Source: Enterprise and SMB Software Survey, North American And Europe, Q3 2007; Enterprise And SMB Software Survey, North America And Europe, Q4, 2008; Enterprise And SMB Software Survey, North American And Europe, Q4 2009; Forrsights Software Survey, Q4 2010; Forrsights Software Survey, Q4 2011; and Forrsights Software Survey, Q4 2012Note: We first included industry-specific software in the Q4 2008 survey; we first included finance and accounting in the Q4 2009 survey.

2008(N = 1,158)

2009(N = 1,021)

2010(N = 455)

2011(N = 913)

2012(N = 1,092)

2013(N = 1,631)

Source: May 27, 2011 , “Forrsights: The Software Market In Transformation, 2011 And Beyond” Forrester report

Business intelligence

Customer relationship management

Collaboration software

Finance & accounting

Industry-specific software

Enterprise resource planning

Human capital management

. . . but they don’t use most of their data

Source: Forrsights Strategy Spotlight: Business Intelligence And Big Data, Q4 2012

Unstructured50TB

Semi-structured

2 TB

Structured12 TB

Utilized

12%

Average data volume per company

9 TB 75 TB

0.6 TB 5 TB

4 TB 50 TB

SMBs: LEs:

Base: 634 business intelligence users and planners


Data sources continue to multiply


Most BI remains backward-looking

Source: September 20, 2011, “Understanding The Business Intelligence Growth Opportunity” Forrester report

Information about


Fundamental shifts in BI and analytics

› The “Google Effect”

› The “good enough” principle

› Self-service tools

› Data visualization

› Predictive analytics

› Big data

› Mobile

› Cloud-based delivery models

“Big data” is:

Techniques and technologies

that make handling data at

extreme scale affordable.

“Big data” is:

Techniques and technologies

that make handling data at

extreme scale affordable.

Several different technologies!

Many different use cases!

Extreme in different dimensions!

What is “extreme scale” in big data?

VOLUMEVELOCITYVARIETY

VARIABILITYVALUE

VERACITYVISCOSITYVIRALITYVITALITY

VISIBILITYVAPOROUS

VALIDITYVIABILITY


Assessing the need for big data tech

DRAFT – WORK IN PROGRESS

Machine-generated, e.g.:Transaction dataCall recordsSmart meter dataLocation informationSystem or web log files

Human-generated, e.g.:TweetsFacebook activityEmailBlog postsPictures, videos

High

LowHigh

Deg

ree

of

stru

ctu

re

Percentage of data elements (potentially) of value

Traditional BI and data warehousing

systems

Content and document management systems

Various content types (doc, xls, ppt, bmp) on PCs and servers; email


Assessing the need for big data tech

DRAFT – WORK IN PROGRESS

Machine-generated, e.g.:Transaction dataCall recordsSmart meter dataLocation informationSystem or web log files

Human-generated, e.g.:TweetsFacebook activityEmailBlog postsPictures, videos

High

LowHigh

Deg

ree

of

stru

ctu

re

Percentage of data elements (potentially) of value

Traditional BI and data warehousing

systems

Content and document management systems

Various content types (doc, xls, ppt, bmp) on PCs and servers; email

What is the question?

What problem are you trying to solve?


There is no single “big data” technology

› Core Hadoop

› The wider Hadoop ecosystem

› Appliances

› Preconfigured/preintegrated systems

› Streaming technologies

› In-memory processing/analytics

› Advances in semantic technologies/NPL

› Cloud-based services

7% 13% 7% 17% 31%

Implemented, not expanding Expanding/upgrading implementation

Planning to implement in the next 12 months Planning to implement in more than 1 year

Interested but no plans

Base: 634 business intelligence users and planners

“What best describes your firm's current usage/plans to adopt big data technologies and solutions?”

Source: Forrsights Strategy Spotlight: Business Intelligence And Big Data, Q4 2012

Big data analytics is growing quickly

20% have implemented

some big data technology

37% are planning a big data technology project in 2013 or beyond

Production Logistics Sales ServiceSourcing

(Singapore bank) high-performance risk

analysis [SAS]. 45,000 instruments with 100,000 parameters: 8.8 billion risks analyzed in less than 1minute (down from 18 hours), aggregated risk portfolio. Upfront strategy evaluation.

(Retailer) price optimization [SAS].

Based on sales and competition -> 270 million price calculations in less than 2 hours (down from 30 hours); now several price changes per day.

(Telecom) churn/ loyalty management

[HP]. Call analysis (more than 500 million/day) combined with social media analysis to assign risk scores to business lines and individual customers.

(Bus service) carrier service optimization

[Fujitsu]. 200,000 input/output operations/second. Response <1 ms: status, position, ETA, consumption, compliance -> all real-time

(Semiconductor) manufacturing

optimization [Exasol]. 5 billion data points for production processes, material, movements, product per production cycle -> monitoring, archiving, comparison, optimization.

(Retailer) workforce scheduling and optimization [Blue-Yonder]. Predictive analysis (450,000 /week) based on sales, weather, traffic -> improved employee/customer satisfaction

(Retailer) inventory optimization

[BlueYonder] Based on weekly sales forecast (135 GB), 300 million data sets (sales, campaigns, products), improved forecast 40% (1 billion/year), real-time

Royal Tech Institute Stockholm [IBM] optimized traffic management. Real-time 250,000 GPS/s (signals) -> 20% less traffic/emissions, 50% shorter trips

High-performance computing for drilling site evaluation [IBM,summer 2010]. 50 TB

per survey. Increased success rate from 1 in 5 to 1 in 3.


Key lessons learned to date

› Skills requirements are often underestimated• BI or research project?

› Many of the emerging tools and technologies

aren’t yet enterprise-grade• Lack of management features and security

› The security and privacy implications are far-

reaching• We’re in uncharted territory, from an ethical as well

as a legal perspective


Never lose sight of the fundamentals

› Always question the source of the data• You may find it’s biased

› Skepticism is as important as statistical skills• The numbers may be telling the wrong story

› Data sets may require specialist expertise• Errors can be very costly

› Data scientists aren’t miracle workers• All findings need business context

› Legal constraints apply, particularly in Europe• A risk-based approach will be key


Make sure the foundations are in place› A close working partnership between business

and IT is not an optional extra• Focus on alignment of goals and objectives

› Work towards business ownership of data • Don’t allow data quality to be made a pure IT issue

› Consider projects within IT as well• A good way to gain familiarity and expertise

› Make room for big data technologies and

techniques in your BI Center of Excellence• No lengthy evaluations or pilots – try it, move on


?Questions

Thank youMartha Bennett

+44 (0) 20 7323 7674

[email protected]

Twitter: @martha_bennett

Blog: http://blogs.forrester.com/martha_bennett

mailto:[email protected]

http://blogs.forrester.com/martha_bennett

Demystify Big Data Breakfast Briefing: Martha Bennett, Forrester

Technology

Transcript of Demystify Big Data Breakfast Briefing: Martha Bennett, Forrester