Table of contentsbarc-research.com/wp-content/uploads/2015/10/bis15-free... · 2015-10-19 · Table...

THE BI Survey 15 – The Results

- 2 -

Table of contents

The Sample ............................................................................................................................................. 3

Sample size and make-up ................................................................................................................... 3

Geographical distribution ..................................................................................................................... 4

Organization sizes by headcount ......................................................................................................... 4

Vertical markets ................................................................................................................................... 6

Featured products ................................................................................................................................... 7

Deployment............................................................................................................................................ 10

Percentage of BI Users in Company ................................................................................................. 10

Completion within Original Budget by Product .................................................................................. 11

Number of Users by Product ............................................................................................................. 14

Percentage of Employees Using the BI Product................................................................................ 16

Departments Using BI ........................................................................................................................ 18

Challenges ............................................................................................................................................. 19

Implementation Problems by Peer Group .......................................................................................... 19

Implementation Problems by Industry ............................................................................................... 20

Most Serious Implementation Problems ............................................................................................ 22

Most Serious Problems Encountered during Use by Business Users ............................................... 23

What Problems Do Customers Typically Encounter? ........................................................................ 24

Reasons for Replacement ................................................................................................................. 25

Reasons for Replacement by Region ................................................................................................ 26

Next Steps ............................................................................................................................................. 27

The BI Survey Analyzer Demo .......................................................................................................... 27

Purchase ............................................................................................................................................ 27


- 3 -

The Sample

Most surveys are conducted or sponsored by an organization based in, and focused on, one country.

However, BI is a worldwide market and we wanted, as far as possible, to capture a large international

sample. This not only presents a more accurate global picture, but also allows international variation to

be analyzed.

The three largest BI markets are the United States, Germany and the United Kingdom, so The BI Survey

15 was produced as a collaboration between organizations in each of these countries, and in partnership

with publishers and vendors in these and other countries. It features not just the well-known US products,

but also products from other regions including Europe and Australia.

Figure 1: Respondents analyzed by region (n=2971)

The net result was an extraordinarily international panel. Respondents were located in 90 countries.

Five countries had 100 or more respondents, and twelve had 50 or more; 29 countries had ten or more

respondents.

Sample size and make-up

Hundreds of thousands of people around the world were invited to participate in The BI Survey 15, using

dozens of email lists, magazines and Web sites. As in previous years, the questionnaire offered different

sets of questions for vendors and users (or consultants answering on behalf of a user). This seems to

produce better quality data as in the past some vendors pretended to be users when they saw they were

not being asked relevant questions.

Participants from last year who indicated that they would like to be part of our panel received a pre-filled

questionnaire with answers from last year’s questions that had remained the same. They were asked to

update their responses, and then to answer the new questions in this year’s Survey.

The results of the online data collected are shown in the following chart, with the numbers of responses

removed also displayed.

Responses

Total responses 3,267 100%

Filtered during data cleansing -196 -6%

Remaining after data cleansing 3,071 94%

Not yet considered buying -119 -4%

Total answering questions 2,952 90%

Figure 2: Responses removed from the samples

61%

24%

10%

3%

1%

Europe

North America

Asia and Pacific

South America

Rest of world


- 4 -

The number of responses is split between users, consultants and vendors. Vendors answered a different

set of questions to those answered by end users. This document focuses on the analysis of the user

results.

Responses

Users 2,110 72%

Consultants 367 12%

All users 2,477 84%

Vendors/Resellers 475 16%

Figure 3: Total responses analyzed in The BI Survey 15

Geographical distribution

One of the key objectives of The Survey is to achieve a geographically balanced sample that reflects

the current global market for BI products. Therefore the online questionnaire was published in four

languages: English, German, French and Spanish.

Having a geographically balanced sample has two major benefits:

Firstly, results of The Survey are more closely representative of the world market, rather than being

largely based on US experience, as is the case with many other surveys.

In regions where knowledge of English is sparse, such as South America and much of Asia and southern

Europe, it is difficult to obtain a good level of feedback and the BI market is less mature in these

countries. Since the fourth edition of The BI Survey, we have significantly boosted the German sample

by specifically targeting users in German-speaking countries, using a fully translated online

questionnaire. A Spanish language questionnaire was included to boost responses from Spain and Latin

America. We also used a French questionnaire, further increasing our European coverage.

Organization sizes by headcount

BI products are most commonly found in large organizations and a high percentage of the responses

we receive are from users in companies with more than 2,500 employees. Nevertheless, responses

from small organizations have been catching up over the years.

The split between respondents from small and large enterprises is well-balanced this year.

Figure 4: Frequency of employee count in respondent organization (n=2297)

32%

53%

14%

More than 2,500

100 - 2,500

Less than 100


- 5 -

The following chart shows the median headcount of respondents’ companies analyzed by the product

they reported on. Of the products defined in the ‘Large international vendors’ and ‘Large deployments’

peer groups there was a higher median number of employees in customer organizations than the sample

average.

Figure 5: Median employee count of user organizations analyzed by product

7500

5000

4200

4100

4000

3500

3000

2750

2500

2350

2000

1500

1500

1000

1000

1000

903

800

800

775

725

720

650

535

528

500

500

500

500

490

400

300

300

160

67.5

60

SAP BO Analysis

SAP BO WebI

Oracle BI FS

SAP BEx

IBM Cognos BI

Information Builders

SAS

SAP BO Design St.

MicroStrategy

arcplan

IBM Cognos TM1

cubus

Tableau

Infor

MS SSRS

Average of all products

Qlik

BOARD

Dimensional Insight

MS Excel & Power pro.

Pyramid Analytics

prevero

Cubeware

TARGIT

Cyberscience

Bissantz

CALUMO

Dundas

Entrinsik

Logi Analytics

Jedox

Corporate Planning

evidanza

Phocas

Yellowfin

Chartio


- 6 -

Vertical markets

We asked all respondents their company’s industry sector. The chart below shows the results of this

question and only includes data from respondents who answered product-related questions in The

Survey.

Manufacturing dominates the list, as it has in previous years.

Figure 6: Responses analyzed by industry (n=2543)

16%

13%

8%

7%

7%

6%

6%

6%

5%

5%

4%

3%

3%

3%

3%

2%

2%

1%

1%

Manufacturing

Retail/Wholesale/Trade

IT

Banking/Finance

Healthcare

Services (except consulting)

Process industry

Consulting

Education

Public sector

Utilities

Insurance

Transportation/Logistics

Other

Telecommunications

Media/Publishing

Construction

Oil/gas/Mining

Agriculture


- 7 -

Featured products

When grouping and describing the products featured in The BI Survey, we did not strictly follow the

naming conventions that the vendors use. In some cases, we combined various products to make

analysis more convenient. In those cases, we named the groups of products as shown in Figure 7. Note

that the names we use in this document are our own and are not always the official product names used

by the vendors.

One of the key reasons for this is that the products we analyze are not necessarily the latest version of

the tool. Vendors often change the product name between versions, making it difficult to have a single

official name for several versions of the same product.

Another reason is that we sometimes bundle related products into a single group to increase the sample

size, even if the vendor prefers to view them as distinct for marketing reasons. In both these cases, the

point is not to challenge the naming conventions of the vendor, but simply to reduce the complexity of

the Survey findings for the convenience of the reader. In some cases, we also shorten the names of the

products to improve the formatting of the charts.

We asked respondents explicitly about their experiences with products from a pre-defined list, with the

option to nominate other products. This list is updated each year and is based on the sample size of the

products in the previous year, as well as additional new products in the BI market. Our pre-defined list

can be found at the end of this document. In cases where respondents said they were using an ‘other’

product, but from the context it was clear that they were actually using one of the listed products, we

reclassified their data accordingly.

We solicited responses on all surviving products with more than a minimal response in the last Survey,

plus a few others whose numbers have potentially grown to the point where there is enough data to be

analyzed.

The following table shows the products included in the detailed analysis. The number of the ‘other’

responses is also included in this chart.

Product labels Product name(s) Respondents

arcplan arcplan Enterprise/Edge 69

Bissantz Bissantz DeltaMaster 88

BOARD BOARD 61

CALUMO CALUMO 37

Chartio Chartio 34

Corporate Planning Corporate Planner 39

Cubeware Cubeware Cockpit 76

cubus cubus outperform 43

Cyberscience Cyberscience Cyberquery 81


- 8 -


Dimensional Insight Dimensional Insight 55

Dundas Dundas Dashboard 38

Entrinsik Entrinsik Informer 50

evidanza evidanza 46

IBM Cognos BI IBM Cognos BI (Cognos 8 and 10) 81

IBM Cognos TM1 IBM Cognos TM1

IBM Cognos Express 39

Infor Infor BI 51

Information Builders Information Builders WebFOCUS 36

Jedox Jedox Base and Premium 58

Logi Analytics Logi Analytics 32

MS Excel & Power pro.

Microsoft Excel

164

Microsoft Power Pivot

Microsoft Power BI

Microsoft Power View

Microsoft SharePoint Server Excel Services

MS SSRS Microsoft SQL Server Reporting Services 83

MicroStrategy MicroStrategy 81

Oracle BI FS Oracle BI Foundation Suite 59

Phocas Phocas 47

prevero prevero 8/prevero Enterprise P7 40

Pyramid Analytics Pyramid Analytics 30

Qlik Qlik QlikView 254

SAP BEx SAP Business Explorer (BEx) 86

SAP BO Analysis SAP BusinessObjects Analysis 31

SAP BO Design St. SAP BusinessObjects Design Studio 30


- 9 -


SAP BO WebI SAP BusinessObjects Web Intelligence 92

SAS

SAS Base

35

SAS Enterprise BI

Tableau Tableau 82

TARGIT TARGIT BI Suite 44

Yellowfin Yellowfin 40

Other 202

Figure 7: Products included in the sample (excluding ‘don’t know')

The last few years have seen an increase in the proportion of German respondents. This is partly due

to cooperation with German vendors and the presence of strong German subsidiaries of international

vendors, reflected through products like arcplan, Bissantz, BOARD, Cubeware and SAP.

This year we also included a few new vendors including Chartio and CALUMO.

The following table contains the products that had responses but are not included in the detailed

analysis. In the BI Survey Analyzer these products are grouped together under the label ‘Others’.

Other products

TIBCO

Pentaho

SiSense

SAP Crystal Reports

Oracle Hyperion Planning

Oracle Hyperion Smart View

SpagoBI

Jaspersoft

prevero Prof. Planner

SAP BW IP

Qlik Sense

SAP Lumira

SAS Visual Analytics

Tagetik

Birst

Actuate


- 10 -

Other products

Dodeca

iDashboards

Jinfonet JReport

Prophix

Adaptive Planning

Corporater

Indicee

Figure 8: Products in the sample but not in the detailed analysis

Deployment

This section focuses on BI deployment, analyzing product usage in small, medium and large

organizations as well as types and numbers of BI users. We compare how widely products are used

and analyze the frequency of BI usage in individual departments and business functions.

Percentage of BI Users in Company

After two years of slight decline, the median percentage of “BI users in a Company” increased by more

than 2 percent on last year to over 13 percent. Out of 2,071 responses, 45 percent of companies have

less than 10 percent of employees who use BI while 12 percent have more than 50 percent of employees

using BI.

Rising use cases and buying preferences around self-service, mobile, ease of use, and support for more

concurrent users foretell growing penetration rates. As a potential early indicator, consider that while the

“less than 5 percent” and ”5 to less than 10 percent” groups in Figure 10 declined from 2014, we see

upticks for the “50 to less than 75 percent” and “More than 75 percent” groups. Though small, the

penetration tiers seeing increases confirm the desire for companies to deploy BI to more employees.


- 11 -

Figure 9: Percentage of BI users in company, timeline (median numbers)

(n=changing basis)

Figure 10: Percentage of BI users in company, 2014 and 2015 (n=2051/2071)

Completion within Original Budget by Product

A BI platform deployed to support a broad range of applications and groups must efficiently address

compliance, user and operational requirements. A lack of flexibility, or performance limitations, can result

in frustration and force costly investments in supporting technologies (i.e. hardware, network, database)

0%

2%

4%

6%

8%

10%

12%

14%

2012 2013 2014 2015

26%

19%

10%

10%

11%

7%

4%

6%

6%

29%

21%

9%

10%

10%

7%

4%

5%

5%

1 to less than 5%

5 to less than 10%

10 to less than 15%

15 to less than 20%

20 to less than 30%

30 to less than 40%

40 to less than 50%

50 to less than 75%

More than 75%

2015 2014


- 12 -

to compensate for a product’s shortcomings. We think operational efficiency and effectiveness will be

an emerging issue as vendors focus on ease of use while putting aside advancements in performance,

integration and core administrative features.

In 2015, six vendors scored above 80 percent in terms of delivering within budget: Phocas (86 percent),

Dimensional Insight (84 percent), Cyberscience (84 percent), Corporate Planning (82 percent), cubus

(82 percent) and Entrinsik (82 percent). The top six companies have a few characteristics in common:

they have spent 20+ years building and implementing their product; they offer solution accelerators

geared to specific system and/or industries; and they value customer success as witnessed by above

average customer satisfaction ratings.

While products from some established vendors often lack modern-looking interfaces and self-service,

they often possess the industry knowledge, performance and functionality that results in achieving

targeted benefits with the best time to value and satisfaction. The results show that small vendors have

better scores for completing projects on budget than large vendors.

The vendors that most frequently fall short in completing projects within budget are IBM Cognos (23

percent), SAS (19 percent), SAP BEx (15 percent), Oracle BI FS (15 percent), SAP BO WebI (14

percent) and MicroStrategy (11 percent).


- 13 -

Figure 11: Completion within original budget by product (n=2187)

Good Moderate Poor

Phocas 86% 14% 0%

Dimensional Insight 84% 14% 2%

Cyberscience 84% 16% 0%

Corporate Planning 82% 15% 3%

cubus 82% 18% 0%

Entrinsik 82% 16% 3%

Tableau 78% 19% 3%

Chartio 77% 23% 0%

Yellowfin 77% 18% 5%

Cubeware 76% 18% 6%

Jedox 75% 22% 4%

prevero 71% 29% 0%

arcplan 71% 23% 6%

TARGIT 71% 22% 7%

Pyramid Analytics 70% 26% 4%

MS SSRS 67% 23% 10%

evidanza 67% 26% 7%

Logi Analytics 66% 28% 7%

Bissantz 65% 31% 4%

MS Excel & Power pro. 63% 29% 8%

BOARD 63% 29% 9%

Information Builders 61% 36% 3%

Dundas 59% 35% 5%

IBM Cognos TM1 59% 35% 5%

Qlik 59% 35% 7%

SAP BO Analysis 58% 35% 6%

Infor 58% 33% 9%

CALUMO 56% 36% 8%

MicroStrategy 42% 47% 11%

IBM Cognos BI 41% 36% 23%

SAP BO Design St. 41% 56% 4%

SAS 41% 41% 19%

SAP BEx 39% 46% 15%

SAP BO WebI 39% 47% 14%

Oracle BI FS 33% 53% 15%


- 14 -

Number of Users by Product

Today, we have a large variety of specialized products tailored to address specific scenarios; sometimes

specific to company size, sometimes to features, and other times to types of users. For example, in the

1990s almost every product focused on data warehouse BI in large enterprises. Today, if a vendor

focuses on customer lifetime value analysis, should we view its lower user counts as negative? Many

people using a product may not equate to it being the right product. Still, with a sample size of 2,000,

the results that follow do provide basis for initial product impressions and follow-up questions.

The BI Survey process stratifies vendor user counts into three bands based on the number of users:

small, medium and large. These bands provide an interesting look at the market for BI products.

Products in the small band are niche or departmental products, while those in the large band are clearly

enterprise BI products. Those in the middle band can go either way, depending on the size of the

company. There are also interesting anomalies in each band, where some products have a wide gap

between their median and mean numbers.

Small Number of Users

BOARD, Logi Analytics and prevero lead the way with the top three positions for median number of

users in the small user band with Chartio, Yellowfin and Corporate Planning bringing up the rear.

Dundas, Logi Analytics and Tableau show a big discrepancy between their median and mean user

counts. Dundas’ whopping 663 mean indicates it has a few customers with a very large number of users.

This is likely due to the fact that some Dundas customers use the product to display performance on big

screens in call centers, factory floors and other settings with lots of users.

Logi Analytics has a high mean count likely because a large percentage of its customers embed the tool

into other applications. Tableau, on the other hand, is a desktop tool but the company has used a land-

and-expand sales tactic to land many large enterprise accounts with thousands of users.

Figure 12: Products with small numbers of users (n=2182)

Products Median Mean

BOARD 45 94

Logi Analytics 45 322

prevero 40 89

Phocas 35 58

Dundas 31 663

evidanza 30 60

Cyberscience 26 99

Bissantz 25 116

Tableau 25 284

CALUMO 21 105

Jedox 20 42

Chartio 18 37

Yellowfin 14 40

Corporate Planning 8 11

Small number of users


- 15 -

Medium Number of Users

Among the group of medium-sized user numbers, four products - arcplan, Microsoft Excel & Power

products, Microsoft SSRS and SAS - top the median user rank with 100 users each. Across the entire

field, arcplan, Microsoft Excel & Power Products and SAP BI Design Studio are well clear of the rest in

terms of mean user numbers, averaging over 1,200 users each.

As mentioned earlier, sometimes it’s important to distinguish between different types of users. For

example, do you need a product that can support 1,000 users who need to customize and create reports

or do they just need to receive and review dashboards that are exported to PDF documents, reducing

both ease of use and platform performance requirements.

Another factor that can influence users is focus on certain departments. For example, if a vendor focuses

on legal solutions, you would expect lower maximum user counts. This is the case with Dimensional

Insight; the company targets hospitals that track various metrics as part of performance and compliance

initiatives. As such, their user counts reflect headcounts that you would typically find at regional hospitals

and healthcare facilities.

Figure 13: Products with medium numbers of users (n=2182)

Large Number of Users

Products running in enterprise environments with a large number of BI users from multiple departments

typically face larger data volumes. They also need to enforce access models designed to govern cross-

team and cross-departmental data access if all users are to leverage a common platform.

If users across different business areas need to create and customize their own reports, the product

needs to support abstraction of source schemas, field names, and allow creation of business-friendly

names and data views.


arcplan 100 920

MS Excel & Power pro. 100 1748

MS SSRS 100 540

SAS 100 297

cubus 90 270

IBM Cognos TM1 80 116

SAP BO Design St. 80 1156

Qlik 78 262

Dimensional Insight 55 240

Cubeware 50 99

Entrinsik 50 83

Infor 50 104

Pyramid Analytics 50 128

TARGIT 50 126

Medium number of users


- 16 -

Finally, the customers with a large number of users need BI products that can scale affordably, ideally

requiring as few hardware and human resources as possible in order to meet requirements and

operational SLAs.

Information Builders, SAP BO WebI and IBM Cognos BI are the three products with the highest median

users in The BI Survey 15 while SAP BEx, SAP BO WebI and Oracle BI FS have the three highest mean

user counts. MicroStrategy is the only other vendor that places on this list with 150 Median and 1,230

Mean users. All of these vendors have strong roots in data warehouse environments including

multidimensional cubes and in-memory options to boost performance.

Figure 14: Products with large numbers of users (n=2182)

Having dominated the large user environments, vendors like IBM and SAP have launched new products

targeting smaller groups. At the same time, mid-market vendors like Pyramid Analytics are beginning to

gain large enterprise customers. BI leaders can expect vendors to continue expanding beyond their

traditional borders, offering buyers an increasing number of options. To gain additional insights from the

product user counts on these charts, consider combining information from other charts in The BI Survey,

information from vendors and other survey results.

Percentage of Employees Using the BI Product

Against an industry backdrop marked by growing demand for self-service, measuring the percentage of

employees using a given product helps inform us which vendors might have products better suited to

empower users. Achieving high employee penetration rates in most corporate environments involves

execution on multiple factors including ease of use, ease of accessibility, price and pricing model.

SaaS cloud BI models have the potential to play a role in all of these factors and many traditional on-

premise vendors are formulating strategies and rewriting products to thrive with this new delivery model.

For 2015, Chartio, a San Francisco-based company that envisions a future where all business users

create dashboards, takes the top spot with customers reporting a median of 29 percent employee

penetration.

Just 1 percent below is Yellowfin, an Australian provider of a full-stack BI platform. Both companies

support cloud BI but in different ways. While Chartio operates a SaaS BI service, Yellowfin makes its

software available on Amazon’s EC2 and Microsoft’s Azure marketplaces with a ‘bring your own license’

(BYOL) model.


Information Builders 350 1459

SAP BO WebI 350 1626

IBM Cognos BI 300 1495

SAP BO Analysis 275 1321

SAP BEx 250 1659

Oracle BI FS 200 1555

MicroStrategy 150 1230

Large number of users


- 17 -

Figure 15: Percentage of employees using the BI product, median numbers

(n=2183)

29%28%

22%20%

17%16%

13%12%

11%11%

10%10%10%10%

8%8%

7%7%

6%6%6%6%6%6%6%6%5%5%

5%5%

4%4%

3%3%3%

ChartioYellowfin

MS Excel & Power pro.Phocas

Dimensional InsightEntrinsikevidanza

DundasIBM Cognos BI

Information BuildersMS SSRS

Oracle BI FSSAP BO Design St.

TARGITQlik

Pyramid AnalyticsBOARD

Logi Analyticsarcplan

BissantzSAP BO WebIMicroStrategy

Cubewareprevero

SAP BExJedox

SAP BO Analysiscubus

CyberscienceCALUMO

SASInfor

TableauCorporate Planning

IBM Cognos TM1


- 18 -

Departments Using BI

Finance, management and sales departments continue to lead in the use of BI, with the latter two having

continued their declines from 2014. Marketing has continued to increase use after pausing growth in

2014 while production/operations continues its fast adoption of BI.

If current trends continue, production/operations could overtake sales and IT to become the third biggest

user of the technology to improve performance within two to four years.

Figure 16: Which departments are using BI? Timeline (n=changing basis)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

2008 2009 2010 2011 2012 2013 2014 2015

Finance

Management

Sales

IT

Production/operations

Marketing

Human resources

Procurement

Logistics

Services

R & D

Legal


- 19 -

Challenges

Implementation Problems by Peer Group

BI projects have the potential to invite many types of problems ranging from confidence issues with

underlying data, to organizational politics, to the product itself. Across large organizations, the top three

problems are: lack of resources (cited by 23 percent of users), software issues (22 percent) and data

migration (20 percent).

When looking at mid-size companies, the top three implementation issues are resources (18 percent),

data migration (17 percent) and unclear requirements (16 percent). With a 12 point spread, the greatest

difference between the two peer groups is in the area of software-related problems, with large

enterprises facing the issue more than twice as frequently.

Vendors such as Dundas focus on providing professional services that address the often-difficult

process of defining KPIs as well as technical services. Many problems can be avoided through sound

SDLC processes that engage business stakeholders early and often. Buyers should find out how

vendors address these issues during the evaluation process.

Figure 17: Implementation problems by peer group (n=2122)

Large

deployments

Mid-sized

deployments

Lack of resources on the project

team 23% 18%

Software-related issues 22% 10%

Data migration 20% 17%

Unclear requirements 20% 16%

Costs higher than expected 16% 7%

Tight deadline 16% 13%

Lack of expertise by the

implementation partner14% 6%

Training-related issues 12% 9%

Customization of the product 11% 8%

Lack of support from

management10% 9%

Lack of project management 7% 6%

Lack of communication in the

project team4% 3%

Lack of resources by the

implementation partner 3% 3%

No significant problems 19% 40%


- 20 -

Implementation Problems by Industry

As mentioned earlier, BI products such as TARGIT and Dimensional Insight achieve the vast majority

of their growth by focusing on challenges in specific industries. Even a general use case like self-service

has different growth ceilings depending on the industry. The chart below provides insights that can help

guide product decisions based on unique challenges within each industry. Key highlights include:

Telecoms need to be especially mindful of BI product capabilities for data quality

Companies across all industries, especially utilities and manufacturing, need to ensure BI

products meet query performance requirements

Companies in the services and telecoms industries should seek solutions that enhance

business user engagement

Transport companies need to pay extra attention to software reliability

Retail and wholesale industry players experience the lowest rate of significant BI problems.


- 21 -

Figure 18: Implementation problems by industry (n=2376)

Financial

Services

Manu-

facturingPublic sector

Retail &

WholesaleServices

Telecom-

municationsTransport Utilities IT

Administrative problems 12% 10% 10% 9% 11% 12% 11% 14% 12%

Company politics 7% 9% 11% 8% 12% 11% 13% 11% 6%

Lack of interest from

business users 14% 13% 14% 13% 18% 17% 10% 12% 12%

Missing key features 12% 5% 5% 6% 8% 11% 6% 7% 8%

Poor data governance 12% 9% 14% 8% 9% 18% 15% 13% 9%

Poor data quality 16% 13% 14% 13% 12% 24% 15% 13% 9%

Product can not handle

number of users4% 2% 0% 2% 2% 5% 0% 1% 3%

Product can not handle

data volume6% 7% 4% 6% 7% 12% 3% 7% 9%

Slow query

performance20% 23% 13% 19% 18% 20% 15% 27% 21%

Product security

limitations7% 2% 3% 3% 4% 0% 4% 1% 7%

Software too hard to

use7% 7% 11% 6% 7% 3% 10% 6% 6%

Software not flexible

enough13% 10% 7% 12% 8% 11% 14% 10% 12%

Unable to get data from

some systems7% 4% 9% 5% 8% 5% 8% 6% 7%

Unreliable software 8% 6% 6% 4% 5% 6% 16% 5% 6%

No significant problems 34% 41% 41% 42% 37% 29% 32% 35% 41%


- 22 -

Most Serious Implementation Problems

The most serious problems encountered during BI implementations are not directly related to the BI

product itself. As a matter of fact, resource and requirements issues are common offenders in any type

of software development project. However, looking down the chart, one can see problems where the

choice of BI product can either mitigate or exacerbate the issue. Below are some ways in which a BI

product can impact on (minimize, exacerbate, provide an alternative) implementation issues.

Data migration: some BI products include ETL tools and databases that could provide a solution; some

products are designed to generate reports and dashboards directly from the source, eliminating the

need to migrate data.

Training-related issues: sometimes accessible classes, on-line tutorials and professional services

consultants are not enough to turn the tide with regard to training-related issues. Ease of use and a

product’s design paradigm play major roles in determining which types of users can quickly learn and

become effective with the product.

Customization of the product: there’s a difference between customizing a data model and changing

a default reporting interface to include or exclude twenty features. While some BI products are built for

out-of-the-box use after configuration and setup, others offer a platform designed to help customers

build and customize applications for deployment.

The interesting fact to keep in mind here is that BI development is increasingly becoming a continuous

process by non-technical people. Perhaps the results shown in this chart reinforce why self-service,

agile models are on the rise.

Figure 19: Most serious implementation problems (n=2330)

19%

17%

16%

14%

13%

10%

9%

9%

9%

7%

6%

4%

3%

4%

35%

Lack of resources on the project team

Data migration

Unclear requirements

Tight deadline

Software-related issues

Training-related issues

Costs higher than expected

Lack of support from management

Customization of the product

Lack of expertise by the implementation partner

Lack of project management

Lack of resources by the implementation partner

Lack of communication in the project team

Other

No significant problems


- 23 -

Most Serious Problems Encountered during Use by Business Users

The most serious issue arising during the use of BI is query performance. At 19 percent, it is a full five

points higher than the next problem (lack of interest by business users). Perhaps the lack of interest

relates to the third most serious issue, which is poor data quality.

Although increased rigor during product evaluations may help in some cases, BI deployments grow

much faster than most managers can envision when making the initial purchase. BI environments

degrade quickly if the overall architecture was not designed to handle increased loads. Most BI products

cannot solve data quality issues, instead exposing them so companies can determine the extent of them.

Reviewing the issues here and structuring projects to account for the ones that apply in a given company

will increase overall project success. Most BI projects experience smooth sailing post deployment with

39 percent of the 2,422 respondents to this question reporting no significant problems.

Figure 20: Most serious problems encountered during use by business users

(n=2422)

19%

14%

13%

11%

10%

10%

9%

7%

7%

7%

6%

6%

3%

2%

5%

39%

Slow query performance

Lack of interest from business users

Poor data quality

Administrative problems

Poor data governance

Software not flexible enough

Company politics

Missing key features

Software too hard to use

Unable to get data from some systems

Unreliable software

Product can not handle data volume

Product security limitations

Product can not handle number of users

Other



- 24 -

What Problems Do Customers Typically Encounter?

Based on 450 responses from the vendor community, the most serious problems that vendors say their

customers report are poor data quality (33 percent), company politics (26 percent), administrative

problems (22 percent), poor data governance (22 percent) and lack of interest from business users (18

percent).

While additional software and hardware investments may be part of the overall solution, solving these

problems require collaboration, process development and business transformation. For example, data

quality issues typically require a combination of business process and upstream systems changes.

Building active enagement necessitates involving users at an early stage in the requirements process.

Software vendors seldom want to work through these thorny issues but it’s important for buyers to

identify these challenges early, allocate sufficient time, and realize that BI software is not a magic eraser

that eliminates problems.

Figure 21: What problems do customers typically encounter? (vendor

responses) (n=450)

33%

26%

22%

22%

18%

13%

11%

10%

9%

7%

4%

3%

1%

1%

2%

21%

Poor data quality

Company politics

Administrative problems

Poor data governance


Unable to get data from some systems

Software not flexible enough

Slow query performance

Software too hard to use

Missing key features

Unreliable software

Product can not handle data volume

Product security limitations

Product can not handle number of users

Other



- 25 -

Reasons for Replacement

Replacing BI software happens for a variety of reasons and is often painful, especially if recent

investments failed to deliver on strategic promises. The BI Survey 15 results indicate that the frequency

of replacements usually increases as the size of the company, in terms of employees, increases. As BI

technology is typically composed of a “portfolio” of tools at larger companies rather than one product,

large companies may be better equipped to weather the impacts of a replacement.

Buyers have many options for reducing the chance that the “reasons” below will surface at their

companies. Methods include increasing rigor during product evaluation, establishing a supportive

political environment, or properly scoping dependencies like data quality training, and security issues

into the overall project scope and costs.

When it comes to buyer’s remorse due to a product’s fitness for the job, cloud-based delivery models

with monthly subscription prices offer the advantage of limiting upfront investment and might be the best

option when it’s not possible to make a quality selection decision.

Figure 22: Replacement of a BI system in the last 12 months by company size

(n=1649)

Figure 23: Reasons for replacing BI systems (n=192)

5%

10%

15%

95%

90%

85%

Less than 100 employees

100 - 2,500 employees

More than 2,500 employees

Replaced a BI system in the last 12 months

Didn't replace a BI system in the last 12 months

44%

27%

27%

26%

25%

25%

21%

17%

16%

14%

13%

12%

8%

7%

18%

Software was not flexible enough

Company politics

Software difficult to use

Missing key product features

Query performance too slow

Too expensive

Product could not handle our data volumes

Bad support

Unreliable software


Unable to get/analyze data from some systems

Poor data quality

Product could not handle large numbers of users

Security limitations in the product

Other


- 26 -

Reasons for Replacement by Region

In Europe, two reasons for replacing BI tools tie for the top spot at 34 percent: company politics and lack

of flexibility, followed by the price being too expensive (23 percent). In North America, 50 percent of

replacements were due to lack of flexibility followed by “difficult to use” (35 percent) and “missing key

features” (33 percent).

With established vendors, take time to understand their track record around major platform upgrades.

Know the product’s roadmap but don’t assume 100 percent of it comes to fruition on time. Instead, work

with the vendor to determine how requirement gaps will be closed in the event that the roadmap

changes. Note that the Vendor Performance Summaries included in The BI Survey 15 indicate whether

a product is “in strategic development”, providing a helpful data point for gauging investment into the

surveyed products.

Figure 24: Reasons for replacement by region (n=168)

34%

34%

23%

21%

21%

21%

19%

17%

14%

13%

13%

11%

6%

6%

50%

17%

23%

21%

27%

35%

33%

13%

19%

13%

10%

13%

10%

8%

Software was not flexible enough

Company politics

Too expensive

Product could not handle our data volumes

Query performance too slow

Software difficult to use

Missing key product features

Bad support

Unreliable software

Poor data quality

Unable to get/analyze data from some systems


Product could not handle large numbers of users

Security limitations in the product

Europe North America


- 27 -

Next Steps

The BI Survey Analyzer Demo

The BI Survey Analyzer is an interactive online tool which enables you to perform your own custom

analysis of the survey data.

Click here to register for an online demo and find out how effective The BI Survey Analyzer can be in

helping you to understand the strengths and weaknesses of BI products and trends in the BI market.

Purchase

To purchase The BI Survey 15, visit the BARC website or contact us at:

Email: [email protected]

Tel: +44 (0)20 8133 8987

http://barc-research.com/research/business-intelligence/the-bi-survey-analyzer-register-for-a-demo/

http://barc-research.com/bi-survey-15/

mailto:[email protected]

Copyright © BARC GmbH 2015. All rights reserved.

Business Application Research Center – BARC GmbH

Germany

BARC GmbH

Berliner Platz 7

D-97080 Würzburg

+49 (0) 931 880651-0

www.barc.de

Austria

BARC GmbH

Goldschlagstraße 172 / Stiege 4 / 2.OG

A-1140 Wien

+43 (1) 8901203-451

Switzerland

BARC Schweiz GmbH

Buchhaldenstraße 7

CH-5442 Fislisbach

+41 (0) 763403516

Rest of the World

+44 1536 772 451

www.barc-research.com

http://www.barc-research.com/

http://www.barc-research.com/

Table of contentsbarc-research.com/wp-content/uploads/2015/10/bis15-free... · 2015-10-19 · Table...

Documents

Transcript of Table of contentsbarc-research.com/wp-content/uploads/2015/10/bis15-free... · 2015-10-19 · Table...