Platform Comuting Test Dev Survey Results FINAL[1]

8/23/2019 Platform Comuting Test Dev Survey Results FINAL[1]

1/35

Copyright 2009 Platform Computing Corporation. All Rights Reserved.1

Research Report onVirtualization and

Cloud Computing inTest/Dev Environments

Platform Computingwww.platform.com
http://www.platform.com/http://www.platform.com/


2/35


Background: Methodology

Objective: Examine the challenges of managing Test/Dev

environments and the opportunity for servervirtualization and cloud computing to address them

Fielded: Jun-Aug 2009

Method: Telephone surveys

Sample: IT architecture and operations managersresponsible for development and test infrastructure

(N=83), at North American firms, cross-industry.

SMB: n=57 (500 Test/Dev servers)

Fielding and data analysis support by third party vendors

Fielding: Primary Point (http://www.primarypoint.com/)

Survey development and analysis support: Taneja Group

(http://www.tanejagroup.com/)
http://www.primarypoint.com/http://www.tanejagroup.com/http://www.tanejagroup.com/http://www.primarypoint.com/


3/35


Key Findings: The Case for Private Cloud

1. Significant investment is being allocated to solve Test/Dev

operational and budgetary challenges: 50% of respondents will

have a funded project start in 2009, while 66% of respondents will

have a funded project started by end of 2010

2. Application teams are using shared infrastructure in production

today, supporting the theory that Cloud Computing is more about a

new business model for IT delivery than new technology

3. Few firms have adopted a hosted environment outside the firewall.

This helps to explain why many firms are starting with private

cloud; get the business model right before introducing additional

complexity involved with transitioning to external infrastructure

4. The top challenge identified in managing Test/Dev environments:

the need to manage virtual and physical resources separately

5. Experienced users confirmed that server virtualization is an

enabler, but adds several layers of control and cost issues that must

be addressed by sharing, process, workflow, and other

management capabilities


4/35


Industry Breakdown

Banking

20%

Gov

18%

Manuf

17%

Telco

14%

S/W

18%

Health

8%

Pharma

1%

Other

2%

Industry Breakdown:

ALL

Banking32%

Gov

8%

Manuf

16%

Telco

12%

S/W 28%

Other

4%

Industry Breakdown:

LARGE (>500)

Takeaway: Broad industry representation. Banking and Software segments dominate

the Large category with a combined 60% of respondents.


5/35


Size of Test/Dev Environment

SMB

< 500

69%

Large >

50031%

Number of PhysicalTest/Dev Servers

0

5

10

15

2025

30

35

40

Distribution

Takeaway: Broad coverage of infrastructure scale. Banking accounts for 40% of the

environments that are greater than 1000 servers.


6/35


Prevalence of Virtualization in Test/Dev

Yes

89%

No

13%

Does your company use

virtualization in Test/Devenvironments?

Takeaway: While virtualization is used heavily in Test/Dev environments, there remains

a significant amount of work being done on non-virtualized infrastructure. Server

virtualization prevalence matches industry estimates of 20-30% market penetration.

3

9

8

14

39

Dont know/Refused

75-100%

50-75%

25-50%

0-25%

If Yes: Approximately what percentage

of physical hosts are supportingvirtualization?

53% of respondents

have virtualized less

than 25% of their

physical servers


7/35Copyright 2009 Platform Computing Corporation. All Rights Reserved.7

Hypervisor Vendor Prevalence in Test/Dev

Takeaway: VMware prevalence is 95%, but 35% run at least one other hypervisor.

Consolidation ratios: 20/host on average is high (with respect to production server

virtualization consolidation ratios), indicating dense, highly shared environments and

significant contention for limited, shared physical server resources.

16

15

18

80

Other

Citrix

Microsoft

VMware

What hypervisor vendors are

you using?

13

9

20

> 20

11-20

1-10

Average Number of VMs per

virtual host?

65% Homogeneous35% Heterogeneous Average: 20


8/35Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Resource SharingObjectives:

Determine the types of resources currently shared

Determine use cases for shared resources

Explore prevalence of internal (private cloud) sharing and

external (hosted, outsourced, public cloud) sharing

Establish current views of limitations and challenges in

Test/Dev for both internal shared and external hosting



Resource Sharing in Test/Dev

Yes

92%

No

8%

Does your company share servers in

Test/Dev environments?

Takeaway: Development teams have a high degree of comfort using shared

infrastructure. Of the small group that dont (8%), most plan to within 2 years. Teams

share across nearly all phases (86% share across at least Dev, Unit Test, Integration Test

& Performance Test phases).

Yes

86%

No

14%

If Yes, do you share servers between

the SDLC phases (Dev, Test, UAT)?



Resource Sharing for Other Activities

Yes

69%

No

31%

Do you currently, or plan to, use your

Test/Dev environment forotheractivities?

Takeaway: Sharing is prevalent, even for non-Test/Dev activities. Disaster Recovery is

likely an emerging use case, based on separate Taneja Group research.

4

47

53

Disaster Recovery

Demos, Prototyping

Training

IfYes, What activities?



Use of Outsourced Infrastructure

No

82%

Yes

18%

Do you use off-premise or hosted

servers for Test/Dev (outsourcing)?

Takeaway: Few firms have adopted a hosted environment outside the firewall.

Lack of Control and Immature Technology are the top functional blockers, but a

significant number have not been effectively sold on benefits (We Dont Have Need)

2

3

3

10

13

19

24

No Time

Too Expensive

Vendors are Too Small

Lack of Security

We Don't Have the Need

Immature Technology

Lack of Control

IfNo: Why Not?



Use of Outsourced Infrastructure

No82%

Yes

18%

Do you use off-premise or hosted

servers for Test/Dev (outsourcing)?

Takeaway: Of those who have outsourced, 71% have outsourced less than 25% of their

Test/Dev servers. Only a few vendors were named: Amazon (3), Savvis (2), Skytap (1),

Surgient (1), ADP (1)

Less

ThanOne

Quarter

71%

More

ThanOne

Quarter

29%

IfYes: What percentage of

your Test/Dev servers arecurrently outsourced?


13/35


Sharing Resources in Production

Yes

76%No

24%

Are you using shared infrastructure or

a private cloud forProduction

enterprise applications?

Takeaway: Application teams are using shared infrastructure in production today,

supporting the theory that Cloud Computing is more about a new business model for IT

delivery than new technology. 30% of shared pools host both Test/Dev and Production

applications, indicating a rising comfort level with sharing overall. Taneja Group

experience confirms emergence of Test/Dev/Prod sharing.

Same

Pools

30%Different

Pools

70%

If Yes: Do Production applications

share the same or different resource

pools as Test/Dev?


14/35


Shared Production Applications

2

3

8

8

13

15

19

20

21

Web 2.0

System Software

Business Critical

HR

Intranet

ERP

Web

Finance

CRM

If Yes, what would you say are your top three Production

applications using shared infrastructure or private cloud?

Takeaway: Supporting the virtualization trend, enterprise applications that are within

the firewall continue to be the primary production applications that are sharing

resources. Business critical applications are at the early stage of moving to a shared

environment.

Top 3 SharedProduction

Applications


15/35


Sharing Resources in Production

Yes

76%No

24%

Are you using shared infrastructure or

a private cloud forProduction

enterprise applications?

Takeaway: Of the 24% who are not sharing, 70% have plans to do so in the next 12

months (70%, N=7/10). Top apps planned to share resources: ERP, Finance, CRM, Web

Yes

55%

No

45%

IfNo, do you know if the company has

any plans to implement shared orprivate cloud for Production?

70% of those with

plans to share in

Production will do

so in the next 12

months


16/35


Types of Applications Being Developed

28

30

30

34

35

38

42

45

50

ERP

HR

CRM

Web 2.0

System Software

Business Critical

Finance

Intranet

Web

Across your enterprise Test/Dev environments, what kinds

of applications are being developed?

Takeaway: Broad mix, focused on web/intranet, finance, and business critical (specific

to the industry/business). Finance and Web apps are also in the top 3 types of apps

currently sharedin production.


17/35

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Test/Dev ChallengesObjectives

Determine the types of user and IT support challenges

Focus on server & virtual machine lifecycle management

issues (vs. SDLC management)

Explore current projects, future planning, and funding to

address Test/Dev challenges


18/35


Test/Dev Operations Challenges

33%

36%

42%

45%

46%

47%

55%

67%

64%

58%

55%

54%

53%

45%

Users do too much configuration work

Users are not returning machines to the pool

Users wait too long for resources

Users complain about lack of control

IT does too much configuration work

Sharing resources among teams is difficult

IT manages virtual and physical resources separately

Please rank the impact of these operational challenges in your

Test/Dev environment:Med/High Impact No/Low Impact

Takeaway: Top challenge: the need to manage virtual and physical resources separately.

The top three challenges relate to lack of cross-domain management, server lifecycle

management, and lab automation.


19/35


Existing Projects to Address Challenges

Yes 41%No

59%

Do you currently have any projects

around any of these challenges?

Takeaway: Top projects focus on configuration (both IT and user configuration efforts),

the need for self-service (wait too long), and better environment sharing across the

software development lifecycle.

8

11

11

12

13

Sharing environments is difficult

Managing virtual and physical

separately

Users spend too much time on

configuration


IT spends too much time on

configuration

IfYes: Which ones? (Top Five)


20/35


Planned Projects to Address Challenges

Yes 33%No

67%

IfNo, do you currently have any

planning around these challenges?

Takeaway: Delivering good configurations to Test/Dev teams quickly, and allowing

them to recreate and share them with a minimum of configuration, are the focus of

most current and planned projects.

4

5

5

6

6

Managing virtual and physical

separately

Sharing environments is difficult


Users spend too much time on

configuration

IT spends too much time on

configuration

If Yes: Which ones? (Top Five)


21/35


Operations Impact: Time Lost

10%< 25%

42%

>25%

8%

87% Claim User Time Is Lost

Due To Ops Challenges.

How Much?

Takeaway: Less time is lost by Test/Dev users than by IT support, but for both the loss is

significant (50% agree that more than 10% of both user and admin time is lost due to

challenges).

10%

< 25%

30%

>25%

19%

89% Claim IT Support Time Is Lost

Due To Ops Challenges.

How Much?

19% say more

than 25% of time

is lost


22/35


Test/Dev Budget Challenges

51%

51%

59%

49%

49%

41%

Unable to charge BU's for what they use

Struggle to track utilization

No budget to add more resources

Please rank the impact of these budgetary challenges

in your Test/Dev environment?Med/High Impact No/Low Impact

Takeaway: Significant recession effect (highest impact of all challenges) is likely

retarding the market for Test/Dev solutions more than any other factor in 2009. Many

respondents indicated they would have explored a solution except for time/budget.

22% Have

Projects

Underway

Around

Budgetary

Challenges


23/35


Cost Analysis of Lost Time

Data:

Dev Team: 87% agree time is lost Of these, 50% say 25%

IT Team: 89% agree time is lost

Of these, 51% say 25%

* Source: Gartner IT Spending and Staffing Report, 2009

Assumptions:

Fully-burdened developer at $150K peryear

Implication: 50% of respondents lose

more than $15K per developer per year

Fully-burdened admin at $100K per year

Implication: 30% of respondents lose

more than $10K per admin per year

Developer/Admin Ratio = 10/1(conservative)

49% of IT staff distribution is in

application development and support ,

and 14% of IT staff distribution is in the

data center *

Cost Analysis: For an environment with 100

developers/testers, with 5 admins

50% will lose at least $1.5M per year in

developer time

49% will lose at least $50K per year in

IT admin time

Takeaway: Lost developer/tester time is a much bigger challenge and cost issue than

lost IT admin time


24/35

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

The Solution OpportunityObjectives

Determine familiarity and impressions of current on-premiseand hosted lab automation and lifecycle management

solutions

Explore existing projects and planning around solution

evaluation

Estimate budget size


25/35


Use of Test/Dev Provisioning Tools

None

34%

HP/

Opsware

17%

BMC

11%

VMware

11%

Other/

Homegrown

27%

Which provisioning tools are you using?

Takeaway: There is a clear lack of vendor penetration for automated provisioning tools

in Test/Dev environments, which explains the high level of reported challenges and the

lack of experience with lifecycle or lab automation (higher-order mgmt tools).

61% Have Homegrown

Tools or No Tools.

Significant Lack of

Investment to Date in

Test/Dev Provisioning

Software


26/35


Effectiveness of Virtualization

No

72%

Yes

28%

Will server virtualization on its own address the

most important challenges in your Test/Dev

environment? Representative Why Not? Answers:

Doesn't take in to account integration and

interoperability issues these are the real

challenges. Need a good service-based

orientation managing business services not

hardware.

You still need process automation;

virtualization is just an enabling technology.

Our biggest challenges in Test/Dev are the

same whether the servers are virtualized or

not.

Our biggest problem now is getting the

configuration[s] right.

Takeaway: Virtualization on its own was expected to solve the challenges only by those

respondents new to using the technology. The majority of experienced users confirmed

that virtualization is an enabler, but adds several layers of control and cost issues that

must be addressed by process, workflow, sharing and other management capabilities.


27/35


Existing Solution Experience

Yes31%No69%

Have you evaluated or tested any

lab automation or virtualization

lifecycle management software?

Takeaway: VMware Lab Manager dwarfs other solutions, but budgetary issues have

stalled a large percentage of evaluations. Few respondents indicated functional

problems with lab management solutions.

2

2

2

4

27

Enigmatec

Microsoft

VMLogix

Surgient

VMware

Which ones?

40% have not

evaluated due to

Lack of Time or

Budget


28/35


Funding to Address Challenges

Yes

42%

No

58%

Do you have a currently funded

project to build or purchaseTest/Dev shared infrastructure

solutions?

Takeaway: Significant investment is being allocated to solve Test/Dev operational and

budgetary challenges.

Don't Know/Refused

0-$50,000

50,000-$100,000

100,000-$250,000

$250,000 +

0 5 10 15 20

IfYes: can you tell us the approximate

budget amount?


29/35


Funding to Address Challenges

Yes

38%

No

62%

IfNo, do you have oneplanned to start this year

or next year?

Takeaway: 38% of respondents that are not investing to solve Test/Dev challenges

today plan to do so in the next two years. 50% of respondents will have a funded

project start in 2009, while 66% of respondents will have a funded project started by

end of 2010.

Yes

42%

No

58%

Do you have a currently funded

project to build or purchaseTest/Dev shared infrastructure

solutions?


30/35


Additional Industry SegmentAnalysis


31/35


Industry Segment Consistency

Virtualization Adoption

Size of environments (Banking outlier)

Number of physical hosts supporting virtualization

Use of virtualization (Manuf outlier)

Hypervisors used

Sharing of Resources

Use of shared servers (SW and Health outlier)

Sharing between stages of SDLC

Using shared servers in production (Telco outlier)

Use Test/Dev for other non-prod work, i.e. demos (Gov and Health outliers)

Application types developed

Solution Opportunity Number that have evaluated lab automation (SW and Health outliers)

Number that have current funding (Banking outlier)

Planned funding for this year


32/35


Industry Segment Outliers

Banking

Heavier outsourcing: 31% (vs. 18% avg.)

Little sharing between Test/Dev and Production: 11% (vs. 30% avg.)

Large: accounts for 40% of the environments that are > 1000 hosts

High projects around operational challenges: 69% (vs. 41% avg.)

15% believe virtualization is enough to solve problems (vs. 28% avg.)

62% have current funding (vs. 42% avg.)

Government

High sharing between Test/Dev and Prod: 56% (vs. 30% avg.)

High use of Test/Dev for other non-prod work, i.e. demos: 91% (vs. 69%

avg.)

Low projects around operational challenges: 18% (vs. 41% avg.)

More believe virtualization is enough to solve problems: 55% (vs. 28% avg.)

High number with planned funding for next year: 50% (vs. 27% avg.)


33/35


Industry Segment Outliers (cont.)

Manufacturing

Only 64% use virtualization (vs. 90% avg.)

No outsourcing: 0% (vs. 18% avg.)

No sharing between Test/Dev and Prod: 0% (vs. 30% avg.)


Software

Only 70% share servers (vs. 92% avg.)

Lighter outsourcing: 10% (vs. 18% avg.)



High number have tested lab automation: 50% (vs. 31% avg.)


34/35


Industry Segment Outliers (cont.)

Telecommunications

Heavier outsourcing: 36% (vs. 18% avg.)

Lighter shared servers in production: 55% (vs. 76% avg.)

High sharing between Test/Dev and Prod: 83% (vs. 30% avg.)



Health Care

Only 57% share servers (vs. 92% avg.)

No outsourcing: 0% (vs. 18% avg.)


Low use of Test/Dev for other non-prod work, i.e. demos: 29% (vs. 69%avg.)

High number have tested lab automation: 71% (vs. 31% avg.)

No planned funding for next year: 0% (vs. 27% avg.)


35/35

For more information Contact Martin Harris, Director Product Management,

[email protected]

Or visit http://www.platform.com/company/contact-us
http://www.platform.com/company/contact-ushttp://www.platform.com/company/contact-ushttp://www.platform.com/company/contact-ushttp://www.platform.com/company/contact-us

Platform Comuting Test Dev Survey Results FINAL[1]

Documents

Transcript of Platform Comuting Test Dev Survey Results FINAL[1]