Turning software risk measurement into business value

June 2013

Konstantin Berger – Regional Sales Consultant

Standardized Risk Measurement for IT Executives 101

CAST Confidential

Agenda

1. Who We Are

2. What We Do

3. Why We Do This

CAST Confidential

Who We Are: Driving software measurement in the industry

2

Key Influencers Recognize CAST

250 Global Leaders Rely on CAST

Institutions Engage CAST SIs Resell CAST SIs Use/Resell CAST

• 20+ years in SAM industry

• $100M+ of R&D investment

• Largest Benchmarking DB

CAST Confidential

Who We Are: How Customers Use Us

3

“We measure software quality at a structural level, in addition to the functional level through testing, to make the right tradeoffs

between delivery speed, business risk, and technical debt.”

Thaddeus Arroyo

CIO, AT&T

“Now we can show our business constituents the ROI from quality improvement.”

Peter de Boel

Head of Global Shipping IT, FedEx

“We got feedback from our clients noticing that our quality has gone up.”

Gene Baker

Director of Application Development, Wells Fargo

“CAST makes the relationship with the suppliers very transparent.” Lester Thomas

Head of Architecture, Vodafone

“I have to continue to deliver the same level of functionality and innovation, at the same quality, but with a significantly lower budget. That’s where CAST

helps us.”

Gil Hoffman

CIO, Maritz

“The architectural assessment of design consequences (on software performance, stability, adaptability, maintainability, and security vulnerabilities) is an area in which CAST excels and successfully

differentiates from static analyzers” Melinda Ballou

ALM Research Director, IDC

CAST Confidential

What We Do: Structural Quality vs. Functional Quality

Functional Quality (not us)

That which we can see

10 to 20% of app dev cost

Structural Quality (what we do)

Architecture Analysis

Standardized SW Characteristics

App Reliability

App Performance Efficiency

App Security

App Changeability

App Size (Function Points)

CAST Confidential

What We Do: A natural complement to testing

Design Develop

QA

Deploy Maintain

NEW

Functional Testing

Performance Testing

CAST: Structural

Testing

CAST Confidential

What We Do: Standardized Risk Management for Execs

CAST Quality Score Scale

Description Grade

Low Risk 4

Moderate Risk 3-4

High Risk 2-3

Very High Risk 1-2

Benchmark

Low

Risk

High

Risk

Health Factors Current Grade Last Delivery Since Baseline

Performance 2.38 0.5% 0.7%

Robustness 2.72 0.2% 0.4%

Security 3.14 0.1% 0.2%

Changeability 3.04 0.2% 0.3%

Transferability 2.77 0.2% 0.3%

Consortium for IT

Software Quality

r1 r2 r3 r4 r5 r6

S

R

P

Ongoing Risk Monitoring

CAST Confidential

Why We Do This: Not just quality for quality’s sake

UNIT LEVEL FLAWS

Downtime caused

by system-level flaws!

Of all

code

defects

Of total

repair

effort

92%

8%

52%

48% 90%

10%

SYSTEM LEVEL FLAWS

Software Risk Prevention:

Focus on critical violations

that matter

Focus resources on areas

of highest impact rather

than pursuing hygiene

“Tracking programming

practices at the Unit Level

alone may not translate into the

anticipated business impact,[…]most devastating

defects can only be detected at the System Level.”

- OMG

8 CAST Confidential

Co

nfi

de

nti

al

Why We Do This: Better Communication with Business

Data Storage

Application

Business Questions

•Why does it always take so long to

make small improvements?

•Why does every new release take

weeks to stabilize no matter how much

it was tested? Why can’t it just work?

•Why do we have to scrap timelines to

fix critical defects every time we get

close to project completion?

•Why are we late again?

IT’s Struggle to Answer

Because the environment is so complex, IT

does not have answers that business can

easily understand. This causes frustration

and trust issues.

9 CAST Confidential

Co

nfi

de

nti

al

Why We Do This: Structural risk requires a 3-tier approach A

rch

itec

ture

Co

mp

lia

nce

Intra-technology architecture

Intra-layer dependencies

Module complexity & cohesion

Design & structure

Inter-program invocation

Security Vulnerabilities

Module Level

Integration quality

Architectural compliance

Risk propagation

simulation

Application security

Resiliency checks

Transaction integrity

Function point & EFP

measurement

Effort estimation

Data access control

SDK versioning

Calibration across

technologies

System Level

Data Flow Transaction Risk

Code style & layout

Expression complexity

Code documentation

Class or program design

Basic coding standards

Program Level

Propagation Risk

Java

JSP

EJB

PL/SQL

ASP.NET

Oracle

SQL

Server

DB2

T/SQL

Hibernate

Spring

Struts .NET

C# VB

COBOL

C++

COBOL

Sybase IMS

Messaging

Java Web

Services

APIs

1

2

3

CAST Confidential 10

Why We Do This: Quality impact at a major services brand

Measured impact in a complex enhancement-heavy environment

304 222 196

385 401

231 198 242 279

167 112

258 274

149 140 245

188

61 56 78 97 62 57 81 75 87 40

260

181 167

225 265

220

151

195

232

154

150

295

385

228 163

223

186

148 84

121 136

96 38 11 10 8 11

0

100

200

300

400

500

600

700

R1

R1.1

R1.2

R2

R2.1

R3

R3.1

R4

R5

R6

R7

R7.1

R8

R9

R9.1

R9.2

R10

R10.1

R10.2

R10.3

R11

R11.1

R11.2

R11.3

R12

R13

R14E

Code No RC Non Code Projected Count

Str

uctu

ral

qu

ality

S

yste

m t

est

defe

cts

Trend line

Before CAST implementation

0

500

1000

1500

2000

2500

3000

3500

R1

R1.1

R1.2

R2

R2.1

R3

R3.1

R4

R5

R6

R7

R7.1

R8

R9

R9.1

R9.2

R10

R10.1

R10.2

R10.3

R11

R11.1

R11.2

R11.3

R12

R13

R14E

Order Management Inventory Management Billing Customer Service

New critical violations

CAST Analysis starting point

CLIENT STUDY OVER 24 MONTHS

$2.7 million payback after 12 months