BCU New Wave in IT Risk Management · PDF fileStructural Quality Limits Business Agility...

A New Wave inIT Risk Management

Dr. Bill CurtisDr. Bill CurtisChief Scientist, CASTChief Scientist, CAST & SVP, CAST Research Labs& SVP, CAST Research LabsDirector, Consortium for IT Software Quality (CISQ)Director, Consortium for IT Software Quality (CISQ)

1 Keynote Curtis © CAST 2011

Increasing Risk in IT Applications

National Research CouncilNational Research CouncilS ft f D d bl S tS ft f D d bl S tSoftware for Dependable SystemsSoftware for Dependable Systems

“As higher levels of assurance areAs higher levels of assurance are demanded…testing cannot deliver the level of confidence required at a reasonable cost.”

“The cost of preventing all failures e cost o p e e t g a a u eswill usually be prohibitively expensive, so a dependable system will not offer uniform levels ofwill not offer uniform levels of confidence across all functions.”

“The correctness of the code is rarely the weakest link.”


Jackson, D. (2009). Communications of the ACM, 52 (4)

Increasing Complexity of IT Applications

ASP/JSP/VB/.NET

User Interface TierMiddle-ware

Enterprise Applications

Application Logic Tier

WebServices

Application Logic TierJava, C++, …

Frameworks Struts MVC, Spring

Legacy Applications

CICS Monitor (Cobol)

CICS Connector

Tuxedo Monitor (C)

Data Management Tier

COBOLCOBOLBatch

Shell Scripts

Database

gEJB – Hibernate

Preventing application level defects requires analysis of all thePreventing application level defects requires analysis of all the

DatabasesDatabasesFilesFiles


Preventing application level defects requires analysis of all the Preventing application level defects requires analysis of all the interactions between components of heterogeneous technologiesinteractions between components of heterogeneous technologies

Application Quality vs. Component Quality

Application QualityApplication quality measures how

well individual components interact t d li b i f tito deliver business functions

Component Quality

Code quality is the measure of individual components for

p y

individual components for compliance with standards and best practices in the t t f ifi lcontext of a specific languageApplication quality is

driven by structural quality

Good code quality Good code quality Good application qualityGood application quality


Good code quality Good code quality Good application qualityGood application quality

Structural Quality Is the Risk Issue

The degree to which a product Quality g pmeets its specified requirementsQuality

ProblemProblemCustomers struggle to state functional requirements. They do not understand non-functional requirements.They do not understand non functional requirements.

“ a failure to satisfy a non functional…a failure to satisfy a non-functional requirement can be critical, even catastrophic…non-functional requirements are sometimes difficult to verify. We cannot write a test case to verify a system’s reliability…The ability to associate code toreliability…The ability to associate code to non-functional properties can be a powerful weapon in a software engineer’s arsenal.”


Structural Quality Limits Business Agility

Ossified, contorted, complex legacy codep g y

Harmfully complex Hard to change Easy to hack into Easy to hack into Costly to maintain

Speed in responding to the market depends on the ease of modifyingthe ease of modifying business applications

S d b t Secure and robust Easy to change Cheap to maintain

Faster to build

Well-engineered, high-quality code

Faster to build


high-quality code

The 4th Wave in Software EngineeringWhat: Architecture, Quality characteristics, ReuseWhen: 2005Wh E ft i t t d t t d d

44Why: Ensure software is constructed to standards

that meet the lifetime demands placed on itProductProduct

What: CMM, ITIL, PMBOK, AgileWhen: 1990-2005Why: Provide a more disciplined environment for

33

Wh t D i th d CASE t l

Why: Provide a more disciplined environment for professional work incorporating best practicesProcessProcess

What: Design methods, CASE toolsWhen: 1980-1990Why: Give developers better tools and aids for constructing

22

What: 3rd & 4th generation languages structured programming

software systemsMethodsMethods

What: 3 & 4 generation languages, structured programmingWhen: 1965-1980Why: Give developers greater power for expressing their

11


programsLanguagesLanguages

Supplementing CMMI

Structural quality engineering supplementssupplements CMMI to unlock even more business value from applications

CMMI focus CMMI focus – process improvement – Six SigmaSQE focus SQE focus – product improvement – Design for Six Sigma

CMMI provides a strong foundation on which to build a rigorous application quality program:rigorous application quality program:

Level 2 – project quality practices Level 3 – standardized quality processesq y p Level 4 – statistical quality management Level 5 – quality innovations


CAST & Structural Quality Measurement

M VOracle PL/SQL

AD GOVERNANCE DASHBOARDAPPLICATION KNOWLEDGE BASEANALYZERS

A H

To assess, monitor, and improve

MANAGEMENT VISIBILITYOracle PL/SQLSybase T-SQL

SQL Server T-SQLIBM SQL/PSM

C, C++, C# Pro C R

ICS

APPLICATION HEALTH

Immediate Impact On-going Impact and improve applications,

development teams and 3rd Party

delivery teams

Pro CCobolCICS

Visual BasicVB.Net A

L M

ET

R Transferability

ChangeabilityPerformance

Security

Robustness

delivery teamsASP.NetJava, J2EE

JSPXML

HTML TE

CH

NIC

A

APPLICATION SIZE

Technical SizeFunctional

W i ht

DRILL-DOWN TO ACTION

Overview…JavascriptVBScript

PHPPowerBuilderOracle Forms

T Technical Size WeightPortfolio

Apps Health TOracle Forms

PeopleSoftSAP ABAP, Netweaver

TibcoB i Obj t

Sub-metrics

Rules

Modules

Objects

factors

CA

TIO

N

AD

ATA

TECHNICAL INVENTORY

Analysis of allBusiness Objects

Universal Analyzer for other

languages

t di ti

Rules

Compliance

AP

PLI

CM

ETA of all

system artifacts


… to remediation

Uses of Structural Quality MeasuresVendor

Managers

IT ExecutivesFIN

HR

IT Executives

DeliverablesCRM

ERP App / ProjectManagers

Deliverablesinsight

Portfolioinsight

ManagersDevelopers

insight

Applicationinsight Remedial


insight

AppmarQThe Missing Product Benchmark

Organization IT Spend St ffiLevel Staffing

Risks Robustness Performance Project

Timelines

CostsProductPerformance Security

Costs Changeability

jLevel

Size

QA

Level


Security Scores Differ by Language

Security scores by language.Distribution of Security Scores.4

3.5

3

ity

2

2.5

Secu

r

1.5

2

1.NET C/C++ COBOL Java EE Oracle

4GL

Bimodal distribution of security scores indicate two types of apps

4GLTechnologies


Apps with security scores are predominantly from Financial Services

Performance Scores Differ by Language

Performance scores by language.Distribution of Performance Scores.4

3 4

3.6

3.8

s

3

3.2

3.4

ance

Sco

res

2.6

2.8

Perfo

rma

2

2.2

2.4

.NET C/C++ COBOL Java EE Oracle 4GL

Technologies

Performance distribution is skewed towards higher scores Newer technologies show lower performance scores


Changeability Scores Differ by Sector

Changeability scores by industryDistribution of Changeability Scores.4

3 4

3.6

3.8

4

ores

2 8

3

3.2

3.4

eabi

lity

Sco

2 2

2.4

2.6

2.8

Cha

nge

2

2.2

litie

s

cial

s

ance

ultin

g

olog

y

ecom

urin

g

men

t

nerg

y &

Util

Fina

n

Insu

ra

IT C

onsu

Tech

no

Tele

Man

ufac

tu

Gov

ernm

Government applications show poor changeabilityO t i G t 75% I d t 50%

E Industry


Outsourcing: Government 75%Industry 50%

Modularity Minimizes the Effect of Size

COBOL Applications (TQI vs Size)

3 3

3,4

3,5

pp ( )

100

120

cts

3,1

3,2

3,3

ndex

(TQ

I)

60

80

mpl

ex O

bje

2,8

2,9

3

al Q

ualit

y In

40

60

of H

igh

Com

R² = 0.45332,6

2,7

2,8

Tota

0

20% o

2,510 100 1000 10000

Size - KLOC

.NET C/C++ COBOL Java EE Oracle 4GL

Technologies

Except for COBOL, size has no impact on application quality M d l it d th ff t f i lit


Modularity reduces the effect of size on quality

CISQ An Industry Initiative

Co-sponsorship

CISQCISQIT E ec ti es

Technical e pertsCISQCISQExecutives experts


CISQ Standards ProcessK l d Di M t d l

Function

Technical Work Groups Knowledge Discovery Meta-modelStructured Metrics Meta-model

ISOFunction Points Defined

Measures

ISO2500014143

MaintainabilityMeasures

27000

CISQExec Reliability &

Performance OMG BestP ti

ISO15939Forum Performance Practices 15939

Security

WeaknessesISO

17799Methods for Metrics Use

Weaknesses& Violations

17799CVSS


Code Pattern Metamodel

BCU New Wave in IT Risk Management · PDF fileStructural Quality Limits Business Agility...

Documents

Transcript of BCU New Wave in IT Risk Management · PDF fileStructural Quality Limits Business Agility...