CAST Confidential 1

CRASH Special Report Impact of JEE frameworks on the structural quality of applications

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

CRASH Special Report – JEE Framework Analysis Overview

Objective: – Understand the impact of frameworks on structural quality – reliability, security and

performance of applications – Understand the impact on structural quality of combining JEE applications with other

technology

Data Sample: – Source: CAST Appmarq benchmarking database – 496 JEE-based business applications – 152 million lines of code – Frameworks analyzed: Hibernate, Spring, Struts

Findings: – Applications with only Hibernate tend to be of higher quality – Applications with Hibernate + Spring tend be of larger size – Huge variation in quality of applications with no frameworks – Majority of JEE applications don’t use frameworks – Overall quality tends to degrade for JEE applications when combined with C/C++ – Pure JEE applications tend to be of higher quality

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

Appmarq benchmarking repository – largest database of its kind with structural quality data from source code analysis

Across Industries

Across Technologies

Across Development

Regions

Application Quality Data From CASTAIP

Central Benchmarking Database

Anonymized / Normalized

Benchmarking

Industry Trends

Best Practices & Standards

Development

Appl

icat

ion

Sour

ce C

ode

Anal

yzed

CAST Confidential

Code, Architecture & Data structure

Analysis

Quality & Sizing

Measures

Compliance Checks

All data gathered from the automated analysis of business applications using CAST Application Intelligence Platform (AIP)

RDBMS

C++

WEB

SAP Siebel Java

Oracle

.NET DB2

BI EAI

Cobol

The CAST Application Intelligence Center

CAST Application Intelligence Platform

Actionable visibility & measures for all:

Reports on application health, team performance, benchmarking … CxOs and VPs

… Dashboard featuring software characteristics,

cost and risk drivers, root cause analysis… Management

… and feedback & advice on software quality and engineering

Dev Teams

Copyright CAST 2011 6

CAST AIP - Leading Software Analysis and Measurement Platform

CAST Confidential

CAST AIP: Structural Quality Characteristics

1. Robustness: the stability of an application and the likelihood of introducing defects when modifying it

2. Performance: the responsiveness of an application 3. Security: an application’s ability to prevent unauthorized

intrusions 4. Transferability: the ease with which a new team can

understand the application and quickly become productive working on it

5. Changeability: an application’s ability to be easily modified 6. Total Quality Index: a composite score computed from the five

Quality Characteristics listed above

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

Data Sample for this Report

JEE Frameworks Groups # Apps # Cos

Sum of KLoC

Struts 1.x 64 33 11,713.39

Hibernate 24 13 3,265.23

Hibernate-Spring 29 20 14,439.43

Hibernate-Struts 1.x 37 19 10,319.72

No Frameworks 170 49 31,323.39

Grand Total 324 - 71,061.16

JEE Technology Mix # Apps # Cos

Sum of KLoC

JEE-.NET 18 11 9,343.66

JEE-C 19 9 8,186.32

JEE-C++ 17 7 15,247.45

JEE-Cobol 50 9 20,975.09

JEE-Mixed 29 16 17,689.99

JEE 120 20 16,914.93

JEE-DB 243 56 63,992.35

Grand Total 496 - 152,349.79

JEE Applications by different frameworks

JEE Applications in combination with other technologies

CAST Confidential

JEE Frameworks: Wide range by application size

Wide range of applications: 10 KLOC to > 2,4 MLOC

CAST Confidential

JEE Frameworks: Primarily from 6 industry segments

Industry Segment Struts 1.x Hibernate Hibernate-

Spring Hibernate-Struts 1.x

No Frame-works

Grand Total

Financial Services 9 3 10 5 67 94

Government 5 3 2 2 9 21

Insurance 9 4 2 7 13 35

Manufacturing 11 7 2 2 12 34

Telecommunications 13 3 8 9 32 65

Utilities 5 0 1 1 13 20

Other 12 4 1 11 17 45

Grand Total 64 24 29 37 170 324

CAST Confidential

JEE Frameworks: Good representation of shoring & sourcing type

Sourcing Type Struts 1.x Hibernate Hibernate-

Spring Hibernate-Struts 1.x

No Frame-works

Grand Total

Primarily In-House 29 9 10 18 62 128

Primarily Outsourced 34 15 15 19 85 168

Grand Total 63 24 25 37 147 324

Shoring Type Struts 1.x Hibernate Hibernate-

Spring Hibernate-Struts 1.x

No Frame-works

Grand Total

Primarily Offshore 16 3 7 8 30 64

Primarily Onshore 47 21 16 29 117 230

Grand Total 64 24 29 37 170 324

CAST Confidential

JEE with Technology Mix: Primarily from 6 industry segments

Industry Segment JEE - .NET

JEE – C

JEE – C++

JEE – COBOL

JEE – Mixed JEE

JEE – DB

Grand Total

Financial Services 9 1 7 44 10 37 70 178

Government 0 0 0 0 0 12 9 21

Insurance 4 1 0 3 1 8 30 47

Manufacturing 0 2 0 0 3 20 26 51

Telecommunications 1 3 8 0 10 14 57 93

Utilities 3 6 1 0 1 10 10 31

Other 1 6 1 3 4 19 32 66

Grand Total 18 19 17 50 29 120 243 496

CAST Confidential

JEE Technology Mix: Good representation of shoring & sourcing type

Sourcing Type JEE-.NET JEE-C

JEE-C++

JEE-Cobol

JEE-Mixed JEE

JEE-DB

Grand Total

Primarily In-House 6 6 12 29 16 37 112 218

Primarily Outsourced 10 12 5 20 12 66 120 245

Unknown 2 1 0 1 1 17 11 33

Grand Total 16 18 17 49 28 103 232 496

Shoring Type JEE-.NET JEE-C

JEE-C++

JEE-Cobol

JEE-Mixed JEE

JEE-DB

Grand Total

Primarily Offshore 7 1 8 7 8 26 48 105

Primarily Onshore 9 17 8 41 20 77 180 352

Unknown 2 1 1 2 1 17 15 39

Grand Total 16 18 16 48 28 103 228 496

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

JEE Frameworks Groups # Apps # Cos

Combined Size of all

Apps (KLoC) Average App Size (KLoC)

Struts 1.x 64 33 11,713 183

Hibernate 24 13 3,265 136

Hibernate-Spring 29 20 14,439 498

Hibernate-Struts 1.x 37 19 10,320 279

No Frameworks 170 49 31,323 184

Grand Total 324 - 71,061 219

Sample indicates that majority of JEE applications still don’t use frameworks

JEE Applications by different frameworks

Majority of the applications that don’t use frameworks could be part of older generation applications

As one of the first frameworks, Struts could be used on more applications than other frameworks

Applications with Hibernate-Spring combination are significantly larger in size than other JEE applications

CAST Confidential

Applications that use Hibernate have higher quality scores than other frameworks

CAST Confidential

JEE applications that don’t use Hibernate or Spring have big variance in quality

Reasons for this variation could be: – These application tend to be of older generation, as discussed earlier (or) – Hibernate and Spring enable developers to build applications in a more

consistent manner

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

Applications with cross technology components are larger than pure JEE applications

JEE Technology

Groups # Apps # Cos Sum of KLoC

Average App Size

KLoC

JEE-.NET 18 11 9,344 519

JEE-C 19 9 8,186 431

JEE-C++ 17 7 15,247 897

JEE-Cobol 50 9 20,975 420

JEE-Mixed 29 16 17,690 610

JEE 120 20 16,915 141

JEE-DB 243 56 63,992 263

Grand Total 496 152,350 307

Applications with C++ components are significantly larger than other combinations of technologies

CAST Confidential

JEE-Technology Mix: Distribution across industries

Industry Segment JEE-.NET JEE-C

JEE-C++

JEE-Cobol

JEE-Mixed JEE

JEE-DB

Grand Total

Financial Services 9 1 7 44 10 37 70 178

Government 0 0 0 0 0 12 9 21

Insurance 4 1 0 3 1 8 30 47

Manufacturing 0 2 0 0 3 20 26 51

Telecommunications 1 3 8 0 10 14 57 93

Utilities 3 6 1 0 1 10 10 31

Grand Total 17 13 16 47 25 101 202 421

Pure JEE applications are spread across industries As expected, JEE-Cobol apps are concentrated in the Financial Services sector C, C++ are more popular in Financial Services and Telecom industries

– They are frequently used in embedded systems in telecom equipment – They are also the language of choice for low-level programming needed for speed in

trading systems

CAST Confidential

JEE applications with C/C++ tend to be of lower quality

This finding is consistent with 2011 CRASH Report C/C++ programming languages tend to be lower level, closer to machine

language, harder to program, and have more scope for making mistakes

CAST Confidential

JEE + .NET behave similar to pure JEE applications

This may be because they both are newer generation languages with higher level of abstraction

They also simplify development and the bulk of common development tasks are standardized

CAST Confidential

Content

1. CRASH Special Report – JEE Framework Analysis Overview 2. Approach and Terminology 3. Data Summary 4. JEE Frameworks Analysis – Key Findings 5. JEE Technology Mix Analysis – Key Findings 6. Conclusions

CAST Confidential

Conclusions

In addition to ease of development, frameworks do enable higher quality of applications

Larger more complex applications tend to use a combination of frameworks in their architecture (Hibernate + Spring)

Special attention should be given to quality when mixing JEE with different technologies – Cross-technology applications tend to be more complex – Developers are often familiar with only one technology – Applications with older generation low-level languages like C, C++

can be of lower quality to begin with

CAST Confidential

Contact Information

Pete Pizzutillo

p.pizzutillo@castsoftware.com

www.castsoftware.com

blog.castsoftware.com/

linkedin.com/company/cast

@OnQuality

slideshare.net/castsoftware