2012 International Conference on Open Source Systems and Technologies (ICOSST)

978-1-4673-3097-8/12/$31.00 ©2012 IEEE

Architecture Based Software System Cost Estimation

Waqar Ul-Hasnain Khokhar

Department of Software Engineering and Computer

Science, Faculty of Basic and Applied Sciences

International Islamic University

Islamabad, Pakistan.

waqarulhk@gmail.com

Shahbaz Ahmed

Department of Software Engineering and Computer

Science, Faculty of Basic and Applied Sciences

International Islamic University

Islamabad, Pakistan.

shahbaz.ahmed@iiu.edu.pk

Abstract—Personal Integrated Process (PIP) is the extended form

of Personal Software Process (PSP). Processes for software

architecture design and evaluation are integrated in the PIP. The

PIP process is supported by a software quality management

system. This paper highlights the detail for estimation of cost for

software development and process improvement. The software

cost can neither be estimated by counting lines of code alone nor

by measuring time in software development alone. Therefore, this

paper proposes an integrated process for software system cost

estimation. Finally, the impact of integration of architecture

design and evaluation on cost and schedule is determined. The

scope of this paper is to highlights results of impact of

architecture design and evaluation on cost. 15% cost of software

development is associated with architecture design and

evaluation activities. This investment is quite reasonable and

helped in precise estimation of project cost and schedule.

Keywords— cost estimation; time estimation; and software

architectures

I. INTRODUCTION

Cost estimation is the core activity for any software development project. Process variations highly impact cost and schedule estimation. The focus of this research was to find the impact of integration of architecture design and evaluation on cost and schedule for Personal Software Process (PSP) engineer. PSP [1] is the class room implementation of CMMI [2]. It contains most of the CMMI processes. However, PSP is designed for individual software engineers or for small team of three to five engineers. Team Software Process (TSP) is designed for managing large team of software engineers and has been integrated with Architecture-Centric Engineering [3]. However, the proposed process is for team managers or coaches and not for individual software system engineers. The related work about the integration or modification of Personal Software Process (PSP) is given in the section 2.

The limitations in PSP were addressed by integrating architecture design and evaluation processes. The Personal Integrated Process (PIP) is the extended form of PSP and contains processes for architecture design and evaluation [4]. The process implementation in brief is given in section 3.

Personal Integrated Process (PIP) is supported by a software tool “FSQMS”. This software quality management system is briefly described in section 4.

To achieve the objectives of the research a case study was conducted. Personal Software Process and Personal Integrated Process were executed in a university computer lab. The details of case study implementation can be found in section 5.

Section 6 is given for addressing impact of architecture design and evaluation on cost and schedule. Research conclusion is given in the section 7. Future work is addressed in section 8, and finally the references are given in the section 9.

II. RELATED WORK

Various solutions for the process integration and modification in PSP have proposed. The detail of process integration, modification and automation can be found in a review paper [5]. The selection of architecture design and evaluation processes was based on an analysis carried out during a literature review. Detail of the analysis can be found in a review paper [6]. The main objective of this research was to find the impact of architecture design and evaluation processes on cost and schedule. The architecture design and evaluation processes involve many non-coding activities. These activities consumes substantial amount of process time as discussed by Capers Jones.

“Indeed, we now know that on some projects (such as large defence systems) the cost to produce paper documents is twice as much as the cost to produce the code itself. [7]”

However, cost estimation in Personal Integrated Process is carried out by counting lines of code and by measuring the time spent in different activities including architecture design and evaluation. The integration of these two concepts helped in better estimation of project cost and time. The PIP is designed for engineering software system. The PIP includes process for software engineering, database engineering, network designing, and computer hardware selection. All these processes are required for better estimation of cost for software system.

III. PIP PROCESS IN BREAF

The core process components of Personal Integrated Process (PIP) are Quality Attribute Workshop (QAW) [8], Attribute-Driven Design (ADD) [9], Active Reviews for Intermediate Designs (ARID) [10], Architecture Tradeoff Analysis Method (ATAM) [11], Cost Benefit Analysis Method (CBAM) [12], Data Modeling [13], Business Modeling, and Requirement Engineering Process [14]. These process components were tailored for the needs of individual engineer or for small team of three to five engineers. The PIP process considers two types of planning i.e. bottom up planning and top down planning. The top down planning is designed for TSP coaches or managers. Top down planning is carried out using historical data such as software size, development time and defects. However, bottom up planning is carried out using standard PSP scripts. The time spent during standard PSP process is estimated by counting lines of code. The bottom up

2012 International Conference on Open Source Systems and Technologies (ICOSST)

and top down historical data is compared for better estimation of software size, time and defects. The software architecture is documented based on the guidelines given in [15].

Personal Integrated Process contains many non-coding process components. The planning and estimation during these activities is carried out based on personal experience, judgment and historical data of time spent in processes.

IV. FIRMSOFT SOFTWARE QUALITY MANAGEMENT

SYSTEM

Firmsoft Software Quality Management System (FSQMS) is the tool design for logging process data and tracking process progress. Personal Software Process (PSP) data or Personal Integrated Process (PIP) data can be logged using the tool. FSQMS is designed using OracleJDeveloper Fusion Web Application [16]. The detail regarding FSQMS components, operations, aims and objectives can be found in [17].

V. CASE STUDY

A single case holistic design case study was conducted in

university computer lab. The guidelines for planning,

execution and reporting case study were taken from articles on

case study research [18], [19], [20]. The research question of

the case study is given below: RQ: What is the impact of architecture design and

evaluation on Risk, Schedule and Cost estimation for PSP Engineer?

Before conducting case study a literature review was carried out. Process components were selected for integration with PSP.

Three types of validity were considered which includes construct validity, internal validity, and external validity. Whole population was considered as sample size. Case study protocol [21] was designed and used during case study execution. The case study domain was business information system. The case study limitations will be addressed by conducing case study for web and database enabled applications.

Retrospective-prospective was the nature of case study. There were two contacts points in the case study design. No control group was considered in the case study design. It was a single case holistic design case study.

OracleJDeveloper Fusion Web Application technology was used in the case study execution. The case study took about one year. In one year time of case study the major activities involve case study protocol design, execution, process learning, analysis, and finally publishing research results and findings.

VI. COST ESTIMATION

The software cost estimation of business information system is not easy. The development of such system involves many activities some of them are coding and other are non-coding. The non-coding activities may include requirement negation, requirement elicitation, requirement engineering, design and evaluation of architecture, reviews and inspections conducted during component and module developments. The coding activities involve implementation of architecture, components and modules. Whether using code generation technology or not. The software cost associated with these activities can be estimated using lines of code, function point,

use case point or UML metrics. The software and software system cost associated with the non-coding activities is the core focus of this research.

Other objective of this research was to find the impact of architecture design and evaluation on cost estimation for PSP engineer. To achieve the objectives of research a case study was conducted in a university computer lab. Results of this case study are discussed here.

Cost Benefit Analysis Method (CBAM) was used for estimation of software system cost, benefit evaluation and for Return on Investment (ROI) calculation. These calculations were used for cost/benefit analysis. The main inputs of the CBAM from ATAM are quality attributes goals, high priority scenarios, architectural views, utility tree and architectural risks. Results and analysis performed during architecture design and evaluations are discussed in [22]. Total nine use case scenarios, two growth scenarios, and three exploratory scenarios were identified during QAW. Out of eleven candidate architectural drivers four are selected as architectural drivers. These four architectural drivers with high stakeholder priority and high impact on architecture are selected for further investigation. Utility tree was created for these candidate architectural drivers. ATAM was applied after ADD activities. 33.34% availability risks, 22.22% modifiability risks, 22.22% reliability risks, 11.11% performance risks, 11.11% security risks were identified.

During first step of CBAM architectural strategies are selected for each use case scenario. The analysis performed during first step is given in the Table I. For each use case scenario appropriate architectural strategy is selected and desired improvement to the architectural strategy is explained.

In step 2 of CBAM quality attribute score is established based on the stakeholder priority and impact on architecture. The analysis performed in second step is given in the Table II. Graphical description of the analysis is shown in the Fig. 1. In step 3 the benefit score was evaluated for each architectural strategy. The benefit score range was from +100 Benys to -100 Benys. There are extensive calculations involves during step 3. Therefore, only the results of this step are discussed here. In step 4 the cost and schedule implications are analyzed. The cost associated with each architectural strategy is calculated. In step 5 the desirability for each architectural strategy is calculated. The result of the analysis performed during these steps is given in the Table III.

TABLE I. ARCHITECTURAL STRATEGIES

Scenario

ID

QA Goal Architectural

Strategy ID

Architectural

Strategy

Description

Desired

Improvement

UCS2 Security AS-001 Using hard

coded security

Reducing

development

time.

UCS2 Security AS-002 Using JAAS

Permission for

business tier.

Increasing

business tire

security.

UCS2 Security AS-003 Using ADF

Authentication

and

Authorization

Reducing

development

time by using

integrated

authorization.

2012 International Conference on Open Source Systems and Technologies (ICOSST)

UCS2 Security AS-004 ADF

Authentication

Using built-in

facility.

UCS2 Security AS-005 HTTP Basic

Authentication

Reducing

development

time.

UCS2 Security AS-006 HTTP Digest

Authentication

Reducing

development

time.

UCS2 Security AS-007 HTTPS Client

Authentication

Using

enhanced

security.

UCS2 Security AS-008 Form-Based

Authentication

Using optimal

security

TABLE II QUALITY ATTRIBUTE SCORE

Scenario

ID

Quality

Attribute

Quality

Attribute

%

QAscore

Stakeholder

Priority

Impact on

Architecture

Total

UCS1 Performance 13.88 M 2 H 3 5

UCS2 Security 16.67 H 3 H 3 6

UCS3 Availability 11.11 L 1 H 3 4

UCS4 Reliability 8.33 L 1 M 2 3

UCS5 Reliability 5.56 L 1 L 1 2

UCS6 Modifiability 5.56 L 1 L 1 2

UCS7 Modifiability 11.11 H 3 L 1 4

UCS8 Reliability 11.11 L 1 H 3 4

UCS9 Availability 16.67 H 3 H 3 6

Total %

Score

100 Total Score 36

TABLE III COST, BENEFIT AND DESIRABILITY RESULTS

Architectural

Strategy

Cost (US

$)

Benefit

(Benys)

Desirability

Benefit/Cost

(Benys/$)

AS-001 583.275 85 0.1457

AS-002 983.235 86.67 0.0881

AS-003 516.615 90 0.1742

AS-004 749.925 90 0.1200

AS-005 549.945 86.67 0.1576

AS-006 549.945 91.67 0.1667

AS-007 949.905 91.67 0.0965

AS-008 749.925 100 0.1333

The cost/benefit analysis performed after this step is shown in the Fig. 2. The architectural strategy AS-3, AS-5 and AS-6 have high benefit/cost results. However, AS-6, AS-7 and AS-8 have high benefit. Therefore, these architectural strategies were filtered for further consideration.

Figure 1. Cost Analysis

Figure 2. Cost-Benefit Analysis

During implementation architectural strategy AS-8 was selected. This strategy was selected based on the security requirement for the software quality management system. The Oracle ADF Form-Based security was implemented. The security was implemented for only two users. However, future work includes the security implementation for 500 users. Similarly, other quality attributes will be evaluated for these constraints.

Figure 3. Activity Based Time Analysis-1

The time spent during architecture design and evaluation is graphically shown in Fig. 3. ATAM and CBAM took most of the time spent during architecture design and evaluation. CBAM took 32.95% of the total time spent during architecture design and evaluation.

2012 International Conference on Open Source Systems and Technologies (ICOSST)

Figure 4. Activity Based Time Analysis-2

The time spend during coding and non-coding activities is shown in the Fig. 4. The 85.16% time was spent in JSF, SQL, and XML based code implementation activities whereas the time spent in QAW, ADD, ARID, ATAM, and CBAM was 14.84%.

Figure 5. Activity Based Time Analysis-3

The time spent in JSF, SQL, and XML code implementation is separately shown in Fig.5. The time spent in non-coding activities QAW, ADD, ARID, ATAM, and CBAM is separately shown in the same figure.

VII. CONCLUSION& FUTURE WORK

Different analysis performed during the case study shows that architecture design and evaluation have impact on cost of software system. About 15% of the cost of software development is associated with the architecture design and evaluation. However, with this investment software system engineer becomes able to estimate the project cost and time at architecture level. If this cost analysis was not performed in the early stage, the estimation of project cost would be difficult and misleading.

The cost of the software system including cost of software development, cost of computer hardwareand cost of network components, cost of software components, and cost of infrastructure will be estimated and compared with the actual results using PIP. The analysis will help in better estimation and planning of large software projects.

REFERENCES

[1]. W.S. Humphrey, A Discipline for Software Engineering. Pearson Education Asia. ISBN 81-7808-435-X

[2]. CMMI Product Team, “CMMI for Development,” Version 1.2, CMU/SEI-2006-TR-008, Carnegie Mellon University, Software Engineering Institute, August 2006.

[3]. R.L. Nord, J. McHale, F. Bachmann, “Combining Architecture-Centric Engineering with the Team Software Process,” Technical Report CMU/SEI-2010-TR-031., Carnegie Mellon University, Software Engineering Institute, December 2010.

[4]. H.K. Waqar, A. Shahbaz, “Personal Integrated Process (PIP),” unpublished.

[5]. H.K. Waqar, A. Shahbaz, “A Literature Review & Recommendations on Personal Software Process Tools," International Journal of Reviews in Computing, vol. 9, no. 4, pp. 26-33, April 10, 2012.

[6]. H.K Waqar, A. Shahbaz, “A Literature Review and Recommendations on Software Architecture Evaluation,” International Journal of Reviews in Computing, vol. 10, no. 4, pp. 28-35, July 31, 2012.

[7]. Jones, “Activity-based software costing,” IEEE Computer, vol. 29, no. 5, pp 103-104, May 1996, doi: 10.1109/2.494092.

[8]. M.R, Barbacci, R. Ellison, A.J. Lattanze, J.A. Stafford, C.B. Weinstock, W.G. Wood, “Quality Attribute Workshops (QAWs),” Third Edition, Technical Report CMU/SEI-2003-TR-016, August 2003.

[9]. R. Wojcik, F. Bachmann, L. Bass, P. Clements, P. Merson, R. Nord, B. Wood, “Attribute-Driven Design (ADD),” Version 2.0. Technical Report CMU/SEI-2006-TR-023, November 2006.

[10]. P.C. Clements, “Active Reviews for Intermediate Designs,” Technical Note CMU/SEI-2000-TN-009, August 2000.

[11]. R. Kazman, M. Klein, M. Barbacci, T. Longstaff, H. Lipson, J. Carriere, “The Architecture Tradeoff Analysis Method,” Technical Report CMU/SEI-98-TR-008, July 1998.

[12]. R. Kazman, Jai Asundi, M. Klein, “Quantifying the Costs and Benefits of Architectural Decisions,” IEEE, ICSE 2001, Proceedings of the 23rd International Conference on Software Engineering, pp. 297-306, 2001, doi: 10.1109/ICSE.2001.919103.

[13]. F.R. McFadden, J.A. Hoffer, Modern Database Management. Fourth Edition, The Benjamin/Cummings Publishing Company, Inc.

[14]. Larman, Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development. Third Edition, Addison Wesley Professional, ISBN: 0-13-148906-2, October 20, 2004.

[15]. P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, R. Nord, J. Stafford, Documenting Software Architectures, Second Review Edition, Addison Wesley, November 1, 2001.

[16]. Fusion Developer's Guide for Oracle Application Development Framework, 11gRelease2 (11.1.2.0.0) E16182-01, May 2011.

[17]. H.K. Waqar, A. Shahbaz, “Software Quality Management System,” Journal of Emerging Trends in Computing and Information Sciences, vol. 3, no. 8, pp. 1205-1212, August 2012..

[18]. Kitchenham, L. Pickard, S.L Pfleeger, “Case Studies for Method and Tool Evaluation” IEEE Software, vol. 12, no. 4, pp 52-62, July 1995, doi: 10.1109/52.391832.

[19]. D.E. Perry, S.E. Sim, S.M. Easterbrook, “Case Studies for Software Engineers,” Proceedings of the 26th International Conference on Software Engineering (ICSE’04) IEEE Computer Society, 2004.

[20]. M. Host, P. Runeson, “Checklists for Software Engineering Case Study Research,” First International Symposium on Empirical Software Engineering and Measurement, pp 479-481, doi: 10.1109/ESEM. 2007.46, IEEE Computer Society.

[21]. P. Brereton, B. Kitchenham, D. Budgen, Z. Li, “Using a Protocol Template for Case Study Planning,” EASE 2008.

[22]. H.K. Waqar, A. Shahbaz, “Personal Integrated Process Based Software Architecture Design and Evaluation,” unpublished.