Project CPSC 542 Summer-08Ali Raza

aliraza@csn.fullerton.edu

I. Introduction

Software systems are becoming more challenging to build day by day especially in the enterprise environment. High qualified staff ensures that software products are built on time, under budget, and are of the highest quality with respect to the attributes such reliability, correctness, usability and the ability to meet all user requirements [1]. For the quality outputs, we perform verification and validation (V&V) process during the software life cycle.

Validation “Validation is the process of evaluating a software system or component during, or at the end of, the development cycle in order to determine whether it satisfies specified requirements.” [2]

In other words, validation is a process where we determine whether the product satisfies the specified user requirements as well as quality attribute such as reliability, scalability, and performance etc during or at the end of the SDLC.

Verification“Verification is the process of evaluating a software system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.”[2]

In other words, verification is a process where we determine whether conditions imposed at the beginning of each phase are satisfied for instance does the code reflect conditions imposed by the design.

Ever since complexity of the software systems have increased in an enterprise environment, it is very hard to release a successful product without using any V&V tools. If we research, we can see there are many tools available now for verification and validation of the software systems.

This paper will discuss two tools for V&V such as NUnit and Mercury Quality Center (MQC)

I. NUnit

URLhttp://www.nunit.org/index.php

Features of NUnit

o Introduction:NUnit is a simple unit-testing framework for the.NET platform which was ported from JUnit framework initially. It is an open source product. We can download from NUnit

website [4]. In SDLC, developers write unit test in order to test their modules and classes, and as a result they verify the code against the design of the software.

o IDE Support

We can integrate the NUnit framework with visual studio and automate our unit testing process. NUnit supports all the visual studio project type such as C#, VB.NET, J# and C++. When tests are run for a Visual studio project, they run just as if the output assembly had been loaded with one exception. The default location for the config file is the directory containing the project file and its default name is the same as the project file with an extension of .config. For example, the following command would load the tests in the nunit.tests assembly using the configuration file. nunit.tests.dll.config located in the same directory as the dll.

nunit.exe nunit.tests.dll

On the other hand, the following command would load the tests using the configuration file nunit.tests.config located in the same directory as the csproj file.

nunit.exe nunit.tests.csproj

The same consideration applies to running tests using the console runner. [4]

o Assertions

This is an important feature of NUnit which provides a central unit testing framework to the .Net applications. NUnit gives a rich set of assertions as static method of the Assert classes [1]. Therefore, if assertions fail during the unit testing process then method doesn’t return and an error is reported.

o Attributes

In earlier version of NUnit, we used to identify tests based on inheritance and naming conventions. But later on, NUnit has used custom attributes for this purpose. NUnit Framework namespace contains all the NUnit attributes. Each source file that contains test must include a using statement for that namespace and the project must reference the framework assembly, nunit.framework.dll. [4]

o Configuration Files

We can run various versions of the .NET framework by using the configuration files. This will be helpful when developers need to know how the system would behavior in different CLRs.

o Multiple-Assembly Support

NUnit allows loading suites of tests from multiple assemblies in both console and GUI. In any case a top level suite is constructed, which contains the root suite for each assembly.

NUnit tool facilitates software verification

As discussed above, NUnit is a product which will be used for unit testing. Unit Testing is a white box testing. White box testing starts at the method, functions and basic class level and progress through complex classes and sometime sub modules. In white box testing our focus will be on statements, branches, path, loops and data follows of the coding. NUnit tool facilities verification because we verify that the code reflects conditions imposed by the design phase.

NUnit tool will be used in development Phase

NUnit will be used during the implementation phase (coding) of the software life cycle. And developers will ensure that code has written as per design. Because by writing the unit tests themselves they will head start towards the quality product and bug free code.

Additional Comments

Unit testing is very useful technique for preventing the defects and NUnit is the perfect tool for .Net framework. I used NUnit way back when I worked on.Net technologies. I used to perform the unit testing before I handed over the code to SQA group, and if I make any changes to the code then I used to perform the regression testing to verify that everything working as expected. As a result SQA group find fewer defects which were mostly related to data issues.

II. Mercury Quality Center (HP Quality Center)

URLhttps://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24^1131_4000_100__

Features of Mercury Quality Center

o Introduction

Mercury Quality Center (MQC) is bought by HP. It is a web-based management tool which provides a centralized control over the entire testing project. MQC is used by the testers as well as managers. It provide an easy interface to manage and organize activities such as requirement coverage, management of test cases, detail reports after the execution of test cases, management of defect and test automation. [7]

We can divide MQC in to two parts. One is for administrator tools called Site Administrator and another one is for user tools called Quality Center Bin.

o Site Administrator (SABin)

It is the important part of the MQC and also the starting point. All the administrative tasks such as creating the project, assign resources or users to the project, creating specific role etc is done from this section. We can also configure other application such as WinRunner or Quality Test Professional (QTP). This section also used as mail servers, and for authentication. For instance we can configure MQC with active directory where user will login to MQC by using her/his windows network password and username. [7]

o Quality Center Bin (QCBin)

This part of MQC is related to the users who interact with the tool. Because it provides following functionality

- Creating test plans- Defining requirements- Creating test cases- Running test cases automatically- Associate requirement with defects. [7]

o Control Access

MQC provides the role based access once user logs into the MQC tool. For instance a test manager can create projects, test lead can prepare test plans and tester can write the test cases. Therefore, it is very easy to control access for various project artifacts and

also distribution of responsibility among the team. We can also manage and create roles by using SABin.

o MQC Modules

There are four important main modules of the MQC. Each module contains information about different projects.

- Requirement ModuleWe can organize the project requirement by using MQC requirement Tab. We can also create folder structure to represent product features. [8]

- Test PlanIn this module we can find all the information about the test cases. We can map those test cases to the product requirement therefore we can get the traceability matrix by using this feature of MQC.

- Test LabBy using test lab we can manage the test execution for specific release. For instance we can identify the set of test cases which are already under test plan and include them while execution. If the test cases are linked to the requirement then after each test cycle management will be able to trace which requirements have been tested as well as the project status

- DefectsDuring execution of the test cases, if any defects occur then they will automatically be logged into the defect tracking system of the MQC. And we get those defects list by using defects module.

Figure 1

o Report Generation

The most key feature of the MQC is the report generation process. Once we are done with the execution of test cases, we can generate many reports for instance list of defects, how many were pass, failed or in complete etc [8]

Requirement Test set (Test Lab)

Test Case(Test Plan)

Defects

o Integrate with other tools

We can also integrate MQC with other products for instance CaliberRM[9], Clear Quest for defect management and load runner.

MQC tool facilitates software verification and validation

By using MQC we can perform black box test. In black box testing approach we assumed that the inner program structure is not known [6]. SQA Staff perform black box testing once the white box testing will be finished and product will be handed over to the QA group. By using MQC tool, SQA team verify code by executing the test cases therefore during that time we are verifying whether the released code is working as expected or not for instance if code has to generate the PDF then QA will verify that PDF link is working or not. Therefore MQC tool can be used as a verification tool.

At the same time we can validate product by using MQC because with the help of this tool we can linked requirement to the test cases and by executing the test cases we would validate the product satisfies the specified user requirements.

We can linked Load Runner with MQC test cases, and validate whether system is performing as per requirement or not and validate all the quality attributes. Therefore MQC tool can be used as a validation tool

MQC tool will be used in testing and requirement Phase

During the requirement gather phase we enter all the requirements by using MQC requirement module therefore this tool can be used in requirement phase. Once code will be released into QA then all the test cases will be executed by using MQC test lab module therefore we use MQC in the testing phase.

Additional Comments

At my company we are using MQC very extensively but instead of using requirement module we are using CaliberRM tool for the requirements, Clear Quest for defect management and we linked those tools with MQC. However, for test cases and reporting purpose we use MQC, where SQA stuff automated the testing process.

SQA staff also runs their regression testing by using this tool. So far we are getting better result with this approach because not only the product quality is improving but also upper

management get the clear picture about the product status by using the reporting feature of the MQC.

III. Reference

[1] Practical Software Testing by Ilene Burnstein

[2] IEEE Standard Glossary of Software Engineering Terminology (Std610.12-1990), Copyright IEEE. All right reserved

[3] Lecture 1

[4] http://www.nunit.org/index.php

[5]https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24^1131_4000_100__

[6] Automated Defect Prevention by Dr. Huizinga and Dr Kolawa

[7] http://www.testinggeek.com/mqc.asp

[8] http://pagkis-software-testing.blogspot.com/2007/10/mercury-quality-centre.html

[9] http://www.borland.com/us/products/caliber/index.html