Integrate model-based testing to find qualityproblems early in developmentAdvantages of using the Rational Rhapsody TestConductorAdd On with Rational Quality Manager throughout the softwaredevelopment process

Skill Level: Introductory

Marc Lettrari, PhD (marc.lettrari@btc-es.de)Vice President, Rhapsody and Statemate Testing ProductsBTC Embedded Systems AG

05 Apr 2011

Take advantage of the integration of the Rational Rhapsody TestConductor Add Onwith Rational Quality Manager to enhance your model-based testing inside apowerful generic framework. This article shows how the integration helps enablerisk-based testing to assist QA teams as they prioritize testing of specific featuresand functions, based on their importance in the overall project and likelihood orimpact of failure. Combining this with new reporting dashboards gives productmanagers a more realistic view of product performance according to set businessobjectives to better ensure that software development projects stay on track.

Managing model-based tests with Rational Quality Manager

Quality assurance teams are often brought into the development lifecycle too late,after requirements are set and errors introduced into the design. To remedy this,IBM has integrated the Rational® Rhapsody TestConductor Add On with Rational®Quality Manager software to provide a live test that span the entire product life cycle.The result provides a consolidated view from requirements to product delivery. Thisarticle describes how to leverage this integration by bringing model-based tests intoyour overall testing process.

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 1 of 13

This integration works by using the Rational Quality Manager solution to managedifferent kinds of tests cases, their execution and results executions. This tooldisplays requirements, test cases, and other resources in one server-based view,which helps geographically dispersed team members exchange information in realtime. The Rhapsody TestConductor Add On uses the UML testing profile toautomatically specify the test architectures, to visualize test cases for improvedunderstanding, and to execute the tests to pinpoint design model failures andimplementation errors.

The article also shows how Rhapsody TestConductor test cases can be managedand incorporated into the overall quality management process within RationalQuality Manager. It explains how the integration helps enable risk-based testing,thus assisting QA teams as they prioritize testing of specific features and functionsbased on their importance in the overall project and likelihood or impact of failure.The ability to prioritize, combined with new reporting dashboards in the software,offers product managers a more realistic view into product performance against setbusiness objectives to better ensure that your project stays successfully on track.

Quality management throughout the software development lifecycle

Today's tech-savvy consumers expect high quality from products they use.Delivering anything less than that can result a company reeling from a perception ofpoor quality for years to come. Consumers also expect advanced features, whichdrives complexity. The amount of software within embedded products is increasingand driving key product differentiation.

With these challenges of quality, complexity, and size, companies need to optimizetheir software testing to achieve desired business outcomes. Often, software testingis done late in the development cycle, which leads to finding the majority of defectsnear the end, when they are the most costly and riskiest to address. However, thesedefects were introduced into the design early in the development process. Softwaredevelopers perform tests while developing software, such as unit tests, but thesetests tend to be very isolated to the developer or inconsistent across developers.Software developers need to focus on developing, so testing becomes an area theytake shortcuts on when a deadline looms.

The advantage of model-driven testing

Fortunately, emergence of the Unified Modeling Language (UML) and SystemsModeling Language (SysML) standards has provided the opportunity forbreakthrough gains in streamlining the development process. These standardsenable engineers and software designers to significantly improve their productivityby transitioning from a code-based development process to model-drivendevelopment (MDD). Using models, systems engineers and software developerscan more clearly understand and analyze complex requirements, automate repetitivetasks, maintain consistency across design activities, test systems concepts earlier

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 2 of 13

by using simulation, and automatically generate code for direct deployment on thetarget hardware.

Model-based testing extends model-based development beyond design to testing.The Rational Rhapsody TestConductor Add On adds model-based testingcapabilities based on the UML Testing Profile to the Rational Rhapsody Developer,Rational Rhapsody Designer for Systems Engineers, or Rational Rhapsody Architectfor Software editions. The testing profile tailors the development environment fortesting by adding concepts such as test architectures and test behaviors into UML.Test architectures extend the existing UML 2.0 structural concepts to describe theelements involved in a test and their relationships. Similarly, the test behaviorextends the existing UML 2.0 behavioral concepts to encompass all observationsand activities during the test.

The TestConductor Add On automates the testing activities by creating a testarchitecture and enabling tests to be graphically captured by using UML sequencediagrams, state charts, or flowcharts. Capturing tests graphically allows bettercommunication of the tests and further elicits the behavior of the design. The testscan be executed and results monitored to automate unit and regression testing.

By extending testing forward in the process to include the design model, then qualitymanagers, and software engineers can check that each feature meets requirementsearly in development so they can assess the overall quality of the design early.

Overview of Rational Quality Manager

IBM® Rational® Quality Manager is a collaborative, role-based, business-drivenenvironment for test planning and execution, workflow control, tracking, and metricsreporting. It provides a web-based centralized test and quality managementenvironment for project managers, quality engineers, and other project stakeholdersto collaborate and share information about the current quality of the project.

Rational Quality Manager quantifies how project decisions and deliverables affectand align with business objectives and outcomes. It helps the team reduce defectsand accelerate product delivery schedules.

Using this software, you can create a customizable dashboard for quick and easyreview of systems quality metrics and project status, such as builds ready to betested. It also provides traceability features that can help test teams ensure that testplans are covering all requirements. The dashboard helps highlight areas wherequality is at risk in the project and where priority should be focused.

Analyzing and elaborating requirements using executablemodels

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 3 of 13

The starting point of a development project is the requirements analysis phase. Theobjective of this phase is to analyze the process inputs. Stakeholder requirementsare translated into system requirements that define what the system must do(functional requirements) and how well it must perform (quality of servicerequirements). In a model-based development process, using IBM RationalRhapsody as the underlying UML modeling tool, this means that the initialstakeholder requirements (which are stored in IBM® Rational® DOORS®requirements management software, for example) are imported into therequirements model, which is depicted in Figure 1.

After the stakeholder requirements are imported, a model-based requirementsanalysis can be performed. This means that a UML model is created that capturesthe given requirements and can be used for analyzing them. With RationalRhapsody, it is possible to create executable models, allowing analysis of therequirements by executing the model derived from these requirements for validationearly in the development lifecycle.

Figure 1. Requirements from Rational DOORS are imported and furtheranalyzed and validated in a Rhapsody model

Validating requirements with model-based testing

As described in the previous section, based on the initial stakeholder requirements,an executable UML requirements model has been created that serves as areference model for the intended behavior of the system to be developed. However,if the requirements model should serve in later development stages as a referencemodel, it must be verified that the behavior of the model indeed fulfills the givenrequirements. This verification step can be done and automated with model-basedtesting.

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 4 of 13

Model-based testing enables dynamic testing of the behavior of the model, meaningthat test cases can be specified (for example, using UML sequence diagrams) andautomatically executed on the model level. The UML Testing profile (see the secondreference in Resources) provides definitions of all concepts and terms needed tomodel test architecture, test behavior, and test data within the UML modelinglanguage. For UML models created with Rational Rhapsody, the TestConductor AddOn provides all needed capabilities and a high degree of automation to help youcreate and execute model-based test cases.

Similar to dynamic software testing, dynamically testing the model ensures that thebehavior of the model is indeed correct with regard to the given requirements.Furthermore, given that both requirements and test cases are part of the model, theycan be linked directly, which ensures complete traceability from requirements to testcases and test results. Figure 2 shows an example of a test case that is describedwith a sequence diagram.

Figure 2. A sequence diagram can specify inputs and expected results of testcases

Integrating model-based testing into the quality managementprocess

It is widely accepted that quality management throughout the whole developmentlifecycle is necessary to get a high-quality product. Rational Quality Manager helpsoptimizing project quality with a single, shared test management hub that provides

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 5 of 13

integrated lifecycle support across many types of testing. It provides a customizable,role-driven solution for test planning, creation, and execution. as well as workflowcontrol, tracking, and end-to-end traceability.

One important aspect of integrated quality management is proactively managingprojects and quality risks with real-time, fact-based insights and reporting to ensurethat project decisions and deliverables remain aligned with your business objectives.At any time during the development lifecycle, quality management should be able toanswer the question: "Are we still on track?"

Especially in early development phases, such as during the functional systemspecification phase, answering this question can sometimes be hard, becausedynamic software testing cannot be performed because there is no softwareimplementation available at this stage. However, if a model-based approach is usedfor system specification, so that an executable model is already available,model-based testing can already be applied at this early stage to assess to whatextent the original stakeholder requirements are met by the current systemspecification. This means that, despite being quite early in the development phase,you can already measure the quality status of the current project.

How to integrate model-based testing into the quality management process

For Rational Quality Manager and the Rhapsody TestConductor Add On, there is aspecific adapter that you can use to access model-based tests directly from RationalQuality Manager. After the adapter is started, it automatically connects to RationalQuality Manager.

Thereafter, when you create a test script in Rational Quality Manager, you canconnect it to an executable model-based test in Rhapsody by following these steps:

1. Create a test script in Rational Quality Manager (see Figure 3). InRational Quality Manager, each test case needs a test script in order toexecute the test case.

Figure 3. Create a test script in Rational Quality Manager to execute the testcase

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 6 of 13

Larger view of Figure 3.

2. After specifying details such as the test script name and so forth, selectRhapsody TestConductor in the Type field of the test script (see Figure4). This tells Rational Quality Manager that the test script is actuallyimplemented as a model-based test in Rational Rhapsody.

Figure 4. Specify Rational TestConductor as the type of test script

Larger view of Figure 4.

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 7 of 13

3. After specifying that the test script is based on a model-based test, youneed to define which model-based test actually implements the test script.To do this, from the Create Test Script dialog window, click Selectadapter.

4. When a dialog window opens that shows all available adapters, simplyselect the adapter that you have started (see Figure 5).

Figure 5. Select the adapter for the test case

Larger view of Figure 5.

5. After you have selected the adapter, the adapter lets you choose theRhapsody model (Figure 6) that contains the model-based test.

6. After selecting the model, select the model-based test case that should beconnected to the test script (see Figure 7).

Figure 6. Specify the Rhapsody model where the test case is located

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 8 of 13

Larger view of Figure 6.

Figure 7. Specify test case to associate with test script

Larger view of Figure 7.

That's it. Now Rational Quality Manager recognizes that this test script is actuallyimplemented by a model-based test that can be executed with RhapsodyTestConductor.

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 9 of 13

When you run the test script, Rational Quality Manager automatically detects thatthis is a model-based test and uses the TestConductor adapter to execute it on thereferenced executable Rhapsody model, as depicted in Figure 8.

Figure 8. Model-based test cases can be run from Rational Quality Manager

Larger view of Figure 8.

After executing the test script in Rational Quality Manager, all of the test results,such as status (passed or failed) and log files, are directly accessible in RationalQuality Manager (see Figure 9). This means that you can use all of the reportingcapabilities of Rational Quality Manager to assess the quality status of the project,even in early development phases.

Figure 9. Results of TestConductor tests can be stored and managed inRational Quality Manager

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 10 of 13

Larger view of Figure 9.

Reusing model-based tests for software verification

The previous section described how model-based tests can be integrated into anoverall quality management process that is based on Rational Quality Manager sothat you can measure the quality status in early development phases. Model-basedtests can also be used to measure the quality status of the software implementationsas soon as they are available. The model-based tests can be adjusted easily so that,rather than testing the underlying model, they test the real system and softwareimplementations, respectively. If these adjustments are made, the model-basedtests can be executed from Rational Quality Manager in exactly the same way asbefore, but not to test the real implementation.

And again, Rational Quality Manager can use the results of these tests along withthe metrics measuring reporting capabilities included to assess the quality status ofthe implementations. Thus, with Rational Quality Manager, model-based testsspecified early can be used throughout the software development lifecycle to helpassess the project's quality.

Summary

Integrating model-based testing with the Rational Rhapsody TestConductor Add Onand using with Rational Quality Manager for quality management helps bring makethe quality status of the project visible earlier in the development phases. Teams canthen identify quality risks in the project earlier, respond to changes quickly, andassess the impact from a quality perspective. By seeing the tests that fail, teams canrecognize critical areas of concern and work to mitigate those risks earlier in thedevelopment lifecycle.

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 11 of 13

Resources

Learn

• To learn more about the Rational Rhapsody collaborative, model-drivendevelopment tool for embedded systems, start with these overviews:Introduction to Rational Rhapsody, the Rational Rhapsody Developer productpage, and the Rational Rhapsody page on IBM developerWorks.

• Explore the various versions, too:

• IBM® Rational® Rhapsody® Architect for Software, a visualdevelopment environment for embedded systems and software

• IBM® Rational® Rhapsody® Architect for Systems Engineers

• IBM® Rational® Rhapsody® Designer for Systems Engineers

• IBM® Rational® Rhapsody® Developer edition. This edition isrequired for Eclipse users and editions are available to createspecialized projects in C, C++, Java, and Ada languages.

• Read the PDF file titled Rhapsody TestConductor Add On Adapter forRational Quality Manager Overview, which is available in the RationalRhapsody installation when the TestConductor Add On is installed.

• Check the Rational Quality Manager page on IBM® developerWorks® for linksto product documentation, articles, tutorials, courses, downloads, and otheruseful areas. Also, explore the Rational Quality Manager Information Center fortechnical details and browse the IBM Quality Management page to learn moreabout what is available. For tips, read these developerWorks articles by MichaelKelly:

• Getting started with IBM Rational Quality Manager (December 2008)

• For Managing your first project with IBM Rational Quality Manager(October 2008)

• Visit the Rational software area on developerWorksfor technical resources andbest practices for Rational Software Delivery Platform products.

• Stay current with developerWorks technical events and webcasts focused on avariety of IBM products and IT industry topics.

• Attend a free developerWorks Live! briefing to get up-to-speed quickly onIBM products and tools, as well as IT industry trends.

• Watch developerWorks on-demand demos, ranging from productinstallation and setup demos for beginners to advanced functionality forexperienced developers.

developerWorks® ibm.com/developerWorks

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 12 of 13

• Improve your skills. Check the Rational training and certification catalog, whichincludes many types of courses on a wide range of topics. You can take someof them anywhere, any time, and many of the "Getting Started" ones are free.

Get products and technologies

• Download Rational Rhapsody Developer and try it free for 30 days.

• Download the free trial version of Rational Quality Manager.

• Evaluate IBM software in the way that suits you best: Download it for a trial, tryit online, use it in a cloud environment, or spend a few hours in the SOASandbox learning how to implement service-oriented architecture efficiently.

Discuss

• Join the discussion in the Rational Rhapsody forum.

• Join the Rational Quality Manager forum, which also includes discussions aboutRational Test Lab Manager.

• Share your knowledge and help others who use Rational software by writing adeveloperWorks article. You'll get worldwide exposure, RSS syndication, abyline and a bio, and the benefit of professional editing and production on thedeveloperWorks Rational website.

• Follow Rational software on Facebook and Twitter (@ibmrational), and add yourcomments and requests.

• Ask and answer questions and increase your expertise when you get involvedin the Rational forums, cafés, and wikis.

• Connect with others who share your interests by joining the developerWorkscommunity and responding to the developer-driven blogs.

About the author

Marc Lettrari, PhDDr. Marc Lettrari received his master’s degree in computer science fromthe University Oldenburg, in Germany, in 2000 and his PhD in computerscience in 2005. He previously worked at the OFFIS computer scienceinstitute in Oldenburg and conducted research in the field’s formalmethods, model-based development, and testing. Since then, he hasbeen working at BTC Embedded Systems AG, in Germany, where he isVice President for Rhapsody and Statemate Testing Products.

ibm.com/developerWorks developerWorks®

Integrate model-based testing to find quality problems early in development Trademarks© Copyright IBM Corporation 2011 Page 13 of 13