Vector Cast PIC Controller Feasibility

VectorCAST for PIC controller

Core Technologies Division

Analysis Report Vectorcast

Vector CAST Analysis Report

Author: Easter Raj S

VectorCAST tool support for 33FJ128MC804 PIC controller - feasibility analysis

Copyright Notice

This document contains proprietary information of HCL Technologies Ltd. No part of this document may be reproduced, stored, copied, or transmitted in any form or by means of electronic, mechanical, photocopying or otherwise, without the express consent of HCL Technologies. This document is intended for internal circulation only and not meant for external distribution.


SummaryVector Cast/C++ is an integrated software test solution that significantly reduces the time, effort, and cost associated with testing C/C++ software components necessary for validating safety- and mission-critical embedded systems.

A one month Vector cast license was procured from Vector cast to evaluate the product for the 33FJ128MC804 PIC controller Firmware and the details are in this report.


Contents1 RELEASE CONTROL RECORD.......................................................................................4

2 DEFINITIONS........................................................................................................................5

3 REFERENCE DOCUMENT................................................................................................6

4 PURPOSE..............................................................................................................................6

5 SCOPE OF THE ANALYSIS..............................................................................................6

5.1 PROBLEM DESCRIPTION..................................................................................................65.2 TYPE OF ANALYSIS..........................................................................................................65.3 ANALYSIS CRITERIA, ACCEPTANCE LEVEL....................................................................65.4 TOOLS USED....................................................................................................................7

6 ANALYSIS APPROACH ASSUMPTIONS.......................................................................7

7 ANALYSIS APPROACH.....................................................................................................7

7.1 INSTALLATION PROCEDURE (VECTORCAST ON THE LICENSE SERVER)....................77.2 BUILDING ENVIRONMENT................................................................................................87.3 EXECUTING THE APPLICATION FOR CODE COVERAGE..................................................87.4 CREATING THE TEST CASES.........................................................................................19

8 RESULT VALIDATION......................................................................................................21

9 Summary / Conclusions /Recommendations..................................................................21

TablesTable 1 : Release Control Record....................................................................................4

Table 2 : Definitions.........................................................................................................5


1 Release Control Record

Table 1 : Release Control Record

Version No. Release Authors Reason for Change Date of Issue

0.1 Easter Raj S Initial Version 21 Sep 20100.2 Easter Raj S Updated for compiler

template customization12 Jan 2012


2 Definitions

Table 2 : Definitions

Term / Abbreviation

Definition

UUT Unit Under Test


3 Reference Document

1. Vector Cast getting started document (vcast_getting_started.pdf)2. Vector Cast tool usage (vecchelp.pdf)

4 Purpose

The purpose of this document is to describe the Vector CAST tool feasibility analysis for 33FJ128MC804 PIC controller Firmware unit testing and code coverage. This document describes the analysis performed and the result of the analysis.

5 Scope of the Analysis

The objective of this analysis is to understand the Vectorcast C/C++ tool, have a hands-on experience with the tool and to analyze the VectorCAST feasibility for 33FJ128MC804 PIC controller on windows platform. The scope also includes analyzing the difference between using Vectorcast C/C++ for unit testing the 33FJ128MC804 PIC controller firmware and performing manual unit testing (Effort analysis is limited to perform branch coverage testing and not for functional testing).

5.1Problem Description

Generally, software-component testing requires generating one line of test code (in the form of stubs, drivers, and test data) for each line of application code to be tested. The necessity to create this "disposable" test software is the main reason manual component testing is so expensive and inefficient. Test software not only has to be written but also has to be debugged to ensure that it performs as expected.

5.2Type of Analysis

Vector cast will be installed in a Windows machine where the code coverage report and test cases will be generated for the 33FJ128MC804 PIC controller Oscillator configuration code and UART receive functionalities. Test cases will be executed and the test execution report will be generated. Also the various features available in Vector cast will be tested / experimented.

5.3Analysis Criteria, Acceptance Level

The following were the acceptance criteria

Software usability.


Time taken to create and execute the unit test cases compared to manual unit testing.

Accuracy and completeness of the basic path and the branches identified with a sample source code.

5.4Tools used

The following tools were used Windows OS PC PIC Module Controller Microchip MP LAB and C30 Compiler VectorCAST C/C++

6 Analysis Approach Assumptions

NA

7 Analysis Approach

7.1 Installation Procedure ( VectorCAST on the License Server )

1. Insert the VectorCAST distribution CD into the CD drive, or unzip the VectorCAST distribution file (vcast.win32.version.zip).2. Double-click on the Setup.exe icon. The VectorCAST software displays a welcome page.3. Click Next and select an installation mode:

• Typical: Select this option if you will be running VectorCAST on the LicenseServer. All VectorCAST and FLEXlm files will be installed in a specifieddirectory (next step) on the local computer.

• FLEXlm Server Only: Select this option if you will NOT be runningVectorCAST on the License Server.

4. Click Next and select an installation directory.The default directory (C:\VCAST) is recommended.Note: The directory path you use must not include spaces or international accent marks or diacritics.5. Click Next.6. Select the program folder.

"VectorCAST" is the default program folder.7. Click Next.

VectorCAST begins installing the licensed software.


7.2 Building Environment

1. Under User Variables:Variable Name: LM_LICENSE_FILEVariable Value: C:\FLEXlm\license.dat

2. Under User Variables:Variable Name: VCAST_MICROCHIP_INSTALL_DIRVariable Value: C:\Program Files\Microchip\MPLAB C 30

3. Under System Variables:Variable Name: VECTORCAST__DIRVariable Value: C:\VCAST\

7.3 Executing the application for Code coverage

1. Copy the content of attached file and append into compiler list data file (C_TEMPLATE.DATA) in the vector CAST installation path “C:\VCAST\DATA”.

2. Verify that the Vector Cast License is valid. (License .dat file to be verified for date)

3. Open the FLEXlm Server


4. Start the FlexLM Server

5. Verify that the Server Started Successfully

6. Start the VectorCast Application

7. Make sure that the config file is copied in to the working folder.

8. Setup Working Directory e.g.: “C:\vcast_tutorial\My_804_SPI-DAC_P”

9. From Menu bar select -> File and open New c/c++ Environment


10.Automatically this configuration is loaded with the Config file

11.All other configurations will be automatically loaded, if we place this config file in working directory.

12.By clicking Next , we see the following, and Enter a Project name say: My_Test123

13.Click Next and select the first option “Traditional Unit Testing”

14.Click Next and select Coverage Type: None

15.Click Next and Set the Current working folders for source and headers


16.Click Next and Select the Main file to be as UUT

17.Click Next and again Next , we see the following summary

18.Press Build, It starts building the project


19.Once built successfully, will load the individual functions of main and are populated such a way


20. If we need to Test and Cover the function OSCCONFIG, right click and Insert TestCase. And on Coverage tab select Branch coverage

21.Right click on the Test Case and select Execute with Debug and also with coverage.


22.Just press OK and continue.It opens MPLAB IDE by itself. Select File—Import

23.By this time, there will be a .cof file created in the working folder just import that


24.Also will see the below screen

25. Just ignore it, DON’T abort or close , Make sure that the Debugger is selected for Simulator


26.Run the project in MPLAB IDE

27. It displays the following message and select option NO


28.Now we can see that the control stops at a location in the file.

29. Now close the MPLAB IDE, It will display the following message


30. May select Yes /No , select Yes

31. In the VectorCast IDE, we may see that the tests executed successfully


32.By clicking on the check box before the test case, we may see the code covered and red shows not covered and Green shows covered. Also may see that the Braches are covered as 46%

33.Now may select the Test—View—Full Report


34.Will generate the report in the .html format.

7.4 Creating the Test cases

Once the environment is created the Vector cast will do the complexity analysis and will provide the details regarding the number of test cases for each function in the module. Also the details regarding the flow of each basic path will be generated by Vector cast which will be taken as an input for assigning input values for variables for test case execution

The following procedure was followed for creating and executing test cases

o In Vector cast, in the “test case tree” window, select the function for which the test case need to be inserted and then right click, and click “Insert Test case”

o Double click on the newly inserted test case. The test case sheet will be opened in the MDI window.

o The values for required variables will be entered to execute the required test path and click the “execute” button to execute the test cases

Report generated by verifying the Test case


7.4.1.1 Effort analysis

Using VectorCast tool:

Sl No

Step Approx effort involved

1 Building the environment 10 min

2 Executing the Code Coverage for complete test stub

2 Hrs

3 Design the test cases for each basic path 1 Hr

4 Develop Test Cases/ each test case 30 Mnts

5 Verify the correctness of the test stub/ test case 30 Mnts

6 Execute the test cases and generate the report / test case

2 Hrs

Total time 6 Hrs 10Mnts

Using Manual Testing approach

Sl No

Step Approx effort involved

1 Construct the flow graph from the source code or flow charts. Complete Test Stub

4Hrs

2 Identify independent paths for complete test stub. 1 hr

3 Calculate Cyclomatic Complexity, V(G). --

4 Design the test cases for each basic path 1 Hr

5 Update the source code and add the required stubs and driver code for each test case testable through UART (serial connection)

1 Hrs

6 Verify the correctness of the stubs and the driver code

2 hrs

7 Execute the Test cases /each test case 2 hrs8 Prepare the Execution report/test case 1 Hr

Total time 12 Hrs


8 Result Validation

The Vector CAST tool unit testing effort comparison with manual testing is given in below.

Effort in Hrs for Unit testing using Vector Cast

Effort in Hrs for Unit testing Manually

6 hrs 10 min 12 hrs

9 Observation

Vector CAST supports fully for lower version of PIC controller and hence the batch execution of test case could feasible. Since Vector CAST is customized for 33FJ128MC804, it could not load the *.cof file directly. Hence, each test case has to be executed with Debug option. This might take little more time for test case execution.

10 Summary / Conclusions /Recommendations

It was observed that using vector cast, the unit testing time was reduced considerably. This analysis was done based on UUT stubbing approach.

Vector cast (with minimal customization) is feasible for 33FJ128MC804 PIC modules for unit testing because of the following reasons

Microchip C30 Support (with minimal support).

Reduced effort and time.

Detailed Code Coverage.

Debugging the Stub Code generated is not required.

Accurate due to automation.

A detailed report is generated.

Vector Cast PIC Controller Feasibility

Documents

Transcript of Vector Cast PIC Controller Feasibility