TJHSST Computer Systems Lab Automation of Testing in the Distributed Common Ground System (Army)
1
TJHSST Computer Systems Lab Automation of Testing in the Distributed Common Ground System (Army) 2008-2009 Michael Eng Mentors: Mark Pumphrey, Greg Cordero Northrop Grumman Corporation: Mission Systems Sector Abstract In most large-scale software systems, it is important to thoroughly test individual modules and components whenever possible. There are many different methods of testing that differ mainly in their approach to the scale of each test. Unit testing is a method of testing that verifies and validates individual units of source code. This project seeks to use different testing methods within unit tests to test the United States Army’s Distributed Common Ground System (DCGS), which will optimize coverage and efficiency as development continues. Background DCGS-A is a joint collaboration between multiple defense contractors and the U.S. Army. It is envisioned as one cohesive system that integrates architecture and input from all surface systems. Intelligence on terrain, personnel, targets, and signals will be integrated into DCGS-A. The portion of the infrastructure that the Northrop Grumman team is handling is the Signals Intelligence (SIGINT) section. The part of DCGS-A that I am testing is written in Java and organized according to the object-oriented programming practice. A Subversion repository is used for version control and as a central bank for stable code. Unit testing has been utilized in countless programs. It is the cornerstone of Extreme Programming, a methodology designed to make development more flexible and to reduce the cost of changing existing software. Unit testing usually relies on an automated testing framework- in this case, JUnit- that facilitates creation and execution of test classes. Within the DCGS-A infrastructure, each class with Methodology JUnit, an automated testing framework, was used in conjunction with Eclipse as an environment for creating and running tests for the various DCGS-A classes. Classes with logic- those with more than simple get and set methods and constructors- were tested. Each class tested had its logical methods and constructors tested in multiple ways. For example, consider a Parser class with a method parse that returns the first number character found in a given String parameter and returns null if there are no numbers in the String. A corresponding JUnit test class might test the parse method by sending it a String with all letters, a blank String, a null value, an Integer, a String array, a String obtained from an Object’s toString method, and a String with all numbers. In each case, there would be an expected output or result from the method. In JUnit, the actual result of each test is usually compared with an expected result by using assertion methods like assertEquals, assertNull, assertTrue, or assertFalse. In order to run test classes en masse, JUnit’s suite method was used to “chain” clusters of tests together. Classes called allTests classes were written at each subdirectory of the project. Each allTests class, depending on its level in the hierarchy, either loaded individual test classes into its suite or loaded the suite of the allTests classes in its subdirectories into its suite. Thus, using Eclipse’s Run as JUnit Test on the allTests class in the root of the directory will run every single test class and method in the project, achieving testing automation and saving systems engineers from having to manually run each test class. created for various DCGS-A classes. Four existing test classes have been modified to increase test coverage or have been updated to remove build errors and reflect updated source classes. As a direct result of testing, five source classes have been modified to remove bugs. Three AllTests classes have been created, and five AllTests classes have been updated to include new test classes. All of the test classes written were designed to meet the criteria that 1) they tested anything with logic- constructors, methods, etc., and 2) tests covered the majority of possible cases for each method being tested. Bugs found using this form of testing usually were a result of bad logic or improper code placement. For example, a class that issued security classifications based on an input erroneously had a line at the end of the method that forced all security classifications to have the default configuration. Without unit testing, this might not have been found until DCGS-A was being demonstrated to the U.S. Army’s representatives. A sample image of Eclipse running JUnit tests. The green bar on the left side shows the progress and success rate of the collection of tests- http://www.onjava.com/pub/a/onjava/2005/07/13/pisces.html?page=2 Conclusion During the mentorship period, test files were successfully written for DCGS-A. They used the JUnit testing framework and methods to carry out test methods and followed the unit test approach to testing software, mainly focusing on testing various permutations of methods by manipulating parameters and arguments. The tests were stored in a test directory equal in structure to the main class directory in order to ease automation and test file location. Automation of the tests was achieved by “chaining” AllTests files together to form a tree-style execution of all test files that could be triggered from the AllTests file at the very top of the testing directory. Running the entire test directory using this method takes about 10 seconds for over 1,000 test methods. Possible means of further automating testing include the implementation of a script into the DCGS-A batch files in order to run all of the tests overnight, though the software group is satisfied with the current implementation. References Buy, U., Orso, A. & Pezze, M. (September 2000). Automated Testing of Classes. ACM SIGSOFT Software Engineering Notes, 25(5), 39-48. from http://delivery.acm.org/10.1145/350000/348870/p39-buy.pdf?key1=348870 &key2=3353531321&coll=GUIDE&dl=ACM&CFID=17425538&CFTOKEN =72653537 ACM Portal. Canna, J. Testing, fun? Really? IBM. Retrieved January 7, 2009, from http://www.ibm.com/developerworks/java/library/j-test.html Carrington, D. (1997). Teaching software testing. ACM International Conference Proceeding Series, 2, 59-64. Retrieved January 7, 2009, from http://delivery.acm.org/10.1145/300000/299369/p59-carrington.pdf? key1=2 99369&key2=9642531321&coll=GUIDE&dl=GUIDE&CFID=17425538&C FTOKEN=72653537 ACM Portal. Chatterjee, S. Testing Java in an Object-Oriented Way. Java.net. Retrieved January 7, 2009, from http://today.java.net/pub/a/today/2006/03/28/testing-java-object-oriented.html Rajendran, R. V. (2008). White paper on Unit Testing. Retrieved January 7, 2009, from http://www.mobilein.com/WhitePaperonUnitTesting.pdf
description
TJHSST Computer Systems Lab Automation of Testing in the Distributed Common Ground System (Army) 2008-2009 Michael Eng Mentors: Mark Pumphrey, Greg Cordero Northrop Grumman Corporation: Mission Systems Sector. Results - PowerPoint PPT Presentation
Transcript of TJHSST Computer Systems Lab Automation of Testing in the Distributed Common Ground System (Army)
TJHSST Computer Systems Lab Automation of Testing in the Distributed Common
Ground System (Army)2008-2009
Michael EngMentors: Mark Pumphrey, Greg Cordero
Northrop Grumman Corporation: Mission Systems Sector
AbstractIn most large-scale software systems, it is
important to thoroughly test individual modules and components whenever possible. There are many different methods of testing that
differ mainly in their approach to the scale of each test. Unit testing is a method of testing that verifies and validates individual units of
source code. This project seeks to use different testing methods within unit tests to
test the United States Army’s Distributed Common Ground System (DCGS), which will
optimize coverage and efficiency as development continues.
BackgroundDCGS-A is a joint collaboration between multiple defense contractors and the U.S. Army. It is envisioned as one cohesive system that integrates architecture and
input from all surface systems. Intelligence on terrain, personnel, targets, and signals
will be integrated into DCGS-A. The portion of the infrastructure that the
Northrop Grumman team is handling is the Signals Intelligence (SIGINT) section. The part of DCGS-A that I am testing is written
in Java and organized according to the object-oriented programming practice. A Subversion repository is used for version control and as a central bank for stable
code.Unit testing has been utilized in countless
programs. It is the cornerstone of Extreme Programming, a methodology designed to
make development more flexible and to reduce the cost of changing existing
software. Unit testing usually relies on an automated testing framework- in this case,
JUnit- that facilitates creation and execution of test classes. Within the
DCGS-A infrastructure, each class with logic- not simple get and set methods- has a corresponding test class. Each test class
tests each method with logic in multiple ways. Testing each method optimizes
code coverage and eases debugging by limiting problems to individual lines of code,
rather than whole classes or packages.
MethodologyJUnit, an automated testing framework, was used in conjunction with Eclipse as an environment for
creating and running tests for the various DCGS-A classes. Classes with logic- those with more than simple get and set methods and constructors- were tested. Each class tested had its logical
methods and constructors tested in multiple ways. For example, consider a Parser class with a method parse that returns the first number character found in a given String parameter and returns
null if there are no numbers in the String. A corresponding JUnit test class might test the parse method by sending it a String with all letters, a blank String, a null value, an Integer, a String array, a String obtained from an Object’s toString method, and a String with all numbers. In each case, there would be an expected output or result from the method. In JUnit, the actual result of each test is usually compared with an expected result by using assertion methods like assertEquals,
assertNull, assertTrue, or assertFalse.
In order to run test classes en masse, JUnit’s suite method was used to “chain” clusters of tests together. Classes called allTests classes were written at each subdirectory of the project. Each allTests class, depending on its level in the hierarchy, either loaded individual test classes into its
suite or loaded the suite of the allTests classes in its subdirectories into its suite. Thus, using Eclipse’s Run as JUnit Test on the allTests class in the root of the directory will run every single test class and method in the project, achieving testing automation and saving systems engineers
from having to manually run each test class.
ResultsFourteen test classes have been created for various DCGS-A classes. Four existing test classes have been modified to increase test
coverage or have been updated to remove build errors and reflect updated source classes. As a direct result of testing, five source classes have been modified to remove bugs. Three AllTests classes have been created, and five AllTests
classes have been updated to include new test classes. All of the test classes written were
designed to meet the criteria that 1) they tested anything with logic- constructors, methods, etc.,
and 2) tests covered the majority of possible cases for each method being tested.
Bugs found using this form of testing usually were a result of bad logic or improper code
placement. For example, a class that issued security classifications based on an input
erroneously had a line at the end of the method that forced all security classifications to have the default configuration. Without unit testing, this might not have been found until DCGS-A was
being demonstrated to the U.S. Army’s representatives.
A sample image of Eclipse running JUnit tests. The green bar on the left side shows the progress and success rate of the collection of tests- http://www.onjava.com/pub/a/onjava/2005/07/13/pisces.html?page=2
ConclusionDuring the mentorship period, test files were
successfully written for DCGS-A. They used the JUnit testing framework and methods to carry out test methods and followed the unit test approach
to testing software, mainly focusing on testing various permutations of methods by manipulating
parameters and arguments. The tests were stored in a test directory equal in structure to the main class directory in order to ease automation
and test file location. Automation of the tests was achieved by “chaining” AllTests files together to form a tree-style execution of all test files that could be triggered from the AllTests file at the very top of the testing directory. Running the entire test directory using this method takes
about 10 seconds for over 1,000 test methods. Possible means of further automating testing include the implementation of a script into the
DCGS-A batch files in order to run all of the tests overnight, though the software group is satisfied
with the current implementation.
ReferencesBuy, U., Orso, A. & Pezze, M. (September 2000). Automated Testing of Classes. ACM SIGSOFT Software Engineering Notes, 25(5), 39-48. fromhttp://delivery.acm.org/10.1145/350000/348870/p39-buy.pdf?key1=348870&key2=3353531321&coll=GUIDE&dl=ACM&CFID=17425538&CFTOKEN=72653537 ACM Portal.
Canna, J. Testing, fun? Really? IBM. Retrieved January 7, 2009, from http://www.ibm.com/developerworks/java/library/j-test.html
Carrington, D. (1997). Teaching software testing. ACM International Conference Proceeding Series, 2, 59-64. Retrieved January 7, 2009, from http://delivery.acm.org/10.1145/300000/299369/p59-carrington.pdf?key1=299369&key2=9642531321&coll=GUIDE&dl=GUIDE&CFID=17425538&CFTOKEN=72653537 ACM Portal.
Chatterjee, S. Testing Java in an Object-Oriented Way. Java.net. Retrieved January 7, 2009, from http://today.java.net/pub/a/today/2006/03/28/testing-java-object-oriented.html
Rajendran, R. V. (2008). White paper on Unit Testing. Retrieved January 7, 2009, from http://www.mobilein.com/WhitePaperonUnitTesting.pdf
