Web 2.0 for Modern Software Testing

Patni Computer Systems Ltd. Web 2.0 for Modern Software Testing of 12

White Paper accepted for publication in STeP-IN SUMMIT 2009

Web 2.0 for Modern Software Testing

Author(s): Manish Jalan Designation: Senior Specialist (Software) Email: [email protected] Sunil Joglekar Designation: Technical Architect Email: [email protected]

Patni Computer Systems Ltd., Unit 18, 4th floor, SDF-VII, SEEPZ, Andheri (East),

Mumbai – 400 096.


Abstract

Introduction Web 2.0 technologies such as wikis, RSS feeds, readers, pipes, tagging etc., by

themselves, are pretty simple to learn and apply. Also, they play well with each other. The challenge then is to create a unique synthesis of information, interaction & intelligence. It is largely up to our imaginations how we go about this.

Here we present how we can apply web 2.0 technologies / elements for making modern

software testing effective in a team environment. First we present usage of a wiki system for testing. Here we address some interesting

challenges while using FitNesse for Testing Second we present usage of wiki for design verification. Here again we present

solutions to some interesting challenges Third we present the usage of the Blog machinery for automating the process of

posting, aggregating, filtering and presenting the build results. Audience Test Architects, CTO, CIO Area of Application Modern software testing Benefits Web 2.0 is not just creating social networks or flashy applications. It can be very

effectively used in enterprise context as well. Issues and Challenges There are 3 aspects of Web 2.0: tools, applications & mindset. Compared to the first

two, it is the mindset that is difficult to achieve. We need to learn to collaborate with the machine, and let the computer do the work for us. And it is up to us to work on inculcating the DNA of Web 2.0 viz. the spirit of sharing, the culture of contribution & the architecture of participation among ourselves & our teams


Table of Contents

ABSTRACT .................................................................................................... 2

1.0 BACKGROUND : MODERN SOFTWARE TESTING ...................................... 4

2.0 USING FITNESSE FOR TESTING ............................................................. 5

.1 CHALLENGE: ALLOWING SYSTEM UNDER TEST TO BE CHANGED FREQUENTLY ..............................5

.2 CHALLENGE: ACCESS CONTROL MECHANISM FOR FITNESSE TESTS .........................................6

.3 CHALLENGE: AUTOMATED TEST EXECUTION AFTER NIGHTLY BUILDS .......................................6

.4 CHALLENGE: TRIGGERING UNIT TESTS FROM FITNESSE ....................................................7

.5 CHALLENGE: MEASURING CODE COVERAGE FOR FITNESSE TESTS ..........................................7

3.0 USING FITNESSE FOR DESIGN VERIFICATION....................................... 7

.1 CHALLENGE: SEAMLESS TRACEABILITY USING FITNESSE ....................................................7

.2 CHALLENGE: DEPENDENCY MANAGEMENT WITH FIT/FITNESSE.............................................8

.3 CHALLENGE: FITNESSE AND TEST DRIVEN DEVELOPMENT ..................................................8

4.0 BLOGGING THE BUILD PROCESS............................................................ 9

.1 THE PROBLEM .............................................................................................9

.2 SOLUTION NEEDED.........................................................................................9

.3 SOLUTION DEVELOPED .....................................................................................9

.4 WEB 2.0 BUILDING BLOCKS USED IN THE SOLUTION .......................................................9

.5 BENEFITS OF THE WEB 2.0 SOLUTION OVER CONVENTIONAL SOLUTION ................................... 10

5.0 CONCLUSION ....................................................................................... 10

6.0 DEFINITIONS, ABBREVIATION AND ACRONYMS.................................. 10

7.0 REFERENCES ........................................................................................ 11

8.0 ACKNOWLEDGEMENTS ......................................................................... 11

BIOGRAPHY OF THE AUTHORS ................................................................... 12


1.0 Background : Modern Software Testing

The Modern Software Systems are growing beyond the boundaries of silo-ed enterprise applications to meet the requirements of rapidly changing markets. They are getting integrated within and across the enterprise thereby metamorphosing into ecosystems. This has resulted in complexity which seems to be ever increasing and also in a high number of changes these systems have to go through. Naturally this has lead to increased risks and has also increased cost of evolving these systems to meet the requirements of rapidly changing business landscape. There are pressures on time to market and there is a need to ensure the agility of the enterprise at several levels : Technical, Organizational and Business. The cost of finding & fixing defects is becoming exponential. There’s a high premium for Enduring Quality in Modern Software Development. The above mentioned challenges can be tackled by working in an iterative / incremental manner. We need to embellish the Continued Integration discipline by embedding & integrating the following activities:

• Source Code Analysis o Common Coding Mistakes o Compliance to Coding Standards o Common Design Mistakes o Compliance to Design / Architectural Policies

• Unit & Integration Testing

• Acceptance Testing

• Load Testing These activities ensure rapid, immediate, useful feedback throughout the span of software development thereby transforming a vicious circle into a virtuous cycle. Consider the case of modern software development using Java / JEE. ANT is a build tool for Java. It provides a facility to carry out tasks such as check out from the source code control, compiling java files, running unit tests et al. Using the build file mechanism, we can compose these tasks in a pipe-line manner. We have developed an ANT pipeline (calling it ‘Java Tool Shed’) using a bunch of open source tools to provide solutions to some of the commonly recurring problems during software development. Many of the available open source tools do a pretty decent job of analyzing source code as well the byte code, jar files at al.

• Compliance to Coding Standards : CheckStyle

• Common Coding Mistakes : PMD, CPD

• Analyzing byte-code : JLint, FindBugs

• Common Design Mistakes : JDepend

• Compliance to Design / Architectural Policies : Utilities developed in-house

• Unit & integration testing : JUnit Family

• Acceptance testing : Fit/Fitnesse

• Database testing : dbFit

• Load / Performance Testing : JUnitPerf, Meter A sample ANT pipeline would look like the figure shown below.


The challenge is how do we make these effective in a team environment? Employing Web 2.0 tools / technologies / concepts will give us that edge. In the forthcoming sections we present some of the examples of employing various Web 2.0 elements (like Blogs, Wikis, RSS, …) for making modern software testing effective in a team environment.

2.0 Using FitNesse for Testing

FitNesse is a Wiki built on top of Fit. Fit stands for Framework for Integrated Test. It is a tool for enhancing communication and collaboration between customers and programmers. Fit creates a feedback loop between them. Wiki is a piece of server software that allows users to freely create and edit Web page content using any Web browser. Wiki supports hyperlinks & has simple text syntax for creating new pages & cross-links between internal pages on the fly. Advantages of FitNesse: Fit combines with FitNesse for effective acceptance testing. We can easily create and edit pages in FitNesse using wiki markup language. FitNesse is also a web server. It helps in enhancing collaboration in software development. Wiki allows the customers, test engineers and programmers distributed across globe to share and execute tests via internet.

.1 Challenge: Allowing System Under Test to be Changed Frequently

Typically, we’ve a single FitNesse server that executes tests on the nightly build. So, a developer might have to wait for a long time before he gets a feedback on her code. We need a way by which developers can frequently execute tests on their code.


Solution We keep local copy of FitNesse at each machine. FitNesse table scripts are kept under source code control system. Developers can then checkout the FitNesse table scripts from source code control system onto their local machines. Then, developers can frequently run the FitNesse tests on the code developed by them.

.2 Challenge: Access Control Mechanism for FitNesse Tests

Since FitNesse is built on Wiki, it provides marginal security. FitNesse allows anyone to run and modify the tests since FitNesse tests are accessed using a web browser. Therefore, we require a robust access control in place. Solution We use FitNesse along with an external source control system such as VSS to provide access control. Using FitNesse along with source control system also provides access control. The figure below shows the necessary setup

.3 Challenge: Automated Test Execution after Nightly Builds

Each time a new build is available, we would like to run FitNesse tests on it. So, it’s really useful if we have a mechanism by which FitNesse acceptance tests are run automatically whenever latest build is available. FitNesse does not provide built in mechanism for this kind of automated test execution. Solution We can use Ant tasks to achieve this.

• Checkouts of latest build of applications and fixtures can be done to “FitNesse Server and Fixture Build Machine” via Ant tasks.

• We can then write an Ant task that causes FitNesse to run the tests automatically.


.4 Challenge: Triggering Unit Tests from FitNesse

Acceptance tests are run only after unit tests are successfully executed. Therefore, we would like to ensure that the unit tests are run before we execute FitNesse acceptance tests. It would be convenient if we’re able to trigger unit tests from within FitNesse. Solution Unit tests are written using frameworks such as JUnit, TestNG etc. JUnit allows running tests from command line. We can use CommandLineFixture to trigger JUnit tests from FitNesse.

.5 Challenge: Measuring Code Coverage for FitNesse Tests

After FitNesse tests are run on latest build, we would like to measure code coverage for FitNesse tests. Code coverage measures number of lines of code executed by tests. FitNesse does not provide built-in facility to measure the test coverage. Measuring code coverage allows us to identify parts of application that are exercised by the FitNesse tests. Solution We can use an open source tool Cobertura to measure code coverage for FitNesse tests. We can write an Ant task to combine FitNesse and Cobertura.

3.0 Using FitNesse for Design Verification

.1 Challenge: Seamless Traceability using FitNesse

We categorize tests according to the feature they are supposed to validate. E.g. authentication, account transactions etc. Each of these tests is based on some artifacts such as use cases, CRC cards, Interaction diagrams and Java docs. These are called as “test basis”. Given a test the reviewer (test engineer or business analyst) would like to examine the basis of the test. E.g. Given a test “TestOpenExistingAddressBook” the reviewer would like to see “OpenAddressBook” use case. Such seamless traceability among tests and their basis offers following advantages:

1. It encourages us to perform the review more times 2. Viewing test basis along with tests themselves allows better understanding of system

Generally, the “test basis” is stored separately from the tests. E.g. Acceptance tests are available on web through FitNesse wiki, whereas other design artifacts such as use cases, CRC cards are stored at some different folders in the file system. The reviewer then needs to constantly switch between the “FitNesse wiki in the web browser window” and “folder structure to view test basis”. This approach is not a smooth approach. Solution The seamless traceability would be achieved if the user can view both tests and their basis without leaving FitNesse environment. For this, we can leverage sub-wiki feature of FitNesse. Using sub-wikis we can structure the FitNesse wiki pages in a standard order. It is not possible to fully automate this task since a test designer must write the Wiki Markup Language script to generate the wiki page. However, we’ve semi-automated this task.


We observed that there’s some similarity in the wiki script that creates linkages between tests and their basis is common. We factored out this common script and stored it as a template. Now anybody, who wants to construct a wiki page should copy the relevant template to FitNesse edit window and should populate it with actual names of the page. We can extend the FitNesse to allow the copying the template through FitNesse edit window UI.

.2 Challenge: Dependency Management with Fit/FitNesse

A Java based application typically contains several components i.e. Java packages. These components may all be placed in a single .jar file, or they be separated into many different .jar files. We want to avoid dependency cycles between these components so that the system is buildable, and that the components are independently deployable. We can find cyclic dependencies among different modules using ‘JDepend – an open source tool’. With the help of ‘JDepend’ api, we can write JUnit tests to ensure that the application has only the desired dependencies. Wouldn’t it be nice if someone writes that code for us? Solution ModuleDependency Fixture does exactly that. It allows us to represent the dependencies of our system as a Fit table. When we run the test, the fixture invokes JDepend and makes sure that only the dependencies we signify in the table are actually present in our application.

.3 Challenge: FitNesse and Test Driven Development

In a typical scenario, we write FitNesse tests when some functionality is implemented. This is in line with the “Waterfall Model”. However, FitNesse is much more effective if used for Test Driven Development (TDD).

We write acceptance tests and fixtures before we write application classes. Here tests and fixtures act as a specification for application. Fixtures guide the development of classes, especially public interface of the classes. This helps to derive a modular design.


4.0 Blogging the build process

.1 The Problem

Each tool in the Java Tool shed produces xml / text results. Although Java Tool Shed is very useful there are some shortcomings, as below.

a. How does one share the results with all team members? Email is not the best way to share the information with multiple people over a long period of time

b. Java Tool Shed results are in xml format which is not meant for humans to interpret / comprehend. In effect, results are not easily understandable

.2 Solution Needed

1. Ability to inform team members when new results are available 2. Present the results in way suitable for human interpretation

.3 Solution Developed

a. Used a Blog server to post the results from the Java Tool Shed. Blogs, by design provide RSS feeds. Team members have to simply subscribe to that. Whenever new results are posted, they are automatically informed. In way, updates are pushed to the team members.

b. Used the light weight APIs of the BlogApps server to create the utility which will post the results from the Java Tool Shed output to the Blog server for broadcast on the Blog web server and RSS feeds

c. Developed filters which will analyze the Java Tool Shed results and create summary information for human interpretation

d. Automated the complete process – From testing, to analyzing to filtering to posting on the Blogs by creating an Ant task

.4 Web 2.0 Building Blocks Used In the Solution

a. ‘Blog the build process’ Server (web based application) was developed using iterative and incremental approach

b. User experience was improved by making the test results available in a readable summary with drill down facility by developing filters that would read the xml/text output from Java Tool Shed

c. Light weight APIs of the BlogApps server was used to develop the posting utility d. Java Tool Shed results were analyzed and filtered (Intelligence was employed) e. RSS feeds were made available for subscription by the development team

Run a test tool from Java Tool

Shed

Post result to blog as blog

entry

Parse RSS feed for

posted entry

Run filter on parsed results

Post the filtered result as

summary blog

Produce Cumulative

Summary as blog


.5 Benefits of the Web 2.0 solution over conventional solution

Conventionally emails would have been used for posting the build results to the intended audience. This solution has two main drawbacks.

a. someone would have to constantly maintain the email list b. Whether an individual is interested or not, he will continue receiving the emails

In the solution presented in this section,

a. The build process has been completely decoupled with the communication of results to the intended audience. This solution simply posts the results on the Blog

b. Whom-so-ever is interested can log on to the Blog and check the results c. If the user wants the luxury of getting informed (results getting pushed to him) he can

opt for the RSS feeds being published by the Blog server. d. Build results are aggregated, filtered, summarized and then presented through intuitive

graphical charts with the drill down facility. This has positively added to the usability of the information from the build process.

5.0 Conclusion

Through the examples presented above we saw Web 2.0 improving agility of our teams. In a major sense, we have come a full circle with Web 2.0. We have rediscovered the good old hacker culture in the 1960’s, 1970’s and 1980’s – a philosophy of software development a la Lisp / UNIX / Smalltalk:

• Let the machine do the work

• Let other people do the work

• Build tools

• Do the work in stages

• Seek simplicity, generality & orthogonality

• Form, contribute, participate in & grow communities. We need to realize that the power of the overall system derives more from the relationships among its constituents, rather than the constituents themselves !

6.0 Definitions, Abbreviation and Acronyms

Acronym Description

Blog Contraction of the term 'Web log' - a Web site, usually maintained by an individual, with regular entries of commentary, descriptions of events, or other material such as graphics or video. Entries are commonly displayed in reverse-chronological order

CPD Copy Paste Detector

Mashups A web application that combines data from more than one source into a single integrated tool


Acronym Description

Pipes Pipes is a hosted service that lets you remix feeds and create new data mashups in a visual programming environment. The name of the service pays tribute to Unix pipes, which let programmers do astonishingly clever things by making it easy to chain simple utilities together on the command line

PMD Project Minus Defects - a static ruleset based Java source code analyzer

RSS Really Simple Syndication - Web feed format used to publish frequently updated work in a standardized format

Tagging A tag is a non-hierarchical keyword or term assigned to a piece of information (such as an internet bookmark, digital image, or computer file). This kind of metadata helps describe an item and allows it to be found again by browsing or searching. Tagging was popularized by websites associated with Web 2.0 and is an important feature of many Web 2.0 service

Wiki a page or collection of Web pages designed to enable anyone who accesses it to contribute or modify content, using a simplified markup language. Wikis are often used to create collaborative websites and to power community websites

7.0 References

Item Description

http://rollerweblogger.org/ Roller: free and open source Java blog software

RSS and Atom in Action - Web 2.0 Building Blocks

Dave Johnson, Manning Publications

https://blogapps.dev.java.net/

Blogapps: RSS and Atom utilities and examples

http://ant.apache.org/ ANT tool

http://fit.c2.com/ Fit – Framework for Integrated Test

http://www.fitnesse.org/ FitNesse - The fully integrated standalone wiki, and acceptance testing framework

http://www.butunclebob.com/ArticleS.UncleBob.JdependFixture

JdependFixture

Fit for Developing Software Rick Mugridge & Ward Cunningham, Prentice hall Publications

8.0 Acknowledgements

We would like to express our sincere thanks to Dr. Prasad Ramanathan for the encouragement, support and feedback.


Biography of the authors

Author 1: (minimum 200 words) Manish Jalan is a Senior Specialist (Software) with the Product and Technology Initiative group at Patni Computer Systems Ltd. He has 8 years experience as a software professional working in global software teams. His core areas of skill are emerging technology adoption, design & development of applications for Web 2.0, Enterprise mobility, VoIP, IP telephony, Speech recognition, IVRS, Call center systems, Push To Talk. His achievements include contribution of SIP residential gateway on http://www.vovida.org (open source project now owned by Cisco). He has also developed frameworks, solutions, tools, protocol stack implementation, guides, courseware, and whitepapers. His areas of interest are: designing innovative solutions on Enterprise Mashups, Enterprise Mobility, Cloud computing et al Author 2: (minimum 200 words) Sunil Joglekar is a Technical Architect with the Product & Technology Initiative group, at Patni Computer Systems Ltd. He has 18 years experience as a Software Professional working in Global Software Teams. He has spent about 5 years in United States mostly as a consultant at Sun Microsystems. Sunil has extensive experience in the development of products / product suites / product lines. Also, he is well-versed in applying the Agile Methods / Practices. He specializes in architecting testable solutions for complex enterprise software systems and their integrations such as EAI / B2Bi / SOA. His primary areas of interest are : strategizing & architecting innovative solutions with Web 2.0, Cloud Computing, Autonomic Computing et al

Web 2.0 for Modern Software Testing

Documents

Transcript of Web 2.0 for Modern Software Testing