Software Testing GroupSoftware Testing Group

Georgia Southern University, North Carolina State University(D)REU Interns 2009

Presented by: Bellanov Apilli, Lydia Richardson, & Cory Alexander

Fault-based Fault-based Combinatorial Testing Combinatorial Testing

of Web Servicesof Web Services

OutlineOutlineMotivationProblemSolutionBackground & ExamplesCurrent Testing TechniquesFault-Based Combinatorial TestingConclusionReferences

MotivationMotivationThe Internet houses

diverse web applications, including: ◦ Banking◦ Networking◦ Auctioning

Web applications are commonly deployed as web services◦ Web services allow for

great flexibility

Motivation Motivation

ProblemWeb services can be very complex

in structure◦Constantly updated

◦ Addition of new features◦ Removal of features◦ Updating features

◦Difficult to assure quality

SolutionSolution

SolutionWeb service testing

◦Exhaustive testing◦Fault-based testing◦Combinatorial testing◦Validation Framework (iTac-Qos)◦Fault-Based combinatorial testing

Web Application ExampleWeb Application Example

Web applications:◦ Are applications accessed via a web browser over a

network◦ Receive input and produce output

Multiple web applications make up a web service◦ Having multiple applications makes web services flexible

Individual applications are updated as opposed to updating one large application

◦ Web applications from different sources interact as a web service

Usernamespasswords

Access to application

Web Service WrappingWeb Service WrappingWeb services are

built upon SOAP (Simple Object Access Protocol)◦ Allows the

transfer of data in XML over the Internet

◦ Enables diverse applications to interact Different languages Different operating

systems

SOAP wraps inputs and outputs, converting them into input and output messages.

Message formats utilize XML

Messages are recognized by other web services

WSDL (Web Service Description WSDL (Web Service Description Language File) & UDDI (Universal Language File) & UDDI (Universal Discovery and Integration)Discovery and Integration)WSDL defines the operands required

for communication with another web service◦Specifies methods web services use when

communicated with other web servicesUDDI specifications store information

about web services, enabling them to interact◦Location – where the service is deployed◦Requirements - predefined rule sets (i.e.,

knowledge of certain inputs)

Web Service ExampleWeb Service ExampleThe user provides

input and receives output ◦ Only aspects

visible to the userNumber of web

applications in the service is unknown to the user, and maybe even to testers

Client Accessing a Web Service

CURRENT TESTING CURRENT TESTING TECHNIQUESTECHNIQUES

Exhaustive TestingExhaustive TestingExhaustive Testing:

◦Suggests testing for every combination of inputs

◦May be impossible in testing all of a certain input (i.e., names)

◦May be too costly or time consuming to test for all inputs.

Considering complexity in web services, exhaustive approaches may be infeasible

Combinatorial TestingCombinatorial TestingTest inputs are

generated considering interactions between input parameters

Minimizes the number of tests by focusing on a certain number of interactions (i.e., 2-way)

Name Birthday

Interests

Cory 1/1/2009

boxing

Lydia 2/2/2009

golf

Bellanov 6/6/2009

sports

2-way interaction

2-way interaction

Fault-Based TestingFault-Based TestingFault-Based techniques have been

applied as a means of testing web services.◦Data perturbation

Messages modified based on predefined rules Modified messages are sent to determine

correct behavior Requires access to WSDL file information

◦Detection of a fault implies that it cannot exist within the web service

◦Limited to white-box cases, where source code is available

Validation Framework (iTac-Validation Framework (iTac-Qos)Qos)iTac Tests and Certifies Quality of

Services:◦Web services are registered,

deployed, and evaluated on a server◦Requires models of web services to

generate tests Models may not always be available

FAULT-BASED FAULT-BASED COMBINATORIAL COMBINATORIAL TESTINGTESTING

Web Service Emulation: Web Service Emulation: iTrustiTrust iTrust: an

application that helps patients keep up with their medical history◦ Prescription information◦ Office visits◦ Etc.

Role-based health care system

Through WSDL, we deploy iTrust as a web service

Testing FrameworkTesting Framework

Monitor: monitors network traffic to and from the requesting machine

Requester: accesses the iTrust systemTraffic: generates random traffic, emulating

the Internet. iTrust: location of iTrust, deployed as web

service

SoftwareSoftwareOperating System(s):

◦Ubuntu Operating System◦Microsoft Windows XP

Web Service related software:◦GlassFish – web application/service

server◦Apache Tomcat - implmentation of Java

Servlet and JavaServer Page technologies (JSP)

◦Traffic – generates random network traffic

Fault-based Combinatorial Fault-based Combinatorial TestingTestingWe combine fault-based and

combinatorial testing techniques in order to expose faults in web servicesA fault is introduced in an iTrust feature

(i.e., falsifying input validation code)Test cases, specific to the feature, are

generatedUsing a test oracle, where expected

outputs for inputs are stored, we compare output from both faulty and non-faulty versions of the feature.

ConclusionConclusionWeb services continuously grow in

complexity throughout their lifetime◦ Web service testing is required to assure

quality iTrust handles very sensitive information, thus

quality must be ensured

Combinatorial testing of web services may be in its infancy◦ By combining both fault-based and

combinatorial testing techniques, our approach may be better able to assess and evaluate web services

ReferencesReferences[1] M. Gudgin, M. Hadley, N. Mendelsohn, J. Moreau,

H. Nielsen, A. Karmarkar, and Y. Lafon. Soap version 1.2.

http://www.w3.org/TR/soap12-part1/, 2007.

[2] D. Kuhn, D. Wallace, and A. G. Jr. Software fault interactions

and implications for software testing. IEEE Transactions on

Software Engineering, 30:418–421, 2004.

[3] C. Mao. Performing combinatorial testing on web

service-based software. In IEEE International Conference on

Computer Science and Software Engineering, pages 755–758,

Nanchang, China, 2008. IEEE Computer Society

[4] J. Offutt and W. Xu. Generating test cases for web services

using data perturbation. SIGSOFT Softw. Eng. Notes,

29(5):1–10, 2004.

ReferencesReferences[5] V. Pretre, F. Bouquet, and C. Lang. Automating uml

models

merge for web services testing. In iiWAS ’08: Proceedings of

the 10th International Conference on Information Integration

and Web-based Applications & Services, pages 55–62, New

York, NY, USA, 2008. ACM

[6] N. C. S. U. RealSearch Research Group. itrust: Role-based

healthcare v7.0.1n. http:

//agile.csc.ncsu.edu/iTrust/wiki/doku.php,

2008.

[7] O. U. specification TC. Uddi version 3.0.2.

http://uddi.org/pubs/uddi_v3.htm, 2005

[8] W. Vogels. Web services are not distributed objects. Web

services are not distributed objects, 7:59–66, 2003.