A Test Case + Mock Class Generator for Coding Against Interfaces Mainul Islam, Christoph Csallner...
-
Upload
hollie-baldwin -
Category
Documents
-
view
214 -
download
0
Transcript of A Test Case + Mock Class Generator for Coding Against Interfaces Mainul Islam, Christoph Csallner...
A Test Case + Mock Class Generator for Coding Against Interfaces
Mainul Islam, Christoph CsallnerSoftware Engineering Research Center (SERC)
Computer Science and Engineering DepartmentUniversity of Texas at Arlington, USA
February 07, 2011.
Published at: 8th International Workshop on Dynamic Analysis (WODA), 2010
Mock Class Generator for Coding Against Interfaces
2
• Test case generation• Dynamic Symbolic Execution• Mock Class• Motivating examples to generate Mock Class• Algorithm to generate Mock Class• Results• Demonstration
Talk Outline
Mock Class Generator for Coding Against Interfaces
3
• What is the goal behind testing ?
Test Case Generation
public void Test(int x) { if (x > 0) {
if (x == 1234567890) { throw new Exception(“Bug”); }
}}
• What can be a good test case to test this method ?
Probability to reach the bug (with a random test input) is: ½ * 1/2^31
Mock Class Generator for Coding Against Interfaces
4
• Explore all feasible execution paths • Initialize the input with symbolic
values and execute the program over symbolic values
• At conditional statements check if either of the branches can be taken (using a constraint solver)
• For each path we get an accumulated path condition
Symbolic Execution
If (s) then … else … C
falsetrue
C` = C ⋀ s C` = C ⋀⌝s
s
• Dynamic symbolic execution: Combine Concrete and Symbolic execution
This example is taken from the slides of Nikolai Tillman (with permission)
5
void TestMe (int a[]) { if (a == null) return; if (a.length > 0) if (a[0] == 123) throw new Exception(“Error”);}
Dynamic Symbolic Execution by Example
a == null
a.length > 0
a[0] == 123
T
T
T
F
F
F
Constraints to Solve Input Observed Constraints
null
{}
{0}
{123}
a != null && !(a.length > 0)
a == null
a != null && a.length > 0 &&a[0] != 123
a != null && a.length > 0 &&a[0] == 123
a != null
a != null && a.length > 0
a != null && a.length > 0 &&a[0] == 123
Solve Execute
Choose Next Path
a == null
Done: No path left
Mock Class Generator for Coding Against Interfaces
6
public interface A {// …
}
public class C { public void foo (A a, int x) {// …// …}
}
Introduction to Mock Class
Mock Class Generator for Coding Against Interfaces
7
When we might need to generate mock-classes ?
• At the initial stage of any development
Mock Class Generator for Coding Against Interfaces
8
• Test case generation is important• Interfaces are very important• Existing techniques are not very good at this
Pex - can generate an instance of a class that implements an interface that is
used in the code *
* http://research.microsoft.com/en-us/projects/pex/
Why is this important ?
Mock Class Generator for Coding Against Interfaces
9
Goal:
• Support interfaces in test case generation for object-oriented programs
Mock Class Generator for Coding Against Interfaces
10
Motivating Example 1public class C {
public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x); }}
}To reach this block of code, ‘a’ must be an instance of A, as well as an instance of B
public interface A {int m1(int x);
}
public interface B {int m2(int x);
}
Mock Class Generator for Coding Against Interfaces
11
Motivating Example 2public class K
implements A, B { // …
}
public class C { public void bar (A a, int x) { if (a == null) return; if ( !(a instanceof B) ) {
B b = (B) a; int z = b.m2(x); }}
}
K implements all of the interfaces referred by the bar
method, but fails reach the code block
public interface A {int m1 (int x);
}
public interface B {int m2 (int x);
}
Mock Class Generator for Coding Against Interfaces
12
Framework
• DSC – Dynamic Symbolic Execution Engine (for Java)
• Z3 – Constraint Solverhttp://research.microsoft.com/en-us/um/redmod/projects/z3/
Mock Class Generator for Coding Against Interfaces
13
Our approach to generate Mock Class
• Determining the type of the Mock Class• Generate meaningful method body if needed
Mock Class Generator for Coding Against Interfaces
14
Sub-/Supertype relation in Java
• Java defines a binary subtype relation• If type B implements/extends type A then,
- A is a direct super type of B - B is a direct sub type of A
Reflexive: A is also a subtype of itselfTransitive: if B is a subtype of A and C is a
subtype of B then C is also a subtype of A
Mock Class Generator for Coding Against Interfaces
15
Sub-/Supertype relation in Java (Cont.)
• A class has one direct class super type and arbitrarily many interface super types.
• Exceptions:type object – has no direct super typetype null – has no direct sub type
Mock Class Generator for Coding Against Interfaces
16
Subtype constraintsObject
A B C
null
Object
A B C
M
nullInitial types in the system
A desired solution: with new type M
public interface A {int m1 (int x);
}
public interface B {int m2(int x);
}
public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x); }}
}
type(a) subtypeof A type(a) != null type type(a) subtypeof B
Constraints:
Mock Class Generator for Coding Against Interfaces
17
Subtype Relation Matrixpublic interface A {
int m1 (int x);}
public interface B {int m2(int x);
}
public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x); }}
}
Null Object A B C M
0 Null x x x x x x
1 Object x
2 A x x
3 B x x
4 C x x
5 M x MA MB MC x
Null Object
0 Null x x
1 Object x
Null Object A B C
0 Null x x x x x
1 Object x
2 A x x
3 B x x
4 C x x
Null Object A B C M
0 Null x x x x x x
1 Object x
2 A x x
3 B x x
4 C x x
5 M x x x x
Solution: MA = true MB = true MC = false
18
Program execution
Found Solution?
No
Yes
Algorithm: Generating Mock Class
Dynamic Symbolic Data Structure Repair
Build Subtype Relation matrix (Generate the constraints)
NO_MC < MAX_MC
Generate Test Case
Introduce a new Mock Class, N0_MC++
Encode the Mock Class in the type system
No
Yes
Solution not found
Yes
Initialization: NO_MC = 0 MAX_MC = 2
Mock Class Generator for Coding Against Interfaces
19
Generating Method Body: for Example 1
public class M implements A, B { int m1 (int x){ return 0;}
int m2(int x){ return 0;}
}
Outline of the source code for Mock Class M
public interface A {int m1 (int x);
}
public interface B {int m2(int x);
}
public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x);
if (z >= 10) { //… }
}}
}
return 10;
Mock Class Generator for Coding Against Interfaces
20
Results
• We have run our solution on JMS (Java messaging system)
• These are all hand-written test casesInterfaces used from
JMS
Number of goals
Number of goals reached
w/o mock-classes
Number of goals reached
with mock-classes
Message 6 0 6ByteMessage 5 0 5MapMessage 4 0 4
StreamMessage 4 0 4TextMessage 5 0 5
ObjectMessage 4 0 4
Mock Class Generator for Coding Against Interfaces
21
Comparison with Pex: Motivating Example 1
public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x); }}
}
public interface A {int m1(int x);
}
public interface B {int m2(int x);
}
Goal 1
Goal 2
PEX reaches 1/2 goal(s)
DSC reaches 2/2 goal(s)
Mock Class Generator for Coding Against Interfaces
22
Future Work
• Extend the solution to support more types, e.g. ‘Array’
• Determine a upper bound to the number of mock classes needed to solve a system
• Evaluation
Mock Class Generator for Coding Against Interfaces
23
Future Work (Cont.)
public class M implements A, B { int m1 (int x){ return 0;}
int m2(int x){ return 0;}
}
Outline of the source code for Mock Class M
public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) {
B b = (B) a; int z = b.m2(x);
if (z >= 10) { //… }
}}
}
return 10;
• Extend method body generation for complex types
Mock Class Generator for Coding Against Interfaces
24
Demo
Mock Class Generator for Coding Against Interfaces
25
Thank you!