Model extraction and test generation from JUnit test suites · Example generated test...
Transcript of Model extraction and test generation from JUnit test suites · Example generated test...
![Page 1: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/1.jpg)
Sophia Antipolis, French Riviera
20-22 October 2015
MODEL & TEST GENERATION FROM JUNITPresented by Pablo Lamela Seijas
© All rights reserved
![Page 2: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/2.jpg)
Overview
2
• Introduction
• Architecture of the solution (James)
• Example
• Pilot study and lessons learned
© All rights reserved
![Page 3: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/3.jpg)
INTRODUCTION
What is James and what does it do?
© All rights reserved
![Page 4: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/4.jpg)
Introduction
4
• JUnit Tests Model Visual diagram
New JUnit Tests
• PROWESS project (testing Web Services)
• James tool: https://github.com/palas/james
• Main contribution: combines two approaches
© All rights reserved
![Page 5: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/5.jpg)
Two approaches: Control vs Data
5
• Control flow: How events affect state
© All rights reserved
![Page 6: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/6.jpg)
Two approaches: Control vs Data
6
• Data flow: How data is reused
© All rights reserved
![Page 7: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/7.jpg)
Our approach James…
7
• Combines both approaches
• So far, targeted at Web Services
• Other interfaces could be tested
© All rights reserved
![Page 8: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/8.jpg)
ARCHITECTURE OF JAMES
Components and output format
© All rights reserved
![Page 9: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/9.jpg)
James structure
9 © All rights reserved
![Page 10: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/10.jpg)
James structure
10 © All rights reserved
![Page 11: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/11.jpg)
James structure
11 © All rights reserved
![Page 12: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/12.jpg)
James structure
12 © All rights reserved
![Page 13: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/13.jpg)
James structure
13 © All rights reserved
![Page 14: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/14.jpg)
James structure
14 © All rights reserved
![Page 15: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/15.jpg)
James structure
15 © All rights reserved
![Page 16: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/16.jpg)
James structure
16 © All rights reserved
![Page 17: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/17.jpg)
Generated diagrams
17 © All rights reserved
![Page 18: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/18.jpg)
Generated diagrams
18
• Rectangular nodes represent:
• Method calls
© All rights reserved
![Page 19: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/19.jpg)
Generated diagrams
19
• Arrows:
• Data dependencies (gray arrows)
• Control dependencies (brown arrows)
© All rights reserved
![Page 20: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/20.jpg)
Generated diagrams
20
• Elliptical nodes show:• Primitive types: int, char…
• String, StringBuffer…
• Alternatives are grouped with:• Diamonds oneOf
(Later translated to QuickCheck’s oneof generator)
© All rights reserved
![Page 21: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/21.jpg)
Generated diagrams
21
• Nodes grouped by
• HTTP requests
• With dependencies
• Classified in:
• Positive
• Negative (pink bg)
© All rights reserved
![Page 22: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/22.jpg)
Generated diagrams
22
• Several other simplifications
• Similar paths are merged• But not positive and negative nodes
• Consecutive numbers are generalized
• Constant strings are concatenated
© All rights reserved
![Page 23: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/23.jpg)
EXAMPLE
Applying James to a simple system
© All rights reserved
![Page 24: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/24.jpg)
CONTENT SLIDE TITLE
24
• Example extracted from book:
F. Cesarini and S. Thompson − Erlang programming
© All rights reserved
API:•start − Starts the server
•allocate − Reserves a frequency
•deallocate − Frees a frequency
•stop − Stops the server, frees all
![Page 25: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/25.jpg)
Example diagram
25 © All rights reserved
![Page 26: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/26.jpg)
Example diagram
26 © All rights reserved
Step 1 Step 2
Step 3Step 4
![Page 27: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/27.jpg)
Example diagram
27 © All rights reserved
Step 1 Step 2
Step 3Step 4
![Page 28: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/28.jpg)
Example diagram
28 © All rights reserved
Step 1 Step 2
Step 3Step 4
![Page 29: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/29.jpg)
Example diagram
29 © All rights reserved
Step 1 Step 2
Step 3Step 4
![Page 30: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/30.jpg)
Example diagram
30 © All rights reserved
Step 1 Step 2
Step 3Step 4
![Page 31: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/31.jpg)
Example generated test
com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer();
com.interoud.freqserver.test.parser.FreqServerResponse var2 = this.allocateFrequency();
// Postcondition: 1
this.checkNoErrors(var2);
// Postcondition: 2
com.interoud.freqserver.test.parser.Result var4 = var2.getResult();
java.lang.Integer var5 = var4.getFrequencyAllocated();
com.interoud.freqserver.test.parser.Result var6 = var2.getResult();
java.lang.Integer var7 = var6.getFrequencyAllocated();
boolean var8 = var5.equals(var7);
// Postcondition: 3
com.interoud.freqserver.test.parser.Result var9 = var2.getResult();
java.lang.Integer var10 = var9.getFrequencyAllocated();
junit.framework.Assert.assertNotNull(var10);
// End of postconditions
java.lang.Integer var12 = var6.getFrequencyAllocated();
com.interoud.freqserver.test.parser.FreqServerResponse var13 = this.deallocateFrequency(var12);
// Postcondition: 1
this.checkNoErrors(var13);
// End of postconditions
int var15 = 0;
java.lang.Integer var16 = new java.lang.Integer(var15);
com.interoud.freqserver.test.parser.FreqServerResponse var17 = this.deallocateFrequency(var16);
// Postcondition: 1
this.checkNotAllocatedError(var17);
// Postcondition: 2
this.checkNotRunningError(var17);
// End of postconditions
Step 1
Step 2
Step 3
Step 4
© All rights reserved
![Page 32: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/32.jpg)
Example generated test
com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer();
com.interoud.freqserver.test.parser.FreqServerResponse var2 = this.allocateFrequency();
// Postcondition: 1
this.checkNoErrors(var2);
// Postcondition: 2
com.interoud.freqserver.test.parser.Result var4 = var2.getResult();
java.lang.Integer var5 = var4.getFrequencyAllocated();
com.interoud.freqserver.test.parser.Result var6 = var2.getResult();
java.lang.Integer var7 = var6.getFrequencyAllocated();
boolean var8 = var5.equals(var7);
// Postcondition: 3
com.interoud.freqserver.test.parser.Result var9 = var2.getResult();
java.lang.Integer var10 = var9.getFrequencyAllocated();
junit.framework.Assert.assertNotNull(var10);
// End of postconditions
java.lang.Integer var12 = var6.getFrequencyAllocated();
com.interoud.freqserver.test.parser.FreqServerResponse var13 = this.deallocateFrequency(var12);
// Postcondition: 1
this.checkNoErrors(var13);
// End of postconditions
int var15 = 0;
java.lang.Integer var16 = new java.lang.Integer(var15);
com.interoud.freqserver.test.parser.FreqServerResponse var17 = this.deallocateFrequency(var16);
// Postcondition: 1
this.checkNotAllocatedError(var17);
// Postcondition: 2
this.checkNotRunningError(var17);
// End of postconditions
Step 1
Step 2
Step 3
Step 4
© All rights reserved
![Page 33: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/33.jpg)
Example generated test
com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer();
com.interoud.freqserver.test.parser.FreqServerResponse var2 = this.allocateFrequency();
// Postcondition: 1
this.checkNoErrors(var2);
// Postcondition: 2
com.interoud.freqserver.test.parser.Result var4 = var2.getResult();
java.lang.Integer var5 = var4.getFrequencyAllocated();
com.interoud.freqserver.test.parser.Result var6 = var2.getResult();
java.lang.Integer var7 = var6.getFrequencyAllocated();
boolean var8 = var5.equals(var7);
// Postcondition: 3
com.interoud.freqserver.test.parser.Result var9 = var2.getResult();
java.lang.Integer var10 = var9.getFrequencyAllocated();
junit.framework.Assert.assertNotNull(var10);
// End of postconditions
java.lang.Integer var12 = var6.getFrequencyAllocated();
com.interoud.freqserver.test.parser.FreqServerResponse var13 = this.deallocateFrequency(var12);
// Postcondition: 1
this.checkNoErrors(var13);
// End of postconditions
int var15 = 0;
java.lang.Integer var16 = new java.lang.Integer(var15);
com.interoud.freqserver.test.parser.FreqServerResponse var17 = this.deallocateFrequency(var16);
// Postcondition: 1
this.checkNotAllocatedError(var17);
// Postcondition: 2
this.checkNotRunningError(var17);
// End of postconditions
Step 1
Step 2
Step 3
Step 4
© All rights reserved
![Page 34: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/34.jpg)
PILOT STUDY
Results and lessons learned
© All rights reserved
![Page 35: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/35.jpg)
Pilot study in Interoud
35
• On an industrial Web Service: VoDKaTV
• Video on Demand
• 20 operations related to:
• Rooms and device management
• Search by criteria
• Example operations
CreateRoom, FindAllRoom, CreateDevice, FindDeviceById, FindDevicesByRoom, UpdateDevice, DeleteDevice, etc.
© All rights reserved
![Page 36: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/36.jpg)
Pilot study in Interoud
36
• Evaluated possible threats to validity:
• How many tests are needed?A single test-case already produces new results
• Are requirements realistic?It requires some manual work, but yes
• Are new tests useful?They helped find an unknown bug!
• Are new tests meaningful?Not so much...
© All rights reserved
![Page 37: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/37.jpg)
Pilot study in Interoud
37
• Implicit objectsid = createRoom(name, location);
checkRoomExists(id, name, location);
• Side effects for client objects
• Classify assertions further
• Primitive values are difficult to track
© All rights reserved
![Page 38: Model extraction and test generation from JUnit test suites · Example generated test com.interoud.freqserver.test.parser.FreqServerResponse var1 = this.startServer(); com.interoud.freqserver.test.parser.FreqServerResponse](https://reader035.fdocuments.in/reader035/viewer/2022062415/5fc736f2d987cd6f81014566/html5/thumbnails/38.jpg)
Questions?
38
James tool:
PROWESS:
Presenter:
Email:
© All rights reserved
https://github.com/palas/james
http://prowessproject.eu/
Pablo Lamela Seijas