ICSE 2014: Unit Test Virtualization with VMVM
description
Transcript of ICSE 2014: Unit Test Virtualization with VMVM
![Page 1: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/1.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Unit Test Virtualization with VMVM
Jonathan Bell and Gail Kaiser Columbia University
Fork me on Github
![Page 2: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/2.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Good news: We have tests!
No judgement on whether they are complete or not, but we sure have a lot of them
Number of testsApache Tomcat 1,734
Closure Compiler 7,949Commons I/O 1,022
![Page 3: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/3.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Bad news: We have to run a lot of tests!
![Page 4: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/4.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Bad news: We have to run a lot of tests!
• Much work has focused on improving the situation:
• Test Suite Prioritization
• E.g. Wong [ISSRE ’97], Rothermel [ICSM ’99]; Elbaum [ICSE ’01]; Srivastava [ISSTA ’02] and more
![Page 5: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/5.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Bad news: We have to run a lot of tests!
• Much work has focused on improving the situation:
• Test Suite Prioritization
• E.g. Wong [ISSRE ’97], Rothermel [ICSM ’99]; Elbaum [ICSE ’01]; Srivastava [ISSTA ’02] and more
• Test Suite Minimization
• E.g. Harrold [TOSEM ’93]; Wong [ICSE ’95]; Chen [IST ’98]; Jones [TOSEM ’03]; Tallam [PASTE ’05]; Jeffrey [TSE ’07]; Orso [ICSE ’09] Hao [ICSE ’12] and more
![Page 6: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/6.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Testing still takes too long.
![Page 7: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/7.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach: Unit Test Virtualization
![Page 8: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/8.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach: Unit Test Virtualization
Reduces test execution time by up to 97%, on average 62%
![Page 9: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/9.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach: Unit Test Virtualization
Reduces test execution time by up to 97%, on average 62%
Apache Tomcat: From 26 minutes to 18 minutes
![Page 10: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/10.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach: Unit Test Virtualization
Reduces test execution time by up to 97%, on average 62%
Integrates with JUnit, ant, and mvn on unmodified JVMs.
Apache Tomcat: From 26 minutes to 18 minutes
![Page 11: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/11.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach: Unit Test Virtualization
Reduces test execution time by up to 97%, on average 62%
Integrates with JUnit, ant, and mvn on unmodified JVMs.
Apache Tomcat: From 26 minutes to 18 minutes
Available on GitHub
![Page 12: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/12.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
![Page 13: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/13.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
![Page 14: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/14.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
1.4 sec (combined)!For EVERY test!
![Page 15: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/15.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
1.4 sec (combined)!For EVERY test!
Overhead of restarting the JVM?
![Page 16: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/16.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
1.4 sec (combined)!For EVERY test!
Overhead of restarting the JVM?Unit tests as fast as 3-5 ms
![Page 17: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/17.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
1.4 sec (combined)!For EVERY test!
Overhead of restarting the JVM?Unit tests as fast as 3-5 ms
JVM startup time is fairly constant (1.4 sec)
![Page 18: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/18.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
JUnit Test Execution
Start JVM
Execute Test
Terminate AppBegin Test
Start Test Suite
1.4 sec (combined)!For EVERY test! Up to 4,153%, avg 618%
Overhead of restarting the JVM?Unit tests as fast as 3-5 ms
JVM startup time is fairly constant (1.4 sec)
*From our study of 20 popular FOSS apps
![Page 19: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/19.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Do applications really use a new JVM for each test?
• Checked out the 1,000 largest Java projects from Ohloh
• 81% of projects with more than 1,000 tests do it
• 71% of projects with more than 1 million LOC do it
• Overall: 41% of all of the projects do
![Page 20: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/20.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence• We typically assume that tests are order-
independent
![Page 21: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/21.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence• We typically assume that tests are order-
independent
• Might rely on developers to completely reset the system under test between tests
• Who tests the tests?
![Page 22: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/22.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence• We typically assume that tests are order-
independent
• Might rely on developers to completely reset the system under test between tests
• Who tests the tests?
• Dangerous: If wrong, can have false positives or false negatives (Muşlu [FSE ’11], Zhang [ISSTA ’14])
![Page 23: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/23.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence
/**#If#true,#cookie#values#are#allowed#to#contain#an#equals#character#without#being#quoted.#*/#public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#####Boolean.valueOf(System.getProperty("org.apache.tomcat.#####util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE","false"))#########.booleanValue();
![Page 24: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/24.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence
/**#If#true,#cookie#values#are#allowed#to#contain#an#equals#character#without#being#quoted.#*/#public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#####Boolean.valueOf(System.getProperty("org.apache.tomcat.#####util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE","false"))#########.booleanValue();
This field is set once, when the class that owns it is initialized
![Page 25: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/25.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Test Independence
/**#If#true,#cookie#values#are#allowed#to#contain#an#equals#character#without#being#quoted.#*/#public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#####Boolean.valueOf(System.getProperty("org.apache.tomcat.#####util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE","false"))#########.booleanValue();
This field is set once, when the class that owns it is initialized
This field’s value is dependent on an external property
![Page 26: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/26.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
A Tale of Two Tests
TestAllowEqualsInValue TestDontAllowEqualsInValue
public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#Boolean.valueOf(## System.getProperty(“org.apache.tomcat.util.http.ServerCookie.## ALLOW_EQUALS_IN_VALUE","false")).booleanValue();
![Page 27: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/27.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
A Tale of Two Tests
TestAllowEqualsInValue TestDontAllowEqualsInValue
Sets environmental variable to true Start Tomcat, run test
public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#Boolean.valueOf(## System.getProperty(“org.apache.tomcat.util.http.ServerCookie.## ALLOW_EQUALS_IN_VALUE","false")).booleanValue();
![Page 28: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/28.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
A Tale of Two Tests
TestAllowEqualsInValue TestDontAllowEqualsInValue
Sets environmental variable to true Start Tomcat, run test
public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#Boolean.valueOf(## System.getProperty(“org.apache.tomcat.util.http.ServerCookie.## ALLOW_EQUALS_IN_VALUE","false")).booleanValue();
Sets environmental variable to false Start Tomcat, run test
![Page 29: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/29.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
A Tale of Two Tests
TestAllowEqualsInValue TestDontAllowEqualsInValue
Sets environmental variable to true Start Tomcat, run test
public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#Boolean.valueOf(## System.getProperty(“org.apache.tomcat.util.http.ServerCookie.## ALLOW_EQUALS_IN_VALUE","false")).booleanValue();
Sets environmental variable to false Start Tomcat, run test
But our static field is stuck!
TestDontAllowEqualsInValue
![Page 30: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/30.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
A Tale of Two Tests
TestAllowEqualsInValue TestDontAllowEqualsInValue
Sets environmental variable to true Start Tomcat, run test
public#static#boolean#ALLOW_EQUALS_IN_VALUE#=#Boolean.valueOf(## System.getProperty(“org.apache.tomcat.util.http.ServerCookie.## ALLOW_EQUALS_IN_VALUE","false")).booleanValue();
Sets environmental variable to false Start Tomcat, run test
TestAllowEqualsInValue
![Page 31: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/31.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Our Approach
Unit Test Virtualization: Allow tests to leave side-effects. But efficiently contain them.
![Page 32: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/32.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
We think in terms of object graphs
![Page 33: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/33.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
Test Runner Instance
We think in terms of object graphs
![Page 34: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/34.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
Test Runner Instance
Test Case 1
references
Accessible!Objects
references
We think in terms of object graphs
![Page 35: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/35.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
Test Runner Instance
Test Case 1
references
Test Case 2
references
Accessible!Objects
references
Accessible!Objects
references
We think in terms of object graphs
![Page 36: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/36.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
Test Runner Instance
Test Case 1
references
Test Case 2
references
Accessible!Objects
references
Accessible!Objects
references
Accessible!Objects
references
Test Case n
references
We think in terms of object graphs
![Page 37: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/37.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
Test Runner Instance
Test Case 1
references
Test Case 2
references
Accessible!Objects
references
Accessible!Objects
references
Accessible!Objects
references
Test Case n
references
We think in terms of object graphs
No cross-talk No cross-talk
![Page 38: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/38.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
We think in terms of object graphs
Class#A
Static#Fields
Class#B
Static#Fields
Static fields: owned by a class, NOT by an instance
![Page 39: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/39.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
How do Tests Leak Data?Java is memory-managed, and object oriented
We think in terms of object graphs
Class#A
Static#Fields
Class#B
Static#Fields
Static fields: owned by a class, NOT by an instanceThese are leakage points
referencesreferences
![Page 40: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/40.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static#Fields
Class%B
Static#Fields
Class%C
Static#Fields
Test 1 Test 2
![Page 41: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/41.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static#Fields
Class%B
Static#Fields
Class%C
Static#Fields
Test 1 Test 2
Writ
es
![Page 42: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/42.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static#Fields
Class%B
Static#Fields
Class%C
Static#Fields
Test 1 Test 2
Writ
es
Reads
![Page 43: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/43.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static#Fields
Class%B
Static#Fields
Class%C
Static#Fields
Test 1 Test 2
Writ
es
Reads Writ
es
![Page 44: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/44.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static#Fields
Class%B
Static#Fields
Class%C
Static#Fields
Test 1 Test 2
Writ
es ReadsReads
Static%Fields
Writ
es
![Page 45: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/45.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static&Fields
Class%B
Static&Fields
Class%C
Static&Fields
Test 1 Test 2
Writ
es ReadsReads W
rites
*Inter
cept
ion*
Static%Fields
![Page 46: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/46.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static&Fields
Class%B
Static&Fields
Class%C
Static&Fields
Test 1 Test 2
Writes Reads
Reads W
rites
*Interception*
![Page 47: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/47.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static&Fields
Class%B
Static&Fields
Class%C
Static&Fields
Test 1 Test 2
Writ
es ReadsReads W
rites
*Inter
cept
ion*
![Page 48: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/48.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static&Fields
Class%B
Static&Fields
Class%C
Static&Fields
Test 1 Test 2
Writ
es ReadsReads W
rites
*Inter
cept
ion*
So, don’t touch them!
These classes had no possible conflicts
![Page 49: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/49.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Isolating Side EffectsClass%A
Static&Fields
Class%B
Static&Fields
Class%C
Static&Fields
Test 1 Test 2
Writ
es ReadsReads W
rites
*Inter
cept
ion*
So, don’t touch them!
These classes had no possible conflicts
Key Insight:!No need to re-initialize the entire application in order
to isolate tests
![Page 50: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/50.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
VMVM: Unit Test Virtualization• Isolates in-memory side effects, just like restarting
JVM
![Page 51: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/51.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
VMVM: Unit Test Virtualization• Isolates in-memory side effects, just like restarting
JVM
• Integrates easily with ant, maven, junit
![Page 52: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/52.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
VMVM: Unit Test Virtualization• Isolates in-memory side effects, just like restarting
JVM
• Integrates easily with ant, maven, junit
• Implemented completely with application byte code instrumentation
![Page 53: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/53.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
VMVM: Unit Test Virtualization• Isolates in-memory side effects, just like restarting
JVM
• Integrates easily with ant, maven, junit
• Implemented completely with application byte code instrumentation
• No changes to JVM, no access to source code required
![Page 54: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/54.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
![Page 55: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/55.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
![Page 56: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/56.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
![Page 57: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/57.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
![Page 58: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/58.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Release lock on TNo
t initia
lized
![Page 59: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/59.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Release lock on TNo
t initia
lized
![Page 60: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/60.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Release lock on TNo
t initia
lized
![Page 61: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/61.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Release lock on T Acquire lock on TNo
t initia
lized
![Page 62: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/62.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Mark init done
Release lock on T Acquire lock on TNo
t initia
lized
![Page 63: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/63.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Mark init done
Release lock on T
Release lock on T Acquire lock on TNo
t initia
lized
![Page 64: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/64.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Mark init done
Release lock on T
Release lock on T Acquire lock on TNo
t initia
lized
First new instance or static reference of T per test
![Page 65: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/65.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Mark init done
Release lock on T
Release lock on T Acquire lock on T
Re-initialize T’s super classes
Not in
itializ
edFirst new instance or static
reference of T per test
![Page 66: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/66.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient ReinitializationEmulate exactly what happens when a class is initialized the first time
First new instance or static reference of T
Acquire lock on T
Check initialization status
Initialize T’s super classes
Run initializer for T
Mark init done
Release lock on T
Release lock on T Acquire lock on T
Re-initialize T’s super classes
Not in
itializ
edFirst new instance or static
reference of T per test
Re-initialize T
![Page 67: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/67.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient Reinitialization• Does not require any modifications to the JVM and
runs on commodity JVMs
![Page 68: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/68.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient Reinitialization• Does not require any modifications to the JVM and
runs on commodity JVMs
• The JVM calls a special method, <clinit> to initialize a class
![Page 69: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/69.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient Reinitialization• Does not require any modifications to the JVM and
runs on commodity JVMs
• The JVM calls a special method, <clinit> to initialize a class
• We do the same, entirely in Java
![Page 70: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/70.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient Reinitialization• Does not require any modifications to the JVM and
runs on commodity JVMs
• The JVM calls a special method, <clinit> to initialize a class
• We do the same, entirely in Java
• Add guards to trigger this process
![Page 71: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/71.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Efficient Reinitialization• Does not require any modifications to the JVM and
runs on commodity JVMs
• The JVM calls a special method, <clinit> to initialize a class
• We do the same, entirely in Java
• Add guards to trigger this process
• Register a hook with test runner to tell us when a new test starts
![Page 72: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/72.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Experiments
• RQ1: How does VMVM compare to Test Suite Minimization?
![Page 73: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/73.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Experiments
• RQ1: How does VMVM compare to Test Suite Minimization?
• RQ2: What are the performance gains of VMVM?
![Page 74: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/74.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Experiments
• RQ1: How does VMVM compare to Test Suite Minimization?
• RQ2: What are the performance gains of VMVM?
• RQ3: Does VMVM impact fault finding ability?
![Page 75: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/75.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ1: VMVM vs Test Minimization
• Study design follows Zhang [ISSRE ‘11]’s evaluation of four minimization approaches
![Page 76: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/76.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ1: VMVM vs Test Minimization
• Study design follows Zhang [ISSRE ‘11]’s evaluation of four minimization approaches
• Compare to the minimization technique with least impact on fault finding ability, Harrold [TOSEM ‘93]'s technique
![Page 77: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/77.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ1: VMVM vs Test Minimization
• Study design follows Zhang [ISSRE ‘11]’s evaluation of four minimization approaches
• Compare to the minimization technique with least impact on fault finding ability, Harrold [TOSEM ‘93]'s technique
• Study performed on the popular Software Infrastructure Repository dataset
![Page 78: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/78.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
0% !10%!20%!30%!40%!50%!60%!70%!80%!90%!
Ant v1!
Ant v2!
Ant v3!
Ant v4!
Ant v5!
Ant v6!
Ant v7!
Ant v8!
JMete
r v1!
JMete
r v2!
JMete
r v3!
JMete
r v4!
JMete
r v5!
jtopas
v1!
jtopas
v2!
jtopas
v3!
xml-s
ec v1!
xml-s
ec v2!
xml-s
ec v3!
Redu
ctio
n in
Tes
ting
Tim
e!
Application!
Test Suite Minimization ! VMVM! Combined!
RQ1: VMVM vs Test Minimization
Larger is bette
r
![Page 79: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/79.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
0% !10%!20%!30%!40%!50%!60%!70%!80%!90%!
Ant v1!
Ant v2!
Ant v3!
Ant v4!
Ant v5!
Ant v6!
Ant v7!
Ant v8!
JMete
r v1!
JMete
r v2!
JMete
r v3!
JMete
r v4!
JMete
r v5!
jtopas
v1!
jtopas
v2!
jtopas
v3!
xml-s
ec v1!
xml-s
ec v2!
xml-s
ec v3!
Redu
ctio
n in
Tes
ting
Tim
e!
Application!
Test Suite Minimization ! VMVM! Combined!
13%
RQ1: VMVM vs Test Minimization
Larger is bette
r
![Page 80: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/80.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
0% !10%!20%!30%!40%!50%!60%!70%!80%!90%!
Ant v1!
Ant v2!
Ant v3!
Ant v4!
Ant v5!
Ant v6!
Ant v7!
Ant v8!
JMete
r v1!
JMete
r v2!
JMete
r v3!
JMete
r v4!
JMete
r v5!
jtopas
v1!
jtopas
v2!
jtopas
v3!
xml-s
ec v1!
xml-s
ec v2!
xml-s
ec v3!
Redu
ctio
n in
Tes
ting
Tim
e!
Application!
Test Suite Minimization ! VMVM! Combined!
13%
46%
RQ1: VMVM vs Test Minimization
Larger is bette
r
![Page 81: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/81.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
0% !10%!20%!30%!40%!50%!60%!70%!80%!90%!
Ant v1!
Ant v2!
Ant v3!
Ant v4!
Ant v5!
Ant v6!
Ant v7!
Ant v8!
JMete
r v1!
JMete
r v2!
JMete
r v3!
JMete
r v4!
JMete
r v5!
jtopas
v1!
jtopas
v2!
jtopas
v3!
xml-s
ec v1!
xml-s
ec v2!
xml-s
ec v3!
Redu
ctio
n in
Tes
ting
Tim
e!
Application!
Test Suite Minimization ! VMVM! Combined!
13%
46%49%
RQ1: VMVM vs Test Minimization
Larger is bette
r
![Page 82: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/82.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
• Previous study: well-studied suite of 4 projects, which average 37,000 LoC and 51 test classes
![Page 83: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/83.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
• Previous study: well-studied suite of 4 projects, which average 37,000 LoC and 51 test classes
• This study: manually collected repository of 20 projects, average 475,000 LoC and 56 test classes
![Page 84: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/84.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
• Previous study: well-studied suite of 4 projects, which average 37,000 LoC and 51 test classes
• This study: manually collected repository of 20 projects, average 475,000 LoC and 56 test classes
• Range from 5,000 LoC - 5,692,450 LoC; 3 - 292 test classes; 3.5-15 years in age
![Page 85: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/85.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
0%! 20% ! 40%! 60%! 80%! 100%!upm!JTor !
Openfire !Trove for Java!
FreeRapid Downloader !JAXX!
Commons Validator !Commons Codec!Closure Compiler !
betterFORM!Apache Ivy !
mkgmap!gedcom4j!
btrace !Apache River !Commons IO!
Jetty !Apache Tomcat !
Apache Nutch !Bristlecone!
Relative Speedup !Larger is better
![Page 86: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/86.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
0%! 20% ! 40%! 60%! 80%! 100%!upm!JTor !
Openfire !Trove for Java!
FreeRapid Downloader !JAXX!
Commons Validator !Commons Codec!Closure Compiler !
betterFORM!Apache Ivy !
mkgmap!gedcom4j!
btrace !Apache River !Commons IO!
Jetty !Apache Tomcat !
Apache Nutch !Bristlecone!
Relative Speedup !
Average: 62%
Larger is better
![Page 87: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/87.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ2: Broader Evaluation
0%! 20% ! 40%! 60%! 80%! 100%!upm!JTor !
Openfire !Trove for Java!
FreeRapid Downloader !JAXX!
Commons Validator !Commons Codec!Closure Compiler !
betterFORM!Apache Ivy !
mkgmap!gedcom4j!
btrace !Apache River !Commons IO!
Jetty !Apache Tomcat !
Apache Nutch !Bristlecone!
Relative Speedup !
Max: 97%
Average: 62%
Larger is better
![Page 88: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/88.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Factors that impact reduction
• Looked for relationships between number of tests, lines of code, age of project, total testing time, time per test, and VMVM’s speedup
![Page 89: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/89.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Factors that impact reduction
• Looked for relationships between number of tests, lines of code, age of project, total testing time, time per test, and VMVM’s speedup
• Result: Only average time per test is correlated with VMVM’s speedup (in fact, quite strongly; p < 0.0001)
![Page 90: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/90.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ3: Impact on Fault Finding• No impact on fault finding from seeded faults (SIR)
![Page 91: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/91.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ3: Impact on Fault Finding• No impact on fault finding from seeded faults (SIR)
• Does VMVM correctly isolate tests though?
![Page 92: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/92.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ3: Impact on Fault Finding• No impact on fault finding from seeded faults (SIR)
• Does VMVM correctly isolate tests though?
• Compared false positives and negatives between un-isolated execution, traditionally isolated execution, and VMVM-isolated execution for these 20 complex applications
![Page 93: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/93.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
RQ3: Impact on Fault Finding• No impact on fault finding from seeded faults (SIR)
• Does VMVM correctly isolate tests though?
• Compared false positives and negatives between un-isolated execution, traditionally isolated execution, and VMVM-isolated execution for these 20 complex applications
• Result: False positives occur when not isolated. VMVM shows no false positives or false negatives.
![Page 94: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/94.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Conclusions
• Most large applications isolate their test cases
![Page 95: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/95.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Conclusions
• Most large applications isolate their test cases
• VMVM provides up to a 97% reduction in testing time through more efficient isolation (average 62%)
![Page 96: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/96.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Conclusions
• Most large applications isolate their test cases
• VMVM provides up to a 97% reduction in testing time through more efficient isolation (average 62%)
• VMVM does not risk a reduction in fault finding
![Page 97: ICSE 2014: Unit Test Virtualization with VMVM](https://reader034.fdocuments.in/reader034/viewer/2022052507/558049b7d8b42ae32c8b4d3c/html5/thumbnails/97.jpg)
@_jon_bell_ICSE 2014 June 5, 2014
Unit Test Virtualization with VMVM
Jonathan Bell and Gail Kaiser Columbia University
https://github.com/Programming-Systems-Lab/vmvm
See a demo of VMVM at 2:30 today! Room MR G1-3
Fork me on Github