Automatically Repairing Test Cases for Evolving Method Declarations
Transcript of Automatically Repairing Test Cases for Evolving Method Declarations
Automatically Repairing Test Cases
for Evolving Method DeclarationsMehdi MirzaaghaeiFabrizio Pastore
Mauro Pezzè
Universitàdella Svizzeraitaliana
http://swiss-landmarks.ch/panos/Lugano9.jpg
Software evolves
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
public class .....
public class AccountUtil { private double dailyInterestRate = 0.00005; private int interestTerm=365; public double interest(BankAccount account return account.getBalance() *dailyInterestRate*interestTerm; } ...
public class AccountFactory {
public static BankAccount create( AccountContext ctx, boolean special ){ BankAccount account = new BankAcccount( ctx.amount ); if ( special ){ account.setInterestRate(0.0001); } ...
public class .....
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; };}
Original classes are modified
public class .....
public class .....
Test cases need maintenance
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; };}
Compilation error:Method deposit(Money) in the type BankAccount is not applicable for the arguments (int)
changes in method declarations: 23% changes
that impact on compilation
changes in method declarations: 23% changes
that impact on compilation
> 80% in maintenance releases
changes in method declarations: 23% changes
that impact on compilation
> 80% in maintenance releases
Automatic Repair to Save Effort
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests
Well suited only for GUI tests
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests Repair Oracles
[B. Daniel et al 2010]
testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40
assertEquals( 40, result );
Well suited only for GUI tests Focus on oracles only
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests
Refactoring Techniques [ReBa, Eclipse]
Repair Oracles[B. Daniel et al 2010]
testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40
assertEquals( 40, result );
//calculate one year interestresult = account.interest();
result = account.interest( 0 );
Well suited only for GUI tests Focus on oracles only
Prevent only some compilation errors
Automatic Test Repair
TestCareAssistant repairs test case compilation errors
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
Repair parameter type change
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
Which parameter to initialize?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
Which parameter to initialize?
Use code diff to identify the parameter1
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to initialize a complex object?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to initialize a complex object?
Find first uses of parameter fields in Version12
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to determine fields values?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to determine fields values?
Diff to identify corresponding variable in Version 03
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
Analyze the def-use chain of the variable back to the definition in the test4
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
Analyze the def-use chain of the variable back to the definition in the test4
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
Instantiate the new object, use original values to initialize fields5
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
Instantiate the new object, use original values to initialize fields5
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
To what extent TestCareAssistant can fix test cases automatically?
To what extent TestCareAssistant can fix test cases automatically?
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
• Static data flow analysis not always effective
• Use of complex data structures, e.g. hash tables
• Changes in method logic
• Changes in interfaces
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
TestCareAssistant
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Automate Test Generation by Identifying and Implementing Test Adaptation Patterns
Questions?public class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
TestCareAssistant