IMPROVED FITNESS FUNCTIONS FOR AUTOMATED PROGRAM REPAIR ZACHARY P. FRY.
-
Upload
dorothy-phillips -
Category
Documents
-
view
214 -
download
0
Transcript of IMPROVED FITNESS FUNCTIONS FOR AUTOMATED PROGRAM REPAIR ZACHARY P. FRY.
3
IMPROVED FITNESS FUNCTIONS
Automatic program repair can fix bugs.
Bugs
Fixes
Fitness Functions
GenProg
4
IMPROVED FITNESS FUNCTIONS
• The current fitness model is imprecise
Ideas:
• Not all test cases are created equal
• Test cases may not describe all relevant program behavior
• Different types of bugs might benefit from different kinds of fixes
We propose to address the naivety of the current fitness representation.
5
FITNESS DISTANCE CORRELATION
“Quantifying the extent to which a GA fitness function approaches an ideal of heuristic search”1
Informally, does a given fitness function produce values that correlate with some grounded notion of “closeness to a fix”?
1) T. Jones and S. Forrest. Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In International Conference on Genetic Algorithms, pages 184–192, 1995.
6
IMPROVED FITNESS FUNCTIONS
• Measuring proximity to a fix• Insert, delete, and swapping lines in the program
Fix d(135)
NO FIX
FIX
i(251,205) i(774,111) s(598,324)
7
IMPROVED FITNESS FUNCTIONS
• Measuring proximity to a fix• Insert, delete, and swapping lines in the program
M1
Fix d(135)
NO FIX
FIX
i(251,205) i(774,111) s(598,324)
i(251,205) i(774,111) s(598,324) d(63)
8
IMPROVED FITNESS FUNCTIONS
• Measuring proximity to a fix• Insert, delete, and swapping lines in the program
M1
Fix d(135)
NO FIX
FIX
i(251,205) i(774,111) s(598,324)
i(251,205) i(774,111) s(598,324) d(63)✓ ✓ ✓ ✗ 75%
9
IMPROVED FITNESS FUNCTIONS
• Measuring proximity to a fix• Insert, delete, and swapping lines in the program
M1
M2
Fix d(135)
NO FIX
FIX
i(251,205) i(774,111) s(598,324)
i(251,205) i(774,111) s(598,324) d(63)✓ ✓ ✓ ✗ 75%
d(84) s(844,265) i(774,111) i(735,431)
10
IMPROVED FITNESS FUNCTIONS
• Measuring proximity to a fix• Insert, delete, and swapping lines in the program
M1
M2
Fix d(135)
NO FIX
FIX
i(251,205) i(774,111) s(598,324)
i(251,205) i(774,111) s(598,324) d(63)✓ ✓ ✓ ✗ 75%
d(84) s(844,265) i(774,111) i(735,431)✓✗ ✗ 25%✗
11
IMPROVED FITNESS FUNCTIONS
The current model of fitness does not correlate well with proximity to a fix (0.145).
Hypothesis: By taking into account previously unused information about test cases, bugs, and fixes we can better inform the evolutionary bug fixing process to fix bugs faster and more often.
12
IMPROVED FITNESS FUNCTIONS
Approach: weight test cases based on known fixes
Test Case 1
Test Case 2
M1
M2
M3
M4
M1
M2
M3
M4
FIX NO FIX
13
IMPROVED FITNESS FUNCTIONS
Approach: weight test cases based on known fixes
Test Case 1
Test Case 2
M1
M2
M3
M4
M1
M2
M3
M4
FIX NO FIX
14
IMPROVED FITNESS FUNCTIONS
Approach: weight test cases based on known fixes
Test Case 1
Test Case 2
M1
M2
M3
M4
M1
M2
M3
M4
FIX NO FIX
0.8 0.2
15
IMPROVED FITNESS FUNCTIONS
Evaluation:
• How much can we speed up fixes?• Computational time and monetary cost• Preliminary results
• How many more bugs can we fix? • Fraction of previously unfixed bugs• Future work
16
PRELIMINARY RESULTS
For a sample of 15 bugs from one program, 31.3% of test cases show no correlation with actual fitness (closeness to a fix)
Bug Avg Percent Time Savings
libtiff-bug-0fb6cf7-b4158fa 24.47%
libtiff-bug-01209c9-aaf9eb3 50.49%
libtiff-bug-10a4985-5362170 35.53%
libtiff-bug-5b02179-3dfb33b 22.08%
libtiff-bug-8f6338a-4c5a9ec 62.24%
Total: 38.96%
17
PRELIMINARY RESULTS
Some test cases are over 23x more correlated with actual fitness than others• Suggests an adequate weighting scheme
using machine learning could fix more bugs, faster
This work and additional efforts to investigate other strategies for improving fitness functions are ongoing.
18
APPLICABILITY
• When might this work?• Programs with expensive test suites – e.g. Php
(12,000+)• When there is heavy overlap between test cases• Test suites/cases that fail to specify the bug
• Assumptions?• Presence of historical bug fix data to mine• Test suites do not evolve drastically from bug to bug• Bugs for a given program are related on some level
19
GOALS
• By providing GenProg a better signal for mutants’ fitness we hope to:• Better direct the search – arrive at fixes
faster, lowering cost (up to 38%)• In the limit, find more fixes for previously
unfixed bugs