Post on 07-Feb-2021
FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage
Caroline Lemieux, Koushik SenUniversity of California, Berkeley
source: https://github.com/carolemieux/afl-rb
F z u z? u z r
F a i r F u z zF u z !
The Rise of Fuzz Testing
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 1
• Programs still have bugs.• Fuzz testing has become very popular in practice and theory
Fuzzing in One Slide
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 2
Fuzzer Program
Feedback
Input
Fuzzing in One Slide
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 3
Fuzzer Program
Feedback
A_VERY_BAD_INPUT
Input
Fuzzing in One Slide
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 4
Fuzzer Program
Feedback
A_VERY_BAD_INPUT
Input
Fuzzing in One Slide
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 5
Fuzzer Program
Feedback
A_VERY_BAD_INPUT
A_VERY_BAD_INPUTThe input:
causes a crash.
Input
What Bugs Can Fuzzing Find?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 6
• Most popular: basic correctness assertions (C/C++)• Segmentation faults• Anything address sanitizer can catch:
• Buffer overflows• Use-after-frees• Etc…
Coverage-Guided (Greybox) Mutational Fuzzing
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 7
Fuzzer Program
Feedback(branches covered)
Input
Coverage-Guided (Greybox) Mutational Fuzzing
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 8
FuzzerSaved inputs
Program
Feedback(branches covered)
Input
Coverage-Guided (Greybox) Mutational Fuzzing
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 9
FuzzerSaved inputs
Program
Mutate saved input
Feedback(branches covered)
Input
Coverage-Guided (Greybox) Mutational Fuzzing
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 10
FuzzerSaved inputs
Program
Mutate saved input
New coverage? Save. Feedback(branches covered)
Input
What’s Missing? Uneven Fuzzing Coverage
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 11
Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
}// ...return process_result;
}
What’s Missing? Uneven Fuzzing Coverage
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 12
Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
}// ...return process_result;
}
Hit by 100k+ inputs
What’s Missing? Uneven Fuzzing Coverage
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 13
Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
}// ...return process_result;
}
Hit by 100k+ inputsà Code under if well-covered
What’s Missing? Uneven Fuzzing Coverage
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 14
Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
}// ...return process_result;
}
Hit by 100k+ inputs
Hit by 1 input
à Code under if well-covered
What’s Missing? Uneven Fuzzing Coverage
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 15
Observation: some parts of the program easier to cover int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
}// ...return process_result;
}
Hit by 100k+ inputs
Hit by 1 input
à Code under if well-covered
à Code under if barely covered
Uneven Fuzzing Coverage à Uncovered Code
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 16
Observation: some parts of the program easier to cover
Result: some functionality wholly uncovered by fuzzing
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Hit by 100k+ inputs
Hit by 1 input
à Code under if well-covered
à Code under if barely covered
Why So Uneven?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 17
Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Input satisfying if:AT_LEAST_10_BYTES
AT_LEAST_10_BYS
Why So Uneven?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 18
Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Input satisfying if:AT_LEAST_10_BYTES
BT_LEAST_10_BYTESAT???_LEAST_10_BYTES
✓✓
✓
Why So Uneven?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 19
Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Input satisfying if:
Why So Uneven?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 20
Some branches hard to hit by naively mutated inputs int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Input satisfying if:
Our Method: FairFuzz
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 21
Program
Feedback
Input
F z u z? u z r
F a i r F u z zF u z !
Utilize existing greybox infoTo target rarely-exercised code à increase coverage
Our Method: FairFuzz
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 22
Program
Feedback
Input
F z u z? u z r
F a i r F u z zF u z !
Utilize existing greybox infoTo target rarely-exercised code à increase coverage
Identify: branches hit by few inputs (rare branches)
Our Method: FairFuzz
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 23
Program
Feedback
Input
F z u z? u z r
F a i r F u z zF u z !
Utilize existing greybox infoTo target rarely-exercised code à increase coverage
Identify: branches hit by few inputs (rare branches)
Identify: where input can be mutated and hit branch
Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 24
Recap: AFL
• AFL [1]: Popular coverage-guided greybox fuzzer• Fuzzes programs taking in file or stdin• Easy to use (just compile program with afl-gcc or afl-clang)• Has found many bugs in practice
[1] http://lcamtuf.coredump.cx/afl/
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 25
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 26
Seeds
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 27
Parent Input Set
Seeds
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 28
Parent Input Set
Parent Input
Select parent to mutate
Seeds
mutant
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 29
Parent Input Set
Parent Input
mutantmutantmutantMutant
Select parent to mutate
Create mutants
Seeds
mutant
mutant
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 30
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Run on instrumented
program
Seeds
mutant
mutant
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 31
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Run on instrumented
program
Get mutants adding new coverage
Seeds
Interesting MutantInteresting MutantInteresting Mutant
mutant
mutant
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 32
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
mutant
mutant
AFL Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 33
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
AFL Mutation Types
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 34
• Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”
values, etc.
AFL Mutation Types
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 35
• Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”
values, etc.
< ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D
mutants
…
AFL Mutation Types
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 36
• Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”
values, etc.
• Random-location mutation• Repeat: choose random mutation, apply at random location
< ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D
mutants
…
AFL Mutation Types
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 37
• Fixed-location mutations• Choose mutation type, apply at all locations in input• Mutation types: byte flips, arithmetic inc/dec, replacing with “interesting”
values, etc.
• Random-location mutation• Repeat: choose random mutation, apply at random location
< ! A T T L I S T B D0 0 A T T L I S T B D< 0 0 T T L I S T B D< ! 0 0 T L I S T B D
mutants
…
< ! A T T L I S T B D < ! A T T L 0 0 T B D < ? C A T L 0 0 T B D
mutant
…(repeat)
mutant
mutant
FairFuzz Method
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 38
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Compute mask
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
mutant
mutant
FairFuzz Method – Key Differences
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 39
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Compute mask
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
mutant
mutant
FairFuzz Method – Selecting Parent Inputs
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 40
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Compute mask
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
FairFuzz Method – Selecting Parent Inputs
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 41
• Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch
FairFuzz Method – Selecting Parent Inputs
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 42
• Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Hit by 100k+ inputs
Hit by 1 input
AT_LEAST_10_BYTES
FairFuzz Method – Selecting Parent Inputs
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 43
• Keep track of # of inputs produced exercising each branch• Pick inputs that exercise a branch hit by relatively few inputs• Rarest branch hit: target branch
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
Hit by 100k+ inputs
Hit by 1 input
AT_LEAST_10_BYTES
mutant
mutant
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 44
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Compute mask
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 45
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 46
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 47
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 48
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D✗
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 49
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D< ? A T T L I S T B D
✗✗
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 50
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D< ? A T T L I S T B D...
✗✗
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 51
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D< ? A T T L I S T B D...
< ! A T T L I S T ? D
✗✗
✓
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 52
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D< ? A T T L I S T B D...
< ! A T T L I S T ? D
✗✗
< ! A T T L I S T B ?✓✓
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
FairFuzz Method – Computing Branch Mask
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 53
• Easily integrated with fixed-location mutation phases of fuzzers• Flip each byte, check if mutated input still hits target branch
? ! A T T L I S T B D< ? A T T L I S T B D...
< ! A T T L I S T ? D
✗✗
< ! A T T L I S T B ?✓✓
int process_xml(char * fuzzed_data, int fuzzed_data_len) {
if (fuzzed_data_len >= 10) {// more code
}// ...if (starts_with(fuzzed_data, “
if (starts_with(&fuzzed_data[10], “ID”)) {// lots more processing code
}}// ...return process_result;
}
< ! A T T L I S T B DParent input hits
Mask: < ! A T T L I S T B D
mutant
mutant
FairFuzz Method – Targeting Mutations
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 54
Parent Input Set
Parent Input
mutantmutantmutantMutant
mutantmutantmutant
Feedback
Select parent to mutate
Create mutants
Compute mask
Run on instrumented
program
Get mutants adding new coverage
Seeds
Add to parent set
Interesting MutantInteresting MutantInteresting Mutant
FairFuzz Method – Targeting Mutations
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 55
• Fixed-location mutation• Don’t produce mutants at locations in mask
• Random-location mutation• Choose random locations outside mask
< ! A T T L I S T B D
< ! A T T L I S T B D
FairFuzz Method – Targeting Mutations
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 56
• Fixed-location mutation• Don’t produce mutants at locations in mask
• Random-location mutation• Choose random locations outside mask
< ! A T T L I S T B DCan’t mutate✗ ✗
…
< ! A T T L I S T B D
FairFuzz Method – Targeting Mutations
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 57
• Fixed-location mutation• Don’t produce mutants at locations in mask
• Random-location mutation• Choose random locations outside mask
< ! A T T L I S T B D
< ! A T T L I S T 0 0 D < ! A T T L I S T 0 0
ok to mutateCan’t mutate✗ ✗
…
< ! A T T L I S T B D
FairFuzz Method – Targeting Mutations
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 58
• Fixed-location mutation• Don’t produce mutants at locations in mask
• Random-location mutation• Choose random locations outside mask
< ! A T T L I S T B D
< ! A T T L I S T 0 0 D < ! A T T L I S T 0 0
ok to mutateCan’t mutate✗ ✗
…
< ! A T T L I S T B D
choose random position here
Evaluation
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 59
Evaluation – Tools Compared
• FairFuzz: our tool, with highest-performing settings• AFL: vanilla AFL, default settings• FidgetyAFL: AFL with highest-performing settings• AFLFast.new: AFLFast with highest-performing settings
[1] Zalewski, Michał. http://lcamtuf.coredump.cx/afl/[2] Böhme et al. Coverage-based Greybox Fuzzing as Markov Chain. CCS’16.
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 60
Evaluation - Benchmarksdjpegreadpngmutool drawxmllinttcpdumpc++filtobjdumpreadelfnm
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 61
AFLFast benchmarks
More complex input structures
FidgetyAFL benchmarks
Evaluation Setup
For each benchmark:• Run each technique 24hrs• Start with 1 valid seed file• No dictionaries• Repeat runs 20x• Calculated confidence intervals
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 62
Summary Results – Coverage Leaders
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 63
Summary Results – Coverage Leaders
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 64
à FairFuzz achieves the highest coverage fast, for nearly all benchmarks
Branch Coverage Over Time
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 65
Where Does FairFuzz Perform Much Better?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 66
Both are programs with nested conditional structure• tcpdump: if this packet type, then if
has this field…• xmllint: byte-by-byte comparisons
Where Doesn’t FairFuzz Perform As Well?
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 67
C++ name demangler: highly recursive-structured program• Covering different branches may
not be best exploration strategy
Conclusion
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 68
code: github.com/carolemieux/afl-rb slides: carolemieux.com/fairfuzz_ase18_slides.pdf
Branch Mask Performance
For a subset of benchmarks, run a cycle with “shadow run”:• For each selected input, create mutants• (1) without branch mask• (2) without branch mask
• Compare % of inputs hitting target branch:• Average over all inputs selected for mutation in cycle
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 69
Branch Mask Performance
UniversityofCalifornia,Berkeley FairFuzz,presentedbyCarolineLemieux 70
• Mask substantially increases % of inputs hitting target branch
Fixed-Location Mutants Random-Location MutantsWith Mask Without Mask With Mask Without Mask
xmllint 90.3% 22.9% 32.8% 2.9%tcpdump 98.7% 72.8% 36.1% 9.0%c++filt 96.6% 14.8% 34.4% 1.1%readelf 99.7% 78.2% 55.5% 11.4%readpng 97.8% 39.0% 24.0% 2.4%objdump 99.2% 66.7% 46.2% 7.6%