Writing acceptable patches: an empirical study of open source project patches
description
Transcript of Writing acceptable patches: an empirical study of open source project patches
![Page 1: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/1.jpg)
Writing Acceptable Patches: An Empirical Study of Open Source Project Patches Yida Tao*, DongGyun Han§ and Sunghun Kim**The Hong Kong University of Science and Technology§KAIST Institute for IT Convergence, Korea Advanced Institute of Science and Technology
![Page 2: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/2.jpg)
Developers submit many patches…but not all of them can be accepted
Code Reviewer
2
![Page 3: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/3.jpg)
Developers submit many patches…but not all of them can be accepted
Patch Reviewer
“IT’S A NO FROM ME!”
3
![Page 4: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/4.jpg)
0%
10%
20%
30%
40%
50%
60%
70%
Firefox(Baysal et al. WCRE’12)
Apache(Rigby and German, 2006)
FLAC,OpenAFS(Weißgerber et al. MSR’08)
Patch rejection & resubmission
56%
40%
60%
4
![Page 5: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/5.jpg)
Patch rejection & resubmission = ? increased interruption and workload
5
![Page 6: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/6.jpg)
Patch rejection & resubmission = increased interruption and workload
6
![Page 7: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/7.jpg)
We hope our patches to be accepted right after the first or early rounds of code review
7
![Page 8: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/8.jpg)
How to write acceptable patches?
8
![Page 9: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/9.jpg)
How to write acceptable patches?
• Learn from “past failures”• Q1: Why are patches rejected?
•Understand the priority• Q2: Which reasons are more decisive to reject a patch?• Q3: Which reasons are more difficult to judge?
•Analyze primary stakeholders• Q4: Do patch reviewers and writers have unanimous
patch evaluation criteria?
9
![Page 10: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/10.jpg)
How to write acceptable patches?
• Learn from “past failures”• Q1: Why are patches rejected?
•Understand the priority• Q2: Which reasons are more decisive to reject a patch?• Q3: Which reasons are more difficult to judge?
•Analyze primary stakeholders• Q4: Do patch reviewers and writers have unanimous
patch evaluation criteria?
10
![Page 11: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/11.jpg)
How to write acceptable patches?
• Learn from “past failures”• Q1: Why are patches rejected?
•Understand the priority• Q2: Which reasons are more decisive to reject a patch?• Q3: Which reasons are more difficult to judge?
•Analyze primary stakeholders• Q4: Do patch reviewers and writers have unanimous
patch evaluation criteria?
11
![Page 12: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/12.jpg)
• Learn from “past failures”• Q1: Why are patches rejected?
•Understand the priority• Q2: Which reasons are more decisive to reject a patch?• Q3: Which reasons are more difficult to judge?
•Analyze primary stakeholders• Q4: Do patch reviewers and writers have unanimous
patch evaluation criteria?
Patch extraction & inspection
Developer survey
Literature12
![Page 13: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/13.jpg)
Patch Extraction & Inspection
13
![Page 14: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/14.jpg)
Extracting Rejected Patches
• Projects• Eclipse and Mozilla
• Bug tracking system• Bugzilla
• Criterion for rejected patch• Patches with explicit review- flag
• Number of patches• 300 randomly inspected
• 162 Eclipse + 138 Mozilla
Patch attachments for Eclipse bug#233156
spell checking performance test (patch)2009-11-23
no flags
performance test (5.69 KB, patch)2009-11-30
no flags
performance test + fix (12.95 KB, patch)2009-12-01
review-
fix (7.63 KB, patch)2010-01-04
review-
performance test (6.53KB, patch)2010-01-04
review+
fix (10.08 KB, patch)2010-01-14
review+
14
![Page 15: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/15.jpg)
Patch Inspection
•Open coding• Review comments of 300 rejected patches
• # open codes: 637
“The patch fixes CCE but introduces a new bug: … Also the parameter should be renamed to …“
Open code:• new bug introduced• bad parameter name
15
![Page 16: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/16.jpg)
Patch Inspection
Open code:“malformed patch”
Open code: “wrong indentation”
Theme:Violating coding styles
16
•Open coding• Review comments of
300 rejected patches
• # open codes: 637
•Axial coding
![Page 17: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/17.jpg)
Patch Inspection
Open code:“malformed patch”
Open code: “wrong indentation”
Theme:Violating coding styles
17
•Open coding• Review comments of
300 rejected patches
• # open codes: 637
•Axial coding• 12 themes why patches
are rejected
![Page 18: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/18.jpg)
Theme Frequency
24.6%
18
Suboptimal solution
![Page 19: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/19.jpg)
Theme Frequency
24.6%
19
Incomplete fix22.3%
![Page 20: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/20.jpg)
Theme Frequency
24.6%
20
22.3%
Including unnecessary changes11.9%
![Page 21: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/21.jpg)
Violating coding styles9.7%
Theme Frequency
24.6%
21
22.3%
11.9%
![Page 22: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/22.jpg)
Bad Naming
Inconsistent, misleadingdocumentationMissing documentation
Test Failure
Introducing New Bugs
Compilation errors
Duplicate code
6.3%
6.1%
5.7%
5.5%
Theme Frequency
22
24.6%
22.3%
11.9%
9.7%
![Page 23: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/23.jpg)
Suboptimal solution, 24.6%
Incomplete fix, 22.3%
Including unnecessary
change, 11.9%Violating coding style guidelines,
9.7%
Bad naming, 6.3%
Inconsistent, misleading
documentation, 6.1%
Missing documentation,
5.7%
Test failures, 5.5%
Introducing new bugs, 4.1%
Compilation errors, 1.9%Duplication, 1.6%
Patch size too large, 0.3%
Developer survey12 patch rejection reasons from manual inspection
• Decisive score of each reason• Difficulty to judge each reason246 / 1785 (14%)
Eclipse and Mozilla developers
23
![Page 24: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/24.jpg)
Q2: How decisive each rejection reason is?
24
![Page 25: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/25.jpg)
Patch reviewers have very low tolerance for patches that fail compilation or tests, or introduce new bugs
25
![Page 26: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/26.jpg)
Inconsistent or misleading documentation is highly unacceptable and way worse than missing documentation
26
![Page 27: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/27.jpg)
Inconsistent or misleading documentation is highly unacceptable and way worse than missing documentation
27
![Page 28: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/28.jpg)
Reviewers reject a large patch not solely because of its size, but mainly because the underlying reasons that induce its large size
28
![Page 29: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/29.jpg)
Reviewers reject a large patch not solely because of its size, but mainly because the underlying reasons that induce its large size
29
![Page 30: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/30.jpg)
Q3: Which patch-rejection reasons are more difficult to judge?
30
![Page 31: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/31.jpg)
Ver
y d
iffi
cult
to
jud
geV
ery
easy
Not decisive Very decisive to reject a patch 31
![Page 32: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/32.jpg)
Test failures
Compile errors
Inconsistent, misleading documentation
Ver
y d
iffi
cult
to
jud
geV
ery
easy
Not decisive Very decisive to reject a patch 32
![Page 33: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/33.jpg)
Ver
y d
iffi
cult
to
jud
geV
ery
easy
Not decisive Very decisive to reject a patch
Unnecessary changes
Duplication
33
![Page 34: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/34.jpg)
Ver
y d
iffi
cult
to
jud
geV
ery
easy
Not decisive Very decisive to reject a patch
Incomplete fixSuboptimal solution
Introduce new bugs
34
![Page 35: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/35.jpg)
Q4: Do patch reviewers and writers have unanimous patch evaluation criteria?
35
![Page 36: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/36.jpg)
Distinguish patch reviewers and patch writers
•Method• Patch writer: write ≥ 1 patch• Patch reviewer: review ≥ 1 patch or both
•Responses• 246 respondents• 131 patch writers, 115 patch reviewers
36
![Page 37: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/37.jpg)
37
2 3 4 5
Compilation errors
Test Failures
Introducing new bugs
Inconsistent Documentation
Suboptimal solution
Incomplete fix
Duplication
Unnecessary changes
Violating coding styles
Bad naming
Missing documentation
Patch size too large
Patch reviewer Patch writer
Avg. Decisive Score
![Page 38: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/38.jpg)
38
2 3 4 5
Compilation errors
Test Failures
Introducing new bugs
Inconsistent Documentation
Suboptimal solution
Incomplete fix
Duplication
Unnecessary changes
Violating coding styles
Bad naming
Missing documentation
Patch size too large
Patch reviewer Patch writer
Avg. Decisive Score
![Page 39: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/39.jpg)
39
2 3 4 5
Compilation errors
Test Failures
Introducing new bugs
Inconsistent Documentation
Suboptimal solution
Incomplete fix
Duplication
Unnecessary changes
Violating coding styles
Bad naming
Missing documentation
Patch size too large
Patch reviewer Patch writer
Avg. Decisive Score
Patch reviewers tend to have stricter criteria on patch quality compared to patch writers
![Page 40: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/40.jpg)
Why are patches rejected?
40
![Page 41: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/41.jpg)
Why are patches rejected?
Patch inspection
41
![Page 42: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/42.jpg)
Why are patches rejected?
Patch inspection Developer survey
+
42
![Page 43: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/43.jpg)
Why are patches rejected?
Patch inspection Developer survey Literature
+ +
43
![Page 44: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/44.jpg)
30 patch rejection reasons in 5 categories
Why are patches rejected?
Patch inspection Developer survey Literature
+ +
44
![Page 45: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/45.jpg)
30 patch rejection reasons in 5 categories
1. Problematic implementation or solution
2. Difficult to read or maintain
3. Lack of communication or trust
4. Deviating from the project focus or scope
5. Affecting the development schedule
45
![Page 46: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/46.jpg)
• Compilation & test failures
• Incomplete
• Suboptimal
• Solution too aggressive or hostile for end users
46
1. Problematic implementation or solution
![Page 47: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/47.jpg)
47
2. Difficult to read or maintain
• Coding style violation
• Inconsistent documentation
• Using deprecated API
• No accompanied test cases
![Page 48: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/48.jpg)
3. Lack of communication or trust• No discussion before patch submission• Patch writers are not responsive
4. Deviating from the project scope or focus• Irrelevant or obsolete
5. Affecting the development schedule• Freeze the progress of other features
48
![Page 49: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/49.jpg)
Please check our paper for more details on patch rejection reasons
49
![Page 50: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/50.jpg)
Summary
Writing acceptable patches
50
![Page 51: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/51.jpg)
Checklist
* Compile error* Test failure* Documentation* Duplication* Unnecessary changes
51
![Page 52: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/52.jpg)
Checklist
* Compile error* Test failure* Documentation* Duplication* Unnecessary changes
52
Communication
* Discussion prior to patch submission
* Timely response to review comments
![Page 53: Writing acceptable patches: an empirical study of open source project patches](https://reader034.fdocuments.in/reader034/viewer/2022052217/559430781a28ab333d8b471b/html5/thumbnails/53.jpg)
Checklist
* Compile error* Test failure* Documentation* Duplication* Unnecessary changes
53
Communication
* Discussion prior to patch submission
* Timely response to review comments
Call for industrial &research efforts
* Introduce new bugs
* Suboptimal solutions
* Incomplete fix