30th International Conference on Software Maintenance and Evolution ... - GitHub … ·...
1
Continuous integration in a social-coding world Bogdan Vasilescu @b_vasilescu / Department of Mathematics and Computer Science Bogdan Vasilescu, Stef van Schuylenburg, Jules Wulms, Alexander Serebrenik, Mark G. J. van den Brand Alexander Serebrenik @aserebrenik How are GitHub developers using CI? 30th International Conference on Software Maintenance and Evolution, Victoria, BC, Canada, September 28 – October 3, 2014 #Developers #Commits #Pull Requests 1 10 100 1000 10000 100000 Sample of large and active GH repos * not forks * ≥ 1 year * both commits and pull requests * Java, Python, or Ruby * ≥ 10 changes last month * ≥ 10 contributors rails elasticsearch ruby gradle 223 projects 2. Commits vs. Pull Requests: Which builds succeed more often? Distribution of p−values from the Chi^2 tests 0.01 0.05 0.1 0.5 1 (45 / 84) statistically significant at 95% Pull request builds are more successful! 0.2 0.5 1.0 2.0 5.0 10.0 20.0 Odds Ratios (39 / 45) > 1 3. Not all projects are the same… Initiate CI process Source Control Build Test Report Commit Tests Developers 1. Usage of the Travis-CI service 0 20 40 60 80 Java Python Ruby Programming.language count configured and using (113 / 223) configured but not using (93 / 223) not configured (17 / 223) • Python and Ruby • Older than two years • Not too many contributors Pull request builds are more successful for: • ≥5 successful pull requests • ≥5 failed pull requests • ≥5 successful commits • ≥5 failed requests (84 / 113) ² Prog. language Age (years) Contributors Java Python Ruby < 2 2–4 > 4 < 17 17—33 > 33 # projects 10 34 40 24 42 18 29 27 28 … s.t. χ2 p < 0.05 3 19 23 9 25 11 18 15 12 reject Stouffer H0? ✗ ✓ ✓ ✗ ✓ ✓ ✓ ✓ ✗ % odds ratio > 1 n/a 89 87 n/a 92 82 89 80 n/a
Transcript of 30th International Conference on Software Maintenance and Evolution ... - GitHub … ·...
Continuous integration in a social-coding world
Bogdan Vasilescu @b_vasilescu
/ Department of Mathematics and Computer Science
Bogdan Vasilescu, Stef van Schuylenburg, Jules Wulms, !Alexander Serebrenik, Mark G. J. van den Brand!
Alexander Serebrenik @aserebrenik
How are GitHub developers using CI?
30th International Conference on Software Maintenance and Evolution, Victoria, BC, Canada, September 28 – October 3, 2014
#Developers #Commits #Pull Requests
Characteristics of the GitHub projects in our sample
1
10
100
1000
10000
100000
Sample of large and active GH repos!* not forks * ≥ 1 year * both commits
and pull requests
* Java, Python, or Ruby
* ≥ 10 changes last month
* ≥ 10 contributorsrails
elasticsearch ruby gradle
223 projects
2. Commits vs. Pull Requests: !Which builds succeed more often?!
Distribution of p−values from the Chi^2 tests
0.01
0.05 0.1
0.5 1
(45 / 84) statistically significant at 95%
Pull request builds are more successful!
0.2 0.5 1.0 2.0 5.0 10.0 20.0
Odds Ratios
(39 / 45) > 1
3. Not all projects are the same…!
Initiate CI process
Source Control
Build
TestReport
Commit
Tests
Developers
1. Usage of the Travis-CI service!
0
20
40
60
80
Java Python RubyProgramming.language
count
configured and using (113 / 223)
configured but not using !(93 / 223)
not configured (17 / 223)
• Python and Ruby
• Older than two years
• Not too many contributors
Pull request builds are more successful for:
• ≥5 successful pull requests
• ≥5 failed pull requests • ≥5 successful commits • ≥5 failed requests
(84 / 113)²
Prog. language Age (years) Contributors
Java Python Ruby < 2 2–4 > 4 < 17 17—33 > 33!
# projects 10 34 40 24 42 18 29 27 28
… s.t. χ2 p < 0.05 3 19 23 9 25 11 18 15 12
reject Stouffer H0? ✗ ✓ ✓ ✗ ✓ ✓ ✓ ✓ ✗
% odds ratio > 1 n/a 89 87 n/a 92 82 89 80 n/a
