Distro Recipes 2013 : Debian and quality assurance
-
Upload
anne-nicolas -
Category
Technology
-
view
2.569 -
download
0
description
Transcript of Distro Recipes 2013 : Debian and quality assurance
![Page 2: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/2.jpg)
Me
I Assistant professor (Maître de conférences) at Univ. de Lorraine� Teach Free Software in a system administration curriculum
(Licence Professionnelle Administration Système, Réseaux etApplications à base de Logiciel Libre)
I Debian involvement:� Collaboration between Debian and Ubuntu
� Ruby packaging
� Archive rebuilds
� Ultimate Debian Database
[email protected] Debian Quality Assurance 2 / 31
![Page 3: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/3.jpg)
Debian
Well known and respected for:
I Its long and bumpy release cycles
I Its quality
But it was not a given:I > 1000 Debian Developers
� Mostly volunteers, with their own agendas
I > 2000 Package maintainers
I 18117 software packages� Some very popular packages� A lot of niche packages
[email protected] Debian Quality Assurance 3 / 31
![Page 4: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/4.jpg)
Debian
Well known and respected for:I Its long and bumpy release cycles
I Its quality
But it was not a given:I > 1000 Debian Developers
� Mostly volunteers, with their own agendas
I > 2000 Package maintainers
I 18117 software packages� Some very popular packages� A lot of niche packages
[email protected] Debian Quality Assurance 3 / 31
![Page 5: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/5.jpg)
Debian
Well known and respected for:I Its long and bumpy release cyclesI Its quality
But it was not a given:I > 1000 Debian Developers
� Mostly volunteers, with their own agendas
I > 2000 Package maintainers
I 18117 software packages� Some very popular packages� A lot of niche packages
[email protected] Debian Quality Assurance 3 / 31
![Page 6: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/6.jpg)
Quality factors
I Culture: "We release when it’s ready"� Many DDs won’t compromise on this
I Packages ownership: individuals or small teams� Many of the maintainers are experts of their packages
I Release Critical (RC) bugs� Prevent packages from being part of a release� Everybody is welcomed to fix them
F Non-Maintainer Uploads
I Debian Quality Assurance team
[email protected] Debian Quality Assurance 4 / 31
![Page 7: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/7.jpg)
Quality factors
I Culture: "We release when it’s ready"� Many DDs won’t compromise on this
I Packages ownership: individuals or small teams� Many of the maintainers are experts of their packages
I Release Critical (RC) bugs� Prevent packages from being part of a release� Everybody is welcomed to fix them
F Non-Maintainer Uploads
I Debian Quality Assurance team
[email protected] Debian Quality Assurance 4 / 31
![Page 8: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/8.jpg)
Quality factors
I Culture: "We release when it’s ready"� Many DDs won’t compromise on this
I Packages ownership: individuals or small teams� Many of the maintainers are experts of their packages
I Release Critical (RC) bugs� Prevent packages from being part of a release� Everybody is welcomed to fix them
F Non-Maintainer Uploads
I Debian Quality Assurance team
[email protected] Debian Quality Assurance 4 / 31
![Page 9: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/9.jpg)
Quality factors
I Culture: "We release when it’s ready"� Many DDs won’t compromise on this
I Packages ownership: individuals or small teams� Many of the maintainers are experts of their packages
I Release Critical (RC) bugs� Prevent packages from being part of a release� Everybody is welcomed to fix them
F Non-Maintainer Uploads
I Debian Quality Assurance team
[email protected] Debian Quality Assurance 4 / 31
![Page 10: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/10.jpg)
Quality factors
I Culture: "We release when it’s ready"� Many DDs won’t compromise on this
I Packages ownership: individuals or small teams� Many of the maintainers are experts of their packages
I Release Critical (RC) bugs� Prevent packages from being part of a release� Everybody is welcomed to fix them
F Non-Maintainer Uploads
I Debian Quality Assurance team
[email protected] Debian Quality Assurance 4 / 31
![Page 11: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/11.jpg)
Debian QA team
I Goal: Improve the quality of Debian as a whole
I Not really a team (no strict membership), more like a central placeto discuss and work on QA
I IRC: #[email protected]
I Mailing list: [email protected]
[email protected] Debian Quality Assurance 5 / 31
![Page 12: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/12.jpg)
Debian QA team: Tasks
I Run archive-wide checks and mass bug filings
I Maintain QA infrastructure
I Take care of the dirty areas of Debian
[email protected] Debian Quality Assurance 6 / 31
![Page 14: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/14.jpg)
Archive-wide checks and MBF
I Developers are volunteers:� Focus on interesting things� Manual testing is boring → not done
I Need automated tests
I Other advantage: treats all packages equallyNiche packages are very important to some users
[email protected] Debian Quality Assurance 8 / 31
![Page 15: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/15.jpg)
Archive rebuilds
I Rebuild all packages from source7 hours on 40 build nodes (using Amazon EC2)
I Detect packages that Fail To Build From Source (FTBFS)� Bugs in packages� Toolchain bugs� Failures caused by changes in other packages
I Ran every 2-3 weeks
I Scripts to analyze failures and file bugs semi-automatically(about 200 bugs filed per hour)
I Also used for test rebuilds with new compilers, linkers, . . .→ Valuable information for upstream
[email protected] Debian Quality Assurance 9 / 31
![Page 16: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/16.jpg)
DebcheckI Check (statically) that dependencies can be satisfied
I Not trivial: versioned dependencies, alternate depends, conflicts
I 2 versions:� "Simple one", in PHP
http://qa.debian.org/debcheck.php
� Better one: edos-debcheck (EDOS project)F Can also work with RPMsF http://edos.debian.net/
[email protected] Debian Quality Assurance 10 / 31
![Page 17: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/17.jpg)
Piuparts
I Testing package installation, upgrade and removal
I Find bugs in maintainer scripts(Shell) scripts executed during package installation/removal
I Developers don’t run it on their systems→ Archive-wide setup
http://piuparts.debian.org
[email protected] Debian Quality Assurance 11 / 31
![Page 18: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/18.jpg)
LintianI Static analysis of packages
I Usually run locally by the developer, before uploading
I Archive-wide setup for continuous testing
I Packages now rejected at upload time for some errors
I Some example findings:� Use of deprecated packaging features� Common packaging mistakes� Embedded copies of libraries� Spelling errors in binaries
usefull visiters treshold targetted superseeded dependanciessplitted maintainance explicitely equiped informations mispelled
http://lintian.debian.org
[email protected] Debian Quality Assurance 12 / 31
![Page 19: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/19.jpg)
DEHS - Debian External Health Status
I Monitor upstream version of packages
I Detect when the Debian package is outdated
I HTML scraping + regular expressionsI Configured in debian/watch
[email protected] Debian Quality Assurance 13 / 31
![Page 20: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/20.jpg)
And more . . .
I Scan build logs for dangerous warnings
I Track security issues
I Track popularity of packages
I . . .
[email protected] Debian Quality Assurance 14 / 31
![Page 22: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/22.jpg)
Infrastructure: mitigating the data hell
I Debian: aggregation of loosely-connected services� No Launchpad!� Everyone can start their own service
I Data everywhereA dozen of places to get interesting data
Tools required to gather data in a central placeand expose it to the maintainers
[email protected] Debian Quality Assurance 16 / 31
![Page 23: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/23.jpg)
Package Tracking System (PTS)
Source package-centric viewhttp://packages.qa.debian.org/package
[email protected] Debian Quality Assurance 17 / 31
![Page 24: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/24.jpg)
Developer Packages Overview (DDPO)
Developer-centric viewhttp://qa.debian.org/developer.php
[email protected] Debian Quality Assurance 18 / 31
![Page 25: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/25.jpg)
Ultimate Debian Database (UDD)
I Gather everything into a single SQL DBI Perfect tool for data-mining Debian
Currently imported in UDD:Debian Sources and Packages, bugs, popcon, testing migrations, upload history,orphaned packages, carnivore, lintian, debtags, translations, NEW queue,screenshots, DEHS, ldap, wanna-build, removals, Ubuntu Sources and Packages,Ubuntu bugs, Ubuntu popcon
Possible questions you can answer with UDD:I RC buggy packages in testing, sorted by popcon?I Packages for which the last 4 uploads were NMUs?I Packages not maintained by official DDs?
http://udd.debian.org/
[email protected] Debian Quality Assurance 19 / 31
![Page 26: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/26.jpg)
Some UDD queries
I Packages with the highest number of different lintian errors orwarnings
select package, count(distinct tag) as cnt
from lintian
where tag_type in(’error’,’warning’)
group by package
order by cnt desc limit 15;
package counttsocks 18
pygopherd 18cyclades-serial-client 17
muddleftpd 17elib 17
[email protected] Debian Quality Assurance 20 / 31
![Page 27: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/27.jpg)
Some UDD queries
I Packages with the highest number of different lintian errors orwarnings
select package, count(distinct tag) as cnt
from lintian
where tag_type in(’error’,’warning’)
group by package
order by cnt desc limit 15;
package counttsocks 18
pygopherd 18cyclades-serial-client 17
muddleftpd 17elib 17
[email protected] Debian Quality Assurance 20 / 31
![Page 28: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/28.jpg)
Some UDD queries
I Who uploaded wheezy’s packages?
select changed_by, count(*) from sources s, upload_history uh
where s.source = uh.source and s.version = uh.version
and s.distribution=’debian’ and s.release = ’wheezy’
group by changed_by order by count desc limit 8;
changed_by countgregor herrmann <[email protected]> 790
Joachim Breitner <[email protected]> 237Christian Perrier <[email protected]> 229Alessio Treglia <[email protected]> 222
Salvatore Bonaccorso <[email protected]> 207Clint Adams <[email protected]> 206
Jonas Smedegaard <[email protected]> 195
[email protected] Debian Quality Assurance 21 / 31
![Page 29: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/29.jpg)
Some UDD queries
I Who uploaded wheezy’s packages?
select changed_by, count(*) from sources s, upload_history uh
where s.source = uh.source and s.version = uh.version
and s.distribution=’debian’ and s.release = ’wheezy’
group by changed_by order by count desc limit 8;
changed_by countgregor herrmann <[email protected]> 790
Joachim Breitner <[email protected]> 237Christian Perrier <[email protected]> 229Alessio Treglia <[email protected]> 222
Salvatore Bonaccorso <[email protected]> 207Clint Adams <[email protected]> 206
Jonas Smedegaard <[email protected]> 195
[email protected] Debian Quality Assurance 21 / 31
![Page 31: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/31.jpg)
Taking care of the dirty areas of Debian
Two complementary approaches:
I Focus on packages neglected by their maintainers
I Focus on maintainers neglecting their packages
[email protected] Debian Quality Assurance 23 / 31
![Page 32: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/32.jpg)
Neglected packages
I Already orphaned:� Do minimalistic maintenance� Problem: > 700 orphaned packages
I Not (yet) orphaned:� Find them, orphan or remove them� Goal: Put maintenance in the hands of people with time
[email protected] Debian Quality Assurance 24 / 31
![Page 33: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/33.jpg)
Bapase: finding neglected packages
I Multi-criteria search for neglected packages
I Based on Ultimate Debian Database
http://udd.debian.org/bapase.cgi
[email protected] Debian Quality Assurance 25 / 31
![Page 34: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/34.jpg)
Inactive maintainers
I Missing In Action (MIA) team
I Find them, track them, orphan their packagesMostly based on reporting by users or fellow developersEchelon: records activity of DDs ; Carnivore: tracks identities
I Collaboration with the Account Managers:possible removal of their account(Unused accounts → possible security problems)
[email protected] Debian Quality Assurance 26 / 31
![Page 38: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/38.jpg)
Following Debian evolutions
Need to improve QA infrastructureI Team-aware
I VCS-aware� Abstraction layer?
If I’ve learned anything, it’s that OSS projects can’t make a choice. Andwhen you can’t make a choice, what do you do? You support all options!Yay! How do we support all options? An abstraction layer! Yay! Abstractionlayers are fun to write! Yay! – – Linux Hater’s Blog
I Standard workflows?� Keep track of sponsorship requests
http://mentors.debian.org + sponsorship-requests
� Track state of packages in VCSF Partial solution: PET (http://pet.debian.net/)
[email protected] Debian Quality Assurance 30 / 31
![Page 39: Distro Recipes 2013 : Debian and quality assurance](https://reader034.fdocuments.in/reader034/viewer/2022042813/54945a53ac79590e2e8b4bb5/html5/thumbnails/39.jpg)
Conclusions
Debian:
I A large-scale volunteer-based distribution
I A continuous effort to organize development and improve quality
[email protected] Debian Quality Assurance 31 / 31