Software Quality Management Practices

of Open Source Projects

By

Hendrik ( a108099 )Phongsathorn Eakamongul ( a108025 )

Software Quality Improvement ( AT70.9016 )Asian Institute of Technology

2009, 17 June

Outlines

- Software change and release management process & platform - Issue tracking - Release Schedule- Tools - Respository - Testing

- Software change and release management process & platform - Issue tracking - Release Schedule- Tools - Respository - Testing

- Compare Process of 2 projects

Software change and release management process & platform

Issue Tracking

• Ubuntu uses Launchpad (http://launchpad.net) for answer, bug tracking and translations.

• Launchpad is collaboration and hosting platform for software projects. As an integrated tool, it supports community building and daily management software process.

• Steps to report a bug :

• Create an account in launchpad.net

• Report the bug

Issue Tracking

• Another way using Apport, a help programs that can found at Help Menu -> Report a problem.

• It is specific to report a bug on running and corresponding application.

Release Schedule

• Based on time based cycle called Time Based Releases process, rather than feature driven.

• Regular release : every six months both for desktop and server edition. • supported for 18 months with security patches, fixes

for critical bugs that could cause data loss, and extra translations.

Release Schedule

• Enterprise release : every 12-24 months. • 3 years support on the Desktop and five years for

Server.

Release Process

• Beginning a new release

• Preparing the infrastructure, tools, and other pieces of the development process needed.

• announce the release officially and open for package uploads.

• Planning

• Specifying the features will be included in the release.

• some come from strategic priorities of the release, an individual idea of the developer, and the list of existing specifications or IdeaPool.

Release Process

• Merging with upstream• Bringing upstream components either directly into

Ubuntu or via Debian.

• Because of Ubuntu based on Debian, the developer team chooses this approach. The reasons are: 1. Its most effective way to keep up to date with upstream code, and 2. The bug-fixes for Debian often same for Ubuntu.

Release Process

• Feature development. • Focusing on development projects which have been

planned.

• Stabilization (freeze). • During this phase, uploads are sometimes held and

need manual approval.

• This is done to make sure the stable release can be delivered on the final release date.

Release Process

• Milestones. • Creating and testing CD images regularly to mark

milestones in the development.

• Four milestones :1. The first alpha milestone. After two months, when the initial flood of new

upstream version, merges with debian, and first set of new features are landed.

2. Three alpha milestones done every three weeks until freeze phase, to get early testing of the new features.

3. Two more alpha milestones in interval of two weeks for stabilizing the new features and fixing the worst bugs.

4. Beta milestone, three or four weeks before final release.

Release Process

• Finalization. Testing every image to ensure the installation methods works as advertised.

• Stable releases.

Release Roadmap

Repository

• Main respository DVCS (Bazaar).

• Mirror• List of archive mirror :

https://launchpad.net/ubuntu/+archivemirrors

• List of CD mirrors: https://launchpad.net/ubuntu/+cdmirrors

• Iso images for testing : http://cdimage.ubuntu.com/

Testing

• Ubuntu Testing Team • objective is to improve Ubuntu through structured

testing of packages and ISO.

• ISO Testing• it is done a few days before releasing any Ubuntu

CD (including alphas, betas, and release candidate)

• Provide to give instruction and directions for using test tracker.

• Testing Day : a special day for the community to come together to test a specific set of ISO images.

Testing

• Category of test cases for ISO testing

• Applications- desktop-level application tests

• System-system-level tests

• Hardware-hardware support tests

• Install-installation test case

• Plans for ISO testing

• Pre release test program

• Smoke testing, to test all desktop variants

• Laptop testing, to test alpha/beta releases on laptops

Testing

• Automated testing can be done with a desktop testing framework like dogtail or LDTP.

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.

Documentation

A content on this slide is summarized from

http://docs.djangoproject.com/en/dev/internalshttp://code.djangoproject.com/

Software change and release management process & platform

Issue tracking

Reports : useful query for ticket

* Tickets needing some work * Accepted tickets that need patches * Accepted tickets that need documentation * Accepted tickets that need unit tests * Accepted tickets that have patches but need to be improved * Tickets by triage stage * Unreviewed tickets * Tickets needing design decisions * Tickets accepted for inclusion * Tickets ready for checkin * Claimed tickets * Tickets by status * open tickets * Grouped by triage stage * Grouped by component * all tickets (including closed) * Grouped by component * Grouped by resolution

Django ticket

Triage's decision on Obvious bugs (i.e. crashes, incorrect query results, non-compliance with a standard)

ticket doesn’t contain enough detail.

Invalid :* user error* describes something other than Django* isn’t a bug report or feature request(i.e. some new users submit support queries as tickets).

Someday/Maybe state

* An enhancement request that might be considered to add to the framework if an excellent patch is submitted.

* not a high priority.

What ticket has or needs in order to be “ready for checkin” ?

Flags :

• “Has patch” patch is “good”.

• “Patch needs improvement” patch no longer applies cleanly, or the code doesn’t live up to standards.• “Needs documentation”• “Needs tests”

Release Process

A : major version number -- happen infrequently (think “years”, not “months”)* major changes * not necessarily backwards-compatible

B : minor version number -- roughly every six months.* new + improve existing features* backwards compatible changes* may deprecate certain features from previous releases.

* After each previous release ( + suitable cooling-off period of a week or two), timeline for the next release is announced.

C : micro version number -- at least once half-way between minor releases, or more often as needed.

* bug and security fixes* 100% backwards-compatible with the previous micro-release.

Version number

A.B or A.B.C.

Deprecated (in minor release)

If a feature in version A.B is deprecated,

version A.B+1 : use of the feature will raise PendingDeprecationWarning. This warning is silent by default.

version A.B+2 : raise a DeprecationWarning.version A.B+3 : remove the feature entirely.

RC Release & pre-1.0

RC Release

Version Number : A.B alpha/beta/rc N

pre-1.0

* no guarantee of backwards-compatibility until the 1.0 release.

Release Schedule

Phase one: feature proposal * What features to include in the next version. + preliminary work on those

features

Phase two: development * alpha release.

Phase three: bugfixes * beta release about halfway through,

* rc complete with string freeze two weeks before the end of the schedule.

* No new features will be accepted during this time.

Phase one: feature proposal

• feature list at the end of part one,

* “Must-have”: critical features that will delay the release if not finished

* “Maybe” : will be pushed to the next release if not finished

i.e. if these features are done before the 1.1 feature-freeze date (March 20), they'll be included in Django 1.1.

* “Not going to happen”: features explicitly deferred to a later release.i.e.- Rejected features - Not in consideration - Small/just bugs- Rejected procedurally -- proposed features that, s of the close date for proposals, lacked

concrete proposals, proof-of-concept implementations, or details needed to start implementing.

Voting style

invented by Apache and used on Python itself,

+1: "I love the idea and I'm strongly committed to it." +0: "Sounds OK to me." -0: "I'm not thrilled, but I won't stand in the way." -1: "I strongly disagree and would be very unhappy to see the idea turn

into reality."

Phase two: development

• At the end of phase two,

* unfinished “maybe” features : postponed until the next release.

* “must-have” features : extend phase two, and thus postpone the final release. ( Though it shouldn’t happen )

Roadmap : version 1.1 roadmap January 15, 2009

"Major" feature freeze for 1.1; work phase ends, and any major incomplete features will be postponed.

February 23, 2009

Django 1.1 alpha.

March 23, 2009

Django 1.1 beta. Feature freeze; only bug fixes will be allowed after this point.

April 2, 2009 Django 1.1 rc 1. and string freeze

April 13, 2009

Django 1.1 final

Sprints work on 1.1 starting in late December.We'll have four or five sprints between now and March.

Some design work in Sprints.

Supported Version• All bug fixes applied to the trunk also be applied to the last minor release, to be

released as the next micro release.

i.e. - now we are at version 1.1 •

- Bugfixes to the main trunk will also applied to previous Micro-release branch i.e. branches/releases/1.0.X.

* The developer who commits a fix to trunk will be responsible for also applying the fix to the current bug-fix branch.

* maintainer who will work with committers to keep them honest on backporting bug fixes.

• Security fixes will be applied to the current trunk and the previous two minor releases.

“release maintainer”

* appointed for each minor* making sure that bug fixes are applied to both trunk and the maintained micro-release branch.* work with the release manager to decide when to release the micro releases

Example 1 : Supported Version

Consider a moment in time halfway between the release of Django 1.3 and 1.4. At this point in time:

* Features will be added to development trunk, to be released as Django 1.4.

* Bug fixes will be applied to a 1.3.X branch, and released as 1.3.1, 1.3.2, etc.

* Security releases will be applied to trunk, a 1.3.X branch and a 1.2.X branch. Security fixes will trigger the release of 1.3.1, 1.2.1, etc.

Example 2 : Supported Version

Imagine, if you will, a point about halfway between 1.1 and 1.2.

* On trunk, development towards 1.2 proceeds with small additions, bugs fixes, etc. being checked in daily.

* On feature branches, development of major features is done. These branches will be merged into trunk before the end of phase two.

* On the branch “branches/releases/1.1.X”, bug fixes found in the 1.1 release are checked in as needed.

* On the branch “branches/releases/1.0.X”, security fixes are made if needed.

Tools

Respository

• Main Respository : SVN• Mirror : DVCS

DVCS Mirror

bazaarLaunchpad mirrors Django's trunk: https://launchpad.net/django

gitsome of which are used to develop patches that will come to SVN.

* Jannis Leidel's git mirror, updated every 30 minutes : http://github.com/django/django/tree/master* Jacob Kaplan-Moss's experimental git mirror : git://djangoproject.com/django (broken as for 2009-03-11)* Matthias Kestenholz's git repositories for Django and for a selection of Django applications: ** Gitweb with complete list of repositories: http://spinlock.ch/pub/git/ (broken as for 2009-03-11, http protocol works fine as of 2009-

03-18, git protocol is stil broken) ** The django changes are sporadically published on repo.or.cz: http://repo.or.cz/w/django.git/ (as for 2009-03-11, last update is 5

months old) * Other people : ** brosner: git://github.com/brosner/django.git ** Marc Fargas (telenieko): git://www.marcfargas.com/django.git/ ** Alex Gaynor: http://github.com/alex/django/tree/master Mercurial (hg) * Django trunk and Django 1.0.x mirrors on Bitbucket: * SVN2HG Gateway of Django and Active branches, updated hourly: http://hgsvn.trbs.net/django/ * GeoDjango Mercurial: includes gis-newforms (a merge of the gis and newforms-admin branches), example code, and other

geospatial goodies. * A experimental Django 1.0 branch using patches from django.bugfixes: http://joelia.gthc.org/django.devel. This is totally at your

own risk and may not work at all. Updated quite not often, but is somewhat stable. See also: http://joelia.gthc.org/django.bugfixes

See updated version at : http://code.djangoproject.com/wiki/DjangoBranches

Commit access

Full committers

- long history of contributions to Django's codebase / being polite and helpful on the mailing lists.

- desire to dedicate serious time to Django's development.

granted by unanimous approval of all existing full committers, and the decision will err on the side of rejection.

Partial committers

- "domain experts" : contributes a large subframework to Django and wants to continue to maintain it.

- giving a formal vote in questions that involve their subsystems.

unanimous approval of all full committers (+ any other partial committers in the same area).

Request commit access

To request commit access, please contact an existing committer privately. Public requests for commit access are potential flame-war starters, and will be ignored.

Testing : python unittest

• Policy to make sure all tests pass at all times

• tests directory of the Django tarball

The tests cover: * Models and database API (tests/modeltests/). * core Django code (tests/regressiontests) * Contrib apps (django/contrib/<contribapp>/tests)

Compare Process of 2 projects

Software change and release management process & platform

Issue tracking http://launchpad.net trac

Release Schedule - Time Based Releases process

- regular release : * every six months * 18 months support both for for desktop and server edition

- enterprise release (LTS) : * every 12 to 24 months * 3 years support on the Desktop and 5 years for Server

- Time Based Releases process

- minor release every six months * bug fixes support for last minor releases * security support for previous two release

- major release in years

Release Process

- planning : specifying features- merging with upstream ( debian )- feature development- testing * first alpha milestone. After 2 months, when the initial flood of new upstream version, merges with debian, and first set of new features are landed. * Three alpha milestones done every three weeks until freeze phase, to get early testing of the new features. * Two more alpha milestones in interval of 2 weeks for stabilizing the new features and fixing the worst bugs. * Beta milestone, 3 or 4 weeks before final release.- Finalization. Testing every image to ensure the installation methods works as advertised.- Stable releases.

- feature proposal - development alpha release.- bugfixes

* beta release about halfway through, * rc complete with string freeze two weeks before the end of the schedule.

Tools

Respository bazaar SVN

Testing - Ubuntu Testing Team for test packages and ISO

- desktop testing framework like dogtail or LDTP

- sprint from hard-core hacking to improving documentation to fixing small bugs. - python unittest