Tracking The Problem By Aaron Jackson

What’s a Problem?

A suspicious or unwanted behavior in a program

Not all problems are errors as some perceived problems are desired features

A problem’s life cycle begins when a person spots the defect or one of the effects of the defect

Problem Life Cycle

1. The user informs the vendor about the problem.

2. The vendor reproduces the problem.

3. The vendor isolates the problem circumstances.

4. The vendor locates and fixes the defect locally.

5. The vendor delivers the fix to the user.

Vendor Responsibilities

Organize the life cycle

Locate which problems are currently open

Find out which problems are the most severe

Find out if any similar problems occurred in the past

Update the user on any progress made

Problem Report

When a problem arises the user sends the vendor a problem report

A problem report contains the information needed to fix the problem

To fix a problem the vendor must be able to reproduce it

A report should contain all relevant facts needed to reproduce the problem

What To Report

Often times a report is bloated with irrelevant information

This can be avoided by providing only relevant information such as:

The Product Release

The Operating Environment

The System Resources

The Problem History

Description of Expected and Experienced Behavior

A One-line Summary

Product Release

The version number or some possible identifier for the program

Using this information, the vendor can recreate test the same copy of the program as the user

If at all possible try to generalize the problem Does the problem only occur on this version of the program?

Does it occur in previous releases too?

Example: Kodi v15.1 “Isengard” Windows OS

Operating Environment

Provide version information about operating system

Provide information on relevant third party components, as interactions between them and the OS may cause problems

Again Try to generalize Does the problem occur on other operating systems?

Does the problem occur with different components attached?

Does the problem in alternate versions of the operating system?

Example: Windows 10 Pro with HP inkjet 3050b installed

System Resources

Sometimes problems can occur under limited resources

As such, it is preferred to provide information on the system’s: Memory

Remaining Hard Disk Space

CPU Load (when problem occurs)

GPU VRAM size (if relevant)

Example: the system has 8GB of memory, the hard has 63GB of free space, and the CPU (intel i5 4 Core) was at 62% load

Problem History

Provides a description of what needs to be done to reproduce the problem

Typically includes input or configuration files

The problem history is the most important fact in troubleshooting

If the problem cannot be reproduced by the vendor it is unlikely to be fixed

Provide all relevant information

Expected and Experienced Behavior

Expected What should have happened according to the user

“The program should have saved the file”

Experienced The symptoms of the problem

“The program crashed”

One-line Summary

Captures the essence of the problem

Used to decide the severity of the problem

Example: Kodi crashes when opening a .wma file

Managing Problems

Two Methods:

1. Problem List

Lists all open or unresolved problems to date

2. Problem Database

Stores all problem reports

Bugzilla

A problem database that organizes all problem reports for Mozilla

Bugzilla stores problem reports using this classification system Severity

Priority

Identifier

Comments

Notification

Severity

Each Problem is assigned a severity

Blocker: Blocks development

Critical: Crashes, loss of data

Major: Major loss of function

Normal: A “standard” problem

Trivial: Cosmetic problem (typos)

Enhancement: Not a problem but a desired feature

Priority

Each problem is assigned a priority

The higher the priority the sooner the problem is looked at

Priority is typically chosen by management

Prioritizing is an important component of development and problem solving

Identifier

Each new problem receives an identifier (AKA bug number or report number)

The bug number is then used in communication when referring to the said problem

Comments

Any user or developer can attach comments to a problem report

Add information you feel is relevant

Suggest possible steps needed to fix the problem

Notification

Developers and users can attach their email to a problem report

When a change is made to the problem they are notified

The Problem Life Cycle

The Problem Life Cycle

Stages of the life cycle: Unconfirmed: A new problem report was just entered into the

database New: The problem report is valid Assigned: The problem is assigned to a developer Resolved: The problem is resolved and placed under one of the

following Fixed – The problem is fixed Invalid – Is not a problem or contains irrelevant facts Duplicate – The same problem has already been reported WONTFIX – The problem will never be fixed WORKSFORME – The developer could not reproduce the problem

Verified: The problem is fixed Closed: The problem no longer exists in a new release Reopened: If a problem occurs again it is assigned reopened rather

than new

Management Resolutions

Who files problem reports

Who classifies problem reports

Who sets priorities

Who takes care of the problem

Who closes the issue

What is the life cycle of a problem

The Software Change Control Board (SCCB)

Many companies use a SCCB to set priorities

Track problem reports and take care of their handling

Typically consists of developers, testers and configuration managers

Problem Driven Development

Create a single problem in the form of a requirement

Divide that problem into sub problems or sub requirements

Solve all problems in order of severity

The product is ready for shipping when all problems are resolved

Managing Clutter

Duplicates are likely to occur when multiple users have the same problem

This causes the problem database to become cluttered with duplicate reports

Reduce clutter by

Encourage users to use the search feature

Identify duplicates when possible

When the original problem is closed all the duplicates are closed with it

Remove unresolved reports that are no longer applicable

Problem Fixes and Releases

How do you manage your versions in order to track bugs?

When a new version is released mark it with an appropriate tag

This creates version control which allows:

A main version where new features are added

A branch version where only fixes are added to stable versions

Problems and Tests

When testing a program there is a high likelihood that a developer will encounter an error

Failures should not be entered in the problem database as that defeats the purpose of the test

Tests make problem reports obsolete

If a developer can reproduce a bug at will there is no need to place it in the database

Effective Problem Reports

Characteristics of an effective problem report:

Is well structured

Is reproducible

Has a descriptive one-line summary

Is as simple as possible

Is as general as possible

Is neutral and stays with the facts

Bug Tracking Tools

Bugzilla – bug tracker used by Mozilla

http://bugzilla.mozilla.org/

Bug Tracking Tools

PHPBUGTRACKER – Lightweight problem tracking software that is simpler to use than Bugzilla

http://phpbt.sf.net/

Bug Tracking Tools

ISSUETRACKER – A bug tracker that focuses on being user friendly

http://issue-tracker.sourceforge.net/

Bug Tracking Tools

TRAC – Comes with integrated version control, making it easier to track problems across multiple versions

http://trac.edgewall.com/

Bug Tracking Tools

SOURCEFORGE – provides automated project organization

http://sourceforge.net/

Bug Tracking Tools

GFORGE – Similar to SOURCEFORGE it provides project organization. Unlike SOURCEFORGE GFORGE does not host projects

https://gforge.com/gf/

Sources

Zeller, A. (2009). Why Programs Fail: A Guide to Systematic Debugging(2nd ed.). Burlington, MA.

https://www.bugzilla.org/docs/2.18/html/lifecycle.html

https://pixabay.com/en/folder-files-paper-office-document-303891/

https://cdn3.iconfinder.com/data/icons/seo-accessibility-usability-2-1/256/Usability_Evaluation-512.png

http://all-free-download.com/free-vector/download/business_meeting_59027.html

http://cliparts.co/clipart/2352617

http://www.thealgonquinway.ca/images/dictionary/img-word3293.jpg