Software Quality Managementpro.unibz.it/staff/mscotto/SQM/SQM1 - Introduction.pdf · 2005-05-30 ·...

Software Quality Management

Software Quality Management

2004-2005Marco Scotto ([email protected])

2Software Quality Management

Course Outline

Introduction to Software QualitySoftware quality frameworks• Product-oriented approaches• Process-oriented approaches

Management topics in Software QualityTestingMeasurement methodologiesSoftware inspection


Logistics

Schedule• Lecture

Wednesday 8.30-10:30Thursday 8.30-10:30

• LabThursday 10:30-12:30

Copies of the slides will be available on Teleacademy web site (http://www.teleacademy.it)


Bibliography

Martin Wieczorek, Dirk Meyerhoff, B. Baltus, Software Quality: State of the Art in Management, Testing, and Tools, Springer-Verlag, 2001N. E. Fenton, S. H. Pfleeger. Software Metrics: a rigorous and practical approach second edition, International Thomson Computer Press, UK, 1997Ron. S. Kenett, Emanuel R. Baker, Software Process Quality, Marcel Dekker, Inc., 1999


Exam

Oral: 40%Project: 60%


Introduction (1/3)

Software presents both an opportunity and a threatIn many applications, software is a critical componentExamples:• Elevators• Airlines• Telecommunications• Medical devices


Introduction (2/3)

From a survey of the Software Engineering Institute (SEI):• 60% of software development organizations

are at the lowest level of capability• The assessment is based on the Capability

Maturity Model (CMM) A framework for achieving process improvement

• The survey involved over 600 organizations worldwide


Introduction (3/3)

The situation is even worse The survey involved organizations that have started improvement efforts or made a commitment to do soIf we add in all the organizations that: • have not assessments performed or • have no plans to implement process improvement

the number of organizations at the lowest level is probably 80%


Software Quality (1/3)

What is the consequence of this low level of process capability in most software companies?• The quality of software!

How many users are really happy with the quality of their software products?• Stories about people losing hours of work

because the computer locked up at an inopportune time are very common



Definition of quality• “Degree of excellence, relative nature or kind

of character, class or grade of thing as determined by this, general excellence" (The New Oxford Illustrated Dictionary)



Software definitions: • Intellectual product consisting of information stored

on a storage device (ISO/DIS 9000: 2000)• Software is "intellectual creation comprising the

programs, procedures, rules and any associated documentation pertaining to the operation of a data processing system"

• A software product is the "complete set of computer programs, procedures and associated documentation and data designated for delivery to a user" (ISO 9000-3)


Why does software have errors?

When executing complex tasks, human beings do commit errors • This cannot be avoidedExperienced programmers commit 1 error in every 10 lines of code (average) About 50% of these errors are corrected during code compilationFurther errors are corrected in testingAbout 15% of the errors are still in the system when delivering it to the customer


Errors and faults in software

And what if the software was (life) critical? How many errors and faults is it acceptable it has? How can we handle errors and faults? What does software quality really mean?


Definitions

Software quality = ???


Definition of Software Quality

Deliverables

ExpectationsQ =


Different focuses over time (1/4)

In the “heroic age of computer science”:

• A program was "good" if it was able to run - at least once-, to terminate running after a time - the length of running-time was not really important -, and to produce results similar to those expected

Around 1960-1975: “micro-efficiency”• A program was "good" if its use was cheap in a certain hardware-

software environment, and it handled the given resources of the computer in an optimal way

Beginning with the ’80’s: ''macro-efficiency''• A "good" computer program is portable, its use can be learned by

common people, and it is highly user-friendly



Nowadays• Correctness, reliability, integrity, interoperability etc.

of the whole system is important

• We usually think of a good system if its documentation is complete and understandableif it interacts with other - computer aided or classic - data-processing systems in an optimal wayif its maintenance can be done at low costsif it is able to cooperate with new software-hardware tools and so on

• Complex organizational frame, well-organized team-work are needed



• Nowadays:

Structured systems' analyzing and developing methods and methodologies have been worked out and complex computer-aided tools have been developed

The use of embedded intelligence

Total Quality Management"to create a system of management procedures that focus on customer satisfaction"



Nowadays:• Software

“drives” airplanes and cars“moves” our money“executes” surgical interventions… interferes with all aspects of our life!

• Bad software/ software with errors: Can kill peopleCan cause financial crashCan lead to worldwide disasters


Importance of Software Quality (1/5)

Several historic disasters attributed to software• 1988 shooting down of Airbus 320 by the USS

VincennesCryptic and misleading output displayed by tracking software

• 1991 patriot missile failure Inaccurate calculation of time due to computer arithmetic errors

• London Ambulance Service Computer Aided Dispatch System

Several deaths• On June 3, 1980, the North American Aerospace

Defense Command (NORAD) reported that the U.S. was under missile attack



Several historic disasters attributed to software• First operational launch attempt of the space

shuttle, whose real-time operating software consists of about 500,000 lines of code, failed

Synchronization problem among its flight-control computers

• 9 hour breakdown of AT&T's long-distance telephone network

Caused by an untested code patch



Ariane 5 crash June 4, 1996• Maiden flight of the European Ariane 5 launcher

crashed about 40 seconds after takeoff• Lost was about half a billion dollars• Explosion was the result of a software error• Uncaught exception due to floating-point error:

conversion from a 64-bit integer to a 16-bit signed integer applied to a larger than expected number

• Module was re-used without proper testing from Ariane 4

Error was not supposed to happen with Ariane 4No exception handler



Mars Climate Orbiter - September 23, 1999• Mars Climate Orbiter, disappeared as it began to orbit

Mars• Cost about $US 125-million• Failure due to error in a transfer of information

between a team in Colorado and a team in CaliforniaOne team used English units (e.g., inches, feet and pounds) while the other used metric units for a key spacecraft operation



Internet viruses and worms• Blaster worm ($US 525 millions)• Sobig.F ($US 500 millions – 1billions)

Exploit well known software vulnerabilities• Software developers do not devote enough effort to

applying lessons learned about the causes of vulnerabilities

• Same types of vulnerabilities continue to be seen in newer versions of products that were in earlier versions.

Usability problems


Garvin’s quality definitions (1984) (1/5)

Transcendent definition

• Quality is absolute and universally recognizable, despite the fact that it cannot be defined precisely

• Only experience can teach to recognize quality



User-based definition

• Quality is "fitness for use”

• This definition starts from the assumption that the individual customers have different needs and those goods that best satisfy their needs are considered to have the highest quality



Value-based definition

• Defines quality in relation to costs

• A quality product provides performance at an acceptable price or conformance at an acceptable cost



The product-based definition • Regards quality as a precise and measurable

variable, as an inherent characteristic of goods



Manufacturing-based definition • Identifies quality as conformity with

specifications• It focuses on the supply side


The elements of software quality (1/5)

• Product• Process• Resource

• Definition• Quality attribute• Metric



Product • Use cases• Design artifacts• Code• ..

Process• Duration• Costs• …

Process• Duration• Costs• …



Software quality attributes are useful to more precisely define customer’s quality requirements Examples• Reliability• Performance• Security• Functionality• Usability• Maintainability• Portability• Reusability• Integrability• Testability



Quality attributes• Attributes may have different measures

describing them from different perspective • Example

Complexity is an internal attribute of products

• There are many complexity measures: to trace the complexity in structure we may consider the cyclomatic complexity of McCabe

• CC cannot track the algorithmic complexity



Quality Attributes• It easier that a a company collects measure of

length and complexity of the code (internal attribute of product) than measure of quality (external attribute) of the product

• Internal attribute: internal characteristics of product, process, and human resources

• External attributes: due to external environment


Measurement

Definition• The process through which we attach

numbers or symbols to the attributes of the real world’s elements, with the scope to describe them according to some, well defined rule (Fenton)


Type of measures

Direct: from a direct process of measuringIndirect: from a mathematical equation in the world of symbolsObjective: the same each time they are taken (e.g. automated collected by some device) Subjective: manually collected by direct human evaluationExample: temperature of coffee - taste of a coffee


The quality framework

Characteristics

Process

Product

Resource

Definition

Qualityattribute

Metric

Objects


Different approaches towards software quality

ISO 9126(Boehm, McCall)...

Process

Product

Resource

Definition

Qualityattribute

Metric

Objects

CharacteristicsGQM

ISO 9001:2000CMM, SPICE

CMMIISO 15504TSP, PSP

Methodologies (RUP, Prince...), Life-cycle support

P-CMM, Weinberg...


What does software quality definition depend on?

…the personality / value-perception / point of view of the evaluator …the type of software production …the development life cycle

…the type of the system / the application domain of the software …business policy


Quality framework, objects and perspectives

objectperspectives

Product Process Resource

user user satisfaction based

participation based

interaction based

engineer system based design based application based

project manager project based control based allocation based


Quality – as depending on the life cycle

Prod

uct r

evisi

on Product transition

Product operation

maintainabilityflexibilitytestability

portabilityreusabilityinteroperability

correctnessreliabilityusability

efficiencyintegrity

(McCall)


Quality as depending on the evaluator (1/4)

User• Software does what I wish

it to do • Quick response time • Ease of use • Should be ready as soon as

possible• Able to follow business

changes• Able to handle erroneous

data• …

Developer• Well specified• Technically correct • Easy to maintain• Easy to develop • Stable• Receives adequate data• ...



User is interested in:• Using the software, its performance, effects of

using the software

User not interested in the internal aspects of software or how it was developedFunctionality, reliability, efficiency, usability, portability



Developer is interested in:• Intermediate product quality and final product

quality• Quality characteristics and metrics might

differ according to the phases of the life cycle• Maintainability• Final product quality characteristics have to

be agreed with the user



Manager is interested in:• Overall quality• Not a specific quality characteristic• Quality improvement with management

criteria (schedule delay, cost overrun)• Will need to assign weights, reflecting

business requirements, to the individual characteristics


Quality according to software application domain

Spaceshuttle orairplaneSatellite

Detectionand alarm

ManagementsystemTest bench

corr

ectn

ess

relia

bilit

y

effic

ienc

y

inte

grity

usab

ility

mai

ntai

nabi

y

flexi

bilit

y

test

abili

ty

porta

bilit

y

reus

abili

ty

inte

rope

rabi

.


Software quality and business policy

Rapid developmentHonoring schedule constraints100% conformity to the specificationComplete coverage during testing...


Total Quality Management (TQM) (1/2)

It represents a style of management aimed at achieving long-term success by linking quality and customer satisfaction

The approach include the creation of a culture in which all members of the organization participate in the improvement of processes, products, and services


Total Quality Management (TQM) (2/2)

Despite variations in its implementation, the key elements of a TQM system can be summarized as follows:• Customer Focus• Process• Human side of quality• Measurement and analysis


TQM Key Elements (1/2)Customer focus• The objective is to achieve total customer satisfaction• Customer focus includes studying customers’ wants

and needs, gathering customer’ requirements, and measuring and managing customers’ satisfaction

Process• The objective is to reduce process variations and to

achieve continuous process improvement• This element includes both the business process and

the product development process• Through process improvement, product quality will be

enhanced


TQM Key Elements (2/2)

Human side of quality• The objective is to create a companywide quality

culture• Focus areas include leadership, management

commitment, total participation, employee empowerment, and other social, psychological, and human factors

Measurement and analysis• The objective is to drive continuous improvement in

all quality parameters by the goal-oriented measurement system


Total Quality Management

Metrics, Models, Measurement, and Analysis

Customer Focus

Process Improvement

Human Side of Quality

Total Quality ManagementContinuous Improvement


Total Quality Management

Various organizational frameworks have been proposed to improve quality Specific examples include:• Plan-Do-Check-Act (PDCA)• Quality Improvement Paradigm/Experience

Factory Organization • Capability Maturity Model (SEI-CMM)• Lean Enterprise Management


Plan-Do-Check-Act (PDCA)

PDCA is based on a feedback cycle for optimizing a single process or product lineCreated by W. Edwards Deming in the 50’s It uses techniques such as feedback loops and statistical quality control • to experiment with methods for improvement• to build predictive models of the product


PDCA BasicsPlan• Plan your improvement, also setting clear numerical

goalsDo• Do your improvement plan, recording the different

stepsCheck• Check what you have done and how closely you have

reached your goalsAct• Act on the basis of the result to get ready to the next

circle of improvement


The circular way

Plan

Check

Do

Act


Quality Improvement Paradigm/Experience Factory Organization (1/2)

It aims at building a continually improving organization based on its evolving goals and assessment of its status relative to those goalsIt uses such techniques as Goal-Question-Metric (GQM)


Quality Improvement Paradigm/Experience Factory Organization (2/2)

The six fundamentals steps are:1. Characterize the project and its environment2. Set the goals3. Choose the appropriate processes4. Execute the processes 5. Analyze the data 6. Package the experience for reuse


Capability Maturity Model (CMM)

The SEI CMM is a staged process improvement, based on assessment of key process areas until you reach level 5, which represents a continuous process improvement


Lean Enterprise Management (1/2)

It is based on the principle of concentration of production on “value-added” activities and the elimination or reduction of “not-value-added”activitiesThe approach has been used to improve factory outputThe goal is to build software with the minimum necessary set of activities and then to tailor the process to the product’s requirements


Lean Enterprise Management (2/2)

The approach uses such concept as:• Technology management• Human-centered management• Decentralized organization• Quality management• Supplier and customer integrationThe implementation of Lean Management in Software Engineering is represented by Agile Methodologies


Summary

Software quality is a complex issueThe need to be familiar with basic concepts and definitions There is no “always good” solution There is no solution “good for everyone”There are many possible approaches • It is not easy to know them all • It is not easy to choose the right one

Software Quality Managementpro.unibz.it/staff/mscotto/SQM/SQM1 - Introduction.pdf · 2005-05-30 ·...

Documents

Transcript of Software Quality Managementpro.unibz.it/staff/mscotto/SQM/SQM1 - Introduction.pdf · 2005-05-30 ·...