Chapter 2- SQA- Conceptes and Standards _STQA

7/28/2019 Chapter 2- SQA- Conceptes and Standards _STQA

1/23

Chapter-2

Software Quality Assurance: Concepts and Standards

Quality Concepts:The concept of software quality is more complex than what common people tend to

believe. However, it is very popular both for common people and IT professionals. According tothe definition of quality in a dictionary, it is usual to find something like the following: set ofcharacteristics that allows us to rank things as better as or worse than other similar ones. In manycases, dictionaries mention the idea of excellence together with this type of definitions.Certainly, this idea of quality does not help engineers to improve results in the different fields ofactivity. In the world of industrial quality in general, a transition from a rigid concept to anadaptive one was performed many years ago. The concept view tend to be more close to thetraditional idea of beauty: it is in the eyes of the observer. So, we reject absolute concepts andtend to use customer satisfaction as main inspiration.

Software qualityDifferent attempts to define software quality as a complex concept that can be

decomposed in more detailed characteristics have been presented since 1970s. The idea was toenable evaluation of quality through the evaluation of more detailed characteristics that aresupposed to be easy to measure or assess. The problem with this work line is that such featuresare not easy to evaluate in a subjective manner or, even, they are not clearly defined.Standardized quality models based on this idea, e.g. ISO 9126 (ISO, ), are only useful as sourceof ideas to establish an agreement for better understanding between customer and developerbecause it is not clear what kind of metrics are really validated and feasible for a correctmeasurement of each characteristic.

Moreover, the different factors or attributes are not independent. For example, highvalues in portability tend to cause low levels of efficiency. There are different positive(reinforcement) and negative (counteracting) relationships clearly identified among the differentattributes in this type of models. When quality has to be measured, it is important to follow thefoundations of every measurement activity to avoid mistakes. Measurement can be defined asthe process by which numbers or symbols are assigned to attributes of entities in the real worldin such a way as to describe them according to clearly defined rules (Fenton & Pfleeger, 1997).

Attributes can be internal (measurable in terms of the entity itself) or external(measurable only with respect to how the entity relates to its environment). Software entities mayrefer to processes, products or resources. Allocation of numbers and symbols can be done in asubjective (mainly based on opinions) or objective way; it is also possible to distinguish betweendirect (involves no other attribute or entity) and indirect measurement (e.g. defect rate =num.defects/size). What it is extremely important for assuring appropriate measurement activity

is to have a clear idea of what are the expected objectives. So, do we think that IQ is really a realmeasure of intelligence or just an indicator of ability to answer specific tests? In the case ofsoftware, is found defects/time a metric of software quality or is it an indicator of quality oftesting? Is LOC/time a measure of productivity or of speed of coding?

Quality Control:


2/23

Software Quality Control is the function that checks whether the software project followsits standards processes, and procedures, and that the project produces the desired internal andexternal (deliverable) products i.e. output. It is the set of procedures used by organizations toensure that a software product will meet its quality goals at the best value to the customer, and tocontinually improve the organizations ability to produce software products in the future.

Software quality control refers to specified functional requirements as well as non-functionalrequirements such as supportability, performance and usability. It also refers to the ability forsoftware to perform well in unforeseeable scenarios and to keep a relatively low defect rate.These specified procedures and outlined requirements leads to the idea of Verification andValidation and software testing.

It is distinct from software quality assurance which includes audits of the qualitymanagement system against a standard. Whereas software quality control is a control ofproducts, software quality assurance is a control of processes. Quality Control Plan implies toanalyze the actions required to fulfill the project requirements such that the end product meets itsspecifications and product quality is maintained. The characteristics of the Quality Plan are asfollows:

Consistent: The plan should follow the standard and guidelines set by LADOTD designmanuals and AASHTO standard.

Complete: The plan should include the overall representation of the project requirements,features, documentation of the project plan such plan should be developed through all thestages of the project development activity.

Clear: The plan, thus developed should be very clear to the developer and also to thestakeholders regarding project requirements and other project details.

Correct: The project details will be very clear to stakeholders regarding product deliverydate of its postponement or cancellation of the product.

Constructible: The project development plan should be such that if by chance somedesign errors occurs in product development then it should be constructible within smalltime span.

To achieve 5 C's it is recognized that communication between stakeholders and developer is verynecessary. The purpose of Quality Control Plan is to assure that the quality of the product beingdeveloped is maintained throughout the development process. The plan also includes theprocedures which assist in controlling the quality of the product. The main objective of QualityControl Plan is to provide mechanism by which all the plans are executed consistently withoutany design errors. It ensures that the procedures are continuously reviewed by the stakeholdersand the designers. To achieve quality control, a project file document is created where feedbackis given at regular intervals. Periodic review of the feedback results in appropriate changes in thedevelopment process. The basic requirements of Quality Control is to fulfill all the validrequirements of the project . It also requires planning, documentation of the project developmentactivities, constant supervision of designer throughout the development process . It also requirethe developer to ensure that all the project activities are co-ordinated and completed as perschedule and reviews are made periodically.

Project quality control requirementsA Project Quality Control Plan is necessary for each project before starting the project work.


3/23

Project Quality Control Plan: This plan gives the detailed information of methods & process thatprovides the good quality control for all work products. This plan is kept updated with therequirements of project. The plan includes the following parts:

Quality Control Staff: Mainly the QC team contains following members:Engineer of Record (EOR)DesignerTechnical Advisors

Checkers/TestersQuality Assurance Manager

Quality Control ReviewsEvery Project will undergo this review step. The reviewer is an experienced person who is

not an active member of project development team. The different reviews are given below.Bidability, Constructibility and Right of Way ReviewsBidability Reviews: These reviews are initialized by Project Management TeamConstructibility and Right of Way Reviews: These reviews allow input from these

departments, for constructibility reviews and assist in the Right of Way Officein reducing right of way costs.

Checking Procedures Checking Reports

Avoid Redundant data Support for focusing on Major issues Makes data & structures consistent

Checking Drawings Provides the design according to the requirement of project Provides Complete & clear idea of project Provides Coordination with other aspects of the project, i.e., structural, civil,

traffic, right-of-way, etc. Gives Compatible standards and good plans preparation practice

Checking Calculations Checking Correspondence

Resolution of DisputesIn review and checking process, if results are not up to mark, then the checker discuss the

issue with design Engineer & tries to solve the issue, If even though the issue is not resolvedbetween the checker and the Designer, then he goes to a senior technical advisor in order to assistin the resolution of the dispute.

Quality Control Activities Check that assumptions and criteria for the selection of data and the different factors

related to data are documented. Check for transcription errors in data input and reference. Check the integrity of database files. Check for consistency in data. Check that the movement of inventory data among processing steps is correct.


4/23

Check for uncertainties in data, database files etc. Undertake review of internal documentation. Check methodological and data changes resulting in recalculations. Undertake completeness checks. Compare Results to previous Results.

Verification & ValidationVerificationandValidationassures that a software system meets a user's needs.Verification: "Are we building the product right". The software should conform to itsspecification.Validation: "Are we building the right product". The software should do what the user reallyrequires.

Quality Assurance:

The software quality is defined as conformance to explicitly stated functional andperformance requirements, explicitly documented development standards, and implicitcharacteristics that are expected of all professionally developed software.Quality assurance is an essential activity for any business that produces products to be used byothers. Prior to the twentieth century, quality assurance was the sole responsibility of thecraftsperson who built a product. The first formal quality assurance and control function wasintroduced at Bell Labs in 1916 and spread rapidly throughout the manufacturing world. Duringthe 1940s, more formal approaches to quality control were suggested. These relied onmeasurement and continuous process improvement as key elements of quality management.

Today, every company has mechanisms to ensure quality in its products. In fact, explicitstatements of a company's concern for quality have become a marketing ploy during the past fewdecades. The implication for software is that many different constituencies have software qualityassurance responsibility- software engineers, project managers, customers, salespeople, and the

individuals who serve within an SQA group. The SQA group serves as the customer's in-houserepresentative. That is, the people who perform SQA must look at the software from thecustomer's point of view.

SQA Activities:Software quality assurance is composed of a variety of tasks associated with two differentconstituenciesthe software engineers who do technical work and an SQA group that hasresponsibility for quality assurance planning, oversight, record keeping, analysis, and reporting.Software engineers address quality (and perform quality assurance and quality control activities)by applying solid technical methods and measures, conducting formal technical reviews, andperforming well-planned software testing. The charter of the SQA group is to assist the softwareteam in achieving a high quality end product. The Software Engineering Institute recommends aset of SQA activities that address quality assurance planning, oversight, record keeping, analysis,and reporting. These activities are performed (or facilitated) by an independent SQA group that:

Prepares an SQA plan for a project:The plan is developed during project planning andis reviewed by all interested parties. Quality assurance activities performed by thesoftware engineering team and the SQA group are governed by the plan. The planidentifies


5/23

evaluations to be performed audits and reviews to be performed standards that are applicable to the project procedures for error reporting and tracking documents to be produced by the SQA group

amount of feedback provided to the software project team Participates in the development of the projects software process description:The

software team selects a process for the work to be performed. The SQA group reviewsthe process description for compliance with organizational policy, internal softwarestandards, externally imposed standards (e.g., ISO-9001), and other parts of the softwareproject plan.

Reviews software engineering activities to verify compliance with the definedsoftware process: The SQA group identifies, documents, and tracks deviations from theprocess and verifies that corrections have been made.

Audits designated software work products to verify compliance with those definedas part of the software process: The SQA group reviews selected work products;identifies, documents, and tracks deviations; verifies that corrections have been made;and periodically reports the results of its work to the project manager.

Ensures that deviations in software work and work products are documented andhandled according to a documented procedure: Deviations may be encountered in theproject plan, process description, applicable standards, or technical work products.

Records any noncompliance and reports to senior management: Noncomplianceitems are tracked until they are resolved.

Software Reviews:Software reviews are a "filter" for the software engineering process. That is, reviews are applied

at various points during software development and serve to uncover errors and defects that canthen be removed. Software reviews "purify" the software engineering activities that we havecalled analysis, design, and coding. Many different types of reviews can be conducted as part ofsoftware engineering. Each has its place. An informal meeting around the coffee machine is aform of review, if technical problems are discussed. A formal presentation of software designto an audience of customers, management, and technical staff is also a form of review. Theformal technical review, sometimes called a walkthrough or an inspection. A formal technicalreview is the most effective filter from a quality assurance standpoint. Conducted by softwareengineers (and others) for software engineers, the FTR is an effective means for improvingsoftware quality.

Cost Impact of Software DefectsWithin the context of the software process, the terms defect and fault are synonymous. Both

imply a quality problem that is discovered after the software has been released to end-users (or toanother activity in the software process). The term error is earlier used to depict a qualityproblem that is discovered by software engineers (or others) before the software is released to theend-user (or to another activity in the software process). The primary objective of formaltechnical reviews is to find errors during the process so that they do not become defects afterrelease of the software. The obvious benefit of formal technical reviews is the early discovery of


6/23

errors so that they do not propagate to the next step in the software process. By detecting andremoving a large percentage of these errors, the review process substantially reduces the cost ofsubsequent steps in the development and support phases.

Defect Amplification and RemovalA defect amplification model can be used to illustrate the generation and detection of

errors during the preliminary design, detail design, and coding steps of the software engineeringprocess. During the step, errors may be inadvertently generated. Review may fail to uncovernewly generated errors and errors from previous steps, resulting in some number of errors thatare passed through. In some cases, errors passed through from previous steps are amplified. Toconduct reviews, a software engineer must expend time and effort and the developmentorganization must spend money. However, the results of the preceding example leave little doubtthat we can pay now or pay much more later. Formal technical reviews (for design and othertechnical activities) provide a demonstrable cost benefit. They should be conducted.

Formal Technical Reviews:A formal technical review is a software quality assurance activity performed by software

engineers (and others). The objectives of the FTR are:1) To uncover errors in function, logic, or implementation for any representation of the

software.2) To verify that the software under review meets its requirements3) To ensure that the software has been represented according to predefined standards.4) To achieve software that is developed in a uniform manner.5) To make projects more manageable.6) The FTR serves as a training ground, enabling junior engineers to observe different

approaches to software analysis, design, and implementation.7) The FTR also serves to promote backup and continuity because a number of people

become familiar with parts of the software that they may not have otherwise seen.

The FTR is actually a class of reviews that includes walkthroughs, inspections, round-robinreviews and other small group technical assessments of software. Each FTR is conducted as ameeting and will be successful only if it is properly planned, controlled, and attended.

The Review MeetingRegardless of the FTR format that is chosen, every review meeting should abide by the

following constraints: Between three and five people (typically) should be involved in the review. Advance preparation should occur but should require no more than two hours of workfor each person. The duration of the review meeting should be less than two hours.

Given these constraints, it should be obvious that an FTR focuses on a specific (and small) partof the overall software. For example, rather than attempting to review an entire design,walkthroughs are conducted for each component or small group of components. By narrowingfocus, the FTR has a higher likelihood of uncovering errors. The focus of the FTR is on a workproduct (e.g., a portion of a requirements specification, a detailed component design, a sourcecode listing for a component). The individual who has developed the work producttheproducerinforms the project leader that the work product is complete and that a review is


7/23

required. The project leader contacts a review leader, who evaluates the product for readiness,generates copies of product materials, and distributes them to two or three reviewers for advancepreparation. Each reviewer is expected to spend between one and two hours reviewing theproduct, making notes, and otherwise becoming familiar with the work. Concurrently, the reviewleader also reviews the product and establishes an agenda for the review meeting, which is

typically scheduled for the next day. The review meeting is attended by the review leader, allreviewers, and the producer. One of the reviewers takes on the role of the recorder; that is, theindividual who records (in writing) all important issues raised during the review. The FTRbegins with an introduction of the agenda and a brief introduction by the producer. The producerthen proceeds to "walk through" the work product, explaining the material, while reviewers raiseissues based on their advance preparation. When valid problems or errors are discovered, therecorder notes each. At the end of the review, all attendees of the FTR must decide whether to:

1) Accept the product without further modification.2) Reject the product due to severe errors (once corrected, another review must be

performed).3) Accept the product provisionally (minor errors have been encountered and must be

corrected, but no additional review will be required).The decision made, all FTR attendees complete a sign-off, indicating their participation in thereview and their concurrence with the review team's findings.

Review Reporting and Record KeepingDuring the FTR, a reviewer (the recorder) actively records all issues that have been raised.

These are summarized at the end of the review meeting and a review issues list is produced. Inaddition, a formal technical review summary report is completed. A review summary reportanswers three questions:1. What was reviewed?2. Who reviewed it?3. What were the findings and conclusions?The review summary report is a single page form (with possible attachments). It becomes part ofthe project historical record and may be distributed to the project leader and other interestedparties. The review issues list serves two purposes:

(1)to identify problem areas within the product.(2)to serve as an action item checklist that guides the producer as corrections are made. An

issues list is normally attached to the summary report.It is important to establish a follow-up procedure to ensure that items on the issues list have beenproperly corrected. Unless this is done, it is possible that issues raised can fall between thecracks. One approach is to assign the responsibility for follow up to the review leader.

Review GuidelinesGuidelines for the conduct of formal technical reviews must be established in advance,distributed to all reviewers, agreed upon, and then followed. A review that is uncontrolled can

often be worse that no review at all. The following represents a minimum set of guidelines forformal technical reviews:1. Review the product, not the producer. An FTR involves people and egos. Conducted properly,the FTR should leave all participants with a warm feeling of accomplishment. Conductedimproperly, the FTR can take on the aura of an inquisition. Errors should be pointed out gently;the tone of the meeting should be loose and constructive; the intent should not be to embarrass or


8/23

belittle. The review leader should conduct the review meeting to ensure that the proper tone andattitude are maintained and should immediately halt a review that has gotten out of control.2. Set an agenda and maintain it. One of the key maladies of meetings of all types is drift. AnFTR must be kept on track and on schedule. The review leader is chartered with theresponsibility for maintaining the meeting schedule and should not be afraid to nudge people

when drift sets in.3. Limit debate and rebuttal. When an issue is raised by a reviewer, there may not be universalagreement on its impact. Rather than spending time debating the question, the issue should berecorded for further discussion off-line.4. Enunciate problem areas, but don't attempt to solve every problem noted. A review is not aproblem-solving session. The solution of a problem can often be accomplished by the produceralone or with the help of only one other individual. Problem solving should be postponed untilafter the review meeting.5. Take written notes. It is sometimes a good idea for the recorder to make notes on a wall board,so that wording and priorities can be assessed by other reviewers as information is recorded.6. Limit the number of participants and insist upon advance preparation. Two heads are better

than one, but 14 are not necessarily better than 4. Keep the number of people involved to thenecessary minimum. However, all review team members must prepare in advance. Writtencomments should be solicited by the review leader.7. Develop a checklist for each product that is likely to be reviewed. A checklist helps the reviewleader to structure the FTR meeting and helps each reviewer to focus on important issues.Checklists should be developed for analysis, design, code, and even test documents.8. Allocate resources and schedule time for FTRs. For reviews to be effective, they should bescheduled as a task during the software engineering process. In addition, time should bescheduled for the inevitable modifications that will occur as the result of an FTR.9. Conduct meaningful training for all reviewers. To be effective all review participants shouldreceive some formal training. The training should stress both process-related issues and thehuman psychological side of reviews.10. Review your early reviews. Debriefing can be beneficial in uncovering problems with thereview process itself. The very first product to be reviewed should be the review guidelinesthemselves. Because many variables (e.g., number of participants, type of work products, timingand length, specific review approach) have an impact on a successful review, a softwareorganization should experiment to determine what approach works best in a local context.

Software Reliability:Software reliability, unlike many other quality factors, can be measured directed and estimatedusing historical and developmental data. Software reliability is defined in statistical terms as"the probability of failure-free operation of a computer program in a specified

environment for a specified time. To illustrate, program X is estimated to have a reliability of0.96 over eight elapsed processing hours. In other words, if program X were to be executed 100times and require eight hours of elapsed processing time (execution time), it is likely to operatecorrectly (without failure) 96 times out of 100.

Whenever software reliability is discussed, a pivotal question arises: What is meantby the term failure? In the context of any discussion of software quality and reliability, failure isnonconformance to software requirements. Yet, even within this definition, there are gradations.Failures can be only annoying or catastrophic. One failure can be corrected within seconds while


9/23

another requires weeks or even months to correct. Complicating the issue even further, thecorrection of one failure may in fact result in the introduction of other errors that ultimatelyresult in other failures.

Measures of Reliability and AvailabilityEarly work in software reliability attempted to extrapolate the mathematics of hardware

reliability theory to the prediction of software reliability. Most hardware-related reliabilitymodels are predicated on failure due to wear rather than failure due to design defects. Inhardware, failures due to physical wear (e.g., the effects of temperature, corrosion, shock) aremore likely than a design-related failure. Unfortunately, the opposite is true for software. In fact,all software failures can be traced to design or implementation problems; wear does not enterinto the picture. There has been debate over the relationship between key concepts in hardwarereliability and their applicability to software. Although an irrefutable link has yet be beestablished, it is worthwhile to consider a few simple concepts that apply to both systemelements.

If we consider a computer-based system, a simple measure of reliability is meantime-between-failure (MTBF), where

MTBF = MTTF + MTTRThe acronyms MTTF and MTTR are mean-time-to-failure and mean-time-to-repair, respectively.Many researchers argue that MTBF is a far more useful measure than defects/KLOC or

defects/FP. As stated simply, an end-user is concerned with failures, not with the total errorcount because each error contained within a program does not have the same failure rate, Thetotal error count provides little indication of the reliability of a system. For example, consider aprogram that has been in operation for 14 months. Many errors in this program may remainundetected for decades before they are discovered. The MTBF of such obscure errors might be50 or even 100 years. Other errors, as yet undiscovered, might have a failure rate of 18 or 24months. Even if every one of the first category of errors (those with long MTBF) is removed, theimpact on software reliability is negligible. In addition to a reliability measure, we must developa measure of availability.Software availability is the probability that a program is operating according to requirements at agiven point in time and is defined as

Availability = [MTTF/(MTTF + MTTR)] * 100%The MTBF reliability measure is equally sensitive to MTTF and MTTR. The availabilitymeasure is somewhat more sensitive to MTTR, an indirect measure of the maintainability ofsoftware.

Software Safety:When software is used as part of the control system, complexity can increase by an order ofmagnitude or more. Subtle design faults induced by human errorsomething that can beuncovered and eliminated in hardware-based conventional control become much moredifficult to uncover when software is used. Software safety is a software quality assuranceactivity that focuses on the identification and assessment of potential hazards that may affectsoftware negatively and cause an entire system to fail. If hazards can be identified early in thesoftware engineering process, software design features can be specified that will either eliminateor control potential hazards. A modeling and analysis process is conducted as part of softwaresafety. Initially, hazards are identified and categorized by criticality and risk. For example, someof the hazards associated with a computer-based cruise control for an automobile might be


10/23

causes uncontrolled acceleration that cannot be stopped does not respond to depression of brake pedal (by turning off) does not engage when switch is activated slowly loses or gains speed

Once these system-level hazards are identified, analysis techniques are used to assign severity

and probability of occurrence.4 To be effective, software must be analyzed in the context of theentire system. For example, a subtle user input error (people are system components) may bemagnified by a software fault to produce control data that improperly positions a mechanicaldevice. If a set of external environmental conditions are met (and only if they are met), theimproper position of the mechanical device will cause a disastrous failure. Analysis techniquessuch as fault tree analysis, real-time logic or petri net models can be used to predict the chain ofevents that can cause hazards and the probability that each of the events will occur to create thechain.

Once hazards are identified and analyzed, safety-related requirements can be specified forthe software. That is, the specification can contain a list of undesirable events and the desiredsystem responses to these events. The role of software in managing undesirable events is then

indicated. Although software reliability and software safety are closely related to one another, itis important to understand the subtle difference between them. Software reliability uses statisticalanalysis to determine the likelihood that a software failure will occur. However, the occurrenceof a failure does not necessarily result in a hazard or mishap. Software safety examines the waysin which failures result in conditions that can lead to a mishap. That is, failures are notconsidered in a vacuum, but are evaluated in the context of an entire computer-based system.

Quality Assurance Standards:Quality assurance system may be defined as the organizational structure, responsibilities,

procedures, processes, and resources for implementing quality management. Quality assurancesystems are created to help organizations ensure their products and services and satisfying

customer expectations by meeting their specifications. These systems cover a wide variety ofactivities encompassing a products entire life cycle including planning, controlling, measuring,testing and reporting, and improving quality levels throughout the development andmanufacturing process.

ISO 9000ISO 9000 describes quality assurance elements in generic terms that can be applied to any

business regardless of the products or services offered. The ISO 9000 standards have beenadopted by many countries including all members of the European Community, Canada, Mexico,the United States, Australia, New Zealand, and the Pacific Rim. Countries in Latin and SouthAmerica have also shown interest in the standards. After adopting the standards, a countrytypically permits only ISO registered companies to supply goods and services to governmentagencies and public utilities. Telecommunication equipment and medical devices are examplesof product categories that must be supplied by ISO registered companies. In turn, manufacturersof these products often require their suppliers to become registered. Private companies such asautomobile and computer manufacturers frequently require their suppliers to be ISO registered aswell. To become registered to one of the quality assurance system models contained in ISO9000, a companys quality system and operations are scrutinized by third party auditors forcompliance to the standard and for effective operation. Upon successful registration, a companyis issued a certificate from a registration body represented by the auditors. Semi-annual


11/23

surveillance audits ensure continued compliance to the standard. The ISO 9000 quality assurancemodels treat an enterprise as a network of interconnected processes. For a quality system to beISO compliant, these processes must address the areas identified in the standard and must bedocumented and practiced as described. ISO 9000 describes the elements of a quality assurancesystem in general terms. These elements include the organizational structure, procedures,

processes, and resources needed to implement quality planning, quality control, qualityassurance, and quality improvement. However, ISO 9000 does not describe how an organizationshould implement these quality system elements. Consequently, the challenge lies in designingand implementing a quality assurance system that meets the standard and fits the companysproducts, services, and culture.

ISO 9001:2000ISO 9001 is the quality assurance standard that applies to software engineering. The standard

contains 20 requirements that must be present for an effective quality assurance system. Becausethe ISO 9001 standard is applicable to all engineering disciplines, a special set of ISO guidelines(ISO 9000-3) have been developed to help interpret the standard for use in the software process.The requirements delineated by ISO 9001 address topics such as management responsibility,

quality system, contract review, design control, document and data control, product identificationand traceability, process control, inspection and testing, corrective and preventive action, controlof quality records, internal quality audits, training, servicing, and statistical techniques. In orderfor a software organization to become registered to ISO 9001, it must establish policies andprocedures to address each of the requirements just noted (and others) and then be able todemonstrate that these policies and procedures are being followed.

ISO 9126ISO 9126 is an international standard for the evaluation of software. The standard is divided

into four parts which addresses, respectively, the following subjects: quality model external metrics internal metrics quality in use metrics

ISO 9126 Part one, referred to as ISO 9126-1 is an extension of previous work done by McCall(1977), Boehm (1978), FURPS and others in defining a set of software quality characteristics.ISO9126-1 represents the latest (and ongoing) research into characterizing software for thepurposes of software quality control, software quality assurance and software processimprovement (SPI). This article defines the characteristics identified by ISO 9126-1. The otherparts of ISO 9126, concerning metrics or measurements for these characteristics, are essential forSQC, SQA and SPI but the main concern of this article is the definition of the basic ISO 9126Quality Model.

The ISO 9126 documentation itself, from the official ISO 9126 documentation, can only bepurchased and is subject to copyright. SQA.net only reproduces the basic structure of the ISO9126 standard and any descriptions, commentary or guidance are original material based onpublic domain information as well as our own experience.The ISO 9126-1 software quality model identifies 6 main quality characteristics, namely:

Functionality Reliability Usability


12/23

Efficiency Maintainability Portability

These characteristics are broken down into sub-characteristics, a high level table is shown below.

It is at the sub-characteristic level that measurement for SPI will occur. The main characteristicsof the ISO9126-1 quality model, can be defined as follows:-

FunctionalityFunctionality is the essential purpose of any product or service. For certain items this isrelatively easy to define, for example a ship's anchor has the function of holding a ship ata given location. The more functions a product has, e.g. an ATM machine, then the morecomplicated it becomes to define it's functionality. For software a list of functions can bespecified, i.e. a sales order processing systems should be able to record customerinformation so that it can be used to reference a sales order. A sales order system shouldalso provide the following functions:

Record sales order product, price and quantity. Calculate total price.

Calculate appropriate sales tax. Calculate date available to ship, based on inventory. Generate purchase orders when stock falls below a given threshold.

The list goes on and on but the main point to note is that functionality is expressed as atotality of essential functions that the software product provides. It is also important to notethat the presence or absence of these functions in a software product can be verified as eitherexisting or not, in that it is a Boolean (either a yes or no answer). The other softwarecharacteristics listed (i.e. usability) are only present to some degree, i.e. not a simple on oroff. Many people get confused between overall process functionality (in which softwareplays a part) and software functionality. This is partly due to the fact that Data FlowDiagrams (DFDs) and other modeling tools can depict process functionality (as a set of datain\data out conversions) and software functionality. Consider a sales order process, that hasboth manual and software components. A function of the sales order process could be torecord the sales order but we could implement a hard copy filing cabinet for the actual ordersand only use software for calculating the price, tax and ship date. In this way thefunctionality of the software is limited to those calculation functions. SPI, or SoftwareProcess Improvement is different from overall Process Improvement or Process Re-engineering, ISO 9126-1 and other software quality models do not help measure overallProcess costs\benefits but only the software component. The relationship between softwarefunctionality within an overall business process is outside the scope of ISO 9126 and it isonly the software functionality, or essential purpose of the software component, that is ofinterest for ISO 9126.Following functionality, there are 5 other software attributes that characterize the usefulnessof the software in a given environment. Each of the following characteristics can only bemeasured (and are assumed to exist) when the functionality of a given system is present. Inthis way, for example, a system can not possess usability characteristics if the system doesnot function correctly (the two just don't go together).


13/23

ReliabilityOnce a software system is functioning, as specified, and delivered the reliabilitycharacteristic defines the capability of the system to maintain its service provisionunder defined conditions for defined periods of time. One aspect of this characteristicis fault tolerance that is the ability of a system to withstand component failure. For

example if the network goes down for 20 seconds then comes back the system shouldbe able to recover and continue functioning. Usability

Usability only exists with regard to functionality and refers to the ease of use for agiven function. For example a function of an ATM machine is to dispense cash asrequested. Placing common amounts on the screen for selection, i.e. $20.00, $40.00,$100.00 etc, does not impact the function of the ATM but addresses the Usability ofthe function. The ability to learn how to use a system (learn ability) is also a majorsub-characteristic of usability.

EfficiencyThis characteristic is concerned with the system resources used when providing the

required functionality. The amount of disk space, memory, network etc. provides agood indication of this characteristic. As with a number of these characteristics, thereare overlaps. For example the usability of a system is influenced by the system'sPerformance, in that if a system takes 3 hours to respond the system would not beeasy to use although the essential issue is a performance or efficiency characteristic.

MaintainabilityThe ability to identify and fix a fault within a software component is what themaintainability characteristic addresses. In other software quality models thischaracteristic is referenced as supportability. Maintainability is impacted by codereadability or complexity as well as modularization. Anything that helps withidentifying the cause of a fault and then fixing the fault is the concern ofmaintainability. Also the ability to verify (or test) a system, i.e. testability, is one ofthe sub-characteristics of maintainability.

PortabilityThis characteristic refers to how well the software can adopt to changes in itsenvironment or with its requirements. The sub-characteristics of this characteristicinclude adaptability. Object oriented design and implementation practices cancontribute to the extent to which this characteristic is present in a given system.

Quality Factors:Traditionally, a quality product is defined in terms of its fitness of purpose. That is, a

quality product does exactly what the users want it to do. For software products, fitness ofpurpose is usually interpreted in terms of satisfaction of the requirements laid down in the SRSdocument. Although fitness of purpose is a satisfactory definition of quality for many productssuch as a car, a table fan, a grinding machine, etc. for software products, fitness of purpose isnot a wholly satisfactory definition of quality. To give an example, consider a software product


14/23

that is functionally correct. That is, it performs all functions as specified in the SRS document.But, has an almost unusable user interface. Even though it may be functionally correct, wecannot consider it to be a quality product. Another example may be that of a product which doeseverything that the users want but has an almost incomprehensible and unmaintainable code.Therefore, the traditional concept of quality as fitness of purpose for software products is not

wholly satisfactory.The modern view of a quality associates with a software product several quality factors such asthe following:

Portability: A software product is said to be portable, if it can be easily made to work indifferent operating system environments, in different machines, with other softwareproducts, etc. Usability: A software product has good usability, if different categories of users (i.e. bothexpert and novice users) can easily invoke the functions of the product. Reusability: A software product has good reusability, if different modules of the productcan easily be reused to develop new products.

Correctness: A software product is correct, if different requirements as specified in theSRS document have been correctly implemented. Maintainability: A software product is maintainable, if errors can be easily corrected asand when they show up, new functions can be easily added to the product, and thefunctionalities of the product can be easily modified, etc.

CMM:The Capability Maturity Model for Software provides software organizations with guidance

on how to gain control of their processes for developing and maintaining software and how toevolve toward a culture of software engineering and management excellence. The CMM wasdesigned to guide software organizations in selecting process improvement strategies by

determining current process maturity and identifying the few issues most critical to softwarequality and process improvement. By focusing on a limited set of activities and workingaggressively to achieve them, an organization can steadily improve its organization-widesoftware process to enable continuous and lasting gains in software process capability. Thestaged structure of the CMM is based on principles of product quality that have existed for thelast sixty years. In the 1930s, Walter Shewart, promulgated the principles of statistical qualitycontrol. These principles have been adapted by the SEI into a maturity framework thatestablishes a project management and engineering foundation for quantitative control of thesoftware process, which is the basis for continuous process improvement.

The Five Levels of Software Process MaturityOrganizing the CMM into the five levels shown in figure below prioritizes improvementactions for increasing software process maturity.


15/23

Fig: The Five Levels of Software Process Maturity

The following characterizations of the five maturity levels highlight the primary process changes

made at each level:1) Initial: The software process is characterized as ad hoc, and occasionally even chaotic. Fewprocesses are defined, and success depends on individual effort. At the Initial Level, theorganization typically does not provide a stable environment for developing and maintainingsoftware. When an organization lacks sound management practices, the benefits of goodsoftware engineering practices are undermined by ineffective planning and reaction-drivencommitment systems.2) Repeatable: At the Repeatable Level, policies for managing a software project andprocedures to implement those policies are established. Planning and managing new projects isbased on experience with similar projects. An objective in achieving Level 2 is to institutionalizeeffective management processes for software projects, which allow organizations to repeat

successful practices developed on earlier projects, although the specific processes implementedby the projects may differ. An effective process can be characterized as practiced, documented,enforced, trained, measured, and able to improve. Basic project management processes areestablished to track cost, schedule, and functionality. The necessary process discipline is in placeto repeat earlier successes on projects with similar applications.3) Defined: At the Defined Level, the standard process for developing and maintaining softwareacross the organization is documented, including both software engineering and managementprocesses, and these processes are integrated into a coherent whole. This standard process is


16/23

referred to throughout the CMM as the organization's standard software process. Processesestablished at Level 3 are used (and changed, as appropriate) to help the software managers andtechnical staff perform more effectively. The organization exploits effective softwareengineering practices when standardizing its software processes. An organization-wide trainingprogram is implemented to ensure that the staff and managers have the knowledge and skills

required to fulfill their assigned roles. The software process for both management andengineering activities is documented, standardized, and integrated into a standard softwareprocess for the organization. All projects use an approved, tailored version of the organization'sstandard software process for developing and maintaining software.4) Managed: At the Managed Level, the organization sets quantitative quality goals for bothsoftware products and processes. Productivity and quality are measured for important softwareprocess activities across all projects as part of an organizational measurement program. Anorganization-wide software process database is used to collect and analyze the data availablefrom the projects' defined software processes. Software processes are instrumented with well-defined and consistent measurements at Level 4.Detailed measures of the software process andproduct quality are collected. Both the software process and products are quantitatively

understood and controlled.5) Optimizing: At the Optimizing Level, the entire organization is focused on continuousprocess improvement. The organization has the means to identify weaknesses and strengthen theprocess proactively, with the goal of preventing the occurrence of defects. Data on theeffectiveness of the software process is used to perform cost benefit analyses of newtechnologies and proposed changes to the organization's software process. Innovations thatexploit the best software engineering practices are identified and transferred throughout theorganization. Continuous process improvement is enabled by quantitative feedback from theprocess and from piloting innovative ideas and technologies.

TQM:

Total Quality Management (TQM) is a management approach to longterm successthrough customer satisfaction. In a TQM effort, all members of an organization participate inimproving processes, products, services, and the culture in which they work. The methods forimplementing this approach come from the teachings of such quality leaders. Total qualitymanagement can be summarized as a management system for a customer-focused organizationthat involves all employees in continual improvement. It uses strategy, data, and effectivecommunications to integrate the quality discipline into the culture and activities of theorganization.

Customer-focused: The customer ultimately determines the level of quality. No matterwhat an organization does to foster quality improvementtraining employees,integrating quality into the design process, upgrading computers or software, or buying

new measuring toolsthe customer determines whether the efforts were worthwhile.

Total employee involvement: All employees participate in working toward commongoals. Total employee commitment can only be obtained after fear has been driven fromthe workplace, when empowerment has occurred, and management has provided theproper environment. High-performance work systems integrate continuous improvementefforts with normal business operations. Self-managed work teams are one form ofempowerment.


17/23

Process-centered: A fundamental part of TQM is a focus on process thinking. A process

is a series of steps that take inputs from suppliers (internal or external) and transformsthem into outputs that are delivered to customers (again, either internal or external). Thesteps required to carry out the process are defined, and performance measures are

continuously monitored in order to detect unexpected variation.

Integrated system: Although an organization may consist of many different functionalspecialties often organized into vertically structured departments, it is the horizontalprocesses interconnecting these functions that are the focus of TQM.

Micro-processes add up to larger processes, and all processes aggregate into thebusiness processes required for defining and implementing strategy. Everyonemust understand the vision, mission, and guiding principles as well as thequality policies, objectives, and critical processes of the organization. Businessperformance must be monitored and communicated continuously.

An integrated business system may be modeled after the Baldrige NationalQuality Program criteria and/or incorporate the ISO 9000 standards. Everyorganization has a unique work culture, and it is virtually impossible to achieveexcellence in its products and services unless a good quality culture has beenfostered. Thus, an integrated system connects business improvement elements inan attempt to continually improve and exceed the expectations of customers,employees, and other stakeholders.

Strategic and systematic approach: A critical part of the management of quality is thestrategic and systematic approach to achieving an organizations vision, mission, andgoals. This process, called strategic planning or strategic management, includes theformulation of a strategic plan that integrates quality as a core component.

Continual improvement: A major thrust of TQM is continual process improvement.Continual improvement drives an organization to be both analytical and creative infinding ways to become more competitive and more effective at meeting stakeholderexpectations.

Fact-based decision making: In order to know how well an organization is performing,data on performance measures are necessary. TQM requires that an organizationcontinually collect and analyze data in order to improve decision making accuracy,achieve consensus, and allow prediction based on past history.

Communications: During times of organizational change, as well as part of day-to-dayoperation, effective communications plays a large part in maintaining morale and inmotivating employees at all levels. Communications involve strategies, method, andtimeliness.

These elements are considered so essential to TQM that many organizations define them, insome format, as a set of core values and principles on which the organization is to operate.

Six Sigma:


18/23

Six Sigma is a set of tools and strategies for process improvement originally developed byMotorola in 1986. Six Sigma became well known after Jack Welchmade it a central focus of hisbusiness strategy at General Electric in 1995 Six Sigma seeks to improve the quality of processoutputs by identifying and removing the causes of defects (errors) and minimizingvariability in manufacturing and business processes. It uses a set of quality

management methods, including statistical methods, and creates a special infrastructure ofpeople within the organization ("Champions", "Black Belts", "Green Belts", "Orange Belts", etc.)who are experts in these very complex methods. Each Six Sigma project carried out within anorganization follows a defined sequence of steps and has quantified financial targets (costreduction and/or profit increase).The term Six Sigma originated from terminology associated with manufacturing, specificallyterms associated with statistical modeling of manufacturing processes. The maturity of amanufacturing process can be described by a sigma rating indicating its yield or the percentageof defect-free products it creates. A six sigma process is one in which 99.99966% of the productsmanufactured are statistically expected to be free of defects (3.4 defects per million),although, as discussed below, this defect level corresponds to only a 4.5 sigma level. Motorola

set a goal of "six sigma" for all of its manufacturing operations, and this goal became a bywordfor the management and engineering practices used to achieve it. MethodsSix Sigma projects follow two project methodologies inspired by Deming's Plan-Do-Check-

Act Cycle. These methodologies, composed of five phases each, bear the acronyms DMAIC andDMADV.

DMAIC: DMAIC is used for projects aimed at improving an existing businessprocess. DMAIC is pronounced as "duh-may-ick". The DMAIC project methodologyhas five phases:

1. Define the problem, the voice of the customer, and the project goals,specifically.

2.

Measure key aspects of the current process and collect relevant data.3. Analyze the data to investigate and verify cause-and-effect relationships.Determine what the relationships are, and attempt to ensure that all factorshave been considered. Seek out root cause of the defect under investigation.

4. Improve or optimize the current process based upon data analysis usingtechniques such as design of experiments, poka yoke or mistake proofing, andstandard work to create a new, future state process. Set up pilot runs toestablish process capability.

5. Control the future state process to ensure that any deviations from target arecorrected before they result in defects. Implement control systems suchas statistical process control, production boards, visual workplaces, andcontinuously monitor the process. Some organizations add a Recognize step atthe beginning, which is to recognize the right problem to work on, thusyielding an RDMAIC methodology.

DMADV or DFSS: DMADV is used for projects aimed at creating new product orprocess designs. DMADV is pronounced as "duh-mad-vee". The DMADV projectmethodology, known as DFSS ("Design For Six Sigma") features five phases:

1. Define design goals that are consistent with customer demands and theenterprise strategy.


19/23

2. Measure and identify CTQs (characteristics that are Critical To Quality),product capabilities, production process capability, and risks.

3. Analyze to develop and design alternatives, create a high-level design andevaluate design capability to select the best design.

4. Design details, optimize the design, and plan for design verification. Thisphase may require simulations.5. Verify the design, set up pilot runs, implement the production process andhand it over to the process owner(s).

SPICE:ISO/IEC15504 - Process assessment, also known as SPICE (Software Process

Improvement and Capability Determination), is a set of technical standards documents for thecomputer software development process and related business management functions. It isanother joint International Organization for Standardization and International Electro technicalCommission standard. ISO/IEC 15504 initially was derived from process lifecyclestandard ISO/IEC 12207 and from maturity models like Bootstrap, Trillium and the CMM. The

ISO/IEC 15504 is the reference model for the maturity models (consisting of capabilitylevels which in turn consist of the process attributes and further consist of generic practices)against which the assessors can place the evidence that they collect during their assessment, sothat the assessors can give an overall determination of the organization's capabilities fordelivering products (software, systems, and IT services). SPICE approach to capabilityassessment and process improvement (Paulk and Konrad, 1994) is more flexible than the SEImodel. It includes maturity levels comparable with the CMM levels, but also identifiesprocesses, such as customer supplier processes, that cut across these levels. As the level ofmaturity increases, the performance of these cross-cutting processes must also improve.

Reference modelISO/IEC 15504 contains areference model. The reference model defines aprocess

dimensionand acapability dimension.The process dimension in the reference model is not the subject ofpart 2of ISO/IEC 15504, butpart 2 refers to external process lifecycle standards including ISO/IEC 12207 and ISO/IEC15288.The standard defines means to verify conformity of reference models.

ProcessesTheprocess dimensiondefines processes divided into the five process categories of:

customer/supplier engineering supporting management organization

With new parts being published, the process categories will expand, particularly for IT serviceprocess categories and enterprise process categories.

Capability levels and process attributesFor each process, ISO/IEC 15504 defines acapability levelon the following scale:

Level Name

5 Optimizing process


20/23

4 Predictable process

3 Established process

2 Managed process

1 Performed process

0 Incomplete process

The capability of processes is measured using process attributes. The international standarddefines nine process attributes:

1.1 Process Performance2.1 Performance Management2.2 Work Product Management3.1 Process Definition3.2 Process Deployment4.1 Process Measurement4.2 Process Control5.1 Process Innovation

5.2 ProcessOptimization.Each process attribute consists of one or more generic practices, which are further elaboratedinto practice indicators to aid assessment performance. Each process attribute is assessed on afour-point (N-P-L-F) rating scale:

Not achieved (0 - 15%)Partially achieved (>15% - 50%)Largely achieved (>50%- 85%)Fully achieved (>85% - 100%).

The rating is based upon evidence collected against the practice indicators, which demonstratefulfillment of the process attribute.

AssessmentsISO/IEC 15504 provides a guide forperforming an assessment.This includes: the assessment process the model for the assessment any tools used in the assessment Assessment processOne of the requirements is to use a conformant assessment method for the assessment

process. The actual method is not specified in the standard although the standard placesrequirements on the method, method developers and assessors using the method. The standardprovides general guidance to assessors and this must be supplemented by undergoing formaltraining and detailed guidance during initial assessments. The assessment process can begeneralized as the following steps:

initiate an assessment (assessment sponsor) select assessor and assessment team plan the assessment, including processes and organizational unit to be assessed (lead

assessor and assessment team)

pre-assessment briefing data collection data validation


21/23

process rating reporting the assessment result Assessment modelTheprocess assessment model (PAM)is the detailed model that is used for an actual

assessment. This is an elaboration of the process reference model (PRM) provided by the processlifecycle standards. The process assessment model (PAM) in part 5 is based on the processreference model (PRM) for software: ISO/IEC 12207. The process assessment model in part 6 isbased on the process reference model for systems: ISO/IEC 15288. The standard allows othermodels to be used instead, if they meet ISO/IEC 15504's criteria, which include a definedcommunity of interest and meeting the requirements for content (i.e. process purpose, processoutcomes and assessment indicators).

Tools used in the assessmentThere exist several assessment tools. The simplest comprise paper-based tools that are

manually used. In general, they are laid out to incorporate the assessment model indicators,

including the base practice indicators and generic practice indicators. Assessors write down theassessment results and notes supporting the assessment judgment. There are a limited number ofcomputer based tools that present the indicators and allow users to enter the assessment judgmentand notes in formatted screens, as well as automate the collated assessment result (i.e. theprocess attribute ratings) and creating reports.

Assessor qualifications and competencyFor a successful assessment, theassessormust have a suitable level of the relevant skills and

experience. These skills include: personal qualities such ascommunication skills relevant education and training and experience.

specific skills for particular categories, e.g. management skills for the managementcategory.

In summary, the ISO/IEC 15504 specific training and experience for assessors comprisecompletion of a 5 day lead assessor training course performing at least one assessmentsuccessfully under supervision of a competent lead assessor and performing at least oneassessment successfully as a lead assessor under the supervision of a competent lead assessor.The competent lead assessor defines when the assessment is successfully performed. There existschemes for certifying assessors and guiding lead assessors in making this judgment.

Uses of ISO/IEC 15504ISO/IEC 15504 can be used in twocontexts:

Process improvement: ISO/IEC 15504 can be used to performprocessimprovementwithin a technology organization.Process improvement is alwaysdifficult, and initiatives often fail, so it is important to understand the initial baselinelevel (process capability level), and to assess the situation after an improvementproject. ISO 15504 provides a standard for assessing the organization's capacity todeliver at each of these stages.

Capability determination: An organization consideringoutsourcingsoftwaredevelopment needs to have a good understanding of the capability of potential


22/23

suppliers to deliver. The organization can determine atarget capabilityfor suppliers,based on the organization's needs, and then assess suppliers against a set of targetprocess profiles that specify this target capability. Part 4 of the ISO/IEC 15504specifies the high level requirements and an initiative has been started to create anextended part of the standard covering target process profiles. Target process profiles

are particularly important in contexts where the organization (for example, agovernment department) is required to accept the cheapestqualifyingvendor. Thisalso enables suppliers to identify gaps between their current capability and the levelrequired by a potential customer, and to undertake improvement to achieve thecontract requirements (i.e. become qualified). Work on extending the value ofcapability determination includes a method called Practical Process Profiles - whichuses risk as the determining factor in setting target process profiles. Combining riskand processes promotes improvement with active risk reduction, hence reducing thelikelihood of problems occurring.

Software Quality Assurance Metrics.

This section should identify the metrics that will be used in the phase to measure the quality ofthe software. Metrics are normally defined in the project standards and plans


23/23

Chapter 2- SQA- Conceptes and Standards _STQA

Documents

Transcript of Chapter 2- SQA- Conceptes and Standards _STQA