SC44315/1/2014 Semester 2/20131 QUALITY MANAGEMENT STANDARDS SC4431 Selected Topic in Software Quality Improvement Dr. Kwankamol Nongpong Department of Computer Science Assumption University Quality Management Standards Focus on the software quality assurance system Organization Infrastructure Requirements Choice of the methods and tools are left to the organization. Focus on the what of SQA not the how Compliance to quality management standards supports the organizations steady efforts to assure an acceptable quality level for its software products. SC44315/1/2014 Semester 2/20132 Importance of Standards The current trend in software development tenders requires certification of participants according to at least one of the dominant quality management standards. Scope of Quality Management Standards Certification Standards Assessment Standards SC44315/1/2014 Semester 2/20133 Classes of SQA Standards Scope of Certification Standards Determined by the aims of certification Enable a software development organization to demonstrate consistent ability to assure that its software products or maintenance services comply with acceptable quality requirements. Serve as an agreed basis for customer and supplier evaluation of the suppliers quality management system. Support the software development organizations efforts to improve quality management system performance and enhance customer satisfaction through compliance with the standards requirements. SC44315/1/2014 Semester 2/20134 Scope of Assessment Standards Serve software development and maintenance organizations as a tool for self-assessment of their ability to carry out software development projects. Serve as a tool for improvement of development and maintenance processes. The standard indicates directions for process improvements. Help purchasing organizations determine the capabilities of potential suppliers. Guide training of assessors by delineating qualifications and training program curricula. ISO 9001 AND ISO 9000-3 SC44315/1/2014 Semester 2/20135 ISO 9000-3: Guiding Principles Customer focus Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision making Mutually supportive supplier relationships ISO 9000-3: Requirements 22 requirements classified into 5 groups (2001): Quality management system Management responsibility Resource management Product realization Management, analysis and improvement SC44315/1/2014 Semester 2/20136 ISO 9000-3 Certification Verifies that an organizations software development and maintenance processes fully comply with the standards requirements. Organization must Develop the organizations SQA system Implement the organizations SQA system Undergo certification audits ISO 9000-3 Certification Process SC44315/1/2014 Semester 2/20137 CMM What is CMM? Capability Maturity Model Initiated by Carnegie Mellon Universitys Software Engineering Institute (SEI) in 1986 Initial version was released in 1992 SC44315/1/2014 Semester 2/20138 Principles of CMM (1) Application of more elaborate management methods based on quantitative approaches increases the organizations capability to control the quality and improve the productivity of the software development process. The vehicle for enhancement of software development is composed of the five-level capability maturity model. The model enables an organization to evaluate its achievements and determine the efforts needed to reach the next capability level by locating the process areas requiring improvement. Principles of CMM Process areas are generic; they define the what, not the how. This approach enables the model to be applied to a wide range of implementation organizations because: It allows use of any life cycle model It allows use of any design methodology, software development tool and programming language It does not specify any particular documentation standard. SC44315/1/2014 Semester 2/20139 CMM Model Evolution of CMM SE-CMM: Software Engineering CMM T-CMM: Trusted CMM SSE-CMM: System Security Engineering CMM P-CMM: People CMM SA-CMM: Software Acquisition CMM IPD-CMM: Integrated Product Development CMM SC44315/1/2014 Semester 2/201310 CMMI Capability Maturity Model Integration Development of specialized CMM models involved development of different sets of key processes Difficulties in cooperation and coordination among departments that applied different CMM variants in the same organization CMMI solves this problem and conforms to ISO/IEC 15504 standard. CMMI Models (v1.1) CMMI-SE/SW System engineering Software engineering CMMI-SE/SW/IPPD/SS System engineering Software engineering Integrated product/process Supplier sourcing engineering CMMI-SE/SW/IPPD System engineering Software engineering Integrated product/process SC44315/1/2014 Semester 2/201311 CMMI Structure Capability Maturity Level 1: Initial Capability Maturity Level 2: Managed Capability Maturity Level 3: Defined Capability Maturity Level 4: Quantitatively managed Capability Maturity Level 5: Optimizing CMMI Process Areas: Level 1 No process area is required SC44315/1/2014 Semester 2/201312 CMMI Process Areas: Level 2 RM: Requirement Management Analyzes requirements of the projects products and verifies that project plans and products of planned activities conform to requirements PP: Project Planning Plans project activities: resources, schedules and outputs are to achieve approval and commitment of all levels involved in the project PMC: Project Monitoring and Control Performs projects progress control: initiates changes and corrections to solve problems and update plans SAM: Supplier Agreement Management Manages acquisition of products and services from suppliers and subcontractors: contracts, progress control and quality assurance CMMI Process Areas: Level 2 MA: Measurement and Analysis Develops, initiates and completes measurements and analyses required to support management progress control and fulfill other information needs PPQA: Process and Product Quality Assurance Develops, implements and follows up application of quality assurance tools for processes and software products CM: Configuration Management Develops, implements and operates a configuration management system: assures integrity of work products, configuration status accounting and etc. SC44315/1/2014 Semester 2/201313 CMMI Process Area: Level 2 RD: Requirements Development TS: Technical Solution PI: Product Integration VER: Verification VAL: Validation OPF: Organizational Process Focus OPD: Organizational Process Definition OT: Organizational Training IPM: Integrated Project Management CMMI Process Areas: Level 3 IT: Integrated Teaming RM: Risk Management DAR: Decision Analysis and Resolution OEI: Organizational Environment for Integration SC44315/1/2014 Semester 2/201314 CMMI Process Areas: Level 4 OPP: Organizational Process Performance QPM: Quantitative Project Management CMMI Process Areas: Level 5 OID: Organizational Innovation and Deployment CAR: Causal Analysis and Resolution SC44315/1/2014 Semester 2/201315 SUCCESS STORIESLevel 5 AssessmentAchieved Success Story 1: Boeings Space Transportation Systems Software Yielded CMMI Level 5 in 3 years. A substantial shift in defect detection, from 89% late detection by testing to 83% early detection by application of various review methods. Earlier detection of defects caused a 31% decrease of rework efforts. Elimination of defects prior to version release increased from 94% to almost 100%. A 140% increase in general productivity. SC44315/1/2014 Semester 2/201316 Success Story 2: Tata Consultancy Services (TCS) TCSs quality project implemented by a South Asian company (14,000 staff) TCS adopted ISO 9000 standard 2 years (1992-1994) to adapt the procedures and the entire quality management system (QMS) and become certified The company continued onto CMM projects. As TCS was ISO 9000 certified, only a few adaptation is needed to reach CMM level 3 assessment (1996). CMM level 4 assessment was achieved in 1998 for a pilot project of 1000 staff. By 2000, significant number of centers had reached level 5 assessment, others had reached level 4 assessment Success Story 2: Tata Consultancy Services (continued) Companys major efforts was certification of the software quality assurance professionals. Three authorized CMM lead assessors Some 77 internally trained CMM assessors Some 678 certified quality analysts Over 300 quality auditors Among the main benefits listed by the company are the following improvements, achieved during 19962000: Reduction of average percentage of rework from 12% to about 4% Reduction of percentage of project schedule slippage from over 3% to less than 2.5% Increase in overall review effectiveness from 40% to 80% defect detection Decreases of 5% in management efforts and of 24% in change request implementation efforts SC44315/1/2014 Semester 2/201317 Success Story 3: Telcordia Technologies Transition from low quality software development to ISO 9000 certification and CMM level 5 assessment in 6 years (1994 1999). ISO 9000 certified (1994-1996) Level 3 assessment (end of 1996) Level 5 assessment for 8 development units (3,500 engineers) (1999) The main benefits garnered by Telcordia during its six-year quality : A 94% reduction in the field faults (release faults) density Percentage of on-schedule major releases reached 98% Overall customer satisfaction rose from 60% in 1962 and 80% in 1994 to over 95% in 1997. Reports by Gartner Inc. Gartner Inc. in a consulting firm specializing in CMM implementation. SC44315/1/2014 Semester 2/201318 ISO/IEC 15504 Compatibility Levels Level 0: Incomplete Level 1: Performed process Level 2: Managed process Level 3: Established process Level 4: Predictable process Level 5: Optimizing process SC44315/1/2014 Semester 2/201319 ISO/IEC 15504 Assessment Model Achievement Grades Scale SC44315/1/2014 Semester 2/201320 ISO/IEC 15504 Model Processes: CUS CUS. 1: Acquire Software CUS. 2: Manage Customer Needs CUS. 3: Supply Software CUS. 4: Operate Software CUS. 5: Provide Customer Service ISO/IEC 15504 Model Processes: ENG ENG. 1: Develop System Requirements and Design ENG. 2: Develop Software Requirements ENG. 3: Develop Software Design ENG. 4: Implement Software Design ENG. 5: Integrate and Test Software ENG. 6: Integrate and Test System ENG. 7: Maintain System and Software SC44315/1/2014 Semester 2/201321 ISO/IEC 15504 Model Processes: SUP SUP. 1: Develop Documentation SUP. 2: Perform Configuration Management SUP. 3: Perform Quality Assurance SUP. 4: Perform Work Product Verification SUP. 5: Perform Work Product Validation SUP. 6: Perform Joint Reviews SUP. 7: Perform Audits SUP. 8: Perform Problem Resolution ISO/IEC 15504 Model Processes: MAN MAN. 1: Manage the Project MAN. 2: Manage Quality MAN. 3: Manage Risk MAN. 4: Manage Subcontractors SC44315/1/2014 Semester 2/201322 ISO/IEC 15504 Model Processes: ORG ORG. 1: Engineer the Business ORG. 2: Define the Process ORG. 3: Improve the Process ORG. 4: Provide Skilled Human Resources ORG. 5: Provide Software Engineering Infrastructure