Risk Analysis Best Practices By Gabriel Rodriguez.

Risk Analysis

Best Practices

By

Gabriel Rodriguez

Copyright 2006-2007. MSQAA Federation Chapter.

Agenda

Explanation of Risk Types of Risk Risk Based Testing Traceability Matrix Q&A Reference


Explanation of Risk


Explanation of Risk

Concepts of Risk Test is a method for control. Controls are used to

reduce risk. A risk is a condition that can result in a loss to an

organization. The risk turns into a loss when some event triggers the

risk.


Explanation of Risk

The Risk is turned into a loss by threat.A threat is the trigger that causes the risk to

become a loss.Threats are reduced or eliminated by controls. Control can be identified as anything that tends

to cause the reduction of risk.Inadequate controls represent Vulnerability.


Explanation of Risk

The Risk is turned into a loss by threat.Vulnerability , therefore, can be defined as a flaw

in the system of control that will enable a threat to be exploited.

The process of evaluating risk, threats, controls, an vulnerabilities is frequently called Risk Analysis.


Types of Risk


Types of Risk

The different types of risks are:– Risks Generated by a computer– Testing Risks– Software Risks– Quality Risks (Test Factors)


Types of Risk

Risks Generated by a computer The risks in a computerized environment include both

the risks that would be present in manual processing and some risks that are unique in a computerized environment:– Improper use of technology.– Repetition of errors.– Cascading of errors.– Illogical processing.


Types of Risk

– Inability to translate user needs into technical requirements.

– Inability to control technology.– Incorrect entry data.– Concentration of Data.– Inability to react quickly.– Concentration of responsibilities.– Erroneous or falsified input data.


Types of Risk

– Misuse by authorized end users.– Uncontrolled system access.– Ineffective security practices for the application.– Program Errors.– Operating system flaws.– Communication system failure.


Types of Risk

Testing Risks Primary testing risks include:

– Budget.– Number of qualified test resources.– Test environment.– Tools and procedures.– Sequence and increments of code delivery.


Types of Risk

Other factors that add risk to the project are– Multi-vendor products that must integrate.– Multi-vendor environments.– Developed with multi-vendor tools.– End user expectations driven by products like MS

Word and Excel.


Types of Risk

Software Risks An effective approach to testing is to identify an

evaluate the risks in a computer system. Those risks deemed important to reduce become the

areas for testing. A decision can be made as to how much risk is

acceptable and then a test plan design to achieve that goal.


Types of Risk

Software Risks Types of risk associated with the development and

installation of a computer system are:– Incorrect Results will be produced.– Unauthorized transactions will be accepted by the

system.– Computer file integrity will be lost.


Types of Risk

– Processing cannot be reconstructed.– Continuity of processing will be lost.– Service provided to the user will degrade to an

unacceptable level.– Security of the system will be compromised.


Types of Risk

Software Risks– Processing will be not comply with the

organizational policy or governmental regulation.– Results of the system will be unreliable.– Programs will be un-maintainable.– System will not be portable to the other Hardware

and Software.


Types of Risk

Software Risks– System will not be able to interconnect with other

computers systems.– Performance level will be unacceptable.– System will be difficult to operate.


Types of Risk

Quality Risks (Test Factors) The failure of the project team to address (control)

these factors may result in loss:– Requirements comply with methodology

(Methodology Test Factor).– Functional Specifications defined (Correctness Test

Factor).


Types of Risk

– Usability Specifications Determined (Ease of Use Test Factor).

– Maintenance Specifications Determined (Maintainable Test Factor).

– Portability needs Determined (Portable Test Factor).


Types of Risk

Quality Risks (Test Factors) System Interface Defined (Coupling Test Factor). Performance Criteria Established (Performance Test

Factor). Operational Needs Defined (Ease-of Operations Test

Factor). Tolerances Established (Reliability Test Factor).


Types of Risk

Authorization Rules Defined (Authorization Test Factor).

File Integrity Requirements defined (File Integrity Test Factor).

Reconstruction Requirements Defined (Audit Trail Test Factor).


Types of Risk

Quality Risks (Test Factors) Impact of Failure Defined (continuity-of-Processing

Test Factor). Desired Service Level Defined (Service Level Test

Factor). Access Defined (Security Test Factor).


Risk Based Testing


Risk Based Testing

The key steps to designing and planning a risk based approach are risk identification, risk strategy, risk assessment and incorporation of the risk analysis into test design.

The objective of risk analysis is to identify potential problems that could affect the cost or outcome of the project. How do we define areas of risk? The objective of the strategy is to define and communicate which risks we will mitigate, and how we will mitigate those risks; what are we going to do about it?


Risk Based Testing

Risk Identification– The activity of identifying risk answers these questions:

Is there risk to this test area or feature? How can it be classified?

– Risk identification involves collecting information about the project and classifying it to determine the amount of potential risk in the test phase and in production (in the future).

– The risk could be related to system complexity, new technology or methodology involved that could cause problems, limited business knowledge or poor design and code quality.


Risk Based Testing

Risk Strategy– After you’ve identified and assessed the risks, you’ll work

toward a strategy. These are the contingency plans for possible alternative project activities or the mitigation of prioritized risks. These plans are then used to direct the management of risks during the software testing activities. It is therefore possible to define an appropriate level of testing per test area based on the risk assessment of that feature or area of functionality. This approach also allows for additional testing to be defined for features that are critical or are identified as high risk as a result of testing (due to poor design, quality, documentation, etc.)


Risk Based Testing

Risk Assessment– Assessing risks means determining the effects (including

costs) of potential risks. A risk assessment involves asking questions such as: Is this a risk or not? How serious is the risk? What are the consequences? What is the likelihood of this risk happening?

– The following sections discuss some of the tools available for performing this assessment.

Risk Assessment Checklist


Risk Based Testing

Risk Assessment Checklist– The Risk Assessment Checklist is the highest level of the

documents use for our assessment. The Checklist helps us prioritize and categorize the identified risks so that we can develop an effective and appropriate test strategy. The other listed documents help you determine, evaluate, and answer the checklist. The categories of the Risk Assessment Checklist are as follows: Complexity, New Feature, User Traffic, Bugginess, Customer Impact, Integration, and the Size of Code Change. These categories should be considered when evaluating the information gathered in your analysis.


Traceability Matrix


Traceability Matrix

Traceability Matrix definition from www.wikipedia.com:– In software development, the term traceability refers to the

ability to link requirements back to stakeholders' rationales and forward to corresponding design artifacts, code,and test cases.

– Traceability supports numerous software engineering activities such as change impact analysis, compliance verification of code, regression test selection, and requirements validation.


Traceability Matrix

Traceability Matrix definition from www.wikipedia.com:– It is usually accomplished in the form of a matrix created for

the verification and validation of the project.– Unfortunately the practice of constructing and maintaining a

requirements trace matrix [RTM] can be very arduous and over time the traces tend to erode into an inaccurate state.

– Alternate automated approaches for generating traces using information retrieval methods have been developed


Traceability Matrix

Traceability Matrix definition from http://www.jiludwig.com/Traceability_Matrix_Structure.htmlstate:

– A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement.


Traceability Matrix

Below is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown.

In traceability, the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.


Traceability Matrix

Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning.

The purpose of the traceability matrix is:– To map relevant Software Development Life cycle

Documentation (SDLC). (Requirements, Functional or Design specifications) with test plan (test cases).

Process to generate a Traceability Matrix:– Requirements, Functional or Design documentation must exist.– Test Plan must be created.


Traceability Matrix

Process to generate a Traceability Matrix:– Requirements documents must exist,– Functional Specification document must exist– Design documentation must exist (If applicable).– Test Plan must be created.– Remember that in order to generate an accurate traceability

matrix all the documents stated above must exist. Make sure to update the Traceability matrix as Change Control

requests are generated.


Traceability Matrix

– Recommendation to create a traceability matrix:– Open a Spreadsheet in Excel and create:

4 or 5 columns for:– Requirements– Functional specifications– Design Specifications– Test Plan– (It is not necessary to have the 3 Software Development Lice

Cycle (SDLC) documentation, the traceability matrix can be created with 2 SDLC documentation. E.g., Requirements and Functional Specifications).


Traceability Matrix

The following slide shows an example of a Traceability Matrix. Please have the following points in mind:

– Every organization designs its traceability matrix template to meet its needs.

– It is not a good practice to mix the concepts of a traceability matrix with a test matrix (Test Matrix is discussed in ‘Test Design’ presentation).


Traceability Matrix


Q&A

Any questions…


Reference

CSTE Study Guide 2002 by QAI CSTE Study Guide 2006 by QAI www.wikipedia.com http://www.jiludwig.com/Traceability_Matrix_Structure.htmlstate


Thank you…

Risk Analysis Best Practices By Gabriel Rodriguez.

Documents

Transcript of Risk Analysis Best Practices By Gabriel Rodriguez.