CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1
description
Transcript of CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1
![Page 1: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/1.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
Lecture #1
![Page 2: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/2.jpg)
Industry facts
Software testing accounts for 50% of pre-release costs,and 70% of post-release costs [Cigital
Corporation]
30-40% of errors detected after deployment are run-time errors [U.C. Berkeley, IBM’s TJ Watson
Lab]
The amount of software in a typical device doubles every 18 months [Reme Bourguignon, VP of Philips
Holland]
Defect densities are stable over the last 20 years : 0.5 - 2.0 sw failures / 1000 lines [Cigital
Corporation]
![Page 3: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/3.jpg)
Critical SW Applications
Critical software applications which have failed :
Mariner 1 NASA 1962Missing ‘-’ in ForTran code Rocket bound for Venus destroyed
Therac 25 Atomic Energy of Canada Ltd 1985-87Data conversion error Radiation therapy machine for cancer
Long Distance Service AT&T 1990A single line of bad code Service outages up to nine hours long
Patriot Missiles U.S. military 1991Endurance errors in tracking system 28 US soldiers killed in barracks
Tax Calculation Program InTuit 1995Incorrect results SW vendor payed tax penalties for users
![Page 4: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/4.jpg)
Introduction
Chapter 1
![Page 5: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/5.jpg)
History
• STEP originally developed out of frustration!
• STEP does not establish rules that must be followed– guidelines
![Page 6: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/6.jpg)
History of testing definitions
![Page 7: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/7.jpg)
Preventive testing
• Philosophy that testing can actually improve the quality of the software tested if it occurs early enough
• The process of writing test cases to test a requirement can identify the flaw in the requirement
![Page 8: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/8.jpg)
Preventive testing
• “Withdraw $200 from an account with $165 in it”
• “Withdraw $168.46 from an account with $200 in it”
![Page 9: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/9.jpg)
Waterfall model
![Page 10: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/10.jpg)
Why is testing so difficult?
• Ambiguous and incorrect requirements
• Tight time schedules
• Many other !!!
![Page 11: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/11.jpg)
STEP
• Introduced in 1985
• Built on IEEE 1008-1987
• Covers a broad activity of software evaluation
![Page 12: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/12.jpg)
Views of testing
![Page 13: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/13.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
End of Lecture
![Page 14: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/14.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
Lecture #2
![Page 15: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/15.jpg)
Introduction
Chapter 1
![Page 16: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/16.jpg)
Views of testing
![Page 17: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/17.jpg)
Elements of STEP
![Page 18: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/18.jpg)
STEP architecture
![Page 19: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/19.jpg)
STEP activities
![Page 20: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/20.jpg)
Activity timing of various levels of test
![Page 21: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/21.jpg)
Activity timing of various levels of test
![Page 22: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/22.jpg)
Parallel Mutually Supportive Development
![Page 23: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/23.jpg)
![Page 24: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/24.jpg)
Roles and responsibilities in STEP
![Page 25: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/25.jpg)
Summary of STEP
![Page 26: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/26.jpg)
Chapter 2
![Page 27: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/27.jpg)
Risk analysis
• No way to guarantee a software system will be perfect
• Small number of projects meet criteria for success
• Most of us realize its impossible to test everything in even the most trivial of systems
• Shortage of trained testers doesn’t help
![Page 28: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/28.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
End of Lecture
![Page 29: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/29.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
Lecture #3
![Page 30: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/30.jpg)
Chapter 2
![Page 31: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/31.jpg)
Risk analysis
• No way to guarantee a software system will be perfect
• Small number of projects meet criteria for success
• Most of us realize its impossible to test everything in even the most trivial of systems
• Shortage of trained testers doesn’t help
![Page 32: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/32.jpg)
Domain of all possible test cases
![Page 33: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/33.jpg)
What is risk?
• “the chance of injury, damage, or loss; a dangerous chance; a hazard”
• Everyone subconsciously performs risk analysis hundred of times a day
• Risk management; risk analysis, avoidance, and control
• IEEE standard defines a section for “Risk and contingencies”
![Page 34: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/34.jpg)
Risk analysis activities
![Page 35: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/35.jpg)
Software risk analysis
• Purpose is to determine what to test, the testing priority, and the depth of testing
• Also determining what not to test
• Should be done by a team of experts from various groups in the organization
![Page 36: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/36.jpg)
Software risk analysis
• Done as early as possible
• First cut after high level requirements
• Results of the analysis should be reviewed periodically because things change
![Page 37: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/37.jpg)
Software riskanalysisprocess
![Page 38: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/38.jpg)
Process
• 1. Form a brainstorming team to increase the number of ideas
• 2. Reduce the list to a workable size
• 3. Compile a list of features
• 4. Assign an indicator of relative likelihood of failure
![Page 39: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/39.jpg)
Likelihood of failure
![Page 40: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/40.jpg)
4. Determine the impact
![Page 41: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/41.jpg)
5. Assign numerical values
• H, M, L
• Is your system safety-critical?
• Add together likelihood of failure and impact of failure
![Page 42: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/42.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
End of Lecture
![Page 43: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/43.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
Lecture #4
![Page 44: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/44.jpg)
Chapter 2
![Page 45: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/45.jpg)
4. Determine the impact
![Page 46: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/46.jpg)
5. Assign numerical values
• H (3), M (2), L (1)
• Is your system safety-critical?
• Add together likelihood of failure and impact of failure
![Page 47: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/47.jpg)
Risk priority
![Page 48: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/48.jpg)
Summed priorities
![Page 49: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/49.jpg)
7. Review/modify the values
• Modify based on additional information from analysis
• Likelihood-of-failure-indicators– Team history– Complexity– Usability– New technology– Defect history
![Page 50: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/50.jpg)
8. Prioritize the features
![Page 51: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/51.jpg)
9. Determine the cut line
![Page 52: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/52.jpg)
10. Consider mitigation
![Page 53: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/53.jpg)
Planning risks and contingencies
• Determine the best contingencies in the event that one of the planning risks occurs
• Schedules are usually ambitious or even impossible
• Planning risks are anything that adversely affects the planned testing effort
![Page 54: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/54.jpg)
Planning risks and contingencies
• Identifying planning risks and contingencies helps you make intelligent, informed decisions
• Possible contingencies– Reduce the scope– Delay implementation– Add resources– Reduce quality processes
![Page 55: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/55.jpg)
Planning risks and contingencies
• Planning risks help us to do the “what if..” and develop contingencies
• The entire testing strategy is planned around the concept of using risk to prioritize the testing effort
![Page 56: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/56.jpg)
Project assumptions
• Project and test plans usually have a section called “assumptions”
• Assumptions become planning risks if they turn out to be false
![Page 57: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/57.jpg)
Summary
• Software risk analysis helps you decide what features and attributes should be tested and helps you assign priorities to these items
• Planning risk analysis helps you decide what to do in the event that an unplanned problem arises
![Page 58: CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1](https://reader036.fdocuments.in/reader036/viewer/2022062423/568143db550346895db06981/html5/thumbnails/58.jpg)
CSE 7314Software Testing and Reliability
Robert Oshana
End of Lecture