Presentation of Andrew Chalmers in 1st International Antibody Validation Forum 2014
Requirements Validation - Chalmers
Transcript of Requirements Validation - Chalmers
![Page 1: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/1.jpg)
Requirements Validation
Lectures 6, DAT230, Requirements EngineeringRobert Feldt, 2012-09-18
tisdag 18 september 12
![Page 2: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/2.jpg)
Recap from last lecture
tisdag 18 september 12
![Page 3: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/3.jpg)
• Specification to refine/specify reqs and reduce risks
• SRS is primarily a communication device
• Also drives development and is baseline for releases
• Modeling for specific situations and reqs
• Many different specification techniques
• Text, Sequence- and state-based models are key
• Use cases, scenarios also quite common
• Formal approaches less used; user communication harder
• IEEE 830 gives basic and common structure
Recap
tisdag 18 september 12
![Page 4: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/4.jpg)
Specification TechniquesWord doc
Excel doc
Text
DB / Req tool
Interaction- /Sequence-based
Scenario
StoryboardUse case
Stimulus-responsesequence
State-based
State transition diagram
UML state diagram
Decision-based
Decision tables
Decision trees
Quality Requirements
PLanguage
Volere
Probabilistic Quality Patterns
UserInterfaces
UI standards Text
PrototypeSketches
Look’n’feelsamples
Formal
Z
Property-based
CSPVDM
tisdag 18 september 12
![Page 5: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/5.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
tisdag 18 september 12
![Page 6: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/6.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
•What if <70?
tisdag 18 september 12
![Page 7: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/7.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
•What if <70?
•What if >100
tisdag 18 september 12
![Page 8: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/8.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
•What if <70?
•What if >100
•70 and 100 are in C or F?
tisdag 18 september 12
![Page 9: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/9.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
•What if <70?
•What if >100
•70 and 100 are in C or F?
•How does this fit with rest? Conflicts?
tisdag 18 september 12
![Page 10: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/10.jpg)
Why validation?
“If temperature is higher than 70 and less than 100, then output should be 3000 watts”
•What if <70?
•What if >100
•70 and 100 are in C or F?
•How does this fit with rest? Conflicts?
•What is missing?
tisdag 18 september 12
![Page 11: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/11.jpg)
Validation Techniques
tisdag 18 september 12
![Page 12: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/12.jpg)
Req Review
tisdag 18 september 12
![Page 13: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/13.jpg)
The Review Formality Spectrum
Formal
Ad Hoc ReviewFormal / Fagan
Inspection
Peer Desk Check
Pair Programming
Team Review
No rules!
tisdag 18 september 12
![Page 14: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/14.jpg)
The Review Formality Spectrum
Formal
Ad Hoc ReviewFormal / Fagan
Inspection
Peer Desk Check
Pair Programming
Team Review
No rules!
7 StagesRoles
PreparationRecorder
Approval/Not
tisdag 18 september 12
![Page 15: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/15.jpg)
[Wikipedia2011]
Fagan Inspection ProcessIBM: 80-90% of defects found
& 25% resource savings
tisdag 18 september 12
![Page 16: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/16.jpg)
• Test-Case Driven Review
• Tester does review to find reqs that are not testable
• Reading techniques
• Ad hoc (most common, focused on experience)
• Check-list based
• Perspective-based (different stakeholders or user types)
Review/Reading Styles
tisdag 18 september 12
![Page 17: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/17.jpg)
Checklist example
tisdag 18 september 12
![Page 18: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/18.jpg)
Selective Homeworkless Review• Challenges when re-introducing Fagan inspections at IBM:
• Managers: High up-front cost (20-30% of dev time), since everything reviewed => Selective reviewing
• Individuals: Preparations seldom happen, since tight schedules => Homeworkless reviews
• Team meets once a week, fixed day&time, 1-1.5 hours
• Artifact selected just before or at meeting
• Roles: Moderator, Reader, Scribe/Recorder
• Hybrid: No preparation => informal, Roles => formal
• Moderator selects specific review technique
[Farchi2008]
tisdag 18 september 12
![Page 19: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/19.jpg)
Selective Homeworkless Review
tisdag 18 september 12
![Page 20: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/20.jpg)
Selective Homeworkless Review
• Moderator monitors metrics:
• Issues found per reviewer per hour
• If below 2, then stop meeting or use other technique
• Does it work?
• 2.17 +/- 0.34 issues/hour/reviewer (90% confidence level)
• “When compared to other review methodologies that in- clude preparation, our method finds fewer issues overall but more major issues per hour. Our opinion is that people working on their own are more effective in finding low-level syntactic problems, as more eyes are watching more places, but less effective in finding real bugs as the understanding is shallower.”
[Farchi2008]
tisdag 18 september 12
![Page 21: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/21.jpg)
Prototyping
tisdag 18 september 12
![Page 22: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/22.jpg)
Prototyping
tisdag 18 september 12
![Page 23: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/23.jpg)
What do industry use?
4 companies used checklist-based and 2 ad hoc review reading
6 used throwaway prototypes, 2 also evolutionary
tisdag 18 september 12
![Page 24: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/24.jpg)
Who do industry involve in reviews?
tisdag 18 september 12
![Page 25: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/25.jpg)
Pros/Cons of Reviews?
tisdag 18 september 12
![Page 26: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/26.jpg)
Improvements to Reviews?
tisdag 18 september 12
![Page 27: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/27.jpg)
Satisfaction with Prototyping?
tisdag 18 september 12
![Page 28: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/28.jpg)
Comparison of Techniques
tisdag 18 september 12
![Page 29: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/29.jpg)
Standards & Process Reqs
tisdag 18 september 12
![Page 30: Requirements Validation - Chalmers](https://reader035.fdocuments.in/reader035/viewer/2022073123/62e5d1c5d1c17f46481e295b/html5/thumbnails/30.jpg)
Standards & Process Reqs
tisdag 18 september 12