Requirement & Release Management in context of IFDK reference product.
-
Upload
hector-small -
Category
Documents
-
view
219 -
download
0
Transcript of Requirement & Release Management in context of IFDK reference product.
Requirement & Release Management in context of IFDK reference product
About this course material- This material if for general training for
requirement and release management- Material is more supportive in class
room - Material will be updated during courses- FreeNest Portable Project Platform is
used to demonstrate things only in practice. This is not limiting usage for material for other training environments (I hope )
About material
Why Requirements?Why Requirements?
IDEALIDEAL
Requirements help
Design
Requirements help
Design
REQ-XREQ-X
REQ-YREQ-Y
REQ-ZREQ-Z
REQ-OREQ-O
REQ
REQ
Yläotsikko
Requirement ManagementRequirement Management
Between 40% and 60% of software failures and defects are the result of poor software management and requirements definition. In plain English, this means that about half of the problems encountered could have been avoided by making it clear, from the very beginning, what the customer expected from the respective project. This is to say that the programming was fine and the developers did their job well – only they did a different job from what they were supposed to.
http://www.softwareprojects.org
Read also http://www.projectshrink.com/why-requirements-change-270.html
Yläotsikko
Example of IFDK product concept
IFDK = Internal Flame Drum Kit
Requirement Management
Requirements and ImplementationRequirements and Implementation
Business and customer
Development Team
Business Requirements
Business Requirements
TechnicalRequirements
TechnicalRequirementsNEEDNEED SOLUTIONSOLUTION
Design & ArchitectureImplentation
Design & ArchitectureImplentation
AgreementAgreement
“One of sides should not dominate in design process”“One of sides should not dominate in design process”
Yläotsikko
ImplentationImplentation
Example of product architecture
Yläotsikko
Yläotsikko
Seller
Customer
Developer
Different aspects to product
Hacker
Nature of a good requirement?- One sentence which can be tested
- Sentence has to be understandable, measurable and testable
Yläotsikko
• All team board should have own color palette • Board can be "soft locked" for further changes by user• Board can be also seen as locked mode without edit• Selected board's can be set in full screen slide show
mode • User will be able to upload own board as a background• All backgrounds are available for all team members• There can be max 20 different backgrounds
• Tractor has four wheels• Tractor has one exhaust pipe• Engine of tractor is capable of use flexi fuel• The tractor has a hook for trailer• The tractor shall have a enhanced driving system
• Tractor has four wheels• Tractor has one exhaust pipe• Engine of tractor is capable of use flexi fuel• The tractor has a hook for trailer• The tractor shall have a enhanced driving system
• Simply ask questions!• You can gather requirements from several sources: Users,
Stakeholders, Business and Development team and many others• Requirements are trying to define nature of
feature/system/solution more specific than common written document does. This information is helping development team to design a solution for a need
• There is several common methods to define and gather requirements.
• Traditional Requirement modeling (http://en.wikipedia.org/wiki/Requirements_management)
• RUP/UML based Use Case modeling (http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process)
• Agile XP oriented User Story’s (http://en.wikipedia.org/wiki/Agile_software_development)
• Your task is to figure out a small difference between them
Read more http://en.wikipedia.org/wiki/Requirements_management
How to find requirements?
FunctionalRequirements
FunctionalRequirements
Non-Functional
Requirements
Non-Functional
Requirements
FunctionalityFunctionality
StabilityStability
SecuritySecurity
PerformancePerformance
UsabilityUsability
User Interface Design?User Interface Design?
Feature XFeature X
Traditional Requirement Modelling and FeaturesTraditional Requirement Modelling and Features
..............
Yläotsikko
ScalabilityScalability
StabilityStability
PerformancePerformance
SecuritySecurity
PerformancePerformance
StressStress
UsabiltyUsabilty
Different points of view on systemRequirement Category‘s
Requirements from product development point of viewRequirements from product development point of view
„scalability“„scalability“
„Stability“
„Functionality“
Yläotsikko
• Different requirement levels
Customer/Business/Stake Holder Requirements
Customer/Business/Stake Holder Requirements
System RequirementsSystem Requirements
Design RequirementsDesign Requirements
Component RequirementsComponent Requirements
ImplementationImplementation
ScalabilityScalability
StabilityStability
PerformancePerformance
SecuritySecurity
PerformancePerformance
StressStress
UsabiltyUsabilty
System TestingSystem Testing
Integration TestingIntegration Testing
Unit TestingUnit Testing
Customer/Business/Stake Holder Requirements
Customer/Business/Stake Holder Requirements
System RequirementsSystem Requirements
Design RequirementsDesign Requirements
Component RequirementsComponent
Requirements
Acceptance TestingAcceptance Testing
Why we need requirements from testing point of view?Why we need requirements from testing point of view?
ImplementationImplementation
„Developer's Area“
„Test Engineers Area“
”Traditional Testing Levels”
Functional and non-functional requirementsFunctional and non-functional requirements
FunctionalRequirements
FunctionalRequirements
Non-FunctionalRequirements
Non-FunctionalRequirements
Functional Requirement
"User can select application from ui
by using wheel button”
”Tractor can be driven both directions”
Functional Requirement
"User can select application from ui
by using wheel button”
”Tractor can be driven both directions”
Non-Functional Requirement
"Performance Requirement"
”Tractor Startup should take minimum 10 seconds”
”Usability Requirement”
”User interface should be able to control using simple wheel quide”
”The hook can last max 20Kkg trailer load”
Non-Functional Requirement
"Performance Requirement"
”Tractor Startup should take minimum 10 seconds”
”Usability Requirement”
”User interface should be able to control using simple wheel quide”
”The hook can last max 20Kkg trailer load”
How it works? How fast it is?How stable it is?
Do some googling!!
Create a wiki page!!
AboutUserStory
Do some googling!!
Create a wiki page!!
AboutUserStory
Yläotsikko
Practice: Define traditional requirements for you feature
Practice: Define traditional requirements for you feature
Functional RequirementsFunctional Requirements Non-Functional RequirementsNon-Functional Requirements
Yläotsikko
Use CaseUse Case
USE CASE
Written scenario for action. Also execeptions included
Use Case: Open IFDK Main Application
Actor: IFDK User
Step1: Gadget User touches home button
Step2: UI wakeup initiated (if standby)
Step3: Home screen is activated
Setp4: User browses applications specific icons using wheel button
Step5. Icons are moving on screen left and right
Step6: User selects application by pushing wheel button
Step7: Application starts up <4 seconds
Execptions:
1. If application cannot start there will be note on screen about problem
USE CASE
Written scenario for action. Also execeptions included
Use Case: Open IFDK Main Application
Actor: IFDK User
Step1: Gadget User touches home button
Step2: UI wakeup initiated (if standby)
Step3: Home screen is activated
Setp4: User browses applications specific icons using wheel button
Step5. Icons are moving on screen left and right
Step6: User selects application by pushing wheel button
Step7: Application starts up <4 seconds
Execptions:
1. If application cannot start there will be note on screen about problem
Use Case
A use case in software engineering and systems engineering, is a description of steps or actions between a user (or "actor") and a software system which leads the user towards something useful.[1]
Wikipedia
Yläotsikko
Why Use Cases are used?Why Use Cases are used?
Use cases are used for define functional requirements!
Yläotsikko
Brief History of Use CasesBrief History of Use Cases
• Defined by Ivar Jacobson http://en.wikipedia.org/wiki/Use_case
• Used with UML (Unified Modeling Language) http://en.wikipedia.org/wiki/Unified_Modeling_Language
• RUP (Rational Unified Process) • http://fi.wikipedia.org/wiki/RUP
Yläotsikko
User StoryUser Story
USER STORY Examples
Simple phrase describes a need. This can lead to several other storys!
"As user I would like to open application easily”
As a user I would like to select application from icon list on screen
As a user I would like to configure amount of application icons on screen
"As a user I would like to use wheel for speeding up selection process"
"As a user I would like to initate application fast enough"
"As a tractor driver I would like to have enhanced driving system”
USER STORY Examples
Simple phrase describes a need. This can lead to several other storys!
"As user I would like to open application easily”
As a user I would like to select application from icon list on screen
As a user I would like to configure amount of application icons on screen
"As a user I would like to use wheel for speeding up selection process"
"As a user I would like to initate application fast enough"
"As a tractor driver I would like to have enhanced driving system”
User StoryUser Story
Yläotsikko
RequirementMeasurable
Testable
RequirementMeasurable
Testable
Use Case, User Story, Requirement
USE CASE
Written scenario for action. Also execeptions included
Use Case: Open Application
Actor: Gadget User
Step1: Gadget User touches home button
Step2: UI wakeup initiated (if standby)
Step3: Home screen is activated
Setp4: User browses applications specific icons using wheel button
Step5. Icons are moving on screen left and right
Step6: User selects application by pushing wheel button
Step7: Application starts up <4 seconds
Execptions:1. If application cannot start there will be note on screen about problem
USE CASE
Written scenario for action. Also execeptions included
Use Case: Open Application
Actor: Gadget User
Step1: Gadget User touches home button
Step2: UI wakeup initiated (if standby)
Step3: Home screen is activated
Setp4: User browses applications specific icons using wheel button
Step5. Icons are moving on screen left and right
Step6: User selects application by pushing wheel button
Step7: Application starts up <4 seconds
Execptions:1. If application cannot start there will be note on screen about problem
USER STORY
Simple phrase describes a need. This can lead to several other storys!
"As user I would like to open main menu using left
button"
"As a user I would like to use mouse wheel for
zoom"
"As a user I would like to initate application fast
enough"
USER STORY
Simple phrase describes a need. This can lead to several other storys!
"As user I would like to open main menu using left
button"
"As a user I would like to use mouse wheel for
zoom"
"As a user I would like to initate application fast
enough"
Non Functional Requirement
"Performance Requirement"
"Application Startup should take minimum 4 seconds"
Non Functional Requirement
"Performance Requirement"
"Application Startup should take minimum 4 seconds"
Functional Requirements
"User can select application from ui
by using wheel button"
Functional Requirements
"User can select application from ui
by using wheel button"
Yläotsikko
Practice 1: Drafting some requirements
Yläotsikko
Define 10 requirements in a team for a selected product
• Card Game
Functional Requirement Non-Functional
Requirement
LOAD SECURITY MAINTENANCE
IMPLEMENTATION
Non-Functional Requirement
Non-Functional Requirement
Non-Functional Requirement
Non-Functional RequirementAgile
Epic / Theme
SCALING
FUNCTION
SAFETY
Non-Functional Requirement
RECOVERY?
NON FUNCTIONAL
NON FUNCTIONAL
PERFORMANCE?
Main {….} ?
Stake Holders? Customer / User ? Business Request ?
Use Case NON FUNCTIONAL
PERFORMANCE?
SCALINGUSABILITY
UC
AcceptanceCriteria? RECOVERY?
NON FUNCTIONAL
User Story ?
User Story Addition +
SCENARIO ?
Traditional Feature
?
Yläotsikko
Who?When?How?Cost?
Who?When?How?Cost?
Product Idea! $$$
• Cat Safehouse with web cam support• Cat’s owner can “communicate” with cat using
“skype” alike technology.
CAT HOUSEService
30€/day Food, drinks and internet connection
includedLive Camfeed
Web portal login
BringCat to safe house
GetCat back home
Heading- Sed posuere interdum sem.- Quisque ligula eros ullamcorper quis,
lacinia quis facilisis sed sapien.- Mauris varius diam vitae arcu. Sed arcu
lectus auctor vitae, consectetuer et venenatis eget velit.
- Sed augue orci, lacinia eu tincidunt et eleifend nec lacus.
Yläotsikko
• Product Management
Idea#1Idea#1
Defining a Feature? Defining a Feature?
Customer
I would like to haveInternal Flame Drum Kit
Could you deliver it to us?
I would like to haveInternal Flame Drum Kit
Could you deliver it to us?
Actually We have severalWow features for it here
Actually We have severalWow features for it here
Ok!What's a plan
Ok!What's a plan
Nice looking feature propoals.We have to do some evaluationNice looking feature propoals.
We have to do some evaluation
Idea#2Idea#2
Idea#3Idea#3 Idea#4Idea#4
Idea#1Idea#1-Technology?-Knowledge-Resource-Solution?-Priority?
FeatureFeature
FeatureFeature
Feature is functionality of product/software which can be seen as one module of whole product
Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface
Feature is functionality of product/software which can be seen as one module of whole product
Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface
Do some googling!!
Create a wiki page!!
AboutUserStory
Do some googling!!
Create a wiki page!!
AboutUserStory
Yläotsikko
Is product a combination of features?Is product a combination of features?
Calory CounterCalory Counter
Drum MetronomeDrum Metronome
Table Drum ModeTable Drum Mode
Standby ModeStandby Mode
MIDI SupportMIDI Support
Touch Screen with single tapTouch Screen with single tap
Yläotsikko
Core Software/Platform
Core Software/Platform
Is product a combination of features?Is product a combination of features?
Calory CounterCalory Counter
Drum MetronomeDrum Metronome
Table Drum ModeTable Drum Mode Standby ModeStandby Mode
MIDI SupportMIDI Support
Touch Screen with single tapTouch Screen with single tap
Yläotsikko
Customer/Business Requirements?Customer/Business Requirements?
Calory CounterCalory CounterDrum MetronomeDrum Metronome
Table Drum ModeTable Drum Mode
Simple Training ModeSimple Training Mode
MIDI SupportMIDI Support
Touch Screen with single tapTouch Screen with single tap
Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4
Who are our target customers?
Customer StrategyCustomer Strategy
Drum MetronomeDrum Metronome Table Drum ModeTable Drum Mode
Simple Training ModeSimple Training Mode
MIDI SupportMIDI Support
Touch Screen with single tapTouch Screen with single tap
Customer Type 1 Customer Type 2 Customer Type 3 Customer Type 4
What is our key customer?
Primary Target
Calory CounterCalory Counter
Secondary Target
Requirement
Requirement
Requirement USE CASE #2
USE CASE #1
USE CASE #3
Requirement
Requirement
Requirement
USE CASE #1 User Story #1
User Story #2
User Story #3
RequirementRequirement
RequirementRequirement USE CASE #2USE CASE #2
USE CASE #1
USE CASE #3USE CASE #3
RequirementRequirement USE CASE #1USE CASE #1 User Story #1User Story #1
Requirement
Requirement
Requirement USE CASE #2
USE CASE #1
USE CASE #3
Requirement
Requirement
Requirement
USE CASE #1 User Story #1
User Story #2
User Story #3
RequirementRequirement
RequirementRequirement
RequirementRequirement USE CASE #2USE CASE #2
USE CASE #1
USE CASE #3USE CASE #3
RequirementRequirement
RequirementRequirement
RequirementRequirement
USE CASE #1USE CASE #1 User Story #1User Story #1
User Story #2User Story #2
User Story #3User Story #3
Requirement
Requirement
Requirement USE CASE #2
USE CASE #1
USE CASE #3
Requirement
Requirement
Requirement
USE CASE #1RequirementRequirement
RequirementRequirement USE CASE #2USE CASE #2
USE CASE #1RequirementRequirement
RequirementRequirement
USE CASE #1USE CASE #1
Features and release planningFeatures and release planning
Release 0.1
Release 1.1 Release 1.2
Feature: Simple Training Mode
Feature: Table Drum mode
Feature Touch Screen with single tap
Release 1.0
TIME TO MARKET!! For Target Group 3TIME TO MARKET!! For Target Group 3
CORE/Platform Software Development
TIME TO MARKET!! For Target Group 2TIME TO MARKET!! For Target Group 2
TIME TO MARKET!! For Target Group 1TIME TO MARKET!! For Target Group 1
Practice: Defining FeaturesPractice: Defining Features
Play problem domain card game with team to
search for features?
Yläotsikko
Yläotsikko
Feature X * nFeature X * n
Feature example 1 (Invented on course 2009-2010)Feature example 1 (Invented on course 2009-2010)
Calory Counter:
Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application
Energy usage
Yläotsikko
Feature example 2 (Invented on course 2009-2010)Feature example 2 (Invented on course 2009-2010)
Table Drummer:
Player drums table board instead of drum can. IFDK kit is able to use DSP algorithm to detect correct drum sound from environment.
In training mode IFDK is trained to detect drum sounds for environment.
Table Drum ModeTable Drum Mode
DSPAlgorithm
DSPAlgorithm
Yläotsikko
DefinesDefines
Feature X * nFeature X * n
FunctionalRequirements
FunctionalRequirements
Non-FunctionalRequirements
Non-FunctionalRequirements
Use CasesUse Cases
Vision of product
Simple Requirement Management ProcessSimple Requirement Management Process
Problem Domain Solution Domain
Solution Proposal
Customer/Marketing/business
User StorysUser Storys
FEATUREVISION/NEED/
PROPOSAL
FEATUREVISION/NEED/
PROPOSAL
YOU!YOU!
Design documents & implementation
Test Case
Yläotsikko
Non-Functional Requirement category examplesNon-Functional Requirement category examples
Scalability
How our implementation is scaling in situation X?
Scalability
How our implementation is scaling in situation X?
Stability
Is our implementation stable on situation like
zzzZZZ?
Stability
Is our implementation stable on situation like
zzzZZZ?
Functionality
Implementation should work like this way
Functionality
Implementation should work like this way
Security
Is our implementation secure enough against
attack type xxx?
Security
Is our implementation secure enough against
attack type xxx?
PerformanceHow good performance our implementation provides against competitor?
PerformanceHow good performance our implementation provides against competitor?
StressHow much we can stress our implentation without
a problems?
StressHow much we can stress our implentation without
a problems?
UsabiltyIs implementation usable for target
customer?
UsabiltyIs implementation usable for target
customer?
MaintenanceIs implementation easy to maintain?
MaintenanceIs implementation easy to maintain?
Yläotsikko
Practice: Create Use CasesPractice: Create Use Cases
ACTOR
Use Case
SYSTEM Actor:
System: IFDK kit
Use Cacenario: Standby mode after boot
1. User turn’s on IFDK by pressing red button on front panel
2. Screen wil flash and show welcome text ”Hello my friend!”
3. User interface opens after seconds
4. Screen will show three selection buttons
5. After 30 seconds user inteface goes to standby mode
Exeption:
1. If user activates screenby tapping it standby counter will be reseted
Actor:
System: IFDK kit
Use Cacenario: Standby mode after boot
1. User turn’s on IFDK by pressing red button on front panel
2. Screen wil flash and show welcome text ”Hello my friend!”
3. User interface opens after seconds
4. Screen will show three selection buttons
5. After 30 seconds user inteface goes to standby mode
Exeption:
1. If user activates screenby tapping it standby counter will be reseted
Yläotsikko
USE CASE:_____________________________________________
Actor:__________________________________
Scenario:
Execeptions:
USE CASE:_____________________________________________
Actor:__________________________________
Scenario:
Execeptions:
Yläotsikko
User StoryUser Story
• http://en.wikipedia.org/wiki/User_story
Yläotsikko
Brief History of User StorysBrief History of User Storys
• Used in agile development process for requirement definition and gathering
• Backlog Item• Simple way to describe needed functionality
Yläotsikko
DefinesDefines
Feature X * nFeature X * n
FunctionalRequirements
FunctionalRequirements
Non-FunctionalRequirements
Non-FunctionalRequirements
Use CasesUse Cases
Vision of product
Simple Requirement Management ProcessSimple Requirement Management Process
Problem Domain Solution Domain
Solution Proposal
Customer User StorysUser Storys
FEATUREVISION/NEED/
PROPOSAL
FEATUREVISION/NEED/
PROPOSAL
YOU!YOU!
Design documents & implementation
Test Case
Yläotsikko
Practice: Create User StorysPractice: Create User Storys
USER STORY:
As a bad behavin person I cannot access IFDK using wlan without encryption
How to test?Acceptance Criteria?
USER STORY:
As a bad behavin person I cannot access IFDK using wlan without encryption
How to test?Acceptance Criteria?
USER STORY:
As a member of audience I can hear effect sound that player is having electrical shocks
How to test?Acceptance Criteria?
USER STORY:
As a member of audience I can hear effect sound that player is having electrical shocks
How to test?Acceptance Criteria?
Yläotsikko
Agile Requirement Management Agile Requirement Management
Epic Story
User StoryUserStory0001
RequirementId0002
RequirementId0003
EpicStory0001
As a userI would like to use
product Which is fast to power
on
As a userI would like to use
product Which is fast to power
on
As a Customer I wouldlike to have topquality product
As a Customer I wouldlike to have topquality product
NOTE:Gadget should
have >30fpsUI performace
NOTE:Gadget should
have >30fpsUI performace
NOTE:Gadget should
Startup <5seconds
NOTE:Gadget should
Startup <5seconds
AcceptanceCriterias?
Yläotsikko
EPIC STORY: As a user I would like to use product which is fast to startup
Story Points:
Notes:
• Gadget should startup <5seconds
How to show it is tested? (eg. Acceptance Criteria)
Use timer to verify startup time. Measure time from power on to main screen activated.
• Time
EPIC STORY: As a user I would like to use product which is fast to startup
Story Points:
Notes:
• Gadget should startup <5seconds
How to show it is tested? (eg. Acceptance Criteria)
Use timer to verify startup time. Measure time from power on to main screen activated.
• Time
USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up
Story Points:
Notes:
• Gadget should startup <5seconds
How to show it is tested? (eg. Acceptance Criteria)
Use timer to verify startup time. Measure time from power on to main screen activated.
• Time
USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up
Story Points:
Notes:
• Gadget should startup <5seconds
How to show it is tested? (eg. Acceptance Criteria)
Use timer to verify startup time. Measure time from power on to main screen activated.
• Time
Yläotsikko
USER STORY:
Story Points:
Notes:
How to show it is tested? (eg. Acceptance Criteria)
USER STORY:
Story Points:
Notes:
How to show it is tested? (eg. Acceptance Criteria)
USER STORY:
Story Points:
Notes:
How to show it is tested? (eg. Acceptance Criteria)
USER STORY:
Story Points:
Notes:
How to show it is tested? (eg. Acceptance Criteria)
Yläotsikko
http://www.sysml.org/http://www.sysml.org/
UML
OMT
OMT
System Engineering
Google: SysML, UML, Systems engineering, Software Design,
Code Code
Software Engineering - ArchitectureSysML
UML
OMT
.NET, JAVA, C++
Software Engineering - Design
Requirements
http://www.sysml.org/
http://www.sysml.org/
http://www.opfro.org/
http://www.google.fi/url?sa=t&rct=j&q=requirement%20specification%20example&source=web&cd=1&ved=0CBgQFjAA&url=http%3A%2F%2Fwww.opfro.org%2FComponents%2FWorkProducts%2FRequirementsSet%2FSystemRequirementsSpecification%2FSystemRequirementsSpecificationExample.doc&ei=aiWETqjnG-P04QTMzNivDw&usg=AFQjCNFd4r5LLJgQj3foixq_jxZHjk78pQ&sig2=ILMCdBta4j8JVTCbQfgZgQ&cad=rja
Interesting Links ?Interesting Links ?
Yläotsikko
You have vision of productWhat means testing in brief?
You have vision of productWhat means testing in brief?
FeaturesFeatures Test CaseTest Case
Use CasesUse Cases
User StorysUser Storys
Product Design & Implementation
RequirementsRequirements
Test CaseTest Case
Test CaseTest Case
Test CaseTest Case
Ready to testReady to test
Testing & Quality Assurance
Can we deliverProduct
Ready to Deliver
Ready to Deliver
? CustomerNot ready
todeliver
Yläotsikko
SW Development Process (Waterfall)SW Development Process (Waterfall)
RequirementGathering/Evaluation
RequirementGathering/Evaluation
DesignDesign
ImplementationImplementation
Verification &Validation
Verification &Validation
MaintenanceMaintenanceError ManagmentProcessError ManagmentProcess
Task1Task1Task1Task1 Task1Task1 Task1Task1
Mile Stone 1Mile Stone 1 Mile Stone 2Mile Stone 2 Mile Stone 3Mile Stone 3
Task1Task1 Task1Task1
SW Development Process (Agile)SW Development Process (Agile)
User Story YUser Story Y
SprintSprint
Task1Task1
Task2Task2
Task3Task3
SprintSprint SprintSprint SprintSprint
Task4Task4
Task5Task5
Task6Task6
User Story XUser Story X
DesignImplementation
Verification
DesignImplementation
Verification
Product Backlog
DesignImplementation
Verification
DesignImplementation
Verification
DesignImplementation
Verification
DesignImplementation
Verification
DesignImplementation
Verification
DesignImplementation
Verification
User Story ZUser Story Z
Task7Task7
Task8Task8
Task9Task9
User Story ZUser Story Z
Yläotsikko
Customer/BusinessRequirements
Customer/BusinessRequirements
Sub SystemRequirementsSub System
Requirements
ComponentRequirementsComponent
RequirementsComponent / Unit
TestingComponent / Unit
Testing
IntegrationTesting
IntegrationTesting
SystemTesting
SystemTesting
AcceptanceTesting
AcceptanceTesting
SystemRequirements
SystemRequirements
IFDK SystemVerification and Validation
IFDK SystemVerification and Validation
Use CasesUse Cases
User StorysUser Storys
FeaturesFeatures
RequirementsRequirements
Validation = Are we building the right product?Verification = Are we building the product right?Validation = Are we building the right product?
Verification = Are we building the product right?
Architecture&Design&
Implementation
Architecture&Design&
Implementation
ProductProductVALIDATIONVALIDATION
VERIFICATIONVERIFICATION
IFDK Product IdeasIFDK Product Ideas
Yläotsikko
V-Model in testingV-Model in testing
Customer Requirements
Sub System Requirements
Component Requirements Component/Unit Testing
Integration Testing
System Testing
Acceptance Testing
System Requirements
Verification and Validation
Verification = Are we building the product right?Validation = Are we building the right product?
Yläotsikko
Component Design
ClassClass
AttributesAttributes
ClassClass
MethodsMethods
ClassClass
AttributesAttributes
ClassClass
MethodsMethods
Yläotsikko
Testing according ISEB standard
Functional Testing
Testing based on an analysis of the specification of the functionality of a component or system. See also black box testing.)
Non-Functional Testing
Testing the attributes of a component or system that do not relate to functionality, e.g. scalabilty, stability, reliability, efficiency, usability, maintainability and portability.
http://www.bcs.org/upload/pdf/glossary-current.pdf
Yläotsikko
Black Box vs White Box Testing
Unit Testing is White Box testingUnit Testing is White Box testingSystem Testing is Black BoxTestingSystem Testing is Black BoxTesting
????
Yläotsikko
We need to capsule all projects as sub projects, which have more independence
• Project Teams has to have own test engineer –> Integration Test Engineer• this lead’s to better defect prevention• Integration Test Engineer is part of project team, part of team at start of project• Test Engineer provides valuable information for design and enables better testability for project product
Define new roles for validation engineers
• Check next page
Lower level for test automation
• Enables regression test (method to block new implementation defects)• Support for executing more complex test scenarios• Continuous test execution (stability, performance gain etc)
Notes
Yläotsikko
Test Level1 – Component/Unit testing
• Component/Class level unit testing (eg. xUnit framework)
Test Level2 - Integration Testing (Feature, Application)
• Integration test for feature/application with “stub” interface components• Functionality SW Testing• Non-Functional SW Testing according needs (eg. feature/application specific performance, stability)
Test Level3 – Regression Testing (Target HW+Android platform, tool, terminal etc.)
• Functional SW testing for all possible features• Non-Functional SW testing according needs• Non-Functional HW Testing according needs
Test Level4 – System Integration Testing (Target Platform, Tool, Terminal etc)
• Functional SW testing for all new features• Non-Functional SW Performance, Stability, Scalability
Test Level5 – Acceptance Testing
• Functional/Non-Functional test according customer requirements
HWT1 – Hardware Testing• HW Integration test with limited environment. • Conducted performance verified
Testing Level descriptions
Yläotsikko
Testing Levels (with TAF = test automation framework)IFDK Release v1.0
Feature Project 2
Regression Testing (ISEB)System Integration
TAF
Component/Unit Testing (ISEB)
IntegrationTesting (ISEB)
System Testing (ISEB)
AcceptanceTesting (ISEB)
Feature Project 1
TAF
TAF Project
TAF enables
TAF
Suppo
rts
Verification and Validation
Yläotsikko
Refrerences & LinksRefrerences & Links
http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdfhttp://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdf
Yläotsikko
VERIFICATION & VALIDATIONVERIFICATION & VALIDATION
Validation = Are we building the right product?
Verification = Are we building the product right?
Yläotsikko
Create a Test Case!
Functional?
Your SourcesFor Test Case
Functional Test Case:
●Verify functionality of XXXX
Non-Functional Test Cases
●Verify Stability of XXXX●Verify Performance of XXX●Verify Security of XXXX●Verify Usability of XXXX●Verify Scalability of XXXX●etc...
Non-Functional?
Which Type?
●Requirement●Use Case●Feature●User Story
●Customer's Idea●Brainstorm●Intitution●Exploratory
Check also.....
●Correct functionality path●Miss-usage of functionality●Boundary Check
Check also.....●Check Possiblity to automated testing?
How to create Test Case???
Regression Test Case??
Write a Case
Write a Case
Yläotsikko
Verify what?Using configuration?
With tools?
What is verdict?
●Verify drum track player pause mode functionality.●Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury”●Test should be done using android emulator environment and using your hands, ears and eyes”
Add Information about case
●Pre State:●Android emulator is running●Release X is installed on emulator
●Test Case Steps:●1. Open drum kit player application●2. Select song ”Show must go on”●3. Start to play●4. Press Pause and check song is paused●5. Check memory usage from system application●6. Press Play●7. jump to 4 several time (<10)●8. Listen song to the end●9. Exit player using ”exit button”
●End State:●IFDK Kit in main screen mode
●Test Case Id●Test Case owner/writer●Date●comments
●If Pause is working result is PASS. If Pause mode failed result is FAIL
Define pre-stateDefine Steps
Define end-state
What Information Test Case should contain?
Yläotsikko
Component /Unit TestingComponent /Unit Testing
ClassClass
AttributesAttributes
ClassClass
MethodsMethods
ClassClass
AttributesAttributes
ClassClass
MethodsMethods
ClassClass
AttributesAttributes
ClassClass
MethodsMethods
ClassClass
AttributesAttributes
TestClassTestClass
MethodsMethods
Unit Test Frame WorkUnit Test Frame WorkResultResult
Yläotsikko
Integration testing using emulatorIntegration testing using emulator
Yläotsikko
Yläotsikko
Test Case
SUT/DUTIFDK
What is generated as results from test case execution
LOG FILE
EVENTS
NOTIFICATIONS
SUT = System Under TestDUT = Device Under Test
ENVIRONMENTANDROID EMULATOR
TOOLSTOOLSTOOLSscripts/grep
TEST CASE
Yläotsikko
●TEST CASE ID XXXXX
●Step1●Step2●Step3.●Step4.
INCIDENT(Huomio)
Bug/Defect(Vika)
Defect Database
IFDKSystem
EXECUTE TEST !
Test Engineer
Notes
Reports
What means error reporting?
Yläotsikko
Component TestingComponent Testing
Customer/BusinessRequirements
Customer/BusinessRequirements
Sub SystemRequirementsSub System
Requirements
ComponentRequirementsComponent
RequirementsComponent / Unit
TestingComponent / Unit
Testing
IntegrationTestingIntegrationTesting
SystemTestingSystemTesting
AcceptanceTestingAcceptanceTesting
SystemRequirements
SystemRequirements
Architecture&Design&
Implementation
Architecture&Design&
Implementation
ProductProduct
VALIDATIONVALIDATION
VERIFICATIONVERIFICATION
Yläotsikko
How to verify component implementation
How to verify component implementation
-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse
-Unit Testing-Code Coverage-Branch Coverage-Complexity Analyse
Yläotsikko
Code ComplexityCode Complexity
Example tool CCCC
Yläotsikko
Branch coverageBranch coverage
The percentage of branches that have been
exercised by a test suite. 100% branch coverage
implies both 100% decision coverage and 100%
statement coverage.
Yläotsikko
Code CoverageCode Coverage
An analysis method that determines which parts of
the software have been executed (covered) by the
test suite and which parts have not been executed,
e.g. statement coverage, decision coverage or
condition coverage.
http://en.wikipedia.org/wiki/Code_coverage
http://www.atlassian.com/software/clover/
Yläotsikko
Release/Configuration Management &Integration Testing
Day 6Day 6
Yläotsikko
Putting all tools together!Putting all tools together!
Continous Integration
http://hudson-ci.org/
Yläotsikko
Design verification -Unit Testing
Design verification -Unit Testing
Yläotsikko
Unit Test example Re-run with MeegoUnit Test example Re-run with Meego
Example of xUnit in QT environment
Yläotsikko
Integration TestingIntegration Testing
Customer/BusinessRequirements
Sub SystemRequirements
ComponentRequirements
Component / UnitTesting
IntegrationTesting
SystemTesting
AcceptanceTesting
SystemRequirements
Architecture&Design&
Implementation
Product
VALIDATION
VERIFICATION
Yläotsikko
Why Integrate first? Avoid Big Bang!Why Integrate first? Avoid Big Bang!
HW ComponentHW Component
Data BaseData BaseComponent/Application 10% testedComponent/Application 10% tested
Web ServiceWeb Service
Tested Component/ApplicationTested Component/Application
Yläotsikko
Integration Test with stubsIntegration Test with stubs
Tested Component/ApplicationTested Component/Application
LogLog
STUB/MOCK ComponentSTUB/MOCK Component
Scripted STUB InterfaceScripted STUB Interface
ControlConfigureControlConfigure
SimulatedInterfaceSimulatedInterface
Messages/EventsMessages/Events
STUB/MOCK ComponentSTUB/MOCK Component
Control InterfaceControl Interface
Yläotsikko
in practice #1 IFDK android setupin practice #1 IFDK android setup
Tested ComponentApplication
Tested ComponentApplication
Activate/ControlActivate/Control
STUB/MOCK ComponentSTUB/MOCK Component
Scripted STUB InterfaceScripted STUB Interface
ControlConfigureControlConfigure
SimulatedInterfaceSimulatedInterface
Messages/EventsMessages/Events
WEB SERVER simulatingFacebook interfaceWEB SERVER simulatingFacebook interface
Control InterfaceControl Interface
Trace/LogTrace/Log
Yläotsikko
in practice #2 server component testingin practice #2 server component testing
Tested ComponentApplication
Tested ComponentApplication
Trace/LogTrace/Log
Activate/ControlActivate/Control
Mock Server/DaemonMock Server/Daemon
Scripted STUB InterfaceScripted STUB Interface
AutomatedTest Interface
AutomatedTest Interface
SimulatedInterfaceSimulatedInterface
Messages/EventsMessages/Events
WEB SERVERWEB SERVER
Control InterfaceControl Interface
Operating SystemOperating System
NeededFake Application
NeededFake Application
Junit Scripted InterfaceJunit Scripted Interface
Yläotsikko
Release Management and Integration Testing
Release Management and Integration Testing
IFDK ApplicationIFDK Application
Facebook Web ServiceFacebook Web Service
HW ComponentCalore Meter
Enabled Drum Stick
HW ComponentCalore Meter
Enabled Drum Stick
Calore Meter SW Component 10% testedCalore Meter SW Component 10% tested
Data Base Schema DesignData Base Schema Design
Stubsneeded
Stubsneeded
Stubsneeded
Stubsneeded
Stubsneeded
Stubsneeded
Stubsneeded
Stubsneeded
First System TestWith all componentsFirst System TestWith all components
Yläotsikko
Trace/Log as feedbackTrace/Log as feedback
Log contains important information. Log
Simple tool for log analysing in Linux ”grep” command ,
TAIL -F /var/log/messages | grep error
-Specific Inhouse log capturing and analyse tools
Yläotsikko
AgendaAgenda
Brief guidance for NEST Project Platform 1.3
Background story for reference project IFDK
Some Definitions
SW Development Process
Release + Configuration Management
Testing Levels and Error Management
Hands On: Test Link + Bugzilla
Error Reporting, Metrics and daily usage
Closed Software Project vs Open Source Project
DiscussionYläotsikko
Error/Bug/Defect ReportError/Bug/Defect Report
●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..
●Defect/Burg/Error ID●Reporter●Time●Founded where●Which way?●Test Case●Test Setup/Configuration●Describe scenario?●Attachements? Picture/Log/etc..
Yläotsikko
About Error ManagementAbout Error Management
RequirementManagementRequirementManagement
ImplementationProcess
ImplementationProcess
Verification&
Validation
Verification&
Validation
FailureReportFailureReport
Yläotsikko
DefinitionsDefinitions
Failure -
Fault, Defect, Bug -
Incident, Failure, Error
Example forum thread: http://www.allinterview.com/showanswers/36257.html
ISTQB syllabus
Yläotsikko
Release & Configuration Management
Release & Configuration Management
Yläotsikko
Release ManagementRelease Management
Version 0.1
Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3
Version 0.2.1Version 0.2.1
Version 0.2.2.1Version 0.2.2.1
Version 0.2.2Version 0.2.2
Version 0.4Version 0.4TrunkTrunk
Customer 1Customer 1
Customer 1Customer 1 Version 0.2.3Version 0.2.3
Version 0.2.2.2Version 0.2.2.2
Version 0.2.2.3Version 0.2.2.3
Yläotsikko
Release & Configuration Managmement
Version 0.1
Version 0.1 Version 0.2Version 0.2 Version 0.3Version 0.3
Version 0.2.1Version 0.2.1
Version 0.2.2.1Version 0.2.2.1
Version 0.2.2Version 0.2.2
Version 0.4Version 0.4TrunkTrunk
Customer 1Customer 1
Customer 1Customer 1
Version 0.2.3Version 0.2.3
Version 0.2.2.2Version 0.2.2.2
Version 0.2.2.3Version 0.2.2.3
FeaturesFeatures
Release 1.0Release 1.0
Release 1.0Release 1.0
Release 1.0Release 1.0
FeaturesFeatures
FeaturesFeatures
Yläotsikko
Release PlanningRelease Planning
Yläotsikko
Test ManagementTest Management
Yläotsikko
Validaton& Verificaton (Testing) Management
Version 0.4Version 0.4
Version 0.2.2.2Version 0.2.2.2
Version 0.2.2
Version 0.2.2
Test PlanTest CasesFor
Features
Test PlanTest CasesFor
Features
Tested Release/configurationTested Release/configuration
Error/DefectReport
Error/DefectReport
Error/DefectReport
Error/DefectReport
Error/DefectReport
Error/DefectReport
Yläotsikko
Feature PackFeature Pack
Project ManagerProject Manager
Designer/CoderDesigner/Coder
IntegrationTest EngineerIntegrationTest Engineer
Test ManagerTest Manager
System TestingSystem Testing
IFDK System TestingIFDK System Testing
Feature Unit/Integration TestingFeature Unit/Integration Testing
IFDK System Acceptance TestingIFDK System Acceptance Testing
SystemTest EngineerSystemTest Engineer
Test AutomationEngineerTest AutomationEngineer
AcceptanceTest EngineerAcceptanceTest Engineer
ValidationValidation
VerificationVerification
IFDK Verification/Validation (Testing Organization)IFDK Verification/Validation (Testing Organization)
Regression Testing
Regression Testing
Integration Testing
Integration Testing
System Testing
System Testing
Acceptance Testing
Acceptance Testing
Test AutomationTest Automation
Unit Testing
Unit Testing
Product ReleaseProduct Release
Yläotsikko
Error and change managementError and change management
Yläotsikko
Example Sources for error reportExample Sources for error report
CRM CRM
Field TestingField Testing
Testing ProcessTesting Process
CustomerFeedback /Customer
Feedback /
Error ReportError Report Change Request?Change
Request?
N x IncidentsN x Incidents
Yläotsikko
Change ManagementChange Management
Sometimes founded defect can lead to change
Bug?Bug?
Change Request?Change Request?Not ClearRequirementsNot Clear
Requirements
Feature ?Feature ?
Yläotsikko
Error Reporting and Process
Yläotsikko
Hands On: Bugzilla Error DatabaseHands On: Bugzilla Error Database
http://www.bugzilla.org/http://www.bugzilla.org/installation-list/
http://www.bugzilla.org/http://www.bugzilla.org/installation-list/
What is Bugzilla?
Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.
What is Bugzilla?
Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.
Yläotsikko
Other Defect Database Solutions
JIRA – Commercial
Requisite Pro – Commercial
Rational Synergy - Commercial
Mantis – Open Source
Yläotsikko
Yläotsikko
BugzillaBugzilla
Yläotsikko
Reporting, Metrics and daily usageReporting, Metrics and daily usage
Yläotsikko
CMMI Process framework
CMMI – covers ”error management” in several process areas
SP3.2 Analyze Verification results
Typical Work Products: Trouble reports
- Analyze the verificationd ata on defects
- Record all results of the analysis in a report
- Provide infromation on how defects can be resolved (including verfiocation methods, criteria, and verification environment) and initiate corrective action
Project Monitoring and Control
SG2 Manage Corrective Action to Closure
SP2.1 Analyze Issues
SP2.2 Take corrective Action
SP2.3 Manage corrective Action
SG2 Validate Product or Product Components
SP2.2 Analyze Validation Results. Change request management & Configuration Management process
SG2 Track and Control Changes
SP2.1 Track Change Requests
SP2.2 Control Configuration Items
Yläotsikko
Traditional SW Project vs Open Source ProjectTraditional SW Project vs Open Source Project
Open Source – Crowd Sourcing
SW Relase tested without coordination by group of volunteers
Release tested by customer
Field Testing
Test Group
Yläotsikko