Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow...
Transcript of Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow...
![Page 1: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/1.jpg)
© 2011 CTG, Inc.
Playing around with
Risks
Jurgen Cleuren
Nov. 24th 2011
![Page 2: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/2.jpg)
Introduction
• Projects are done in a probabilistic environment
– Incomplete information
– Parameters change over time
– What is true in the beginning of the project, can be false some time
later
• Games in a probabilistic environment
– Incomplete information (cardgames,…)
– Random element (dice, cards,…)
![Page 3: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/3.jpg)
Which games ?
• Poker
• Monopoly
• Backgammon
![Page 4: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/4.jpg)
Succes:
First learn the rules, then play better than everyone else
- Albert Einstein -
• Rules -> requirements
• Every game starts with learning and agreeing on the rules -> every
project starts with defining and agreeing on the requirements
![Page 5: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/5.jpg)
TEXAS HOLD‟EM POKER
![Page 6: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/6.jpg)
Rules of Texas Hold‟em
• 2 Blind cards
• Betting round
• 3 Open Community cards (flop)
• Betting round
• 1 Open Community card (Turn)
• Betting Round
• 1 Open Community Card (River)
• Betting Round
• Best hand of 5 cards wins
![Page 7: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/7.jpg)
Hand Ranking
• Highest Card
• Pair
• 2 Pair
• 3 of a kind
• Straight (5 consecutive cards)
• Flush (5 cards of the same suit)
• Full House (3 and 2 )
• 4 of kind
• Straight Flush (5 consecutive cards of the same suit)
![Page 8: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/8.jpg)
General test flow vs Poker round
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
![Page 9: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/9.jpg)
Similarities / Differences
3 Recurring phases
1 big phase (Complete Functional testing vs Flop)
2 small phases (retest + regression vs Turn + River)
Determine Risk before 1st test run (Risk Matrix or FTT vs 1st bet)
Adapt Risk Matrix or FTT after each test run
![Page 10: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/10.jpg)
Should the FTT or Risk Matrix be adapted ?
• Every Test run gives new information
• Likelihood of risks change
– Failed test cases get a higher likelihood
– Passed test cases in unchanged code have a lower likelihood
• Closer to deadline => Risks can change
• Reporting is more clear
– Each Risk Matrix/FTT tree is a snapshot of the project at a certain time
![Page 11: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/11.jpg)
![Page 12: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/12.jpg)
![Page 13: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/13.jpg)
![Page 14: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/14.jpg)
![Page 15: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/15.jpg)
Who ?
• Test manager should take the lead
• Preferably Project Board
• Test manager can do it by himself, but the board should at least be aware
that this activity is done
![Page 16: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/16.jpg)
Justification
• A Poker hand needs justification at any point in the game
– Having the best hand in the beginning doesn‟t imply that you are going
to win
– You must be prepared to fold when you are not winning anymore
– The money you‟ve already bet doesn‟t count
• It is in the pot so it is not your money anymore
• Don‟t put more money in a losing hand
– No justification anymore: get out of the hand
– Cut your losses
![Page 17: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/17.jpg)
Project justification
• A project needs justification at any point in time
• During testing: justification after each test run
– New information is given
– Risks are updated
• No Justification means project should be closed
• Previous investments do not count
– Don‟t put more money in a failing project
• Cut your losses
![Page 18: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/18.jpg)
Test process justification
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Justification
![Page 19: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/19.jpg)
Result Oriented Thinking
• A decision can be right even if the outcome is not favourable
• Focus more on the decision and the „why‟ instead of the result
– Tester A completes a test set in 4 days by skipping tests
– Tester B completes the same test set in 6 days through thoroughly
testing
– No defects were discovered in production
– Who would you reward the most ?
![Page 20: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/20.jpg)
Thought process
• In poker, to anticipate and understand your opponents actions, you need
to think as your opponent and not as you in his situation
• What motivates you, does not necessarily motivate another person
• Successful people ask better questions
– WHY? is more important than HOW? or WHO?
“Someone who knows HOW will always have a job
Someone who knows WHY will always be his boss”
![Page 21: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/21.jpg)
Luck
“ Luck is when preparation meets opportunity”
- Seneca –
• Be prepared to get lucky
– In poker, sometimes you need to get lucky. When you get lucky, be sure
to take a big pot out of it.
– Sometimes a best case scenario happens, but we need to take
advantage of it.
– Being ahead of planning is more valuable if you can actually do
something with this situation
![Page 22: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/22.jpg)
MONOPOLY
![Page 23: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/23.jpg)
![Page 24: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/24.jpg)
Aspects of the game
• Investing in houses
• The higher the investment, the bigger the payoff
• Some spaces have a higher probability
• Random element: dice
![Page 25: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/25.jpg)
Analogy to Risk Based Testing
• Different probability of landing on spaces = Likelihood
• Higher Payoff = Impact
• What can monopoly teach us ?
– What is more important: Impact or Likelihood ?
![Page 26: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/26.jpg)
Impact and likelihood on the board
![Page 27: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/27.jpg)
Winning Monopoly strategies
• Complete Orange Colour group and invest as soon as possible
– Why Orange ? It has the biggest probability of other players landing on
it
– Be prepared to even trade down to get this colour group
• Complete Red Colour group as a second priority
– Why Red ? It has the second biggest probability of other players
landing on it
![Page 28: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/28.jpg)
What lessons are to be learned
• Likelihood >>>>>> impact
– The weight of Likelihood should be > 50%
– The weight of Impact should be < 50%
![Page 29: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/29.jpg)
BACKGAMMON
![Page 30: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/30.jpg)
Backgammon
![Page 31: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/31.jpg)
Important rules of Backgammon
• Goal: get all your tiles from one end to the other
• Only tiles that are standing alone on a pillar can be captured
• A captured tile has to be brought back in play at the beginning of the
board
• Random element: Dice
![Page 32: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/32.jpg)
Translation to risks
• Your own checkers: Risks
• Opponents checkers: Causes
• Whenever one of your checkers is captured it‟s in fact a cause hitting a
risk
• A risk is mitigated when the checker cannot be captured (2 or more on
one pillar)
![Page 33: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/33.jpg)
Translation of risk priority
• Low risk: checker in the first quadrant
• Medium risk: checker in the second or third quadrant
• High risk: checker in the fourth quadrant
![Page 34: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/34.jpg)
What can Backgammon teach us?
• Which risks should be mitigated and which are low priority ?
• There might be a possibility to remove a cause, but in the same way
creating a new risk. Should we do it ?
– Eg: Back-up testmanager vs full time testing
![Page 35: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/35.jpg)
Backgammon Strategies
• Checkers in the 1st quadrant don‟t have to be protected. Moving forward
is the better play.
Risks with low priority don‟t have to be mitigated. The correcting cost is
usually way less than the mitigation costs
• Checkers in the 4th quadrant need to be protected at all costs.
Risk with high priority need to be mitigated at all costs
• For checkers in the 2nd and 3rd following rule applies: Always take a
chance to capture
If you can remove a cause and therefore create a medium of low risk,
do so
![Page 36: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/36.jpg)
CONCLUSIONS
![Page 37: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/37.jpg)
Conclusions
• What Poker taught us:
– Adapt FTT/Risk tree after each test run
– Priorities of test items can change
– Justification has to be done after each test run
– Don‟t focus on results, focus on decisions
– Be prepared to get lucky!
![Page 38: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/38.jpg)
Conclusions
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Adapt FTT/Risk Matrix
• Justification
![Page 39: Playing around with Risksconference.eurostarsoftwaretesting.com/wp-content/... · General test flow vs Poker round • Create FTT/Risk Matrix • Software Delivery • Full functional](https://reader034.fdocuments.in/reader034/viewer/2022051806/5ffd46ff51768c1b9e784326/html5/thumbnails/39.jpg)
Conclusions
• Monopoly
– Likelihood >>>>> Impact
• Backgammon
– Don‟t mitigate low priority risk
– Always mitigate high priority risks
– Removing a cause and creating a medium or low risk is the way to play