Evidence based decision-making - lean product development
-
Upload
sage-software-consulting-inc -
Category
Software
-
view
103 -
download
0
Transcript of Evidence based decision-making - lean product development
![Page 1: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/1.jpg)
Evidence-based Decision-making for Lean Product Development
presented byKevin Burns@ MN AEG
Jan. 17, 2017
[email protected], @kevinbburns
![Page 4: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/4.jpg)
[email protected], @kevinbburns 4
![Page 5: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/5.jpg)
Peace Corp Recruitment and Public AffairsStory of how we used technology to improve Peace Corp recruitment.
Switch from USPS to email
Switch from manual data entry to wild-card search in gopher email system and screen scraping results, an early for of ETL.
Conduct direct email campaigns when spam still meant ‘meat in a can’
[email protected], @kevinbburns 5
![Page 6: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/6.jpg)
[email protected], @kevinbburns 6
![Page 7: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/7.jpg)
Open Discussion
Who’s measuring value, outcomes, and/or impacts today?
How are you measuring them?
If you’re not measuring them, why not?
Who’s measuring cost?
How are you measuring cost?
[email protected], @kevinbburns 7
![Page 8: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/8.jpg)
The 1st Agile Principle
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
(should we change valuable to beneficial impact?)
How do we define value (impact) and how do we measure it?
Not all Projects (or Features) are created equal.
[email protected], @kevinbburns 8
![Page 9: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/9.jpg)
Is value determined by delivery on time, on budget, and on scope?
Is the scope delighting the customer?Are they using everything we delivered?
[email protected], @kevinbburns 9
![Page 10: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/10.jpg)
In a survey of 4 products, 65% of the features were rarely or never used.
How much money could have been saved if we never built them?
In the Waterfall project world, we have to ask for everything we can think of because capital will end at the end of the project. Instead we should be asking what has the most value in terms of the business outcome and/or impact and how are we going to measure it.
[email protected], @kevinbburns 10
![Page 11: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/11.jpg)
[email protected], @kevinbburns 11
![Page 12: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/12.jpg)
[email protected], @kevinbburns 12
![Page 13: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/13.jpg)
Marty Cagan Quotes
• Customers don’t know what they want. It’s very hard to envision the solution you want without actually seeing it.
• At least 2/3 of our ideas are never going to work. The other 1/3 will take 3-4 iterations to get right.
• The role of the product manager is to discover a product that is valuable, usable, and feasible. Product, design, and engineering work together to arrive at optimal solution.
[email protected], @kevinbburns 13
![Page 14: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/14.jpg)
[email protected], @kevinbburns 14
![Page 15: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/15.jpg)
What we measure is changing
Business CustomerPO, SM, BL
Software EngineeringAD, DD, DA
UserUX, BA, QA, SME
BusinessValuable
Design Usable
TechnicallyFeasible
INNOVATIVESOLUTION
[email protected], @kevinbburns 15
![Page 16: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/16.jpg)
Lean Startup
• Are we asking what are Minimum Viable (Valuable) Product and how do we know when we’ve delivered it?
• Use a scientific method to measure, learn and pivot or preserver.
• Use meaningful quantitative objective measure to evaluate impact.
• Can you use A/B testing?
[email protected], @kevinbburns 16
![Page 17: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/17.jpg)
MVPInnovation
UserUX, BA, QA, SME
Business Valuable
Design Usable
Software EngineeringAD, DD, DA
Business CustomerPO, SM, BL
Use scientific method (measurable) to learn
and discovery your Minimum Viable
(Valuable) Product (MVP)
Technically Feasible
MVP innovations emerge from Conversations
[email protected], @kevinbburns 17
![Page 18: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/18.jpg)
Value and/or Impact driven culture
• Are we measuring the Cost vs Benefit at all levels of our work items?• Portfolio• Program• Project• Feature/Capability• Story/Requirement• Tasks/Test
• Are we measuring the Impact our features have on our customers?
• The act of sizing helps us define done and value
• Use story telling and test statements create understanding of value and DoD
[email protected], @kevinbburns 18
![Page 19: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/19.jpg)
Don Reinertsen’s Seven Big Ideas2nd Generation Lean Product Development1. Understand your economics2. Manage your queues3. Exploit variability4. Enable smaller batches5. Control WIP and start rates6. Prioritize based on economics7. Accelerate feedback
[email protected], @kevinbburns 19
![Page 20: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/20.jpg)
1. Understand Your Economics
• In product development all difficult decisions involve multiple variables.
• Making decisions that affect multiple variables requires quantification.
• Doing quantification, isn’t as hard as you might think.
[email protected], @kevinbburns 20
![Page 21: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/21.jpg)
A Typical Question
Should we operate our testing process at 80% utilization with a 2-week queue, or 90% utilization with a 4 week queue?
[email protected], @kevinbburns 21
![Page 22: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/22.jpg)
Making Economic Decisions
• Waste• Cycle time• Variability• Efficiency• Revenue• Unit Cost• Value-Added
Proxy Variable Space
Tran
sfor
mat
ions
Life Cycle Profits
Economic Space
[email protected], @kevinbburns 22
![Page 23: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/23.jpg)
Modeling Process
Model Expense Overrun
Model Cost
Overrun
Model Value
Shortfall
Model Schedule
Delay
Model Risk
Change
Create Baseline Model
Determine Total Profit Impact of Missing a MOP
Calculate Sensitivity Factors
[email protected], @kevinbburns 23
![Page 24: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/24.jpg)
Guidelines for decision-making
• Set strategic goals/guidelines for decision-making so low-level tactical decision can be decentralized while still being aligned within strategic goals/objectives.
• While good intentioned, centralized decision-making is often slow and suboptimal because it lacks the context of all the variable at play at tackle level
[email protected], @kevinbburns 24
![Page 25: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/25.jpg)
2. Manage Your Queues
• Many product developers assume higher utilization leads to faster development.
• They neither measure nor manage the invisible queues in their process.
• Consequently, they underestimate the true cost of overloading their processes.
• Such overloads severely hurt all aspects of development performance.
[email protected], @kevinbburns 25
![Page 26: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/26.jpg)
Effect of Capacity Utilization
10 20 30 40 50 60 70 80 90 100
05
1015
20
Que
ue S
ize
% Capacity [email protected], @kevinbburns 26
![Page 27: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/27.jpg)
Managing Queues
Cost of Excess Delay
Total Cost
Dollars
Excess Product Development Resources
Minimize Total Cost to Maximize Profits
[email protected], @kevinbburns 27
![Page 28: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/28.jpg)
Why Queues Matter
• Queues Create…• Longer cycle time• Lower Quality• More variability• Increased risk• More overhead• Less motivation
Managing queues is the key to improving product development economics
[email protected], @kevinbburns 28
![Page 29: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/29.jpg)
3. Exploit Variability
• In manufacturing it is always desirable to reduce variability
• In product development eliminating variability eliminates innovation
• We must understand the specific conditions that make variability valuable and manage our process to create these conditions
• We need development process that function in the presence of variability
[email protected], @kevinbburns 29
![Page 30: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/30.jpg)
Asymmetric Payoff and Option Pricing
Expected Price Payoff vs Price
Expected Payoff
Expe
cted
Pay
off
Price
PricePrice
Prob
abili
ty
Payo
ff Strike Price
Strike Price
[email protected], @kevinbburns 30
![Page 31: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/31.jpg)
Higher Variability Raises the PayoffEx
pect
ed P
ayof
f
Price
Strike Price
Payoff SD = 15
Payoff SD = 5
Option Price = 2, Strike Price = 50, Mean Price – 50, Standard Deviation = 5 and 15
[email protected], @kevinbburns 31
![Page 32: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/32.jpg)
4. Enable Smaller Batches
• When work products are invisible, batch sizes are invisible• When batch sizes are invisible, product developers pay little attention
to them• Many companies institutionalize large batch sizes• Batch size reduction is attractive because it is fast, easy, cheap,
granular, leveraged, and reversible• It is a great starting point for LPD
Batch Size Queues Cycle Time
X 0.5 X 0.5 X [email protected], @kevinbburns
32
![Page 33: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/33.jpg)
Drawing Review Process
200
10 Weeks
20
1 Week
Unreviewed Drawings
Large Batch Small Batch
[email protected], @kevinbburns33
![Page 34: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/34.jpg)
Benefits of Small Batch Testing
Higher ValidityFewer Open Bugs
Faster Cycle Time
Early Feedback
Less Debug Complexity
More Efficient Debugging
More Uptime
Smaller Change
Fewer Status Reports
Less Requirement Changes
Faster Learning
Lower Cost Changes
Cheaper Debugging
Cheaper Testing
Less Non-Value-Added
Better Code
Cheaper Correction
Better Economics
[email protected], @kevinbburns 34
![Page 35: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/35.jpg)
Setting Batch Size
Transaction Cost
Cost
Items per Batch
Economic Batch Size
1 2 3 4 5 6 7 8 9 10
05
1015
20
Total Cost
[email protected], @kevinbburns 35
![Page 36: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/36.jpg)
5. Control WIP and Start Rates
• Many developers incorrectly assume that the sooner they start work, the sooner they will finish it
• They are constantly tempted to start too much work
• This dilutes resources and causes long transit time through their processes
• A long transit time hurts efficiency, quality and responsiveness
[email protected], @kevinbburns 36
![Page 37: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/37.jpg)
Little’s Formula
• By constraining WIP in development processes we can control cycle time
• This approach, which is known as Lean Kanban, is currently growing rapidly in software development
https://en.wikipedia.org/wiki/Little%27s_law
MeanResponseTime = MeanNumberInSystem / MeanThroughput
[email protected], @kevinbburns 37
![Page 38: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/38.jpg)
Control Number of Active Projects
1
2
3
4
1
2
3
4
COD Savings of Project 1 and 2 Late Start Advantages for Project 3 and 4
Time to Deliver
Time to Deliver Time to Deliver
[email protected], @kevinbburns 38
![Page 39: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/39.jpg)
Avoid Long Planning Horizons
• The further out you plan, the less likely your forecast will be accurate
• Don’t do detailed analysis on things beyond a quarter
• Market conditions change everyday, this can change requirements
• Changing requirements cause churn (waste)
[email protected], @kevinbburns 39
![Page 40: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/40.jpg)
Visual WIP Control Boards
Ready Queue Coding Ready to Test Testing Done
WIP constraints = 10
13
14
15
16
11
10
9
8
7
6
5
4
3
2
112
?
[email protected], @kevinbburns 40
![Page 41: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/41.jpg)
6. Sequence Work Correctly
• The sequence in which work is processed is called the queuing discipline
• By changing the queuing discipline we can reduce the cost of a queue without decreasing the size of the queue
• Since manufacturing has homogeneous flows it always uses FIFO (First-In-First-Out)
• For the non-homogeneous flows of product development other approaches have better economics
[email protected], @kevinbburns 41
![Page 42: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/42.jpg)
Use FIFO for Homogeneous Flow
First-In First-Out
Cost of
Delay
12
3A
B
Time
Cost
Delay CostLast-In First-Out
Cost of
Delay 12
3
AB
Time
Cost
Project Duration Cost of Delay
1 3 3
2 3 3
3 3 3
[email protected], @kevinbburns 42
![Page 43: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/43.jpg)
Weighted Shortest Job First (WSJF) for Non-homogenous flow
High Weight First
Cost of
Delay
1
23
AB
Time
Cost
Delay Cost
Low Weight First
Cost of
Delay
A
B
Time
Cost
Project Duration Cost of Delay
Weight = COD/Duration
1 1 10 10
2 3 3 1
3 10 1 0.1
1
23
160 796 % Reduction in COD
[email protected], @kevinbburns 43
![Page 44: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/44.jpg)
7. Create Faster Feedback
• When queues and batch sizes are large feedback is slow
• Slow feedback hurts quality, efficiency, and cycle time
• Feedback speed has enormous economic leverage in product development, but it is rarely explicitly managed
[email protected], @kevinbburns 44
![Page 45: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/45.jpg)
The Front-Loaded Lottery
• A lottery ticket pays $3000 to winning three digit number• You can pick the number in two ways:
• Pay $3 to select all three digits at once• Pay $1 for the first digit, find out if it is correct, then choose if you wish to pay
$1 for the second digit, and then choose if you wish to pay $1 for the third digit.
[email protected], @kevinbburns 45
![Page 46: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/46.jpg)
Value of Feedback
100%
Spend $1
Savings = $0.90Savings = $0.99
10%1%
0 $1 $2 $3
Probabilityof
Occurrence
Cumulative [email protected], @kevinbburns 46
![Page 48: Evidence based decision-making - lean product development](https://reader031.fdocuments.in/reader031/viewer/2022030214/589a416a1a28ab8c588b6437/html5/thumbnails/48.jpg)
References
• Don Reinertsen• https://www.youtube.com/watch?v=L6v6W7jkwok&spfreload=1#t=16.242347• https://www.amazon.com/gp/product/1935401009/ref=as_li_tl?ie=UTF8&camp
=1789&creative=9325&creativeASIN=1935401009&linkCode=as2&tag=reinertassoci-20&linkId=U56AHKXXQVN4VZFY
• https://en.wikipedia.org/wiki/Little%27s_law
[email protected], @kevinbburns 48