together with
Benchmarking R&D Portfolio
Management Best Practices
Michael Menke & Keith Duncan
November 14th, 2013
© 2013 Planisware
Introductions
Keith Duncan, NPDP Host & Presenter
Director, Innovation & PPM Practices
Planisware
Michael Menke, Ph.D. Featured Presenter
President, Value Creation Associates
Fellow, Society of Decision
Professionals
2
3
Welcome and Introduction
Study Overview, Significance, and Key Findings
The Most Critical Best Practices
Final Thoughts
Benchmarking R&D Portfolio
Management Best Practices.
Agenda
Poll:
Have you formally
benchmarked your
portfolio management
practices and process?
4
5
Welcome and Introduction
Final Thoughts
Benchmarking R&D Portfolio
Management Best Practices.
Agenda
Study Overview, Significance, and Key Findings
The Most Critical Best Practices
Portfolio Management is a decision process that is
clearly associated with value creation in Pharma.
*This includes client organizations who have fully implemented a value-based approach
to resource allocation and portfolio management. (Timeframe mid-1990s to early 2000s.)
Pharma Client* Stock Price Performance Relative to Dow pharma and S&P 500 indices
-50%
0%
50%
100%
150%
200%
250%
300%
350%
400%
450%
Year 0 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7
S&P 500
Index
Dow Jones
Pharma
Index
Portfolio
Adopters
Index
6
Good decisions begin with a conducive culture, effective
behaviors, a DQ framework, and a good decision process.
The PPM Accelerate benchmarking study focuses on behaviors,
framework and process to identify PPM best practices.
Culture & Incentives
Decision Behaviors
Decision Quality (DQ) Framework
Decision Process
DQ Methodology & Tools
7
Poll:
Does your organization
have a formal decision
quality process and use
tools and frameworks to
improve your decision
making?
8
This Framework applies to Portfolio Decisions.
• Although culture and incentives are very important, they are very
hard to change
• It is easier to study and to change decision behaviors and
practices, DQ framework and decision process
• Since 2011, Value Creation Associates has been co-sponsor of a
global Project Portfolio Management (PPM) best-practices
benchmarking study—PPM Accelerate
• The objective has been to establish importance, execution quality
and performance benchmarks for 50 PPM best practices
• In order to establish a standard of excellence (―benchmark‖), we
deliberately included a number of PPM leaders
• We now have data from over 70 organizations, many PPM leaders,
and are continuing to enroll new organizations
9
The following organizations are included in PPM Accelerate
Rounds 1-3, with a current total of over 70 participants.
They represent a wide range of industries and functions. The study is on-going.
Dow
AgroSciences
Dow Advanced
Materials
10
Resource
Information
Analytics,
Reporting &
Risk Assessment
Behavior
Financial
Information
Strategic
Value
Information
PPM Organization
& Governance
PPM
Processes
PPM Accelerate used this framework to organize
and assess the 50 Best Practices.
The 50 Best Practices
are organized into 8
categories:
Best practices and Pitfalls are
based on 30+ years of
practical experience and
extensive literature research.
Most of them had been
discovered and validated in
prior benchmarking studies.
A
B C
D E F
G H
11
Some best practices and pitfalls are almost common sense; others are more subtle but still important.
Additional best practices are included throughout the presentation. The complete list is
included in my recent paper in the Sept-Oct. Issue of Research Technology Management.
• A1 - Pursue three overarching objectives in portfolio management: strategic alignment, strategic
balance, and maximum return
• B5 - Show impact of project risk on future project and portfolio value
• C2 - Decision making by management is knowledge-based, transparent, and consistent
• D3 - Measure the strategic & financial value of portfolio decisions using a business case
• E3 - Do not overload the project pipeline or the people (resource projects adequately)
• F1 - Have a well-defined business strategy and communicate it to all employees clearly and often
• G1 - Portfolio governance should be clearly defined and understood
• H1 - Use a consistent PPM process, language, and tools across all levels and functions
Example: A Best Practice from each Category
12
Each participant scored all 50 Best Practices on 4
criteria. Actualization is our performance metric.
This scoring mechanism is quite simple yet produces many powerful insights when analyzed.
Relevance / Core
Contribution (1-7)
Frequency of Use
(0-100%)
Quality of Execution
(0-100%)
Scoring Mechanism
Frequency
of Use
Quality of
Execution
Actualization
(0-100%) X =
Criteria
13
Participant answers validate the best practices as
important, establish a clear performance benchmark.
• All except 5 practices are considered Core by 15 or more orgs and ten by 30+
• Every practice is considered relevant by at least 25 organizations and 16 by 42+!
• 6 of the 50 best practices have an average contribution of 6.0 or higher (on a scale
of 1-7, only including the Y and C scores) and only 6 have average contribution
below 5. This reconfirms that these best practices are a valid and powerful set.
• Average best practice actualization drops rapidly from 66% to 33%, a significant
‗range‘
• However the actualization of the top 3 organizations is usually above 80% and is
occasionally 100%! Organizations can do these practices if they want to.
• Best practices with actualizations below 45% are fertile ground to explore for
gaining competitive advantage, especially when they are considered important.
• One third scored their PPM performance against peer organizations as 6 or 7 (out
of 7), one third scored it 5, and one third scored it 4 or lower.
• There are many synergies among the best practices – they work well together
14
There are performance differences between functional and
industry subgroups, but all are well below the Top 3 benchmarks.
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Average Actualizations over all 50 Best Practices
IT Orgs Total Pop. R&D Orgs Life Science Orgs. Top 3
15
The Top 3 average actualizations are quite high, establishing a true benchmark for excellence in Portfolio Management.
These practices can be done by those who decide to do them!
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Ave
rag
e A
ctu
aliz
atio
n R
ate
Average of Top 3 performers
Average performance
Best Practices
16
C1
A1
C2 G4 H5
D3
G1
F1
H2
H4
B1
F3
C5
C3
G3
C4
B9
H3
G5
F4
D1
G2
D2 A3
D5
H1
D4
E2 A5
F2
E3
G6
E1
A2
F5
B5
B8 B2
H6
B7
B4 G7
A4
E5 B3 E4
D6 H7
B6
30%
35%
40%
45%
50%
55%
60%
65%
4.3 4.5 4.7 4.9 5.1 5.3 5.5 5.7 5.9 6.1 6.3
Actu
aliz
atio
n
Contribution to portfolio management value
Best Practice Linear (Best Practice)
Best practices that score high on both average contribution and
average actualization are ―essential for excellence.‖
D3
G1
H2
H4
B1
G3
H3
G5
D1
D2
A3
H1
50%
55%
60%
65%
5.4 5.5 5.6 5.7 5.8 5.9 6.0 6.1 A
ctu
aliz
atio
n
Contribution to portfolio management value
Best Practice
Best Practices: Contribution vs. Actualization High Contribution, High Actualization
17
F1
C3
B9
F4
D5
E2
A5
F2
E3
E1
A2
B5
30%
35%
40%
45%
5.3 5.5 5.7 5.9 6.1 A
ctu
aliz
atio
n
Contribution to Portfolio Management Value
Best Practice
C1 A1
C2
G4 H5
D3
G1
F1
H2
H4
B1
F3
C5
C3
G3
C4
B9
H3
G5
F4
D1
G2
D2 A3
D5
H1
D4
E2 A5
F2
E3
G6
E1
A2
F5
B5
B8 B2
H6
B7
B4 G7
A4
E5 B3 E4
D6 H7
B6
30%
35%
40%
45%
50%
55%
60%
65%
4.3 4.5 4.7 4.9 5.1 5.3 5.5 5.7 5.9 6.1 6.3
Actu
aliz
atio
n
Contribution to Portfolio Management Value
Best Practice Linear (Best Practice)
Best practices with high average contribution but low average
actualization are good places to seek ―competitive advantage.‖
Best Practices: Contribution vs. Actualization
High Contribution, Low Actualization
18
Summary of Key Findings
Who does portfolio management well? • Pharma / Healthcare is the best performing industry
• Organizations executing mostly R&D projects score well, whereas organizations doing
mostly IT projects do worse
What is being done well? • Use of both quantitative and qualitative measures
• Having well established / implemented decision criteria
• PPM well aligned with regular Planning & Control processes
• PPM Process. Organization and Governance
What are the main areas needing improvement? • Insufficient integration of PPM and resource management
• Portfolio not well aligned with strategy; strategy unclear
• Risk considerations/elements not reflected explicitly
• Portfolio management not aligned across tiers/functions
• Benefits management frequently not executed
The results from all Rounds are very similar.
19
20
Welcome and Introduction
Final Thoughts
Benchmarking R&D Portfolio
Management Best Practices.
Agenda
Study Overview, Significance, and Key Findings
The Most Critical Best Practices
Most critical best practices for portfolio
management success
1. Objectives:
Strategic alignment
Strategic balance
Maximize return
2. Make
excellent
portfolio
selection
decisions
3. Manage
aggregate
resources
effectively
21
Pursuing the three objectives helps achieve a holistic portfolio
selection decision, not just a financially-driven project ranking
• Strategic alignment requires a necessary and sufficient
portfolio of projects, guided by a clear strategy.
F1 — Have a well-defined business strategy and communicate it to all employees clearly and often
F3 — Translate strategic goals and gaps into necessary projects (―building-in strategic alignment‖)
F4 — Confirm that the projects in the portfolio are sufficient for the strategy to succeed
22
Strategic portfolio management aims to produce an innovative, aligned, valuable, and balanced portfolio
Project
Creation
Project Strategy
Evaluation (DA)
Portfolio
Analysis
Innovative
Aligned
Opportunities
High-
Value
Projects,
with Options
Strategy/Market/
Business Needs* Commercial
Assessments
Commercial
Sensitivities and
Dependencies
Technology
Opportunities/
Challenges
Technology
Assessments
Technology
Sensitivities and
Dependencies
Optimal,
Balanced
Portfolio
Individual
Creativity
Business and
Platform Plans,
and Strategies
Function Plans,
Strategies and
Resources
*Strategic alignment should be built-in from the start.
23
Define the Strategic Roadmap
24
Gather Ideas
25
Know Your Business Case
26
Balancing the portfolio across many dimensions, such as risk
and return, helps ensure superior portfolio decisions.
• Several of the best practices support achieving strategic
balance:
• Balance can also be examined across many other
strategic dimensions, such as timing, geography,
markets, business areas, technology capabilities etc.
B2 — Use effective visual displays to convey portfolio information (e.g. Risk-Return grid)
B5 — Show impact of project risk on future project and portfolio value.
27
Displaying two characteristics of projects—technical difficulty
and commercial potential—helps balance risk and return.
Low High Hig
h
Lo
w
Maintain
Competitiveness
Te
ch
nic
al D
ifficu
lty
(Ho
w to
ug
h is
it?
)
Gain Strategic
Advantage
Commercial Potential
(Why do it?)
28
Project Portfolio Matrix
Manage projects in different quadrants differently.
Project Portfolio Matrix
―Deliver it‖ • Manage specs
• Manage budget
• Manage schedule
―Move it‖ • Up: Simplify technology
• Over: Expand commercial scope
• Out: Re-deploy resources
―Exploit it‖ • Make time top priority
• Target best markets
• Explore full range of apps /
upgrades
―Prove it‖ • Resolve hardest hurdles first
• ID tech gaps
• Verify potential applications
29
Value maximization is very powerful, but it should not
be done in isolation from the other portfolio objectives
• You can‘t maximize value if you can‘t measure it
• There are proven methods to optimize portfolio value
• Optimizing portfolio value requires the careful
consideration of alternative project strategies
• Optimizing portfolio value in isolation can lead to a very
unaligned and unbalanced portfolio
A3 — Use a value/return measure that is aligned with shareholder value (e.g. eNPV)
A4 — Take explicit steps to maximize portfolio return (e.g. ―Efficient Frontier‖ approach)
E4 — Examine alternative strategies and resource levels to achieve project objectives
30
The optimal (“efficient frontier”) portfolio would create $2.6
billion more value for the same development investment!
Cumulative Expected Development Investment ($ millions)
Sh
are
ho
lde
r V
alu
e (
$
mill
ion
s)
$2.6 billion added value
Current
Portfolio
Expanded
Current
Reduced
Minimal
4,000
1,000
2,000
3,000
5,000
6,000
7,000
8,000
9,000
10,000
11,000
0 0 250 500
Highest
Value
Portfolio
13,000
14,000
15,000
16,000
12,000
750 1,000
With 25 projects and 100 alternatives in total there are in excess of 1015 (one
quadrillion) possible portfolios; the green points simulate a range of these.
31
Optimize the Portfolio
32
Run the Scenarios
33
Poll:
Which of these 3 practices
would you say you routinely
and explicitly use in your
portfolio management?
34
Most critical best practices for portfolio
management success
1. Objectives:
Strategic alignment
Strategic balance
Maximize return
2. Make
excellent
portfolio
selection
decisions
3. Manage
aggregate
resources
effectively
35
Since portfolio management is a decision process, it is
crucial to make excellent portfolio selection decisions.
• The portfolio governance process must be well defined
• Second the portfolio decision process and resulting recommendations
should be transparent
• Finally, to make the decision real, there must be an allocation of
resources to execute the selected projects
C1 — All stakeholders are disciplined and reliable in following the agreed PPM processes
G1 — Portfolio governance should be clearly defined and understood.
C2 — Decision-making by management is knowledge-based, transparent, and consistent
C3 — Portfolio management results in an allocation of resources to projects and programs.
C4 — Once portfolio decisions are made, they are supported by all involved parties.
36
Management behavior is hard to change, but it
can be done!
• Experience, organizational culture, and incentives drive senior management behavior
• It is impossible to change people‘s experience, since that reflects their past
• Organizational culture can be changed, but it is a long slow process demanding continuous senior executive involvement and modeling of the desired new culture
• Incentives operate in the present and can be used to impact current behavior, but are frequently too quantitative and too short term
• Incentives are the shortest route to behavior change, however, culture is the most durable
37
The HP Way was one of the most admired and
emulated corporate cultures of the 20th century
• The HP Way comprises values, objectives and strategies & practices
• The five enduring organization values of the HP Way are:
— Trust and respect for individuals
— High level of achievement and contribution
— Uncompromising integrity
— Teamwork
— Flexibility and innovation
• The seven corporate objectives (ca 1997) were: Profit, Customers,
Fields of Interest, Growth, People, Management and Citizenship, and
were guiding principles for all decision-making by HP people
• The strategies and practices consist of shared plans and actions for
working, managing and leading. Key categories are 1) open
communication, 2) MBO and 3) personal responsibility & initiative
Source: The HP Way, David Packard, Harper Business, 1995 38
Intel’s Eight Decision Making Principles reflect
their culture
• MANAGERS IDENTIFY DECISIONS THEIR ORGANIZATIONS OWN AND THE DECISION MAKERS. They are accountable for assigning the most capable/competent individual; they are accountable for developing this competency in their group.
• DECISIONS ARE OWNED BY THE INDIVIDUAL(S) ACCOUNTABLE FOR THE RESULTS. Delegate decisions to the lowest organization level possible - impact of the decision determines the level.
• EVERY DECISION HAS ONE (OR AT MOST TWO) DECISION MAKERS. Everyone else involved is either a stakeholder or a consultant to the decision maker with responsibility to agree or "disagree and commit."
• THE INDIVIDUAL ACCOUNTABLE FOR RESULTS IS EMPOWERED TO DECIDE. When you are accountable for a decision you have the authority to act - you are empowered; decision making happens where information comes together with personal accountability.
• DECISION MAKERS MUST CLEARLY DEFINE THE PROCESS, TOOLS AND ROLES. Team members know which process will be used, who the ratifiers, stakeholders, etc. are before the process begins; consensus is desirable but not mandatory - ―disagree and commit‖ when appropriate; methodical decision making processes are appropriate for high risk, complex decisions.
• DECISION MAKERS MUST BALANCE DATA WITH TIMELINESS AND JUDGMENT TO MAXIMIZE BUSINESS IMPACT. Don't wait to get ALL the data; wait until you have ENOUGH; don‘t accept ―go get another rock‖ requests for information that won't change decision outcome.
• DECISIONS ARE SUPPORTED ONCE THEY ARE MADE. Decision makers are informed if the environment or data changes; ratified decisions are not vetoed or revisited without new data.
• DECISIONS ARE CLEARLY COMMUNICATED TO THOSE AFFECTED. Final reasoning and criteria are communicated to stakeholders and consultants.
39
Another large technology organization includes management decision behaviors in its performance review process, creating an incentive.
• Results & Performance Driven — How I achieve business results
− Understands & balances the ST & LT impacts of decisions
− Makes data-driven, fact-based decisions
− Makes sound decisions which take into account broad system-wide implications
− Makes timely decisions, balancing analysis with decisiveness
− ―Sets the bar high, makes the difficult decisions‖
• Customer centric—How I deliver on customer needs
− Understand customer needs from their business & Industry perspective
− Define & develop innovative solutions to customer problems
• Business Acumen & Thought Leadership—How I apply my
knowledge to making decisions
− Understands how the business operates …
− Has a deep understanding of the business strategy …
− Drives innovation, champions change
− Provides an environment where risks can be taken to create extraordinary return
− Makes ethical decisions, with the utmost integrity
• Team Leadership and Personal Effectiveness − ―Communicates the logic behind decisions‖
40
Most critical best practices for portfolio
management success
1. Objectives:
Strategic alignment
Strategic balance
Maximize return
2. Make
excellent
portfolio
selection
decisions
3. Manage
aggregate
resources
effectively
41
Finally it is essential for success to manage
resources effectively across the portfolio.
• Anticipate / Manage bottlenecks.
• Balance Aggregate Resource Supply & Demand
• DO NOT Overload the Pipeline
E1 — Identify and monitor resource bottlenecks
B7-B10 — Identify the key bottleneck time, money, people, and materials/facilities clearly and
transparently.
E2 — Manage the balance between resource demand and resource supply (manage capacity)
C5 — Prioritization is done according to a clear set of rules.
E3 — Do not overload the project pipeline or the people (resource projects adequately). E3 has
one of the lowest average actualizations of all 50 practices, yet it is recognized as very important.
(Contribution = 5.6)
42
Manage the balance between resource demand &
resource supply
Recommendations: Demand-Side Resource Management
• Resist trying to control the detail; only model reality as far as is relevant for strategic planning (I.e. at an aggregated level)
• Focus planning and management efforts on large and/or mission critical projects (the ones that really need to get done)
• Group portfolio resources into three categories: skills, facilities and technology environment
• There are four levers to manage resources:
- Change time scales for lower priority projects to flatten demand
- De-couple development from roll-out to help flatten demand
- De-scope to reduce absolute resource needs (but beware of value erosion)
- If these are not enough, defer or drop some lower priority projects
Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11) 43
Manage the balance between resource demand &
resource supply
Recommendations: Supply-Side Resource Management
• Be realistic in assessing how many resources you actually have, given
training, holidays, vacation time, sick time etc.
• Differentiate core competencies from commodity skills
• Staff/train enough people to cover these core competency needs
• For commodity skills, develop standard role descriptions, cross-train
people and develop good external relationships
Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11) 44
Do not overload the project pipeline (resource
projects adequately).
Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11)
Requirements
• A shared understanding that overloading the pipeline degrades
productivity
• A clear understanding of what the resource supplies are (money,
people, special skills)
• Realistic estimates of available time
• An effective portfolio prioritization/decision process that leads to clear
in/out decisions
45
Do not overload the project pipeline (resource
projects adequately).
Recommendations
• Become familiar with the research and experience that proves
overloading is detrimental to productivity
• Also follow Best Practices C3, C5, G4, H5 to ensure clear prioritization
and crisp decision making (including project ―kill‖ decisions)
• This can be facilitated by software tools and by implementing the
bottleneck management Best Practices B7, B8, B9, B10 and E1
46 Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11)
Manage the balance between resource demand &
resource supply
47
Maximize Operational Efficiency
48
Track Progress
49
Gain Business Intelligence
50
51
Welcome and Introduction
Final Thoughts
Benchmarking R&D Portfolio
Management Best Practices.
Agenda
Study Overview, Significance, and Key Findings
The Most Critical Best Practices
These are the primary PPM improvement areas we saw, organized
by the frequency of occurrence and the priority of the challenge
Many organizations should focus on the challenges in the upper right.
PPM Governance
Benefits Management
Resource Management
Strategy & Strategic Alignment
Added Value of PPM—Purpose & Objectives
PPM Organization, Integration & Alignment
PPM Effectiveness and Improvement
Analytics, Reporting, and Risk Assessment
Communication and Reporting
Project Evaluation, Business Case & DQ
PPM Decision Behavior
Fewer than 10 organizations 10 or more organizations
Ave
rag
e P
rio
rity
1 to 3
4+
Frequency of Occurrence
52
Questions?
53
Recording and slides will be made available in follow-up emails
Other Questions? E-mail [email protected]
Keith Duncan, NPDP [email protected]
Michael Menke, Ph.D. [email protected]
References
1. Managing R&D Portfolios for Improved Productivity and Profitability, Michael M. Menke et al., Journal of Science Policy and Research Management, Vol. 4 (No. 4), pp. 400-412, 1989
2. Portfolio Management for New Products, Robert G. Cooper, Scott J. Edgett and Elko J Kleinschmidt, Perseus Books, 1998.
3. “How SmithKline Beecham Makes Better Resource-Allocation Decisions”, Paul Sharpe and Tom Keelin, Harvard Business Review, March-April 1998
4. The Smart Organization, David Matheson & Jim Matheson, Harvard Business School Press, 1998.
5. “Portfolio Management in an Upstream Oil & Gas Organization”, Mazen A. Skaf, INTERFACES 29: 6 November-December 1999 (pp. 84-104)
6. Business Portfolio Management, Michael Allen, John Wiley & Sons, 2000
7. Project Portfolio Management, Harvey A. Levine, Jossey-Bass, 2005
8. Optimizing Corporate Portfolio Management, Anand Sanwal, John Wiley & Sons, New York et al., 2007
9. Executing Your Strategy, Mark Morgan, Ray Levitt and William Malek, Harvard Business School Press, 2008.
10. ―Seeing the Forest As Well As the Trees: Creating Value with Portfolio Optimization,‖ Chris Reinsvold, Eric Johnson and Michael Menke, SPE 116419, 2008
11. Project Portfolio Management: A View from the Management Trenches, The EPMC, Wiley, 2009
12. ―Making R&D Portfolio Management More Effective,‖ Michael Menke, Research-Technology Management, Sept.-Oct. 2013 (pp. 34-44)
54
Singular focus on PPM
Software designed for NPD organizations
Fully integrated,
End-to-End PPM software
Built-in industry best practices
16+ years of experience
Proven methodology,
flexible delivery
Clients include 250+ innovation leaders
Benchmarking
R&D Portfolio Management
Best Practices brought to you by
55
Top Related