together with

Benchmarking R&D Portfolio

Management Best Practices

Michael Menke & Keith Duncan

November 14th, 2013

© 2013 Planisware

Introductions

Keith Duncan, NPDP Host & Presenter

Director, Innovation & PPM Practices

Planisware

Michael Menke, Ph.D. Featured Presenter

President, Value Creation Associates

Fellow, Society of Decision

Professionals

2

3

Welcome and Introduction

Study Overview, Significance, and Key Findings

The Most Critical Best Practices

Final Thoughts

Benchmarking R&D Portfolio

Management Best Practices.

Agenda

Poll:

Have you formally

benchmarked your

portfolio management

practices and process?

4

5

Welcome and Introduction

Final Thoughts

Benchmarking R&D Portfolio

Management Best Practices.

Agenda

Study Overview, Significance, and Key Findings

The Most Critical Best Practices

Portfolio Management is a decision process that is

clearly associated with value creation in Pharma.

*This includes client organizations who have fully implemented a value-based approach

to resource allocation and portfolio management. (Timeframe mid-1990s to early 2000s.)

Pharma Client* Stock Price Performance Relative to Dow pharma and S&P 500 indices

-50%

0%

50%

100%

150%

200%

250%

300%

350%

400%

450%

Year 0 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7

S&P 500

Index

Dow Jones

Pharma

Index

Portfolio

Adopters

Index

6

Good decisions begin with a conducive culture, effective

behaviors, a DQ framework, and a good decision process.

The PPM Accelerate benchmarking study focuses on behaviors,

framework and process to identify PPM best practices.

Culture & Incentives

Decision Behaviors

Decision Quality (DQ) Framework

Decision Process

DQ Methodology & Tools

7

Poll:

Does your organization

have a formal decision

quality process and use

tools and frameworks to

improve your decision

making?

8

This Framework applies to Portfolio Decisions.

• Although culture and incentives are very important, they are very

hard to change

• It is easier to study and to change decision behaviors and

practices, DQ framework and decision process

• Since 2011, Value Creation Associates has been co-sponsor of a

global Project Portfolio Management (PPM) best-practices

benchmarking study—PPM Accelerate

• The objective has been to establish importance, execution quality

and performance benchmarks for 50 PPM best practices

• In order to establish a standard of excellence (―benchmark‖), we

deliberately included a number of PPM leaders

• We now have data from over 70 organizations, many PPM leaders,

and are continuing to enroll new organizations

9

The following organizations are included in PPM Accelerate

Rounds 1-3, with a current total of over 70 participants.

They represent a wide range of industries and functions. The study is on-going.

Dow

AgroSciences

Dow Advanced

Materials

10

Resource

Information

Analytics,

Reporting &

Risk Assessment

Behavior

Financial

Information

Strategic

Value

Information

PPM Organization

& Governance

PPM

Processes

PPM Accelerate used this framework to organize

and assess the 50 Best Practices.

The 50 Best Practices

are organized into 8

categories:

Best practices and Pitfalls are

based on 30+ years of

practical experience and

extensive literature research.

Most of them had been

discovered and validated in

prior benchmarking studies.

A

B C

D E F

G H

11

Some best practices and pitfalls are almost common sense; others are more subtle but still important.

Additional best practices are included throughout the presentation. The complete list is

included in my recent paper in the Sept-Oct. Issue of Research Technology Management.

• A1 - Pursue three overarching objectives in portfolio management: strategic alignment, strategic

balance, and maximum return

• B5 - Show impact of project risk on future project and portfolio value

• C2 - Decision making by management is knowledge-based, transparent, and consistent

• D3 - Measure the strategic & financial value of portfolio decisions using a business case

• E3 - Do not overload the project pipeline or the people (resource projects adequately)

• F1 - Have a well-defined business strategy and communicate it to all employees clearly and often

• G1 - Portfolio governance should be clearly defined and understood

• H1 - Use a consistent PPM process, language, and tools across all levels and functions

Example: A Best Practice from each Category

12

Each participant scored all 50 Best Practices on 4

criteria. Actualization is our performance metric.

This scoring mechanism is quite simple yet produces many powerful insights when analyzed.

Relevance / Core

Contribution (1-7)

Frequency of Use

(0-100%)

Quality of Execution

(0-100%)

Scoring Mechanism

Frequency

of Use

Quality of

Execution

Actualization

(0-100%) X =

Criteria

13

Participant answers validate the best practices as

important, establish a clear performance benchmark.

• All except 5 practices are considered Core by 15 or more orgs and ten by 30+

• Every practice is considered relevant by at least 25 organizations and 16 by 42+!

• 6 of the 50 best practices have an average contribution of 6.0 or higher (on a scale

of 1-7, only including the Y and C scores) and only 6 have average contribution

below 5. This reconfirms that these best practices are a valid and powerful set.

• Average best practice actualization drops rapidly from 66% to 33%, a significant

‗range‘

• However the actualization of the top 3 organizations is usually above 80% and is

occasionally 100%! Organizations can do these practices if they want to.

• Best practices with actualizations below 45% are fertile ground to explore for

gaining competitive advantage, especially when they are considered important.

• One third scored their PPM performance against peer organizations as 6 or 7 (out

of 7), one third scored it 5, and one third scored it 4 or lower.

• There are many synergies among the best practices – they work well together

14

There are performance differences between functional and

industry subgroups, but all are well below the Top 3 benchmarks.

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Average Actualizations over all 50 Best Practices

IT Orgs Total Pop. R&D Orgs Life Science Orgs. Top 3

15

The Top 3 average actualizations are quite high, establishing a true benchmark for excellence in Portfolio Management.

These practices can be done by those who decide to do them!

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Ave

rag

e A

ctu

aliz

atio

n R

ate

Average of Top 3 performers

Average performance

Best Practices

16

C1

A1

C2 G4 H5

D3

G1

F1

H2

H4

B1

F3

C5

C3

G3

C4

B9

H3

G5

F4

D1

G2

D2 A3

D5

H1

D4

E2 A5

F2

E3

G6

E1

A2

F5

B5

B8 B2

H6

B7

B4 G7

A4

E5 B3 E4

D6 H7

B6

30%

35%

40%

45%

50%

55%

60%

65%

4.3 4.5 4.7 4.9 5.1 5.3 5.5 5.7 5.9 6.1 6.3

Actu

aliz

atio

n

Contribution to portfolio management value

Best Practice Linear (Best Practice)

Best practices that score high on both average contribution and

average actualization are ―essential for excellence.‖

D3

G1

H2

H4

B1

G3

H3

G5

D1

D2

A3

H1

50%

55%

60%

65%

5.4 5.5 5.6 5.7 5.8 5.9 6.0 6.1 A

ctu

aliz

atio

n

Contribution to portfolio management value

Best Practice

Best Practices: Contribution vs. Actualization High Contribution, High Actualization

17

F1

C3

B9

F4

D5

E2

A5

F2

E3

E1

A2

B5

30%

35%

40%

45%

5.3 5.5 5.7 5.9 6.1 A

ctu

aliz

atio

n

Contribution to Portfolio Management Value

Best Practice

C1 A1

C2

G4 H5

D3

G1

F1

H2

H4

B1

F3

C5

C3

G3

C4

B9

H3

G5

F4

D1

G2

D2 A3

D5

H1

D4

E2 A5

F2

E3

G6

E1

A2

F5

B5

B8 B2

H6

B7

B4 G7

A4

E5 B3 E4

D6 H7

B6

30%

35%

40%

45%

50%

55%

60%

65%

4.3 4.5 4.7 4.9 5.1 5.3 5.5 5.7 5.9 6.1 6.3

Actu

aliz

atio

n

Contribution to Portfolio Management Value

Best Practice Linear (Best Practice)

Best practices with high average contribution but low average

actualization are good places to seek ―competitive advantage.‖

Best Practices: Contribution vs. Actualization

High Contribution, Low Actualization

18

Summary of Key Findings

Who does portfolio management well? • Pharma / Healthcare is the best performing industry

• Organizations executing mostly R&D projects score well, whereas organizations doing

mostly IT projects do worse

What is being done well? • Use of both quantitative and qualitative measures

• Having well established / implemented decision criteria

• PPM well aligned with regular Planning & Control processes

• PPM Process. Organization and Governance

What are the main areas needing improvement? • Insufficient integration of PPM and resource management

• Portfolio not well aligned with strategy; strategy unclear

• Risk considerations/elements not reflected explicitly

• Portfolio management not aligned across tiers/functions

• Benefits management frequently not executed

The results from all Rounds are very similar.

19

20

Welcome and Introduction

Final Thoughts

Benchmarking R&D Portfolio

Management Best Practices.

Agenda

Study Overview, Significance, and Key Findings

The Most Critical Best Practices

Most critical best practices for portfolio

management success

1. Objectives:

Strategic alignment

Strategic balance

Maximize return

2. Make

excellent

portfolio

selection

decisions

3. Manage

aggregate

resources

effectively

21

Pursuing the three objectives helps achieve a holistic portfolio

selection decision, not just a financially-driven project ranking

• Strategic alignment requires a necessary and sufficient

portfolio of projects, guided by a clear strategy.

F1 — Have a well-defined business strategy and communicate it to all employees clearly and often

F3 — Translate strategic goals and gaps into necessary projects (―building-in strategic alignment‖)

F4 — Confirm that the projects in the portfolio are sufficient for the strategy to succeed

22

Strategic portfolio management aims to produce an innovative, aligned, valuable, and balanced portfolio

Project

Creation

Project Strategy

Evaluation (DA)

Portfolio

Analysis

Innovative

Aligned

Opportunities

High-

Value

Projects,

with Options

Strategy/Market/

Business Needs* Commercial

Assessments

Commercial

Sensitivities and

Dependencies

Technology

Opportunities/

Challenges

Technology

Assessments

Technology

Sensitivities and

Dependencies

Optimal,

Balanced

Portfolio

Individual

Creativity

Business and

Platform Plans,

and Strategies

Function Plans,

Strategies and

Resources

*Strategic alignment should be built-in from the start.

23

Define the Strategic Roadmap

24

Gather Ideas

25

Know Your Business Case

26

Balancing the portfolio across many dimensions, such as risk

and return, helps ensure superior portfolio decisions.

• Several of the best practices support achieving strategic

balance:

• Balance can also be examined across many other

strategic dimensions, such as timing, geography,

markets, business areas, technology capabilities etc.

B2 — Use effective visual displays to convey portfolio information (e.g. Risk-Return grid)

B5 — Show impact of project risk on future project and portfolio value.

27

Displaying two characteristics of projects—technical difficulty

and commercial potential—helps balance risk and return.

Low High Hig

h

Lo

w

Maintain

Competitiveness

Te

ch

nic

al D

ifficu

lty

(Ho

w to

ug

h is

it?

)

Gain Strategic

Advantage

Commercial Potential

(Why do it?)

28

Project Portfolio Matrix

Manage projects in different quadrants differently.

Project Portfolio Matrix

―Deliver it‖ • Manage specs

• Manage budget

• Manage schedule

―Move it‖ • Up: Simplify technology

• Over: Expand commercial scope

• Out: Re-deploy resources

―Exploit it‖ • Make time top priority

• Target best markets

• Explore full range of apps /

upgrades

―Prove it‖ • Resolve hardest hurdles first

• ID tech gaps

• Verify potential applications

29

Value maximization is very powerful, but it should not

be done in isolation from the other portfolio objectives

• You can‘t maximize value if you can‘t measure it

• There are proven methods to optimize portfolio value

• Optimizing portfolio value requires the careful

consideration of alternative project strategies

• Optimizing portfolio value in isolation can lead to a very

unaligned and unbalanced portfolio

A3 — Use a value/return measure that is aligned with shareholder value (e.g. eNPV)

A4 — Take explicit steps to maximize portfolio return (e.g. ―Efficient Frontier‖ approach)

E4 — Examine alternative strategies and resource levels to achieve project objectives

30

The optimal (“efficient frontier”) portfolio would create $2.6

billion more value for the same development investment!

Cumulative Expected Development Investment ($ millions)

Sh

are

ho

lde

r V

alu

e (

$

mill

ion

s)

$2.6 billion added value

Current

Portfolio

Expanded

Current

Reduced

Minimal

4,000

1,000

2,000

3,000

5,000

6,000

7,000

8,000

9,000

10,000

11,000

0 0 250 500

Highest

Value

Portfolio

13,000

14,000

15,000

16,000

12,000

750 1,000

With 25 projects and 100 alternatives in total there are in excess of 1015 (one

quadrillion) possible portfolios; the green points simulate a range of these.

31

Optimize the Portfolio

32

Run the Scenarios

33

Poll:

Which of these 3 practices

would you say you routinely

and explicitly use in your

portfolio management?

34

Most critical best practices for portfolio

management success

1. Objectives:

Strategic alignment

Strategic balance

Maximize return

2. Make

excellent

portfolio

selection

decisions

3. Manage

aggregate

resources

effectively

35

Since portfolio management is a decision process, it is

crucial to make excellent portfolio selection decisions.

• The portfolio governance process must be well defined

• Second the portfolio decision process and resulting recommendations

should be transparent

• Finally, to make the decision real, there must be an allocation of

resources to execute the selected projects

C1 — All stakeholders are disciplined and reliable in following the agreed PPM processes

G1 — Portfolio governance should be clearly defined and understood.

C2 — Decision-making by management is knowledge-based, transparent, and consistent

C3 — Portfolio management results in an allocation of resources to projects and programs.

C4 — Once portfolio decisions are made, they are supported by all involved parties.

36

Management behavior is hard to change, but it

can be done!

• Experience, organizational culture, and incentives drive senior management behavior

• It is impossible to change people‘s experience, since that reflects their past

• Organizational culture can be changed, but it is a long slow process demanding continuous senior executive involvement and modeling of the desired new culture

• Incentives operate in the present and can be used to impact current behavior, but are frequently too quantitative and too short term

• Incentives are the shortest route to behavior change, however, culture is the most durable

37

The HP Way was one of the most admired and

emulated corporate cultures of the 20th century

• The HP Way comprises values, objectives and strategies & practices

• The five enduring organization values of the HP Way are:

— Trust and respect for individuals

— High level of achievement and contribution

— Uncompromising integrity

— Teamwork

— Flexibility and innovation

• The seven corporate objectives (ca 1997) were: Profit, Customers,

Fields of Interest, Growth, People, Management and Citizenship, and

were guiding principles for all decision-making by HP people

• The strategies and practices consist of shared plans and actions for

working, managing and leading. Key categories are 1) open

communication, 2) MBO and 3) personal responsibility & initiative

Source: The HP Way, David Packard, Harper Business, 1995 38

Intel’s Eight Decision Making Principles reflect

their culture

• MANAGERS IDENTIFY DECISIONS THEIR ORGANIZATIONS OWN AND THE DECISION MAKERS. They are accountable for assigning the most capable/competent individual; they are accountable for developing this competency in their group.

• DECISIONS ARE OWNED BY THE INDIVIDUAL(S) ACCOUNTABLE FOR THE RESULTS. Delegate decisions to the lowest organization level possible - impact of the decision determines the level.

• EVERY DECISION HAS ONE (OR AT MOST TWO) DECISION MAKERS. Everyone else involved is either a stakeholder or a consultant to the decision maker with responsibility to agree or "disagree and commit."

• THE INDIVIDUAL ACCOUNTABLE FOR RESULTS IS EMPOWERED TO DECIDE. When you are accountable for a decision you have the authority to act - you are empowered; decision making happens where information comes together with personal accountability.

• DECISION MAKERS MUST CLEARLY DEFINE THE PROCESS, TOOLS AND ROLES. Team members know which process will be used, who the ratifiers, stakeholders, etc. are before the process begins; consensus is desirable but not mandatory - ―disagree and commit‖ when appropriate; methodical decision making processes are appropriate for high risk, complex decisions.

• DECISION MAKERS MUST BALANCE DATA WITH TIMELINESS AND JUDGMENT TO MAXIMIZE BUSINESS IMPACT. Don't wait to get ALL the data; wait until you have ENOUGH; don‘t accept ―go get another rock‖ requests for information that won't change decision outcome.

• DECISIONS ARE SUPPORTED ONCE THEY ARE MADE. Decision makers are informed if the environment or data changes; ratified decisions are not vetoed or revisited without new data.

• DECISIONS ARE CLEARLY COMMUNICATED TO THOSE AFFECTED. Final reasoning and criteria are communicated to stakeholders and consultants.

39

Another large technology organization includes management decision behaviors in its performance review process, creating an incentive.

• Results & Performance Driven — How I achieve business results

− Understands & balances the ST & LT impacts of decisions

− Makes data-driven, fact-based decisions

− Makes sound decisions which take into account broad system-wide implications

− Makes timely decisions, balancing analysis with decisiveness

− ―Sets the bar high, makes the difficult decisions‖

• Customer centric—How I deliver on customer needs

− Understand customer needs from their business & Industry perspective

− Define & develop innovative solutions to customer problems

• Business Acumen & Thought Leadership—How I apply my

knowledge to making decisions

− Understands how the business operates …

− Has a deep understanding of the business strategy …

− Drives innovation, champions change

− Provides an environment where risks can be taken to create extraordinary return

− Makes ethical decisions, with the utmost integrity

• Team Leadership and Personal Effectiveness − ―Communicates the logic behind decisions‖

40

Most critical best practices for portfolio

management success

1. Objectives:

Strategic alignment

Strategic balance

Maximize return

2. Make

excellent

portfolio

selection

decisions

3. Manage

aggregate

resources

effectively

41

Finally it is essential for success to manage

resources effectively across the portfolio.

• Anticipate / Manage bottlenecks.

• Balance Aggregate Resource Supply & Demand

• DO NOT Overload the Pipeline

E1 — Identify and monitor resource bottlenecks

B7-B10 — Identify the key bottleneck time, money, people, and materials/facilities clearly and

transparently.

E2 — Manage the balance between resource demand and resource supply (manage capacity)

C5 — Prioritization is done according to a clear set of rules.

E3 — Do not overload the project pipeline or the people (resource projects adequately). E3 has

one of the lowest average actualizations of all 50 practices, yet it is recognized as very important.

(Contribution = 5.6)

42

Manage the balance between resource demand &

resource supply

Recommendations: Demand-Side Resource Management

• Resist trying to control the detail; only model reality as far as is relevant for strategic planning (I.e. at an aggregated level)

• Focus planning and management efforts on large and/or mission critical projects (the ones that really need to get done)

• Group portfolio resources into three categories: skills, facilities and technology environment

• There are four levers to manage resources:

- Change time scales for lower priority projects to flatten demand

- De-couple development from roll-out to help flatten demand

- De-scope to reduce absolute resource needs (but beware of value erosion)

- If these are not enough, defer or drop some lower priority projects

Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11) 43

Manage the balance between resource demand &

resource supply

Recommendations: Supply-Side Resource Management

• Be realistic in assessing how many resources you actually have, given

training, holidays, vacation time, sick time etc.

• Differentiate core competencies from commodity skills

• Staff/train enough people to cover these core competency needs

• For commodity skills, develop standard role descriptions, cross-train

people and develop good external relationships

Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11) 44

Do not overload the project pipeline (resource

projects adequately).

Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11)

Requirements

• A shared understanding that overloading the pipeline degrades

productivity

• A clear understanding of what the resource supplies are (money,

people, special skills)

• Realistic estimates of available time

• An effective portfolio prioritization/decision process that leads to clear

in/out decisions

45

Do not overload the project pipeline (resource

projects adequately).

Recommendations

• Become familiar with the research and experience that proves

overloading is detrimental to productivity

• Also follow Best Practices C3, C5, G4, H5 to ensure clear prioritization

and crisp decision making (including project ―kill‖ decisions)

• This can be facilitated by software tools and by implementing the

bottleneck management Best Practices B7, B8, B9, B10 and E1

46 Source: Project Portfolio Management: A View From the Management Trenches (Ref. 11)

Manage the balance between resource demand &

resource supply

47

Maximize Operational Efficiency

48

Track Progress

49

Gain Business Intelligence

50

51

Welcome and Introduction

Final Thoughts

Benchmarking R&D Portfolio

Management Best Practices.

Agenda

Study Overview, Significance, and Key Findings

The Most Critical Best Practices

These are the primary PPM improvement areas we saw, organized

by the frequency of occurrence and the priority of the challenge

Many organizations should focus on the challenges in the upper right.

PPM Governance

Benefits Management

Resource Management

Strategy & Strategic Alignment

Added Value of PPM—Purpose & Objectives

PPM Organization, Integration & Alignment

PPM Effectiveness and Improvement

Analytics, Reporting, and Risk Assessment

Communication and Reporting

Project Evaluation, Business Case & DQ

PPM Decision Behavior

Fewer than 10 organizations 10 or more organizations

Ave

rag

e P

rio

rity

1 to 3

4+

Frequency of Occurrence

52

Questions?

53

Recording and slides will be made available in follow-up emails

Other Questions? E-mail webinar@planisware.com

Keith Duncan, NPDP keith.duncan@planisware.com

Michael Menke, Ph.D. mmenke@value-creation.com

References

1. Managing R&D Portfolios for Improved Productivity and Profitability, Michael M. Menke et al., Journal of Science Policy and Research Management, Vol. 4 (No. 4), pp. 400-412, 1989

2. Portfolio Management for New Products, Robert G. Cooper, Scott J. Edgett and Elko J Kleinschmidt, Perseus Books, 1998.

3. “How SmithKline Beecham Makes Better Resource-Allocation Decisions”, Paul Sharpe and Tom Keelin, Harvard Business Review, March-April 1998

4. The Smart Organization, David Matheson & Jim Matheson, Harvard Business School Press, 1998.

5. “Portfolio Management in an Upstream Oil & Gas Organization”, Mazen A. Skaf, INTERFACES 29: 6 November-December 1999 (pp. 84-104)

6. Business Portfolio Management, Michael Allen, John Wiley & Sons, 2000

7. Project Portfolio Management, Harvey A. Levine, Jossey-Bass, 2005

8. Optimizing Corporate Portfolio Management, Anand Sanwal, John Wiley & Sons, New York et al., 2007

9. Executing Your Strategy, Mark Morgan, Ray Levitt and William Malek, Harvard Business School Press, 2008.

10. ―Seeing the Forest As Well As the Trees: Creating Value with Portfolio Optimization,‖ Chris Reinsvold, Eric Johnson and Michael Menke, SPE 116419, 2008

11. Project Portfolio Management: A View from the Management Trenches, The EPMC, Wiley, 2009

12. ―Making R&D Portfolio Management More Effective,‖ Michael Menke, Research-Technology Management, Sept.-Oct. 2013 (pp. 34-44)

54

Singular focus on PPM

Software designed for NPD organizations

Fully integrated,

End-to-End PPM software

Built-in industry best practices

16+ years of experience

Proven methodology,

flexible delivery

Clients include 250+ innovation leaders

Benchmarking

R&D Portfolio Management

Best Practices brought to you by

55