How to Evaluate An Investment Manager Jessica N. Portis, CFA Senior Vice President

1

How to Evaluate An Investment Manager

Jessica N. Portis, CFASenior Vice President

Summit Strategies Group8182 Maryland Avenue, 6th FloorSt. Louis, Missouri 63105

2

LiquidityInflation RiskAsset AllocationMarket VolatilityRisk Management

Manager SelectionLiability ManagementFunding Rate Volatility Government RegulationRate of Return and Funding Issues

* Greenwich Associates, Market Trends 2013, USII-13

3

PUBLIC FUND MANAGER HIRING ACTIVITY*

2010

2011

2012

2013

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Expect to HireHave Hired

* Greenwich Associates, Market Trends 2013, USII-13. “Have Hired” refers to mandates or assignments US institutions have hired for – additional or replacement – in the past 12 months. “Expect to Hire” refers to anticipated hiring for mandates or assignments in the next 12 months.

4

CLIENTS FIND IT EASY TO HIRE MANAGERS

Intellectually Challenging Exercise

“Doing”Something

Attention from Managers

It’s Fun to Go Shopping

Thrill of the Chase

Good Source ofInformation for

Personal Accounts

5

CLIENTS FIND IT DIFFICULT TO TERMINATE MANAGERS

● Admitting a mistake.

● Personal relationships established.

— Probably less today than 10 years ago.

● The crowd mentality.

● Politics— Local— Minority

● Timing

— Don’t want to sell at the bottom.

● It’s costly to switch managers.

● Manager’s reason for poor performance or personnel changes appears reasonable.

YOU’RE FIRED!

6

Fama’s Nobel Work Shows Active Managers Fated to Lose

— 10/15/2013 – Charles Stein

Active Managers Stink? Blame These All-Too-Common Fund Flaws

— 10/22/2013 – Focus on Funds by Brendan Conway

Why 'active' investment is a losing bet

— 3/23/2012 – Moneywatch by Larry Swedroe

Medieval Medicine And Active Fund Management

— 7/25/2013 – Personal Finance by Rick Ferri

The Problem for Active Managers— 4/29/2014 – Opinion by Pauline Skypala

ACTIVE MANAGEMENT MAKES FOR AN INTERESTING MEDIA TOPIC

7

-600

-400

-200

0

200

400

600

800

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Med

ian

Exce

ssRe

turn

(Net

of F

ee a

t $50

mill

ion)

Percentage of Rolling 3-Yr Observations Generating ER

US Large Cap ValueThree Year Quarterly Rolling Excess Return Consistency

Summit Top Tier StrategySummit Second Tier

ALPHA DOES EXIST; YOU JUST HAVE TO KNOW WHERE TO FIND IT

Relatively consistent andhigh excess return

Very consistent and positive excess return

Closet index strategies

Very consistent and high excess return

8

Detail Client Constraints

Qualitative Review

Defining Expectations

for the Mandate

Quantitative Review

MULTI DIMENSIONAL APPROACH TO DUE DILIGENCE PROCESS

9

CLIENT CONSTRAINTS = SEARCH CRITERIA

● Desirable Manager Attributes:

— Experience o Firm tenureo Investment team “pedigree”o Product history

— Historical Performanceo Relative to peerso Relative to benchmark

— Level of Assets Under Managemento Minimum sizeo Number of clientso Client diversity

— Preference for local, emerging or minority manager

Candidate Pool



Qualitative Review


for the Mandate

Quantitative Review

10

DEFINE EXPECTATIONS FOR THE MANDATE AND MANAGER

Alpha Expectations Macro Impact on Sub Strategies

Evaluation Period Tracking Error Budget

Integration with Existing Mandates


Qualitative Review


for the Mandate

Quantitative Review

11

WHY SETTING EXPECTATIONS IS IMPORTANT

● Helps frame your goals and objectives.

● Focus your efforts on meaningful/impactful strategies.

● Improves your odds of being “satisfied” with the manager.

● Narrows down the list of managers you have to perform in-depth due diligence.

— Makes your due diligence process more efficient.

— Tedious process that involves the review of both quantitative and qualitative aspects of firm.


Qualitative Review


for the Mandate

Quantitative Review

12

EFFICIENCY OF ASSET CLASS – ESTABLISHING REALISTIC ALPHA EXPECTATIONS

Emerging MarketsUS Large Cap Value

64%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Dec

-99

Dec

-00

Dec

-01

Dec

-02

Dec

-03

Dec

-04

Dec

-05

Dec

-06

Dec

-07

Dec

-08

Dec

-09

Dec

-10

Dec

-11

Dec

-12

Dec

-13

Perc

enta

ge o

f Uni

vers

e Be

ating

Ben

chm

ark

Annual Chance of Beating Benchmark

52%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Mar

-94

Mar

-95

Mar

-96

Mar

-97

Mar

-98

Mar

-99

Mar

-00

Mar

-01

Mar

-02

Mar

-03

Mar

-04

Mar

-05

Mar

-06

Mar

-07

Mar

-08

Mar

-09

Mar

-10

Mar

-11

Mar

-12

Mar

-13

Perc

enta

ge o

f Uni

vers

e Be

ating

Ben

chm

ark

Annual Chance of Beating Benchmark

Median Annualized Excess Return: 0.18% Median Annualized Excess Return: 1.75%

*Rolling 1-yr Quarterly 1994-201. Source Evestment Alliance, Large Value - 733 Strategies/ Emerging Markets - 497 Strategies (active & inactive)


Qualitative Review


for the Mandate

Quantitative Review

13

UNDERSTANDING WHERE YOU ARE IN THE ECONOMIC CYCLE AND THE ALPHA POTENTIAL FOR SUB STRATEGIES

Annualized ER vs. R1kV (2000-2013) Tracking Error

Deep Value 2.5% 7%Defensive 2.4% 8%Relative Value 1.7% 4%Dividend 1.0% 5%Closet Index 0.9% 3%


for the Mandate

Large Cap Value Manager Sub Strategy Performance


Qualitative Review


for the Mandate

Quantitative Review

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013Deep Value

17.3%Deep Value

8.6%Russell 1000 V

22.2%Yield / Income

5.4%Defensive -18.1%

Deep Value 41.5%

Deep Value 16.7%

Defensive 7.0%

Russell 1000 V 17.5%

Deep Value 36.9%


Closet Index 8.4%

Yield / Income 20.9%

Relative Value 5.2%

Yield / Income -30.8%

Relative Value 25.9%

Defensive 15.7%

Yield / Income 6.3%

Closet Index 17.1%

Closet Index 34.2%


Relative Value 8.4%

Closet Index 19.6%

Closet Index 4.0%

Relative Value -33.1%

Closet Index 22.7%


Relative Value 1.0%

Deep Value 15.9%


Closet Index 15.6%

Russell 1000 V 7.1%

Deep Value 18.8%

Defensive 2.2%

Closet Index -35.1%



Closet Index 0.6%



Defensive 14.4%

Yield / Income 6.5%


Deep Value 1.6%

Russell 1000 V -36.8%



Russell 1000 V 0.4%




Defensive 3.4%

Defensive 16.6%

Russell 1000 V -0.2%

Deep Value -40.8%

Defensive 18.5%

Closet Index 14.1%

Deep Value -3.2%

Defensive 11.9%

Defensive 25.5%

14

IMPACT OF TRACKING ERROR ON RESULTS: WHAT ARE YOU COMFORTABLE WITH?

● Observation: Historically, strategies utilizing a high-conviction investment styles performed best within the US Large Cap Value market segment.

● Takeaway: Align your alpha expectations with your risk tolerance.

Risk Tolerance: Relationship Between Excess Return and Tracking Error.

Relationship between excess return and tracking error. 27,889 rolling 3-year periods for 723 US large cap value strategies 1994-2014. Source: Evestment Alliance.

1-3% 3-5% 5-10% 10%+

-0.5%

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

Tracking Error

Med

ian

Exce

ss R

etur

n


Qualitative Review


for the Mandate

Quantitative Review

15

PERFORMANCE EVALUATION DILEMMA

Bottom Quartile

Bottom Decile

0% 10% 20% 30% 40% 50% 60% 70%

% Large Cap Value Managers That Ranked Low for At Least One 3-Year Period in Prior

10 Years

● Good managers sometimes appear “bad”.

● The gut reaction is to terminate poor-performing managers.

● What makes a poor manager?

● How to evaluate track record?


Qualitative Review


for the Mandate

Quantitative Review

16

HOW TO EVALUATE TRACK RECORDS

● Avoid simply relying on recent performance to evaluate a manager – it can be dangerous.

● Incorporate elements of risk and return.— Standard Deviation— Information Ratio— Tracking Error— Batting Average— Up/Down Market Capture

● Rolling period evaluation is much more telling than a single point in time.— Consistency of results.— Helps isolate luck versus skill.— Not constrained by point in time comparisons.— Assess the effect of macro economic conditions on results.


Qualitative Review


for the Mandate

Quantitative Review

17

A CASE STUDY

● Question: Should this manager be terminated?

● Answer: It Depends! Performance only captures one dimension. Need to qualitatively understand the manager to better assess future prospects for success.

1-Year 3-Year 5-Year0%

5%

10%

15%

20%

25%

30%

Manager Index

Trailed 8 of last 12 quarters with no positive alpha

Trailed 13 of last 20 quarters with no positive alpha

Annualized Results Ending 9/30/2007

Mar

-01

Jul-0

1N

ov-0

1M

ar-0

2Ju

l-02

Nov

-02

Mar

-03

Jul-0

3N

ov-0

3M

ar-0

4Ju

l-04

Nov

-04

Mar

-05

Jul-0

5N

ov-0

5M

ar-0

6Ju

l-06

Nov

-06

Mar

-07

Jul-0

7

Manager Index25%

Median

75%

100%

0%One-Year Rolling/Quarterly Percentile Ranking


Qualitative Review


for the Mandate

Quantitative Review

18

MANAGER AVOIDANCE ≥ MANAGER SELECTIONPe

rson

nel I

ssue

sProduct Assessment

Philosophy & Process

Firm

Spec

ific I

ssue

s

Due Diligence Is Focused on Risk Management.

“I’d rather be roughly right than precisely wrong.” - John Meynard Keynes


Qualitative Review


for the Mandate

Quantitative Review

19

QUALITATIVE ASSESSMENT: FIRM

FIRM

Ownership

Leadership

Compensation

Litigation

Product

Integration

Business

Mgmt.

Operational

Infrastru

cture

“I don’t think culture is something you can describe.” – Bill Gates


Qualitative Review


for the Mandate

Quantitative Review

20

QUALITATIVE ASSESSMENT: PRODUCT

AUM

StrategyHistory

Client Type

Diversity

CapacityManagement

Culture

Star vs.

Team

PRODUCT

ASSESSMENT

“I don’t want a lot of good investments; I want a few outstanding ones.” – Philip Fisher


Qualitative Review


for the Mandate

Quantitative Review

21

QUALITATIVE ASSESSMENT: PERSONNEL

Team

Pedigree

Team

Stability

Add’ lTeam

Resources

TeamExperience

TeamSize &

Depth

PERSONNEL

“Coming together is a beginning. Keeping together is progress. Working together is success.” – Henry Ford


Qualitative Review


for the Mandate

Quantitative Review

22

QUALITATIVE ASSESSMENT: PHILOSOPHY AND PROCESSPh

iloso

phy

Investment

Process

Performance

Drivers

Trading

Portfolio

Characteris

tics

Decision Process

Risk Cultu

re

PHILOSOPHY &

PROCESS

“If you don’t have a competitive advantage, don’t compete.” -Jack Welch


Qualitative Review


for the Mandate

Quantitative Review

23

TRIGGERS FOR A FORMAL REVIEW OF A MANAGER

● Firm – Material ownership change or regulatory concern.

● Product – Change in assets under management, style drift, or dilution/orphan of product.

● Personnel – Departure of key investment professional or change in team structure and decision process.

● Philosophy and Process – Change in investment philosophy, shift in market environment not conducive to style, or relaxation of risk controls.

● Performance – Three-year excess return outside of original investment thesis or deterioration in quantitative criteria – information ratio, tracking error, standard deviation, batting average, up-/down- capture ratio, etc.

● Strategic Fit in Client Portfolio – Does the strategy still make sense in terms of your needs?

24

REVISITING THE CASE STUDY

25

Five-Year Rolling/Quarterly Percentile Ranking

Growth of $1.00

Universe: International Value Equity (Separate Accounts & Commingled Funds)

CASE STUDY: THE COMPLETE PICTURE A high conviction manager underperforms badly: do you stay or go?

— Was the firm sold? No.— Had the AUM changed dramatically? No.— Did the team experience turnover? No.— Did the investment process change? No.— Were the markets rewarding their

approach? No! Quality and valuation were out of favor for several years.

26

CONCLUSION

● Clearly define the role the manager and strategy play in the overall portfolio.

● Understand both the opportunities and threats associated with a manager prior to investing.

● Define performance expectations for each manager.— What does the historical pattern of returns for a manager look like?— Understand which strategies work best in the asset class and when.

● When issues arise, investigate thoroughly and compare to documented expectations.

● Act quickly when determining that a termination needs to occur.

QUANTITATIVE REVIEW + QUALITATIVE REVIEW + ONGOING MONITORING = SUCCESS

27

QUESTIONS AND ANSWERS

Disclaimer: Summit Strategies Group (Summit) has prepared this report for the exclusive use by the client for which it was prepared. The information herein was obtained from various sources, such as the client’s custodian(s) accounting statements, commercially available databases, and other economic and financial market data sources. While Summit believes these sources to be reliable, Summit does not guarantee nor shall be liable for the market values, returns or other information contained in this report. The market commentary, portfolio holdings and characteristics are as of the date shown and are subject to change. Past performance is not an indication of future performance. No graph, chart, or formula can, in and of itself, be used to determine which securities or investments to buy or sell. Any forward-looking projection contained herein is based on assumptions that Summit believes may be reasonable, but are subject to a wide range of risks, uncertainties and the possibility of loss. Accordingly, there is no assurance that any estimated performance figures will occur in the amounts and during the periods indicated, or at all. Actual results and performance will differ from those expressed or implied by such forward-looking projections. Any information contained in this report is for information purposes only and should not be construed to be an offer to buy or sell any securities, investment consulting or investment management services.

How to Evaluate An Investment Manager Jessica N. Portis, CFA Senior Vice President

Documents

Transcript of How to Evaluate An Investment Manager Jessica N. Portis, CFA Senior Vice President