Download - Fi een Years of De ned Contributions: Assessing the ......women, the corresponding ﬁgure is 36.3% (see [10]). In contrast, the equiva-lent average values for OECD countries are 62.9%

www.clapesuc.cl

Fifteen Years of Defined Contributions: Assessing the Chilean Pension Experience

Documento de Trabajo Nº 43

Fifteen Years of Defined Contributions: Assessing

the Chilean Pension Experience

Hans Schlechter

Pontificia Universidad Catolica de Chile

Santiago, CHILE

Bernardo K. Pagnoncelli

Universidad Adolfo Ibanez

Santiago, CHILE

Arturo Cifuentes

CLAPES UC

Santiago, CHILE

and

Columbia University

New York, USA

March 2018

Abstract

In 1980 Chile switched from a state-managed defined-benefit pension system to

a defined-contribution scheme based on individual capital accounts. The new

system was further refined in 2002 with the introduction of five investment funds,

with, allegedly, di↵erent risk-return profiles. The funds di↵er in their portfolio

composition which is driven by strict minimum and maximum limits (mostly

related to stocks and bonds), dictated by the regulator. We have examined the

performance of these funds over a fifteen-year period looking at their returns

and actual risk profiles, aided by three rank-order metrics. Unfortunately, our

results are unambiguously distressing: while the regulator succeeded in creating

five funds with clearly di↵erent risk profiles, their risk-adjusted returns as well

as their cumulative (absolute) returns are completely at odds with the desired

goal. In fact, during long stretches of time the funds exhibited a performance

that was exactly the opposite of what it was intended: an indictment on the

idea of controlling portfolio risk via asset allocation limits.

1 Introduction

The early social security system in Chile started in the 1920s and it was designed

to provide retirement benefits for the elderly, as well as other social benefits.

Under this system, di↵erent pension schemes were developed to attend the needs

of the di↵erent occupational groups in the country. By the 1970s, these schemes

had resulted in significant disparities in terms of the benefits received by each

of these groups. The system was based on a pay-as-you-go (PAYG) structure,

where active workers financed the pensions of the retirees. And pension obliga-

tions were met through withdrawals from the stock of accumulated savings, as

well as from the returns provided by those savings.

During the 1980s, the ine�ciencies associated with this arrangement, plus

doubts over its long-term financial feasibility, pushed the government to reform

the social security system. And in November 1980 a law introducing a new

defined contribution (DC) pension scheme based on individual capital accounts

managed by private institutions, was approved. The new system had two main

objectives. First, it established a clear link between the savings the worker had

accumulated during his active life and his pension. And second, it aimed at pro-

viding the future retiree a stable income with a high replacement rate1. Under

this arrangement, the workers’ monthly contributions are deposited in individual

(segregated) accounts and are managed by private institutions known in Chile

as AFPs (a Spanish acronym derived from their o�cial name, Administradoras

de Fondos de Pensiones). The AFPs are regulated by the Superintendencia de

Pensiones (SP), which dictates the guidelines that the AFPs must adhere to,

when investing the funds of the future retirees, also known as a�liates (afilia-

dos). Ideally, the savings deposited in these capital accounts, plus their accrued

earnings, should be su�cient to provide adequate long-term pension benefits to

the future retirees.1The replacement rate is the ratio obtained by dividing the retiree’s monthly income by a

representative value of his last years’ monthly earnings.

1

In August 2002 the pension system was further modified with the intro-

duction of five funds (multifunds, or multifondos in Spanish), known as A, B,

C, D and E. Fund A was supposed to be the riskiest, and Fund E the most

conservative. These funds were supposed to deliver long-term returns commen-

surate with their respective risk profiles. From the regulator’s perspective, the

rationale behind the multifund system was to o↵er the future retirees a rea-

sonable variety of risk-return investment options, in a setting simpler than the

full complexity of the stock and fixed income markets. The thought was that

younger workers could benefit from taking more risk, and therefore achieving

higher returns, while workers close to their retirement age should move grad-

ually to more conservative options, represented by Fund E. All in all, it was

hoped that by providing di↵erent investment options that could match di↵erent

investment horizons and risk preferences, the system could o↵er the a�liates a

chance to obtain a better pension.

Recently, critics of the current pension system have focused on the low

replacement rate attained by many retirees; the lack of competition (and high

concentration) in the AFP industry; the gap between the pensions received by

civilians and military personnel; the di↵erence in pensions between men and

women; and the fact that until 2008 most independent workers failed to make

regular contributions to their future pensions (since January 2012 it is manda-

tory for freelance workers to make monthly contributions to their retirement

accounts). Furthermore, the consequences of having inadequate pensions have

been one of the main topics of discussion in Chile as the country exhibits one

of the lowest replacement rates among the OECD members. In Chile, pensions

for men are equivalent to 40.1% of their pre-retirement earnings; in the case of

women, the corresponding figure is 36.3% (see [10]). In contrast, the equiva-

lent average values for OECD countries are 62.9% and 62.2% respectively, with

Turkey having the highest percentages (102.1% and 97.9%). The causes behind

the low replacement rate are complex, and involve a combination of factors

such as low contributions (the 10% minimum is likely to increase in the coming

2

months), and the fact that some workers stop contributing for long periods of

time, due to several reasons. Addressing those issues is fundamental to have a

healthy system, but the remedies fall into the realm of public policy and better

communication with the population, which are beyond the scope of this paper.

The regulator defines the risk-return profile of each of the funds via port-

folio constraints. They essentially deal with the percentage of the portfolio that

can be invested in equities and bonds. Of the five funds, Fund A is the one that

is allowed the highest percentage in equities (up to 80% of its holdings). This

percentage is reduced progressively as we move from Fund A to Fund E, where

it reaches a 5% value. By the same token, the percentage of the portfolio that

can be invested in fixed income securities increases from Fund A to Fund E. The

composition of each portfolio is also determined by several other constraints in

addition to the limits by asset class (that is, stocks, bonds, alternative invest-

ments). The SP also imposes limits on maximum exposures to issuers, as well

as asset managers (in case investments are made in mutual funds). In essence,

the restrictions imposed by the SP aim at o↵ering the a�liates the possibil-

ity of having access to di↵erent risk-return alternatives while maintaining an

adequate level of diversification within each option. The key point is that the

regulator has attempted to control the risk-return profile of the multifunds in-

directly, that is, via the asset allocation limits already mentioned. Our findings

show that such portfolio constraints fail to rank the multifunds according to the

desired order of decreasing returns, from Fund A to Fund E, when considering

periods of time greater than 5 years.

With this as background, our main goal in this paper is to o↵er a thorough

understanding of the true risk-return profile of the multifunds, making exten-

sive use of the data available in the period 2002-2017. More precisely, we want

to assess if the multifunds are performing the way the regulator intended, a

problem of paramount importance since the Chilean pension system has served

as a blueprint to modify the pension system of other countries, mostly in Latin

3

America and Eastern Europe. As described in [13], life-cycle funds have specific

characteristics that di↵erentiate them from other financial instruments and the

evidence in other countries indicate that a correct performance of these funds de-

pends on a well-designed system. For example, evidence from Turkey [4] shows

that having a passive investment system outperforms an active investment one,

or from Poland [6], which shows that the supervisory board structure has an im-

portant e↵ect on the funds performance. In the context of our research, the very

long investment horizon implies that the composition of an individual’s portfolio

has to change as retirement approaches, and in a DC system the regulator must

o↵er a su�cient number of alternatives that can accommodate those needs. Our

main finding is that the current system fails to do so, and the consequences for

the a�liates are severe.

In the following sections we evaluate the historic performance of the mul-

tifunds, we look at their true risk profile, and we propose some metrics to assess

whether their actual performance was consistent with the objectives spelled out

by the regulator. This is necessary to evaluate the suitability of the current

regulatory constraints. We conclude with some suggestions on how to modify

the current regulation so that risk-return profiles of the funds actually match

the desired objective.

2 Performance of the Funds

2.1 Returns

The pension of a worker is a function of both, the savings accumulated during his

active working life plus the profitability achieved by those savings. Thus, in this

section we focus on the returns achieved by the multifunds. For the purpose of

this study we have relied on the performance data of the multifunds as reported

in the regulator website (www.spensiones.cl). Specifically, the data gathered

4

start in October 2002 (the beginning of the multifund system) and end in July

2017. The data reflect the monthly returns of each fund (A, B, C, D and E),

measured based on inflation-adjusted Chilean pesos (a unit known as unidad

de fomento, or UF). Hence, these returns are actual (real) returns, not nominal

returns. Additionally, the returns used in this study are the monthly industry

averages for each fund. The Chilean AFPs exhibit strong herd behavior and

thus, working with the average industry returns (as opposed to the returns of

a particular AFP) makes more sense. The strong herd behavior has been the

result of an ill-designed performance benchmark that encourages the AFPs to

mimic each other’s portfolios in order to avoid the penalties associated with

deviations from the industry average. This topic has been treated in detail in

[3].

2.1.1 Average returns

With this in mind, we turn first to Figure 1, which shows the monthly returns

for each fund. We can observe that Fund A displays the highest volatility, while

Funds D and E display the lowest. It can also be seen that the riskier funds

fall and recover together, which suggest that their behavior is highly correlated.

Table 1 reports some basic statistics based on the funds monthly returns (178

data points for each fund). These statistics are consistent with the trends shown

in Figure 1, namely, Fund A exhibits the highest volatility (measured by the

standard deviation of returns), while D and E the lowest. Table 1 also indicates

that in terms of minimum and maximum monthly returns, as well as average

returns, the five funds are rank-ordered according to the intended risk profile,

namely, Fund A on top, and Fund E at the bottom.

These results might seem to indicate that the asset allocation constraints

designed by the regulator had indeed achieved their intended goal. However,

we will see that such conclusion is premature. The fact of the matter is that

the average monthly return is a poor proxy for long-term performance. More

5

20022003

20042005

20062007

20082009

20102011

20122013

20142015

2016

-20

-10

0

10R

etu

rns

in p

erce

nta

ge

(%)

Fund A

20022003

20042005

20062007

20082009

20102011

20122013

20142015

2016

-20

-10

0

10

Ret

urn

s

in p

erce

nta

ge

(%)

Fund B

20022003

20042005

20062007

20082009

20102011

20122013

20142015

2016

-20

-10

0

10

Ret

urn

s

in p

erce

nta

ge

(%)

Fund C

20022003

20042005

20062007

20082009

20102011

20122013

20142015

2016

-20

-10

0

10

Ret

urn

s

in p

erce

nta

ge

(%)

Fund D

20022003

20042005

20062007

20082009

20102011

20122013

20142015

2016

Period (year)

-20

-10

0

10

Ret

urn

s

in p

erce

nta

ge

(%)

Fund E

Figure 1: Monthly returns for the five funds (October 2002 - July 2017).

precisely, what is critical in terms of a pension, is really the cumulative return

over the relevant time-period, and not the average monthly returns and its

corresponding fluctuations (volatility). Thus, we now look at the cumulative

long-term return of the multifunds.

6

Table 1: Funds monthly returns, descriptive statistics, expressed in percentage

(%), October 2002 - July 2017.

Fund Mean Return St. Dev. Min Max

A 0.61 3.59 –21.28 9.50

B 0.50 2.59 –14.25 6.47

C 0.44 1.74 –8.00 4.25

D 0.38 1.10 –4.00 2.90

E 0.32 0.89 –2.81 3.40

2.1.2 Cumulative returns

Considering that a working person’s active life is roughly forty years, and that

we have five funds, it is reasonable to assume—as a first approximation—that

on average a typical worker would stay eight years on each fund, as he moves

sequentially from Fund A to Fund E. Thus, we consider the 8-year cumulative

return as an appropriate parameter to assess the multifunds’ performance. Fig-

ure 2 depicts for each of the five funds, the cumulative 8-year return, based on

di↵erent starting dates, beginning on October 2002, and ending on July 2009.

That is, we consider all the possible 8-year time-windows allowed by our data.

Therefore, each point in the graph represents the cumulative return a worker

would have obtained, had he entered that specific fund on the date indicated

on the horizontal axis, assuming he remains there for the entire 8-year period.

We observe that the funds returns for the period ranging from the end of

2002 until the end of 2003, and from the end of 2008 until the beginning of 2010,

are rank-ordered according to the sequence the regulator intended. That is, a

person who had entered the system at any time within those periods, would

have obtained higher returns if he had chosen Fund A over Fund B, or Fund

B over C, etc. In other words, the riskier the funds, the higher the cumulative

7

20022003

20042005

20062007

2008

Starting period (year)

0

10

20

30

40

50

60

70

80

90

100

110

120

Cu

mu

lati

ve

retu

rns

in p

erce

nta

ge

(%)

Fund A Fund B Fund C Fund D Fund E

Figure 2: Cumulative 8-year returns for the five funds, as a function of the

starting period (October 2002 - July 2009).

return. However, it is the period from the end of 2003 until the end of 2008

the one that draws our attention. During this period, there is a high variability

in terms of which fund enjoys the highest return. Moreover, throughout most

of the period, rank-ordering the funds in terms of their returns results in a

sequence which is exactly the opposite of what it was intended. For example,

a young person who had entered the system at the beginning of 2006 would

have received a higher return if he had chosen Fund E (the most conservative)

instead of Fund A (the one that is supposed to be the most adequate choice for a

young worker). As a matter of fact, there are many instances within this period

in which the cumulative return of Fund A was close to zero. It is impossible

to overlook the damage that a situation like this would have inflicted on those

workers. In summary, Figure 2 shows a very disturbing phenomenon—during a

significant period of time, the cumulative (long-term) returns of the multifunds

exhibited a pattern which was the exact opposite of what the regulator had in

8

mind. Longer (10- or 12-year) as well as shorter (5-year) time-windows result

also in graphs retaining the essential features identified in Figure 2, namely, an

inverted relationship between the risk and return of the multifunds that persists

for a significant length of time.

2.1.3 Sharpe ratio

Figure 3 is analogous to Figure 2, except that it shows the Sharpe ratio (SR)

instead of cumulative return, computed over 8-year time-windows. In this cal-

culation the risk-free rate was assumed to be zero, thus, the SR is actually the

average observed return divided by its corresponding standard deviation. The

20022003

20042005

20062007

2008


0

0.1

0.2

0.3

0.4

0.5

Sh

arp

e ra

tio


Figure 3: Sharpe ratio (SR) for each of the five funds, considering 8-year time-

windows, as a function of the starting period (October 2002 - July 2009).

SR reflects the return adjusted by risk (i.e. normalized by units of risk). One

could argue that, in theory, after adjusting for risk, all five funds should exhibit

a similar performance. Clearly, this is not the case. It is clear from Figure 3 that

9

Fund E is always on top, while Fund A is always at the bottom. Considering

that the SR is actually obtained by dividing two numbers, it is di�cult—at least

initially—to attribute a low SR solely to poor return performance. We must

be mindful of the potential distortions caused by a denominator approaching

zero, a frequent case when dealing with low volatility portfolios such as funds

D and E. Therefore, although it is tempting to attribute to Fund E a superior

performance based on this metric, we must refrain, at this point, from deriving

sweeping conclusions based only on this metric. We will revisit this issue later.

2.1.4 Holding periods

Figure 4 shows the cumulative returns considering di↵erent holding periods,

expressed in months (along the horizontal axis), for each of the funds. The

holding periods—the time between entering and leaving a specific fund—range

between 1 to 96 months, and we consider all the possible starting months given

our data. Thus, each point on the graph represents the n-month cumulative

return achieved by a person who entered that specific fund at some time be-

tween October 2002 and July 2009. For each point in the x-axis we have 82

observations, corresponding to each of the possible starting months given our

data. For example, when considering 96-month holding periods (8 years), the

vertical points for each fund correspond to the curves in Figure 2. Finally, the

last frame (f) shows the average returns for all funds.

Several observations are in order. First, we notice that the cumulative

returns of Fund A are marked by a high dispersion with respect to the average.

This dispersion decreases as we move from Fund A to B, from B to C, and so

on. This situation appears to be in line with the regulator intention, namely,

volatility of returns should decrease from Fund A to Fund E.

However—once again—this conclusion is premature for frame (f) reveals

another worrying pattern. We note that the average cumulative returns of the

10

12 24 36 48 60 72 84 96-50

-30

-10

10

30

50

70

90

110

Cum

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Average

(a) Fund A

12 24 36 48 60 72 84 96-50

-30

-10

10

30

50

70

90

110

Cum

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Average

(b) Fund B

12 24 36 48 60 72 84 96-50

-30

-10

10

30

50

70

90

110

Cum

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Average

(c) Fund C

12 24 36 48 60 72 84 96-50

-30

-10

10

30

50

70

90

110C

um

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Average

(d) Fund D

12 24 36 48 60 72 84 96-50

-30

-10

10

30

50

70

90

110

Cum

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Average

(e) Fund E

12 24 36 48 60 72 84 96

Length of holding period (months)

0

10

20

30

40

50

60

Cum

ula

tive

retu

rn i

n p

erce

nta

ge

(%)

Fund A

Fund B

Fund C

Fund D

Fund E

(f) Average return for each fund

Figure 4: Cumulative returns for each of the five funds considering several

holding periods.

11

five funds, while easily distinguishable for short—and medium—term holding

periods, start to converge after five years. In fact, the longer the time horizon,

the more similar the returns of the five funds. This situation is quite revealing.

First, it indicates that it is misleading to evaluate the performance of the funds

without paying attention to long-term horizons. And second, it shows that when

matters the most—that is, for periods longer than five years—the funds behave

fairly similarly. This is problematic for the typical holding period of any given

fund is likely to be longer than five years. This convergence in performance

certainly undermines the rationale for selecting Funds A or B, as they appear

to o↵er only higher risk without adequate return compensation for taking this

additional risk. In fact, this revelation is consistent with what the SR hinted in

the previous section. Thus, we are now in a position to state that the riskier

funds o↵er poor risk-adjusted returns.

In summary, while the average returns paint a picture which appears to

be consistent with the regulator intentions, the cumulative returns and the risk-

adjusted returns (far more relevant metrics from the workers perspective) o↵er a

dramatically di↵erent view: the multifunds have exhibited during an important

part of their existence a performance which is almost the opposite of what it

was intended.

2.2 Risk Profile of the Funds

We now turn to the risk associated with the funds. For this purpose, we consider

two risk metrics commonly used in risk management and financial engineering,

the Value-at-Risk (VaR) and the Conditional Value-at-Risk (CVaR). The VaR

is the maximum loss that a portfolio can su↵er in a specific period of time,

estimated with a given (normally very high) level of confidence [9]. If the VaR

is estimated with a confidence 1 � ↵, it follows that the probability of having

losses exceeding the VaR is ↵. More formally, the VaR of a random variable

X with cumulative distribution function (cdf) F (·) and with a confidence level

12

↵ 2 [0, 1] is defined as

VaR↵[X] := min{t|F (t) � ↵} = min{t|P (X t) � ↵}.

The CVaR, which is an alternative risk metric to the VaR, considers only those

losses that exceed the VaR [12]. The CVaR of a random variable X with cdf

F (·) and with confidence level ↵ 2 [0, 1] is defined as

CVaR↵[X] :=1

1� ↵

Z 1

↵VaR� [X]d�.

In short, the CVaR (also known as expected shortfall) is the expected value

of the losses that exceed the VaR. When X is a discrete random variable with

support points z1 < z2 < . . . < zN and associated probabilities p1, . . . , pN , it

follows from [11] that

CVaR↵[X] :=1

1� ↵

" k↵X

k=1

pk � ↵

!zk↵ +

NX

k=k↵+1

pkzk

#,

where ↵ 2 (0, 1) and k↵ is such that

k↵X

k=1

pk � ↵ >k↵�1X

k=1

pk.

In this study we will focus on the VaR and the CVaR of the funds’ returns.

In this context, a fund is riskier if its VaR (or CVaR) is lower (more negative).

Relying solely on the VaR somehow limits the scope of the analysis, since the

VaR does not fully capture the tail end of the distribution associated with lower

returns. The CVaR, which focuses on the values exceeding the VaR, does.

Additionally, another shortcoming of the VaR is that it violates the so-called

subadditivity condition. The CVaR, a coherent risk metric [1], does not have

this limitation.

2.2.1 VaR

Figure 5 shows the 95%-confidence VaR, based on monthly returns, for the five

funds, considering 6-year time-windows (top frame) and 8-year time-windows

13

(bottom frame). The horizontal axis indicates the beginning of the period con-

sidered. For the 6-year time-windows the VaR of the funds ranges from –6.3%

to –0.9%, with Fund E consistently exhibiting a value fluctuating around –1%.

The variability of the VaR of the funds increases as we move from Fund E to

Fund A. We also notice that the rank order of the funds, which is in line with the

regulator expectations, is stable over time, although the numerical di↵erences

among the funds’ VaRs change widely. The 8-year VaR, shown in the bottom

panel of Figure 5, reveals the same patterns. However, the numerical di↵erences

among the VaR values tend to show greater consistency.

20022003

20042005

20062007

20082009

2010


(a) 6-year time window

-8

-7

-6

-5

-4

-3

-2

-1

0

VaR

in p

erce

nta

ge

(%)

20022003

20042005

20062007

2008


(b) 8-year time window

-8

-7

-6

-5

-4

-3

-2

-1

0

VaR

in

per

centa

ge

(%)


Figure 5: VaR, based on monthly returns, for each fund, considering (a) a 6-year

time-window and (b) an 8-year time-window.

14

2.2.2 CVaR

Figure 6, which is similar to Figure 5, shows the corresponding 95%-CVaR

values, considering, again, 6-year (top frame) and 8-year time-windows (bottom

frame). The same pattern applies: we see a consistent rank order according

the regulator’s intentions, with some discrepancies in terms of the actual CVaR

values. These discrepancies are more significant when using the 6-year time-

window, which seems to suggest that over longer time-periods, the relationship

among the risk of each of the funds is more stable. It is also apparent—in

agreement with Figure 5—that starting in 2012, Funds D and E show an almost

identical performance in terms of risk.

20022003

20042005

20062007

20082009

2010



-14

-12

-10

-8

-6

-4

-2

0

CV

aR i

n p

erce

nta

ge

(%)

20022003

20042005

20062007

2008



-14

-12

-10

-8

-6

-4

-2

0

CV

aR i

n p

erce

nta

ge

(%)


Figure 6: Monthly returns CVaR, considering (a) 6-year and (b) 8-year time-

windows.

15

In summary, we can conclude that in terms of risk, either by looking at

the absolute value of the relevant metrics, or the rank order they imply, the

funds behaved in the manner expected: the funds exhibited decreasing levels of

risk from Fund A to Fund E.

3 Rank Order Metrics

Having considered di↵erent criteria to evaluate returns and risk, we now examine

the rank order imply by these criteria. To this end, we focus on the two most

relevant parameters, the CVaR and the cumulative return, measured both using

di↵erent time windows from 6 to 12 years. Recall that the intention of the

regulator was twofold: (i) in terms of risk, the funds should exhibit decreasing

levels of risk, from Fund A to Fund E; and (ii) in terms of returns, the funds

should deliver decreasing returns when moving from Fund A to Fund E. We

focus on the CVaR (instead of the VaR) since as explained before the CVaR

is a more encompassing metric as it captures the behavior of the tail end of

the (return) distribution. And we select the cumulative return (instead of the

average return), since this is the factor that really dictates the magnitude of

the replacement rate. Therefore, an adequate performance of the system would

mean that the funds (A, B, C, D, E), judged by these two criteria, should be

rank-ordered as (1, 2, 3, 4, 5), if not always, at least most of the time.

3.1 Definition of the metrics

For this purpose, we consider three di↵erent rank order metrics. Each addresses

a di↵erent aspect of the departure from the correct (desired) rank order.

(i) Hamming distance. This metric assigns a value of 0 if a fund is in the

correct position and 1 otherwise (see [5]). Thus, the maximum possible

16

value is 5, corresponding to a situation in which all the funds are in the

“wrong” position. For example, the sequence (1, 2, 3, 4, 5) obviously

results in a value equal zero. However, the sequences (5, 3, 4, 2, 1) and

(3, 4, 1, 5, 2) are assigned a value of 5, while (1, 2, 3, 5, 4) would get a 2.

Let us note that the sequence (5, 4, 3, 2, 1), which in our case represents

the worst-case scenario, is assigned a value of 4, explained by the correct

position of Fund C. A shortcoming of this metric is that only focusses

on whether a fund is in the correct position, but not the distance to its

“correct” position.

(ii) Spearman footrule. This metric considers the absolute di↵erence be-

tween the position in which a fund is, and the position it should have, and

adds all five numerical values (see [2]). In short, it attempts to capture

the magnitude of the deviation from the correct rank order as well. For

example (3, 4, 1, 5, 2) results in a value equal to |3 � 1| + |4 � 2| + |1 �

3| + |5 � 4| + |2 � 5| = 10. If the funds are rank-ordered in exactly the

reverse sequence, i.e. (5, 4, 3, 2, 1), something we can describe as the

worst possible situation, the value is 12, which is indeed the maximum

possible value this metric can have.

(iii) Kendall Tau rank distance. This metric counts the number of pairwise

discrepancies between the correct rank order and the actual rank order (see

[7], [8]). Since we have five funds, the possible pairwise comparisons are

ten (1 with 2, 3, 4, and 5, and then 2 with 3, 4 and 5, and so on), and

hence the maximum possible value is 10. The following example clarifies

the calculation. Suppose the funds have been rank-ordered in the following

sequence: (3, 4, 1, 2, 5). In this case the Kendall Tau is 4 because the

pairs (3,1), (3,2), (4,1) and (4,2) represent pairwise disagreements with

respect to the original list (1, 2, 3, 4, 5).

17

3.2 Results

In all three metrics, higher values are associated with higher levels of discrepancy

in terms of the rank order. To facilitate the comparisons, we have normalized

all metrics by their maximum value. Thus, a value of 0 reflects a perfect rank

order, (1, 2, 3, 4, 5) in this case, whereas a value of 1 indicates the maximum

discrepancy with respect to the desired benchmark.

20022003

20042005

20062007

20082009

2010



0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

20022003

20042005

20062007

2008



0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

Hamming Spearman Kendall Tau

20022003

20042005

2006


(c) 10-year time window

0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

20022003

2004


(d) 12-year time window

0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

Figure 7: Rank order performance metrics based on CVaR, based on (a) 6-year,

(b) 8-year, (c) 10-year and (d) 12-year time-windows.

Figure 7 shows the normalized values of the three rank order metrics ap-

plied to the CVaR risk metric, while Figure 8 shows the values of the metrics

applied to the cumulative returns. We consider 6-, 8-, 10- and 12-year time-

windows. This is analogous to the situation described in Figures 2 and 3 (the

18

20022003

20042005

20062007

20082009

2010



0

0.2

0.4

0.6

0.8

1P

erfo

rman

ce m

etri

c (n

orm

aliz

ed)

20022003

20042005

20062007

2008



0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

Hamming Spearman Kendall Tau

20022003

20042005

2006


(c) 10-year time window

0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

20022003

2004


(d) 12-year time window

0

0.2

0.4

0.6

0.8

1

Per

form

ance

met

ric

(norm

aliz

ed)

Figure 8: Rank order performance metrics based on cumulative returns, based

on (a) 6-year, (b) 8-year, (c) 10-year and (d) 12-year time-windows.

dates along the horizontal axis mark the beginning of the time-window consid-

ered). Average values of the three di↵erent rank order metrics are summarized

in Tables 2 and 3 for the CVaR and cumulative returns, respectively, for the

various time-windows considered in the analysis.

Table 2: CVaR rank order metrics.

Length of time window

Distance 6y 8y 10y 12y

Hamming 0.127 0.048 0 0

Spearman 0.053 0.020 0 0

Kendall Tau 0.032 0.012 0 0

19

Table 3: Cumulative returns: rank order metrics.

Length of time window

Distance 6y 8y 10y 12y

Hamming 0.512 0.533 0.448 0.291

Spearman 0.578 0.576 0.461 0.181

Kendall Tau 0.547 0.542 0.417 0.120

In terms of absolute risk (CVaR, Figure 7), except for 34 out of 107 periods

at the end of the 6-year time-window (Figure 7(a)), and 10 out of 83 at the end

of the 8-year time-window (Figure 7(b)), the funds are rank-ordered correctly.

Table 2 shows a pattern which is consistent with Figure 7, namely, that in terms

of risk the funds are ordered, most of the time, in a satisfactory manner, as the

metrics are much closer to 0 than 1. Moreover, for 10-year periods or longer,

the order is perfect.

In terms of cumulative returns (Figure 8), all three metrics suggest a wor-

risome pattern. Let us focus on the 8-year time window (Figure 8(b)). We note

that in October 2003 the rank order starts to depart from the correct sequence

and gets increasingly distorted. Then, from July 2005 through October 2008,

the funds are consistently rank-ordered in a manner which departs significantly

from the desired sequence as two of the three metrics remain anchored at the

worst possible value, that is, 1. For this time-window, the funds are rank-ordered

in an undesirable manner in 60 out of 83 cases, and in 41 out of 83 cases the

funds are rank-ordered in the worst possible way: (5, 4, 3, 2, 1), in essence,

a rank order which is the exact opposite of what the regulator had in mind.

Table 3 presents a pattern consistent with what is shown graphically in Figure

8, that is, a rank order of the funds at odds with the regulators goal (metrics

values higher than, or close to, 0.5). Although for 12-year periods the metrics

improve (closer to 0 than 1), the significance of this improvement is arguably

20

very debatable as most people stay in a fund for a period much shorter than 12

years.

As a final observation, the fact that the funds are correctly rank-ordered

in terms of risk, but not in terms of cumulative returns, explains the pattern

exhibited by the SR (Figure 3). In short, this figure reveals that as we move

from Fund A to Fund E, the risk-adjusted returns improve, which—again—it is

exactly the opposite of what we should expect. In other words, investors in the

riskier funds received a poor compensation for taking more risk.

4 Conclusions

In 2002 the Chilean pension regulator introduced a multifund scheme: five funds

(A, B, C, D, E), which were supposed to o↵er the future retirees di↵erent risk-

return options according to their investment profiles and preferences. Specifi-

cally, the funds were designed to achieve increasing long-term returns commen-

surate with their risk profiles, with risk decreasing from Fund A to Fund E. The

ultimate goal of this multifund structure was to improve the pensions replace-

ment rate. Fifteen years after the implementation of this concept (almost twice

the average time a worker remains in each fund), it seems fitting to examine if

this idea has been successful.

Unfortunately, based on all the analyses already discussed, the verdict

is quite negative. Yes, the funds designed by the regulator—chiefly through a

number of upper and lower limits in terms of asset classes—have been successful

in the sense that the five funds are correctly rank-ordered in terms of risk.

However, their cumulative returns over long time periods have not been in line

with their risk profile. In fact, during significant stretches of time, the five

funds have been rank-ordered in terms of returns in a manner which is the

opposite of what it was intended (with Fund A exhibiting the lowest returns

21

and Fund E the highest). In essence, participants in Funds A and B took more

risk, but they did not receive returns that compensated them for this risk. The

consistency exhibited by the rank order of the funds based on their SRs (with

Fund A always at the bottom and Fund E on top), coupled with the strong

convergence of returns displayed after holding periods longer than five years,

add to a troubling picture. It leaves one wondering whether it would have been

better just to o↵er one fund. This situation has been particularly damming for

those young workers who entered the system around 2007 and most likely join

Fund A. For example, 100 UF invested in Fund A in November 2007 would have

meant to have only 102.12 UF in November 2015. In summary, the evidence

indicates that attempting to control the risk-return profile of the funds by means

of time-invariant asset allocation constraints has not worked in the way it was

intended. The risk-return profiles of the funds are at odds with the intended

goal.

The idea that attempting to define (and control) the risk-return profile

of a portfolio via concentration limits does not work should not be surprising.

This idea lacks both, a sound theoretical basis, and some credible empirical

evidence. If one wants to control risk, the obvious approach is to do so via some

of the commonly accepted risk metrics, applied to the entire portfolio under

consideration. The indirect approach taken by the SP, namely, manipulating the

asset concentration limits with the hope of obtaining some consistent risk-return

profiles, is somehow based on the notion that the key attributes of di↵erent asset

classes are time-invariant. Clearly, there is no evidence to support this claim.

In fact, the evidence points in the opposite direction. For example, return

correlation values between di↵erent asset classes are notoriously unstable and

change significantly as a function of time.

Finally, it must be clear that this study should not be taken as an in-

dictment of the Chilean pension system, and more broadly, an indictment of

the conceptual basis at the root of privately-managed DC schemes in general.

22

Neither is a call to return to a PAYG system or a push to promote the alleged

benefits of a state-managed pension fund system. This study is really a call to

re-examine the criteria currently employed by the regulator to control the risk

profile of the multifunds. At the very least, the idea of abandoning the practice

of attempting to control risk via asset allocation limits, in order to replace it

with sound portfolio-level risk metrics should be considered. Failure to do so

will continue to inflict permanent damage on the pensions of the future retirees.

References

[1] Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, and David Heath. Co-

herent measures of risk. Mathematical finance, 9(3):203–228, 1999.

[2] Persi Diaconis and Ronald L. Graham. Spearman’s footrule as a measure of

disarray. Journal of the Royal Statistical Society. Series B (Methodological),

pages 262–268, 1977.

[3] Viviana Fernandez. Profitability of Chile’s defined-contribution-based pen-

sion system during the multifund era. Emerging Markets Finance and

Trade, 49(5):4–25, 2013.

[4] Umut Gokcen and Atakan Yalcın. The case against active pension funds:

Evidence from the Turkish private pension system. Emerging Markets Re-

view, 23:46–67, 2015.

[5] Richard W. Hamming. Error detecting and error correcting codes. Bell

Labs Technical Journal, 29(2):147–160, 1950.

[6] Krzysztof Jackowicz and Oskar Kowalewski. Crisis, internal governance

mechanisms and pension fund performance: Evidence from Poland. Emerg-

ing Markets Review, 13(4):493–515, 2012.

[7] Maurice G. Kendall. A new measure of rank correlation. Biometrika,

30(1/2):81–93, 1938.

23

[8] Maurice G. Kendall. Rank correlation methods. Hafner, New York, 2nd

ed., rev. and enl. edition, 1955.

[9] Thomas J. Linsmeier and Neil D. Pearson. Value at risk. Financial Analysts

Journal, 56(2):47–67, 2000.

[10] OECD. Pensions at a glance 2017. 2017.

[11] R. Tyrrell Rockafellar and Stanislav Uryasev. Conditional value-at-risk for

general loss distributions. Journal of banking & finance, 26(7):1443–1471,

2002.

[12] R. Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional

value-at-risk. Journal of risk, 2:21–42, 2000.

[13] Luis M. Viceira. Life-cycle funds, in Annamaria Lusardi, ed., Overcoming

the saving slump: How to increase the e↵ectiveness of financial education

and saving programs, 2008.

24