Managed Funds Performance with Alternative Benchmarks: The Carhart Four- Factor Model versus Traditional Models

Managed Funds Performance with Alternative Benchmarks: The Carhart
Four- Factor Model versus Traditional Models


Lakshman Alles
School of Economics and Finance
Curtin University of Technology
GPO Box U1987
Perth 6001, Australia
Tel. (61) 89266 7811Fax. (61) 89266 3026
Email: AllesL@cbs.curtin.edu.au

and

V.C. Hong
Gold Corporation
310 Hay Street
Perth 6004
Tel. (61) 89421 7445
Email:

chih@goldcorp.com.au


ABSTRACT

This study examines the ten-year performance of Australian managed funds in three different
investment categories using monthly data that is free from survivorship bias. Three different
investment categories were selected to capture any unique features that may be attributed to the
particular type of fund. The performance of the funds was evaluated based on the Carhart Four-
Factor Model, and the results compared with two traditional benchmarks: the market index and
Jensen’s alpha. Results of performance based on the Carhart model showed little difference to the
results based on the traditional models.

1

---

Introduction

The managed fund industry is large and growing. For instance, in Australia it had a total of $618
billion in funds under management as at 30 September 20011 composed of 3 main sources; the
superannuation sector accounting for 47%, life insurance offices supplying 27%, public unit
trusts contributing 19% and the rest providing 7%. The superannuation funds grew from $285
billion in September 2001 to $309 billion in September 2002.2 and the Public Unit Trusts also
reported an increase of an additional $1.6 billion in the level of investment in the year ending
September 2002.

Investors in managed funds often base their fund selection decision on the past performance and
performance persistence of funds. Professional fund managers recognise the keen interest that
potential investors have towards past fund performance, and eagerly publicise their good
performance, particularly performance persistence. Good performing fund managers are routinely
glamorised in the popular media. The proper assessment of fund performance is therefore a
matter of crucial importance to fund managers and the general investing public alike.

The study of managed fund performance is not new. Numerous studies on the performance
evaluation of managed funds have been carried out in USA, Australia and around the world.
Researchers have been focussing on the issue of fund performance evaluation due to a number of
unsettled issues associated with it. One of the difficulties lies in the absence of a common
methodology for performance evaluation. Performance must be based on risk adjusted returns.
Adjusting for risk gives rise to many difficulties. One is in choosing the method for risk
adjustment. When adjusting for risk, a choice must be made regarding the underlying asset
pricing model and the appropriate comparative benchmarks. The absence of a consensus
regarding the applicable asset pricing model or the appropriate comparative benchmark gives rise
to inconsistencies in performance evaluation. This is because the choice of the pricing model
presupposes the systematic risk assumed for the fund evaluated, and it can have a significant

1 Australian Bureau of Statistics (ABS) 30/09/01, AusStats:5655.0 Managed Funds, Australia
2 Australian Bureau of Statistics (ABS) 28/02/01 & 30/09/01, AusStats:5655.0 Managed Funds, Australia

2

---

effect on the estimates of its performance attributes, and potentially inconsistent rankings of
performance evaluation.

Using Australian managed fund data, this study evaluates performance based on the Carhart 4-
factor risk loading model and compares the results with traditional evaluation procedures:
performance relative to the market index and performance measured by Jensen’s alpha. The
Carhart 4-factor model is motivated by previous research evidence indicating that market
equilibrium is better represented by the Fama-French 3-factor model and an additional factor
attributed to Jegadeesh and Titman’s (1993) one-year momentum anomaly (Carhart (1997)).
Performance evaluation measures based on the 4-factor model could alternatively be interpreted
as a performance attribution model after allowance is made for the simple trading strategies of
holding high versus low beta stocks, large versus small market capitalization stocks, value versus
growth stocks and momentum versus contrarian stocks. It would be instructive to investigate
whether fund managers demonstrate any superior performance ability, once the performance
attributes of these simple trading strategies are filtered out. To do so, results of performance
measures based on the Carhart 4-factor model are compared with the more traditional methods of
Jensen’s alpha and performance relative to the market index.

This study also improves upon previous Australian studies of fund performance because it uses a
larger number of funds, a longer time period, more recent data and overcomes the survivorship
biases that afflicted most previous studies. This study concentrates on the performance of public
unit trusts as opposed to wholesale funds or superannuation funds.

The rest of the paper is arranged as follows. Section II reviews the literature relating to this study.
Section III details the methodology used to evaluate fund performance and section IV outlines the
sources of data used in this study. Section V presents the results and the final section VI provides
the conclusions.





3

---

II Literature Review

The current evidence in Australia suggests that fund managers are unable to outperform the
market. Robson (1986) analysed how unit trusts and mutual funds in Australia performed
between the periods of 1969 to 1978 comparing performance to a market benchmark and an
industry benchmark. In both cases, funds were unable to outperform either of the benchmarks.
Although the study provided an important insight into the fund industry in Australia, the findings
were weakened by the quality of the data used. The lack of data forced the use of two different
market proxies over two sub-periods. This underlying difference would make the comparability
very difficult and therefore the conclusion questionable because results can be bias depending on
the proxy(ies) used. The different proxies were a result of the lack of available information to be
used as the market proxy. As the reasoning is valid, it casts scepticism over the results of the
study. Another issue that may also affect the integrity of the findings is the small size of the
sample, couple this with the admission from the author that there is a small survivorship bias and
there may be concerns over the results. This study overcomes some of the problems experienced
by Robson (1986). Firstly, this study utilises consistent data for the whole duration of the sample
period. The market proxy used over the entire 10 year period remained unchanged and was
consistently applied. The sample size of the funds used in this study is much larger, 390
compared to 76. Including terminated funds in the sample overcomes the issue of survivorship
bias.

Confirmation of poor managed fund performance in Australia has also been observed in the
superannuation sector.3 Sinclair (1990) found that the performances of the funds did not exceed
the market proxy. Gallagher (2000) provided further evidence of fund performance in Australia
using more recent fund data. His conclusion is consistent with those of Robson (1986) and
Sinclair (1990), in that funds were unable to outperform the market. Furthermore, there was no
indication that fund managers were not able to accurately time the market nor did they have
superior stock/investment selection abilities. Despite the outcome of the study, it too lacked in
sample size and it did not account for survivorship bias. Notwithstanding these shortcomings, the
study added to the knowledge base for superannuation funds. Although this study does not extend

3 also see Drew, M and Noland, J. 2000, ‘EMH is Alive and Well’, JASSA, issue 1, Summer, pp15-18

4

---

to the superannuation fund industry, nonetheless the principles of managed fund performance and
persistence are still valid and the findings transferable to this study.


Fund Performance Benchmarks

The performance of a managed fund is hard to quantify accurately. The difficulties experienced
in trying to quantify the performances of a managed fund lie with the lack of standardised
methodologies used for performance evaluation. This loose perception of performance and the
lack of standardised evaluation techniques inherently elevates the role benchmarks play in all
managed fund studies because the choice of performance evaluation can produce substantial
differences in the overall conclusion. A commonly used market benchmark to determine
performance in Australia is the All Ordinaries Accumulated Index (AOAI). The AOAI is
representative of the performance of the whole Australian stock market. Therefore, the index is
frequently used as an absolute benchmark to gauge performance.

The sensitivity of performance conclusions due to the choice of evaluating techniques was
examined in a study conducted by Grinblatt and Titman (1994). The study identified the ease
with which misrepresentation may occur when examining fund performance depending on the
type of benchmark used. The study tested four different benchmarks against three different
performance regression models and found inconsistencies in the outcome of the fund
performance. The authors argued that because the fund data was ex post, the calculated outcome
should be the same irrespective of the performance measures or the absolute benchmarks used.
Although the evidence clearly indicated that there were variances in the calculated results, the
conclusions of how the fund performed overall could not be refuted. In addition, to maintain the
creditability of the tests, the study also used a controlled group of 109 passive funds to check the
integrity of the test.

Lehmann and Modest (1987) also conducted a study which examined whether managed fund
performance, as benchmarked to the CAPM and alternate APT would result in different
outcomes. The same argument was put forward, in that if a performance measure was both

5

---

accurate and efficient, then the conclusions should be identical. Unfortunately, the study
concluded that there were reported discrepancies between the choice of benchmarks used and the
performance of managed funds.

A more recent study addressing the complexities of selecting an efficient benchmark for the
purposes of accurately measuring fund performance was Kothari and Warner (1997). The authors
conducted a comprehensive study into five commonly used performance measures in fund
studies. The evaluation models tested include the Sharpe measure, Jensen alpha, Fama & French
3 factor model, the Treynor measure and the appraisal ratio by simulating 336 portfolio
performances between January 1964 and December 1991. The simulated testing found fund
performances were easily misspecified. Some of the measures reported superior abnormal returns
when there was not any. This was also true when testing for market timing abilities of the funds.

Kothari and Warner (1997) went on to conclude that the results derived from using the Fama-
French 3 factor model was “better” than the CAPM. Although the Fama French 3 factor model
showed evidence of misspecification and reported abnormal returns, the misspecification was
smaller than the Jensen Alpha (CAPM). It was noted that the additional variables, including size
and book to market value could help to explain the performance of funds. The conclusion derived
from the study indicates the need for care when using an absolute benchmark to compare the
performance of managed funds.

Many fund managers measure their performance and promote their ability to out perform the
market simply via a comparison of the raw returns of the fund with a market index (AOAI).
Likewise, other industry specific indices have also been used depending on the type of fund; for
instance, a fund that “specialises” in small stocks might be benchmarked against the Small Index.
The attractive feature of using these indexes to measure performances is the simplicity and the
ease of conveying information to a potential investor about the performance of the fund.
Although there are merits in using indices as a benchmark, the real question of what is the most
appropriate index to use is largely subject to interpretation.


6

---

This study will also consider a fund managers’ performance relative to the AOAI because it is a
commonly accepted benchmark in the fund industry. In addition, the All Industrial Accumulation
Index (AIAI) will be used as an alternative benchmark. The motive for using an alternative
benchmark is related to a recent study by Finn and Koivurinne (2000), whereby they analysed the
ex ante efficiency of Australian stock market benchmarks. As part of their study, the AOAI,
AIAI, All Resources Accumulation Index (ARAI) and 23 industry sector Accumulation Indices,
were examined for their ex ante efficiency. The study found evidence that the AOAI was ex ante
inefficient under no limitation of short selling. However, the results could not substantiate
inefficiencies given a limitation on short selling; as is the current situation in Australia. As the
study was unable to provide evidence that the AOAI was ex ante inefficient (assume limited short
selling), then the index could be used as one method of benchmarking fund performances.

The study further suggested that the AIAI (excluding the property sector) would be a more
efficient benchmark because the index excluded both mining and resource sector stocks. It was
shown that both mining and resource sector stocks could not positively contribute to the
performance of the fund. Thus, the systematic exclusion, as observed in the AIAI would represent
a “more efficient” benchmark. Moreover, the property sector, as reported in the study, also could
not contribute to the ex ante efficiencies of the AIAI. Finally, Finn and Koivurinne (2000) noted
that should a fund manager shift their portfolio weights from mining, resources and property
sector stocks towards industrial sector stocks then the portfolio should derive performances above
that of the AOAI. Based on the aforementioned findings, managed funds which have a distinct
weighting towards the industrial sector should perform better than funds which do not and should
outperform the AOAI. It is clear that from the findings of Finn and Koivurinne (2000), any model
using input variables as an explanatory proxy for the market needs to exercise caution because an
inappropriate proxy can be easily chosen.

The process of comparing a fund’s return against a market index has the merit of simplicity.
However, the process does not account for the risk exposure of the fund. To overcome this
inadequacy, a risk adjusted model was initially developed in the Jensen (1968) study. Jensen
(1968) evaluated the performance of managed funds in the United States for the period between
1945 and 1964. Each fund was individually regressed using the model and the value of the

7

---

intercept derived from the regression indicates the performance relative to the market. A positive
(negative) intercept was evidence that given the level of risk, the fund was able to outperform
(under perform) the market. Based on this simple model, Jensen (1968) found evidence that fund
managers were unable to achieve returns above those of a naive investor. The popularity of the
Jensen model has largely been driven by both the simplicity of implementation and the ease of
interpretation. The attractive nature of the model does not come without criticism. Roll (1978)
tested the robustness and the validity of the CAPM and was able to present a strong case
highlighting the weaknesses of the model. The evidence produced from the Roll (1978) study
was able to illustrate the ambiguities of the model from the interpretation of how the market
index should be formed. It was proven that the index used as the foundation of the model would
generate misleading conclusions about the performances of an asset. Specifically, the assets,
through the weaknesses of the model were able to either plot above or below the market line.
This represented either superior (above the line) or inferior (below the line) performances by the
asset. It was also argued that if the model used an ex ante efficient index, then all the securities
should therefore fall on the market line in an efficient market. If the assets did not fall on the
market line, then the market index had to be inefficient. Presumably, the evidence provided by
Finn and Koivurinne (2000) should indicate that the index used in the CAPM for this study
should be ex ante efficient. Given this presumption, the funds should lie along the market line
with an estimated alpha of zero. Despite the fact that this study presents information about the
inadequacies of the CAPM as a measure for fund performance, the CAPM can still serve as a
generalist benchmark for an investor to make an informed decision about the performance of a
fund.

A more sophisticated model used to assess managed fund performance was developed in Carhart
(1997), which is now known as the Carhart 4 factor model. The model evolved from a
combination of several earlier independent studies. Effectively, the Jensen model was used as the
foundation and known additional explanatory variables were added to form a multi factor model.
An additional three variables were added to make the model, which included factors to account
for the size effect, book-to-market variable and a momentum trading variable. These additional
variables provided an increase in the explanatory power over the CAPM (Carhart 1997).


8

---

Size and book to market factors were included into the model because Fama and French
(1993,1994) found that both these variables had explanatory power in the cross sectional return of
stocks. The extent of these two effects in the Australian financial market was also undertaken by
Halliwell, Heaney and Sawicki (1999). The book to market factor did not appear to present any
significant explanatory powers in the Australian study. Size however, was able to provide some
explanation, particularly in small to medium size companies. The explanatory powers of size
factor did however diminish as the size of the companies increased. Furthermore, the study made
mention that the number of significant findings changed depending on whether the portfolios
were formed on a value weighted or equal weighted portfolio. This again illustrates the role that
the size of a company has on the outcome.

The other factor which accounted for the increase in the explanatory powers of the CAPM was
the introduction of a momentum trading strategy to the model. Momentum trading strategy works
on the premise of buying last period winners and selling last period losers (Jegadeesh & Titman
1993, Chan, Jegadeesh and Lakonishok 1996). Jegadeesh and Titman (1993) examined the
implications of momentum trading; the research found evidence that following a momentum
strategy generates significant returns. The abnormal positive return under the momentum trading
strategy, as reported in the study, resulted from taking a long position in stocks which exhibited
positive performances in the prior six months and short the stocks which performed poorly over
the same period. Their results however, noted that the positive abnormal return dissipated
between the second and third year. Nevertheless, the study drew attention to the documented
advantages of a simple trading strategy.

Jegadeesh and Titman (1999) re-examined their initial 1993 study by testing the momentum
strategy hypothesis with data from 1990 to 1997. The reported outcome from the study
highlighted that a simple momentum trading strategy was continuing to be profitable. The
evidence derived from this secondary study rejects the hypothesis of Conrad and Karl (1998) that
returns are related to cross-sectional factors of stocks instead of time-series trends. The
subsequent study (Jegadeesh and Titman, 1999) exhibited no evidence of return reversal after the
initial two to three year period post the investment, yet for periods thereon after, the positive
abnormal returns dissipated.

9

---

The identified properties of a simple trading strategy to explain the abnormal returns for stocks
was then adopted by Grinblatt, Titman and Wermers (1995) to explain American mutual funds.
The findings from their study found that mutual funds had a tendency to purchase stocks based
upon past positive performances. This simple momentum strategy employed by mutual funds
achieved a significant positive return. In addition, the study also wanted to ascertain whether fund
managers exhibited any herding type symptoms. Herding occurs when fund managers follow a
lead and trade in the same trend; where funds buy and sell the same stock at the same time. The
herding phenomenon was less pronounced when compared to the momentum trading strategy.


III Methodology

Three models are used to evaluate the performance of Australian managed funds. Initially, the
monthly returns of the managed funds are compared with the market index. The other two
approaches adjust for risk based on two alternative models. The first is the traditional Jensen
Measure based on the CAPM and the second is the model developed in Carhart (1997). Despite
the criticism associated with the CAPM, the Jensen model has been selected due to its simplicity
and the wide acceptance of the model in the finance industry.

The Jensen Measure

Under the Jensen model the CAPM is used for assessing the performance of funds by simply
rearranging the model in excess return form:-

Rjt – Rft =αj + βj[Rmt – Rft] + εjt

where Rjt = the return of a security or portfolio j at time t
Rft = the market’s risk free rate
βj = the risk variable of the model
Rm – Rf = the premium expected return of the m