Tag Archives: returns-based style analysis (RBSA)

Smart Beta and Market Timing

2 Replies

Why Returns-Based Style Analysis Breaks for Smart Beta Strategies

Smart beta (SB) strategies tend to vary market beta and other factor exposures (systematic risk) over time. Consequently, market timing is an important source of their risk-adjusted returns, at times more significant than security selection. We have previously discussed that returns-based style analysis (RBSA) and similar methods fail for portfolios that vary exposures. Errors are most pronounced for the most active funds:

  • Estimates of a fund’s historical and current systematic risks may be flawed.
  • Excellent low-risk funds may be incorrectly deemed poor.
  • Poor high-risk funds may be incorrectly deemed excellent.

Due to the variation in Smart Beta strategies’ exposures over time, returns-based methods tend to fail for these strategies as well.

Three Smart Beta Strategies

We analyze the historical risk of three SB strategies as implemented by the following ETFs:

SPLV indexes 100 stocks from the S&P 500 with the lowest realized volatility over the past 12 months. PRF indexes the largest US equities based on book value, cash flow, sales, and dividends. SPHQ indexes the constituents of the S&P 500 with stable earnings and dividend growth.

All three smart beta strategies varied their factor exposures including their market exposures.

Low Volatility ETF (SPLV) – Market Timing

The low-volatility smart beta strategy has varied its market exposure significantly, increasing it by half since 2011. As stocks with the lowest volatility – and their risk – changed over time, the fund implicitly timed the broad equity market.  The chart below depicts the market exposure of SPLV over time:

Chart of this historical U.S. market exposure of the low volatility smart bet (SB) strategy as implemented by PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV)

PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV) – Historical U.S. Market Exposure

Low Volatility ETF (SPLV) – Historical Factor Exposures

SPLV’s market exposure fluctuates due to changes in its sector bets. Since the market betas of sectors differ from one another, as sector exposures vary so does the fund’s market exposure:

Chart of the historical exposures to significant risk factors of the low volatility smart bet (SB) strategy as implemented by PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV)

PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV) – Significant Historical Factor Exposures

Low Volatility ETF (SPLV) – Returns-Based Analysis

The chart below illustrates a returns-based analysis (RBSA) of SPLV. A regression of SPLV’s monthly returns against U.S. Market’s monthly returns estimates the fund’s U.S. Market factor exposure (beta) at 0.50 – significantly different from the historical risk observed above:

Chart of the regression of the historical returns of PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV) against the Market

PowerShares S&P 500 Low Volatility Portfolio ETF (SPLV) – Historical Returns vs. the Market

This estimate of beta understates SPLV’s historical market beta (0.55) by a tenth and understates current market beta (0.70) by more than a third. RBSA thus fails to evaluate the current and historical risk of this low volatility smart beta strategy. Performance attribution and all other analyses that rely on estimates of historical factor exposures will also fail.

Fundamental ETF (PRF) – Market Timing

The market risk of the Fundamental ETF has been remarkably constant, except from 2009 to 2010. Back in 2009 PRF increased exposure to high-beta (mostly financial) stocks in a spectacularly prescient act of market timing:

Chart of the historical exposures of the fundamental smart beta (SB) strategy as implemented by the PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) to U.S. and Canadian Markets

PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) – Historical Market Exposure

Fundamental ETF (PRF) – Historical Factor Exposures

The historical factor exposure chart for PRF illustrates this spike in Finance Factor exposure from the typical 20-30% range to over 50% and the associated increase in U.S. Market exposure:

Chart of the exposures of the fundamental smart beta (SB) strategy as implemented by the PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) to significant risk factors

PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) – Significant Historical Factor Exposures

This 2009-2010 exposure spike generated a significant performance gain for the fund. PRF made approximately 20% more than it would have with constant factor exposures, as illustrated below:

Chart of the historical return from market timing (variation in factor exposures) of the PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF)

PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) – Historical Risk-Adjusted Return from Market Timing

By contrast, PRF’s long-term risk-adjusted return from security selection is insignificant:

Chart of the historical returns from security selection (stock picking) of the PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF)

PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) – Historical Risk-Adjusted Return from Security Selection

Factor timing turns out to be more important for the performance of some smart beta strategies than security selection.

Fundamental ETF (PRF) – Returns-Based Analysis

A returns-based analysis of PRF estimates historical U.S. market beta around 1.05:

Chart of the regression of the returns of PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) against the U.S. Market

PowerShares FTSE RAFI US 1000 Portfolio ETF (PRF) – Historical Returns vs. the Market

This 1.05 beta estimate only slightly overstates the fund’s current and historical betas, but misses the 2009-2010 exposure spike. Returns-based analysis thus does a decent job evaluating the average risk of a fundamental indexing smart beta strategy, but fails in historical attribution.

Quality ETF (SPHQ) – Market Timing

The market exposure of the quality smart beta strategy (SPHQ) swung wildly before 2011. It has been stable since:

Chart of the U.S. and Canadian Market exposures of the quality smart beta (SB) strategy as implemented by the PowerShares S&P 500 High Quality Portfolio ETF (SPHQ)

PowerShares S&P 500 High Quality Portfolio ETF (SPHQ) – Historical Market Exposure

Quality ETF (SPHQ) – Historical Factor Exposures

As with the other smart beta strategies, market timing by SPHQ comes from significant variations in sector bets:

Chart of the historical exposures of the quality smart beta (SB) strategy as implemented by the PowerShares S&P 500 High Quality Portfolio ETF (SPHQ) to significant risk factors

PowerShares S&P 500 High Quality Portfolio ETF (SPHQ) – Significant Historical Factor Exposures

Quality ETF (SPHQ) – Returns-Based Analysis

A returns-based analysis of SPHQ estimates historical U.S. market beta around 0.86:

Chart of the regression of the historical returns of PowerShares S&P 500 High Quality Portfolio ETF (SPHQ) against the U.S. Market

PowerShares S&P 500 High Quality Portfolio ETF (SPHQ) – Historical Returns vs. the Market

Given the large variation in SPHQ’s risk over time, this 0.86 beta estimate understates the average historical beta but slightly overstates the current one. While the current risk estimate is close, RBSA fails for historical risk estimation and performance attribution.

Conclusions

  • Low volatility indexing, fundamental indexing, and quality indexing smart beta strategies vary market and other factor exposures (systematic risk) over time.
  • Due to exposure variations over time, returns-based style analysis and similar methods tend to fail for smart beta strategies:
    • Funds’ historical systematic risk estimates are flawed.
    • Funds’ current systematic risk estimates are flawed.
    • Performance attribution and risk-adjusted performance estimates are flawed.
  • Analysis and aggregation of factor exposures of individual holdings throughout a portfolio’s history with a capable multi-factor risk model produces superior risk estimates and performance attribution.
The information herein is not represented or warranted to be accurate, correct, complete or timely.
Past performance is no guarantee of future results.
Copyright © 2012-2015, AlphaBetaWorks, a division of Alpha Beta Analytics, LLC. All rights reserved.
Content may not be republished without express written consent.

Returns-Based Style Analysis – Overfitting and Collinearity

1 Reply

Plagued by overfitting and collinearity, returns-based style analysis frequently fails, confusing noise with portfolio risk.

Returns-based style analysis (RBSA) is a common approach to investment risk analysis, performance attribution, and skill evaluation. Returns-based techniques perform regressions of returns over one or more historical periods to compute portfolio betas (exposures to systematic risk factors) and alphas (residual returns unexplained by systematic risk factors). The simplicity of the returns-based approach has made it popular, but it comes at a cost – RBSA fails for active portfolios. In addition, this approach is plagued by the statistical problems of overfitting and collinearity, frequently confusing noise with systematic portfolio risk. 

Returns-Based Style Analysis – Failures for Active Portfolios

In an earlier article we illustrated the flaws of returns-based style analysis when factor exposures vary, as is common for active funds:

  • Returns-based analysis typically yields flawed estimates of portfolio risk.
  • Returns-based analysis may not even accurately estimate average portfolio risk.
  • Errors will be most pronounced for the most active funds:
    • Skilled funds may be deemed unskilled.
    • Unskilled funds may be deemed skilled.

These are not the only flaws. We now turn to the subtler and equally critical issues – failures in the underlying regression analysis itself. We use a recent Morningstar article as an example.

iShares Core High Dividend ETF (HDV) – Returns-Based Style Analysis

A recent Seeking Alpha article provides an excellent illustration of problems created by overfitting and collinearity. In this article, Morningstar performed a returns-based style analysis of iShares Core High Dividend ETF (HDV).

Morningstar estimated the following factor exposures for HDV using the Carhart model:

Morningstar: Returns-Based Analysis of the iShares Core High Dividend ETF (HDV) Using the Carhart Model

iShares Core High Dividend ETF (HDV) – Estimated Factor Exposures Using the Carhart Model – Source: Morningstar

The Mkt-RF coefficient, or loading, is HDV’s estimated market beta. A beta value of 0.67 means that given a +1% change in the market HDV is expected to move by +0.67%, everything else held constant.

The article then performs RBSA using an enhanced Carhart + Quality Minus Junk (QMJ) model:

Morningstar: Returns-Based Analysis of iShares Core High Dividend ETF (HDV) Using the Carhart + Quality Minus Junk (QMJ) Model

iShares Core High Dividend ETF (HDV) – Estimated Factor Exposures Using the Carhart + Quality Minus Junk (QMJ) Model – Source: Morningstar

With the addition of the QMJ factor, the market beta estimate increased by a third from 0.67 to 0.90. Both estimates cannot be right. Perhaps the simplicity of the Carhart model is to blame and the more complex 5-factor RBSA is more accurate?

iShares Core High Dividend ETF (HDV) – Historical Factor Exposures

Instead of Morningstar’s RBSA approach, we analyzed HDV’s historical holdings using the AlphaBetaWorks’ U.S. Equity Risk Model. For each month, we estimated the U.S. Market exposures (betas) of individual positions and aggregated these into monthly estimates of portfolio beta:

Chart of the historical market exposure (beta) of iShares Core High Dividend ETF (HDV)

iShares Core High Dividend ETF (HDV) – Historical Market Exposure (Beta)

Over the past 4 years, HDV’s market beta varied in a narrow range between 0.50 and 0.62.

Both of the above returns-based analyses were off, but the simpler Carhart model did best. It turns out the simpler and a less sophisticated returns-based model is less vulnerable to the statistical problems of multicollinearity and overfitting. Notably, the only way to find out that returns-based style analysis failed was to perform the more advanced holdings-based analysis using a multi-factor risk model.

Statistical Problems with Returns-Based Analysis

Multicollinearity

Collinearity (Multicollinearity) occurs when risk factors used in returns-based analysis are highly correlated with each other. For instance, small-cap stocks tend to have higher beta than large-cap stocks, so the performance of small-cap stocks relative to large-cap stocks is correlated to the market.

Erratic changes in the factor exposures for various time periods, or when new risk factors are added, are signs of collinearity. These erratic changes make it difficult to pin down factor exposures and are signs of deeper problems:

A principal danger of such data redundancy is that of overfitting in regression analysis models.
-Wikipedia

Overfitting

Overfitting is a consequence of redundant data or model over-complexity. These are common for returns-based analyses which usually attempt to explain a limited number of return observations with a larger number of correlated variable observations.

An overfitted returns-based model may appear to describe data very well. But the fit is misleading – the exposures may be describing noise and will change dramatically under minor changes to data or factors. A high R squared from returns-based models may be a sign of trouble, rather than a reassurance.

As we have seen with the HDV example above, exposures estimated by RBSA may bear little relationship to portfolio risk. Therefore, all dependent risk and skill data will be flawed.

Conclusions

  • When a manager does not vary exposures to the market, sector, and macroeconomic factors, returns-based style analysis (RBSA) using a parsimonious model can be effective.
  • When a manager varies bets, RBSA typically yields flawed estimates of portfolio risk.
  • Even when exposures do not vary, returns-based style analysis is vulnerable to multicollinearity and overfitting:
    • The model may capture noise, rather than the underlying factor exposures.
    • Factor exposures may vary erratically among estimates.
    • Estimates of portfolio risk will be flawed.
    • Skilled funds may be deemed unskilled.
    • Unskilled funds may be deemed skilled.
  • Holdings-based analysis using a robust multi-factor risk model is superior for quantifying fund risk and performance.
The information herein is not represented or warranted to be accurate, correct, complete or timely.
Past performance is no guarantee of future results.
Copyright © 2012-2015, AlphaBetaWorks, a division of Alpha Beta Analytics, LLC. All rights reserved.
Content may not be republished without express written consent.