Selecting Look-back Periods
The key adjustable parameters used in the Ranking spreadsheets used here at ITA Wealth are the short and long term momentum look-back periods (ROC1 and ROC2) and the period over which volatility is measured. Secondary parameters are the relative weights assigned to these primary parameters. Thus, we have 6 variables and a large number of possible combinations that will influence the output of our rankings.
Although the spreadsheet is designed to allow users the freedom to change all these parameters to satisfy their own personal preferences, versions up to, and including, revision 7.1.3 have been released with “default” settings of 91 and 182 calendar days for the momentum look-back periods and 10 days mean variance or 63 days semi variance for the volatility calculations. Weightings are set at 50% assigned to the short term momentum period (ROC1), 30% assigned to the longer term momentum period (ROC2) and 20% to Volatility.
The above “suggestions” were set, by me, based on my personal understanding and “feeling” for what I was measuring, my review of available literature on momentum investing, including the strengths and weaknesses of using momentum as the primary “indicator” and limited practical experience using the ranking spreadsheet to manage momentum portfolios successfully over the prior four to five years. These parameters have also been used in most of the examples reported on this site.
Despite the fact that I have always claimed that the default values represented a robust system I have never had any strong evidence to support this statement. In this study, we set out to stress test the system using, to the maximum extent possible, best practices in systematic back-testing. I stress the need to adopt best practices because back-testing inevitably leads to at least some degree of optimization – and this can be dangerous and unreliable for future applications.
Because we have six major parameters and a wide range of variable values for each parameter, there are literally thousands of possible combinations and we need to break down the possible combinations into sensible sub-tests that we can analyze. Since momentum is our key indicator it seems obvious that we should focus first on the ROC1 and ROC2 look-back periods. We therefore fix (almost) all other parameters such that we have:
1. Rutherford Portfolio: (VTI, VEA, VWO, TIP, TLT, PCY, VNQ, RWX, GLD, DBC and SHY);
2. Ranking Parameters: – ROC1 = 20-170 trading days in steps of 5 days; ROC2 = 20 -170 trading days in steps of 5 days; Volatility = 63 day Mean Variance/SD;
3. Weighting: ROC1 = 50%; ROC2 = 30%; Volatility = 20%.
4. Review cycle: 23 -2/+5 (Trading day)
5. Assets for inclusion in the portfolio: Top 2 ranked assets (and ties) selected with Equal Weighting;
a. 2 Assets ranked higher than SHY – 50% Asset 1, 50% Asset 2;
i) 2 Assets tied ranked 2nd – 34% Asset 1, 33% Asset 2, 33% Asset 3;
b. 1 Asset ranked higher than SHY – 50% Asset 1, 50% SHY;
c. No Assets ranked higher than SHY – 100% invested in SHY;
6. Test period: 06/30/2006 – present
7. EMA smoothing: None – raw data of daily closing prices
Before proceeding we should note that the 20 -170 (trading) day range selected for these tests was based on the results from a very long (over 2 days of continuous computing time) pre-test that covered look-back periods of up to 252 (trading) days since published results in the literature suggest that look-back periods of 3 – 12 months are most effective. The purpose of the pre-test was to consider look-back periods of up to 12 months (252 trading days) and to see if we could narrow down the most effective range in order to reduce computing time to an acceptable level – in this case it left us with “only” 14-hour tests going forward. We need to recognize that, although we have reduced the look-back period to a maximum of 170 days (~8 months) this does not necessarily mean that using longer look-back periods may not result in “superior” performance for different portfolios over longer periods of time – this will depend on how we measure superior. I’ll try to remember to come back to this issue at the end of the study.
Also, note that, in list item #4 above, we have included check-date noise (https://itawealth.com/2015/01/12/back-testing-issues-remain-skeptical-results/) in the tests since we feel strongly that this is a very important parameter that is generally ignored and we are not aware of any published studies that have addressed this issue anywhere in the literature. Again, in order to keep computing times to acceptable levels, the number of random sequences has been set at 10 rather than the 100 sequences used in other tests.
The result of a single back-test using the above parameter values is shown below:
The above “heat map” provides a two-dimensional picture of the relative value of total returns based on the 10 sequence average of back-tests using the 31 x 31 combinations of ROC1 (vertical axis) and ROC2 (horizontal axis).
The heat map is colored in relation to the percentile in which the total return lies within the complete range of values. The percentile range is divided into deciles, i.e. values falling in the top ten percent of the total range are colored dark blue while values lying in the bottom ten percent are colored bright red. Values lying in the deciles between the top and bottom ten percent are colored in various shades of blue through red with values in the mid percentile range being colored yellow:
Definition of “Robust”
When we say that we are looking for a system that is robust, what we are saying is that we want a set of parameters that, while not necessarily generating optimal performance under all conditions, might be expected to put us in a position whereby performance is ranked in maybe the top two deciles (20%) of possible outcomes. Thus we are looking for the areas that have the highest density of color in the blue range of the color spectrum.
Defining robust in this way means that, as market conditions change, we have relevant tolerance to changes in optimal settings and being slightly off the optimal settings (that we cannot pre-determine) will not hurt us too badly.
Important Special Features of the Heat Map
Although the heat map is a two dimensional picture of multi-variant momentum look-back periods, there is additional information “hidden” in the heat map that is useful:
- The values contained in the diagonal of the heat map (white line in the above figure) contain information on the “best” look-back values that might be used if an investor preferred to use a simpler single period look-back system similar to Antonacci’s Dual Momentum system.
The figure below shows the values of total returns along the diagonal of the above heat map:
From the above we can see that look-back values between 60 and 110 days might be expected to generate the best results.
- Since we have fixed the weighting of the ROC1 and ROC2 momentum periods at 50% and 30% respectively and since we have chosen to vary the ROC1 and ROC2 look-back periods between 20 and 170 along both axes, not only do we get information on what might be the best look-back periods to use, but we also get (some) information on the weightings that might produce the best results.
The white diagonal line in the above heat map acts as a mirror in terms of ROC1 and ROC2 look-back periods – in other words, we can find an X,Y combination of ROC1 and ROC2 both to the left and to the right of the diagonal. However, since our weightings of ROC1 and ROC2 are not equal, the “reflection” is not a symmetrical “mirror image” and we have a distortion in the patterns. We will come back to this in future parts of this study.
Part 2 Summary
In this part 2 of the study we have introduced readers to heat maps, described how and why they have been used and identified special features of interest contained in these plots.
However, the above heat map shows the output from only a single run (or a single average of one set of data with 10 different check date sequences) and, as such, is simply a single test optimization. This is not what we are looking for and does not represent best back-testing practice, so, in future parts of this study we will be extending the tests to include different price-time profiles and different portfolios/asset lists.
Special thanks to Herb Haynes for developing the software to plot the heat maps – they contain so much information that is far easier to visualize when presented in this way and you will be seeing many more of them in future parts of this study. My apologies to Herb for changing his original heat map colors – I actually prefer Herb’s color scheme – but it has been easier for me to define and explain the decile breakdown in the percentile groupings than it would have been to explain Herb’s original groupings and also easier (for me) to use Excel’s standard formatting capabilities. The differences are minimal other than in the coloring and have no impact on the interpretation of test results.
Herb Haynes, Ernie Stokely, Lowell Herr, David (HedgeHunter)