Learn To STOP Curve Fitting!
Download this free guide on how to stop curve fitting. Following these four simple steps can improve your trading dramatically!
In a recent article entitled, “Backtesting the MAC-System – How Long is Long Enough?“, the author presented a simple rotational trading system based upon moving averages. The system is one of the most basic rotational systems you could devise. Based upon signals generated from the moving averages, your investment is either within the S&P index or a bond fund. Several readers voiced their concerns over the possibility of the system being over optimized to the historical data. This is a valid point and worth looking into.
When it comes to making a profitable trading system we depend upon finding market edges we can exploit. All successful trading relies on recurring patterns found in the historical market data and exploiting those patterns to make a profit. This is true for discretionary traders as well as system traders. Optimizing our system to take advantage of market edges is not the problem. We want to optimize our system to take advantage of an edge.
Yet, much of the historical market movement is noise or random movement. So, the problem of over optimizing our system to the historical data starts to happen when our system begins to key-into patterns within the noise instead of exploiting a true market edge. Finding a pattern in the noise can produce great looking equity curves on historical data, but on out-of-sample data the equity curve can break down rather quickly.
In summary, we want a trading system that exploits a true market edge and avoids keying-off patterns in the market noise. Optimization is not the problem. Over fitting is the problem and having the ability in separating a true market edge from the noise is what this game is all about.
So, how can we test the MAC system to help us determine if it’s overly optimized? When looking at a trading model, it’s important that the parameters demonstrate robustness across a wide range. That is, the system should remain profitable over many different values. We should be able to change some of the key input values and we should have a viable looking trading model. The output (net profit in this case) should not drastically change.
In order to explore the robustness of the basic premise of this trading model I created a stripped-down version in EasyLanguage that buys and sells the S&P cash market only. The complete trading model, as presented by the author of the MAC system, has rules to move cash into a bond fund during “bear” markets, but I’m particularly interested in testing the buy/sell signals generated by the various moving averages as these seemed to produce the most amount of questions by readers.
Our trading model is very simple with signals to go-long the S&P cash market and when to close our position. The buy/sell rules are below.
The 34-day exponential moving average (EMA) of the S&P 500 becomes greater than 1.001 times the 200-day SMA.
The 40-day simple moving average (SMA) of the S&P crosses below the 200-day SMA.
I coded the above rules in EasyLanguage and tested them on the S&P cash market. Before getting into the details of the results, let me say this: all the tests within this article are going to use the following assumptions:
Shares = $5,000 per trade / 5 * ATR(10)
Just using the simple rules above we get the following equity curve and results.
The baseline rules call for a 200 SMA to act as a major trend filter. Is this 200 value just an outlier? Or, is this parameter robust for a wide range of values? To test the robustness of this input, I will use TradeStation’s optimization feature which will allow me to quickly test a range of values. I will test the range 60 – 260 in increments of 10 for the input value used in the SMA calculation. The results of my test are below. The x-axis displays the look-back period in days while the y-axis displays the net profit.
The optimal value (260) is not the default value (200). Many values show profitable results. The default value and above all show very similar results.
The baseline rules call for a 34 EMA to rise above the 200 SMA a given amount to trigger a buy signal. Is the 34 period used for the EMA calculation overly optimized? To test the robustness of this input, I will use TradeStation’s optimization feature which will allow me to quickly test a range of values. I will test the range 5 – 75 in increments of 5 for the input value used in the EMA calculation. The results of my test are below. The x-axis displays the look-back period in days while the y-axis displays the net profit.
The optimal value (20) is not the default value (34). All values show profitable and very similar results.
The baseline rules call for a 34 EMA to rise above the 200 SMA by a factor over 1.001. Is this factor overly optimized? Again, I will use TradeStation’s optimization feature. I will test the range 0.990 – 1.010 in increments of .001 for the input value used in the factor calculation. The results of my test are below. The x-axis displays the factor value while the y-axis displays the net profit.
The optimal value (.98) is not the default value (1.001). All the values show profitable and very similar results to the default value.
The baseline rules call for a 40 SMA cross below the 200 SMA to trigger a sell signal. Is the 40 period used for the SMA calculation overly optimized? Once again, I will use TradeStation’s optimization feature to quickly test a range of values. I will test the range 5 – 75 in increments of 5 for the input value used in the SMA calculation. The results of my test are below. The x-axis displays the look-back period in days while the y-axis displays the net profit.
The optimal value (55) is not the default value (20). All values show profitable results. Values between 35 and 60 all show similar results.
It appears this trading model is not overly optimized. Through my simple testing I was able to demonstrate that the core of the MAC trading system shows profitability over a wide range of values. In fact, some combinations produce better results. The default values do not appear overly optimized. Note, many different values appear to work just fine.
It’s my opinion the MAC system will likely work into the future. It works because it positions a trader into bull trends as seen from 2009 to today (late 2014) while more importantly, it gets a trader out of the market during prolonged bear markets such as seen in 2007-2008. The exact parameters don’t matter! You can use a 260 SMA or a 180 SMA for the major trend filter. You can enter the market using a 40 EMA or a 20 EMA. You can use a factor of 1.001 or .0098 as the buy threshold. They all produce positive results which speaks to the robustness of the system.
It’s true the testing period within this article only produced 16 trades (samples) and this is a very low value. Part of this is due to the fact I don’t have all the historical data at my disposal to test. However, based upon the authors original work of going back to 1955, I would feel confident that we would have a sufficient number of samples.
Simple systems on daily charts applied to a broad market index provide a systematic approach that often will do far better than what most people do. That is, make buying and selling decisions on emotional reactions.
Several people have asked about this model on SPY. Strictly speaking, the MAC-System presented in this article is not the complete MAC-System. The EasyLanguage code I wrote for the above tests was designed strictly to test the robustness of the input values – not to test the overall performance of the strategy. The trading model presented is a stripped down version of the MAC-System. I would refer you to the article “Backtesting the MAC-System – How Long is Long Enough?” for a more complete description of the performance of SPY on the MAC-System
However given the strategy code presented in this article is not the complete MAC-System, below are the results when applied to SPY. Profits are reinvested. All trading equity is invested for each trade. $16 commissions and $.04 slippage per share was deducted per round trip. The TradeStation Performance Report can be download here MAC_Test_SPY.
Jeff is the founder of System Trader Success – a website and mission to empowering the retail trader with the proper knowledge and tools to become a profitable trader the world of quantitative/automated trading.
Please log in again. The login page will open in a new window. After logging in you can close it and return to this page.