Learn To STOP Curve Fitting!
Download this free guide on how to stop curve fitting. Following these four simple steps can improve your trading dramatically!
In Part 1 of this series, found here, I ran a simple study of a trading system, in order to look at some of the common components (such as end of day closes, short versus long bar periods) that many system developers add to their trading system. Could some of these strategy building blocks be hurting their development efforts? To answer the question, I analyzed one trading system over 7 different markets for a 5 year period. Even though I was able to extract some conclusions from the study, I also said the following:
“Of course, I made these conclusions based on one study. What if the strategy was different? What if the timeframes or markets were different? What if different years were used for the test period? Will the conclusions reached here still hold? I’ll examine those questions in Part 2.”
To summarize the earlier work, the conclusions from Part 1 were:
In this second part of the study, I’ll now look at a different group of markets, a completely different strategy and some different timeframes. I’ll also try to answer the following questions: Will the conclusions still hold up? What type of insights, if any, do conducting studies like these actually provide? Finally, can studies like these improve your own strategy development?
Previously in Part 1, I tested 7 different markets across a range of commodities.
I picked these at random, one from each major market group. The test period will be from 1/1/2007 to 12/31/2011, a 5 year period that includes some quiet, and some very chaotic, markets.
For Part 2, I will test another basket of commodities.
I will also test the period 1/1/2012 to present, a smaller test period outside of the range previously tested.
Finally, I will assume $5 round turn commissions and $30 round turn slippage.
I will use two extremely simple base strategies for the tests I am running: a simple breakout based on closing prices (Strategies A and B), or a reverse breakout (countertrend – Strategies C and D). In all cases, the breakout length X will be varied from 5 bars to 100 bars.
Here is the code for all four versions of the strategy:
First, I will conduct a test of Strategies A and B for the same period, with the second group of commodities. To keep things manageable, only the group summary results are shown below:
The results of Group 2 are generally in line with Group 1 results, which show that swing trading (strategy A) is better than intraday trading (Strategy B), and that longer timeframes are slightly better than short timeframe bars.
Of course, concluding that long timeframe bars and swing trading is superior based on only a few tests is dangerous. So, let’s look at Strategies C & D (the opposite Strategies of A & B), and see if the results have the same tendencies for different timeframes and test period, on a random subset of Group 1 and Group 2 instruments.
For timeframes, the results show a tendency for the smaller timeframes to be better than the longer timeframes, exactly the opposite of what was found earlier. But this effect is small, which could mean it is not actually true, and could be just an artifact of the limited testing. This is a good example why it is dangerous to draw conclusions from only a few tests.
For the swing vs. intraday testing, the results are pretty clear – intraday testing is not nearly as good as swing trading. This agrees with the earlier conclusion.
Through both Part 1 and Part 2 of this study, a lot of tests have been run over various bar timeframes, instruments, strategies and time periods. Yet, it would be dangerous to make any sweeping conclusions based on these tests. Although these results show that intraday trading is less profitable than swing trading, there are surely cases where intraday trading is preferable over swing trading.
So, what can this study actually teach us? I think the following lessons come out of this study:
The key to successful strategy testing and development is to keep an open mind, and allow the data to point you in the right direction. At the same time, though, don’t let the data trick you into making false conclusions or sweeping generalizations. Remaining flexible in your development approach can lead to new and profitable strategies – ones that you might now be dismissing based solely on past testing.
If you would like to learn more about building trading systems be sure to get a copy of my latest book, Building Winning Algorithmic Trading Systems.
— Kevin J. Davey of KJ Trading Systems
Kevin Davey is a professional trader and a top performing systems developer. Kevin is the author of “Building Algorithmic Trading Systems: A Trader’s Journey From Data Mining to Monte Carlo Simulation to Live Trading” (Wiley Trading, 2014.) . He generated triple digit annual returns 148 percent, 107 percent, and 112 percent in three consecutive World Cup of Futures Trading Championships® using algorithmic trading systems.
His web site, www.kjtradingsystems.com, provides trading mentoring, trading signals, and free trading videos and articles. He writes extensively in industry publications such as Futures Magazine and Active Trader and was featured as a “Market Master” in the book The Universal Principles of Successful Trading by Brent Penfold (Wiley, 2010).
Active in social media, Kevin has over 15,000 Twitter followers. An aerospace engineer and MBA by background, he has been an independent trader for over 20 years. Kevin continues to trade full time and develop algorithmic trading strategies.
Please log in again. The login page will open in a new window. After logging in you can close it and return to this page.