I recently had someone email me about the performance of a strategy I created back in late 2005/early 2006 and traded for a few years. I remember the strategy being a daily mean reversion set up with an intraday pullback entry. I figured it probably had not done well over the last decade. I stopped trading in the middle of 2008 because I did not like how it was behaving. In the backtest it did well in bear markets but was not doing so in the middle of 2008.
I ran the strategy from 2007, using the rules as they were published and was pleasantly surprised by the results. A CAR of 25%. Overall not too bad. Wish I had still been trading it. This is an 11-year out of sample test.
Then I remembered, didn’t this have insane results from 2000 to 2005? Here are those results.
Did you notice that 595% return in 2003? Now I am thinking what the heck happened since 2007. Look at the drop in yearly returns. Was the original strategy overfit and flawed? Did the markets change? If so, what is the difference? What follows is my investigation into trying to determine if the strategy is broken or the markets have changed.
I cannot share the rules because of the NDA I signed when it was created. What I can say is that it is a very typical mean reversion strategy that trades stocks.
I will include 2006 in the In-sample period because the return for the year was in line with previous years.
Was the initial strategy curve fit? I do not have the original code so I made a best guess of the parameters I would have optimized on and their values. The strategy CAR was 1.3 standard deviations from the average of all the variations. Even the worst variation still had a very good CAR from 2000 to 2005 and saw similar drop in performance. This variation may have been a little overfit but that is not the entire story.
Is there a decline in the number of trades? Average number of trades per year:
Wow, a drop of 50%! That starts to explain a lot. But why less trades?
Are fewer stocks passing our liquidity filter? This is the average per year:
A small decline but it does not explain the large drop.
Since the strategy depends on intraday sell off, maybe we are seeing less setups. This is the average per year:
That explains the lower number of trades per year. We have less setups. This means we are seeing fewer stocks sell off and setting up for a mean reversion trade. This explains a lot. But let us see if it is the entire story.
Is the quality of the trade dropping too? Meaning what is the average % profit/loss per trade?
Another drop of 50%. A double whammy. These last two stats explain the drop in the performance. But I wanted to know more.
Are the trades less volatile and is that why they are not as big? These are the average 100-day historical volatility of the trades:
That is not leading to the decrease. What is causing the decrease in average % p/l? I don’t know what else to look at.
Is this strategy broken? I don’t think so. But what has changed is the market, which has greatly reduced the returns. Having fewer setups and smaller gains on the trades seems to have caused the reduction in returns. As to why, my theory is that it’s because of the popularity of mean reversion strategies and quantified trading. Do you have any ideas on what is going on or other tests you want to see? Post them in the comments below.
Remember we had eleven years of out-of-sample data to work with to determine if the strategy was originally overfit or if it had broken, or if the markets had changed. Even with all these data it was not completely easy to figure out what was going on. The next time your strategy starts to perform poorly after 3 to 12 months, remember how hard it can be. I dropped the strategy after 6 months. It was not broken, the markets had changed.
--by Cesar Alvarez from Blog Alvarezquanttrading
Contributing authors are active participants in the financial markets and fully engrossed in technical or quantitative analysis. They desire to share their stories, insights and discovers on System Trader Success and hope to make you a better system trader. Contact us if you would like to be a contributing author and share your message with the world.
Please log in again. The login page will open in a new window. After logging in you can close it and return to this page.