Testing A Simple Gap Strategy

November 10, 2014 5:00 am19 commentsViews: 7010

I was recently reading a book (more like a pamphlet, it’s only 93 pages), “Seven Trading Systems for the S&P Futures” by David Bean. I’m always interested in “older” books that have defined rules based systems in them to see if they really stood up to the test of time, and to kind of have an extended “Walk Forward Test” without having to wait around a couple of years. The book was written and published in 2010 and has 7 pretty basic gap trade systems. Quickly, for those unfamiliar with gaps, we are simply talking about the distance, or the “gap” in price, from the previous day’s close to the following day’s open. Pictured below is an example of a “gap-down” from the 9/10/14 close to the 9/11/14 open. As you can see, the closing price from 9/10 was 1987.25, the following morning on 9/11 we gapped down to open at 1979.

Gap-Example

This is also an example of a trade that would have been executed using this strategy, going long on the open of 9/11 at 1979.00 and exiting 4 points or $200 dollars later at 1983.00.

I’m only going to be looking at the Gap Fill #1 strategy from the book for the purposes of this article. There are several things that I really like about this strategy. First, it’s dead simple. Your hamster could manually trade it. Second, because it’s so simple, we can easily test the robustness of all the inputs and make sure the strategy was not overly optimized just to create a pretty equity curve (which for the purposes of the book, if Mr. Bean is being honest w/ himself, he did in fact do because he didn’t include any slippage or commissions in the book, and choose some rather, shall we say, convenient start and stop times for the back test). Anyway, let’s jump right in and see if Gap Fill #1 stands the test of time.

Trading Model Rules:

  1. Uses a 100 day SMA bull/bear regime on the daily chart
  2. If the S&P opens inside of the previous day’s range, and above the previous day’s close, a short will be taken (in a bear regime, aka S&P is under the 100 day SMA)
  3. If the S&P opens inside of the previous day’s range, and below the previous day’s close, a long will be taken (in a bull regime, aka S&P is above the 100 day SMA)
  4. There is a $500 stop loss, a profit target of $200, or exit on the close.

Notes:

  • $20 round trip was taken out for slippage and commission
  • Assumes $100,000 account
  • Only trades 1 contract and does not reinvest profits
  • Data1 is the ES 1-minute chart
  • Data2 is the ES daily chart

Baseline Results

The original in-sample test from the book was conducted from 06/02/2005-05/28/2010.

  • Net Profit: 13,152.50
  • Profit Factor: 1.39
  • No. of trades: 358
  • Annual rate: 2.48%
  • % time in the market: 1.61 %
  • % winners: 75.42%
  • Avg. Trade Net Profit: $36.74
  • Sharpe: 0.13
  • Max Drawdown: $2,175

Baseline-eq-curve

So that’s a pretty looking equity curve even with a taxing deduction for slippage and commission on 358 trades, but you probably figured it would test okay for the in-sample data, otherwise it wouldn’t have been published in a book. So let’s test the out of sample data from before the in-sample start, and after the in-sample ends, running up to September of this year. Basically, the same sample Jeff likes to use, inception of the ES up until now. This is where the rubber will really meet the road and we can see if the idea really holds up to the test of time.

In-Sample & Out-of-Sample Results

Testing the whole sample, out-of-sample previous, in-sample, and out of sample after 01/01/1998-09/02/2014 (side note: 1st trade in this test was on 10/1998 due to maxbar setting).

  • Net Profit: $28,837
  • Profit Factor: 1.21
  • No. of Trades: 1,335
  • % winners: 72.43%
  • Avg Net Profit: 21.60
  • Annual rate of return: 1.52%
  • Sharpe: -0.05
  • Max Drawdown: 4,552.50

All-sample-eq-curve

It looks like we might just have a winner here! BUT (there’s always a but) clearly we have some good news AND bad news here. The good news is that it literally, this month, (doing this study in the middle of Sept/2014) just made new all-time equity highs. The bad news, is that it was basically flat for the first 5 years of the study, from the beginning of 1998 to June of 2003 to go along with another long period of no new highs from the middle of 2011 to the middle of 2013. On a side note, never ever believe a day trading system’s results without commission and slippage included. As with this study that has 1,300 trades taken over 16 years, if I hadn’t added commission and slippage you would probably think this was the holy grail of systems, as you pay just as much in slippage and commission ($26,000) as you make in net profit. So now that we’ve seen that the baseline system has stood the test of time, let’s dig deeper and see if we can improve the variables and make sure they are robust enough to trust.

Testing Regime Lookback Period

Let’s start by testing the regime filter to make sure that it does in fact improve the performance of the strategy. We’ll first test the effectiveness by removing it from the trading rules. Below is the equity curve with the regime filter removed.

EQ-curve-without-regime

So it clearly helps, you get about 10k more in profit, with half the trades and a much better equity curve. Next let’s test the robustness of the regime filter by using TradeStation’s optimization feature. We will test values from 50-400 in increments of 50.

Testing-Regime-Lookback

So as you can see above, 100 is in fact the best value with the net profit tailing off ever so slightly the larger you make the regime filter. The good news is that all values are positive and everything over 100 performs very well with very little difference between 100 through 250. Next let’s test the profit target.

Testing Profit Target

We’ll test from $100 through $1,500 in increments of $100.

Testing-Profit-Target

More good news again, from the value of 200 and beyond, basically every value is incredibly robust with all targets testing positive. This is really where you can mold the system more to your personality without curve fitting. If you like more winners and small profits, that’s just fine. If you like going for the homeruns with less winners, that’s fine as well…whatever suits your trading personality. So clearly 400 is the best, but it is almost certainly an outlier. BUT…..I actually don’t have a problem if you want to use 400 as your profit target, as long as you don’t expect it do as well as it did in the back test. It made about 39k in the test, but I think a more realistic expectation would be in the neighborhood of 30k which seems to be top of the line for several other values.

Here is a quick look at the percentage of winning trades for each profit target tested. As you can expect, the higher the profit target, the lower percentage of winners you’ll have. It is interesting to note that after about a 600 dollar profit target, the percentage of winners really doesn’t drop off at all.

Testing-Profit-Target_And_Prcnt_Winners

Lastly, let’s test the stop loss.

Testing Stop Loss

We’ll test $100 through $1,000, using increments of $100.

Testing-Stop-Loss

So it’s very interesting that 400 is both the best value for a stop and a profit target. (I wonder if that has anything to do with the historical ATR of the ES?) As you can see, a stop that is too tight hurts the performance and a stop that is too large, while profitable, also hurts performance. The good news being that there is a large “sweet spot” for a stop, anywhere from 300 through 700 which is plenty robust. This is yet again another way you can mold the strategy to your trading personality.

Conclusion

So all in all I think this is a great strategy that has stood up to the test of time as well as our own testing. BUT (I told you there is always a but) there is so much more that could be tested on this strategy to improve the results and weed out some of those really bad periods. There are clear market “regimes” where the market gaps and then backfills the gap, and there are clear “regimes” where the market may be trending and leaves a lot of gaps open. A couple ideas to explore might be to test “equity curve” trading to capitalize on times when the market is in the correct regime for this strategy. Another idea is testing a profit target and stop loss that adjust to volatility or ATR to minimize risk and maximize possible return. What about using the gap fill itself as a target and half that value as a stop? A good position sizing model that can improve leverage in the regimes where this system thrives and decrease it or not trade at all during the bad regimes could really improve results and filter out a lot of unproductive trades. Anyway, that’s just a couple of ideas, I would love to hear others.

The ATR (Average True Range) for the ES in the entire sample is just over 18 handles. So my very simple and offhand theory on why $400 is the best stop and profit target, assuming 18 is the average range so 9 handles would approximately be the daily midpoint 9*50=450….so the theory then being your stop is on average just big enough to not get stopped out and just large enough to capture the optimum profit assuming two way trading (as only about 28% or so of days end up as trend days). Again, just my cocktail napkin theory.

— By Ben Little

You can follow me on Twitter @TRUmav. I always enjoy sharing and brainstorming ideas with like-minded traders.

Download

Other Article In This Series

Ben has been a private trader for over 7 years and is now a TradeStation Trading App Developer. Finding and exploiting edges is his game, as he’s not only a trader but a professional sports handicapper. You can find him on Twitter @TRUmav for financial commentary, or @BtheHouse for sports betting commentary. If your interested in more models like the one above, check out BtheHouse.com. Ben always enjoys sharing and brainstorming ideas with like-minded traders, bettors, and investors.

Twitter LinkedIn 

19 Comments

  • An interesting study!

    The system described here shares a common feature with so many other successful strategies (many of which Jeff has covered here in the past): it combines short term reversion to the mean (the gap fill in this case) with a long term movement away from the mean (the trend/regime identified by the MA and giving directional bias).

    Is the max drawdown that you give the maximum intraday?

  • Is this based on cash gap or 4:15pm EST close gap?

  • Thanks BlueHorsehoe! To answer your question, yes, the max drawdown is the Intra-day peak to valley.

  • Shawn, the strategy is based on the 4:15 EST close.

  • How do you get negative sharpe ratio with positive returns.
    I suspect your definition is not based on geometric average return

    In-Sample & Out-of-Sample Results
    Testing the whole sample, out-of-sample previous, in-sample, and out of sample after 01/01/1998-09/02/2014 (side note: 1st trade in this test was on 10/1998 due to maxbar setting).

    Net Profit: $28,837
    Profit Factor: 1.21
    No. of Trades: 1,335
    % winners: 72.43%
    Avg Net Profit: 21.60
    Annual rate of return: 1.52%
    Sharpe: -0.05
    Max Drawdown: 4,552.50

    • Steve – I think the negative comes about because TradeStation defaults to a risk free return setting of 2%. Over the test period that would have been > $30k, and the strategy return is < $30k.

  • Interesting article thanks. I agree it’s useful to come across older strategies that now have walk forward data – results tend to show how difficult this trading game is!

    It looks like it’s best to avoid going long in August or short in December. That could be curve fitting; or maybe it makes sense.

  • Steve, notice at the top of the article that the testing we ran assumed a 100,000 dollar account. For the purposes of our test, also remember that we are only trading 1 contract the entire time for every trade and never re-investing profits. You really only need a 5,000 dollar account to trade this strategy, 100,000 is just the TradeStation default. By changing nothing in the system, and only changing the amount of money we start with to 10,000 instead of 100,000, the Sharpe Ratio changes from -0.05 to 0.14. Really for individual strategies like this the Sharpe and annual rate of return are not useful as they can both be changed dramatically by simply adjusting leverage, or the amount of capital you start with. I could make all the stats and the equity curve look way better just by making a simple tweak to the position sizing, but that would be misleading.

  • Thanks Pete, I agree nothing “proves” an edge like an extended walk-forward test…the longer the better. I’ve noticed several times in other systems that I’ve tested that the equity curves will bottom out right at the end/beginning of a year. Market seasonality is very real, Jeff has done several well written articles on it, and I also have a study on the subject on my blog if you would like to check that out as well, you can find it on my twitter page, @TRUmav

  • Finally something good on gaps trading. I’m looking forward for a following post simply because this is a perfect add to my overnight trading.
    http://nightlypatterns.wordpress.com

  • Robert Jacobson

    I’ve actually been trading the Bean Gap strategy (with some modifications) for two years. I’ve generally had good luck with it. The downside is that it only trades a few times each month.

    I have one quibble: You’re testing this using Data1 on five-minute bars. That means that the opening order is placed (if the conditions are met) at 8:36. However, the book itself calls for trading this on one-minute bars, so the opening trade should be placed at 8:31.

    In my experience, the one-minute bars are much more profitable. It might be worth re-running your tests on one-minute bars. You can also optimize this by testing the results with 8:31, 8:32, 8:33 … as the time of your opening trade.

  • Thanks for the comment Robert! I just went back and double checked the workspace, everything was tested on the ES 1 min for data 1, it was just a typo on my part for the article, thank you for pointing it out, it will be corrected.

  • It would be good to provide the out-of-sample test results after 05/28/2010. Results before the in-sample may be useless because they may have been used to train an NN or genetic algorithm. It seems that after 05/28/2010 the performance is not good. Please provide results.

  • Alex, the results are in the article (2nd equity curve).

  • Tradestation can create virtual reality, if you end up walking into the trap of any kind of optimization. Unless of-course safe guards are implemented.

    Since in this exercise, you have optimized PT/SL, it helps to use “exceed limit price” setting.

    This is more so applicable for intraday trades which hinge on a small edge, trying to make a point on an average over many trades/months.

    And many times it happens so that the optimization engine will record a profit target hit, even when the price just touches it, which in reality you might never have been filled on your profit target.

    If you test this or any other of you strategy which you have a profit limit target, exceed limit price setting will give a more real picture of its performance. ( this ofcourse is not applicable for limit entries, which can never be backtested effectively to show real world performance and hence all entries if you are backtesting should be market and with appropriate slippage and commissions which I think you have done to some extent, but personally I prefer to use $30 roundtrip slippage+commisions which is on the worst case side, but it helps me to avoid looking at a pretty curve in the initial stages and something else in reality later.)

    just my 2 cents.

  • Hi Guest, thanks for the comment. The Profit target and stop loss were not optimized in the article, the optimization tests were simply ran to make sure that the default profit target and stop loss had not been overly optimized in 2010 when the system was published. But you make a great point in the exceed limit setting. I’ve personally seen several “good” scalping systems that did not have that feature enabled, and it greatly inflated the results. You also alluded to another good point that back tested results are almost never reflections of real world results…they are simply a baseline of expectations. I heard Perry Kaufman once say that he generally only expected 50% of backtested profits in the real world. I think that is a bit on the pessimistic side, but when crafting a strategy it certainly is better to prepare for the worst-case scenario, and hope for the best.

  • Interesting study. With a profit factor of 1.21, is this strategy even worth doing? I guess what I’m also asking is whether profit factor can vary like Sharpe Ratio and annualized rate of return can [vary in proportion to leverage]?

  • Hi Mark, while I would be the first to say that your not going to get rich trading this strategy, it does grind out a decent profit over the long haul, even if you only use the baseline. Profit Factor can definitely be affected by several factors though, leverage included. This particular strategy is really just a good building block to either improve upon or develop your own strategy off of the core idea that we know has an edge.

  • Thanks for a great post.
    I wonder if each trade always was 100k size?
    And if yes; is 0.4% profit target at all standard when it comes to morning gap targets, i thought it was more 1-2%?

Leave a Reply


Time limit is exhausted. Please reload CAPTCHA.

LEARN HOW TO OPTIMIZE WITHOUT 
CURVE FITTING

Download these four simple steps to help ensure you make money on the live market!

x