The post Are your backtest results fooling you? appeared first on System Trader Success.

]]>Have you ever started trading a strategy that performs well in the backtests but delivers a very different result when you begin trading it with real money?

Could your backtest reports be fooling you by indicating a strategy is great but really only showing you part of the overall picture?

How do you give yourself a better chance of developing trading systems that are robust and perform well going forward?

Kevin Davey (not the guy pictured above!), World Cup Trading champion from kjtradingsystems.com, has been creating trading strategies for over 25 years. In Episode 5 of the BetterSystemTrader podcast he says:

It’s amazing how easy it is to create systems that you think are good that just fall apart. –...

To reduce the chance this could occur he completes Monte Carlo analysis on all his systems to ensure they are robust and meet his risk requirements BEFORE he puts his money on the line.

What is Monte Carlo analysis and how can it be used to improve your own trading results? Read on, we’re going to show you.

Monte Carlo analysis is a process that allows you to get a more accurate picture of the performance of a trading strategy beyond what a standard backtest report can provide.

A backtest report shows the results of a series of trades in a specific order, but the problem is that’s just history, you don’t know what’s going to happen as it goes forward. What if a lot of losing trades all show up in a row, what type of drawdown will you experience? What’s the chance that you could get a drawdown larger than anticipated, or a string of losing trades longer than expected?

Monte Carlo analysis basically lets you scramble the order of the trades in a backtest to provide a better understanding of possible future performance, based on the assumption that future trades will have similar characteristics to historical trades but in an unknown order.

The results allow you to determine the probabilities of drawdown and profit levels and the chance your trading account could be completely wiped out.

Your worst drawdown is always in front of you.– Kevin Davey

Yes, even the seasoned pros like Kevin use it and this is why:

I’ve actually found cases where the walk forward equity curve looked great – probably a lot of people just made the decision, “Hey, I’m going to trade it.” But when I ran the Monte Carlo simulation I found out that there was really a lot more risk in the system and it was a lot riskier than I had anticipated. So basically the amount of return that I was getting compared to the amount of risk I could have, that didn’t necessarily show up in that historical equity curve, was just too much for the profit I was getting and so I basically said, “Well I can’t trade this particular system.”

Kevin has kindly offered a free copy of the Monte Carlo analysis tool he’s developed in Excel, for all Better System Trader podcast listeners. There is a link to download the tool at the end of this article but let’s first see how it works and how to apply the results to our own trading.

When you open the simulator, there are a few values you need to enter based on your own personal trading parameters. (If it prompts you to enable the macros you will need to say yes otherwise the simulator won’t work).

To setup the simulator enter your trading details in the light blue sections, starting in the top left with the base starting equity, the level at which you would stop trading the system if the account equity fell below it and the average number of trades per year:

To enter your trades into the simulator press the ‘Clear’ button and paste the list of trade profit and loss in $ from your backtest report.

For this example we’ll use a list of 1805 trades over 10.5 years. Based on a $10,000 starting balance the CAR is 31% and Maximum Drawdown is 11%, which results in quite a smooth equity curve:

The results may seem impressive but let’s run it through the Monte Carlo simulator. By adding the trades into the simulator and pressing the Calculate button, the simulator runs through the list of trades 2500 times, randomizing the sequence of trades each time. We’ve set a starting equity of $10,000 to match the backtest and the stop trading level has been set to $8,000.

The results from the simulator are very interesting.

We’ve run the trade list through the Monte Carlo simulator and now it’s time to compare the results with the backtest:

The first thing to notice in the Median Drawdown for the Monte Carlo simulations is 24.6%, however, the backtest reported a Maximum Drawdown of 11%. How can this be?

By switching the order of the trades we’ve identified that the strategy actually contains more risk than the backtest report shows. The favorable sequence of trades in the backtest is understating actual risk!

Also, if the backtest report only produces a drawdown of 11% but the Monte Carlo Median Drawdown is 24.6%, there are likely sequences of trades that have produced 50% drawdowns or larger, much higher than the drawdown limit of 20%.

Note that trading this strategy with a $10,000 starting balance has a 33% chance that it will meet or exceed the 20% drawdown limit. This risk of ruin is much too high.

The Monte Carlo results have shown that starting with a $10,000 account and a 20% drawdown limit we have a 33% chance of ruin and the Median Drawdown of 24.6% is higher than our drawdown limit. What can we do about this?

Without adjusting the strategy rules or risk per trade it seems the best approach is to start with a higher account balance. By checking the yellow results table in the Monte Carlo simulator we can see that we should probably trade this strategy with $25,000 or higher:

We can now see the importance of Monte Carlo analysis in the system development process. This basic example has shown us how backtest results, which only show the performance of one order of trades, may not be showing the full picture.

By running the trade list through the Monte Carlo simulator we’ve determined:

- The Maximum Drawdown value in the backtest report (-11%) was based on a favorable run of trades and was understating the actual risk of drawdowns, with the Monte Carlo simulations showing a Median Drawdown of -24.6%
- The Risk of Ruin when trading a $10,000 account size was 33%, much too risky to trade, therefore a larger account size or smaller trade risk would be required to reduce the possibility of ruin.

To get your free copy of the Excel document please visit this link and scroll to the bottom of the page. The Excel document is provided by Kevin Davey and Better System Trader. Be sure to listen to Kevin Davey’s interview on Better System Trader.

**UPDATE**: It looks like the download link for Kevin Davey's free Monte Carlo tool is not working so here is a directly link to download it: >> Free Monte Carlo Spreadsheet <<

— by Andrew Swanscott from Better System Trader

The post Are your backtest results fooling you? appeared first on System Trader Success.

]]>The post A Simple S&P System appeared first on System Trader Success.

]]>The post Testing A Simple Gap Strategy appeared first on System Trader Success.

]]>The post The Overnight Edge appeared first on System Trader Success.

]]>The post Inverting The RSI and VIX Strategy appeared first on System Trader Success.

]]>The post Capture The Big Moves! appeared first on System Trader Success.

]]>The post Finding The Optimal Period appeared first on System Trader Success.

]]>Today’s topic is on finding the optimal periodicity for the RSI indicator and the techniques should apply to other sorts of indicators as well. The good news is that it won’t require any rocket science. John Ehlers has written a lot about this topic, and his articles spurred my interest in it, which frankly I had never considered. I might should note now that I have not reviewed the actual methodology that Ehlers used in any depth but my impression was that the methods were very advanced. However, I realized I could tackle this problem using just basic statistics.

Before we start analyzing specific solutions, I want to take a moment to describe what we are actually attempting to do. The question that we are trying to answer is how do we find the optimal length to use when calculating an indicator. A poor length value may result in the RSI spending too much time in either the overbought or oversold states, or failing to reach the proper levels.

Obviously, the optimal lookback will change as market conditions change and therefore our methods should be adaptive and responsive to the market. The goal is to find a metric to rank how well behaved the RSI is. A well behaved RSI is one that spends about an equal amount of time in overbought and oversold levels. During a trending market, if the RSI gets stuck into overbought conditions then that would be an example of a poorly behaved RSI. The solution thus is simply to measure the amount of time that the RSI spends in overbought and oversold conditions.

My original inspiration came from the idea of a histogram. We simply add 1 for every bar that the RSI is above our threshold and subtract 1 for every bar that the RSI is below our threshold. The absolute value of the resulting summation is our error metric, which we seek to minimize. Key Concept: Sum the amount of time that the RSI spends above 50 and below 50, and take the difference. The absolute value of the difference is our error metric, which we seek to minimize.

Next, we will simply compute our error metric for every RSI length in our search space, and select the RSI with the minimum error. In this example, I searched from RSI(2) to RSI(14). There is one more consideration which is determining our lookback. Too short of a lookback will lack statistical relevance while too long of a lookback will defeat the benefits of adaptation. Some manual trial and error were used before I settled on a lookback of 90 days.

Test We conducted the test using the S&P 500 Emini and have selected the long only RSI(2) with standard 30/70 overbought/sold thresholds as our baseline. The results are below:

@ES Daily 1/1/2004 - 1/29/2017

RSI(2) Simple | RSI (2..14) 50/50 | |
---|---|---|

Net Profit | $86,075.00 | $82,812.50 |

Max DD: | $16,937.50 | $17,887.50 |

Profit Factor: | 1.84 | 1.98 |

Avg Profit Per Trade | $304.15 | $363.21 |

Number of Trades | 283 | 228 |

Win % | 74.91 | 75 |

While the results are not too different, we do see our profit factor and average profit per trade has improved. The net profit is slightly reduced and the drawdown is slightly higher but essentially similar.

Tightening The Threshold We can get better results by making our threshold more restrictive. Instead of using the median line, we can use the amount of time that our indicator spends in actual overbought and oversold levels. This makes some sense as the overbought/sold conditions are what’s really important. We use the same technique but now we try to balance the amount of time we spend below 30 and above 70. We see significantly better results. The results improve even more when we try to balance the extreme overbought/oversold conditions using 20/80 as our thresholds: notice that we do not change our actual buy/sell conditions.

@ES Daily 1/1/2004 - 1/29/2017

RSI (2..14) 30/70 | RSI (2..14) 20/80 | |
---|---|---|

Net Profit | $73,387.00 | $83,375.00 |

Max DD: | $16,937.50 | $16,937.50 |

Profit Factor: | 2.53 | 2.54 |

Avg Profit Per Trade | $824.59 | $850.77 |

Number of Trades | 89 | 98 |

Win % | 74.16 | 77.55 |

Final Analysis & Summary Both the 30/70 and 20/80 optimization thresholds show significant improvement in the profit per trade over the baseline, fewer trades, and higher profit factor. This suggests that our RSI optimizations are improving trade quality significantly and suggests that our indicators are better synchronized to the more dominant cyclical structure of the market.

While the net profits are not quite as strong, the strong increase in average profit per trade is especially appealing. It appears that we can find significant cyclical structure in the markets using simple statistical techniques. However, we did have to use some trial and error in determining the lookback to calculate our statistics over, and that may be one area for improvement.

––Curtis White from Beyond Back Testing

The post Finding The Optimal Period appeared first on System Trader Success.

]]>The post Two Swing Trade Systems – Part 2 appeared first on System Trader Success.

]]>Yesterday I discussed two swing-trade systems that work pretty well in out-of-sample data. While each works differently, they overlap enough that you don’t get any benefit from running them both at the same time. One great thing about these two systems is that they’re dead simple to manage. Trade at the open or the close, simple math, etc., etc.

I will repeat the caveat from yesterday: these trades average <1% gain per trade. You must have sufficient capital and/or a low/nonexistent commission fee to make these work. While you can use leveraged ETFs or account leverage to help increase the profit/commission ratio, you also increase your chance of a catastrophic hole in your money.

In the lead image, you can see that I have indicators for both RSI and PIRDPO. PIRDPO occurs more frequently, and the RSI trades are a complete subset of the PIRDPO trades (during this particular time frame). There is no benefit to trading both systems.

- Calculate the two-period RSI value of SPY at the end of the day. You could for example use this free online chart.
- When the value is less than or equal to 12, there’s your signal.
- Buy at the open of the next day.
- Sell at the first close that is higher than your buy price, but you can’t sell the same day you buy.
- Exit ten days later at the close, if still not profitable.
- Don’t enter into a second trade if you get a new signal while still in a previous trade.

That’s it. “WTF? Where are the stops?!” you say. To which I reply, watch your language. Sure there’s a stop…it’s a time stop. You get out after a set period of time. It has been my experience that swing trades rarely benefit from defined stop loss exits. They sometimes don’t even benefit from profit targets, but in this case the profit target is simply “any”.

If you think this sounds stupid, please refer back to the chart in the previous post.

First off, what does “PIRDPO” stand for? “Position In Range Detrended Price Oscillator”. Just say “purd-poh” for ease of use. This one requires that you keep a spreadsheet.

- Keep track of the closing prices of SPY for every day.
- Calculate the average of the last 11 days in another column. This will be a moving average (“MA”).
- In the next column, calculate the detrended price, which is ( Close – MA ) / MA
- In the next column, calculate the minimum of the last 11 days worth of detrended prices.
- If today’s detrended price is the lowest of the last 11 days, there’s your signal.
- Buy at the next day’s open.
- Sell at the first close that is higher than your buy price, but you can’t sell the same day you buy.
- Exit ten days later at the close, if still not profitable.
- Don’t enter into a second trade if you get a new signal while still in a previous trade.

Pretty flippin’ simple, eh? Now go make some money.

--By Matt Haines from blog Throwinggoodmoney

The post Two Swing Trade Systems – Part 2 appeared first on System Trader Success.

]]>The post Two Swing Trade Systems (Part 1) appeared first on System Trader Success.

]]>In my recent posts I’ve made reference to various swing-trade systems I’ve developed, which I’ve used as data to discuss things like correlation, diversification, and the value of leveraged ETFs. I’ve had a number of people say, “Hey, what are these systems you’re referencing?”. The short answer is: they’re not something I’m discussing publicly. “But that’s not fair!”, you cry. “Everything should be free!”.

Yeah OK, I see your point. I’m still not going to divulge those systems, but I do have two interesting swing-trade systems that I *will* share with you. They’re simple to execute using nothing more than online tools and/or spreadsheets. More importantly, they were optimized using data from 2000-2009, and perform well in out-of-sample (“OOS”) data (2010-2017).

Disclaimer: I am not your financial advisor. I advise you not to trade at all because it’s dangerous and sometimes you lose money. Put your money in the bank instead. Both systems trade SPY, the S&P 500 ETF. The graph below shows the growth of $30,000 with the entire account invested in each trade, and no commissions or fees were taken out. The drawdowns (“DD”) in the graph are on a cash basis, so they naturally get larger as the account gets bigger.

I especially like how these two systems handled the **Kerfluffle of 2008**. Sure there were some bouncing around but money was made none the less. In fact, the **Inconvenience of 2011** actually showed worse results. I originally slapped together the RSI system– I mean really, it took me like 10 minutes – so that I had a baseline to compare other systems with. I was fully expecting it to fall apart in OOS testing, but it didn’t. Something as long in the tooth as a short-period RSI system still had the potential to make gains. Who knew? The only caveat is that the RSI system is somewhat flat over the past two years.

You could say the same thing about the PIRDPO system in 2004-2005 though, so it happens. Doesn’t mean it’s broke. “But Matt, what is this PIRDPO system?”, you might ask. Looks pretty good, eh? It trades more frequently than the RSI system, and from the graph it looks more profitable but with some bigger drawdowns. How do they compare? Before you trade either one of these systems, there’s something you should know: they make tiny profits on each trade. I mean TINY. You can not be putting on $1000 positions with a $10 commission each way and expect to make any money.

You must either have a large account or a small commission fee (or both). If you decide to use one of these systems, you probably wouldn’t want to use BOTH of these systems. Why? They overlap quite a lot. In fact, 76.62% of all RSI days in a trade are also in play using PIRDPO. They are correlated time-wise so that you get little benefit from running them at the same time. Here are some stats for each system:

RSI 2000-2017 | RSI- 2010-2017 | PIRDPO2000-2017 | PIRDPO 2010-2017 | |
---|---|---|---|---|

% Profit | 235.82% | 43.83% | 354.89% | 74.51% |

#Of Trades | 250 | 96 | 327 | 142 |

Avg. % gain/loss | 0.51% | 0.40% | 0.48% | 0.41% |

Winners | 217(86.80%) | 83(86.46%) | 282(86.24%) | 121(85.21%) |

Max consecutive wins | 25 | 13 | 22 | 15 |

Losers | 33(13.20%) | 13(13.54%) | 45(13.76%) | 21(14.79%) |

Max Consecutive Losses | 2 | 2 | 3 | 2 |

Max Trade % DD | -17.17% | -14.04% | -17.17% | -13.20 |

Max System % DD | -17.14% | -15.19% | -17.15% | -14.10% |

CAR/MAxDD | 0.43 | 0.35 | 0.54 | 0.58 |

Sharpe Ratio | 1.58 | 1.34 | 1.66 | 1.61 |

A helpful tip to figure out your position size: First, determine your commission costs for the in/out trade. Let’s say they’re $10 each way, so a total of $20. Then determine what percentage of your average profit can be eaten up by commissions. Let’s say 10%. $20 / 0.1 = $200. Your average profit per trade needs to be $200. If the average profit is 0.40% (worst case in the chart above), your position size needs to be $200 / 0.004 = $50,000. pos size = ( commission * 2 ) / ( % of profit ) / ( average % gain per trade ) You did see the title of this post, right? It says “Part 1”. Next week will be the actual details of the systems…

–By Matt Haines from blog throwinggoodmoney

The post Two Swing Trade Systems (Part 1) appeared first on System Trader Success.

]]>The post Discover Best Trading Rules with Perceptron Search appeared first on System Trader Success.

]]>Every system developer has a list of rules and indicators that they want to test out. The problem every developer faces is narrowing down that list of rules and indicators to the select few that are best worth focusing on. A quite similar and related problem that every developer has to deal with is having some inputs or indicators that they believe to be relevant and important but not knowing exactly in what way they are significant. I will show you how you can quickly find the best rules and indicators to focus on using an automated search approach which will work on most traditional retail trading platforms (such as Tradestation, Multicharts, Ninjatrader, Amibroker, etc.). But first I should point out that there are two basic approaches to developing systems.

The manual method also known as the hypothesis and test method (guess and check), quite similar to the scientific method, and the generative method also often thought of as the “data mining” approach. It seems that most professionals use the scientific method because the generative method lacks sufficient control over the process. The problem with the traditional method is that it can be really slow and self-limiting.

Computers today are tremendously powerful and fast, and yet the system developer using the traditional approach derives very little benefit from all that increased computing power and resources. Clearly, a developer who can’t utilize the advances in technology will be at a significant handicap.

Fortunately, it is not an either-or scenario because the optimizers found in retail trading platforms can be used for not just optimizing inputs but also for discovering the best trading inputs and rules to use. The key idea is to use the optimizer to find the most relevant rules or indicators. One of the first ideas that I had was to chain conditions together using operators such as “or” and “and”. This can be achieved by creating an array of rules with associated bit states which the optimizer can turn on and off.

Tradestation has the limitation that it can’t optimize Boolean fields. The solution is to simply convert everything to “0” and “1” numeric. Create a list of rules and allow the optimizer to turn them on or off.

` Inputs:` |

A rule must both be turned on and have its corresponding input set to true for it to trigger. This allows the optimizer to turn rules on and off. Each associated “buy condition” is optimized (0 to 1, step 1). The method works and is quite powerful in itself. But the problem is you might start to think maybe we should link the rules together using an AND statement instead. OR means any rule can be own whereas AND requires all the rules to be true.

Also, maybe some conditions should signal to the system not to take a trade; that they have negative input. The number of inputs multiplies and the method quickly become unworkable. Clearly, while this method is powerful, it has too many limitations. The solution is to change the Boolean “on/off” conditions to weights, as shown below:

`BuyConditions[5](0);` |

We sum the weights and a buy is only triggered when the combined weights are greater than our threshold value. This is known as a perceptron. The perceptron is the precursor to the much more advanced neural network (a question for mathematical folks, what’s the difference between this and a linear function approximator?).

For simple searches, you optimize the weights with values 0 or 1. However, you can create partial weighted functions where multiple inputs can contribute to the output by optimizing from 0 to 1 in fractional increments. By changing the step size, you can define the minimum contribution that each input needs to have for the perceptron to “fire”. For example, if your threshold is 1 and your minimum step is .5 then any indicator would need to contribute at least 50% to have an impact or else would be forced into the off state.

An additional reason why this model is more powerful is that you can optimize from -1 to 1 which enables indicators to have a negative or inhibitory effect. Perceptron search can quickly reveal which rules and indicators are most likely to be relevant and worthy of additional exploration. As for techniques for additional development, converting the trading signals into normalized continuous output functions could eliminate the need to even define the initial rules.

— by Curtis White from blog Beyond Backtesting

The post Discover Best Trading Rules with Perceptron Search appeared first on System Trader Success.

]]>