One of the biggest issues with system development is that many trading strategies do not hold up into the future. This could be due to several reasons:
There are several approaches and methodologies to asses robustness and increase likelihood of positive real life trading performance including:
One of the most robust ways of testing the reliability of a trading system and making sure the program will have the highest likelihood of performing well in real live trading is to use walk forward optimization (WFO), a method first described in the book “Design, Testing, and Optimization of Trading Systems” by Roberto Pardo.
Walk forward analysis is the process of optimizing a trading system using a limited set of parameters, and then testing the best optimized parameter set on out-of-sample data. This process is similar to how a trader would use an automated trading system in real live trading. The in-sample time window is shifted forward by the period covered by the out-of-sample test, and the process is repeated. At the end of the test, all of the recorded results are used to assess the trading strategy.
In other words, walk forward analysis does optimization on a training set; tests on a period after the set and then rolls it all forward and repeats the process. We have multiple out-of-sample periods and look at these results combined. Walk forward testing is a specific application of a technique known as Cross-validation. It means taking a segment of data to optimize a system, and another segment of data to validate. This gives a larger out-of-sample period and allows the system developer to see how stable the system is over time.
The picture below illustrates the walk forward analysis procedure. An optimization is performed over a longer period (the in-sample data), and then the optimized parameter set is tested over a subsequent shorter period (the out-of-sample data). The optimization and testing periods are shifted forward, and the process is repeated until a suitable sample size is achieved.
In order to demonstrate the concept we will perform in this article a walk forward optimization on a volatility breakout trading system (VBO). For the test we will use the German DAX futures, NinjaTrader, CQG historical 1-minute data, and we will assume 3 points of slippage for each R/T trade to cover trading frictions.
The process consists of three main steps:
Definition of the in-sample and out-of-sample periods
We will choose as in-sample 1/1/2001 to 12/31/2009 for system design and in-sample optimization and 1/1/2010 to 12/31/2012 as out-of-sample period to evaluate the in-sample optimization robustness and execute the walk forward. We will then use a 3:1 ratio for the WFO (walk forward optimization):
Define the robust parameters area in the in-sample period
In this section we will define the “robust area” of the system parameters. We will optimize only 3 system parameters:
Other system parameters that we will not optimize are:
As we can see the average SQN (system quality number) tends to decrease as we increase the time period of the chart in minutes. We will choose 14 minutes for all simulations going forward.
For more info on SQN:
The NinjaTrader code for the system quality number can be downloaded here:
We define “robust” a parameter surface area that does not have major peaks or valleys and has generally a good performance.
Definition of the robust parameter for the volatility filter
Now that we have identified the robust parameters area it is worth performing a full in-sample optimization to see how the system would have performed between 2001 and 2009.
The system generated a net profit of $120,000 between 2001 and 2009 with a profit factor of 1.56, performing 756 trades with an average of 41% trades profitable. The systems exhibits certain desirable characteristics such as a high win/loss ratio of 2.28.
As anticipated we will now proceed with a walk forward optimization.
Step 1: We’ll optimize between 2007-2009 and find the best parameters.
The best parameters for the 2007-09 period are:
We apply these parameters to the out-of-sample period in 2010 with the following results:
Net profit: $12,300
% profitable: 43%
Profit factor: 1.38
Step 2: We’ll optimize between 2008-2010 and find the best parameters.
The best parameters for the 2008-10 period are:
We apply these parameters to the out-of-sample period in 2011 with the following results:
Net profit: $27,900
% profitable: 43%
Profit factor: 1.61
Step 3: We’ll optimize between 2009-2011 and find the best parameters.
The best parameters for the 2009-11 period are:
We apply these parameters to the out-of-sample period in 2012 with the following results:
Net profit: $17,540
% profitable: 41%
Profit factor: 1.58
In this article we have shown how to perform a walk forward optimization on an intraday mechanical system. The walk forward results are in line with the in-sample results and this builds confidence in the robustness of the strategy.
For more information on algorithmic trading strategies please visit: http://www.vbosystems.info
— By Amon Licini of VbO Systems. VbO Systems is a developer of 100% automated trading systems coded in NinjaTrader that can be auto-traded on almost all asset classes. Amon Licini, the founder of Vbo Systems, has been a private trader for 15 years and a senior manager with various corporates in Italy. Amon’s main trading interests lie in the area of volatility and open range breakouts for intraday systems. He lives in Milan with his wife and 2 children and loves traveling when he’s not developing new systems. Amon has a degree in mechanical engineering from the Polytechnic University of Milan.
Contributing authors are active participants in the financial markets and fully engrossed in technical or quantitative analysis. They desire to share their stories, insights and discovers on System Trader Success and hope to make you a better system trader. Contact us if you would like to be a contributing author and share your message with the world.
Please log in again. The login page will open in a new tab. After logging in you can close it and return to this page.