The Top Three Pitfalls of Stock And ETF System Development

During my nearly 25 years of experience in the trading business, I have talked to many traders about system development. Futures trading systems and back-adjusted contacts are fairly well understood. Recent issues with the pits closing and markets being only electronic have created some issues but it’s still relatively straight forward to design a trading system which can beat, for example, the Barclay Systematic Traders Index.

I have found the same to not be true for stock and ETF systems, even though both security types are more accessible to the general public. This seems to be for several reasons. First, the standard split-adjusted data series used by itself in backtesting has severe limitations when testing on a portfolio. I’ll detail more about this later in the article, but the summary is that you need to use unrealistic trading assumptions just to get a backtest which is moderately distorted from what the real results would have been for a long backtest of 15-20 years. Certain trading types are off-limits with standard split-adjusted data in a portfolio. For example, you can’t just exit one of the stocks or ETFs being traded on a protective stop and continue trading the others.

Secondly, another issue is that stock and ETF system developers do not understand how to test a stock/ETF system to make valid comparisons to buy and hold. It is true that it is, indeed, hard to develop systems which outperform buy and hold on a return basis depending on your testing window. For example, if we don’t include the 2008-2009 crash, then it is very difficult. If this period is included, then the comparisons are easier. Outperforming buy and hold is a bit of a myopic goal, however. One thing that systems designers can do is build a system which makes almost as much as buy and hold without as much risk. In the trading world, reducing risk is always critical.

Please note that I am not saying mechanical trading systems cannot beat buy and hold. I am simply saying that you need a good system to beat buy and hold by a sizable margin over a long time period. In fact, I have designed many excellent strategies which greatly outperform buy and hold with a lot less risk. A later installment of this article series will focus on how to develop these strategies.

For now, let’s look at the beginning of how to develop stock and ETF strategies. Fundamental to designing these systems is understanding the concepts behind split-adjusted data and some other general issues relating to equity data.

Split-Adjusted Data

A major reason why traders believe that they cannot build mechanical trading systems that outperform “buy-and-hold” is that they do not understand the issues in terms of preparing the data and the effects that differences in the data can have on the results.

Most stock traders use what is called “split-adjusted” data, which is similar to “ratio-adjusted” data in the commodities world. The problem in using this type of data is that the adjustment destroys the dollar returns, historical daily range, and the original price levels. The only advantage it has is that it correctly calculated the percent return.

If the price of a stock moves too high or too low, management of a company can issue splits in the stock to encourage trading. The vast majority of stock splits occur because the price has gotten too high. Consider the case when the price rallies to $50 per share and management decides to split 2 shares for 1 share. This does not affect the company valuation, but there are two shares of $25 stock on the books for every one share of $50 stock that previously existed. If the gap on the chart cannot be smoothed, false trades occur in testing because the unadjusted chart looks like the price dropped from $50 to $25. If you had a protective stop at $40 on a trade, it might trigger in the backtested results. However, this is actually a false order because the company valuation didn’t change with the drop from $50 to $25. “Split-adjusted” data is a way to solve this problem.

The split-adjusted data stream is produced by dividing the prices prior to a stock split by the factor of the stock split. For example, in our case above, the dividing factor would be 2 (for the 2 for 1 split). This sounds like a great solution, right? Not quite. This has the effect of cutting the previous daily ranges in half. Instead of one day being between $45 and $55 per share, that day now traded between $22.50 and $27.50. The problem really occurs when a stock has been split many times. When this occurs, the split-adjusted data can get ridiculous. An extreme example of this is Microsoft. The split-adjusted price is $0.11 per share if stock splits are handled all the way back to 1988. The real price per share at that time was about $34!

Another problem has to do with the way the major markets have been operated in the past. Stocks were priced in fractions until quotes were changed to the current decimal format. Many of the available public stock databases are set up to calculate to two decimal places. When a stock like Microsoft has had so many splits, a split-adjusted $0.01 move scales out to $3.40! To compensate, a minimum of 4 decimal places needs to be saved and more would be better in many cases. Another problem with split-adjusted data is accounting for the calculation of commissions. The actual dollar values are meaningless and the results of backtesting can only be seen in terms of percent returns.

The core premise of split-adjusted data is that the percent return is correct. The next step in this logic is that always buying the same amount of each stock traded will make percentage return numbers also correct. However, what happens when you don’t want to buy the same dollar value of each stock? What if you want to use a percent risk model whereby position size is based upon how much risk is assumed in a given trade? For example, suppose you decide to risk 1% of your account on a given position. For a $100,000 account, the risk would be $1,000 on a given trade.

Disregarding split-adjusted data for the moment, presume that our system rules are to exit a long position at a 10-day low. If that low is $1 per share away, it is possible to buy 1000 shares of that stock. With split-adjusted data, this is not possible (remember, the Microsoft disparity in prices). The problem becomes amplified when looking at a portfolio of stocks where risk analysis needs to be performed on each of them. Since portfolio-level analysis in backtesting software is a relatively new development, many of these issues have yet to be addressed.

A percent-risk money management strategy might require that you place a large percentage of your account into one stock position. What if the example stock with the $1.00 per share risk was $100 per share stock? In this case, 100% of the account would have to be placed in the stock position while we might be trying to trade a basket of 100 stocks. We are within the directives of our system because we are risking 1% of our account’s value on that trade. In summary, it’s essential to not just know the split-adjusted price of the stock for the correct prices on the trades, but also the unadjusted price is needed for entries and exits so dollar returns, percent returns, and the amount of money committed to taking a position can be accurately calculated.

Business Survivorship

Another issue in a stock trading system is the “business survivorship” test. For example, how valid is a test on the current S&P 500 in which results are overstated due to the fact that stocks such as WorldCom and Enron are not included in the backtest? The “business survivorship” test is an issue that many traders choose to completely ignore. However, it is one that they will likely have to face in real life because old data for many de-listed stocks is not readily available. Whether you choose to consider this in your testing or not is naturally entirely up to you; however, you have to realize that it exists when you are evaluating a trading strategy.


Dividends can create problems as well with split-adjusted data. This used to be an issue more so for Dow 30 companies and utility stocks. Nowadays, changes in tax treatment where options have to be expensed and dividends are better tax-wise has helped to make dividends more popular. The price of a stock drops on the day the dividend is assigned to the existing owner of the stock, which causes a downtick on the chart. When analyzing mature companies like those in the S&P500, it must be remembered that dividends can account for as much as one half of the return of buy-and-hold strategies. When profits from dividends are compounded and reinvested back into stocks, dividends can have a very powerful effect on those compound returns.

Other dividend-related problems could cause major changes to system results even if dividend-adjusted data is used. Assume that our system rules tell us to buy at the highest high of the last 12 months. During that time, the stock has had a high of $40, but has paid a $2 dividend since that high. Most system developers would buy if the price exceeds $40, but since the dividend was subtracted since the high price was made, the real adjusted 12-month high is $38. If the stocks were bought at $38 instead of $40 across hundreds of shares over decades, the results of the system would be drastically different!

Another issue to consider when trading stocks is the use of fundamental data. Most data vendors do not have access to fundamental data and the ones who do typically only maintain one year of it in their databases. This creates a problem since it is very difficult to backtest a strategy using such a limited amount of data. This is only one of the problems of trading stocks using fundamentals. Long-term histories of fundamental data are available, but it is usually quite expensive.

As you can see, stock traders can have a tendency to not believe in mechanical systems due to the difficulty of isolating the issues involved in obtaining realistic backtest results that would compare with real-world performance. Analyzing these problems objectively results in an easy loss of faith in the system. After all, if the data is not clean, how can we realistically test any system and know how it will hold up with real money on the line? A trader must have a tremendous amount of faith in the system and that only comes after extensive backtesting on clean data, with the right backtesting platform.

How Can We Solve These issues in Backtesting ETF and Stock Systems?

How we solve these issues will be discussed in next week’s article.  In that article I’m going to show you how to correct these issues we discused and in doing so, create systems that accurately backtest so you can become more successful in trading stocks and ETFs.

About the Author Murray Ruggiero

Murray Ruggiero is the chief systems designer, and market analyst at TTM. He is one of the world’s foremost experts on the use of intermarket and trend analysis in locating and confirming developing price moves in the markets. Murray is often referred to in the industry as the Einstein of Wall Street.He is a sought-after speaker at IEEE engineering conventions and symposiums on artificial intelligence. IEEE, the Institute of Electrical and Electronics Engineers, is the largest professional association in the world advancing innovation and technological excellence for the benefit of humanity. Due to his work on mechanical trading systems, Murray has also has been featured on John Murphy’s CNBC show Tech Talk, proving John’s chart-based trading theories by applying backtested mechanical strategies. (Murphy is known as the father of inter-market analysis.)After earning his degree in astrophysics, Murray pioneered work on neural net and artificial intelligence (AI) systems for applications in the investment arena. He was subsequently awarded a patent for the process of embedding a neural network into a spreadsheet.Murray’s first book, Cybernetic Trading, revealed details of his market analysis and systems testing to a degree seldom seen in the investment world. Reviewers were universal in their praise of the book, and it became a best seller among systems traders, analysts and money managers. He has also co-written the book Traders Secrets, interviewing relatively unknown but successful traders and analyzing their trading methodologies. Murray has been a contributing editor to Futures magazine since 1994, and has written over 160 articles.As chief systems designer, Murray digs into the depths of niche and sub-markets, developing very specialized programs to take advantage of opportunities that often escape the public eye, and even experienced high level money managers.