Simulation: Beyond Backtesting

beyond backtesting

One problem with traditional backtesting is that it relies on the presupposition that there are repeating predictive patterns in the market. In fact, most trading methodologies rely on this assumption. And yet we know the disclaimer that past performance is not indicative of future results.

And yet backtesting largely assumes that the future will be similar to the past. Yet, we can imagine the possibility for non-repeating but predictable profit opportunities. Even without getting into those possibilities, we can imagine that if we can model the dynamics of the market accurately, we can predict new outcomes that cannot be extrapolated from the past.

The way this is accomplished is simulation. Simulation offers the powerful promise of allowing us to make use of historical market data under varying conditions of future similarity. Simulation, massive simulation is also poised to impact every aspect of our lives.

Imagine for a moment that you are a world class MMA fighter or boxer and you’re competing against a similar well ranked fighter. What should your strategy be? In the past, you might have studied your opponent and intuited a strategy. Perhaps, if you were more sophisticated you might have even used crude statistics such as counting to figure up the risk of and probability of a given move working. But today, it is surely possible to feed in your moves into a computer with precise timing and force calculations. Next, it is possible to infer the same regarding your opponent by using previous fight videos. In addition, by using the fighters height, weight, and other statistics it is possible to model how well they could perform even moves that were not recorded. Once all the data is put into the computer then you can run thousands or hundreds of thousands of fight simulations. The best performing simulations will yield the best strategies. The strategies that are discovered may be non-intuitive and completely innovative. These can be used, with human cognition and consideration, as the basis for your game plan.

Now, imagine how this would work for the trader. It is not just running thousands of simulations on past data. But you must infer how future traders will react to changing market conditions. This is the difficult part because you need to know how the combination of variables will impact their behavior.

Even if that level of simulation is beyond the average developers capability or can only provide rough approximations due to the difficulty in modeling, it is still possible to start thinking more along the lines of simulation to explore creative opportunity and risk management.

Some ideas for how you might do this:

  • Use random and partially randomized entries and exits to try to find more universal or robust settings for your strategies.
  • Create synthetic market data where you change the amount of volatility, trend, and mean reversion to see how it might impact your strategies.
  • Create models of how traders might act in certain scenarios and look for situations that might offer predictive advantage.
  • Use Monte Carlo analysis with randomized entries to come up with pessimistic capital requirements.
  • Try to find optimal strategies for given market conditions.
  • Build self-learning strategies with limited capacity for memory and try to find the optimal rules for trading.

–by Curtis White from blog, Beyondbacktesting

About the Author Curtis White

Curtis is a market enthusiast and programmer who enjoys predicting markets, developing trading systems, and tape reading. He offers a blog at where he shares his original ideas for developing and enhancing trading systems. He is a creative,synthesis thinker who likes to combine ideas and techniques from a broad palette. One of his core goals is the creation of graybox systems, software, and decision support tools for enhancing the capabilities of high performance discretionary traders.

Leave a Comment: