How to Design and Backtest a Trading System

I have been working for some time on various systems for trading stocks, and thought I would share some of my observations and the lessons I have learned.

The first thing to figure out, of course, is what type of trader you want to be. Are you looking for high returns with correspondingly greater risk, or slow and steady returns, with lower risk?

It’s important to remember that it’s possible to obtain great returns, even with very low accuracy rates; that is, the returns may be high, but the risks fairly low, because no losing trade is very large. For example, a system that is accurate only 25% of the time, but returns an average win of 5% and an average loss of less than 1% might outperform one that boasts 75% accuracy, but that risks large losses relative to its gains. Keep this in mind, if you decide to “buy” a system and even if you design your own. Remember, also, that most systems you can “buy” lose their effectiveness by being on the market. A truly great system is not going to be for sale – there’s more money to be made in trading it than in selling it.

There are several ways a system can go wrong. First of all, there are theoretical flaws. For example, a system that calls for buying a stock at the opening price of a bar, but relies on knowledge of the closing price of that bar, is unrealistic. This kind of theoretical problem can arise in many ways; perhaps you don’t actually use a future value in your system, but use a moving average or other indicator, incorporating future values.

Another problem is that of unrealistic stopping. Say you set a trailing stop of .1% of the entry price. Many software packages will show that this results in excellent – unbelievable – results, because they will assume that you can exit just below the high value in a bar, without dealing with the possibility that the stock will actually hit its in-bar low before its in-bar high.

Perhaps the greatest problem is that of over-optimization. This can occur in two ways. The most familiar way is to use the built-in optimization feature of a software package to find some combination of parameters that results in excellent results when backtesting, but fails utterly when applied to future data. A more subtle way to over-optimize is to find a system that works well, sometimes regardless of the parameters used, because you are not testing over a large enough data sample. For example, I have found several systems which appear, when tested over a couple months’ data, to result in annual returns of one million percent or better, regardless of which stocks are traded or which parameters are used. These same systems often fail miserably over larger time frames, however. Be sure you test your system over plenty of data and hundreds of simulated trades.

As for software, I recommend AmiBroker. It’s not perfect, but it has several advantages over its competitors:

  • It is very inexpensive (one-time cost of only $149 for the standard version, or $229 for the professional version).
  • It offers a language for programming custom indicators and systems, backtests, scans, and stock screening, which is very easy to learn and fairly flexible.
  • Frequent upgrades
  • An active user community.
  • Support for DLL plugins.

This is just an introduction; there is much more to designing and testing a good system than I have covered here. Readers may be interested in my finance pages and the Elliott Wave Theory.





Leave a Reply

Your email address will not be published. Required fields are marked *