Algorithmic Trading Tips 20.

CHECKING ROBUSTNESS WITH MONTE CARLO NOISE.

Put your strategy on the test bench.

This Trading Tips issue focuses on checking the robustness of a trading system, using Monte Carlo simulations in Tradesignal, when noise is added to the price data. For this purpose the underlying price history is modified by adding noise. This procedure can be repeated many times by using randomly generated values and shows how stable the results of your strategy are, if the price history changes.

DOWNLOAD WORKSPACE. (For users with Thomson Reuters.)
DOWNLOAD WORKSPACE. (For users with Bloomberg.)

By downloading you agree to the terms of use below.

THE PROBLEM OF CURVE FITTING.

The amazing computer power we have at our hands nowadays makes it feasible to backtest limitless ideas, optimize them and find which ones (and which parameters of them) worked best in the past and so “fitting” by a great amount the systems’ parameters onto the past “curve” of price data. Of course you can’t altogether eliminate the curve fitting of a system to historical data because the systems are in the first place designed by examining what worked well in the past.

Overfitting, however, is a problem. This is because the greater the fitting of a system to the past, the greater its historical performance but also the greater the risk that it will not perform well in the future because the future price movements are not exactly the same as the past ones.

 

OVEROPTIMIZED OR NOT? ROBUSTNESS CHECK PROVIDES THE ANSWER.

The greatest fear of all systematic traders is that their trading system will stop working when the technical conditions change. This is because the systems are designed on past data and there is the danger of what is called “Curve Fitting”. The use of so-called robustness checks provides information about how the trading strategy is “immune” towards changes in the price data. Apart from using as few parameters as possible for a system and extensively testing it on Out-Of-Sample data (that is, on data which was not used to find its best set of parameters) there are also three common tests you can do to increase the odds that a system is not overly curve fitted and thus it is robust to changes in future price behavior.

The first test regards examining the system’s performance in symbols of the same nature as the one your system is designed to trade. So, for example, suppose that you designed a system to trade commodity A. When it performed well on A (In-sample and even Out-Of-Sample) but failed on a commodity B with the same characteristics as A then this is a cautionary flag that the system may be prone to future losses if you use it to trade commodity A since it failed to capture the primary forces which move the prices of B. You can’t expect a system to be profitable in all kind of markets but at least it should be able to have a decent performance in all symbols of the same nature because such symbols are usually traded by the same market participants and share common traits.

The second (and most popular in technical analysis software) robustness test regards checking the performance of a system when its parameters are changed a bit (for more information download How To 03 “Stresstesting”). Tradesignal’s optimization feature does a pretty good job in giving the user sophisticated means to perform such a test. When for example a simple moving average (SMA) crossover system performs pretty well using a short term SMA of 5 bars and a long term SMA of 35 bars but performs miserably for a short term SMA of 6 bars and a long term SMA of 36 bars it is an indirect indication that the 5/35 combination is curve fitted and thus the odds it will fail in the future are quite increased.

The third robustness test (an example of which we will focus below) regards checking the performance of a system when the price data are changed. This is a more direct test and more sophisticated than the previous ones since there are numerous ways for price data to be changed. To perform it you need to conduct a vast number of backtests using Monte Carlo simulations.

 

MONTE CARLO SIMULATION.

Broadly speaking, Monte Carlo simulations (a.k.a Monte Carlo methods or Monte Carlo experiments) refer to repeated simulations to obtain numerical results. For conducting the third robustness test this translates to defining what general kind or price changes you want your system to be tested upon, make as many such changes as possible (from very small to very big) to the price data, backtest your system on these changed data and finally examine the magnitude of change of your system’s performance in relation to the magnitude of price changes. In the sequence I will show you an example of such a test in Tradesignal.

 

ADDING A FRACTION OF ATR AS NOISE TO THE PRICE.

Consider a trading symbol named SYMBOL. We want to create a new artificial symbol (let’s name it SYMBOLnew) by changing the OPEN, HIGH, LOW, CLOSE values of each price bar of the SYMBOL. The new values for them (namely OPENnew, HIGHnew, LOWnew and CLOSEnew respectively) will be the same as the old ones plus or minus a random fraction of the Average True Range (ATR) provided that all new values are positive numbers and the logical order is maintained (that is, the LOWnew is positive and equals their minimum and the HIGHnew equals their maximum). To create the SYMBOLnew in Tradesignal all you need to do is write an indicator which will construct the OPENnew, HIGHnew, LOWnew, CLOSEnew and it will draw a Candlestick chart based on these values. I have created such an indicator (namely Noisy Symbol). By dragging and dropping the Noisy Symbol onto the chart of a trading symbol a new sub chart showing the SYMBOLnew is created. The indicator has three input parameters:

ATR_PERIOD

Its default value is 1 and defines the period of the ATR which will be used. The native AvgTrueRange function of Tradesignal is used to calculate the ATR.

ATR_FACTORCUTOFF

Its default value is 0.3 and defines the maximum fraction of the ATR which will be used to generate the noise. A zero value for this parameter results in zero noise, that is, the Noise Symbol will be the same as the initial symbol. For example, if this parameter is 0.3 then the random numbers which will be added or subtracted from the OPEN, HIGH, LOW, CLOSE to produce their new values won’t be higher than 0.3 times the ATR.

MAINTAINOCORDER

This parameter determines whether the initial order of Open and Close should be maintained or not (default: false). If set to TRUE it maintains the initial order of OPEN and CLOSE.

ITERATIONS

This is a dummy input which will be used in the Optimizer to perform the Monte Carlo simulation. When the indicator is applied in a chart the value of this input doesn’t affect anything.

TIP:
In Figure 1 (next page) you can see the Noisy Symbol applied in a daily chart of ADIDAS with its default values. Each time the Noisy Symbol is calculated or recalculated (for example when you change one of its parameters or you press the Reload button) a new “noisy” version of ADIDAS is created.

Since the Noisy Symbol is not a static symbol but an indicator which changes each time it is calculated you can now use the Tradesignal’s optimization feature to perform a Monte Carlo simulation for a trading strategy and see how the performance of the strategy changes when an ATR noise is added to the price.

FIGURE 1: THE NOISY SYMBOL APPLIED IN A DAILY CHART OF ADIDAS WITH ITS DEFAULT VALUES.

PRACTICAL EXAMPLE:
PERFORMING A MONTE CARLO SIMULATION.

As an example let us consider the daily chart of ADIDAS and the Tradesignal’s “Adaptive MA Cross – Entry” (with default values) strategy.

  • 01. Open a daily chart of ADIDAS and drag and drop the Noisy Symbol indicator onto the chart of ADIDAS to create the candlestick chart of Noisy Symbol which is of course a noisy version of ADIDAS
  • 02. Proceed to drag and drop the “Adaptive MA Cross – Entry” strategy onto the chart of Noisy Symbol. Note that if you press the Reload button (or change the input parameters of the Noisy Symbol) the Noisy Symbol is recalculated and a new “noisy” version of ADIDAS along with new updated strategy signals are created.

FIGURE 2: YOU CAN APPLY A STRATEGY TO THE NOISY SYMBOL BY SIMPLY DRAGGING AND DROPPING.

  • 03. You can now click the Optimize button,  select Brute Force in the Optimizer Wizard and click Next. In the next window of the Optimizer Wizard remove all parameters from the Selected Parameters menu except ATR_FactorCutoff and Iterations.

FIGURE 3: SELECTING PARAMETERS IN THE OPTIMIZER

  • 04. In the next step use the following Start, End and Step values for ATR_FactorCutoff and Iterations parameters:
  • 05. Starting the optimization procedure (which is now designed to work as a Monte Carlo simulation) will backtest the “Adaptive MA Cross – Entry” strategy against 20,000 versions of noisy ADIDAS charts. In figure 4 you can see the dispersion of Total Net Profit metric of such an optimization I made with 2000 bars of historical data presented as a heat map with the aid of Tradesignal’s “3D (Surface) Diagram” visualization feature.

FIGURE 4: DISPERSION OF PROFITS FOR THE MONTE CARLO SIMULATION EXAMPLE ON ADIDAS.

Tradesignal’s “3D (Surface) Diagram” visualization feature offers a quick way to see the dispersion of profitability for a strategy when it is applied on thousands of noisy versions of a symbol.

ENDLESS POSSIBILITIES OF MODIFICATION.

The technique presented here opens a door of endless possibilities. You can tweak the code of Noisy Symbol to produce any type of changes to the original price data to test your strategy’s robustness from various angles. You can also create new metrics to monitor for your Monte Carlo simulations. For example, if you want to examine maximum drawdown, consider adding the following three variables in the beginning of your strategy’s code:

var:​​ HHsf(0),OEq(0);

var:​​ AllTrades::MaxDD(0),TMAXDD(0);

If you add the following in the end of your strategy code, you will have a global variable MaxDD at your disposal:

OEq:=OpenEquity;

HHsf:=Maxlist(OEq,HHsf);

If​​ (HHsf-OEq>TMAXDD)​​ then​​ TMAXDD:=HHsf-OEq;

if IsLastBar then AllTrades::MaxDD:=-TMAXDD;

The MaxDD calculates the Maximum Equity DrawDown (largest peak to trough distance) of the open profit equity curve for the strategy and it will be available from the optimization heatmap.

Meta:​​ Synopsis("Adds ATR noise to the price data"),

 ​​ ​​ ​​ ​​ ​​​​ Categories("Monte Carlo, Intalus Trading Tips"),

 ​​ ​​ ​​ ​​ ​​​​ Author  ​​ ​​​​ ("Giorgos Siligardos");​​ 

 

Input:​​ ATR_Period(1,1),

 ​​ ​​ ​​ ​​ ​​ ​​​​ ATR_FactorCutOff(0.3,0.0),

 ​​ ​​ ​​ ​​ ​​ ​​​​ MaintainOCorder(FALSE),

 ​​ ​​ ​​ ​​ ​​ ​​​​ Iterations (1,1);

 

var:​​ nO,nH,nL,nC,

 ​​ ​​ ​​ ​​​​ NoiseAbsSpan(0),FoundPrice(FALSE),noise,randsign,x;

 

//Set Noise Span

NoiseAbsSpan:=AvgTrueRange(ATR_Period)*ATR_FactorCutOff;

 

//Randomizer

x:=Iterations;

 

{OPEN}

FoundPrice:=FALSE;

While​​ FoundPrice=FALSE​​ do

 ​​​​ begin

 ​​​​ If​​ NoiseAbsSpan=0​​ then​​ noise:=0​​ else​​ noise:=RANDOM(NoiseAbsSpan);

 ​​​​ If RANDOM(1)>0.5​​ then​​ randsign:=+1​​ else​​ randsign:=-1;

 ​​​​ nO:=randsign*noise+OPEN;

 ​​​​ If​​ nO>0​​ then​​ FoundPrice:=TRUE;

 ​​​​ end;

 

{HIGH}

If​​ NoiseAbsSpan=0​​ then​​ noise:=0​​ else​​ noise:=RANDOM(NoiseAbsSpan);

If RANDOM(1)>0.5​​ then​​ randsign:=+1​​ else​​ randsign:=-1;

nH:=MaxList(randsign*noise+HIGH,nO);

 

{LOW}

FoundPrice:=FALSE;

While​​ FoundPrice=FALSE​​ do

 ​​​​ begin

 ​​​​ If​​ NoiseAbsSpan=0​​ then​​ noise:=0​​ else​​ noise:=RANDOM(NoiseAbsSpan);

 ​​​​ If RANDOM(1)>0.5​​ then​​ randsign:=+1​​ else​​ randsign:=-1;

 ​​​​ nL:=MinList(randsign*noise+LOW,nO);

 ​​​​ If​​ (nL>0)​​ then​​ FoundPrice:=TRUE;

 ​​​​ end;

 

{CLOSE}

If​​ MaintainOCorder​​ and​​ C=O​​ then​​ nC:=nO

else

 ​​ ​​​​ begin  ​​​​ 

 ​​ ​​​​ FoundPrice:=FALSE;

 ​​ ​​​​ While​​ FoundPrice=FALSE​​ do

 ​​ ​​ ​​ ​​​​ begin

 ​​ ​​ ​​ ​​​​ If​​ NoiseAbsSpan=0​​ then​​ noise:=0​​ else​​ noise:=RANDOM(NoiseAbsSpan);

 ​​ ​​ ​​ ​​​​ If RANDOM(1)>0.5​​ then​​ randsign:=+1​​ else​​ randsign:=-1;

 ​​ ​​ ​​ ​​​​ nC:=MaxList(MinList(randsign*noise+CLOSE,nH),nL);

 ​​ ​​ ​​ ​​​​ FoundPrice:=TRUE;

 ​​ ​​ ​​ ​​​​ If​​ MaintainOCorder​​ AND​​ ((C-O)*(nC-nO))<0​​ then​​ FoundPrice:=FALSE;

 ​​ ​​ ​​ ​​​​ end;

 ​​ ​​​​ end;

//Draw New Symbol

DRAWCANDLESTICK(nO,nH,nL,nC,COLORWHITE,COLORRED,COLORBLUE);

FIGURE 5: FREQUENCY DISTRIBUTION HEATMAP AND FURTHER ANALYSIS

Clicking the Export Data you can export all the results from the Monte Carlo simulation in csv format which can be immediately accessed from MS EXCEL to conduct further statistical analysis. This Frequency Distribution Heatmap (along with the bar chart) was created by EXCEL and refers to the results of the Monte Carlo simulation example on ADIDAS.

 

DATA EXPORT ALLOWS FURTHER ANALYSIS OPTIONS.

Finally, clicking the Export Data you can export all the results from the Monte Carlo simulation in csv format which can be immediately accessed from MS EXCEL to conduct further statistical analysis. In figure 4 for example you can see a frequency distribution heatmap created with EXCEL after exporting the results of the ADIDAS Monte Carlo simulation.

That’s it for today. Take care, take profit and “Eις το επανιδείν”.
Giorgos Siligardos

DOWNLOAD ARTICLE AS PDF.

DISCLAIMER.

© Copyright Tradesignal Ltd., London.
Distribution allowed under a Creative Commons Attribution-Noncommercial license
 http://creativecommons.org/licenses/by-nc/3.0/

Tradesignal® is a registered trademark of Tradesignal GmbH. Unauthorized use or misuse is specifically prohibited. All other protected brands and trademarks mentioned in this document conform, without restriction, to the provisions of applicable trademark law and the copyrights of the respective registered owners.

Tradesignal Ltd. obtains information from sources it considers reliable, but does not guarantee the accuracy or completeness of its information contained therein. Tradesignal Ltd. and its affiliates make no representation or warranty, either express or implied, with respect to the information or analysis supplied herein, including without limitation the implied warranties of fitness for a particular purpose and merchantability, and each specifically disclaims any such warranty. In no event shall Tradesignal Ltd. or its affiliates be liable to for any decision made or action taken in reliance upon the information contained herein, lost profits or any indirect, consequential, special or incidental damages, whether in contract, tort or otherwise, even if advised of the possibility of such damages. This material does not constitute an offer or a solicitation of an offer or a recommendation to buy or sell securities. All expressions of opinion are subject to change without notice.

This code is provided free of charge and “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of fitness for a particular purpose and noninfringement. In no event, shall Tradesignal Ltd. be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, or of or in connection with this code or the use of this code.

.

non Donec commodo ut Nullam fringilla