Robustness Testing in Forex Trading
Hello traders, in this lecture, I will show you an example of a robustness testing check using the Reactor in EA Studio.
So as I’ve already said a few times, what the Reactor does is automate the whole process. We have the Generator where the trading strategies are generated. Then if we want, they go through the optimization process and then they go through the robustness testing check.
And something very important to note is that when we use the optimization, any of the 3, this will change the strategy that comes from the Generator. But the robustness testing will not change it. It will just test if it is robust.
And this is possible again with 3 different options. One is the Walk Forward validation, very similar to what I described in the previous lecture for the Walk Forward. But the difference is that it doesn’t change the strategy.
It just proves if the strategy is robust or not, if the result of the Walk Forward is worse than the initial strategy, then this initial strategy that came from the Generator is over-optimized. And if the strategy fails, the Walk Forward validation, or any of the other 2 robustness tests, then we will not see it in the Collection at the end.
Perform Robustness Testing With Monte Carlo Validation
In the collection, I have that strategy that I used previously to show you.
But I will leave it because I want to show you how the Monte Carlo works and the Multi Market. Monte Carlo is the tool that I always use when generating trading strategies. So this box is always checked for me.
What Monte Carlo does in very simple words, it tries to break the strategy before we start trading with it. It runs a variety of robustness tests.
If I click on Monte Carlo, you will see that I have the option for different simulations that are separated.
We have market variations, execution problems, and strategy variations.
How to know if your strategy is over-optimized
So how I usually use it is with the last 2 – Randomize Indicator parameters and Randomize backtest starting bar. And what that means is that the Monte Carlo will perform a variety of simulations with different or randomized indicator parameters, and it will simulate backtest from different starting bar back in time.
This way, we will know if the strategy was optimized for the Indicator parameters. Just what I explained about over-optimizing. And we will know if we start at any moment at any bar with this strategy with this Expert Advisor, is it going to be still profitable or not? And this way, we will know if the strategy was over-optimized exactly for the period tested.
Robustness Testing Check
So if I go to Collection and I select that strategy again, you see that it showed that it was over-optimized for that period.
And let’s go to Monte Carlo and I will click on start and let’s see what will be the results in the robustness testing check. Here they are.
We have 19 out of the 20 tests validated. So all of these tests here are with different parameters, different backtest starting bar. But one of the results is losing here.
The Optimizer
So there is a very huge chance that this strategy was over-optimized. This is because I actually optimized it in another lecture. With a step of 1 using the Optimizer. And we saw that it optimized it very well for this period.
But then when I added previously 8 years, then the strategy is not really what we want to see. And actually, I changed the data a little bit, but it’s very similar. So that’s the one that I have set, for XM. And now, I will go back to Monte Carlo and I will tell you about the market variations. This is where we have Randomize spread, Randomize history data.
You can use that if you’re using different historical data or for different brokers with different spreads.
Execution Problems
Well, this will be applicable if you have some execution problems with your broker. It often skips positions or there is a vast slippage, which is not really the best thing we want to have with the broker. You can check as well the strategy with such simulations. But this is how I use it. The last 2.
Robustness Testing With Multi-Market Validation
And if I go back to the Reactor and I go to Multi Market, this is the last robustness testing checks that we have on the list.
A strategy that comes from the Generator, that passes the optimization, the Monte Carlo, everything on the top, and reaches the Multi Market, we will be able to check if this strategy is performing well on the other markets as well.
We can change these markets to whatever we want. Usually, if we are generating for EURUSD, for example, we want to know how the strategy performs on other currency pairs. Not on gold or stock or oil or anything like that. So I have clicked randomly on a few currency pairs and if I use the Multi Market, it will perform backtest on these currency pairs, the same strategy and it will validate a number of tests.
So if we have like 5 here and we select 3 that are validated, it means that it will validate for sure for EURUSD. And then we want another 2 to be validated.
Avoid Over-Optimization
Let me demonstrate very quickly how that works with the current strategy that we have in the editor. I will go to Multi Market and let’s change the brand to EURGBP, for example, and as well, I will select AUDUSD. And then I have AUDNZD, it doesn’t really matter. I just want to have 4 pairs and I will click on start. Let’s see how this strategy performs in robustness testing examples on other 5 markets. 1 of 3, 1 of 4. And we are waiting for the last one, here we go. All the others are losing.
The initial strategy for GBPUSD is profitable.
But one more time, I have optimized that and I think I over-optimized that. This is not like a 100% event. I cannot say 100% that I have over-optimized the strategy. But from what I saw after adding the previous 8 years is that the strategy is losing for most of the time and I see that the Multi Market shows me losses on the other markets. Probably the best way to say it is it’s most likely that I have over-optimized the strategy. We are not 100% sure but the chances are big.
So that’s why I don’t recommend using the Optimizer, it’s better to use the Walk Forward or the normalization. And don’t get me wrong, it doesn’t mean this is a bad thing. Not at all. You just need to know how to use it carefully and what robustness testing exactly you need to perform after that to make sure that the strategy is not over-optimized.
Robustness Testing in EA Studio
Some of the features in EA Studio are so good and the algorithm behind it is so smart. I can say that it becomes dangerous for some of the strategies because it finds quickly the best parameters. It optimizes them very quickly. This is the work of the Optimizer, but at the same time, this is risky not to over-optimize it, just like driving a very fast car, which is dangerous.
People want to drive always fast cars, but this is when usually the accidents happen. So you need to be very careful when using it or just play more with it, see how it’s working. And if you have any questions, of course, feel free to ask me in the forum.
Setting Data Horizon
I will remove now this strategy from the collection and I will run the Reactor. This is how I perform a robustness check most of the time. Usually, it works faster.
But again, I have a few more running on the same computer. So when you run 3, 4, 5, or more, they might get slow. Keep in mind that EA Studio works in the browser and it loads from the server the 1st time and then it works on the browser. It’s web-based, which gives you the opportunity to run it on multiple browsers at the same time.
So what do I usually do? I set 1 for EURUSD. And now just next to the recording screen, I have a few more for GBPUSD, EURGBP, and this is how I am generating strategies. One thing I forgot to do is to reduce the data. I will go to Data, Data Horizon and I will do it just for the last 5 years.
I will start again.
And one thing, keep in mind that the more data you use, a little bit slower it will be, the less data, the faster it will be. It’s logic. So this is how I am using the Reactor.
It’s a lot of possibilities through the options. You can test it, use whatever makes sense to you. But definitely, it’s the best thing we can use in algorithmic trading nowadays to have robust strategies and to have many strategies.
How does it work?
It will generate strategies if they pass the Acceptance criteria, the Out of Sample, the Normalization, and the Monte Carlo robustness check. I will see them in the Collection.
So basically, they will go through a tough robustness testing. And at the end of the day, I will have hundreds of strategies, maybe thousands of strategies, and I will see the best ones into the collection.
I will leave it now running for some time. Actually, I have set exactly 10 hours. And after 10 hours, I will show you what results I have in the Collection and you will see how easy it happens. I don’t do anything.
I will leave it overnight and in the morning I will have some ready strategies in the collection.
Thank you very much for reading. If you found that article useful, leave a comment below and if you haven’t subscribed to our YouTube channel, make sure to do it now.
If you have any questions, let me know.
Cheers.