EA Forex Academy – Algorithmic & Manual Trading Courses › Forums › Forex Trading Forum › Use of Historical data & Data Horizon
- November 19, 2020 at 6:29 #66733
Sounds good……November 18, 2020 at 13:12 #66728
Always nice to hear (read) your method of thinking Richard.
Only now I understand why and how you’re gathering so many strategies over time. Nice tactic!
I am systematically testing the exact 4 pairs you mention but I’ll be testing XAUUSD (gold) and XAGUSD (silver) as well.
Will use two computers: One with 16 GB RAM and another with 32 GB RAM.
I’ll be running a the live accounts on my new laptop. The idea is that the laptop will continue running on battery in case of power failure, provided Internet connection is still working.
Started with Tools > Settings where I chose $1000 as initial account and 1/500 leverage (since my broker allows it).
In Generator, I chose an “Entry lot” of 0.03 since I’ll be using 4 pairs and 10 EAs on each pair.
Regarding Historical data:
My two brokers delivered excellent data via EA Studio Data Export Script.
One broker delivered 33-58 months of data (M15, M30 & H1) and the other delivered 16-29 months of data for the same TFs (Timeframes).
I checked the accuracy of the data, they had a max. days off of 2 & 3 (better than Dukascopy. Not for longer period, but long enough)
Decided to stick to the scripts (in stead of Historical data App) because I’d save adjusting the data using Symbol Settings and avoid entering wrong values.
I also noticed that one broker has normal commission (about $7.5 per lot on gold and silver) and the other has ZERO commission on gold and silver. So I’m excited to test the profits on gold and silver there.
Started by running the Reactor for 10 minutes with almost no criteria & no robustness to see how many strategies I can get.
Then I increased the working time to 30 minutes with more stringen Acceptance criteria & Monte Carlo.
30 minutes was enough to test-generate strategies and filter them down to 10 strategies which I added as a PF-EA (Portfolio Expert Adviser).
Recalculating the PF-EA gave me a pretty good idea about what kind of profit I’d expect per day and per month.
If and when all goes well, I’ll let the Reactor run for many hours and thus create solid strategies with confidence.November 18, 2020 at 10:58 #66727
Yes I always use monte carlo as Petko teaches…. Set it to run within the reactor. The reactor can hold a maximum of 300 into the collection. I run up to 10 reactors at a time. 6 on one PC and 4 on another. I run them continuously at times and it has taken weeks and maybe months to get the number of strats I am talking about. Of course there will be a point when it is no longer a benefit to keep adding more and more strats and may even have negative effects…… I don’t know yet. I just keep running multiple side by side tests all the time to identify any consistent edges that I can add to the system….. I would probably just stick with 4 pairs like the EURUSD GBPUSD EURJPY GBPJPY if you are fairly new….This will give you the 4 main pairs evenly traded. (Each currency appears twice each) Create a few 100 strats for each pair and then experiment on how you will filter them to say a top 10 of each. Then trade all 40 together on one MT4. One of the key things I have learnt from Petko and my years of manual trading before coming here is to keep the currencies balanced for each MT4….. One of the bigger questions I have pondered lately is do you filter the trades more thoroughly initially and then possible run them for a shorter period of time (1 week) or do you not filter them so stringently and let them run for say a month but then apply some filtering as you go along to cull the poor performers…. If you have read through my posts in the last month I feel more comfortable creating a large amount of strats, filtering them more stringently and then let them run for 1 week. I am aiming to start them Monday morning and then shut them down at the end of the week and be out of the market over the weekend. I will have to see how I go……. Without getting to philosophical I believe that you have to ultimately make your own system that aligns with your personality. Any way enough ramblings for now……..November 18, 2020 at 0:21 #66717jbcdkParticipant
Thank you very much for sharing your thoughts and ideas, it is very useful and helpful, as well as very inspiring. When you start the reactor to generate EA’s and first stop after maybe 300 or 2000 EA’s, do you then also use robustness testing eg Monte Carlo, and how long are you about to generate eg 1000 EA’s ?
ThanksNovember 17, 2020 at 11:42 #66526
You’re welcome……November 17, 2020 at 10:46 #66524
Useful information indeed. What a great guy Richard is to share his thoughts in a very comprehensive manner. Thank you Richard.November 17, 2020 at 9:36 #66521IlanModerator
This is really useful information for generating strategies.
IlanNovember 17, 2020 at 5:50 #66516
Hi Asser. This is what Popov says regarding the correlation setting:
(Hope it is OK to copy and paste his remarks here….)
“The Correlation shows how equally trade two strategies. It compares the Balance lines of the strategies for the complete testing period.
A correlation coefficient of 1 means that two strategies have absolutely equal balance lines. It is possible to have strategies with different trading rules to rise very similar signals and we want to eliminate such cases.
When the Correlation analysis is on, it checks every strategy pushed to the Collection against all other included strategies. If the program finds that two strategies has correlation greater than the set value, it marks them as correlated. EA Studio removes the strategy with the the lower profit in such cases.
For example, a Correlation Coefficient of 0.96 means that the strategies differ by 4 percent.”
This week I am running some side by side tests to compare .9 vs .98 correlation…. At this stage .9 is about 20 – 30 % more profitable then .98 and tends to open approx 20% more trades, but early days yet obviously……
The top 5 vs top 10: Top 5 seems to out perform Top 10 by about 30% across a dozen test variations I have run over the last 3 weeks…. (this means that you are doubling position size on the top 5 obviously to give you the same margin % / risk as the top 10.)
Without getting to far ahead…… You might want to look at lot size for each pair regarding ADR. I am running some tests with fixed lot size (.03) vs variable (.01 – .08) depending on expected return for the week from each portfolio of 10. Again I am always looking for balance and smooth equity line. So that each pair is delivering a relatively even contribution to the overall system regardless of ADR…..November 17, 2020 at 5:40 #66515
Petko’s demonstration of how 10 EAs work together in a PF as one EA with a pretty good united equity line (because they complement each other’s drawdowns) inspired me to focus on creating exactly PF of 10 EAs. I could narrow down to 5 PF EAs, inspired by you, but I must have enough successful simulations prior to doing that.November 17, 2020 at 5:34 #66514
My current goal (if it holds) is to set up a routine (a system) consisting of creating 10 NEW Portfolio Expert Advisers (PF EAs) each month, so that I avoid uploading old strategies and recalculate or validate them, making the routine a tad simpler. I’d rather avoid demo-trading but having enough successful simulations would be similar to demo-trading in my view.November 17, 2020 at 4:44 #66513
Always happy to read your wise comments Richard. Yes, I’m experimenting with a few ideas. But now, I’m even more inspired by your thinking which I’ll certainly take into consideration.
My current plan is to turn everything into a routine – which (as you point out) always will have room for improvement. I’m getting closer to my goal everyday.
My theory (always based on Petko’s teachings) is to work with a Portfolio EA of 10 (will maybe move to 5 inspired by you if it works for me), because I could see the equity line of 10 strategies united into one PF EA equity line – almost regardless the weaknesses of each individual strategy because they complement each others.
Imagine if I could routinely turn creating a PF EA of 10 each month and run them live because I’ve tested them enough times with simulation. My current idea is NOT to use old EAs and validate them, but create 10 COMPLETE new EA each month. That would be my dream… But first, I’ll have to satisfy myself with enough successful simulations. In my view, that’s better than demo-trading.
Regarding playing with the correlation %:
Great idea. However, are you sure that .90 is stead of .98 will give you more different strategies? I’d think the opposite is true (but I could be wrong).
Thank you Richard & good luck to you too.November 17, 2020 at 3:50 #66512
So this is my latest way of doing things. I don’t think you ever should stop evolving your systems. Because I feed in so many strats to get my top 10 / 5 (I am actually starting to find the top 5 more profitable lately…..) I also have started playing with the correlation %. I bring it up from the standard .98 to .90 to ensure that all my strats are significantly different from each other. More diversity within the portfolio.
Anyway it is good to see you experimenting with different idea’s based on what you are learning here. This is what I do as well. Good luck….November 17, 2020 at 3:45 #66511
Then I end up with some pretty good looking strats into the portfolio:November 17, 2020 at 3:42 #66510
Then bring up the PF till I get the best 10. Create portfolio:November 17, 2020 at 3:39 #66509
So if I read it correctly you are ultimately creating your portfolio of 10 from the 3 years of data. Of course at the end of the day you have to practice and find your own system. Which you are doing…. I prefer to create the strats leaving off the last month. Then run them on the last months data. And then select my top 5/10 based on the last months performance. In my mind that means that the strats have proved themselves over 5 years to be profitable and they have also proved themselves to be profitable in the last month. Because I use very stringent criteria they will be consistantly profitable over the whole of the month without any significant dips in the equity line…..
Example attached: Here I feed in over 1000 strats created over 5 years of data. Then run them over the last month. You see only 146 pass the criteria for the month. This is currently set to COT 8, Losses in a row 1. When I was only feeding in around 50 – 100 strats before I had to use much lower criteria to get enough strats to pass. (COT 5, Losses in a row 2. ) As I get more and more strats to feed in the tighter I make the criteria.
- You must be logged in to reply to this topic.