EA Forex Academy – Algorithmic & Manual Trading Courses › Forums › Forex Trading Forum › Use of Historical data & Data Horizon
November 8, 2020 at 22:53 #65581
Thank you for pointing out to me that I was blocked because I shouldn’t upload more than one image per post. I have therefore created a one in all image.
I hope I’m wrong, but if I am right and if Mr Popov can update the Historical data App, it would become SUPER ACCURATE for all traders.
I’m adopting your best and newest gameplan which you showed in your course:
Forex Trading with Expert Advisors + 30 Best Strategies (Every Month)
The plan consists of (in my words):
– Uploading 5 years (or so) of data with the Historical data App (Remember to adjust the timezone)
– Creating a portfolio of 10 strategies with the Reactor,
– Doing nothing for one month
– Recalculating the 10 strategies on this last month’ data and go live.
So I made a gameplan of my own:
– I upload 5 years of data (UP TO 13. January 2020)
– I create a portfolio of 10 strategies,
– Using Data Horizon, and simulating that today is the 13. of January 2020, I run the 10 EAs on January month (13. January through 31. January)
– Next I do the same with February, March, … up to 30. October 2020.
– By recalculating & adjusting the 10 strategies every month (just like doing it live), I can see (month-by-month) what is the potential of this excellent strategy and what kind of profit I can make. That would boost my confidence and allow me to start trading live for the first time in my life.
Therefore the problem (as explained in the image) of synchronizing data with my broker with Historical data App is of a huge importance to me.
Please take it seriously!
November 10, 2020 at 10:01 #65669
Let me have a look at it in detail later today, and I will give you the answer.
Petko ANovember 10, 2020 at 14:59 #65701
Just going over it again I noticed you are talking about the summer and wintertime.
That is not a glitch. The data from the app is one, it is bars.
It is about if your broker actually changes it or not. Some brokers change it, others do not. Some synchronize it with your local time.
So a solution is to set your time on the computer to UTC, and then install a new MT platform and see if the broker will synchronize it.
If it does, great, you will have the perfect match in the data.November 10, 2020 at 20:22 #65713
You say: “That is not a glitch. The data from the app is one, it is bars.”
If that’s the case, great! But then, why do we have to synchronize at all?
As you put it in your course: Even professional traders forget to synchronize the data with their broker.
You say: “It is about if your broker actually changes it or not.” (“it” meaning summertime to wintertime and vise versa)
I don’t understand your point, but I followed your instructions:
– I adjusted the timezone of my PC to UCT.
– I downloaded a new MT4 demo.
– My PC time showed: 17:32 (when it’s actually 18:32 where I live)
– My broker’s time showed: 19:32
– So I opened a EURUSD H1 chart on MT4.
– On Historcial data App, I loaded data from the App with (GMT) in the settings (GMT=UTC)
– The Price chart of EURUSD H1 showed a bar of 10. November 2020 at 16:00
– My broker’s MT4 showed the same bar of 10. November 2020 at 18:00
– So I adjusted the App’s settings to GMT+2
– The Price chart showed now the same bar of 10. November 2020 at 18:00 (PERFECTLY SYNCHRONIZED)
– Going back on MT4 to 14. July 2020 at 10:00 (there is a distinguished bar)
– The same distinguished bar is showing at 09:00 on the App’s Price chart (SYNCHRONIZING FAILED)
My conclusion: Even though I don’t understand the point of changing my PC’s time zone to UTC and installing a new MT4, I did what you said and the problem remained the same.
If what you’re saying is that DATA IS BARS (meaning without timestamp), so why do we have to synchronize?
Am I over-synchronizing?
What if my broker changes the time from summer to winter and back again (as do many brokers)? Does that mean that the Historical data App is safe as long as the broker doesn’t change the time? In which case, the historical data App only applies to a few brokers.
Still confused Petko!
Can you or anybody else explain to me (in simple words) what am I doing wrong?
Thank you.November 11, 2020 at 5:19 #65714
I get it now (I think) …
The most important thing is that charts in MT4 match the charts from Historical data App of the same time-frame.
The synchronizing must match with a RECENT distinguished bar.
When going half a year back in time, the charts still match but there may be a difference in the time stamp due to winter- and summertime. This difference SHOULD NOT affect back-testing.
If understand it right this time, and in order to maintain my sanity (sorry – I’m like 62,000 years old), I’ll have to partially (only partially) disagree with you when you say in your course:
Forex Trading with Expert Advisors + 30 Best Strategies (Every Month)
Video: The best solution for historical data
“but before you download the data, you need to do something VERY-VERY important. You need to SYNCHRONIZE the time zone. Something EVEN professional traders are missing. They ask me why their back-test in EA Studio doesn’t match with the back-test in Metatrader. The reason for that is the TIME ZONE.”
Any comments?November 12, 2020 at 12:27 #66007
I will try to make it very simple for you:
If your broker doesn’t synchronize with your time and is changing summer to wintertime, you can do nothing. You can not set in the App from this to this date to be one zone, then from this to this date to be another zone, and do that for 13 years.
If you are with UTC, there will be no changes with winter and summertime, and if your broker synchronizes it with you, you will have the data matched because the DATA IS BARs – open, low, high, and close.
I do not get into these details in the courses because the difference of the backtest is so minimal even though a winter-summer change doesn’t worth the time to do anything about it.
I hope that makes sense.November 12, 2020 at 16:29 #66197
Your explanation is indeed simple and clear. Thank you sooo much!
As you say: There’s nothing I can do about it because whether I set my computer time zone to UTC or to GMT+1, the discrepancy with summertime and wintertime is the same. Meaning my broker does not synchronize with me. So, as you say, I cannot set the App to be from this to this date to apply on one time zone and from that date to that date to apply on another time zone… Understood!
Now: Can I do the following with the “Data Horizon” function? (regardless the discrepancy in summer-/wintertime)
In stead of training on a demo account for weeks and months, I had the idea of testing 10 best strategies I generate every month from the beginning of 2020 and up to the end of October 2020. I was going to do so, by forcing (via Data Horizon) a start date and an end date of 3 years (not 5) up to every month of 2020.
Here’s an Excel illustration of my plan if that’s easier to understand:
It should be doable since the historical data is one huge block of data (timeframes and bars) – only synchronized at the end – Right?
Have you (Petko) or anybody had the same idea? Is it a good plan or a waste of time – honestly?
Please comment, fellow traders! Don’t leave everything to Petko.
Thanks.November 12, 2020 at 19:57 #66204
It is very simple to what I show in some of the videos.
You can generate EAs for any period of time using the Data Horizon and then just recalculate the recent period’s strategies.
That is what works best for me, and I really do not need to trade many EAs on Demo accounts this way.November 13, 2020 at 2:08 #66206
I do something similar. To test my ideas for example on H1. I use 5 years of data COT 5 and PF 1.2. I go back 5 years and 2 months. I set the data horizon for 5 years and exclude the last 2 months. I keep the reactors running until I have at least 300 strats and sometimes up to 2000… I then use the validator to “feed in” all my strats (the validator does not have the limitation of 300 like the collection does). I set the validator settings to something a bit broader like COT 5 and 2 consecutive losses. I then set the data horizon to 1 month ago so 1 month following on from the five years that the strats were created on. I then experiment with different filters on the collection tab. COT 5 – 10, sort by PF, WL ratios etc….. Then i select the top 5,10,20 and create the portfolio. Then change the data horizon to the last month and recalculate. Now you will see how your portfolio would have run in the last month with the previous months filters you applied. This gives you the ability to test 100’s of ideas and get a pretty good idea of how they will work in the future. Of course I am sure that this will not be quite as effective as running your ideas forward in actual time but it will let you know if you are on the right track. Because i have now created about 1000 strats for each of the 11 pairs that i trade i am setting the validator setting more stringently. Currently COT 8 and Losses in a row 1. This normally takes my 500 – 1000 strats to about 50 that meet the criteria for the month. From there I am finding the best ones are top 5 sorted initially by NB and then filtered by PF but I am still testing. I then run these top 5/10 forward on a demo with a balanced selection of currencies. So always the currency appears an equal amount of times: – EU, GU,GJ, EJ. that covers the 4 main currencies. I am now experimenting with adding AUS and CAD so You need 9 pairs to get even exposure: AC, AJ, EC, EJ, EU, GA, GJ, GU, UC. I have only just started to realise the power of the validator so I encourage to have a play with it if you haven’t already….November 14, 2020 at 13:07 #66243
Is it possible to change the title of this forum topic
Glitch in Historical data App
Use of Historical data & Data Horizon
Thank you in advance.
Of course it’s similar to what you teach in your videos. I learned almost everything I know from you!
Not only do you teach me how to operate EA Studio, you also teach me how to think! I’m so grateful for that.
Love your teaching methods, your spirits and your optimism.
Thank you for showing me your own method. Very interesting and inspiring indeed.
I totally agree when you say:
“This gives you the ability to test 100’s of ideas and get a pretty good idea of how they will work in the future. Of course, I am sure that this will not be quite as effective as running your ideas forward in actual time but it will let you know if you are on the right track. “
This is exactly my missing link to getting started as a newbie.
Ilan (our teacher and moderator) has already introduced me to the Validator, and I studied everything I found about this tool in EA Studio. But pointing out that the Validator is not limited by 300 strategies as do the Collection, was new to me. An eye opener. Thank you again Richard.November 17, 2020 at 2:30 #66508
Hey Petko, Richard & everybody
YOUR BEST, LATEST & EASIEST METHOD (condensed – in my words):
1. Create 10-25 strategies over 3 years of Historical data
2. Wait one month
3. Add the that one month to the Historical data
4. Recalculate everything and create a Portfolio Expert with NEW 10 best strategies
5. Run the Portfolio Expert with these 10 EAs on the new month LIVE
6. Rinse & repeat (without waiting one month)
MY METHOD (inspired from and built on your method):
1. Create a Portfolio Expert with 10 best strategies over 3 years of Historical data
2. Wait one month
3. VALIDATE the Portfolio Expert with 10 strategies on that month ONLY (BEFORE adding that month to the 3 years)
so that we can see what would have happened if we run them LIVE on that same month
4. Now we add the last month to the historical data (and subtract one month from the beginning of the historical data
to keep the period length of 3 years constant)
5. RECALCULATE and CREATE a NEW Portfolio Expert of 10 best strategies
6. Run the Portfolio Expert with these 10 EAs on the new month LIVE
7. Rinse & repeat (without waiting one month)
Is my method useful or full of holes?
Thanks.November 17, 2020 at 3:39 #66509
So if I read it correctly you are ultimately creating your portfolio of 10 from the 3 years of data. Of course at the end of the day you have to practice and find your own system. Which you are doing…. I prefer to create the strats leaving off the last month. Then run them on the last months data. And then select my top 5/10 based on the last months performance. In my mind that means that the strats have proved themselves over 5 years to be profitable and they have also proved themselves to be profitable in the last month. Because I use very stringent criteria they will be consistantly profitable over the whole of the month without any significant dips in the equity line…..
Example attached: Here I feed in over 1000 strats created over 5 years of data. Then run them over the last month. You see only 146 pass the criteria for the month. This is currently set to COT 8, Losses in a row 1. When I was only feeding in around 50 – 100 strats before I had to use much lower criteria to get enough strats to pass. (COT 5, Losses in a row 2. ) As I get more and more strats to feed in the tighter I make the criteria.November 17, 2020 at 3:42 #66510
Then bring up the PF till I get the best 10. Create portfolio:November 17, 2020 at 3:45 #66511
Then I end up with some pretty good looking strats into the portfolio:November 17, 2020 at 3:50 #66512
So this is my latest way of doing things. I don’t think you ever should stop evolving your systems. Because I feed in so many strats to get my top 10 / 5 (I am actually starting to find the top 5 more profitable lately…..) I also have started playing with the correlation %. I bring it up from the standard .98 to .90 to ensure that all my strats are significantly different from each other. More diversity within the portfolio.
Anyway it is good to see you experimenting with different idea’s based on what you are learning here. This is what I do as well. Good luck….
- You must be logged in to reply to this topic.