Are you still full of doubts that testing a strategy on historical data can be useful in real trading? In this article, we will compare achievements of the previous Championships' leading participants with their results shown in MetaTrader 5 Strategy Tester during the automatic tests.
The Championship observers have always been curious about the results shown by the leading trading robots in the Strategy Tester before the competitions. Are they genuine winners or was it a pure luck?
Results of the following participants will be analyzed in the article:
Not all participants have responded to our request to use their competition data. Therefore, we have not been able to include the entire TOP-10 into our analysis. So, "we have what we have."
First of all, we should compare results of the leading participants on a test period and during the Championship. Let us start from four Expert Advisors from ATC 2011. The first graph (already familiar to you) displays the changes of the participants' balance during the Automated Trading Championship 2011.
The next graph shows the results of the same participants when testing their Expert Advisors on historical data during the automatic tests:
As you can see, all Expert Advisors performed well during the tests. Most participants steadily increased their balances. Total balance ranged from $80 000 to $230 000 in the Strategy Tester at the end of the seven-month test period. The Expert Advisors worked well though no fabulous results of millions of dollars were detected.
Now, let us examine similar graphs from ATC 2010.
Let's see how Expert Advisors performed during the tests and the competition.
Like in 2011, trading results are good and steady enough. Expert Advisor of Boris Odintsov (bobsley) made $375 000 for the seven-month test period. Balance curves of Dimitar Manov (Manov) are almost similar on both graphs displaying a steady movement with a sharp drop at the very end caused by forced closing of positions.
Analyzing the balance graphs we can conclude that leading positions at the Championship are not just pure luck.
Now, let us find out what strategies are most popular among the Championship leaders - lots of deals with small profits or vice versa. To achieve that, we should compare the number of performed deals and total profit.
As you can see, the dots' locations are similar on both graphs. Most trades have been performed by Yedelkin and Manov. They are followed by the group of five participants with the average number of deals. The winners of the both Championships are detached from the rest of the group, though it seems they have switched places on the graphs.
The data table is shown below to simplify the analysis:
||Trades (АТС/Tests)||Deals (АТС/Tests)||Ratio of profit to deals (АТС/Tests)|
||103 135 / 223 655||11 / 36
||33 / 97
||3125 / 2305
||27 826 / 135 013||49 / 49
||99 /123||281 / 1097
||49 140 / 70 674||115 /307||231 /614||212 / 115
||58 810 / 102 922
||108 /211||229 / 445||256 / 231
|tmt0086||45 529 / 62 766
||84 / 213
||168 / 426
||271 / 147
|| 22 773 / 87 866
||307 / 896||1478 /4674||15 / 22
||67 102 / 367 670
||142 /337||551 / 1205||121 / 305
||36 534 / 76 329
||159 /346||319 /692||114 / 110
||30 553 / 108 249
||585 /828||1031 / 1662
||29 / 65
The closer the values of the ratio of profit to the number of deals on ATC and during the automatic tests, the less fitting in Expert Advisors. The ratio values of almost all participants correlate with each other except Ilnur Khasanov's one (aharata). His ratio was 4 times higher during the automatic tests.
We can conclude that the total profit is not affected by the number of performed trade operations too much. An Expert Advisor can take the lead making only 11 trades, like Igor Korepin's one, or trading frequently (551 deals and 142 trades), like Boris Odintsov's one.
Finally, let us compare another two basic parameters of trading robots' operation - profit factor and equity relative drawdown.
Equity relative drawdown is the largest equity drawdown in a certain percentage of the local equity maximum. It means that a relative percentage instead of an absolute monetary value is calculated. Relative drawdown on historical data is an important value for estimating a trading strategy during its development and optimization.
The left graph shows that most participants have similar profit values in Strategy Tester and during the Championship. None of the leaders gained fabulous profits that we could see when analyzing all the participants in the article Optimization VS Reality: Evidence from ATC 2011 where this parameter often exceeded 12-15. None of the Expert Advisors with overstated profitability in the Strategy Tester were able to repeat their results during the Championship.
As for the equity drawdown, it is impossible to draw some definite conclusions. Some participants took heavy risks waiting out the losses, but still managed to gain profit, while some participants were more cautious. It should be noted that all winners have similar drawdowns during the tests and the Championship comparing to other contestants.
In this article, we carefully examined operation results of TOP-10 Expert Advisors of the last two Championships shown in the Strategy Tester and during the competitions. The analysis turned out to be very interesting. It appears that you cannot just climb to the top of the Automated Trading Championship with only pure luck. Surely, there is some part of luckiness in that but a well thought-out trading strategy is always the key factor.
All Expert Advisors from ATC 2010 and 2011 TOP-10 showed excellent results during the preliminary tests. Besides, they did not show any signs of fitting which often leads to displaying unlikely results during the tests and ends with deposit loss in real trading.
Develop and optimize your trading robot wisely. Win the Automated Trading Championship 2012 and turn your virtual test profit into real $40 000!
Optimization VS Reality: Evidence from ATC 2011
You want it, you've got it! After the publication of the article "The Mystery of Changing Leaders Solved?" we have received many requests to write an article having a greater analytical bias. Today we have a hot topic - comparing the Expert Advisors' parameters obtained during the automatic tests before the Automated Trading Championship 2011 with the ones shown during the Championship itself.
Each newcomer wants to create a trading system capable of making maximum profit on each price movement. As time passes by, traders gain more and more experience finally reaching more realistic expectations and abandoning their dreams to buy exactly on the very bottoms and sell on the very tops. Everyone knows that it is impossible to trade with the accuracy of ZigZag indicator often shown on history data...
|← Previous||Next →|