How to Backtest Expert Advisors in MT4 Strategy Tester to Reach Every Tick Modelling Quality of 99% and Have Real Variable Spread Incorporated
MetaTrader 4 can reach 90% modelling quality at its best by default and it cannot incorporate real variable spread. But there's a better way to run backtests.
Written by www.Autotrading.Academy
99% modelling quality backtest basically shows the ugly truth and reveals weak side of the strategy
Below are two backtests run for the same Expert Advisor. First backtest is of 90% accuracy and the next one is of 99% accuracy.
It's obvious that this EA is not worth wasting anyone's time and money.
EURUSD history center data, default settings, 2010-2015
EURUSD Dukascopy data, default settings, 2010-2015
Tools you'll need to perform a 99% modelling quality Every Tick backtest
It's obviously you'll need to have MT4 platform installed on your computer in order to run any kind of backtest using MT4 Strategy Tester. However, note that MT4 alone can reach 90% modelling quality at its best and it even won't have real spread incorporated. MT4 alone can only use fixed spread which gives inaccurate results.
MetaTrader 4 trading platform is free of charge and available at mostly any Forex broker.
You will need history price data for backtesting and Tick Download software is exactly what you need. It allows you download history price tick data from Dukascopy, which is free to use for everyone, and even converts it to your preferred timezone. Usually you want to convert the price data to the same timezone used by your broker.
Tick Downloader is free of charge.
Birt's CSV2FXT Script
You'll need CSV2FXT script to convert the Dukascopy history price data to a MetaTrader 4 format (.FXT files) which are used by the MT4 Strategy Tester. This script has quite a big range of parameters which give a lot of flexibility when preparing your backtesting environment.
Birt's CSV2FXT script is free of charge.
Tick Data Suite
Tick Data Suite is the only paid software you'll need to reach 99% modelling quality. The software is quite easy to install and easy to use. TDS not only offers 99% modelling quality backtests, but also adds real variable spread that have actually happened in the past, allows simulation of slippage, and running multiple MT4 instances at the same time.
TDS is the only software available for 99% modelling quality backtests with real variable spreads incorporated. Tick Data Suite is not free, but it's price is quite reasonable.
This guide is for older Tick Data Suite version, not the new TDSv2.
How to prepare your backtesting environment?
Before you can run any backtest you'll need to prepare your backtesting environment which is to download, install and configure required software.
Download History Price Tick Data
First we need to download history price tick data. It takes quite a while to download full history data files as they are quite large, so it's wise to start downloading them now.
Download free "Tick Downloader" from here: http://www.strategyquant.com/tickdownloader/
Start the installer you've downloaded and follow on-screen instructions to finish the installation.
Usually it's wise to choose installation folder outside "Program Files" folder, like on a separate disk drive, Desktop or your Documents folder.
Set Data Output Folder
Once you open "Tick Downloader" make sure you set your preferred data output folder in the SETTINGS window.
I recommend you using separate folder or separate disk drive as data output folder.
Select Pairs & Range
Select pairs you want to download price tick data for and then select the range. Usually I select "All Time", but obviously you can download smaller ranges if you need to.
Note that full history price tick data file (~13 years) comes in CSV format and takes around 11 GB of disk space. If you'll convert the file into different time zone then "Tick Downloader" will create additional file of similar size.
Choose Timezone & Download
Next click "Configure" to choose your preferred time zone and make sure you select "Automatic export to CSV" so that "Tick Downloader" create CSV files automatically after tick data is downloaded.
After you set the parameters hit the "Start download" button and your data will start downloading. It will take quite a while so be patient.
Tick Data Downloaded
Here's my history price tick data downloaded and converted into CSV files.
EURUSD folder holds the tick data files and additionally two CSV files were created. One file for default GMT-0 timezone and the other for my chosen GMT+2 timezone with European DST enabled.
These files hold history price tick data for 13 years and they take 22 GB of disk space. That's only for one instrument.
CSV files can be deleted later after you convert them into MT4 FXT files.
Download & Install MetaTrader 4 platform
Download MetaTrader 4 for PC, install it and create a demo account. For maximum convenience and easy of use install MT4 outside of the Program Files folder. Good idea to install on a separate hard drive, because history price tick data will use a lot of disk space.
Download & Start MetaTrader 4 installation
Download MT4 from your broker or directly from here: http://www.metatrader4.com/
Once downloaded, execute the MT4 installation wizard, which is usually named mt4setup.exe
If you download MT4 from your broker the file name might be different.
Follow MT4 on-screen instructions
Once you agree with the terms and conditions click the SETTINGS button to choose custom installation destination folder.
Choose custom MT4 installation folder
This step is very important. You need to choose installation destination folder outside the "Program Files" or any other system folder.
Best practice is to use external disk drive because history price tick data will use a lot of disk space. If you do not have external disk drive you can simply install MT4 on your desktop or inside Documents folder as an example.
Enabling Portable Mode for MT4
After MT4 installation is finished the platform will load in just a few moments. You need to close the platform for now and enable Portable Mode before you continue to use it. Do not skip this step to make the whole process easier.
Select Properties of your new MetaTrader 4 shortcut and then add /portable at the end of the text you'll find in the "Target" field. Click OK to save changes and double-click the shortcut to start MT4 in Portable Mode.
IMPORTANT! From now on, start this MT4 only using this shortcut.
Choose MT4 server to create account
Once MT4 loaded you need to create a demo account. But first you need to select a demo server of your broker or simply add MetaQuotes-Demo server to the list by clicking the "add new broker ..." and typing "MetaQuotes".
Once the server appears on the list select it and click NEXT, select "New demo account" and click NEXT again and follow on-screen instructions to finish opening your demo account.
If you have an account already just click CANCEL and log into your account.
Log into your MT4 account
After login window appears enter your MT4 account number (login) and password to log into your MetaTrader 4 account.
MT4 accounts has two passwords (main and investor) and they both will work for converting history price data into FXT format.
Install Birt's CSV2FXT script & convert price tick data to FXT files
Download and install Birt's CSV2FXT script, copy CSV files to your MetaTrader 4 folder and convert them into FXT files. This is a special MT4 file format used by Strategy Tester module to perform backtesting of Expert Advisors.
Install Birt's CSV2FXT Script
Open the Data Folder of your MT4 platform from the top menu "File -> Open Data Folder" and then go inside "MQL4" folder.
Now download Birt's CSV2FXT script into this folder from here: http://eareview.net/tick-data/downloads
Make sure you choose the latest script version for MT4 build 545 or higher. Birt's CSV2FXT script comes in three files:
1) CSV2FXT.mq4 goes into MQL4Scripts
2) FXTHeader.mqh goes into MQL4Include
3) CsvReader.dll goes into MQL4Libraries
Refresh scripts list in MT4 Navigator window
To make the CSV2FXT script appear in your MT4 without restarting the platform you need to refresh the scripts file list.
In the Navigator window click on "Scripts" and them "Refresh". This will update the list with all the recently installed Scripts, Expert Avisors and Indicators.
If Navigator window is not visible you can turn it on by pressing CTRL+N or through the top menu "View -> Navigator".
Copy tick data CSV files into MQL4 Files folder
In the same Data Folder of your MT4 platform go inside "MQL4" and "Files" folder.
Now move your CSV files here so that CSV2FXT script can see them and convert into FXT files.
It's highly recommended that you move the CSV files, not copy, but move them so that you do not create additional copied taking up additional disk space. Also moving files is faster than copying them and usually takes just several seconds.
Start CSV2FXT script and configure
Attach CSV2FXT script onto EURUSD chart (because we convert the price data for EURUSD) and configure.
In the COMMON tab check the "Allow DLL imports" so that the script can work.
Set CSV2FXT script Inputs
In the INPUTS tab set your desired parameters. Here's how I do this:
CSV filename = EURUSD_tick_UTC+2_00.csv
Create HST files = trueUse real (variable) spread = true
CSV GMT offset = GMT+00
CSV DST setting = None
Remove duplicate ticks = false
Create H1 FXT = true
There are parameters for each time frame, so you can choose the ones you need for your backtesting environment and set them to "true".
For detailed explanation about each parameter refer to the author's website.
Answer NO after CSV2FXT script finishes creating FXT files
After CSV2FXT script finishes creating FXT files it will ask if you want those files to be moved to the right folders.
Click NO. We will do this manually to avoid HST files being overwritten by MT4 platform and updated with the most recent price tick data from your broker server.
Disconnect from MT4 account
It is very important that you disconnect from your MT4 account at this stage, before moving the newly created FXT and HST files into designated folders.
This way you'll make sure MT4 does not overwrite price data with the one from broker servers.
The most convenient way for me to disconnect from MT4 account is to open Login window (File -> Login to Trade Account) and add zero at the end of the login number.
You should see "Invalid account" message at the bottom right corner once disconnected.
Protect HST and FXT files (Step 1)
It is very important to make HST and FXT files read-only to protect them from being overwritten by MT4.
Open MT4 Data Folder again, navigate inside the
and select all FXT and HST files.
Protect HST and FXT files (Step 2)
Open Properties window and set "Read-Only" attribute to these files.
To make sure the "Read-Only" box is really selected, first you should clear it and then select.
Move FXT and HST files into designated folders
In the same MQL4Files folder and sort the files by "Type".
You have a list of HST files and FXT files that needs to be transferred to designated folders.
CSV file can be removed now, but it is recommended to leave it in case you'll need to create FXT files again.
Move HST files
HST files should be moved to the folder named:
Note that instead of MetaQuotes-Demo you might actually see your broker's server name.
For example on HotForex MT4 platform this folder would be named:
This folder name should match the server name from the MT4 Login window where you've entered your login and password previously.
Move FXT files
FXT files should be moved to the folder named:
These are the files MT4 Strategy Tester will use for back-testing.
This is it. You've converted and prepared history price tick data.
Install Tick Data Suite for MT4
Purchase and install Tick Data Suite software for your MetaTrader 4 platform. This software is mandatory for running Every Tick 99% modelling quality backtests with real variable spread incorporated.
Purchase & Download Tick Data Suite
Purchase & download Tick Data Suite from this website:
TDS is the only software available for 99% modelling quality backtests with real variable spreads incorporated.
Install Tick Data Suite
When installing Tick Data Suite simply follow on-screen instructions and leave the "Launch Tick Data Suite Configuration" box selected at the end.
When you click Finish the configuration window will appear.
Activate and Connect TDS with MT4 platform
Activate TDS by entering your Serial number and clicking Update.
Then click "Select Path" and select your MT4 platform which should be connected with TDS. Click "Copy TDS" and then YES when it will offer to create a shortcut to this TDS instance.
Note: You can connect unlimited number of MT4 platforms on the same computer.
Enable Portable Mode for Tick Data Suite
After TDS is installed into specific MT4 platform you need to enable Portable Mode before you start using it. Do not skip this step to make the whole process easier.
Select Properties of your new TDS shortcut and then add /portable at the end of the text you'll find in the "Target" field. Click OK to save changes and double-click the shortcut to start MT4 with TDS in Portable Mode.
IMPORTANT! From now on, start this MT4 with TDS only using this shortcut to enable 99% backtesting quality with real spread.
Launch MetaTrader 4 with Tick Data Suite
Start MetaTrader 4 platform with Tick Data Suite using the newly created shortcut.
Once you will be prompted to login it is very important that you don't. Click Cancel. Never log into MT4 account unless you need to convert the price data again.
Take a look at the top of your MT4 window. If it says "MetaTrader 4 with Tick Data Suite" it means you've done it correctly and now your MT4 is ready for 99% backtest with real spread.
Note: You can also launch multiple MT4 platforms with TDS and run multiple backtests, which is normally not possible without TDS.
Running 99% modelling quality Every Tick backtest with real spread
Now you are ready to run Every Tick backtests with real spread incorporated and reach 99% modelling quality. Having real spread in your backtesting process makes your strategy test more precise.
Open MT4 Strategy Tester window
Backtesting of automated trading strategies (Expert Advisors) is done in the MT4 Strategy Tester window. You can open it from the top menu (View -> Strategy Tester) or by pressing CTRL+R.
Select EA you want to backtest
Choose which Expert Advisor you want to test and click "Expert Properties" to set your desired parameters.
In the "Testing" tab enter initial deposit amount, choose currency and make sure "Long & Short" positions are selected to allow both, BUY and SELL operations.
Set your desired EA inputs
Most of the Expert Advisors have at least few parameters you can set. In the "Inputs" tab you can set them the way you want for this particular test.
Select instrument, time frame and modelling type
Next step is to select currency pair (Symbol) and its time frame. Then you need to select "Every Tick" as your modelling type.
It has no effect if you change the spread here as Tick Data Suite will override this setting and use real variable spread which actually happened in the past. In this example "Current" is selected and that's what will show up on the report.
Set the date range
Strategy Tester allows you to select date range for the test. If it's not selected, like in this example, backtest will be performed on all available history price data.
But if you need to test the strategy only during specific date range you can easily do that.
When you done setting parameters click "Start" to begin the test. It might take a while depending on how long the date range is, EA algorithm and your computer power.
Backtest finished. Here are the results
After backtest is finished you can see the results. In this equity chart we see modelling type and quality is printed, which was "Every Tick" with 99% quality.
Report of the backtest
In the "Report" section you can see more results of the backtest including the modelling quality number as well.
Additionally you can find full trade list generated during the test in the "Results" tab, find if there were any errors (Journal) or restart the test again in the "Settings" tab.