Data study

MT5 Strategy Tester accuracy: real ticks vs synthetic, and why backtests disagree

Tommy J.Founder, RoboticEA

17 September 2026·13 min read

If you have ever run the same Expert Advisor through the MetaTrader 5 Strategy Tester twice, on two different price histories, and watched the equity curve change shape, you have met the most common reason a backtest “differs”: the tester can only replay the price path it is handed. We checked how much that matters by running 100 strategies through both RoboticEA’s engine and the MT5 Strategy Tester, first on a tick stream built from bars, then on 24,596,614 real ticks.

A Strategy Tester run has three inputs that people rarely write down: the price path the tester replays inside each bar, the costs charged on it, and the account model that books the trades. Change any one and a strategy that looked identical on a chart will not produce the same trades. This post is about the first, because it is the one that is invisible in the report and the one we could measure cleanly.

The tester offers modelling modes that differ in where the ticks come from. “Every tick” builds them from bar data; “Every tick based on real ticks” replays recorded ticks. We measured a bar-derived stream that our own tooling wrote and a real-tick stream, so treat what follows as evidence on bar-derived versus real, not as a measurement of the tester’s built-in generator, which we did not test. The tester did exactly what it was asked in both cases, and that is the point: MetaTrader 5 is a faithful replay engine, and a replay is only as informative as the tape.

The first run gave MT5 a custom EURUSD symbol whose only history was bars. To let the tester run in its tick-based mode, our importer wrote four points per five-minute bar — open, high, low, close, in that order, a quarter of the bar apart — with the ask a fixed two points above the bid. That is 48 points per hourly bar, and the tester’s journal confirmed it: 273,200 ticks over 5,695 bars, or 47.97 per bar. The second run replayed real Dukascopy ticks, loaded into a custom symbol, with the spread each tick actually carried.

Real EURUSD does not look like that. In the same window a typical five-minute bar held 283 ticks (mean 360), and only 0.2% of bars held four or fewer. An hourly bar held a median 3,461. The count also swings through the day, from about 1,151 ticks in the 21:00 UTC hour to about 10,074 at 14:00 — and the spread swings with it, averaging 1.38 pips at 21:00 against a median of 0.30 pips overall. A fixed spread cannot represent that, and a flat 48 points a bar cannot represent any of it.

For a concrete week, take Monday 6 to Friday 10 March 2023: 671,703 real ticks, where a four-point stream would offer 5,660. Everything a strategy does between the four points — whether a stop was touched before a target, what a fill cost — is decided by the stream, not by the strategy.

• Same strategies, both tools. 100 strategies from RoboticEA’s library (presets, composed strategies and stateful sizing and basket families), each run in the C++ engine and exported as an Expert Advisor and run in the MT5 Strategy Tester.
• Same window and account. EURUSD H1, 2023-02-01 to 2023-12-31, $10,000 deposit, leverage 1:100, seed 42. Swap and commission were zero on both sides so a cost difference could not hide a fill difference.
• Same tick file in the real-tick run. Both legs consumed one file of 27,553,506 Dukascopy records. 24,581,968 fall inside the window; MT5 reported 24,596,614, a 0.06% difference. Each side then builds its own candles from those ticks, as a live terminal does.
• Every row is on disk. The figures on this page are recomputed from the raw run records by a script, and the published /parity-study table is the same 100 rows, gap for gap.

On real ticks, the median absolute gap in total return was 6.06 pp across the 98 strategies that traded on at least one side (2 took no trades in either tool). The headline figure of 6.07 pp above is the same data on the 97 strategies that also ran on the bar-derived stream, no-trade rows included; a slightly different row set, not a different result. The spread of outcomes matters more than the median, so here are the deciles, then the same strategies on the bar-derived stream beside them.

The engine and the tester also agree on which strategies did well: across the 98 strategies the correlation between the two total returns is 0.92, and they agree on whether the strategy finished up or down for 89 of them.

Read that as agreement of the same order of magnitude, not identity. The worst gaps are concentrated in strategies whose result compounds: the median gap was 3.65 pp where the MT5 return was under 20% in size (66 strategies) and 12.58 pp where it was 20% or more (32). Stateful strategies, whose position size reacts to earlier trades, had a lower median gap (3.14 pp against 6.18 pp for the rest) but the fattest tail.

The reasons attached to those rows in the run record are mundane: a stateful sizing strategy where one different fill is amplified, a volume-reading strategy, or a large compounded return that turns a small fill difference into many points. One row has no cause we could measure, and we say so rather than inventing one.

Before the real-tick run we had three separate anomalies on the books, each with its own theory. All three moved when only the tick stream changed.

1. Stops that filled at the wrong price

In a four-point stream, a price between the open and the high has nothing to fill against, so the tester fills such a stop at the bar's extreme instead. Across the paired strategies the median bias (engine minus MT5) fell from +4.92 pp to +1.59 pp, the engine was the higher of the two on 67 strategies before and 60 after, and 55 of 97 strategies were closer to the tester on real ticks. The clearest single case, preset x25 oscillator stoch kd cross regime eurusd rr2: engine +69.0% against MT5 +9.3% on the four-point stream (+59.64 pp), and +39.3% against +39.4% on real ticks (−0.08 pp).

2. A tick-volume series that was a constant

Volume-reading indicators use the tester’s tick count. A four-points-per-bar stream makes that count 48 for every hourly bar, so a volume indicator sees a flat line. The strategies that read volume (11 of them traded on at least one side) had a median gap of 18.35 pp against 7.26 pp for the rest on the bar-derived stream, a penalty of 11.09 pp. On real ticks it was 8.42 pp against 5.95 pp, a penalty of 2.47 pp: 78% of it gone, not all of it. What remains is plausibly a different volume series (recorded size in our engine, tick count in the tester); we have not confirmed that.

3. Grid strategies that made impossible money

On the bar-derived stream, 10 grid strategies returned between +190% and +5508% in the MT5 tester while the engine, on the same strategies, reported between −44% and +32%. A grid adds to a position as price moves against it and closes the basket on a small retracement. One plausible mechanism is that a stream visiting the high before the low inside every bar hands it a tidy retracement after every excursion; we saw the effect, and did not isolate that mechanism.

Only two of those grids have a real-tick MT5 report on disk. Both fall to ordinary losses:

Two strategies are not a distribution, and we are not claiming one. What they show is direction: the absurd figures were a property of the stream, and they disappear when the stream is real.

On the bar-derived stream, the engine and the tester took exactly the same number of trades on 43 of 97 strategies. On real ticks that dropped to 5. If you judge agreement by trade count, the real-tick run looks much worse. If you judge it by money, it is better. Both are true, and they are the same fact.

The mechanism is simple. When both tools replay the same coarse path, they evaluate every signal on the same bar closes and reach the same decisions by construction — and the same coarseness is what mis-fills the stops, so the two agree with each other and both are wrong about the money. On real ticks, each tool builds its own candles from a very large tick set (the two counts differ by 0.06%), a marginal signal can land on either side of its threshold, and the trade lists drift apart while the fills become realistic. Even the price a candle is built from matters: building the engine’s bars from the mid rather than the bid, which is what the tester charts, took exact matches from 5 to 9 of 100 while making the money gap worse (6.84 pp against 5.95 pp).

There is also a residual we cannot explain from the records: on real ticks the engine took more trades than the tester on 88 of the 95 strategies where the counts differed. The direction is consistent, so it is probably systematic, and we have not found its cause.

• One instrument, one timeframe, one window. Every row is EURUSD H1, 2023-02-01 to 2023-12-31, one seed, one $10,000 account, on a curated 100-strategy set. Nothing here speaks to gold, indices, other timeframes or other years.
• Agreement with MT5 is not proof of live fills. It shows two simulators agree when fed identical ticks. Both legs used Dukascopy ticks; your broker’s feed, spreads and slippage differ, and a live account will differ again.
• About 6 pp is the honest number, not one point. Only 47% of strategies were within 5 pp of the tester. Use the engine to rank and stress strategies, and the tester to confirm before you trust a number.
• One suspect row. preset_x25_mean_reversion_williams_cci_double_extreme_eurusd_rr2 shows the widest trade-count gap. An earlier audit found that row contaminated by tester-container reuse, so the MT5 side of it is probably a harness artefact. It was not re-run for this post; leaving it out moves the median to 6.05 pp.
• The volume set is a classification, not a proof. The strategies flagged as volume-reading come from an earlier audit’s list rather than a fresh read of every spec.
• Earlier boards looked tighter. Boards we published earlier showed smaller gaps under a different, bar-fed set-up. We could not reconcile them with the run records that remain, so we do not quote them.

• Read the tester journal before the report. It states how many ticks and bars were generated. A count near 48 per hourly bar means the run replayed roughly four points per five-minute bar, not recorded ticks.
• Choose the real-ticks model when you want fills to mean something. In our harness the tester’s real-ticks mode on a custom symbol that held only bars produced no trades at all, so if a run comes back empty, check the tick history before the strategy.
• Charge the spread the instrument really had. On the ticks we used it averaged 0.34 pips, 1.30 at the 99th percentile, and 1.38 in the 21:00 UTC hour. A constant will flatter or punish a strategy depending on when it trades.
• Set swap, commission and account type deliberately. We zeroed swap and commission on both sides to isolate the tick effect. Baskets and grids are booked per leg on a hedging account, so compare their returns, not their trade counts.
• Be careful with volume indicators. They read tick counts, which are only meaningful when the ticks are real.
• Run it twice and treat the spread as your error bar. The same MT5 strategy moved a median 7.75 pp between the two tick models (the largest move was 121.6 pp). A single tester run is one draw.

RoboticEA is built around that loop: backtest with a per-year record and the drawdown beside every return, export readable MQL5, then confirm in your own MetaTrader 5. See how the workflow fits together, the full parity study table, and start free.