The 7 ways backtests lie — and how to avoid them

Every backtest is wrong. The useful ones are wrong in known, bounded ways. The dangerous ones are wrong in ways you don't notice until you've wired real capital into them. This is a tour of the seven failure modes you'll meet in retail backtesting, and the cheap mitigations that keep most of them at bay.

1. Lookahead bias

The classic. Your strategy at bar t accidentally peeks at data from bar t+1 or later, and you don't notice because the simulator doesn't enforce a strict causal boundary. The most common form is computing an indicator on the full series before slicing — your EMA at bar 100 is now influenced by what happened at bar 500.

Fix: every indicator value must be computed using only data available at or before the bar it's evaluated on. A well-built simulator (the one in this platform included) walks the bars in chronological order and only exposes the past to the rule evaluator.

2. Survivorship bias

You test on the current S&P 500. The S&P 500 from 2010 contained Enron-shaped holes — companies that delisted, went to zero, got acquired. By testing only on today's survivors, you're testing only on companies that didn't fail. Your backtest looks great. The strategy would have died with Lehman.

For crypto this matters less but it's not zero. Test on a basket of "the top 10 by market cap as of 2020" and you're missing Terra Luna's collapse, FTT's wipeout, and a dozen smaller token failures. Use point-in-time symbol lists, not today's snapshot.

3. Overfitting

You searched 10,000 parameter combinations and found one with Sharpe 2.5. Congratulations — you've found the lottery winner, not a strategy. With enough degrees of freedom you can find a curve fit for any historical series; the test is whether the same parameters generalise to data you didn't optimize on.

Walk-forward is the standard mitigation. See the dedicated piece on walk-forward optimization for the mechanics, and the walk-forward docs for the implementation in this platform.

4. Missing slippage and fees

The default simulator fills you at the close of every signal bar with no spread, no maker/taker fee, no impact cost. Reality charges 5-15 bps each side on a liquid crypto pair, more on illiquid alts, and more again if your order is large enough to move the book. A strategy that turns over 200% per month dies at 5 bps round-trip if its edge was 12 bps per trade.

Round-trip fees (bps)   Net edge per trade   Annual return (52 trades)
0                       1.00%                 +67%
5                       0.95%                 +63%
10                      0.90%                 +59%
20                      0.80%                 +51%
50                      0.50%                 +30%
100                     0.00%                  +0%

Always model fees explicitly. If you're unsure of your venue's fee schedule, start with 10 bps round-trip and tune from there once you see live fills.

5. Take-profit and stop-loss fills

Many backtesters fill TP and SL at the next bar's open, not at the price you specified. On a 4h candle that's an enormous discrepancy — a wick that touched your stop at $42,000 may be reported as a fill at the next-bar open of $39,500. Free $2,500 per trade in fake P&L.

The correct behaviour is intra-bar: if the high of bar t exceeds your TP, you got filled at the TP price during that bar. If the low pierces your stop, you got filled at the stop. This platform's simulator does both. Verify your simulator does too.

6. Regime change

Markets are not stationary. The BTC of 2017 (parabolic retail mania) is a different market than the BTC of 2022 (institutional + macro), and that's a different market than the BTC of 2025. A strategy fitted on the regime of one period may simply not apply to another.

Defences: test across at least two distinct regimes. Track rolling Sharpe on out-of-sample data and watch for drift. Be willing to re-fit periodically and accept that no strategy is forever.

7. The equity-curve fallacy

The equity curve is a single random walk through possibility space. If you run the same strategy on a Monte Carlo permutation of the trades (random order, same multiset), you'll get a fan of equity curves — and many of them look nothing like the one you saw. Yet you'd make the deployment decision based on a single curve.

Better diagnostics: max drawdown over the trade-shuffled distribution, probability of a 6-month underwater period, time-to-recover. A strategy with a 95th-percentile max drawdown of 35% will at some point produce a 35% drawdown — plan capital so that doesn't end you.

Pre-deployment checklist

• Indicators are causal (no peeking forward).
• Symbol list is point-in-time, not today's survivors.
• Walk-forward results are within striking distance of in-sample.
• Fees and slippage are modelled to at least 10 bps round-trip.
• TP / SL fill intra-bar at the trigger price.
• Strategy was tested across at least one bull and one bear regime.
• Monte Carlo drawdown distribution is sized for your account.

Next steps

Read the backtesting docs for the exact fee, slippage, and fill model the platform uses, then move strategies that survive backtest into paper trading before risking capital. The simulation-vs-reality gap is real, and paper is the cheapest way to measure it.