Methodology
How our numbers are made
Every backtest, dataset, and comparison on this site comes out of one pipeline: real exchange data in, an event-driven engine in the middle, and a robustness check before anything is published. This page documents that pipeline — so you can judge our numbers instead of having to trust them.
Data
All results are computed on real OHLCV candles fetched from crypto exchanges via CCXT — primarily Binance for the public datasets. Public pages use daily bars from 2021-01-01; the studio backtester also runs intraday timeframes (15m, 1h, 4h). We never use synthetic or interpolated price data.
Data fetching is fail-loud by design: if a venue cannot serve a requested window, the run errors instead of silently filling the gap. A backtest on patched data is worse than no backtest.
One engine, everywhere
The same event-driven backtest engine powers the product, every public page, and our published research runs. It processes bars as a stream: indicators update incrementally, warmup periods are respected, and a rule can only ever see data that existed at that moment — no lookahead, by construction.
Indicator implementations are verified against independent references (TA-Lib and TradingView) with an automated parity harness; core indicators match TA-Lib to floating-point precision.
Costs are part of the result
Every simulated fill pays trading fees — the public datasets apply standard exchange fees to a $10,000 account. Fee-free equity curves flatter strategies that trade often; a strategy that only wins before costs is a losing strategy.
Validation before publication
A good-looking equity curve is the beginning of the analysis, not the end.
Chronological 70/30 split
History is split in time order: a strategy is scored on the first 70% (in-sample) and re-scored on the last 30% it never saw (out-of-sample). Memorizing the past shows up as an out-of-sample collapse.
Walk-forward windows
Beyond a single split, runs are re-evaluated across rolling windows, so a verdict never hinges on one lucky period.
Monte Carlo & deflated Sharpe
Trade sequences are resampled to estimate how much of a result is luck, and Sharpe ratios are deflated for the number of configurations tried before anything is reported.
Overfitting probability
Parameter sweeps are scored with combinatorially purged cross-validation (PBO) — the probability that the winning configuration only looks best by chance.
What we don't do
- No trading signals for sale and no copy-trading — we publish methods and tools, not "follow me" calls.
- No cherry-picking: our house research runs are published win or lose, marked with a HOUSE badge, and every public share carries a hash-verifiable stamp.
- No paid placements in comparison or "best" pages — nobody can buy a better verdict.
- No financial advice. Backtests describe the past; they are not a promise about the future.
Corrections
If you find a number that looks wrong — a dataset row, a chart, an indicator value — tell us. We would rather fix an error than defend it.
Report an issue