Backtesting, done honestly
Backtesting is how you estimate whether a strategy has an edge — and it is the easiest thing in trading to get wrong. These pages explain the workflow and, just as importantly, the biases and statistical traps (survivorship, look-ahead, data snooping, overfitting) that make a backtest lie, plus the techniques — walk-forward, Monte Carlo, out-of-sample and forward testing — that make it more trustworthy.
Backtesting: Backtesting is simulating a trading strategy on historical data to estimate how it would have performed. Done honestly it needs clean, survivorship-free data, no look-ahead bias, out-of-sample and walk-forward validation, and awareness that curve-fitting and data-snooping produce backtests that look brilliant and fail live. A good backtest reduces uncertainty; it never proves future profit.
What is Backtesting?
Core conceptBacktesting is the process of simulating a fully-specified trading strategy on historical market data to estimate how it would have performed, so you…
The Backtesting Workflow
ProcessThe backtesting workflow is the disciplined, repeatable pipeline that turns a trading hypothesis into a validated strategy: form a hypothesis, prepar…
Historical Data
DataHistorical data is the record of past prices, volumes and related information used to backtest strategies, and its granularity, cleanliness, adjustme…
Survivorship Bias
BiasSurvivorship bias is the distortion that arises when a backtest uses only instruments that survived to the present, silently excluding delisted, merg…
Look-Ahead Bias
BiasLook-ahead bias is the error of allowing a backtest to use information that would not actually have been available at the moment a decision was made,…
Data Snooping
StatisticsData snooping is the statistical error of testing many strategies, parameters or variations on the same data and then selecting the best, which almos…
Curve Fitting
OverfitCurve fitting is tuning a strategy's parameters so tightly to the specific ups and downs of the tested history that the backtest looks excellent but …
Overfitting
OverfitOverfitting is when a model or strategy is complex enough to memorise the noise in the data it was built on rather than learn the underlying structur…
Walk-Forward Testing
ValidationWalk-forward testing repeatedly optimises a strategy on an in-sample window and then tests the chosen parameters on the immediately following out-of-…
Monte Carlo Simulation
RobustnessMonte Carlo simulation resamples or randomises a strategy's historical trades or returns many times to generate a distribution of possible outcomes, …
Out-of-Sample Testing
ValidationOut-of-sample testing evaluates a strategy on data that was never used to design, tune or select it, giving the most honest available estimate of how…
Forward Testing
ValidationForward testing runs a finalised strategy forward on genuinely new data as it arrives in real time, either on paper or with small live capital, provi…
Paper Trading
SimulationPaper trading is the practice of running a strategy on live market data in real time with simulated, no-money execution, letting you test the logic, …