Home / Backtesting

The Backtesting Checklist

A structured checklist for building a backtest you can trust: the biases that inflate results and must be eliminated, and the validation steps that establish whether an edge is real.

Backtesting: A trustworthy backtest is defined less by the profit it shows than by the biases it has ruled out. The five biases that most often manufacture fake performance are survivorship, look-ahead, data-snooping, overfitting, and unrealistic fills and costs. Once those are controlled, an edge is validated — not proven — through progressively harsher tests: out-of-sample data the model never saw, walk-forward analysis that re-optimises on a rolling basis, Monte Carlo resampling to see the range of possible paths, and finally forward (paper) testing on live data before any capital is committed. Passing every step reduces the chance of self-deception; it never removes it.

Work through this checklist in order. The biases come first because no amount of validation rescues a backtest that was contaminated at the data or logic stage — you would simply be validating a fantasy. For the underlying concepts, see What is backtesting and the backtesting workflow.

Part 1 — Biases to eliminate

Each of these inflates backtest performance in a way that will not survive contact with live markets. Treat them as defects to be actively disproven, not risks to be vaguely acknowledged.

BiasWhat goes wrongHow to eliminate it
SurvivorshipTesting only on instruments that still exist today ignores the delisted, merged, and bankrupt names, biasing results upward.Use a point-in-time universe that includes dead symbols. See Survivorship bias.
Look-aheadUsing information not available at decision time — e.g. today's close to trade today's open, or restated fundamentals.Enforce strict as-of timestamps; only use data time-stamped before the bar you act on. See Look-ahead bias.
Data-snoopingTrying many variations on the same data until one looks good; the winner is often chance, not edge.Limit the number of trials, adjust for multiple testing, and reserve untouched data. See Data-snooping.
Overfitting / curve-fittingA model with too many parameters memorises the noise in one sample and fails out-of-sample.Favour few parameters, wide stable plateaus, and simple logic. See Overfitting and Curve-fitting.
Unrealistic fills & costsAssuming perfect fills at the signal price with zero cost overstates every return metric.Model brokerage, STT, exchange and stamp charges, plus slippage and partial fills.

India-specific cost reality

On Indian equities and F&O, frictions are not trivial: Securities Transaction Tax (STT), exchange transaction charges, GST on brokerage, SEBI turnover fees, and stamp duty all apply. An intraday options strategy that ignores STT on the sell side and realistic bid-ask spreads on illiquid strikes can look profitable in a backtest and lose money live. Model costs per leg, per trade.

Concrete look-ahead traps

  • Filling an order at a bar's close when the signal was computed from that same close. Trade the next bar's open instead, or model the fill realistically.
  • Using an indicator that repaints — e.g. a value that changes after the bar completes.
  • Splitting or dividend adjustments applied across the whole series so past prices already “know” a future corporate action. See Corporate actions and Adjusted prices.
  • Normalising or scaling features using statistics computed over the full dataset, leaking future distribution into the training window.

Part 2 — Validation steps

Validation cannot prove an edge exists; it can only fail to reject it. Run these in increasing order of severity, and stop developing the moment a step reveals the edge was fragile.

StepQuestion it answersHow to run it
Out-of-sample testDoes the edge hold on data the model was never fitted to?Hold back a contiguous block of history, develop only on the rest, then test once. See Out-of-sample testing.
Walk-forward analysisWould the strategy have adapted through changing regimes?Repeatedly optimise on a training window, test on the next unseen window, then roll forward. See Walk-forward testing.
Monte CarloHow lucky or unlucky was the single realised equity path?Resample or reshuffle trade sequences to build a distribution of outcomes and drawdowns. See Monte Carlo simulation.
Forward / paper testDoes it work on genuinely live, unseen data with real latency?Run the finished system in paper-trading or forward-test mode before committing capital.

The one-shot rule for out-of-sample data

Out-of-sample data protects you only while it stays untouched. Each time you peek, tweak, and re-test, it quietly becomes in-sample. Treat your holdout as a single, final exam — if you fail it, the honest move is to go back to hypothesis generation, not to keep trying variants against the same holdout.

Part 3 — Before you risk capital

  • Confirm the number of independent trades is large enough for the metrics to mean anything.
  • Check parameter sensitivity: small changes in inputs should not collapse performance. A sharp peak is a warning; a broad plateau is reassuring.
  • Re-run with costs doubled. A robust edge degrades gracefully; a fragile one disappears.
  • Verify the strategy's logic matches your execution engine assumptions — order types, timing, and liquidity.
  • Document every parameter, data source, and assumption so the result is reproducible.

Related concepts

Deepen any row via Historical data, Backtesting workflow, and the Performance Metrics Cheat Sheet.

Frequently asked questions

What is the single most damaging backtesting bias?
Look-ahead bias is often the most dangerous because it can be subtle and produces spectacular, entirely fake results. Using any information not available at the moment of the decision — a same-bar close, a restated fundamental, a full-sample normalisation — lets the strategy 'see the future', and the equity curve looks flawless until it trades live.
How is survivorship bias avoided?
Test on a point-in-time universe that includes instruments which were later delisted, merged, or went bankrupt. If your data vendor only supplies currently-listed symbols, your backtest silently excludes the losers and overstates returns. Reconstructing the historical index membership is the correct fix.
What is the difference between out-of-sample and walk-forward testing?
Out-of-sample testing holds back one block of data and tests a fixed, already-chosen model on it once. Walk-forward analysis repeatedly re-optimises on a rolling training window and tests on the next unseen window, simulating how you would actually re-tune the system over time. Walk-forward is harsher and more realistic.
Why run a Monte Carlo simulation on a backtest?
A backtest produces one realised sequence of trades and therefore one drawdown. By resampling or reshuffling the trades, Monte Carlo builds a distribution of possible equity paths and drawdowns, revealing how much of the result was luck and what a plausible worst case looks like. The realised path is usually kinder than the median simulated one.
How do costs affect a backtest?
Frictions — brokerage, STT, exchange fees, GST, stamp duty, and slippage — subtract from every trade. High-frequency or thin-margin strategies are especially sensitive: a system with a small per-trade edge can be net-negative once realistic costs are applied. Always model costs per leg and stress-test by doubling them.
What is data-snooping and how do I control it?
Data-snooping is running so many strategy variations on the same data that one succeeds by chance. Control it by limiting the number of trials, keeping a genuinely untouched holdout, adjusting significance for the number of tests performed, and preferring an idea motivated by an economic rationale over one found purely by search.
Does passing every validation step prove a strategy works?
No. Validation reduces the probability of self-deception but never removes it. Markets change, and a strategy that passed out-of-sample, walk-forward, Monte Carlo, and forward testing can still fail live. The steps establish that the edge is not obviously fake, not that future profit is assured.
Is paper trading a substitute for a proper backtest?
No — they answer different questions. A backtest evaluates many years of history quickly but on modelled fills. Paper (forward) trading runs on genuinely unseen live data with real latency and data feed quirks, but slowly. Use the backtest to develop and the forward test to confirm before risking capital.
How many trades do I need for the results to be meaningful?
There is no fixed number, but metrics computed on a few dozen trades are dominated by noise. More independent trades tighten the estimates of win rate, expectancy, and Sharpe. Be especially sceptical of a high Sharpe or profit factor built on a small sample — it is usually overfitting.
What is a parameter plateau and why does it matter?
A plateau is a broad region of parameter values that all perform similarly well. It suggests the edge is robust and not tuned to one lucky setting. A sharp, isolated peak — where a tiny change in an input collapses performance — is a classic sign of curve-fitting and is unlikely to persist live.

Last reviewed 11 July 2026. Educational content only — not investment advice.

Educational content only — not investment advice. See our Risk Disclosure and Methodology.