Home / Backtesting / Historical Data
DataIntermediate

Historical Data

Historical data is the record of past prices, volumes and related information used to backtest strategies, and its granularity, cleanliness, adjustment and point-in-time correctness set the hard ceiling on how trustworthy any backtest can be.

Quick answer: Historical data is the record of past prices, volumes and related information used to backtest strategies, and its granularity, cleanliness, adjustment and point-in-time correctness set the hard ceiling on how trustworthy any backtest can be.

In simple words

Your backtest can only be as honest as the data underneath it. If the prices are wrong, unadjusted for splits, missing the stocks that later got delisted, or secretly contain information no one had at the time, then every result on top is corrupted. Getting the data right is unglamorous and it is where most of the real work lives.

Purpose

This page exists because data problems are the silent, invisible cause of most misleading backtests, and understanding data quality is a prerequisite for trusting any result.

Visual explanation

Historical Data

Raw feeds flowing through cleaning, corporate-action adjustment and point-in-time alignment into a backtest-ready series.

Data PipelineSource / FeedIngestNormaliseCorp-actionAdjustValidateStoreServe

Professional explanation

Granularity: from daily bars to ticks

Data comes at different resolutions, and the right one depends on the strategy. End-of-day OHLC bars suit positional and swing systems. Minute bars suit intraday strategies. Tick data, every trade and quote update, is required for microstructure and execution research but is enormous and hard to store and process. Higher granularity is not automatically better: it is larger, noisier, more expensive, and can tempt you into strategies whose apparent edge lives entirely inside the bid-ask spread and vanishes after realistic costs.

Sources and their trade-offs

Historical data comes from exchange feeds, broker APIs, and third-party vendors. Exchange data is authoritative but can be costly and comes with usage terms; broker APIs are convenient but often provide limited history and may have gaps or vendor-specific adjustments; third-party vendors vary widely in quality and in how they handle corporate actions. No source is automatically clean. You must know exactly how your source timestamps bars, whether prices are adjusted, and how it treats missing sessions and auctions.

Corporate-action adjustment

Splits, bonuses, dividends, rights issues and mergers change the raw price without changing economic value. A 1-for-1 bonus halves the quoted price overnight, and an unadjusted series will show that as a catastrophic loss the strategy never actually suffered. Back-adjusted series correct historical prices for these events so returns are continuous, but adjustment introduces its own subtlety: adjusted absolute price levels are not the prices that traded historically, which matters if your rules reference specific price levels. You must know which convention your data uses and align your logic to it.

Point-in-time correctness

A point-in-time dataset stores what was known at each moment, not what we know now. This matters most for fundamentals: earnings are reported with a lag and often revised later, so a naive database that stamps the final restated figure on the original date lets your strategy 'know' numbers that were not yet public, a classic look-ahead leak. It also matters for index membership and delistings. Point-in-time data is harder to obtain and more expensive, but without it, fundamental and event-driven backtests are systematically inflated.

Quality problems to hunt for

Real data is dirty. Watch for bad ticks and spikes, zero or negative prices, duplicate bars, timezone and daylight errors, missing sessions, mislabelled adjusted versus unadjusted prices, and volume that does not reconcile. Each of these can create fake trades or fake profits. A rigorous data-validation step, range checks, gap detection, cross-source reconciliation, is not optional; it is where you catch the errors that would otherwise masquerade as edge.

Indian (NSE) data realities

For Indian markets, specific issues recur. F&O contracts expire, so building a continuous futures series requires an explicit roll convention. Lot sizes change over time, so a fixed lot assumption misprices older trades. Circuit limits, trading halts and the pre-open auction create sessions that behave differently from continuous trading. Nifty and Bank Nifty index composition changes as stocks are added and removed, so a survivorship-free study must reflect historical membership. Corporate actions on individual NSE stocks are frequent and must be adjusted consistently.

Storage, versioning and reproducibility

Because a backtest must be reproducible, the data itself needs versioning. If a vendor silently revises history, or you re-download and get a different series, your results change without any code change. Serious research pins a dated snapshot of the data, records its provenance and adjustment convention, and treats the dataset as a versioned artefact alongside the code. Column-oriented formats and careful timestamp handling make large minute and tick datasets tractable.

Practical example

Illustrative example (Indian market)

You backtest a Nifty stock rotation using ten years of daily data downloaded free from a broker API. It shows strong results, but two problems hide inside the data. First, the universe contains only stocks listed today, so companies that were delisted or merged, often after poor performance, are simply absent, and your winners are overrepresented. Second, a stock that had a 1-for-2 bonus shows an unadjusted 33 percent overnight drop your strategy 'bought into'. After you switch to a survivorship-free, corporate-action-adjusted, point-in-time dataset, the annualised return falls by several percentage points, which is the more honest figure.

NSE changed the Bank Nifty and Nifty lot sizes several times over the past decade. A futures backtest that assumes today's lot size throughout will misstate position value and therefore risk and returns for older years, so the lot size must be looked up as of each historical date rather than hardcoded.

Advantages

  • High-quality data is the foundation that every reliable backtest depends on
  • Point-in-time data eliminates a whole class of look-ahead leaks at the source
  • Corporate-action adjustment makes returns continuous and comparable across time
  • Versioned data snapshots make results reproducible and auditable

Limitations

  • Truly clean, point-in-time, survivorship-free data is expensive and sometimes unavailable
  • Free broker or vendor data often has gaps, adjustment errors and short history
  • Tick data is huge and demands serious storage and processing infrastructure
  • Adjusted prices are not the prices that historically traded, which can break level-based rules

Why it matters in practice

  • Data quality sets the maximum trustworthiness of any downstream result
  • Most surprising, too-good backtests trace back to a data defect rather than a real edge

Common mistakes

  • Using a universe of only currently listed stocks, which bakes in survivorship bias
  • Backtesting on unadjusted prices, so splits and bonuses appear as huge fake losses
  • Stamping restated fundamentals on their original date, leaking future information
  • Assuming a single fixed F&O lot size across years when it actually changed
  • Skipping data validation, so bad ticks and duplicate bars create phantom trades
  • Not recording which adjustment convention the data uses, then writing rules that assume the other

Professional usage

Institutional desks treat data as a first-class, versioned asset with a dedicated pipeline: authoritative sources, rigorous cleaning, documented corporate-action handling, point-in-time storage, and reconciliation across vendors. They snapshot and version datasets so a backtest run today and re-run next year returns identical results, and they invest heavily in the unglamorous validation layer because they know that most spurious edges are data artefacts, not genuine signals.

Key takeaways

  • Backtest trustworthiness is capped by data quality, adjustment and point-in-time correctness
  • Match granularity to the strategy; more resolution is not automatically better
  • For India, handle F&O rolls, changing lot sizes, index membership and frequent corporate actions
  • Version and snapshot your data so results stay reproducible

Frequently asked questions

What historical data do I need to backtest a strategy?
You need price and volume data at a granularity matching your strategy, daily bars for positional, minute bars for intraday, ticks for execution research, that is clean, corporate-action adjusted, survivorship-free and point-in-time correct. The lowest-quality attribute among these sets the ceiling on how much you can trust the backtest.
What is point-in-time data?
Point-in-time data stores what was actually known at each historical moment, including as-reported (not later restated) fundamentals and the universe of names then listed. It prevents your strategy from using information that was not yet public, which is a common and subtle source of look-ahead bias.
Why do I need corporate-action-adjusted prices?
Splits, bonuses, dividends and rights change the raw quoted price without changing value. Without adjustment, a bonus issue looks like a huge overnight loss your strategy never suffered. Back-adjusted series make returns continuous, though the adjusted price levels no longer equal the prices that historically traded.
Where can I get NSE historical data?
Sources include exchange data products, broker APIs, and third-party vendors, each with trade-offs in cost, history length, and how they handle adjustments and gaps. Free broker data is convenient but often has limited history and adjustment quirks, so you should verify its timestamp and adjustment conventions before relying on it.
What granularity should I use?
Match it to the holding period: end-of-day bars for swing and positional strategies, minute bars for intraday, and tick data only if you are studying microstructure or execution. Finer data is larger, noisier and can tempt you toward apparent edges that live inside the spread and vanish after costs.
How does survivorship bias relate to data?
It arises when your dataset only contains instruments that survived to today, silently dropping delisted, merged or bankrupt names that were usually poor performers. This inflates backtested returns. The fix is a point-in-time universe that includes every name as it existed historically.
What data problems should I check for?
Look for bad ticks and spikes, zero or negative prices, duplicate or missing bars, timezone and daylight-saving errors, mislabelled adjusted versus unadjusted prices, and volumes that do not reconcile. A validation step with range checks, gap detection and cross-source reconciliation catches most of these before they create phantom trades.
How do I handle F&O expiry in historical data?
Build a continuous futures series using an explicit roll rule, for example rolling a set number of days before expiry or on a volume/open-interest trigger, and adjust for the price gap at each roll. Without this, expiry creates phantom gains or losses that no real trader could have captured.
Do adjusted prices cause any problems?
Yes. Because back-adjusted historical prices are shifted to make returns continuous, the absolute price levels are not the levels that actually traded. Any rule that references a specific price, a round number, or an absolute stop distance can behave differently on adjusted versus raw data, so you must know which series your logic assumes.
Why version my historical data?
Because vendors revise history and re-downloads can differ, a backtest can change with no code change if the data shifts underneath it. Pinning a dated snapshot with recorded provenance and adjustment convention makes results reproducible and lets you audit exactly what produced a given outcome.
Is free historical data good enough for backtesting?
It can be adequate for learning and rough screening, but free data often has short history, gaps, adjustment errors and no point-in-time fundamentals or survivorship correction. For results you intend to trade on, you should verify quality carefully or use a reputable paid source.
How much history is enough for a backtest?
Enough to span multiple market regimes, including at least one stress period, and enough trades that results are not driven by a few lucky ones. A strategy validated only across a single calm bull phase reveals little about how it behaves in a crash or a prolonged range.

Voice search & related questions

Natural-language questions people ask about Historical Data.

Why is data quality so important in backtesting?
Because your backtest can only be as honest as the data under it. Wrong, unadjusted or survivor-only data will corrupt every result no matter how good your code is.
What does point-in-time data mean?
It means the data only contains what was actually known at each past moment, so your strategy cannot accidentally use future information like a later-restated earnings figure.
Do I need to adjust for stock splits and bonuses?
Yes. Without adjustment a split or bonus looks like a giant overnight price drop that your strategy never really experienced, which ruins the results.
Where do I get Indian market data for backtesting?
From exchange data products, broker APIs, or third-party vendors. Just check how each one handles adjustments, gaps and history length before you trust it.
Is free stock data good enough to backtest?
For learning, often yes. For strategies you plan to trade with real money, free data's gaps, short history and adjustment errors make it risky, so verify it carefully.
What granularity of data should I use?
Match it to your holding period: daily bars for swing trades, minute bars for intraday, and tick data only for microstructure work. Finer data is not automatically better.

Sources & references

    Last reviewed 11 July 2026. Educational content only — not investment advice. Markets and rules change; verify current conventions with SEBI, NSE/BSE and your broker.

    ← Previous
    The Backtesting Workflow
    Educational content only — not investment advice. Examples use illustrative numbers and simplified models. Algorithmic trading and derivatives involve substantial risk. See our Risk Disclosure and SEBI Disclaimer.