Backtesting

The CFTC will publish the Commitments of Traders report twice per week from the end of 2026. The series keeps its name, but the day it describes, the delay before it's public, and the number of rows in a year all change. A backtest built on the weekly report has been testing that structure as well as the rule.

A backtest that runs back to 1926 only helps if what it measures existed for the whole period. We tested which edges lasted through changes in the market and which didn't, using a century of daily US returns, 16 years of ES intraday data, the closing-auction seconds that moved in 2018 and 2019, and the fee that widened ETF spreads in April 2026.

Many aspiring traders are told that using daily bars is good enough for all but short-term intraday strategies. We tested the guess many backtesting engines make about the path inside a daily bar against 16 years of one-second data in six futures markets. It gets the order of the high and low right four days in five, as it would on a random walk, but for a stop and a target near the entry it's close to a coin toss, and it flatters the tight-stop, wide-target brackets an optimiser is drawn to.

1. Almgren, R., & Chriss, N. (1999). Optimal execution of portfolio transactions. Journal of Risk, 3(2), 5--39.

The current state of retail backtesting practice is characterised by a significant gap between the apparent sophistication of the tools and the actual rigour of the analysis they produce.

Based on the foregoing analysis, I propose the following minimum standards for a backtest to be considered credible evidence of a potentially viable trading strategy.

Novice and retail trading system developers suffer from what might be termed the signal isolation blindspot: an overwhelming focus on a single dimension of the problem, generating a profitable entry…

Off-the-shelf backtesting frameworks are not neutral tools.

Python's dominance in retail algorithmic trading is a sociological phenomenon, not the result of a deliberate technical evaluation.

Traders who are aware of the overfitting problem described in the preceding section often turn to a family of validation techniques (Monte Carlo simulation, synthetic data generation, and…

Harvey, Liu, and Zhu (2016) demonstrated that the threshold for statistical significance in backtested strategies must be adjusted for the number of strategies tested.

Every OHLC bar, regardless of its timeframe, is a lossy compression of the true price path. A bar records the open, the high, the low, and the close, but nothing about the trajectory between them.

The most fundamental question in any backtest is deceptively simple: would this trade have been filled, and at what price?

A backtest is only as reliable as the data on which it is built, and the quality of freely available and even commercially distributed historical market data is far worse than most traders realise.

The barriers to entry for quantitative trading research have fallen dramatically over the past decade.