Draft The Backtesting Illusion: how p-hacking and selection bias inflate published Sharpe ratios
Abstract
Published Sharpe ratios in systematic-trading research are systematically inflated. We identify three distinct sources of inflation — selection-on-significance, parameter-grid search, and walk-forward design contamination — and show through replication of three widely-cited studies that the corrected estimates fall by an average of 0.6 in Sharpe terms.
The paper proposes a deflation methodology applicable to existing published results, requiring only the original parameter grid and the number of variants tested.
Keywords
backtesting, selection bias, p-hacking, Sharpe ratio, walk-forward analysis, multiple testing
Download
The PDF is available here.