Stats › Backtesting and evidence › Backtest overfitting
After a thousand backtests the best Sharpe ratio is 3.26 even when nothing works
| What was fitted | In-sample Sharpe | Negative out of sample | Overfitting probability |
|---|---|---|---|
| A seasonal rule on a pure random walk | 1.27 | About 53% | 55% |
| A random walk with a real seeded monthly effect | 1.54 | About 13% | 13% |
| An illustrative strategy set, all in-sample Sharpes between 1 and 3 | 1 to 3 | About 78% | 74% |
| A real investment strategy | Not stated | About 3% chance of loss | About 4% |
Measured probability of backtest overfitting, defined as the chance that the configuration which looks best in sample will rank below the median out of sample. The first two rows are constructed examples and the difference between them is the whole lesson: a 1.27 Sharpe on noise and a 1.54 Sharpe on a real effect look almost identical from the in-sample side.
After a thousand backtests the expected best Sharpe ratio is 3.26 even when every strategy has a true Sharpe of zero. On a seasonal rule fitted to a pure random walk, the probability of backtest overfitting measures 55% and 53% of out of sample Sharpe ratios are negative.
Members only
The rest of this page is for members
Below this point there are 5 sections and 4 named sources, roughly 800 words of it. Every figure carries the source it came from and the date the data is from.
You can keep browsing every statistic in the library for free. The intro and the summary are always open.