LibraryCalculators

Backtest overfitting check

A backtest can look good for two reasons that have nothing to do with skill. It may have seen data it could not have had at the time, which is a leak. Or it may be the best of many tries, which is luck. This check asks about both, and computes everything from the numbers you type in.

The Sharpe ratio of the version you kept.

Count every rule, setting, asset list and model you tested on this data, including the ones you threw away.

0 if you do not know it.

3 if you do not know it.

Optional. Leave it empty to assume the versions had no skill, so their spread is sampling noise, 1/√years.

Every number is computed on this page from your inputs. Nothing is sent to an AI model and no market data is used.

Education, not advice. Inputs are your own numbers; no market data is used.

Result

Not yet distinguishable from luck. The deflated Sharpe ratio is 90.17%, under the 95% bar: the best of 50 skill-less tries could plausibly look this good.

MeasureValue
Deflated Sharpe ratio (DSR)90.17%
Best Sharpe that luck alone would give (SR0, annualised)1.1382
Probabilistic Sharpe ratio (PSR, against 0)99.98%
History needed for this many tries (minimum backtest length)1.6 years
Return observations used1008
Spread of trial Sharpe ratios (assumed: 1/√years)0.5000

What to check

What this computes

The luck half uses the deflated Sharpe ratio of Bailey and López de Prado (2014). It asks how likely your Sharpe ratio beats the best that the same number of skill-less tries would show over the same history. It also gives the minimum backtest length of Bailey, Borwein, López de Prado and Zhu (2014): how many years of data that many tries need before a Sharpe ratio this high stops being expected from luck alone.

The leak half is seven yes or no questions. Each one is a mistake that made a strategy look far better than it was, in our own runs on four markets or in published research. A leak changes the Sharpe ratio itself, so when any leak answer is yes or not sure, the verdict says to fix that first. No AI model reads your answers: the verdict follows fixed rules shown on this page.

How to read the result

Read the verdict first, then the flags under it. A flag names the mistake and links to the fix.

The deflated Sharpe ratio is a probability. Near 100% means the result is hard to explain as the luckiest of your tries. Near 50% or lower means the best of that many coin flips would look about as good.

Compare the minimum backtest length with the years you have. If it needs more history than you have, then with this many tries a Sharpe ratio this high is what luck alone is expected to produce.

Sanity check: set the versions tried to 1. Then there was no search, the luck benchmark is 0, and the deflated Sharpe ratio equals the probabilistic Sharpe ratio. Raise the versions and it falls.

The formula

SR0=V[(1−γ) Φ−1 ⁣(1−1N)+γ Φ−1 ⁣(1−1Ne)]SR_0 = \sqrt{V}\left[(1-\gamma)\,\Phi^{-1}\!\left(1-\tfrac{1}{N}\right) + \gamma\,\Phi^{-1}\!\left(1-\tfrac{1}{N e}\right)\right]

The best Sharpe ratio that N tries with no skill are expected to show: the spread of their Sharpe ratios times how far the best of N lands above the middle.

DSR=Φ ⁣((SR−SR0)T−11−γ3 SR+γ4−14 SR2)DSR = \Phi\!\left(\frac{(SR - SR_0)\sqrt{T-1}}{\sqrt{1 - \gamma_3\,SR + \frac{\gamma_4 - 1}{4}\,SR^2}}\right)

The probability that the true Sharpe ratio is above that luck benchmark, given T observations and the skew and kurtosis of the returns.

MinBTL≈((1−γ) Φ−1 ⁣(1−1N)+γ Φ−1 ⁣(1−1Ne)SR)2\text{MinBTL} \approx \left(\frac{(1-\gamma)\,\Phi^{-1}\!\left(1-\tfrac{1}{N}\right) + \gamma\,\Phi^{-1}\!\left(1-\tfrac{1}{N e}\right)}{SR}\right)^{2}

The years of history needed before the best of N skill-less tries is no longer expected to reach your annualised Sharpe ratio.

SR is your Sharpe ratio, N the number of versions tried, T the number of return observations (years times periods per year). In the first two lines SR and the spread √V are per period: the page divides your annualised figures by the square root of the periods per year.

γ is the Euler-Mascheroni constant, about 0.5772, and e the base of the natural logarithm. Φ is the standard normal distribution function and Φ⁻¹ its inverse. γ₃ is the skewness and γ₄ the raw kurtosis of returns, 3 for a normal distribution.

When you leave the spread empty, the page assumes the versions had no skill. A skill-less version's annualised Sharpe ratio over y years then varies by about 1/√y from noise alone, and that is the spread used. The minimum backtest length rests on the same assumption, with SR annualised.

The verdict rule: any leak answer of yes or not sure gives "fix the leak first". Otherwise a deflated Sharpe ratio of 95% or more gives "passes this check", and anything lower gives "not yet distinguishable from luck". The cost and survivorship questions raise a flag but do not set the verdict.

Limits

Questions

Can AI tell me if my backtest is overfit?

Not by reading it. A chatbot cannot see how many versions you tried or what your code did on each date unless it runs the code. The deflated Sharpe ratio turns the number of tries into a probability, and a truncation test, described on our leakage page, shows whether code uses future data.

How many strategy versions can I test on five years of data?

Bailey, Borwein, López de Prado and Zhu (2014) give an example: with 5 years of data, no more than 45 independent configurations should be tried, or a strategy with an in-sample annualised Sharpe ratio of 1 is expected from luck alone. Type a Sharpe ratio of 1 and 45 versions into this page to see the same figure.

What counts as a version?

Anything you would have kept if it had looked better: a new rule, a different moving-average length, another asset list, another model, a new cost assumption. If a choice was made after seeing a backtest, it was a version.

Why does a leak override a good deflated Sharpe ratio?

The deflated Sharpe ratio corrects a real Sharpe ratio for luck. A leak makes the Sharpe ratio itself wrong, because the strategy used information it could not have had. Correcting a wrong number for luck still leaves a wrong number.

Related

Also in the library