PBO, introduced by Bailey, Borwein, López de Prado and Zhu, treats strategy selection itself as the thing to validate. It repeatedly splits the data, picks the configuration that looks best in-sample, then checks how that same configuration ranks out-of-sample. If the in-sample winner routinely lands in the bottom half out-of-sample, your selection process is overfitting.
The output is a probability in [0, 1]. Low PBO means the configuration that wins in testing tends to keep winning on unseen data; high PBO (above ~0.5) means your 'best' settings are likely a product of luck and trying many combinations. It is the natural companion to walk-forward testing.
Beispiel
After scanning 200 parameter sets, the top set shows a PBO of 0.6 — more likely than not curve-fit — so it is rejected despite a great in-sample Sharpe.
Wie Noon Barbari Probability of Backtest Overfitting (PBO) nutzt
Jedes Konzept hier ist in der Plattform umgesetzt. Öffne die entsprechenden Docs oder das Tool, um es in Aktion zu sehen.
Validate a strategy in noonbarbari →Verwandte Begriffe
- Backtesting
Overfitting
Fitting a strategy so closely to past data that it captures noise, not signal.
- Backtesting
Walk-forward optimization
Optimize on a rolling in-sample window, validate on the next out-of-sample slice.
- Backtesting
Out-of-sample
Data the strategy was not allowed to see during parameter selection.
- Statistik
Sharpe ratio
Excess return over the risk-free rate per unit of total volatility.