What is the probability of backtest overfitting (PBO)?

PBO estimates the probability that the strategy configuration you selected because it ranked best in-sample will actually rank below median out-of-sample. It works by repeatedly partitioning the backtest data into training/testing combinations and checking how often the in-sample winner disappoints out-of-sample.

A PBO near 50% means in-sample ranking is a coin flip — your selection process adds nothing. Our own cross-template study measured a median rank correlation of just 0.37 between in-sample and out-of-sample rankings, which is why 'I picked the best backtest' is such a weak strategy-selection method.