What does out-of-sample mean in backtesting?

Out-of-sample (OOS) data is history the strategy was never tuned on. A common split is chronological 70/30: design and optimize on the first 70% (in-sample), then run the frozen strategy once on the final 30%. The OOS result is the honest one — the in-sample number is what the strategy was shaped to produce.

The discipline is in the "once": if you peek at the OOS result, adjust the strategy, and test again, the out-of-sample data quietly becomes in-sample and its evidential value is gone. Count your shots.