AdvancedMultiple choice
The best of a hundred backtests · Part 3 of 3
A researcher tries 100 variants of a signal on five years of daily data. In truth every variant has a Sharpe ratio of zero. The best variant shows an annualised in-sample Sharpe ratio of 1.3.
What should you expect from the chosen variant out of sample, and how should its 1.3 be judged?
- AExpect about 1.3, since the in-sample Sharpe is unbiased for each variant
- BAbout zero; judge 1.3 against a best-of-100 null
- CExpect about 0.65, because the standard haircut is to halve a backtest Sharpe
- DRerun the 100 variants on the same data with new random seeds to confirm
The worked solution is in Premium
The answer, the full working and the one idea to take away – for this and all 1,322 questions in the bank. Answer it in practice and your working is marked, with a known mistake named when you make one.
More alpha and signal research questions
- You backtest 20 independent strategies with no true skill over one year, where…Applied
- A researcher checks each new idea against the same held-out period and keeps…Applied
- A colleague shows you a backtest with a Sharpe ratio of 2.1 over three years.Advanced
- What does the probability of backtest overfitting measure?Expert
- You winsorise a standardised cross-section at ±3.Foundation
- A stock has a beta of 1.2, the risk-free rate is 3% and the equity risk premium is 5%.Foundation