Research case studies
A backtest too good to be true, part 3 of 3
AdvancedMultiple choice
A backtest too good to be true · Part 3 of 3
A colleague’s daily stock-selection model uses company fundamentals and shows a Sharpe ratio of 4 from 2005 to 2020. The universe is today’s S&P 500 constituents, and each fundamental is stamped with its fiscal period end date.
With point-in-time data the Sharpe ratio is 1.2. What test best checks for any remaining leak?
- ADelay the features a day and rerun on shuffled labels
- BAdd more features and check whether the Sharpe ratio rises further
- CCheck that the in-sample R² is small, as a leak would inflate it
- DExtend the backtest back to 1995
The worked solution is in Premium
The answer, the full working and the one idea to take away – for this and all 1,322 questions in the bank. Answer it in practice and your working is marked, with a known mistake named when you make one.
Report an issue with this question
Learn the method
More research case studies questions
- A series has median 10 and median absolute deviation 2.Applied · Premium
- Small companies are much likelier to leave a financial field blank when their…Applied · Premium
- Ten features are each missing in 5% of rows, independently.Applied · Premium
- You build a factor from quarterly fundamentals downloaded from a vendor today.Advanced · Premium
- Your model degrades sharply after a known market structure change.Advanced · Premium
- Using daily closing prices, you find that large-cap returns predict next-day…Advanced · Premium