Machine learning
Cross-validating a return forecaster, part 3 of 3
AdvancedMultiple choice
Cross-validating a return forecaster · Part 3 of 3
You train a model on ten years of daily data. Each feature uses a trailing 20-day window, and each label is the stock’s return over the next 5 days. You plan to choose hyperparameters by cross-validation.
With purged folds in place, you try 200 hyperparameter settings and report the best cross-validated Sharpe ratio. What is wrong with that number?
- ANothing, because purging already removed the leakage
- BIt is biased upward by the search, so it needs a held-out check
- CIt is biased downward, since each fold trains on less data
- DIt is fine as long as the Sharpe ratio is averaged across the folds
The worked solution is in Premium
The answer, the full working and the one idea to take away – for this and all 1,322 questions in the bank. Answer it in practice and your working is marked, with a known mistake named when you make one.
Report an issue with this question
Learn the method
More machine learning questions
- Why is ordinary shuffled k-fold cross-validation wrong on a financial time series?Applied · Free solution
- What does embargoing do that purging does not?Advanced · Free solution
- A classifier flags 500 cases, of which 40 are true positives, and misses 60 positives.Applied · Premium
- An anomaly detector has sensitivity 90% and specificity 95%.Applied · Premium
- You apply a strictly increasing transformation to a classifier’s scores.Applied · Premium
- A classifier has 30 true positives, 20 false positives and 10 false negatives.Foundation · Premium