Goodness of fit: R squared, F tests and information criteria
REG · Chapter 311 min readAsked at Two Sigma, Citadel, AQR, QuantCo
Assumes Gauss–Markov: what each assumption buys and what breaks it.
1:05The lesson in a minutePremium
The key idea, drawn and talked through in a short video before you read. Lesson videos come with Premium.
Start the 7-day free trialUnlockAfter this lesson you should be able to
- Interpret and say why it always rises.
- Use an F test to compare nested models.
- Choose between AIC and BIC, and say what each is targeting.
Fit statistics answer "how much of the variation did I explain?" and are routinely over-read. In return prediction an of one per cent is a strong result, so the absolute number carries almost no information – what matters is comparing models on the same data, honestly penalised for complexity.
Equation 3.1
R squared and its adjustment
The share of variance explained, and the version that charges for parameters.
- Residual sum of squares – what the model failed to explain.
- Number of regressors. Adjusted can fall, and can even go negative.
Proposition 3.2
Why never falls
Adding a regressor enlarges the space you are projecting onto, and the projection onto a larger space is at least as close to the target. In the worst case the new coefficient comes out zero and nothing changes; in any finite sample it will be slightly non-zero by chance, so rises strictly. That makes useless for comparing models of different sizes.
Holds when
- In a simple regression ; in a multiple regression it is the squared correlation between and .
- It is not comparable across different dependent variables – regressing returns and regressing prices produce incomparable numbers.
Why a one per cent is good news. Returns are mostly unforecastable, so nearly all their variance is noise you were never going to explain. A daily cross-sectional signal with an information coefficient of 0.10 – very strong – has an of one per cent, and that is enough to build a business on because you apply it across thousands of positions and thousands of days. Reading as a quality score imported from a physics or marketing context is one of the fastest ways to sound unfamiliar with financial data.
Equation 3.3
The F test for nested models
Tests whether the extra regressors in the unrestricted model add anything jointly.
- Number of restrictions being tested.
- Restricted and unrestricted residual sums of squares.
The rest of this lesson is in Premium
You have read the opening. 10 more sections follow, including 4 worked examples and 3 quick checks.
Nothing is charged for 7 days, and you can cancel before then. Not ready for a trial? A free account, with no card, saves your progress and opens the free questions, the drills and a mock paper. Or read The law of large numbers and the central limit theorem in full, free.