Generalised models: logistic, Poisson and quantile regression

Linear regression models a conditional mean on an unbounded scale. When the outcome is a probability, a count or a tail, that is the wrong target — and each generalised model is the same linear predictor pushed through a function that makes the range come out right.

Equation 8.1

Logistic regression

Model the log-odds linearly, which maps the whole real line onto $(0,1)$ so the fitted probability is always valid.

\ln\frac{p}{1-p} = \beta_0 + \beta^\top x \quad \Longleftrightarrow \quad p = \frac{1}{1 + e^{-(\beta_0 + \beta^\top x)}}

$p/(1-p)$: The odds. A coefficient of $\beta$ multiplies the odds by $e^\beta$ per unit.
$\beta_0$: Log-odds at $x = 0$ , which is rarely a meaningful point unless you centre.

Figure 8.2 · The link, and where the slope lives. The coefficient is constant in log-odds and anything but constant in probability: the same one-unit move is worth

25

percentage points at the middle and almost nothing in the tails. Quoting a logistic coefficient as "an effect on probability" without saying where is the standard error.

Proposition 8.3

Reading a logistic coefficient

A coefficient of $0.7$ means the odds multiply by $e^{0.7} \approx 2$ per unit increase. Near a probability of $0.5$ the effect on the *probability* is about $\beta/4$ per unit — the derivative of the logistic at its midpoint is one quarter — which is a fast way to translate a coefficient into something interpretable.

Holds when

The divide-by-four rule is an upper bound: the effect on probability is smaller away from $0.5$ .
Odds ratios are not probability ratios, and conflating them overstates effects on common outcomes.
No $R^2$ exists; use log-likelihood, AUC or a calibration plot instead.

Example 8.4

A logistic model of whether a trade is profitable gives a coefficient of $0.4$ on a signal. The base rate is $50\%$ . What does a one-unit increase do?

Show the worked solutionHide the worked solution

Worked solution

Formula $\text{odds} \times e^{\beta}, \qquad \Delta p \approx \beta/4 \text{ near } p = 0.5$
Substitute $e^{0.4} = 1.49$
Solve $\text{odds: } 1.0 \to 1.49 \Rightarrow p = \frac{1.49}{2.49} = 0.598$
$\text{rule of thumb: } 0.5 + 0.4/4 = 0.60$
Answer $\text{about } 60\%, \text{ up from } 50\%$

Sanity check. The divide-by-four approximation gives $0.60$ against an exact $0.598$ — close enough to quote, and it fails gracefully by overstating as you move away from the midpoint.

Generalised models: logistic, Poisson and quantile regression

After this lesson you should be able to

Logistic regression

Reading a logistic coefficient

The rest of this lesson is in Premium