Statistics and inference formulas

The law of large numbers and the central limit theorem

The central limit theorem

\sqrt{n}\,\frac{\bar{X}_n - \mu}{\sigma} \ \xrightarrow{d}\ \mathcal{N}(0, 1)

The standardised sample mean converges in distribution to a standard normal, whatever the shape of the underlying distribution — provided the variance is finite.

Chebyshev’s inequality

\Pr\big(|X - \mu| \ge k\sigma\big) \le \frac{1}{k^2}

Holds for any distribution with a finite variance — no normality required. It is loose, but it is the guarantee you fall back on when you cannot trust the tails, and it is what proves the weak law of large numbers in one line.

The standard error of a Sharpe ratio

\operatorname{SE}(\widehat{SR}) \approx \sqrt{\frac{1 + \tfrac{1}{2} SR^2}{n}}

For independent, normal returns, measured per period (Lo, 2002). Annualise both the ratio and its error by $\sqrt{\text{periods per year}}$ . Fat tails and autocorrelation both make the true error larger.

Remember

LLN: the sample mean converges to the true mean, given a finite mean.

Estimation: bias, variance and maximum likelihood

The decomposition

\mathrm{MSE}(\hat\theta) = \mathbb{E}\big[(\hat\theta - \theta)^2\big] = \mathrm{Bias}(\hat\theta)^2 + \mathrm{Var}(\hat\theta)

Squared error splits cleanly into a systematic part and a noisy part. Minimising the total is what you actually want; unbiasedness constrains only the first term.

The Cramér–Rao bound

\operatorname{Var}(\hat\theta) \ge \frac{1}{n\,I(\theta)}, \qquad I(\theta) = \mathbb{E}\!\left[\left(\frac{\partial}{\partial\theta}\ln f(X;\theta)\right)^2\right]

No unbiased estimator can beat this variance. The MLE achieves it asymptotically, which is why its standard error is read off the curvature of the log-likelihood — flat likelihoods mean little information and wide errors.

Remember

$\mathrm{MSE} = \mathrm{Bias}^2 + \mathrm{Var}$ ; you want the total, not zero bias.

Confidence intervals, and the interval you would trade

An interval for a mean

\bar{x} \pm z_{\alpha/2}\,\frac{s}{\sqrt{n}}

Point estimate, plus or minus a multiple of the standard error. Use $t$ rather than $z$ when $n$ is small and the variance is estimated.

An interval for a variance

\left[\frac{(n-1)s^2}{\chi^2_{0.975,\,n-1}},\ \frac{(n-1)s^2}{\chi^2_{0.025,\,n-1}}\right]

For normal data, $(n-1)s^2/\sigma^2$ has a chi-square distribution with $n - 1$ degrees of freedom. The interval is asymmetric and wide for small samples — and it depends heavily on normality, so for returns a bootstrap interval is usually safer.

Remember

Estimate $\pm z \times$ standard error; $z = 1.96$ for 95%.

Hypothesis testing: errors, power and which test to use

Sample size from power

n \approx \frac{2\left(z_{\alpha/2} + z_{\beta}\right)^2\sigma^2}{\delta^2}

The two-sample size needed to detect a difference $\delta$ with power $1-\beta$ at level $\alpha$ .

Remember

Type I is a false alarm at rate $\alpha$ ; type II is a miss at rate $\beta$ .

p-values, p-hacking and the multiple-testing problem

The family-wise error rate

\mathrm{FWER} = 1 - (1 - \alpha)^m

The probability of at least one false positive across $m$ independent tests at level $\alpha$ .

Holm’s step-down procedure

\text{reject } H_{(1)}, \ldots, H_{(k)} \text{ while } p_{(i)} \le \frac{\alpha}{m - i + 1}

Sort the p-values and compare the smallest with $\alpha/m$ , the next with $\alpha/(m-1)$ , and so on, stopping at the first failure. It controls the family-wise error rate exactly as Bonferroni does, and it is never less powerful — so there is no reason to use plain Bonferroni.

Remember

Write the hypothesis down before running the test.

Resampling: the bootstrap, permutation tests and where they break

Key rules

The bootstrap estimates the variability of an estimate, not the estimate itself.
It works for statistics with no closed form — medians, Sharpe ratios, quantiles.
A permutation test assumes only exchangeability under the null.

Bayesian statistics: conjugacy, shrinkage and credible intervals

The update

p(\theta \mid x) \propto p(x \mid \theta)\,p(\theta)

Posterior is proportional to likelihood times prior. The constant of proportionality is whatever makes it integrate to one, and for conjugate pairs you never need to compute it.

Remember

Posterior $\propto$ likelihood $\times$ prior.

Experiment design: randomisation, peeking and minimum detectable effect

Minimum detectable effect

\mathrm{MDE} = (z_{\alpha/2} + z_\beta)\sqrt{\frac{2\sigma^2}{n}}

Run the power calculation the other way: given the sample you can actually get, what is the smallest effect you could reliably detect?

Remember

Randomisation makes confounders irrelevant in expectation; stratify on the ones you know.

Detailed formula cards