STAT

Statistics and inference

Estimation, intervals and tests — and the multiple-testing problem that quietly decides whether a backtest means anything.

1
Limit theorems
LLN, CLT and the delta method, plus the cases where they fail.
- 1.1The law of large numbers and the central limit theoremFree12 min
2
Estimation
Bias, variance, MSE, consistency, MLE and method of moments.
- 2.1Estimation: bias, variance and maximum likelihood12 min
3
Confidence intervals
Construction, the standard misinterpretation, and the z-scores you need cold.
- 3.1Confidence intervals, and the interval you would trade11 min
4
Hypothesis testing
Errors, power, and the z, t, chi-square and F tests.
- 4.1Hypothesis testing: errors, power and which test to use12 min
5
p-values and multiple testing
p-hacking, Bonferroni, FDR — the central problem in signal research.
- 5.1p-values, p-hacking and the multiple-testing problem13 min
6
Resampling
Bootstrap, jackknife, permutation tests and block methods.
- 6.1Resampling: the bootstrap, permutation tests and where they break12 min
7
Bayesian statistics
Conjugacy, credible intervals and Bayesian A/B testing.
- 7.1Bayesian statistics: conjugacy, shrinkage and credible intervals12 min
8
Experiment design
A/B tests, sequential testing and minimum detectable effect.
- 8.1Experiment design: randomisation, peeking and minimum detectable effect12 min