Probability

Conditional probability

The fraction of the world in which $B$ happens that also has $A$ in it. Everything else follows from this one definition.

Bayes in odds form

Posterior odds are prior odds times the likelihood ratio. It is the fastest form to use at speed: no denominator to compute, and repeated independent evidence just multiplies more likelihood ratios in.

The law of total probability

The denominator in Bayes, and often the whole question. Split the world into cases that do not overlap and cover everything, compute the probability of the evidence in each, and weight by how likely each case is.

Normal tail figures

And in the other direction, the z-scores for the intervals you are asked to quote:

Sums of exponentials

The waiting time until the $k$-th event of a Poisson process is a sum of $k$ independent exponential gaps. It connects the three: Poisson counts, exponential gaps, gamma waiting times.

The statement

For any random variables $X_1,\dots,X_n$ on the same probability space, and any constants $a_i$:

With constants

Variance by indicators

Linearity gives the mean for free; the variance needs the pairwise covariances, which are usually just as easy because each needs only a joint probability of two events. For the hat-check problem it gives a variance of exactly one, for every $n \ge 2$.

Law of total expectation

The inner expectation is a function of $Y$, so it is itself a random variable. The outer expectation averages it over $Y$.

The form you will actually write

A weighted average of the conditional answers, weighted by how likely each case is.

Law of total variance

Total variability splits into the average spread inside each scenario, plus the spread of the scenario averages.

The biased game

The probability of reaching $N$ before $0$, starting from $k$, when each step is up with probability $p$.

How long it lasts

The expected number of steps before absorption in a fair game — maximised in the middle, and larger than most people guess.

The martingale behind the biased formula

For a walk that steps up with probability $p$, $(q/p)^{S_n}$ has constant expectation: one step multiplies it by $p\,(q/p) + q\,(p/q) = 1$. Optional stopping then gives the biased ruin probability in one line, just as $S_n$ itself does for the fair game.

Turn arrows into a matrix

Rows are today’s state and columns are tomorrow’s, in the order Clear, Storm. Matrix multiplication sums over the possible intermediate states. Every row of a transition matrix sums to one.

The two that factorise

The maximum is below $x$ exactly when every draw is; the minimum is above $x$ exactly when every draw is.

The full distribution of the $k$-th smallest

For $n$ uniforms, the $k$-th smallest has a beta distribution. It gives variances as well as means, and shows that extremes are pinned down much more tightly than the median.

The inverse transform

Feed a uniform through the inverse CDF and you get the distribution you wanted. This is the general answer to "how would you sample from this?", and it is why a uniform generator is all a library needs.

Box–Muller

Two independent uniforms give two independent standard normals. It works because the joint normal density is rotationally symmetric: the angle is uniform and the squared radius is exponential with mean two, and both are easy to generate.

Control variates

If you can simulate a quantity $X$ whose mean you know exactly and which moves with the thing you want, subtract its error. With the optimal coefficient the variance falls by the factor $1 - \rho^2$ — a correlation of $0.9$ cuts it by $81\%$, worth five times as many paths.