Conditional probability and Bayes

Bayes is not a formula to recall — it is what you get by writing the probability of two things happening in the two orders available and setting them equal. Every classic trap in this area comes from ignoring the base rate, and every fix comes from counting the cases.

Equation 1.1

Conditional probability

The fraction of the world in which $B$ happens that also has $A$ in it. Everything else follows from this one definition.

\Pr(A \mid B) = \frac{\Pr(A \cap B)}{\Pr(B)}

$\Pr(A \cap B)$: Both happen.
$\Pr(B)$: The event you are conditioning on; it must have positive probability.

Derivation 1.2

Bayes in two lines

The intersection is symmetric, so expand it both ways and rearrange.

$\Pr(A \mid B)\Pr(B) = \Pr(A \cap B) = \Pr(B \mid A)\Pr(A)$
$\Pr(A \mid B) = \frac{\Pr(B \mid A)\Pr(A)}{\Pr(B)}$
$\Pr(B) = \Pr(B \mid A)\Pr(A) + \Pr(B \mid A^{c})\Pr(A^{c})$
The law of total probability, which is almost always how you get the denominator.

\Pr(A \mid B) = \frac{\Pr(B \mid A)\Pr(A)}{\Pr(B \mid A)\Pr(A) + \Pr(B \mid A^{c})\Pr(A^{c})}

Term	Name	Usually the easy part?
$\Pr(A)$	Prior	Yes — the base rate
$\Pr(B \mid A)$	Likelihood	Yes — the stated accuracy
$\Pr(B)$	Evidence	No — build it from total probability
$\Pr(A \mid B)$	Posterior	The answer

Table 1.3 · What the pieces are called.

Example 1.4

The taxicab problem

A taxi was involved in a hit-and-run. $85\%$ of the city’s taxis are green and $15\%$ are blue. A witness says the taxi was blue, and tests show they identify a colour correctly $80\%$ of the time. What is the probability the taxi was blue?

Show the worked solutionHide the worked solution

Worked solution

Formula $\Pr(B \mid \text{"blue"}) = \frac{\Pr(\text{"blue"} \mid B)\Pr(B)}{\Pr(\text{"blue"} \mid B)\Pr(B) + \Pr(\text{"blue"} \mid G)\Pr(G)}$
Substitute $= \frac{0.8 \times 0.15}{0.8 \times 0.15 + 0.2 \times 0.85}$
Solve $= \frac{0.12}{0.12 + 0.17}$ The witness is right about a blue cab in 12% of cases and wrong about a green one in 17%.
$= \frac{0.12}{0.29}$
Answer $\Pr \approx 0.41$

Sanity check. Below one half, despite an $80\%$ accurate witness — because green cabs are so much more common that the witness’s mistakes about them outnumber their correct calls on blue ones.

The base rate does the work. Think in counts rather than percentages. Out of a hundred cabs, fifteen are blue and the witness correctly calls twelve of them; eighty-five are green and the witness wrongly calls seventeen of them blue. Twenty-nine "blue" calls, twelve of them right. Nothing about that requires a formula, and expressing it this way is usually the fastest route to the answer under pressure.

Figure 1.5 · Where the positives come from. Both highlighted paths end in a positive test. The test is doing its job on each branch; what decides the answer is how many people started down each one.

Conditional probability and Bayes

After this lesson you should be able to