Pricing a game: EV, re-rolls and when to stop

A trading interview is full of games whose value you are asked to quote. Almost all of them reduce to one idea: at every decision point, compare what you have against what continuing is worth, and take the larger. That comparison is the whole of optimal stopping, and it is also exactly how an American option is priced.

Proposition 1.1

The continuation value

Define $V$ as the value of the game from here on if you keep playing optimally. At any point where you may stop, the value is $\max(\text{what you hold now},\ V)$ . Compute $V$ first, then the policy — "accept anything above $V$ " — falls out for free rather than being guessed.

Holds when

With finitely many rounds, work backwards from the last one.
With infinitely many, $V$ satisfies an equation in terms of itself.
The threshold always equals the continuation value, never the mean of the distribution.

Figure 1.2 · The continuation value divides keep from re-roll. With one re-roll available, a fresh die is worth 3.5. Faces 1–3 should be exchanged for that continuation value; faces 4–6 are worth keeping. Averaging the six displayed values gives 4.25 before the first roll.

Example 1.3

One re-roll

You roll a fair die and may re-roll once, keeping the second value if you do. What is the game worth?

Show the worked solutionHide the worked solution

Worked solution

Formula $V = \frac{1}{6}\sum_{f=1}^{6}\max(f,\ V_{\text{cont}}), \qquad V_{\text{cont}} = 3.5$
Substitute $\max(f, 3.5) = 3.5, 3.5, 3.5, 4, 5, 6$
Solve $V = \frac{25.5}{6}$
Answer $V = 4.25$

Sanity check. Re-roll on 1, 2 or 3; keep 4, 5 or 6. The threshold is 3.5 because that is what a fresh roll is worth, not because it is the midpoint.

Derivation 1.4

Infinite horizon: solving $X = f(X)$

When the game can go on forever, condition on the first step and write the value in terms of itself.

$V = \frac{1}{6}\sum_{f=1}^{6}\max(f,\ V)$
Unlimited re-rolls: the continuation value is the game itself.
$\text{Guess } 4 < V < 5 \Rightarrow V = \frac{4V + 5 + 6}{6}$
Faces 1 to 4 are re-rolled; 5 and 6 are kept.
$6V = 4V + 11 \Rightarrow 2V = 11$

V = 5.5

Proposition 1.5

Always verify the bracket

The step above *assumed* the threshold sat between 4 and 5 in order to write the equation. Having solved it, check: $V = 5.5$ means you re-roll anything at or below 5, so faces 1 to 5 are re-rolled and only a 6 is kept. That contradicts the assumption, so the equation must be rewritten.