Data quality: missingness, outliers, regimes and leakage

Most of the time in a research project goes on data, and most of the errors that survive into production started there. The useful framing is that every data problem is a question about the mechanism — why is this value missing, why is it extreme, why does this series change character in 2008 — and the mechanism decides the treatment.

Mechanism	Meaning	Safe treatment
MCAR	Missing at random, unrelated to anything	Dropping or imputing both work
MAR	Missingness depends on observed variables	Impute conditional on those variables
MNAR	Missingness depends on the missing value itself	Model it; the fact of missing is a feature

Table 5.1 · The three missingness mechanisms. Financial data is overwhelmingly the third kind. A firm that stops reporting a metric, or a fund that stops publishing returns, is telling you something — and imputing a mean erases exactly that information.

Missingness as a feature. The instinct is to fill the gap so the model can run. But if the value is missing *because* of what it would have been, the pattern of missingness carries signal that the filled value does not. Add an explicit indicator for "was missing" alongside whatever imputation you use: the model can then learn from the fact as well as the value, and you will sometimes find the indicator is the stronger feature. It also makes the assumption visible to anyone reading the code, which a silent forward-fill does not.

Figure 5.2 · One bad tick, and what trimming it does. A single mis-scaled return in a hundred moves the mean by thirty basis points and quadruples the standard deviation. Winsorising at the first and ninety-ninth percentile is not cosmetic — an unchecked tick like this will dominate any covariance matrix built from the series.

Proposition 5.3

Outliers on fat-tailed data

A four-sigma daily return is not an error, it is Tuesday. Deleting extremes from financial data removes the observations that carry most of the information about risk, and it biases every subsequent estimate of the tail.

Holds when

Distinguish data errors — a price of zero, a 10,000% return, a stale repeat — from genuine extremes.
Winsorise the *features* to cap leverage; leave the *target* alone so the tail is still modelled.
Report results with and without the extremes; if the conclusion flips, that is the finding.

Proposition 5.4

Regimes and structural breaks

Market structure changes: decimalisation, the rise of electronic trading, a tick-size pilot, a new regulation. A model fitted across a break is fitting two different processes, and one of them no longer exists.

Holds when

Plot every feature over the whole history before modelling — breaks are usually obvious to the eye and invisible in a summary statistic.
A rolling-window estimate that jumps at a known date is confirmation, not a coincidence.
Deciding to start the sample after a break is legitimate; deciding it after seeing which start date works is not.

Data quality: missingness, outliers, regimes and leakage

After this lesson you should be able to

Outliers on fat-tailed data

Regimes and structural breaks

The rest of this lesson is in Premium