Foundations: stationarity, autocorrelation and the Wold decomposition

Time series analysis begins by asking whether the thing you are modelling has a stable mean and a stable dependence structure. Almost everything else — estimation, forecasting, inference — assumes it does, so establishing stationarity is not a preliminary but the first substantive step.

Definition 1.1

Stationarity

Weak (covariance) stationarity, $\mathbb{E}[x_t] = \mu, \quad \mathrm{Var}(x_t) = \sigma^2, \quad \mathrm{Cov}(x_t, x_{t+h}) = \gamma(h)$ — Constant mean and variance, and a covariance that depends only on the gap between observations. Strict stationarity asks that the whole joint distribution be shift-invariant, which is stronger and almost never needed — second moments are what the estimators use.

Proposition 1.2

Reading the ACF and PACF

The autocorrelation function gives $\mathrm{Corr}(x_t, x_{t-h})$ at each lag; the partial version gives the same after removing the effect of the intervening lags. Their shapes identify the model: an AR process has a PACF that cuts off sharply at its order and an ACF that decays; an MA process is the mirror image.

Holds when

AR( $p$ ): PACF cuts off after lag $p$ , ACF decays geometrically.
MA( $q$ ): ACF cuts off after lag $q$ , PACF decays.
ARMA: both decay, and the orders are chosen by information criteria rather than by eye.
Bands of $\pm 2/\sqrt{n}$ mark what is distinguishable from zero.

Why the partial version exists. In an AR(1) with coefficient $0.8$ , $x_t$ correlates with $x_{t-2}$ at $0.64$ — but only because both are tied to $x_{t-1}$ , not through any direct link. The ordinary ACF cannot distinguish a genuine two-lag dependence from one transmitted through the intervening value, so it decays smoothly and tells you little about the order. The partial autocorrelation removes the intermediaries and shows the direct relationships only, which is why it cuts off cleanly at the true order.

Figure 1.3 · What an $\mathrm{AR}(1)$ autocorrelation looks like. Geometric decay, never a cut-off — that is the signature that separates an

\mathrm{AR}

from an

\mathrm{MA}

, whose autocorrelation goes to exactly zero after lag

q

. Reading which of the two you are looking at is most of identification.

Foundations: stationarity, autocorrelation and the Wold decomposition

After this lesson you should be able to

Stationarity

Reading the ACF and PACF

The rest of this lesson is in Premium