The ARMA family: identification, estimation and forecasting

TS · Chapter 212 min readAsked at Two Sigma, Citadel, DE Shaw, QuantCo

Assumes Foundations: stationarity, autocorrelation and the Wold decomposition.

After this lesson you should be able to

  • State the stationarity and invertibility conditions.
  • Forecast from an AR model and describe how the forecast decays.
  • Choose orders without overfitting.

ARMA models express a series as a combination of its own past and past shocks. They are the workhorse for anything with linear memory, and on financial returns they usually fit almost nothing — which is itself the useful finding, since it says the predictability is not in the conditional mean.

Equation 2.1

The models

AR terms carry the series’ own memory; MA terms carry the memory of past shocks.

xt=∑i=1pϕixt−i+εt+∑j=1qθjεt−jx_t = \sum_{i=1}^{p}\phi_i x_{t-i} + \varepsilon_t + \sum_{j=1}^{q}\theta_j\varepsilon_{t-j}
ϕ\phi
AR coefficients. Stationarity requires the roots of the AR polynomial to lie outside the unit circle.
θ\theta
MA coefficients. Invertibility requires the same of the MA polynomial.

Proposition 2.2

Stationarity and invertibility

For an AR(1), stationarity is simply ∣ϕ∣<1|\phi| < 1: shocks must die away rather than accumulate. Invertibility, the MA-side condition, means the model can be rewritten as an infinite AR — which is what makes the shocks recoverable from the observed data and therefore the parameters estimable.

Holds when

  • AR(1) at ϕ=1\phi = 1 is a random walk: non-stationary, and the boundary the unit-root tests examine.
  • Two different MA parameterisations can give the same ACF; invertibility picks the one you can estimate.
  • For higher orders, check that the polynomial roots lie outside the unit circle.

Derivation 2.3

Forecasting from an AR(1)

Iterate the model forward, replacing unknown shocks by their mean of zero.

  1. x^t+1=μ+ϕ(xt−μ)\hat{x}_{t+1} = \mu + \phi(x_t - \mu)

    One step ahead: the deviation from the mean shrinks by ϕ\phi.

  2. x^t+h=μ+ϕh(xt−μ)\hat{x}_{t+h} = \mu + \phi^h(x_t - \mu)

    The deviation decays geometrically.

  3. Var(error)=σ21−ϕ2h1−ϕ2\mathrm{Var}(\text{error}) = \sigma^2\frac{1 - \phi^{2h}}{1 - \phi^2}
forecasts converge to μ; error variance converges to the unconditional variance\text{forecasts converge to } \mu; \text{ error variance converges to the unconditional variance}
0102000.51φ = 0.9φ = 0.5Steps ahead hForecast, in deviations from the mean
Figure 2.4 · Every forecast collapses to the mean. A stationary AR(1)\mathrm{AR}(1) forecasts ϕh\phi^h times today’s deviation, so the horizon at which the model stops saying anything is set entirely by ϕ\phi. At 0.90.9 that is a few dozen steps; at 0.50.5 it is about five.

The rest of this lesson is in Premium

You have read the opening. 12 more sections follow, including 5 worked examples and 3 quick checks.

Start the free 7-day trialSign in

Nothing is charged for 7 days, and you can cancel before then. Or read The law of large numbers and the central limit theorem in full, free.

The ARMA family: identification, estimation and forecasting · QuantMax