ARMA

The useful outcome

ARMA separates two kinds of memory: persistence in the observed series and temporary information carried by recent innovations. The distinction is fundamental to ARIMA and SARIMAX. The practical goal is a path that another person can reproduce from a declared cutoff—not a line whose origin, state, or future assumptions are hidden.

The question this tutorial answers is: How do lagged values and lagged forecast errors jointly determine an ARMA conditional mean?

What you will build

You will calculate the model by hand, implement it in Python and TypeScript, inspect eight long synthetic scenarios, and learn where correct recursion ends and empirical forecast validation begins.

AutoReg establishes recursive lag substitution. ACF and PACF help diagnose orders but do not prove a fitted model.

Freeze the variant before fitting anything

Fixed-coefficient ARMA(p,q) with an intercept, contiguous lags, zero pre-sample innovations, recursively reconstructed in-sample residuals, and zero future innovations.

This matters because forecast libraries often share a model-family name while differing in constants, trend representations, signs, initialization, missing data, or optimizer behavior.

Intuition before notation

Known residuals can affect early horizons. Unknown future shocks have conditional expectation zero, so their contribution disappears as the forecast moves forward. The hardest part is known past innovations versus unknown future innovations. The forecast origin is therefore a visible data boundary, not just the point where a chart changes color.

Formula and symbols

$\hat y_{T+h|T}=c+\sum_{i=1}^{p}\phi_i\tilde y_{T+h-i}+\sum_{j=1}^{q}\theta_j\tilde\varepsilon_{T+h-j},\quad \tilde\varepsilon_t=0\text{ for }t>T$

The formula is a conditional point forecast. Future random innovations use zero because zero is their conditional expectation—not because the future is known to contain no shocks.

Data that the formula is allowed to see

A finalized equally spaced series, ordered AR and MA coefficient vectors, an intercept, a declared pre-sample innovation policy, and a horizon. Observations must be equally spaced, finite, ordered, and on one consistent scale. The model version and cutoff are frozen before the holdout. Revisions require a new forecast version rather than silent history replacement.

Algorithm

1. Validate the series, coefficient vectors, intercept, and horizon.
2. Set residuals before the eligible AR origin to zero under the selected conditional convention.
3. Reconstruct one-step fitted values and residuals in time order.
4. At each future horizon, combine lagged observed/forecast values with available past residuals.
5. Append a zero future innovation after every conditional-mean forecast.

Hand calculation

The following is author-derived arithmetic over synthetic values:

{
  "values": [
    10,
    12,
    11
  ],
  "config": {
    "ar": [
      0.5
    ],
    "ma": [
      0.25
    ],
    "intercept": 5,
    "horizon": 2
  }
}

• At t=1: fitted=5+0.5(10)=10, so epsilon_1=2.
• At t=2: fitted=5+0.5(12)+0.25(2)=11.5, so epsilon_2=-0.5.
• h=1: 5+0.5(11)+0.25(-0.5)=10.375.
• h=2: 5+0.5(10.375)+0.25(0)=10.1875.

The expected path is [10.375, 10.1875]. Independent tests pin these values before the larger fixtures are generated.

Calculation flow

The order matters. Revealed actuals may score the path, but they may not alter the version that was issued at the cutoff.

Guided experiment

The standalone lab contains eight scenarios and 1,440 calendar-month-end observations. It trains on 144 values and forecasts a 36-value holdout. Start at horizon one, predict what state should persist, then Step through twelve model-specific checkpoints. Change the principal parameter and watch the complete path recompute from the frozen training data.

The scenarios isolate persistence, mean reversion, linear trend, seasonal structure, a late level shift, one gross outlier, a variance-regime change, and seasonal phase/amplitude drift. They are not selected trades or provider histories.

The state an expert audits

For this model, keep the AR value stack, reconstructed observed innovations, remaining MA memory, and the intercept-implied mean visible.

Choose ARMA only after the modeled level is plausibly stationary and both value memory and innovation memory are defensible. Residual ACF, innovation initialization, invertibility, and the implied mean deserve explicit review.

A large first-horizon adjustment that vanishes quickly is innovation memory. A path that converges to the wrong level is intercept or mean specification. Do not confuse those two failure signatures.

When implementations disagree, use this order: Check MA sign convention first, then pre-sample residual initialization, intercept semantics, AR/MA coefficient ordering, and whether fitted residuals were reconstructed over the same training span.

The workbench is deliberately quantitative. It connects the model's unique state to the declared benchmark rather than asking the learner to judge whether the purple line “looks right.”

Failure diagnosis

• Changing pre-sample residual initialization changes early fitted values.
• MA signs differ across software and textbook conventions.
• Near-noninvertible MA coefficients make estimation and interpretation fragile.
• A single outlier can enter several early forecasts through residual memory.
• Future innovations are zero in expectation, not known to be literally zero.

Describing the conditional mean as the realized future path or hiding the residual initialization convention. Correct arithmetic answers “did we implement the selected model?” It does not answer “is this a good model for this series?”

Implementation contract

The Python reference favors readable state transitions. TypeScript implements the same contract independently. Shared fixtures and focused tests require matching numeric paths within 1e-9 and exact agreement on lengths, nulls, states, and invalid-input behavior.

Estimation is intentionally outside the public function. In a real workflow, fit or select parameters using only training data, record the method and version, pass the frozen result into this recursion engine, and evaluate using rolling origins against simple benchmarks.

Practical evaluation

Use a loss function that matches the decision, report by horizon, and include baseline forecasts. Preserve the refit schedule and origin timestamps. If prediction intervals are needed, choose and validate an uncertainty model; do not manufacture intervals from point forecasts alone.

The lab's declared baseline is nonseasonal naïve last-value forecast. At active horizon k, it calculates errors only for the first k revealed holdout values:

• MAE = mean(abs(actual - forecast));
• RMSE = sqrt(mean((actual - forecast)^2));
• bias = mean(actual - forecast);
• MAE skill = 1 - model_MAE / benchmark_MAE.

Positive skill means the model's revealed-prefix MAE is lower than the frozen benchmark's. Negative skill is useful evidence: the sophisticated recursion is losing to a simple rule on this synthetic path. When benchmark MAE is zero, skill is undefined rather than infinite. One origin still cannot establish general performance; repeat the complete procedure over rolling origins before making an empirical claim.

Related methods

AutoReg introduces lag memory. ARMA adds innovation memory. ARIMA changes the modeled scale through differencing. SARIMA/SARIMAX adds seasonal polynomials and future driver scenarios. Holt-Winters maintains level, trend, and seasonal states directly. Theta combines a smoothed level with constrained drift.

Next: ARIMA.

Historical and evidence boundary

A historical example is not useful for this mechanism lesson. A defensible case requires licensed point-in-time observations, identity, frequency, adjustment and revision basis, cutoff, parameters frozen before inspection, benchmark, holdout, and publication rights.

This tutorial establishes definition fidelity. It does not establish stationarity, model adequacy, forecast calibration, historical association, causation, profitability, or investment suitability.

Primary references

• statsmodels.tsa.arima.model.ARIMA — AR, MA, ARMA, ARIMA, seasonal, trend, and exogenous special cases plus stationarity and invertibility options.
• Forecasting: Principles and Practice — ARIMA models — ARIMA notation, differencing, AR and MA orders, stationarity/invertibility, and recursive point forecasting with future errors set to zero.
• Forecasting: Principles and Practice — Evaluating point forecast accuracy — Forecast accuracy must be measured on genuine forecasts outside the training sample; MAE, RMSE, scaled errors, and benchmarks answer different evaluation questions.
• Forecasting: Principles and Practice — Time series cross-validation — Rolling forecasting origins preserve temporal order and can evaluate horizon-specific errors without using future observations.
• Forecasting: Principles and Practice — Evaluating regression and residual behavior — Residual plots and residual autocorrelation diagnose information left in the fitted model, but residual size is not a substitute for genuine forecast error.