Using ARIMA and Box-Jenkins methodology for time series forecast - Part 1
The Box-Jenkins method, named after George Box and Gwilym Jenkins, is an approach used in time series analysis to find the best fit of a time-series model to past values of a time-series applying Autoregressive Moving Average (ARMA) and Autoregressive Integrated Moving Average (ARIMA).
Approach
The approach has been referred as a stochastic model building, an iterative approach which consists of the following steps:
- Identification
- Estimation
- Diagnosis Checking
1. Identification
The identification step can be broken down into:
- Assess whether the time series is stationary, and if not, how many differences are required to make it stationary.
- Identify the parameters of an ARMA model for the data.
1.1 Differencing
Use unit root statistical tests on the time series to determine whether or not it is stationary. Repeat after each round of differencing.
Avoid over differencing.
Differencing the time series more than is required can result in the addition of extra serial correlation and additional complexity.
1.2 Configuring AR and MA
Two diagnostic plots can be used to help choose the p and q parameters of the ARMA or ARIMA.
Autocorrelation Function (ACF). The plot summarizes the correlation of an observation with lag values. The x-axis shows the lag and the y-axis shows the correlation coefficient between -1 and 1 for negative and positive correlation.
Partial Autocorrelation Function (PACF). The plot summarizes the correlations for an observation with lag values that is not accounted for by prior lagged observations.
Both plots are drawn as bar charts showing the 95% and 99% confidence intervals as horizontal lines. Bars that cross these confidence intervals are therefore more significant and worth noting.
- The model is AR if the ACF trails off after a lag and has a hard cut-off in the PACF after a lag. This lag is taken as the value for p.
- The model is MA if the PACF trails off after a lag and has a hard cut-off in the ACF after the lag. This lag value is taken as the value for q.
- The model is a mix of AR and MA if both the ACF and PACF trail off.
2. Estimation
Estimation involves using numerical methods to minimize a loss or error term.
We will not go into the details of estimating model parameters as these details are handled by the chosen library or tool.
I would recommend referring to a textbook for a deeper understanding of the optimization problem to be solved by ARMA and ARIMA models and optimization methods like Limited-memory BFGS used to solve it.
3. Diagnostic Checking
The idea of diagnostic checking is to look for evidence that the model is not a good fit for the data.
Two useful areas to investigate diagnostics are:
- Overfitting
- Residual Errors.
3.1 Overfitting
Check whether the model overfits the data. Generally, this means that the model is more complex than it needs to be and captures random noise in the training data.
This is a problem for time series forecasting because it negatively impacts the ability of the model to generalize, resulting in poor forecast performance on out of sample data.
==Careful attention must be paid to both in-sample and out-of-sample performance and this requires the careful design of a robust test harness for evaluating models.
3.2 Residual Errors
Forecast residuals provide a great opportunity for diagnostics.
A review of the distribution of errors can help tease out bias in the model. The errors from an ideal model would resemble white noise, that is a Gaussian distribution with a mean of zero and a symmetrical variance.
For this, you may use density plots, histograms, and Q-Q plots that compare the distribution of errors to the expected distribution. A non-Gaussian distribution may suggest an opportunity for data pre-processing. A skew in the distribution or a non-zero mean may suggest a bias in forecasts that may be correct.
Additionally, an ideal model would leave no temporal structure in the time series of forecast residuals. These can be checked by creating ACF and PACF plots of the residual error time series.
The presence of serial correlation in the residual errors suggests a further opportunity for using this information in the model.
ARIMA forecast on Part 2
