ARIMA Time Series Prediction Model

The ARIMA (AutoRegressive Integrated Moving Average) model is a widely used statistical method for time series forecasting. It combines three components: AutoRegressive (AR), Integrated (I), and Moving Average (MA). The ARIMA model is denoted as ARIMA(p, d, q), where $p$, $d$, and $q$ are parameters that need to be determined.

Components of ARIMA

1. AutoRegressive (AR) Component (p):

   • The AR component models the relationship between an observation and a number of lagged observations (i.e., previous values in the time series).
   • The parameter$p$represents the number of lag observations included in the model.
   • For example, if$p = 2$, the model uses the two previous observations to predict the current value.

2. Integrated (I) Component (d):

   • The I component represents the degree of differencing (i.e., the number of times the data have had past values subtracted).
   • The parameter$d$indicates the number of non-seasonal differences needed to make the time series stationary.
   • Stationarity means that the statistical properties of the time series, such as mean and variance, are constant over time.

3. Moving Average (MA) Component (q):

   • The MA component models the relationship between an observation and a residual error from a moving average model applied to lagged observations.
   • The parameter$q$represents the number of lag residual errors included in the model.
   • For example, if$q = 1$, the model uses the error from the previous time step to predict the current value.

Determining the Proper$p$,$d$, and$q$

1. Determining$d$(Differencing Order):

   • The first step is to check if the time series is stationary. If not, differencing is applied to make it stationary.
   • Use statistical tests like the Augmented Dickey-Fuller (ADF) test to check for stationarity.
   • If the series is not stationary, apply differencing (i.e., subtract the previous value from the current value) and repeat the test.
   • The number of times differencing is applied until the series becomes stationary is the value of$d$.

2. Determining$p$(AR Order):

   • Use the Autocorrelation Function (ACF) plot to identify the number of significant lags.
   • The ACF plot shows the correlation between the time series and its lagged versions.
   • The value of$p$is determined by the number of significant lags in the ACF plot.

3. Determining$q$(MA Order):

   • Use the Partial Autocorrelation Function (PACF) plot to identify the number of significant lags.
   • The PACF plot shows the correlation between the time series and its lagged versions, after removing the effects of shorter lags.
   • The value of$q$is determined by the number of significant lags in the PACF plot.

Example

Suppose you have a monthly sales time series and you want to build an ARIMA model. Here are the steps to determine$p$,$d$, and$q$:

1. Check for Stationarity:

   • Apply the ADF test. If the series is not stationary, apply differencing once (i.e.,$d = 1$) and check again.
   • If the series becomes stationary after one differencing,$d = 1$.

2. Determine$p$Using ACF Plot:

   • Plot the ACF for the differenced series.
   • If the ACF shows significant lags at lags 1 and 2, but not beyond, then$p = 2$.

3. Determine$q$Using PACF Plot:

   • Plot the PACF for the differenced series.
   • If the PACF shows significant lags at lag 1, but not beyond, then$q = 1$.

Thus, the ARIMA model for this time series would be ARIMA(2, 1, 1).

Model Selection and Validation

• Model Selection: Once$p$,$d$, and$q$are determined, fit the ARIMA model to the data and evaluate its performance using metrics like Mean Absolute Error (MAE), Mean Squared Error (MSE), or Root Mean Squared Error (RMSE).
• Cross-Validation: Use techniques like rolling forecasting origin or time series cross-validation to validate the model's performance on out-of-sample data.
• Parameter Tuning: Experiment with different combinations of$p$,$d$, and$q$to find the model that minimizes the chosen error metric.

Conclusion

The ARIMA model is a versatile tool for time series forecasting, but its effectiveness depends on the proper selection of the parameters$p$,$d$, and$q$. By carefully analyzing the ACF and PACF plots and ensuring the series is stationary, you can determine the appropriate ARIMA model for your time series data.