Consider an AR(1) process \(X_t = \alpha X_{t-1} + W_t\). We consider stationarity and autocorrelation for this process.
- We have already seen that we need \(\mu_t=0\) to have first order stationarity.
Now consider the variance. Since \(X_t = \alpha X_{t-1} + W_t\), \[
\sigma_t^2=\text{Var}(X_t) =\text{Var}(\alpha X_{t-1} + W_t)
=\text{Var}(\alpha X_{t-1}) + \text{Var}(W_t)\\ = \alpha^2 \text{Var}( X_{t-1}) + \text{Var}(W_t) = \alpha^2 \sigma_{t-1}^2 + \sigma^2
\]
- (Here we used that \(\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y)\) when \(X\) and \(Y\) are independent).
If the variance is constant, then \(\sigma_t^2 = \sigma^2_{t-1}\) and \[
\sigma^2_t = \alpha^2 \sigma_t^2 + \sigma^2
\]
We see that the variance can only be constant if \(-1<\alpha <1\). In this case \(\sigma_t^2 = \frac{\sigma^2}{1-\alpha^2}\).
For \(|\alpha| \geq 1\), the variance will increase over time. The process is cannot be stationary (including random walk).
To find the autocorrelation, first observe \[
X_{t+h} = \alpha X_{t+h-1} + W_{t+h} = \cdots = \alpha^h X_t+ \sum_{i=0}^{h-1} \alpha^i W_{t+h-i}
\]
Then we find the autocovariance: \[
\gamma(t,t+h) = \text{Cov}(X_t, X_{t+h})
= \text{Cov}(X_t, \alpha^h X_t + \sum_{i=0}^{h-1} \alpha^i W_{t+h-i})\\
= \text{Cov}(X_t, \alpha^h X_t) + \text{Cov}(X_t, \sum_{i=0}^{h-1} \alpha^i W_{t+h-i})
= \alpha^h\text{Cov}(X_t,X_t) + 0
= \alpha^h \text{Var}(X_t)
\]
- (Here we used the computation rules \(\text{Cov}(X,Y+Z) = \text{Cov}(X,Y) + \text{Cov}(X,Z)\) and \(\text{Cov}(X,aY) = a\text{Cov}(X,Y)\).)
If the variance is constant, we can calculate the autocorrelation: \[
\frac{\text{Cov}(X_t, X_{t+h})}{\sigma_t\sigma_{t+h}} = \frac{\alpha^h\sigma^2/(1-\alpha^2)}{\sigma^2/(1-\alpha^2)} = \alpha^h.
\]
So: the AR(1)-model is stationary if \(-1<\alpha <1\) and \(\sigma_t^2 = \sigma^2/(1-\alpha^2)\) - otherwise not.
The autocorrelation decays exponentially for a stationary AR(1)-model. This is illustrated for 3 different \(\alpha\) values:
h = 0:20
acf1 = 0^h # AR(1) with alpha = 0 (or white noise)
acf2 = 0.5^h # AR(1) with alpha = 0.5
acf3 = 0.9^h # Ar(1) with alpha = 0.9
plot(matrix(rep(h,3),3),cbind(acf1,acf2,acf3),col=rep(1:3,each=length(h)),
pch=rep(1:3,each = length(h)),xlab="h",ylab="ACF")
