Welcome back to Practical Time Series Analysis. In these lectures, we're looking at some of the fundamental building blocks, some of the fundamental stochastic processes that give rise to the sorts of time series we're likely to encounter in our professional practice. We've already seen moving average processes, and now we explore autoregressive processes. When you're done with this lecture, you should be able to describe to a friend or a colleague what an autoregressive process is, what it's seeking to model. You should be able to simulate with AR or similar environment an autoregressive process. You should be able to discuss qualitatively what the ACF of some simple autoregressive processes look like and we'll tie the autoregressive process back to our concept of a random walk. Just to recall the notational environment that we're in, for a moving average process, we start with white noise, and we then take a linear combination of several preceding terms in time of white noise, and build our new state X sub t from those. We're going to do something that looks similar, but is actually rather different with the auto regressive process. We'll see what we mean by looking similar in a moment. Right now let's say that X sub t is going to be some sort of a piece that we can't model very well, an innovation, a random shock to a system. Plus a term that depends upon the history of the system. In other words, the several states preceding. When we look at history what we mean is take xt- 1 down through xt- p. We're going to look at these several history terms. We'll form, again, a linear combination by multiplying by coefficients. And we'll then put in the piece that we don't know quite how to model, this random term Zt, in order to create our new state of our system. The random walk is almost trivially seem to be of this sort. The state of the system at time t looks like what it looked like one period ago, so the coefficient in front of the Xt- 1 can be just taken as a 1, plus some Z of t. We present this right now just as a quick caution that the auto-regressive processes won't necessarily be stationary. In fact, we'll try to come up with some basic conditions that tell us when an auto-regressive process is stationary. Moving onto simulation, and we can't stress enough how important it is for you to run many simulations. Look at the resulting traces, trajectories, realizations, to look at the resulting time series. And also look at the ACF that you're generating. So we'll set a random seed so that everybody will have the same data when they run their simulations. We'll take 1000 terms with Phi = 0.4. There;s a little bookkeeping to be done. We'll set up with z as a family of independent high ID random variables. We're going to say x equals null so that we have the variable named x that we're going to now start filling. We'll take our first x to be z1, and more importantly, how do we start building states in our system? x at time t will look like some noise component, plus phi xt-1, and we'll fill in slots from time 2 to time n. A little bit of housekeeping or bookkeeping on plotting, we'll create a time series object as we've done before. We'll set up our plots so that there are two rows and one column. We'll plot the time series and we'll plot its estimated ACF. When phi = 0.4, there is some dependence upon neighbors. You can look, and at this point, we should have the intuition to see that this is not just noise, but that there are some correlations. It's hard to get a quantitative measure of that just be looking at a time series like this. So we're led to the ACF where we look at correlations at different lag spacings. And we can see that after two or three periods, the correlations seem to be getting down to noise, but we do have a fairly health relationship over a couple side periods. If we bring phi up to 1, we're just moving past stationarity and also we'll discuss that later and also you can see that the ACF, here there is some various decay in the ACF. But the ACF would stay theoretically right constant up there at 1. There's strong dependence on history. Let's look at an AR(2) process, an auto regressive process of order 2, let's go back and look at the formula. We've got some random piece. And I'm taking 0.7 and 0.2 here as coefficients. But again, you should be running these codes and varying the coefficients. And making observations on how the time series looks, and how the ACF looks. We're trying to increase our sophistication. So now we'll call arma.sim. This is a routine available to us in the stance package. It's going to want a list of coefficients. And arima.sim, as the name suggests, will give you auto regressive integrated moving average simulations. We haven't really talked about i yet. But we can put in autoregressive terms or moving average terms and you can see we're putting our 0.7 and 0.2 in for the autoregressive piece. And then we'll do the same sort of plotting that we do. You can look at the time series up here and look at the ACF down here, and see that the correlations, you can see that very strongly in different pieces, the correlations are going to stick around for some time. The correlations decay at a slower rate now, than they did just a moment ago. We like our time series to be stationary for several reasons. There are conditions, and we'll develop conditions involving the unit circle in a little bit. Right now the geometry suggested is more of a triangle. And in order for an AR(2) process to be stationary, we'll state here without proof that phi 2 should be between negative 1 and 1. And there's a relationship between phi 2 and phi 1 that must be satisfied as well. We'll simulate an AR(2) process. As we move here there's a very small difference. We're going to have phi1 be 0.5. We'll let phi2 be negative now and look at the time series in the ACF. We'll call arima.sim as before, but now we're going to put phi1 and phi2 in as variables. That's, perhaps, obvious in that line. A little bit more interestingly, when we do our plotting, we can put those variables right into our plot command by using Paste. So Paste is going to create a nice characterated feed to main, the title of our plot. We'll give it a character string and then we'll separate by comma, but now we can actually put in phi1 and phi2 just as variables into the plotting command. That'll save you a lot of time and a lot of trouble by going back and hoping that you find every phi1 and phi2 as an argument or as a label. When we look now, we can see that the time series is jumping around quite a bit. In fact, by phi2 being negative, we're actually introducing negative correlations into our ACF. So you can see the correlation of neighbors one step away is positive. 2 and 3 negative. And then it's very hard to tell if you don't have a formula, but it looks like we get into noise pretty quickly after that. In this introductory lecture, we've defined what an autoregressive process is. We've seen how to write it down. We've explored simulations. We've begun discussing qualitative features of the ACF, but again, not to be pedantic or redundant about it. Can't stress enough that running these simulations yourself will give you a lot of insight into how these processes behave, so try to run as many as you can.