[MUSIC] Hello again, and welcome back. In this lecture we're going to start managing the data that we've obtained for this project. Now, I'm going to do a lot of it really quickly, because it could be a little tedious for you to follow along with all the data I'm downloading and extracting and managing. But I'll describe to you what I'm doing, and then I'll provide as an attachment to this lecture, the actual data that I compile. So, you can follow along by just watching what I'm doing if you want, and then download the data that we've brought together here. To start our I'm going to obtain the full set of land use data from the Department of Water Resources in California. And then next up I'm going to move all of this data to a new project folder I'll create. And I tend to like to put all the data into a data subfolder and a project folder and then a raw data folder for all of this data. I'll also keep the zip files in a separate folder after i extract them. So I'll extract all the data. I'll throw the zip files in a folder, and then I'll put them all into a raw data folder so that after I start processing them, I'll have all of this original data stored somewhere that I can access. Okay. So, to start this out, I'm going to go make a new project folder. I'll call it Agriculture Project. Then I'll paste my data in there and then I'll extract all visit files. So I'll use 7-zip for that, and just extract to the same folder. And I'll overwrite the files, because that file is a metadata file, but if it's the same name, it should be the same file. We can always go back and retrieve it from the zip file if we need to. And then, now I have all the data, I'm going to sort it by data type and then I'll select my zip files again. And then I'm going to move them to a folder called Original Zips. And that way I can access them later if I need to without having to go back and download them. So it's kind of my backup if I mess up the data. And then I'm actually going to move all this data into a subfolder called Data Now. And I'll make a Land Use folder and then ultimately I'm going to put that into a raw folder where I store all of my raw data. And I'll select everything except the data folder and I'm going to cut it and then I'll move it into the raw folder after I move the Land Use folder into there. This is a little backwards but just know that I'm working with the data and I'm putting it all inside a data folder in my project and the a raw folder and then a sub folder for the land used data slot. When I extract the rest of my data I can put it in a separate folder and I can find things easily still. So once my data is extracted usually what I end up doing is exploring it in our catalog and then starting to add it to a map document. It looks like I have a number of different datasets here because I did ext from different zip files and they all come in as shapefiles. But this isn't all of them here because a bunch of them are in separate folders, too, and that's just the way of it sometimes. [LAUGH] Sometimes you get data in different folders, and then apparently different geographic coordinate systems. For now, I'm going to ignore these errors, and I'll tell it not to warn me again in this session. Because, what we're going to ultimately do, is merge all these data sets into one problem. What I'm most looking for here in getting this all into a map right now is I just want to be able to explore the data. I want to see if they have similar table schemes, that is, do they have a similar set of attributes so I can merge them and then what's the data coverage like. So, let's, Take all these. And for management purposes, I'm going to group these layers and I'll call them Land Use Shapefiiles. And now I'll zoom to the layer. Now let's collapse our catalog here. And we'll see what kind of coverage we're getting. Okay. So it's not perfect, but It's decent enough that California Central Valley runs probably right around through here. We're missing the southern parts of the state in Sierra Nevada. Mostly places that don't have a lot of agriculture. So far it's still possible to answer our question of the floodplane, or agriculture grown in a floodplane. The next thing I usually do in cases like this is I save a working map document. This map document is not for anything except data analysis. I'm not going to do any sort of, any sort of cartography with it's a map document that I sort of expect will eventually get cluttered even if, I'll do my best to keep it organized and so what I tend to do is I make a folder I call mxds for mxd files. You can call it maps or something else but I call it mxds and that's where I store all of my map documents that I work on, and I'll call it agriculture working. And to me working is my signal word that I don't expect a lot from this map document but it is a spot that might have all of my data in it. It might be set up nicely for viewing lots of different parts of my data and for some analysis. Okay so next up, lets take a look at these attribute tables and see if they seem to have approximately the same, the same field. So that's one from 2006. The six prefix there, let's look at one from 2000. And then let's find a more recent one, 2013 here. And sub class one. Class1 and Multiuse I think are the fields that we wanted from last time. So, we seem to have those fields. So that's a good start. We'll verify that more as we go on, but that's a good place to be right now. The next thing I want to do is see how I can remove all these extra big polygons. They don't seem like they're actually the data. The data seems to be at the smaller scale here but it seems like they fill out somethng like a quadrangle or a quarter quadrangle or something like that with the rest of the boundary of it whether or not it has data. So let's select one of them and see what happens. And again, make those easier to render. And then I have overlapping data sets here, apparently. So let's select this one instead. So, that's one thing to definitely take note of is that this data overlaps. So that's potentially a problem here. We'll identify this later and figure out which one it's in. And it looks like it's in 00FR. If I was going to spend much time like this, I probably want to organize these layers in the table of contents there. So I have one selected and, as I was suspecting, it seems like maybe the subclass and class are defined in such a way that we could potentially remove these. So let's see what happens if I select everything in Class Z. Let's turn off everything except this layer. So, let's go to 00FR. And now let's try the class = Z. Okay, so that gets us some of the way there. It looks like not all of them are selected though. Let's identify this one and see what kind of class it is. So it says class 1 nv but sub class 1 is still star star so let's see again what happens if we select the sub-class = and then the star star. It gives us all those outer ones but it also gives us a bunch in the middle. So now is when I think I should go look at the metadata and see what's going on. Okay so poking around in the folder with the data I did find a file that has the metadata for these class types and sub-class types. And when I bring it up, which I already have it up in the background now, I get standard land use legend. And it sounds like, looking at this document, there is one of these for years 2009 and after and there are other ones. We could see it in the folder. There's an 09 legend and there's an 05 legend. There are other ones for each year. Now we know we need to be careful about merging each different year, because maybe they're a little different. But it breaks up the classes for us. So, the letter classes that we had, and then, these look like they're the sub-classes, so G1 would be barley's being grown. And we have all through these different agricultural classes, and then urban classes, and then native vegetation classes, and unclassified. And we saw that Z outside the area boundary. So, let's think of a plan of attack now. How do we want to approach this problem? What I'm thinking I'm going to do next is take only the stuff we really care about. We're not interested in urban right now. We're not interested in the industrial or the vacant or the native vegetation or the unclassified. So we can approach this a couple ways. We can make a query that specifically excludes all these classes or a query that includes the rest of them. And I think that probably what's easier is a query that excludes. Because we can do a partial match on starting with U and starting with N and then E or Z. So let's go see if we can subset our data Just as a test right now. Okay. So, let's go back to select by attributes, we're on the correct data layer this time. And let's go to, CLASS1, in, instead of a not in, we're going to do an in really quickly just to make sure that we only are picking up the correct items in our query. So we're going to we're going to find the set of things we actually want to avoid at first, and lets start with nr, nv, nv for the natural vegetation, and then we'll do U, C, UI, UL, UR, UV, Z for now. Let's see what we got. Apply. Okay. So the query seems to be good enough. Class 1 here looks like it's only picking up the things we want. Okay, so then let's make sure that we have all of them now. So, we're looking at, I'm missing ENTRY DENIED. So, that's Class E, and then I think the rest of them are agricultural, so I'll do, E and Apply. It looks like we have about 5200 records selected. So now let's switch this to a not query. We'll negate it by going Class 1 not in this, so it'll give me all the records where Class 1 is not one of these values. And I'll click Apply. And I do, in fact, get the inverse of that. So close that out and now I have a layer and a query that seem like they're going to work for this. So, I'm going to just go save out this query somewhere in some metadata so that I know what I did to subset it and then when I merge all of them into one I'll actually do my subsetting. So rather than going to data, I'll go to new, I'm just going to make a text document called queries. I don't have a system for this, but, let's edit that and say, sub setting land use layers to agricultural types. And just save that aside for now, and that way we can come back to it later. Okay I'm going to pause this working here, and we'll pick up where we left off in the next video. That way you don't have videos that are crazy long and you have to remember your place in. So, see you in the next video.
