0:00

Hi. In this lecture, we're gonna talk about a very simple class of models that

helps us make sense of data and these are known as categorical models. In a

categorical model, what you do is you basically bin reality into different

categories, and you hope these categories help you make better sense of the data.

That they explain some of the variation in the data. I wanna start out by just

describing what a categorical model's like and we'll talk about how they can help us

make sense of data. So let me give an example. A long time ago, over a decade

ago I was at a conference in Amazon, which is a company that, you know, sells all

sorts of stuff over the web, [laugh] right? Had, was just going public. And

there was a discussion whether Amazon was a good investment or not. So one person

who was a Wall Street investor said, you know, I think it's a horrible investment.

If you think of Amazon all it is, is it's just a delivery company. Like they just

got a big warehouse, you order stuff, they deliver it. The margins in that industry

are really small. Like there's already UPS and EPX, [laugh] UPS and FedEx and DHL and

all those sorta places. I just don't think there's any money in it. Now, another

person said, you know what, I'm gonna put Amazon in a very different box. I'm gonna

put this box over here in a box that says information, because I think its part of

the new information economy. They're gonna gather all this information about what

consumers want. It's all going to be centrally held. They're gonna be worth a

ton of money. Now it turns out if you put Amazon in this information box, you

probably would have invested in it, and you'd have made a lot money. If you put

Amazon in this delivery box, you wouldn't have invested in it, and you wouldn't have

made a lot of money. So which box you use [inaudible] how you categorize things

affects how you think about things. And, again, what sort of decisions you make. So

this leads to a phrase that one of my friends who's a physcholy, a psychologist

[laugh] once said, lump to live. What my friend meant is this, is that, we create

these lumps, these boxes, these categories in order to make sense of the world. So I

look out there in the street and I see a vehicle. I don't say oh it looks like a

1997 Ford F150 pickup truck, right? Instead I just say truck or I just say

car. Or if I look at a piece of furniture I just say it's a dresser. I don't say

it's an 1874 Chip N Dale dresser. I don't completely break it down. I just put

things in categories. And these [inaudible]. They're short cuts, right?

They help us make sense of the world. And let's think about why we model again,

right? One of the reasons we model is to help us decide, strategize and design.

Right. So one of the reasons we lump is it helps us make quicker faster decisions

where we just put things in categories and say this is something I like, this is

something I don't like, this is something that's risky, this is something that's not

risky. Let me give some examples. Give some fun examples. So the first one is

let's suppose you?re a kid and you gotta decide what am I gonna eat and what am I

not gonna eat. Well one sorta categorization you might use is the green

categorization. So you might say anything that's green. Proxy screen. A

grasshopper's green, asparagus is green. All these things are green, right?

Everything else, bananas, those are yellow. Candy bars, brown, orange, they're

orange. Pears, pears can be green, but we'll assume they're yellow. And

strawberries, they're red. These other things aren't green. And so your rule

could be, I'm gonna eat anything that's not green and I won't eat anything that's

green. And so that rule will keep you safe from things like grasshoppers and

asparagus. Right, so that's a rule you might follow. Now it's not an optimal rule

because you might run into a green pear and that green pear might be something

you'd really like but if you've been avoiding green things, you may decide,

well not gonna risk it. So that the same, an example of a simple rule. Let's now

show how you can use a rule like that to make sense of data. So now let's suppose

I've got a bunch of data here, and these are different food items, and what you've

got in this column right here are calories. So this is how many calories

there are in each of these, these food items. So what I want to do is I'm trying,

I wanna make sense of why do some things have a lot of calories and some things not

have a lot of calories. And so I've got this list of items. Well the first thing

that I need to try to make sense of is how much variation is there in this data. Well

to understand how much variation there is, first I've got to find out, sort of what's

the average value? And then variation tells me, how far are things, on average,

from that value? So if I add all this up, I've got a 100+250 that's 350, 440, 550,

900 right? So we've got 900 divided by five, so that means the mean here Is 180,

so on average, everything in this group has about 180 calories. And I want to ask

some things are higher. Right? This is 350 and some things are lower, this is 90. I

want some understanding of how much variation is in that data. So one way to

do that we just subtract the mean from everything. So if we take 100 minus 180

that's gonna be minus 80. 250 minus 80 that's gonna be 70. 90 minus 180 that's

minus 90, right? 110 minus 180 is minus 70 and 350 minus 80 is 170. Well if I add all

these things up I'm gonna get minus 80, plus 70, minus 90, minus 170, plus 70,

it's be zero because it's gonna be the same as a mean. So what I need is I need

all these differences to be positive. So one thing I can do is I can just take the

absolute value. Of all these things. Right? And then I could add up the

absolute value. And I could get 80 plus 70 is 150. Plus 90 is 240. Plus 70 is 310.

Plus one 70 is 480. So we could say, the total difference from the mean is 480. But

what we do in statistics, is we tend to do something different. We actually tend to

take the difference and square it. And the reason we square it is really twofold. One

is that again it makes everything positive, which is what it did before. And

the other thing is that it amplifies larger deviations. Because what we'd

really like to do is prevent those huge deviations. This is gonna amplify large

deviations. So if I look at the pair, I would have 100 minus 180, which is 80

squared which is 6,400. So that's how much variation there would be. So that's the,

how much the. Difference from the pair, pair to the mean squared. And if

[inaudible] for the cake, I'm gonna get 250-180 which is 70. And if I square that,

right, I'm gonna get 4900. Now I could do this for everything. All of them right? So

for the pear, I get sixty four hundred, for the cake I get forty nine hundred, for

the apple eighty one, for the banana forty nine, for the pie, twenty eight [laugh],

thousand, nine hundred. So this is again, a long way from the mean, and you square

it at a huge effect, so square amplifies larger mistakes. Now if I add up all these

numbers, I'm gonna get fifty three twenty. That's what we call a total variation. So

I plotted that data, this tells me sort of, how much variation is in that data,

what I'd like to do is keep. [inaudible] categories [inaudible] that reduces that

variation that somehow explains why something are high and some things are

low. So what's the obvious. Categorization. The obvious categorization

here is that pears and apples and bananas are fruit and cakes and pies, right, are

desserts. So let's create a fruit category and a dessert category. So in the fruit,

I've got one thing that's 90, one thing that's 100 and one thing that's 110. And

in the dessert category, I've got one thing that's 250 and one thing that's 350.

So let's look at them in more detail. I've got 90,100, 110, the mean Is gonna be 100

here, right? The average is also 100. What's the total variation? Well, 90 minus

100. Is just ten, so if I square that, I get 100. 100 minus 100. Right. Is zero, so

if I square that, I get zero. And one ten minus 100. Is also ten, so if I square

that, I get 100. So the total variation here is just gonna be 100+100 or 200. So

now what I've done is I've got a mean of 100, and a total variation of 200. And

now, if I go to this case, the mean is gonna be 300, right, for the desserts. And

what's the total variation? Well, for the cake, it's 250-300, which is 50 squared,

which is 2500. And for the pie, it's 350-300, which is also 50 squared. Which

is 2500 [inaudible]. Add those up, I get 5000. Alright, so let's clean this up a

little bit. So what I did is by creating two categories, a fruit category and a

dessert category, I now have a mean in the fruit category of 100 and a variation of

200, and a mean in the dessert category of 300 and a variation of 5000. Now one

[inaudible] I started out with, right? When I had all the stuff together. I had a

mean of 180 and I had a variation. Of 53,200. Now look at how much my variation

has gone down. It went from 53,000 to 5,000. So here's to the idea. These

categories substantially reduce the amount of variation I have left over. So think of

the variation as what's unexplained. So initially I say look I can just say things

on average of 180 calories and we've got 53,000 units of variation that's [laugh]

unexplained. Now I say wait a minute, I'm gonna create a categorical model that says

there's fruit in desserts and fruits have few calories than desserts. And you can

say we?ll look it appears to be the case. Fruits have a mean of 100. Desserts have a

mean of 300 and the variation in the fruits is only 200 and the variation in

the desserts is 5,000. So I've reduced variation a ton. What we want is we want a

formal measure of how much we've reduced variation. That's actually fairly simple,

right? So that a total variation of 5300, fruit variation is 200, dessert variation

is 5000. So that gives me 5200. So 53,000 start and I get 5200 left. So what we

wanna ask is how much did I explain. That's the question. How much of that

radiation did I explain? Well, then I started out with 53,000, right 200. And

now I only have 5,200 left over and so the amount I explained is just 53,000 minus

5,000 right which is 48,000 right. >> Over 53, two. So the percentage of [inaudible]

I explained was 48,000 divided by 53,000 which is a huge amount. And I can write

this more simply as just one minus the amount I didn't, that's left over. One

minus 5,200 over 53,000. So, right? Because that's just a simpler way to do

it. And so when I get that the amount of [inaudible] I explained was 90 thou- 90

percent. So 90.2%. So that's how much of that variation I explained. This is equal

to, again, that 48,000 right, divided by 53,200, the amount of variation that I

explained. Now, formally, this is called the R squared. So this is the [inaudible],

the percentage of variation that I explained just by that simple

categorization. So, if the R squared is near one. That means I explained almost

all the variation, so the model explains a lot. Right? If the R squared is near zero,

that means I didn't explain any of the variation really, and the model doesn't

explain very much at all. Now the better the model, the more R large R squared

it'll have. But depending on, there could be so much variation in the data that even

a great model. Only has an r squared of five or ten%. There also could be

situations where the thing you're trying to explain in pretty understandable and a

good model has to have an r squared of 90%. So there's no fixed rules whether,

you know, what a good r squared is. It depends what the data looks like. But with

a class, you know sort of class of models, or you know that's [inaudible] data class,

you can sorta figure out this is a good model, this is a bad model. Based on

experience. Let's push this a little bit further. We had, you know, fruit and

desserts, right? Those are our two categories. But if I had, you know, a

whole kitchen worth of food, it may be the case, that, like, I'd wanna have more

categories. So I might create a vegetable category and a grains category. And then I

could put everything in one of these four boxes. So one of the differences between

sort of experts and nonexperts is experts tend to have more boxes. They also tend to

put the right things in the right boxes, so they tend to have useful boxes. So if

you want to be good at sort of predicting things or understanding how the world

works, what you have to have is a lot of categories and you have to have those

categories be the right categories. They've got to explain a lot of the

variation. And we can measure how much of the variation it explains, your model

explains, by using that R squared. One last point. Even if you explain a lot of

variation, it doesn't mean you've got a good model. Let's go back to the schools

case. So suppose I'm trying to figure out what makes a good school, what really

leads to a good school performance. So I try all sorts of different boxes. I look

at schools that spend a lot of money versus schools that don't spend a lot of

money. Schools that have small class sizes and big class sizes. Schools that are big

and schools that are small, right? And nothing really seems to explain too much

of the variation. And then I create a box that I call the equestrian box. And I put

all the schools in here that have equestrian teams. And I find, oh my

goodness, every school with an equestrian team is great. Well, the thing is, that

doesn't mean that the equestrian team made the school good, right? So statisticians

make a distinction between correlation. Which is, is there a statistical

relationship between having an equestrian team and being a good school? And

causation, did the equestrian team cause the school to be good? So remember when we

draw, when you think about putting it in this box. Like, this box right here is a

bunch of good outcomes. And this box here has mostly bad outcomes. That doesn't

necessarily mean that the thing that created this box, if it's the equestrian

box. Is the reason that the schools were good. It could be that there's some other

reason. So why would you have an equestrian team? Well you only have an

equestrian team if you had a lot of money. And you probably also only have an

equestrian team if you have a lot of parental involvement, things like that.

Like a lot of support from the community. And you only have an equestrian team if

you have a lot of open space. So it could be that having an equestrian team is a

proxy for things like money, parental involvement, open space, right, those

sorts of things that actually do make a school good. So even if your boxes work,

that's no guarantee that they're actually the cause of why it works. Okay, so what

do we have? We have one way, [inaudible] the simplest model you can possibly have

is a character model. You can say, Amazon is a delivery company. Amazon's an

information company. You can say things are either fruits or desserts. Right? And

by creating these boxes it can help you sort of explain the variation in data.

What we saw then is a simple way to measure how good a categorization is, is

how much of that you explained the percentage of the variation you explained

and that's what we called R squared, right? R squared was just take all the

variation that was there and then ask how much was left how much was left over. That

means and then we subtract those two that tells you how much you explained. So you

ask what percentage of all that variation do you. Let me start over. Okay, what have

we learned. What we've learned is this. We've learned that the simplest kind of

model you can have is just a category based model, right where you just sorta

lump the world in different categories, and you place your data in different boxes

depending on what different data it is. So that could be information companies versus

delivery companies. That could be fruits and desserts, right? And in doing that,

what you could do is you can reduce the amount of variation you see in the data.

So there's a total variation which is sorta of just like how much unexplained

variation there was out there in the world, by putting it in boxes you organize

it in such a way that you reduce the variation. The amount at which you reduce

the variation is what we call the r squared. That's the percent of variation

explained. And the more variation explained the better your categorization

is. Of course if you create more boxes you can explain more of the variation. Where

we're going next, if you've got linear models, which in effect can create a

different box for each value of x, our dependent variable. Okay, thanks.