案例学习：预测房价

Loading...

来自 University of Washington 的课程

机器学习：回归

3576 个评分

案例学习：预测房价

从本节课中

Nearest Neighbors & Kernel Regression

Up to this point, we have focused on methods that fit parametric functions---like polynomials and hyperplanes---to the entire dataset. In this module, we instead turn our attention to a class of "nonparametric" methods. These methods allow the complexity of the model to increase as more data are observed, and result in fits that adapt locally to the observations. <p> We start by considering the simple and intuitive example of nonparametric methods, nearest neighbor regression: The prediction for a query point is based on the outputs of the most related observations in the training set. This approach is extremely simple, but can provide excellent predictions, especially for large datasets. You will deploy algorithms to search for the nearest neighbors and form predictions based on the discovered neighbors. Building on this idea, we turn to kernel regression. Instead of forming predictions based on a small set of neighboring observations, kernel regression uses all observations in the dataset, but the impact of these observations on the predicted value is weighted by their similarity to the query point. You will analyze the theoretical performance of these methods in the limit of infinite training data, and explore the scenarios in which these methods work well versus struggle. You will also implement these techniques and observe their practical behavior.

- Emily FoxAmazon Professor of Machine Learning

Statistics - Carlos GuestrinAmazon Professor of Machine Learning

Computer Science and Engineering

[MUSIC]

We've talked at great length about models where you specify some set of features.

And what that results in is a model with fixed flexibility.

There's a finite capacity to how flexible the model can

be based on the specified features.

But now we're gonna take a totally different approach.

There are approaches that are called nonparametric approaches.

And the one that we're gonna look at in this module

is called k-nearest neighbors and kernel regression.

And what these methods allow you to do is be extremely flexible.

The implementations are very, very simple and they allow

the complexity of the models you infer to increase, as you get more data.

This really simple approach is surprisingly hard to beat.

But as we're gonna see,

all of this relies on us having enough data to use these types of approaches.

To start with,

let's talk about this idea of fitting a function fit globally versus fit locally.

So let's imagine that we have the following data that represent

observations of houses with a given square feet and their associated sales price.

And maybe just for the sake of argument in this module,

let's imagine that for small houses there tends to

be a linear relationship between square feet and house value.

But then that relationship tends to taper off and

there's not much change in house price for square feet.

But then you get into this regime of these really large houses where as

you increase the square feet,

the prices can just go way up because maybe they become very luxurious houses.

In the types of models that we've described so far, we've talked

about fitting some function across the whole input space of square feet.

So for example, if we assume a really simple model,

like just a constant fit, we might get the following fit to our data.

Or if we assume that we're gonna fit some line to the data,

we might get the following or a quadratic function.

And we might say, well, this data kinda looks like there's some cubic fit to it.

And so the cubic fit looks pretty reasonable, but

the truth is that this cubic fit is kind of a bit too flexible,

a bit too complicated for the regions where we have smaller houses.

It looks like it fits very well for our large houses but

it's a little bit too complex for lower values of square feet.

Because, really for this data you could describe it

as having a linear relationship like, I talked about for low square feet value.

And then, just a constant relationship between square feet and value for

some other region.

And then, having this maybe quadratic relationship between square feet and

house value when you get to these really, really large houses.

So this motivates this idea of wanting to fit our function locally to

different regions of the input space.

Or have the flexibility to have a more local description of what's

going on then our models which did these global fits allowed.

So what are we gonna do?

So we want to flexibly define our f(x) the relationship in this

case between square feet and house value to have this type of local structure.

But let's say we don't want to assume that there is what are called structural

breaks that there are certain change points where the structure of our

regression is gonna change.

In that case, you'd have to infer where those break points are.

Instead let's consider a really simple approach that works well when you have

lots of data.

[MUSIC]