Recall our diamond data set that we're looking at where

our x was carat and our y was price.

We fit the least squares equation minimizing y

minus beta naught plus beta one x.

We minimize that least squares equation.

We discussed how there was at least two ways to think about this.

One was to find this best line that fits the data, minimizing the sum of

the squared vertical distances between the data points and the line.

The second was to think about why as a point in n dimensional space,

which in this case we're just gonna visualize it with R3.

y is a point in n dimensional space and the plane,

Beta naught Jn + beta 1 x was the two dimensional plane cutting

through the n dimensional space.

And the beta-hat whereas the specific combination of points,

j, n and x, that led to the point right here,

the projection of y onto that plane, which we'll call y hat.

So y hat is equal to beta naught hat + beta 1 hat x.

So in this case, it's this whole vector.

And then if we're visualizing it in this left-hand case,

the y hat each component is just the projections, or is not the projections,

is the straight line distance,

minimum distance points onto this line right here.

So if this point, if this combination is x5 Y 5,

then this point right here is y hat sub 5.

And y hat is the vector of y hats.

So y hat is defined as beta, well, if I'm being technically correct,

which is the best kind of correct, it should be y naught, beta naught,

times j and + b1 times X.

So y hat are the fitted values.

It is the projection of the data point y, the n-dimensional data point y,

onto the two dimensional space defined by X in the constant vector.

Another vector of interest is E, the residual, which is y- y hat.

So every element of E for example if this distance.

So E 5 would be the distance between Y 5 and Y hat 5.

E visualized in this left hand picture, is the vector right here.

Is the vector that is orthogonal to the plane that points up to the data point y.

So it's probably best visualized if we drew it from the origin, it's over there.

So e, it makes sense by the way, and this'll come up later on, that

the vector e is orthogonal to every point in this purple plane that I've drawn here.

Okay. So E are called E,

the vector of differences between y and y hat is the vector of so-called residuals.

If you can think about our least square as criteria,

is actually well we could think about it as minimizing y minus y hat squared,

where y hat takes on this specific form with a beta naught hat and

by finding the best beta naught hat and data one hat.

Or we can also think about our least squares equation

as minimizing some of the squared residuals, okay?

So these are two important concepts, the fit of value's y hat and the residuals E.

So let's go through a computing example where we just compute

each of these quantities.

We'll revisit these very frequently throughout the class,

as they're essential to regression.

And one last comment on y hat.

If we wanted to predict a price for a given carat value, that would be y hat.

And notice, we don't have to only predict at where we observed x values.

If we wanted, for example, to predict right here.

We could plug in an x value that we didn't observe and

get a predictive y value, now that wouldn't be a y hat value,

because y hat is specifically the vector predicted values at the observed x's,

but this illustrates how we would get predictions at x's we didn't observe,

just plugging straight into the line that we fit.

Fitted values and residuals

Advanced Linear Models for Data Science 1: Least Squares

与讲师见面