[MUSIC]

So up until this point we've talked about functions of just one variable and

finding there minimum or maximum.

But remember when we were talking about residual sums of squares,

we had two variables.

Two parameters of our model, W zero and W one.

And we wanted to minimize over both.

Let's talk about how we're going to move to functions defined over

multiple variables.

Moving to multiple dimensions here,

where when we have these functions in higher dimensions

we don't talk about derivatives any more we talk about gradients in their place.

And what a gradient is.

Let me write this, this is the notation for

the gradient of a function where this W is really a vector of different W's.

Let's say W zero, W one, all the way up to some W,

I didn't leave enough room there, sorry.

It's the sum WP.

And what the gradient is it's pretty straightforward actually.

The definition it's gonna be a vector,

where we're gonna look at what are called the partial derivatives of G.

We're going to look at the partial with respect to W zero.

The partial of G with respect to W one.

W one all the way up to the partial of G with respect to some WP.

And let's describe what one of these looks like.

So, for example here.

This partial derivative.

It's exactly like a derivative

where we're taking the derivative with respect to in this case W one.

But what are we going to do with all the other W's?

W zero, W two, W three all the up to WP?

Well we're just going to treat them like constants.

It's as if there were just numbers, five, seven and ten,

when we first talked about taking derivatives in one dimension.

The partial derivative [SOUND] is like

a derivative with respect to W one, in this case.

[SOUND].

Treating all other variables as constants.

Just to be clear, this gradient is in this case if we assume that

there's some variables W zero, all the way up to WP, there's

a reason that I'm using this notation we'll see it later in this course.

But this represents a P plus one, cause we're indexing starting at zero.

This is a P plus one

dimensional vector.

Let's work through a little example here, where I defined a function

G of W to just be five W zero plus ten W zero W one plus two W one squared.

And let's compute the gradient of W.

And in this case they're just two variables, so

I have to take the partial of G with respect to W zero.

I'm gonna take the derivative of this thing here, only interested in W zero.

This term I get a five the next term, well,

I"m treating W one like a constant.

Just like I would take the derivative and get a constant as the result,

that's what I'll get, so there's a ten, and there's a W one.

And then I take the derivative of this term.

It has nothing to do with W zero.

It's just like a constant.

So, the result is gonna be zero.

Now I take the partial with respect to W one and

this first term is just a constant with respect to W one so it doesn't appear.

This next term, well, I end up with

these two terms as the coefficient for W one.

And then I get to this last term, and

again using the same derivative property's we talked about before.

When I have something to a power, I'm gonna bring not powered down, so

I'm gonna get four W one, raise to the one power.

Okay, so my gradient of G

is going to be five plus ten w one,

ten W zero plus four W one.

So what does this mean?

Well, if I want to look at, let me switch to the red color so

that you can see it on this plot, If I want to look at the gradient

at any point on this surface well I'm just going to plug

in whatever the W one and W zero values are at this point.

So there's some W zero, W one value, and I'm going to compute the gradient.

And it'll be some vector.

It's just some number in the first component,

some number in the second component, and that forms some factor.

[MUSIC]

Gradients: derivatives in multiple dimensions

机器学习：回归

与讲师见面