Multivariate Taylor

Loading...

来自 Imperial College London 的课程

Mathematics for Machine Learning: Multivariate Calculus

729 个评分

Imperial College London

Mathematics for Machine Learning: Multivariate Calculus

729 个评分

课程 2（共 3 门，Specialization Mathematics for Machine Learning）

This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future.

从本节课中

Taylor series and linearisation

The Taylor series is a method for re-expressing functions as polynomial series. This approach is the rational behind the use of simple linear approximations to complicated functions. In this module, we will derive the formal expression for the univariate Taylor series and discuss some important consequences of this result relevant to machine learning. Finally, we will discuss the multivariate case and see how the Jacobian and the Hessian come in to play.

Linearisation5:16

Multivariate Taylor6:27

See you next module!0:28

与讲师见面

Samuel J. Cooper
Lecturer
Dyson School of Design Engineering
David Dye
Professor of Metallurgy
Department of Materials
A. Freddie Page
Strategic Teaching Fellow
Dyson School of Design Engineering

So far in this module I've introduced you to the concept of Power Series, and

then shown you how we can use them to approximate functions.

We've also seen how Power Series can then be used to give you some indication

of the size of the error, that results from using these approximations,

which is very useful when applying numerical methods.

The last idea that we're going to very briefly look at on this topic

is upgrading the Power Series from the one-dimensional version that we've seen so

far to its more general multivariate form.

Just to recap on the notational options from the previous video.

We saw that we can re-express the Taylor series from a form that emphasises

building the approximation of a function at a point p to a totally equivalent form

that emphasises using that function to evaluate other points

that are a small distance delta x away.

Just to make sure you can see that they're the same, we can write out the first four

terms, one above the other, to make the comparison clear.

In this video, we are going to continue using the delta x notation

as it's more compact, which will come in handy when we're in higher dimensions.

We won't be working through the multivariate Taylor series derivation

in any detail, as all I really want you to take away from this video is what

a multidimensional Power Series will look like, both as an equation and in a graph.

So keeping in mind our one-dimensional expression, let's start by looking at

the two dimension case, where f is now a function of the two variables x and y.

So our truncated Taylor series expressions will enable us to approximate the function

at some nearby point x plus delta x, y plus delta y.

Let's look at the simple two-dimension Gaussian function, which many of you may

be familiar with from studying the normal distribution in statistics.

It's a nice, well-behaved function with a simple maximum at the point (0,0).

Now in a one dimension analysis, our Power Series will give us an expression for

a one dimensional function.

However, in a two dimensional case our power series would of course now give us

a two dimensional function, which we would more intuitively refer to as a surface.

Just as with the one dimensional case, our zeroth order approximation

are simply the value of the function at that point applied everywhere,

which means in 2D, this would just be a flat surface.

So a zeroth order approximation at the peak would look like this.

And is zeroth order approximation somewhere on the side of the bell

would look a bit more like this.

This is fairly straightforward, but

now let's think about the first order of approximation.

drawing the analogy from the 1D case again,

the first order approximation should have a height and a gradient.

So we're still expecting a straight surface, but

this time it can be at an angle.

Now if we are calculating it at the peak, it wouldn't be a very exciting case

as it's a turning point and so the gradient is zero.

However, let's look again at the point on the side of the slope.

Taking the first order approximation here gives us a surface with the same height

and gradient at the point.

Finally, let's look at the second order approximation for these two points.

We're now expecting some kind of parabolic surface.

However, if I calculate it at the peak nothing seems to appear.

But that's just because we've created a parabola inside our bell curve, so

we need to look from underneath to even see it.

Finally, if we put up the second order approximation at a point at

the side of the bell curve,

you can see that we end up with some kind of interesting saddle function.

But it's still fairly clear even by eye that the gradient and

the curvature are matching up nicely at that point.

Although, this approximation is evidently not useful

outside of a fairly small region around the point.

We're now going to take a look at how we would write expressions for

these functions.

So if we want to build a Taylor series expansion of the two dimensional function

f at the point x, y.

And then use it to evaluate the function at the point x plus delta x,

y plus delta y, our zeroth order approximation

is just a flat surface with the same height as the function

at our expansion point.

The first order approximation incorporates the gradient information

in the two directions.

And once again, we are thinking about how rise equals the gradient times the run.

Notice here for compactness, I'm using the partial symbol with the subscript

to signify a derivative with respect to a certain variable.

If we look now at the second order approximation, we can see

that the coefficient of one-half still remains as per the one dimensional case.

But now we have three terms, all of which are second derivatives.

The first has been differentiated with respect to x twice.

The last with respect to y twice.

And the middle term has been differentiated with respect to each

of x and y.

Now there are of course higher order terms, but

we've already got more than enough to play with here.

So let's look again at the first order term.

It's the sum of the product of the first derivatives with the step sizes, but

hopefully this will remind you of the discussion our Jacobian.

So we can actually re-express this as just the Jacobian multiplied by a vector

containing delta x and delta y, which we can write as delta bold x,

where the bold x signifies a vector containing those two terms.

Finally, if we look at the second order term in the same way and

notice that these second derivatives can be collected into a matrix,

which we've previously defined as our Hessian.

Then to make the sum we need,

we now have to multiply our delta x vector by the Hessian, and

then again by the transpose of the delta x vector, and that's it.

So we now have a nice compact expression for

the second order multivariate Taylor series expansion, which brings together so

much of our calculus and linear algebra skills, and makes use of the Jacobian and

Hessian concepts which we defined earlier in the course.

And of course, although we've been talking about the two dimensional case

in this video so far, we could actually have

any number of dimensions contained in our J, H, and delta x terms.

So this immediately generalises from 2D to multi-dimensional hyper surfaces,

which is pretty cool.

See you in the next one.

[MUSIC]

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2018 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取