So far in this module I've introduced you to the concept of Power Series, and
then shown you how we can use them to approximate functions.
We've also seen how Power Series can then be used to give you some indication
of the size of the error, that results from using these approximations,
which is very useful when applying numerical methods.
The last idea that we're going to very briefly look at on this topic
is upgrading the Power Series from the one-dimensional version that we've seen so
far to its more general multivariate form.
Just to recap on the notational options from the previous video.
We saw that we can re-express the Taylor series from a form that emphasises
building the approximation of a function at a point p to a totally equivalent form
that emphasises using that function to evaluate other points
that are a small distance delta x away.
Just to make sure you can see that they're the same, we can write out the first four
terms, one above the other, to make the comparison clear.
In this video, we are going to continue using the delta x notation
as it's more compact, which will come in handy when we're in higher dimensions.
We won't be working through the multivariate Taylor series derivation
in any detail, as all I really want you to take away from this video is what
a multidimensional Power Series will look like, both as an equation and in a graph.
So keeping in mind our one-dimensional expression, let's start by looking at
the two dimension case, where f is now a function of the two variables x and y.
So our truncated Taylor series expressions will enable us to approximate the function
at some nearby point x plus delta x, y plus delta y.
Let's look at the simple two-dimension Gaussian function, which many of you may
be familiar with from studying the normal distribution in statistics.
It's a nice, well-behaved function with a simple maximum at the point (0,0).
Now in a one dimension analysis, our Power Series will give us an expression for
a one dimensional function.
However, in a two dimensional case our power series would of course now give us
a two dimensional function, which we would more intuitively refer to as a surface.
Just as with the one dimensional case, our zeroth order approximation
are simply the value of the function at that point applied everywhere,
which means in 2D, this would just be a flat surface.
So a zeroth order approximation at the peak would look like this.
And is zeroth order approximation somewhere on the side of the bell
would look a bit more like this.
This is fairly straightforward, but
now let's think about the first order of approximation.
drawing the analogy from the 1D case again,
the first order approximation should have a height and a gradient.
So we're still expecting a straight surface, but
this time it can be at an angle.
Now if we are calculating it at the peak, it wouldn't be a very exciting case
as it's a turning point and so the gradient is zero.
However, let's look again at the point on the side of the slope.
Taking the first order approximation here gives us a surface with the same height
and gradient at the point.
Finally, let's look at the second order approximation for these two points.
We're now expecting some kind of parabolic surface.
However, if I calculate it at the peak nothing seems to appear.
But that's just because we've created a parabola inside our bell curve, so
we need to look from underneath to even see it.
Finally, if we put up the second order approximation at a point at
the side of the bell curve,
you can see that we end up with some kind of interesting saddle function.
But it's still fairly clear even by eye that the gradient and
the curvature are matching up nicely at that point.
Although, this approximation is evidently not useful
outside of a fairly small region around the point.
We're now going to take a look at how we would write expressions for
these functions.
So if we want to build a Taylor series expansion of the two dimensional function
f at the point x, y.
And then use it to evaluate the function at the point x plus delta x,
y plus delta y, our zeroth order approximation
is just a flat surface with the same height as the function
at our expansion point.
The first order approximation incorporates the gradient information
in the two directions.
And once again, we are thinking about how rise equals the gradient times the run.
Notice here for compactness, I'm using the partial symbol with the subscript
to signify a derivative with respect to a certain variable.
If we look now at the second order approximation, we can see
that the coefficient of one-half still remains as per the one dimensional case.
But now we have three terms, all of which are second derivatives.
The first has been differentiated with respect to x twice.
The last with respect to y twice.
And the middle term has been differentiated with respect to each
of x and y.
Now there are of course higher order terms, but
we've already got more than enough to play with here.
So let's look again at the first order term.
It's the sum of the product of the first derivatives with the step sizes, but
hopefully this will remind you of the discussion our Jacobian.
So we can actually re-express this as just the Jacobian multiplied by a vector
containing delta x and delta y, which we can write as delta bold x,
where the bold x signifies a vector containing those two terms.
Finally, if we look at the second order term in the same way and
notice that these second derivatives can be collected into a matrix,
which we've previously defined as our Hessian.
Then to make the sum we need,
we now have to multiply our delta x vector by the Hessian, and
then again by the transpose of the delta x vector, and that's it.
So we now have a nice compact expression for
the second order multivariate Taylor series expansion, which brings together so
much of our calculus and linear algebra skills, and makes use of the Jacobian and
Hessian concepts which we defined earlier in the course.
And of course, although we've been talking about the two dimensional case
in this video so far, we could actually have
any number of dimensions contained in our J, H, and delta x terms.
So this immediately generalises from 2D to multi-dimensional hyper surfaces,
which is pretty cool.
See you in the next one.
[MUSIC]
