And when we do that out, we come up with 2.75 it's value, so it's lower than the
mean, we expect it to be lower than the mean.
And, the intuition there is that movie three isn't as, quote unquote, much
better than the average than D is quote unquote more of a harsh critic than the
average. So they don't quite cancel each other out
and we do end up going below the average here for this pair, D3.
let's try User A, movie two. So we have A2 and that's a value that we
don't already know. For D3 we do have that value.
Now we want to try A2, something that we don't know.
so we do A2, we get, we start with the mean, again, of 3.5, then we add the bias
of A, which is 0.83 and then we add, the bias of movie two, which is -1.17, so
again, this is more negative than this is positive, so.
We're going to go below the average and we get 3.16.
It's here, say it's little, again, a little more below the average.
And if we continue that out, we do the baseline for all the values, this is what
we'll get in this table here. And just a note, for instance some of
values are with this, only one value actually, is 5.00.
the reason that it's not, if you did the calculation out for four you would have
bias of 8.83 and movie 3 0.75. It would go above five.
But we clip them at five because it's not possible to be greater than five.
So therefore we just clip and that's kind of an important point.
the reason we clip them at five is because we know the rating can't be
greater than five so we'd only be hurting ourselves by pushing it above five.
So if it goes above five then we just hit it at five.
And if it goes below one, then we just assume that it's one, so, we, remember we
can only be between one and five here. So, now, once you do this out, we can do
the RMSE, oh and just let me say here, also, this is a five, so that's clipped
and this is a one, so that's also clipped.
so you can do that out. you can see from D, D2 for instance, D,
that's going to go below because 2 minus 1.5 minus 1.17 is already actually a
negative number, it's below 0, even. So, we just clip that at one.
So now, the RMSE, we'll do the RMSE of the test set.
I'm just going to do it, I'm not going to do every step out this time, I'm just
going to do it all in one single step. So again, we do the square root and then
we have to find the mean of the squared errors.
So now we're right at the squared errors terms so we have 4 and 2.80, so this is 4
minus 2.80 squared plus, now the second one.
This is 2 minus 1 squared plus in the third one we have 5 minus 5 squared.
Which is just 0 because 5 minus 5 is 0. The fourth one we have 3 minus 3.5
squared. And the fifth one, it's kind of running
out of room over here, but now the fifth one.
We have 4 minus 3.85 squared, and we divide that by 5 because we're averaging
everything. So, write out, I'm going to write out
what each of these terms are when we square them in here.
We get, this is 1.44. This is 1 clearly because 2 minus 1 is 1.
1 squared is just 1. We have zero, as we said.
We have a 0.25, and, we have a, 0.0225. That's all divided by 5 to average it,
and then we take the square root of the whole thing.
And then that, when we do that out, we get 0.7365.
[SOUND] Now, it's the RMSE for the test set.
If we do the training set, we will get 0.4709, and we won't do that out again,
because there's 20 of those terms. Which would be a lot to write on paper,
but that's the idea. So now we can compare this with the RMSE
that we had before. And so now we have an improvement.
If we do an improvement over that 1.0247 which is what we had before, an RMSE, the
test set just when we did the raw averaging we're down to 0.7365 now.
So if we want to find the percent improvement over that we just do, we
start at one because everything is around, starts at 100% so we're trying to
find a percent improvement. So basically what we're going to do is
we're going to find out what fraction we are of that right now.
So 0.7365 is where we are now, and we were at 1.0247.
Right. And so we do 1 minus that because we're
trying to find the im- percent improvement.
So the lower that we are relative to this, the better.
And let's multiply that by 100%. And we get 28.13%.
So, we're, we're improved almost 30% over what we had before.
And then for this guy the improvement's actually more like 59 %.
So you can see that again we're we did much better than we did when we just did
a simple raw average. And so that somewhat explaining the
utility of using this approach, but know that the percentage improvements are
going to be way different depending upon what example, this is only one example.
And in fact the improvement could be negative, in which case we're actually
getting worse. and so it's not always guaranteed that we
are going to be getting better. Especially if there's some unknown,
patterns in the data. but on average we expect intuitively that
incorporating this is going to make our prediction better.
Christopher BrintonLecturer
Mung ChiangProfessor
