Then once we have the bias values we want to actually take them and we want to come up with the baseline predictor. And so we just basically add the bias values in from that. And so here's a tabulation of the bias values that we just computed just in table form. The users are over here and the movies are over here. And so we'll start with user D, and movie three. And, we'll just, we'll just illustrate, just a few of these computations. Just two of them. We'll just do, for instance, user D, movie three. so, D and three. Remember we start with the, the baseline or the, the raw average value of, 3.5. Then we add in the bias of user D, which is negative 1.5, and then we add in the bias of movie three, which is 0.75. So, remember D is a harsh critic. And three is a, good, quote unquote, good movie. And when we do that out, we come up with 2.75 it's value, so it's lower than the mean, we expect it to be lower than the mean. And, the intuition there is that movie three isn't as, quote unquote, much better than the average than D is quote unquote more of a harsh critic than the average. So they don't quite cancel each other out and we do end up going below the average here for this pair, D3. let's try User A, movie two. So we have A2 and that's a value that we don't already know. For D3 we do have that value. Now we want to try A2, something that we don't know. so we do A2, we get, we start with the mean, again, of 3.5, then we add the bias of A, which is 0.83 and then we add, the bias of movie two, which is -1.17, so again, this is more negative than this is positive, so. We're going to go below the average and we get 3.16. It's here, say it's little, again, a little more below the average. And if we continue that out, we do the baseline for all the values, this is what we'll get in this table here. And just a note, for instance some of values are with this, only one value actually, is 5.00. the reason that it's not, if you did the calculation out for four you would have bias of 8.83 and movie 3 0.75. It would go above five. But we clip them at five because it's not possible to be greater than five. So therefore we just clip and that's kind of an important point. the reason we clip them at five is because we know the rating can't be greater than five so we'd only be hurting ourselves by pushing it above five. So if it goes above five then we just hit it at five. And if it goes below one, then we just assume that it's one, so, we, remember we can only be between one and five here. So, now, once you do this out, we can do the RMSE, oh and just let me say here, also, this is a five, so that's clipped and this is a one, so that's also clipped. so you can do that out. you can see from D, D2 for instance, D, that's going to go below because 2 minus 1.5 minus 1.17 is already actually a negative number, it's below 0, even. So, we just clip that at one. So now, the RMSE, we'll do the RMSE of the test set. I'm just going to do it, I'm not going to do every step out this time, I'm just going to do it all in one single step. So again, we do the square root and then we have to find the mean of the squared errors. So now we're right at the squared errors terms so we have 4 and 2.80, so this is 4 minus 2.80 squared plus, now the second one. This is 2 minus 1 squared plus in the third one we have 5 minus 5 squared. Which is just 0 because 5 minus 5 is 0. The fourth one we have 3 minus 3.5 squared. And the fifth one, it's kind of running out of room over here, but now the fifth one. We have 4 minus 3.85 squared, and we divide that by 5 because we're averaging everything. So, write out, I'm going to write out what each of these terms are when we square them in here. We get, this is 1.44. This is 1 clearly because 2 minus 1 is 1. 1 squared is just 1. We have zero, as we said. We have a 0.25, and, we have a, 0.0225. That's all divided by 5 to average it, and then we take the square root of the whole thing. And then that, when we do that out, we get 0.7365. [SOUND] Now, it's the RMSE for the test set. If we do the training set, we will get 0.4709, and we won't do that out again, because there's 20 of those terms. Which would be a lot to write on paper, but that's the idea. So now we can compare this with the RMSE that we had before. And so now we have an improvement. If we do an improvement over that 1.0247 which is what we had before, an RMSE, the test set just when we did the raw averaging we're down to 0.7365 now. So if we want to find the percent improvement over that we just do, we start at one because everything is around, starts at 100% so we're trying to find a percent improvement. So basically what we're going to do is we're going to find out what fraction we are of that right now. So 0.7365 is where we are now, and we were at 1.0247. Right. And so we do 1 minus that because we're trying to find the im- percent improvement. So the lower that we are relative to this, the better. And let's multiply that by 100%. And we get 28.13%. So, we're, we're improved almost 30% over what we had before. And then for this guy the improvement's actually more like 59 %. So you can see that again we're we did much better than we did when we just did a simple raw average. And so that somewhat explaining the utility of using this approach, but know that the percentage improvements are going to be way different depending upon what example, this is only one example. And in fact the improvement could be negative, in which case we're actually getting worse. and so it's not always guaranteed that we are going to be getting better. Especially if there's some unknown, patterns in the data. but on average we expect intuitively that incorporating this is going to make our prediction better.
