So here there are some of the things we have to worry about based on our discussion, and then how do we choose the various parameters of the k-neighbor classifier? What's the training time that's easy? Because to make sure you've chosen the right k and it's working very well, you will have to find the neighbors of every point in your training set. So it's short but it's not quadratic. It is linear in the number of points that are in your training set, because for each point you will have to find its neighbors, so for each point you live with search and find the three or four nearest neighbors. Maybe if you have an efficient way of finding that, you can do this in linear time. It could be worse but that's roughly what the time could be. What's the scoring complexity? The scoring complexity is easy. Once I found the neighbors, I just have to find the majority word. So if I have n and not nine neighbors, I just have to see which class is more, and then say this point belongs to that class. Therefore question that comes up is, having loosely using the word distance or similarity. So how do you know two points are close? How do we know they are similar? Through a different functions available, we'll see the Euclidean function just stands from one point to another would work if everything is a numerical value, but it may not work if things are categorical, then how do you measure distances? The next question that comes up, how big is the k? So first of all, if we don't use k equal to even numbers, because if I use k equal to say 6, and I get three of blue and three of pink. Now we got to toss a coin, so breaking the tie is an issue. So typically we start use odd numbers; 1, 3, 5, 7, 9. Now, if that is noisy data, which means the blues and the pinks are merged together in some random way, then the larger the value of k. It is better. As I told you that a problem of looking up because you will have to look up who the neighbors, and you have to have a cunning trick of doing this. So there are tricks which people use, which are more database, data organization tricks, which will allow you to rapidly search and find the nearest neighbors, instead of simply looking at every point and then sorting them and say which other neighbors, there are cooler ways of doing this. This method is not very good when the data is extremely noisy. So that may be in those situations, you may take the noisy data and project it down to something lower dimensionally, which filters the noise and then find the nearest neighbors on that. Finally, we never use the actual face value of the distance. So we just say this is closed, is not closed. But unlike regression, or logistic regression, or PCA, whatever you have done so far, when k-means distance is more notional, is this similar or is this not similar? So this is probably the easiest methods to explain. It works fairly well, and it's easy to use when you just want to test your algorithms and say, "Hey, I want to beat this one."
