Hello and welcome back. Let us now introduce a different concept into imaging painting. Basically, we're going to do a Smart Cut and Paste. We're going to first introduce the concept and then see how we obtain excellent results by merging that with the type of variation and formulations that we just talked about in the previous videos. So, why do we need an another type of impainting? There's a number of reasons. But one of them is in the example that we showed in the previous video. We discussed the fact that. PDEs and variational formulations are excellent tools for impainting smooth regions. They're not as good for impainting texture as we see here. That we need to basically extend this nice texture inside the region to be impainted. And that's where this other type of techniques become extremely powerful. Let me blow this region so we can explain what's going on. We need now to inpaint inside here and we see that there are plotters that we want to extend. We could actually write partial differential equations that extend that flow towards this region. But it would be much harder to write a partial differential equation that extends this type of texture that goes in multiple directions. But we have another technique. Remember we talk about non-local means for image denoising. We happen to exploit the same type of approach but for image impainting. Here is the basic idea. Let us assume that we take a a patch here. This patch is very particular. It contains some known information and some unknown information. Now in the same way as the non-local means, I can go around and looking for similar patches but, in this case, I do the comparison only in the region of known pixels. So, for example, I basically take a patch of the same size as here but I only look at this region. And I say is this region similar to this region and I keep going around, for example I might take a byte here. And once again I look only at this region. So, at least they translate the known region towards dark patch and I keep going and I find one or more patches dark in the known region, they're basically equal or very similar. Let us assume that I find one, for example I find it here. And we draw the region and we say okay, this is very similar to this here. So how can I fill this region? I basically copy this. The unknown becomes the known part of the matching part. I can copy the whole region. I can copy only a few border pixels and I can copy only from one part or I can find a number of patches that are similar and copy their average or any other type of function. But the concept is the same concept as in all local means. Find similar patches but now similarity's based only on the known region. Once you find similar patches. Copy the unknown region towards here. When we do that, we finish and we know here. And we can keep propagating. This now becomes known because I just filled that in. And then we can keep going inside and keep going inside and keep going inside. So a basic idea is that we look at every unknown pixel. We start from pixels that have a lot of known area surrounding it. We fill them. And then we keep going with the other and the other pixels, until we finish basically filling in everything. And let me go back to the previous slide. And we basically see. That we feel nicely the texture, we also feel nicely the texture here. And we actually have feel the texture in a very nice fashion all around. Just by the same concept as the non-local means. Now, that, we can actually combine with variational formulations. What I just told you, we can write in a variational formulation, but we can keep adding additional constraints. For example, we could say okay, if this patch found a friend here, meaning a patch that is very similar to it, I want to make sure that the patch next to it, let me just use a different pen. I want to make sure. That the bunch next to it finds a similar part next to the previous one. So that would help me. In this search it will make it faster but it will also force that this region is smoothly copied here and that can be posed in a variational formulation by for example by asking. That the translation vectors. So this part got translated from here. This part got translated from here. We have two translation vectors. We can force the translation vectors to change smoothly. And that's, it's a variation in formulation, for example, we can take the derivatives of the coordinates of the translation vectors and penalize when those derivatives are too high. That means that basically, you're jumping all around to get those. New patches. All these concepts can be incorporated into one variation or formulation, can become very rich. We can incorporate the PDEs type, or the variation and formulation type of smooth continuation that we saw in the previous videos. We can incorporate this Smart Cut and Paste all in one large variation and formulation. And then you obtain very nice results as we see here, we basically have an object that has been removed, as we see here, by this combination of variational formulations with Smart Cut and Paste. Here is another example and we are going to see this example again in the next video is going to exp, as I'm going to explain soon. Again here, remember this image looks perfect that's because impainting, but we basically have removed this lamp from this image and got a very, very nice result. Here is yet another example where we basically remove the bars. And we get a nice result by this combination of Cut and Paste in a variational formulation. By now we have basically seen two type of, or three type of concepts. PDEs and variational formulations, that's almost one group because the standard variation of formulation leads to PDEs. And then we saw this Smart Cut and Paste. And we saw how we can combine them all together to get very, very nice image and painting results. Before I show you some video impainting concepts, I want to demo in painting in real time. And I'm going to do that in the next video. Looking forward to seeing you again then. Thank you.
