here. This patch is very particular.
It contains some known information and some unknown information.
Now in the same way as the non-local means, I can go around and looking for
similar patches but, in this case, I do the comparison only in the region of
known pixels. So, for example, I basically take a patch
of the same size as here but I only look at this region.
And I say is this region similar to this region and I keep going around, for
example I might take a byte here. And once again I look only at this
region. So, at least they translate the known
region towards dark patch and I keep going and I find one or more patches dark
in the known region, they're basically equal or very similar.
Let us assume that I find one, for example I find it here.
And we draw the region and we say okay, this is very similar to this here.
So how can I fill this region? I basically copy this.
The unknown becomes the known part of the matching part.
I can copy the whole region. I can copy only a few border pixels and I
can copy only from one part or I can find a number of patches that are similar and
copy their average or any other type of function.
But the concept is the same concept as in all local means.
Find similar patches but now similarity's based only on the known region.
Once you find similar patches. Copy the unknown region towards here.
When we do that, we finish and we know here.
And we can keep propagating. This now becomes known because I just
filled that in. And then we can keep going inside and
keep going inside and keep going inside. So a basic idea is that we look at every
unknown pixel. We start from pixels that have a lot of
known area surrounding it. We fill them.
And then we keep going with the other and the other pixels, until we finish
basically filling in everything. And let me go back to the previous slide.
And we basically see. That we feel nicely the texture, we also
feel nicely the texture here. And we actually have feel the texture in
a very nice fashion all around. Just by the same concept as the non-local
means. Now, that, we can actually combine with
variational formulations. What I just told you, we can write in a
variational formulation, but we can keep adding additional constraints.
For example, we could say okay, if this patch found a friend here, meaning a
patch that is very similar to it, I want to make sure that the patch next to it,
let me just use a different pen. I want to make sure.
That the bunch next to it finds a similar part next to the previous one.
So that would help me. In this search it will make it faster but
it will also force that this region is smoothly copied here and that can be
posed in a variational formulation by for example by asking.
That the translation vectors. So this part got translated from here.
This part got translated from here. We have two translation vectors.
We can force the translation vectors to change smoothly.
And that's, it's a variation in formulation, for example, we can take the
derivatives of the coordinates of the translation vectors and penalize when
those derivatives are too high. That means that basically, you're jumping
all around to get those. New patches.
All these concepts can be incorporated into one variation or formulation, can
become very rich. We can incorporate the PDEs type, or the
variation and formulation type of smooth continuation that we saw in the previous
videos. We can incorporate this Smart Cut and
Paste all in one large variation and formulation.
And then you obtain very nice results as we see here, we basically have an object
that has been removed, as we see here, by this combination of variational
formulations with Smart Cut and Paste. Here is another example and we are going
to see this example again in the next video is going to exp, as I'm going to
explain soon. Again here, remember this image looks
perfect that's because impainting, but we basically have removed this lamp from
this image and got a very, very nice result.
Here is yet another example where we basically remove the bars.
And we get a nice result by this combination of Cut and Paste in a
variational formulation. By now we have basically seen two type
of, or three type of concepts. PDEs and variational formulations, that's
almost one group because the standard variation of formulation leads to PDEs.
And then we saw this Smart Cut and Paste. And we saw how we can combine them all
together to get very, very nice image and painting results.
Before I show you some video impainting concepts, I want to demo in painting in
real time. And I'm going to do that in the next
video. Looking forward to seeing you again then.
Thank you.
Guillermo SapiroProfessor
