5 - Smart Cut and Paste - Duration 07:51

Loading...

来自 Duke University 的课程

Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital

362 个评分

Duke University

Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital

362 个评分

In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.

从本节课中

Image and video inpainting

Students will get involved with a very exciting topic, since image and video inpainting is one of the most used tools in the movie industry. They will learn the problem, and also how they can approach it from multiple directions. This will also help to illustrate how the same problem can be approached from multiple mathematical angles. We will connect this with Shanon’s work providing yet another angle. If you watched the lectures on PDEs you will have more mathematical background, but you will enjoy this unit and learn without it as well.

1 - Introduction to Image Inpainting - Duration 08:168:16

2 - Inpainting in Nature - Duration 05:015:01

3 - PDEs and Inpainting - Duration 21:59 - Optional break at 13:3121:59

4 - Inpainting via Calculus of Variations - Duration 15:32 - Optional break at 10:0615:32

5 - Smart Cut and Paste - Duration 07:517:51

6 - Demo - Photoshop Inpainting Healing Brush - Duration 02:312:31

7 - Video Inpainting and Conclusions - Duration 05:135:13

与讲师见面

Guillermo Sapiro
Professor
Electrical and Computer Engineering

Hello and welcome back. Let us now introduce a different concept into

imaging painting. Basically, we're going to do a Smart Cut

and Paste. We're going to first introduce the

concept and then see how we obtain excellent results by merging that with

the type of variation and formulations that we just talked about in the previous

videos. So, why do we need an another type of

impainting? There's a number of reasons.

But one of them is in the example that we showed in the previous video.

We discussed the fact that. PDEs and variational formulations are

excellent tools for impainting smooth regions.

They're not as good for impainting texture as we see here.

That we need to basically extend this nice texture inside the region to be

impainted. And that's where this other type of

techniques become extremely powerful. Let me blow this region so we can explain

what's going on. We need now to inpaint inside here and we

see that there are plotters that we want to extend.

We could actually write partial differential equations that extend that

flow towards this region. But it would be much harder to write a

partial differential equation that extends this type of texture that goes in

multiple directions. But we have another technique.

Remember we talk about non-local means for image denoising.

We happen to exploit the same type of approach but for image impainting.

Here is the basic idea. Let us assume that we take a a patch

here. This patch is very particular.

It contains some known information and some unknown information.

Now in the same way as the non-local means, I can go around and looking for

similar patches but, in this case, I do the comparison only in the region of

known pixels. So, for example, I basically take a patch

of the same size as here but I only look at this region.

And I say is this region similar to this region and I keep going around, for

example I might take a byte here. And once again I look only at this

region. So, at least they translate the known

region towards dark patch and I keep going and I find one or more patches dark

in the known region, they're basically equal or very similar.

Let us assume that I find one, for example I find it here.

And we draw the region and we say okay, this is very similar to this here.

So how can I fill this region? I basically copy this.

The unknown becomes the known part of the matching part.

I can copy the whole region. I can copy only a few border pixels and I

can copy only from one part or I can find a number of patches that are similar and

copy their average or any other type of function.

But the concept is the same concept as in all local means.

Find similar patches but now similarity's based only on the known region.

Once you find similar patches. Copy the unknown region towards here.

When we do that, we finish and we know here.

And we can keep propagating. This now becomes known because I just

filled that in. And then we can keep going inside and

keep going inside and keep going inside. So a basic idea is that we look at every

unknown pixel. We start from pixels that have a lot of

known area surrounding it. We fill them.

And then we keep going with the other and the other pixels, until we finish

basically filling in everything. And let me go back to the previous slide.

And we basically see. That we feel nicely the texture, we also

feel nicely the texture here. And we actually have feel the texture in

a very nice fashion all around. Just by the same concept as the non-local

means. Now, that, we can actually combine with

variational formulations. What I just told you, we can write in a

variational formulation, but we can keep adding additional constraints.

For example, we could say okay, if this patch found a friend here, meaning a

patch that is very similar to it, I want to make sure that the patch next to it,

let me just use a different pen. I want to make sure.

That the bunch next to it finds a similar part next to the previous one.

So that would help me. In this search it will make it faster but

it will also force that this region is smoothly copied here and that can be

posed in a variational formulation by for example by asking.

That the translation vectors. So this part got translated from here.

This part got translated from here. We have two translation vectors.

We can force the translation vectors to change smoothly.

And that's, it's a variation in formulation, for example, we can take the

derivatives of the coordinates of the translation vectors and penalize when

those derivatives are too high. That means that basically, you're jumping

all around to get those. New patches.

All these concepts can be incorporated into one variation or formulation, can

become very rich. We can incorporate the PDEs type, or the

variation and formulation type of smooth continuation that we saw in the previous

videos. We can incorporate this Smart Cut and

Paste all in one large variation and formulation.

And then you obtain very nice results as we see here, we basically have an object

that has been removed, as we see here, by this combination of variational

formulations with Smart Cut and Paste. Here is another example and we are going

to see this example again in the next video is going to exp, as I'm going to

explain soon. Again here, remember this image looks

perfect that's because impainting, but we basically have removed this lamp from

this image and got a very, very nice result.

Here is yet another example where we basically remove the bars.

And we get a nice result by this combination of Cut and Paste in a

variational formulation. By now we have basically seen two type

of, or three type of concepts. PDEs and variational formulations, that's

almost one group because the standard variation of formulation leads to PDEs.

And then we saw this Smart Cut and Paste. And we saw how we can combine them all

together to get very, very nice image and painting results.

Before I show you some video impainting concepts, I want to demo in painting in

real time. And I'm going to do that in the next

video. Looking forward to seeing you again then.

Thank you.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2018 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取