In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.
3 - Brain Imaging, Diffusion Imaging, Deep Brain Stimulation - Duration 26:28
Loading...
来自 Duke University 的课程
Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital
415 个评分
Duke University
415 个评分
从本节课中
Medical imaging
This is a bonus unit. Enjoy it. Image processing has been very successful in medical imaging, and we will use examples from HIV and brain research to illustrate the importance of image processing in solving societal problems. We will describe the basic tools in these exciting applications, from the acquisition to the analysis.
与讲师见面
Guillermo SapiroProfessor
Electrical and Computer Engineering
Hello and welcome back to our image and video processing class.
This video, we're going to be talking about brain imaging.
Brain imaging is an extremely exciting area with applications in basic science,
try to understand our brain, how we develop,
and also application in many scene as we're going to see in this video.
We are going to start with a bit of a discussion on a relatively new modality,
which is called diffusion-weighted MRI, diffusion-weighted Magnetic Resonance
Imaging. MRI is very old and it has been a
tremendous contribution to medicine. Diffusion-weighted MRI is a, as I said, a
relatively new technique. The reason I'm going to talk about it is
because we're going to see a very surprising application of one of the
tools that we have learned, actually in the segmentation week, apply now to
diffusion imaging in a very very different scenario.
Like most of our weeks, during this class, we could just speak this topic of
medical imaging, even the topic of medical imaging for MRI and basically
talk for months about it. So, as before, I'm giving you a few
examples to provide you some motivation and some excitement about this area.
So what is diffusion-weighted imaging? The basic idea is relatively simple, the
basic concept. So, you have a brain,
actually, we're talking about brain imaging, but you can do diffusion imaging
for other organs. You go into a magnet, an MRI machine,
same type of machine, so it's kind of the same classical hardware. We're going to
just use it differently and what we're going to get out is measurements about
the brain connectivity. How is a given region in the brain
connected to a different region in the brain? Something very cool, how is this
connected to this? Is it connected at all, for example?
That's what we are trying to do with diffusion-weighted imaging and it has
been used for neurological disorders, you know, if the connections are not what we
expect, that might actually mean something.
It has been used for study, as I say, normal brain development,
and in general, it's used to study the structure of fiber bundles,
basically, connections in the brain. What is the underlying concept in
diffusion-weighted imaging? The underlying concept is the following.
Let me just run this. Let's assume, I'm going to make a drawing
here. Let's assume that you have a region in the brain that is connected to a
different region in the brain. So, grey matter, basically, and another
region of grey matter in the brain, and we assume that they're connected by
brain fiber bundles. Imagine, just cables, okay? So you have
two regions in the brain and you have a cable connecting between them.
Now, we have water and one efficient way to imaging is trying to measure the water
diffusion in the brain. Now, assume that they're trying to make
water go through the bundle in this direction.
Because the bundle is there, water will have a challenge to go and to move in
that direction. Now, if I try to make it go in this
direction, then, it would actually move in a much
easier fashion. So if there is a presence of a
connectivity in a given direction, we can actually try to push the water in
different directions, and according to how successful we are in
pushing that water, we can infer the direction of these
bundles and then basically the connectivity of different regions in the
brain. So for example, if we try to diffuse the
water, and it goes in every direction, there's basically no cables.
If we try to diffuse the water and it goes in every direction, but one,
basically there is a cable in kind of the orthogonal direction, in the
perpendicular direction. So if it goes every place, but it just
has really difficulties going here, in this direction, then something is
stopping it in this direction. So it's kind of the perpendicular
direction. Now, you can always, you can also find
crossings, basically find that more than one direction, it's difficult to make the
water go through. And in general, you can find basically
the probability of diffusion in every direction and that gives you the
probability of having connections, cables in the brain,
this cable, this bundles in the brain between different directions.
How is that done? The person goes into the magnet, and basically, there are
gradiants created in different directions.
So it's kind of you're creating magnetic fields in different directions.
So, you go around, let's say a circle and you try to create that field in this
direction to push water in that direction,
then you try to create a field in this direction to push water in that
direction. So you basically do that for multiple
directions and you measure if you can move those water molecules in that
direction, then, what you get at the end is
something that looks like this. You, you get multiple directions, and
basically, people take 32 or 64, sometimes 128 of these directions
depending how much time can you spend inside the magnet and created these
gradients in multiple directions. From that, you can infer the probability
of having a connection in a given direction, because you infer the
probability of moving basically the water in a given direction.
So you get all these multiple directions and from that, as which is said you could
have every single voxel in the image. So we are in 3D, our brain is 3D, so it's
got pixels without voxels. Basically it's kind of three-dimensional
pixels, and,
this is just a slice of a brain and we here, have here a zoom-in version of one
of the regions and then at every voxel, you get concept probability of diffusion
in a given direction or a probability of having these cables in the brain, these
bundles in the brain in a given direction.
And then, you can consider the whole, so this, I'm drawing here, the more
round, it is the more kind of isotropic uniform
probabilities. The more stretched it is, the more
anisotropic in that direction, and if you see that there is kind of a cross, then
there is anisotropy in more than one direction.
You can consider it completely or as we see here, with this red, which is, can
pick the maximal, what is actually very strong and that means that probably there
are fibers, there is cables..
I keep using the word cables, so, because it's kind of, those that
connect or make basically give the connectivity or the conductivity between
regions. So you can only pick the maxima between
those and say, hey, I believe there is a connection in this direction and maybe,
also in this direction. We're going through this vault set,
so, in this vault set, we might have connections going in two directions,
basically, strong connections going in two different directions.
Now, that's per coxel. What happen if I want to say, which regions are connected
to this region of the brain or is this region of the brain connected to this
region of the brain? So, that's basically, I want to go from
this local measurements that I get from the machine.
There is some image processing tool, actually get these denoising,
but basically, most of the work, basically, is done by these very smart
acquisition. So you're here,
and you say, I want to know the probability of connection between now far
away regions. is this region of my brain connected to
this region? How can I measure these from these data
that I have just obtained? And look what we can do,
there are many things that we can do, but here is one of them.
We can do a Hough transform. Now, we use, we learn about the Hough
transform to detect straight lines or curves.
Now, we are actually going to do a Hough transform for something slightly
different, but these are still curves.
We want to find, basically, cables in three-dimensions.
So, this is how we do it. We basically start, let's say from a
point, and we say, let us find the strong curves going through this particular
point. So you can throw, you can just start with
a few samples. You just put many curves,
you can parametrize this. You can parametrize almost any function that you
desire. One of the ways to parametrize is just to
write polynomial function and say, I want to run, I want to just use a
function that is of this style. Let me just write an a1+a2, x+a3x^2, and
so on, maybe you go up to six exponential, seventh,
and then the half is going to vote for this coefficients. Remember, Hough
transform is great when we have parametrized curves, and here, I'm
going to just parametrize a curve by a polynomial, by basically, this function
and x is just it coordinates in xyz in three-dimension, so these are kind of
vectors. Now, how do you do the voting? Very simple.
Remember, at every voxel, we have the probability of going in every direction.
So I take this curve and I say, is this a good curve?
I go through the voxel and I say, what's the probability of going at that voxel in
this direction? So basically, the tangent to the curve
and then you go to the next voxel, you know, go to the next voxel and you
keep going. So you add the probabilities that you got
from the data of the diffusion MRI. When we talk about a Hough transform, we
were kind of adding the edge content. If there is an edge, we're voting for it.
If there wasn't, we weren't voting for it.
Here, we actually do a weighted voting. We vote by the probability of going in
that direction, but it's a Hough transform.
This is, I think, is very beautiful, that we can use some very standard theory for
a very cool application in a modern image acquisation technology.
So, you go, and vote, and vote, and vote, and then you take another one, and you
vote, and you pick the curve that has the maximal vote.
So you say, this is the one that most of it or at least the majority or in a very
strong case, the tangents go in the direction that the forces tell me there's
a lot of probability, a high probability of going in that direction, a fiber or a
cable as we have been saying, then you pick another point and you do
the same, you pick another point and you do the same.
And in that way, you get these tracks you get from every point, basically, you get
this probability of having this cables and connections.
And then you say, is this point connected to a point, let's say, here?
If there is not a strong cable, a strong curve, then you say they're not
connected. You can actually make a binary system,
yes or no, or you can make a probability decision.
Oh, this is the probability that they are connected and the probability is as in
Hough transform, the addition of the local probabilities at each voxel.
In the same way that when we were detecting simple or straight lines,
we were saying what's basically the chances that this straight line had many
local edge detectors along it's way. And when you get that the end is a
beautiful image like this that gives you a lot of tracks.
It basically gives you a lot of connections between different regions in
the brain. And that, you can use for many, many
problems in basic neuroscience in, for disease, for development or for other
areas. So to recap, you're going to demand it,
you get these magnetic fields in different directions,
you compute the probability of having that cable or a fiber in a given
direction, and when you want to get information about two points, you use the
Hough transform to do a voting process and you can get that connection.
And this is not the only way to do that, it's just one way, and think is a nice
way to illustrate how some tools that work, developed for image segmentation,
basically, are also applicable in this new modality.
Now, this has many, many uses, and for example, you can use diffusion
imaging for the very exciting area of deep brain stimulation.
And I want to discuss what this deep brain stimulation and some of the image
processing challenges there. So let's do that now.
What is deep brain stimulation? Deep brain stimulation, first of all, is used
for things like Parkinson, essential tremor, and at the more experimental
level, for clinical depression, and for OCD, and other neurological
problems. The basic idea is that, as we see in this
image, an electrode is inserted in the brain and pulses are sent.
So the electrode is inserted and there's also a generator that is normally put in
the chest with a battery and that generator sense electric pulses and
basically stimulates a given region in the brain.
So it's brain surgery, the patient for most of the cases is actually awake, and
we are going to see the reason for that. So there is only local anesthesia, but
the person is actually awake during the surgery where this electrode is inserted
in the brain and we see kind of a schematic drawing here.
And I strongly recommend you to go for example in YouTube and do a search on
deep brain stimulation and you're going to see the fantastic results of
this surgery and some of the symptoms of, for example, Parkinson or essential
tremor basically stop immediately when surgery is finished.
It's a fascinating area. Now, the big challenge here is to find
the target, to find where do you want to put that
electrode? That's the big challenge here. that is normally done with the help of a
number of components, two of them being a model.
Basically, brain neurosurgeons know about the brain,
they know the literature and they have kind of a model of the different regions
in the brain and some are basically expressed here in a schematic way.
And then they use an atlas, they basically have an atlas of brains
and they say they know where those different regions are in the atlas.
And then they get some imaging for that particular patient, and then, they try to
locate those regions in the images of that particular patient using this atlas.
Now, one the challenges with the help of new acquisition techniques, a new image
processing techniques, is not to use the atlas anymore and to basically do
everything with images from that subject. The reason that the atlas is sometimes
used, as well as of course the models, is because the images and image processing
is not great and then you need to use an atlas to kind of help you navigate inside
the brain. Now, if we have better images and if we
have better image processing, we might be able and as going to show you in a
second, we are able to basically localize the target per subject and then do a
better job. Now, the way that people are looking into
advancing the brain stimulation is fascinating for image processing,
because it uses everything that you can get from an MR machine.
And of course, this is not a class on magnetic resonance imaging, so I'm just
going to name a few. We just saw a few minutes ago, diffusion
imaging and we're going to see its role in a second.
But people use different, basically, modalities inside the MR,
so the subject is in the magnetic resonance machine.
It's like, the same hardware, but you change the software a bit,
and you can get different things like T1, T2,
susceptibility weighted imaging, and of course, diffusion imaging as we just saw.
And, to try to combine all of this, what do I mean by that, you basically register
them, you align that, basically, we saw, we talk about
registration when we were talking about HIV and image processing for viruses.
So you put all of them in a coordinated system, in the same coordinated system.
You might be able to bring the atlas as well if you wish to do so, and then, when
you have all that, you basically, which is already one of the images processing
challenges, you start doing other image processing
things, like for example, image segmentation.
You might be able to automatically or semi-automatically, for example, using
active contours, segment, and mark the regions that a neurosurgeon knows are the
target regions. So you put, your data log active contours
that are specialized to this type of challenges.
And the beauty of that is that, when we talk about active contours, normal,
normally, we talk about gradients of one image.
Now you can take gradients of multiple images,
because you have multiple modalities together.
So you can, we talked about T1, T2, the susceptibility weighted image,
you can actually compute gradients with all of them together.
Remember, we talked about vectorial gradients,
so now at every voxel and I have one scalar number, I have multiple numbers,
the T1, the T2, the SWI, the susceptibility weighted imaging, and then
I compute vector gradients and I do this vector active contours to segment the
regions. You can, of course, and you get segments like these.
Here are different regions of interest. The neurosurgeon knows more or less the
regions, where they are, but this gives are very precise segmentation of the
different regions of interest to help the neurosurgeon targeting the one that he or
she knows that should be targeted and that's where the electrode should
basically be touching, so you can give kind of a future reality of the brain.
It's superimposing these segments that you get with active contours, basically,
inside a three-dimensional visualization of the brain.
You can of course, add the diffusion-weighted imaging that we saw
earlier in this video and this is represented here with gray, these are the
fibers that we computed. So the neurosurgeon can start thinking, okay, if
I touch this region because it's connected to this, then when I put
basically electric pulses there, because I inserted my, basically, my electrode
there, then that would also affect this region because they're actually
connected. So it's not only that you're giving a
good target localization, you're also telling if you touch here, what's
going to be affected by you touching there and that's the beauty of putting
all these image processing all together. And then at the end,
the neurosurgeon did a lot of planning, and basically say, okay, this is my
target because you help me localize and you help me figure out the connections.
After surgery, the subject goes into a CT, and basically, we take a picture of,
this is actually the cable that is going to be basically going to the chest
for the battery and this is the electrode that was inserted.
We have segmentations of the regions and then the neurosurgeon can basically,
after surgery, see, was I in the right place?
As I say, the subject is awake and the subject is giving feedback to the
neurosurgeon. For example, if it's a surgery for
Parkinsons, then you can basically experiment different behaviors to see if
they improve with the position of the electrode.
And for example, one is the tremor, if the tremor stops when you localize the
electrode in a certain place and start sending certain electric pulses, then you
say, it looks like I'm in a good region there.
But to work even further, you can, after surgery visual, visualize everything that
happen, and here is actually real data, these are segmentations that were
obtained by type of active contours, type of techniques with all these modalities.
These are the electrodes, so you go and register one more modality
which is the CT from after surgery and then the neurosurgeon can say, looks like
I was where I wanted to be, that's great. I actually saw that during surgery,
because the subject was getting better, and then, electrodes actually have
multiple contacts, some have four, some new ones coming to
the market have 16. And then this post-op images can help the
programmer to say, which one of the contacts would be more effective
according to the position that I have localized there?
So a lot of image processing, we do registration from the different
modalities, we do segmentation,
and that helps the neurosurgeon to localize himself or herself inside the
brain of the subject to better do the surgery,
of course, doing it safer as well. And, in the same fashion, that is maybe
after surgery to do a better programming and understanding what was the actual
outcome of the surgery. Once again, deep brain stimulation is
extremely exciting and I strongly recommend you, if you just search, for
example, in YouTube, you're going to see actual patients that talk about how their
life became much better after deep brain stimulation.
From our perspect of image processing, tremendous challenges and, and I say, as
I said earlier, basic are the contributions to humanity are great.
So we're doing very very challenging image processing and the outcome is
absolutely outstanding and it's hard to ask for something better than that. I
hope you enjoyed this short video that shows you a few of the challenges in
brain imaging, which is as I say, one of the very active areas in medical imaging.
I'm looking forward to see new in the next video.
Thank you very much.
