Hello and welcome back to our image and video processing class. This video, we're going to be talking about brain imaging. Brain imaging is an extremely exciting area with applications in basic science, try to understand our brain, how we develop, and also application in many scene as we're going to see in this video. We are going to start with a bit of a discussion on a relatively new modality, which is called diffusion-weighted MRI, diffusion-weighted Magnetic Resonance Imaging. MRI is very old and it has been a tremendous contribution to medicine. Diffusion-weighted MRI is a, as I said, a relatively new technique. The reason I'm going to talk about it is because we're going to see a very surprising application of one of the tools that we have learned, actually in the segmentation week, apply now to diffusion imaging in a very very different scenario. Like most of our weeks, during this class, we could just speak this topic of medical imaging, even the topic of medical imaging for MRI and basically talk for months about it. So, as before, I'm giving you a few examples to provide you some motivation and some excitement about this area. So what is diffusion-weighted imaging? The basic idea is relatively simple, the basic concept. So, you have a brain, actually, we're talking about brain imaging, but you can do diffusion imaging for other organs. You go into a magnet, an MRI machine, same type of machine, so it's kind of the same classical hardware. We're going to just use it differently and what we're going to get out is measurements about the brain connectivity. How is a given region in the brain connected to a different region in the brain? Something very cool, how is this connected to this? Is it connected at all, for example? That's what we are trying to do with diffusion-weighted imaging and it has been used for neurological disorders, you know, if the connections are not what we expect, that might actually mean something. It has been used for study, as I say, normal brain development, and in general, it's used to study the structure of fiber bundles, basically, connections in the brain. What is the underlying concept in diffusion-weighted imaging? The underlying concept is the following. Let me just run this. Let's assume, I'm going to make a drawing here. Let's assume that you have a region in the brain that is connected to a different region in the brain. So, grey matter, basically, and another region of grey matter in the brain, and we assume that they're connected by brain fiber bundles. Imagine, just cables, okay? So you have two regions in the brain and you have a cable connecting between them. Now, we have water and one efficient way to imaging is trying to measure the water diffusion in the brain. Now, assume that they're trying to make water go through the bundle in this direction. Because the bundle is there, water will have a challenge to go and to move in that direction. Now, if I try to make it go in this direction, then, it would actually move in a much easier fashion. So if there is a presence of a connectivity in a given direction, we can actually try to push the water in different directions, and according to how successful we are in pushing that water, we can infer the direction of these bundles and then basically the connectivity of different regions in the brain. So for example, if we try to diffuse the water, and it goes in every direction, there's basically no cables. If we try to diffuse the water and it goes in every direction, but one, basically there is a cable in kind of the orthogonal direction, in the perpendicular direction. So if it goes every place, but it just has really difficulties going here, in this direction, then something is stopping it in this direction. So it's kind of the perpendicular direction. Now, you can always, you can also find crossings, basically find that more than one direction, it's difficult to make the water go through. And in general, you can find basically the probability of diffusion in every direction and that gives you the probability of having connections, cables in the brain, this cable, this bundles in the brain between different directions. How is that done? The person goes into the magnet, and basically, there are gradiants created in different directions. So it's kind of you're creating magnetic fields in different directions. So, you go around, let's say a circle and you try to create that field in this direction to push water in that direction, then you try to create a field in this direction to push water in that direction. So you basically do that for multiple directions and you measure if you can move those water molecules in that direction, then, what you get at the end is something that looks like this. You, you get multiple directions, and basically, people take 32 or 64, sometimes 128 of these directions depending how much time can you spend inside the magnet and created these gradients in multiple directions. From that, you can infer the probability of having a connection in a given direction, because you infer the probability of moving basically the water in a given direction. So you get all these multiple directions and from that, as which is said you could have every single voxel in the image. So we are in 3D, our brain is 3D, so it's got pixels without voxels. Basically it's kind of three-dimensional pixels, and, this is just a slice of a brain and we here, have here a zoom-in version of one of the regions and then at every voxel, you get concept probability of diffusion in a given direction or a probability of having these cables in the brain, these bundles in the brain in a given direction. And then, you can consider the whole, so this, I'm drawing here, the more round, it is the more kind of isotropic uniform probabilities. The more stretched it is, the more anisotropic in that direction, and if you see that there is kind of a cross, then there is anisotropy in more than one direction. You can consider it completely or as we see here, with this red, which is, can pick the maximal, what is actually very strong and that means that probably there are fibers, there is cables.. I keep using the word cables, so, because it's kind of, those that connect or make basically give the connectivity or the conductivity between regions. So you can only pick the maxima between those and say, hey, I believe there is a connection in this direction and maybe, also in this direction. We're going through this vault set, so, in this vault set, we might have connections going in two directions, basically, strong connections going in two different directions. Now, that's per coxel. What happen if I want to say, which regions are connected to this region of the brain or is this region of the brain connected to this region of the brain? So, that's basically, I want to go from this local measurements that I get from the machine. There is some image processing tool, actually get these denoising, but basically, most of the work, basically, is done by these very smart acquisition. So you're here, and you say, I want to know the probability of connection between now far away regions. is this region of my brain connected to this region? How can I measure these from these data that I have just obtained? And look what we can do, there are many things that we can do, but here is one of them. We can do a Hough transform. Now, we use, we learn about the Hough transform to detect straight lines or curves. Now, we are actually going to do a Hough transform for something slightly different, but these are still curves. We want to find, basically, cables in three-dimensions. So, this is how we do it. We basically start, let's say from a point, and we say, let us find the strong curves going through this particular point. So you can throw, you can just start with a few samples. You just put many curves, you can parametrize this. You can parametrize almost any function that you desire. One of the ways to parametrize is just to write polynomial function and say, I want to run, I want to just use a function that is of this style. Let me just write an a1+a2, x+a3x^2, and so on, maybe you go up to six exponential, seventh, and then the half is going to vote for this coefficients. Remember, Hough transform is great when we have parametrized curves, and here, I'm going to just parametrize a curve by a polynomial, by basically, this function and x is just it coordinates in xyz in three-dimension, so these are kind of vectors. Now, how do you do the voting? Very simple. Remember, at every voxel, we have the probability of going in every direction. So I take this curve and I say, is this a good curve? I go through the voxel and I say, what's the probability of going at that voxel in this direction? So basically, the tangent to the curve and then you go to the next voxel, you know, go to the next voxel and you keep going. So you add the probabilities that you got from the data of the diffusion MRI. When we talk about a Hough transform, we were kind of adding the edge content. If there is an edge, we're voting for it. If there wasn't, we weren't voting for it. Here, we actually do a weighted voting. We vote by the probability of going in that direction, but it's a Hough transform. This is, I think, is very beautiful, that we can use some very standard theory for a very cool application in a modern image acquisation technology. So, you go, and vote, and vote, and vote, and then you take another one, and you vote, and you pick the curve that has the maximal vote. So you say, this is the one that most of it or at least the majority or in a very strong case, the tangents go in the direction that the forces tell me there's a lot of probability, a high probability of going in that direction, a fiber or a cable as we have been saying, then you pick another point and you do the same, you pick another point and you do the same. And in that way, you get these tracks you get from every point, basically, you get this probability of having this cables and connections. And then you say, is this point connected to a point, let's say, here? If there is not a strong cable, a strong curve, then you say they're not connected. You can actually make a binary system, yes or no, or you can make a probability decision. Oh, this is the probability that they are connected and the probability is as in Hough transform, the addition of the local probabilities at each voxel. In the same way that when we were detecting simple or straight lines, we were saying what's basically the chances that this straight line had many local edge detectors along it's way. And when you get that the end is a beautiful image like this that gives you a lot of tracks. It basically gives you a lot of connections between different regions in the brain. And that, you can use for many, many problems in basic neuroscience in, for disease, for development or for other areas. So to recap, you're going to demand it, you get these magnetic fields in different directions, you compute the probability of having that cable or a fiber in a given direction, and when you want to get information about two points, you use the Hough transform to do a voting process and you can get that connection. And this is not the only way to do that, it's just one way, and think is a nice way to illustrate how some tools that work, developed for image segmentation, basically, are also applicable in this new modality. Now, this has many, many uses, and for example, you can use diffusion imaging for the very exciting area of deep brain stimulation. And I want to discuss what this deep brain stimulation and some of the image processing challenges there. So let's do that now. What is deep brain stimulation? Deep brain stimulation, first of all, is used for things like Parkinson, essential tremor, and at the more experimental level, for clinical depression, and for OCD, and other neurological problems. The basic idea is that, as we see in this image, an electrode is inserted in the brain and pulses are sent. So the electrode is inserted and there's also a generator that is normally put in the chest with a battery and that generator sense electric pulses and basically stimulates a given region in the brain. So it's brain surgery, the patient for most of the cases is actually awake, and we are going to see the reason for that. So there is only local anesthesia, but the person is actually awake during the surgery where this electrode is inserted in the brain and we see kind of a schematic drawing here. And I strongly recommend you to go for example in YouTube and do a search on deep brain stimulation and you're going to see the fantastic results of this surgery and some of the symptoms of, for example, Parkinson or essential tremor basically stop immediately when surgery is finished. It's a fascinating area. Now, the big challenge here is to find the target, to find where do you want to put that electrode? That's the big challenge here. that is normally done with the help of a number of components, two of them being a model. Basically, brain neurosurgeons know about the brain, they know the literature and they have kind of a model of the different regions in the brain and some are basically expressed here in a schematic way. And then they use an atlas, they basically have an atlas of brains and they say they know where those different regions are in the atlas. And then they get some imaging for that particular patient, and then, they try to locate those regions in the images of that particular patient using this atlas. Now, one the challenges with the help of new acquisition techniques, a new image processing techniques, is not to use the atlas anymore and to basically do everything with images from that subject. The reason that the atlas is sometimes used, as well as of course the models, is because the images and image processing is not great and then you need to use an atlas to kind of help you navigate inside the brain. Now, if we have better images and if we have better image processing, we might be able and as going to show you in a second, we are able to basically localize the target per subject and then do a better job. Now, the way that people are looking into advancing the brain stimulation is fascinating for image processing, because it uses everything that you can get from an MR machine. And of course, this is not a class on magnetic resonance imaging, so I'm just going to name a few. We just saw a few minutes ago, diffusion imaging and we're going to see its role in a second. But people use different, basically, modalities inside the MR, so the subject is in the magnetic resonance machine. It's like, the same hardware, but you change the software a bit, and you can get different things like T1, T2, susceptibility weighted imaging, and of course, diffusion imaging as we just saw. And, to try to combine all of this, what do I mean by that, you basically register them, you align that, basically, we saw, we talk about registration when we were talking about HIV and image processing for viruses. So you put all of them in a coordinated system, in the same coordinated system. You might be able to bring the atlas as well if you wish to do so, and then, when you have all that, you basically, which is already one of the images processing challenges, you start doing other image processing things, like for example, image segmentation. You might be able to automatically or semi-automatically, for example, using active contours, segment, and mark the regions that a neurosurgeon knows are the target regions. So you put, your data log active contours that are specialized to this type of challenges. And the beauty of that is that, when we talk about active contours, normal, normally, we talk about gradients of one image. Now you can take gradients of multiple images, because you have multiple modalities together. So you can, we talked about T1, T2, the susceptibility weighted image, you can actually compute gradients with all of them together. Remember, we talked about vectorial gradients, so now at every voxel and I have one scalar number, I have multiple numbers, the T1, the T2, the SWI, the susceptibility weighted imaging, and then I compute vector gradients and I do this vector active contours to segment the regions. You can, of course, and you get segments like these. Here are different regions of interest. The neurosurgeon knows more or less the regions, where they are, but this gives are very precise segmentation of the different regions of interest to help the neurosurgeon targeting the one that he or she knows that should be targeted and that's where the electrode should basically be touching, so you can give kind of a future reality of the brain. It's superimposing these segments that you get with active contours, basically, inside a three-dimensional visualization of the brain. You can of course, add the diffusion-weighted imaging that we saw earlier in this video and this is represented here with gray, these are the fibers that we computed. So the neurosurgeon can start thinking, okay, if I touch this region because it's connected to this, then when I put basically electric pulses there, because I inserted my, basically, my electrode there, then that would also affect this region because they're actually connected. So it's not only that you're giving a good target localization, you're also telling if you touch here, what's going to be affected by you touching there and that's the beauty of putting all these image processing all together. And then at the end, the neurosurgeon did a lot of planning, and basically say, okay, this is my target because you help me localize and you help me figure out the connections. After surgery, the subject goes into a CT, and basically, we take a picture of, this is actually the cable that is going to be basically going to the chest for the battery and this is the electrode that was inserted. We have segmentations of the regions and then the neurosurgeon can basically, after surgery, see, was I in the right place? As I say, the subject is awake and the subject is giving feedback to the neurosurgeon. For example, if it's a surgery for Parkinsons, then you can basically experiment different behaviors to see if they improve with the position of the electrode. And for example, one is the tremor, if the tremor stops when you localize the electrode in a certain place and start sending certain electric pulses, then you say, it looks like I'm in a good region there. But to work even further, you can, after surgery visual, visualize everything that happen, and here is actually real data, these are segmentations that were obtained by type of active contours, type of techniques with all these modalities. These are the electrodes, so you go and register one more modality which is the CT from after surgery and then the neurosurgeon can say, looks like I was where I wanted to be, that's great. I actually saw that during surgery, because the subject was getting better, and then, electrodes actually have multiple contacts, some have four, some new ones coming to the market have 16. And then this post-op images can help the programmer to say, which one of the contacts would be more effective according to the position that I have localized there? So a lot of image processing, we do registration from the different modalities, we do segmentation, and that helps the neurosurgeon to localize himself or herself inside the brain of the subject to better do the surgery, of course, doing it safer as well. And, in the same fashion, that is maybe after surgery to do a better programming and understanding what was the actual outcome of the surgery. Once again, deep brain stimulation is extremely exciting and I strongly recommend you, if you just search, for example, in YouTube, you're going to see actual patients that talk about how their life became much better after deep brain stimulation. From our perspect of image processing, tremendous challenges and, and I say, as I said earlier, basic are the contributions to humanity are great. So we're doing very very challenging image processing and the outcome is absolutely outstanding and it's hard to ask for something better than that. I hope you enjoyed this short video that shows you a few of the challenges in brain imaging, which is as I say, one of the very active areas in medical imaging. I'm looking forward to see new in the next video. Thank you very much.
