Deep learning added a huge boost to the already rapidly developing field of computer vision. With deep learning, a lot of new applications of computer vision techniques have been introduced and are now becoming parts of our everyday lives. These include face recognition and indexing, photo stylization or machine vision in self-driving cars. The goal of this course is to introduce students to computer vision, starting from basics and then turning to more modern deep learning models. We will cover both image and video recognition, including image classification and annotation, object recognition and image search, various object detection techniques, motion estimation, object tracking in video, human action recognition, and finally image stylization, editing and new image generation. In course project, students will learn how to build face recognition and manipulation system to understand the internal mechanics of this technology, probably the most renown and oftenly demonstrated in movies and TV-shows example of computer vision and AI.
Color models
Loading...
来自 National Research University Higher School of Economics 的课程
Deep Learning in Computer Vision
59 个评分
National Research University Higher School of Economics
59 个评分
课程 5（共 7 门，Specialization Advanced Machine Learning）
从本节课中
Introduction to image processing and computer vision
Welcome to the "Deep Learning for Computer Vision“ course! In the first introductory week, you'll learn about the purpose of computer vision, digital images, and operations that can be applied to them, like brightness and contrast correction, convolution and linear filtering. These simple image processing methods solve as building blocks for all the deep learning employed in the field of computer vision. Let’s get started!
与讲师见面
Anton KonushinSenior Lecturer
HSE Faculty of Computer Science
Alexey ArtemovSenior Lecturer
HSE Faculty of Computer Science
Now we know what grayscale digital images is.
The next question. How can we obtain and represent color images. What is color?
Color is a psychological property of
our visual experience when we look at objects and lights,
and it is not a physical property of those objects and lights.
Color is the result of interaction between
physical light in the environment and our human visual system.
Visible light is an electromagnetic radiation in the range
from 380 nanometers to 780 nanometers.
Our eyes are sensitive to electromagnetic radiation in this range
and are insensitive to look for magnetic radiation outside this range.
The reason why it has happened so probably lies in the fact that almost half of
the solar energy that reaches Earth is transferred by light in this range.
Almost the same amount falls to the infrared light,
and the rest falls to ultraviolet radiation.
To describe the source of light,
we need to build that spectrum.
To do it, we need at each wavelength in the range of
visible light to measure the amount of metered energy per time unit.
The amount of metered energy can be measured for example
in number of photons emitted per millisecond.
When light reaches the eye,
photoreceptor cells in the retina reacts to it.
Each rod and cone acts as a filter on the spectrum.
To get the output of this filter we multiply
its response curve by the spectrum and integrate over all wavelengths.
Each photoreceptor cell yields one number.
There are three types of cones with different responses.
They can be described by the modes of their responses as long,
medium, and short cones.
So in our eyes they represent spectrum with three numbers from three types of cones.
How can you represent a full spectrum with only three numbers?
Of course we can't, and most information in the signal is lost.
So two very different spectra may appear indistinguishable to a human observer.
On the slide, I give two examples of such spectrums,
which look like a purple color.
You can see from the curves that they are vastly different.
One curve is very smooth and one has a lot of peaks.
But to the human observer,
these two very different light sources produce the same color sensation.
Such spectra that produce the same color spectra are called metamers,
though these two examples are two metamers of purple color.
Let's select three primary colors P1, P2, P3.
We can try to represent all other colors with linear combination of these primary colors.
Such primary colors are called primaries.
And the weights that they used to represent other colors can
be color coordinates in this color model on color coordinate system.
There is a theory called trichromatic color theory that says that we can
go get all visible colors by combining just those three primaries.
To verify this theory,
you can make color matching experiments.
In these color matching experiments,
the human observer is presented these different lights.
On one part of the screen,
you see a light spot from the test light.
On the other part of the screen,
you see a spot that is combined by three primary light sources.
Human observer can change their relative power of these power sources.
The goal of the observer is,
by changing relative powers,
to match the colors of test light to the color of the combination of primary lights.
We can do the same experiments again and again with different observers and compare
the powers that are used by different subjects to match the same test light,
and then we can average the results of them.
Based on these color matching experiments a set of heuristic laws,
which are called the Grassman laws, are derived.
The basic idea behind the Grassman laws is that color matching appears to be linear.
The following properties of color matching are violet.
If two test lights can be matched with the same set of weights,
then these two test lights can match each other. The second property.
If we mix two test lights,
then mixing the matches will result in matching the result of mixing two test lights.
And if we scale the test lights,
then the matches gets scaled by the same amount.
Based on these laws,
we can obtain different linear color models.
It's enough to select three primary sources,
and then we can obtain all other colors by mixing these primary sources.
For example we can select cone perception modes as primaries.
These modes correspond to red,
green and blue colors.
So such model will be called RGB model.
In 1931, International Commission of illumination proposed an CIE RGB 1931 color model.
In this model, primaries are three monochromatic lights.
If you want to use this model in computer display,
they will correspond to the three types of phosphors in the computer display.
We can visualize all colors that can be represented by
RGB model with a 3D cube aligned to this coordinate system.
The origin 0, 0,
0 corresponds to black color.
The primaries are located on the three corners of the cube along the coordinate axis.
The corner which is opposite to black with coordinates 1,
1, 1 corresponds to white color.
Three other corners, correspond to a combination of just two primary colors,
and they look like blue,
magenta, and yellow colors.
The next question is,
can these RGB model represent all colors that are visible to a human observer.
To verify this, we can continue color matching experiments to match
RGB primaries to single-wavelength light sources in the range of visible light.
The weights from these experiments will provide us matching functions.
For each wavelength, we can get
three weights that are used to match the single-wavelength light.
We average the result of the matching functions of many observers,
and during this experiment we will find that
sometimes it's impossible to match test color with available primaries.
To solve this problem, the primary color needed to be added to the test source,
rather, to the basic primaries.
To match color in this example,
we need to add some amount of P2 primary to the test color side.
This correspond to the negative weight.
If you need to add some primary color to the test source,
then we'll write this down as negative coordinate of
this primary in color matching functions.
So, the result of this experiment is that RGB model can't represent all visible colors.
Substractive matching is required for some wavelengths.
The other problem is RGB model that it is not very intuitive.
Intuitively, we can describe visible light usually
using intensity of light or brightness as a measure of strength of the light,
and color or chromaticity.
Sometimes, chromaticity can we break down to the hue and saturation factor.
And in RGB model,
we have no specific chromaticity information,
we have only weights of the primary colors,
and there is no brightness information.
To solve this problem, in 1931,
together with standard RGB model,
another model, which is called XYZ was proposed.
The goal was to get a linear additive model
XYZ where component Y correspond to perceived brightness.
This model should cover all visible colors,
and all matching function should be everywhere positive.
So, there is no such thing for XYZ model as substractive matching.
X and Z components in this model capture chromaticity component of light.
And points (zero, zero, one),
(zero, one, zero), and (one,
zero, zero) will be imaginary primaries.
All values of X, Y,
and Z are from zero to infinity.
Dimension theorems for XYZ are shown in this slide.
To visualize all possible colors using XYZ model,
we will use normalized values X and Y,
where X is ratio of X to the sum of XYZ,
and Y is ratio of capital Y to the sum of XYZ.
By varying X and Y in the range from zero to one,
we can build color diagram or called color gumut.
From this color gumut,
we can make several observations.
First observation is that single wavelength light sources
are placed on the bounding curve of this gumut.
Bottom line segment of purple colors correspond to
colors that are unachievable by single wavelength light sources.
So there are some colors that human can only see you if
they're produced by a light source,
not a laser but some range wavelengths.
And from the shape of the gumut,
we can see that no three real primaries can cover all visible colors.
So, all linear models will either have
imaginary primaries or they will not cover all possibly visible colors.
On the slide, I display several variants of RGB model.
The main difference between the models is the selection of primaries.
You can see the difference in color presentation between different models.
Depending on the selected primaries,
the model cover different areas in color gumut.
There is also one additional important property of light perception.
It is non-uniformity of intensity perception.
On the slide, there are two samples.
First, is physically uniform brightness samples,
and the second is perceptually uniform brightness samples.
For the physically uniform brightness samples,
you can see that the difference between
black and dark gray is very significant for the human observer.
It's much more significant than the difference between the bright samples on the right.
From this, we can make
the following observation that it is ineffective to store a brightness linearly
because our human eyes distinguish differences in
dark areas much better than difference in bright areas.
So we can use gamma transformation and store gamma-corrected values
instead of linear RGB values.
To store gamma-corrected values is the main idea behind the sRGB model.
sRGB model is a standard RGB model used in high definition television,
digital cameras, display, et cetera.
To obtain values of this sRGB model from XYZ model,
we need to do two steps.
First step is linear transformation from XYZ coordinates to linear RGB coordinates.
Then we apply per channel on non-linear transformation.
If we want to estimate light intensity or brightness from sRGB model,
we need to apply this two transformation in reverse.
First, we apply non-linear transformation to obtain linear RGB values,
and then reapply linear transformation from linear RGB values to
XYZ values and the Y value will give us the brightness of this color.
There are other type of color models.
For example, the YIQ model.
This model was first introduced for analog colored TV.
This is also linear model with separated the brightness component.
Y corresponds to brightness component,
and I and Q correspond to chromaticity information.
Because brightness is separated,
we can transmit it on a grayscale channel
and color and information can be transmitted on additional bandwidth.
This allows us to simultaneously translate signal,
which can be received by grayscale TV and colored TV.
YIQ model is used in NTSC standard.
In PAL standard, there is a bit different model.
Now, we can understand how we can obtain colored images.
To obtain colored images,
we need to measure R,
G, and B value in each point of optical image.
This can be done for example by taking three pictures with red,
green, and blue filter of the same scene.
In Russia, the history of color photography is started by Sergei Prokudin-Gorsky.
He build special projector to display color photographs
on three separate images in red, green, and blue channels.
He also received the grant from the last Russian emperor
Nicholas II to make a color gallery of the Russian empire.
You can see images reconstructed from photos of
Sergei Prokudin-Gorsky on the link displayed on the slide.
In digital photography, instead of taking three images with different filters,
we can use another technique,
which is called Bayer pattern.
In this case, each pixel in a sensor are if it its own filter.
So, in each pixel, we obtain value for only one color channel,
and the values for other color channel are missing.
So, they need to be reconstructed.
We can interpolate values of missing channels using information from neighboring pixels.
This procedure is called demosaicing.
Demosaicing can lead to image artifacts,
as demonstrated in the slide.
For example, fine black and white details can be interpreted as changes in color,
not a small details during demosaicing.
And we will receive raw in colors in the image.
To solve this problem,
specific techniques of demosaicing should be used
and implemented actually in most of the modern cameras.
So even this simple procedure of taking color images is not very simple.
Now, we can understand what the digital color image is.
Digital color image is three dimensional array where each pixel is coordinate.
X, Y is represented with color vector of lens C. C is the number of channels.
Usually, the number of channels is three,
for example for RGB model or YIQ model, or some other model.
But the number of channels can be different, for example,
we can try to store full description of spectrum so the vector will be very large.
But usually, you we have only three channels.
Usually, each component is discretized to the interval
from zero to 255 and stored in one bit word.
This is usually enough for the recognition purposes.
What if you want to have more precision in each channel?
You can use 10 bit or 16 bit words,
and obtain better accuracy in color representation.
But for the recognition,
its rarely required so we will think that most
of images that we're working these are three channels by eight-bit words.
