Learn the fundamentals of digital signal processing theory and discover the myriad ways DSP makes everyday life more productive and fun.
5.5.a Stochastic signal processing
Loading...
您将学习的技能
Signal Processing, Discrete Fourier Transform, Data Transmission, Ipython, Fourier Analysis, Convolution, Linear Algebra, Digital Signal Processing
审阅
4.7（473 个评分）
- 5 stars373 ratings
- 4 stars83 ratings
- 3 stars10 ratings
- 2 stars5 ratings
- 1 star2 ratings
BB
Dec 03, 2017
Excellent course with lots of interesting real world applications as examples. The course moves quite fast though, and assumes students have some basic knowledge of signals and systems.
GC
Sep 18, 2017
This is a really great class; quite challenging, but very rewarding. Very much focused on theory, but with some practical applications.
从本节课中
Module 5: Sampling and Quantization
教学方
Paolo Prandoni
Lecturer
Martin Vetterli
Professor
So why stochastic signal processing?
So far in the class, we have assumed that every discrete time signal we used could
be described exhaustedly, either via a close form representation like so
or, and this is really the advantage of discrete time signals,
simply by enumerating its non-zero values.
How ever interesting signals are not known in advance.
For instance what im going to say next contains information that is not known in
advance.
But I do know what im going to say next is going to be a speech signal.
So I can try to describe stochastic signals in terms of a probabilistic model.
The good news is that we can do signal processing with stochastic signals
using the same tools that we have developed so far.
In this module, we will not try to treat the subject of stochastic signal
processing, either exhaustively or
very rigorously, but we will try to give you enough intuition, and
the mathematical tools, to deal with ubiquitous random signals, such as noise.
So let's start with a simple example of a stochastic signal.
Suppose we generate a discrete time signal, by tossing a coin for each sample,
and setting the sample's value to plus one if the outcome is head, and
minus one if the outcome is tail.
So because of the mechanics of the coin toss,
each sample is independent from all others and each sample has a 50%
probability of being plus one and a 50% probability of being minus one.
So this our signal generator and every time we turn on the generator,
every time we repeat this experiment of coin tossing.
We get what we call a different realization of the signal.
We can plot it and so for instance, the first run of coin toss gives us this
series of plus ones and minus ones, and then if we repeat the experiment again,
we toss the coin another 32 times, we get a different realization.
And we can repeat the experiment as many times as we want and
the outcome will likely be different every time we run the machine.
Although we cannot describe in advance the values of the signal,
we know the mechanics behind the generation of the signal.
And the question is, can we analyze the signal?
So, for instance, can we get a special representation of a random signal.
So let's try by taking the DFT of a finite set of random samples.
If we do that, we just run the machine for 32 samples and then we take a DFT.
But every time we repeat the experiment, we get a different plot and
the values don't really seem to follow any pattern.
So maybe we tell ourselves,
maybe we don't have enough data to discover a pattern in the DFD.
So maybe we should take longer realizations.
Instead of 32 points, maybe we should take more.
And so let's try and take 64 points.
And still we don't see any pattern.
So let's try and take 128 points.
And again, the spectrum doesn't seem to show any trait
that we can readily understand.
So we need a new strategy.
When faced with random data an intuitive response is to take averages.
So, we average out the values in order to eliminate the fluctuations.
In probability theory, the average is computed across the realization,
not along the time max's, but across different repetitions of the experiment.
So, for the coin toss signal, the expectation of each sample is -1[P(n-th
toss is tail)] + 1[P(n-th toss is head)].
But we know that these probabilities are one half each, so
this sum ends up being zero.
Since the DFT is the leader operator,
averaging the DFT values will not work either.
If we try and do that,
we quickly realize that the average of each DFT sample will be 0 as well.
However, the signal does move from minus 1 to plus 1, so its energy, or its power,
must be non-zero.
If you remember the definition of energy and
power of a signal from module 2.1, we can readily see that the energy is infinite.
Because the limit for N that goes to infinity of the sum
from -N to n of the values of the sequence squared.
A simple 2N plus one, because each value squared will be equal to one.
And so as N goes to infinity, this will diverge.
However, the signal does have finite power over any interval.
We take the energy over a minus Into an interval, and
we normalize that by the length of the interval.
And we find out that the power is actually 1, regardless of the interval's length.
Let's try the following strategy.
Let's try to average the DFT's square magnitude, normalized.
So we pick an interval N, we pick a number of iteration,
so a number of times we will repeat the experiment.
We run the signal generator n times, and we obtain n, m points realizations.
We compute the DFT of each realization, and
we average their square magnitude divided by the length of the interval.
So if we do that, of course, the first DFT will be this random pattern that we have
seen before, but as we increase the number of realizations,
we seem that the points seem to convert to something.
And indeed by the time M hits 5000, we see that the average
of the squared magnitude of the DFT seems to converge to the constant 1.
So have defined a quantity here P of k, which is the expected value of the squared
magnitude of the kth bin of the DFT over N points divided by capital N.
And it looks very much as if P of k, this expectation, is equal to 1 for all ks.
So if the square magnitude of the DFT tends to the energy
distribution in frequency of a signal, then the normalized square magnitude of
the DFT tends to the power distribution, or the power density in frequency.
So what we have just derived is a new frequency representation for
signals that have infinite energy, the finite power,
and it is called the power spectral density.
Let's try to develop some intuition about the power spectral density
of the coin toss signal.
The fact that it is constant means that the power
is equally distributed over all frequencies.
In other words we cannot predict if the signal will move slowly or super-fast.
We cannot predict that because each sample is independent of each other.
So we could actually have a realization where, just by luck,
we have a constant signal because all coin tosses are heads or we could have
a realization in which at each coin toss we have a different outcome.
And so the signal will oscillate at the maximum digital frequency.
The power spectral Embodies this behavior
by distributing the probability of power over the entire frequency access.
Lets try now to filter a random process and
to run this experiment we take our coin toss signal once again and we take what is
probably the simplest filter we can think of just a two point moving average filter.
So the output of the filter will be the average
of two neighboring points in the input random signal.
And the question we want to find an answer to is what is the power
spectral density of the output.
So we compute this numerically.
We run the experiment as before.
We generate a random signal.
This time we'll filter it and then we take the average
over several realizations of the DFT squared of the filtered output.
So we'll run the experiment like before.
We choose an interval n, we generate the random signal.
This time we filter it and
then we take the average over several realizations of the DFT squared.
For n equal to 1 we don't really see a pattern.
But as M increases,
we see that the power spectral density converges to a very precise shape.
Now if you remember the frequency response of the two point moving average,
that is just H of e to the j omega, equal to 1 plus e to the j omega divided by two.
And this shape here It's nothing but the square magnitude of this frequency
response evaluated at the DFT points, ie at 2 pi over M.
So, indeed if we plot this, we see that the match is almost perfect.
The power spectral density of the output seems to be the power spectral
density of the input, IE, the constant one, multiplied by the square magnitude
of the frequency response of the filter, this time computed on the DFT grid.
We can generalize these results beyond numerical experiments and
move it into the world of infinite support stochastic processes.
The details are in the book but here we will summarize the key points.
A Stochastic process is characterized in frequency
by it's power spectral density and it can be shown that the power spectral density
is the DTFT of the autocorrelation of the process.
Where each sample of the autocorrelation is obtained by taking the expectation
of the product of the stochastic signal times a delayed copy of itself.
For a filtered stochastic process general result is that
power spectral density of the output.
Is equal to the power spectral density of the input times the frequency response
squared.
The good news is that this result guarantees that the filters that we
design in the deterministic case can still be used with stochastic signals.
A low pass will remain a low pass and a high pass will remain a high pass.
We do however lose the concept of phase and this is understandable since we don't
really have any advance information on the shape of the stochastic signal.
That will depend on the particular realization.
All we know is where the power is distributed in the frequency.
Now that we have some stochastic tools in place, let's attack the concept of noise.
So noise is everywhere.
It appears in the form of thermal noise in circuits.
It could be the sum of various extraneous interferences in communication systems.
Or the quantization and numerical errors that complex digital systems produce and
that we cannot predict in advance.
Because of our lack of knowledge about the sources of the noise.
We will model the noise as a stochastic signal and
the most important type of noise is white noise.
With the term white we indicate a stochastic process where
all the samples are uncorrelated.
If the samples are uncorrelated the article relation of the process
will be zero every where except at zero.
Where it will take the value of the variant.
As a consequence the power spectral density is the constant sigma square.
Where is sigma is the variance of the stochastic signal.
Graphically the power spectral density of a white signal couldn't be any simpler.
We have seen an example of this in the coin toss experiment.
The power spectral density of white noise is independent
of the probability distribution function for the single sample.
Distribution however will be important to estimate
the bounds of the noise signal in the time domain.
Very often we use a Gaussian distribution to model the underlying probability
distribution function for the sample.
The reason for that is that the Gaussian distribution is the model of choice
when we want to represent the effect of many unknown superimposed sources,
as is the case for noise.
In this case is the noise where we called additive white Gaussian noise, or
AWGN for short.
