0:00

Hello, welcome back to the course on Model Signal Processing for Music Applications.

In the previous programming lecture,

we talked about the HPR model, the Harmonic Plus Residual model.

That was the idea of modeling a sound with a harmonic and residual component.

And we talked about the implementation in the SMS tools package.

Now, we want to go a step further and talk about the hpsModel,

the Harmonics plus Stochastic model.

So now, we are modelling the residual that we obtained with

the stochastic modeling approach.

So, we obtain a harmonic and stochastic representation of a particular sound.

So the diagram, the block diagram, is very similar.

The whole thing for the analysis is the same

until we obtain the residual spectrum of X, Xr[k].

And then instead of just going and summing it with a harmonic component,

what we are doing is approximating it with a stochastic approximation.

So, by assuming the magnitude spectrum and

then the synthesis is done by generating random numbers for

the phase, and combining it with a sort of an interpolated version

of these approximations so that we generate a complete spectrum.

And this stochastic spectrum can then be added back to the harmonic component.

The great advantage of these stochastic approximation is that now

we are more flexible, we can do many more things on this

representation apart from being much more compact.

See, this is stochastic approximation,

reduces the amount of information that is required to reconstruct a signal.

So apart from being a compactor presentation, is a more flexible,

and will be able to do quite a few things with that.

So let's start from the same code that we looked at in the last lecture,

which was this code in which, well now I changed the sound.

Now I use the foot sound, but basically it performs

the harmonic analysis by taking a 50 and detecting the zero,

so it performs the harmonic analysis steps.

And then it synthesizes those harmonics and

it subtracts them from the orignal signal.

It recomputes the spectrum of the original signal with the same window size and

type than the one that is in this generated sinusoidal spectrum.

So we need to multiply it by a window, and

we need to choose the window size that was the same

then was used for the sinusoidal synthesis.

And, we can then subtract the harmonics from

the original signal and obtain the residual.

And now in this lecture,

basically we focus on the stochastic approximation of these residuals.

These three lines are doing that.

So, the first is we are converting it to a DV scale.

So, we are converting half of the magnitude spectrum to a DV scale and

we obtain a new magnitude spectrum.

And then we call this function called resample with this, my new spectrum,

and of course with values that are above a certain threshold.

And we are converting it to a shorter array.

And the size of this new array,

is the size that we started from multiplied by stocf.

And stocf is our stochastic factor approximation.

Basically, is moving approximation or down sampling factor.

So basically, it means that the output array

will be 40% of the original spectrum.

And then, we are up sampling again to the size that we

started with in order that then we can do the resynthesis.

So in fact, resample can be use both for down sampling and for up sampling.

And it uses the DFT approach.

So, let's talk a little bit about that.

So, we can go to do the documentation on the resample of function within scipy.

So this is the documentation, so it takes an array,

it takes the size of the output array.

So either less or more, so you can down sample or up sample.

And well, there is some optional arguments also that allow us to give a particular

window or to give more multidimensional array and process only part of that.

And since Python of course is all open source,

we can also see the source code of this function.

And as I said, well, every sample uses an frequency-response

approach to do downsampling and upsampling.

And what it does is, it performs first the FFT

of the input signal then it will basically cut the spectrum or

append samples depending if its down sampling or up sampling.

And then, once we have recomputed a longer or a shorter spectrum by cutting or

appending zeroes, we perform the inverse of 50.

That's a quite straightforward, simple way of performing

interpolation or downsampling or upsampling.

So we go back here, so this returning for

the first three samples, we will return less samples and

then afterwards we will convert it to this up sample version

that will be a smooth version of the spectrum we started with.

Let's run this, so we can see how that works.

So, let's run test4.

Okay and now we can plot the original spectrum.

So, mxr is already the dB version of the spectrum and half of it.

Okay, so this is our residual spectrum.

And then we plot on top of that the stocEnv, that's of the same size but

a smooth version because it's a result of having first down tempo.

And then, up sample with interpolation.

Okay so, as you see we are very much in fact if we zoom into any of these regions,

we see that the green line is the smooth version but

it is of course is smoother than the original one,

but it follows quite well, the original signal.

So, let's change the stochastic factor of 0.4 and let's put maybe 0.1.

So then, that will be a stronger smoothing factor.

And if we save this, and then we compute it.

8:08

Okay, now we plot again the original residual spectrum.

And now, let's plot the smooth envelope.

Okay, and this clearly shows the smooth envelope on top of that,

and is much smoother than before.

Okay, so that's basically what the stochastic approximation does.

And we now go into the HPS model code of the SMS tools but

we'll see the analysis function the synthesis function.

And also one function that does analysis and synthesis at the frame level.

And the analysis for example is basically the same then for the HPR model.

We only add one input parameter which is this stop f,

factored as a moving factor and then in the analysis,

we first perform the harmonic analysis then we subtract

the sinusoids from the original signal to obtain the residual.

And then we run the stochastic model now,

the one that we already discussed when we talked about the stochastic model.

So we just, the stochastic model receives the signal, the residual signal.

And it has the hope size, the same hope size that we're using and now the window

size that we used for stochastic approximation is just twice a hope size,

so in fact, the stochastic approximation does not require big windows.

In fact it's the other way around, the smaller the better so

we can capture time information, the stochastic components, the residual

components, the time information, the attacks of the nodes, etc.

Are most critical.

So, the smaller the window, the better.

In fact, if H many times is by default 128 samples,

here we are using windows of 256 samples.

So, we will be computing an FFT of 256 samples and approximating that, okay?

And of course then we return the harmonics.

And instead of returning the residual, we return this envelopes.

This sequence of envelopes that we are computing.

And in the synthesis part, well it's a similar situation.

We synthesize the sinusoids, and then synthesize the stochastic

component using the stochastic model synth that also we talked about.

So from these envelopes, we can synthesize the stochastic component.

And the function that puts it all together,

there is this hps model function that includes analysis and

synthesis using the harmonic plus a stochastic model.

So, it has some parameters.

The full parameters and then it performs analysis, and then it performs synthesis.

And in the process,

it plots certain intermediate values.

Okay so, let's go to the terminal and now instead of running the code or

running the code by calling the file, we can also run the function directly,

and that allows us to be able to change the parameters.

So if we import this function,

the HPS model function.

Okay, now we have imported.

We have executed.

Now, well we better to just imported with a given name.

For example, HPS.

Okay and now we can call HPS and the function within HPS.

So within this file there is this function, main.

So, we can just call main.

And here, now we'll be able to change the values of these parameters,

so let's just get the values by default.

12:35

Okay, so this computes this analysis and synthesis of this sound,

of this sax-phrase with the default parameters.

And now, we can play around.

So of course,

we could use the interface that we have been showing in the demo classes.

But here, maybe sort of more intuitive in terms of understanding

what is going on and being able to go deep into the code.

To just replace parameters here, so for example,

now if I want to change the threshold and make it a smaller threshold.

Instead of minus 100 let's put minus 40,

then we will run the same code but changing the threshold,

and of course, changing the threshold has meant that it found only few harmonics.

In fact, it can see that everything is stochastic.

And in here, the stochastic representation is the one that is more prominent.

Okay.

And you can change any parameter.

You can change the sound, for example.

Okay, we can change the sound file or

we can change the minimum duration of the harmonics, etc, etc.

So, no need to explain that anymore.

Basically, we can just execute in any way we want.

Okay, and that's all I wanted to say.

So, this is basically an extension of the Harmonic plus Residual Model.

And we have seen how it works.

So basically,

this is the most complex model that we're going to see in this course.

And all the others can be understood as related with this one.

So if you understand this model, I think it's a good way to

understand a lot of things we have been talking about.

So, that's all for today.

So, I hope to see you next lecture.

Thank you.