Week 3.5: Graphic representation. Classification of states-3

Loading...

来自 National Research University Higher School of Economics 的课程

Stochastic processes

29 个评分

National Research University Higher School of Economics

Stochastic processes

29 个评分

The purpose of this course is to equip students with theoretical knowledge and practical skills, which are necessary for the analysis of stochastic dynamical systems in economics, engineering and other fields. More precisely, the objectives are 1. study of the basic concepts of the theory of stochastic processes; 2. introduction of the most important types of stochastic processes; 3. study of various properties and characteristics of processes; 4. study of the methods for describing and analyzing complex stochastic models. Practical skills, acquired during the study process: 1. understanding the most important types of stochastic processes (Poisson, Markov, Gaussian, Wiener processes and others) and ability of finding the most appropriate process for modelling in particular situations arising in economics, engineering and other fields; 2. understanding the notions of ergodicity, stationarity, stochastic integration; application of these terms in context of financial mathematics; It is assumed that the students are familiar with the basics of probability theory. Knowledge of the basics of mathematical statistics is not required, but it simplifies the understanding of this course. The course provides a necessary theoretical basis for studying other courses in stochastics, such as financial mathematics, quantitative finance, stochastic modeling and the theory of jump - type processes.

从本节课中

Week 3: Markov Chains

Upon completing this week, the learner will be able to identify whether the process is a Markov chain and characterize it; classify the states of a Markov chain and apply ergodic theorem for finding limiting distributions on states

Week 3.1: Definition of a Markov chain. Some examples17:16

Week 3.2: Matrix representation of a Markov chain. Transition matrix. Chapman-Kolmogorov equation11:30

Week 3.3: Graphic representation. Classification of states-110:36

Week 3.4: Graphic representation. Classification of states-24:43

Week 3.5: Graphic representation. Classification of states-37:14

Week 3.6: Ergodic chains. Ergodic theorem-16:11

Week 3.7: Ergodic chains. Ergodic theorem-215:21

与讲师见面

Vladimir Panov
Assistant Professor
Faculty of economic sciences, HSE

Another characteristic of a state of Markov Chain is a so called period.

Let me give a definition.

Period of a state i is

the greatest common divisor of all n such that Pii (n) is not equal to zero.

With this you should think about how many steps you can return from i to i,

and then you should find are greatest common divisor of all such

n. We will denote period by d(i),

and if period is equal to one,

then we will say that the state number i is aperiodic.

Let me provide an example.

If you look look at this picture we can definitely calculate the periods of all states.

For instance the period of a state number one is equal to one just

because P111 is not equal to zero.

You'll see there is an arc from one to one.

Therefore, the greatest common divisor is also equal to one.

If you think about state number two,

you carry from two to two in two steps,

in four steps, you should make two circles in four steps and so on.

So this period was a state number

two is equal to two and the same situation with the state number three.

Well, as for the state number four,

there is no circles,

so you cannot return to four if you start from four.

In this case, there is no n such that this condition is fulfilled,

and in this case we will also say that the period is equal to one.

As for the state number five,

you can return from five to five in two steps,

you can go from five to six and to six to five,

but also in three steps because you can go from five to six,

make a circle here,

and then return back.

So, there are numbers two and three in this brackets,

and the greatest common divisor between two and three is equal to one.

This means that the state number five is also aperiodic.

Also aperiodic is the state number six,

and so we have a very interesting situation.

So inside any class,

the period of all states is the same.

Let me formalize this fact as a theorem,

all elements in one class of equivalents

have the same period.

Let me prove this fact, it's a little difficult.

Let me take two elements,

i and j, from one class.

There is some walk from j to i,

and there's some walk from i to j.

Let me denote the length of the walk from i to j by n,

I mean you have to do n steps to access j from i,

and let me denote the length of a walk from j to i by m. Since you can return

from i to i by

making n + m steps,

then according to our definition of a period,

n + m should be divisible by the period of the state i.

Let me now take some k,

such that you can return from j to j by making k steps,

so Pjj(k) is not equal to zero.

This means that you can actually return from i to i by making n + m + k steps.

In fact you can go from i to j by making n steps,

then you can make a circle from j to j by making k more steps,

and then you can go from j to i by making m steps.

So n + m + k,

is also a number such that by Pii (n +m +k)is not equal to zero,

and therefore, n + m + k shall be divisible by d(i).

From these two statements we conclude that also

the number k shall be divisible by d(i).

But what is a number k?

This is a number such that Pjj(k) is not equal to zero.

We know that if k is such that this is fulfilled,

then k is divisible by d(i).

But you know that the greatest common divisor of all such case is equal to d(j).

So d(i) is one divisor of all such case,

and d(j) is the greatest common divisor.

Therefore, d(i) should be divisible by d(j).

But what is the difference between i and j?

There is almost nothing different between them and therefore we can just

say that d(j) shall be also divisible by of d(i).

From this two statements we conclude that d(i) is equal to d(j).

So the appearance coincide.

So within one class is a period,

is the same for all notes and therefore

period is also a characteristic of the following of the complete class.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2018 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取