Lately I have stopped reading “Probability and computing”, since I found some gaps in the exposition of the text, especially at chapter 7 “Markov Chains and Random Walks”-the authors left undefined some terminologies such as absorption. Certainly this is not the text for anybody who has little background of probability and want to learn it rigorously (though it is a good introductory text for randomized algorithm). So I bought “Markov Chains” of James Norris.

Exercises in “Markov Chains” are easy (at least for the first chapter ), though there are some problems that I am not quite sure. Here is one of them:

A particle moves on the eight vertices of a cube in the following way: at each step the particle is equally likely to move to each of the three adjacent vertices, independently of it past motion. Let be the initial vertex occupied by the particle, the vertex oppostite . Calculate each of the following quantities:

(i) the expected number of steps until the particle returns to ;

**(ii) **the expected number of visits to until the first return to ;

**(iii) **the expected number of steps until the first visit to .

This exercise is placed under “Invariant distribution” section. My answer is **(i)** 8, **(ii) **1, **(iii)** 16 (?). I did not use anything from this section to solve **(iii) **and the answer also is suspicious, at least counter-intuitive to me. So I would like to know if there is any connection between the expected number of steps until the first visit to , , and the invariant distribution .

### Like this:

Like Loading...