**Exercise 7.12: **Let be the sum of independent rolls of a fair dice. Show that, for any , .

**Solution: **Let be a Markov chain on state space consist of states: , where the chain reaches state if and only if .

The transition matrix is for and . The claim is equivalent to

We know that if a Markov chain has an irreducible, aperiodic transition matrix and an invariant distribution , then for all of the state space.

We will prove the defined above is irreducible, aperiodic and then find the invariant distribution of (it will turn out that as we need).

**Exercise 7.21:** Consider a Markov chain on the states , where for we have and . Also, and . This process can be viewed as a random walk on a directed graph with vertices , where each vertex has two directed edges: one that returns to and one that moves to the vertex with the next higher number(with a self-loop at vertex ). Find the stationary distribution of this chain.(This example shows that random walks on directed graphs are very different than random walks on undirected graph).

**Solution: **

Posted by tom on February 28, 2012 at 11:37 pm

Can you give solution of 5.10 exercise? urgent!

thanks

Posted by haya on June 12, 2013 at 2:42 am

Can you give a solution for exercise 7.21?