## Sunday, 18 September 2016

### Reducible state space Markov chains

While reviewing a paper last night, I started thinking about reducible state space Markov chains.  Most undergraduate probability students are fed theorems about irreducible Markov chains.  To summarize these theorems.

• If two states communicate, then they are either both recurrent or both transient.
• If a MC is irreducible and all states are positive recurrent, then there exists a unique stationary distribution.
• If a MC is irreducible, aperiodic, and has a stationary distribution, then it converges to that.

We can ask a set of questions about reducible Markov chains to test our intuition for the subject.
• Let's setup a trivial 4 state Markov chain where the first two and last two states form separate communicating classes.
• How many stationary distributions exist?
• Do limiting distributions exist?
• Does the limit distribution depend on the initial distribution?  How many possibilities are there?