Markov Chains

I. Discrete Parameter.-
1. Fundamental definitions.-
2. Transition probabilities.-
3. Classification of states.-
4. Recurrence.-
5. Criteria and examples.-
6. The main limit theorem.-
7. Various complements.-
8. Repetitive pattern and renewal process.-
9. Taboo probabilities.-
10. The generating function.-
11. The moments of first entrance time distributions.-
12. A random walk example.-
13. System theorems.-
14. Functionals and associated random variables.-
15. Ergodic theorems.-
16. Further limit theorems.-
17. Almost closed and sojourn sets.- II. Continuous Parameter.-
1. Transition matrix: basic properties.-
2. Standard transition matrix.-
3. Differentiability.-
4. Definitions and measure-theoretic foundations.-
5. The sets of constancy.-
6. Continuity properties of sample functions.-
7. Further specifications of the process.-
8. Optional random variable.-
9. Strong Markov property.-
10. Classification of states.-
11. Taboo probability functions.-
12. Last exit time.-
13. Ratio limit theorems; discrete approximations.-
14. Functionals.-
15. Post-exit process.-
16. Imbedded renewal process.-
17. The two systems of differential equations.-
18. The minimal solution.-
19. The first infinity.-
20. Examples.