Materials for book: "Markov Chains for programmers"
The Lighting model, a simple two state model to simulate a light switching on and off.
The Lighting model, using probabilities (DTMC version):
The pushkin model developed by A. A. Markov (according to Snell’s book).
The land of oz model.
The Belfast weather model (by Stewart).
The Generic model.
The Lily Pad model (frog in the pond).
Software Aging and Rejuvenation model (CTMC).
The Mouse Maze model.
The Ehrenfest model.
A six state Birth and Death model.
A larger Mouse Maze model, with 9 states.
Look at spreadsheets/Chapter04-CTMC-26states-Shakespeare.xlsx for more information about this model.