Markov Chain

CORE DEFINITION

A Markov chain is a stochastic process that undergoes transitions from one state to another on a state space. It is named after the Russian mathematician Andrey Markov. The process requires the property of "memorylessness": the probability distribution of the next state depends only on the current state, not on the sequence of events that preceded it. This specific type of "memorylessness" is called the Markov property. Markov chains have many applications as statistical models of real-world processes. At each step of a Markov chain, the system may change its state according to a probability distribution, or it may remain in the same state. The change of state is called a transition, and the probabilities associated with various state changes are called transition probabilities. A random walk is an example of a Markov chain. In a random walk, the state at each step is a point in a graph, and the walker can move to any adjacent point with equal probability, regardless of the path taken so far.