Why must each column of the transition matrix sum to 1, not each row?

It depends on the convention, and this course uses columns as the 'from' state. Each column collects every place the system could go next FROM a given state; since it must go somewhere, those probabilities total $1$. If instead you laid the matrix out with rows as 'from', the rows would sum to $1$ and you would multiply as $\mathbf{s}_n = \mathbf{s}_0 T^n$ with a row vector. Pick one convention and stay consistent; in this course we keep columns summing to $1$ and multiply $T\mathbf{s}$.

What is the difference between $\mathbf{s}_n = T\mathbf{s}_{n-1}$ and $\mathbf{s}_n = T^n \mathbf{s}_0$?

They give the same answer. $\mathbf{s}_n = T\mathbf{s}_{n-1}$ is the one-step recurrence: it advances the chain a single step from the previous state. $\mathbf{s}_n = T^n \mathbf{s}_0$ is the closed form: it jumps straight to step $n$ from the start. For anything beyond a couple of steps, raise $T$ to the power on the GDC rather than multiplying repeatedly.

How high a power do I need for the steady state?

There is no magic number, but for typical exam matrices a power of $50$ to $100$ is plenty. A good check is to compute $T^{50}$ and $T^{51}$: if the columns agree to several decimal places and are identical to each other, you have reached the steady state and any column is $\mathbf{s}$.

Should I solve $T\mathbf{s} = \mathbf{s}$ algebraically or use powers of $T$?

Both are valid on Paper 2. The algebraic route turns $T\mathbf{s} = \mathbf{s}$ into a small system of linear equations together with the constraint that the entries of $\mathbf{s}$ sum to $1$, which you can solve on the GDC. The high-power route is usually faster: compute $T^{50}$ and read off a column. Use whichever your calculator makes easier, but know that the steady state is precisely the eigenvector of $T$ for eigenvalue $1$.

Does every Markov chain settle into a single steady state?

No. Regular chains, those for which some power $T^k$ is all-positive, have a unique steady state reached from any start. Chains with absorbing states or with cyclic behaviour need not: an absorbing chain ends up trapped, and a strictly cyclic chain oscillates forever without converging. In AI HL the contexts are usually regular, but read the problem before assuming convergence.

Transition matrices and Markov chains — IB…

Revise Transition matrices and Markov chains for IB Mathematics: Applications and Interpretation (4.10)

Looking for free IB Mathematics: Applications and Interpretation notes on Transition matrices and Markov chains? This Higher Level lesson covers official syllabus point 4.10 on Maths AI HL. Use the visual steps, key terms, and quick checks below — then practise with IB criterion marking on Transition matrices and Markov chains. Built for students searching "IB Maths AI 4.10", "Transition matrices and Markov chains IB revision", or a free alternative to paid IB note sites.

Transition matrices and Markov chains

Some systems evolve in clean, predictable jumps: today's weather nudges tomorrow's, this month's phone brand shapes next month's, a car sitting in one depot is likely to be returned to another. A Markov chain models exactly this kind of step-by-step, probability-driven change, on one deliberately simple assumption: the future depends only on the present, never on the full history of how you arrived. That assumption lets a single transition matrix $T$ carry the whole model. This lesson covers the full 4.10 syllabus, building $T$ correctly, stepping the system forward with $\mathbf{s}_n = T^n \mathbf{s}_0$ , and finding the long-run steady state $\mathbf{s}$ where the distribution stops changing. Because 4.10 is a Higher Level Paper 2 topic, the GDC handles every matrix power, so the marks reward setting up the model and reading it in context. As in every calculation topic, the worked examples below are laid out exactly the way our engine marks your own practice.

BY THE END, YOU CAN…

1Construct a transition matrix $T$ from a described system, checking that each column is a probability distribution summing to 1.
2Set up the initial state vector $\mathbf{s}_0$ and use $\mathbf{s}_n = T^n \mathbf{s}_0$ on the GDC to find the state after $n$ steps.
3Find and interpret the long-run steady state $\mathbf{s}$ satisfying $T\mathbf{s} = \mathbf{s}$ , as the eigenvector for eigenvalue 1 or from a high power of $T$ .
4Apply Markov chains to contexts such as weather, brand-switching and population movement, interpreting the numbers in words.
5Present working so that method marks and follow-through protect the final answer under our Paper 2 marking engine.

In simple terms

A friendly intro before the formal notes — no formulas yet.

Predicting the Future, One Step at a Time

A Markov chain tracks a system that moves between a few fixed states at each time step, where the only thing that matters for the next step is the current state. A single matrix $T$ stores all the switching probabilities, and multiplying by it walks the system forward in time. Push it far enough and many systems settle into an unchanging steady state.

THINK OF IT LIKE…

Suppose you only ever use one of two streaming services, StreamOne or FlixTwo, and each month your choice depends only on last month. If you were on StreamOne, there is an 80% chance you stay and a 20% chance you switch; if you were on FlixTwo, a 30% chance you jump to StreamOne and a 70% chance you stay. Those four numbers are the transition matrix. Knowing today's market split, the matrix predicts next month, the month after, and, if you keep pushing, the long-run split that stops changing no matter how many more months pass.

1
Name the states of the system (weather types, brands, locations) and fix their order once, for good.
2
Build the transition matrix $T$ so each COLUMN lists the probabilities of where you go FROM one state; every column must sum to 1.
3
Write the initial state vector $\mathbf{s}_0$ as a column showing the starting split across the states.
4
Step forward with $\mathbf{s}_n = T^n \mathbf{s}_0$ on the GDC; for the long run, raise $T$ to a high power or solve $T\mathbf{s} = \mathbf{s}$ to read off the steady state.

Explore the concept

Use the live diagram and synced steps — play it or tap a step card to walk through.

Key formulas

Tap any symbol to reveal exactly what it means and its units.

$1, 2, \ldots$

$T\mathbf{s} = \mathbf{s}$

Full topic notes

Formal explanation with the rigour you need for the exam.

States, transitions and the Markov property

A Markov chain has two ingredients. First, a finite list of STATES, the distinct conditions the system can occupy, such as 'sunny' and 'rainy', or the brands a customer might buy. Second, TRANSITIONS between those states over discrete time steps: a day, a month, a generation. The defining rule, the Markov property, is that the probability of the next state depends only on the current state. The chain has no memory: a run of ten sunny days changes nothing about tomorrow if today is sunny. That single simplification is what makes the whole model fit into one matrix.

Building the transition matrix $T$

The transition matrix $T$ stores every one-step probability. It is square, with one row and one column per state. In this course the convention is that each COLUMN is the current ('from') state and each ROW is the next ('to') state, so the entry in row $i$ , column $j$ is the probability of moving to state $i$ given you are now in state $j$ . Read a column top to bottom and you are reading a full probability distribution: all the places the system might go next, starting from that column's state.

For states ordered $1, 2, \ldots$ , the transition matrix is $T = \begin{pmatrix} p_{11} & p_{12} & \cdots \\ p_{21} & p_{22} & \cdots \\ \vdots & \vdots & \ddots \end{pmatrix}$ , where $p_{ij} = P(\text{to } i \mid \text{from } j)$ and each column satisfies $\sum_i p_{ij} = 1$ .

Square: one row and one column per state, in a fixed order you choose once.
Columns are 'from', rows are 'to': the entry in row $i$ , column $j$ is $P(\text{go to } i \mid \text{now in } j)$ .
Every column sums to 1: each column is a probability distribution, because from any state the system must transition somewhere (possibly staying put).

The state vector and stepping forward in time

The distribution of the system at time $n$ is stored in the state vector $\mathbf{s}_n$ , a column whose entries are the proportions (or counts) in each state, listed in the same order as the matrix. To advance one step you multiply on the left by $T$ : $\mathbf{s}_1 = T\mathbf{s}_0$ . Advancing again gives $\mathbf{s}_2 = T\mathbf{s}_1 = T^2\mathbf{s}_0$ , and repeating the pattern produces the closed form for any number of steps. On Paper 2 you never chain the multiplications by hand; you raise $T$ to the power directly.

State after $n$ steps: $\mathbf{s}_n = T^n \mathbf{s}_0$ . \n The matrix sits on the LEFT and the state vector on the RIGHT; the product is again a column vector.

EXAM TIP

On Paper 2 you always have your GDC, so learn to enter a matrix, raise it to a power, and multiply it by a vector fluently. Store $T$ and $\mathbf{s}_0$ in named variables, then evaluate $T^{n}\mathbf{s}_0$ in one line. Doing the arithmetic on the calculator both saves time and removes the multiplication slips that cost accuracy marks.

The long-run steady state

Push a regular Markov chain far enough and the state vector stops changing: it reaches a STEADY STATE (or equilibrium) $\mathbf{s}$ , where applying $T$ once more returns the same vector. That is the defining equation $T\mathbf{s} = \mathbf{s}$ : the flow into each state exactly balances the flow out. Written as $T\mathbf{s} = 1\cdot\mathbf{s}$ , it says $\mathbf{s}$ is an eigenvector of $T$ for eigenvalue $1$ , and every transition matrix has $1$ as an eigenvalue. Two routes find it: solve $T\mathbf{s} = \mathbf{s}$ as linear equations with the entries constrained to sum to $1$ , or raise $T$ to a high power and read off a column, since for a regular chain the columns of $T^n$ all converge to $\mathbf{s}$ .

Steady state: solve $T\mathbf{s} = \mathbf{s}$ with $\sum \text{entries of } \mathbf{s} = 1$ . \n Equivalently, $\mathbf{s}$ is the eigenvector of $T$ for eigenvalue $1$ , normalised so its entries sum to 1; or read a column of $T^{n}$ for large $n$ .

Common mistakes examiners penalise

Rows that sum to 1 instead of columns — in this course columns are the 'from' state and each COLUMN must sum to $1$ . Writing $T$ so the rows sum to $1$ transposes the matrix and every later answer is wrong.
Multiplying in the wrong order — the step is $T\mathbf{s}$ , matrix on the left and column vector on the right. Writing $\mathbf{s}T$ with a column vector is undefined, and swapping to a row vector silently changes the convention.
Using $nT\mathbf{s}_0$ instead of $T^n\mathbf{s}_0$ — you raise the matrix to the power $n$ , you do not multiply the matrix by $n$ . $\mathbf{s}_n = T^n\mathbf{s}_0$ .
Solving $T\mathbf{s}=\mathbf{s}$ without the constraint — the equation $T\mathbf{s}=\mathbf{s}$ alone has infinitely many solutions (any multiple of $\mathbf{s}$ ). You must add $\sum \text{entries} = 1$ (or the population total) to pin down the unique steady state.
Confusing eigenvalue $1$ with eigenvalue $0$ — the steady state is the eigenvector for eigenvalue $1$ (since $T\mathbf{s}=1\cdot\mathbf{s}$ ), not the null space.
Assuming every chain converges — the high-power method needs a regular chain. Absorbing or cyclic chains need not settle to a single interior steady state, so check the context before quoting one.
Rounding populations too early or leaving fractions of people — carry the GDC's full figures, and round headcounts to whole numbers only at the final step.

Model answer — marked the way our engine marks it

On Paper 2 the marks are analytic: each is tied to a specific line of working, a method mark (M) or an accuracy mark (A), and an accuracy mark depends on the method mark it follows. Follow-through (FT) means an earlier wrong value need not cost you the marks that depend on it, provided the later step is done correctly on your own figure. For matrix work the engine expects to see the set-up $\mathbf{s}_n = T^n\mathbf{s}_0$ , the matrix power computed (GDC allowed), and the answer read in context. Study how each mark below is earned by a specific line.

Where this leads

Markov chains tie the probability you met earlier to the matrices and eigenvectors of the HL algebra strand: the steady state IS an eigenvector, so the eigenvalue-eigenvector machinery you use elsewhere pays off directly here. The same modelling instinct, list the states, capture the one-step rule, then iterate, underlies coupled systems of differential equations later in the course, where matrices again drive a system forward in time and the long-run behaviour is read from eigenvalues. Master the two moves of this topic, stepping forward with $\mathbf{s}_n = T^n\mathbf{s}_0$ and settling down with $T\mathbf{s} = \mathbf{s}$ , and a wide family of dynamic models becomes a single, familiar calculation.

Worked examples

See the formulas applied — reveal one step at a time, like the exam.

EXAMPLE 1

A survey tracks customer loyalty between two coffee shops, Bean Scene (B) and Daily Grind (D). Each month, 80% of Bean Scene's customers return and 20% switch to Daily Grind; of Daily Grind's customers, 30% switch to Bean Scene and the rest stay. Initially Bean Scene holds 60% of the market and Daily Grind 40%. \n (a) Write down the transition matrix $T$ . \n (b) Find the market share after 2 months. [6]

1
Fix the state order as B then D, and set columns as 'from', rows as 'to'.\n\n**(a) Transition matrix.** From B: stay $0.8$ , switch to D $0.2$ , giving the first column $\begin{pmatrix} 0.8 \\ 0.2 \end{pmatrix}$ . From D: switch to B $0.3$ , stay $0.7$ , giving the second column $\begin{pmatrix} 0.3 \\ 0.7 \end{pmatrix}$ .\n $T = \begin{pmatrix} 0.8 & 0.3 \\ 0.2 & 0.7 \end{pmatrix}.$ \nEach column sums to $1$ , as required. [A2: fully correct matrix; A1 for one error]\n\n**(b) After 2 months.** The initial state vector is $\mathbf{s}_0 = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}$ , and $\mathbf{s}_2 = T^2 \mathbf{s}_0$ . [M1: correct set-up]\nOn the GDC, $T^2 = \begin{pmatrix} 0.70 & 0.45 \\ 0.30 & 0.55 \end{pmatrix}$ , so\n $\mathbf{s}_2 = \begin{pmatrix} 0.70 & 0.45 \\ 0.30 & 0.55 \end{pmatrix}\begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix} = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}.$ [M1: compute $T^2 \mathbf{s}_0$ ]\nAfter 2 months Bean Scene holds $60\%$ and Daily Grind $40\%$ . [A1]\n\nThe share happens to return to the start here; that is a coincidence of these numbers, not yet the steady state.

✓ FULLY WORKED

EXAMPLE 2

A city has three internet providers: ConnectFast (C), StreamEasy (S) and PingKing (P). Monthly, ConnectFast keeps 85% of its customers, losing 10% to StreamEasy and 5% to PingKing; StreamEasy keeps 75%, losing 15% to ConnectFast and 10% to PingKing; PingKing keeps 80%, losing 10% to ConnectFast and 10% to StreamEasy. Initially C has 20 000 customers, S has 50 000 and P has 30 000. \n (a) Write down the transition matrix $T$ . \n (b) Find the number of customers with each provider after 6 months. \n (c) Find the long-term number of customers with each provider, assuming the total stays constant. [8]

1
Order the states C, S, P, with columns 'from' and rows 'to'.\n\n**(a) Transition matrix.\n $T = \begin{pmatrix} 0.85 & 0.15 & 0.10 \\ 0.10 & 0.75 & 0.10 \\ 0.05 & 0.10 & 0.80 \end{pmatrix}.$ \nEach column sums to $1$ . [A2: fully correct; A1 for one error]\n\n(b) After 6 months.** The total is $20\,000 + 50\,000 + 30\,000 = 100\,000$ , and $\mathbf{s}_0 = \begin{pmatrix} 20000 \\ 50000 \\ 30000 \end{pmatrix}$ . Compute $\mathbf{s}_6 = T^6 \mathbf{s}_0$ on the GDC. [M1]\n $\mathbf{s}_6 \approx \begin{pmatrix} 42085 \\ 36193 \\ 21722 \end{pmatrix}.$ \nAfter 6 months, approximately $42\,085$ with ConnectFast, $36\,193$ with StreamEasy and $21\,722$ with PingKing (rounded to whole customers). [A2: A1 for one slip]\n\n**(c) Long term.** Raise $T$ to a high power, e.g. $T^{50}$ , whose columns become identical. [M1]\n $T^{50} \approx \begin{pmatrix} 0.4651 & 0.4651 & 0.4651 \\ 0.3488 & 0.3488 & 0.3488 \\ 0.1860 & 0.1860 & 0.1860 \end{pmatrix},$ \nso the steady-state proportions are $\mathbf{s} \approx \begin{pmatrix} 0.4651 \\ 0.3488 \\ 0.1860 \end{pmatrix}$ . Multiplying by the constant total $100\,000$ : [M1]\nConnectFast $\approx 46\,510$ , StreamEasy $\approx 34\,880$ , PingKing $\approx 18\,610$ . [A1]\n\nIn the long run ConnectFast dominates the market, even though it started with the fewest customers.

✓ FULLY WORKED

EXAMPLE 3

A predator population is modelled as a Markov chain over two states, 'high' (H) and 'low' (L), from year to year with transition matrix $T = \begin{pmatrix} 0.7 & 0.4 \\ 0.3 & 0.6 \end{pmatrix}$ (states in order H, L). Find the long-run proportion of years in which the population is high, giving your answer as a fraction. [5]

1
Let the steady state be $\mathbf{s} = \begin{pmatrix} x \\ y \end{pmatrix}$ with $x + y = 1$ . It satisfies $T\mathbf{s} = \mathbf{s}$ . [M1: set up $T\mathbf{s} = \mathbf{s}$ ]\n\nThe top row gives\n $0.7x + 0.4y = x \;\Rightarrow\; -0.3x + 0.4y = 0 \;\Rightarrow\; 0.3x = 0.4y.$ [M1: form a usable equation]\nSo $x = \dfrac{0.4}{0.3}\,y = \dfrac{4}{3}\,y$ . Substituting into $x + y = 1$ : [M1: use $x+y=1$ ]\n $\tfrac{4}{3}y + y = 1 \;\Rightarrow\; \tfrac{7}{3}y = 1 \;\Rightarrow\; y = \tfrac{3}{7}, \quad x = \tfrac{4}{7}.$ [A1]\n\nSo $\mathbf{s} = \begin{pmatrix} 4/7 \\ 3/7 \end{pmatrix}$ , and the population is high in the long run for $\dfrac{4}{7}$ of years (about $57.1\%$ ). [A1]\n\nCheck on the GDC: $T^{50}$ has both columns equal to $\begin{pmatrix} 0.5714\ldots \\ 0.4286\ldots \end{pmatrix} = \begin{pmatrix} 4/7 \\ 3/7 \end{pmatrix}$ , confirming the algebra.

✓ FULLY WORKED

EXAMPLE 4

The weather is modelled by a Markov chain with transition matrix $T=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}$ (states: sunny, rainy). If today is sunny ( $\mathbf{s}_0=\begin{pmatrix}1\\0\end{pmatrix}$ ), find the probability it is sunny in 2 days. [4]

1
Model answer — full working.\n\nThe states are ordered sunny then rainy, and each column of $T$ sums to $1$ . Starting sunny, $\mathbf{s}_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ , and the state after 2 days is\n $\mathbf{s}_2 = T^2\mathbf{s}_0.$ \n\nCompute $T^2$ on the GDC:\n $T^2 = \begin{pmatrix} 0.8 & 0.4 \\ 0.2 & 0.6 \end{pmatrix}^2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}.$ \n\nThen\n $\mathbf{s}_2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}\begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0.72 \\ 0.28 \end{pmatrix}.$ \n\nThe first entry is the probability of sunny, so $P(\text{sunny in 2 days}) = 0.72$ .\n\n---\nHow our marking engine awards the 4 marks:\n\n- M1 — set-up. Awarded for using $\mathbf{s}_2 = T^2\mathbf{s}_0$ (matrix raised to the power, on the left of the initial vector). This is the method mark; it stands even if a later arithmetic value slips.\n- M1 — compute $T^2$ . A method mark for correctly squaring the matrix (GDC allowed), $T^2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}$ . The engine checks the power was formed, not that it was done by hand.\n- A1 — state vector. Awarded for $\mathbf{s}_2 = \begin{pmatrix} 0.72 \\ 0.28 \end{pmatrix}$ , the product read off correctly. This accuracy mark depends on the M1s above, and FT applies: a candidate with a slightly different but correctly-multiplied $T^2$ still earns it on their own figures.\n- A1 — final probability. Awarded for identifying the sunny probability as the first entry, $0.72$ . Because $\mathbf{s}_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ simply selects the first column of $T^2$ , a candidate who quotes $0.72$ directly from that column also earns full marks; ISW means later restatements do not lose it.\n\n**'Accept equivalent forms and correct rounding.'** The engine accepts $0.72$ written as $\frac{18}{25}$ or $72\%$ , and accepts the answer whether reached via $T^2\mathbf{s}_0$ or by reading the first column of $T^2$ .\n\nBottom line: of the 4 marks, two are method marks that survive an arithmetic slip, and the accuracy marks are shielded by follow-through. A student who writes only ' $0.72$ ' with no matrix set-up risks losing the two method marks; a student who shows $\mathbf{s}_2 = T^2\mathbf{s}_0$ , the squared matrix and the product keeps the method regardless of a slip in the final number.

✓ FULLY WORKED

How it all connects

The big idea sits in the middle — tap a linked idea to explore the link.

MAIN IDEATransition matrices and Markov chains

Tap a linked idea to see how it connects back to the main topic — that connection is what examiners reward.

Glossary

Try to recall each definition before you reveal it.

Quick check

Answer in your head first — then tap to check. No pressure.

Revision flashcards

Flip the card. Test yourself before the exam.

QUESTION · 1 / 10

Markov chain

TAP OR SPACE TO FLIP

ANSWER

A model of a system that moves between a fixed set of states at discrete time steps, where the probability of the next state depends ONLY on the current state, not on the earlier history. This 'memoryless' property is what lets a single matrix describe the whole process.

TAP OR SPACE TO FLIP BACK

1 / 10 · ← → space

Key takeaways

Review these before you close the topic — retrieval beats re-reading.

✓
Square: one row and one column per state, in a fixed order you choose once.
✓
Columns are 'from', rows are 'to': the entry in row $i$ , column $j$ is $P(\text{go to } i \mid \text{now in } j)$ .
✓
Every column sums to 1: each column is a probability distribution, because from any state the system must transition somewhere (possibly staying put).

Practice — then mark it

The whole point: a real Cambridge question, marked mark-by-mark.

Real past paper[0]

Get a Paper 2 Markov-chain question marked: set up $T$, compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Get a Paper 2 Markov-chain question marked: set up $T$ , compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Do it on paper → mark itMARKED MARK-BY-MARK · B1 / M1 / A1 · OFFICIAL SCHEME

Extra simulations & links

PhET, GeoGebra and other curated tools — open in a new tab.

Frequently asked

✓

Checkpoint

One marked question is worth ten re-reads — close the loop before you move on.

Checkpoint

Reading it isn’t knowing it — prove it.

Before you move on: do Get a Paper 2 Markov-chain question marked: set up $T$, compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working on paper, snap a photo, and get examiner-style feedback on exactly where you win and lose marks.

Attempt & get marked →Takes about a minute · you’ll land back on this lesson

← PREVIOUS4.9 Confidence intervals and further hypothesis testing NEXT →5.1 Introduction to differentiation and rates of change

Transition matrices and Markov chains

BY THE END, YOU CAN…

1Construct a transition matrix $T$ from a described system, checking that each column is a probability distribution summing to 1.
2Set up the initial state vector $\mathbf{s}_0$ and use $\mathbf{s}_n = T^n \mathbf{s}_0$ on the GDC to find the state after $n$ steps.
3Find and interpret the long-run steady state $\mathbf{s}$ satisfying $T\mathbf{s} = \mathbf{s}$ , as the eigenvector for eigenvalue 1 or from a high power of $T$ .
4Apply Markov chains to contexts such as weather, brand-switching and population movement, interpreting the numbers in words.
5Present working so that method marks and follow-through protect the final answer under our Paper 2 marking engine.

In simple terms

A friendly intro before the formal notes — no formulas yet.

Predicting the Future, One Step at a Time

THINK OF IT LIKE…

1
Name the states of the system (weather types, brands, locations) and fix their order once, for good.
2
Build the transition matrix $T$ so each COLUMN lists the probabilities of where you go FROM one state; every column must sum to 1.
3
Write the initial state vector $\mathbf{s}_0$ as a column showing the starting split across the states.
4
Step forward with $\mathbf{s}_n = T^n \mathbf{s}_0$ on the GDC; for the long run, raise $T$ to a high power or solve $T\mathbf{s} = \mathbf{s}$ to read off the steady state.

Explore the concept

Use the live diagram and synced steps — play it or tap a step card to walk through.

Key formulas

Tap any symbol to reveal exactly what it means and its units.

$1, 2, \ldots$

$T\mathbf{s} = \mathbf{s}$

Full topic notes

Formal explanation with the rigour you need for the exam.

States, transitions and the Markov property

Building the transition matrix $T$

Square: one row and one column per state, in a fixed order you choose once.
Columns are 'from', rows are 'to': the entry in row $i$ , column $j$ is $P(\text{go to } i \mid \text{now in } j)$ .
Every column sums to 1: each column is a probability distribution, because from any state the system must transition somewhere (possibly staying put).

The state vector and stepping forward in time

State after $n$ steps: $\mathbf{s}_n = T^n \mathbf{s}_0$ . \n The matrix sits on the LEFT and the state vector on the RIGHT; the product is again a column vector.

EXAM TIP

The long-run steady state

Common mistakes examiners penalise

Rows that sum to 1 instead of columns — in this course columns are the 'from' state and each COLUMN must sum to $1$ . Writing $T$ so the rows sum to $1$ transposes the matrix and every later answer is wrong.
Multiplying in the wrong order — the step is $T\mathbf{s}$ , matrix on the left and column vector on the right. Writing $\mathbf{s}T$ with a column vector is undefined, and swapping to a row vector silently changes the convention.
Using $nT\mathbf{s}_0$ instead of $T^n\mathbf{s}_0$ — you raise the matrix to the power $n$ , you do not multiply the matrix by $n$ . $\mathbf{s}_n = T^n\mathbf{s}_0$ .
Solving $T\mathbf{s}=\mathbf{s}$ without the constraint — the equation $T\mathbf{s}=\mathbf{s}$ alone has infinitely many solutions (any multiple of $\mathbf{s}$ ). You must add $\sum \text{entries} = 1$ (or the population total) to pin down the unique steady state.
Confusing eigenvalue $1$ with eigenvalue $0$ — the steady state is the eigenvector for eigenvalue $1$ (since $T\mathbf{s}=1\cdot\mathbf{s}$ ), not the null space.
Assuming every chain converges — the high-power method needs a regular chain. Absorbing or cyclic chains need not settle to a single interior steady state, so check the context before quoting one.
Rounding populations too early or leaving fractions of people — carry the GDC's full figures, and round headcounts to whole numbers only at the final step.

Model answer — marked the way our engine marks it

Where this leads

Worked examples

See the formulas applied — reveal one step at a time, like the exam.

EXAMPLE 1

1
Fix the state order as B then D, and set columns as 'from', rows as 'to'.\n\n**(a) Transition matrix.** From B: stay $0.8$ , switch to D $0.2$ , giving the first column $\begin{pmatrix} 0.8 \\ 0.2 \end{pmatrix}$ . From D: switch to B $0.3$ , stay $0.7$ , giving the second column $\begin{pmatrix} 0.3 \\ 0.7 \end{pmatrix}$ .\n $T = \begin{pmatrix} 0.8 & 0.3 \\ 0.2 & 0.7 \end{pmatrix}.$ \nEach column sums to $1$ , as required. [A2: fully correct matrix; A1 for one error]\n\n**(b) After 2 months.** The initial state vector is $\mathbf{s}_0 = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}$ , and $\mathbf{s}_2 = T^2 \mathbf{s}_0$ . [M1: correct set-up]\nOn the GDC, $T^2 = \begin{pmatrix} 0.70 & 0.45 \\ 0.30 & 0.55 \end{pmatrix}$ , so\n $\mathbf{s}_2 = \begin{pmatrix} 0.70 & 0.45 \\ 0.30 & 0.55 \end{pmatrix}\begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix} = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}.$ [M1: compute $T^2 \mathbf{s}_0$ ]\nAfter 2 months Bean Scene holds $60\%$ and Daily Grind $40\%$ . [A1]\n\nThe share happens to return to the start here; that is a coincidence of these numbers, not yet the steady state.

✓ FULLY WORKED

EXAMPLE 2

1
Order the states C, S, P, with columns 'from' and rows 'to'.\n\n**(a) Transition matrix.\n $T = \begin{pmatrix} 0.85 & 0.15 & 0.10 \\ 0.10 & 0.75 & 0.10 \\ 0.05 & 0.10 & 0.80 \end{pmatrix}.$ \nEach column sums to $1$ . [A2: fully correct; A1 for one error]\n\n(b) After 6 months.** The total is $20\,000 + 50\,000 + 30\,000 = 100\,000$ , and $\mathbf{s}_0 = \begin{pmatrix} 20000 \\ 50000 \\ 30000 \end{pmatrix}$ . Compute $\mathbf{s}_6 = T^6 \mathbf{s}_0$ on the GDC. [M1]\n $\mathbf{s}_6 \approx \begin{pmatrix} 42085 \\ 36193 \\ 21722 \end{pmatrix}.$ \nAfter 6 months, approximately $42\,085$ with ConnectFast, $36\,193$ with StreamEasy and $21\,722$ with PingKing (rounded to whole customers). [A2: A1 for one slip]\n\n**(c) Long term.** Raise $T$ to a high power, e.g. $T^{50}$ , whose columns become identical. [M1]\n $T^{50} \approx \begin{pmatrix} 0.4651 & 0.4651 & 0.4651 \\ 0.3488 & 0.3488 & 0.3488 \\ 0.1860 & 0.1860 & 0.1860 \end{pmatrix},$ \nso the steady-state proportions are $\mathbf{s} \approx \begin{pmatrix} 0.4651 \\ 0.3488 \\ 0.1860 \end{pmatrix}$ . Multiplying by the constant total $100\,000$ : [M1]\nConnectFast $\approx 46\,510$ , StreamEasy $\approx 34\,880$ , PingKing $\approx 18\,610$ . [A1]\n\nIn the long run ConnectFast dominates the market, even though it started with the fewest customers.

✓ FULLY WORKED

EXAMPLE 3

1
Let the steady state be $\mathbf{s} = \begin{pmatrix} x \\ y \end{pmatrix}$ with $x + y = 1$ . It satisfies $T\mathbf{s} = \mathbf{s}$ . [M1: set up $T\mathbf{s} = \mathbf{s}$ ]\n\nThe top row gives\n $0.7x + 0.4y = x \;\Rightarrow\; -0.3x + 0.4y = 0 \;\Rightarrow\; 0.3x = 0.4y.$ [M1: form a usable equation]\nSo $x = \dfrac{0.4}{0.3}\,y = \dfrac{4}{3}\,y$ . Substituting into $x + y = 1$ : [M1: use $x+y=1$ ]\n $\tfrac{4}{3}y + y = 1 \;\Rightarrow\; \tfrac{7}{3}y = 1 \;\Rightarrow\; y = \tfrac{3}{7}, \quad x = \tfrac{4}{7}.$ [A1]\n\nSo $\mathbf{s} = \begin{pmatrix} 4/7 \\ 3/7 \end{pmatrix}$ , and the population is high in the long run for $\dfrac{4}{7}$ of years (about $57.1\%$ ). [A1]\n\nCheck on the GDC: $T^{50}$ has both columns equal to $\begin{pmatrix} 0.5714\ldots \\ 0.4286\ldots \end{pmatrix} = \begin{pmatrix} 4/7 \\ 3/7 \end{pmatrix}$ , confirming the algebra.

✓ FULLY WORKED

EXAMPLE 4

1
Model answer — full working.\n\nThe states are ordered sunny then rainy, and each column of $T$ sums to $1$ . Starting sunny, $\mathbf{s}_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ , and the state after 2 days is\n $\mathbf{s}_2 = T^2\mathbf{s}_0.$ \n\nCompute $T^2$ on the GDC:\n $T^2 = \begin{pmatrix} 0.8 & 0.4 \\ 0.2 & 0.6 \end{pmatrix}^2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}.$ \n\nThen\n $\mathbf{s}_2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}\begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0.72 \\ 0.28 \end{pmatrix}.$ \n\nThe first entry is the probability of sunny, so $P(\text{sunny in 2 days}) = 0.72$ .\n\n---\nHow our marking engine awards the 4 marks:\n\n- M1 — set-up. Awarded for using $\mathbf{s}_2 = T^2\mathbf{s}_0$ (matrix raised to the power, on the left of the initial vector). This is the method mark; it stands even if a later arithmetic value slips.\n- M1 — compute $T^2$ . A method mark for correctly squaring the matrix (GDC allowed), $T^2 = \begin{pmatrix} 0.72 & 0.56 \\ 0.28 & 0.44 \end{pmatrix}$ . The engine checks the power was formed, not that it was done by hand.\n- A1 — state vector. Awarded for $\mathbf{s}_2 = \begin{pmatrix} 0.72 \\ 0.28 \end{pmatrix}$ , the product read off correctly. This accuracy mark depends on the M1s above, and FT applies: a candidate with a slightly different but correctly-multiplied $T^2$ still earns it on their own figures.\n- A1 — final probability. Awarded for identifying the sunny probability as the first entry, $0.72$ . Because $\mathbf{s}_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ simply selects the first column of $T^2$ , a candidate who quotes $0.72$ directly from that column also earns full marks; ISW means later restatements do not lose it.\n\n**'Accept equivalent forms and correct rounding.'** The engine accepts $0.72$ written as $\frac{18}{25}$ or $72\%$ , and accepts the answer whether reached via $T^2\mathbf{s}_0$ or by reading the first column of $T^2$ .\n\nBottom line: of the 4 marks, two are method marks that survive an arithmetic slip, and the accuracy marks are shielded by follow-through. A student who writes only ' $0.72$ ' with no matrix set-up risks losing the two method marks; a student who shows $\mathbf{s}_2 = T^2\mathbf{s}_0$ , the squared matrix and the product keeps the method regardless of a slip in the final number.

✓ FULLY WORKED

How it all connects

The big idea sits in the middle — tap a linked idea to explore the link.

MAIN IDEATransition matrices and Markov chains

Tap a linked idea to see how it connects back to the main topic — that connection is what examiners reward.

Glossary

Try to recall each definition before you reveal it.

Quick check

Answer in your head first — then tap to check. No pressure.

Revision flashcards

Flip the card. Test yourself before the exam.

QUESTION · 1 / 10

Markov chain

TAP OR SPACE TO FLIP

ANSWER

TAP OR SPACE TO FLIP BACK

1 / 10 · ← → space

Key takeaways

Review these before you close the topic — retrieval beats re-reading.

✓
Square: one row and one column per state, in a fixed order you choose once.
✓
Columns are 'from', rows are 'to': the entry in row $i$ , column $j$ is $P(\text{go to } i \mid \text{now in } j)$ .
✓
Every column sums to 1: each column is a probability distribution, because from any state the system must transition somewhere (possibly staying put).

Practice — then mark it

The whole point: a real Cambridge question, marked mark-by-mark.

Real past paper[0]

Get a Paper 2 Markov-chain question marked: set up $T$, compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Get a Paper 2 Markov-chain question marked: set up $T$ , compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Do it on paper → mark itMARKED MARK-BY-MARK · B1 / M1 / A1 · OFFICIAL SCHEME

Extra simulations & links

PhET, GeoGebra and other curated tools — open in a new tab.

Frequently asked

✓

Checkpoint

One marked question is worth ten re-reads — close the loop before you move on.

Checkpoint

Reading it isn’t knowing it — prove it.

Attempt & get marked →Takes about a minute · you’ll land back on this lesson

← PREVIOUS4.9 Confidence intervals and further hypothesis testing NEXT →5.1 Introduction to differentiation and rates of change

Revise Transition matrices and Markov chains for IB Mathematics: Applications and Interpretation (4.10)

In simple terms

Explore the concept

Key formulas

Full topic notes

States, transitions and the Markov property

Building the transition matrix $T$

The state vector and stepping forward in time

The long-run steady state

Common mistakes examiners penalise

Model answer — marked the way our engine marks it

Where this leads

Worked examples

How it all connects

Glossary

Quick check

Revision flashcards

Key takeaways

Practice — then mark it

Get a Paper 2 Markov-chain question marked: set up $T$, compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Extra simulations & links

Frequently asked

Checkpoint

Reading it isn’t knowing it — prove it.

In simple terms

Explore the concept

Key formulas

Full topic notes

States, transitions and the Markov property

Building the transition matrix $T$

The state vector and stepping forward in time

The long-run steady state

Common mistakes examiners penalise

Model answer — marked the way our engine marks it

Where this leads

Worked examples

How it all connects

Glossary

Quick check

Revision flashcards

Key takeaways

Practice — then mark it

Get a Paper 2 Markov-chain question marked: set up $T$, compute $\mathbf{s}_n = T^n\mathbf{s}_0$ and find the steady state with full working

Extra simulations & links

Frequently asked

Checkpoint

Reading it isn’t knowing it — prove it.

Discuss Transition matrices and Markov chains