What is a multi-state model?
What a multi-state model is: transition hazards, the illness–death model, state probabilities, length of stay, clock forward or reset, and the data set-up.
A multi-state model describes how people move between states over time, such as event-free, relapsed and dead. Each possible move is a transition with its own hazard, and from those hazards the model gives the probability of being in each state at any time, and the expected time spent there. Ordinary survival analysis is the simplest case, with two states, alive and dead.
States, transitions and hazards
A transition hazard is the rate of a move among the people currently in the state it leaves. A state that people can leave is transient; one they cannot leave, such as death, is absorbing (Putter, Fiocco and Geskus, Stat Med 2007). The allowed moves are set out in a transition matrix, with a row and a column for each state and a number for each allowed transition (de Wreede, Fiocco and Putter, J Stat Softw 2011).
The standard example is the illness–death model. Everyone starts event-free, for example after surgery for cancer, and can relapse and later die, or die without relapsing; a covariate can act differently on each of the three transitions. Competing risks are the special case with one transient state, the starting one (see what are competing risks?). A multi-state model adds intermediate states.
What you can estimate
- Transition hazards and covariate effects, from a survival model for each transition, such as a Cox or a parametric model, fitted to the people at risk of it.
- State occupation probabilities, of being in each state at time , and transition probabilities, the same given the state occupied at an earlier time . The Aalen–Johansen estimator, which extends Kaplan–Meier to several states, estimates them non-parametrically within groups of patients; a regression model builds them from all the fitted hazards for any covariate pattern.
- Expected length of stay in a state up to a time horizon, the area under its state occupation probability curve, which feeds into life-years, QALYs and costs.
These probabilities depend on every transition hazard, so a hazard ratio for one transition does not give its effect on the chance of being in a state. Take constant hazards of 0.10 per year for relapse, 0.02 for death without relapse and 0.25 for death after relapse. After 10 years, 30% are alive and relapse-free and 53% have died. Halving the relapse hazard alone raises the first to 50% and lowers the second to 39%, because it keeps people in the state with the lower death hazard. Expected time alive in the first 10 years, the area under the survival curve to 10 years, rises from 7.5 to 8.2 years. A hazard ratio of 0.5 for relapse, with no death hazard changed, cuts the 10-year risk of death by only 27%.
Clock forward or clock reset?
Each hazard needs a time scale. With clock forward, time runs from the start, such as surgery, for every transition. With clock reset, it restarts at zero on entry to each state, so the hazard of death after relapse depends on the time since relapse. Clock-forward models are often assumed to be Markov, meaning that the time at which a patient relapsed does not change their later hazard of death, given the time since surgery. A clock-reset model whose hazards depend only on the current state and the time since entering it is called semi-Markov (Putter, Fiocco and Geskus, 2007).
Choose from the clinical question, whether prognosis after relapse depends more on the time since relapse or on the time since surgery. In Putter and colleagues’ experience, the choice changes the estimated regression coefficients little. In a clock-forward model, add the time of entry into a state as a covariate for the transitions out of it; a clear effect means the Markov assumption does not hold, although no effect does not prove it does (Titman and Putter, 2022). In a clock-reset model the same check tests the semi-Markov assumption.
For prediction, the Aalen–Johansen estimator of transition probabilities assumes a Markov model. Its estimates of state occupation probabilities stay valid without that assumption when censoring is unrelated to patients’ states and histories, as with administrative end of follow-up (Datta and Satten, 2001); under the same condition a landmark version gives transition probabilities (Putter and Spitoni, 2018). Predictions from a clock-reset model can be made by simulating many paths through the states (Crowther and Lambert, Stat Med 2017).
How the data are set up
The data have one row per person for each transition they are at risk of, from each state they enter, with a start time, a stop time and a status, 1 if that transition happened and 0 if not. A patient who relapses at 2 years and is alive at 5 has three rows: relapse, from 0 to 2, status 1; death without relapse, from 0 to 2, status 0; and death after relapse, from 2 to 5, status 0 (clock forward; with clock reset this row runs from 0 to 3). Each transition’s model is then fitted to its own rows (Putter, Fiocco and Geskus, 2007).
Our multi-state package pendragon, which is in development and not yet released, will do this set-up with pendragon set in Stata and pendragon_set() in R, from one row per person or from one row per recorded state and time; the second form also allows recovery and repeated visits to a state. The mstate package for R covers the same steps for non-parametric and Cox models (de Wreede, Fiocco and Putter, 2011). Our tutorials show how to define a transition matrix and fit a Cox model for each transition.
Common mistakes
- A transition matrix that does not match the data. It must include every transition the model allows, even one that nobody in the data makes, and each fitted model must be matched to its transition number. Check the count of each transition against the model you meant.
- Leaving people out of a transition they were at risk of. A person at risk of several transitions contributes a row to each. Someone who dies without relapsing belongs in the relapse model, censored at death; dropping them overstates the relapse hazard.
- Forgetting delayed entry. With clock forward, a patient is at risk of death after relapse only from the time they relapse. Counting them at risk from time zero is a form of immortal time bias.
- Reading a hazard ratio as an effect on risk. Report predicted probabilities or length of stay as well.
Where multi-state models are used
- Disease progression, such as relapse followed by death in cancer, or the events that follow a bone marrow transplant (Putter, Fiocco and Geskus, 2007; Andersen and Keiding, Stat Methods Med Res 2002).
- Health economic models. Cost-effectiveness models often use three states, progression-free, progressed and dead. A multi-state model estimates each transition from patient data; a partitioned survival model takes time progressed from the gap between two survival curves. In the CLL8 trial of first-line treatment for chronic lymphocytic leukaemia, partitioned survival, Markov and multi-state models gave incremental cost-effectiveness ratios of about £16,000, £13,000 and £29,000 (Williams et al., Med Decis Making 2017).
- Recurrent hospitalisation. Ieva, Jackson and Sharples modelled serial hospital admissions and death in heart failure, and estimated readmission rates, length of stay in hospital and expected total time in hospital over five years (Stat Methods Med Res 2017).