A deep dive into Stonehaven's 2024 UK General Election MRP and what it tells us about voters
Luke Betham
Stonehaven’s 2024 UK election MRP accurately predicted the outcome of the election, only 38 seats out from the overall seat totals, making it the closest model in the UK.
How the model performed in more detail
Our MRP model demonstrated remarkable accuracy. When examining the results of the four major political parties of Great Britian, our predictions deviated by an average of only 7 seats, the closest of any other published model in the election. By constituency, the outcomes were also within our probable range (above 5% chance) for 95% of constituencies, again indicating the model’s strength.
Our model effectively captured the significant shifts in voter behaviour and the changing political landscape, correctly anticipating the successes of the Liberal Democrats, and Reform getting a higher level of the popular vote and only securing a handful of seats.
Challenges with this election for MRP modelling
Predicting the 2024 UK general election posed several challenges due to its dynamic - and in some cases unprecedented - nature:
Turnout predictions
Voter turnout was particularly difficult to predict for several reasons. Firstly, this topic exacerbates the divide between what people say they will do and the action they eventually take. If taken at face value, surveys typically indicate a turnout of above 90%, though turnout in the last few elections have been below 70%. This meant that we needed to build sub-models to predict who will act on their poll answers, versus those who will not.
On the day, the final turnout figures were just under 60%, the lowest since 2001. In Leeds South, the turnout was only 42% of the electorate. This indicates how vital this sub-model was in providing accurate predictions.
Polling biases
Predicting the actual percentage of votes for each party can be challenging due to inherent biases in the raw polling data. While MRP is effective at addressing these issues, the variability still poses a significant challenge. The visual below, using the West Midlands as an example, showcases how exaggerated these biases can be. In our surveys, about 51% of respondents said they would be voting for Labour, but on polling day this figure was 34%. As you can see, our MRP is able to correct for this bias by more accurately modelling predicted behaviour of particular subgroups in constituencies. It shows that MRP is useful not only for granular breakdowns of a population but also for correcting survey biases.
Tactical voting
We began the project with a tactical voting model, but its complexity quickly increased as the election approached and more people reacted to polls. The increased number of viable parties in constituencies (mainly Reform UK and the Liberal Democrats) also led to more vote splitting and an increased number of tactical voting scenarios. Several seats ended up as near three-way ties, complicating predictions.
In addition, in the week before the election, our data indicated that approximately 5% of the electorate were considering changing their vote from Labour back to Conservative, concerned about a Labour “supermajority” and the need to consider a viable opposition. We then built and monitored a scenario which showed what would happen if all these voters decided to switch back and show how large an effect this inverse tactical voting could have.
In addition to tactical voting, we have been closely monitoring a few sets of swing voter segments which we predicted in early 2023 were going to define this election. These segments are values-based segments of new Labour voters, and it allowed us to understand which policies and campaign messages were most important to each group, which vary largely in the reason they are voting Labour. These are also now the groups that the Labour party will need to keep onside over the next 5 years.
Vote swing and maginality
Models often start with the previous election as a baseline. However, this election saw the largest swing in decades, with Reform UK (or its predecessor, the Brexit Party) not standing in any seats in the previous election.
The results were much more marginal than in the past, with many seats decided by tens or hundreds of votes.
The rise of independents
The rise of independent candidates focusing on single-issue campaigns added another layer of complexity to the political landscape. The election resulted in several seats being won by independent parties in key locations such as Leicester South, Birmingham Perry Barr, Blackburn, and Bradford West, underscoring the need for robust modelling techniques and adaptable algorithms to capture the nuances of voter behaviour and political dynamics around these issues.
Why Stonehaven's MRP model performed so well
The range of predictions for Labour varied significantly, from 418 to 470 seats, with 470 being outside our model's credible intervals. So why do these MRP models differ so much?
Bayesian Modelling Methodology
Rather than using on very large 20,000 respondent snapshot poll (as most pollsters do), our approach used:
- - ~70 polls over 36 months (2,000-respondents each)
- - 5 polls weekly polls from the date the election was announced (2,000-respondents each)
- - A final poll for 28-30th June 2024 (5,000 respondents)
Our Bayesian methodology resulted in a poll that allows us to synthesise data from approximately 120,000 unique respondents over three years, with more recent data weighted more heavily, making it more like a weighted moving average (i.e. most of the model will be based on the most recent polls). This resulted in a snapshot of voter intention without being too heavily influenced by a specific date. It allows us to see how the electoral picture changes over time, without having to run incredibly large polls each time.
Data Quality
We invested in high data quality by eliminating bots, minimizing survey drop-outs, and removing respondents speeding through the survey, which ensures more reliable and representative data, leading to cleaner model outputs. Unlike some MRP polls that prioritize huge sample sizes and overlook potential data quality issues, our focus on quality can have a larger impact than smarter modelling.
Survey Design and Question Framing
Our survey design significantly impacted model accuracy. Different pollsters treat respondents who select “I don’t know” differently, with many opting to create sub-models to predict how these people will vote on election day. Our survey design forced these respondents into a hypothetical decision, thus eliminating the need for additional sub-models for this issue.
Model Specifications
How an MRP is designed, including the assumptions it makes, can significantly impact its predictions. For example, if a pre-election poll was taken at face value, it might have predicted a +90% voter turnout and over 500 seats for Labour. We needed to account for local nuances within each constituency to ensure our model reflected real-world complexities and recognized smaller parties.
Among other factors, we considered by-election results, the correlation between religious populations and protest votes, and the “hero/villain” effect of different candidates (e.g., Jeremy Corbyn and Nigel Farage) and tactical voting (detailed below). These elements added complexity but also enhanced the model's accuracy by incorporating the full range of influences on voter behaviour.
Tactical Voting
An example of a specific model specification would be the addition of tactical voting, which became more complex and self-referential throughout the election period, as MRP models became increasingly published and discussed in the media, which, in turn, may have impacted how people intended to vote.
We built a tactical voting sub-model to handle the increased number of viable parties, and accommodated people who may be inclined to vote both for and against one of these parties. This sub-model considered not only respondents’ awareness of their constituency’s results in the last election, but their understanding of what might be happening in their constituency in the current election. It then further assessed how likely individuals were to act on this knowledge, understanding that, like turnout, respondents often overestimate their own likelihood to tactically vote.
We correctly identified that voters were going to tactically vote against the SNP in Scotland, and this led to our model projecting a collapse of SNP seats from 48 to 22. Ultimately, this impact was even larger than respondents were reporting, and the SNP ended up with only 9 seats.
THE VALUE OF MRP USAGE BEYOND ELECTIONS - ISSUES BASED MODELLING
Outside of the field of political polling, it is rare that the findings from a nationally representative research project are validated by asking an entire population to vote – especially within a week of fieldwork.
As a strategic advisory firm, we regularly utilise MRP modelling to help our clients gain deeper understanding of issues that matter to the public and use this knowledge to help them run effective campaigns. The quality of our work depends on our ability to accurately identify, measure and quantify a broad range of influences on public opinion, as we use this to predict how people will react to different issues. For example, see our recent modelling work on onshore wind farms below:
However, few cases include the variety of factors which are as fast-moving, wide-ranging, or complex as the factors which influence voting in the run up to the UK general election. So although Stonehaven is not a political polling agency, the 2024 UK General Election offered us an irresistible testing ground for our modelling capabilities. Through the complexities of the 2024 UK general election, we have put our survey design, modelling capabilities and data utilisation to the test, and found them to be remarkably precise.