Sign in. It’s quick, free and it’s up to you.
An account is an optional way to support the work we do. Find out more.
OPINION POLLS ARE one of the main talking points of the recent local and European Parliament elections. Tánaiste Micheál Martin and Jack Chambers, Fianna Fáil Director of Elections, publicly criticised pollsters using online panels.
Fianna Fáil has good reason to be frustrated. General election polls have significantly underestimated support for Fianna Fáil and vastly exaggerated support for Sinn Féin. For them, this creates a mistaken media narrative about popular support for Sinn Féin.
But does this underestimation of support for FF in general election polls mean that we should completely ignore opinion polls? Do they still have value for politicians, journalists and voters? How can the media, politicians and voters engage more constructively with public opinion polls? The starting point is to recognise that general election polls do not measure support for parties in local elections. Irish general election polls have always been a poor predictor of local elections, but are good predictors of general election results.
One criticism raised by politicians and commentators after these elections relates to the huge difference between support in current opinion polls for the upcoming general election and the local election results.
Let’s untangle local and general election polls. Are opinion polls for general elections accurate? This is the first question I will address.
The Irish Polling Indicator project, which Tom Louwerse (Associate Professor, Leiden University) and I started 10 years ago, aggregates all opinion polls into a single estimate of public support.
We have collected all opinion polls published since the mid-1980s, resulting in a sample of 717 surveys.
This tedious exercise involved going through newspaper archives, checking various websites and contacting academics. The resulting dataset not only allows us to present aggregated estimates but also helps us understand how well polls predict elections.
To assess the accuracy of polls, I extracted the Polling Indicator estimate on the day of the election and compared the figures to the actual first preference vote shares. Next, I calculate the absolute mean error, which is the difference between the first-preference vote share at the election and the aggregated estimate. If a party obtained 25 per cent of first-preference votes, but the poll of polls estimated support at 21 per cent, the absolute error for this party amounts to 4 percentage points. This analysis is performed for all parties to calculate the average absolute difference.
The graph below reveals that the absolute mean error results to be just 1.32 percentage points across the eight general elections between 1989 and 2020. In other words, on average, the poll of polls is just 1.3 points off the election result. This is very accurate.
From a political science perspective, it is also very encouraging: most estimates fall within the margin of error. There are, of course, exceptions to the rule. In 2002, the polls overestimated support for Fianna Fáil by 4.6 percentage points but underestimated the party’s support by 4.4 points in 2007 and 3.4 points in 2016.
In 2020, the aggregated polls overestimated support for Fianna Fáil by 2.4 points. In 2016, the polls overestimated support for Sinn Féin and Fine Gael by around 3 points each. Yet, overall, polls have done a pretty good job in predicting party support in Irish general elections over the past 25 years.
What about Local Elections? Pollsters received a lot of criticism for underestimating Fianna Fáil support and vastly overestimating the Sinn Féin vote last week. Yet, opinion polls do not intend to predict local election results.
Looking at the second figure here supports this argument. The absolute mean differences range from 2.46 (2004) to 3.77 (1991) points. Last weekend, the mean difference was 3.18 points. At almost every election, the estimates for at least one party are completely different to the outcome.
Why are the polls so far off? Irish local elections are unique. Incumbency plays a huge role in Local Elections. Our research shows that even incumbents who got elected on the last count face a huge advantage in rerunning and getting re-elected compared to the candidate who lost on the last count. Canvassing, name recognition and community ties are highly relevant. Door-to-door campaigning is a unique feature of Irish politics and it is a crucial condition for getting elected.
An Ireland Thinks poll fielded on the day of the local and European Parliament elections mirrors these claims. 75 per cent of Fianna Fáil voters cast their first preference for a candidate rather than the party the candidate represented. 66 per cent of Fine Gael voters ranked the party over the candidate.
The focus on candidates is lower for smaller parties – possibly because they ran fewer incumbents and their candidates were simply not as well known.
Fianna Fáil and Fine Gael also canvassed very successfully. 35 per cent of respondents said they met a Fianna Fáil candidate at their doorstep. 34 per cent of respondents had direct contact with a Fine Gael candidate. Only 18 per cent met a Sinn Féin representative. This might be part of the explanation why Sinn Féin performed even more poorly than anticipated: they are not only losing support in opinion polls, but they had few incumbents and their candidates were simply not as active on the ground as Fianna Fáil and Fine Gael. To sum up, incumbency and local ties are crucial, and Fianna Fáil and Fine Gael outperformed Sinn Féin in this hugely important aspect of campaigning.
We need to keep in mind that the electorate is becoming more volatile. According to Exit Polls from the past 25 years, around half of the voters make up their minds about the first preference vote in the last month before an election. Since the financial crash, around four in 10 voters have switched between parties from one general election to the next. Campaigns and canvassing are crucial in general elections and can reshuffle the Irish political landscape.
The debate between the efficacy of online and face-to-face polls continues to be contentious. While some, like Fianna Fáil’s Micheál Martin and Jack Chambers, argue that online polls tend to underestimate their support, face-to-face polls also present challenges, such as low response rates and social desirability biases. Red C Research, despite being criticised at the moment, accurately predicted Fianna Fáil’s performance in the 2020 general election.
Ideally, we rely on a mix of methodological approaches to better understand Irish party politics. Telephone polls, face-to-face surveys, and online surveys all have advantages and shortcomings. Michael A. Bailey, Professor at Georgetown University, describes these issues in an excellent recent book “Polling at a Crossroads”. Having an open mind towards public opinion will ultimately improve our understanding of public opinion.
What can we learn from this debate on opinion polls and how can we move forward the debate? I have four recommendations:
1. Media outlets should adjust their reporting. We must consider structural differences between polling companies, focus on trends rather than individual polls and not overplay small “changes” lying within the margin of error as “bounces”, “rebounds”, and “recoveries”. Sampling variation drives most of these “changes”, which have usually been within the margin of error of ±3 percentage points. Political scientists Erik Gahner Larsen and Zoltán Fazekas show that Danish news reports frequently interpret deviations within the margin of error as substantive changes. The authors conclude that “a horserace coverage of polls about change can rest on a foundation of stability.” Irish newspapers often apply the same rhetoric. A move away from horserace coverage is necessary.
2. In addition to changing the language, polling companies and newspapers could present party support as within windows (i.e., the 95 confidence interval). Reporting single numbers projects a false sense of precision. Other countries are slowly adopting this approach. For example, Italian media reported the results from the exit poll for the European Parliament elections in intervals. Rather than focusing on a single number – and suggesting a wrong sense of certainty – their headline finding was that “Meloni’s FdI to get 26-30% in EU elections – exit poll Centre-left Democratic Party to come second with 21-25%”. Irish media should follow suit.
3. We should focus more on trends. For example, the polling trends clearly showed the loss of support for Sinn Féin in general election polls – across all companies. While pollsters deviate systematically in their survey mode and baseline results, these developments are unambiguous. Voters should approach poll results critically, considering the broader context and acknowledging uncertainty.
4. We need to keep in mind that polls provide a single snapshot and that using polls to predict elections is extremely difficult. Paul Perry, back then Vice Chairman of Gallup Polling, focused on these problems in a Public Opinion Quarterly paper published 45 years ago:
All one has to do is use a properly drawn sample of the electorate large enough to minimise random sampling error, get honest answers from everyone, do the questioning close enough to the time of voting to minimise changes in voting intentions, anticipate how the undecided will vote, and, finally, distinguish between voters and non-voters in the electorate – (Perry, 1979: page 312).
Pollsters should need to be even more transparent in their reporting methods. While most outlets now list the poll’s margin of error and time when the survey was in the field, the exact approach of selecting households for face-to-face polls, selecting respondents from the online panel, non-response rates, and the weighting of respondents is still a black box. This is understandable because polling is highly complex and relies on cutting-edge statistical methods. Media outlets could add a few sentences about the sample selection and weighting.
Finally, it is important to note that polls are extremely useful for understanding voter preferences and behaviour. Journalists and politicians need polls to understand the electorate. I believe the current debate on the usefulness of polls is constructive and ought to lead to important reflections among all actors involved. I hope that mentioning methods and uncertainty will increase trust in public opinion. Ultimately, these changes would help us to focus on policy issues and the solutions that parties offer.
Stefan Müller is Assistant Professor in the School of Politics and International Relations at University College Dublin. He maintains the Irish Polling Indicator and the Irish Politics Data project.
To embed this post, copy the code below on your site
Siding with the government in calling concerned citizens racists and bigots when busloads of migrant men are being forced on communities all over Ireland certainly didn’t help their cause. They need to drop the woke nonsense and the Hamas cheerleading to gain some credibility back.
@sean mcnamara: I’m no fan of SF, but they and the government are right to oppose what Israel is doing in Gaza. To call this ‘Hamas cheerleading’ is incorrect and dishonest.
@Comments Section Closed: At least I have only one profile, unlike some people.
@Comments Section Closed: Well, I’ve never seen your profile before, yet you’re obviously not new to the site.
I understand that you probably have indigestion over the far right’s abysmal election showing. I sympathise insincerely.
@Brendan O’Brien: There’s a big difference between opposing what Israel is doing in Gaza and the SF policy of cosying up to the terrorist Hamas organisation as well as their effective chants for the abolishment of Israel as seen at their and fheis.
And there’s a big gap between their housing promises and their support for open borders and uncontrolled immigration.
@Thomas Sheridan: support for open borders and uncontrolled migration? You do spout a load of nonsense
@sean: then how did the govt win while calling ‘concerned citizens’ racists? None of your anti immigrant parties won a seat. Wipeout.
@Thomas Sheridan: doesn’t Simon Harris side with Palestine now lol? No one bought your open border uncontrolled immigration because your anti immigrant parties didn’t win one seat in the locals that has the most seats and easiest to win.
Really hope mary looper stays on she is doing a fantastic job at destroying SF lol
@9QRixo8H: Simon Harris doesn’t side with the Hamastinians or other associated terrorist thugs in the ME
He has called for the implementation of a two state solution – something your Ham Ass friends have rejected.
https://avalon.law.yale.edu/20th_century/hamas.asp
@sean mcnamara: As soon as I see the word ‘woke’ I automatically stop reading.
@Thesaltyurchin:
Is this not a term that “woke” folk termed themselves ? As I soon as I see “woke” I just think gullible.
Mary Lou took the party to WOKE if we want FF we can vote FF…..she has been a disaster
@Renshai Renshai: You do know that SF presents itself as a left-wing party? Why would you expect right-wing policies from a left-wing party?
@Renshai Renshai: none of your anti-WOKE (bigoted) parties won a seat. Not one seat. The woke FFG won the election. Even with their failed referenda and apparently “open border” migration policies, they won.
@9QRixo8H: your lovely new immigrant’s are mutilating women at record levels but your probably down with that send them all home, the right isn’t gone anywhere you know
@Paul Hackett: You want to deport the women who are being mutilated?
@Brendan O’Brien: what age are you ,dope
@Paul Hackett: it’s a pity you are still here
@Joe Kelly: I’d say there’s a lot of pity in your life
I assume that many commenters realise that the rise of a political group is a reaction to the attempt to control human behaviour too much, which can be construed as far-right.
The Journal and RTE may take their cues from the academic community operating through social politics, but there is a limit to what people will accept as model behaviour, especially unreasonable hype from the modelling community and their dire predictions passed off as facts.
I hope that politicians regain a sense that they represent the people rather than being driven by models and academic convictions that have no place in a productive and creative society.
People may call me whatever they wish; however, I will call out dehumanisation reactions when I see it.
@Gerald Kelleher: You are so obsessed with the academic community and ‘modelling’ that you apparently ignore the real world completely.
Here, for instance, is what climate change is doing to wine production in Italy:
@Bren:
Prejudice has never been rejected, soundly or otherwise, in Irish society, for although the academic community is unable to hide Victorian natural selection behind eugenics or social darwinism any longer, there is no organisation or group of people willing or able to see terms like racism, racial, biracial, racist as invalid biological language.
Natural selection is about human breeding and not evolution, the concept that some people deserve to thrive at the expense of less favoured ‘races’.
‘On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life’ Darwin
Irish people are behaving like Brexiteer politicians, especially the Victorian imitator. there is no left or right politics here, just sordid academic politics
@Brendan O’Brien: I remember in 1983 when climate change modelling started to appear and was eventually weaponised by computer modelling. At that time, some theorists were also seeing the rise of phenomena.
“A Lagrangian [model] is not a physical thing; it is a mathematical thing – a kind of differential equation, to be exact. But physics and maths are so closely connected these days that it is hard to separate the numbers from the things they describe. Nobel Prize winner Burton Richter of Stanford said: ” Mathematics is a language that is used to describe nature” he said “But the theorists are beginning to think it is nature. To them, the Lagrangians/models are the reality ” Discover Magazine,1983
@Brendan O’Brien: the Guardian has become biased. I gave up a subscription to them after nearly 5 years. Every article is doom and gloom and only from one side of the argument. Gerald is correct about the influence of academia on social issues. Not every one ascribes to the far left view of the various NGOs and academics which have too much policy influence. It’s a bit like the Government leaving all policy and governance to Doctors and Modelers during COVID. No political risk in absolving all responsibility to a third party even when the dogs in the street realised it was not the black death after year one . Look how the Greens hobbled nuclear power back in the day causing the Carbon Crisis today .
@Comments Section Closed:
Your dehumanisation comment to O’Brien marks you as compliant in the same way as the SS once were.
It doesn’t offend me as there are many commenters like you out there, to crude to know what cruelty looks like.
@gregory pym: ‘The Guardian has become biased’ is a case of shooting the messenger. Are you going to tell Paola Ferrari that she’s not really losing her grape crop?
@gregory pym:
The direction of travel with COVID-19 is that the experimental community were attempting to make a virus more infectious, so whether it escaped from the lab or not, it was a disaster even before it can be considered an accident where millions died and lives were disrupted.
This is the second serious misadventure with biology, with Victorian natural selection being the first. It was swept under the carpet after WWII as social darwinism despite the original invasion and extermination imperatives.
It is a juncture in European and world history where politicians are asked to represent society rather than being shills for the indulgences of mathematical modellers.
@Comments Section Closed:
Buy yourself an English keyboard.
My voice is cultural, the ability to see where experimental theorists have taken society by using social politics as a conduit. What is seen as far-right is a reaction to this societal imbalance where people wake up every day to hear that by controlling human behaviour, they can control the weather/temperatures from media outlets acting on behalf of academic and social politicians.
Too crude to know what cruelty is, I leave you to your own devices.
@Brendan O’Brien:
You may not see it, but I am certain more considerate commenters will.
“Climate models are systems of differential equations based on the basic laws of physics, fluid motion, and chemistry.” Climate models
https://en.wikipedia.org/wiki/Climate_model
I cannot offer to demonstrate what went wrong with any specific group, as I have no financial or reputational gain other than it is highly dangerous for a productive society. It would be helpful to have people involved but academic politics has made certain that it protects itself beginning with its hapless supporters and their noise.
Mary Loo is Fianna Fail lite and does not represent the change she keeps banging on about
It’s a great result for the media.
@087 bed: You poor thing. Thoughts and prayers.
@087 bed: It’s a great result for democracy.
@087 bed: The media report and also comment. The comment part will have plenty to say about mistakes and failures.
Ti them it does not matter whose, except whose will get the most eyeballs on it
The right wing have made gains all over Europe including Ireland and FG and FF are part of that cohort. They have let immigration and refugees be an issue to gain populist support. Its the same all over Europe.
Anti-immigrant bigoted parties didn’t win one seat across the nation. There is zero anti-immigrant momentum.
@9QRixo8H: There is and they won a few seats on councils, mainly Dublin
@Gary Kearney: Dzopers think the ‘hard-right’ is going to stop immigrants taking the jobs they don’t want to do. Beautifully played by the rich, effortlessly chumming poor waters.
There is no comparison between local and general elections which was shown 5 years ago.
@hans vos: You can cling to that if you wish, but SF didn’t run enough candidates in the GE, now their attempt to rectify that has spectacularly blown up in their faces.
SF didn’t do the grunt work on the hustings, and their core support is really between 17 and 20% , the media campaign for this election was amateur and they won’t form or be part of any government.
@Paul O’Mahoney: Not with this result but they still have time to rectify their mistakes before the next GE. So just don’t cling on to your hopes yet.
@hans vos: My hopes? , I don’t have any when it comes to politics I simply don’t care.
Nothing will change and democracy, the style we have, is not designed to help anyone except political parties and the members.
And expecting elections to help those who truly need it by voting is laughable.
There is only one key takeaway and that is immigration. Irish people just want someone to say that We do not mind people coming here to work but we do mind people coming here to sponge off us . We have been taken for a ride regarding immigration . Australia have strict immigration policies but no one calls them far right or racists.
If SF had have proclaimed they are on our side in this , they would have had a landslide . Alas , they did not listen and choose to ignore
Oh dear, oh dear, oh dear, what a tumble from grace lol
@Duffman: OHHHHHH yeah……
To sum up the analysis – SF are going to check the weather vane to see which way the wind is blowing and produce a populist stance.
Anyone notice that SF faired best in the border counties. Is that the result of the last batch of immigrants.
Who was the director of Elections in SF, as that is the buck stops too as well as the General Council.
They got destroyed at the last locals and struggled for candidates for the General election, which they did brilliantly in. This time they over estimated and put in too many candidates. We had four and they had a reasonable chance of two.
Yet the sitting councillor who is a great councillor nothing to do with party support, barley scrapped in. That is bad planning!
@Gary Kearney: Sinn Fein will have to put their thinking balaclavas on after this disaster.
@honey badger: thinking balaclavas! Lol
Singing ooh ahh probably doesn’t help.
It’s all their game plan to win Gaza elections in 2025. They have been so focused on Palestine, it’s obvious. Smart move.
A few points I took from the local count regarding SF were that the vote was more based on individuals rather than parties, the SF candidates were very poor in our area (young with no work done on the ground to build their profiles) and transfers went everywhere, some independents took a lot of the votes that would have went to SF in the past.
@The next small thing: We live in Kildare and as of this morning SF had 1 elected to KCC, this is a county that could have returned 4 TDs in GE had they run 4 candidates.
But it points to another flaw SF have and it’s experience in Government they are going to become under increased scrutiny in the run up to the GE. 300,000 house prices in Dublin type statements will be laughed at, but their economics will also come under the spotlight.
Fine saying ” our budget is full costed” for a year, but governments last 4/5 years and things like wage inflation, national debt interest etc need costing and then any proposals added to see the true figures.
They are Junior C hurlers try to play Senior
@Paul O’Mahoney: They are hurlers on the ditch. The current SF has its roots in the sectarian divide in NI and has no place in Southern politics. What place has an abstentionism MLA to represent the Southern people in the European Parliament. SF has one objective and that’s Irish unity. It doesn’t matter what price we pay for it.
In need of a Reshuffle
have your say