Support from readers like you keeps The Journal open.
You are visiting us because we have something you value. Independent, unbiased news that tells the truth. Advertising revenue goes some way to support our mission, but this year it has not been enough.
If you've seen value in our reporting, please contribute what you can, so we can continue to produce accurate and meaningful journalism. For everyone who needs it.
OPINION POLLS ARE one of the main talking points of the recent local and European Parliament elections. Tánaiste Micheál Martin and Jack Chambers, Fianna Fáil Director of Elections, publicly criticised pollsters using online panels.
Fianna Fáil has good reason to be frustrated. General election polls have significantly underestimated support for Fianna Fáil and vastly exaggerated support for Sinn Féin. For them, this creates a mistaken media narrative about popular support for Sinn Féin.
But does this underestimation of support for FF in general election polls mean that we should completely ignore opinion polls? Do they still have value for politicians, journalists and voters? How can the media, politicians and voters engage more constructively with public opinion polls? The starting point is to recognise that general election polls do not measure support for parties in local elections. Irish general election polls have always been a poor predictor of local elections, but are good predictors of general election results.
Polls for general elections are accurate
One criticism raised by politicians and commentators after these elections relates to the huge difference between support in current opinion polls for the upcoming general election and the local election results.
Let’s untangle local and general election polls. Are opinion polls for general elections accurate? This is the first question I will address.
The Irish Polling Indicator project, which Tom Louwerse (Associate Professor, Leiden University) and I started 10 years ago, aggregates all opinion polls into a single estimate of public support.
We have collected all opinion polls published since the mid-1980s, resulting in a sample of 717 surveys.
This tedious exercise involved going through newspaper archives, checking various websites and contacting academics. The resulting dataset not only allows us to present aggregated estimates but also helps us understand how well polls predict elections.
To assess the accuracy of polls, I extracted the Polling Indicator estimate on the day of the election and compared the figures to the actual first preference vote shares. Next, I calculate the absolute mean error, which is the difference between the first-preference vote share at the election and the aggregated estimate. If a party obtained 25 per cent of first-preference votes, but the poll of polls estimated support at 21 per cent, the absolute error for this party amounts to 4 percentage points. This analysis is performed for all parties to calculate the average absolute difference.
The graph below reveals that the absolute mean error results to be just 1.32 percentage points across the eight general elections between 1989 and 2020. In other words, on average, the poll of polls is just 1.3 points off the election result. This is very accurate.
Advertisement
Stefan Müller
Stefan Müller
From a political science perspective, it is also very encouraging: most estimates fall within the margin of error. There are, of course, exceptions to the rule. In 2002, the polls overestimated support for Fianna Fáil by 4.6 percentage points but underestimated the party’s support by 4.4 points in 2007 and 3.4 points in 2016.
In 2020, the aggregated polls overestimated support for Fianna Fáil by 2.4 points. In 2016, the polls overestimated support for Sinn Féin and Fine Gael by around 3 points each. Yet, overall, polls have done a pretty good job in predicting party support in Irish general elections over the past 25 years.
Not so for Local Elections – this is no surprise
What about Local Elections? Pollsters received a lot of criticism for underestimating Fianna Fáil support and vastly overestimating the Sinn Féin vote last week. Yet, opinion polls do not intend to predict local election results.
Looking at the second figure here supports this argument. The absolute mean differences range from 2.46 (2004) to 3.77 (1991) points. Last weekend, the mean difference was 3.18 points. At almost every election, the estimates for at least one party are completely different to the outcome.
Stefan Müller
Stefan Müller
Why are the polls so far off? Irish local elections are unique. Incumbency plays a huge role in Local Elections. Our research shows that even incumbents who got elected on the last count face a huge advantage in rerunning and getting re-elected compared to the candidate who lost on the last count. Canvassing, name recognition and community ties are highly relevant. Door-to-door campaigning is a unique feature of Irish politics and it is a crucial condition for getting elected.
An Ireland Thinks poll fielded on the day of the local and European Parliament elections mirrors these claims. 75 per cent of Fianna Fáil voters cast their first preference for a candidate rather than the party the candidate represented. 66 per cent of Fine Gael voters ranked the party over the candidate.
The focus on candidates is lower for smaller parties – possibly because they ran fewer incumbents and their candidates were simply not as well known.
Fianna Fáil and Fine Gael also canvassed very successfully. 35 per cent of respondents said they met a Fianna Fáil candidate at their doorstep. 34 per cent of respondents had direct contact with a Fine Gael candidate. Only 18 per cent met a Sinn Féin representative. This might be part of the explanation why Sinn Féin performed even more poorly than anticipated: they are not only losing support in opinion polls, but they had few incumbents and their candidates were simply not as active on the ground as Fianna Fáil and Fine Gael. To sum up, incumbency and local ties are crucial, and Fianna Fáil and Fine Gael outperformed Sinn Féin in this hugely important aspect of campaigning.
We need to keep in mind that the electorate is becoming more volatile. According to Exit Polls from the past 25 years, around half of the voters make up their minds about the first preference vote in the last month before an election. Since the financial crash, around four in 10 voters have switched between parties from one general election to the next. Campaigns and canvassing are crucial in general elections and can reshuffle the Irish political landscape.
Online vs. face-to-face polls
The debate between the efficacy of online and face-to-face polls continues to be contentious. While some, like Fianna Fáil’s Micheál Martin and Jack Chambers, argue that online polls tend to underestimate their support, face-to-face polls also present challenges, such as low response rates and social desirability biases. Red C Research, despite being criticised at the moment, accurately predicted Fianna Fáil’s performance in the 2020 general election.
Related Reads
The Explainer: How does Ireland's system of voting work?
Elections 2024: The seeds of an Irish 'Stop the Steal' campaign are being sown
Elections: Should you vote the whole way down the ballot paper?
Ideally, we rely on a mix of methodological approaches to better understand Irish party politics. Telephone polls, face-to-face surveys, and online surveys all have advantages and shortcomings. Michael A. Bailey, Professor at Georgetown University, describes these issues in an excellent recent book “Polling at a Crossroads”. Having an open mind towards public opinion will ultimately improve our understanding of public opinion.
Moving forward: suggestions for journalists, politicians and voters
What can we learn from this debate on opinion polls and how can we move forward the debate? I have four recommendations:
1. Media outlets should adjust their reporting. We must consider structural differences between polling companies, focus on trends rather than individual polls and not overplay small “changes” lying within the margin of error as “bounces”, “rebounds”, and “recoveries”. Sampling variation drives most of these “changes”, which have usually been within the margin of error of ±3 percentage points. Political scientists Erik Gahner Larsen and Zoltán Fazekas show that Danish news reports frequently interpret deviations within the margin of error as substantive changes. The authors conclude that “a horserace coverage of polls about change can rest on a foundation of stability.” Irish newspapers often apply the same rhetoric. A move away from horserace coverage is necessary.
2. In addition to changing the language, polling companies and newspapers could present party support as within windows (i.e., the 95 confidence interval). Reporting single numbers projects a false sense of precision. Other countries are slowly adopting this approach. For example, Italian media reported the results from the exit poll for the European Parliament elections in intervals. Rather than focusing on a single number – and suggesting a wrong sense of certainty – their headline finding was that “Meloni’s FdI to get 26-30% in EU elections – exit poll Centre-left Democratic Party to come second with 21-25%”. Irish media should follow suit.
3. We should focus more on trends. For example, the polling trends clearly showed the loss of support for Sinn Féin in general election polls – across all companies. While pollsters deviate systematically in their survey mode and baseline results, these developments are unambiguous. Voters should approach poll results critically, considering the broader context and acknowledging uncertainty.
4. We need to keep in mind that polls provide a single snapshot and that using polls to predict elections is extremely difficult. Paul Perry, back then Vice Chairman of Gallup Polling, focused on these problems in a Public Opinion Quarterly paper published 45 years ago:
All one has to do is use a properly drawn sample of the electorate large enough to minimise random sampling error, get honest answers from everyone, do the questioning close enough to the time of voting to minimise changes in voting intentions, anticipate how the undecided will vote, and, finally, distinguish between voters and non-voters in the electorate – (Perry, 1979: page 312).
Pollsters should need to be even more transparent in their reporting methods. While most outlets now list the poll’s margin of error and time when the survey was in the field, the exact approach of selecting households for face-to-face polls, selecting respondents from the online panel, non-response rates, and the weighting of respondents is still a black box. This is understandable because polling is highly complex and relies on cutting-edge statistical methods. Media outlets could add a few sentences about the sample selection and weighting.
Finally, it is important to note that polls are extremely useful for understanding voter preferences and behaviour. Journalists and politicians need polls to understand the electorate. I believe the current debate on the usefulness of polls is constructive and ought to lead to important reflections among all actors involved. I hope that mentioning methods and uncertainty will increase trust in public opinion. Ultimately, these changes would help us to focus on policy issues and the solutions that parties offer.
Stefan Müller is Assistant Professor in the School of Politics and International Relations at University College Dublin. He maintains the Irish Polling Indicator and the Irish Politics Data project.
Readers like you are keeping these stories free for everyone...
A mix of advertising and supporting contributions helps keep paywalls away from valuable information like this article.
Over 5,000 readers like you have already stepped up and support us with a monthly payment or a once-off donation.
To embed this post, copy the code below on your site
Close
52 Comments
This is YOUR comments community. Stay civil, stay constructive, stay on topic.
Please familiarise yourself with our comments policy
here
before taking part.
@Andrew Kenny: I’d be on the same page as you but there’s a 300 million younger vote in the US so her word will definitely impact on Trumps campaign so it definitely carries weight.
@Tony Skillington: average age of her concert goers is 40+. Taylor has outlined her reasons and research behind the decision to endorse Harris, not that any of you would care to read them.
Mute another one? what's going on is the semi state sec
Favourite another one? what's going on is the semi state sec
Report
Sep 11th 2024, 1:34 PM
@Tony Skillington: In 2020 there was 168.31 million people registered to vote in the US. Sure the total population is around 345 million…… Did you get mixed up with the amount of worldwide followers she has or something?…..
The general jist of what you’re saying is right
@Andrew Kenny: It’s great to see even as a non-Swiftie! Not my bag but hugely influential. The Trump’s of the world have had the run of the place for too long. Male, stale and beyond the pale, in his case. Stick together sisters and bring the old guard down! I don’t agree with everything Harris stands for but she’s not so much an upgrade as hi-tech cutting-edge futurism beside the cat-crazed mob.
@John Kenny: they closed the article where FG want to bring in public-funded childcare!!! Socialist childcare!!! From a party that’s supposed to be for privatisation/capitalism!!!
@cup of tea: You mean most Americans with half a brain will vote for Trump. Elon Musk may be intelligent but he has no compassion or empathy, which may be part of his autism. Just watch the way he fired people at Twitter – there are even videos of him doing it. If he is the type of leader which encompasses Trump thinking then Attila the Hun would be considered a nice chap.
Not supporter of trump or Harris but if a person young or old need to follow some pop star to help them to decide who to vote for well that is a sad state of affairs for that person who can’t think for themselves.
@Ned: If you actually read her statement she did not tell people to vote for Harris. She told people to look at what both candidates and decide for themselves who to vote for. What she recommended her followers was for them to make sure they register to vote.
Genuine question: if a man gets a woman pregnant should the man have a say in what happens to the foetus, unborn child, sprog, spawn, or whatever people want to label the result.
@AnthonyK: Genuine answer: Ideally the man should be allowed state his case. But be shouldn’t ever be allowed to dictate what a woman does with her body.
@AnthonyK: what if the man feels like he’s a woman now surely that’ll give him rights he can argue he has very same rights as a woman then with equality crap.
She was criticised because she’s friends with someone who “liked” a Trump comment on social media. So there are sad people who actually check on what comments people like. The inquisition has returned.
Meanwhile being endorsed by a singer is a major thing! Granted Harris is a better choice than Trump (so is my cat). But the dumbing down of politics is rather worrying.
@Mike: Add moaning to the list of things you don’t understand. Calling me ‘girl’ is great. You’re another low IQ, low info Dim Bulb and you’re not afraid to hide it.
I wish trump would clear up one thing? Are the Dems talentless, incompetent bunglers or are they complex, genius, masterminds?
I bet the Trump supporters onhere believe they are both.
@Louis Jacob: You just reminded me of this paragraph from an article I read recently about Trump supporters – Boebert took the opportunity to boast on Trump’s behalf about one of his favorite topics: his supreme intelligence—as evidenced by the fact that, as Trump loves to mention, he allegedly once “aced” some cognitive test.
But here’s what Boebert actually said: “President Trump, when he was in office, underwent testing for his cognitive dissonance.”
Taylor Swift, well, maybe. Let’s hope she doesn’t get endorsed by Bouncey, Pinky, Dolly and the other pretentious “feminists”. That would sink her with a lot of voters!
Stock markets jump after Trump tariff pause, but Taoiseach warns we're 'not out of the woods yet'
Updated
1 hr ago
11.1k
58
Oliver MacCormack
Lisburn man (71) given seven-year sentence for human trafficking offences against nine victims
2 mins ago
12
u-turn
Trump announces 90-day tariff 'pause' on countries that 'have not retaliated', ratchets up China rate to 125%
Updated
13 hrs ago
58.8k
174
Your Cookies. Your Choice.
Cookies help provide our news service while also enabling the advertising needed to fund this work.
We categorise cookies as Necessary, Performance (used to analyse the site performance) and Targeting (used to target advertising which helps us keep this service free).
We and our 164 partners store and access personal data, like browsing data or unique identifiers, on your device. Selecting Accept All enables tracking technologies to support the purposes shown under we and our partners process data to provide. If trackers are disabled, some content and ads you see may not be as relevant to you. You can resurface this menu to change your choices or withdraw consent at any time by clicking the Cookie Preferences link on the bottom of the webpage .Your choices will have effect within our Website. For more details, refer to our Privacy Policy.
We and our vendors process data for the following purposes:
Use precise geolocation data. Actively scan device characteristics for identification. Store and/or access information on a device. Personalised advertising and content, advertising and content measurement, audience research and services development.
Cookies Preference Centre
We process your data to deliver content or advertisements and measure the delivery of such content or advertisements to extract insights about our website. We share this information with our partners on the basis of consent. You may exercise your right to consent, based on a specific purpose below or at a partner level in the link under each purpose. Some vendors may process your data based on their legitimate interests, which does not require your consent. You cannot object to tracking technologies placed to ensure security, prevent fraud, fix errors, or deliver and present advertising and content, and precise geolocation data and active scanning of device characteristics for identification may be used to support this purpose. This exception does not apply to targeted advertising. These choices will be signaled to our vendors participating in the Transparency and Consent Framework.
Manage Consent Preferences
Necessary Cookies
Always Active
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work.
Targeting Cookies
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
Functional Cookies
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then these services may not function properly.
Performance Cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not be able to monitor our performance.
Store and/or access information on a device 111 partners can use this purpose
Cookies, device or similar online identifiers (e.g. login-based identifiers, randomly assigned identifiers, network based identifiers) together with other information (e.g. browser type and information, language, screen size, supported technologies etc.) can be stored or read on your device to recognise it each time it connects to an app or to a website, for one or several of the purposes presented here.
Personalised advertising and content, advertising and content measurement, audience research and services development 146 partners can use this purpose
Use limited data to select advertising 116 partners can use this purpose
Advertising presented to you on this service can be based on limited data, such as the website or app you are using, your non-precise location, your device type or which content you are (or have been) interacting with (for example, to limit the number of times an ad is presented to you).
Create profiles for personalised advertising 85 partners can use this purpose
Information about your activity on this service (such as forms you submit, content you look at) can be stored and combined with other information about you (for example, information from your previous activity on this service and other websites or apps) or similar users. This is then used to build or improve a profile about you (that might include possible interests and personal aspects). Your profile can be used (also later) to present advertising that appears more relevant based on your possible interests by this and other entities.
Use profiles to select personalised advertising 85 partners can use this purpose
Advertising presented to you on this service can be based on your advertising profiles, which can reflect your activity on this service or other websites or apps (like the forms you submit, content you look at), possible interests and personal aspects.
Create profiles to personalise content 39 partners can use this purpose
Information about your activity on this service (for instance, forms you submit, non-advertising content you look at) can be stored and combined with other information about you (such as your previous activity on this service or other websites or apps) or similar users. This is then used to build or improve a profile about you (which might for example include possible interests and personal aspects). Your profile can be used (also later) to present content that appears more relevant based on your possible interests, such as by adapting the order in which content is shown to you, so that it is even easier for you to find content that matches your interests.
Use profiles to select personalised content 35 partners can use this purpose
Content presented to you on this service can be based on your content personalisation profiles, which can reflect your activity on this or other services (for instance, the forms you submit, content you look at), possible interests and personal aspects. This can for example be used to adapt the order in which content is shown to you, so that it is even easier for you to find (non-advertising) content that matches your interests.
Measure advertising performance 136 partners can use this purpose
Information regarding which advertising is presented to you and how you interact with it can be used to determine how well an advert has worked for you or other users and whether the goals of the advertising were reached. For instance, whether you saw an ad, whether you clicked on it, whether it led you to buy a product or visit a website, etc. This is very helpful to understand the relevance of advertising campaigns.
Measure content performance 61 partners can use this purpose
Information regarding which content is presented to you and how you interact with it can be used to determine whether the (non-advertising) content e.g. reached its intended audience and matched your interests. For instance, whether you read an article, watch a video, listen to a podcast or look at a product description, how long you spent on this service and the web pages you visit etc. This is very helpful to understand the relevance of (non-advertising) content that is shown to you.
Understand audiences through statistics or combinations of data from different sources 76 partners can use this purpose
Reports can be generated based on the combination of data sets (like user profiles, statistics, market research, analytics data) regarding your interactions and those of other users with advertising or (non-advertising) content to identify common characteristics (for instance, to determine which target audiences are more receptive to an ad campaign or to certain contents).
Develop and improve services 84 partners can use this purpose
Information about your activity on this service, such as your interaction with ads or content, can be very helpful to improve products and services and to build new products and services based on user interactions, the type of audience, etc. This specific purpose does not include the development or improvement of user profiles and identifiers.
Use limited data to select content 37 partners can use this purpose
Content presented to you on this service can be based on limited data, such as the website or app you are using, your non-precise location, your device type, or which content you are (or have been) interacting with (for example, to limit the number of times a video or an article is presented to you).
Use precise geolocation data 47 partners can use this special feature
With your acceptance, your precise location (within a radius of less than 500 metres) may be used in support of the purposes explained in this notice.
Actively scan device characteristics for identification 27 partners can use this special feature
With your acceptance, certain characteristics specific to your device might be requested and used to distinguish it from other devices (such as the installed fonts or plugins, the resolution of your screen) in support of the purposes explained in this notice.
Ensure security, prevent and detect fraud, and fix errors 93 partners can use this special purpose
Always Active
Your data can be used to monitor for and prevent unusual and possibly fraudulent activity (for example, regarding advertising, ad clicks by bots), and ensure systems and processes work properly and securely. It can also be used to correct any problems you, the publisher or the advertiser may encounter in the delivery of content and ads and in your interaction with them.
Deliver and present advertising and content 100 partners can use this special purpose
Always Active
Certain information (like an IP address or device capabilities) is used to ensure the technical compatibility of the content or advertising, and to facilitate the transmission of the content or ad to your device.
Match and combine data from other data sources 73 partners can use this feature
Always Active
Information about your activity on this service may be matched and combined with other information relating to you and originating from various sources (for instance your activity on a separate online service, your use of a loyalty card in-store, or your answers to a survey), in support of the purposes explained in this notice.
Link different devices 55 partners can use this feature
Always Active
In support of the purposes explained in this notice, your device might be considered as likely linked to other devices that belong to you or your household (for instance because you are logged in to the same service on both your phone and your computer, or because you may use the same Internet connection on both devices).
Identify devices based on information transmitted automatically 91 partners can use this feature
Always Active
Your device might be distinguished from other devices based on information it automatically sends when accessing the Internet (for instance, the IP address of your Internet connection or the type of browser you are using) in support of the purposes exposed in this notice.
Save and communicate privacy choices 69 partners can use this special purpose
Always Active
The choices you make regarding the purposes and entities listed in this notice are saved and made available to those entities in the form of digital signals (such as a string of characters). This is necessary in order to enable both this service and those entities to respect such choices.
have your say