Home » News » Fact check: How reliable are election polls? | Germany | DW

Fact check: How reliable are election polls? | Germany | DW

“The surveys have not become more uncertain, but rather more elaborate,” says Thomas Wind, founder and managing director of Institute for Target Group Communication (IfZ). The opinion research expert has “a problem with the handling of the polls”: “Every day a new number or a new poll is published, it creates a suction,” says Wind.

More polls, fewer participants

The political scientist and publicist Frank Brettschneider, who has already written several studies on opinion polls and polls and teaches at the University of Hohenheim, confirms the trend. According to his surveys, both the number of polls and the reporting on them have increased more than tenfold in Germany since 1980.

Media researcher Frank Brettschneider

One consequence of this is that fewer and fewer people are willing to take part in polls. According to the international Studie “Election polling errors across time and space”(Errors in polls in time and space) 20 years ago 30 percent of those questioned took part in the polls, today the rate is less than ten percent.

The study, updated in June 2020, analyzed 351 national elections in 45 states with more than 30,000 polls between 1942 and 2017. The two authors, Will Jennings, Professor of Political Science at the University of Southampton in the UK, and Christopher Wlezien, Professor of opinion polls at the University of Texas, consider the development to be alarming: “This jeopardizes the representativeness of the surveys and leads to errors and deviations”.

Quality of the samples

The quality of the samples determines whether a survey is representative. One of the systematic problems is how to deal with the undecided group.

Infographic infratest dimap Brexit 02 German

As recently as April 2016, the supporters of the EU predominated. In the referendum on June 23, however, 51.89% of the British voted in favor of leaving the EU. The undecided group was included in this survey

“The lessons from the polls on the election of US President Trump or the referendum on Brexit show that certain target groups cannot be reached,” explains Wind expert. The undecided group would be excluded from many surveys, including the Sunday question of the ARD Germany trend. Wind: “That’s a problem, because that can be 20 percent or more.”

Political scientist Brettschneider explains further: For example, many Trump supporters would in principle not take part in polls because, in their eyes, they belonged to the establishment like the “lying press”.

“Since they didn’t vote in the past, that didn’t matter,” said Brettschneider. “Now they take part in elections – and that leads to discrepancies between the polls and election results.”

Evaluation and transparency

The type of survey can also affect the representativeness of surveys. Conversations over the fixed network are considered to be particularly meaningful because they are regionally assigned and the interviewees can take time for a conversation. In the case of surveys on the mobile phone, however, it is unclear where the person is currently and whether they can be spoken to. Online surveys are easier to collect, but they are often anonymous and only record people who use the Internet.

General election polls infographic

Not a forecast, just a snapshot: a lot can change before the elections at the end of September

In order to improve the sample quality, underrepresented groups are weighted more heavily by the opinion research institutes. For opinion polling expert Wind, it is now beginning to “become mysterious” because the institutes would rarely make their weighting formulas transparent.

In addition to the weighting formulas, the evaluation and analysis of election surveys are also susceptible to sources of error. Because the statistically calculated deviation rate between two and three percent is often only in the small print in the information on the surveys and is not always mentioned in the publication.

With a possible deviation of 2.5 percent, a party that is 50 percent in the polls could in reality also have a 47.5 percent or 52.5 percent share of the vote – so in this case it could even mean that a party has an absolute majority, or not.

Infographic Maximum of 4 percent survey differences DE

In the polls, the Union appeared to be more popular than in actual voter favor

Precision of the results

Despite the numerous factors of uncertainty, the polls on the federal elections over the past 20 years have been surprisingly precise. As from the ones on the platforms wahlrecht.de and dawum.de According to published data, the surveys only exceeded the usual margin of error of three percent twice.

From a Evaluation of survey data since 2001 by ZEIT ONLINE shows that the election results of the individual parties were predicted with an average accuracy of 1.74 percentage points. If you only look at the last ten years, i.e. 2011 to 2021, the deviation has increased by an average of 0.41 percentage points.

The Union was clearly overestimated by all polling institutes in the 2017 federal elections; the largest deviation was 4.1 percent (see graphic). In 2005, the deviation rates for the Union even reached up to seven percent.

Infographic - Error rate per institute - DE

“No change in survey quality”

ZEIT data journalist Christian Endt suspects that the increasing deviation could be related to the success of the AfD and the increasing fragmentation of the party spectrum. “Otherwise,” says Endt, “there has been no change in the quality of the survey over time since 2001.”

The data analysis by ZEIT ONLINE took into account almost 500 surveys that were published in the 30 days before election day. The parties examined included CDU / CSU, SPD, Greens, AfD and Left.

The international meta-analysis “Election polling errors across time and space” comes to the same conclusion: “There is no evidence for the increase in errors”, according to the study. Between 1940 and 1950 and 1960 and 1970 there was a deviation rate of 2.1 Percent, since 2000 this has been constant at two percent.

The authors note, however, that the size of political parties can influence the precision of election polls. “The bigger the party and its struggle to stay in power, the greater the deviation rates,” says the study, “especially when the polls are close together, as in 2015 in Great Britain or 2016 in the USA.”

Conclusion: Even if polls are better than their reputation, there are always deviations that are above the usual rate of two to three percent. Because the political imponderables have increased despite the constant quality of the survey. This became particularly clear in the state election of Saxony-Anhalt on June 6, 2021. The polls had not predicted the CDU’s victory for the ballot, but a head-to-head race between the CDU and AfD. The sometimes imprecise evaluation when reporting on surveys can also increase inaccuracies.

– .

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.