The Alberta elections in Canada, California’s governor elections, the Tory victory in Britain, Brexit, and the Trump victory are just a few examples of how pollsters all around the world have been disappointing us. While the growing failure of polling is only now becoming a hot topic, it has been an overlooked, recurring trend for years.
The 1980s were an optimistic time for the polling industry. Most people were easily accessible by their landline phones and pollsters rarely came across skeptical respondents who were repulsed by the ads, cold calls, and scam artists that we see today. When technological and social change arose from the invention of the cellular phone, the accuracy of polls began a steady decline. This change deeply affected the industry’s efficacy and, ultimately, its reputation. As time passed, it became increasingly inefficient for pollsters to pay for live interviews. In addition, lower response rates affected sample pool accuracy and participants began to expect compensation for their participation. Higher spontaneity triggered by increasingly turbulent politics also made people much more unpredictable and created a larger disparity between poll estimates and results. Even Nate Silver, from the famous polling site FiveThirtyEight admitted that “Polls in the UK, and in other places around the world, would appear to be getting worse as it becomes more challenging to contact a representative sample of voters.”
The shortcomings of the polling industry seem to be infinite and widely discussed. However, the true question lies not in why the industry failed – that we seem to know – but where these consistent errors will move the polling industry in the coming years.
Some speculate that this marks the end of the polling industry as we know it. Pollsters like David Coletto, strategist and CEO of the polling and research firm Abacus Data, defend their profession. “[Recent events were] not a polling disaster,” Coletto argues, “If anything it was a failure of interpretation.” While this is a compelling argument, the unfortunate reality is that people care less about the numbers than they do the story. When the story is incorrectly skewed by the media, the pollsters will be the ones to take the hit, even if their numbers were not wrong. Given the industry’s dependence on public interpretation, few people can deny that the political polling is at risk. At the end of the day, the industry will not crumble overnight, but it’s turbulent past few years, and especially turbulent past few months, are undoubtedly a call to action for the trade to shift its gears. That being said, there are several places the industry might go.
As previously alluded to, one extremist point of view argues that the polling industry might completely dissolve in the next few years. In addition to its recent stigmatization from wide-scale poor interpretation, there appears to be a growing number of confounding variables in technology and social transformations that prevent the polling companies from doing their job as well as they used to when these variables were nonexistent. To some, it may seem like the perfect storm for people to their demand for polling data altogether and put pollsters out of business. However, this is a more dramatic, short-term, and unrealistic reaction.
Some suggest that academic institutions should take matters into their own hands and lead the way in sorting out the industry. Pollster universities, as they are often called, have no motivation to skew interpretation and are generally transparent about their models and data since they genuinely want to figure things out, instead of push forward an opinion like many media and poll sources desire. Don Levy from the polling institute at Siena College in New York elaborates, “We don’t do this for money. We’re not working for candidates. We will continue to study the process.” Still, university pollsters have a long way to go due to limited resources and experimental models.
A more realistic path is to have the polling industry study specific adjustments it needs to make in order to regain its accuracy and reliability. While more realistic, this “solution” is much easier said than done because of the countless number of unknowns regarding data gathering, adjusting, and analyzing; the industry would practically have to start studying methods from scratch and this is difficult, expensive, and time-consuming. All the same, pollsters have succeeded in proving their desire for improvement through a number of adjustments already. Some pollsters are re-testing the old-fashioned door-to-door polling technique. Meanwhile, the Canadian Association of Public Opinion is already planning to set new, higher standards around transparency and polling methods, and to better hold firms accountable when results go awry. Unfortunately, these gestures are geographically independent and insufficient in solving the overarching limitations of current political polling strategies and a number of adjustments have yet to be recognized by pollsters.
For example, it is becoming more and more common for people to see political polling as a tool to push a particular perspective or agenda, adding skepticism to its perception and pressure on the polling practice. This stigmatization has significantly damaged the credibility of the polling industry due in large part to media sources skewing data presentation. Pollsters need to consider taking more responsibility in presenting their data independently and objectively from media. Polling companies also need to be less threatened by each other and collaborate in order to avoid herd behavior, where pollsters are pressured into producing similar results to what most other companies say, even if it is incorrect. Most importantly, they need to re-capture as many voters as they can, which is now limited due to cell phone contact restrictions that did not exist when the polling industry thrived. This offers opportunities for anthropologists and other social scientists to add input in voter and societal behavior that can adjust polls, make them more accurate, and intrinsically incentivize more people to participate in the data gathering.
In the meantime, pollsters should move their focus from political polling to polling for companies’ new product research projects, where it is slightly more difficult to let emotions and biases control results. When testing a product, population samples are also smaller and therefore more compatible with currently available polling methods. Pollsters should continue to look into improving their political polling strategies, but until drastic improvements are made, the industry’s opportunity lies elsewhere. Of course, polling should not be eliminated from news and politics because without it, people would feel lost. However, it is clear that the industry has dipped into a dark period that it needs to address.
“Our paradigm has broken down and we haven’t figured out how to replace it,” says Cliff Zukin, professor of public policy and political science at Rutgers University and former president of the American Association for Public Opinion Research. “We’ll have to go through a period of experimentation to see what works, and how to better hit a moving target,” he agrees.
Andrew-Gee, E. (2015, September 10). Margin of Error. Retrieved from The Globe and Mail: http://www.theglobeandmail.com/news/politics/survey-says-the-future-of-polling-is-hard-topredict/article26042112/
Balnaves, M. (2016, November 9). This is the end of industrial political polling as we know it . Retrieved from Quartz: http://qz.com/832565/election-2016-this-is-the-end-of-industrial-political-polling-as-we-know-it/
Coletto, D. (2016, November 14). In Defense of My Craft: 2016 U.S. Election Polling Did Not Fail. Retrieved from Huffington Post: http://www.huffingtonpost.ca/david-coletto/election-polls-did-not-fail_b_12943996.html
Craighill, P. M., & Clement, S. (2014, August 27). What’s right with political polling. Retrieved from The Washington Post : https://www.washingtonpost.com/news/the-fix/wp/2014/08/27/whats-right-with-political-polling-a-response-to-nate-silver/
Durden, T. (2016, August 17). New York Times’ Dire Warning About Polling Accuracy . Retrieved from ZeroHedge: http://www.zerohedge.com/news/2016-08-17/new-york-times-dire-warning-about-polling-accuracy
Kolowich, S. (2016, November 10). Academic Pollsters Didn’t See All Those Trump Voters Coming, Either. Retrieved from The Chronicle of Higher Education: http://www.chronicle.com/article/Academic-Pollsters-Didn-t/238362
Lebeau, J. (2015, November 7). Forget Trump and Clinton: The Polling Industry has a Lot Riding on the Election. Retrieved from Forbes: http://www.forbes.com/sites/jordanlebeau/2016/11/07/forget-trump-and-clinton-the-polling-industry-has-a-lot-riding-on-the-election/#5eac38386783
Maggi, Q. (2016, November 9). Why was everyone wrong? Retrieved from iPolitics: https://ipolitics.ca/2016/11/09/why-was-everyone-wrong/
Reports of the Death of Polling have been Greatly Exaggerated. (2016, November 9). Retrieved from The Conversation: http://theconversation.com/reports-of-the-death-of-polling-have-been-greatly-exaggerated-68504