Are voters lying to pollsters, or to themselves?

So the polls were wrong, again. Just how wrong remains to be seen, but by my calculations, Joe Biden’s advantage was overestimated by more than 5 percent in numerous battleground states. This comes after similar mishaps not just in the 2016 presidential election, but also in several recent UK elections, most notably 2017.

Pollsters are and should be judged by such results. Yet you only need to look at their infernal bickering over methods to see that polling remains more of an art than a science. In large part, this is because they are faced with a moving target: new social phenomena are appearing all the time that confound assumptions about voting behaviour.

Lately much of the blame has been placed on voters “lying” to pollsters. The theory is that when one party or candidate has a social stigma attached to it, some percentage of their supporters will be unwilling to admit their intentions until they reach the privacy of the voting booth. This is has been called the “shy Tory” or “shy Trump voter” effect, since these are the voters most likely to mislead or avoid pollsters on such grounds.

But this merely points to a more fundamental problem with trying to predict voting behavior. The idea that people are being dishonest about their intentions assumes they know what those intentions are – that they are consciously aware of what they believe and what decisions they are likely to make. Yet there are plenty of psychological findings, not to mention centuries worth of introspective and observational literature, which suggest that people are not entirely transparent to themselves in this way.

Take the “shy Tory/ Trump voter” effect. When we imagine how social pressure works, we tend to envisage a person who thinks one thing and says something different, and obviously this does occur. But generally there is no bright, Goffmanesque line between the private thoughts of individuals and the social world where such pressure is brought to bear. The influence of other people’s attitudes tends to be manifest within our thoughts.

When society is riven by deep conflicts of belief, then for a substantial number of people those conflicts will be raging in their minds as well. That has certainly been my subjective experience at various points over the last decade – what do I actually believe about this or that issue? If your peer group is exerting strong pressure in one direction, your desire to fit in or avoid censure doesn’t just affect how you speak and act. The voice of the group will to some extent be internalized, forming part of the obscure cacophony that is the human conscience. In some cases, you may genuinely persuade yourself that you hold a view which, when the chips are down and you are required to act, you don’t appear to hold at all.

So when people say they will vote in a certain way, or that they don’t know how they will vote, they may well believe it in some sense (“believe” being an incredibly slippery concept here) – even if their eventual behavior suggests they were being dishonest. Perhaps they weren’t lying to the pollster, but lying to themselves without knowing it.

Of course it’s possible that polling companies can find ways around this problem. As the advertising industry has demonstrated for more than a century, smart analysts can predict our behaviour better than we ourselves can, and they don’t even need a rigorous theory of mind to do it. They can just use empirical observation to draw connections between certain indicators and certain outcomes.

Thus more successful pollsters in recent times have learned that asking tangential questions, about a person’s views or politics in general, can offer insights into voting intention. Such giveaways are probably effective even when the person has misled him or herself about who they will vote for. One pollster commented on suburban women who were outspoken about how much they disliked Trump, but would then agonize over issues like civil disorder. He concluded, “it’s almost like they’ve been looking for a reason to hold their nose and vote for Trump.”

Of course failure to measure a “shy Tory/ Trump voter” effect is not the main reason we should pay much less attention to polls – it’s just part of the wider problem that they don’t seem very reliable. But if you accept that people are often unclear or self-deceiving about their intentions, there are also important implications for how we interpret data like polls, surveys and election results, especially when it comes to understanding motivation.

As we have seen after numerous recent elections, the question of why people voted as they did can be seriously inflammatory. Was it because of a clearly held principle or stance on some issue? Or was it because they are just prejudiced? We have a habit of projecting onto voters the motivations that suit our own assumptions. But just as someone may lie to themselves about how they will vote, the same chaos of deceptions and rationalizations can make it unclear why they will vote that way. Some number of people won’t be able to grasp what their decisive motivation was. In which case, claiming to know their outlook better than them seems like a shaky proposition.

This doesn’t mean we should stop trying to gather information on these questions; an informed guess is still better than an uninformed one. But we should make greater allowances for the opaqueness and complexity of the political mind. At moments of discord, it is tempting to see everyone as committed one way or the other – especially when you are confronted not with actual people, but statistics. In fact, these are just the moments when people might be least clear about their own beliefs and intentions.