Explaining the unreliability of surveys in the 2012 Catalan elections

Somehow deviating from the original purpose of this blog, this post will focus on the possible explanations behind the lackadaisical victory obtained by CiU in the recent Catalan elections, which was not predicted by any of the numerous pre-election surveys and polls. I argue that there is no single reason behind this major surprise, but a whole combination of factors making all predictions worthless.

Recent results in the Catalan regional elections pose a very interesting challenge for anyone interested in political science and voting dynamics, as well as with the predicting power of surveys and polls. This time, it's nothing like the usual 'Gallup against the world' issue, but outcomes remarkably deviating from all polls, predictions and expectations (including those of the candidates themselves!)

According to all pre-election polls, performed by news outlets and official sources apiece, the ruling party, Artur Mas’ Convergence and Union (CiU), would at least repeat the results it obtained in 2010, with 62 seats out of the 135-strong parliament. However, when it was all said and done, CiU was again the prevailing force, but only took 50 seats. Talk about a bitter victory.

How could such a reversal of sorts happen? Why did not just one, but all polls fail to reflect this fiasco? The answer obviously lies in the extraordinary high number of undecided voters reflected by all polls, up to an average 30% of the respondents. Let's imagine a survey with a sample of 1,000 respondents. Given that the number of potential voters in Catalonia hovers around 5 million, this combination would normally produce results with a 4-5% margin of error and a 99% confidence level, meaning that we should be 99% sure that the results will be the ones obtained, within a +/- 4-5% range.

However, if 30% of respondents report being undecided, only 700 real answers will be given. If, out of those 700, 330 report their intention to vote for CiU, projections will assign this party almost half the seats in the Parliament. In fact, even a lower number should suffice, as the Catalan voting system, a modified version of the d'Hondt method, strongly overrepresents the less populated, more rural provinces of Girona, Tarragona and Lleida, traditional CiU strongholds. In other words, the number of undecided voters practically equaled that of CiU-leaning respondents, thus making any projections wildly unreliable: if 30% of the sample are unable to give what we should consider a statistically valid answer, the reliability or validity of any such survey should be seriously doubted.

Or shouldn't we? Despite reporting relatively high levels of undecided voters, such pre-election polls tend to get the results fairly right, both thanks to the overgenerous  margin of error or projected bracket we obtain when taking all the major polls combined (in this particular election, CiU was projected to obtain anywhere between 60 and 70 deputies, a a +/- 8% margin of error) and to the fact that undecided voters usually end up distributing their final choices in a rather regular and predictable pattern across the whole political spectrum – barring major campaign breakouts or breakdowns. Even if the campaign was a especially vicious one, there was only a very modest downward trend for CiU results in polls published up until the very end of the 2-week campaigning period. In short, nobody was able to predict what was actually coming. Beside the inordinate number of undecided voters, what are the reasons behind this highly unusual predictive breakdown?

First – and maybe foremost – the record-breaking turnout, which reached 70% of the electorate, benefited left–wing parties, all of them obtaining far better results than initially predicted. The biggest gains went to the Republican Catalan Left (ERC), a pro-independence left-wing party who collected thousands of votes of disaffected former CiU voters. Even the Socialist Party (PSC), despite taking a hard beating, ended up with 20 deputies, up from the predicted 16-18. The green-socialist party Iniciativa per Catalunya Verds (ICV) went from 10 to 13 seats, and even the radical anti-capitalist, pro-independence popular movement CUP took home 3 wholly unexpected deputies. The fact that CiU has lost just 90,000 votes (or 7.5% of the total votes they obtained in 2010) but 12 deputies (nearly 20% of the previous total), while left-wing parties collected an extra 531,000 votes (going from a mere 447,000 in the 2010 election to 978,000 now, more than double) tells the whole story here.

A second reason is the newfound strength of the so-called 'hidden vote': namely, people who are ashamed to acknowledge they support unpopular political options, such as anti-separatist parties in Catalonia. Social polarisation due to the mainstream pro-separatist discourse set up the breeding ground for an unusually high number of hidden votes: the populist Ciutadans (C's) wildly beat all projections by tripling its representation, going from 3 to 9 deputies, while Mariano Rajoy's People's Party (PP) also did well, gaining an extra seat despite the tough austerity, centralising policies coming from Madrid. In the case of C’s, their explosive rise – which was only partially predicted in the surveys – demonstrated that the PSC was indeed a 'mixed bag' until now. Its power base in the Barcelona area industrial towns, mainly made of former internal migrants and their working-class offspring, deserted en masse. Some went for the alternative socialist option, ICV, but most undecided former socialist voters, and also some traditional CiU voters – scared off by its newfound separatist thrust – switched to Ciutadans.

Finally, last-minute hardball tactics by the right-wing Madrid press, wholly supported by relevant figures of the PP-controlled central government and highlighted by the leak of an alleged police report incriminating Artur Mas, Home Affairs Minister of Catalonia Felip Puig and CiU patriarch and former President Jordi Pujol, also played their part. Despite the still unclear validity (or even existence) of this document, it surely scared off many undecided voters who might be leaning towards CiU, believing in Mas' rhetoric, but were definitely disappointed at even the rumors tying his name to a further corruption scandal – something all too common when it comes to Spanish and Catalan politics, including CiU. Most of these disappointed people moved left to ERC, seen as a more genuine pro-independentist, clean force, but some others moved to the populist C’s or even the PP.

To sum up, an unusually high number of undecided voters in all pre-election polls combined with an unusually high turnout and the unexpectedly uneven allocation of the votes of those undecided citizens – which massively opted for left-wing, pro-independence ERC, ICV or CUP or populist, anti-separatist C's – shattered all predictions. Right-wing parties, such as CiU and the PP, traditionally enjoy a strong, loyal base of supporters: this army of loyalists always comes handy, but it can also fall short of breaking the bank, especially when turnout is unusually high. Austerity, recession, unemployment, dirty politics and miscalculations did the rest.