Methods of Failure: How Political Journalism lost the US Presidential Election to Nate Silver
There are plenty of reasons to feel smug for the vast majority of us who subscribe to and believe in the importance of social and human sciences in the week the Obama family was returned to 1600 Pennsylvania Avenue by the American electorate –or, as an meme surfacing across social networks today put it, the fact that Ann Romney is secretly celebrating not having to move to a smaller house today. The re-election of Barack Obama and the defeat of his Republican challenger Mitt Romney is good news for the vast majority of scholars who dedicate their working lives to studying the interplay between structure and agency, between self and society, between economy and culture. Or to put it more drastically, it is good news for anyone who broadly subscribes to an enlightenment vision scientific enquiry, truth and critique.
But there was a second reason to celebrate for social scientists. If Wednesday morning left both the electoral map and Republican politicians feeling a little blue, there was another occupational group in need of collective introspection: the class of political journalists, commentators and pundits, who in the cause of the campaign had increasingly wilfully disregarded the lessons of the academic disciplines that form journalism’s very foundation.
Obama’s victory was decisive, winning 332 votes in the Electoral College compared to Romney’s 206 (assuming Obama will hold his nearly 50,000 votes advantage in Florida). His margin in the popular vote will be around three million, taking eight out of the nine states news media had identified as swing states throughout the campaign. The clarity of this victory appeared to have been surprising to many. For months many news media had promised a nailbiter, talked of a race that was “too close to call.” Some predicted a Romney victory or even a Romney landslide, leaving Jed Levison to gleefully list 34 blown election predictions on Daily Kos. From Glen Beck to Newt Gingrich reviewing these predictions promises a great deal of liberal Schadenfreude, but they are unsurprising. They were attempts by individuals who had openly pinned their colors to the GOP mast to present the Republican ticket as competitive for obvious strategic reasons. And while I am aware of the dangers of taking Fox News blatant disregard of fundamental journalistic ethics in their entirely partisan perspective, few will disagree if I rank Fox News among these campaigners and lobbyists rather than among actual news media.
The denial of Republican activists and supporters upon learning of Obama’s victory which we could witness across Twitter and broadcast news alike the day after the election is as much an expression of the first stage of grief as it is of the fragmentation of the public sphere which has allowed audiences to construct textual boundaries in their engagement with news that limit the discourses they encounter to those that correspond with their own partisan perspective and horizons of expectation.
This crisis of public discourse is not a new insight. Yet, it is confounded by a failure of political journalism that includes the standards of many, though not all professional journalists. The most shocking aspect of the campaign coverage was that mainstream media’s staunch reluctance to indicate the way the race was leaning and developing. After the first debate between the candidates in Denver, the notion of “Mittmentum” captured the journalistic imagination. Predictions commonly identified all nine swing states as ‘toss up.’ As little as a week ago, the Washington Post moved Ohio back into this category. CNN’s Wolf Blitzer, to the bewildered amusement of Obama supporters, continued calling polls showing the President leading by 2 or 3% a “statistical tie.” Right up to the election pundits from NPR to the BBC stressed how in such a close race any outcome was possible.
But was it? While the old hacks of the trait peddled the story of a “too close to call” election, those contributing to the debate from different professional backgrounds, often via the blogosphere, offered alternative ‑ and as it turns out far more accurate assessments ‑ of the state of the race. Nate Silver, economics graduate and baseball analysts, is only the most prominent exponents of the many who approached the polling data with the systematic approach that was not only absent among many political journalists, but that also proved entirely accurate. On his blog Fivethirtyeight, licensed by the New York Times two years ago, Silver quantified the chances of an Obama re-election at 90.9% while many political journalists continued to resort to the old “toss up” line. In the end, Silver called every single of the fifty states correctly, including closely fought North Carolina, Florida, Virginia and Ohio. Other’s such as the Princeton Electoral Consortium, run by Sam Wang, neuroscientist and regular contributor to the BBC’s outstanding statistics radio program More or Less, were similarly accurate.
Ezra Klein summarized the backlash against Silver by political commentators and politicians Silver’s model suggested were loosing last week in the Washington Post. Now that his predictions have been proven highly accurate again, those who rushed to his defense before have plenty of cause to celebrate the triumph of maths. Anthony Goldbloom in the Sidney Morning Herald echoes many such reactions when he claims “the ability to analyze large amounts of data is starting to replace expert knowledge.”
It is tempting to agree with Goldbloom’s claims. Too often during the campaign was political journalism lacking in basic literacy and numeracy. My personal highlight in this respect came early on election night with CNN’s Gary Tuchman exemplifying political journalism’s crisis in just three words: trying to compare the handwritten returns from three polling stations in Virginia to the respective 2008 results, Tuchman concluded they are “almost exactly similar” – a phrase that in its tripart oxymoronic denial of numerical and linguistic logic is a multifaceted complex of stupidity and ignorance reminiscent of an Escher painting.
But it is a false dichotomy. This is not about data crunching Wunderkinder with degrees in economics, sociology or statistics whose computer-powered precision faces off with the gut and intuition of aging political journalists. While Silver’s model (whose code he understandably doesn’t disclose) appears to be remarkably accurate and he is to be congratulated on his remarkable achievements, it does not spell the redundancy of political journalists. It doesn’t prove the supremacy of statistics over other forms of analysis. Rather it calls for something else: better political journalism and a thorough reflection on what political journalism is for and how it ought to be conducted.
The accuracy of Silver’s predictions is not a triumph of maths and statistics, it is an illustration of the need to reflect on methods and epistemology. What much of political punditry over the past months failed to recognise are the basics of methodology that any student on our undergraduate courses in sociology or Media Studies will learn in their first year: there is no universally accurate method, and different research questions require different methods of study. If we are interested in question of “how many?” and “who?” (which tend to be fundamental to predicting election results), political commentator’s gut feeling or “having talked to the people here in Ohio” are a poor substitute for systematic analysis of different polls and polling average. And while no one asks for the bulk of political journalists to share the depth of statistical literacy that mark Silver’s or Wang’s work, asking about sampling strategies of different pollsters – who was being interviewed and by what means – is no prerogative of experts but are questions that our said undergraduates seem to have rather greater confidence in answering that many professional journalists across major networks and national newspapers on both sides of the Atlantic.
The point is that it is not the job of political correspondents, pundits and commentators to sit around tables and predict elections in the same way that retired athletes get to make a living by speculating about the outcome of sporting competitions. They are, evidently, no good at it. Instead, we need commentators and journalists who understand the nature of evidence (be it statistical or otherwise) and focus on the questions they are better equipped to answer ‑ not the quantitative, but qualitative questions of the campaign: the “whys” and “hows”, not the “who” and “how manys.”
There are two possible explanations for their failure to do so, both of which I believe are contributing factors. Firstly, broadcasters have a double incentive for portraying a close race. Again, much like sporting contests that are billed as epic battles full of suspense and surprise, covering a closely fought presidential election is likely to attract higher ratings than a foregone electoral conclusion. Yet, even more importantly, portraying the contest as close is the premise for sustaining the enormous influx of revenue from campaigns buying airtime with donors on both sides being much less likely to contribute the cash that ultimately ends up in media organizations’ pockets if there appears little to play for. Someone like Nate Silver only spoils the party here.
Beyond this institutional failure, however, also lies a failure of journalistic integrity and competency on an individual level, as many professional journalists fail to approach their work in a systematic and indeed scientific fashion; a failure to question the empirical basis of their assumptions and conclusions and to engage with the plethora of information and knowledge from the academic community and other expert citizens that via the world wide web has become easily accessible to journalists and the general public alike. This failure is reflective of a disengaged, complacent and lazy attitude towards the nature of knowledge. I am not quite, like Judith Lichtenberg, raising the spectre of positivism here. But in face of a cultural and political movement that has appropriated the lessons of deconstructionism to set out to construct its own reality, political journalists have to learn that the old party trick of ‘balancing’ won’t do. Truth is not the mid-point between Karl Rove’s opinion and that of a morally sane person. Instead journalists need to take the real lesson from the accuracy of Silver’s predictions: not that numbers beat words, not that quantitative research is inherently superior to qualitative investigation, but that whatever type of knowledge and information we deal with, not least professionally, we need to critically examine its empirical and epistemological premises. From election predictions to reporting on social deprivation, global warming, and a range of other topics in which journalists seem unable to penetrate the fog of political spin through an almost hysterical reluctance to engage with the science behind the claims, leaving them to simply recycle talking points and poorly understood statistics, it’s logic, not balance that matters. Now that the election is over, those who make a living from creating and distributing knowledge, information and evaluation, shouldn’t marvel at the “magic powers” of maths – they should appreciate the indispensable need to critically reflect on the processes by which they do so. As the godfather of political punditry James Carville himself would have said: “It’s methodology, stupid!”
This is a good piece, and a gentle corrective to those trumpeting “math” as the winner of last night’s election. To be fair, “sound methodology” doesn’t fit as well on a macro.
I think the shaming of the punditocracy should also cause people to ask whether the best role of journalists, even “opinion journalists,” is to keep close tabs on the “horse race” aspect of politics and elections. It often seems that media outlets have abandoned the job of understanding and explaining policies and policy proposals and their effects in favor of “who’s winning now” stories. The latter are typically lower-cost, can easily be outsourced, and are often easier to sell to readers and viewers, who are naturally intrigued by narratives of competition. But their prominence also seems a result of news media searching for subject matter that is safe from complaints of “bias” (nine times out of ten, you can add “liberal” to that word). Explaining that there is a growing consensus among economists about the folly of supply-side policies, or that Romney’s “deficit-reduction plan” is incoherent, might be seen by some as an partisan intervention in the election, while simply reporting on the ins and outs of polls is often construed as “objective.” But ironically, even the latter has not proven safe from knee-jerk accusations of bias, particularly from the right–as your post discusses.
This is an old complaint, of course, but I thought it was worth adding here.
Nicely done Cornel. I wholeheartedly agree!
I entirely agree, Jonah – there is another paper to be written about the “sportization” of political coverage!