Exploring the Tone of the 2016 Campaign

By undergraduate students Megan Bayagich, Laura Cohen, Lauren Farfel, Andrew Krowitz, Emily Kuchman, Sarah Lindenberg, Natalie Sochacki, and Hannah Suh, and their professor Stuart Soroka, all from the University of Michigan.


The 2016 election campaign seems to many to have been one of the most negative campaigns in recent history. Our exploration of negativity in the campaign – focused on debate transcripts and Facebook-distributed news content – begins with the following observations.

Since the advent of the first radio-broadcasted debate in 1948, debates have become a staple in the presidential campaign process.  They are an opportunity for voters to see candidates’ debate policies and reply to attacks in real-time. Just as importantly, candidates use their time to establish a public persona to which viewers can feel attracted and connected.

Research has accordingly explored the effects of debates on voter preferences and behavior. Issue knowledge has been found to increase with debate viewership, as well as knowledge of candidates’ policy preferences. Debates also have an agenda-setting effect, as the issues discussed in debates then tend to be considered more important by viewers. Additionally, there is tentative support for debate influence on voter preferences, particularly for independent and nonpartisan viewers. While debate content might not alter the preferences of strong partisans, it may affect a significant percentage of the population who is unsure in its voting decision. (For a review of the literature on debates effects, see Benoit, Hansen, & Verser, 2003).

Of course, the impact of debates comes not just from watching them but also from the news that follows. The media’s power to determine the content that is seen and how it is presented can have significant consequences. The literatures on agenda setting, priming, and framing make clear the way in which media shape our political reality. And studies have found that media’s coverage of debates can alter the public’s perception of debate content and their attitudes toward candidates. (See, for instance, Hwang, Gotlieb, Nah & McLeod 2006, Fridkin, Kenney, Gershon & Woodall 2008.)

This is true not just for traditional media, but for social media as well. As noted by the Pew Research Center, “…44% of U.S. adults reported having learned about the 2016 presidential election in the past week from social media, outpacing both local and national print newspapers.” Social media has become a valuable tool for the public to gather news throughout election cycles, with 61% of millennials getting political news from Facebook in a given week versus 37% who receive it from local TV. The significance of news disseminated through Facebook continues to increase.

It is in this context that we explore the nature of the content and coverage of the presidential debates of 2016.  Over the course of a term-long seminar exploring media coverage surrounding the 2016 presidential election, we became interested in measuring fluctuations in negativity across the last 40 years of presidential debates, with a specific emphasis on the 2016 debates. We simultaneously were interested in the tone of media coverage over the election cycle, examined through media outlets’ Facebook posts.

To test these hypotheses, we compiled and coded debate transcripts from presidential debates between 1976 and 2016. We estimated “tone” using computer-automated analyses. Using the Lexicoder Sentiment Dictionary (LSD) we counted the number of positive and negative words across all debates. We then ran the same test over news articles posted on Facebook during the election cycle, taken news feeds of main media outlets including ABC, CBS, CNN, NBC, and FOX. (Facebook data are drawn from Martinchek 2016.)

We begin with a simple measure of the volume of tone, or “sentiment,” in debates.  Figure 1 shows the total amount of sentiment – the total number of positive and negative words combined, as a percentage of all words – in all statements made by each in candidate across all debates.  In contrast with what some may expect, the 2016 debates were not particularly emotion-laden when compared to past cycles. From 1976 through to 2016, roughy 6.9% of the words said during debates are included in our sentiment dictionary. Hillary Clinton and Donald Trump’s speeches were essentially on par with this average; neither reached the peak of 8% (like 2004) or the low of 6% (like 2012).

Figure 1: Total Sentiment in Debates, 1976-2016

Figure 2 shows the percent of all sentiment words that were negative (to be clear: negative words as a percent of all sentiment words), and here we see some interesting differences.  Negativity from Democratic candidates has not fluctuated very much over time. The average percent of negative sentiment words for Democrats is 33.6%.  Even so, Hillary Clinton’s debate speeches showed relatively high levels of negativity, at 40.2%. Indeed, Clinton was the only Democratic candidate other than Mondale to express sentiment that is more than 40% negative.

Figure 2: Negativity in Debate Speeches, By Political Party, 1976-2016

Clinton’s negativity pales in comparison with Trump’s, however.  Figure 2 makes clear the large jump in negativity for Donald Trump in comparison with past candidates. For the first time in 32 years, sentiment-laden words used by Trump are nearly 50% negative – a level similar to Reagan in 1980 and 1984. Indeed, when we look at negative words as a proportion of all words, not just words in the sentiment dictionary, it seems that nearly every one in ten words uttered by Trump during the debates was negative.

The 2016 debates thus appear to be markedly more negative than most past debates. To what extent is the tone of debate content reflected in news coverage? Does negative speech in debates produce news coverage reflecting similar degrees of negativity?  Figure 3 explores this question, illustrating negativity (again, negative words as a proportion of all sentiment words) in the text of all Facebook posts concerning either Trump or Clinton, as distributed by five major news networks.

What stands out most in Figure 3 are the differences across networks: ABC, CNN, and NBC show higher negativity for Trump-related posts, while Fox shows higher negativity for Clinton-related posts.  CBS posts reflect a more neutral position.

Figure 3: Negativity in Facebook News Postings by Major Broadcasters, By Candidate, 2016

Clearly, political news content varies greatly across news sources. Trump’s expressed negativity in debates (and perhaps in campaign communications more generally) does not necessarily translate to more negative news content, at least by these measures. For instance: even as Trump is expressing more negative sentiment than Clinton, coverage in Fox is more positive towards Trump.  Of course, news coverage isn’t (and shouldn’t be) just a reflection of what candidates say. But these make clear that the tone of coverage for candidates needn’t be in line with the sentiment expressed by those candidates.  Expressing negative sentiment can produce negative coverage, or positive coverage, or (as Figure 3 suggests), both.

This much is clear: in line with our expectations, the 2016 presidential debates were among the most negative of all US presidential debates.  The same seems true of the campaigns, or at least the candidates’ stump speeches, more generally.  Although there was a good deal of negativity during debates, however, the tone of news coverage varied across sources.  Depending on citizens’ news source, even as candidates seem to have focused on negative themes, this may or may not have been a fundamentally negative campaign cycle. For those interested in the “tone” of political debate, our results highlight the importance of considering both politicians’ rhetoric, and the mass-mediated political debate that reaches citizens.

 


This article was co-authored by U-M capstone Communication Studies 463 class of 2016, which took place during the fall election campaign. Class readings and discussion focused on the campaign, and the class found themselves asking questions about the “tone” of the 2016 debates, and the campaign more generally. Using their professor Stuart Soroka as a data manager/research assistant, students looked for answers to some of their questions about the degree of negativity in the 2016 campaign.