Does the media convey accurate information about what policymakers do?

Post developed by Lauren Guggenheim 

Citizens in a democracy must have access to reliable information about what is happening in government policy in order to form meaningful preferences and hold politicians accountable. But do media consumers get the information they need to have informed policy preferences?

Past research suggests that public opinion on policy issues is thermostatic; that is, the public adjust their preferences for policy spending downward when spending increases and upward when spending decreases. In their new book, Information and Democracy: Public Policy in the News, Stuart Soroka and Christopher Wlezien argue that the public respond in this way because they are getting information about policy spending from the news.

For anyone paying attention to academic or public debates about news quality, it may not seem like this could be the case. Consider that the news media are regularly accused of having a partisan bias, failing to cover important issues, or relying on sensationalism to draw in viewers. Moreover, researchers and other observers also have concerns about the ability of the public to process the information they do receive; many people are not interested in following the news, and those that do may interpret information through a lens of pre-existing beliefs.

Nevertheless, the public does seem to be picking up on cues about policy. Soroka and Wlezien set out to measure the accuracy of media coverage of policy and identify the domains where the media tend to provide more or less faithful coverage of policy. In the book, they investigate the frequency and reliability of media coverage in five spending domains: defense, welfare, health, education, and the environment. Examining decades of government spending, media coverage, and public opinion in the US, Information and Democracy assesses the accuracy of media coverage, and assesses its direct impact on citizens’ preferences for policy.

To measure media accuracy, the authors use a two-step process. First, they create a “media policy signal” which is based on using a dictionary to code sentences about spending in each of the five policy domains across different news outlets. The dictionary codes each sentence as positive/upward changes in spending, negative/downward changes in spending, or no change in spending, scored +1, -1, or 0, respectively. The “media policy signal” is a sum of these codes by year, so that years where there are more upward changes in spending, the signal is positive, and years in which there are more downward changes, the signal is negative. The second step involves modeling the “media policy signal” against actual changes in the budget for these policy domains over time. Actual changes in the budget are drawn from the Historical Tables released by the Office of Management and Budget (OMB).

A website based on the book is available at http://mediaaccuracy.net, where users can examine both the volume and accuracy of media coverage of policy by news outlet across the five topic domains. These volume and accuracy measures are based on the method described above and cover the computer-automated content analyses of millions of newspaper articles and television transcripts. Data displayed on the website compare the “signal” in media coverage to actual budgetary policy change. When media coverage matches policy change, media accuracy is high. When media coverage does not match policy change, media accuracy is low. Through a series of infographics, readers can examine the accuracy of 17 major US newspapers and 6 television networks over the past 20-30 years.

The results show a more nuanced picture of the relationship between media coverage and actual spending on policy than critics might expect. In some domains, news coverage indeed does not reflect changes in spending especially well. However, there are other domains, such as defense, welfare, and health where the relationship between news coverage and actual spending is strong. http://mediaaccuracy.net shows how this extends to individual news outlets. These results suggest that, while not always accurate, the media can convey accurate and useful information about government spending, and can play a role in the political accountability necessary in a modern democracy. The degree to which this is true does vary across outlets, however, sometimes in substantial ways.

Newspaper coverage of welfare

The chart above shows volume and accuracy scores for newspaper coverage of welfare for 17 different newspapers.

The chart above shows volume and accuracy scores for newspaper coverage of welfare, with media outlets in alphabetical order.

Data on the Russian invasion of Ukraine available in near-real time

Post developed by Katherine Pearson 

In order to track and share data on events unfolding in Ukraine, Yuri Zhukov, Associate Professor of Political Science and Research Associate Professor at the Center for Political Studies, launched VIINA: Violent Incident Information from News Articles on the 2022 Russian Invasion of Ukraine. VIINA is a near-real time multi-source event data system for the invasion. map of Ukraine showing location of violent incidents on March 7, 2022

“I wanted to make these data available immediately because media sites in both countries are already being shut down, due to either censorship (in Russia) or military operations (in Ukraine),” said Zhukov. “It is thus essential that researchers have access to information about the war, as reported across media organizations and other actors in the information space.” While different media cover different types of events, VIINA’s multi-source approach will capture a more accurate picture of events as they unfold.

This platform allows researchers to access data based on news reports from Ukrainian and Russian media, which have been geocoded and classified into standard conflict event categories through machine learning.

VIINA is freely available for use by students, journalists, policymakers, and researchers. Using an automated web scraping routine that runs every 6 hours, VIINA extracts the text of news reports published by each source and their associated metadata, including publication time and date, web urls. GIS-ready data can be downloaded from VIINA, with temporal precision down to the minute.

VIINA draws on news reports from a variety of Ukrainian and Russian news providers. Data sources currently include news wires, TV stations, newspapers, and online publications in both countries. Zhukov plans to expand these sources as the conflict unfolds, to include OSINT social media feeds and other key sources. The set of sources may also change as the war unfolds — due to interruptions to journalistic activity from military operations, cyber attacks, and state censorship, as well as the availability of new data from other information providers.

VIINA: Violent Incident Information from News Articles on the 2022 Russian Invasion of Ukraine.