ICYMI (In Case You Missed It), the following work was presented at the 2017 Annual Meeting of the American Political Science Association (APSA). The presentation, titled “The IMF and the Political Economy of GDP Data Production” was a part of the session “Economic Growth and Contraction: Causes, Consequences, and Measurement” on Sunday, September 3, 2017.
Political economists often theorize about relationships between politics and macroeconomics in the developing world; specifically, which political or social structures promote economic growth, or wealth, or economic openness, and conversely, how those economic outcomes affect politics. Answering these questions often requires some reference to macroeconomic statistics. However, recent work has questioned these data’s accuracy and objectivity. An under-explored aspect of these data’s limitations is their instability over time. Macroeconomic data is frequently revised ex post, or after the fact, and as such one could ask the same question of (ostensibly) the same data, and get different answers depending on when the question was asked.
We set out to explore the political economy of data production by examining a newly available dataset of ex post revisions to World Development Indicators (WDI) data. Ex post revisions occur when newly available information changes national statistical offices’ beliefs about the nature of the economy. Those revisions extend into the past, effectively rewriting history and, in the process, providing a reasonable proxy for the inaccuracy of the initial reports. These revisions affect a wide swath of data, but we focus on Gross Domestic Product (GDP) and GDP-derived measures, like GDP per capita and GDP growth. GDP revisions are common—most GDP data available for download at the WDI are different now than they were at the time of its initial release. Normally these changes are subtle; other times they are substantial enough to condemn prior data releases as misleading.
We use these revisions to answer two related questions. First, how sensitive are political-economy relationships to GDP revisions? Should researchers worry about revisions-driven instability in the state of political-economic knowledge? We show that they should. To illustrate, we subject a simple, bivariate statistical relationships between democracy and growth to re-estimation using alternative versions of the “same” data. The democracy-growth relationship has been a topic of sufficient interest in economics and political science that instability in this relationship should give us reason for pause. Seen in this light our estimates are worrisome. As we show in Figure 1 below, our estimates are unstable across different “observation years” and further, they are unstable in ways that suggest that initial estimates were biased. Rather than simply a diminution of standard errors as more heavily revised data are introduced (which is what we would expect to see if revisions simply reduced random “noise in the data”), the estimated coefficients for Democracy change substantially across models estimated with different revisions of the same country-year GDP growth data.
Figure 1: GDP Growth ~ Democracy
This finding anticipates our second question: Given the likelihood that GDP revision are non-random, what accounts for ex post revisions? What does the “political economy” of revisions look like? We show using Kolmogorov-Smirnov tests (see Figure 2) and random forest models (see Figure 3) that the International Monetary Fund (IMF) influences the magnitude of revisions for GDP and GDP-related measures. That is not entirely surprising. Our suspicion that the IMF would have such an effect is a straightforward recognition of its well-publicized efforts to provide financial and human resources to the national statistical offices of the countries in which it works. What we have “uncovered” in this exercise is simply one consequence of the IMF doing precisely what it has publically said it is doing. But this finding’s (retrospective) obviousness does not diminish its importance. Consider the empirical challenges that this presents. Political economists often ask if the IMF affects the way economies functions, but the IMF’s independent effect on the way economies are measured substantially complicates our ability to know if it does. And it doesn’t just complicate our ability to know if the IMF’s policies affect the economy, it complicates our ability to know if anything correlated with IMF participation affects the economy. Many important things correlate with IMF participation, including, for example, democracy, a country’s relationship with the UN, and whether or not a country is an ally of the United States.
Figure 2: Distributions of GDP Growth Changes
Figure 3: Predictors of GDP Growth Revisions
Of course, politics likely affects the way the economy is measured in a variety of ways that have nothing to do with the IMF. Our random forest analysis suggests that democracy might also have an effect, for example, as might public sector corruption, and it is not hard to tell a plausible post hoc story for why that might be. But our aim is not to provide a comprehensive picture of the political economy of data production, but simply to show that it exists, and that it exists in a manner that should alert us to its importance. Taking seriously the political provenance of ostensibly apolitical data is an important (and, we believe, interesting) step towards refining the state of political economy knowledge.
 The raw data used in this paper are available at http://databank.worldbank.org/data/reports.aspx?source=WDI-Archives. To facilitate researcher use of this data, we will make it available in an R package, revisions. This package will contain long- and wide-format data sets.