post developed by Katherine Pearson

There are people all over the world who need the social sciences to reconcile the problems of their lives. At the Open Social Science Conference in Mannheim, Germany, Arthur Lupia told attendees that this is why social scientists must work to ensure that their work is transparent and credible. Lupia delivered his keynote address “It’s Up to Us – Transparency and the Public Value of Science” on Friday, January 25, 2019. A recording of the talk is available online.

Lupia studies how people make decisions when they don’t have much information. Quantitative researchers, he notes, make scientific claims backed up by proofs. A proof allows others to understand the thought process and produce the same result. It allows researchers to go beyond their own understanding, and bring others along.

It’s not always sufficient to show evidence, however. The scientist must also consider the person who receives the information: what they pay attention to, how they make sense of that information, what they understand. In order to create change, social science research must first reach its audience. The way scientists communicate knowledge makes a difference.

What makes social science research valuable to people who use it?

The mission of social science researchers, according to Lupia, is to offer people a better understanding of the relationship between things they can see and things they can do.

The amount of information available has increased greatly within our lifetimes. An abundance of information is available online. So why should anyone seek out rigorous scientific research? If information can be found for free, why should anyone fund social science?

To answer these questions, Lupia starts by explaining that decisions rely on evaluations of evidence. The essential criteria for evaluating evidence are:

  • Credibility – there is something about the claim that is trustworthy?
  • Legitimacy – is this claim developed in accordance with recognized principles?

Challenges facing scientists

While scientific inquiry fosters greater honesty in evaluations, the incentives that scientists face present challenges, as well. Scientists have a strong incentive to discover novel findings and publish them. The marketplace rewards claims that go viral. On the other hand, the incentives to explain how these discoveries emerge are very weak.

These incentives threaten the meaning and the value of the research. If scientists publish only astonishing results, but ignore more mundane studies that disagree, scientific credibility and legitimacy are eroded. If social scientists want to advance research that improves quality of life in the world, then credibility and legitimacy matter a lot.

Opportunities to improve transparency

Transparency in scientific research can be improved by creating better incentives. Because researchers already have strong incentives to publish, Lupia suggests that it makes sense to focus efforts on academic journals. He gave several examples of incentives that help journals increase transparency.

The Data Access and Research Transparency (DA-RT) Statement increases transparency in social science. The statement was signed by 28 academic journals that promised to “commit to greater data access and research transparency, and to implementing policies requiring authors to make as accessible as possible the empirical foundation and logic of inquiry of evidence-based research.”

The Transparency and Openness Promotion Guidelines are a model for increasing transparency in publishing. Created by the Center for Open Science, over 8,000 journals have committed to these guidelines. CoS badges

The Center for Open Science has created badges to show when a study has preregistered, uses open data, or uses open materials. Research has shown that implementing these badges dramatically increases the rate of data sharing.

The Election Research Preacceptance Competition, led by Lupia and Brendan Nyhan, asked researchers to design a paper using data from the 2016 American National Election Studies (ANES) and submit the design to a participating journal before the data were available for analysis. In return, selected entries were offered an award of $2,000 and placement on a panel at a major political science conference. This challenged researchers to share ideas regardless of whether the analysis yielded the expected results. Lupia noted that they received fewer submissions than they expected. He suspects that it’s difficult for researchers, especially younger ones, to take the risk to publicly share an idea without knowing what the outcome will be.

The value of social science is up to us

When was the golden age of social science? Lupia says it is dawning right now. Scientists must ensure their work has value to those who can use it to make change, and that’s why they need to commit to greater transparency. Increasing transparency requires creating incentives and improving infrastructure for openness. Most people who consume scientific research will never read the evidence that went into the findings. Instead they take the results on faith. Lupia cautions that if researchers don’t uphold standards of transparency, they lose their value as credible sources of knowledge.