Of the Importance to Publish Negative Results

I had a rough time during my PhD with many experiments that did not support a common hypothesis in my field of research. However, I was able to successfully submit a manuscript describing my negative data. Recently I even won a prize for publishing them.

When scientists embark on a new study, they formulate a hypothesis that they want to test. Sometimes the experiments do not support the hypothesis the researchers set out to test. If the obtained data are unable to confirm a hypothesis or replicate previous results, they are called negative results. Sometimes they are also called NULL results, as the Null hypothesis H₀ (the hypothesis that there will be no difference between experimental and control group) was not rejected. Most of the time, negative results are more accurate and give more informative than results that support a new hypothesis. 

If a test of experimental data comes up significant with p < 0.05, we reject H₀ and accept H₁ (the hypothesis that the results show an effect). Notably, we only tested H₀ and the p-value says nothing about the probability of H₁ being true. However, a non-significant p-value means that H₀ is true (or the data didn’t have enough power to reject it). In a Bayesian sense, data underlying a non-significant p-value can be strong evidence for the H₀. 

Negative data are obviously not very spectacular, because we want to find out what is true, not what isn’t. Positive results seem more interesting and more important than NULL results. The latter are often not submitted for publication, because they are believed to generate less value to scientists and academic publishers. Indeed, they are less likely to open new avenues of research that generate funding opportunities. Manuscripts reporting negative data are also more likely to get rejected, because they appear less exciting. Traditionally it is difficult to publish negative data, unless they refute a spectacular claim. Studies that do not confirm a new hypothesis often get literally filed away in a drawer. Therefore this is also called the “file drawer phenomenon”.

PUBLISH ALL RESULTS TO FIGHT THE PUBLICATION BIAS! 

Unfortunately, the negative data get lost to the scientific community. If ever another group of researchers has a similar hypothesis, they are likely to tap into the same dead end. The fact that such negative data are rarely published, leads other scientists to waste time and effort by unnecessarily repeating experiments. It is estimated [1] that this costs the US economy alone, $28bn each year, similar in scale to the total $35bn National Institute of Health annual budget [2]. Moreover, the bias towards positive results can lead to an overestimation of biological phenomena or efficacy of drugs. It is devastating and frustrating, if the biased representation of preclinical work compromises the outcome of drug trials. Thus, publishing more negative results will have a positive impact on the development of new drugs and healthcare solutions. 

by Maklay 62 via pixabay

Another current problem is reproducibility. Even though it is fundamental to scientific progress, replication of studies carries little prestige in academic research. Especially in neuroscience, reproducibility has come under particular focus due to some spectacular cases, where data could not be reproduced [3]. Recently, systematic studies demonstrated that current biomedicine has a serious replication problem. It is shocking that more than half of the published biomedical data could not be reproduced [1]. This led to the declaration of a reproducibility crisis. It is necessary to value the effort to reproduce and publish studies regardless of their outcome.

 SCIENCE IS MOST EFFECTIVE WHEN BOTH POSITIVE AND NEGATIVE RESULTS ARE PUBLISHED

Fortunately, many journals now publish reproduction studies and negative data; for example PeerJ, PlosONE, J Neg Res Biomed, Scientific reports and others. Furthermore the necessity to reproduce experiments and publish negative results gets now also recognized by funding agencies that award publications that do not confirm the expected outcome or original hypothesis. The prizes aim to emphasize the value in publishing all the results, as science is most effective when both positive and negative results are published. Another way to fight publication bias and focus on the scientific process and soundness are “Registered Reports”. For this type of journal article, methods and proposed analyses are pre-registered prior to research being conducted. Thereby the results are accepted for publication before data collection commences and without regard to their positive or negative outcome.  
These efforts show, that the recognition to publish negative results and replication studies is growing. Hopefully this will contribute to the soundness of science and retrieve research from the reproducibility crisis.

QUEST is giving away 15 awards of € 1,000  to first/last/corresponding authors (BIH, MDC or Charité affiliation) of preclinical or clinical research papers in which the main result is a NULL or ‘negative’ or in which the replication of own results or the results of others is attempted. Futher information can be found here.

The ECNP’s Preclinical Data Forum created the “ECNP Preclinical Network Data Prize”, a prize for published “negative” scientific results, of €10,000. Aimed initially at neuroscience research, it encourages publication of data where the results do not confirm the expected outcome or original hypothesis. The ECNP’s Preclinical Data Forum is a mixed industry and academic group which aims to improve the replicability and reliability of scientific data, especially in drug development. Futher information can be found here.

by Claudia Willmes, PhD Alumna AG Eickholt / AG Schmitz  

[1] sciencemag, 2015 http://bit.ly/2E5ho01
[2  sciencemag, 2017 http://bit.ly/2uWuFTt
[3] nature news, 2014 http://go.nature.com/2rAME4b