“Everything we think we know may be wrong. The correct results could be sitting in people’s file drawers because they can’t get them published” David Lehrer.
The issue about whether negative results, otherwise known as non-significant results, should be published is a very controversial issue. It has been suggested by The All Results Journal that approximately 60% of experiments conducted fail to produce the desired results and significant findings, however the proportion of these findings that are published is extremely low. In some parts of the USA 95-100% of studies published are of positive results, indicating that very few negative results are published (Morgan, 2010).
Many of you may be wondering why I am implying this is bad. We conduct experiments to try and find a difference between groups and these are the results that are being published so what’s the problem? Well, as this is a stats blog I’d better include a bit of maths in it. If in studies we are looking for results where the alpha level is set at .05 then there is a 1 in 20 chance that a difference will occur due to chance. So using this statistic we can assume that if 20 experiments are conducted then there is also a 1 in 20 chance that 1 of these will turn out to have significant findings due to chance. Therefore, if a study finds positive results then it appears to be significantly valid when it stands alone, but if it was presented in context with the other 19 studies which have negative results (and therefore are unlikely to be published) then we would have a different outlook on it, as explained by Steffer (2011). This demonstrates how the publication bias towards positive results influences perspective on positive results and creates an unbiased outlook.
Journal publishers say that both negative and positive results are equally considered when deciding what to publish, however research has shown that results take significantly longer to get published if they are negative rather than positive, if they get published at all, see Stern and Simes (1997). The graph below, taken from their research, shows that the amount of results unpublished remains higher for negative results over increasing periods of time compared to positive results.
The awareness of the bias of publication is increasing and there are now journals which are dedicated to publishing negative results, such as the Journal of Articles in Support of the Null Hypothesis for psychology. However, as a researcher planning a study are you likely to go and look for literature in a journal of negative results rather than a more reputable journal such as the Journal of Experimental Psychology which is more likely to publish positive results? I doubt it!
So what are the major implications of the bias of not publishing negative results? Well for starters it can hinder scientific research. If studies are not published then different researchers will continue to run almost identical studies and continue finding the same negative results which remain unpublished, which wastes a lot of time, effort and money. Sometimes it can be just as important to know that one variable does not affect another, as it is to know that it does, and to show that there is obviously something wrong with the paradigm you are investigating. Even if these results may not appear relevant to current research, future research can stem from this and may lead to other findings, whether positive or negative (Rice, 2011). The increased publication of negative results may also take the pressure off researchers to find positive results in their data so may reduce the likelihood of exaggerated results and the manipulation of data by only reporting findings from certain participants or parts of the methodology, therefore creating a more realistic image of the findings. But no matter what, the lack of publication of negative results creates a biased view of studies and can lead to a Type 1 error.
But let’s try and view this from another perspective, because other than positive results being far more exciting there must be other reasons why this bias exists. Firstly, can you imagine the size of journals if they were to publish the 60% of negative results as well as the 40% of positive results they already publish (assuming all studies are reported)? It would be a nightmare for researchers, or students, to sift through all this literature to find the material that is relevant and useful for what they want. Writing up a report to be published takes a lot of time, so would it be more productive to leave these studies the way they are and begin on a new investigation instead of wasting time writing up a failed one? Negative results can also have psychological effects. The reputation of a discipline can be greatly impacted on if the majority of the results published were negative. People may no longer give money to a charity for research if it appears like it is being wasted on studies that are not finding anything significant. This can reduce the motivation and faith in research, which can be especially important for many areas of research, such as treatments for cancer where the hope that research will find a cure is what can spur patients on.
So what can be done to reduce this issue? It has been suggested that researchers should enter their hypothesis and methodology into a database before conducting a study, and must therefore insert the data found afterwards regardless of the outcome, and even if it isn’t written up as a report (Schooler, 2011). This method has already been used successfully in clinical and educational research fields in the US.
After the information presented above I hope as fellow students you can now appreciate how all findings are important and should have equal opportunities for publishing in order to create a more balanced and realistic view of research. Negative results make up a large proportion of the data that is obtained however it is not adequately represented in literature so has a major impact on our perceptions, and can influence future research.
This article shows many varying views regarding whether negative results should be published or not.