What is "publication bias"

n. (context statistics English) The bias to meta-analysis resulting from statistical studies with low statistical power tending to remain unpublished and inaccessible to the analyst.

Publication bias is a type of bias occurring in published academic research. Publication bias is of interest because literature reviews of claims about support for a hypothesis or values for a parameter will themselves be biased if the original literature is contaminated by publication bias. While some preferences are desirable—for instance a bias against publication of flawed studies—a tendency of researchers and journal editors to prefer some outcomes rather than others (e.g., results showing a significant finding) leads to a problematic bias in the published literature.

Studies with significant results often do not appear to be superior to studies with a null result with respect to quality of design. However, statistically significant results have been shown to be three times more likely to be published than papers with null results. Multiple factors contribute to publication bias. For instance, once a result is well established, it may become newsworthy to publish papers with reasonable power that fail to reject the null hypothesis. It has been found that the most common reason for non-publication is investigators declining to submit results for publication. Factors cited as underlying this effect include investigators assuming they must have made a mistake, to not find a known finding, loss of interest in the topic, or anticipation that others will be uninterested in the null results.

Attempts to identify unpublished studies often prove difficult or are unsatisfactory. One effort to decrease this problem is reflected in the move by some journals to require that studies submitted for publication are pre-registered (registering a study prior to collection of data and analysis). Several such registries exist, for instance the Center for Open Science.

Strategies are being developed to detect and control for publication bias, for instance down-weighting small and non-randomised studies because of their demonstrated high susceptibility to error and bias, and p-curve analysis.