Advanced lesson on fallacies arising from biases in research publication and reporting. Students learn to recognize publication bias, funding influences on research, selective outcome reporting, and the problematic practice of hypothesizing after results are known. These fallacies are critical for evaluating scientific literature and research claims.
The systematic tendency for research with positive, novel, or statistically significant results to be published preferentially over research with negative, null, or non-significant findings. This creates a distorted literature where published research overestimates effect sizes and gives a misleading impression of scientific consensus.
The accumulation of unpublished studies with null or negative results in researchers' 'file drawers,' leading to a published literature that systematically overrepresents positive findings. This is the mechanism that produces publication bias, where failed studies remain unknown and unavailable for meta-analysis.
The systematic tendency for research funded by sources with financial or ideological interests in the results to produce findings favorable to those interests, while still appearing methodologically sound. This bias operates through subtle choices in study design, outcome selection, analysis decisions, and interpretation rather than through outright fraud.
The tendency for researchers to selectively cite studies that support their hypothesis or theoretical position while failing to cite or adequately discuss contradictory findings. This creates citation networks and review articles that misrepresent the balance of evidence by amplifying certain findings through repeated citation while marginalizing others.
The selective reporting, emphasizing, or switching of outcome measures based on the results, where pre-specified primary outcomes that showed null effects are downplayed while unplanned secondary outcomes showing positive effects are promoted to primary status. This includes reporting only favorable outcomes from among multiple measured outcomes.
Formulating hypotheses or theoretical explanations after examining the data and observing patterns, then presenting these post-hoc hypotheses as if they had been predicted a priori. This involves reverse-engineering theory from results while maintaining the appearance of confirmatory hypothesis testing.