The Erosion of Causal Inference in Systematic Reviews in Epidemiology

Abstract

The assessment of disease causation is a complex process with a decades-long history of development and discussion.  The family of methods involved had been in place for at least 30 years when meta-analysis and the systematic narrative review emerged to be added to study designs and statistical methods.  More recently, methods to evaluate bias and quality have been added.  Traditionally, near the end of a causal assessment, that is, after all the studies have been collected and described and sometimes meta-analyzed, investigators apply a set of conditions (or criteria or considerations) to evaluate whether an association observed in epidemiological studies supports a causal association.  The criteria proposed by A.B. Hill—Hill’s criteria—are arguably the best-known example.  In this paper we describe and critically examine a trend in the epidemiological literature wherein some practitioners have been chipping away at this final step.  In some instances, the use of these criteria-based methods has been totally rejected; in other situations, some of the traditional criteria (or considerations) have been eliminated while others remain.  It is important to point out that these eliminations and exclusions are not replaced with some presumably better approach.  Rather, there is a sense that these so-called “criteria” are no longer relevant.  We see this process as eroding the reliability and validity of causal claims.