The Erosion of Causal Inference in Systematic Reviews in Epidemiology
Main Article Content
Abstract
The assessment of disease causation is a complex process with a decades-long history of development and discussion. The family of methods involved had been in place for at least 30 years when meta-analysis and the systematic narrative review emerged to be added to study designs and statistical methods. More recently, methods to evaluate bias and quality have been added. Traditionally, near the end of a causal assessment, that is, after all the studies have been collected and described and sometimes meta-analyzed, investigators apply a set of conditions (or criteria or considerations) to evaluate whether an association observed in epidemiological studies supports a causal association. The criteria proposed by A.B. Hill—Hill’s criteria—are arguably the best-known example. In this paper we describe and critically examine a trend in the epidemiological literature wherein some practitioners have been chipping away at this final step. In some instances, the use of these criteria-based methods has been totally rejected; in other situations, some of the traditional criteria (or considerations) have been eliminated while others remain. It is important to point out that these eliminations and exclusions are not replaced with some presumably better approach. Rather, there is a sense that these so-called “criteria” are no longer relevant. We see this process as eroding the reliability and validity of causal claims.
Article Details
The Medical Research Archives grants authors the right to publish and reproduce the unrevised contribution in whole or in part at any time and in any form for any scholarly non-commercial purpose with the condition that all publications of the contribution include a full citation to the journal as published by the Medical Research Archives.
References
2. Hill AB. Statistical evidence and inference. Chapter 24 in: A Short Textbook of Medical Statistics. London:Hodder and Stoughton 1971:283-296.
3. Susser M. Rules of inference in epidemiology. Reg Tox Pharm. 1986;6:116-28.
4. Aschengrau A, Seage GR. The epidemiologic approach to causation. Chapter 15 in: Essentials of Epidemiology in Public Health. Sudbury, MA:Jones and Bartlett, 2003;375-401.
5. Beaglehole R, Bonita R, Kjellstrom T. Causation in epidemiology. Chapter 5 in: Basic Epidemiology. Geneva:World Health Organization, 1993;71-81.
6. Gordis L. From association to causation: Deriving inferences from epidemiologic studies. Chapter 13 in: Epidemiology. 2nd ed. Philadelphia:W.B. Saunders, 2000;184-203.
7. Goodman SN, Samet JM. Causation and causal inference. Chapter 1 in: Schottenfeld D, Fraumeni Jr, JF. Cancer Epidemiology and Prevention. 3rd ed. New York:Oxford, 2006;3-9.
8. Weed DL. Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related. Int J Epidemiol. 2000;29:387-390.
9. Weed DL, Hursting SD. Biologic plausibility in causal inference: current method and practice. Am J Epidemiol. 1998;147:415-425.
10. Weed DL. Analogy in causal inference: rethinking Austin Bradford Hill’s neglected consideration. Ann Epidemiol. 2018;28:343-346.
11. Cogliano VJ, Baan RA, Straif K, et al. The science and practice of carcinogen identification and evaluation. Environ Health Perspect. 2004;112:1269-1274.
12. United States Environmental Protection Agency (EPA). Guidelines for Carcinogen Risk Assessment. Risk Assessment Forum. U.S. Environmental Protection Agency, Washington, DC. EPA/630/P-03/001F March, 2005.
13. MacMahon B, Pugh TF. Concepts of cause. Chapter 2 in: Epidemiology: Principles and Methods. Boston:Little, Brown, 1970.
14. Mausner JD, Bahn AK. The search for causal relations: Observational studies. Chapter 5 in: Epidemiology: An Introductory Text. Philadelphia:W.B. Saunders. 1974;91-111.
15. Kleinbaum DG, Kupper LL, Morgenstern H. Fundamentals of Epidemiologic Research. Chapter 2 in: Epidemiologic Research. Belmont, CA:Lifetime Learning. 1982;19-39.
16. Rothman KJ. Causal Inference in Epidemiology. Chapter 2 in: Modern Epidemiology. Boston: Little, Brown. 1986;7-21.
17. Weed DL. Causal and Preventive Inference. Chapter 17 in: Greenwald P, Kramer BS, Weed DL. Cancer Prevention and Control. New York:Marcel Dekker, 1995;285-302.
18. Kelsey JL, Pettiti DB, King AC. Key Methodologic Concepts and Issues. Chapter 2 in: Brownson RR, Pettiti DB. Applied Epidemiology: Theory to Practice. New York:Oxford University Press, 1998;35-69.
19. Rothman KJ and Greenland S. Causation and Causal Inference. Chapter 2 in: Modern Epidemiology. 2nd ed. Philadelphia:Lippincott, Raven. 1998.
20. Vetter N, Matthews I. Causation. Chapter 3 in: Epidemiology and Public Health Medicine. London: Churchill, Livingstone, 1999;23-30.
21. Rothman KJ. What is Causation? Chapter 2 in: Epidemiology: An Introduction. New York:Oxford University Press, 2002;8-23.
22. Bhopal R. Cause and effect: The epidemiological approach. Chapter 5 in: Concepts of Epidemiology. New York:Oxford University Press, 2002;98-132.
23. Kundi M. Causality and the interpretation of epidemiologic evidence. Environ Health Perspect. 2006;114:969–974.
24. Ward AC. The role of causal criteria in causal inferences: Bradford Hill's "aspects of association.” Epidemiol Perspect Innovat. 2009, 6:2.
25. National Toxicology Program (NTP). Handbook for conducting a literature-based health
assessment using OHAT approach for systematic review and evidence integration. National Institute for Environmental Health Sciences (NIEHS), 2019.
26. Fedak KM, Bernal A, Capshaw ZA, et al. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015;12:14 DOI 10.1186/s12982-015-0037-4.
27. Carbone M, Klein G, Gruber J, et al. Modern criteria to establish human cancer etiology. Ca Res. 2004;64: 5518–5524.
28. Shapiro S. Causation, bias and confounding: a hitchhiker’s guide to the epidemiological galaxy. J Fam Plann Reprod Health Care. 2008;34:261-264.
29. Ioannidis JPA. Exposure-wide epidemiology: revisiting Bradford-Hill. Stat Med. 2015;doi:10.1002/sim.6825.
30. Van Reekum R, Streiner EL, Conn DK. Applying Bradford-Hill’s criteria for causation to neuropsychiatry. J Neuropsych Clin Neurosci. 2001;13:318-325.
31. International Agency for Research on Cancer (IARC). Preamble. IARC Monographs on the Evaluation of Carcinogenic Risks to Humans. 2006. World Health Organization: Lyon, France.
32. Susser M. Criteria of judgment. Chap. 11 in Causal Thinking in the Health Sciences. Oxford:University Press, 1973:140-162.
33. Checkoway H, Pearce N, Kriebel D. Research Methods in Occupational Epidemiology. 2nd ed. Oxford:University Press; 2004.
34. Baker D. Review of environmental health and epidemiological principles. In: Baker D, Nieuwenhuijsen MJ. Environmental Epidemiology: Study methods and applications. Oxford:University Press; 2008:15-40.
35. Hall W. A simplified logic of causal inference. Aust NZ J Psychiatry. 1987;21:507-513.
36. Adami HO, Berry CL, Breckenridge CB, et al. Toxicology and epidemiology: Improving the science with a framework for combining toxicological and epidemiological evidence to establish causal inference. Tox Sci. 2011;122:223-234.
37. Schlesselman JJ. “Proof” of cause and effect in epidemiological studies: criteria for judgment. Prev Med 1987;16:195-210.
38. Thygesen LC, Andersen GS, Andersen H. A philosophical analysis of the Hill criteria. J Epidemiol Commun Health. 2005;59:512-516.
39. Lucas RM, McMichael AJ. Association or causation: evaluating links between “environment and disease.” Bull World Health Org. 2005;83:792-795.
40. Rothman KJ and Greenland S. Causation and causal inference. Am J Pub Health. 2005;95(Suppl 1):S144-150.
41. Weed DL. The nature and necessity of scientific judgment. J Law Policy. 2007;15:135-164.
42. Weed DL. Commentary: On the reliability of causal claims. Global Epidemiol. 2022;(in press).