Translational roadblock: are publication guidelines the way around it?
Main Article Content
Abstract
With regards to new insights, the scientific community has to rely on transparent and accurate reporting of design, conduct and outcomes of studies in order to allow readers, peer reviewers and editors an assessment of methodology and relevance. In animal experimentation, it was found that, to a disturbing extent, vital information such as the method of randomization and blinding is not or inadequately disclosed in publications. To make matters worse, a sizable number of animal studies is not reported at all, leading to an overstatement of intervention effects, unnecessary repetitions and premature clinical studies.
Both factors contribute to translational problems where clinical trials do not reflect the results of preclinical findings and seriously impede therapy development. In addition, unnecessary in vivo studies present a considerable animal welfare issue. In 2010, two sets of guidelines were published with the aim of improving on project design and reporting, making meta-analyses and systematic reviews more feasible and reducing animal use: the Gold Standard Publication Checklist (GSPC) and the Animal Research: Reporting In Vivo Experiments (ARRIVE) guidelines. This mini-review focuses on the ARRIVE-Guidelines.
Article Details
The Medical Research Archives grants authors the right to publish and reproduce the unrevised contribution in whole or in part at any time and in any form for any scholarly non-commercial purpose with the condition that all publications of the contribution include a full citation to the journal as published by the Medical Research Archives.
References
Alfaro, V. (2005). Specification of laboratory animal use in scientific articles: current low detail in the journals' instructions for authors and some proposals. [Review]. Methods Find Exp Clin Pharmacol, 27(7), 495-502. doi: 10.1358/mf.2005.27.7.921309
Altman, D. G. (2005). Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors. [Research Support, Non-U.S. Gov't]. BMJ, 330(7499), 1056-1057. doi: 10.1136/bmj.330.7499.1056
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., . . . Lang, T. (2001). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. [Consensus Development Conference Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S. Review]. Ann Intern Med, 134(8), 663-694.
Arrowsmith, J. (2011). Trial watch: phase III and submission failures: 2007-2010. [News]. Nat Rev Drug Discov, 10(2), 87. doi: 10.1038/nrd3375
Baker, D., Gerritsen, W., Rundle, J., & Amor, S. (2011). Critical appraisal of animal models of multiple sclerosis. [Research Support, Non-U.S. Gov't Review]. Mult Scler, 17(6), 647-657. doi: 10.1177/1352458511398885
Baker, D., Lidster, K., Sottomayor, A., & Amor, S. (2014). Two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLoS Biol, 12(1), e1001756. doi: 10.1371/journal.pbio.1001756
Bebarta, V., Luyten, D., & Heard, K. (2003). Emergency medicine animal research: does use of randomization and blinding affect the results? [Comparative Study Review]. Acad Emerg Med, 10(6), 684-687.
Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., . . . Stroup, D. F. (1996). Improving the quality of reporting of randomized controlled trials. The CONSORT statement. [Research Support, Non-U.S. Gov't]. JAMA, 276(8), 637-639.
Ben-Cherif, W., Dridi, I., Aouam, K., Ben-Attia, M., Reinberg, A., & Boughattas, N. A. (2013). Circadian variation of Valproic acid pharmacokinetics in mice. [Research Support, Non-U.S. Gov't]. Eur J Pharm Sci, 49(4), 468-473. doi: 10.1016/j.ejps.2013.05.009
Bracken, M. B. (2009a). Why animal studies are often poor predictors of human reactions to exposure. J R Soc Med, 102(3), 120-122. doi: 10.1258/jrsm.2008.08k033
Bracken, M. B. (2009b). Why are so many epidemiology associations inflated or wrong? Does poorly conducted animal research suggest implausible hypotheses? Ann Epidemiol, 19(3), 220-224. doi: 10.1016/j.annepidem.2008.11.006
Briel, M., Muller, K. F., Meerpohl, J. J., von Elm, E., Lang, B., Motschall, E., . . . Bassler, D. (2013). Publication bias in animal research: a systematic review protocol. [Research Support, Non-U.S. Gov't]. Syst Rev, 2, 23. doi: 10.1186/2046-4053-2-23
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafo, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. [Research Support, Non-U.S. Gov't]. Nat Rev Neurosci, 14(5), 365-376. doi: 10.1038/nrn3475
Cao, L., Liu, X., Lin, E. J., Wang, C., Choi, E. Y., Riban, V., . . . During, M. J. (2010). Environmental and genetic activation of a brain-adipocyte BDNF/leptin axis causes cancer remission and inhibition. [Research Support, N.I.H., Extramural]. Cell, 142(1), 52-64. doi: 10.1016/j.cell.2010.05.029
Chalmers, I. (1990). Underreporting research is scientific misconduct. [Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S.]. JAMA, 263(10), 1405-1408.
Christopher, M. M. (2007). Improving the quality of reporting of studies of diagnostic accuracy: let's STARD now. [Editorial]. Vet Clin Pathol, 36(1), 6.
Coppus, S. F., van der Veen, F., Bossuyt, P. M., & Mol, B. W. (2006). Quality of reporting of test accuracy studies in reproductive medicine: impact of the Standards for Reporting of Diagnostic Accuracy (STARD) initiative. [Evaluation Studies Research Support, Non-U.S. Gov't]. Fertil Steril, 86(5), 1321-1329. doi: 10.1016/j.fertnstert.2006.03.050
Crossley, N. A., Sena, E., Goehler, J., Horn, J., van der Worp, B., Bath, P. M., . . . Dirnagl, U. (2008). Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. [Evaluation Studies Research Support, Non-U.S. Gov't]. Stroke, 39(3), 929-934. doi: 10.1161/STROKEAHA.107.498725
Dirnagl, U. (2006). Bench to bedside: the quest for quality in experimental stroke research. [Research Support, Non-U.S. Gov't Review]. J Cereb Blood Flow Metab, 26(12), 1465-1478. doi: 10.1038/sj.jcbfm.9600298
Dirnagl, U., & Lauritzen, M. (2011). Improving the quality of biomedical research: guidelines for reporting experiments involving animals. [Editorial Introductory]. J Cereb Blood Flow Metab, 31(4), 989-990. doi: 10.1038/jcbfm.2010.219
Drummond, G. B., Paterson, D. J., & McGrath, J. C. (2010). ARRIVE: new guidelines for reporting animal research. [Editorial]. Exp Physiol, 95(8), 841. doi: 10.1113/expphysiol.2010.053785
Erb, H. N. (2010). Changing expectations: Do journals drive methodological changes? Should they? Prev Vet Med, 97(3-4), 165-174. doi: 10.1016/j.prevetmed.2010.09.011
Fisher, M., Feuerstein, G., Howells, D. W., Hurn, P. D., Kent, T. A., Savitz, S. I., & Lo, E. H. (2009). Update of the stroke therapy academic industry roundtable preclinical recommendations. [Review]. Stroke, 40(6), 2244-2250. doi: 10.1161/STROKEAHA.108.541128
Grindlay, D. J., Dean, R. S., Christopher, M. M., & Brennan, M. L. (2014). A survey of the awareness, knowledge, policies and views of veterinary journal Editors-in-Chief on reporting guidelines for publication of research. BMC Vet Res, 10, 10. doi: 10.1186/1746-6148-10-10
Hackam, D. G., & Redelmeier, D. A. (2006). Translation of research evidence from animals to humans. [Letter Research Support, Non-U.S. Gov't Review]. JAMA, 296(14), 1731-1732. doi: 10.1001/jama.296.14.1731
Hill, C. L., LaValley, M. P., & Felson, D. T. (2002). Secular changes in the quality of published randomized clinical trials in rheumatology. [Evaluation Studies Research Support, Non-U.S. Gov't]. Arthritis Rheum, 46(3), 779-784. doi: 10.1002/art.512
Hirst, A., & Altman, D. G. (2012). Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. [Research Support, Non-U.S. Gov't]. PLoS One, 7(4), e35621. doi: 10.1371/journal.pone.0035621
Hooijmans, C. R., Leenaars, M., & Ritskes-Hoitinga, M. (2010). A gold standard publication checklist to improve the quality of animal studies, to fully integrate the Three Rs, and to make systematic reviews more feasible. [Research Support, Non-U.S. Gov't]. Altern Lab Anim, 38(2), 167-182.
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014). SYRCLE's risk of bias tool for animal studies. [Research Support, Non-U.S. Gov't]. BMC Med Res Methodol, 14, 43. doi: 10.1186/1471-2288-14-43
Hopewell, S., Dutton, S., Yu, L. M., Chan, A. W., & Altman, D. G. (2010). The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. [Research Support, Non-U.S. Gov't]. BMJ, 340, c723. doi: 10.1136/bmj.c723
Horn, J., de Haan, R. J., Vermeulen, M., Luiten, P. G., & Limburg, M. (2001). Nimodipine in animal model experiments of focal cerebral ischemia: a systematic review. [Review]. Stroke, 32(10), 2433-2438.
Hotopf, M., Lewis, G., & Normand, C. (1997). Putting trials on trial--the costs and consequences of small trials in depression: a systematic review of methodology. [Review]. J Epidemiol Community Health, 51(4), 354-358.
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Med, 2(8), e124. doi: 10.1371/journal.pmed.0020124
Ioannidis, J. P., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., . . . Tibshirani, R. (2014). Increasing value and reducing waste in research design, conduct, and analysis. Lancet, 383(9912), 166-175. doi: 10.1016/S0140-6736(13)62227-8
Jerndal, M., Forsberg, K., Sena, E. S., Macleod, M. R., O'Collins, V. E., Linden, T., . . . Howells, D. W. (2010). A systematic review and meta-analysis of erythropoietin in experimental stroke. [Meta-Analysis Research Support, Non-U.S. Gov't Review]. J Cereb Blood Flow Metab, 30(5), 961-968. doi: 10.1038/jcbfm.2009.267
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. [Guideline Research Support, Non-U.S. Gov't Review]. PLoS Biol, 8(6), e1000412. doi: 10.1371/journal.pbio.1000412
Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F., Cuthill, I. C., Fry, D., . . . Altman, D. G. (2009). Survey of the quality of experimental design, statistical analysis and reporting of research using animals. [Research Support, Non-U.S. Gov't]. PLoS One, 4(11), e7824. doi: 10.1371/journal.pone.0007824
Knight, J. (2003). Negative results: Null and void. [News]. Nature, 422(6932), 554-555. doi: 10.1038/422554a
Kokolus, K. M., Capitano, M. L., Lee, C. T., Eng, J. W., Waight, J. D., Hylander, B. L., . . . Repasky, E. A. (2013). Baseline tumor growth and immune control in laboratory mice are significantly influenced by subthermoneutral housing temperature. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't]. Proc Natl Acad Sci U S A, 110(50), 20176-20181. doi: 10.1073/pnas.1304291110
Kola, I., & Landis, J. (2004). Can the pharmaceutical industry reduce attrition rates? Nat Rev Drug Discov, 3(8), 711-715. doi: 10.1038/nrd1470
Korevaar, D. A., Hooft, L., & ter Riet, G. (2011). Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments. Lab Anim, 45(4), 225-230. doi: 10.1258/la.2011.010121
Krauth, D., Woodruff, T. J., & Bero, L. (2013). Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. [Research Support, N.I.H., Extramural Review]. Environ Health Perspect, 121(9), 985-992. doi: 10.1289/ehp.1206389
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., . . . Silberberg, S. D. (2012). A call for transparent reporting to optimize the predictive value of preclinical research. [Research Support, N.I.H., Extramural]. Nature, 490(7419), 187-191. doi: 10.1038/nature11556
Macleod, M. R., Fisher, M., O'Collins, V., Sena, E. S., Dirnagl, U., Bath, P. M., . . . Howells, D. W. (2009). Good laboratory practice: preventing introduction of bias at the bench. Stroke, 40(3), e50-52. doi: 10.1161/STROKEAHA.108.525386
Macleod, M. R., Lawson McLean, A., Kyriakopoulou, A., Serghiou, S., de Wilde, A., Sherratt, N., . . . Sena, E. S. (2015). Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. [Research Support, Non-U.S. Gov't]. PLoS Biol, 13(10), e1002273. doi: 10.1371/journal.pbio.1002273
Macleod, M. R., O'Collins, T., Howells, D. W., & Donnan, G. A. (2004). Pooling of animal experimental data reveals influence of study design and publication bias. [Meta-Analysis Research Support, Non-U.S. Gov't]. Stroke, 35(5), 1203-1208. doi: 10.1161/01.STR.0000125719.25853.20
Macleod, M. R., van der Worp, H. B., Sena, E. S., Howells, D. W., Dirnagl, U., & Donnan, G. A. (2008). Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. [Meta-Analysis Research Support, Non-U.S. Gov't]. Stroke, 39(10), 2824-2829. doi: 10.1161/STROKEAHA.108.515957
Martin, B., Ji, S., Maudsley, S., & Mattson, M. P. (2010). "Control" laboratory rodents are metabolically morbid: why it matters. [Research Support, N.I.H., Intramural]. Proc Natl Acad Sci U S A, 107(14), 6127-6133. doi: 10.1073/pnas.0912955107
Masoro, E. J. (2009). Caloric restriction-induced life extension of rats and mice: a critique of proposed mechanisms. [Review]. Biochim Biophys Acta, 1790(10), 1040-1048. doi: 10.1016/j.bbagen.2009.02.011
Mignini, L. E., & Khan, K. S. (2006). Methodological quality of systematic reviews of animal studies: a survey of reviews of basic research. [Comparative Study Meta-Analysis Research Support, Non-U.S. Gov't]. BMC Med Res Methodol, 6, 10. doi: 10.1186/1471-2288-6-10
Mills, E., Wu, P., Gagnier, J., Heels-Ansdell, D., & Montori, V. M. (2005). An analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently. J Clin Epidemiol, 58(7), 662-667. doi: 10.1016/j.jclinepi.2005.01.004
Nevalainen, T. O., Nevalainen, J. I., Guhad, F. A., & Lang, C. M. (2007). Pair housing of rabbits reduces variances in growth rates and serum alkaline phosphatase levels. [Research Support, Non-U.S. Gov't]. Lab Anim, 41(4), 432-440. doi: 10.1258/002367707782314247
O'Collins, V. E., Macleod, M. R., Donnan, G. A., Horky, L. L., van der Worp, B. H., & Howells, D. W. (2006). 1,026 experimental treatments in acute stroke. [Comparative Study Evaluation Studies Research Support, Non-U.S. Gov't]. Ann Neurol, 59(3), 467-477. doi: 10.1002/ana.20741
Perel, P., Roberts, I., Sena, E., Wheble, P., Briscoe, C., Sandercock, P., . . . Khan, K. S. (2007). Comparison of treatment effects between animal experiments and clinical trials: systematic review. [Comparative Study Research Support, Non-U.S. Gov't Review]. BMJ, 334(7586), 197. doi: 10.1136/bmj.39048.407928.BE
Phillips, C. V., MacLehose, R. F., & Kaufman, J. S. (2008). (Errors in statistical tests)3. Emerg Themes Epidemiol, 5, 9. doi: 10.1186/1742-7622-5-9
Plint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006). Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. [Review]. Med J Aust, 185(5), 263-267.
Pound, P., Ebrahim, S., Sandercock, P., Bracken, M. B., & Roberts, I. (2004). Where is the evidence that animal research benefits humans? [Review]. BMJ, 328(7438), 514-517. doi: 10.1136/bmj.328.7438.514
Prager, E. M., Bergstrom, H. C., Grunberg, N. E., & Johnson, L. R. (2011). The importance of reporting housing and husbandry in rat research. Front Behav Neurosci, 5, 38. doi: 10.3389/fnbeh.2011.00038
Ross, M. H., & Bras, G. (1971). Lasting influence of early caloric restriction on prevalence of neoplasms in the rat. J Natl Cancer Inst, 47(5), 1095-1113.
Salvarrey-Strati, A., Watson, L., Blanchet, T., Lu, N., & Glasson, S. S. (2008). The influence of enrichment devices on development of osteoarthritis in a surgically induced murine model. ILAR J, 49(4), 23-30.
Saver, J. L., Albers, G. W., Dunn, B., Johnston, K. C., & Fisher, M. (2009). Stroke Therapy Academic Industry Roundtable (STAIR) recommendations for extended window acute stroke therapy trials. [Congresses Research Support, N.I.H., Extramural]. Stroke, 40(7), 2594-2600. doi: 10.1161/STROKEAHA.109.552554
Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. [Guideline Research Support, Non-U.S. Gov't]. Obstet Gynecol, 115(5), 1063-1070. doi: 10.1097/AOG.0b013e3181d9d421
Schwarz, F., Iglhaut, G., & Becker, J. (2012). Quality assessment of reporting of animal studies on pathogenesis and treatment of peri-implant mucositis and peri-implantitis. A systematic review using the ARRIVE guidelines. [Research Support, Non-U.S. Gov't Review]. J Clin Periodontol, 39 Suppl 12, 63-72. doi: 10.1111/j.1600-051X.2011.01838.
Sena, E., van der Worp, H. B., Howells, D., & Macleod, M. (2007). How can we improve the pre-clinical development of drugs for stroke? [Research Support, Non-U.S. Gov't Review]. Trends Neurosci, 30(9), 433-439. doi: 10.1016/j.tins.2007.06.009
Sena, E. S., Briscoe, C. L., Howells, D. W., Donnan, G. A., Sandercock, P. A., & Macleod, M. R. (2010). Factors affecting the apparent efficacy and safety of tissue plasminogen activator in thrombotic occlusion models of stroke: systematic review and meta-analysis. [Meta-Analysis Research Support, Non-U.S. Gov't Review]. J Cereb Blood Flow Metab, 30(12), 1905-1913. doi: 10.1038/jcbfm.2010.116
Sena, E. S., Currie, G. L., McCann, S. K., Macleod, M. R., & Howells, D. W. (2014). Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. [Review]. J Cereb Blood Flow Metab, 34(5), 737-742. doi: 10.1038/jcbfm.2014.28
Sena, E. S., van der Worp, H. B., Bath, P. M., Howells, D. W., & Macleod, M. R. (2010). Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. [Research Support, Non-U.S. Gov't]. PLoS Biol, 8(3), e1000344. doi: 10.1371/journal.pbio.1000344
Siegel, V. (2011). Reproducibility in research. [Editorial]. Dis Model Mech, 4(3), 279-280. doi: 10.1242/dmm.008037
Smith, J. A., Birke, L., & Sadler, D. (1997). Reporting animal use in scientific papers. Lab Anim, 31(4), 312-317.
ter Riet, G., Korevaar, D. A., Leenaars, M., Sterk, P. J., Van Noorden, C. J., Bouter, L. M., . . . Hooft, L. (2012). Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions. PLoS One, 7(9), e43404. doi: 10.1371/journal.pone.0043404
Ting, K. H., Hill, C. L., & Whittle, S. L. (2015). Quality of reporting of interventional animal studies in rheumatology: a systematic review using the ARRIVE guidelines. Int J Rheum Dis, 18(5), 488-494. doi: 10.1111/1756-185X.12699
Tsilidis, K. K., Panagiotou, O. A., Sena, E. S., Aretouli, E., Evangelou, E., Howells, D. W., . . . Ioannidis, J. P. (2013). Evaluation of excess significance bias in animal studies of neurological diseases. PLoS Biol, 11(7), e1001609. doi: 10.1371/journal.pbio.1001609
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012). Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. [Review]. Syst Rev, 1, 60. doi: 10.1186/2046-4053-1-60
van der Worp, H. B., de Haan, P., Morrema, E., & Kalkman, C. J. (2005). Methodological quality of animal studies on neuroprotection in focal cerebral ischaemia. [Research Support, Non-U.S. Gov't Review]. J Neurol, 252(9), 1108-1114. doi: 10.1007/s00415-005-0802-3
van der Worp, H. B., Sena, E. S., Donnan, G. A., Howells, D. W., & Macleod, M. R. (2007). Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. [Meta-Analysis Research Support, Non-U.S. Gov't Review]. Brain, 130(Pt 12), 3063-3074. doi: 10.1093/brain/awm083
Varga, O., Hansen, A. K., Sandoe, P., & Olsson, I. A. (2010). Improving transparency and ethical accountability in animal studies: three ways to link ethical approvals to publications. EMBO Rep, 11(7), 500-503. doi: 10.1038/embor.2010.91
Vesterinen, H. M., Egan, K., Deister, A., Schlattmann, P., Macleod, M. R., & Dirnagl, U. (2011). Systematic survey of the design, statistical analysis, and reporting of studies published in the 2008 volume of the Journal of Cerebral Blood Flow and Metabolism. [Research Support, Non-U.S. Gov't]. J Cereb Blood Flow Metab, 31(4), 1064-1072. doi: 10.1038/jcbfm.2010.217
Vesterinen, H. M., Sena, E. S., ffrench-Constant, C., Williams, A., Chandran, S., & Macleod, M. R. (2010). Improving the translational hit of experimental treatments in multiple sclerosis. [Meta-Analysis Research Support, Non-U.S. Gov't Review]. Mult Scler, 16(9), 1044-1055. doi: 10.1177/1352458510379612
Weindruch, R., & Sohal, R. S. (1997). Seminars in medicine of the Beth Israel Deaconess Medical Center. Caloric intake and aging. [Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, P.H.S. Review]. N Engl J Med, 337(14), 986-994. doi: 10.1056/NEJM199710023371407