Bibliometric and Metadata Indicators to Support Literature Screening in Clinical Practice Guidelines: Identification and External Validation

Abstract

Background:
Clinical practice guidelines (CPGs) play a major role in implementing evidence-based medicine into clinical practice. However, guideline development requires screening hundreds to thousands of candidate articles to identify literature supporting recommendations, imposing a substantial burden on physicians involved in guideline development. Indicators that characterize candidate articles from multiple perspectives could help reduce this literature-screening burden.
Aims:
To identify bibliometric indicators and metadata associated with article citation in clinical practice guidelines and to evaluate their external validity.
Methods:
The published search strategies of the 2023 Japanese Urological Association and 2025 European Association of Urology guidelines for upper tract urothelial carcinoma were rerun to construct derivation and validation datasets. Bibliometric indicators (citation counts and journal characteristics) and metadata (study design) were collected using values available at each guideline's literature search. Associations between guideline citation and each indicator were evaluated using multivariable logistic regression, and discrimination was assessed by the area under the receiver operating characteristic curve (AUC).
Results:
In the derivation dataset, three factors were independently associated with guideline citation: average citations per year (odds ratio [OR], 1.017; 95% confidence interval [CI], 1.007-1.025), journal Cited Half-Life (CHL) (OR, 1.219; 1.110-1.340), and randomized controlled trial (RCT) or meta-analysis (OR, 10.775; 5.668-20.482). In the validation dataset, the same three factors showed associations in the same direction (ORs, 1.093, 1.047, and 2.827). The AUC was 0.766 (95% CI, 0.691-0.841) in the derivation dataset and 0.768 (95% CI, 0.730-0.806) in the validation dataset. These factors reflected three aspects of candidate articles: citation density, persistence of citation in the publishing journal, and study design.
Conclusion:
Average citations per year, journal CHL, and RCT or meta-analysis were associated with guideline citation across two guidelines in the same disease area, supporting the external validity of these factors. These three indicators capture complementary aspects of candidate articles and may help reduce the burden on physicians involved in clinical practice guideline development.