Beyond Confidence Scores: Calibrating Physician Reliance on AI Decision Support Through Localized Reliability Signals

Artificial intelligence is entering clinical practice faster than we understand how physicians actually use it. Two failure modes threaten patient safety: following recommendations that are wrong, and dismissing recommendations that are right. Both stem from a mismatch between how much a clinician trusts a system and how reliable that system actually is for a given case. We argue that the information these systems provide about their own limitations is central to closing that gap. We illustrate this using results from a controlled study of 272 participants that compared five ways of communicating an AI's performance. It found that stating a system's overall accuracy, the most common approach in practice, did not improve decisions at all. What did help was providing information about the system's performance on prior cases resembling the current one. This localized reliability signal was especially valuable when the AI was wrong but confident. Notably, participants made better decisions without reporting better understanding of the system, suggesting calibration develops through exposure rather than explanation. We translate these findings into design and training recommendations for clinical decision support.